Yio Remote Community

Added ROON integration and media browsing

This weekend I implemented ROON integration. In my opinion ROON is one of the best music servers (https://roonlabs.com/) but I found out that the ROON community is more audiophile than IT/smart home enthusiastic. I tried to arouse interest for the YIO remote in the ROON community, but a DIY remote is for most of them too heavy. Anyhow the ROON integration for YIO is available now at https://github.com/ChristianRiedl/integration.roon. Discovery is not available until now, the port and IP address of the server must be configured. Important: the entities friendly_name must be the name of the ROON instance :
“media_player”: [
{
“entity_id”: “media_player.work”,
“friendly_name”: “Arbeitsplatz”,
“integration”: “roon”,
“type:”: “speaker”,
“supported_features”: [

  {
       "url": "ws://192.168.1.100:9100/api",
        "imageurl": "http://192.168.1.100:9100/api/image/",
        "friendly_name": "RIC's Roon Control",
        "id": "roon",
        "log": "info"
   } 

I added configurable logging using QLoggingCategory. Later I will redirect the output to a log file with QMessageLogger. I recommend it for other integrations. It allows to change logging without recompiling, important for supporting users.

Additionally I started with implementation of media browsing based on the current YIO remote-software, dev branch. This version is at https://github.com/ChristianRiedl/remote-software, dev branch.
I made some (small) extensions to the media-player entity :

  • Adding a property browseItems. It is a list of media entry objects containing
    Item_key key of the entry
    title
    sub_title
    image_url for albums, and artists, (accessing the ROON server)

  • Adding a property browseCmds. Up to 3 commands which are currently use for 3 buttons on the head of the list. Currently I work only with simulation maybe on the real YIO we need only the hardware buttons. This commands are necessary as ROON provides several possible commands on the different hierarchy levels :
    Artist play tracks from this artists randomly
    Album play whole album immediately, add to queue
    Track play now, next, queue

  • Adding a function browse (cmd)
    cmd is any of the commands from browseCmds (TOP, BACK, PLAY, QUEUE) or the selected enries item_key.

To test it, I modified the current implementation of components/media_player :

  • I implemented a rudimentary Card.qml (modifying the blinds card.qml). I added a “LIST” button which invokes the CardList.qml. In the real version the LIST function can be invoked with hardware button of course.

  • I implemented a CardList.qml (in this version a quite rough appearance, I will try to improve it) it allows to browse the media and select album, tracks for playing.

To be discussed : currently CardList.qml uses a scrolling list for the media items. This list can be quite long. This can be a performance problem when album images are displayed. I am not sure if Qt ListView loads and renders only visible images. If not, we need another solution (paging or optimisation of image loading)

1 Like

That is great @ChristianRiedl! Thank you for your contribution. Roon would be a nice integration to have. I was also looking at it at some point to use it myself, but the hefty price tag scared me away :slight_smile:

Would you be able to move the roon integration to run in a separate thread? Like the Home Assistant and other integrations? I experienced it with Home Assistant, when a lot of data coming through websocket, the UI hangs. Running it in a different thread prevents this problem. Unfortunately the websocket class is not running in a different thread by default.

The use of QLogger would be a nice addition. I just throw in a primitive logging class that writes to a file, but I think that could be definitely improved with the addition of categories, etc.

The design of the media player, including browsing, album view, playlist view, etc., is underway. So hopefully in a couple of days I can post it up here so you can also see. The aim is still to create a unified look for all the integrations.

I tested the scrolling with list with my spotify integration and seems like it works well. As far as I know Qt only renders visible elements, otherwise we can do lazy loading. Also what I was doing lately is to resize images in the backend and provide the appropriate size to the qml side. This makes the UI much more responsive.

One thing I don’t really understand is the browse(cmd) addtion to the media player entity. We have the supported features that defines the UI as well. So basically if PLAY or QUEUE is in the supported features list, then the UI reacts to it and displays those UI elements/features. We do this in all the various entities and so far it works great.

But all in all, I really appreciate your work and addition to the project!

Ok I will implement it in a thread !
Regarding browse (cmd) : We need some function to navigate to a selected item, to top, back. ROON server provides different play feature on different levels and supports different hierarchies (with or without genre, I prefer with to keep the album list shorter but some dislike this kind of categorysation). I use the browseCmds array to change the available play and navigation features dynamically. I want to show you ROON’s browse hierarchy and the possible play modes.

Library
    Artists                                 
        Artistxy                                   Play Artist (Play Now, Radio)
            Albumxy                                Play Album (Play Now, Add Next, Queue, Radio)
                 Trackxy                           Play Track (Play Now, Add Next, Queue, Radio)
    Albums   
            Albumxy                                Play Album (Play Now, Add Next, Queue, Radio)
                 Trackxy                           Play Track (Play Now, Add Next, Queue, Radio)

Genres
    Genrexy                                         Play Genre (Shuffle, Radio)
        Artistxy                                    Play Artist (Play Now, Radio)     
            Albumxy                                 Play Album (Play Now, Add Next, Queue, Radio)
                 Trackxy                            Play Track (Play Now, Add Next, Queue, Radio)

Playlists
    Playlistxy                                      Play list  (Play Now, Add Next, Queue, Radio)
         Tracks                                     Play Track (Play Now, Add Next, Queue, Radio)

Internet Radio
    Station xy                                      Play now

Play Now : Stop current and play immediately
Play Next : Play after current finished
Queue : Add at the end
Shuffle : play random
Radio : play random and continue with similar Music

Of course it is not really necessary to implement all this stuff. But the hierarchy should not be restricted to genre - artist - albums as I use normally, different servers and people use different hierarchies. I also sometime start with artists when the genre is not clear. Some systems offer internet radio, playlist other not. I know systems offering “favorites”. And some systems allow playing artists, albums other not. Radio mode is really ROON specific I think. But Play Now or Queue is a required feature I think.

All devices until now have the feature list, But for media browsing it is more flexible to
offer generic browsing supporting what the server is offering. When using only the feature list, a ROON user will want the features
PLAY_ALBUM, PLAY_ARTIST, INTERNETRADIO, PLAYLIST
BROWSE_ARTIST, BROWSE_GENRE, BROWSE_ALBUM

We will have one GUI for multiple media servers, but multiple media server integration implementations with different hierarchies and features. More complicated than lights with or without brightness control.

Great, thank you!

Play Now, Play Next, Queue, Shuffle, Radio are quite common these days. Spotify, Tidal and my Bang and Olufsen speakers have the same and work similar way like Roon does.

I am thinking of these features while designing the UI. It might not cover everything from the get-go, but I think it will be a good base to build on.

I think these features can be included in the supported features list. If the integration offers internet radio or not, etc. So we can use that and show the UI that is needed. The media player is going to be the most complicated one regarding the entity type and UI representation, that is for sure. But I think we are talking about the same thing. The UI will have all the necessary elements to work with as many integrations as possible. The integration will handle the logic.

I have created a repository on the YIO Remote github page. Feel free to join there:

I joined, thank you. I was thinking about the separate thread in the integrations.

I found out (google) that QtWebsockets open can take some time, but in theory sendmessage should be fast. Generally adding threads for poor single kernel CPU’s does not bring an advantage, as long as all operations are asynchronous. I will make some perftests on the PI Zero to see what is going on.

I also read about updating Qt GUI updating from a thread. The answers I found where that the update should be done using the signal / slot mechanism which is working accross thread borders. As I see in the home_assitant integration only the sendCommand is done via signal / slot.

Probably the constructs

Q_PROPERTY (QColor color READ color NOTIFY colorChanged)

creates something similar which switches the notification event to another threads. But such thread switchings also costs time. I will test it also on the Pi Zero. I like to figure such things out. When I started my software carrier we controlled a whole powerplant with a processor having less than 10 % of the PI Zero power and memory.

Glad to have you on board.

When I tried the home assistant integration without QThread, the UI was blocked when I was getting a lot of state updates. After that I moved that to QThread, the issue was gone. Same with hardware handling. I started to move all hardware interfacing to a separate thread to avoid UI blocking. For example while waiting for ambient light measurement, etc.

The UI can be updated from a thread via signals and slots. The buttons also work like this. They emit a signal when there was a button press. And the various UI elements connect to this signal and act on it.

I’m loving this, it’s like Christmas morning already.
As a frequent ROON user and having all my music inside ROON including my Tidal subscription this news make me more then happy! Great Job Christian :wink:
Keep up the good work and make it the best ROON experience you can together with the help from Marton. Good job! I myself am not a developer, but can really appreciate everyones real honest effort to make YIO Remote the best experience and solution that there is. Keep it up.

1 Like

I implemented ROON discovery. Unfortunately they don’t support mDNS, but something named “SOOD”, based on multicast UDP.

To integrate it into remote software there are some questions.

Currently discovery and integration are both handled by the Integrations class. Dock is “discovered”, the other integrations require configuration via config.json.

Discovery of real hubs and their entities is a very complicated thing. It is a process requiring user interaction and can only be integrated later and somehow in the web configurator, I think.

@marton: I have an easier question. Roon integration requires some file storage (tokens delivered by the pairing procedure). The appPath is available as root contextProperty but I did not find a way to access it from the intergration dll.

Great addition. Thanks @ChristianRiedl!

Yes. We have plans to do the integration setup from the web configurator. The plan is to expand the YIO API with methods, so the web configurator can ask the api to discover this and that etc. We are focusing on other aspects but this is on the list.

The appPath variable is not available globally yet. We haven’t had the need for it, until now. Feel free to change the code to make it available. Let’s keep the files required by the integrations in their own directory, so it’s well organized.

@marton: I think the best way is to add a getContextProperty function to Config class and ConfigInterface. It is somehow related with configuration.

How shall we deal on github with the remote-software repo.
Shall I work with my private github repository ?
Can you create a “dev-christianriedl” branch for me ?

I also would like to check in my intermediate media-list solution until you replace it with something new.
Quite often I switch my development workstation (work, home, notebook at weekend) and use github to stay synced.

@ChristianRiedl sounds good with getContextProperty.

I’ll add you to the repo. If you would create a feature branch with the changes, that would be great.
We create feature branches and then create a pull request to the dev branch.

Ok. I added some features (getContextProperty, yioapi button simulation) and a command line argument “simulate” which allows to run YIO remote in a development environment without hardware and dock without changing sources.

I continue with “generic media browsing” as intermediate solution because it is the best way to test ROON integration. Later we can replace it.

Great additions! Thank you @ChristianRiedl.

Here is a preview of the media player I am working on. This shows the open state when you tap on the “button” and also the search functionality. The background would change according to the average color of the album art. I already have this working in the mini media player and it looks really good :slight_smile: For the search, I imagined a feature where you can filter the results. There’s an album/playlist view as well. I’ll be building this hopefully this weekend in QML. The three dot, next to the track, could bring up a contextual menu with more options.

2 Likes

@marton: I continued working on “generic browsing”.

Searching : For me even the virtual smartphone keyboards are painfully small. Whenever I use TIDAL with ROON I use the iPad. The big amount of results TIDAL search delivers are easier to check on a bigger device. But anyhow I will add search support to my ROON integration.

I found out that it is better to deliver the BROWSE result in a single object. It is more efficient and avoids synchronization problems (sequence of updates in QML).

  QVariant  m_browseResult;  // Better to return in one structure (perf, synchronisation) :
                             // items :          Array of { item_key, title, sub_title, image_url }
                             // playCommands:    Array of play commands for the items like PLAY, QUEUE
                             // type:            Result type Genre, Album, Artist, Track, Playlist, Radio
                             // title:           Name of the genre, album, artist, track
                             // level:           Top is 0

playCommands : Especially when browsing TIDAL it is unpredictable which PLAY commands are supported. I propose to keep it as optional item.

ROON has a strange browsing philosophy. You have to drill down in the hierarchy to get a “key” for a PLAY NOW command : artist -> album -> PLAY ALBUM -> PLAY NOW. But I think we should be able to execute any available PLAY operation immediately on the displayed item in a artist, album, track list. I was able to implement it in this way.

I also was thinking about supported_features for media players. I think the integration knows best what is supported on which media server. The customer wants to have all possible features and we will have more troubles when he thinks it is sufficient to add some feature (when the media server does not support it).

Here my current solution, for me sufficient. Not very beautiful, no transitions. I never was a UI guy.

You get the popup menu when you select an item via touch or hardware button.The menu item also can be selected via touch or hardware button.

I think most of the features should work with the hardware buttons. This is the biggest advantage of YIO compared to a smartphone app.
Some keys can be used in an intuitive manner (up / down, OK, volume up / down).
but some things are not so intuitive. For such things maybe we could implement a popup menu on the OK hardware button which shows in “beginner mode” additionally all available “non intuitive” buttons. I am not sure if it is a good idea. I added it mainly as option for easier testing, the red drawings are retouche.

2 Likes

I think the browseResult object is a good idea. Will be easy to connect to QML.

I know search is cumbersome on small screens, but I think it’s something we should have.

I didn’t have as much time as I wanted to keep building the UI parts, but I managed to build some things for the media player. It’s in the dev branch.

The overlay button helper screen is something I also thought about and I think it’s good to have.

I’ve added more screens. Will be building these in the coming days.

1 Like

@ChristianRiedl Amazing work man!

As you know we need to standarize the library stuff while providing as much of the needed features. I made a github issue describing an example intergration / Proposal draft thingy. :wink:
I cannot call it a design though…

Could you have a look at it?

What do you think about this approach?
Are there substantial things missing?
I know the contextual menu is missing, I believe we should focus on the browsing stuff first to folowup with the contextual stuff. what are your thoughts.

My current status : (I will answer Niels question seperately) :

I implemented searching (1), playing from TIDAL works (2) added “Play From”(3) and internet radio is also working (4).

Searching : ROON allows searching on your own local library and on any subscribed service like TIDAL, QOBUZ. For this reason searching is an attribute of an item in the list.
When you select it you get a input popup.

Play from : It allows to “Play Now” a track and appends the rest (following tracks) of the album to the queue. “Play Now” deletes the queue.

@marton : 2 not related questions

  • How do you debug on the Pi Zero, using remote debugging ? Or do you run QtCreator on the poor PI (I tryed it but it is really slow).

  • Will it be possible to add a private component (QML only) without compiling whole remote software. Is it possible to mix quickcompile with uncompiled QML? Similar as we have it now for the integrations it would be great to make the UI also modular.

Browse hierarchy :
@Nklerk

My ideas are a bit ROON influenced. But I think they can be applied to other systems too.
When using pathes there is the problem that the same item can have different pathes because you can start browsing with genres, artist, albums. Anyhow every item must have some unique key.
Browsing is navigating in a tree where you do not know the children before drilling down.

For example when you browse TIDAL service you cannot predict what you will get (favorites, news, top 10 …). If we want to to support it we cannot rely on strict pathes.

Let me explain my browse / play model (you find it in my dev branch / media player). I tried to keep it as simple as possible, of course it can be exended with additional attributes.

The result of any browsing is always delivered as one structure (browseResult) consisting of

  • array of items

  • array of play commands which can be applied for each of the items (Play Now, Play from here, Add Next, Shuffle, Queue, Start Radio)

  • item type (genre, album, artist, playlist, track)

  • title shown in the header (artist name, album name ,…)

  • level in the hierarchy

Every item consists of :

  • a unique item key used for next browse or play operations (can also be a path).

  • a title and a subtitle intended for a two line text in the browse list (artist/album, album/track …)

  • an optional image url

  • an optional input prompt for searchable items

There are only 3 functions required for browsing and starting media playing

  • browse (target). The target is either a reserved word TOP (of hierarchy) BACK (one level) or an item key. After the function is called, browseResult will be updated.

  • playMedia (command, item key). Command is one of the play commands. The function is applied on the selected item.

  • search (search text, item key). The search text is applied by the user. The result is again delivered in the browseResult. TIDAL delivers an additional hierarchy level (albums, artists)

More advanced attributes (lyrics, stories, links…) are not accessible via ROON API. As the space on the YIO screen is quite limited I will not miss it.

I should better describe the proposal. What i describe is exacly what you are describing. minus the search. please have a look at it again and look into the data what’s missing.

The search isn’t documented yet… My idea there is to let the integration return an array named groups that include the groups that are available for filtering, i.e. album, tracks etcetra. the items also contain a group (array) where they could be for instance Movie and Watched or Episode and unwatched. the UI can filter the results.