Remote control, Voice control, options? Server?

To discuss development of addons / skins / customization of MediaMonkey.

Moderator: jiri

Linwood
Posts: 63
Joined: Sat Jul 13, 2019 6:49 pm

Remote control, Voice control, options? Server?

Post by Linwood » Wed Nov 20, 2019 10:04 am

I'm not quite sure where to put this so apologies if it is in the wrong place.

I've become a real fan of Google Play Music, and MM5's sync to it is great. And yes I know You Tube may screw that up eventually.

Have you considered, or is there now (here or from third parties) any way to directly control MM5 (desktop or server) from Google Voice Assistant. I.e. to play tracks locally instead of from GPM?

One reason I need it is I have a ton of player piano music I'd like to play directly to the piano by voice. Putting it in GPM is not good as it makes a horrible squeal if you accidentally play it on regular speakers, so I'd like to put a dedicated library up hooked to the piano.

Lacking that, is there any plan (or maybe you already do) to open the protocol up to 3rd party integrations. I use Home Assistant for automation, and Google Assistant can control Home Assistant. If I could do an integration for Home Assistant to MM5 (server I guess?) then I could indirectly control it from Google Assistant. Voice search of tracks may be hard in that case, but it could easily invoke named playlists by just naming automations. In fact turning playlist names into routines dynamically (which you can do in HA) would let Google index those and you could "Activate Playlist Christmas" for example, without Google having to index anything directly.

Anyway... anyone have thoughts on paths forward on this?

Linwood
Posts: 63
Joined: Sat Jul 13, 2019 6:49 pm

Re: Remote control, Voice control, options? Server?

Post by Linwood » Fri Nov 22, 2019 10:59 pm

I did discover that I can sort of control MM5 from Home Assistant, at least for casting, by building a script like this:

Code: Select all

play_mm5:
  alias: Play on mm5
  sequence:
  - data:
      entity_id: media_player.master_bedroom_mini
      media_content_id: http://my_mm5_ip_address:21628/ItemID=5864.flac
      media_content_type: audio/mp3
    service: media_player.play_media
  

If MM5 is running, chromecast will contact it and play the indicted track. It doesn't obtain any of the accompanying metadata (title, artist, cover art), and it is not really home assistant that is doing anything with MM5, it is Home Assistant telling Chrome to suck the track from MM5.

But offering it just in case anyone wants to experiment a bit.I thought it was cool when I could issue a command on Home Assistant (including by Google) and music started. Unfortunately this kind of script isn't parameterized, i.e. it would take a separate script for every song, times the number of media players. So I am not suggesting the approach is practical, just interesting.

PS. I got the URL by sniffing traffic, but I suspect it's pretty easy to get all of it including the item id programmatically.

Ludek
Posts: 3074
Joined: Fri Mar 09, 2007 9:00 am

Re: Remote control, Voice control, options? Server?

Post by Ludek » Mon Nov 25, 2019 6:36 am

Hi,
I guess this topic is rather for the addons developer forum, but note that entering http://my_mm5_ip_address:21628 (mm5 server port) into address bar of internet browser (like Chrome) allows you to access web version of MM5 -- i.e. MM5 is running in the internet browser directly ;-)
See: https://www.dropbox.com/s/ma4q4g3f84exe ... 4.png?dl=0

This way you can also sniff the commands for getting e.g. playlists list, tracks and all the metadata that you need in JSON format.
i.e. in the mode the MM5 UI is running in the browser and all the data are read from the MM5 server (as JSON batches) for the MM5 UI (HTML5/JS/CSS3) client.
MM5 client uses https POST request with input body like "data:|playlist:0|getChildren" (can been seen in the DbgView)

Let us know whether it is usable for your needs and feel free to ask for more info / description as you like (as the web version is rather an experiment ATM and isn't documented yet, and also it isn't actively maintained so not fully working yet).

Linwood
Posts: 63
Joined: Sat Jul 13, 2019 6:49 pm

Re: Remote control, Voice control, options? Server?

Post by Linwood » Mon Nov 25, 2019 8:21 am

(Feel free to move the topic if more appropriate).

That's interesting way to see how to get some of the data. I'll have to try it to see if I can sniff out the post commands that might do things like "play this playlist". And if it will even respond -- some brief experiments just ended up with my browser page hung.

I'd still need to do the integration on the smart home side (Home Assistant in my case) but that might be viable. Their media player commands do not look that complicated (or that capable -- basically just things like play this, pause, resume and volume).

But it becomes very cool when those get rolled up into other automation.

Now... if you could get the linux version of the client working, and finish up the MMS, I can get VLC out of the mix. :roll:

Linwood
Posts: 63
Joined: Sat Jul 13, 2019 6:49 pm

Re: Remote control, Voice control, options? Server?

Post by Linwood » Mon Nov 25, 2019 11:02 pm

If anyone is curious how neat this can be, here's a 1 minute video showing a player piano (and associated lights) being invoked from a voice command to google, which runs through Home Assistant, invokes VLC on a linux box running the piano to pull a song from Media Monkey as a DLNA server on a windows server (off in the middle of the house in a closet).

VLC also appears on the monitor on the piano so you can intercede there if needed, plus the monitor has a music reader program (display sheet music if you are playing yourself).

These kind of setups will become more common (not necessarily pianos of course) and is why I think MM5 would benefit from some remote control options (I haven't yet sniffed the window connection but will).

https://youtu.be/Wf-EY2eFK5I

OK, it won't win any oscars, just a demonstration. If you have a google assistant nearby the video does contain a few plays I say "Hey Google"...

Linwood

Ludek
Posts: 3074
Joined: Fri Mar 09, 2007 9:00 am

Re: Remote control, Voice control, options? Server?

Post by Ludek » Tue Nov 26, 2019 6:23 am

Hey, that's really cool!! ;-)

In which format is the music served by MM5 that it can recognize the tunes?
I guess that it is the "music reader program" on the piano that does the sheet/tune recognition and not the VLC, right?

Linwood
Posts: 63
Joined: Sat Jul 13, 2019 6:49 pm

Re: Remote control, Voice control, options? Server?

Post by Linwood » Tue Nov 26, 2019 9:15 am

I probably ran a couple things together...

The player piano can play analog music in a special form, the right channel has keystroke information modulated into an audio signal much like an old style modem squeal. The left channel can have a (mono) audio accompaniment played through a speaker under the piano. This is special music bought from the manufacturer (Pianodisc). It presents as a normal MP3 (or similar) file, so I can catalog and "play" it from any media program, so long as I can output it to the headphone type jack on the piano.

The sheet music reader display on the piano I put together with custom software to display PDF's from self-scanned sheet music. That mostly requires a human to read and play, though some I have run through music recognition software to turn it into a music score, then from there (via Musescore) to Midi. So while it can't happen on the fly, and requires a lot of manual cleanup, I can go from sheet music to an actual track that can play on the player system, though those midi files (and associated PDF so you can see the score) will not "fit" in MediaMonkey. I keep them in Calibre which is a bit more format-agnostic than audio server software with MM, and access them from my sheet reader software (Qt based on linux).

But the interesting part to me is being able now to make the piano part of the overall house system. When I tell it to play, lights change around the room and inside the piano, I can stop other music on other players (if any), power up the outlets with the player system and speaker, and start the music. All from a voice command.

But for the future what I'd love to see this simplified, one music server somewhere which can play to any renderer (the piano would be one, albeit a bit of a special one). Just no music player/server I've found really has all the pieces in one place (Plex was close but lacked a good player/renderer for linux). MM5 (or better MMS + MM5) is by far a better media manager, but lacks linux support (so far) and remote control.

Incidentally, by installing gmrender-resurrect on the linux box, I also have it as a DLNA renderer in and of itself (no player), so I can play straight from MM5 on another system to the piano, no VLC in the middle. I just can't remote control MM5. But this is handy for sitting at my desk and organizing music, playing a bit of a track to see if I like it, etc.

Sorry, I ramble. Just trying to give some encouragement for thinking "home automation" as you work on MM5/MMs.

Linwood
Posts: 63
Joined: Sat Jul 13, 2019 6:49 pm

Re: Remote control, Voice control, options? Server?

Post by Linwood » Tue Nov 26, 2019 9:21 am

Incidentally for those who play piano, and having nothing to do with MM5...

This setup also allows one to play manually and record the keystrokes as midi, then play it back so you can hear yourself. It's truly depressing, especially since with a midi player you can slow it down and listen to every mistake really slow and actually see yourself pressing the wrong keys. :cry:

But on the other side, with sheet music -> midi, you can also play and follow along on the sheet music, and slow the tempo WAY down so you can learn exactly how the timing works on complicated passages. Midi players (from sheet music) play without any expression really, it is not very artistic, but it is exactly and precisely the right timing and notes, so handy for learning.

That lack of artistic quality is why Pianodisc sells music -- it's a recording from a real pianist, not computer generated. There's a huge difference in quality - one is perfect in terms of notes and timing but dull, the other has feeling, even the occasional mistake.

Post Reply