The trouble with Google Hangouts...

The Trouble with Google Hangouts...

I have been an enthusiastic user and supporter of Google Hangouts since they were launched. I am one of the early members and admins of the popular G+ Tech And Coffee Community who pretty much started out because of Hangouts and, via that group, have participated in Hangout On Air shows  about technology.

I've also taken part in many other shows/conferences including VoIP Users Conference and others.
I also fairly regularly use standard (not "on air") public/private hangouts both for business and pleasure. I have played games, watched Youtube videos, discussed topics both serious and silly, demonstrated software, and had business meetings using video Hangouts. It is a great facility and, in my view, far surpasses tools like Skype.

I have also been an extensive user of Hangout (text) Chat as it's the primary communication mechanism used by many Ingress players. Some of the Hangouts I'm in have more than 80 members.

Although I like Hangouts, a few things about it bug me (and others I know) and I think Google are missing some tricks which would make Hangouts much more popular and useful.

A brief history

Hangouts started a few years ago as a feature of Google+. At that point it was a pure video-conferencing service (there was an accompanying chat application, but you needed to be part of the video conference to use it). It was revolutionary because it support multi-user video conferencing, and all you needed to use it was a decent web browser and a Google account. You could also use it on Android and iOS devices via an app. At the time it was probably the best cross-platform video conferencing/chat service and, arguable, still is.

Later, Google introduced a replacement for their popular Google Talk service, and named this "Hangouts" too, whilst semi-merging the video chat into it. Hangout chats are great as they are a much more modern chat system than Google Talk was, supporting more emoji, picture sharing, location sharing, selectable notifications and notification snoozing, cross-device synchronisation, and an optional searchable history. Hangouts also have probably the best system on any chat service for showing where other users have read to.

So, what's the problem?

I should point out that I'm focussing on functionality here. There are, from time to time, other problems such as app stability, bandwidth utilisation, useability and so on but I don't intend to discuss these here.

I'm also mainly focussing on text chat Hangouts, as I think video Hangouts are pretty good.

I think there are some very basic capabilities missing from chat Hangouts which limit its usefulness and which have pushed people to other platforms, such as Slack. I hear a lot of good things about Slack, for instance, but people tend to suggest it is fundamentally different from Hangouts suggesting it has more in common with IRC. I don't see that: I think Hangouts and Slack are more fundamentally similar to each other than they are to IRC, and with a few additions, Hangouts could do a lot of what can be done in Slack.

So, what do I think could be changed?

Fixed URLs

It would be great if you could fix the URL for a text Hangout to make it semi-permanent. In this way you could post the URL (for instance, in  G+ community) and people could opt to join rather than having to be invited (or re-invited). This would make Hangouts much more like IRC, where users could join and leave as they wish.

The dumb thing is, Video Hangouts do have this option. Text hangouts do not.

Moderation/Ownership

Text Hangouts are sorely missing any sort of moderation capability. Once someone has been invited to a Hangout, the only way to remove them is for them to leave voluntarily. This is a problem because one abusive member (or an unwanted outsider) can completely mess up a Hangout.

What happens in practice is a new Hangout is started, and all of the old members are invited to it except, of course, the troublemakers. This is awkward and very hit and miss when Hangouts can have up to a hundred participants. It's made worse because the original hangout still remains unless all of the users leave it, leaving the possibility of fragmented discussions when people cling onto the old hangout.

A number of things are needed here:

• Firstly, there is a need to have some sort of "ownership" of the Hangout. Ideally this should be shareable amongst several members who can then act as moderators
• There needs to be a way for a hangout admin/moderator to be able to kick and ban users from the Hangout
• There should be a way for an owner to suspend or delete a hangout, or archive it whilst making it inactive
• It should be possible to restrict the ability to invite new people to a hangout to owners/admins
• Ideally, there should be a way to restrict access to a hangout to a specific group of people. Some have suggested G+ Circles would be a way to do this, but I disagree. Circles are generally a way to capture social connections, not to enforce security. A better way, IMO, would be to allow Hangout membership to be restricted to a Google Group, which would also support integration with Google Drive and, of course, Groups itself.

For those not familiar, Google Groups allows a group of people to be associated with a single email address. The primary purpose of this is for mailing lists and collaborative inboxes, but Groups can also be used for access control for Google Drive, Google Sites and some other Google products. Using the same capability for Google Hangouts makes sense, and provides an integrated permissions management capability across multiple Google services.

Again, some of these capabilities are already available to Video Hangouts, but not to chat.

Support direct sharing of more stuff

Whilst it's possible to share links, pictures, screenshots, locations, and even Youtube videos within Chat hangouts, it would be good to support other types. For instance, it would be useful if it was easy to share a Google Doc from within a Hangout chat (there is support for this in Video Hangouts) and have it deal with the permissions automatically.

A public API

One of the huge benefits of Google Talk was that it used an open protocol (XMPP) which made it possible to integrate lots of different systems into it.

It would be beneficial to have some sort of public API to support the two-way integration of external services into chat Hangouts.

For instance, it would be great to have the ability to have IRC like "bots" to help with things like moderation, inline information services, and so on. You could have bots which collected external information (such as specific tweets, alarms from monitoring systems, etc.) and pushed them into the Hangout, bots which checked for bad language and ejected people from the Hangout if they use it, a bot which collects history to display on a scrolling blog widget, and so on.

It would also support the integration of chat into online games, as part of the Google Play Games service.

There are some API capabilities for video Hangouts, but not for chat Hangouts.

Sort out the Video Integration

Individually Hangouts Video and Hangout Chat are great products, but the way they are glued together is, frankly, a bit broken. As I've pointed out above, many of the capabilities supported in Video Hangouts are not available in chat hangouts, which makes the integration very lop-sided and limited. But there are other reasons to separate them.

For instance, if I create a Video Hangout, I automatically get an associated chat hangout whether I want it or not. Even if I leave the Video Hangout, the associated text hangout persists and I need to leave it separately. This leads to unwanted side effects. For instance, notifications are enabled by default so I get alerts as new people join the video hangout even if I left it some time ago. Depending on how they join the Video Hangout, I may even get a video call invite.

Conversely, if I create a new chat Hangout, there is always the option to do a video call, and at one point or another someone is bound to click that button (usually accidentally).

This is a common meme amongst Hangout users.

Video and Chat hangouts need to be separated. There should definitely be the option to integrate them, but it shouldn't be automatic. Perhaps as part of the moderation capabilities, it should be possible to say whether video calling is supported on a chat Hangout.

Conclusion

With the enhancements I describe, Hangouts could become a lot more useful and support a lot more use cases. For instance, some hangouts could work similar to IRC, with a public URL, public access, and moderation. Others could be private/secure with the option to remove unwanted members and prevent them from being added in the first place. It would support the capability for closed Hangout groups with much more integration into other Google services like Drive.

Hangouts could become the default in-game chat capability for online games (obviously subject to privacy controls). For instance, in Ingress if someone had linked their G+ account, and enabled unsolicited Hangout requests, it would be good to be able to launch a Hangout with them directly from the game.

What functional changes would you like to see in Hangouts?

Sonos UPnP Development - Let's play some music!

So far we have seen how to discover media devices, how to monitor what is being played on them, and how to pause a media device.

I present here a way to play something on a Sonos Zoneplayer (or, for that matter, any other media renderer).

The code here is based on the code in my post Sonos UPnP Development - Controlling Playback which has been modified so that, instead of pausing the Zoneplayers, it now tries to play a file on one of them.

Firstly, an important point to consider here is that, unlike UPnPClient5 which searched for, and addressed all of the zoneplayers, this version needs to target a specific zoneplayer. We, therefore need a way to specify this.

We also need to specify a file to play. Browsing your Sonos library, looking for a specific file, selecting it, and playing it is a subject in it's own right and, hopefully, I'll get around to covering it eventually. I intend to keep this simple so I'm going to play a URI from the Internet which points to an MP3 file. So our program needs a way for use to specify this file.

The code below include these enhancements:


Rather than hardcode this information, I have made them parameters so you call the program thus:
./UPnPClient7 <zoneplayer> <uri>

The code to get these parameters is mostly at lines 128 to 135. These are held in variables declared at lines 18 and 20 along with another variable, rendererFound, which is used as a flag for error reporting.

As before in UPnPClient5 we discover all of the media render devices and create a list of them. The callback function device_proxy_available_cb is used for this as before.

However, in this case we do a test for the media renderer name. This is at lines 98 to 103. Not we are only matching on the first few characters, so we don't have to provide the full name of the ZP. If we are not careful, we could match multiple ZPs, and in this case the program will play the file on all of them. Note this is NOT the same as linking the zones.

When a match is made, we call a new utility function play_stream which is defined at lines 38 to 86. This takes two parameters: a pointer to the renderer, and the URI.

The instruction to set the AVTransport to use this URI is at lines 68 to 72. This is a call to the AVTransport service "SetAVTransportURI", which does exactly as it says on the tin. This requires a few parameters:

• InstanceID - Always 0
• CurrentURI - Our URI
• CurrentURIMetadata - This describes the URI in more detail

The CurrentURIMetadata parameter is a string containing an encoded DIDL-Lite description of the file. I have hacked this into a couple of strings "metadata1" and "metadata2" and used strcat at lines 59 to 61 to create the final metadata string which contains our URL. This is a nasty, hacky way of doing this, but it works and it also exposes the raw data so you can see what it is.

We then set this as the CurrentURI for our AVTransport. If this is successful, we then instruct the renderer to play (lines 74 to 77).

Job done!

If I call this as follows:

./UPnPClient7 Office http://www.archive.org/download/jj2008-06-14.mk4/jj2008-06-14d1t03_64kb.mp3

I will have Jack Johnson start to play in my office.

Note that this is actually not added to the Sonos queue. It is played directly. Adding stuff to the Sonos Queue is something which involves some Sonos-specific capability. At the moment I'm concentrating on standard UPnP capability. I hope to cover the Sonos queue in a future article.

Sonos UPnP Development - Getting more information from the renderer

This next version is an evolution of the version in my previous post Sonos UPnP Development - Accessing the AV Transport service.  It has been extended to display some basic metadata for any tracks that are found.

The main changes from the previous version (UPnPClient4) are:

1.  A new callback function on_didl_object_available (lines 31-44) which prints out metadata for a track object including the full URL of the album art.
2. New code in the existing on_last_change callback which creates a parser for the XML track object (lines 86-96)
3. A change to the service subscription to indicate we want to get the "CurrentTrackMetaData" information (line 69).
4. I have also added the ability to specify the timeout on the command line (lines 155-158 & line 194). For brevity I haven't done any serious condition/error checking here.

If you run this program with no parameter it will default to run forever. You will need to force close it with CTRL-C. If you specify a parameter in seconds, it will run for that many seconds before exiting.

The way this works is similar to UPnPClient4, on which it is based. However, within the last_changed callback handler (which, as you recall, is called whenever the AVTransport service changes state) we request and process the track metadata.

The track metadata is contained in an XML document which looks like this:

If you uncomment the "g_print" on line 88 it will print out this XML document for you to see (it prints on a single line - I have formatted the above sample to make it easier to read).

Looking through this document you can see it contains quite a lot of information about the currently playing track. This is contained in an XML schema known as "DIDL-LIte" which is a cut-down version of DIDL. In UPnP AV this is used for a lot of things including content directories.

Luckily in our case the gupnp toolkit provides us with some nice capabilities to make it easy to parse this and extract information from it. The way this works is you needs to create a parser (line 58). When the parser is run it scans the DIDL document and signals every time it gets a new sub-object. In our case, the sub-object is a DIDL Item containing the track metadata. We need to create a callback function to handle this (which we have called on_didl_object_available and connect this to the parse before we execute it (line 84). We then run the parser and let it do it's thing.

In the parser callback handler on_didl_object_available (lines 30 -38) we receive the metadata object from the parser as a GUPnPDIDLLiteObject object. We can then use the gupnp convenience functions to extract specific data from this.

If you run this, you should get something similar to:

If you play tracks whilst the program is running, or if the track changes, the new details will print out.

Of particular note is the "Album Art:" information. The metadata returns a partial URL of the Album Art. This URL is relative to the Device URL. To get the full url we have to append the device URL to it (http://<ZP_IP_ADDRESS>:1400). This version of the code does this for us so that the URL output (highlighted above) can be cut and paste into the address bar of a web brower in order to view the album art.  If you are listening to Internet radio you should get the station logo for the station, if there is one.

The change in this updated version is achieved by pushing a pointer to the renderer object into the on_didl_object_available callback so that the callback code can reference it (the change here is made at line 89). Then in on_didl_object_available we dereference the renderer object and use it to look up the base URL (line 39) and then construct a new URL using the device base URL and the album art partial URL (line 40).

Clearly this basic capability could be extended to a number of uses, such as:

• Keep a GUI client up to date with current playing information
• Create a logging service to monitor what tracks had been played
• Create a "most played" or "recently played" playlist from the log
• Create a screensaver or media centre display based on what a specified zone was playing
• Look up artist information, artwork, or lyrics on the Internet and display them
• Push Album Art of currentply playing track to a UPnP TV or picture frame

As well as many more applications of this.

Sonos UPnP Development - Controlling Playback

In previous posts I have introduced the important concepts of UPnP Services, and that they offer Events and Actions. We have seen how it is possible to subscribe to an Event in order to be notified of when something interesting happens.

Now we are going to look at how we can use Actions to actually control something. This is a very simplistic example, but should illustrate the basic concepts.

The following code, when run, will do a "Pause All" of the renderer devices it finds on your network. This code is based mostly on the version from my previous post UPnP Discovery with Sonos, event driven:


The main changes to this version are that that we no longer print anything out. The device_info method which was used to print out device details in v3 has been replaced by a new function pause_device which pauses the device instead (lines 20-35). The key function call is on lines 32-34. This requires a little explanation.

The call takes multiple parameters including the service being addressed (in our case, the AVTransport service which we have previously obtained a reference to, the action requested, which "Pause", a flag to pass back en error and then a bunch of parameters. The parameters depending on the action being triggered, and this is within the device description document. In the case of AVTransport it's also within the UPnP specifications at upnp.org.

The following image shows how this looks when using a development tool such as Device Spy. The left column shows the Actions supported by AVTransport. I have selected "Pause" and the right column shows me the parameter this Action requires. It needs a single parameter called "InstanceID" of type "ui4".

Using Device Spy developer tool to examine the AVTransport interface

So in our call on line 32 (and according the the GUPnP documentation for "gupnp_service_proxy_send_action") we need to specify this parameter a as a "tuple" comprising the parameter name, the parameter type and the parameter value. In this case we want the first (and only) AVTransport control. The count starts at 0. We finish wth a "NULL" to indicate there are no more parameters.

This call is a simple one with no return parameters. If we had return parameters to deal with we would need to handle this differently.

Also, the function returns an error if the action failed. In our case it will fail if the "Pause" cannot be carried out. One reason for this would be that the devices is already paused. In this case we don't care about this so we ignore it. A more complete version might keep track of the transport state of the player, and only send a "Pause" action to those which were not already paused.

When run, this program does not give any output. It simply pauses every discovered player on the network. Clearly this could be used to connect to a home automation system, such that when the phone rang or the doorbell was pressed, all of the zones were paused.

Sonos UPnP Development - Accessing the AV Transport service

This is an iteration of the code I produced in my post UPnP Discovery with Sonos - Event Driven. Now we have discovered the devices on the network, we should try to do something useful with them. In this post I show how to pull some interesting information from them.

More specifically, I am going to reference the AVTransport service which is part of the MediaRender specification. This introduces some other another important concepts in UPnP: Services, Events and Actions.

Services

Functionality in UPnP devices is primarily exposed via services. Services are advertised on the network and can be discovered. The advertisement for the service includes the technical document which explains its capabilities. This document may be viewed using UPnP Device Spy, or in the case of standards based services like AV the specifications are also published at upnp.org.

Services expose two types of capability: actions and events.

Actions

If you want to perform some function on the device, you call an appropriate action. For a media player device, for example, one action might be to start playing the current track. We will look at actions in a later article.

Events

When a device does something that the rest of the world might want to know about, it can trigger an event. The events that a device may trigger are part of it's device specification document which can be viewed using a UPnP Development toolkit like Device Spy, or in the case of AVTRansport by looking at the UPnP AV specifications at upnp.org.

Anything else on the network that is interested can register it's interest in knowing about that event by subscribing to it. It may also unsubscribe if it is no longer interested. If it is subscribed to an event, it will receive an event every time the target device does the thing which causes the event.

For instance, n the case of a MediaRenderer object there is a service called AVTransport which represents the status and capability of the renderer transport.

In this article we are going to look at this and used the event called "last_changed" which is triggered whenever the Transport state changes.

The code below is based on the version in my last article with some modifications:

The first thing to notice is I have removed a bunch of code:

device_info as been removed and the code to print the hashtable out in main_loop_timeout has also gone. This code was really only there in the previous version so that you could see what was going on.

I have also removed the "Device added:" and "Device removed:" print statements from device_proxy_available_cb and device_proxy_unavailable_cb so none of the console messages that were in the last version are there any more.

The major additions here are as follows:

Firstly we have added a new callback function on_last_change (lines 30 to 75). This function is passed the UDN of the device as well as some data. The data is an XML document describing the transport state.

A couple of things are important here. Firstly we used the UDN (Unique Device Name) of the device as the key when we stored it in the GHashTable. This means we can use the UDN to quickly find the renderer object.

Secondly, the XML doucment needs to be parsed to pull useful information from it. Luckily GUPnP provide a set of libraries (gupnp-av) to help with this.

So, this function parses the "TransportState" information from the XML user data, looks up the renderer from our GHashTable using the UDN, and prints out the device and state information.

We then need to subscribe to the AVTransport service. We do this in the device_proxy_available_cb. Whenever this is called it represents a new renderer device being found. We do the following:

1. look up the AVTransport service (line 92)
2. register our interest in the "LastChange" event, and point this at our on_last_change callback function (line 97-101)
3. and turn on the subscription (line 102)

From then on, any time the renderer transport state changes, it will fire a "LastChanged" event at us which will run our little function.

A few other, minor changes are needed to support this:

• I have #included the gupnp-av headers (line 8)
• I have created a parser object and initialised it (lines 16 and 135)
• I have also moved the timeout value to a #define called "RUN_TIME" (lines 12 and 162) so it is easier to change and set it to 10 seconds

If you run this you should get something like this:

If you clear the queue you will get:


As you can see this starts to offer some interesting possibilities. You could build a GUI around this which not only listed all of the zones, but also listed whether they were currently playing or paused, all in real time.

You could also use this (with the appropriate hardware and drivers) to send on/off messages to an amplifier on a specific zone via Infrared (lirc) or a 12V trigger (parallel port?)

UPnP Discovery with Sonos, event driven

Following on from my post UPnP Discovery with Sonos Players I present a revised version which doesn't add much in terms of functionality, but which is a bit tidier and has some changes to how we handle discovery.

In the last versions, we simply printed out whatever we discovered. Processing discovery on-the-fly like this will not suit a lot of real-world programs, especially if it involves something like a GUI. What we need to do is to store the results of out discovery so that we can reference them whenever we want.

This raises an important point about discovery. UPnP discovery is not a "one shot" activity. It is a continuous processes. You start it, and it then runs continuously until you stop it. There is no magic flag that says "discovery has finished".  Normally the start and end of discovery is dictated by the start and end of your program. In my program, discovery (and the rest of the program) is stopped by me running a 2 second timer, after which time discovery is stopped and the program exits.

This is a very important concept to understand as, during the continuous discovery process devices can come and go from the network. Your program has to be able to accommodate devices appearing and disappearing. Of course, we are used to that with the Sonos controllers: if we add a new player to the system it magically appears in the zone menu. If you disconnect power from a Zoneplayer it (eventually) disappears.

So what we need to do in any long-lived UPnP program is use the discovery to maintain a local copy of what is currently on the network.

This next piece of code does that. It is based on the previous versions but with some significant changes. I am using a GHashTable from the Glib collections library. Once again this is common on Linux systems. In fact it's a dependency for GUPnP so it will be there if GUPnP is.

GHashTable gives us a nice key-value table with the ability to easily add and remove items.

Here is the code:


Firstly, I have added some new functions as follows:

device_info (lines 19-21) is a simple print utility function which, given a key value and a renderer reference, prints out some information about that renderer. This is used by main_loop_timeout (lines 26-34) which has been altered to print out the contents of the GHashTable. Remember, main_loop_timeout is called to signal the end of the program.

We already had the callback function device_proxy_available_cb (lines 40-50) but now we have added a new callback function device_proxy_unavailable_cb. These now respectively add or remove the discovered renderer device to the GHashTable. What this means is that at any time our GHashTable should contain a current list of the available devices in the network.

(Note that a device which has its power removed will not normally announce it's departure from the network, so this list is not 100% correct).

These callback functions also print data about the added or removed device to the screen, so you can see what it's doing.

To accommodate these changes, we need to make some simple changes to the main program. Firstly we need to create our empty GHashTable, which I have called renderers (line 79). Then we need to register our interest in the device-proxy-unavailable signal which occurs when a device leaves the network, and point it to use our new callback handler (lines 97-99).

I think the only other change worth mentioning is that I have made the MediaRenderer URN into a constant (line 10 and line 87) as it's better coding practice.

On running this, you should get something similar to the following:

The first block shows the devices first being discovered. Then the 2 second timeout kicks in and this prints out the current contents of the renderers list before stopping discovery. Note that before it stops discovery the system automatically tidies up for us, removing all of the discovered devices (the final block).

If you wish, change the timeout value at line 106 from 2 seconds to something longer if you want to make this more obvious.

Also note that, in this case, a new media renderer showed up on my list "Intel AV Renderer (laptop)". This is the Intel AV Renderer which is part of their UPnP developer toolkit. I ran this on my Windows laptop to show that this is a generic UPnP AV capability that's being used.

If you set the timeout to a longer value, you can launch and close the Intel AV renderer and see it being added to, and removed from the list of devices.

As you can imagine, this could be wrapped in some sort of GUI to give a constantly updating list of the Media Renderer devices on the network, similar to the Zone list on the Sonos controller.

UPnP Discovery with Sonos players

Following on from my previous article on Discovering UPnP Devices this post describes how to be a bit more specific about what we are searching for. Specifically, I will be searching for Sonos music streaming devices.

A crude way of doing this would be to alter the device_proxy_available_cb callback function with an if/then statement that selectively printed out device information based on the values in the model or Friendly Name information, but UPnP provides us with a better approach: When you initiate a discovery, your discovery request includes a specification of what you want to discover. Previously we used the specification upnp:rootdevice which discovered every device. We can limit this by saying we are only interested in certain types of devices.

UPnP devices have a hierarchy. Typically there is a root device which can contain one or more sub-devices. These sub-devices represent specific functionality. Sonos Zoneplayers contain several sub-devices which do different things, but in this case I am interested in the part which plays music. In UPnP terms this is a UPnP AV MediaRenderer.

Each UPnP device type has a URN which identifies what type of device or sub-device it is. The standard URNs for things like UPnP AV are described in the UPnP specification documents at www.upnp.org but these are also available directly from the device itself.

The easiest way to get these is by using a developers too like UPnP Device Spy. Note that Device Spy and similar tools are not "sniffers" as some uninformed people like to claim. They are developer tools. They discover devices and provide an easy way to query them for their capabilities, status and even to execute actions on them.

In this case, examining the UPnP documentation, or using Device Spy and looking at a Zoneplayer, reveals the URN for the UPnP AV Media Renderer is urn:schemas-upnp-org:device:MediaRenderer:1. So if we use that in our discovery, we will only get devices which match this specification returned.

The previous code I present discovered every UPnP device on the network. By modifying this slightly, we can make it so it only returns media player devices. This modified version is shown below with the lines that have changed highlighted:


As you can see, the change is very simple. If we now run this we will get something like the following:
A few things are noteworthy here:

• We only have Media Renderer devices. The firewall and the Sonos WD100 dock no longer show up
• The format of the output is slightly different. This is because we have been returned the MediaRenderer sub-device instead of the root device
• We no longer get the IP address. This isn't a problem as you really should never need it. If you really must have it (e.g. for display info), there are ways to find it.
• In this case we get the Zoneplayer zone name
• In the example above there are no non-Sonos MediaRenderer devices on the network. If there were they would show up in this list

So that is how to restrict UPnP discovery to a specific type of device.