DE.2006-12-27.00102 Photo by DigitalTribesThe 26th Chaos Communication Congress (26C3) is over and I could watch a lot of talks through video streaming. But one thing bothered me: The only type of stream which could be provided without major outages was the WMV stream. For most of the people watching the streams this was no big concern but for me it was a downturn as the open variant of the streams, served by Icecast2 and encoded in Theora were far from reliable!

Theora streams can easily be relayed by any server running Icecast2. I myself provided such a relay (and I'm doing it still for the unofficial FM4 OGG-Stream) and at times when the stream was available this server served around 100 people with live video and audio data.

At the 26C3 there were two relays (one provided by me, the other one by FEM) accessed through a simple DNS round robin record. While this is easy to set up I think it's not feasible for such scenarios where there is high demand for the streams. DNS-RR does not know anything about the load any of the participating server takes at the moment and it has no way of reliably determining the availability of the streams served by the servers.

Having some experience with Icecast2 I quickly wrote a simple HTTP load-balancer which is able to determine the status of each relay which serves a particular stream.

I came up with a quick solution in Python/Django, using PostgreSQL as the database backend. It currently allows for the creation of Streams, Servers and Mounts which together form a map of different servers providing identical streams. How it works: The end-user requesting the stream is given the URL for his stream which points to the HTTP load-balancer. Upon receiving a HTTP request for any of the configured streams the system replies with a HTTP 302 code redirecting the user to a server with free slots for the desired stream. In case no server is currently reachable or none of the defined mounts is present (in case the master-source went offline) it will reply with a HTTP error.

The free slots for each combination of server and mount are determined by querying a special XSLT stylesheet served by each Icecast2 relay. A sample output can be seen at my own server. It only contains the currently available streams and their free slots. This information is then polled by the server running the load-balancer and is written into it's database. I have to admit that the polling takes away the possibility to base balancing decisions upon live data but it has so far turned out to be effective enough even at a polling interval of one minute.

The load-balancer itself requires not much bandwidth (in comparison to the relays) and since it only serves a few requests (typically only one redirect) for each user it remains relatively lightweight when it comes to CPU and IO.

So far these media players have been confirmed to be able to handle the HTTP 302 redirect:

  • Totem/Gstreamer
  • Xine
  • Mplayer
  • VLC

Source code is available through SVN: http://subversion.fladi.at/opencompany/trunk/django-icecast-balancer/

A short introduction on how to set it all up will be included soon.