In the modern world, analytics are an essential driver for almost all business decisions. Before you build anything, you need to first figure out how you’re going to measure your work and define what success would look like.
I’ve been asked numerous times about live streaming stats. What are the most important live streaming stats from a technical perspective? What metrics should I be thinking about? What should I monitor? How can I measure and evaluate my stream quality using quantitative data?
With so many people interested in the subject, I decided to do some deeper digging. Having spoken with numerous customers, partners and other streaming professionals about live streaming stats, here’s my takeaway.
Session Length is pretty straightforward and is the de facto metric for overall performance. If your Session Length drops dramatically, all other things being equal, your streams are having problems. To find the cause, you’ll have to dig deeper down into other metrics but, if you’re only going to track one metric, this is probably the single best one since it’s all encompassing.
How long does it take to load a video from user interaction (e.g. clicking play) to the first frame? This may or may not show up in Session Length depending on how you calculate your variables. If you see Loading Time spike, you know you’re having a problem, most likely related to your origin server or CDN. Alternatively if you shrink Loading Time, and everything else remains unchanged, your streams will almost certainly see increased engagement. A normal video Loading Time is pretty short, so this metric is very good at predicting whether users will leave the page without even starting a stream. With all of the effort that live streaming applications make to expand their user bases, nothing is more disheartening than acquiring new users, only to have them leave without using your service because they couldn’t load the stream.
What is the quality of the stream? Think about the difference between watching a sporting event on an old school 480p TV and a modern 4K one. Obviously these experiences exist at opposite ends of the viewing spectrum. With modern adaptive streaming and a full spectrum of available Bitrates, the importance of this performance metric has grown considerably. It’s no longer good enough to just deliver a stream, it needs to be a high Bitrate one. Watching something in Ultra HD is simply a better, clearer experience. Especially on mobile devices, the stream’s resolution can be the defining factor in a user’s viewing experience. In the future, expect users to become even pickier about the bitrate that they’re viewing live streams at.
And then, of course, there’s the dreaded Rebuffering. How many times have you been streaming a game and then, right as something important is about to happen, your stream stalls and starts to buffer? This is easily one of the most frustrating experiences that a viewer has to deal with. Unfortunately, it’s also an incredibly common occurrence. If you want happier viewers, minimize Rebuffering as much as possible.
There are so many other possible performance metrics, why did I end up with these four? The reason I like to focus on these four is that together they give you an overview and then the full cycle of your live stream. The overview is Session Length. Session Length is the simplest way to see if your users are engaged or if your stream is struggling from start to finish. Then Loading Time, Bitrate and Rebuffering show the streaming cycle in between the beginning and the end. Starting with Loading Time, then adding Bitrate and Rebuffering, they combine to cover the quality of the live stream throughout the whole session.
So how do we measure these metrics? For Session Length, Loading Time and Bitrate, you have numerous options:
Looking at averages is the most direct option. The mean is a simple and concise way to get a clear overview of how your live stream is performing for all of your users. Though the mean doesn’t show a great deal of detail, it makes up for that in its clarity.
This measurement option is very similar to the mean, but I slightly prefer median. In any large data set, you’ll have quite a few outliers. Assuming your streams have a large number of viewers, the median will represent a more accurate average experience than the mean.
- Percentile Ranges
Percentile-based ranges are my personal favorite measurement tool. They take a little bit more effort to create, but I think they’re worth it. I’ve seen numerous different ranges used, but I prefer the 25-75 interquartile range. This gives you a much better idea of your overall live streaming stats. Are you performing well enough? Are the ranges small or large? Even if the median statistic seems fine, a significant portion of your viewer base may be having issues. The interquartile range can highlight this. Some companies, looking for more advanced breakdowns, will track results at numerous other percentages along the spectrum, but I think the 25-75 range is a great place to start.
And lastly, to measure Rebuffering, I’m a fan of measuring Rebuffering Duration as a Percent of Total Session Duration and the Percent of Streams That Experience Rebuffering Instances. The former indicates the percent of your user’s viewing time that is spent Rebuffering. The latter shows how Rebuffering affects all viewers and whether it’s causing viewers to immediately exit the page. Though some people prefer other measurements, like the number of rebuffering events or total rebuffering time, I think these live streaming stats provide the most useful insights.
Of course, there are numerous other metrics that can be used to measure your video streams. Interesting statistics can include everything from gigabytes delivered or total viewership, to play rate or average and maximum concurrent viewership. The key is to figure out what matters most in your use case and then find the best way to test it.
Moral of the Story
Information is power. Without data, or even worse with bad data, the chance that your business makes good decisions shrinks considerably. As the old saying goes, “garbage in, garbage out.”
So, in my opinion, the best live streaming stats you can use to measure your video streams are:
- Session Length - Interquartile Range
- Loading Time - Interquartile Range
- Bitrate - Interquartile Range
- Rebuffering - Rebuffering Duration as a Percent of Total Session Duration & Percent of Streams that Experience Rebuffering Instances
Peer5’s Serverless CDN supports massively-scaled live and on demand video streaming. As the demand for ever increasing amounts of delivery capacity continues to grow unabated, no single CDN is able to provide all of the capacity, features, geographic coverage and uptime that broadcasters need. Hence, the growing momentum behind a Multi-CDN architecture and the load-balancing, redundancy and feature completeness advantages that this approach affords.
Our novel peer-to-peer (P2P) solution solves the peak demand problem by creating a network that actually gets stronger as viewership increases. This means that we perform our best at the exact moment when traditional CDNs struggle the most – making Peer5 the perfect complement to a broadcaster’s existing CDN infrastructure. All of our accounts come with Peer5’s own, in-depth analytics dashboard to help you better understand how your video streams are performing. Register for your own Peer5 account here.