Transcript
[ Music ]
>> Hello. Welcome
to this session
on Validating HTTP Live Streams.
My name is Eryk.
I'm a media engineer at Apple.
Let's get started.
What are we doing when
we validate a stream?
We're trying to ensure
the stream is correct.
At least in terms
of its structure.
Structural checks can't find
all the potential problems,
but it will eliminate
a lot of them.
To be more precise, we first
want the playlist and segments
to be properly formatted.
That is, is the software going
to be able to parse the stream?
Second, we have some
additional requirements.
For example, the HLS spec does
not require I frame playlists.
But we want to require them,
because they improve the
experience for the user.
Third, we want these streams
to reflect established
best practices.
Ultimately, this
is all in service
of making sure the stream
plays well on your devices.
So let's talk about the
specifications we have.
The most important is the
HLS specification itself.
If you've been working with
HLS for any length of time,
you've probably read this
document a time or two.
It is an Internet
Engineering Taskforce --
IETF, internet draft.
You can find it just
by searching
on the web for "draft pantos".
The first version was
published seven years ago.
We updated two or
three times a year.
It details the structure
of playlists.
What kinds of tags
exists, and their format.
And explains the
responsibilities
for the clients and the server.
More recently we
introduced a new document.
More recently we
introduced a new document.
The HLS Authoring Specification.
This is available in
the reference library.
We first published it
last fall just for tvOS.
For WWDC, we've updated it to
cover iOS and OS X as well.
This document tries to cover
what constitutes well-structured
HLS content.
The requirements
in the authoring specification
come from several sources.
Some echo requirements
from the HLS specification.
For example, where
discontinuities should be.
What your playlist
duration should be.
Some have to do with
device support.
For example, that all your
video needs to be H.264.
Or that you can't have more than
60 frame per second content.
Others come from the
desired user experience.
For example, the I-Frame
intervals that we expect.
Or, the fact that we want
you to support trick play.
A number are informed
by the past seven years
A number are informed
by the past seven years
of our experience
and our analysis.
I'm talking about the
recommendations we have
around delivery.
The bitrates we recommend.
And our recommendations
around privacy and security.
The requirements I've listed
here are just some examples
to give you a flavor of
what's in the document.
The specification itself
is broken into categories.
Like video encoding.
Trick Play.
Media segmentation.
Delivery and so on.
Now of course we
don't expect you
to do this validation by hand.
So let's look at the
tools we provide.
But first, you need
to get the tools.
Everything important about
HLS can be found here
at the developer streaming page,
developer.apple.com/streaming.
Now if we scroll
down a bit, down --
halfway down on the
right, there is a link
to the "Tools" download.
Now please note you
will have to log
in with a developer
enabled Apple ID
in with a developer
enabled Apple ID
in order to access this page.
But remember that
developer accounts are free.
Now once you've done that, you
end up on the downloads page.
The download itself
is a disk image
which contains an
installer package.
Now the screenshot here
is a little out of date.
You should see a newer disk
image dated June 2016 or later.
We do typically update the
tools several times a year.
Now by the way, if you are
viewing this during WWDC,
the new version may not
have been posted yet.
After you've run the installer,
you end up with sever tools.
And I want to remind you that
these tools run only on OS X.
The first two -- the
validation are what I'll cover
in the rest of this talk.
The next three tools are
segmenters for files,
streams, and subtitles.
And I wanted to mention
as a quick side
that the subtitle segmenter
is useful for breaking
up the large web VTTT into a
set of segmented web VTT's.
up the large web VTTT into a
set of segmented web VTT's.
The last two tools, Variant
Playlist Creator is used
to create master playlists.
And the last one is used
to format timed metadata
that you can then insert
using the segmenters.
Now all of these tools
have manned pages.
And you can pass any of these
attached H option to get help.
But before we talk about
the validation tools,
I want to make a short pitch
for mediafilesegmenter.
It's simple to call, you
just pass it a movie file.
It produces a media
playlist and media segments.
While this tool has
a lot of options,
it's not as complicated
as it might seem.
The options break
into a few categories.
Options having to do
with naming files.
With structuring playlists.
How to do encryption and so on.
Even if you don't use
this introduction,
the mediafilesegmenter
can be a good cross check
the mediafilesegmenter
can be a good cross check
on whatever process
you do use to segment.
So on to validation.
Media stream validator
is the core tool.
You can call it with
your master playlist.
And it produces output on the
terminal and the json file.
Now this json contains
everything that was important
in the output, and
more data besides.
A couple of options
that I want to call out.
First of all you
can adjust the path
and the name of the json file.
And you can adjust the timeout.
Well, what's the timeout?
By default, the validator
times out.
That is, it stops
processing after five minutes.
Now this may not be enough time
to process a big video
on demand stream.
And for live content,
while five minutes is fine
for your initial checks,
you'll want to do some checks
over a much longer
period, once you determine
that the stream is
basically correct.
that the stream is
basically correct.
Here's what the media stream
validator output looks like.
So here we're calling media
stream validator with URL.
And first it tells you
the version number.
And then it outputs some trace
as it's downloading
playlists and files.
And then it outputs information
about every variant, rendition
and I-Frame variant
that it found.
Now this can be hard to digest.
So we created HLS report.
This is a python script
that takes the json
and summarizes and condenses it.
And it produces an
HTML file as output.
The name of that file
is the same as the name
of your json file, except
the extension is changed.
And again, there are some
options I want to call out.
We can change the
name of the HTML file.
The dash I or dash ID option
is helpful when you want
The dash I or dash ID option
is helpful when you want
to match HTML output to
portions of the json file.
And I'll give you some more
details about how that works
when I go through the examples.
Now the verbose option
is a very handy option
that gives you a
lot of information
about every variant
and rendition.
Unfortunately, I
don't have time to go
into detail about that output.
But I wanted to call it out as
something that is available.
One quick aside here.
You should validate your
streams before you submit
to the App Store.
And also, when you
submit to the App Store,
you should include a test
stream URL in your notes field
so that the reviewers have a
stream that they can look at.
I'd like to run some demos.
But rather than run all this
on the screen and have you try
to follow along and
squint at tiny text,
I've captured the output and
I'll guide you through it.
First, a little command
line sample.
First, a little command
line sample.
Run media stream validator.
And notice I'm giving the
name of the json file.
And I've just got a fake URL
here to give you the flavor.
Now we've seen the output
before, so I'll skip that.
But remember that this might
take several minutes to run.
Now once it's completed,
we can run HLS report.
Passing at the json that media
stream validator created.
If HLS report works well,
we don't have any output.
Just the HTML file.
So if we look in our
current directory,
we'll see there are
just two files around.
The json that media
stream validator created.
And the HTML file that
HLS report created.
So let's take a look at
some example reports.
First, I'm going to
start with a playlist
that has several common errors.
This is what a report
looks like.
This is what a report
looks like.
The first thing you want to
examine is the stream type.
Now usually, this is
VOD, or LIVE, or EVENT.
These reflect a three kind --
basic kinds of streams
that HLS supports.
But you could get
the value "UNKNOWN".
And this means that we weren't
able to read the playlists.
Or worse, you could
get the value mixed.
Now this can mean that
you had some unknowns.
That is, that some of the media
playlists couldn't be read.
Or worse, it could be that you
actually have conflicting types.
Like for example, some of your
variants claim to be EVENT,
and some claim to be LIVE.
Now the next item in the report
is the variant overview table.
It has one entry for
each video variant.
Now if we happened to have
our stream type as mixed,
that will be an added
column in this table
that will be an added
column in this table
that indicates the
playlist type.
If we look over to the right,
we can see that there
are more columns.
And in particular, if
it's a VOD we're going
to have a percent
processed column.
Now if this is less than 100%,
that means that the
timeout wasn't long enough.
Now if you have live content,
you won't see the
percent process.
Instead you'll see two columns.
Average segment count, and
average playlist duration.
Now the next thing you want
to check is the percent
difference column.
This shows you how much
greater the measured bitrate is
than the value that was
declared in the master playlist.
Now these values
are way too high.
Notice that variant one has a --
is 40% over what it should be.
Now these values
should be in the range
of minus 10% to positive 10%.
If the value here is positive,
that means your declared
value should be increased.
that means your declared
value should be increased.
And if the value is negative,
your declared value should
probably be decreased.
Now notice we've
actually got two of these.
Because in our playlist we
declared both the bandwidth
and the average bandwidth.
We are able to compare
the declared
and measured values for both.
Now sometimes looking
at this in terms
of percentage can
be a little hard
to see what the real
difference is.
So if you add the -- pl
option, meaning playlist,
to your HLS report command line,
you'll see two new columns.
Playlist rate.
And playlist average rate.
So with this, we can see that
for example, for variant one,
the declared value is
392 kilobits per second.
But the measured value was
551 kilobits per second.
Now let's switch back to a
version without the -p option
and scroll down a bit.
There's another table,
the rendition overview.
In this master playlist we have
audio as a separate stream.
That makes it a rendition,
and so it gets reported here.
Following this, we have
our must fix issues.
Now must fix issues are things
which are essential
for correct playback.
Or things which we
consider so important
that we made them
hard requirements.
You see item #2 and #4 here are
what we were just talking about.
The fact that the peak
and average bit rates,
that the declared and
measured values don't match up.
Now item 3 we'll address
in a later example.
Scrolling down farther, we now
have our should fix issues.
Should fix issues are things
which we strongly
encourage you to do.
Now these are followed lastly
by the report information.
The report information is
so that we can backtrack
The report information is
so that we can backtrack
to which version of
media stream validator
and HLS report was used.
This is important if you
pass us in a bug report,
the output from HLS report.
Or if you're even going back
and looking at something
after you've downloaded a more
recent version of the tools.
Now item 5 here is telling us
that we don't have any captions
or subtitles in our playlist.
We'll fix this in
our next example.
Let's look at item number 7.
So item number 7 says that our
default video variant is not
what we suggest.
And it's telling us
about variant #8.
So let's scroll back up
to the variant overview.
And we can see that item #8 has
the item number highlighted.
That's how we can tell
it's the default variant.
Now remember the default
variant is the variant
Now remember the default
variant is the variant
that the client's going
to play initially.
It's the first variant
in the master play list.
And notice that this one, the
max rate is 8 MB per second.
That's way too high
as a default.
Okay. Those are some common
problems that you might have.
In our second example, we've
taken the first example
and fixed the problems
I pointed out.
We've adjusted the
declared values,
so now the percent
difference is nearly zero.
We've fixed the default variant
to be something more reasonable
-- variant #4 in this case --
by moving a different media
play list to the first position
in the master playlist.
And if we look over on
the right-hand side,
we fixed our timeout, so
we're now processing the
entire playlist.
Now we had a should fix
item about captions.
We fixed that by adding
a subtitle playlist.
We fixed that by adding
a subtitle playlist.
Now notice as a subtitle's
rendition -- rendition #12 --
doesn't have a max
or average rate.
That's because the subtitle
playlist is not processed.
And we put an n/a over in
the percent processed column
to remind us that
this is in process.
But we still have
one big must fix,
and that's no I-Frame playlist.
So how does the report change
when we add I-Frame playlist in?
The first part looks the
same, but let's scroll down.
Now, in addition to the
other overview tables,
there's an I-Frame
variant overview.
Now notice we're a little off
on our percent difference
values here.
But these are well
within the range.
Remember, that's +/- 10%.
Now let's talk about these
two columns over here.
Scaled average and multiplier.
Scaled average is the affective
worst case average data rate
Scaled average is the affective
worst case average data rate
of the I-Frame playlist.
Yeah, that's kind of a mouthful.
Remember that the measured,
average bit rate
represents the data rate
at a 1 X playback speed.
But the whole point of I-Frames
of Trick Play is to play back
at greater than 1 X speed.
Now the multiplier tells
us the lowest playback rate
at which we will hit
that worst case value.
Higher rates are
going to be no worse,
because we will drop
frames in order
to keep the actual displayed
frame rate at a constant value.
What you should do with these
values is compare the scaled
average data rate here, against
the normal variant data rate
that has the same resolution.
These should be close in value,
or the I-Frame variant
should be lower in value.
Otherwise, Trick Play could
cause a drop in resolution.
Now in this report, I've
also got another new column.
Now in this report, I've
also got another new column.
That's because I added the
-- id option to HLS report.
Now as I mentioned before,
this helps you find the
same data in the json file.
So let's look for this
first one, data ID 16.
Which is our lowest
resolution I-Frame playlist.
Let's look for that in the json.
So here's an abbreviated
text view of the json.
The top level of the
json is a dictionary.
And we're interested
in the variance value.
Which is an array
of dictionaries.
Now one of those dictionaries
will have data ID 16.
And I want to note here
that these are not
in order by data ID.
Now this one with data ID 16,
that's the json variant entry
whose data matches the row
in the report.
That is the lowest
I-Frame variant.
Now, I've skipped
a lot of the data.
There's much more information
than this slide shows.
Use your favorite tool for
digging around json files,
or write yourself some
special purpose tools.
Now, these json's
can get very large.
It's not uncommon for them to
be over 100,000 lines in size.
For our last example,
we took that playlist
and we added multiple
kinds of audio to it.
So the top looks pretty similar.
Let's scroll down and
look at our renditions.
Now, rather than just
that single, stereo audio,
we've added two multi-channel
audio renditions as well.
One is AC-3 that
is Dolby Digital.
That one's called the aud2.
And one is EC-3,
Dolby Digital Plus.
One we call aud3.
So let's go up and look
at our variance section.
Now, notice that because AC-3
and EC-3 are not substitutable,
Now, notice that because AC-3
and EC-3 are not substitutable,
and neither are they
substitutable with AAC,
each one has its
own default variant.
That is, once we
choose aud2, the AC-3,
we will default to variant #12.
Because that variant is the
first one in the aud2 group
that appears in the
master playlist.
Similarly, if we chose EC-3, we
would default to variant #20.
Now notice there's
a bitrate difference
between the aud2
and aud3 variance.
For example, variant
#9 and variant #17.
This is due to the different
bitrates of the audio.
The bitrate you'll see here
for the variant is the
combined bitrate including
the renditions.
Okay. That's it for my examples.
Where can you find some
examples of your own to look at?
Well, there are three
examples pointed to off
Well, there are three
examples pointed to off
of the streaming page.
If you go to
developer.apple.com/streaming,
there's a pointer to
the examples page.
And you could dig the
URLs out of this page.
Or, you could just read
them on this slide.
The first two will have
complaints when you validate.
That's because these streams
were created a long time ago,
and we deliberately
decided not to upgrade them.
But that third variant
is a new one,
and should be almost perfect.
All right, it's time to wrap up.
Here are the only three things I
absolutely want you to remember.
Obey the authoring spec.
Validate, validate,
validate your streams.
And include a test stream
URL when you submit your app.
For more information,
go to the WWDC site,
and use this session's
number, 510.
There's a lot of cool
new stuff this year
There's a lot of cool
new stuff this year
in HTTP live streaming.
So I encourage you to
attend or watch the talk,
What's New in HTTP
Live Streaming.
Thank you, and have
a great conference.