Transcript
[ Music ]
>> Hello. Good morning,
everyone.
[ Applause ]
And welcome to this year's
session on HTP Light Streaming.
My name is Emil Andriescu.
Today's talk is about measuring
and optimizing HLS performance.
First, let's reflect for a
second on why we should care and
why it is essential for
application.
Let's pretend it's Saturday
night, you're in your favorite
spot on the couch, you've
skillfully browsed through all
the reviews, title is set,
popcorn is ready, and you
eagerly tap play when this
happens.
Faced with this mysteriously and
never-ending animation, you ask
yourself, what could be worse.
Let's face it, you know it, it's
a playback error.
But what do customers really
expect from HTP Light Streaming?
Well, they expect
high-definition image, high
fidelity sound, and instant
media response when they tap
play.
Yet, streaming applications over
the internet are always at the
mercy of the network, so how do
we reconcile?
Well, HTP Light Streaming was
designed to address this, that
is to provide the best possible
uninterrupted streaming
experience in an unpredictable
network environment.
So why are we here?
Well, there's more.
Over time, HLS has evolved into
a more dynamic ecosystem,
supporting new offering features
such as I-frame playback, new
media formats, and of course new
codecs.
At the same time, we're
constantly adding powerful iOS,
tvOS, and macOS APIs such as you
can tune and adjust playback to
your target audience and provide
a much richer user experience.
Delivery patterns and transport
protocols, they are also
evolving, so it is important to
look at your server side
performance in connection to how
content is being consumed,
either on a mobile device or in
the living room.
Given all these options, how can
you be sure that you are
providing the best possible user
experience to your audience?
Well, the first step is to
understand and quantify the user
experience in conjunction to
changes that you make to your
content, application, or
delivery.
This is an area where we believe
it is imperative to measure
rather than to guess which
configuration is optimal.
What is this session about?
First, we want to establish a
common language for discussing
streaming quality of service.
Second, we want to discuss how
to objectively measure your
application streaming
performance.
Third, we want to help you
identify and solve some of the
problems that impair streaming
quality of service.
And finally, we want to get
those master playlists right.
This is because many of the
problems and issues that we see
with streaming quality are
actually rooted in the authoring
of the master playlist.
Before going into detail, let's
begin with a brief overview of
an HLS playback session.
As you'd expect, it begins with
the download of a master
playlist.
Once the playlist is passed by
AV Player, it knows what content
it refers to.
In this case, we have two
bitrates, 1 megabit and 2
megabit.
AV Player will pick one of
these, will go ahead and
download a media playlist
together with additional
artifacts such as keys, and then
continue to download media
segments until the buffer level
is sufficient for playback.
When that happens, the AV Player
item will communicate a
prediction of playability by
setting the is playback likely
to keep up property to true.
If you've preset the AV Player
Rate to 1, so you're using the
Autoplay feature of a AV Player,
the player will go ahead and
start playback immediately.
We call this Time Interval
Startup Time.
From this point on, the wall
clock, also known as real time,
and the player item time base
will advance at the same speed,
with one condition, which is
that content must arrive at an
equivalent or faster rate than
that which AV Player is
consuming.
If that's not the case, AV
Player will try and switch down
to the 1 megabit here.
If network still cannot keep up
with real time at 1 megabit,
well the buffer will eventually
run dry, and AV Player has no
choice here, it needs to stop
playback, event which we call a
stall.
The player will remain in this
state not only until data starts
flowing again, but up until
there's a sufficient level of
buffer for the player item to
trigger another positive
playability prediction.
After that, playback can
continue normally.
Now let's discuss about
quantifying the user experience
for such a session.
We do that by defining a set of
Key Performance Indicators or
KPIs.
We picked five of them that we
believe are most representative
for HTP Light Streaming.
One question you may ask is how
much time do my users spend
waiting for playback to start.
Is it one second?
Is it five seconds or maybe 30
seconds?
This is an essential point in
terms of user experience.
Further, playback stalls, like
the one we just saw, they are
disruptive to the user.
We care both about how often
they occur, but maybe more
importantly, how long does it
take to recover from a stall?
Yet, the best strategy without
knowledge of the future to not
stall is to deliver content at
the lowest available bitrate.
But of course that's not what we
want.
We want to deliver the best
audio and video quality while
still not stalling.
So there's a clear tradeoff
between risk of stalling and
media quality, right.
For that, we need another
measure of the overall media
quality for a session.
And finally, playback errors.
We talked about that.
They are more disruptive than
stalls, right.
What can we do to track playback
errors?
Okay.
Let's begin with startup time.
There are multiple APIs that you
can use to obtain or compute
startup time.
First, don't use the AV Player
status changing to ready to
play.
That doesn't tell you that
playback is going to start.
However, if you are using
Autoplay, so you're setting the
rate, the player's rate in
advance, you can use the AV
Player item status changing to
ready to play or the AV Player
item is playback likely to keep
out changing to true.
These are observable properties.
When that happens, you know that
playback is about to start, but
there might be a few
milliseconds before playback
actually starts.
So what we recommend is to
either use the AV Player time
control status changing to
playing or to track the player
item time base, and there's a
notification that allows you to
do that.
AV Player relies on heuristics
to avoid stalls, but we know it,
sometimes they're unavoidable.
You can monitor as stalls occur
by registering to the AV Player
item playback stall
notification.
The suggestion here is to count
the occurrence of stalls.
Of course, if you want to
compare and aggregate stall
behavior across sessions of
different duration, then you
need to normalize this.
How do you do that?
Well, we recommend that you use
the total duration watched and
compute the stall rate in terms
of stalls per unit of time
watched, such as stalls per
hour.
A stall of 30 seconds is much
worse to the user than a stall
of one second.
This is why we also care about
rebuffering time or stall
duration.
By measuring the time interval
between playback stalled
notification and when the player
item time base changes back to
1, you can compute an accurate
stall duration.
Again, the total duration can be
normalized using the duration
watched of the session.
Well, you might be wondering at
this point, how do I compute the
duration watched of a session?
And the answer is, through the
Access Log.
Let's see how we do that.
So this is a snippet of code.
First, we get a reference to the
Access Log from the player item.
We iterate through the events in
the Access Log, and we simply
sum up each events duration
watch.
And there you have it.
We computed a total duration
watch for a session.
And now you may be wondering,
well what is this event?
What is an event in the access
log mean?
Well, for that let's look at how
AV Player Item Access Log works.
So the AV Player Item Access Log
provides a history of your
session.
It is initially null, but as
playback occurs, you're going to
receive an AV Player Item New
Access Log Entry notification,
and by that time, you'll have an
Access Log.
You'll see that events in the
Access Log contain information
on various areas such as the
current variant URL, the current
bitrate, duration watch, number
of stalls, and so on.
These values are initially by
convention initialized to a
negative value or null.
As playback occurs, they are
updated with actual measurement
data and the actual variant URL
that you're playing.
There are two cases in which we
will add new events to the
Access Log, and that is variant
switch, like in this case, or a
playback seek.
But before a new event is added,
the old one becomes immutable,
and then we add the new event.
Now, keep in mind that while
these values here are constantly
updating as playback occurs, so
the values in the last event,
they are not observable
properties.
We also mentioned that we care
about media quality.
How do we compute that?
A way to measure if the user is
presented with the best possible
media quality is of course to
look at the video bitrate being
served.
Here we don't really care about
the startup time or the stall
duration, so let's remove those.
So we're left with the playback
state.
In this example, we see that we
played for a longer time on the
2 megabit variant and less time
at 1 megabit.
By time weighting each bitrate,
we can obtain a single value of
video quality that we can
compare across sessions.
We call this measure a
Time-Weighted Indicated Bitrate,
and computing it is just as
simple as with the total
duration.
Once again, we get a reference
to the Player Items Access Log.
We iterate through the events in
the log.
We compute the time weight of
each event with respect to the
total duration watch we computed
earlier, and finally, we sum up
the weighted bitrate value.
Now keep in mind that some of
these properties may not be
initialized, so do the
appropriate checks in your code.
Another event which you must
absolutely track is of course
playback failure.
To do that, you observe the AV
Player item status.
If the value ever changes to
false, it means AV Player
encountered an unrecoverable
error.
A good way to transform this
observation into a KPI?
Well, one way to do it is to
look at the percentage of failed
sessions with respect to total
sessions, but there might be
other ways to do it.
One thing I want to stress here
is that not all errors in your
stream may be fatal.
Some may impact media quality
while some might not even be
perceivable by the user.
But nonetheless, if there are
errors, they convey that there
is an issue with your stream.
So how do I get more insights on
the stream, right, what
happened?
And the answer is from the
Player Item Error Log.
The AV Player Item Error Log.
The Error Log conveys failures
with varying degrees of user
impact.
It works in a similar fashion as
the Access Log except that
events represent error rather
than player access states.
They cover various areas, such
as delivery issues, network
issues, content authoring
errors, and so on.
For instance, they can give you
an insight on why a stall
occurred, such as no response
for a media file for about ten
seconds.
So we talked about startup time
that you can track for every
session.
We encourage you to take a look
at the distribution of startup
times for your application.
We also talked about stall
occurrence and stall duration.
We mentioned that Time-Weighted
Indicated Bitrate is a good
indication of experienced media
quality across a session, and
finally, you probably want to
keep the percentage of failed
sessions as low as possible.
Keep in mind that not all KPIs
are comparable across sessions.
One example of that is that AV
Player foundation uses the AV
Player layer size on the screen
to evaluate illegible variants
for HLS.
So for instance, if you've got
10 ATP content, it will probably
not be displayed on a 200 pixel
view, but it doesn't mean the
user experienced poor image
quality.
What to do then?
We recommend that you gather
additional context information
along with your streaming
metrics.
This will allow you to later
partition your playback sessions
in classes that make sense for
your application.
Sample code for this section is
available on the Apple developer
website as part of the HLS
catalogue sample.
Now, please let me welcome
Zhenheng Li, who will talk to
you about ways to improve HLS
performance.
Thank you.
[ Applause ]
>> Thank you, Emil.
Hello, everyone.
My name is Zhenheng.
We have discussed all the KPIs
that our users care the most.
In this part of talk, let's
focus on ways to improve these
APIs.
We will look deeper in three
areas.
One, how to reduce the startup
time.
Two, how to investigate and
avoid stalls.
Three, how to investigate and
avoid errors.
Let's get started.
So what can delay start of
playback?
Here is an example of the
operations from the user clicks
play until the video start to
play back.
The application create every
asset and start inspection of
the asset to find out durations
and awardable media options of
the asset.
It takes a few round trip
between the device and the
content server to download the
master playlist and [inaudible]
playlist.
After that, the application
create AV Player and AV Player
Item.
Buffering starts.
Oftentime, buffering is
interrupted, content is
encrypted.
It takes a few round trips
between the device and
[inaudible] to fetch the
decryption keys.
Once the keys are fetched,
buffering resumes.
However, it may be interrupted
again.
Let's say the application offers
a feature, resumes from the
previously watched point.
Application sets a sic time, a
set [inaudible] time on the
player on behalf of the user.
Every player discard the
existing buffer and start
download from a new location.
Segment 100.
Again, it maybe interrupted.
Users has a language preference
setting in the application.
She or he prefers Spanish audio.
Thus, application sets media
selection on the player item,
existing audio buffer being
discarded, player start
downloading from a different
language variant.
In a few seconds later, player
item notifies playback is like
to keep up, application sets a
rate.
Playback starts, and it
continues.
All this time, user is waiting.
So as we can see, it does take a
few time-consuming operations to
start up, run a trip between the
device and the content server
and the key servers.
Round trip times between AV
Player and applications,
oftentimes these two sit at
different processes.
So how the application measures
the time cost and startup time?
It may measure the time spent
between the API calls and the
player and the Player Item
status change notifications.
Every player item also offers
startup time in the Access Log.
This time is measured by the AV
Player item, represents the time
for buffering only.
It's measured from the start
media downloading until the
first playback is selected to
keep up.
So our user wants the video to
start fast, in at most a few
seconds.
There are ways to achieve that.
One option, we can move some
operations to a different stage
before the user clicks play.
For example, AV Asset creation
and inspection can be moved out.
Key fetching can be moved out.
Thus, when the users starts a
video playback, there is less
waiting time.
So where do we move those
operations to?
While your user is viewing the
video catalogue or video info,
it's a good time to create an
inspect AV Asset before the user
decides to play.
Now, last year we had introduced
AV Content Key Session API.
This new API decouples the media
load from key fetching.
It gives the application total
control on key management.
It offers ways to optimize key
fetching, such as bundling up
multiple key requests back to
the key server.
If you happen to adopt AV
Content Key Session, spending a
few hours of engineering hours,
your user will notice a faster
startup time.
So we have moved the AV Asset
creation and key fetching out of
startup time.
Now what's left is mainly the AV
Player Item buffering time and
the communication time from AV
Player and your application.
Oftentime, app may be able to
avoid buffering, such as due to
[inaudible] or due to media
options.
We can even try to reduce the
round trip time between the
player and the application.
Thus the startup is further
reduced.
Let's take a look.
When you create AV Player Item,
if you know where your user is
intending to start the playback,
set the current time on player
item.
If you know what are the media
options such as which language
to download for playback, set
that as well before you set the
AV Player item onto the player.
Same with the AV plyer.
As soon as the user click play,
set rate before the start
downloading for the AV Player
Item.
Thus, the playback will start
automatically as soon as Player
Item has enough to play back.
In summary, set up AV Player
before buffering.
Set AV Player rate before
setting the player item onto the
player.
A lot of application offers a
feature to allow the user choose
multiple videos and play one
after another, such as binge
watching TV episodes.
We have seen implementation such
as one player and one player
item per video.
There's always a startup
buffering time for each new
video.
You may reduce that buffering
time for the new video by AV
Queue Player.
Create multiple player items,
include them all on the play
queue.
While the player is playing the
current item, when the media
download finishes for the
current item, player will start
downloading for the next one
while the current one is still
playing.
Thus, the next player item will
start playback as soon as
current event play to the end.
So do use AV Queue Player to
play multiple items and enqueue
second AV Player Item well in
advance.
So what's left now?
Buffering time.
First, what determine network
buffering time.
Four factors.
The choice of your variant, the
content bitrate, your playlist
target duration, and of course,
last, the network bandwidth.
Let's take a look a few examples
of buffering time.
First, it's a simple master
playlist.
It specifies an ATP video at
about 5 mbps.
Let's assume the network
bandwidth is around 6 mbps.
Our target duration is 10
seconds.
In most of the cases, player
item buffers one segment before
it notifies playback it like to
keep up.
However, the same master
playlist, almost the same
network condition, the user may
observe slower startup.
The reason is, remember, the
network bandwidths change, and
the content bitrate also change.
In this case, there are a few
segments take longer to
download.
Thus, it takes longer to start.
To solve this problem, offering
a variant with lower bitrate may
help.
Player may decide to switch down
and start up sooner.
When all other information is
absent, the first listed variant
will be your startup variant.
So in this example, same two
variants.
The lower bitrate is listed
first with same network
condition.
Player will start faster, start
up faster and also switch up
pretty quickly given the network
bandwidth is sufficient for
playback.
In summary, to reduce network
buffering time, make a wise
choice of initial variant.
Lower content bitrate means
shorter buffering time, but it
is a tradeoff of video quality.
If you are offering multiple
media formats such as HDR and
SDR videos or stereo audio and
multiple-channel audios, make
sure the initial variant for
each media format are on similar
level of bitrate so your user
will have a similar experience
regardless what kind of viewing
setup they have.
That's all about reduce startup
time.
Our video has started.
Next, let's talk about stalls.
To be really clear, stalls can
happen, especially when the
network bandwidth is really low.
So in this part of talk, let's
focus on how to investigate
stalls and how to improve or
avoid stalls.
How the application investigate
stalls.
The application should be
listening to the stall
notification at all time.
And the application should be
also checking the AV Player
status such as is playback
likely to keep up.
AV Player Item also offers Error
Log and Access Logs.
The application should be
listening to an exam those logs
when the stall happens.
Next, let's take a look two
stall examples.
First, stall notification has
been received by the
application.
The application should have
received the Error Log as well.
The error comments give you
detailed information on what has
happened.
In this case, it says media file
not received in 15 seconds.
Application checks Access Log to
find out what the AV Player was
playing at the moment when the
stall happened.
It tells you detailed
information such as the player
was playing what content and
such URI.
The indicated bitrate is the
content bitrate.
In this case, 36 mbps, and that
is a [inaudible] content.
An observed bitrate is the
current network bandwidth.
In this case is 2.8 mbps.
It's obviously due to the
network bandwidth can't catch up
with the content bitrate.
So to deal with variable
networks, remember to provide a
full set of bitrate.
Remember some of your users may
have a slower network
connection, or your user may be
on the go, such as on cellular
while viewing the video.
If you're offering multiple
video, multiple media formats,
each codec combination needs
it's own set of tiers.
Not all stalls are due to
network condition.
Let's look at this one.
Stall happened, Error Log tells
you a different story this time.
It says playlist file unchanged
for two consecutive reads.
If you check the Access Log at
the time, player was playing
live.
They indicated the bitrate is
rather low.
The content is about 400K, and
the network bandwidth is 3.7
mbps.
This look like a content
delivery issue.
So to reduce or to avoid stalls
due to content delivery, content
server and CDN must deliver
media files, segments, keys
without any delay.
Update live playlist at least
every target duration.
The CDN [inaudible] must be
configured to deliver most
recent playlist to avoid stale
playlists.
Synchronized discontinuity
sequence number between
playlist.
Indicate server-side failure
clearly using right HTTP status
code.
That's all about stall.
What about error?
How do we investigate errors?
There are a few ways.
We have Error Log and Access Log
from AV Player Item.
We also have error property from
every player and player item
that the application can
observe.
In addition, we have some media
validation tools for you to
detect the content issue.
Let's look at them one by one.
AV Player Item Error Log, they
have talked a little bit about
[inaudible] in this one.
This type of Error Log is an
indication that there is a
problem with network or content
format.
However, they are not always
fatal.
When the error is indicated,
playback may be perfectly fine
at that moment.
However, the application showed
the check in the error comments
to find out details, such as
this one.
We have seen it before, media
file not received in 15 seconds.
So it's an indication that your
user may have observed or will
observe stalls.
Now next one is HTTP error, it
says file not found.
This an indication of a content
delivery issue.
The user may observe audio loss,
video loss, or both.
[inaudible] specified bandwidth
for variant.
Now this is an interesting one.
It's an indication of a stall
risk.
However, the playback may be
perfectly fine when the error is
indicated.
It means some of the segments
bitrate is higher than what is
specified in the master
playlist.
Last example, crypto format
error, unsupported crypto
format.
This may be an indication of a
failure, a playback failure.
All this error message and a few
more that are not talked about
here are very helpful when we
have AV Player and Player Item
errors.
Let's take a look.
The application should be
observing AV Player Item status
and AV Player Item error
property to find out this type
of error.
These errors are fatal errors.
When the error indicated
playback has been terminated
already, so what should we do?
How do we find out the cause?
Here is example.
The application is observing
player item status when the
status changed to failed.
Application go off to check the
AV Player error properties as
well as the Error Log from the
AV Player Item.
Here is the error property from
the player item.
It provides some useful
information.
Error code from AV foundation
error domain.
It also provides some hint, go
off and check the Error Log from
AV Player Item.
So corresponding AV Player Error
Log gives you much more details.
It tells you on this data and
the time and what URI with what
type of error.
So in this case, it's
unsupported crypto format.
It also tells you what type of
network interface the device was
on when the error happens.
Next type of error, HDCP.
If you are offering content that
requires HDCP protection, your
application should be observing
this long property name,
property.
It's output obscured due to
insufficient external
protection.
The value of this property
changes to two means three
things.
Current item requires external
protection.
Device does not meet the
protection level.
User will observe or is already
observing video loss, like
through [inaudible] for example.
To avoid this issue, your master
playlist should offer at least
one variant that does not
require HDCP for fallback.
Remember, not all your users has
the viewing setup that is HDCP
capable.
App user interface should
reflect the property change to
timely hint the user.
A lot of playback issues are
introduced by content authoring
such as audio and video out of
syncope or glitches while
[inaudible] switching.
In addition to the error
investigation and handling that
we have talked about, we would
encourage you to use our media
stream validator too, which is
available on the developer
website.
That's all I want to talk about
it today.
Now let's welcome my colleague,
Eryk Vershen, to talk about how
to author the master playlist
the correct way.
Thank you.
[ Applause ]
>> Thanks, Zhenheng.
My name's Eryk Vershen.
I'm an engineer working on HLS
Tools.
We've spoken about how to
measure your performance and
also how to address many of
those concerns.
However, one of the key elements
to successful and error-free
playback experience is to ensure
that your master playlist is
authored correctly.
The master playlist is what
allows the player to make
intelligent decisions both
before and during playback.
So getting it right is critical.
There we go.
That's my advice.
No, I'm just kidding.
I think I need to give you a
little more background to
understand what Roger meant.
We want you to put all of the
encoding options you have into
your master playlist and to
describe them as completely as
possible.
Let's pretend you're asking me
questions.
This is the crucial question and
the main thing you have to get
right.
Now, first you have to remember
that just because a master
playlist works doesn't mean it's
right.
I've actually seen master
playlists that look remarkably
like this.
This is technically legal, and
it's next to useless.
I say, okay, well how about this
one?
It has a few more variants.
Well, it's a little bit better,
but it's still terrible.
Can we even play this?
What codec is it using?
Is it HDR?
Is it 60 fps?
You need to tell us everything.
We want you to tell us
everything.
For example, average bandwidth.
Average bandwidth enables us to
make better decisions about
which variant to switch to.
It's a better predictor or
whether we'll be able to keep up
with a stream.
Codecs is what enables us to
filter out things that we can't
play, and resolution allows us
to make good decisions about
which variant to choose.
Remember, we don't read the
media playlists or the media
segments until we have to.
So you need to tell us things
ahead of time in your master
playlist.
So here's a sample of a simple
master playlist.
This playlist allows the player
to adapt to bandwidth changes
and make good choices about
which variant to use.
Now, everything that we've done
here is invisible to the user.
It just makes the stream play
better.
Okay.
Let's look at a common problem.
Your stream plays, but you're
not seeing any images in fast
forward, or your not seeing a
thumbnail in the scrubber bar.
Here's the Apple TV scrubber
bar.
You can see how long your
content is.
You can see where you are in the
content, where you want to go.
Now, in order to get that
thumbnail image, you need to
give us an I-frame playlist, and
the I-frame playlist is also
what allows us to provide images
in fast forward and reverse
playback on your iPad or your
iPhone.
Now in order to talk about
I-frame playlist, we first need
to talk just for a moment about
normal video.
Now, here's a way of visualizing
regular video segments in HLS.
Each segment has content for a
number of frames, so it has a
duration in frames, and it has a
particular average bitrate, and
that bitrate varies from segment
to segment.
Now, because of compression
techniques, most frames in a
video can only be decoded
relative to other frames.
But I-frames, the I stands for
intercoded frames, these are
frames that are independently
decodable, and they're the base
frames that allow everything
else to be decoded.
Now, as I've shown you here, you
might have more than one I-frame
in a single segment, and the
I-frames need not be in a, occur
at regular intervals.
An I-frame playlist is a
playlist which just points to
the I-frame content, that is
only the I-frame data will be
downloaded.
And when we talk about the
duration of an I-frame, we
always mean the time from that
I-frame till the next I-frame.
Now, this particular set of
I-frames, I've shown as
extracted from my normal
content, but you can also make
what we call a high-density
I-frame playlist.
This isn't something just
extracted from your normal
content.
Instead, you make it
deliberately with more evenly
spaced I-frames.
This will allow us to work
better.
It allows us to give a much
smoother result when you're fast
forwarding.
Now, here I'm showing you a
master playlist without I-frame
playlist added.
Now, notice that the I-frame
playlist has almost exactly the
same tags as the normal
playlist.
The only difference is the
I-frame playlist does not
support the frame rate attribute
because it doesn't make any
sense in that context.
Now, a good test for your
I-frame playlist is to try and
play it directly.
That is, take the URI of your
I-frame playlist and paste it
into Safari.
It should play at 1X speed, and
you should see the I-frames
displayed one after another in a
slowly changing sequence.
Now also I want to point out the
difference in the bitrate.
Notice that the I-frame bitrate
is much lower than the normal
bitrate.
That should always be the case.
Now, speaking of bitrates, we've
defined how to compute the peak
bitrate in the HLS
specification.
Make sure you do it that way.
Otherwise, you may get that
segment exceeds playlist,
exceeds specified bandwidth
error.
Now, we're going to move away
from video and talk about audio
for a little bit.
Now the most common question is
how do I support multiple
languages?
Here's what the interface looks
like.
I've got a list of languages,
and the user can select one.
And here's a sample playlist.
Now, notice what we did is we've
added an audio tag, sorry, an
audio attribute onto each of our
video variants, and we've added
a group, the media tags with
group ID's.
The group ID is simply a tag
that allows you to associate the
audio renditions with the video
variants.
So notice there are a number of
differences between the two
audio renditions.
Just as with your variants, we
want you to tell us as much as
you can about your media.
Now, there are two attributes
that people tend to have trouble
with on the media tags, and
that's Default and Autoselect.
Okay.
So Autoselect says that the
media selection code in the
player is allowed to choose this
rendition without any special
input from the user.
Most of the time, you want
Autoselect set to yes.
If you don't set this, the
user's going to have to make an
explicit choice to get that
rendition.
The default on the other hand is
what to pick when the user has
not given a preferred language.
Generally this should be the
original language of your video,
and the default must be
autoselectable because the
system has to do the choosing.
Now, this default has nothing to
do with the default video
variant.
This is the default within the
group of renditions.
So, okay, great.
I've got multiple language, but
I'd really like to have some
multichannel audio.
I've got 5.1.
Okay.
Well the first thing to remember
is not all devices can play
multichannel audio.
So you want to also provide the
user with a stereo option.
And you should think of this
always as filling out a matrix.
You need to have every format
having every language.
You may say, well, I don't have
a multichannel original for my
French.
I don't have a 5.1 French.
In that case what you should do
is put stereo, your stereo
content in that group instead.
You need to have something in
every slot of this matrix.
So let's see a sample playlist
again.
This one is just like the
previous example except I've
changed the group ID, and
remember that's perfectly fine
because the group ID just serves
to connect the audio renditions
with the video variants.
Now, here we had the
multichannel group, and then
I've set this up with French as
stereo, so you can see how
that's done.
What you need to do is make sure
that the codecs tag indicates
all the different codecs that
can occur within that rendition
group.
Now, notice that we had to
duplicate our video variant.
So now we've got two entries,
one pointing to one audio group,
and the other pointing to the
other audio group.
And you'll see this kind of
duplication again in later
slides.
Well let's say rather than 5.1,
I've got several audio bitrates.
I've got some high bitrate
audio, and I know I need to
provide a low bitrate for some
users.
So in terms of the renditions,
this is similar to what we had
before.
We still got a matrix.
We want to fill it out with
every language for every
bitrate.
And since these are both AAC,
they're considered the same
format.
So if I also want to have
another format, all I do is
extend that matrix.
Now, I want to mention that I've
been saying language for
convenience, but you should
remember that it's the name
attribute which is the unique
attribute, not the language
attribute.
Now, in this playlist I'm not
going to show you the media
tags.
I'm just going to show you the
video variants with their audio
group names.
Now you want your low bitrate
video associated with your low
bitrate audio, and you want you
high bitrate video associated
with your high bitrate audio.
And you'll always want to split
this up like this.
Don't do a situation where you
have a complete set of video
variants associated with your
low bitrate audio and a complete
set of variants associated with
your high bitrate audio.
Because if you do that, you can
be at a high video bitrate and
be bouncing between high and low
audio bitrates.
Now, here I've added in the AC3
content.
Notice that again we had to
duplicate our video variant
entries, but they point to the
same video playlist.
Now, notice also that the
bitrate on the video variants
changes.
Remember that's because the
video, I'm sorry, the bitrate
associated with the video
variant is the bitrate of the
video itself plus any associated
renditions.
Now, let's go back to the video
for just a second because I want
to have multiple video formats.
I like to have HEVC, so I can
have better quality at the same
bitrate or I'd like to have
Dolby Vision so I can have some
HDR content.
Again, we're kind of filling out
a matrix.
In this case, no matter which
video format we choose, we want
to end up with a reasonable set
of variants.
So the rows here are tiers based
on quality, and we want to fill
out the matrix with a variant in
each tier in each format.
Now, we don't have to
necessarily fill out the higher
portions of the tiers on our
older formats.
You can skimp a little bit
there.
But similar to audio, not every
device supports things like
Dolby Vision, so you want to
provide an H.264 variant as a
fallback.
The main thing to remember is
that in each column you want to
have the bitrate form a nice
progression.
Now, this playlist has gotten a
little too big to show on one
slide, so I'm going to split it
over three slides.
This one shows you the H.264
variant.
On this slide, we have the HEVC
variant.
Now, notice everything has a
different video playlist that
it's pointing to, and here's our
Dolby Vision variant, and notice
that everything here has had the
same audio group.
So, again, if we wanted to have
multiple audio formats, we would
need to replicate the video
variants for each audio format.
And again this wouldn't increase
the number of video playlists we
had to have.
It would just increase the
number of entries that we had in
the playlist.
Okay.
We're almost done.
Our last bit is about subtitles
and closed captions.
Now, you can probably guess how
this works.
Our variants need to point at
the subtitle and closed caption
groups that we're using.
So we need to add an attribute
to our video variant, and we
need to describe the renditions.
Now, notice that the closed
caption rendition does not have
a URI attribute.
That tells the system that the
closed caption data is found
within the video content, not in
a separate playlist.
So, there you go, the right
thing to do is to give us
everything you've got.
See, now you understand better
what I meant.
Okay.
I'd like to quickly summarize
the talk we've given today.
Emil talked about key
performance indicators, about
how to get or compute the values
and what they mean.
And Zhenheng talked about ways
to reduce startup time and how
to go about resolving stalls and
other errors.
And I've talked about how to do
master playlists.
I'd like to briefly mention the
HLS validation tools.
They do identify many issues
with master and media playlists,
and it's worth your time to use
them.
As always, you can get more
information from the WWDC app or
the developer website.
That's all we have today.
Thanks very much for your
attention and time.
[ Applause ]