WWDC2003 Session 711

Transcript

Kind: captions
Language: en
morning everyone welcome to the audio in
QuickTime session I hope you enjoyed
that slight demo that we had of some of
the things that we're going to be
showing you today as well so this
session is covering changes to the
quicktime audio foundation basically the
big changes were using Core Audio
natively we're removing the code that's
depending on sound manager this is
bringing a lot of new capabilities to
quick times audio infrastructure and as
well we're going to talk a little bit
about audio codec news so we're going to
be focusing today on playback export
some changes to DV audio that we made
and as well news from the audio codec
team and the technology frameworks that
were covering today are quicktime and
core audio and without any further pause
i'm going to introduce greg chapman
who's going to be talking about audio
playback so we've been having a lot of
fun as you might imagine so the main
thing we wanted to do with this revision
of quicktime was to remove all those
restrictions that the sound manner
manager put on us we wanted to have more
than two channels per track you've been
listening to some five dot one the
linkin park and the bugs bunny was all
five dot one so you can hear center
right and left right and left surround
and into the subs we wanted to have high
sample rates Sal manager limited us to
basically a 16-bit number for the sample
rate we're talking to core audio now so
we're doing much better on this front we
wanted more than 16 bits per integer
sample we could support playback of 24
and 32 bit integer but we would always
bring it back down to 16-bit to play it
out so you'd lose all that nice depth I
we also wanted to support floating-point
samples we used to be able to do this
again we convert to 16-bit you lose all
that nice deb the other thing we wanted
to do was to be able to do multi tracks
around where you could have for example
four five dot one you could have six
mono tracks each one labeled as being
left right center whatever and then have
them mixed together sample accurate
come out all the appropriate speakers so
let me go to demo two and let's show you
what we've got I'm gonna play this whole
trailer template part of it in the
quicktime state of the union we want to
see the whole thing
since five dot one
[Music]
here we go
[Music]
he doesn't think that we perceive you
[Music]
all of our line we involve tonight I
believe we can
mr. Anderson
[Music]
please agree
employment services
[Music]
and for fall
[Music]
good on you
[Music]
so you might ask what do you have to do
to be able to play such content in your
app and the answer is absolutely nothing
your app can play this as soon as we
shift this version of quicktime let me
show you iTunes itunes doesn't know
about quicktime support for
multi-channel iTunes will play itunes
does not use quicktime to play mp3s or
the new music store content but if you
handed a movie that doesn't know how to
play it'll it'll use quicktime to play
it here's an example of this is a linkin
park song i think it's the same one we
played when you came in takes a little
bit
so I'm not gonna play that whole song
again as fun as it was so that's an
example of an app that you know we just
installed a different QuickTime
underneath it plays back this movie it
gets surround sound no worries okay
let's go back to slides so how did we do
that there's a couple of things there's
some file format things we have to do
there's a new version of the sound
description that's in the tracks in the
soundtrack I've described sample rate as
a 64-bit float for example there's a new
optional sound description extension
that tells you what the channel layout
is for the track is this left and right
is it you know five dot one is it just
left this is we've rewritten the sound
media handler so that it can understand
these things and it talks to core audio
instead of the sound manager as you
might imagine and the new multitrack
mixture that i mentioned that does
sample accurate mix between sound tracks
so that you can have your surround
between tracks we talked a little bit
about channel mapping and how that works
historically movies can I have one or
more soundtracks usually they're
alternates but you could have more than
one soundtrack playing at the same time
each track could only have one or two
channels and if you had to we would just
assume that it was left followed by
right and if you had a multitrack movie
it would always mix two mono or stereo
depending what your output device was so
if you had three stereo tracks you'd end
up with stereo out and make certain
amount of sense but it's kind of
limiting in this lovely new world tracks
going to have any number of channels
there are assigned two speakers using a
sound description extension as I
mentioned and each track can contribute
to the movies channel layout for example
five dot one could be one track with six
channels in it could be six mono tracks
labeled according to what speaker they
should go to or it could be a 5-channel
track and then a 1 channel LFE track or
you know free stereo tracks or whatever
just you have to mark them appropriately
and then we'll play them back correctly
the order of the track in the movie is
not important as long as the channel
assignments are labeled and your old
movies that have unlabeled mono and
stereo they behave as before as you
would
back and if you're playing to a stereo
device if i took that matrix trailer and
you played it back on your computer with
no you know six channel device plugged
in that would down mix the stereo and
you would hear you know left all the
left side and the left all the right
side on the right use your Center in
both sides so let me show you a little
bit about that will go back to demo 2
i've got three movies here there these
are actually quad let me show you what
they look like so the one change we made
to player was to this particular panel
so that it could display what the layout
was he doesn't need to know the layouts
in order to play it but in order to
display the name of the layout he has to
be able to do that and we'll talk a
little bit later about how you can do
that too so this one here this has got
one soundtrack and that soundtrack has
four channels and they're labeled left
right left surround right surround
lobster on the Rights around all the
back speakers that's cool let's play
that now this is very boring content but
you'll be able to hear let it right
that's array right Sarang okay thank you
Kevin here's this one this is one has
four tracks let's take a look at each
one of those this first track is a mono
track it goes in right second ones left
surround the strange order but rights
around and then left that's kind of cool
is that going to go in the right places
the right then sir ring right sir ring
there's always that little hesitation
before that one comes in that makes me
nervous and this one's just to be
different it's also for tracks this one
goes left right left around rights
around as you might expect and this want
to play by correctly too but right then
there we go right Saran okay so you can
see that we're actually following the
labeling we're not just doing something
in the order of the tracks go back to
slides okay let me talk about a few
interesting new API is in QuickTime now
that we're talking to core audio there's
some new things you may need to do the
first one is QT converts sound
description we now have three different
versions of sound descriptions if you
look at sound descriptions yourself this
is starting to become a nightmare so we
put in an API where you can take any
version sound description convert it to
any other version sound description we
have a mode where you can convert it to
the lowest possible version of sound
description given the format let's
describe so that you can get the best
possible backward compatibility when
you're creating movies so old versions
of QuickTime can play them and you can
go to and from an audio stream basic
description which is core audios struck
that describes a stream of audio so
that's very useful for just taking a
sound description you convert it to an
ASB D and carry on with core audio these
are the the obvious ones you might want
to have to set a movie to play to a
particular audio output device device
you IDs are a core audio concept it's a
CF string so unique identifier for the
device we used to have mediaset sound
output component where you could grab a
sound manager sound output component and
say please play to this device if you
call the old API we will try underneath
to determine what the actual core audio
device is that that sound output
component was going to talk to and then
we'll play to that guy will try and do
the right thing we have two levels of
API here one is to set the movie to play
and then Mediaset is for individual
tracks so you can as before set
individual tracks and movie to play two
different devices extract movie audio
this is something that people have asked
for for a while have a way to take a
movie and just get uncompressed audio
out of it mix across track decompress
the tracks that are necessary this is
the one that's going to allow you to do
this and it returns a core audio buffer
list which means you can then take that
put it in a queue for some au graph to
you know have a ball with and play it
out or whatever you want to do
these api's begin and I'm not really
going to go through these I'm just going
to show you some sample code that will
be posted you can take a look at let's
go to demo too okay yes it's unix i
wrote a hook excuse me i wrote a command
line app and i'm going to show you the
routine that I wrote it just takes a
movie extracts the audio uncompressed
and writes it to a pay iff file which is
a boring thing to do but I could imagine
that you guys might think of something
more interesting to do with the audio
buffer list you get out of the movie so
it takes a movie and it takes the name
of the FF file you want we come down you
begin movie audio extraction you can ask
the audio extraction session what if the
default output format is going to be and
that default is generally 32-bit float
it's all the channels in the movie from
all the soundtracks in the movie it does
no mix down by default and I think it
picks the highest sample rate in the
movie if there are multiple sample rates
otherwise it's B sample rate so but I
that's not what I want I want to put it
in an aiff file so I want 16-bit
big-endian samples so that's what I say
here I do that by hand core audio has
much nicer c++ classes for dealing with
audio stream basic descriptions you tell
the extraction session what you want so
I send in what I want I wanted 16 bits
per channel when I wanted signed big
Indian blah blah blah allocate your core
audio buffer list minds real simple I've
only got one buffer because it's all
interleave but you could get the
interleaved float32 buffers if you
wanted set up my aiff file and I call
extract movie audio this is the
interesting loop you watch this flags
word that comes out of extract movie
audio for this bit to be set when that
bit is set you're done and each buffer
you pull you write it out to the file or
pass it off to your au graph whatever
you want to do and then we do that so
let me just show you this running I
wrote a little bad script to do it
because the command line tool is really
ugly but let's see I'm in the same
folder here and you'll watch that show
up do it and over here we have suddenly
an aiff file this is an AAC movie which
is actually people who have tried to get
audio out of movies by themselves it's
really hard when you're dealing with a
BB r codec there are seeming issues
between buffers and you get pops and
it's just a pain in the bud but here we
are we've got this and it plays
beautifully and loudly now that
interestingly enough with stereo and it
is a stereo and you just heard what we
do with stereo when we have a
multi-channel output which is we put it
in writing the left as you would expect
alright that was fun go back to slides
now I'm going to bring up Brad Ford and
he's going to talk to you about
exporting these new kinds of audio
[Applause]
thanks Greg and thank all of you for
caring enough to get out of bed early
Greg said that we've been having a lot
of fun I would use a different word to
describe it probably pain but that's the
whole mantra of QuickTime we write the
hard code so that you don't have to and
a great deal of pain has gone into
making these bits playback to the right
place at the right time and I we would
really hate it if all this work were
wasted by not giving you a way to author
this content so thus we needed to give
you a way to export to these formats as
well so the QuickTime that ships with
panther is quicktime 6.4 so let me give
you an overview of the world before
quicktime 6.4 our exports were limited
in the same manner as our playback we
could only do what the sound manager
could do we also had problems with
standard
sound compression you may have seen this
you may have called it if you are a
programmer or if you're a Content
developer you may have seen it when you
export to AIF or wave it's the dialogue
that comes up and gives you a list of
all the compressors and the sample rates
and options that you can compress to
that's the standard sound compression
dialogue or component I've had some
problems with it it requires displaying
quick times own dialogue to work well
and right it actually has some property
calls so you can use it with your own UI
and just sort of try to use it as your
state machine underneath but it was
really envisioned to be driven by the
dialogue so we did some things wrong we
didn't always keep track of our state
correctly we sometimes gave you the
wrong codec settings and so this
required you to have an intimate
understanding of what the codec was
doing underneath that was a pain for you
the dialogue also offered some strange
choices for compression formats sample
size choices for compressed audio which
is just wrong and also sometimes would
show you various flavors of uncompressed
masquerading as codex which is also very
weird and wrong so now in QuickTime 6.4
we have a new standard sound compression
component I'm kind of loathe to say it's
a new dialogue because it's oh so much
more it's a dialogue in one case but
it's also just a sound configurator it
can export to and from all the new
formats we can play it back we'll let
you export to it standard sound works
well with or without quick times default
dialogue which makes it easy for you to
put custom UI on top of it so any
application you might develop that would
need to export audio you can use your
own beautiful UI and just use our
standard sound component to drive it and
let it keep track of all the state ask
it questions and we make this possible
with new property api's which i'll be
talking about in a minute also the
default dialogue is now a lot smarter we
don't make the same mistakes that we
made before it's easier for me to show
this to you then to talk about it so
let's go to demo to
okay we've got the same lovely riveting
Warner Brothers sound effects content
that you heard earlier I'm just going to
bring up the export dialog I'm going to
export to a QuickTime movie and this is
the new standard sound dialog this is
what the UI looks like you'll notice a
lot of changes from the old one first of
all gone are 32-bit float floating-point
24 bit integer and all the other
uncompressed formats pretending that
they were really compressed formats now
if your uncompressed your uncompressed
all the time and you get two different
flavors of yourself by going down to
this codec specific settings area where
you can set your sample size to be what
it really is and you can tell us whether
you're floating point or unsigned or
little endian things like that also of
note are and I expect some oohs and ahhs
here ooh that's a lot more channel
layouts than we ever supported before
you want to do mono or stereo that's
fine you can still do that but you can
also tag the content in your movie we'll
let you export to as many tracks or as
many channels as you have in your input
movie so if you've got six tracks or six
channels of audio in your input movie
will prevent you know all the 50 s all
the five dot ones makes sense also
here's another la moment wow sample
rates that go up beyond 64 so we'll let
you go to 24-bit 96 kilohertz audio no
problem and as a first let's go back to
AAC where for the first time will let
you export multi-channel AAC which opens
up a lot of really interesting
possibilities for web content delivery
you know you can export AAC in the range
of say 192 to 320 kilobits per second
that's low enough that you could
transfer it over the Internet
sound pretty interesting okay so let's
go ahead and save this out to the
desktop
okay here we go and let's quickly look
at the info now we've taken all the
tracks and we've put the audio in each
track down into one track but it's still
got all the channels they're just all in
one track so we can play this back you
get the idea AAC 5 dot 1 you witness it
first in public ok let's go back to
slide sleep that actually was the first
public demo of five dot one AAC export
in QuickTime feel special so what
application changes are necessary many
of you probably use these old friends
these at ap eyes convert movie to file
or convert movie to data ref if you do
you'll know that the it has a lot of
parameters and the last parameter is a
component instance which is the
component instance of the exporter that
you're going to use and there are three
ways that you can do this you can pass
in nil for the last parameter which
tells quick time I don't care what
exporter you use just do the right thing
you can also pass in a component not a
component instance which means you find
the right component of the type and
subtype that you want but you don't open
it and you say i know i really want to
use this component but i don't care
about special settings so still do the
right thing it's the end the third one
that you can do is you can actually open
that component yourself do some special
configuration to it and then pass it as
that last parameter if you if you do the
first case pass nil or the second case
passing just the component you don't
have to make any changes your export
will work with the new settings you'll
get the new audio exporting by default
now if you do open your own movie export
directly you must opt-in you must be
explicit otherwise we don't know if
you're prepared to handle the new
properties and the new settings that
we're going to give you so the way that
you opt-in is you call this lovely new
called QT set component
property I'll be talking about these
component properties in more in a minute
but you can see you just tap in the
exporter that you're talking about the
class and the property ID is K movie
exported property ID enable 2003 audio
features pretty long property name and
it's a boolean and if you say yes then
you are telling us that you're ready to
handle these new settings now if you
have opted-in you may just need to opt
in and you're done you will only need to
make further changes if you call movie
export get settings as Adam container
and parse the settings yourself if you
do that so I don't want to tell you not
to do that but it's kind of risky to do
that because we might do things like we
just did when we changed audio
underneath you and give you all
different settings so if you part if you
have code to parse them you need to be
ready because you're going to get
different settings also if you open the
standard sound compression dialogues so
just as a sidebar here the old one was
of type s CDI standard compression
dialogue and of sub type f oun sound so
because it use the sound manager the new
one is also SCD I but the subtype is and
no offense to the German automotive
industry audi aud I because it uses core
audio so if you open up this new
standard sound component by yourself
because you want to configure settings
or you want to use our dialogue you also
have to change your code because we
don't use the same calls as the old
standard sound component so now let's
address the bigger issue of the
properties problem we in QuickTime are
very lazy people much like yourself and
if we solve a problem we don't want to
solve it once we want to solve it for
many other problems we want to get
software reuse so there was a general
problem here that we have a lot of
components and they expose their
properties in different ways so wouldn't
be nice if we had a standard way for all
components to expose their properties
getting them setting them finding out if
they have such a property listening to
properties and we didn't have a way to
do this and standard
was sort of one of the first test beds
for such a facility because it had
roughly 50 new properties and we needed
to decide how we were going to do this
so that they wouldn't conflict with the
old properties so what were our options
I'll give you three options that we
could have gone with we could have gone
with the old if you've used standard
compression before you know this call
well sv get info and FC set info so this
is a lovely call it lets you take your
component instance of the step of the
sound standard sound compression
components and pass in a for care code
that is the property ID you care about
and then the don't void star so you have
to know exactly what that type is if you
don't bad things will happen you might
crash or worse you might think you got
the right information but you actually
got some garbage so there has to be a
special understanding contract between
you and the component implementer so
that was a little bit scary here's
another option we could have gone with
this is something that sequence grabber
does create new get set pairs and shinbe
existing selectors so you'll see in
sequence grabber there are a lot of
these pairs of get this guy set this guy
this is nice because you can have custom
parameters for each call but the
downside is that you have a lot of
interfaces to learn and you make typing
mistakes and it does not address
listenable properties these are just get
as and setters and some existing API is
required different data types the other
option was to adopt a core audio model
we liked what karate 0 is doing a lot
because they give they have actually
three standard calls the get info get
and set and if you're talking about the
how hardware abstraction layer they also
have property listeners this was very
close to the kind of thing that we
wanted to do as well the only drawback I
can think of is that each property class
requires its own suite of calls so you
have a set of gettin set and get info
properties for audio units and audio
streams and audio hardware we were
hoping that we could make it general
enough that all components could use the
same calls so here is our solution a
suite of standard come
at property api's these are being
inherited by core services so it's not
just a quick time thing this is new and
Panther and it's a system-wide thing so
these are going to be in the negative
range if you've ever written a component
you know what I'm talking about there
are some required selectors that you
have to put in open closed version and
there are some optional ones but also in
the negative range these are optional
negative range property selectors
there's a component get property info
component get property component set
property and then two for adding
property listeners and removing property
listeners they can be implemented by any
component including yours and we would
encourage you to use them now there is
one thing to know about these the new to
Panther calls are prefixed by call
component so it looks like call
component getcomponent property info
these are going to be shipped in Panther
and forward if however you want to ship
these properties in Jaguar then we have
also released cutie prefix versions of
these which have the glue is in quick
time so we can ship it in Jaguar as well
the property type is specified by a
class as well as an ID that means you
get to for care codes or eight care care
codes to work with instead of just one
for care code just kind of good because
it sort of gives you a little bit of a
buffer in the area of namespace
collision so if I'm a component your
component and I have sample rate
property and you have sample rate
property but yours is a float64 and
mines and unsigned fixed I can still
protect myself if our property names are
the same because i have a class or care
code that puts mine in a different space
than yours also all of the data is
explicitly typed in the get info call so
if you make a get info call and you
don't have to know anything about me you
can just query me what are your
properties I have to tell you what the
type of the property is and the size so
it's very explicit even if you don't
have the header file what the property
is ok so the get info call
also gives you some flags that tell you
some important things this is a logical
extension of what they were doing poor
audio as well they have a great call and
get info that lets you see if the
property is settable so ours gives you a
bitmap of flag field that lets you see
if perhaps yes the property is settable
but I can't set it now maybe my hardware
is not ready so maybe it'll return can
set later or can get later so you can
get meta information about the property
by getting these flags out including one
of the most important ones will notify
listeners and I should mention standard
sound compression the new dialogue has
like I said about 50 new properties and
I think all of them but five are
listenable so you can build a very
sophisticated app that's very dumb
underneath because it's all it has to do
is listen for property changes what do
they look like so here are the actual
calls I've already talked about them so
I won't stay on this long it has a class
it has an ID has the type which is also
a for care code it has a size and then
it has that flag field that tells you
can get now can get later here is how
you would implement a property listener
let's say you are listening for my
available sample rate list my available
samples sample rate may change based on
whether or not the compressor changes so
you would set up my listener proc and
you typically have a big switch
statement in it in which you would
handle all of the property IDs that
you'd registered for so I just did a
simple little printf statement there and
go and update my UI and then you would
tell the component like standard
compression you would first make a new
component property listing or upp and
then you would register for that
notification by saying call component
add component property listener or if
you're using the QuickTime version for
Jaguar you would say cutie add component
property listener with the exact same
parameters and once you do that you are
listening for that property and you will
continue listening for that property
until you remove it there is one
important thing I should mention about
these property ap is due to a
minor oversight the glue code got into
Panther got into the feed CDs that you
that you received on Monday but the
header file didn't make it there in time
wasn't promoted to public status so that
was a little bit of a minor oversight
but we are making it available to you
via sample code so if you go to the ADC
developer site and look under quicktime
you will find it there it's called QT
component properties dot H and there you
can get all of the calls and start using
them so let's bring up a demo and show
you these audio properties and youth can
we go to demo too okay here is a great
little app that was written by Adrian
bear Locker another member of the
QuickTime engineering team and I'm very
proud of this app because he wrote it
very quickly and it's the most slick
under thousand line code app that one
could write because it keeps no state
all of its state is managed by the
standard sound compression dialogue so
maybe you're like Adrian you think that
the standard sound compression dialogue
is swell but you really just you know
can't get enough metal in your life you
want more metal or maybe you wanted to
have your own custom UI look like he did
so you are able to use standard sound to
do all of your configuration and put up
your own UI on top of it so let's take
the same content we heard before that
has the left right left surround rights
around and we'll use his you I now
remember this is being driven entirely
by the properties so every time I
tweaked a setting here it's doing a set
component property call or its listening
for changes and updating the UI so let's
go ahead and do this export I suppose I
should select an output location
and we'll export it was a pretty full
featured app and it's under a thousand
lines of code but right right so there
we took a quad and we mixed it down to
stereo and it still plays in the right
places if you'd like to learn more about
the component property api's or standard
sound compression in general please come
to our lab between 12 and two today
where you'll hear more thanks and I'll
bring back Tim so feature was really
really good we worked really really hard
we were a little bit confused we might
have been drinking strange things so we
want to show you a little bit but we are
going through can we go back to demo two
please
lefts around gone again oh man wish I
didn't have to work on audio
[Music]
that's not right that's better oh that's
much better quick comes back to normal
[Music]
[Applause]
thank you so that was that was a stupid
movie that a lot of people on the quick
19 work really really hard to film and
direct and dream up I guess that was
probably the eyes are the easiest of the
hardest so I'm going to talk a little
bit about some DV audio changes back to
you know real content so we made some
changes in QuickTime 63 the issue of DV
audio was it as two basic ways that you
can store DV audio inside of a QuickTime
movie the first way is you can store
inside track as PCM audio and basically
extracting it from the DV frames as you
capture that's what final set pro does
like what QuickTime sequence crabber
does and there's been generally no
problems with long-term ad sink in that
case the other way you could do it is
you could have the audio track index
every frame of DV and reference to VV
audio inside the DV frame and that's
what I movies doing when it captures and
that's what the QuickTime DV importer is
doing and the issue there is that audio
in that case is represented as a fixed
number of samples for DV frame which is
a problem because it isn't if you look
at the duration of a frame in NTSC it's
basically a variable number of audio
samples per frame and when we
represented as a fixed value which is
basically the maximum number that could
occur we're basically accumulating
adrift in some cases so in playback is
fine but in our export and our editing
cases we were accumulating about a one
frame of video error in the audio for
about every two and a half minutes of
export or or editing depending on the
case we had a fairly large problem with
some AV sink in some applications so the
solution that we decided to do is
represent DV frames having variable
duration which is really the correct
thing to do so we have a new type which
is v dva and it's a variable duration DV
and that's new and click x 63 it's also
going to be in 64 and forever after
that's the default when you're importing
it a DV clip when you use a little bit
QuickTime importer and in iMovie it's
now going to be representing them as
variables through
so a couple of reminders when you're
looking at tvs you're playing around and
actually looking at TV audio in the
tracks you're going to see the new type
k variable duration TV audio format you
may see the old type because files that
were created the 463 will still have the
old type and the important thing is if
you're playing with sound descriptions
and you're using the new type you must
set the compression ID to variable
compression and that's how we basically
make this work and so we've actually
achieves a really good audio video sink
and I just wanted to point out a couple
of things a lot of the support that
you're seeing will be available when
quicktime six warships on jaguar as well
that's something that we're working on
to make sure that the multi-channel
support etc is going to be working on
that platform we're using a mobile I Oh
audience to drive the audio the six
channel audio connected via firewire and
of course we're using the core audio
support and their drivers to make that
happen and with that I'd like to
introduce oh one more thing one more
thing which is for those interested in
AV startup sync issues we have a session
this afternoon at two where we're going
to be talking about some changes we're
making for audio video startup saying we
figured that's more advanced programming
in audio so we put it in that session
and now I'd like to introduce Eric
Aldrich she's going to talk about audio
codec development thank you Tim
[Applause]
this is wonderful quicktime 6.4 built on
top of core audio but now we're going to
take a little journey to core audio land
and talk about audio codecs this is what
is an audio codec essentially it's the
business end of the QuickTime on the
audio side it's going to actually handle
your encoding your decoding of your
audio format on audio codecs are going
to replace your s cons and your ethics
they're now called 8x an ax and right
now in your feed CD we have AC mp3 and
IMA and we've already shipped AAC before
quick time since QuickTime six okay so
how do you go about lighting and audio
codec well we make a little bit easier
for you there's a lot of code involved
but we provide you an audio codec SDK
it's available on the air on the connect
on apple com site your core audio we
provide you a framework basically just
needed to implement a codec it's a
fairly basic but you can do quite a bit
with it we give you some sample codec
which is IMA for one important point to
note is you do not have to implement
both an encoder and decoder you can
implement one and not the other example
this is that mp3 we've only implemented
to decode we have not implemented the
encode another thing that's important
about audio codecs is we use a push and
pull model you could push the data in
and you pull the resulting
transformation out and keep that in mind
when I'm talking about this further here
okay required files for writing an audio
codec fairly straightforward big one is
there is a sea-based codec which is
pretty much the be a base class for
everything you're going to use this in
every codec you're right we put this
basically all these files provide the
framework for what you will need to
implement handles all the dispatching
all the nitty gritty dirty details of
talking to
the sound converter and like our excuse
of the audio converter which was
detailed in session 404 okay we have an
optional file which is a AC simple codec
and this implements a basic buffer for
very basic sound codecs audio codecs we
provide buffer the buffering
implementation for you you don't have to
do it but for things that are fairly
elaborate you're going to want to use
something else and then you're going to
have to deal with codec specific files
for instance you're going to want if
you're doing AC or mp3 you're going to
want an AAC codec file you can inherit
directly from either base or simple
codec then you have just set you have
several methods you'll have to implement
and these are analogous to a few things
that were already in the sound codex of
course you have your get your property
management which is get property info
get property setproperty you have your
initialize an uninitialized which is
basically locked down the codec and then
unlock it you have a pen input data
which is the push of the push/pull model
where you push your data in then you'll
produce output packets which is your
pull on the push-pull model and that's
where you're going to get your decoded
or encoded data out of ok a few things
you need to think about like before you
had open a component now that is handled
mostly in the constructor your input
your default formats we handle the end
instructor as well ok you need to handle
the encoder and decoder separately a
model that we have used as we create if
you have a lot of common functionality
and codecs with your encoding or
decoding so a lot of that we put into a
generic you know here we see my codec
and then we inherit directly from that
we create my decoder my encoder and
that's the model you want to think about
it allows us to add for instance on mp3
we have you know an mp3 codec then
inherent the mp3 decoder inherence mp3
codec and if we later choose to
implement a 10 color for instance will
also inherit directly from the mp3 codec
file okay when dealing with properties I
can't stress enough that this is
probably the most one of the most
important features of what you'll be
dealing with this is basically you're
configuring your codex before you lock
it all down so if you're get property
info it tells you how much data is
required for each property and more
importantly it tells you whether or not
you can set it there will be two cases
of this a you cannot set it or B you can
set it when it the codec is not
initialized and get into that little bit
get property not all that much to say
you just have to realize that that at
any time this could be called and it is
alright to answer I don't know if
someone asks you for your bit rate your
bit rate hasn't been configured you
could say you know zero say I don't know
or you can say just report the default
but it's important understand that this
can be called at any time there are some
properties that won't even be known
until after run times so it's possible
you can just say I don't know
setproperty again is not always
available depends if the Codex been
initialized or not initialize an
uninitialized again initialize is just
basically walking down the codec at that
point when someone calls initialize
whatever they have set you take that and
that's how you set up the audio codec if
they have not set all the appropriate
properties it is if you can use default
values and if you can make something out
of it you can initialize at whatever
settings have been set or you can or you
can fail that's perfectly legal they
haven't provided you enough data you can
fail and enough uninitialized just
simply removes this lock basically a
boolean that I'm i initialize or am I
okay a pen input data fairly
straightforward you basically take your
input data and you put it in internal
buffer important thing to note though is
you have to deal with audio packet you
know audio stream packet descriptions
all vbr codecs should handle these very
important for mp3 AAC and the like CBR
codecs you don't have to it's nice you
can you can handle them if you choose
but it's not really required produce
output packets does the actual encoding
or decoding whatever you don't have to
have the actual encode decode right
there using your call separate function
or a separate library whatever to do it
that's where you actually handle it and
again I'm an encode side if you're a vbr
codec you have to fill out the audio
stream packet descriptions another thing
to note here this is where you're going
to actually handle clearing your input
buffer even though their pen input data
you deal with you know handling handing
the data in and produce output packets
you're going to update all your buffers
you have to remember to do that here and
backward compatibility kind of an
important thing there's a lot of very
well there's a lot of quick times fix
their copies a QuickTime sticks out
there and Panther hasn't shipped yet we
allow you so you can deal the old escala
mess deck world we give you sound
manager audio components which allows
you to wrap an audio codec so two old
versions of QuickTime it looks just like
an old Sound Manager component as
deckhand s calm and we're going to be
providing an SDK that does this on our
monocyte anyway you will need to
implement this if you want that support
it's kind of important but very
straightforward and we've been shipping
these for a while this is how we
implemented AAC from the get-go so this
is proven technology and it's been
working for quite a while so this point
one important thing some documentation
on all this
read this here core audio overview which
you can get basically connect on apple
com take a look at it provide any
feedback we'd appreciate it thank you
[Applause]
the non-technical is still here for the
roadmap these are the sessions on
technical quicktime we have a bunch of
sessions going on also for content
developers on a quick time one thing
that is not noted here is that if you
really care deeply about audio there is
a feedback form starting just after we
finish this session across the aisle
here so if you have any comments that
you want to provide to the audio team
this is the right place to go and
provide that we have a quick country
that formed tomorrow afternoon we will
welcome you all there if you have any
questions so if you have any comments
that you want to make about quicktime
does my name and my address that you can
use if you have any further comments
analytic questions after the conference
is finished for more information you can
go to the developer slide quicktime if
you have any up you wanna see where
quicktime is in general you can go to
the Apple side quicktime and for
information you can go to the
documentation side there is a QuickTime
section area that has been updated just
for this conference as I see us being
present here there is documentation
there is sample code that is only
available to you if you go to the ABC
side download place and you go to the
QuickTime stuff so it should be
available right now for you also we have
mentioned this many times this morning
there is a quick timeline
out going on downstairs so if you have
any questions for instance today is the
special day for any audio questions all
the engineers from the audio team will
be down there are they loud today all
day you can go down there is in the
first floor Sacramento room there is
also some assistance for content
development if that's what you're
interested in
you