WWDC2001 Session 209

Transcript

Kind: captions
Language: en
thank you for coming this is a session
on audio processing and sequencing
services and just before I bring Chris
Rodgers up is going to be doing most of
conversation just thought I'd welcome
you my name is Bill Stewart on the
engineering manager of the Chorio - if
you missed the talk yesterday we talked
just very briefly about the audio
hardware layer that we have in Mac OS 10
and all of this sort of fits in the
general frame of what we call core audio
and aside from the audio hardware stuff
yesterday we talked about the default
output unit as a way of very easily
having your application get up sound out
the current output device is selected by
the control panel so what we're going to
be doing today is to talk about Audio
Units and Chris will go through that in
a moment and the default output unit is
one of them so if you were at the talk
yesterday and a little bit confused
about well that fits in you should be
quite clear by the end of this talk we
hope so without any further ado I'll
bring Chris Rogers up from the cordilla
- thank you good afternoon pleased to be
here to speak with you a little bit
about some of the higher level services
available for audio on Mac OS 10
overview the talk is first thing I'm
going to talk about is audio unit
architecture audio units are components
for processing audio or just dealing
with audio in some general manner and
then I'm going to talk about the audio
toolbox hooking audio units together and
processing graphs and lastly I'll be
speaking about some timing and
scheduling services that are available
using music sequencing services so let's
take a look at where these higher level
services sit in our audio stack Audio
Units are one level above the core audio
audio how api's which Jeff spoke about
yesterday and then above that we have
what are known as the a you graph
services which are a set of API is for
connecting these audio units together
and the music sequencing services will
see can talk to au graphs and audio
units and core MIDI is off to the side
there and later on Doug will be going
over that in quite some detail I'm
actually showing how there are some
relationships between the MIDI services
and in some of the audio unit technology
that we provide so audio units are
components components that are handled
by the component manager that QuickTime
makes wide use of like all components
individual components are identified by
a four character type subtype and
identifier field and the component
manager can query a list of available
components of a given type Audio Units
are the ones we're interested in this
case so you can call find next component
to go through the list of available
components and then once you've gotten
to the one that you're interested in
opening you can call open a component
and to close it you call closed
component
and the component manager has some
pretty good documentation so you can
refer to our developer documentation
about the component manager for that the
audio unit component in general it
generates receives processes translates
or otherwise manipulates streams of
audio as I have my slide and they're
good high level building blocks that can
be connected together just to create
networks of audio processing and you can
also use them singly and just the for
instance instantiate a a software synth
and up and wire it up so this output
goes directly to the hardware we provide
a set of audio units that ships with OS
10 and we'll be adding more in future
and future releases and developers are
free to create their own Audio Units and
will be providing an SDK to make that a
lot easier than it would be otherwise so
what kinds of Audio Units are there
there are other units that just provide
audio data so they're sources they have
no inputs to them an example of that
type of audio unit would be what's known
as a music device it's a software
synthesizer and Apple ships DLS to
soundfont compliant software synths in
OS 10 also there are other units which
interface with the hardware they provide
a higher level abstraction of hardware
IO devices so they talked to the audio
how and one of the most common types of
audio units on the category of DSP
processors reverbs filters mixers this
type of thing so these type of audio
units typically they take input and
process it somehow and output the audio
stream another kind of audio unit is
format converter converting between 16
bits floating point
interleaving stereo streams into
discrete mono streams sample rate
converters and there you know quite a
few other types as well and then another
class
is what I call it after class of audio
unit and this is a little bit strange
because it doesn't actually process the
audio in any way using DSP and also
doesn't change the format of the audio
stream but instead it somehow does some
other kind of impedance matching between
the input and output for instance it may
do some kind of internal buffering so
that the input buffer size is different
from the output buffer size or it may
lay on top of a thread scheduling scheme
so that everything before that audio
unit in the chain would get deferred to
another thread and this way you could
potentially take advantage of multi
processing on multiprocessor machines is
very interesting application so I guess
we're gonna try running a little demo
here and I should probably explain a
little bit what I've done
oh this is your demo okay so if we can
have the machine oh I can't see anything
oh and neither can you
I can see this okay let me just get this
to run so
since the big tease
so what I'm actually running here is a
Java program the Java program is quite
simple and me just this is very
interesting trying to do it this way
this is dis demonstrating how to use the
component description and the component
services from Java it's the same as you
would do in C as we've been talking
about in those sessions you have both a
Java and C API to this I thought it
might be interesting just to show you
some simple Java code that is quite
clear about the concept of what we're
doing so we create a component
description this describes to the
component manager the type of components
we're looking for the type is okay audio
unit type and then what I'm going to do
here is I've just told it to look for
the first component of this type and
I'll use the au component which is
actually in this package in Java and
this will find the first one and then
I'm going to print out some information
about it and then I'm going to keep
finding the next component from the one
that I've already found until I don't
find any more and then my program
finishes so this will give you an idea
of the audio units that we ship with the
current version of system of this system
so there's delay some filters as a
matrix reverb in de levers D into
louvers a mixer unit there's the DLS
music device you can see it's type
subtype manufacturer there's the audio
device output to default output unit
which is actually an extension of the
basic unit that will talk to any device
to default output device will track a
default output that the user might set
and various others thanks go back to the
slides
and of course I can't see my screen okay
I just just before the demo I went over
some of the categories of audio units
and this is just a little simple diagram
showing that not even it's a basically a
box that can have an arbitrary number of
inputs arbitrary number of outputs and I
think this diagram is showing that DSP
FX would have both inputs and outputs
with some kind of processing going on
software synth music device may only
have outputs because it's actually
synthesizing sound was creating it from
scratch a default output you know which
Doug talked about yesterday it would
just take inputs and it would send that
audio to a hardware device talk about
audio unit properties properties are
very very fundamental to the way that
Audio Units work and a very general and
extensible way to pass information to
and from audio units out of units
themselves they implement a very small
set of functions which which can be
called to deal with getting audio in and
out and setting parameters and so on but
we created this property mechanism to
allow other units to be extensible so
when we come up with some new feature
that audio units need to implement we
can always use the the property api's
and pass in data via void star pointer
and the length of the data is indicated
by a byte size parameter the we define
several different properties today and
the type of property is identified by an
audio unit property ID
it's just 32 bit integer both the MIDI
API is in the audio how API is also
support this concept of properties it's
almost identical type of mechanism in
the API between a three
secondary to the the property identifier
identifying which property we have
addressing within the audio unit to tell
which part to specify which part of the
audio unit is to deal with the property
so there's a scope which can be global
input output or group global means that
the property applies to the audio unit
as a whole input scope means the
property applies to a particular input
output is pretty clear and group is a
particular type of scope that is
relevant for software synthesizers music
devices audio unit element is an index
specifying which particular input output
or group we're identifying so if we want
to talk to input number two then this
argument would be 2 or actually it would
probably be depending on how we're
numbering starting at 1 or 0 the IDs are
defined in audio unit properties that H
and it's important to understand what
the data formats of the properties are
because we're just passing information
as a void start pointer and a length
parameter you have to understand the
structure of the data so those data
formats are defined also in the header
file we have a couple properties defined
right now and additional properties will
be added as needed and developers are
free to implement their own private
properties for their own Audio Units and
there's a range of reserved property IDs
for that but all audio units should
support the general system properties so
yesterday at Jeff's talk somebody was
interested in knowing about the
performance capabilities of our system
and somebody asked about well how many
audio units can you connect together and
is there going to be a problem well the
que audio unit property fast dispatch
property is a way of getting access to
function pointers which call directly
into the
audio unit and this bypasses the
component manager dispatch overhead that
is typically encouraged so basically
you're just dealing with function call
overhead when you're doing your
processing it's no different from any
other kind of DSP you might be doing in
your own code so really the time the the
bandwidth limiting factor is actual DSP
that is occurring so the answer to the
question of how many units can you
string together is really a function of
well how much processing are you doing
among all these audio units it's not a
limitation of the Audio Units themselves
this particular property can be used as
an optimization but you're not required
to use it here's the second example of a
property and that's the the K music
device property sound bank FS spec this
is a way of telling the software
synthesizer which sample bank it should
use so you identify the sample bank with
an FS spec pointer and you pass that
information in by calling audio unit set
property and the first argument is the
actual component instance the second
argument is the property ID the sound
Bank at that spec ID then it's global
scope because it applies to the simp as
a whole the element doesn't really
matter in the case of global scope
second to last argument is pointer to
the file spec and then the size of the
of the data is the size of the FS spec
so that that shows you how you can
communicate property information to an
audio unit without a unit step property
and all properties are dealt with in a
similar way
of course the specifics of what the
property information is and what it does
will vary parameters are less for
configuring attributes of audio units
and more for changing real time control
points and we have the audio unit get
parameter and audio units set parameter
functions for this
allows real-time control of processing
algorithms the parameters are identified
with audio unit parameter ID and the
value is a 32-bit floating-point number
and these parameters are very similar to
MIDI continuous controllers but they're
much more resolute there are 32-bit
floating-point numbers as I pointed out
and it's much much higher resolution
than seven bit MIDI value the set
parameter function includes a timestamp
allowing the audio unit to schedule the
parameter change for a very specific
time within a buffer of audio so the
timestamp is in units of sample frames
on the next buffer of audio to be
processed an audio unit supplies its
list of supported parameters with the ke
audio unit property parameter list and
individual parameters can be queried
with the parameter info property and
information such as the name of the
parameter what units the parameter uses
whether it be Hertz cents decibels
whatever min Max and default value for
that parameter and some flags as well
and information provided by this
property could be used by a client to
put up some kind of user interface
representing this particular parameter
so if the parameter represents volume in
decibels or something like that from
maybe I don't know - 150 decibels up to
zero DB or who knows the control is
actually able to display the the units
and the minimum and maximum value
appropriately I'm just information Audio
Units are in one of several states and
it's important to understand the state
transitions that Audio Units go through
when an audio unit is first opened with
open a component it's considered to be
an open State once the component is
closed then it doesn't even exist
anymore
so that's kind of like the base level
constructor destructor
if you're thinking in object-oriented
terms but there's an additional level of
construction and destruction and Audio
Units that's the initialized and
uninitialized state and we have the
audio unit initialized and audio unit
uninitialized functions for that
once the ADI unit is initialized you can
actually start it start at running with
audio unit start and that applies
specifically to io audio units such as
the default output unit actual actually
start the hardware through the audio Hal
so let's go into a little more detail
about what's the difference between just
opening the component in the first place
and initializing it it seems like it's
the same thing well when you open the
component you have a component instance
and you can make calls to the component
but at this time the component is not
expected to allocate any significant
amount of resources such as memory but
you are able to make property calls to
the audio unit to configure it in some
particular way for example you might
want to configure it sample rate or bit
depth or maybe you're interested in
setting up some other aspect of the
audio unit before you actually
initialize it then once you call audio
unit initialize and you've made your
property calls to configure it just like
you want then the appropriate
allocations can occur optimizing for the
particular configuration so if you're
configuring a reverb for a certain
number of delays or something like that
it's able to allocate exactly the amount
of memory it needs so the running state
this is what audio is actually being
processed it's actually going through
from input/output an audio unit or the
hardware it's running whatever the case
may be and the audio output unit
implements the audio units start and
other units stop functions
and if you have a graph of audio units
connected together this this is where
the audio actually starts being
processed or stops being processed and
it the audio output unit generally
represents the terminal node in a
connection and as Doug talked about a
bit yesterday the default audio unit it
starts the hardware device that the
audio Hal is connected to here's another
API to Audio Units audio unit reset it
basically is used when an other unit is
initialized and it may even be running
but if you want to clear the state of
the audio unit it's typically used with
DSP Audio Units
so for delay effects you would clear the
delay line for reverbs you would clear
out the reverb memory for filters you
clear the filter State and this is
important if if the graph is actively
processing audio but you perhaps
reposition the playhead to a different
part and the audio stream and you don't
want reverb tail from from a later part
of the of the score to be appearing in
the in the mix
audio unit IO management audio units
process audio or deal with audio using a
pole IO model and what I mean by that is
that there's a function a particular
function you call to be the audio output
stream from the audio unit and when you
do that the audio unit has to go and
fetch its audio input so it typically
does that by calling the same function
on the output of another audio unit and
then that audio even has to get its
audio input from somewhere so if you
imagine a chain of these audio units
connected together one after the other
you would pull on the very end one and
in turn it would go down the chain each
one would fetch its input from the one
before and you pull the audio through
the chain it's possible to connect these
audio units up in more complicated
configurations than just simple linear
chains but the idea is still the same
the audio is pulled through to the
terminal node which is typically ending
with the hardware output device all
units specify their number of inputs and
outputs through properties and the
format of the audio data is canonically
it's N channel interleaved audio or it
could be signed sideband data such as
spectral analysis data and this is
linear PCM 32-bit floating-point
okay I've been talking about connecting
these outer units together and this is
just a picture showing that so I'm going
to talk about how to get audio into an
audio unit and then I'm going to talk
about how do you get audio out of an
audio unit so there are two ways to
provide an audio unit with its audio
data the first way is using a property
called a audio unit property make
connection and with this property you
specify a source audio unit and its
output number and then that connection
is made and the audio unit knows that
it's to read its audio data for a
particular input from another Audio
Units output a second way of providing
audio data to an audio unit is by
registering a client callback and this
is the standard sort of callback that we
get in the audio Haller and the sound
manager or in many other audio systems
such as a CO and the in this using this
method the the client just specifies a
callback function and then every buffers
life cycle this callback function is
called and the client is able to provide
the audio for a particular input of an
audio unit and in the case where audio
units have multiple inputs each input
may be configured separately using
either of these two methods so our unit
has two inputs
first may be connected to the output of
another audio unit in the second input
may be connected up with a client
callback so that audio is provided
directly by the client in this case I'm
going to be talking soon about the a you
graph API which is a higher level
abstraction for connecting these audio
units together so I've just gone over
two ways that you provide audio input
data to the audio unit now it's
important to understand how do you
actually get the rendered audio out of
the output
and you have to use the audio unit
render slice function call for this it
pulls the audio or reads audio from a
particular audio unit output and it must
be called once per output so if there
are three outputs then it would be
called three times each audio output can
be an n-channel interleaved stream like
I said before so it's possible that an
audio unit might just have one output
and provide actually eight discrete
channels internally in that one output
let me go over some of the important
arguments to the audio unit render slice
function it's actually very similar to
the the audio Hal callback arguments the
audio timestamp it's exactly the same
audio timestamp structure as an audio
Hal and it specifies the start time of
the buffer that is to be rendered it
provides information about host time the
current host time of the machine or the
host time that corresponds to the start
of the buffer along with the sample time
that that corresponds to so it's
possible to synchronize other system
events such as MIDI events which use
this host time to the audio stream which
are synchronizing based on sample time
the audio buffer argument is where you
can pass a buffer that you expect the
audio unit to render into or optionally
you can pass a null for this and it will
pass back a returned buffer that it did
it caches and let me just
the audio buffer is also a structure
that's used in audio house so we we
share we share the same data structures
between the high level and the low level
parts of the audio stack so with audio
unit render slice
it's optionally possible to schedule
events before a given IO cycle so before
audio unit render slice is called you
may want to do some scheduling parameter
changes with Audio Units set parameter
or in the case of software synthesis you
might want to schedule some notes to
start or stop playing during that that
slice of audio so there's its two phase
scheduled render model and step one you
would schedule events and in step two
you call audio unit render slice an
advanced feature of the rendering model
is the pre and post rendering
notification mechanism it's an optional
callback that a client can install on an
output of an audio unit which gets
called before rendering occurs and after
rendering occurs so every time that
audio unit render slice is called the
audio unit will call this notification
callback do performance rendering and
then call the callback again to say that
it's done rendering and this allows the
client who installs this notification to
perform arbitrary tasks some examples
would be scheduling which you might do
in exactly the same way as it's as I
described here before every render slice
and another use that you might make of
this is to perform CPU load measurement
you can actually measure the time that
the audio unit takes to performance
rendering so how do you write one well
how do units are components
the same kind of components that have
been used in QuickTime for a very long
time so anybody who's familiar with
writing components in QuickTime should
be able to get up and running pretty
quickly with Audio Units for people who
haven't written components before it can
be a little bit difficult unless you
have some help and we have an SDK that
makes this very simple all you really
have to do is override one or two basic
functions and in fact at the base level
you really only have to override one and
that's the function which does your
actual DSP processing if your audio unit
is as a processing type of audio unit
and this SDK will be placed on our
website soon in the next few days we
hope so finish talking about Audio Units
at the lower level I'm going to talk
about a you graph this stands for audio
unit graph and this is set of API is for
connecting these components together
represents is a high level
representation of a set of other units
and the connections between them allows
you to construct arbitrary signal paths
fairly arbitrary anyway and represents a
modular routing system and it can deal
with large numbers of them why use a you
graph one of the reasons is that
connections and disconnections between
audio units can be performed while audio
is actually being pulled through the
chain and if you are talking to the
audio units at the lower level you have
to take great care at at changing
connections distant making connections
and disconnections while the audio is
being pulled through you can get all
kinds of bad things happening there's a
lot of synchronization involved and a
you graph takes care of this for you
another reason you might want to use a
you graph is that it maintains a
representation of the graph even when
the audio units themselves are not
instantiated and it also takes care of
and embodies the state that the audio
units are in as we'll see
hey you note is primary object that's
represented in a you graph an au note
actually represents a particular audio
unit in the graph whether that outer
unit is instantiated yet or not it's a
placeholder representing that audio unit
to create a node you can call a u graph
new node and argument to this function
is a component description which
specifies the type subtype and ID of the
component this is this information that
you would get from fine next component
component manager call it's also
possible to pass this what we call class
data to initial initialize an audio unit
width and we'll talk about that in a
minute
may you graph remove node just takes it
out of the graph to create connections
between these nodes once you've added a
bunch of nodes you've got to connect
node input disconnect note input clear
connections that clears all the
connections in the graph and update for
synchronizing real time changes you can
also walk through the graph and look at
all the nodes with get node count get
ant node get node info these are in the
hey you graph H header file in our audio
toolbox framework and I think that the
api's are fairly self-explanatory if you
looking at header file so we're not
going to go into detail with all the
arguments just like the audio units
themselves the a you graph mirrors the
state of the other units in them and I
and this slide is just pretty much going
over what I talked about concerning the
states of audio units here an important
thing is that a you graph state
corresponds with the audio unit state
so the state API is through our graph
our graph open close so a you graph open
when you call that it actually opens all
of the audio units represented by the
nodes in the graph and close closes all
the audio units and correspondingly
initialize on initialize it calls the
audio unit functions and start and stop
same thing in addition there inspection
or introspection API is how you graph is
open is initialized is running so you
can determine what state the graph is in
oh this class data when an au node is
added to the graph it's possible to pass
arbitrary data in with a void star
pointer and a length similar to the
properties and if you add this
information this optional class info
when the audio unit is opened a property
call will be made on the audio unit with
this data and the audio unit is able to
make use of this data in any way that it
wants it's there's no data format that's
defined currently for it but it's a way
it's a way to maintain persistence
because when the graph is closed
data can be gotten from the audio unit
and stored in the node so it can be
recreated in the same configuration that
it was so a little bit earlier I tried
to explain the notion of this head audio
unit in a graph in a very simple case
you would have a chain of audio units
say you might have three first one
connected in a second
connected to the last one the third
audio unit which is maybe an output
device and this last one represents the
head of the graph and we talked about
the pole map model the head audio unit
is very important because it controls
the hardware it controls what the
processing is is being pulled through
the graph or not and so there are
special audio unit API is our unit start
and stop which I talked about before and
the AU graph
has corresponding calls a you graph
start and stop okay done talking about
audio units and a you graph so finally
I'm going to talk about some sequencing
services for performing scheduling a few
slides back I had a diagram up showing
the schedule render model we're in step
one you schedule events to audio units
and secondly you would render a slice of
audio the music sequencing services
actually would be one mechanism where
the scheduling could occur it's a simple
API for creating and editing multitrack
MIDI sequences individual events in the
tracks can address Audio Units and
software's synthesize their music
devices it provides a runtime for the
playback of these sequences and has a
bunch of api's for doing live edits cut
copy paste merge this type of thing so
while the sequence is playing you can be
editing it's important in a sequence
their application be able to do that for
those developers who already have a
sequencing engine in their application
this set of API is might not be so
interesting but for somebody developing
a new application because the api's are
there for doing the basic edits and for
actually providing the runtime system
for playing back the sequences it's only
necessary to write a UI which is not
necessarily easy but if you're good at
writing UI
at least you don't have to write all of
the underlying engine which can be
difficult sometimes the scheduling uses
units of beats and it's floating-point
32-bit numbers representing
beats and it can also represent
represent seconds to in certain cases
there's a tempo map for converting
between the beat time and real time
seconds API is for reading writing
standard MIDI files tracks have
attributes you can mute solo loop tracks
the music player is the highest level
object and it provides transport so you
can you can seek the sequence to a
particular time and start the sequence
playing as top of the sequence playing
with the music player and it provides
the appropriate synchronization between
host host base processing and MIDI music
sequence is the next level object and a
music sequence contains multiple
playback tracks which are called music
tracks each music track contains a
series of time ordered time-stamped
events and each music track with these
events addresses a particular au node in
in the a graph that we spoke about
before so for instance if you have a
graph with a software synth
there would be a node representing the
software synth and one of these tracks
might be addressing its events to the
software synth there might be another
audio unit in a graph which is a filter
of some kind there might be another
track which is addressing parameter
changes to the filter to sweep to the
filter cutoff
there are also attributes to the track
commute solo as I said before you can
set up the loop length number of
repetitions of the loop and also offset
the start of the track relative to the
other tracks there are several different
types of events the most basic one is a
MIDI note message
it's MIDI note number with velocity MIDI
Channel and a duration the note off is
not a music event as such it's implied
in this MIDI note message using the
duration there's a MIDI channel message
for all the other channel messages
there's raw data for a system exclusive
there's a tempo event for doing tempo
changes and there's this extended note
API which is very much like the MIDI
MIDI note message but it provides for
more sophisticated control and esoteric
applications MIDI note event only has
the note number and velocity and a MIDI
channel associated with it and this
extended note event allows for an
arbitrary number of floating-point
arguments to be associated with the
instantiation of a note not everybody
may be interested in that but it's there
the extended control represents a 32-bit
floating-point parameter change and
ultimately it may end up calling Audio
Units set parameter at the exactly the
right moment in the audio processing for
sweeping filter for changing volumes for
panning positioning audio all different
kinds of parameter changes there's also
a user event to arbitrary data can be
placed and meta event for MIDI so here's
the one example there are there's an API
corresponding to each one of these types
of events and all of them are very
similar to this one so I'm just going to
go over the one this is how you add a
MIDI note event the first argument
is the the music track you're adding the
event to the second argument to think
they got the arguments wind up a little
bit funny but the second argument if you
can make sense of that is the timestamp
it's 32 bits actually I think it's a 64
64 bit it's a double precision timestamp
representing the beat and then you have
a pointer to a structure the MIDI note
message structure which essentially just
has the note number velocity
MIDI channel and the duration of the
note in it so this is a diagram showing
how you can have a music sequence with
multiple tracks three in this case where
each track is addressing a separate
audio unit in an a you graph and this
corresponds with the previous diagram
the two-phase schedule render model if
you remember there are some editing
api's for copy cut paste this type of
thing and I think if you look into music
player dot H header file and audio
toolbox framework it will be very clear
how to use these one of the lowest level
objects in music sequencing services is
a music event iterator and it allows you
to actually walk through the events in a
track see what they are and it could be
useful if you're writing sequence or
application you're writing the UI
sitting on top of this engine for just
walking through the events seeing what
they are so you can display them also if
you want to create custom editing
operations on track you could use this
music event iterator to walk through all
of the events in a particular time range
so there are a couple different api's
for dealing with these iterators you can
the music event and Reuter is a
first-class object so you create it with
new and dispose and you can seek the
iterator to a particular point in the
track it finds the first event at or
after the time that you're seeking to
you can move the iterator one event
forward in time with next event one
event previous with previous event and
you can actually ask the iterator if
it's at the end or beginning of a track
what has previous event and has next
event then the music event iterator get
event info function can be used to find
out what type of events you have at that
points so you can find out what the
event type is what time the event occurs
at and you get a void star pointer which
you cast to the appropriate event type
to get access to the particulars of that
event for instance if it's a MIDI note
event you can find out what it's no
numbers so on you can delete an event at
the point where the iterator is or you
can change the events time okay I'm
going to play a demo it's kind of kind
of a little bit vague what I'm trying to
illustrate here but I think what I'm
trying to show is how all of these
services work together so in the example
I've created a graph I have a software
synth the DLS music device it's loaded
up with a twelve string guitar sample
Bank and it's playing through kind of an
arpeggio really quickly and then I have
some additional audio units in the graph
I think there's a digital delay in there
there's a limiter look-ahead limiter
node to keep the volume pretty steady
and there's a reverb at the end and I
also have a bandpass filter in the graph
whose center frequency I'm sweeping
using the music sequencing services so
I've set up music sequence and have one
of the tracks addressing the node in the
graph
which is the bandpass filter and
sweeping that around and it basically
just ends up sounding like some kind of
ethereal wash but you should hear that
there are these kind of filter sweeps
going in and out and there's a reverb in
there for sure and let's see if I can
weight this machine up and get this
going here okay got my cursor
unfortunately it's not nearly as
impressive as Jeff Moore's multi-channel
demo the other day but this will give
you an idea okay it's I'm only launching
an application actually it's it's
nothing too visual it's just to hear but
I'll I'll show you double click and you
should hear something
[Music]
[Applause]
[Music]
[Applause]
[Music]
so it's a little bit vague but thanks
but it's showing it's showing all the
parts of the system that I talked about
I think bill I'd like Bill Stewart to
come up and he's gonna show us some some
more demonstrations and talk about the
Java interfaces thought I'd might
mention that all of that sound you hid
was just some nerds coming from the tall
string guitar so it's pretty interesting
I think now if I can have the demo
machine but I've got to get my mouse to
somewhere
okay so okay I'll just get this guy
running and
this is cool I can just show you what I
want to show you so this is my java
application and do we have sound coming
out from this
so I'll very quickly walk through the
code with you after this basically I'm
using a sound font that we got the same
sound font the criticism from mu which
is a 12 string guitar and I've written a
java application and we'll go through
the MIDI side of this in the next
session and I'll go through the audio
unit or graph cited in this session and
basically the media application is
listening to the MIDI point getting the
MIDI in messages from this keyboard and
telling the music device to play notes
in response to keyboard input I'm
actually guitarist so I had to use the
12 string and I'll probably have to do
[Music]
[Laughter]
so I'm going to quit this guy and let me
just bring up the project and you have
to just bear with me while I find it and
this will give you sort of some feel for
how this this code looks that Chris has
been taking talking about things so let
me just move it over here look what's
going on
okay so I can barely see what I'm doing
here
alright so there's a bunch of stuff here
just to sort of set the thing running
I'm loading the sound Bank here which
I've just installed with Mac OS 10 we
also wanted to make sure that there were
directories that were available as
standard places for you to put standard
resources that you may want to share
between different applications so in
this case I'm putting the sound bar a
sound Bank in the slash library audio
sounds banks and then this is the name
of the sound font file here and there's
also a mirror of this directory in the
users directory as well so sound banks
that were stored here would be available
to any user that came onto the machine
so you might imagine like a lab where
you could have a setup that you want
available to every user and then
individual users can have their own
sound banks in their own user version of
this thing and there are fine folder
constants that you can use for look for
these in systems that have been
localized and there's other folders
there for audio units themselves if you
want to make them available for
different types of sounds there's a
whole bunch of MIDI stuffie that I'll go
through later on and what I want to show
you is just basically how the graph is
set up and we'll just ignore the first
thing so this constructor here is
creating the graph and that just calls
through to the C API to create the graph
here and then this is me creating two
synth two nodes one for the DLS music
device and i give it a component
description and I already know what I'm
looking for here I could use the find
component stuff to actually hunt around
for components I'm just creating two
nodes I'm creating a music device and
I'm going to use the DLS synthesis the
the that one for that and yeah we're
very hopeful to see other music devices
from you guys like you know some
acoustic modeling synthesizer devices
and
all that kind of stuff and if I get the
mousepad here and then the second one is
the default output unit I'm just going
to create that because I want the sound
to actually come out of the computer and
then to connect those I just tell the
graph to connect the node the output
from the synth into the input of the
output node and just to show you that
there's some stuff going on with the
graph after I create the graph I'm going
to get the note I get the node count
here and I get the index node starting
from 0 to the number of nodes and I'm I
print that out and I can show you what
that guy looks like so you can see there
that it's telling me I've got two nodes
and there's a component which is the
music node and the default output node
and I've lost my mouse again notice ok
so after I do the connection of the
nodes I'm going to open the graph and
this will open the audio units and that
will actually it's really a constructor
until you do open on the graph you don't
have an audio unit you just have the
node itself it's an abstract
representation of the graph so when I
open the graph I can ask the graph to
give me the audio unit that's associated
with a particular node and if we have a
look back up here you'd see that I had
kept this guy around I could actually go
and look in the graph and find out the
nodes here and I could ask her for
information but I've kept this note here
so I know that what I want to do is get
the music device which is a subclass of
audio unit for that particular node and
what we do in the cord
Audio Driver API is is we actually give
you the subclasses that correspond to
the extensions of the component
selectors so you could have a basic
audio unit which is your parent class
and you've got an extension of that was
your output audio unit another extension
is the music device and that in
component terms just provides additional
component selectors additional
functionality if I've specified a sound
Bank which I'd be kind of bit lame if I
didn't then I'm going to set the
property of the sound Bank on the music
synthesis is a convenience routine in
Java does the same thing as what I
crocheted in the C file and it just
takes the Java file i/o object it does
all the necessary conversions to the
native structures and instantiate that
and you notice that I'm doing this when
I've opened the graph but before I
initialize it so this is one of these
things where this is a property that we
want the synthesizer to have when it's
initialized and until the Center is
initialized it's not going to try to use
a particular sound bang until you
initialize it and then just to get the
synth sort of up and running I'm going
to set some default volume and channel
stuff and there's some commented out
code there I'm not going to worry about
and because I want this to be responsive
I must have acquitted tonight so because
I want this to be responsive what I'm
actually doing at this point is to get
the the audio device output unit which
is the default output unit that I'm
sending out to and from that guy I'm
going to get the device that it's
attached to and I'm going to set its
buffer size so that's going to basically
take the default callback buffer which
is about 11 milliseconds 512 sample
frames and I'm going to set that to 64
sample frame so the responsiveness of
the system is about 1 and 1/2
milliseconds so I get creating a memory
object that I can pass that information
in and then I just set the output
properties straight on the device
so I get the device from the output unit
and it's actually a device output unit
and I set the property to 64 frames
which is 64 times 2 because it's a
stereo device times 4 which is the
number of the size of the float and of
course in Java you don't do size of
light because
it's Java and not serious math and then
I actually get that property as well and
I print it out and you can see if I can
find my mouse and you can see there that
it's telling me that the buffer size for
the i/o clock is 512 which is what we'd
expect 64 by 2 by 4 so that's all we do
and then after I've set the buffer size
up and I've initialized the synthesizer
I then just start the graph and that
will start the device that will start
pulling through the chain and away we go
and then in the next session I'll go
through how I you set up the MIDI
pausing stuff so that I can actually
send MIDI events to the music device so
I can we go back to slides ok so just
some notes about the Java there's also
Java doc available on the on the website
we're also preparing a PDF of just the
core to architecture itself and they'll
be available at the web site early next
week
and as we stressed yesterday we sort of
look at coreos basically an architecture
that you should understand how it works
and that's really a language neutral
feature that the job and the C API is
represent the same API to the same
functionality that's implemented
underneath and in a variety of different
ways so we also have a mailing list it's
at least our apple.com it's called core
do api and we welcome you to involve
yourself in that mailing list and it's a
good way to send information to us and
we will answer questions on that list
and there's a website I don't know if
it's completely up yet if it's not up
today will be over the next few days
developer.apple.com slash audio and
there's some related sessions for the
DVD viewers of this wonderful thing and
the next session which should be MIDI
is at five o'clock today audio services
and Mac os10 was yesterday and there's a
sound networking ghost before this one
if you're have any questions related to
firewire in USB
there's a lot of fire wire a lot of USB
audio devices and many devices and stuff
that's coming out you can contact Craig
Keithley Keith lead Apple comm if you're
interested in in seeding we obviously
you know constantly working on these
technologies if you're interested in
getting access to that before we release
it publicly you can contact us at audio
Apple comm or talk to your developer
tech developer relations manager and
inquire about seeding
you