WWDC2000 Session 175

Transcript

Kind: captions
Language: en
hello I'd like to thank you all for
coming this afternoon this is a very
exciting session to be involved in and
it's actually one of three sessions to
do with Mac os10 support for audio and
MIDI this session will cover both the
MIDI support that we're providing an
hourish 10 and it's the first time in a
number of years that Apple has made a
serious commitment in this area and as I
said it's very exciting to be involved
in this project there's two more
sessions on Friday there late in the
afternoon starting at three-thirty are
covering both the iokit area of audio
support audio device support and the
user application interface to audio
devices and encourage you to go to that
and then after those sessions on Friday
they'll actually be a party that will
give you some more details on later so
without any further ado I'd like to
introduce Doug white will be talking
about the media services on hours 10
thank
[Applause]
good afternoon I'm Doug wide I'm a
software engineer in the core audio
group and I'm here to tell you about the
new MIDI system services on Mac OS 10
before coming to apple iwork tadap code
systems for about 12 years working on
MIDI applications like many of you and I
was the author of all math there so
here's what I'm going to talk about
today I'm going to talk a little about
the history and the goals of our new
system services to the history of MIDI
on the Macintosh because that will help
you understand our goals I'll go over
some of the key concepts in the midi API
we'll look at some of the functions in
it I'll talk a little about the
performance challenges of getting MIDI
working on a multitasking modern
operating system like OS 10 and i'll
tell you about the availability of the
new midi system services so going back
to the mid-1980s the first macintosh
MIDI interfaces connected directly to
the serial hardware and developers would
write to the serial ports directly
because it was most efficient and I
don't think the serial drivers supported
external clock in those days and that
was fine and good for a while until the
late eighties we started to see
multi-port MIDI interfaces like Mark of
the unicorn Smitty timepiece we started
to see sound cards like digidesign smack
proteus from sample cell and they're
started that there became a need for
system software for applications to deal
with these different kinds of hardware
in a hardware independent way so in 1989
we saw apples MIDI manager which was the
first attempt to solve this problem
unfortunately MIDI manager had some
performance problems and it got kind of
unwieldy on when you tried to use it in
large studio environments opcodes nice
OMS which came out in 1990 to address
some of these limitations with MIDI
manager and in the course of a few years
it became a de facto standard
most most programs on the Macintosh not
supported on ls9 people making MIDI hard
work tend to write OMS drivers for their
hard work and it's the way it's be only
weighs a lot of these applications in
draw and hardware talking to each other
now but because OMS was controlled by a
competitor and now that it isn't being
supported anymore as far as I can tell
or it well it continues to work on OS 9
but it's not being developed any further
and eternally the prospects for it to
work law in the light stand are not good
so that combined with the fact that
developers have been sort of chipping
away at MIDI compatibility with the
problem of software since and so on we
see a need for Apple to provide a single
set of MIDI system services for OS 10 so
that's our main goal I know it's time is
to provide a single standard so that
everyone's hard work and software can
play together nicely again we so too in
support of that goal we want to focus
just on those basic midi i/o services
and doing those basic video services
with highly accurate timing meaning low
latencies and low jitter also we want to
make the midi services open source so
that you as developers can see what
we're doing help us fix things if we're
not doing them right and not have to be
afraid of repeats of the fates of midi
manager and OMS so looking at the midi
system services in the big picture of
the rest of the OS they are layered
above the eye out yet in the colonel I
oak it is where we see drivers for
talking to hardware we have a concept of
many drivers applications can talk to
the MIDI services which in turn clocks
committee drivers for higher level
applications of MIDI like just playing
MIDI files you can use the quicktime
music device api's and Chris Rogers will
be calling you a bit about that in the
second half
discussion so here are the main pieces
of the new MIDI services we have a
driver model for many drivers
applications can share access to
hardware meaning that multiple MIDI
applications can send simultaneously to
the same device and a MIDI device can
send MIDI into the computer and multiple
applications can all receive it we
timestamp all the MIDI input we
scheduled all MIDI output in advance for
applications we want to do that and all
that scheduling is done using the most
accurate timing hardware on the computer
the host clock as returned by uptime the
MIDI services provide a small central
repository of information about the MIDI
hardware that's present they don't try
to replicate the full functionality of
the UC and OMS and free midi where the
user you know with 15 synthesizers can
enter information about all that when I
say there's a repository of device
information that's just about the actual
dinner MIDI interfaces and cards for
president we don't at least for now
become concerned about the devices that
are attached externally to MIDI
interfaces and the MIDI services provide
some basic inter-process communication
so if you have a small midi utility
maybe a transpose azure arpeggiate sore
something you can you can use the MIDI
services to create an application like
that so now I'd like to get into some of
the objects and functions in the midi
API first there's the MIDI client and
for those of you who've used midi
manager in lms this is a familiar
concept the first thing you typically do
is create a MIDI client object and
that's done with MIDI client create
after creating your clients then you can
create midi port objects this is also
similar to many manager in lms reports
or objects through which your programs
can send and receive midi messages
and they're created at startup so here
we see midi input port creates and many
output port create which create an input
and output ports obviously the my read
proc parameter is passed to my MIDI
input port create is a procedure through
which that port will let me rephrase
that but that procedure will get called
when MIDI comes into your program
through that port all the MIDI io
functions use the structure MIDI packet
list when sending and receiving many
it's simply a list of MIDI packet
structure can be any length for now and
those MIDI package structures are
themselves variable length structures
maybe packet contains one or more
simultaneous MIDI events with one
exception if you have a system exclusive
message it has to be in its own MIDI
packet and the reason for that is just
to make our own internal parts inc
simpler and for similar reasons running
status is not allowed inside a MIDI
packet but otherwise comes the data
array portion of the MIDI packet is a
little mini stream directed at one
device and many packets are time-stamped
and I'll talk a bit about that later so
dealing with variable like structures
like MIDI packet and MIDI packet list
can be a little annoying so we provided
a few simple helper functions there's
next committee packet which you can use
when dealing with the MIDI packet list
you can get a pointer to the first
packet in the packet list then use next
midi packets to advance to the next one
and so on when sending MIDI you can use
MIDI packet list in it and MIDI packet
list add to dynamically build up a MIDI
packet list and here's an example of
using them here I'm creating a buffer of
1k bites on the stack I'm casting it to
MIDI packet list
I call MIDI packet listen it on it to
initialize it and then I'm adding a
simple MIDI note on event to it with
MIDI Tackett let's add that this for
more complex examples and I could go
adding more events we played at
different time and build up a MIDI
packet list when MIDI packet list add
returns null then I know that it's
become full and it's time to send it and
I can call me a packet list in it and
start calling maybe tag and looks bad on
it again okay now that we've looked at
the structures representing MIDI data
itself here's a subject that represent
MIDI sources and destinations in the
system Wow the lowest level object is a
MIDI endpoint which is a simple MIDI
source or destination and a single 16
channel midi stream so your program can
have one simple view of the system has
an array of sources and destinations and
here's some example of here is an
example of finding all the destinations
in the system and sending a MIDI packet
see each one midi get number of
destinations and MIDI get destination
are used to walk through all the
destinations and then MIDI send takes as
its first argument an output port which
you created it start up the destination
which was just returned from MIDI get
destination and then a MIDI packet list
so in this example we're sending the
same MIDI packet list to all the
destinations similarly here's an example
of how to find and receive all the MIDI
sources in the system find an open input
connections to all the MIDI sources in
the system we call mitigate number of
sources and MIDI gets forced to walk
through all the sources and then we call
midi port connect source to establish a
connection from that midi source to your
programs input port now the reason we
asked you to create such connections
explicitly and this concept is familiar
to those of you who've used
math it says that if we've got a bunch
of MIDI sources sending stuff into the
computer there's we only incur the
overhead of delivering that midi to your
application when it's from a source that
your application cares without listening
to and if you remember when we created
that input port at the beginning of the
program we passed my read proc and it
gets called when that midi comes in a
note about the read proc the MIDI
library creates a thread on your
programs behalf to receive that data so
similarly to the way on Mac OS 9 for
those of you who've done many
programming there before your MIDI would
come in at interrupt level and you'd
have to be careful about critical
regions and not accessing memory I know
it's 10 you need to be aware that your
read proc is called from a separate
thread and you may have synchronization
issues with any data you access from
that thread and also be aware it's a
high priority thread don't do too much
work there okay so those are many
sources and destinations the next higher
level object in the midi api is midi
entity which is a logical sub component
of a device which groups together some
number of endpoints for example you
might have a USB device which has a
general midi synthesizer in it and a
pair of MIDI ports that device can be
thought of as having two entities
synthesizer and the pair of MIDI Jack's
an eight port MIDI interface with eight
ends and eight outs might be thought of
as having a tentative each of them with
a source and a destination that point
and the reason we have this concept of
an entity is so that if your program
wants to communicate in a bi-directional
manner with some piece of hardware out
there you have a way of associating the
sources and destinations you know which
wich one constitute a pair
the next level up in the middie API is a
MIDI device which represents an actual
physical device like a MIDI interface a
card something that's on fire wire it's
something that's controlled by a driver
the driver for that device will have
located it and registered it with the
system and so this diagram here
illustrates how many devices contain
excuse which contain endpoints and so
here's a quick look at the functions you
would use to walk through the system and
locate the devices and entities that are
present midi get number of devices and
midi get device will inter iterate
through the devices and mitigate number
of entities and maybe get entity will
walk through the entities that are
associated with the device okay now
we've looked at devices entities and end
points there's a set of calls to find
out information about those devices
entities and endpoints and we call these
attributes properties the property
system is extensible meaning that anyone
can make up a property to attach to
their device but we've designed a few
simple ones like its name its
manufacturer name its model number the
MIDI channels that it's listening on it
if someone knows that so the system is
extensible properties can be inherited
which means that if you ask for example
an endpoint what is your manufacturer
name we'll probably end up with the
devices manufacturer name because the
driver rider will probably just that
here's my device I'm the ABC corporation
and here's my d4 device and you know
he's just attached that to the device
but the entity will inherit that
property from the device and as I said
mom for now properties are most likely
only going to be set by drivers here's a
simple example of obtaining a property
of an object we use the MIDI object get
string property call Pat
the constant K MIDI property name and we
get back a CF string which is the name
we can convert it to a C string print it
and then release the cs string rep that
we got back CF string wrapped this part
of core foundation which you can read
about in our documentation okay is the
highest level object in MIDI API is the
MIDI setup which represents a saved or
savable state of the system it's
essentially just a list of the devices
the drivers located the MIDI interfaces
and cards that are present but we do
have facilities there for keeping track
of some other details like which drivers
device on the serial port which device
does the user prefer for playing back
general MIDI files or whatever here are
some examples of how your program can
manipulate midi setups those of you
who've used on that might be afraid of
the term setup because there was that
own a setup program that not everyone
liked but we don't have any user
interface involved here although at
worse I could envision a dialog where
the user has to authorize serial ports
to be sort of searched but in any case I
don't think we're going to have any any
user interface here MIDI setup create
simply tells the system to go
interrogate all the drivers find out
what hardware is present make a MIDI
setup containing all those devices and
return it then you would almost always
call MIDI setup install after calling
MIDI setup create which just tells the
system here's the MIDI setup make that
the current state until someone else
tells you otherwise MIDI setup get
current returns a reference to the
current MIDI setup and then there's MIDI
setup to data and MIDI setup from data
which allow you to convert a MIDI setup
object to and from a textual
representation which is an XML and can
be saved to a file
okay so that's the tour of some of the
objects to which you send and receive
many now I'd like to talk about some of
the issues of timing when sending MIDI
and receiving it your programs will
probably want to schedule MIDI output a
little bit in advance and you can do
that by using timestamps the timestamps
and the MIDI packets we looked at
earlier use the host clock time as
returned by up time we suggest you don't
schedule events too far in advance if
you schedule a whole five minute MIDI
file to be played it'll play and you
won't have any way to tell it to stop
lets the user quits your program
probably so we suggest that you use a
number say 100 milliseconds or so as a
guideline of how far in advance to
schedule that's a short enough period of
time to be relatively responsive as user
says stop but it's also far enough in
advance so that if you're talking to a
piece of hardware that's got some
latency and talking to it we can still a
timing accuracy and sending events for
that device an important thing you're
about whoops pushing buttons here an
important thing about scheduling MIDI
output is that we have support for
devices for pieces of hardware that are
capable of doing their own scheduling
and sending of MIDI a driver may attach
a property to its device that says I
want to schedule event for this device
some number of milliseconds in advance
so you as an application rider should
check that property and see if the
driver rider has has attached that
property to the device and respect it so
if the driver is saying i want my maybe
five milliseconds in advance please then
you as an application writer can get the
best timing from the system by making
sure that you're milking your MIDI
events get sent five milliseconds in
advance or more
a few timing issues on incoming midi we
time stamp it with the host clock as
soon as possible and to schedule your
own tasks there's a lot of different
ways to do this in Mac OS 10 there's a
number of AP is that we're recommending
that you use the calls and
multiprocessing that H which is part of
carbon for those of you who are writing
MIDI drivers for your own many hardware
[Music]
many drivers are packaged as CF plugins
which are a little bit intimidating at
first glint at first glimpse when I
first looked at it I said this looks
like calm I'm scared but it's not that
bad we have some example drivers you can
build on and it makes it pretty easy
usually you won't need a kernel
extension and this is true if you're
writing a driver for a USB firewire or
serial midi driver if you're writing a
pci cards driver then you will need a
kernel extension but usually like four
USB in my example drivers on just a USB
user client and these terms are familiar
to those of using the i/o kids sessions
and for those of you writing drivers I
recommend you go find out more about IO
kit if you haven't already the driver
programming interface from the MIDI side
of things is pretty simple there's just
a few calls to implement there's one to
locate your hardware their calls to
start and stop communicating with your
hardware there's a call to send some
MIDI events to your hardware and when
you receive incoming MIDI events and
there's a way you can call back into the
MIDI system to have those many events
delivered I just wanted to make a quick
note here about how having looked at the
source code for USB drivers on OS 9 and
written one on ls10 it's an order of
magnitude easier least I thought so I
know I stand to write USB driver and
that's very encouraging
okay here's a diagram that shows the
pieces of the committee implementation
at the top we see your client
applications in green we supply a MIDI
framework which is a client-side dynamic
library that library communicates with
the MIDI server process and the reason
we have that server processes so that
incoming MIDI from some piece of
hardware can be efficiently distributed
to multiple applications below the MIDI
server we see that it loads and controls
the MIDI driver plugins and those driver
plugins communicate with IO kit now
you'll notice in the diagram the
horizontal gray lines both indicate
address space boundaries for different
processes so we have the colonel and the
midi server and your client applications
address spaces which brings us to one of
the main performance issues in dealing
with committee which is moving data
between different protected address
spaces now we've got some pretty good
and fast mechanisms for doing it but
nonetheless it's still important to be
aware of that and there are some things
you can do in your program to squeeze
extra performance out of the system when
possible do schedule your output a few
milliseconds in advance and look at that
property of the driver to see if it
wants to get data a little bit ahead of
time and this will especially enable you
to send multiple MIDI events that happen
close together in time with a single
call to MIDI send so instead of sending
one MIDI event at the time if you
package up just even the few
milliseconds of data at a time was
called committee send that will help the
system be a bit more efficient and we do
that i should mention i just wanted to
say we have a tendency on the core OS of
scheduling mechanism and things are good
there but they're getting better okay to
show you that midi is actually up and
running i know it's been to some extent
i've got a demo setup here
at the bottom you see I've got a MIDI
keyboard and sound module I've got a
MIDI interface that blue thing and that
connects to the computer by yo speak and
over on the right there we see the
various layers of software through which
midi messages travel when i run these
programs okay first I've got a simple
program which just plays a series of
MIDI notes at very regular intervals
using the scheduler built into the MIDI
server and hopefully we'll hear that
they're very nice and regular a little
latter place that's pretty good in
regular founding I think I've got
another program here here I've just got
a keyboard playing its own internal
sounds and now i'm going to play i'm
going to run a program that'll take the
midi from the keyboard send it to the
USB interface to the computer through
the whole stack of software back down to
the interface and after the sound module
so when i play the keyboard along when i
slide the keyboard i should be hearing
it sound as long as as well as one in
the sound module and we shouldn't be
hearing any delays or variations in that
delay
[Music]
that sounds pretty good too i think i
don't think the ladies are excessive or
anything I've got a midi file i really
like here i'm going to play a little bit
of it
[Music]
so it seemed pretty pretty been Smitty
big-play got one more little mini pilot
like to play for you this time i also
i'm going to play along with a little
bit using mid-eastern midi through the
computer that is I'm not showing you
what I'm actually doing here because
it's ugly and just learning terminal
based programs on Austin seems like to
remember what the type
[Music]
I hope that shows is that video is real
on Mac OS 10 so the next thing you're
probably wondering is how can I start to
make my applications work with MIDI on
OS 10 the MIDI services are not part of
developer preview for their just they've
just been coming together in the last
couple of weeks but we are just about
ready to start speeding so please write
to Dan Brown who's here in the front row
and we've given you an easy-to-remember
email address audio at apple.com we are
still holding out the possibility of
tweaking the api's a little bit based on
your feedback and our own release
process but we're basically in a mode of
optimizing stabilizing and getting ready
to release it as part of the mac OS x
public beta this summer and i want to
remind you that it is open source so I
hope I've given you get in good
introduction to MIDI on Mac OS 10 and
I'm really looking forward to seeing
your applications to use it thank
[Applause]
thanks Doug I'd like to bring Chris
Rogers out now Chris has been working
with Apple for about a year and has been
doing quite a lot of work on the
quicktime music architecture and is also
going to be discussing some of the
high-level audio services that we're
providing to applications developers in
general good afternoon as bill said my
name is chris rogers and i'm happy to be
here to talk to you today about music
services available on ls10 the topic
that we'll be covering today our new
synthesizer replacing the current
quicktime music architecture synthesizer
it's a dlf software synthesizer and
we'll be discussing that in some detail
we'll talk about the audio unit and
music device component architecture how
to actually hook these guys together in
different configurations and what that
means will become clear later on the
sequencing services and downloadable
sounds toolbox so let's get on with it
ok so where do music services fit in
with the rest of the core audio system
the music services are higher level
services that's it both on top of the
the MIDI server the rest of the MIDI
services that Doug has presented and
also the audio i/o devices and Jeff more
will be discussing this in great detail
in a later talk on friday i really
encourage you to go to that that would
be the core audio multi-channel and
beyond presentation at two o'clock on
friday and both of those systems
actually sit on top of i/o kit and you
may be interested in how to implement
audio drivers
and there will be a talk also on Friday
about audio family iokit drivers so the
music services are available to all
clients and also the MIDI server and the
audio i/o devices are directly
accessible by the client so depending on
the level of access that you require
very low level control you may just want
to go right down to the MIDI server and
I o devices or higher level control you
can talk to the MIDI services we're
dedicated to supporting open standard
that includes MIDI course standard MIDI
files are mid files are midi files are
standard MIDI files with a dls section
in the file pls dance or downloadable
sounds and that's a sample bank format
where people can include their own
custom samples sound effects
high-quality sample 16-bit stereo that
you want if you want and also we're
incorporating some of the ideas included
in mpeg-4 structured audio we're not
going as far as implementing fail or
anything like that for those who know
what mpeg-4 is about but we've
incorporated some of the better ideas
and mpeg-4 the dls soft sent this is a
synthesizer software synthesizer that's
been completely rewritten from scratch
to replace the synthesizer currently in
QT ma it's among other things it's got a
much better reverb several different
types basically just sound smoother and
what's even better is the reverb isn't
hard coded into the sense it's it's
implemented in a modular way so that
third parties can conflict in their own
reverb and other effects we'll see how
that works in a little bit what else is
in this off chance has much tighter
scheduling of notes so that you don't
get this kind of slop that you might
have seen on other synthesizers
scheduling it sample accurate it is a
downloadable sound sense and it allows
for easy importation of high-quality
third-party sample banks so don't
necessarily have to be stuck with cheap
8-bit sound set you can load in not only
general MIDI sound sets but arbitrary
sound sample banks for your own customs
music it's very general sample bass
synthesizer and below pure exponential
as per the downloadable sounds
specification there's a two-pole
resident filter in their unlimited key
ranges velocity ranges and layers the
layers let you actually stack multiple
samples when you hit the same key and
you can have individual panning and
modulation parameters on each one so you
can get nice fat rich pads that way and
also dls provides for much more flexible
modulation routing possibilities in the
old cutie ma sent okay now we're going
to talk about audio units now what is an
audio unit we're going to see in the
next couple slides what that really
means at its most abstract level it's
kind of a box that deals with audio in
some way it would take in n audio
streams and output m audio streams and
the number of inputs and outputs can be
variable in fact you may have an audio
unit that has no input or no output and
in at least one case there may be an
audio unit that has no input and and no
output and you may think what would that
be and that would be maybe an audio unit
that represents an external MIDI device
and we'll see how that can wrap up a
MIDI endpoint through the MIDI many
services that Doug Doug spoke about
earlier there are other types of audio
units that have have no inputs a an
audio unit is representing a hardware
input device would only have output and
vice versa hardware output device would
have only
put and PSP processors would typically
have both inputs and outputs for
processing audio some examples of DSP
processing modules would be reverb
chorus delay ring modulator parametric
EQ put a stereo mixer and there it's not
really probably didn't really belong in
that category but stereo mixer would be
an audio unit that takes in multiple
inputs and mixes according to volume and
pan information to to stereo output
another type of audio unit would be
format converters for sample rate
conversion bit depth conversion this
type of thing and also codecs like mp3
code coders and decoders another type of
audio unit is one which at high level
abstract the notion of a hardware input
device that would be layered on top of
the audio i/o device aight guys that
Jeff will be talking about on Friday
another audio source would be a software
synthesizer and this is an audio unit
which is called a music device which
supports some additional API is over the
audio unit an audio destination that's
another type of audio unit and that
could at a high level abstract the
notion of a hardware output device and
that would be implemented in terms of
the low level audio i/o api's and also a
file you could have an audio unit which
just writes its output directly to a
file so this just gives you kind of a
flavor of what the range of behavior
that an audio unit can can exhibit ok so
these individual audio units they're the
kind of interesting on their own but
they get even more interesting when
you're able to hook them together an
arbitrary configurations it was our goal
to have an architecture that lets
developers connects these modules up in
arbitrary ways not to
in linear chains like the current sound
manager can do and not just some kind of
a monolithic mixer architecture with
fixed send returns but we were going for
a fully modular approach where these
audio units can be connected in and
pretty sophisticated ways to create all
kinds of different interesting high
level high level software the
connections between these these audio
units is represented by an au graph
object and there's a whole set of api's
we're dealing with a you graph I'm not
going to go over too many specific api's
and in my top because there are so many
of them and I'm covering so much
different so many different topics
actually put the a you graph it in
essence it represents a set of audio
units and their connections like I said
there's a simple API to create and
connect them together and api's for
actually persisting the state of the
graph so that you could save the state
to a file or from memory and then
reconstruct the graph based on that
let's look a little bit at the client
API of the audio unit are you going to
have properties and you can get at those
properties with are you going to get
properties that property and has
property properties are keyed by ID and
an idea is really just an integer that's
all it is some of our IDs are predefined
and others can be defined by particular
implementers of audio units or third
parties can define their own custom
properties some examples of properties
would be a name number of inputs so the
client if the client is interested in
how many inputs and what kind of data
format these inputs take the client
would call get property with the
appropriate ID data is pacified void
star and length so arbitrary data could
be passed back and forth between the
client and audio unit and third parties
can pass custom data back and forth in
this way a real time parameters often
it's interesting at a real-time system
to to make parameter changes in real
time where these parameter changes occur
at very specific times so the other unit
get parameter and set parameter are the
api's that are used for this the
parameters are also key by ID and our
32-bit floating point values unlike the
MIDI continuous controllers which are
limited to a resolution of 7 bits which
is positioned for some things but very
insufficient for other types of
parameters like like pitch and filter
frequency are our values are 32-bit
floating point we feel that's resolute
enough to cover almost any need the set
parameter call includes a timestamp for
when that parameter change should take
effect so you so that client is able to
very accurately schedule very precise
parameter changes some examples of
parameters are channel gain and pan for
a stereo mixer filter cutoff frequency
or for that matter residence in a
low-pass filter delay time for a course
delay effect there are many others one
of the most important things you want to
do with these other units is actually
get access to be rendered the rendered
audio coming out of one of the audio
output streams and the audio unit render
call is used for this the client passes
in time stamping information for when
the audio buffer is to be presented in
the audio stream the audio is also
rendered for a specific output so if not
unit has say for different outputs audio
unit render would be called four times
once for each output
and in the internal implementation of an
audio unit in order to do signal
processing like come say a filter
low-pass filter how does the audio unit
actually read it input in order to do
the processing and then pass the result
back to the client who calls audio unit
render well the audio unit actually
reads its input by calling audio unit
render on another audio unit which
provides its input and audio unit knows
which one that is which is its source
because the connection has been
established for a head of prime okay let
me talk about the two-phase scheduled
render model on the bottom of this
diagram you'll see there's a timeline
time is progressing from left to right
and this is representing an audio stream
for a particular output of an audio unit
and we see that the audio stream is
divided into Little Miss diagrams
there's three different time slices but
conceptually can imagine an audio stream
being divided up into many different
small time slices for which processing
occurs so first of all for each time
slice events are scheduled time very
specific time stamping information is
provided for these events and secondly
the audio is rendered for each of the
outputs so for instance in the first
phase if this is a software synthesizer
all note events which apply for this
given time slice are scheduled and then
the audio is rendered the music device
is actually an audio unit which extends
upon the audio unit api's with
additional AP is that are specific to
synthesis and the music device also
replaces the note allocator and music
component that currently exists in
quicktime music architecture what kind
of additional api's does the music
device support mainly the api's center
around scheduling notes when notes start
one note stop the first protocol that's
used is just that the MIDI protocol
which everybody's familiar with and all
music devices would be expected to
support this protocol the second
protocol is an extended protocol which
allows for variable argument note
instantiation what does that mean really
in the MIDI protocol there there's only
two bits of information which are
provided for a note on event that is a
note number and a velocity so which note
on the keyboard is and how hard did you
hit it but that may be insufficient for
certain types of more complex
instruments and there may be certain
interesting applications where more
information could be provided for a note
instantiation for instance where to
position a note in 3d space so
additional information can be provided
in this variable argument note
instantiation another example would be
in a physical modeling synthesizer of a
drum it's for anybody who's actually
played a hand drum they know how subtle
changes in the position and how hard you
hit it and how flat your palm is or if
you hit it with the tip of your fingers
it makes very subtle changes in the
resonances that come out of the drum
that sounds completely changes in the
character of the tone so this type of
information could be passed in the
variable argument no substantiation this
extended protocol also supports more
than 16 midi channels and more than 128
controller messages music sequencing
services we're moving on to a different
topic here this represents a whole
nother set of api's which once again I
not able to go through in detail because
there are so many of them and I don't
have a limited time to talk but
essentially this is a set of a GIS for
constructing and editing multitrack
sequences whether they're MIDI sequences
or
using this extended protocol there's
also a run time for the real-time
playback of these sequences and that's
otherwise known as a sequencer the
events themselves can be like I said
MIDI events they can be in the extended
format and that can also be user events
which have user defined data in them and
that's up to the developer to decide how
to use the events actually address Audio
Units and music devices and external
MIDI gear through MIDI endpoints
indirectly through a music device
encapsulation so what can we do with
these sequences I've already shown
previously a slide where there are these
Audio Units connected together in
arbitrary configurations and here we
have three Audio Units so you know we
have one and two feeding into the third
one and now to the side here we see a
sequence that has three tracks and
events from track one are addressing
audio unit one track tues addressing
number two and track freeze addressing
audio unit number three the yellow the
sick yellow arrows represent the flow of
audio through the system and these blue
lines extend blue lines represent
control information scheduling
information being supplied to the audio
units if you remember back to the render
schedule diagram that I had a few slides
earlier you'll see that the sequence is
actually providing the schedule part of
this and the render part is actually
being pulled through by the audio units
themselves here are some features that
the sequencing service would slide this
basic cut copy paste merger place as you
would expect in a in a sequence or
application once again this does not
supply any user interface this is just
the low-level engine which will perform
this editing so you just slap a UI on
top of this and you're ready to go
track oh yeah also these edits can be
done live while the sequence is playing
so there's no difficulty there each
track in the sequence can have
attributes mute solo and looping
attributes so you can have a track which
is actually looping over and over again
on the same event and the loop time is
of course completely configurable the
sequencing services could be used as a
core sequencing engine for a sequencing
application so one of the most difficult
things in a sequencing a sequencing
application is to write this sequencing
engine not that writing user interface
code is easy but at least this much work
is done so this is an opportunity for
developers to leverage our our
technology here the scheduling uses
units of beets in floating-point format
there's an implicit temple map in the
sequence and the sequence formatted
persistent can be saved either to
standard MIDI file format or a new data
format in QuickTime which we're in
process of defining and we welcome your
input there as well okay now I'm going
to show you a demonstration using the
sequencing services it's actually a
simple little C program I wrote it's
just one or two pages long and it's just
basically calling into these sequencing
ati's it's not meant to be a musical
composition just kind of a basic
run-through of of what the sequencer can
do what we going to here is a cycling
through of general midi percussion and
after a while you'll hear a resonant
filter come in with the filter sweeping
through back and forth and on top of
that you'll hear
an AM module amplitude modulator and I
implemented this by actually connecting
the dls soft scent which is an audio
unit to a resonant filter and following
the resident filter I put an amplitude
modulator audio unit and then I created
a sequence that actually addresses are
ammeter changes in real time sweeping
the frequency of the filter and changing
the modulation of the amplitude
modulator I realized its kind of echo we
in this place so please bear with me
hope you can get the gist of what I'm
doing
[Music]
once again this is really nice
this this is just a simple simple
example it's C program one or two pages
of code something I slapped together
pretty quick didn't have any user
interface available to me to author
anything more interesting but you can
imagine if you had more complicated
setups of audio units representing a
number of different kinds of some
processing and it's reverbs and delays
and so on you could get quite a lot more
interesting set of effect so we got to
kind of wait until we have more of a
library of audio units built up let's
see where did I leave my little remote
okay now we're going to move on to
talking about the downloadable sounds
toolbox once again this represents a
whole set of api's which I don't have
time to go over individually but I can
talk about at a broad level downloadable
sounds is both a sample bank data format
and it's a sample based sense model the
tool box provides for reading and
writing VLS level two files and creating
arbitrary VLS instruments it could be
used as a foundation for a really nice
custom instrument editor application so
that users can drag in their own samples
apply envelopes and LFOs and panning and
layering and so on so this is a really
good opportunity for third parties this
format also replaces quicktime music
architecture atomic instruments format
at the top level the dls toolbox uses a
number of objects in its api's the dls
collection is at the top the collection
references the number of instruments
instrument representative reference
number of regions and the collection
also references the wave data as VLS
wave objects a dls collection contains a
set of instruments as I said it
references the wave data and it also
includes text based information could
include copyright information the name
of the collection author comment any
kind of tagged text that user wants to
put in there an instrument is assigned
to a particular MIDI bank and program
number and contains a set of regions and
also some articulation parameters low
frequency oscillators envelopes etc and
also contains text information like the
collection does dls region actually
references the sample data that's going
to be played and contains a loop points
and defines where in the key range we're
on the keyboard the sample will play and
in what velocity range and like I said
these regions can be stacked for
layering and also these regions can
contain articulation information like
envelopes and Ella phones which would
override those found in the instrument
and text information the dls wave object
contains the actual sample data and the
sample format for that data and the DLS
articulations object that's what
actually contains the LFOs that envelope
the reverb sends level and all the other
modulation information panning and these
objects can be attached to the DLS
regions or the dls instruments as we saw
and there's a simple set of api's for
accessing and setting the relevant
information in each one of these objects
and connecting them together and
it's really a lot easier to use this API
than to try to create a dls collection
by doing a low-level bite lunch and
leave me backwards compatibility I want
to say that we are supporting the old
quicktime music architecture components
they have been reimplemented the note
allocator and the music component the
old software sense have been
reimplemented on top of all of this new
technology but we are deprecating the
api's for these components they continue
to work but we are really encouraging
developers to move over to the new audio
unit music device api's and the
sequencing service API so I would want
to wrap up my my presentation here and I
guess we can move on to Q&A session with
Bill thanks for giving me time and hope
you all find a use for music services
just before we actually get started on
the Q&A I'd like to say that the
synthesizer that's been quick time on
the OS 10 discs that you have on dp4 is
actually the new synthesizer we haven't
publicized the api's yet us were still
actually going to review stages for
those eight guys but the sample sets us
on the DP for CD is that an a big sample
said it's the same sample sympathy I
can't give you my engine users but the
actual synthesize the synthesis engine
itself is the new music device
components with chris has been talking
about today and you know that will be
available seceding a little bit later as
as dark discussed in his talk as well if
you send email to audio at AFRICOM
you'll be able to get access to setting
information will be setting up setting
with and you must feel
you