WWDC2001 Session 209

Transcript

Kind: captions Language: en thank you for coming this is a session on audio processing and sequencing services and just before I bring Chris Rodgers up is going to be doing most of conversation just thought I'd welcome you my name is Bill Stewart on the engineering manager of the Chorio - if you missed the talk yesterday we talked just very briefly about the audio hardware layer that we have in Mac OS 10 and all of this sort of fits in the general frame of what we call core audio and aside from the audio hardware stuff yesterday we talked about the default output unit as a way of very easily having your application get up sound out the current output device is selected by the control panel so what we're going to be doing today is to talk about Audio Units and Chris will go through that in a moment and the default output unit is one of them so if you were at the talk yesterday and a little bit confused about well that fits in you should be quite clear by the end of this talk we hope so without any further ado I'll bring Chris Rogers up from the cordilla

• thank you good afternoon pleased to be

here to speak with you a little bit about some of the higher level services available for audio on Mac OS 10 overview the talk is first thing I'm going to talk about is audio unit architecture audio units are components for processing audio or just dealing with audio in some general manner and then I'm going to talk about the audio toolbox hooking audio units together and processing graphs and lastly I'll be speaking about some timing and scheduling services that are available using music sequencing services so let's take a look at where these higher level services sit in our audio stack Audio Units are one level above the core audio audio how api's which Jeff spoke about yesterday and then above that we have what are known as the a you graph services which are a set of API is for connecting these audio units together and the music sequencing services will see can talk to au graphs and audio units and core MIDI is off to the side there and later on Doug will be going over that in quite some detail I'm actually showing how there are some relationships between the MIDI services and in some of the audio unit technology that we provide so audio units are components components that are handled by the component manager that QuickTime makes wide use of like all components individual components are identified by a four character type subtype and identifier field and the component manager can query a list of available components of a given type Audio Units are the ones we're interested in this case so you can call find next component to go through the list of available components and then once you've gotten to the one that you're interested in opening you can call open a component and to close it you call closed component and the component manager has some pretty good documentation so you can refer to our developer documentation about the component manager for that the audio unit component in general it generates receives processes translates or otherwise manipulates streams of audio as I have my slide and they're good high level building blocks that can be connected together just to create networks of audio processing and you can also use them singly and just the for instance instantiate a a software synth and up and wire it up so this output goes directly to the hardware we provide a set of audio units that ships with OS 10 and we'll be adding more in future and future releases and developers are free to create their own Audio Units and will be providing an SDK to make that a lot easier than it would be otherwise so what kinds of Audio Units are there there are other units that just provide audio data so they're sources they have no inputs to them an example of that type of audio unit would be what's known as a music device it's a software synthesizer and Apple ships DLS to soundfont compliant software synths in OS 10 also there are other units which interface with the hardware they provide a higher level abstraction of hardware IO devices so they talked to the audio how and one of the most common types of audio units on the category of DSP processors reverbs filters mixers this type of thing so these type of audio units typically they take input and process it somehow and output the audio stream another kind of audio unit is format converter converting between 16 bits floating point interleaving stereo streams into discrete mono streams sample rate converters and there you know quite a few other types as well and then another class is what I call it after class of audio unit and this is a little bit strange because it doesn't actually process the audio in any way using DSP and also doesn't change the format of the audio stream but instead it somehow does some other kind of impedance matching between the input and output for instance it may do some kind of internal buffering so that the input buffer size is different from the output buffer size or it may lay on top of a thread scheduling scheme so that everything before that audio unit in the chain would get deferred to another thread and this way you could potentially take advantage of multi processing on multiprocessor machines is very interesting application so I guess we're gonna try running a little demo here and I should probably explain a little bit what I've done oh this is your demo okay so if we can have the machine oh I can't see anything oh and neither can you I can see this okay let me just get this to run so since the big tease so what I'm actually running here is a Java program the Java program is quite simple and me just this is very interesting trying to do it this way this is dis demonstrating how to use the component description and the component services from Java it's the same as you would do in C as we've been talking about in those sessions you have both a Java and C API to this I thought it might be interesting just to show you some simple Java code that is quite clear about the concept of what we're doing so we create a component description this describes to the component manager the type of components we're looking for the type is okay audio unit type and then what I'm going to do here is I've just told it to look for the first component of this type and I'll use the au component which is actually in this package in Java and this will find the first one and then I'm going to print out some information about it and then I'm going to keep finding the next component from the one that I've already found until I don't find any more and then my program finishes so this will give you an idea of the audio units that we ship with the current version of system of this system so there's delay some filters as a matrix reverb in de levers D into louvers a mixer unit there's the DLS music device you can see it's type subtype manufacturer there's the audio device output to default output unit which is actually an extension of the basic unit that will talk to any device to default output device will track a default output that the user might set and various others thanks go back to the slides and of course I can't see my screen okay I just just before the demo I went over some of the categories of audio units and this is just a little simple diagram showing that not even it's a basically a box that can have an arbitrary number of inputs arbitrary number of outputs and I think this diagram is showing that DSP FX would have both inputs and outputs with some kind of processing going on software synth music device may only have outputs because it's actually synthesizing sound was creating it from scratch a default output you know which Doug talked about yesterday it would just take inputs and it would send that audio to a hardware device talk about audio unit properties properties are very very fundamental to the way that Audio Units work and a very general and extensible way to pass information to and from audio units out of units themselves they implement a very small set of functions which which can be called to deal with getting audio in and out and setting parameters and so on but we created this property mechanism to allow other units to be extensible so when we come up with some new feature that audio units need to implement we can always use the the property api's and pass in data via void star pointer and the length of the data is indicated by a byte size parameter the we define several different properties today and the type of property is identified by an audio unit property ID it's just 32 bit integer both the MIDI API is in the audio how API is also support this concept of properties it's almost identical type of mechanism in the API between a three secondary to the the property identifier identifying which property we have addressing within the audio unit to tell which part to specify which part of the audio unit is to deal with the property so there's a scope which can be global input output or group global means that the property applies to the audio unit as a whole input scope means the property applies to a particular input output is pretty clear and group is a particular type of scope that is relevant for software synthesizers music devices audio unit element is an index specifying which particular input output or group we're identifying so if we want to talk to input number two then this argument would be 2 or actually it would probably be depending on how we're numbering starting at 1 or 0 the IDs are defined in audio unit properties that H and it's important to understand what the data formats of the properties are because we're just passing information as a void start pointer and a length parameter you have to understand the structure of the data so those data formats are defined also in the header file we have a couple properties defined right now and additional properties will be added as needed and developers are free to implement their own private properties for their own Audio Units and there's a range of reserved property IDs for that but all audio units should support the general system properties so yesterday at Jeff's talk somebody was interested in knowing about the performance capabilities of our system and somebody asked about well how many audio units can you connect together and is there going to be a problem well the que audio unit property fast dispatch property is a way of getting access to function pointers which call directly into the audio unit and this bypasses the component manager dispatch overhead that is typically encouraged so basically you're just dealing with function call overhead when you're doing your processing it's no different from any other kind of DSP you might be doing in your own code so really the time the the bandwidth limiting factor is actual DSP that is occurring so the answer to the question of how many units can you string together is really a function of well how much processing are you doing among all these audio units it's not a limitation of the Audio Units themselves this particular property can be used as an optimization but you're not required to use it here's the second example of a property and that's the the K music device property sound bank FS spec this is a way of telling the software synthesizer which sample bank it should use so you identify the sample bank with an FS spec pointer and you pass that information in by calling audio unit set property and the first argument is the actual component instance the second argument is the property ID the sound Bank at that spec ID then it's global scope because it applies to the simp as a whole the element doesn't really matter in the case of global scope second to last argument is pointer to the file spec and then the size of the of the data is the size of the FS spec so that that shows you how you can communicate property information to an audio unit without a unit step property and all properties are dealt with in a similar way of course the specifics of what the property information is and what it does will vary parameters are less for configuring attributes of audio units and more for changing real time control points and we have the audio unit get parameter and audio units set parameter functions for this allows real-time control of processing algorithms the parameters are identified with audio unit parameter ID and the value is a 32-bit floating-point number and these parameters are very similar to MIDI continuous controllers but they're much more resolute there are 32-bit floating-point numbers as I pointed out and it's much much higher resolution than seven bit MIDI value the set parameter function includes a timestamp allowing the audio unit to schedule the parameter change for a very specific time within a buffer of audio so the timestamp is in units of sample frames on the next buffer of audio to be processed an audio unit supplies its list of supported parameters with the ke audio unit property parameter list and individual parameters can be queried with the parameter info property and information such as the name of the parameter what units the parameter uses whether it be Hertz cents decibels whatever min Max and default value for that parameter and some flags as well and information provided by this property could be used by a client to put up some kind of user interface representing this particular parameter so if the parameter represents volume in decibels or something like that from maybe I don't know - 150 decibels up to zero DB or who knows the control is actually able to display the the units and the minimum and maximum value appropriately I'm just information Audio Units are in one of several states and it's important to understand the state transitions that Audio Units go through when an audio unit is first opened with open a component it's considered to be an open State once the component is closed then it doesn't even exist anymore so that's kind of like the base level constructor destructor if you're thinking in object-oriented terms but there's an additional level of construction and destruction and Audio Units that's the initialized and uninitialized state and we have the audio unit initialized and audio unit uninitialized functions for that once the ADI unit is initialized you can actually start it start at running with audio unit start and that applies specifically to io audio units such as the default output unit actual actually start the hardware through the audio Hal so let's go into a little more detail about what's the difference between just opening the component in the first place and initializing it it seems like it's the same thing well when you open the component you have a component instance and you can make calls to the component but at this time the component is not expected to allocate any significant amount of resources such as memory but you are able to make property calls to the audio unit to configure it in some particular way for example you might want to configure it sample rate or bit depth or maybe you're interested in setting up some other aspect of the audio unit before you actually initialize it then once you call audio unit initialize and you've made your property calls to configure it just like you want then the appropriate allocations can occur optimizing for the particular configuration so if you're configuring a reverb for a certain number of delays or something like that it's able to allocate exactly the amount of memory it needs so the running state this is what audio is actually being processed it's actually going through from input/output an audio unit or the hardware it's running whatever the case may be and the audio output unit implements the audio units start and other units stop functions and if you have a graph of audio units connected together this this is where the audio actually starts being processed or stops being processed and it the audio output unit generally represents the terminal node in a connection and as Doug talked about a bit yesterday the default audio unit it starts the hardware device that the audio Hal is connected to here's another API to Audio Units audio unit reset it basically is used when an other unit is initialized and it may even be running but if you want to clear the state of the audio unit it's typically used with DSP Audio Units so for delay effects you would clear the delay line for reverbs you would clear out the reverb memory for filters you clear the filter State and this is important if if the graph is actively processing audio but you perhaps reposition the playhead to a different part and the audio stream and you don't want reverb tail from from a later part of the of the score to be appearing in the in the mix audio unit IO management audio units process audio or deal with audio using a pole IO model and what I mean by that is that there's a function a particular function you call to be the audio output stream from the audio unit and when you do that the audio unit has to go and fetch its audio input so it typically does that by calling the same function on the output of another audio unit and then that audio even has to get its audio input from somewhere so if you imagine a chain of these audio units connected together one after the other you would pull on the very end one and in turn it would go down the chain each one would fetch its input from the one before and you pull the audio through the chain it's possible to connect these audio units up in more complicated configurations than just simple linear chains but the idea is still the same the audio is pulled through to the terminal node which is typically ending with the hardware output device all units specify their number of inputs and outputs through properties and the format of the audio data is canonically it's N channel interleaved audio or it could be signed sideband data such as spectral analysis data and this is linear PCM 32-bit floating-point okay I've been talking about connecting these outer units together and this is just a picture showing that so I'm going to talk about how to get audio into an audio unit and then I'm going to talk about how do you get audio out of an audio unit so there are two ways to provide an audio unit with its audio data the first way is using a property called a audio unit property make connection and with this property you specify a source audio unit and its output number and then that connection is made and the audio unit knows that it's to read its audio data for a particular input from another Audio Units output a second way of providing audio data to an audio unit is by registering a client callback and this is the standard sort of callback that we get in the audio Haller and the sound manager or in many other audio systems such as a CO and the in this using this method the the client just specifies a callback function and then every buffers life cycle this callback function is called and the client is able to provide the audio for a particular input of an audio unit and in the case where audio units have multiple inputs each input may be configured separately using either of these two methods so our unit has two inputs first may be connected to the output of another audio unit in the second input may be connected up with a client callback so that audio is provided directly by the client in this case I'm going to be talking soon about the a you graph API which is a higher level abstraction for connecting these audio units together so I've just gone over two ways that you provide audio input data to the audio unit now it's important to understand how do you actually get the rendered audio out of the output and you have to use the audio unit render slice function call for this it pulls the audio or reads audio from a particular audio unit output and it must be called once per output so if there are three outputs then it would be called three times each audio output can be an n-channel interleaved stream like I said before so it's possible that an audio unit might just have one output and provide actually eight discrete channels internally in that one output let me go over some of the important arguments to the audio unit render slice function it's actually very similar to the the audio Hal callback arguments the audio timestamp it's exactly the same audio timestamp structure as an audio Hal and it specifies the start time of the buffer that is to be rendered it provides information about host time the current host time of the machine or the host time that corresponds to the start of the buffer along with the sample time that that corresponds to so it's possible to synchronize other system events such as MIDI events which use this host time to the audio stream which are synchronizing based on sample time the audio buffer argument is where you can pass a buffer that you expect the audio unit to render into or optionally you can pass a null for this and it will pass back a returned buffer that it did it caches and let me just the audio buffer is also a structure that's used in audio house so we we share we share the same data structures between the high level and the low level parts of the audio stack so with audio unit render slice it's optionally possible to schedule events before a given IO cycle so before audio unit render slice is called you may want to do some scheduling parameter changes with Audio Units set parameter or in the case of software synthesis you might want to schedule some notes to start or stop playing during that that slice of audio so there's its two phase scheduled render model and step one you would schedule events and in step two you call audio unit render slice an advanced feature of the rendering model is the pre and post rendering notification mechanism it's an optional callback that a client can install on an output of an audio unit which gets called before rendering occurs and after rendering occurs so every time that audio unit render slice is called the audio unit will call this notification callback do performance rendering and then call the callback again to say that it's done rendering and this allows the client who installs this notification to perform arbitrary tasks some examples would be scheduling which you might do in exactly the same way as it's as I described here before every render slice and another use that you might make of this is to perform CPU load measurement you can actually measure the time that the audio unit takes to performance rendering so how do you write one well how do units are components the same kind of components that have been used in QuickTime for a very long time so anybody who's familiar with writing components in QuickTime should be able to get up and running pretty quickly with Audio Units for people who haven't written components before it can be a little bit difficult unless you have some help and we have an SDK that makes this very simple all you really have to do is override one or two basic functions and in fact at the base level you really only have to override one and that's the function which does your actual DSP processing if your audio unit is as a processing type of audio unit and this SDK will be placed on our website soon in the next few days we hope so finish talking about Audio Units at the lower level I'm going to talk about a you graph this stands for audio unit graph and this is set of API is for connecting these components together represents is a high level representation of a set of other units and the connections between them allows you to construct arbitrary signal paths fairly arbitrary anyway and represents a modular routing system and it can deal with large numbers of them why use a you graph one of the reasons is that connections and disconnections between audio units can be performed while audio is actually being pulled through the chain and if you are talking to the audio units at the lower level you have to take great care at at changing connections distant making connections and disconnections while the audio is being pulled through you can get all kinds of bad things happening there's a lot of synchronization involved and a you graph takes care of this for you another reason you might want to use a you graph is that it maintains a representation of the graph even when the audio units themselves are not instantiated and it also takes care of and embodies the state that the audio units are in as we'll see hey you note is primary object that's represented in a you graph an au note actually represents a particular audio unit in the graph whether that outer unit is instantiated yet or not it's a placeholder representing that audio unit to create a node you can call a u graph new node and argument to this function is a component description which specifies the type subtype and ID of the component this is this information that you would get from fine next component component manager call it's also possible to pass this what we call class data to initial initialize an audio unit width and we'll talk about that in a minute may you graph remove node just takes it out of the graph to create connections between these nodes once you've added a bunch of nodes you've got to connect node input disconnect note input clear connections that clears all the connections in the graph and update for synchronizing real time changes you can also walk through the graph and look at all the nodes with get node count get ant node get node info these are in the hey you graph H header file in our audio toolbox framework and I think that the api's are fairly self-explanatory if you looking at header file so we're not going to go into detail with all the arguments just like the audio units themselves the a you graph mirrors the state of the other units in them and I and this slide is just pretty much going over what I talked about concerning the states of audio units here an important thing is that a you graph state corresponds with the audio unit state so the state API is through our graph our graph open close so a you graph open when you call that it actually opens all of the audio units represented by the nodes in the graph and close closes all the audio units and correspondingly initialize on initialize it calls the audio unit functions and start and stop same thing in addition there inspection or introspection API is how you graph is open is initialized is running so you can determine what state the graph is in oh this class data when an au node is added to the graph it's possible to pass arbitrary data in with a void star pointer and a length similar to the properties and if you add this information this optional class info when the audio unit is opened a property call will be made on the audio unit with this data and the audio unit is able to make use of this data in any way that it wants it's there's no data format that's defined currently for it but it's a way it's a way to maintain persistence because when the graph is closed data can be gotten from the audio unit and stored in the node so it can be recreated in the same configuration that it was so a little bit earlier I tried to explain the notion of this head audio unit in a graph in a very simple case you would have a chain of audio units say you might have three first one connected in a second connected to the last one the third audio unit which is maybe an output device and this last one represents the head of the graph and we talked about the pole map model the head audio unit is very important because it controls the hardware it controls what the processing is is being pulled through the graph or not and so there are special audio unit API is our unit start and stop which I talked about before and the AU graph has corresponding calls a you graph start and stop okay done talking about audio units and a you graph so finally I'm going to talk about some sequencing services for performing scheduling a few slides back I had a diagram up showing the schedule render model we're in step one you schedule events to audio units and secondly you would render a slice of audio the music sequencing services actually would be one mechanism where the scheduling could occur it's a simple API for creating and editing multitrack MIDI sequences individual events in the tracks can address Audio Units and software's synthesize their music devices it provides a runtime for the playback of these sequences and has a bunch of api's for doing live edits cut copy paste merge this type of thing so while the sequence is playing you can be editing it's important in a sequence their application be able to do that for those developers who already have a sequencing engine in their application this set of API is might not be so interesting but for somebody developing a new application because the api's are there for doing the basic edits and for actually providing the runtime system for playing back the sequences it's only necessary to write a UI which is not necessarily easy but if you're good at writing UI at least you don't have to write all of the underlying engine which can be difficult sometimes the scheduling uses units of beats and it's floating-point 32-bit numbers representing beats and it can also represent represent seconds to in certain cases there's a tempo map for converting between the beat time and real time seconds API is for reading writing standard MIDI files tracks have attributes you can mute solo loop tracks the music player is the highest level object and it provides transport so you can you can seek the sequence to a particular time and start the sequence playing as top of the sequence playing with the music player and it provides the appropriate synchronization between host host base processing and MIDI music sequence is the next level object and a music sequence contains multiple playback tracks which are called music tracks each music track contains a series of time ordered time-stamped events and each music track with these events addresses a particular au node in in the a graph that we spoke about before so for instance if you have a graph with a software synth there would be a node representing the software synth and one of these tracks might be addressing its events to the software synth there might be another audio unit in a graph which is a filter of some kind there might be another track which is addressing parameter changes to the filter to sweep to the filter cutoff there are also attributes to the track commute solo as I said before you can set up the loop length number of repetitions of the loop and also offset the start of the track relative to the other tracks there are several different types of events the most basic one is a MIDI note message it's MIDI note number with velocity MIDI Channel and a duration the note off is not a music event as such it's implied in this MIDI note message using the duration there's a MIDI channel message for all the other channel messages there's raw data for a system exclusive there's a tempo event for doing tempo changes and there's this extended note API which is very much like the MIDI MIDI note message but it provides for more sophisticated control and esoteric applications MIDI note event only has the note number and velocity and a MIDI channel associated with it and this extended note event allows for an arbitrary number of floating-point arguments to be associated with the instantiation of a note not everybody may be interested in that but it's there the extended control represents a 32-bit floating-point parameter change and ultimately it may end up calling Audio Units set parameter at the exactly the right moment in the audio processing for sweeping filter for changing volumes for panning positioning audio all different kinds of parameter changes there's also a user event to arbitrary data can be placed and meta event for MIDI so here's the one example there are there's an API corresponding to each one of these types of events and all of them are very similar to this one so I'm just going to go over the one this is how you add a MIDI note event the first argument is the the music track you're adding the event to the second argument to think they got the arguments wind up a little bit funny but the second argument if you can make sense of that is the timestamp it's 32 bits actually I think it's a 64 64 bit it's a double precision timestamp representing the beat and then you have a pointer to a structure the MIDI note message structure which essentially just has the note number velocity MIDI channel and the duration of the note in it so this is a diagram showing how you can have a music sequence with multiple tracks three in this case where each track is addressing a separate audio unit in an a you graph and this corresponds with the previous diagram the two-phase schedule render model if you remember there are some editing api's for copy cut paste this type of thing and I think if you look into music player dot H header file and audio toolbox framework it will be very clear how to use these one of the lowest level objects in music sequencing services is a music event iterator and it allows you to actually walk through the events in a track see what they are and it could be useful if you're writing sequence or application you're writing the UI sitting on top of this engine for just walking through the events seeing what they are so you can display them also if you want to create custom editing operations on track you could use this music event iterator to walk through all of the events in a particular time range so there are a couple different api's for dealing with these iterators you can the music event and Reuter is a first-class object so you create it with new and dispose and you can seek the iterator to a particular point in the track it finds the first event at or after the time that you're seeking to you can move the iterator one event forward in time with next event one event previous with previous event and you can actually ask the iterator if it's at the end or beginning of a track what has previous event and has next event then the music event iterator get event info function can be used to find out what type of events you have at that points so you can find out what the event type is what time the event occurs at and you get a void star pointer which you cast to the appropriate event type to get access to the particulars of that event for instance if it's a MIDI note event you can find out what it's no numbers so on you can delete an event at the point where the iterator is or you can change the events time okay I'm going to play a demo it's kind of kind of a little bit vague what I'm trying to illustrate here but I think what I'm trying to show is how all of these services work together so in the example I've created a graph I have a software synth the DLS music device it's loaded up with a twelve string guitar sample Bank and it's playing through kind of an arpeggio really quickly and then I have some additional audio units in the graph I think there's a digital delay in there there's a limiter look-ahead limiter node to keep the volume pretty steady and there's a reverb at the end and I also have a bandpass filter in the graph whose center frequency I'm sweeping using the music sequencing services so I've set up music sequence and have one of the tracks addressing the node in the graph which is the bandpass filter and sweeping that around and it basically just ends up sounding like some kind of ethereal wash but you should hear that there are these kind of filter sweeps going in and out and there's a reverb in there for sure and let's see if I can weight this machine up and get this going here okay got my cursor unfortunately it's not nearly as impressive as Jeff Moore's multi-channel demo the other day but this will give you an idea okay it's I'm only launching an application actually it's it's nothing too visual it's just to hear but I'll I'll show you double click and you should hear something [Music] [Applause] [Music] [Applause] [Music] so it's a little bit vague but thanks but it's showing it's showing all the parts of the system that I talked about I think bill I'd like Bill Stewart to come up and he's gonna show us some some more demonstrations and talk about the Java interfaces thought I'd might mention that all of that sound you hid was just some nerds coming from the tall string guitar so it's pretty interesting I think now if I can have the demo machine but I've got to get my mouse to somewhere okay so okay I'll just get this guy running and this is cool I can just show you what I want to show you so this is my java application and do we have sound coming out from this so I'll very quickly walk through the code with you after this basically I'm using a sound font that we got the same sound font the criticism from mu which is a 12 string guitar and I've written a java application and we'll go through the MIDI side of this in the next session and I'll go through the audio unit or graph cited in this session and basically the media application is listening to the MIDI point getting the MIDI in messages from this keyboard and telling the music device to play notes in response to keyboard input I'm actually guitarist so I had to use the 12 string and I'll probably have to do [Music] [Laughter] so I'm going to quit this guy and let me just bring up the project and you have to just bear with me while I find it and this will give you sort of some feel for how this this code looks that Chris has been taking talking about things so let me just move it over here look what's going on okay so I can barely see what I'm doing here alright so there's a bunch of stuff here just to sort of set the thing running I'm loading the sound Bank here which I've just installed with Mac OS 10 we also wanted to make sure that there were directories that were available as standard places for you to put standard resources that you may want to share between different applications so in this case I'm putting the sound bar a sound Bank in the slash library audio sounds banks and then this is the name of the sound font file here and there's also a mirror of this directory in the users directory as well so sound banks that were stored here would be available to any user that came onto the machine so you might imagine like a lab where you could have a setup that you want available to every user and then individual users can have their own sound banks in their own user version of this thing and there are fine folder constants that you can use for look for these in systems that have been localized and there's other folders there for audio units themselves if you want to make them available for different types of sounds there's a whole bunch of MIDI stuffie that I'll go through later on and what I want to show you is just basically how the graph is set up and we'll just ignore the first thing so this constructor here is creating the graph and that just calls through to the C API to create the graph here and then this is me creating two synth two nodes one for the DLS music device and i give it a component description and I already know what I'm looking for here I could use the find component stuff to actually hunt around for components I'm just creating two nodes I'm creating a music device and I'm going to use the DLS synthesis the the that one for that and yeah we're very hopeful to see other music devices from you guys like you know some acoustic modeling synthesizer devices and all that kind of stuff and if I get the mousepad here and then the second one is the default output unit I'm just going to create that because I want the sound to actually come out of the computer and then to connect those I just tell the graph to connect the node the output from the synth into the input of the output node and just to show you that there's some stuff going on with the graph after I create the graph I'm going to get the note I get the node count here and I get the index node starting from 0 to the number of nodes and I'm I print that out and I can show you what that guy looks like so you can see there that it's telling me I've got two nodes and there's a component which is the music node and the default output node and I've lost my mouse again notice ok so after I do the connection of the nodes I'm going to open the graph and this will open the audio units and that will actually it's really a constructor until you do open on the graph you don't have an audio unit you just have the node itself it's an abstract representation of the graph so when I open the graph I can ask the graph to give me the audio unit that's associated with a particular node and if we have a look back up here you'd see that I had kept this guy around I could actually go and look in the graph and find out the nodes here and I could ask her for information but I've kept this note here so I know that what I want to do is get the music device which is a subclass of audio unit for that particular node and what we do in the cord Audio Driver API is is we actually give you the subclasses that correspond to the extensions of the component selectors so you could have a basic audio unit which is your parent class and you've got an extension of that was your output audio unit another extension is the music device and that in component terms just provides additional component selectors additional functionality if I've specified a sound Bank which I'd be kind of bit lame if I didn't then I'm going to set the property of the sound Bank on the music synthesis is a convenience routine in Java does the same thing as what I crocheted in the C file and it just takes the Java file i/o object it does all the necessary conversions to the native structures and instantiate that and you notice that I'm doing this when I've opened the graph but before I initialize it so this is one of these things where this is a property that we want the synthesizer to have when it's initialized and until the Center is initialized it's not going to try to use a particular sound bang until you initialize it and then just to get the synth sort of up and running I'm going to set some default volume and channel stuff and there's some commented out code there I'm not going to worry about and because I want this to be responsive I must have acquitted tonight so because I want this to be responsive what I'm actually doing at this point is to get the the audio device output unit which is the default output unit that I'm sending out to and from that guy I'm going to get the device that it's attached to and I'm going to set its buffer size so that's going to basically take the default callback buffer which is about 11 milliseconds 512 sample frames and I'm going to set that to 64 sample frame so the responsiveness of the system is about 1 and 1/2 milliseconds so I get creating a memory object that I can pass that information in and then I just set the output properties straight on the device so I get the device from the output unit and it's actually a device output unit and I set the property to 64 frames which is 64 times 2 because it's a stereo device times 4 which is the number of the size of the float and of course in Java you don't do size of light because it's Java and not serious math and then I actually get that property as well and I print it out and you can see if I can find my mouse and you can see there that it's telling me that the buffer size for the i/o clock is 512 which is what we'd expect 64 by 2 by 4 so that's all we do and then after I've set the buffer size up and I've initialized the synthesizer I then just start the graph and that will start the device that will start pulling through the chain and away we go and then in the next session I'll go through how I you set up the MIDI pausing stuff so that I can actually send MIDI events to the music device so I can we go back to slides ok so just some notes about the Java there's also Java doc available on the on the website we're also preparing a PDF of just the core to architecture itself and they'll be available at the web site early next week and as we stressed yesterday we sort of look at coreos basically an architecture that you should understand how it works and that's really a language neutral feature that the job and the C API is represent the same API to the same functionality that's implemented underneath and in a variety of different ways so we also have a mailing list it's at least our apple.com it's called core do api and we welcome you to involve yourself in that mailing list and it's a good way to send information to us and we will answer questions on that list and there's a website I don't know if it's completely up yet if it's not up today will be over the next few days developer.apple.com slash audio and there's some related sessions for the DVD viewers of this wonderful thing and the next session which should be MIDI is at five o'clock today audio services and Mac os10 was yesterday and there's a sound networking ghost before this one if you're have any questions related to firewire in USB there's a lot of fire wire a lot of USB audio devices and many devices and stuff that's coming out you can contact Craig Keithley Keith lead Apple comm if you're interested in in seeding we obviously you know constantly working on these technologies if you're interested in getting access to that before we release it publicly you can contact us at audio Apple comm or talk to your developer tech developer relations manager and inquire about seeding you