---
title: WWDC2004 Session 222
framework: wwdc
role: article
path: wwdc/wwdc2004-222
---

# WWDC2004 Session 222

## Transcript

Kind: captions Language: en good morning everybody 9am after the party not too bad sure we'll get some stragglers coming in anyway I want to make one program note if you had to choose between this session and the GL shading language session which unfortunately is at the same time because we had a last-minute scramble after both sessions after this session and the GL went upstairs everyone's going to be back in the graphics and media lab if you had questions about that and had to pick so anyway well I'm really excited about what we have for you this morning we're going to be talking about quartz composer which is a new tool on the developer DVD pretty exciting way to play around with all the technology will have so without further ado I'll bring up PR le ly of Olivier Latour good morning everyone welcome to the quartz composer WWDC session so the first thing we are going to look at is obviously what is quartz composer so it's a brand new tool we introducing in Tiger developer tool and exciting visual the visual programming tool and what you do with it is you create compositions which basically our little programs that somehow process and run the graphical data quartz composer is able to actually package a bunch of technologies into one signal environment and you have technology is like opengl quartz 2d or brand new technologies we introducing in Tiger like core video or core image everything in real time from editing preview to debugging and the compositions you create are really easy to integrate into your applications we provide an ID palette and we the system is completely compatible with the coco bindings the first thing i'm going to show you are some simple examples of compositions can we see a demo machine number two please thank you so here we have a simple but typical compositions it contains a mix of 2d and 3d elements like in that case we have a sweetie cube that is rotating with some text on it and in the background we have some simple text that is being blurred in real time using some cordage filters okay I'm going to let you read to the the end of the text you are able to this composition is actually it's at the foundation of it is a simple 2d image that was to which were applied a bunch of filters and at the end you opted that completely different results from the original image which is animated with the time and it's interesting to see that the quality of what you generate with quartz composer using general lying technologies we have graphic technologies we have in your braiding system it's very very good very good and it's running very fast you might not see it but at the bottom left corner we got a frame right counter and we're running at 60 frames per second right now because we're sinking to the display but it would actually run even faster this composition is interesting because we have we don't have any image as its source it's basically generated from scratch using core image filters that are assembled and animated with time and once again you can see that the quality is very good and it's completely per pixel computers this release of quartz composer also allows you to do some simple 3d animations and which could be interactive and respond to the mouse I'm using the mouse right now to actually move around in a 3d world this other composition and here we go is actually a slide show where we have a folder full of images and the composition is responsible for putting them on screen and computing the transition between each between the images the last composition I want to show you is a composition that gets data from the outside world the the internet so this composition is getting the RSS feed from the apple website and showing the hot news weather the simple text animation back to slide please so what are we going to learn to that basically five points we're going to have some theory at the beginning basically the quad computer concepts then we're going to look at the application itself we're going to go through a tutorial for that you can see a typical example of building a composition and obviously we'll look at how to play back compositions in your applications at the beds of the court composer concepts we have what we call patches bachelors are like basic simple processing units their role in life is simply to execute and produce some results this result can be sent to their output ports which are which we are simply going to call outputs or to some rendering destinations then to control dispatchers you pass parameters input parameters actually through their input ports of the patches or we're going to call input ports simply input so we can say that factual they're like little function that produce some results according to a bunch of parameters and the current time the ports the input ports and output ports of the patches are actually typed which means the kind of data you pass does better the we can pass some values like simple numbers boolean values or strings even colors and we can also pass around complex objects like bitmaps OpenGL textures or core image images let's look at some example patches the first one we're going to look at is the Edit full patch which stands for a low-frequency OC 80 theater the goal of this patch is actually to produce a cinema a web on its output which is determined by the current time the period of the wave the website so that would be for example a square wave or sinusoidal wave and it's included this other patch we have here does not have an input parameter it doesn't need any it's simple role is to produce on its outputs the current mouse position the x and y coordinates and the last patch spritebatch does not have any output port because it is rendering to a destination and in that case this patch is actually drawing a quad at a given x and y position and with a given color and texture on it so how do we put everything together well let's say we have kind of work space and we put all the patches on it patches we want to use these patches are going to have some inputs and outputs we can build some connections between the inputs and outputs so that for example the patch in the middle is retrieving the data its input data from the other patches on the left so we can say that the answer the patch on the further right is actually pulling the data from the patches on the left so what we bill here is a kind of simple data flow model and this is at the core of the court composer concept we're going to put these patches into actually a kind of macro patch and when it becomes interesting is that for example the patch we have at the middle of the macrophage the big one could actually be itself another micro patch made of several patches and some of these patches may themselves be other macro patches and so on so what you end up with is a lyrical patch tree now if we call the top patch sub macro patch at the top of the tree the root patch then the answer your patch tree and the entire your data flow is it describes is what we call the court computer composition now i'm going to show you the application itself because that's in a theory for now so you will find the application into developer folder applications graphic tools squad composer so it's a kind of standard Coco looking application where you can work on multiple compositions at the same time and in that case we have one document for composition so let's look in detail at this document window what we have here is the workspace where actually we put the patches and we interconnect them on the right we have the list of patches we can actually use there are pre-sorted by categories so that it's easier to find them for example some of them are going to be controlling the objects by sending data for it for the position for example some other patches are going to be used to import data like images from 50 from a video camera then we have several numeric patches that are used to perform abuse clematis some mathematical operations and then we also have rendering patches because we want to render something eventually at the bottom of the list you will find several categories that start with a dot and these are all the quarry image filters which are natively supported by quartz composer so you can sign here all the compositing filters distortion effects generators and so on ok now we're going to build a simple composition the first thing we want to have is actually render some single screens so I'm going to go to the renderer and pick up for example the sprite it has more parameters than the abstract version you've seen on the slide but it's not very difficult to understand if you can wait what they're that we have basically a bunch of input parameters to define the position orientation of the sprite and then some other parameters to define the width the height and also the kind of texture you want to have on it we can run the composition because as I said at the beginning this entire system is completely real time now we have our sprite I would like for example to control the position of the sprite with the mouse so for that i'm going to use the mouse controller and drag and drop it there now I would like to have the x and y position of the sprite to be determined by the mouse coordinates so for that I actually drug connection from the x and y outputs of the mouse to the x and y inputs of the sprite and now i got my sprite which is kind of following the mouth at the same time weird is that we have a huge trial the reason is that we are not clearing the rendering area so each time the sprite is drawn over the previous frame and obviously we want to add something that looks a lot nicer for that i'm going to go back to the renderer category and pick up the clear patch which obviously clear the screen but you will notice that i don't see my sprite anymore the reason is that since we're performing drawing operations wait to define which ordered operations are performed and in that chest you will notice on the top right corner of these kind of patches that they have a number indicating in which order they are executed we have number one here number two there and obviously we want to do the inverse which is rendering first I mean clearing first the rendering area and then drawing out sprite for that you display a contextual menu on the sprite you are interested in and you can change it's a rendering layer which basically defined the order and now it's working fine the next step I'm going to use the third patch and we introduced earlier which is the Edit foot patch so the electro is outputting a web according to the time and I'm going to drive the let's say the width of the sprite with that wave so now we get we can see something that is animated I might want to also drive the height of the sprite with that web but do some other versions on it the way you would do it for example is use a matte patch we go and I'm going to take the value from the output of the LFO turn it through the mat patch and get the fur the next result and connect it to the height of the sprite now we've seen so far how you connect patches and have values being transmitted dynamically through pact between patches but of course you might not want always do that and have some value is defined statically so how do you do that well you can simply double-click on an input which is going to bring up an editor and then you can set the value on the input so in that case that's a color input so we get a color whale and I can then pick up some green color for example then another way to edit the parameters is to show the inspector there and go to the input parameter Spain where you can see all the parameters at once you will notice that parameters that are defined by because they are connected to some outputs are not editable obviously so what I what I might want to do there is simply I don't know define take the force P despite not to go below 0 point 5 for a twist for example for tight I mean I'm to go above 0 point 5 and now we got something like this let's go back to the to the slide please okay we still have some some theory to look at can we go back to the slides please thanks so within earlier that we have what we call a year or she go patch tree now we are going to have a rough look of regarding how exactly it is being evaluated by the suspense so the top of the tree we have our we have an acrobat which is basically like the root patch and then during evaluation this patch is being traversed and each time and we're going to the system is going to execute the sub patches that are content into this micro patch and each time one of these sub patches is a macro patch itself it's going to be traversed and all its sub patches are being executive and so on and so on for the trees drivers upside down now what exactly happens when we are inside a micro patch and we need to execute the set patches the first thing to know is that not all patches are kind of born equal we have consumer patches these are the essential the most important patches there are the one which renders something for right now they're all rendering something to the destination area which is going to be like a screen or preview rendering window each time you render a frame of the compositions they're going to be executed there are cyclical you give in a defined order as we've seen earlier and you can look at the order by looking at the other number added on the top right corner and they actually are the one which pull the data from the other patches what kind of other patches do we have well we have the professor patches which ask which are kind of slave patches they run on the mound and they simply execute to when it when the output when their inputs I've changed and they need their outputs to be updated the system only execute them kind of in a lazy mode the third type of factors we have our providers and this one's their role is to get data from outside sources into the system in that example it would be a mouse patch and they also run on demand which means only when the outputs are needed okay so now how is that simple example going to actually execute as we said earlier the consumer patches are the most important patches and they are the one driving everything so they get executed first and first we have the clear path to execute then we have the sprite patches can execute but this one is interesting because some of its inputs are actually defined because by the by the fact they are connected to other outputs to some outputs of other patches so before running the sprite patch the system must make sure that all the inputs that are connected have up-to-date values so first it's going to execute the mouse patch so that it's updating its outputs that get copies to the input of the sprite patch and then we have the elephant patch that is executive and eventually the mat patch now all the values on the sprites are up to date and we can execute the sprite match let's go back to the DML please so I'm going to show you a little more about the evaluation system and how you can look at it inside the application so here we have a composition studies far more advanced than the one we looked at earlier let's look at it for a second full screen so it's a kind of DVD scene if you want that was built using court composer and we have live video playing being masks in real time from real-time coordinate filters also and some flying hearts and everything and everything is goofy completely editable for example which I want to change the title are just double click on a string input and I say WWDC 2004 ok now it's very good as we said earlier we have some it's it's a patch free gender composition is a patch tree the way you can look at it is by displaying the bruiser and here you can see we have the root patch and I can look at the background patch the flying hard patch title and menu some of them are going to contain other sub patches so on back to the root patch let's look at the background patch which is a macro that was created to simplify the so that you don't end up with thousands of patches at the root level you can create macros and put them in order to clean up your workspace we're going to see that later in inner tutorial so let's look at the exact evaluation of the system in that case we would have some movie coming in which is our started DV progressive 24 frames per second it's going through this patch here the movie positioning which actually it's not the real name but I renamed it in order simply to put the movie at the correct position on screen then it goes through a series of filters to display to correct the gamma to address the collar controls and also to to mix the two masks out the bottom movie we're not interested in using an image that was imported from from the hard drive and we end up with the part of the movie that we're interested in that its position on the correct place and mask out then we have our original background image that is then some more color corrected and on top of that we add a kind of white mask that is going to be used to generate a hello hello hello and signed a ring and eventually when we combine everything we end up with our final background image now you can see that this is a very part the tooltip system is very powerful because it allows you to see exactly what's happening inside the data flow but we can even do better if you press the debug button here what's going to happen is that it's going to call our eyes the patches depending on the weather are currently being executed we have three colors green means the patch is currently activated and is being is running red means a patch is not even being activated or running or anything at all for example if I just drag and drop the patch here nobody's using it so it just read it's useless then orange patches are the one that are being activated but they're not running why is that because the data here is never changing so the system is obviously small enough to only reacts acute the part of the data so that are changing which is the case here because the in the movie images that arrive at the at the entry of the patch is changing every frame so this enter 85 here needs to be reevaluated now let's look at what happens if I change the color here for example pick up some green stuff as you can see this part of the pipeline just blink indicating that because some parameters are change somewhere it's revaluing all the necessary parts back to slices so up to the tutorial this simple tutorial is going to take us through building a composition from scratch and what we are going to build is a composition session under a simple real-time glow effect and we're gonna learn through that how to build such a gothic using crow image filters and also how to render a simple animated cube that we're going to fit through that grill effect because we're going to be wandering in to create a texture out of it and we're going to obtain the result that you see at the bottom of the screen with the original cube and then the cube with the glow effect on top of it so how would we do a simple glow effect it's only a two step process we have the original image and then on top of it we put a broad version we add a blood version so we hope that this we obtain this nice glowing effect so let's get started so I'm going to launch quartz composer start up with a brand new composition first thing I'm going to do is import an image to work with so I go to the generators and I use bitmap with file there an inspector if I go to the settings pane I can click on the import file button and pick up go and pick up an image file in that case this one then the next thing I want to do is actually render this image on screen you can get started here we go for that I'm going to use the image render which give us extensive control over rendering a simple image on screen and I'm simply going to connect the bitmap output to the image input of the image render the connection is red meaning there is some conversion going on because the bitmap we spider generating kind of a bitmap object and the image renderer except the core image image but the system is smart enough to do conversions between the objects or the various types you may be transmitting whenever it's possible for the right just to indicate that a conversion is occurring is happening otherwise you would get like green connection we're not displaying the entire image because the image render give a specific control over which part of the image we want to display the reason is that because it's rendering core images Corey majors may be infinite and you might often end up with infinite images when you apply some filters so in that case you already need to specify I want to render this area of the image so we could specify manually here and input parameters that our original image is 1024 x 768 so that will do for now then to achieve a blur effect we said we needed to use a blur and to add that on top of the original rendering so for that I'm going to use the Gaussian blur which is into the coordinates hitters and the addition compositing batch all i have to do is collect the bitmap to the input of the Gaussian blur connect the result of the Gaussian blur to one of the input of the addition patch ok let's that's going to be better there we go and the result is what we want to display ok and once i'm done with my blur i want to add it to the original image now we got our nice glow aside but it's if we look at it full screen what I notice is that it's very bright and one way to fix that is simply to insert a color adjustment and by playing on the gamma before we apply the blur so I'm going to insert a gun I just before the blur and then I can play with the ok the value of the power on the gamma correction to actually kind of fix the brightness of the of the glow effect now it's much better so that will be our first part of this tutorial second part is going to be to create some animated backgrounds because it's obviously more interesting to have our burlas like being applied to something that is animating that something that's completely static so first thing we want to do is we're going to just create a brand new compositions as usual clear the screen and then when their acute on top of it the cube as several parameters and you can control position orientations and with the dimensions of every every axis and so on first thing we're going to do is simply change the dimensions to make it a little smaller say all 75 we do the trick and then we want to have this cube simply animate with time one way to do that is would be to use the let's see it turns interpolation patch which simply interpolates between two values on a given duration and I'm going to control with that the X and Y rotation okay i want the interpolation to start at zero in that case we're going to be in degrees finish at 360 degrees over at say ten seconds and I want the interpolation to loop now we got our knife rotating cube next step is to usually add a texture on the faces of the cube so I'm going to go back to generators and use bitmap which file and import an image so i should have one for this we go nice break picture and to display this kind of brand new image on the faces of the cube I simply have to connect the bitmap to the various faces of the cube there we go and the last one so it's getting better but one thing is still missing it's the the fact our tube is not even lighted so it's not very realistic so let's look at the environment patches we have here and one of them is the lighting so we're going to drag and drop it there we don't see any difference yet the reason is that you need to specify how the lighting is going to influence the battery I mean which patches on the workspace here are going to be influenced by the lighting you might not want all the patches to be specified to be intersected so the way we do it is because it is by using the fact that the lighting patch is a macro patch itself and it's only going to affect the patches that are inside this micro patch one way to navigate through the patch for you seen earlier is to use the bruiser bear another way is simply to double click on a title bar in a patch and then you can go inside it and if you want to go up one level you just click on the Edit parent better so what we're going to do is just cut this go into the lighting patch and touch it and I mean past it here we go so now we got our cube which is lighter than rotating and everything that's going to be part 2 all right now how do we put the these two compositions together what we need to do is produce some kind of image we're gonna with the rotating cube that we're going to feed through the glow effect and it's convenient because we have a patch specifically for that which is called which is a generator patch and which generated texture with what is inside itself let's look at the so I just added it to the workspace and let's look at the settings so for the OpenGL folks you're going to notice that you have access to all the underlying open G of settings in that yet it's a picture so I can specify the target the meat mapping the filtering mode everything so it's important to notice that this tool even if it's a daunting to abstract all the various technologies it encompasses it's not hiding everything so you can still have access to the low level settings most of the time so in that case I'm going to use a rectangle fixture and i also want to i will turn it to specify what is the size of the texture so we can either hard-coded hard coded or you can just put 0 0 which in that case sells the system well generate the texture that is whatever the size of the rendering destination so if you're wondering i don't know 320 x 240 which is the case for this preview area this picture is going to be 320 x 240 and so on then what i want to do is that will click here to go inside the texture patch could be what i just created and pass it there so now on to this texture i'm going to have my rotating cube and the next step is obviously to fit that texture in through the sis to the glow effect instead of the original image ok save the razor last 43 remember we set the dimensions manually here which is kind of annoying because if you change some images at the beginning or anywhere in the tight junction operations you might want to have these values to be defined automatically one way to do that is to use a tool we have around here which is the for example texture dimensions as the name implies you simply pass in a texture and you can get the width and height in pixels of that texture so now it's set automatically and we have our nice glowing effect there is a plan on the cube and by playing with the the power of the gamma you can see the effect of the glow if I go fullscreen you're going to notice it's not running extremely smooth the reason is that remember our original texture on which we applied the Gaussian blur is the same size of the screen so in that case that's going to be 12 90 x 960 or something and it's a lot of data to be processed by the by the GPU because computing a brewer is a very expensive operation so there are ways to optimize that I'm not going to go into details for this example and I'm simply going to show you the end result of that optimization this is the version you're actually going to find any examples provided on the tiger DVD example compositions so this one is the final at riddles and it's very smooth and we got a nice flow effect basically the difference with what we just build is after the texture after the step where we generate the texture with the cube we basically down sample the texture to one that is always 256 256 and we feed that downsampled version of the texture through the through the the gaussian blur and the glow effect and because it's only 256 x 256 it's going to be very fast to be computed back to the slide please so now it's time to look at how we are going to play the composition there are three possibilities you can use the QC view which is customized and if you will provide with court composer and you can use it directly in interface builder you can do kind of advanced playback still in interface builder using the QC pad controller to use the bindings and then the final way of playing back a composition is kind of the hard way where you have more control over the composition and that's going to be using the low-level QC 10 class and you have to use that programmatically okay back to the emulation please so let's launch interface builder the first time you launch it you're going to have to add the quad composite palette which is not loaded by default so for that you show the the preferences of interface builder you go to the palette area and you simply click on ADD and you will find a quad composer palette into developers a developer extras palettes and then court composer ok now we up and running so let's create an empty application and I'm simply going to drag and drop an instance of the quad composer view onto my window and then display the interface builder inspector in the attributes pan you're going to find a load button which allows us to obviously specify the compositions we're going to be playing back so I'm going to take the optimized version of the glow effect which is created and now I can just go directly to text interface and it's up and running so so far we haven't even type of seeing a line of code to do all of this what might work it's worth noticing it ok that was for very simple playback now we're going to do something that is a little more advanced and use the cocoa bindings so let's go back for a minute to the required composer application and I'm going to hide interface builder there we go and let's open our optimized low so that's this one so we said at the very beginning that the system was completely compliant with the coco bindings that means key value coding and key value observing and to use bindings you need to use loosely keys to specify the objects when you look at the tooltips you will notice that you may not actually see clearly on the screen but the third item is the P because each patch in the system in a batch tree as a unique key to identify it and then each input port or output port also has a key and you can see these kids by simply displaying the tooltips so in that case the the key for the patch is rendering texture underscore one and the key for its output is output texture so we have a way to identify ports and and patches in a tree what we are going to do now is in order to use that composition with bindings we're going to transform it into some parameterize composition get excited for example we want to we might want to have to parameterize the background color or the intensity of the glow effect how would we do that well let's go inside the rendering texture and the color we interested to change is defined by the clear patch so what we are going to do is bring that input to its parent patch so for that I display the contextual menu on the patch and I have an option publish inputs and I can select the input some interested in which in that case is clear color and specify a name for the years for the new input simply color now if I go back to the parent patch you will notice it now has a color input which is actually with corresponds to the the clear color input of the clear patch that is inside this macrophage so that is called publishing inputs outputs you know an input is published to its upper level by because it's going to be like a full field dot instead of an MP dot let's publish but remember we said at the beginning we have a patch tree and at the top of the patch tree we have a macro patch like any other micro patch which means we can also publish inputs and outputs to this micro patch so let's do it we're going to publish the new color input we just created to its parent patch which is the root patch and we're going to keep the same name and now I have an input that is at a very top level and it's kind of an input of the composition and it's the color input so you can use the the display pen here to actually see all the top level inputs of the compositions which can be considered like the parameters of the composition itself let's publish another input like the intensity of the of the glow effect let's save the result as our part 4 ok now I'm going to go back to IB and this time we're going to do playback using bindings to use bindings we need a controller and we provide one which is the QC patch controller and which you instantiate simply by dragging by drug and dropping it onto the document window remember the way binding works is that you have a model in that case that's going to be the composition then you have several views that are displaying that are interacting with that model and interaction is made through the controller that is that is standing in the middle and doing the interment yet on the controller I can obviously know the composition and I'm going to load the one I just created then I'm going to go back to my QC view just unload the composition that it on it I news now bindings so I display the bindings area of the attributes inspector and then we have a property that is patch which obviously determines the patch that is displayed by the view so I'm going to bind that to the patch controller i just created and i want to retrieve the root patch of the composition and displayed on the view and the way you do that is simply by using the touch controller key which is going to return the root batch object so remember the tub patch had to compare the top level of the composition so now I can test the interface and it's the same residence for nothing has changed except we going through bindings which give us a lot of flexibility because what I can do now is for example pick up a slider and use that to control our power input we just created on the composition remember it's this one okay so the way we do that is simply by binding the value on the slider to the patch control to to the patch controller then which we've the root patch object and then we need to specify the key pass for that to access that bad color input if we look at the court composer we can see that the color input if you look at the tools if you can see you can look at the key and you can look also at the key for the published version of the input and it's that case it is co somatically computed by the name so because I use the name color the key conveniently set to color it's always going to be the case unless you have all right you already have a color input and then the system is going to ask to pick up another name for you so we know that that input is identified with color so now that gives me the input and what I'm interested on the input is the value itself or am sorry it's not actually colloids its power in that case so its power and what I'm interested in is the valuable power input so I add that value okay now i can run the 9260 attributes of the slider so that it goes from 0 to 1 only and that it is in continuous mode okay that's there so now I got live interaction without typing a single line of code still of the composition we created and anime and controlled use i have in my window let's go a step further and drive the the color now so same principle we go to the bindings area you bind the value to the pad controller we go you retrieve the patch object itself and then we say color switch with the coloring put that value to reach with the value on that input okay we can do better than that if we go back to a composition I can even take go let's get it running I might want to change the images the image i use as a texture on the cube so the way to do that would be to go back to the cube and we would like to have the same texture be put on all the faces on the q and here we have like six connections going on if you want to simplify your life and this happens quite often when you want to add the same value be said on several inputs at the same time and when you change one of these values you don't want to change for example of six of them each time for that we provide a little tool which is the little utility patch as you'd say which is the input splitter if you display inspector you can select the the type of data it's transmitting so the input filter is so in that case we want to transmit a texture and the input splitter is simply meaning the value that arise on its input to its output so I'm going to connect the six meses to that single output and now i can i can set that value from one single and foodborne instead of sending it for six obviously I can do this and I get the same reason as before and I'm going to publish that input and name it texture go back to the parent patch get rid of that one into useless now go back to the parent patch it's right there the texture input we just created publish it again to the top level to the upper level I mean and publishing publishing it one last time to the very roots I publish the output instead we go texture and keep the same end now our composition has three input parameters the background color the power of the the glow effect and a texture on the cube I'm going to set that and stop it go back to IB we load the composition on the controller for the composition is actually stored inside the controller and set with the knit side so you need to reload it when you change it now has my brand new composition still working the same ok and in that case I'm going to draw a net image view and bind it as usual I go to the bindings panel and display the value pick up the pad controller which with the patch and type this texture to get the texture port text your input port and that value to a tree the value itself one last thing I need to do is make sure the here we go NS image view is actually editable so now it's up and running and what I can do is just drop any kind of image file there you know and [Applause] you see our starting image will affect unum back to the side please alright so that was playing back a composition without even typing a single line of code but what if you really want to type some code well you got to use the QC renderer class unfortunately it's still very simple to do so only three steps you need to have an NS OpenGL context around because this entire system quartz composer is basically running on top of OpenGL at the primary backbone and then we need to have a QC renderer instance and we simply run a friends using runner at times the run the right time method let's look at some sample code so how does it what does it look like well I'm going to assume I have an NS open G of you standing around which is called my NS sngl view and I retrieve its NS OpenGL context from it then you create an instance of the Qt renderer using that context and using the file the path to a fine compared to a composition side somewhere on your hard drive now if I want to render ten seconds of that compositions with like 25 frames each second for one second you can do it in a very ugly manner in that simple for loop or you have to do is call renderer you pass the appropriate time and then you need to flush the buffers on the OpenGL context to display what was just drawn on screen more in a minute about why we need to do that and eventually when you're done you just release the renderer and all the cleanup is going to be done automatically now why exactly is the renderer not flushing the OpenGL buffers itself so that just after a frame is rendering it will be seen on screen well the goal is that this allows a lot of flexibility where you can interleave the composition rendering with your own OpenGL code and if that says I can do some opal Drouin OpenGL drawing before then we are another composition and do more OpenGL drawing afterwards to create kind of underlay overlay so it's very easy to integrate composed the quad composer system into your application if you into your already existing OpenGL application if you want to add add some examples from flying logos all around the screen of stuff like that you can design them easily in other compositions and then import them into your application we've seen that one way to communicate with the composition is by doing everything using bindings in interface builder now you can also do that programatically and the equivalent calls would be set value for input key or value for output key to which with data from an output of the composition and the way you would be doing that is simply you pass an object corresponding to the value and you specify the key corresponding to the input port of the composition or the output port you may have noticed earlier that the runner at time takes an optional dictionary of arguments and what we are passing here are what we may pass here actually because this is completely optional is the current NS event that is being processed by the application and you can also pass the mouse location in normalized coordinates you find more details into the header corresponding to this QC renderer class you may wonder why you have to actually pass this and it's even stuff and the damask location once again is optional but the real reason is that you might want to have the QC one or system running in to come online tool or some application where you don't have any UI so it's not even it's not even there I mean this data is not available so the system cannot retrieve it another reason might be that you're running that into your application but you don't want the QC renderer to actually steal even from your system so that's the downside of using a low-level API like this is that if you playing back a composition that expect user events and in that case only the mouse patch I think is actually using them to obviously detect mouse up and mouse down you will need to pass this events manually so what kind of value should you pass when you use set value for input key well to each type of pork there is a corresponding nsobject you can pass if it's boolean index or number court simply personalized number yoga sleep ass and its core object for co report and so on what you need to pass for now for the texture bitmap or image port is you can pass NS images or in the case of the image port you can directly past the CI image you obtained from some other place so what if I want to control for grammatically the power of my glow effect well the way you would do it is by simply adding a called runner or set value and the power input is a number input so as we said earlier we're going to use a simple NS number for that so we back up the next number with the value interested in and we pass that to the renderer for the correct input key the system is kind of small enough so you might actually you don't even need to pass with NS number anything that will any objective c object that is going to respond to float value or double value or int int value this kind of stuff is going to work so back to the demo please i'm going to be going to show you very simple application of the QC 10 class and that is going to be the simple playback application you've seen at the very beginning of this presentation so the simple application is basically playing full screen and compositions which creating a full swing and expand GL context create a QC renderer on it load the composition and play continuously and it's like two pages of code and any portion of it is actually setting up the OpenGL contact capturing the screen detecting the fact that the user is drawn dropping a file on the application icon and so on the quad composer part of like couple of lines of code another example is this screensaver that is now provided with Tiger you might have seen during an veterans keynote I think so what exactly is inside a screensaver well let's have a look at it so I can show the original there which is somewhere into the system folder and if you look at the package content go to resources you will find a swell composition trial and it's right there so all that animation was done would it change the corner or something all that kind of complex looking screen server was done without typing a line of code except for the simple playback part into the screensaver bundle back to the slides please ok can we see the slide please okay thanks and the last thing I would like to show you is kind of open some doors or what exactly you can do with quartz composer and Willian says on the point that it's really an open system and you can bring data from many many places you've seen earlier that it was there was there were some interactive compositions retrieving data from the mouse for example or from an RSS feed but we can also use MIDI controlling devices video cameras and even more the composition we built the compositions would build kettle to respond to their execution environment somehow so that should be the capabilities of the OpenGL renderer or the dimensions of the rendering area so demo machine please okay for the last demo I'm going to still modify our composition now-famous glow effect composition and add more interactivity to it okay so let's see let's get a picture first that's not there anymore okay ok so for example here I have an immediate device controller which is simply as a bunch of sliders and knobs and on its on it on each of this knob and each of the slider there is a corresponding midi controller ID and when you touch them it's simply signing the current value of that control to the midi system and we have here a midi controller patch i can drag and drop and by looking at the inspector specifically at the settings panel you can configure it completely and i want to listen to my MIDI input here you can select several sources you can filter the MIDI channels you want to listen to and obviously select the controllers you want to observe in that case I want to observe controller 32 actually so I'm going to select 32 in the list and now we have a new output that was generated with the 32 controller and I'm going to delete the the control number one it's outputting a value that is normalized between two on one cause it's more convenient that the 01 27 values that usually get from media equipment so i have to do is connect it there and now i can use my slider here and as you can see I control the blow effect simply with sighs all right let's do a little more so instead of using a static image or something that is generated from some other compositions let's get some video in so I just use the texture with video touch we go and I'm going to feed that on the let's say well the cube the faces of the cube so as to do is connect this connect the the texture output to the texture input of the which was eventually going to end up on lie on my cube itself you can see there are several settings you can set for video captures and I need to turn on the video camera I guess let's try this okay I might have to restart the composition so it detects the video input okay so now i have okay- video input there and control voice via subscribe now we don't it be cool if this composition was still working when there is no video input but what happens if I actually turn off the video camera and so the image is still is frozen and when I restored or caters to some image remaining try again ok so if no camera is connected this patch is not going to output anything so we don't mean it wouldn't it be nice if we were able to display some text saying connect a video camera or something but only if there is no video camera connected for the way we detect if there is a video camera connected or not is by the fact some object is defined on it out on the output for that we could use a tool that is the multiplexer the multiplexer is simply a number of inputs and you select by using the source index input you select which one of these inputs it can be forwarded to the output so in that case i'm going to set the multiplexer to manipulate texture objects reduce the number of inputs to only to connect the output of the next lecture to the input of my of my rendering texture thing macro connect for example the output of the texture here of my video texture to the source number one and on source number zero let's just put an image with some text on it so there is a specific generator for that which create a bitmap with some text so I'm going to connect it there ok and I can change the text like no video ok I have no video on my cube how would I so it's easy you know you just return on back the video camera there it's running ok and I restart now by changing the input I'm going to get either the video input all the texts now we would like you see that to be done automatically there is one little hack you can use to do that it's simply using let's fish it is conditional or logic here we go because you can actually connect an object like a texture or something like that to a boolean input and basically if there is an object the boolean input is going to be set to true and if there is no objects can be set to false so here are the simple patch which is doing a logic comparison between two billions and I'm going to use that so I'm going to set it to or comparisons set the second input too true so it always pass and connect the texture object to the input there so what we have now is on that'd put i'm going to have true if there is a texture coming or false is there is no texture and i can take that boolean and connect it to an index input and two is simply go electron set to one and false is going to transfer to zero which is exactly what I need so now I got a system that is automatically defined and if i have a video camera connected i'm going to get the video camera image if I don't have a video camera connected I'm going to get the no video picture we go I kind of saturated thing oh it is long oh that's right that's right I do their true that is incorrect and go oh oh that's right yeah ain't that tension there there we go that's better so that was kind of an interactive composition and we can do the last thing I'm going to show you is how to have that composition respond to the OpenGL render capacity capabilities because all that part here is basically using core image and it's using in it hardware so it's only going to work if you have a video card that is supporting the proper expansion set so what is the composition is going to run on it whether it's not going to crush or anything but all this part here is going to do strictly nothing so you might want to display something else instead so let's go back to the tools and I have a convenient opengl info patch which is returning me for example the renderer vandor or an renderer name of the door of the car under the version and can check also for the existence of various OpenGL extensions the way you use it is that you display the settings pane and you can add some fancy extensions names here for example I don't know my fancy extension okay add it there and now you get a boolean it's a boolean output so if the extension is supported you get true if it's not supported you get false so busey GL ARB vertex program is supported on this computer fragment program is also supported but my fancy extension well is not supported for some reason okay now what I'm going to do is do a simple comparison on the renderer version so that would be numeric and then conditional so the renderer version is a number and in that case I think it's OpenGL one point let's see 1.4 and what I want to test is do we have an OpenGL renderer that supports OpenGL 1.4 or later so I just do is greater or equal than 1.4 and you may have notice that all the consumer patches automatically have an unable input added by the system which is a simple boolean and you can turn on turn off the consumer patches from that input so that enable input is at the top of the patch now what I'm going to do is connect my result to this unable input so as we can see doesn't change anything because if you see this video card support OpenGL 1.4 now what is for some reason I wanted to test for the support of OpenGL 1.5 well I would simply set one point oops I don't even put the one place for them here we go I would simply put one point five instead of 1.4 and greater or equal and okay and I must have done something wrong here oh that's right that's pretty bad here we go so now you can see that simply changing the the version you want a drag for it's going to turn off dynamically part of the data flow so it's important to I'm going to conclude on that that compositions you create are absolutely not something that is close you can really interact with then you can feed data in you can retrieve data from then they can be dynamic dynamically responding to the environment and it's a great word system where you can experiment with the brand new technologies we are adding in tiger without typing a line of code and you can enter and you can add them easily to your applications back to the slides please so who you might want to contact about this there is a threaded round with our and graphics and imaging evangelists or possibly myself for more information you may want to look at at to documentation we have visual computing with what composer which is a simple introduction followed by a tutorial which is pretty much the tutorial we just did right there and the other documentation is image processing with the cordage framework and it actually contains a description an extensive description of all the features provided back your image which are natively supported by the system by the courts computer system you will also find the sample code for the quad composer player ID mode today which is going to be on the ADC website and you will also find some demo compositions located in to developer examples quad composer and you may obviously look at the screen savers