WWDC2003 Session 716

Transcript

Kind: captions Language: en thank you all for coming to our advanced quicktime programming techniques topic so advanced we brought our own exer which all our demos will not be running on the xserve in fact what we're going to be talking about today is a topic that a lot of people find pretty important one is thread safety at multi threading in quicktime a second topic we're going to cover is some integration we've done in QuickTime 64 and Panther for color sink and lastly we'll be talking about the complicated topic of audio video startups inc and just an overview of some of the technology frameworks will cover today a quick time obviously color sync quartz quartz 2d and core services for some of our belief reading api and now I'd like to introduce Sam bushel to come and talk about multi-threading thanks Tim good afternoon I think I could get him troubles up with my water bottle on that thing quick slant heritage on Mac OS version 6 7 8 and 9 meant that many of its api's are designed to be run from a single thread macular sin brings us not only not multiprocessing the ability to do multiple things in multiple applications at once but also it removes some of the traditional limitations on what we could do with multiple threads within an application I want to talk about some of the work we've been doing inside quick time to allow to use use multi-threading with the QuickTime ap is to make your application more responsive and more powerful first let's talk about the basic problem is single froze ap I'd if everything we're instantaneous no one would care will be okay if everything had to run from the same thread but many common operations are slow and it's a bad user experience when these slow operations delay application responsiveness it's inconvenient to users when they can't export to movies at once or export a movie and play a movie at the same time so from our perspective the urgent need is to allow developers to move these slow jobs to background threads so they can free up the main thread to to provide a responsive user interface in Panther the particular operations we have in mind are still image import and export opening movies rendering movies re compressing movies exporting movies but in Panther we are not changing the way that movies are played back playback has already been is already implemented in a way that avoids blocking the user interface it's been that way since QuickTime 10 it's the other these other operations that have become you I hogs we will make these things safe to perform on background threads so in pants of the model we're aiming for is to allow independent threads to work on disjoint sets of QuickTime objects this is the most important thing I want you to take away from from my session today dis joined us if one thread has its set of QuickTime objects and another thread has its own set of quick two objects and never the twain shall meet everything's going to be fine but we're not making it safe for two threads to access the same objects simultaneously if two threads need to manipulate the same object then those threads need to perform some kind of negotiation or locking around this now this is no different from any unprotected data structure you might implement in your own code you need to have some kind of thread protection before you allow simultaneous access the exception to the destroying this rule arrives with the file system the file system has always been designed to allow multiple readers of the file and so in order to understand this this slide we need to make the distinction between a movie which is a data structure in memory with a capital m and a movie file which sits out on a disk somewhere and may or may not be a QuickTime movie that maybe is some other for the format that's been imported and is viewed as a quicktime movie it's okay for two threads to each have their own movie data structure that points to the same movie file and okay for them to be displaying or rendering different parts of that movie same time and moving around it male may not be okay for one of those threads to be modifying that file obviously but that's actually come as no surprise so let me repeat this really important point again don't let two threads work on the same QuickTime object at once it's you as the developer who has responsibility for making sure that this is the case it's also the developers responsibility to avoid going creating a dialogue or a window in a background thread if you go and look in the headers you'll see that the human interface toolbox is not thread-safe it's the main run Luke the receives event with weather in carbon or cocoa you I needs to basically stay in the main thread one more restriction this stuff is only going to be safe to do from from threads in Panther even if you install a spanking new version of quicktime on jaguar you're still not going to get these thread safety improvements why not because there's a lot of plumbing underneath quicktime that we rely on a lot of api's and core services that have been improved not just to enable the stuff and that stuff is only available on Panther so now that I'm done telling you about all these restrictions it's over to Tim to show you how great it is because it is substantially nice to be able to run the stuff from multiple threads great thank you so I have three applications i'm going to show you demos up and they're going to be available on the Apple Developer connection website under the QuickTime etc so WWDC will put up some URLs at the end so what we did was we wanted to be able to make some clear applications that showed not only the code that you need to do to make your applications thread safe but also demonstrate visually so that people can understand exactly what we're trying to enable the first application thread tester pro is basically trying to emulate what finder or I photo would be doing in the preview mode of viewing graphics important images and that's what this demo is showing so what I've done is I've opened a window and I'm going to open a second window off this application and before i go on i'm going to set each window to use the main thread as opposed to the p thread so right now there's one thread in this application that serves both windows and then I'm going to select some images to preview so I'm going to pick some smaller images that I took when I was on vacation in Southeast Asia far too long ago and i'm going to select some larger images in this other one and while that one's loading you can see I can basically load up these images and it's fairly responsive but when I tell it to autorun which is basically going through every image and I'll tell this one to autorun now these large images are what effectively are doing they're blocking the other thread from processing that image because the images in the front window are much larger they take longer to decode and not only is that going on but you can see that the UI is pretty non-responsive I can't really click it's kind of like waiting for things to get done so let me stop that and again even stopping takes a little bit of time so just to give you an idea this is running on a dual 14 2 g 4 so what I'm going to do now is I'm going to switch these applications to each use their own pthread worker thread so now the operations in the UI are still going to be it's going to be basically using a background thread to decompress the image and when that image is decompressed it's going to transfer it and copy it to the display done in the foot in the main thread so the main threads doing all the display work but the background thread is doing the decompression so again I'm going to do the autorun and now i'm going to start this one you can see the first changes as i do the large images the background images keep on going that's because there is a multi processing machine at each has their own thread so it's distributing the work that's one advantage of using the thread safe multithreading quick time the second thing is you can see that the UI is very responsive even though it's doing work I can click and accepts the clicks I can go to the menus etc and I can stop it so that is the first demo I wanted to show that's graphics and quarters using the same application doing two sets of images on two different threads so the second demo on showing is that we've done the same work for graphics exporters one thing that you'll see here is that in the application as a menu that says only use thread-safe components so we're able to detect through some techniques and we'll talk about which components are thread safe in the OS and here I'm going to limit to the ones that are and again I'm going to switch to the main thread and select a folder export file so now it's loaded in a bunch of images and I'm going to do the same autorun it's created a folder basically spewing these all out in a different format and while this is going on you can see it it's responsive because the images are small but it's not exactly superb so let me stop that and get rid of that guy so back to the main thread I'm going to open another window and again select the same export folder and switch this window to use a piece red so now both windows are using P threads and let me just get them both going and again now we have two applications bringing the images and exporting them to different formats what is basically doing is it exploiting them to all the formats that are shown here so this is writing and we see that there's going to be two folders created here and let me just stop that and stop the other one and you can see what's done here a couple of these images were loaded up in all the different formats so once again you can think of applications like iphoto and other applications which are exporting images they can now do these in the background so that's pretty much the story for importing and exporting and where did my window go so my last demo is so with movies you kind of the same ideas you want to be able to do things in the background you want to be able to do multiple operations in your application when you're doing things like exporting movie so again i have an application that called a threads export movie and i can select whether i want to use the main thread or pthread in this case i'm selecting a movie using the main thread and here's the movie and as you have a mark Sun once again and now I'm going to export and oh I did that earlier today no worries so let me take the settings I'm going to pick Sinopec's nice and fast it's thread safe so now i'm exporting this movie to sinha pack and you can see it's taking quite some time and while it's doing this thing I can't do anything with the application not very very good user experience so let's just wait for this to be it so what we're going to then do is when to switch this and be able to do at our background thread and when you're doing that the application will be responsive so let's switch this guy to be on a P thread and I'm going to open a new window it's time i'm going to select a long version of andrew and again it's using AP thread so let's export this guy test move yes replace lat and cinna packs the settings are good so I'm going to start exporting that while I'm doing that I'm going to go back so it's responsive and I'm going to go an export test to move and replace that one and again i'm going to use tuna packed so you can see that i'm able to do that export it just for fun I'm going to open a new window like Andrew and play this and just so you can see what's going on we're able to actually export two different movies and play one of the same movie by the same time so basically well be threaded rendering and multi-threaded transcoding in QuickTime six core thank you thank you guys thanks a lot perfect thank you Tim so Tim gets to show you all the good news I get to tell you about the bad news what's the bad news bad news is some components are not thread-safe even if we could make every component that we ship on quicktime in panther threadsafe quick down is an extensible system and users can install third-party components and they might be last year's components they may not be thread-safe so we're going to have to cope with this fact that we have thread-safe code and thread unsafe code in the system when you do a high level operation in QuickTime that may be like opening a movie and signed up to display it or converting an image from one file format to another that can involve maybe a dozen lower level components and often you don't know which components you're going to use until you actually try and do it so that means that some media files and some convert some conversions cannot be safely performed from background threads and your application is going to have to cope with finding that out dynamically now a bad a bad way for your application to find out dynamically would be for it to crash at random on users machines a better way is for it to get some kind of message so the new rule is for applications to create threads to call quick time is you have to call this API to say from this thread please don't let me open any non thread-safe components it's a component manager call it's in core services since the cs prefix and it's in Panther if the component manager is about to open a component and it notices the component is not thread-safe then it will return it won't open the component because even the open call might do things that are unsafe and instead it will return this error code and that error code is likely to propagate back through other api's to you and if you receive that error from any API in quick time then you need to cope with that and shift the work over to the main thread in the Panther seed there's a mixture of thread safe and non threadsafe components and you can see some of those on this chart others you'll have to work out for yourself you can test that by checking the component thread safety flag just with fine mix component or get component in folds it's a flag to find in the component manager header file so the most important still image formats are the ones we've gone and made the thread safe first mag paint is not not because we hate mac paint it brings a smile to all of our faces the real reason is so that you have some well-defined test cases for exercising the migration feature if you should use the components that are thread safe to exercise code like Tim destroy it didn't demonstrated but you should use the components that are on the right to exercise your code so as to make sure that it's safe even if you encounter media that's supported by components that aren't thread safe in some situations you're going to want to move a QuickTime object from one thread to another thread now with graphics importance in graphics exporters you can manage that with locking that you perform on your own just make sure that it's only being called from one thread of the time but movies are kind of special because of the way that they work movies need to know which thread they belong to at any given time so there's a pair of AP is that you call to detach the a movie from one thread and then to attach it to another thread a bunch of QuickTime api's lets you install callback routines these forward to several categories there are some that have always been called back from funny threads that hasn't changed but there are other ones that in general get called back from whatever thread is doing the work now in the past that was always the main thread but now you might move that work to another thread and if you do you need to take care with your callbacks and rethink whether they're actually thread safe as well in particular you should take care with your progress callbacks don't just call set control value to adjust the progress slider because that could cause the progress labs to be redrawn and like I said earlier we've header file say that the component managers the control manager and the dialog manager are not thread-safe and that could cause quality Emily calling set control value although we think about it it's just setting a value it actually would cause quite a lot of code to be run in order to draw that and it may not always be immediately obvious that the code that you're running is not thread-safe it thread safety bugs can be nicely tricky and hard to reproduce and it's better to be safer than sorry if you're a developer who writes QuickTime components good for you please make the next version of your components red safe and when you do that you should set the thread safety flag on your component it's just a component flag in the global space what kind of things you need to do in your component to make it thread safe and what kind of things you need to do in your application to make sure your thread safe well a bunch of that plumbing that I've talked about in beneath quick time here are some good examples it's now safe to create G worlds and drawing them from dream to threads it's now safe to open and close components from background threads it's safe to resolve aliases it's safe to create and manipulate handles from threads in order to make that work safely we made the what was previously a global error variable for the memory manager memory we made that from a global variable into a per thread variable so the area you get back your code you get back from from an API call on one thread one interfere with the API call you make on another threat you may have read advice saying stop using handles use malloc instead and there's a little bit of confusion about that if you're using new handle to allocate private data structures inside your application then it's a good idea to examine those and see if you can switch them over to use malloc or kallik instead if they never change size that might be the right thing to do because malik is faster on mac OS 10 handle the handle implementation is built on top of it and it has to maintain some extra information that isn't necessary for some elekid color it has to maintain the master pointer blocks for example however we use handles in quick time and in the alias manager and so forth as a part of our dialogue of our API we have over 480 is that pass handles around in quick time and they're not going away so your use of handles if you're using it for those purposes is obviously the right thing to do there are some other managers and ap is that won't be becoming thread-safe the resource manager ap is involve a global state called the resource chain it's a list of open resource files and effectively it's an implicit parameter passed to most of the resource manager ap is and as such it's not thread-safe however the one-shot component manager calls that getcomponent resources such as getcomponent resource and get component public resource getcomponent public resource list these guys are thread safe so for example in your component if you come and thing that happens in image decompressor components is that they need to get the CDC I resource to answer the question in get codec info you fill in this codec info slab and a way that often this has been done is by calling open a resource file get get resource and close a resource file because those API is nickel eight the resource chain they're not safe to do instead you can use a one-shot call getcomponent resource and that is actually safe and that's what we recommend you do inside your gate code again phone call there are the examples of this in other component pipe another common practice in components has been to have some shared global state that shared between confirmed instances and generally people would use the component rest call retcon as a way of storing where the state was now when all components were used on the same thread and memory was scarce on your Macintosh LLC it was a good idea because you wanted to save memory however if you want your component to be used by multiple threads at once it's now a bad idea and it's time to upgrade that code now if you really need to share dynamic state between components and those component instances might be used by multiple threads then you need to pick up whatever pthread library or MP library you api you'd like to use and use some kind of protection mechanism to coordinate that some kind of locking on the other hand if you are doing if you're using shared Global's for constant tables and maybe allocating them once and then hanging on to them later on then the best thing you can do is to turn take those tables generate some source code to build those tables label it as const and just build it and put it in your executable it might seem a little bit eager or a little bit weird but it is actually a better thing to do for a number of reasons if your data is constant and you tell a compiler here is a big constant array then instead of putting it in a place in private memory to one application it'll put it in the read-only section of the executable which for a eunuch see reasons is called the text segment I don't understand that history but this is the same place where your executable code is and the nice thing about that code is that it can be shared between multiple applications so as well as sharing these tables between multiple instances of a component inside one application these are shared between multiple instances of multiple of components across applications that stuff needs only to be resident in physical memory once and even better when those met women if that memory needs to be pushed out for paging reasons it doesn't need to be written back to the disk it doesn't need to take up space and swap files because it's read-only and so the colonel knows that it can bring it back anytime at once just by mapping it back from from the read-only file so that's the best thing to do in cleaning up your code in that area so to summarize the thread safety part of this talk we're giving you the opportunity to make your applications more responsive and more powerful by doing work in background threads in return your part of the bargain is that you need to structure your code so that it will cope with legacy non threadsafe components component developers please make a component thread safe so that the application developers can take advantage of them and this is a panther only feature now there's one thing I haven't met we have mentioned on the slide which is that we have a seed if the opportunity to secede some of you and we have set up a mailing list website instead of an address that you can send malsu to request involvement which is QT thread seed at apple com that email address is going to be around just for next week so you should send email to us in the next week I have some reminder cards you should come to me after the session and I'll give you a reminder card and you can take it back and arrange to tell us what you're going to do and we can see if we can see you so now I'd like to hand things over to me so if you've been asleep through the multi threading part of the talk it's time to wake up again we're going to talk about cook times integration with color think and I want to begin this by taking a sip of water so let's start with a jog and update the QuickTime has a P is for drawing still image files and we have a P is for writing still image files we talked about those earlier their graphics importers and graphics exporters color thing provide services for matching colors between profiles if you haven't heard of these terms a profile describes how pixel values relate to the colors we see it's a mathematical model of this relationship different devices like cameras printers scanners and monitors they each have different profiles it turns out that the relationship between a pixel value stored in a some kind of buffer and the color you end up seeing is going to be different from each of these different kinds of devices and often for different individual devices color matching the big thing that color thing provides color matching is hands leading pixel values to compensate for the differences between profiles so that you see the same color so many kinds of still image files have different ways of embedding color singh profiles and part of the graphics importer and graphics eight exporter API since QuickTime 40 has been affordances to read these profiles and to write these profiles and new files but that was pretty much all we provided to draw an image with color matching you'd had to write some extra code and to preserve a profile when converting between one image format in another you had to write a little bit of extra code it wasn't much but because color Fink has often been viewed as a pro feature applications that didn't view themselves as pro imaging applications often didn't spend that effort and this meant that the results are patchy and even non pro users a beginning to notice this and start to get a little bit tetchy about the fact that this picture they've got doesn't the same everywhere so here's what you had to do in order to draw an image using a graphics in Portal with color matching you'd get a graphics importer for the file you'd draw it into an off-screen G world or buffer you'd get the color thing profile by making the call on the graphics importer and then you choose some destination profile often a generic RGB profile then you create a color world which is a color sync API and then you would match which means actually performing the translation it match from the G world to another G world and then copy that to a window this is a little bit more work and if you really cared you probably did it but it will still some work if you wanted to preserve a profile that wasn't quite such a big deal you had to get it from the importer and set it on the exporter and you know in QuickTime 40 this doesn't seem like such a big deal for people to have to do but if you didn't do it then you lost some kind of important information and people who are relying on that as part of their process or people who just were messing around with some images could become a little bit unhappy if they lost that so here's what we're doing in Panther graphics importers will now draw using color matching by default and we'll use the CMYK to RGB conversion that color singh provides by default because it looks better than what we were doing which was somewhat naive furthermore when you convert image formats using graphics exporters we will preserve the profile we can only do this if you are plugging in a graphics imported directly into a graphics exporter or using some other mechanism that lets us or automatically detect the profile and push it through if you draw into some off-screen G world and then export out that off-screen G world the G world is not tagged other kinds of things like CD images and bitmap context in core graphics ah but the quick tour equivalents are not so we can help when we can help otherwise you may have to do some work still yourself but this is what you do now in order to draw an image using color matching you just draw and we will draw a generic RGB if you haven't told do anything else will go and call color sink and I might as well show you so let's go to demo one Wow the exports finished so all of the sample code that Tim showed you earlier and the sample code that I'm showing you now is available but because it's a panther the school requires a panther api's you have to go to the Apple Developer connection site and login so you go to connect our com login and then if you look under download software and then the name of the developers conference then you'll find a bunch of stuff there's more stuff each day and here is I'm going to show you an application that you could be playing with if you'd learn that I because it's an image of application you drop images on and they get drawn i called it drop draw that's the kind of an imaginative person i am i mentioned the CMYK rendering because this we have had substantial complaints about and previously feedback forum if i have turned off this flag use color think matching to pretend that we're back in jaguar and this is what you get this is a CMYK tiff file it's a picture of a palm tree and and it is recognizably a palm tree but if you are a photographer you probably wouldn't be very happy with that if it was a sort of photograph you taken this is what it looks like if you're using the color sync CMYK to RGB conversion and it looks a whole lot better you can see more depths of detail in the shadows and the waterless looks less radioactive so this is now what you'll get by default out of if you're destroying using QuickTime that's kind of nice but because you're drawing using graphics importers there's all these other features that we give you like we give you the ability to crop and masterworks this is a cropping using source extraction in source space you can apply a rotation matrix do that rotate the image we can you can clip we can give it a clipped region and there's a fairly complicated clip region we can do them all the obvious thing to do there we go so CMYK images look a lot better and you can still do all of the things that you used to be able to do with graphics importers we've made this we've introduced this feature in those orthogonal away as we can the difference between unmatched and matched images is a bit more subtle when you come to other to RGB images and and there we can be forgiven for thinking that there wasn't much of a deal if you look at this image this is a picture that Kemba risk and took on holiday he says that he was on safari it's billy hard to tell whether he was in a zoo but will take probably to him it here's a picture of some wild beast the in looks it looks great you know if I got a bug report tank look how bad you made by image draw I wouldn't believe that until maybe I saw what it looked like side by side with the matched image and you know if if you saw this image if you were there in person and saw the animal and you know that it looked browner then you'd be unhappy if it wasn't that round but i think this i think it even looked area here's another example this one this one's pretty good here's a picture of a lion and here's the same line with color thank matching and that looks a whole lot sexier ones much more romantic picture of a lion so the photographer's side of this is look how bad these things look this is this is terrible the programmer who you know isn't the photo photographer doesn't know what they're complaining about but in fact it's something so sorry a professional who's really going to care about this I think anyone who's really getting into the photography once their images to look like they were it's a otherwise you feel it's your your memory of the event was a little bit the face but the problem is if you only see one of these how do you know woody did the right thing so coming back to slide we've got this problem how do you know whether it's right or wrong could you pick if you only saw one of these did you pick the right one I'm not sure that I could so there's a bit of a problem for developers and the color think team has come up with a good solution they've created some files with special profiles it turns out that people have discovered that you can use color sync matching because it does all of this color space conversion you can use that for doing explicit image adjustment you can say make the thing that's a bit blue or green make the sky blue that kind of adjustment and people use it for doing kinds of color correction on purpose not just to make things look the same and they've created some image files that have embedded profiles that do a whole lot of changing in fact they switch the colors red green and blue around they all switch them around and then they make one of them vanish so the image at the top is what it looks like if you don't do any color matte color mad matching and I don't know if you can read it it says the embedded test profile is not used and is not double matched but the second one where the text has gone green is if you do the correct compensation using color sink or something else and it has switched the colors around and the first word not has vanished because of this special profile and so it says the embedded test profile is used and not double matched and should the compensation of the transformation between these profiles be applied again then you'll find the image at the bottom which is turned red and the word not has vanished and now it says it is double mashed and but this is all coming from the same image file so to work out whether you're doing the right thing or the wrong file you can just drag this into your much into your application so it's a couple of this these image files are also available if you go to connect at apple com there's these images of trick profiles we've got a bunch of different image formats here and I can go into my little application so there's one which always looks blue because it doesn't actually have a profile it's there as a baseline it's there if you want to create other image formats by taking the profile out of one of the other images so here's a gift and here's the test text the embedded test profile is not used and let's make it be used and look it's used whoo so we've got a bunch of images here I'll turn off that one to show you that they all work so this is in my application which since you haven't downloaded the sources to you don't know what I've actually doing anything very fancy but I can show you in a few other applications that this is is really a widespread effect and we can make sure the things that are doing the right thing the finders preview here you can see that the finder is displaying these with matching the finder does not call a color thing to do this stuff here's another application that doesn't call color think as far as I can tell which told me it didn't you say morning and I believe him bbedit actually lets you open the images and movies I think well you go to there's an option in the preferences to do this try this for the copy see if this one works oh did up here we are okay so here's an image in bbedit in the preferences here there's a choice whether you can what you want to display images using QuickTime and even open movies kind of interesting for a text editor but I've heard it doesn't suck so if your applications like bbedit and it's just calling the graphics importer so this is already working for you another apple application but not one that you think of as an imaging application is iTunes itunes actually does call traffic supporters I have it on good authority state they use graphics importers to draw the album artwork so I can just drag my image file in and there we go and be playing in the background really very 80's song should get your g 5 upgrade today okay what if your application already does color fig matching well some of you saying oh my god my application already does this is he broken it and some of you are probably saying what's the problem well you don't apply that color singh image adjustment twice that would be a bad idea because then you'd overcompensate and the photographers would be unhappy again so it's difficult for you might be concerned if you have already have an application the good news is that we did think of this and we do maintain binary compatibility with applications that are already calling graphics and folders and doing the matching themselves as we showed before the way we do this is a little bit tricky the graphics important Owl Eyes well it tells you the profiles that it's going to match to when you ask you what the profile and better than in the image file is so now if it's going to draw the image matched to generic RGB it'll tell you when you are say what's the color thing profile of this image it'll tell you oh it's generic RGB and that means that if you then match again to generate RGB color sink will say that's a no op these are the same image for the same profile and so the image will still only have been matched once it's worth if you have an application that does this it's worth checking that this is the case because if you grabbed that profile ahead of time and store that somewhere else or if you go through some other path that isn't quite this simple then it's possible that there might be a situation in which you did match twice if you get an erroneous double match but you can easily detect that case and work out how to fix it by using those image files which as I said are up on the ADC connect side on the other hand if you just want to tell us Doug do color matching I'll do it instead we have an API to do that that's what we were using to turn off things and pretend we're on Jaguar it's just setting a flag on the graphics importer and since it's just a flag it's safe to do that on all the way back to click x 40 in fact in the Panther feed that you have received automatic color management only happens when you draw images it doesn't happen in the utility api's for getting images as pick handles or picture files or saving them as quicktime images we also have not yet done anything to modify importing still images or sequences of still images as movies if you're a graphics imported developer you need to opt in to get this feature because you need to enable the code that sets that returns a different profile from gate kalafina profile it's not very hard that i'm not going to go into it here if you want to talk to me about it I'll be downstairs in Sacramento which is not up 80 i'll be in sacramento tomorrow morning and so all other people on the ICM team so we are making color management the default because we think that it's a better user experience not just for pro customers but for all of them we're also making a strong effort to preserve binary compatibility because we think that's also pretty important user experience so one more thing from me I want to tell you about a couple of new API is in quick time to help integrate with quartz 2d so we have a new API that's in the Panther seed to create a CG image from an image file and this is great if you don't want to draw that into a CG context this was requested by a bunch of people if you don't know what a CG context is I encourage you to learn go to some court sessions pick up a book one tip though quartz uses a floating point coordinate space so if you want to send to that image somewhere you should be careful to round down the coordinates when you're dividing some number by 2 or else you might draw that image at a half pixel offset and did you get a blurry image so round is your friend floor is your friend explore as a function we've also introduced an API to allow you to provide a CG image as the input to a graphics export operation and so using this you can take the smooth images that you've rendered by quartz and write the amount of Mac Paint files if you like so that's it for me i will have these little reminder cards if you're interested in writing applications that take advantage of the threads a thread multi-threaded abilities please come see me in and i'll give you one of these cards and then you'll remember that you have to send an email to us go to the quicktime labs downstairs it's not very obvious you have to look for the word sacramento and then you have to know that that's where you have to go but there's a whole bunch of QuickTime engineers there it's great great opportunity to meet the team and ask for help with your tricky questions and I'd like to introduce John Michell bill to he's going to talk to you about another interesting feature that's up on the board hi so let's switch the toilet from topeka right now from all this place FC color think and what you do support in a quick and help enter and let me talk about audio video started change before I do that let's take a look at the concrete time everything model today today we have movie time which is driven by time basis and those time basis are driven by clogs basically time basis as a definition of rate they can stop the time that it can move the time forward or back for all whether the clerk provide real-time clock information which mean that the assign is always moving so this clogs can be provided by external hardware device today such as audio or video if your movie is not attached to any of these devices to examine some we will use cpu clock but the fact that we time is capable of using this external clock to drive the time length of the movie let us play audio and video within think of a long period of time the issue we have today is that this model make the assumption that the rate of the time base can change right away and by making this assumption what kind of trouble can we run into well if you take the example of video Clerk which is lower there assuming this video clerk is currently driving the time base of your movie when your application as good chance to start this movie basically internally we're going to turn around and change the rate of the standard and assume is going to change right now but actually this external device cloth we really would like to see the rain changing not right now but on the edge of the video thing because it will be totally incapable of outputting the first sample before this age well if you look at this growing the arrow is pointing and when the movie time is already started changing so the audio track has already seen the time moving into the second arrow when the video one will be talking not possible for it to display any sample before the third arrow so by making this assumption that the rice can change we interview to get a tiny gap between audio and video devices today and we're not talking about second or minutes in this gap this is what a couple of minutes ago today but when you are in the high-end audio and video space then we don't care about what the scale of younger trees they want to see any capital so what do we need to do to fix this issue and get rid of this gap well quick 10 need to be able to understand this club constraints and as soon as we understand this constraint will be able to provide some Headroom before the right of the movie change so do we make that happen in center well we've introduced two new ID is on the clock component side the first one is clogged yet time for a change and basically this API we used to ask the clock what time is it and say what will change the time now what we're going to do we're going to say well we are about to change the rate from this value to this new one tell us what's the best time to change this value the other API on the clock component side as well is club get red change constraint which basically let quicktime knows that the clock does that such a constraint there is also intend on this new time the API could get time or change status so this this one you'll be able to call it after you change the rate of the movie in global you will be able to tell when the time will restart moving there is also one you bitch which has been added to an existing API call get time-based statutes it's called time base rate changing that we were telling you that where you are winning this gap I was just talking about so what does that mean for your application well if you use high level movie toolbox API you don't channel that is all that you need to be aware that the time this red can be done the room by the time is not moving and actually the slide is talking about clutch and this me out of sync with my slightly this should be real time babe if you use low level time this API directly outside of the movie context you might have more work to do you probably need to check for this new flag return by to get time date status because it could generate some using behavior you never seen before so the last epi added intenta is a movie to box AP angkor get movie right change constraints so before you start a movie can query the minimum and maximum delay that you will see when you were to change the right of your movie so let's switch to the demo machine to show you what this stuff is about so of course i am trying to demo something which is showing a millisecond delay when you start a movie so that's not really something easy so in order to have you reimbursed and what's going on i modified the building audio clock on this machine just because i can and i advertised the constraint of 27 rather than realistic constraint which is about a couple of minutes agone in real hardware devices so you'll be able to see what I'm talking about when the movie when the movie race is changing the other thing i have added to this stone on quicktime player either custom panel which is shown you a couple of thing the first one is the current rate of the movie and the two other permits are basically are using the the new Europeans was talking about the first one is asking the movie to box with the constraining when your birth to stop and this is the to second I was talking about building to this sake audio device clutch and the other one will basically money tell the time changing after ice change the rate of the movie so the player has not been modified from all that has been bona fide in this demo is the clock component which is used and because this movie contain both another one video track by deform Christian will pick up the the gog o clock to tick the time of this movie so when I'm about to play this movie you'll see that the movie controller believe that the race has changed while we're going to see that the rate actually will change in in the future so if i do that nothing is happening and you see the slider going down and let me do that again so you can understand what's going on everybody believed the movie had started and the clock is driving the 10 days of the movie so even if the constraint is applied on the audio track actually the video is going to wait patiently for this time to start playing to show that Apple expert has over the cell yeah idea so to show you that it's really coming from the from the audio clock what I can do I can extract the video track from the original movie and if I just show you the panel the everything tunnel again will see that there is no more constraint because quicktime is not using the built-in audio clock anymore and if I just play this movie it does behave at any movie today which mean that the video can starboard away and there is no nothing happening that the rate change instantaneously if I strike just the audio track of the original movie then one more time because quicktime is using the building of your clothes it's going to wait before the stuff happen are you now or have you ever used with you go with them I just experiment I have thought but all i would have loved for now and that's all other times i am as well to talk about the services of secant enter thank you so ye mo e mo is not here Jia with hearing coming up to rapid decision okay so now for the ending of this session we're going to talk about what is still available for you sessions and we have for tomorrow we have quicktime streaming server programming and we have a quicktime alternative programming environments on friday but the thing that when i highlight is the feedback forum tomorrow is a 335 remember correctly and the important thing here is that it is scheduled to be in haight-ashbury and we have moved it to marina because we think that we need more room for all the QuickTime people who are at the conference so high actually was going to be just too small for we expect you all to be there in this is enough to film and I actually already so please take note of that we will be modifying all the displays around the conference I'm sorry that's an excellent time actually to have a conference on Friday but it will be the morning of friday okay does my contact information if some happens to run out of his cards you can write to me and i'll make sure that he gets the information also more information this is all the sample code that we have been talking about here is already available for you to download from the ABC side there is also new quicktime documentation available that you can go and download we have every time we are trying to catch up with the new api's and make that available to you so you as attendees of the conference can go today and downloaded we we'll be making it available to all developers as sponsor goes out to more people at this moment only you have it for documentation you can go and get the new stuff right now more information and thread tester drop draw those are available from the download site already and the quick and LA we have mentioned this many times for API developers this is an important opportunity for you to talk directly with the QuickTime engineering team and have any questions or consultation that you want to make answer we still have the rest of this afternoon and tomorrow and Friday for you