WWDC2004 Session 203

Transcript

Kind: captions Language: en good afternoon welcome to the second day of www DC my name is Derek Clegg and I'm a course engineer I've been working in courts for quite some time now hence my title today I'm going to talk about the new things that we've added to Tiger to help you as a developer do more cool stuff I'm in addition we're going to have my colleague John berkey come up and for the second half to show you more about courts to D and how it's actually leveraging the quartz extreme part of the system to do even more cool stuff and then along the way we're going to have a number of demos so the first thing that you of course need to remember as we've emphasized already in other sessions is that our focus on Mac oh and Mac os10 is quartz 2d it's no longer quick draw cord draw is a great technology service really well for 20 years its crystals going dark it's time to sort of move to courts 2d so we really want you to think toward doing that if you haven't already and of course if you have done that we feel they want to continue supporting you and making more and more stuff part of course duty to help you out there's a special session on friday at two transitioning the court CD which will go into this in more depth and help those of you who haven't already started to transition make the transition now i'm also not going to talk today too much about the details of court city the architecture the basic type tell all that fits together there's a session that i gave last year WWDC that talks a lot about all of that information so you can certainly pick up the DVD and check that out and in addition prompts more importantly this year we've added a lot of really great documentation there's some brand new stuff that's all about drawing the courts 2d the transition of course duty there's a full reference of all the api's and there's a lot more so if you have any issues like that sort of basic understanding we really urge you to go to the documentation it's great and of course check out the DVD last year as well the my focus today is going to be all the new api's we've added and tiger sort of the new things that are coming down the road that we can use that you can expect to start using now just though to give you some context we have a cg text is the basic type in court CD that's what everything uses to do drawing so TG context is sort of the coin of the realm in a certain way it's the it's by nature by its architecture device-independent what that means is that you use a CG context if you're talking to a printer or to the screen or to a bitmap so the same basic type for all of those things so it's sort of abstract the drawing environment for you in addition very importantly it's a resolution independent you zach lee ji context with the same API to draw to a you know 100 DPI screen or a 1600 DPI printer and in addition it's staple it maintain state for you it has what we call the graphic state which is things like the the color what color you're going to fill shape in or whether you're lying is dashed or not things like that and most of the API is in courts take a CG context and work with that so what are the new API is that we're going to be adding for tiger well we're going to be some new drawing api's that work with CG context directly a new thing at layer API which is a sort of a more efficient representation of off-screen bitmap drawing and some new PDF input on outside api's for those of you who are used to using those ok so this is sort of the list of what I'm going to cover for the drawing API is a lot of things so let's just go into each one independently so the first is a new clipping API the idea here very straightforward concept you have a mask of soft mask of some sort and you want to use that as a clipping region in the past and Panther clipping was always sort of path based you could clip to a circle or clip two rectangles or to a much more complex path but it wasn't very easy to do a clip directly to a mask so we've added API in Tiger to let you clip to a mask and it works just like every other clothing region clipping in court it's intersected with a current clipping region a base G saved you restore semantics so in that respect is identical it's just that now you can sort of get the power of a soft mask for your drawing operations in addition to sort of the standard clip half concept that we've already had so we think this will be very useful for certain people who need to be able to do this type of operation we've also added some new paths api's and the idea the sort of the overarching theme of some of the things going to be talking about is make simple things simple course is a very very powerful API it has lots of things available in it but the problem is that sometimes a little bit the water gets deep really fast it's hard to know where to start sometimes we've had people say well I want to draw a circle AB you learn about bebier pads and cubic equations I don't know what to do about that so that's a bad thing we want to make it easy for people to do simple things so the one of the goals for some of the api's you've added to make simple things even simpler so for tiger we're adding api's let you do circles just directly without having to sort of worried about bez a path and so on of course the lip Caesar just a variation of a circle we've also added api's let you do lots of lines strokes very very rapidly and you'll see a very good demo of how that used a little bit later when john comes up to talk but of course we want to keep simple things simple but we also don't want to neglect people who have more advanced needs and so one of the things we've added for have is hit testing so you can find out if a point is inside a path so you can do you know we want to drag oh yay and also those are some of the advanced developer and also we want to be able to allow you to replace the current path with a stroke version of the path again for hit testing or for special shading stuff like that now again it's along the same theme of keeping simple things simple we've added some api's to let you get access to some of the basic fundamental color pink color spaces the in color sink has sort of three fundamental color spaces the generic gray generic RGB generic CMYK and it used to be and Panther sort of difficult to get access to these you'd have to sort of go to color sync and muck around and color saying come back to core graphics muck around in core graphics and after you've done all that mucking around you finally oh look up color space so what we've done is made it very very simple to get access to the color space without having to go through all of that sort of gyration and that way you get you can guarantee that you can easily get a color space that gives you high fidelity color everywhere now that's important because our future direction is high-quality color high fidelity color everywhere across all devices we are no longer trying to model sort of the device idea where you get a different color if you're printing versus a different color versus on your monitor we want to make sure that you get a high quality match color everywhere you go to that end we want you to avoid using device color spaces and to help you with that we're actually going to replace the use of device color spaces automatically for you with the generic RGB genetic gray and so on so that even if you don't care too much about color and a lot of people sort of are willing to sort of let the system handle a lot of that up for them you'll still get the same results on all monitors on all printers without actually having to do anything very difficult yourself now an example of how easy it is though to transition to use the sort of not use device color but instead use some more specific color spaces in the past you might use the function on cg color space create device RGB by its name it's a device color space the results will look different on different output devices that's a bad way don't do that anymore the good way the tiger way is to call CG color space create with name and you pass in the name of the color space in this case generic RGB so it's very simple not a lot of code changes not much difference and what you end up with is a color that will look the same across all platforms so this is definitely a direction we're headed and so we want you to be able to come along with us and that for image api's we've added a number of new ways to do image drawing and work with images and so I'll go into each one of these independently so the first is chroma key masking this is you know some people call this blue screen it's a very straightforward idea you have someone stand in front of a blue screen and here we have two men shaking hands and then when you display it you subtract out the blue color and everything that was blue becomes transparent so this is something that you know everybody sort of is familiar with from TV and so on and so we have what we've added intent and fusing and tiger is a simple way to create an image and specify a key color the color you want to make transparent and end up with an image works just like every other image which has those parts transparent the function of CG image create with masking colors and it's a very easy way to get chromakey masking for your images if you will in addition this is something that falls in the category making simple things simple it's a very obvious idea assistant to the previous example we have a clipping GG image create with mask lets you take an image and apply a mass to it a soft mask any depth that sort currently supported in with image masks and what you get out as a mask image so this is something that you know maybe some people are familiar with from from quick-draw it's definitely something that took a lot more code too much code to do and Panther so for tiger we've added a single function one shot to GG image crate with mask and you get exactly this effect very powerful but very simple as well good reading it for you another another API to help work with images in certain cases particular cases is the idea of a sub image of an image this is a pretty common idea if you maybe have cast some complicated drawing and a single bitmap and perhaps you want to use that different regions of that didn't that four different drawing as you're doing your work and here we imagine an example where someone's drawn a plot maybe it's for their cash and they have you know nine letters and what they want to do is sort of pick out each one independently so there's a now new function cg image create with Imogen wreck you start out with an image and you choose a rectangle and what you end up with is an image that works just like every other imaging course it's just a piece of that book parent image the nice thing is that all these some images sort of share the same parent you're not making copies of all the data you can keep it all in one place so that's a very powerful API for those cases and then finally with images again this is something that we really want to make much easier bitmap context we told people in courts 2d if you want to do a screen drawing if you want to do some sort of caching like that use a bit of context later we'll tell you about something new that's even better than up Conte but this low cases where it's appropriate to use a bit of context and you do your drawing is a bit enough context and we just sort of you know blissfully say oh and then create an image from it and draw with that image and it turns out that's a lot of code it's not so simple and particularly it's not so simple to make sure the color spaces match your destination and all that sort stuff so what we've done for tiger is added one function that does all that work for you cg bitmap context create image you pass in a Content bitmap context and out pops an image from it so it's very simple now to sort of do what we've always just wave our hands about in the past this is a very powerful function but the thing you need to remember is that it is a copy operation now that's not as bad as it sounds I'll tell you why in a second but the point is that it's a copy operation because the CG images must be immutable we expect that an image will be reused or rather excuse me we expect that a CG image will be uploaded to video RAM once and then every time that image is drawn multiple times we don't re-upload it that saves a huge amount of traffic through the bus to the into the video card if your images has changed then we can't do that we have to re-upload every single time that's really inefficient so we are enforcing it's always been true in the past that we are definitely enforcing now the concept that a CG image must be immutable so that's something that's very important for you to check out in your application make sure that you're not doing something where your changes the context of an image because if you start doing that then your hardware acceleration will really suffer now when I say it's a copy operation it's not really a copy it's a copy on write it uses the you know vm magic of mach to do a copy on write so you don't actually really make us a copy instead you have two things that point to the same in memory destination and if anybody writes on one of them then a copy happens but until something is written than nothing bet you don't get a copy you have you supposed to show the same underlying data structures so that's really useful because you can sort of create an image from a bit enough context to draw it somewhere release it and you actually have never made a true copy another thing that we wanted we're adding for images again sort of more of it the advanced category it's what we call deep pixel support this is deep in the sense that it's now floating point pixels for both image content and for bitmap context so we already support in Panther the standard image integer formats you know 12 well actually up to 32 1 to 32 bits per pixel sorry bits per component for Panther we're now letting you specify a 32-bit floating point value and the floating point value is can be you know you have red green blue alpha as your floating point value as you can see Mike a floating point pretty much any support anything that we currently support an integer we also support and floating point same thing for output for bitmap context you can create a floating point bitmap context you might use now excuse me the the advantage of this is that now for image formats that are starting to come online where you're actually bringing in you know more of these higher resolution images we now can support those natively now the other thing that's important to note is that this is not sort of high dynamic range imaging that's a very specialized thing it's used in its sort of more color seeing color science type of thing to sort of figure out how to map your big floating point range just sort of visible that's a problem that's best left to the colors think guys is a special session on that where they talk about that more in depth what we do is something very simple we just take the values between 0 and 1 and that becomes the image anything less than 0 we clamp 20 anything greater than 1 we clamp 21 so we're not actually mucking around with the data ourselves we let you do that or coercing to that other parts of the system that are more capable excusing now another thing that we have had sort of missing in the past is image I 0 input and output of image data as you know some of you we've been able to support as input jpg natively PNG natively in two courts 2d raw image data is sort of a big buffer of red green blue alpha data not encoded in a particular file format for output we haven't had a way to do any image outputs of course we've been able output PDF files which are tremendously rich and complicated but for something like a simple jpg output nothing so for Tiger we want to remedy that and Tiger we're going to add support for jpg PNG gif gifs a 50,000 a bunch of other formats supported by quicktime and other formats as we need of course i'll obviously as well the floating-point types subscribe to see deploying point formats such as openexr and the floating-point version of TIFF for output and the WWDC build we support jpeg and tiff output and we're going to certainly add more of those to come for the final Tiger release there's going to be a session that talks about some of this and combines it with a previous topic of high dynamic range on looks like on Wednesday so if you're interested in more details about that it's only go to that session I'm going to go a little bit into sort of an overview of the api's because we cover them briefly so there are two types there's a way to get images in a way to get images out the wedding at image is in is the cg image source that brings data into the system and some particular file format we decode it for you and draw it so you can bring images obviously files we've got a JPEG file a URL a raw data buffer similar to what we already support just a big wad of JPEG and our tip or whatever that you might have mapped into memory and of course we support the standard 2g data provider which is the way you get information into the courts 2d typically what an image source does is it creates EG images so you create an image source from a file and out pops a teaching image now not all images file formats are just single images some are multiple like jiff files have animations we will have multiple images for that just files may have different images for different resolutions so we support from an image source you can also sort of get all of the images in the image itself in the in the file format the file data itself plus you can get access to the metadata so if your camera were to embed its make in the file then we would be able to return oh this is a canon camera or if it had longitude and latitude we could return that so you can't get access to the meditative very easily as well but the most important point about the cgm is force and what it creates is that this will be the best and fastest and the primary way we're going to be supporting to draw images and tiger that means all parts of the system are going to go through this we're putting lots and lots of emphasis on optimizing making it you can do in all those altivec tricks and so on and so it's very important that this be the primary way we get images drawn and they'll leave the bat best and fastest and so it's worth looking at from that perspective of course the higher-level frameworks will also adopt this and sort of use it indirectly but if you're sort of working at the courts to lee level it's definitely worth looking into for output sort of the opposite basically the same you can output to a file to URL to raw data buffer a memory you have to put a CG data consumer which is a standard way that you can output two things in courts 2d and similarly to the input you when you create an image destination you supply cg images and you can supply metadata and of course your image format supports multiple images you can supply multiple images and write out the results so then you can write out a tiff file from cg or you can write out our ports to d you can write out of jpg and so on so just to give you a quick demo I have to say it fronted the this demo like all types of demos like this is a little bit of trivial because we're supporting a new file format hard to demo that tremendously well but I'll do what I can we have a simple app here and notice that here we have a tip file and oh this is Panther we can't drop it in because CG cork 2d doesn't support TIFF so now we're going to turn on image I oh and now the TIFF file is accepted ray so that's very important the power of image I oh now just as an additional thing this is a simple example of pulling up the metadata from this file in this case it has a bunch of fields things like the height in pixels and so on the file size the color model and so on there's a lot of information like that that will be embedded in and images that you can pull out again the very trivial example of getting the metadata there just to tie it into the previous topic about color spaces notice that this in this image this image doesn't actually have a color space associated with it we're not actually we're not using the closest place but when we start using generic RGB it doesn't change why because in Tiger we're automatically making sure that anything that's untagged gets tagged with generic RGB so you get the same fidelity across all platforms and then just as a simple example of to prove that there really is something going on this is a specialized color space which swaps blue and red and as you can see now you have a very odd maybe sick ladybug ok so that's the demo with that ok so what else have we added let me get some water and I'll tell you ok another thing that's very important and is very powerful is of course the resolution independence in quartz 2d the fact that you know any resolution any output any scale factor looks just the same looks just as good as other scale factors now it's not surprising that of course you know down the road we expect monitors to change in resolution we also know that people who are currently disabled use resolution to scale factors to do too sumup their UI so they get some better ability to look at the content and so on and the thing that we are very very focused on is making sure that in all those cases it still looks great that your app still looks just as beautiful it does today on the current monitors that's an apparent very important thing now that means that you need to actually sort of play with the idea that your application may be scaled may need to zoom up and so on what that means that have been at a technical level is that will set up the context for you the ctm will be set up so that the scale factor will be built in to the context so you don't actually have to worry about that too much but that means you do have to worry about it in that you can't undo that if you undo it and something you're not drawing to the right scale factor you're drawing at some other scale factor and everything's going to look like you know bad things on the screen what that means is that we don't want you to sort of undo the transformation by this is a little bit technical but we're taking the ctm and inverting and caddington counting it back to pretend like you're an identity don't do that if you understand what I'm saying if you don't good and instead we really think that's important to consistently ug seiji restore that's a standard way to save the graphic state restore the graphic state we've spent a lot of effort in tiger optimizing the G saves you mr. operation so it's very cheap and so we think that that's the right way to do and it sort of will save you once we do get down to the road where we are still going to start scaling the applications and now all that said there are still some cases where you really do need to know what pixels am i touching you want to draw a thin line for example you don't want it to straddle a pixel boundary or you need something do exactly a blood against something else those things do happen where you sort of need that and help you with those cases when you may have a scaled-up user interface we have sort of a big workers function cg contacts get user space through the divide space transform that is the full transform from the current user space that you're drawing in all the way down to the pickles to the actual pixels that you're working with the device space now often you don't need something so heavyweight so we have some convenient functions a function that will take a point in userspace convert it to divide space two pixels take that point in pixel space to put it back to user space and we have similar functions for size and rectangles as an example a simple example of how you might use that here we have somebody who wants to do some sort of specialized rounding in device place where they really want to round to the middle of the pixel or something like that so they have a point in user space they call CG context convert point to the x space that converts that point through the current transformation matrix all the way down to pixels they do their alignment the way they want and then they take that point convert it back to user space and now when they draw with it they know that what will come out will be precisely aligned so for those special cases worth important this is a very powerful API and general though we think that you'll be just fine using the standard courts to do things for scalable you I but cases where it's necessary this will help you out now let me give you a quick demo of scalable UI and how it works with apps so we may have seen this yesterday in Peters talk so we have many of you are familiar with quartz debug very powerful tool for development and we have now a user interface resolution slider so you can change the slider and you'll get a different resolution for your application so for example here we might let's say we'll go up to two times and let's launch Safari they see so far is twice as big now you can do this with your app to you can take your app change this slider and see what happens when you show up when you bring it up on screen it should have brought up apple.com maybe they're announcing something else here knows but the the point here is that notice how Safari looks really good it's still as high-quality as it was when it was at a lower resolution the text looks great all the drawing and someone looks great so that's this is something that will help you out to make sure that your application doesn't have any problems with rendering as you can see there are some issues like up in the upper right-hand corner clearly I've changed the resolution but the nobody told the clock or the sound bar and I would show you some other apps where it doesn't look this good at all but that would be embarrassing so I won't so make sure your app is not one of those let's let's see Safari is not quitting I'll put it here and the other thing to remember make sure that you quit the ok we can push back make sure you quit that app otherwise everything will launch too big and that's difficult to read okay so that's sort of a lot of the things we've added so far for for drawing for new ways to do drawing you better ways to work with CG context little ways network with images and paths and so on so now we're going to talk about a layer which is a new thing we're adding for tiger to help out it's really designed to sort of help optimize the traditional case of where you want to do off-screen rendering but you still want it to be hardware accelerated and the past what we've done is said oh if you want to do some sort of off-screen rendering just create a bit of I'm context draw into that and then take the result and composite to your destination that's a good way to do caching and it's very powerful and very straightforward thing to do the one problem though is that the data always lives on your side of the of the bus essentially right it's all sort of in your address space so any time out drawing you do there which you then try to draw on to say the video card will have to be uploaded that's not efficient but and by design by the bitmap kind of design it's sort of limited to that so digi layers are sort of an optimized version of sort of a better way to work with sort of do this type of you know bitmap context type caching now the the key thing too is that in addition to just sort of being a convenient way to do to sort of do off-screen rendering will when you create it you specify a reference to another context and what we'll do is we'll set up set up the layer that you that we create for you so that sort of matches the the destination context for example it will match the color space so that means you don't really need to know about the color spaces of the context you're working with for example of screen color space and so on instead we'll sort of create that set that up for you so when you draw into the lair what will happen is that it will already be matched to that color space so when it's actually finally used you don't have to a specialist separate match so it makes sure that things stay highly efficient and you can sort of think of it it's really in many ways sort of like you know as this example shows just like a rubber stamp on some sense you can take the layer you draw into it and then you can sort of use that over and over and over here sample we created a layer or drawn the butterfly into it and now we can just use that and what's great is if the butterfly sorry if the layer is hardware-accelerated so that you it's already up on the card when that stamp happens as all none of that comes back over to the CPU and so those all done in the GPU so that's very very efficient and very high-powered for those types of cases where you need to do sort of off-screen caching and you don't want to have things sort of live on your side of the the the bus does it work so like I mentioned we sort of imagine that one common use probably the majority will be cashing so you would use a layer instead of a bitmap context for off-screen caching another one which may be a little bit less likely is sort of a buffering where you might be in the middle of the screen update on this side so you can sort of go ahead and start drawing into a layer so that the next time you get a chance to composite to the destination you you you're ready to go that might be a maybe a little bit less common use so the short demo will go back to my little app so here we have our app again and here's a PDF file or a nice PDF file but it's got a lot of data and it's a slow thing to render now if I wanted to animate that I would be fired because this is not really animation this is sort of chunky blob eNOS so instead of course obviously what you want to do in Panther we would have said oh well to solve this problem create a bitmap context very good solves the problem for Panther the problem though is of course that's not hardware accelerated for tiger you can create a layer and what this does is will draw the PDF file into the layer and then the layer is now hardware accelerated so what we do animation it's going to be completely smooth it's all however accelerated everything looks beautiful now the thing to remember of course is that it's still in some sense bits so if you think about it it's not the original high resolution PDF file that if you scale up you'll get you know perfectly beautiful results at any scale factor you will start seeing you know perhaps some sort of image artifact is you if you were to scale up the this particular layer because it is this but in many cases for something like this that doesn't make any difference you don't care about that so much you just simply want to have high-quality fast efficient rendering their Safari still okay so that's that demo so that's a lot of the of the api's for both drawing content for high efficiency off-screen rendering and now we wanted to also added some stuff for PDF support as you know many the PDF is the metadata file format in court and Mac os10 it's our high fidelity rendering of drawn content it's a great format it works really well for us the problem is that sometimes people have trouble sort of both in the creation slide they want to do more things and on the sort of input side where they want to really look and introspect more into the pef itself so that it's an API to help do that so let's go through those the first is links an acre is very comparable to HTML again it works just for the PDF context that is able to record this information and you can do sort of the basic stuff you can do with links and anchors you can specify a URL and say well when somebody clicks on that point in the PDF page 17 I want you to open up this URL in safari so you can do that and then of course the anchor type model where you can say well if I click here on page 47 go back to page eight I'll go forward to page 75 I think you can sort of do forward and backward references and it's very simple API it's very easy to use but it gives you a lot of power to sort of preserve that and of course the far i will be using this and tiger to preserve links and anchors when they actually go to print from safari in addition for creation a lot of times people have wanted to create encrypted PDF this follows the PDF specification put a published by Adobe works with a PDF context and at creation time you specify a password or more than one password and the permissions associated with the passwords so you can sort of say well for this password don't let them print the file or don't let them copy anything from the file for the WWDC build we support 40-bit encryption and of course we expect to extend that up to i think 128 which is part of the people to standard in PDF for tiger okay so that sort of output so new things you do without but for input the what we've added for Panther was something really powerful it lets you sort of introspect about the document structure a PDF file anything about it was a giant tree of of objects and you can use the API is that available in Panther to sort of walk through that tree and look at all the content the one problem is that the real meat of the file the actual content stream for the page would said you know draw the circle here and put this text here and draw this circle here and put this text over here it was just sort of the big black box you could look at the content you could print it out but you couldn't actually interpret all the pieces of it unless you wrote a park to yourself a little bit left some I don't know a little bit less attractive for most people and in fact the it's not something you can just sort of read either and sort of figure out exactly what it what it element so for Tiger what we've added is a new way to take the content stream itself and parse through it and call your functions back for each operator your is interested in so that way you can really take for this page look at all the pieces of the PDF file and get essentially sort of blow up in the whole file so you can look at every bit in the file itself this good glad you like it this is it's very easy to use the idea here is that that for each operator you're interested in you supply a callback function you don't you can do it for none you can do it for all of them and we'll just run through it through the content stream itself and call you back here's an example again a very simple example the idea here is you create an operator able where you specify both the callback function and the operator you're interested in in this case we're imagining that someone wants to look at the BMC operator the begin marked content operator well that's why they just want to do that you create the content stream of course from the page that's that's new as well where you get the content stream out of the page and create a CG PDF scanner that takes a content stream that you're interested in the operator table with your callbacks and an optional info parameter and you call CG PDF scanner scan that will parse through all the PDF do all the mucky work inside of parsing and so on and call back your function so each time it sees a BMC operator called you or the arguments on the stack for the BMC off for the sort of the BMC op that you're looking at you can do what you might want to do maybe you're counting the number that times they appear so like that when that function returns in CG PDF scanner scan returns the full context cream has been parsed so now you're done you release the scanner you clean up after yourself and you now have the information you want so it's actually very simple used and require a lot of work and you can do really powerful things with it for example this is what we use inside of courts GD to do both all of our PDF rendering and the text extraction zone that's part of previews so we use this in two very different ways but again it's very powerful for with just very simple API in addition for an addition for Panther we start excusing for tiger what am I saying we we now have a new thing called PDF kit the special session devoted to it is a little bit higher level it's sort of a way to it's actually what preview uses now to sort of do all of its PDF rendering and it lets you sort of do things like you know do to up drawing to sort of one up drawing do text selection so on all in your own application so it's sort of a cover for a lot of complicated functions the preview used to only be able to do but now you'll be able to do with PDF kit so if that is something you want to do if you want to bring PDF into your own application a lot of people do select text selection searching all that type of thing within your own application and it's definitely worth attending the session right after this session in a different room okay so this is a lot of stuff that we've added for tiger we've added some both easy to use and some advanced pass AP is of course some new colors may see color space api's lots of image support various ways to work with images floating point images of course as well a floating point bitmap context a better way to get an image data in and out of quartz 2d new layer support for a high-efficiency high quality off-screen rendering and blue and and also a bunch of functions for PDF input and output so that's sort of the new API is in Tiger the new things you can work with and now John Burke is going to come up here and talk about courts to the going extreme and what what that means for you hi everybody I'm John berkey i'm here to talk to you about courts Chiddy and quartz extreme so before I start I want to sort of give an idea what we would be talking about I want to talk about the architecture enough that you get an idea of what we're doing so you sort of know what to watch for when you're doing your apps and then we're going to do some demos so you'll see that it's real and a lot of interesting different ways and then finally we'll be talking about rules of the road we're ending with that because there's a lot of very specific things we need you to do so that your apps really really Rock so let's get to it so of course extreme for Tiger the big change is that 2d is now hardware accelerated the thing for us that was important was that we need to maintain courses very very high quality but we wanted to speed things up a lot we use the GP when we can we use the CPU when we need to so the big thing that we did to start with before we began this project was we analyze a lot of applications what we did was we made sure that the things that the applications do we worked on first and what you'll find is you get your apps going is that basically all the things that your typical application does are all in hardware and on GPU so you'll see a lot of speed and that's where it will be calling out as we talk about specific operation and then the less common operations things like stroke lines with dashes things like that that are less common will be accelerating later if at all because those things will be coming up very very rarely and importantly why say celebrated that I'm talking about on hardware the other thing we're doing though which items really exciting is you'll find that software performance is actually quite a bit higher and tiger as well and the reason for that is that as we've optimized the pipeline for hardware we found a lot of opportunities to continue to optimize the software pipeline to so I'm very excited about that and wheres noting about that all the numbers you'll have on screen here will be tiger software versus hardware you'll find that even we saw four numbers as I'm saying they're quite a bit higher so so here's our course extreme architectural diagram and I want to just point out a couple things just to sort of set the idea of what we're talking about we remember that the three big things we do in the course extreme engine are we deliver a few d.video and 3d to the screen we use the GPU and then we see this word surfaces so we use Hardware surfaces to deliver video and 3d so focusing in on 2d there's just a few main concepts so first of all the application as you know renders into the window we call the backing store the bitmap for the window and then it says flush so when we say flush what we mean is we talked to quartz extreme the course compositor and it flushes the content to this to the screen so what that does is it actually talks to OpenGL and OpenGL execute the flesh in behalf of quartz extreme so focusing in men this is the key point here is that course 2d up until Tiger was using software was running with the CPU so we were still very limited by the CPUs performance our CPUs are really really great today we all know but still compared to the GPUs are more limited then the great thing of our courts extreme was that we use the DMA engine on the graphics cards to do an asynchronous VMA pull of the bits across to the GPU and that's been great because we get a synchronous behavior and the GPU is really good at this so we get a lot of speed as you know it's quartz extreme so that was awesome but there was more to do the first thing that Peter alluded to in his presentation is that GPUs have become this other animal GPUs are now these amazing things that can do these core image effects no blurs all these different things and 2d has been isolated from that part of the capability because we've been using it as a blitar to get our content to the video screen the other thing is is that window back towards are very big so even if the CPU was capable of writing those pixels into this window backing store very very quickly we're still to shovel those bits across the bus to the screen so there's still a lot of work to be done and then last thing is that the window actually is a synchronization bottleneck because while the CPU is shoveling the bits into the window remember where DMA flushing from that same bucket to the screen and so so we don't have tearing we they have to wait for each other that's what we talked about we talked about over flushing so we'll see that in each of these cases we have solutions so as I've been saying it's all about bison BAM list we're software rendering here and this is a great number the g5 really really rock but it's 5 gigabytes per second and as we know there are other things for the CPU to do on the system so and also i'm using these big blocks because we're talking about a lot of data window back stores are huge and we using to talk to render is a lot to do here we are with the hardware DMA flushing it's 2.1 gigabytes per second but still we're shoveling big black says he knows a lot to do here and finally the bottleneck in the window vacuum store here's the key point look at that 30 gigabytes per second and climbing the charts are all you know point in the same direction here you've seen in previous presentations so this is our key we want to focus on that that's our deliverer so here's the punch line then this is our plan right we used want to stop using the cpu and we want to move towards using the GPU GPU as peter showed in this presentation has a lot of pipes it's very very wide it's very good at this and importantly by using the two together we can keep the GPU as busy as possible the GPU is a single purpose piece of equipment for doing graphics so by keeping the CP is that pipeline we can keep it really busy so what does this mean it means the backing store is no longer in DRAM so where does it go goes to the graphics chip OpenGL we work with OpenGL to deliver things to the hardware and this gets pretty interesting here so there we go I'm not going to clicking OpenGL commands are very small hence the small purple boxes that's the key point here for this part is that we're delivering a lot less fights across the bus and this is a very optimal stream the OpenGL guys have worked very hard as you know our gaming engines today deliver millions of polygons so what we're talking about here we use in a very tight stream and then uploads this is important we'll be focusing on issues this issue a lot as we go forward in the presentation uploads are very infrequent and we will be working together to ensure that they're much smaller than window backing stores very key point right the aqua artwork and some would fill a quarter of a screen but we fill the whole vacuum store with it so that's the point as we upload the artwork cross and reuse it we have less to do there then finally this is great back to the flushing bottleneck with the window backing store on the GPU the flushes are occurring all on hardware running to hardware with the small commands and then all the flushes are on hardware so that whole flushing biomech goes poof so that's really really great as the key to our performance so how does it look like looks pretty good it's pretty amazing to me I work with this stuff every day but I'm always astounded by how much performance you can get from the system what's amazing is this is still CPU limited as fast as I can shovel stuff with the present day she'd be used i can still travel faster in the future and what you'll see then as you test it yourselves as you'll see that performance will scale more in hardware as you use your cpu gets faster so we have numbers where the Lions performance scaled between my powerbook and the g5 number that's here is scalable like three or 4x when I went to the g5 and only scale by 2x this sort of the CPU normal number so I think it's really interesting I want to call it a couple things this is about still right here the point here is is that with the big rectangle we're not limited by so rate still so we're really really fast that's important because this is what we do first and we render a window you know we fill it with stuff so that's important text rendering is very important there's a lot of discussion on the net about that and I just want to call it out we're really working hard on this we actually spend a lot of man might sign making this great and we're basically almost breaking 5 million here so that's really great then Lions this is something we'll be talking about today in detail we have a new API to really make that easier for you to rather live a lot of line performance on your machine so sort of defocusing here stepping back one step here's PDF rendering which is a user of the core primitives and what you see here is is that there's two documents here in two different ways of running the same documents on the top of the screen is a core graphics benchmark and we're getting above 2 x 2.2 x with that we're getting the same values below they're about twice as fast a little bit less but i'm quite a bit slower when it's in preview that's okay because preview is an application it has to do an patience and stuff you know some of us are speed freaks but there's a lot of really great value that's delivered by the higher and apps so it's not always about that also wanted to point out that on the run remember so anyway application resizing this is a very interesting slide for those who are focusing in on the numbers so see the red this is something that we're using as a reminder for you when we go into the rules of road section so we have lots of speed here but it's sort of burying and sometimes the blue line is the hard work number is actually slower so this is a thing if some of these applications aren't cashing their image rest still we all work together at Apple but we're all you know still pitching in we're not shipping Tiger yet so we thought it was great to show you some of these cases where our apps are slower to and to call this out and I'll have a demo to explain this to so again you see the number on the bottom text edit is crazy fasted its frames per second they just have to turn off being sink and your monitors to even take this measurement because it's just a big flickery craziness I'll show you that in a minute spree neat so the key is as we move forward you'll start to experience more and more of this performance because as I showed you the benchmarks the poor primitives you're actually using or are fast so this is great I think this means we can deliver and this is my experience in bringing it apps up and looking around the system and what their wild nekton we should experience 2x across the board with GUI performance for tiger so uh yeah kind of demo machine not that one okay so for the first one I'm just going to show you a little finder performance that's pretty fast it looks good and then here's a little make sure yeah good there's a little even faster so that's all good there's a lot of light little demos just starting to give you a flavor for what we're talking about your Safari so it's getting pretty fast I'll show you here with it off you'll see that it's still fast it's a little slower it's about this number is about 1.5 x and so we're pretty happy with that for where we are here's another who's the one of the flickery crazy ones and they make sure I'm set here so that number you can actually some of the guys in the team taught me just you can measure by the lot of terrorists you see what the frame rate is but it's looking at around 200 here that's really great this is an interesting one okay actually I do that right okay so here's the Java app which is cool because they just told me they were doing this little demo and so we grabbed it to see what it looked like it's kind of cute what it does the tries to render as many balls and keep the frame rate at 30 so you can see with hardware with this great g5 we can do 430 two balls per second it's a new benchmark number balls per second great so but now when we turn on hardware this is actually with no pain co changes from them they're still I'm working with them to crank up their pipe a little bit more I still got beam stink off so that's why this will tarry but anyway we'll keep going so it should flow it up there about 3 X 2 what's offer was that's really great is there quite there's a quite a deep pipeline here to get the code from job all the way down that's pretty sweet so here's our friend the wireframe this is an important demo to us because this is something Joseph Mars been using for quite a few years to talk about performance in quick draw and Mac OS 9 and part of our message to you is we take this very seriously when you tell us that you need better performance in these areas so you can switch over to court so this is a big deal for us so you can see already with course software performance things are looking great 1.3 million on a g5 at me I'm amazing and then you know hardware is crazy and what's really neat about this for you GL people like me this is immediate mode so I'm actually using the slowest version of OpenGL primitive rendering here so when we switch to things like vertex buffer objects people learning about this week we'll be able to crank this number up even higher and it's already in the stratosphere so that's pretty great the cards are very very fast ok so this is sort of back to kind of pseudo this is going to get us into the rules of the road so pay attention to this one in sort of a different way make sure I do this right yeah ok so software hardware really really great that numbers like 20 X and then now so what about this this is strange behavior this is hardware with no caching look at that frame rate whoo so this is our first example of how we need to be compliant with our API the good news is there's not a new message here just we actually need you to be compliant now to your app rocks so here's a scaling performance g5s do great that's three times however my powerbook that's nice so here's hardware whoo so all right back to slides OOP where's the clicker okay so caching this is the important thing this is sort of a typical command stream you just saw go by contain some commands and contain some resources to be uploaded and we're just showing the cute little picture of a resource being cached so now we light it up because we're going to reuse the resource and see it was all little blocks that's the technical term and what we want to focus on is making lots of little box so we want our resources to be cash we want to reuse our resources and here's another one so you know the point is is this is the kind of behavior you'll see a lot of the things you'll be doing there'll be a nice clean command pipe and then the idea is that rarely you'll be also doing resource uploads so using be rammed of room this is actually I think a nice simple story for you everything is automatic all you need to do is use the retain release semantics that same core foundation and if you do that what you're doing is volunteering to be a part of our caching strategy importantly what you're not doing is volunteering to have all of your images put on the card all the time well we'll do for you behind the scenes is dynamically take care of that to make sure that when i put putting too much up there or too little so that takes a way that worried that you have a Vaux I got to retain just the right amount so that my app doesn't freak out what you need to do is just retain all your image rest all your pattern rights law your color dress all the things you're using and again that's how we know because we can see that and we can see the hardware you want and what we'll do is take care of the rest so in a typical picture whose finder we have images and patterns text Lions rectangle these are the primitives that I gave you the numbers on in typical case that will be everything on the screen will take care of everything there refer that as a visual working set the idea here is that you know you have this set of resource you'll be using and those will sort of stay cash for you so rules the road this is the big one cash your rest this isn't exactly how it would look for you but you get the point right here in the first top part we have a loop and every time through the loop we do the cheesy thing bad thing which is make an image ref render it and then release it don't do that instead some port in your code where you're loading your images whatever you doing create your images hold on to them that's your cash your cash is to hold rest your cash is not the whole bit if you want to hold bit that's fine just pull the rest with them and then as many times you want to render the image render the image just don't do what you did up above there so layers great new strategies their hard work silver to bitmap and bitmap context they're very easy to use the key point is again they're dynamic will make as many of them Hardware cell rate as we can based on the card you have what you're asking us to do otherwise and they're really easy to use I really like how the guys did this because they're actually easier to use them the old bitmap context anyway so these are great importantly right bitmap context always have to go across the bus and layers will be in hard or whenever possible so here's one of our new performance AP is the key point here is you know you see the top here good and bad this looks like nice clean code just that by iterating a loop like that and giving us one line at a time we have to create a lot of temporary storage to handle anything you might give us with the second one here you tell us how big your storage is and you give it to us all at once so in actually actuality with the hard to render work it basically doesn't get touched until we're ready to talk to GL that's very powerful here Rex case this API is existed for a while but we're encouraging to use it more same thing right storage all at once tell us the number it travels all the way through the system don't catch your caches this is another thing that we found in working with the operating system people will find this a lot of caches around for different reasons rendering has changed a lot in last few years so some things are faster than they used to be this is the key point check your caches we've all got them and try turning them off and see if they still help it might not anymore so that's what I'm talking about here also very important this is an awesome session the shark technology we have is where the best technologies has come out of Apple and less five years this might sound silly but really really really measure your app before you think you know what it's doing it will look very different with hardware acceleration number one and number two as you know the scale of applications has really grown West ten years so we need to measure our stuff and then simplify the scenario that you're measuring until you're sure you understand it and you know what to do that where you're working on the right stuff so this one is important to me I use quick draw for a long part of my career and I really appreciated Joseph Mara and Mike Frankovitch working hard to make this run great on 10 but here's the deal quick draw it's as we're saying here too fragile and undocumented it's just got a lot of funny stuff about it from the past if now's the time we're all have to say goodbye this is the key point for us we're faced with this so if quick draw is rendered in a window we're going to turn off hardware acceleration so that's how or what we're ended up with so more good news it's really easy to turn on you've seen a bunch of demos here you have the stuff in your computer kamandi it's right here and all you see that it's listed there also importantly only some of our products support quartz extreme with 2d these are the one the key of this fragment program stuff we have a very modern new graphics pipeline and we're taking advantage of these new features so we actually do need them so final thoughts first core imaging it's a great new technology it works with us very well we supporters a real image this is the link was one of those top secret sessions so I wanted to make sure you saw where it was when it was actually very soon now this is really great though you can make an image using their api's and well it'll be clicked with our clipping and everything so it's really great it's right in and then here's the rest so the whole system is now hardware accelerated and we really this enables us to do a lot of really amazing things so this would be a lot of great things in the future here's our mailing list we love discussion and we've got documentation from a quartz game amazing so we have two different documents a lot of people working with Artemis and this is where you can find them they're actually on your computers and like to bring up Travis she's going to lead QA