WWDC2003 Session 211

Transcript

Kind: captions Language: en bdc so kicking the graphics track off this morning with session to 11 which is introduction to court services and the primary focus of the session is going to be the non drawing api's that are part of the courts architecture these api's do things like manage the displays even facilitate doing certain common operations the developers need to do such as what we call screen scraping are pulling pixels back out of the frame buffer and the key thing is that an important part of the session is that these api is that we're going to be talking about in the session they're essentially the the API is that lie underneath other API is to do display management that you might be familiar with like a QuickTime api's draw sprocket api's display manager eight guys and we really when the reasons we put the session together is because we really want developers to migrate to the display services API is because that's really where all the work actually happens and you're getting closer to the native API and getting more functionality and doing it that way so it's my pleasure invite Mike Poquette graphics imaging engineer to the stage to take you to the presentation good morning I'm Mike talk yet I'm the person in the torch team who takes care of all the non drawing parts of quartz I'm guys working way down there in the bowels of the system fixing the plumbing wrapping in a little duct tape here in there as your requests come in implementing all those nice features you need and i'll be going over quartz services now quartz surfaces what can port services do for you we're the non drawing part of quartz yeah there's there's code that doesn't draw in there it's lots nice we handle all the display management stuff we give you the functionality you need to take over a display to do a presentation we let you reconfigure the displays we can let you position displays on your desktop we can let you adjust the resolution of the displays change the refresh rate do all those fun little tweaks the display preferences panel is an example of someone who uses our services we also handle some special / tasks we give you the basics you need to read the frame buffer contents off that screen scraping thing we also have support for doing remote control as a computer programs like apple remote desktop and Timbuktu can take advantage of this stuff now most apps don't need cord services this low-level stuff most apps will run just fine and don't even need to know this stuff exists where does this fit in in court where the display management code where your screen scraping support and we're event injection event injections a fancy way of saying take an event and stick it into the event chain and a little dribble on down to all the applications we're going to give you full screen application support information here and display configuration information and we'll go over a few the more obscure api's that are in there so I'm just help you out with backward compatibility and some to help you do some neat things that you might not have been able to do very easily before we're also going to cover the remote operation services that's the screen scraping part I've got some sample code for you will walk you through a little bit of tricks and techniques there and show you how to post some events for remote control purposes and also give you guys who were writing user space device drivers some tips on how to make that work right now we've got some definitions we use some funny terms in court sometimes display ID that's not well it's probably something identify the display I figured that much out okay just lay IDs are numbers that go along with displays there 32-bit 'ish and of things there's a typedef in there for them the numbers stay the same for any specific exact hardware configuration even across reboots any change changes in the horror configuration you unplug a display and replace it with a different model you move the display to a different connector on the back of the machine you're going to get a different display ID now all these displays that we hook up to the machine you'll query in court services using that display ID that's how you identify them to courts and that's how courts tells you about the displays display States we have a bunch of different states monitors can be in a bunch of states the frame buffers can be in we combine all that to get there just refer to this as a display state states include active and active display is one you can draw on that's pretty straightforward it's there and it's nice and drawable we can have two slaves to go to sleep you're familiar with that goes blank one of the Energy Star mode usually kicks in in a monitor gets all nice and quiet don't draw on a sleep displays now again if you're an ordinary application you don't need to worry about that it's all covered for you these things only apply the only time you really need to be concerned about these is if you're writing something that's drawing directly to the screen itself through one of our fancy api's we're gonna get into here mirroring Oh mirroring has changed so much over the years we've introduced things like hardware mirroring and we've got software mirroring we have hardware generated display mats and also to other wacky things I'll be getting into all those different states in detail and giving you some tips on how to deal with that we have online displays an online display is pretty much a monitor plugged into the computer it may be awake or asleep and may be mirrored around near but it's connected off wine well that's the connector on the back of your computer with no monitor there it's just offline all alone a placeholder in the system don't try to talk to the system about offline monitors it that they aren't there you'll see all through our programming interfaces this CG prefix CG is of course as is intuitively obvious the most casual observer the prefix for courts now the display services part of the system is going to cover display enumeration it's going to tell you that is what are all the displays get me list please it's going to cover display configuration place this display here change this one's resolution change the refresh rate change the bit depth we're going to cover palette or display support now there is something you haven't heard about in a while pallets so 70s and we got some new goodies for Panther our displays the API now competitive technologies by competitive I don't mean that those something those people up north of building what I mean is other technologies inside Mac os10 that do a lot of the same functions that you see in this code there's the sprockets tough draw sprocket has a lot of display management goodies in it quicktime has a few obscure APA's in the corners for handling displays as does the AGL apple graphics library package and of course there's our old buddy display manager now these api is our president Mac os10 oh why are they in Mac OS 10 oh so you could bring your apps forward to assist you in porting so they're so CFM test based apps that you want to run on nine and ten can run but you can use cg direct display these new API is instead those older functions on Mac OS 10 are all built on top of the CT direct display API so you don't really need to put up with those things if they don't do happy what you want if you cut right down to the CT director slaap eyes you can get some finer control over the system you might be able to make it do just what you want I'll give you some tips on how to do that clients of these display services games we like games full screen applications like keynote configuration tools display preferences your own apps new functionality that we have mirroring I told you about mirroring before right mirroring software mirroring is something that pretty much everyone who used to we all remember writing those good old loops run through the G device list find those displays figure out which ones intercept our window and draw on all of them over and over and over again well you know you don't actually have to do that in the carbon or cocoa AB that's covered for you the courts compositor does that work but if you are talking directly to the displays yourself you still get to do that which might be one good reason to think about sticking with windows systems and not bothering to go right to the display something to keep in mind if we're doing software mirroring software mirroring will happen on all these machines with different graphics cards monitor on each graphics card or if displays are in different depths software mirroring you have to go through if you are talking right to the display and draw on each display in turn going down the line hardware mirroring is something that's shown up just in the past few years hardware mirroring happens when all those displays are running at the same depth and they're attached to the same controller you'll see Hardware mirroring when you connect up external monitors to the newer I books to the shiny power books to those funny new graphics cards that support two monitors hardware mirroring one display is the active display the primary display keyword the other displays are mirrored and hardware you don't need to draw on those so you just draw on the active or the primary display in a hardware marriage set we have more programming interfaces to tell you which ones are the primary which ones are active and which ones are Hardware mirrors now we also had hot plugging in the system I'm going to get into that in some more detail in the world a little while hot plugging turns mirroring on and off automatically in some cases in other cases you get extended desktops in either case if you're going to do full screen access using what we call display capture you need to worry about all these things display capture is something really cool it lets you steal the display away from the windows system it's yours you got it you have lots of control that way but you know what the window system doesn't know about it so anything that normally goes into a window isn't going to work there like views eh I've usns views controls buttons all those fancy high level widgets you know work on capture displays because you can't put a window on that you've just taken over that hardware for your own purposes now when you do take over the hardware like that though there are advantages OpenGL for example can put that display into some very unusual modes of operation that are extremely fast game developers you probably notice the sorority huh so you've got more control but less high level functionality in particular because you're not going through the window system you don't have that view hierarchy and all those other goodies there all those accessibility API eyes that are going to be covered in another session today don't work so that's something to bear in mind captured displays are normally used by games that's a big use of them particularly the OpenGL games like to take over the display and go as fast as you can now there's another way to do full screen access and this will be familiar to lots of folks coming from the mac OS 9 world carbon developers hide menu if you call carbons hide menu function the menu bar disappears the doc goes off the screen it looks big and empty you then create a full screen window and put it up there and you draw through a window now your app still ring in the window system this is used by some other presentation apps such as PowerPoint to do it's full screen thing also used by a number of games this is the way a lot of the games that are based on the Pepsi FM technology that run on both mac OS 9 a mac OS 10 work because this sort of functionality is available on both you are going through the window system to do this you're doing a full screen window that has advantages for accessibility purposes it has disadvantages in that it might not be as fast as for example running opengl and a full screen mode and display capture we've got functions for you cg capture display grab one display take control the display is removed from the window system the app that calls this is forced to the foreground you wouldn't believe how bad it is to leave somebody taking over an entire display in the background so the events go to some other app it's really hard to recover situation so if you capture display we're going to make sure you're the foreground app you'll be pulled as workaround it's your responsibility than to process events and respond now once you capture the display you can draw directly to that screen using GL quartz and other techniques you can get a port to it capture all displays capture all this place is kind of handy this just takes all the displays in a multi display desktop and captures them now by default while capturing the distal a caption all the plays are going to color them all black they take it over capture all displays has another neat thing that might be soon lead some of the draw straw get folks if you've captured the displays you release them to go bring up some UI elements you can do that and then you capture them again it will apply the last captured mode to the displays again so you'll be right back in the last mode of operation you were running full screen now after you're done using a display you might want to release it you can do this in the middle of your program just so that you can go ahead and use those UI elements bring up some control panels for setting up game options that sort of thing cg display release releases one display lets it back into the window system cg release all displayed is really handy this releases all the displays okay that was obvious it also restores all the display modes to the ones they use your last set up and their preferences it puts the gamma table back the way colors sync thinks they should be and you can use it after a display capture or capture all displays one function boom puts everything back the way it was you don't have to keep notes yourself of what the display layout used to be you don't have to go through all its yeah I think this one was over here I think this one might have been up here none of that stuff just call one function boom you're all set drawing to a capture display we have a couple of new functions you repent their see Jesus play get drawing context let's get you a quartz drawing context for a display the context is owned by the framework notice it doesn't say create there so don't release it opengl you can use opengl take a pixel format set up a full screen GL context those see ggl AP is when you're done destroy your context put everything back the way it was opengl is usually going to give you the best overall graphics performance if you've been going any of the GL sessions you've seen you can use it to 2d animations you can use it to bleed images to the screen you can use it to do all sorts of interesting special effects feels pretty cool stuff and even better you can now get a CG context from your opening yell context to get an accelerated context that's been covered in a few sessions here too now some people might need to change their display configuration if you're playing a game and oh you're on one of those nice big cinema displays wow he probably didn't do all your artwork for that 1280 x 900 or whatever that thing is you probably did a lot of artwork Pharaoh 640 by 480 so you got two choices you can do a little bitty square in the middle of the display or you can change the display mode and fill the screen the display configuration API lets you do this it's a transactional model you tell the system I want to begin a display configuration i want to accumulate a series of operations i want to set a display mode i want to set a position i want to set a refresh rate all these goodies and then once you've accumulated this new state that you want all the displays in you apply it with one function call and the system will then go through and do whatever is needed to get there now the system has constraints built into it we don't let you put gaps between your displays you can lose your mouse cursor that way we don't know a while overlapping displays that can look kind of odd be very confusing to a user we don't allow you to make up your own display modes you not see I 42 is a good number i'll use that one I wonder what that does no you can't do that the system is going to reject these sort of things if you're on for instance an ibook that's doing mirroring of displays the hardware doesn't handle not mirroring the displays so if you try to do that the system is going to say no I don't do that and you can't you can't on your so we have these constraints that get applied one other constraints that gets applied as it's going to look at all the displays you're setting the one that shows up at 0-0 for its origin point becomes the main display that's the one that gets the menu bar in the login window if you don't set any displays 200 we get to pick one so we'll just do it ourselves yeah cg began display configuration is how you get the whole process rolling this is going to return to you ACG just lake and cigarette in a pointer this little rest thing is what you're going to pass to all the other configuration functions it's going to be the accumulator for all the state you want to build up and it's going to be what you use to apply the state change you're going to call CG complete display configuration to complete the display configuration or if you've done all this work and decided who this an asshole I don't want to do this anymore you can throw it away with canceled display configuration to set the display origin you use the mysteriously named function cg configure display origin this sets the origin point the upper left corner of that display you can lose them in X you can move them in why I'm Z you can configure a display mode what's a display mode a display mode code wise looks like a CF dictionary ref a display mode describe a particular display resolution a particular refresh rate a particular bit depth and some options display modes come from a couple of different api's that we have we have AP eyes that will let you get a list of all available with bleh modes for particular display that comes back as a CFR a rift full of CF dictionary reps you can look through those yourself just enumerated in the dictionaries there are some keys defining the headers so for the width the height the refresh rate all that sort of good stuff you can pick out one of those dictionaries and you can feed that back in as a display mode by calling CG configure display mode pretty straightforward you can also use our little chooser CD display best mode for parameters and a set of related functions with really long names longer than that even can be used to pick a mode these are sets of functions which take a desired width height pixel depth refresh rate and some other options you can feed in and they'll look through the list of available modes and try to pick the best one in general when they're picking a mode they'll try to match your depth that you requested they'll go for a size equal to or enclosing the size you requested if they have to go for and closing they'll go for as small as possible it encloses they also have some options that let you specify I want only modes that are known safe for this monitor and graphics card so you don't have to bother putting up that confirmation panel that can be handy there are options to pick stretched modes to fill in gaps on the sides of your monitor you can turn on mirroring now some interesting things that happen here when you mirror a display in another display the system is going to do some interesting things it's going to try for Hardware mirroring so if you're not explicitly setting the mode of some displays that you're putting into a mirror set the system is going to auto configure those to match in resolution and to match in depth so that it can use that Hardware mirroring I mentioned so if you don't explicitly set the mode of the display we will try to match them all up to avoid those mapping borders whats am adding border you might have seen these we get these nifty widescreen gadgets that's am adding border a little black band we put on the sides to sort of complete the screen we also have what are called stretch modes that you can get into a stretch mode will actually fill all this space in it'll actually use a little bit of hardware to expand out the image so when you go to set up displays as a mirror the system is going to try and change the modes of some of the displays in the set to match the smallest display in the set to match the depth of the displays together so we can use our Hardware marry tricks now on mirroring some displays cannot be unmarred for example the ibooks that secondary to slave for it is only for mirroring so you really can't break them apart the hardware doesn't handle that one other thing if you look in the display mode dictionary you might see numbers in there for things like the display row bites that's sort of a value taken in isolation when you mirror displays together the robots values for the individual modes may change because the device driver and the hardware might require certain amounts of padding if we do Matt generation we may do that Matt generation in hardware so the only video portion that's live is the only piece of frame buffer memory that actually exists those black bands are just an artifact of the video system they don't actually exist in the frame buffer so things like robots may change after display configuration where you set mirroring setup now once you set up your display mode display positions you've got your mirroring sets the way you want it you then call CG complaint display configuration that takes that ref pulled all the data in grinds on it for a while apply some of the system constraints and applies it it then releases that configuration riff there's a couple of different options well there's three different options actually this takes configure for appt only this is a really handy little mechanism if you tell the system I want this configuration just to apply to my app what that tells the system is this is not a permanent configuration this program wants it it might be a presentation app like this it might be a game it wants its particular display mode if something bad happens and a game is ceases to function for some reason say it failed quit escapes dies blows up the systems is home well that was a moat only for that app and I better go back to the user's preferences and it restores everything for you so if you're just setting things up for your own apps purposes please use we can figure for a half only option that gives us the subtle recovery mechanism one of the other cool things utilises in this configurations for a nap only if you need to get back to the user's preferences say at the end of the game you can use CG restore permanent display configuration and that it's a long one that'll put things back the way they were that restores the permanent display configuration catch you by surprise didn't it now configure for session is kind of interesting this applies a configuration change that lasts while that users logged in but it does not go into permanent storage so if the user logs out or reboot the machine it comes back up with this last permanent configuration setting this is kind of handy for experimenting with different configurations and putting up an option panel tum usual you want to cancel this one later you can cancel it and configure permanently it anyway anyway eular it configures it permanently that writes it in display preferences and it comes back the next time the user logs in or the machines rebooted interesting tidbit for you if you're an administrative user and you do eight configure permanently that also becomes the boot configuration for the machine the configuration is adjusted by the system after you feed it in so you should after setting the configuration no check what did it really do did you get what you expected I'm gonna do some tweaks on you that you didn't realize we actually have functions in the system you can use to read back the configuration you can read back the origin point for display you can read back its width and height you can read back all sorts of interesting things you can get the current display mode out of this twin see oh is this the Mona XS for now that's pretty close in addition we also have some other api's you might find handy you can ask for a displays vendor as who made the thing which comes back as an integer value and model code even the serial number of many displays to actually in code serial numbers this is handy if you're doing some specialized calibration apps typically you might want to put in certain kind of profiles color sync uses this stuff you can ask for the display resolution that's the width and height you can get the origin point one interesting thing is CG display uses OpenGL acceleration that's our little way of saying you got quartz extreme CG display i/o service port now was that that's an i/o service port yeah right an i/o service port is a little representation that you can pass into io kit io kick and then look at that and say oh yeah this display and you can read back all sorts of interesting really low-level things from i/o kit things like eat it data and everyone's eyes just glazed over next slide we have display change notifications you can subscribe to now quickdraw pays attention to these it does its own thing in reconfigures hooker also has the implementation for display manager by the way many of you might be calling display manager to get display configuration state changes that works I guess what it's built on it's Friday morning isn't it it's built on this stuff CG display a register reconfiguration call back but you can pass a function into it registers it and it calls you back when something happens in fact it calls you back when something's about to happen is that cool or what we know in advance and something's going to happen the callback is invoked for each display will call it back this display what's that display look the other display one at a time with a flag telling you something's about to happen and then after something happened displays are reconfigured it gets called again for each display this display its origin change this display its mode changed this display it's a mirror of this display and this display was removed from the system what we support hot plugging of displays now yeah I get that a DC or DVI connector on the back of the powerbook you plug it a sleigh an act you don't have to close the Lydia to put the thing to sleep first we notice that oh look something changing the hardware quick reorganize the system we reorganize the system we bring up the new display States and you're good and this you have to be prepared for those darn users can just take that connector pull it right out all your machines running displays can vanish on you boy that doesn't mess up a game so you have to be aware that these state changes can happen at any time it can be really shocking for you we do have some cool tricks by the way if you've captured displays to capture the display hot plugging and unplugging isn't going to perturb your system that much you capture to display your running on the built-in display in a powerbook somebody plugs in another display disconnects it will note that oh look something's connected we have this change state after he's done after he's let that display go he'll change state so we hold off for a little while that keeps your game coach from exploding on you it's a good thing now when somebody actually hot plugs that monitor into a system fun stuff happens I oak it gets pained by the hardware's and take the loop something changed the device drivers go rattling along if you know this one hey look there's a monitor on it now I do 11 before and the hardware and the software will work together to pull the monitor and I know what kind of monitor you okay you're one of the ISA and you run in which display modes okay I'll just pick one and it fires up the monitor and puts it into some mode of operation usually a mode of operation that's marked as a default state for that monitor at this point we send notifications out all the application single look look something changed you've got a new monitor and your application can do something with that or just ignore it most absolutist ignore it it's not a big deal if you're a carbon or cocoa app now care some big deal if you're some kind of full screen presentation if you might be interested miss you might be able to do something special with that extra display the one that's going to get you though is when they unplug the display what if you're doing something on that too what if you thought you had captured but you didn't enforce price if you actually captures the display and it gets unplugged now that's not so good if you're capturing this place you might want to think about keeping your UI on a display that's marked as the built-in display and yeah there's API to tell you see G is display built in true ok so you can check for that display fade feeding into the distance now the old way of fading a display was grab the gamma table stop some values in okay that's pretty good take those values what some slightly smaller values in okay keep doing that over and over and over again hundreds and hundreds of times burning up cpu cycles until you finally fade the screen out to black or whatever you were going to say to well that has a couple of bad effects first you burn a lot of CPU cycles and let's face it monitors have refresh periods they aren't going to see that change with 70 or 80 times a second you can do a thousand times second doesn't help its CPU intensive you lose your color sync data yeah what you guys went to a lot of trouble to calibrate those displays and now look what you've done oh so we have a new API nupur penser well I can't wake him up at all the display fate api's lets you tell the system I want to do a fade operation now please reserve the hardware and do a nice smooth transition it's driven off the hardware and off of some fancy timers it does a really nice job of fading displays in and out for you to handle all those transitions you can use this instead of doing the gamma table tricks on your own very easy to use built-in now we already have a fade effect built in for display mode changes all right we'd oh yeah I think you've seen that fade the blue thing that happened that's the sage transition point three seconds 5.5 fate in you can override it when you're doing a configuration for that configuration operation only by calling cg configure display fade effect now this wonderful function takes the config ref that you using the time to fade out the time to fade back in and the color to use for the Fayette transition the default that's built into the system will fade to that lovely shade of French blue when you do a display change unless you capture to display in which case we fade to black this probably covers what most of what you need if you're typically just setting the slate configurations up you don't really need to worry about anything else the default probably is good enough for most of you now there's other places where people like to do phase if you're in a game you've been running some sort of cinematic piece and you want to do a transition over to say is your GL based animations often people will do a fade out perhaps a harbor mode change bring up their full screen context and do a fade in we can automate all this for you you can acquire the hardware to do the state operation by calling a choir fade reservation when you're done you can release the display Fayed reservation the reservation is good for up to fifteen seconds if it takes longer than 15 seconds to the your fade transition I'd think about that that's a long time you should get impatient now you can do the fade synchronously or asynchronously you can tell the system I want to do a fade I want to start with a normal display I want to end up with a solid color and I want the color to have this RGB value this is just in the display color space 000 would happen to be black and true yes I want this to be a synchronous operation you'll call this and two seconds later this call will return and the display will then faded to black very straightforward you can then do some set up operations for instance stop your cinematic start your GL sequence running you might want to fade back in while you're already rendering in that case you want to start the fade in and then go right to doing your animations let the fade proceed on its own not worry about it to do that you would start off with for instance we're starting from a solid color in this case we want to end up with the display looking normal the starting color was black so we're doing a fade in from black falls meaning asynchronous in this case cg display sade returns immediately the fade occurs over the next two second which is what we specified while you get your GL animations up and running so you do nice fade in your game figures are already running around digitally stabbing each other having a grand old time this can actually look pretty slick and this is easier than trying to do a lot of the asynccallback stuff yourself all the actual call back and work to drive this is handled outside of your process in the fade machinery we've added to the system palliative displays oh here we go oh here uses pallid displays in their new app their coding up good i got some news for you this stuff in the system for legacy support there's minimal to no hardware acceleration available when you're using 8-bit pallets all at fancy hardware just loves 32-bit mode it goes past then they go down to 8-bit modus what I've got these huge honking buses and you want me to move eight bits at a time it's got a low precision doesn't composite very well at all it's barely supported by the hardware come on guys for past the Stephanie's we're past the 80s even we can get better the configuration functions will let you select an 8-bit mode they won't let you make it a permanent setting there is a reason for that anyone know ah alright we do have palette functions for you to play with this is for legacy support by the way I don't know if I mentioned that you can create a palette you can create a pellet from the displays content you can create a pal with an array of sample you provide you can get the color an index you can set the color at an index you can tell a display here's your palate and you can release appellate any questions no good common developer issues what sort of issues could developers possibly have with this stuff well as recognizing those mirroring and display configuration changes if you're playing with the display directly you need to be aware of mirroring that old device loop comes back to haunt you you're going to need to look at all the displays in the mirror ring set and make sure you draw on all and in turn or things can look bad if you're in Hardware mirroring oh lucky you you just have to worry about the active display in the hardware mirror set most of the newer hardware will be doing hardware mirroring that's what you're going to get and all the nice shiny power books in the newer iBooks the white ibooks that's what you're going to get those funny cards that have to monitor connectors on a single card both will normally do hard remembering unless you work really hard to make it not do it by setting the displays different depths explicitly don't do that unless you really like to work at things if you're capturing displays you must be aware of the mirroring state to get the correct behavior now capturing displays and changing the display configuration some people have been capturing the displays just so they could change the modes and then releasing them that looks funny the entire screen goes black for a couple of seconds your app comes to the foreground even though you thought you were doing this from some faceless UI element in the background looks bad don't do that please please there is no need to capture the display in order to reconfigure them yeah ucg capture all displays to avoid funny desktop layout effects if you capture all the displays there's no desktop showing and we won't bother telling things on the desktop that oh yeah the displays change sighs quick creme all your icons in the one corner so we can stop that behavior we can do it people we can work together capture all the displays please that heís display mode changes from the rest of the system only your app needs to worry about them in that case display changing let's take a look over on demo machine one and see what this is all about I mentioned we have some API is that let us actually look at the display state here you can see we can look at well there's the origin that makes sense oh yes 1024 x 768 sure ok yeah 32 bit too deep uh-oh look refresh rate came back LCD panel why is that well for some arcane reason very deep in the system I don't know why we get back a refresh rate of 0 on LCD panels so that's a good tip is oh you look at must be an LCD panel new for panther we have a new function that returns you the display width and height in millimeters no this is not actually three hundred and ten millimeters this one here is this number here is the display ID for this particular display I mentioned you can get information about particular models this is vendor ID 610 I wonder who that could be and product ID 9215 yeah display modes we have desks and mechanisms in the system to let you actually pick the sleigh modes up it looks like we've got a 1024 x 1658 32-bit display suppose I wanted to let us say try for oh yeah there's a mullet for 640 480 I want I go for saya no that's close 800 x 600 is the nearest mode that would enclose that size 1024 x 768 the nearest mode that wouldn't close this size oh look we can get to 8-bit let's see what happens if we apply it kind of grainy looking in it and a compositor doesn't care for that that much we can always be a lot by restoring our permanent display configuration with that one line function and we're back now some displays have stretched modes they can support suppose I asked for 800 x 600 on this one yep this displays near a stretched mode is 1024 768 so it would take that and try to stretch things safe modes ah we have a safe mode this is all the modes that are known to function for this hardware and the graphics card that's in here safe mode these are options you can feed into that get me the best mode for a particular configuration now to get the information about a display we needed to get a display ID alright well let's get the main display ID in this case we can feed in things this returns us the physical bounds and millimeters for the display this returns us the pixel bounds for that display we can ask for the current mode that returns us that CF dictionary rest we can pull information out of that anything in ACF dictionary rep is accessed by key in this case the values that you'll get back we'll all be CF number CF number rep and you can use that to fetch out the value by as an integer or as a double whatever is most convenient for you the CF number rep will do the right thing and as far as converting those other API so lets you get the vendor ID the model number very straightforward stuff searching for a display mode this is a rough one this is the longest version of the function cg display best mode for parameters and refresh rate what property shot that it takes a display ID a desired width a desired height depth a desired refresh rate an optional property value you can just feed in null for the property by default or if you're looking for only safe displays so you don't have to put up any kind of confirmation dialog you can feed in okay so you just label to say for hardware and it returns you a mode the function will always return a mode if it can't figure out what on earth you're asking for it will hand back the current display mode I don't know use this it works right now tries for a depth first match on those now we change the configuration I told you this is rough stuff we found the mode with that best mode for stuff begin configuration we get back our configuration to handle configure the display mode for the main display to that mode these modes are only valid for the display you query daman and you don't get to make up your own the mode will only work that's returned by one of the API is that bends modes or lists of modes and once we've got things set up all we're doing a change in the load in this case we call CG complete display configuration configure for appt only so if something bad happens in the app crashes we get back capturing all the displays we have a new for penser item you can specify an option capture with no fill that actually captures the display but leaves the content as you see them think you can do your fade transition it looks like you're fading out from the desktop really slick going to your app I'm going to show you an example of capturing a display doing a fade out we're going to draw a color bars pattern and then we'll do that asynchronous fade back in one of the interesting things are the ways you can start a fade back in and release your reservation token if the fade back in ends with a normal display if it ends with a normal display state then releasing that token will have no effect on the displays appearance the system will automatically say okay this is a deferred release i'll release it once the state in is done and once we're done we're going to use release all displays to release the displays let's capture the display you'll see we're going to capture it but not actually fill it and we'll do a fade out fade back in with our color bar pattern it's all capture it's ours now we can do with it as we wish now click we restored everything that easy didn't have to record all the display locations didn't have to try and put them all back ourselves very straightforward this sample code will be appearing on our website soon in the next few weeks now onto something completely different remote operations remote operations is all the stuff you need to do remote control over the machine hello look display drawing activity monitor and generation hosting events look at that name display drawing activity monitor let you listen for state changes on the display oh look a window just flushed over here oh look quick on movies playing over in that corner stuff happening it tells you the areas that are changed on the display will also let you post events into the system for remote control purposes or for userspace device drivers now how is all this going to work why do you have two straight the bits off the string why can't I go get them somewhere else well all the apps in the system are drawing into their own private buffers you don't get to play with those through the apps buffers not your buffers the buffers all go into the courts compositor which takes all the bits blends them together seasons and lightly and sends them to the frame buffer for display some bits can be transparent some of the bits from the frame buffer maybe as a result of several windows accumulated together the only place where you can get a consistent picture of what all the pieces look like is at the frame buffer there's a lot of processing that can happen between here and there if you looked at some of the accessibility interfaces we have things like display zoom that can actually occur so what you see on the display doesn't actually correlate very closely with what you started with in the individual app frame buffers and the individual app buffers the final frame buffer product that's what you need you pretty much have to go to that the remote operation services is the best way of getting these finished bits clients live remote services include apple remote desktop Timbuktu the old DNC some accessibility applications like to play with us too they may need to post events into the system so they'll be using the event posting interface to post into a particular session and tell the apps in that session to show certain behaviors screen scraping all I gets bits well we have this complicated pipeline it's really hard to get to the individual dish from the window if you wanted to get the bits on the screen but not go to the screen it'd be an awful lot of work you'd basically we went to reinventing the whole courts compositor and there's more tooth and it looks Oh to assist you in finding what's change on the screen we give you a system of asynchronous notifications async means that we tell you something has changed your app goes and picks it up and gets the current state of what's change on the system it's a sink so that applications that don't behave well don't get to hang the entire system it's very straightforward if you have delays in your processing delays that occur systems system-wide scheduling or load that cause you to be a little slow about getting the current state back that's not a battery problem we keep coalescing the state changes for you they're built up and your app will get the latest image of whatever states on the display there are new functions added to Panther that tell you not about just changes but about areas that are moving on the screen this will help speed up certain operations that you might get into if you're doing screencasting good morning now strains rating performance now that screen is at the end of a long bus that long pipeline so out there's a distant end of a GP or PCI bus connector and if you go out there and read those pixels yourself it's going to be slow the fastest way to read the pixels off the screen is not obvious at all is use OpenGL you build a full train GL context select the screen with GRE buffer Jill front you read the screen with GL read pixels into your own buffer that's going to get you DMA transfers from the screen right into your applications buffer if you do it right if you don't do it so right it's still going to get you a fast software path but it won't be as fast as doing the DMA trick things to be aware of GL is a little different view of the world that you might be used if you're used to dealing with frame buffers it puts things in quadrant one the origin point being down in the lower left corner upside down now if you're just processing the bits to pack them up and send them across or a screen casting application that's not actually if that big a deal it just means you have to walk through the bitmap in a different order just set your pointers up correctly and set up your stride as you traverse the data and you'll be all set source code showing you how to do this is going to be online i'm going to show you right now what this looks like over here on one little bitty grab application we can grab the spring contacts go ahead and save it to a file there we are and look it's a screen grab well that was exciting wasn't it can go back to sleep now but I hooked up something interesting to this version of grip we can time it will do 10 grabs in a row and averaged out and see how many milliseconds it takes to go ahead and scrape this 1024 768 32-bit screen in software okay 140 milliseconds that's not too bad let's try it with OpenGL oh oh I got faster didn't it that's more time for your application to get the data compressed shove it over the wire and catch the next screen update now that I factor a ten in this case now you won't always see that big of a change but pretty much any of the hardware we built in the last couple of years this is faster than doing it and software DMA would so how does this code do it it's rough we set up for our main display a mask fit we say we'd like a full-screen context for that display we choose a pixel format have GL create a full-screen context get rid of the pixel for us you don't leak set that context to be the current context tell the system this is full train here we go point GL at the front buffer that is the display ask it for the width and height get a place to put the data we're going to grab make sure that all the DL commands are finished in the display is correct and current tell the system we'd like the data packed please call GL read pixels with the mystical incantation govt GRA GL unsigned integer GGG rub these are apple extensions their present on all Mac os10 systems so I didn't bother checking for that first naughty me if you ask for this pattern this will match 32 bit deep display hardware you'll get the DMA effect GL unsigned int 1 555 rev the only other rev depth in there will get you the 16-bit display format and you'll get DMA transfers in that case also when you're all done release stuff that's all there is to it less than two pages of code in really big type to capture the string contents and back to the slides leave over in the event side of systems you need to post events into the works right this is interesting we have keyboard and mouse events coming in from the kernel or to the colonel from our hardware the colonel and his system pick up those events chew on them a little bit put them into a nice format for us pass those into the courts compositor what the courts compositor knows we're all your windows live it knows what you app is foreground so it can be used to route these events out to all your applications there's a couple of ways to push events into the system io ahead post event is a call from user space into the i/o kit that takes an event structure and shoves it right in as though we're coming from other hardware if you are writing a user space device driver or something that wants to pretend to be a device driver synthesizing events system-wide use i/o hit post event there are also some techniques can be just inject events at the courts compositor level these are actually not appropriate for userspace device drivers they are used in some assistive technology though these lets you post a keyboard event in a scroll wheel event or a mouse event in at the course compositor level now why are these not appropriate we have multiple users sessions on the machine now there can be a second user login with the second set of windows a second window system second set of apps a second doc all that post keyboard event posts roll wheel event and post mouseevent are going to post into the session they are called in so if you're emulating hardware you missed got the difference there I don't want anyone to get tripped up on this posting Mouse advance is pretty straightforward CT post mouseevent you give it a first position you tell a boolean that tells it I want to move the cursor to this position normally a pass true in you pass an account of buttons one two three and a set of boolean's for the button state if you push a button down don't forget to post another event to let the button up thank you that has a really bad effect on the system I'm going to get to in a minute there's also an interface CG warp cursor you can use just to move the cursor around on the display posting keyboard events there are key codes which are the decrypted glyph codes that you see in a system they're also character I'm sorry there are key codes oh is this terminology fuzzy key codes are virtual key codes their scan codes coming off the keyboard character codes are the encoding for a particular character a glyph the thing you need to get right passing into this call is the key code most environments ignore the character code part the virtual key code is the part you need to pass in there's actually a dummy flag you can pass in for the character cuz it'll cause the system to guess at the best key code to use with it application environments like carbon and cocoa use the key mapping and the key char API is part of carbon to actually do the translation from that raw virtual key code into the character code modifier flags are set by posting virtual key codes for the modifier keys all the keys and the people have these key codes you pass them into this interface and it's just like you're pushing the keys and letting them up on the keyboard don't forget to release the key after you pressed it the events Depression when you post events into the system we turn off the hardware event pass for a little while by default that's to avoid Mouse Wars a student helpdesk system you want to drive a sister for little by posting events into it you don't want the eight-year-old grabbing his mouse and fighting you so we'll turn off the local mouse for a short period of time the force quit key combination reconnects the local hardware things get really bad say you goof when you're debugging things you forget to let the mouse button up there are functions to override the default behavior now I suggest you always set the state as you want just in case the default behavior changes in the future the event state combining as one of these things that combines the flags from your keyboard modifiers from the real keyboard with the events from the remote system you might want to keep this off to keep local modifier keys to the local shift key being held down from getting into your event the most frequent question we get is how do we turn this off to do that you just want to set the suppression interval to zero so it doesn't suppress things for any length of time tell the system yeah I want event combining I want my local shift key and my remote ship heat all act the same and you'll want to tell the system don't bother blocking local hardware events permit all events during the suppression interval and permit all local events during a remote mouse drag a tells you you had to let that mouse button up if you leave that mouse button down you're doing a drag operation and this is will normally block all local events during that drag to keep things from going horribly astray if you don't post that mouse up you're sticking in a drag state bad don't do that common developer problems questions we run into I want to filter all the events in the system well you don't do that at the courts level that's the kind of thing that goes Ethan I oak it or you might want to think about what you're doing what does you really need to accomplish for example some of the questions we've gotten on this our operations that are better handled as input managers in the system then as a generic low-level I filter every event and pick out the ones I'm interested in sort of operations ah another little common for people only use the functions in the public headers undocumented interfaces may change they may go away they may stop working undocumented parameters undocumented functions may do undocumented things to the system you may let the smoke out of some parts the stuff in the public headers is actually better when we do our development we often build things versus a private FB I and then once we figured out all the quirks and what we've gotten our act together we turn it into a public interface please use the public interfaces who here is calling a private interface any volunteers yeah yes yes I'll change that for you I'll take care of that one and for more information let's get Travis up here [Applause] thank you Mike I think we have some reference documentation available for you guys we're after you going to aggregate it's all aggregated on sort of reorganized ADC website where tech pubs is move the documentation from where it was before we began w WC so everything should be you know documentation should be appearing on a lot of stuff into the graphs and imaging area on the revised ad see website we also have some tech notes that are available and so it's gonna be taken up to a seven it talks about the GG direct display API and then well probably very many people won't be going to too late to cg direct solid api considering how unpopular eight that palliative tech is actually becoming and should become we have some additional QA so such as many relating to patrol and see how to use it if I'm a carbon coral app and I'm you need to use cg direct display you know several q and A's up here they're available and we have lots of sample codes that are going to be appearing I don't think all of these are up quite yet but check the ADC website we plan to push a lot of samples that you saw on today's demo out very shortly and then obviously if you need more information on how to use GL + GL read pixels you can look at a couple different GL references pry it open GL doubt at work which is a good resource for OpenGL in general you