WWDC2003 Session 211

Transcript

Kind: captions
Language: en
bdc so kicking the graphics track off
this morning with session to 11 which is
introduction to court services and the
primary focus of the session is going to
be the non drawing api's that are part
of the courts architecture these api's
do things like manage the displays even
facilitate doing certain common
operations the developers need to do
such as what we call screen scraping are
pulling pixels back out of the frame
buffer and the key thing is that an
important part of the session is that
these api is that we're going to be
talking about in the session they're
essentially the the API is that lie
underneath other API is to do display
management that you might be familiar
with like a QuickTime api's draw
sprocket api's display manager eight
guys and we really when the reasons we
put the session together is because we
really want developers to migrate to the
display services API is because that's
really where all the work actually
happens and you're getting closer to the
native API and getting more
functionality and doing it that way so
it's my pleasure invite Mike Poquette
graphics imaging engineer to the stage
to take you to the presentation good
morning I'm Mike talk yet I'm the person
in the torch team who takes care of all
the non drawing parts of quartz I'm guys
working way down there in the bowels of
the system fixing the plumbing wrapping
in a little duct tape here in there as
your requests come in implementing all
those nice features you need and i'll be
going over quartz services now quartz
surfaces what can port services do for
you we're the non drawing part of quartz
yeah there's there's code that doesn't
draw in there it's lots nice we handle
all the display management stuff we give
you the functionality you need to take
over a display to do a presentation we
let you reconfigure the displays we can
let you position displays on your
desktop we can let you adjust the
resolution of the displays change the
refresh rate do all those fun little
tweaks the display preferences panel is
an example of someone who uses our
services we also handle some special /
tasks we give you the basics you need to
read the frame buffer contents off that
screen scraping thing we also have
support for doing remote control as a
computer programs like apple remote
desktop and Timbuktu can take advantage
of this stuff now most apps don't need
cord services this low-level stuff most
apps will run just fine and don't even
need to know this stuff exists where
does this fit in in court where the
display management code where your
screen scraping support and we're event
injection event injections a fancy way
of saying take an event and stick it
into the event chain and a little
dribble on down to all the applications
we're going to give you full screen
application support information here and
display configuration information and
we'll go over a few the more obscure
api's that are in there so I'm just help
you out with backward compatibility and
some to help you do some neat things
that you might not have been able to do
very easily before we're also going to
cover the remote operation services
that's the screen scraping part I've got
some sample code for you will walk you
through a little bit of tricks and
techniques there and show you how to
post some events for remote control
purposes and also give you guys who were
writing user space device drivers some
tips on how to make that work right now
we've got some definitions we use some
funny terms in court sometimes display
ID that's not well it's probably
something identify the display I figured
that much out okay just lay IDs are
numbers that go along with displays
there 32-bit 'ish and of things there's
a typedef in there for them the numbers
stay the same for any specific exact
hardware configuration even across
reboots any change changes in the horror
configuration you unplug a display and
replace it with a different model you
move the display to a different
connector on the back of the machine
you're going to get a different display
ID now all these displays that we hook
up to the machine you'll query in court
services using that display ID that's
how you identify them to courts and
that's how courts tells you about the
displays
display States we have a bunch of
different states monitors can be in a
bunch of states the frame buffers can be
in we combine all that to get there just
refer to this as a display state states
include active and active display is one
you can draw on that's pretty
straightforward it's there and it's nice
and drawable we can have two slaves to
go to sleep you're familiar with that
goes blank one of the Energy Star mode
usually kicks in in a monitor gets all
nice and quiet don't draw on a sleep
displays now again if you're an ordinary
application you don't need to worry
about that it's all covered for you
these things only apply the only time
you really need to be concerned about
these is if you're writing something
that's drawing directly to the screen
itself through one of our fancy api's
we're gonna get into here mirroring Oh
mirroring has changed so much over the
years we've introduced things like
hardware mirroring and we've got
software mirroring we have hardware
generated display mats and also to other
wacky things I'll be getting into all
those different states in detail and
giving you some tips on how to deal with
that we have online displays an online
display is pretty much a monitor plugged
into the computer it may be awake or
asleep and may be mirrored around near
but it's connected off wine well that's
the connector on the back of your
computer with no monitor there it's just
offline all alone a placeholder in the
system don't try to talk to the system
about offline monitors it that they
aren't there you'll see all through our
programming interfaces this CG prefix CG
is of course as is intuitively obvious
the most casual observer the prefix for
courts now the display services part of
the system is going to cover display
enumeration it's going to tell you that
is what are all the displays get me list
please it's going to cover display
configuration place this display here
change this one's resolution change the
refresh rate change the bit depth
we're going to cover palette or display
support now there is something you
haven't heard about in a while pallets
so 70s and we got some new goodies for
Panther our displays the API now
competitive technologies by competitive
I don't mean that those something those
people up north of building what I mean
is other technologies inside Mac os10
that do a lot of the same functions that
you see in this code there's the
sprockets tough draw sprocket has a lot
of display management goodies in it
quicktime has a few obscure APA's in the
corners for handling displays as does
the AGL apple graphics library package
and of course there's our old buddy
display manager now these api is our
president Mac os10 oh why are they in
Mac OS 10 oh so you could bring your
apps forward to assist you in porting so
they're so CFM test based apps that you
want to run on nine and ten can run but
you can use cg direct display these new
API is instead those older functions on
Mac OS 10 are all built on top of the CT
direct display API so you don't really
need to put up with those things if they
don't do happy what you want if you cut
right down to the CT director slaap eyes
you can get some finer control over the
system you might be able to make it do
just what you want I'll give you some
tips on how to do that clients of these
display services games we like games
full screen applications like keynote
configuration tools display preferences
your own apps new functionality that we
have mirroring I told you about
mirroring before right mirroring
software mirroring is something that
pretty much everyone who used to
we all remember writing those good old
loops run through the G device list find
those displays figure out which ones
intercept our window and draw on all of
them over and over and over again well
you know you don't actually have to do
that in the carbon or cocoa AB that's
covered for you the courts compositor
does that work but if you are talking
directly to the displays yourself you
still get to do that which might be one
good reason to think about sticking with
windows systems and not bothering to go
right to the display something to keep
in mind if we're doing software
mirroring software mirroring will happen
on all these machines with different
graphics cards monitor on each graphics
card or if displays are in different
depths software mirroring you have to go
through if you are talking right to the
display and draw on each display in turn
going down the line hardware mirroring
is something that's shown up just in the
past few years hardware mirroring
happens when all those displays are
running at the same depth and they're
attached to the same controller you'll
see Hardware mirroring when you connect
up external monitors to the newer I
books to the shiny power books to those
funny new graphics cards that support
two monitors hardware mirroring one
display is the active display the
primary display keyword the other
displays are mirrored and hardware you
don't need to draw on those so you just
draw on the active or the primary
display in a hardware marriage set we
have more programming interfaces to tell
you which ones are the primary which
ones are active and which ones are
Hardware mirrors
now we also had hot plugging in the
system I'm going to get into that in
some more detail in the world a little
while hot plugging turns mirroring on
and off automatically in some cases in
other cases you get extended desktops in
either case if you're going to do full
screen access using what we call display
capture you need to worry about all
these things display capture is
something really cool it lets you steal
the display away from the windows system
it's yours you got it you have lots of
control that way but you know what the
window system doesn't know about it so
anything that normally goes into a
window isn't going to work there like
views eh I've usns views controls
buttons all those fancy high level
widgets you know work on capture
displays because you can't put a window
on that you've just taken over that
hardware for your own purposes now when
you do take over the hardware like that
though there are advantages OpenGL for
example can put that display into some
very unusual modes of operation that are
extremely fast game developers you
probably notice the sorority huh so
you've got more control but less high
level functionality in particular
because you're not going through the
window system you don't have that view
hierarchy and all those other goodies
there all those accessibility API eyes
that are going to be covered in another
session today don't work so that's
something to bear in mind captured
displays are normally used by games
that's a big use of them particularly
the OpenGL games like to take over the
display and go as fast as you can now
there's another way to do full screen
access and this will be familiar to lots
of folks coming from the mac OS 9 world
carbon developers hide menu if you call
carbons hide menu function the menu bar
disappears the doc goes off the screen
it looks big and empty you then create a
full screen window and put it up there
and you draw through a window now your
app still ring in the window system this
is used by some other presentation apps
such as PowerPoint to do it's full
screen thing
also used by a number of games this is
the way a lot of the games that are
based on the Pepsi FM technology that
run on both mac OS 9 a mac OS 10 work
because this sort of functionality is
available on both you are going through
the window system to do this you're
doing a full screen window that has
advantages for accessibility purposes it
has disadvantages in that it might not
be as fast as for example running opengl
and a full screen mode and display
capture we've got functions for you cg
capture display grab one display take
control the display is removed from the
window system the app that calls this is
forced to the foreground you wouldn't
believe how bad it is to leave somebody
taking over an entire display in the
background so the events go to some
other app it's really hard to recover
situation so if you capture display
we're going to make sure you're the
foreground app you'll be pulled as
workaround it's your responsibility than
to process events and respond now once
you capture the display you can draw
directly to that screen using GL quartz
and other techniques you can get a port
to it capture all displays capture all
this place is kind of handy this just
takes all the displays in a multi
display desktop and captures them now by
default while capturing the distal a
caption all the plays are going to color
them all black they take it over capture
all displays has another neat thing that
might be soon lead some of the draw
straw get folks if you've captured the
displays you release them to go bring up
some UI elements you can do that and
then you capture them again it will
apply the last captured mode to the
displays again so you'll be right back
in the last mode of operation you were
running full screen
now after you're done using a display
you might want to release it you can do
this in the middle of your program just
so that you can go ahead and use those
UI elements bring up some control panels
for setting up game options that sort of
thing cg display release releases one
display lets it back into the window
system cg release all displayed is
really handy this releases all the
displays okay that was obvious it also
restores all the display modes to the
ones they use your last set up and their
preferences it puts the gamma table back
the way colors sync thinks they should
be and you can use it after a display
capture or capture all displays one
function boom puts everything back the
way it was you don't have to keep notes
yourself of what the display layout used
to be you don't have to go through all
its yeah I think this one was over here
I think this one might have been up here
none of that stuff just call one
function boom you're all set drawing to
a capture display we have a couple of
new functions you repent their see Jesus
play get drawing context let's get you a
quartz drawing context for a display the
context is owned by the framework notice
it doesn't say create there so don't
release it opengl you can use opengl
take a pixel format set up a full screen
GL context those see ggl AP is when
you're done destroy your context put
everything back the way it was opengl is
usually going to give you the best
overall graphics performance if you've
been going any of the GL sessions you've
seen you can use it to 2d animations you
can use it to bleed images to the screen
you can use it to do all sorts of
interesting special effects feels pretty
cool stuff and even better you can now
get a CG context from your opening yell
context to get an accelerated context
that's been covered in a few sessions
here too now some people might need to
change their display configuration if
you're playing a game and oh you're on
one of those nice big cinema displays
wow he probably didn't do all your
artwork for that 1280 x 900 or whatever
that thing is you probably did a lot of
artwork Pharaoh 640 by 480
so you got two choices you can do a
little bitty square in the middle of the
display or you can change the display
mode and fill the screen the display
configuration API lets you do this it's
a transactional model you tell the
system I want to begin a display
configuration i want to accumulate a
series of operations i want to set a
display mode i want to set a position i
want to set a refresh rate all these
goodies and then once you've accumulated
this new state that you want all the
displays in you apply it with one
function call and the system will then
go through and do whatever is needed to
get there now the system has constraints
built into it we don't let you put gaps
between your displays you can lose your
mouse cursor that way we don't know a
while overlapping displays that can look
kind of odd be very confusing to a user
we don't allow you to make up your own
display modes you not see I 42 is a good
number i'll use that one I wonder what
that does no you can't do that the
system is going to reject these sort of
things if you're on for instance an
ibook that's doing mirroring of displays
the hardware doesn't handle not
mirroring the displays so if you try to
do that the system is going to say no I
don't do that and you can't you can't on
your so we have these constraints that
get applied one other constraints that
gets applied as it's going to look at
all the displays you're setting the one
that shows up at 0-0 for its origin
point becomes the main display that's
the one that gets the menu bar in the
login window if you don't set any
displays 200 we get to pick one so we'll
just do it ourselves
yeah cg began display configuration is
how you get the whole process rolling
this is going to return to you ACG just
lake and cigarette in a pointer this
little rest thing is what you're going
to pass to all the other configuration
functions it's going to be the
accumulator for all the state you want
to build up and it's going to be what
you use to apply the state change you're
going to call CG complete display
configuration to complete the display
configuration or if you've done all this
work and decided who this an asshole I
don't want to do this anymore you can
throw it away with canceled display
configuration to set the display origin
you use the mysteriously named function
cg configure display origin this sets
the origin point the upper left corner
of that display you can lose them in X
you can move them in why I'm Z you can
configure a display mode what's a
display mode a display mode code wise
looks like a CF dictionary ref a display
mode describe a particular display
resolution a particular refresh rate a
particular bit depth and some options
display modes come from a couple of
different api's that we have we have AP
eyes that will let you get a list of all
available with bleh modes for particular
display that comes back as a CFR a rift
full of CF dictionary reps you can look
through those yourself just enumerated
in the dictionaries there are some keys
defining the headers so for the width
the height the refresh rate all that
sort of good stuff you can pick out one
of those dictionaries and you can feed
that back in as a display mode by
calling CG configure display mode pretty
straightforward you can also use our
little chooser CD display best mode for
parameters and a set of related
functions with really long names longer
than that even can be used to pick a
mode these are sets of functions which
take a desired width height pixel depth
refresh rate and some other options you
can feed in and they'll look through the
list of available modes and try to pick
the best one in general when they're
picking a mode they'll try to match your
depth that you requested they'll go for
a size equal to or enclosing the size
you requested if they have to go for and
closing they'll go for as small as
possible it encloses they also have some
options that let you specify I want only
modes that are known safe for this
monitor and graphics card so you don't
have to bother putting up that
confirmation panel that can be handy
there are options to pick stretched
modes to fill in gaps on the sides of
your monitor you can turn on mirroring
now some interesting things that happen
here when you mirror a display in
another display the system is going to
do some interesting things it's going to
try for Hardware mirroring so if you're
not explicitly setting the mode of some
displays that you're putting into a
mirror set the system is going to auto
configure those to match in resolution
and to match in depth so that it can use
that Hardware mirroring I mentioned so
if you don't explicitly set the mode of
the display we will try to match them
all up to avoid those mapping borders
whats am adding border you might have
seen these we get these nifty widescreen
gadgets that's am adding border a little
black band we put on the sides to sort
of complete the screen we also have what
are called stretch modes that you can
get into a stretch mode will actually
fill all this space in it'll actually
use a little bit of hardware to expand
out the image so when you go to set up
displays as a mirror the system is going
to try and change the modes of some of
the displays in the set to match the
smallest display in the set to match the
depth of the displays together so we can
use our Hardware marry tricks now on
mirroring some displays cannot be
unmarred for example the ibooks that
secondary to slave for it is only for
mirroring so you really can't break them
apart the hardware doesn't handle that
one other thing if you look in the
display mode dictionary you might see
numbers in there for things like the
display row bites that's sort of a value
taken in isolation when you mirror
displays together the robots values for
the individual modes may change because
the device driver and the hardware might
require certain amounts of padding if we
do Matt generation we may do that Matt
generation in hardware so the only video
portion that's live is the only piece of
frame buffer memory that actually exists
those black bands are just an artifact
of the video system they don't actually
exist in the frame buffer so things like
robots may change after display
configuration where you set mirroring
setup now once you set up your display
mode display positions you've got your
mirroring sets the way you want it you
then call CG complaint display
configuration that takes that ref pulled
all the data in grinds on it for a while
apply some of the system constraints and
applies it it then releases that
configuration riff there's a couple of
different options well there's three
different options actually this takes
configure for appt only this is a really
handy little mechanism if you tell the
system I want this configuration just to
apply to my app what that tells the
system is this is not a permanent
configuration this program wants it it
might be a presentation app like this it
might be a game it wants its particular
display mode if something bad happens
and a game is ceases to function for
some reason say it failed quit escapes
dies blows up the systems is home well
that was a moat only for that app and I
better go back to the user's preferences
and it restores everything for you so if
you're just setting things up for your
own apps purposes please use we can
figure for a half only option that gives
us the subtle recovery mechanism
one of the other cool things utilises in
this configurations for a nap only if
you need to get back to the user's
preferences say at the end of the game
you can use CG restore permanent display
configuration and that it's a long one
that'll put things back the way they
were that restores the permanent display
configuration catch you by surprise
didn't it now configure for session is
kind of interesting this applies a
configuration change that lasts while
that users logged in but it does not go
into permanent storage so if the user
logs out or reboot the machine it comes
back up with this last permanent
configuration setting this is kind of
handy for experimenting with different
configurations and putting up an option
panel tum usual you want to cancel this
one later you can cancel it and
configure permanently it anyway anyway
eular it configures it permanently that
writes it in display preferences and it
comes back the next time the user logs
in or the machines rebooted interesting
tidbit for you if you're an
administrative user and you do eight
configure permanently that also becomes
the boot configuration for the machine
the configuration is adjusted by the
system after you feed it in so you
should after setting the configuration
no check what did it really do did you
get what you expected I'm gonna do some
tweaks on you that you didn't realize we
actually have functions in the system
you can use to read back the
configuration you can read back the
origin point for display you can read
back its width and height you can read
back all sorts of interesting things you
can get the current display mode out of
this twin see oh is this the Mona XS for
now that's pretty close in addition we
also have some other api's you might
find handy you can ask for a displays
vendor as who made the thing which comes
back as an integer value and model code
even the serial number of many displays
to actually in code serial numbers this
is handy if you're doing some
specialized calibration apps typically
you might want to put in certain kind of
profiles color sync uses this stuff you
can ask for the display resolution
that's the width and height you can get
the origin point one interesting thing
is CG display uses OpenGL acceleration
that's our little way of saying you got
quartz extreme CG display i/o service
port now was that that's an i/o service
port yeah right an i/o service port is a
little representation that you can pass
into io kit io kick and then look at
that and say oh yeah this display and
you can read back all sorts of
interesting really low-level things from
i/o kit things like eat it data and
everyone's eyes just glazed over next
slide we have display change
notifications you can subscribe to now
quickdraw pays attention to these it
does its own thing in reconfigures
hooker also has the implementation for
display manager by the way many of you
might be calling display manager to get
display configuration state changes that
works I guess what it's built on
it's Friday morning isn't it it's built
on this stuff CG display a register
reconfiguration call back but you can
pass a function into it registers it and
it calls you back when something happens
in fact it calls you back when
something's about to happen is that cool
or what we know in advance and
something's going to happen the callback
is invoked for each display will call it
back this display what's that display
look the other display one at a time
with a flag telling you something's
about to happen and then after something
happened displays are reconfigured it
gets called again for each display this
display its origin change this display
its mode changed this display it's a
mirror of this display and this display
was removed from the system what we
support hot plugging of displays now
yeah I get that a DC or DVI connector on
the back of the powerbook you plug it a
sleigh an act you don't have to close
the Lydia to put the thing to sleep
first we notice that oh look something
changing the hardware quick reorganize
the system we reorganize the system we
bring up the new display States and
you're good and this you have to be
prepared for those darn users can just
take that connector pull it right out
all your machines running displays can
vanish on you boy that doesn't mess up a
game so you have to be aware that these
state changes can happen at any time it
can be really shocking for you we do
have some cool tricks by the way if
you've captured displays to capture the
display hot plugging and unplugging
isn't going to perturb your system that
much you capture to display your running
on the built-in display in a powerbook
somebody plugs in another display
disconnects it
will note that oh look something's
connected we have this change state
after he's done after he's let that
display go he'll change state so we hold
off for a little while that keeps your
game coach from exploding on you it's a
good thing now when somebody actually
hot plugs that monitor into a system fun
stuff happens I oak it gets pained by
the hardware's and take the loop
something changed the device drivers go
rattling along if you know this one hey
look there's a monitor on it now I do 11
before and the hardware and the software
will work together to pull the monitor
and I know what kind of monitor you okay
you're one of the ISA and you run in
which display modes okay I'll just pick
one and it fires up the monitor and puts
it into some mode of operation usually a
mode of operation that's marked as a
default state for that monitor at this
point we send notifications out all the
application single look look something
changed you've got a new monitor and
your application can do something with
that or just ignore it most absolutist
ignore it it's not a big deal if you're
a carbon or cocoa app now care some big
deal if you're some kind of full screen
presentation if you might be interested
miss you might be able to do something
special with that extra display the one
that's going to get you though is when
they unplug the display what if you're
doing something on that too what if you
thought you had captured but you didn't
enforce price if you actually captures
the display and it gets unplugged now
that's not so good if you're capturing
this place you might want to think about
keeping your UI on a display that's
marked as the built-in display and yeah
there's API to tell you see G is display
built in true ok so you can check for
that
display fade feeding into the distance
now the old way of fading a display was
grab the gamma table stop some values in
okay that's pretty good take those
values what some slightly smaller values
in okay keep doing that over and over
and over again hundreds and hundreds of
times burning up cpu cycles until you
finally fade the screen out to black or
whatever you were going to say to well
that has a couple of bad effects first
you burn a lot of CPU cycles and let's
face it monitors have refresh periods
they aren't going to see that change
with 70 or 80 times a second you can do
a thousand times second doesn't help its
CPU intensive you lose your color sync
data yeah what you guys went to a lot of
trouble to calibrate those displays and
now look what you've done oh so we have
a new API nupur penser well I can't wake
him up at all the display fate api's
lets you tell the system I want to do a
fade operation now please reserve the
hardware and do a nice smooth transition
it's driven off the hardware and off of
some fancy timers it does a really nice
job of fading displays in and out for
you to handle all those transitions you
can use this instead of doing the gamma
table tricks on your own very easy to
use built-in now we already have a fade
effect built in for display mode changes
all right we'd oh yeah I think you've
seen that fade the blue thing that
happened that's the sage transition
point three seconds 5.5 fate in you can
override it when you're doing a
configuration for that configuration
operation only by calling cg configure
display fade effect now this wonderful
function takes the config ref that you
using the time to fade out the time to
fade back in and the color to use for
the Fayette transition the default
that's built into the system will fade
to that lovely shade of French blue when
you do a display change unless you
capture to display in which case we
fade to black this probably covers what
most of what you need if you're
typically just setting the slate
configurations up you don't really need
to worry about anything else the default
probably is good enough for most of you
now there's other places where people
like to do phase if you're in a game
you've been running some sort of
cinematic piece and you want to do a
transition over to say is your GL based
animations often people will do a fade
out perhaps a harbor mode change bring
up their full screen context and do a
fade in we can automate all this for you
you can acquire the hardware to do the
state operation by calling a choir fade
reservation when you're done you can
release the display Fayed reservation
the reservation is good for up to
fifteen seconds if it takes longer than
15 seconds to the your fade transition
I'd think about that that's a long time
you should get impatient now you can do
the fade synchronously or asynchronously
you can tell the system I want to do a
fade I want to start with a normal
display I want to end up with a solid
color and I want the color to have this
RGB value this is just in the display
color space 000 would happen to be black
and true yes I want this to be a
synchronous operation you'll call this
and two seconds later this call will
return and the display will then faded
to black very straightforward you can
then do some set up operations for
instance stop your cinematic start your
GL sequence running you might want to
fade back in while you're already
rendering in that case you want to start
the fade in and then go right to doing
your animations let the fade proceed on
its own not worry about it to do that
you would start off with for instance
we're starting from a solid color in
this case we want to end up with the
display looking normal the starting
color was black so we're doing a fade in
from black falls meaning asynchronous in
this case cg display sade returns
immediately the fade occurs over the
next two second
which is what we specified while you get
your GL animations up and running so you
do nice fade in your game figures are
already running around digitally
stabbing each other having a grand old
time this can actually look pretty slick
and this is easier than trying to do a
lot of the asynccallback stuff yourself
all the actual call back and work to
drive this is handled outside of your
process in the fade machinery we've
added to the system palliative displays
oh here we go oh here uses pallid
displays in their new app their coding
up good i got some news for you this
stuff in the system for legacy support
there's minimal to no hardware
acceleration available when you're using
8-bit pallets all at fancy hardware just
loves 32-bit mode it goes past then they
go down to 8-bit modus what I've got
these huge honking buses and you want me
to move eight bits at a time it's got a
low precision doesn't composite very
well at all it's barely supported by the
hardware come on guys for past the
Stephanie's we're past the 80s even we
can get better the configuration
functions will let you select an 8-bit
mode they won't let you make it a
permanent setting there is a reason for
that
anyone know ah alright we do have
palette functions for you to play with
this is for legacy support by the way I
don't know if I mentioned that you can
create a palette you can create a pellet
from the displays content you can create
a pal with an array of sample you
provide you can get the color an index
you can set the color at an index you
can tell a display here's your palate
and you can release appellate any
questions no good common developer
issues what sort of issues could
developers possibly have with this stuff
well as recognizing those mirroring and
display configuration changes if you're
playing with the display directly you
need to be aware of mirroring that old
device loop comes back to haunt you
you're going to need to look at all the
displays in the mirror ring set and make
sure you draw on all and in turn or
things can look bad if you're in
Hardware mirroring oh lucky you you just
have to worry about the active display
in the hardware mirror set most of the
newer hardware will be doing hardware
mirroring that's what you're going to
get and all the nice shiny power books
in the newer iBooks the white ibooks
that's what you're going to get those
funny cards that have to monitor
connectors on a single card both will
normally do hard remembering unless you
work really hard to make it not do it by
setting the displays different depths
explicitly don't do that unless you
really like to work at things if you're
capturing displays you must be aware of
the mirroring state to get the correct
behavior now capturing displays and
changing the display configuration some
people have been capturing the displays
just so they could change the modes and
then releasing them that looks funny the
entire screen goes black for a couple of
seconds your app comes to the foreground
even though you thought you were doing
this from some faceless UI element in
the background
looks bad don't do that please please
there is no need to capture the display
in order to reconfigure them yeah ucg
capture all displays to avoid funny
desktop layout effects if you capture
all the displays there's no desktop
showing and we won't bother telling
things on the desktop that oh yeah the
displays change sighs quick creme all
your icons in the one corner so we can
stop that behavior we can do it people
we can work together capture all the
displays please that heís display mode
changes from the rest of the system only
your app needs to worry about them in
that case display changing let's take a
look over on demo machine one and see
what this is all about I mentioned we
have some API is that let us actually
look at the display state here you can
see we can look at well there's the
origin that makes sense oh yes 1024 x
768 sure ok yeah 32 bit too deep uh-oh
look refresh rate came back LCD panel
why is that well for some arcane reason
very deep in the system I don't know why
we get back a refresh rate of 0 on LCD
panels so that's a good tip is oh you
look at must be an LCD panel new for
panther we have a new function that
returns you the display width and height
in millimeters no this is not actually
three hundred and ten millimeters this
one here is this number here is the
display ID for this particular display I
mentioned you can get information about
particular models this is vendor ID 610
I wonder who that could be and product
ID 9215 yeah display modes we have desks
and mechanisms in the system to let you
actually pick the sleigh modes up it
looks like we've got a 1024 x 1658
32-bit display suppose I wanted to let
us say try for
oh yeah there's a mullet for 640 480 I
want I go for saya no that's close 800 x
600 is the nearest mode that would
enclose that size 1024 x 768 the nearest
mode that wouldn't close this size oh
look we can get to 8-bit let's see what
happens if we apply it kind of grainy
looking in it and a compositor doesn't
care for that that much we can always be
a lot by restoring our permanent display
configuration with that one line
function and we're back now some
displays have stretched modes they can
support suppose I asked for 800 x 600 on
this one yep this displays near a
stretched mode is 1024 768 so it would
take that and try to stretch things safe
modes ah we have a safe mode this is all
the modes that are known to function for
this hardware and the graphics card
that's in here safe mode these are
options you can feed into that get me
the best mode for a particular
configuration now to get the information
about a display we needed to get a
display ID alright well let's get the
main display ID in this case we can feed
in things this returns us the physical
bounds and millimeters for the display
this returns us the pixel bounds for
that display we can ask for the current
mode that returns us that CF dictionary
rest we can pull information out of that
anything in ACF dictionary rep is
accessed by key in this case the values
that you'll get back we'll all be CF
number CF number rep and you can use
that to fetch out the value by as an
integer or as a double whatever is most
convenient for you the CF number rep
will do the right thing and as far as
converting those other API so lets you
get the vendor ID the model number very
straightforward stuff
searching for a display mode this is a
rough one this is the longest version of
the function cg display best mode for
parameters and refresh rate what
property shot that it takes a display ID
a desired width a desired height depth a
desired refresh rate an optional
property value you can just feed in null
for the property by default or if you're
looking for only safe displays so you
don't have to put up any kind of
confirmation dialog you can feed in okay
so you just label to say for hardware
and it returns you a mode the function
will always return a mode if it can't
figure out what on earth you're asking
for it will hand back the current
display mode I don't know use this it
works right now tries for a depth first
match on those now we change the
configuration I told you this is rough
stuff we found the mode with that best
mode for stuff begin configuration we
get back our configuration to handle
configure the display mode for the main
display to that mode these modes are
only valid for the display you query
daman and you don't get to make up your
own the mode will only work that's
returned by one of the API is that bends
modes or lists of modes and once we've
got things set up all we're doing a
change in the load in this case we call
CG complete display configuration
configure for appt only so if something
bad happens in the app crashes we get
back
capturing all the displays we have a new
for penser item you can specify an
option capture with no fill that
actually captures the display but leaves
the content as you see them think you
can do your fade transition it looks
like you're fading out from the desktop
really slick going to your app I'm going
to show you an example of capturing a
display doing a fade out we're going to
draw a color bars pattern and then we'll
do that asynchronous fade back in one of
the interesting things are the ways you
can start a fade back in and release
your reservation token if the fade back
in ends with a normal display if it ends
with a normal display state then
releasing that token will have no effect
on the displays appearance the system
will automatically say okay this is a
deferred release i'll release it once
the state in is done and once we're done
we're going to use release all displays
to release the displays let's capture
the display you'll see we're going to
capture it but not actually fill it and
we'll do a fade out fade back in with
our color bar pattern it's all capture
it's ours now we can do with it as we
wish now click we restored everything
that easy didn't have to record all the
display locations didn't have to try and
put them all back ourselves very
straightforward this sample code will be
appearing on our website soon in the
next few weeks
now onto something completely different
remote operations remote operations is
all the stuff you need to do remote
control over the machine hello look
display drawing activity monitor and
generation hosting events look at that
name display drawing activity monitor
let you listen for state changes on the
display oh look a window just flushed
over here oh look quick on movies
playing over in that corner stuff
happening it tells you the areas that
are changed on the display will also let
you post events into the system for
remote control purposes or for userspace
device drivers now how is all this going
to work why do you have two straight the
bits off the string why can't I go get
them somewhere else well all the apps in
the system are drawing into their own
private buffers you don't get to play
with those through the apps buffers not
your buffers the buffers all go into the
courts compositor which takes all the
bits blends them together seasons and
lightly and sends them to the frame
buffer for display some bits can be
transparent some of the bits from the
frame buffer maybe as a result of
several windows accumulated together the
only place where you can get a
consistent picture of what all the
pieces look like is at the frame buffer
there's a lot of processing that can
happen between here and there if you
looked at some of the accessibility
interfaces we have things like display
zoom that can actually occur so what you
see on the display doesn't actually
correlate very closely with what you
started with in the individual app frame
buffers and the individual app buffers
the final frame buffer product that's
what you need you pretty much have to go
to that the remote operation services is
the best way of getting these finished
bits clients live remote services
include apple remote desktop Timbuktu
the old DNC some accessibility
applications like to play with us too
they may need to post events into the
system so they'll be using the event
posting interface to post into a
particular session and tell the apps in
that session to show certain behaviors
screen scraping all I gets bits well we
have this complicated pipeline it's
really hard to get to the individual
dish from the window if you wanted to
get the bits on the screen but not go to
the screen
it'd be an awful lot of work you'd
basically we went to reinventing the
whole courts compositor and there's more
tooth and it looks
Oh to assist you in finding what's
change on the screen we give you a
system of asynchronous notifications
async means that we tell you something
has changed your app goes and picks it
up and gets the current state of what's
change on the system it's a sink so that
applications that don't behave well
don't get to hang the entire system it's
very straightforward if you have delays
in your processing delays that occur
systems system-wide scheduling or load
that cause you to be a little slow about
getting the current state back that's
not a battery problem we keep coalescing
the state changes for you they're built
up and your app will get the latest
image of whatever states on the display
there are new functions added to Panther
that tell you not about just changes but
about areas that are moving on the
screen this will help speed up certain
operations that you might get into if
you're doing screencasting good morning
now strains rating performance now that
screen is at the end of a long bus that
long pipeline so out there's a distant
end of a GP or PCI bus connector and if
you go out there and read those pixels
yourself it's going to be slow the
fastest way to read the pixels off the
screen is not obvious at all is use
OpenGL you build a full train GL context
select the screen with GRE buffer Jill
front you read the screen with GL read
pixels into your own buffer that's going
to get you DMA transfers from the screen
right into your applications buffer if
you do it right if you don't do it so
right it's still going to get you a fast
software path but it won't be as fast as
doing the DMA trick things to be aware
of GL is a little different view of the
world that you might be used if you're
used to dealing with frame buffers it
puts things in quadrant one the origin
point being down in the lower left
corner upside down now
if you're just processing the bits to
pack them up and send them across or a
screen casting application that's not
actually if that big a deal it just
means you have to walk through the
bitmap in a different order just set
your pointers up correctly and set up
your stride as you traverse the data and
you'll be all set source code showing
you how to do this is going to be online
i'm going to show you right now what
this looks like over here on one little
bitty grab application we can grab the
spring contacts go ahead and save it to
a file there we are and look it's a
screen grab well that was exciting
wasn't it can go back to sleep now but I
hooked up something interesting to this
version of grip we can time it will do
10 grabs in a row and averaged out and
see how many milliseconds it takes to go
ahead and scrape this 1024 768 32-bit
screen in software okay 140 milliseconds
that's not too bad let's try it with
OpenGL oh oh I got faster didn't it
that's more time for your application to
get the data compressed shove it over
the wire and catch the next screen
update now that I factor a ten in this
case now you won't always see that big
of a change but pretty much any of the
hardware we built in the last couple of
years this is faster than doing it and
software DMA would so how does this code
do it it's rough we set up for our main
display a mask fit we say we'd like a
full-screen context for that display we
choose a pixel format have GL create a
full-screen context get rid of the pixel
for us you don't leak set that context
to be the current context tell the
system this is full train here we go
point GL at the front buffer that is the
display ask it for the width and height
get a place to put the data we're going
to grab make sure that all the DL
commands are finished in the display is
correct and current tell the system we'd
like the data packed please call GL read
pixels with the mystical incantation
govt GRA GL unsigned integer GGG rub
these are apple extensions their present
on all Mac os10 systems so I didn't
bother checking for that first naughty
me if you ask for this pattern this will
match 32 bit deep display hardware
you'll get the DMA effect GL unsigned
int 1 555 rev the only other rev depth
in there will get you the 16-bit display
format and you'll get DMA transfers in
that case also when you're all done
release stuff that's all there is to it
less than two pages of code in really
big type to capture the string contents
and back to the slides leave
over in the event side of systems you
need to post events into the works right
this is interesting we have keyboard and
mouse events coming in from the kernel
or to the colonel from our hardware the
colonel and his system pick up those
events chew on them a little bit put
them into a nice format for us pass
those into the courts compositor what
the courts compositor knows we're all
your windows live it knows what you app
is foreground so it can be used to route
these events out to all your
applications there's a couple of ways to
push events into the system io ahead
post event is a call from user space
into the i/o kit that takes an event
structure and shoves it right in as
though we're coming from other hardware
if you are writing a user space device
driver or something that wants to
pretend to be a device driver
synthesizing events system-wide use i/o
hit post event there are also some
techniques can be just inject events at
the courts compositor level these are
actually not appropriate for userspace
device drivers they are used in some
assistive technology though these lets
you post a keyboard event in a scroll
wheel event or a mouse event in at the
course compositor level now why are
these not appropriate we have multiple
users sessions on the machine now there
can be a second user login with the
second set of windows a second window
system second set of apps a second doc
all that post keyboard event posts roll
wheel event and post mouseevent are
going to post into the session they are
called in so if you're emulating
hardware you missed got the difference
there I don't want anyone to get tripped
up on this
posting Mouse advance is pretty
straightforward CT post mouseevent you
give it a first position you tell a
boolean that tells it I want to move the
cursor to this position normally a pass
true in you pass an account of buttons
one two three and a set of boolean's for
the button state if you push a button
down don't forget to post another event
to let the button up thank you that has
a really bad effect on the system I'm
going to get to in a minute there's also
an interface CG warp cursor you can use
just to move the cursor around on the
display posting keyboard events there
are key codes which are the decrypted
glyph codes that you see in a system
they're also character I'm sorry there
are key codes oh is this terminology
fuzzy key codes are virtual key codes
their scan codes coming off the keyboard
character codes are the encoding for a
particular character a glyph the thing
you need to get right passing into this
call is the key code most environments
ignore the character code part the
virtual key code is the part you need to
pass in there's actually a dummy flag
you can pass in for the character cuz
it'll cause the system to guess at the
best key code to use with it application
environments like carbon and cocoa use
the key mapping and the key char API is
part of carbon to actually do the
translation from that raw virtual key
code into the character code modifier
flags are set by posting virtual key
codes for the modifier keys all the keys
and the people have these key codes you
pass them into this interface and it's
just like you're pushing the keys and
letting them up on the keyboard don't
forget to release the key after you
pressed it the events Depression when
you post events into the system we turn
off the hardware event pass for a little
while by default that's to avoid Mouse
Wars a student helpdesk system you want
to drive a sister for little by posting
events into it you don't want the
eight-year-old grabbing his mouse and
fighting you so we'll turn off the local
mouse for a short period of time the
force quit key combination reconnects
the local hardware things get really bad
say you goof when you're debugging
things you forget to let the mouse
button up
there are functions to override the
default behavior now I suggest you
always set the state as you want just in
case the default behavior changes in the
future the event state combining as one
of these things that combines the flags
from your keyboard modifiers from the
real keyboard with the events from the
remote system you might want to keep
this off to keep local modifier keys to
the local shift key being held down from
getting into your event the most
frequent question we get is how do we
turn this off to do that you just want
to set the suppression interval to zero
so it doesn't suppress things for any
length of time tell the system yeah I
want event combining I want my local
shift key and my remote ship heat all
act the same and you'll want to tell the
system don't bother blocking local
hardware events permit all events during
the suppression interval and permit all
local events during a remote mouse drag
a tells you you had to let that mouse
button up if you leave that mouse button
down you're doing a drag operation and
this is will normally block all local
events during that drag to keep things
from going horribly astray if you don't
post that mouse up you're sticking in a
drag state bad don't do that
common developer problems questions we
run into I want to filter all the events
in the system well you don't do that at
the courts level that's the kind of
thing that goes Ethan I oak it or you
might want to think about what you're
doing what does you really need to
accomplish for example some of the
questions we've gotten on this our
operations that are better handled as
input managers in the system then as a
generic low-level I filter every event
and pick out the ones I'm interested in
sort of operations ah another little
common for people only use the functions
in the public headers undocumented
interfaces may change they may go away
they may stop working undocumented
parameters undocumented functions may do
undocumented things to the system you
may let the smoke out of some parts the
stuff in the public headers is actually
better when we do our development we
often build things versus a private FB I
and then once we figured out all the
quirks and what we've gotten our act
together we turn it into a public
interface please use the public
interfaces who here is calling a private
interface any volunteers yeah yes yes
I'll change that for you
I'll take care of that one and for more
information let's get Travis up here
[Applause]
thank you Mike I think we have some
reference documentation available for
you guys we're after you going to
aggregate it's all aggregated on sort of
reorganized ADC website where tech pubs
is move the documentation from where it
was before we began w WC so everything
should be you know documentation should
be appearing on a lot of stuff into the
graphs and imaging area on the revised
ad see website we also have some tech
notes that are available and so it's
gonna be taken up to a seven it talks
about the GG direct display API and then
well probably very many people won't be
going to too late to cg direct solid api
considering how unpopular eight that
palliative tech is actually becoming and
should become we have some additional QA
so such as many relating to patrol and
see how to use it if I'm a carbon coral
app and I'm you need to use cg direct
display you know several q and A's up
here they're available and we have lots
of sample codes that are going to be
appearing I don't think all of these are
up quite yet but check the ADC website
we plan to push a lot of samples that
you saw on today's demo out very shortly
and then obviously if you need more
information on how to use GL + GL read
pixels you can look at a couple
different GL references pry it open GL
doubt at work which is a good resource
for OpenGL in general
you