WWDC2004 Session 203

Transcript

Kind: captions
Language: en
good afternoon welcome to the second day
of www DC my name is Derek Clegg and I'm
a course engineer I've been working in
courts for quite some time now hence my
title today I'm going to talk about the
new things that we've added to Tiger to
help you as a developer do more cool
stuff I'm in addition we're going to
have my colleague John berkey come up
and for the second half to show you more
about courts to D and how it's actually
leveraging the quartz extreme part of
the system to do even more cool stuff
and then along the way we're going to
have a number of demos so the first
thing that you of course need to
remember as we've emphasized already in
other sessions is that our focus on Mac
oh and Mac os10 is quartz 2d it's no
longer quick draw cord draw is a great
technology service really well for 20
years its crystals going dark it's time
to sort of move to courts 2d so we
really want you to think toward doing
that if you haven't already and of
course if you have done that we feel
they want to continue supporting you and
making more and more stuff part of
course duty to help you out there's a
special session on friday at two
transitioning the court CD which will go
into this in more depth and help those
of you who haven't already started to
transition make the transition now i'm
also not going to talk today too much
about the details of court city the
architecture the basic type tell all
that fits together there's a session
that i gave last year WWDC that talks a
lot about all of that information so you
can certainly pick up the DVD and check
that out and in addition prompts more
importantly this year we've added a lot
of really great documentation there's
some brand new stuff that's all about
drawing the courts 2d the transition of
course duty there's a full reference of
all the api's and there's a lot more so
if you have any issues like that sort of
basic understanding we really urge you
to go to the documentation it's great
and of course check out the DVD last
year as well the my focus today is going
to be all the new api's we've added and
tiger sort of the new things that are
coming down the road that we can use
that you can expect to start using now
just though to give you some context we
have a cg
text is the basic type in court CD
that's what everything uses to do
drawing so TG context is sort of the
coin of the realm in a certain way it's
the it's by nature by its architecture
device-independent what that means is
that you use a CG context if you're
talking to a printer or to the screen or
to a bitmap so the same basic type for
all of those things so it's sort of
abstract the drawing environment for you
in addition very importantly it's a
resolution independent you zach lee ji
context with the same API to draw to a
you know 100 DPI screen or a 1600 DPI
printer and in addition it's staple it
maintain state for you it has what we
call the graphic state which is things
like the the color what color you're
going to fill shape in or whether you're
lying is dashed or not things like that
and most of the API is in courts take a
CG context and work with that so what
are the new API is that we're going to
be adding for tiger well we're going to
be some new drawing api's that work with
CG context directly a new thing at layer
API which is a sort of a more efficient
representation of off-screen bitmap
drawing and some new PDF input on
outside api's for those of you who are
used to using those ok so this is sort
of the list of what I'm going to cover
for the drawing API is a lot of things
so let's just go into each one
independently so the first is a new
clipping API the idea here very
straightforward concept you have a mask
of soft mask of some sort and you want
to use that as a clipping region in the
past and Panther clipping was always
sort of path based you could clip to a
circle or clip two rectangles or to a
much more complex path but it wasn't
very easy to do a clip directly to a
mask so we've added API in Tiger to let
you clip to a mask and it works just
like every other clothing region
clipping in court it's intersected with
a current clipping region a base G saved
you restore semantics so in that respect
is identical it's just that now you can
sort of get the power of a soft mask for
your drawing operations in addition to
sort of the standard clip half concept
that we've already had so we think this
will be very useful for certain people
who need to be able to do this type of
operation
we've also added some new paths api's
and the idea the sort of the overarching
theme of some of the things going to be
talking about is make simple things
simple course is a very very powerful
API it has lots of things available in
it but the problem is that sometimes a
little bit the water gets deep really
fast it's hard to know where to start
sometimes we've had people say well I
want to draw a circle AB you learn about
bebier pads and cubic equations I don't
know what to do about that so that's a
bad thing we want to make it easy for
people to do simple things so the one of
the goals for some of the api's you've
added to make simple things even simpler
so for tiger we're adding api's let you
do circles just directly without having
to sort of worried about bez a path and
so on of course the lip Caesar just a
variation of a circle we've also added
api's let you do lots of lines strokes
very very rapidly and you'll see a very
good demo of how that used a little bit
later when john comes up to talk but of
course we want to keep simple things
simple but we also don't want to neglect
people who have more advanced needs and
so one of the things we've added for
have is hit testing so you can find out
if a point is inside a path so you can
do you know we want to drag oh yay and
also those are some of the advanced
developer and also we want to be able to
allow you to replace the current path
with a stroke version of the path again
for hit testing or for special shading
stuff like that now again it's along the
same theme of keeping simple things
simple we've added some api's to let you
get access to some of the basic
fundamental color pink color spaces the
in color sink has sort of three
fundamental color spaces the generic
gray generic RGB generic CMYK and it
used to be and Panther sort of difficult
to get access to these you'd have to
sort of go to color sync and muck around
and color saying come back to core
graphics muck around in core graphics
and after you've done all that mucking
around you finally oh look up color
space so what we've done is made it very
very simple to get access to the color
space without having to go through all
of that sort of gyration and that way
you get you can guarantee that you can
easily get a color space that gives you
high fidelity color everywhere
now that's important because our future
direction is high-quality color high
fidelity color everywhere across all
devices we are no longer trying to model
sort of the device idea where you get a
different color if you're printing
versus a different color versus on your
monitor we want to make sure that you
get a high quality match color
everywhere you go to that end we want
you to avoid using device color spaces
and to help you with that we're actually
going to replace the use of device color
spaces automatically for you with the
generic RGB genetic gray and so on so
that even if you don't care too much
about color and a lot of people sort of
are willing to sort of let the system
handle a lot of that up for them you'll
still get the same results on all
monitors on all printers without
actually having to do anything very
difficult yourself now an example of how
easy it is though to transition to use
the sort of not use device color but
instead use some more specific color
spaces in the past you might use the
function on cg color space create device
RGB by its name it's a device color
space the results will look different on
different output devices that's a bad
way don't do that anymore the good way
the tiger way is to call CG color space
create with name and you pass in the
name of the color space in this case
generic RGB so it's very simple not a
lot of code changes not much difference
and what you end up with is a color that
will look the same across all platforms
so this is definitely a direction we're
headed and so we want you to be able to
come along with us and that for image
api's we've added a number of new ways
to do image drawing and work with images
and so I'll go into each one of these
independently so the first is chroma key
masking this is you know some people
call this blue screen it's a very
straightforward idea you have someone
stand in front of a blue screen and here
we have two men shaking hands and then
when you display it you subtract out the
blue color and everything that was blue
becomes transparent so this is something
that you know everybody sort of is
familiar with from TV and so on and so
we have what we've added intent and
fusing and tiger is a simple way to
create an image and specify a key
color the color you want to make
transparent and end up with an image
works just like every other image which
has those parts transparent the function
of CG image create with masking colors
and it's a very easy way to get
chromakey masking for your images if you
will in addition this is something that
falls in the category making simple
things simple it's a very obvious idea
assistant to the previous example we
have a clipping GG image create with
mask lets you take an image and apply a
mass to it a soft mask any depth that
sort currently supported in with image
masks and what you get out as a mask
image so this is something that you know
maybe some people are familiar with from
from quick-draw it's definitely
something that took a lot more code too
much code to do and Panther so for tiger
we've added a single function one shot
to GG image crate with mask and you get
exactly this effect very powerful but
very simple as well good reading it for
you another another API to help work
with images in certain cases particular
cases is the idea of a sub image of an
image this is a pretty common idea if
you maybe have cast some complicated
drawing and a single bitmap and perhaps
you want to use that different regions
of that didn't that four different
drawing as you're doing your work and
here we imagine an example where
someone's drawn a plot maybe it's for
their cash and they have you know nine
letters and what they want to do is sort
of pick out each one independently so
there's a now new function cg image
create with Imogen wreck you start out
with an image and you choose a rectangle
and what you end up with is an image
that works just like every other imaging
course it's just a piece of that book
parent image the nice thing is that all
these some images sort of share the same
parent you're not making copies of all
the data you can keep it all in one
place so that's a very powerful API for
those cases and then finally with images
again this is something that we really
want to make much easier bitmap context
we told people in courts 2d if you want
to do a screen drawing if you want to do
some sort of caching like that use a bit
of context later we'll tell you about
something new that's even better than up
Conte
but this low cases where it's
appropriate to use a bit of context and
you do your drawing is a bit enough
context and we just sort of you know
blissfully say oh and then create an
image from it and draw with that image
and it turns out that's a lot of code
it's not so simple and particularly it's
not so simple to make sure the color
spaces match your destination and all
that sort stuff so what we've done for
tiger is added one function that does
all that work for you cg bitmap context
create image you pass in a Content
bitmap context and out pops an image
from it so it's very simple now to sort
of do what we've always just wave our
hands about in the past this is a very
powerful function but the thing you need
to remember is that it is a copy
operation now that's not as bad as it
sounds I'll tell you why in a second but
the point is that it's a copy operation
because the CG images must be immutable
we expect that an image will be reused
or rather excuse me we expect that a CG
image will be uploaded to video RAM once
and then every time that image is drawn
multiple times we don't re-upload it
that saves a huge amount of traffic
through the bus to the into the video
card if your images has changed then we
can't do that we have to re-upload every
single time that's really inefficient so
we are enforcing it's always been true
in the past that we are definitely
enforcing now the concept that a CG
image must be immutable so that's
something that's very important for you
to check out in your application make
sure that you're not doing something
where your changes the context of an
image because if you start doing that
then your hardware acceleration will
really suffer now when I say it's a copy
operation it's not really a copy it's a
copy on write it uses the you know vm
magic of mach to do a copy on write so
you don't actually really make us a copy
instead you have two things that point
to the same in memory destination and if
anybody writes on one of them then a
copy happens but until something is
written than nothing bet you don't get a
copy you have you supposed to show the
same underlying data structures so
that's really useful because you can
sort of create an image from a bit
enough context to draw it somewhere
release it and you actually have never
made a true copy another thing that we
wanted we're adding for images again
sort of more of it
the advanced category it's what we call
deep pixel support this is deep in the
sense that it's now floating point
pixels for both image content and for
bitmap context so we already support in
Panther the standard image integer
formats you know 12 well actually up to
32 1 to 32 bits per pixel sorry bits per
component for Panther we're now letting
you specify a 32-bit floating point
value and the floating point value is
can be you know you have red green blue
alpha as your floating point value as
you can see Mike a floating point pretty
much any support anything that we
currently support an integer we also
support and floating point same thing
for output for bitmap context you can
create a floating point bitmap context
you might use now excuse me the the
advantage of this is that now for image
formats that are starting to come online
where you're actually bringing in you
know more of these higher resolution
images we now can support those natively
now the other thing that's important to
note is that this is not sort of high
dynamic range imaging that's a very
specialized thing it's used in its sort
of more color seeing color science type
of thing to sort of figure out how to
map your big floating point range just
sort of visible that's a problem that's
best left to the colors think guys is a
special session on that where they talk
about that more in depth what we do is
something very simple we just take the
values between 0 and 1 and that becomes
the image anything less than 0 we clamp
20 anything greater than 1 we clamp 21
so we're not actually mucking around
with the data ourselves we let you do
that or coercing to that other parts of
the system that are more capable
excusing now another thing that we have
had sort of missing in the past is image
I 0 input and output of image data as
you know some of you we've been able to
support as input jpg natively PNG
natively in two courts 2d raw image data
is sort of a big buffer of red green
blue alpha data not encoded in a
particular file format for output we
haven't had a way to do any image
outputs of course we've been able output
PDF files which are tremendously
rich and complicated but for something
like a simple jpg output nothing so for
Tiger we want to remedy that and Tiger
we're going to add support for jpg PNG
gif gifs a 50,000 a bunch of other
formats supported by quicktime and other
formats as we need of course i'll
obviously as well the floating-point
types subscribe to see deploying point
formats such as openexr and the
floating-point version of TIFF for
output and the WWDC build we support
jpeg and tiff output and we're going to
certainly add more of those to come for
the final Tiger release there's going to
be a session that talks about some of
this and combines it with a previous
topic of high dynamic range on looks
like on Wednesday so if you're
interested in more details about that
it's only go to that session I'm going
to go a little bit into sort of an
overview of the api's because we cover
them briefly so there are two types
there's a way to get images in a way to
get images out the wedding at image is
in is the cg image source that brings
data into the system and some particular
file format we decode it for you and
draw it so you can bring images
obviously files we've got a JPEG file a
URL a raw data buffer similar to what we
already support just a big wad of JPEG
and our tip or whatever that you might
have mapped into memory and of course we
support the standard 2g data provider
which is the way you get information
into the courts 2d typically what an
image source does is it creates EG
images so you create an image source
from a file and out pops a teaching
image now not all images file formats
are just single images some are multiple
like jiff files have animations we will
have multiple images for that just files
may have different images for different
resolutions so we support from an image
source you can also sort of get all of
the images in the image itself in the in
the file format the file data itself
plus you can get access to the metadata
so if your camera were to embed its make
in the file then we would be able to
return oh this is a canon camera or if
it had longitude and latitude we could
return that so you can't get access to
the meditative very easily as well but
the most important point about the cgm
is force and what it creates is that
this will be the best and fastest
and the primary way we're going to be
supporting to draw images and tiger that
means all parts of the system are going
to go through this we're putting lots
and lots of emphasis on optimizing
making it you can do in all those
altivec tricks and so on and so it's
very important that this be the primary
way we get images drawn and they'll
leave the bat best and fastest and so
it's worth looking at from that
perspective of course the higher-level
frameworks will also adopt this and sort
of use it indirectly but if you're sort
of working at the courts to lee level
it's definitely worth looking into for
output sort of the opposite basically
the same you can output to a file to URL
to raw data buffer a memory you have to
put a CG data consumer which is a
standard way that you can output two
things in courts 2d and similarly to the
input you when you create an image
destination you supply cg images and you
can supply metadata and of course your
image format supports multiple images
you can supply multiple images and write
out the results so then you can write
out a tiff file from cg or you can write
out our ports to d you can write out of
jpg and so on so just to give you a
quick demo I have to say it fronted the
this demo like all types of demos like
this is a little bit of trivial because
we're supporting a new file format hard
to demo that tremendously well but I'll
do what I can we have a simple app here
and notice that here we have a tip file
and oh this is Panther we can't drop it
in because CG cork 2d doesn't support
TIFF so now we're going to turn on image
I oh and now the TIFF file is accepted
ray so that's very important the power
of image I oh now just as an additional
thing this is a simple example of
pulling up the metadata from this file
in this case it has a bunch of fields
things like the height in pixels and so
on the file size the color model and so
on there's a lot of information like
that that will be embedded in and images
that you can pull out again the very
trivial example of getting the metadata
there just to tie it into the previous
topic
about color spaces notice that this in
this image this image doesn't actually
have a color space associated with it
we're not actually we're not using the
closest place but when we start using
generic RGB it doesn't change why
because in Tiger we're automatically
making sure that anything that's
untagged gets tagged with generic RGB so
you get the same fidelity across all
platforms and then just as a simple
example of to prove that there really is
something going on this is a specialized
color space which swaps blue and red and
as you can see now you have a very odd
maybe sick ladybug ok so that's the demo
with that ok so what else have we added
let me get some water and I'll tell you
ok another thing that's very important
and is very powerful is of course the
resolution independence in quartz 2d the
fact that you know any resolution any
output any scale factor looks just the
same looks just as good as other scale
factors now it's not surprising that of
course you know down the road we expect
monitors to change in resolution we also
know that people who are currently
disabled use resolution to scale factors
to do too sumup their UI so they get
some better ability to look at the
content and so on and the thing that we
are very very focused on is making sure
that in all those cases it still looks
great that your app still looks just as
beautiful it does today on the current
monitors that's an apparent very
important thing now that means that you
need to actually sort of play with the
idea that your application may be scaled
may need to zoom up and so on what that
means that have been at a technical
level is that will set up the context
for you the ctm will be set up so that
the scale factor will be built in to the
context so you don't actually have to
worry about that too much but that means
you do have to worry about it in that
you can't undo that if you undo it and
something you're not drawing to the
right scale factor you're drawing at
some other scale factor and everything's
going to look like you know bad things
on the screen
what that means is that we don't want
you to sort of undo the transformation
by this is a little bit technical but
we're taking the ctm and inverting and
caddington counting it back to pretend
like you're an identity don't do that if
you understand what I'm saying if you
don't good and instead we really think
that's important to consistently ug
seiji restore that's a standard way to
save the graphic state restore the
graphic state we've spent a lot of
effort in tiger optimizing the G saves
you mr. operation so it's very cheap and
so we think that that's the right way to
do and it sort of will save you once we
do get down to the road where we are
still going to start scaling the
applications and now all that said there
are still some cases where you really do
need to know what pixels am i touching
you want to draw a thin line for example
you don't want it to straddle a pixel
boundary or you need something do
exactly a blood against something else
those things do happen where you sort of
need that and help you with those cases
when you may have a scaled-up user
interface we have sort of a big workers
function cg contacts get user space
through the divide space transform that
is the full transform from the current
user space that you're drawing in all
the way down to the pickles to the
actual pixels that you're working with
the device space now often you don't
need something so heavyweight so we have
some convenient functions a function
that will take a point in userspace
convert it to divide space two pixels
take that point in pixel space to put it
back to user space and we have similar
functions for size and rectangles as an
example a simple example of how you
might use that here we have somebody who
wants to do some sort of specialized
rounding in device place where they
really want to round to the middle of
the pixel or something like that so they
have a point in user space they call CG
context convert point to the x space
that converts that point through the
current transformation matrix all the
way down to pixels they do their
alignment the way they want and then
they take that point convert it back to
user space and now when they draw with
it they know that what will come out
will be precisely aligned so for those
special cases worth important this is a
very powerful API and general though we
think that you'll be just fine using the
standard courts to do things for
scalable you I but
cases where it's necessary this will
help you out now let me give you a quick
demo of scalable UI and how it works
with apps so we may have seen this
yesterday in Peters talk so we have many
of you are familiar with quartz debug
very powerful tool for development and
we have now a user interface resolution
slider so you can change the slider and
you'll get a different resolution for
your application so for example here we
might let's say we'll go up to two times
and let's launch Safari they see so far
is twice as big now you can do this with
your app to you can take your app change
this slider and see what happens when
you show up when you bring it up on
screen it should have brought up
apple.com maybe they're announcing
something else here knows but the the
point here is that notice how Safari
looks really good it's still as
high-quality as it was when it was at a
lower resolution the text looks great
all the drawing and someone looks great
so that's this is something that will
help you out to make sure that your
application doesn't have any problems
with rendering as you can see there are
some issues like up in the upper
right-hand corner clearly I've changed
the resolution but the nobody told the
clock or the sound bar and I would show
you some other apps where it doesn't
look this good at all but that would be
embarrassing so I won't so make sure
your app is not one of those let's let's
see Safari is not quitting I'll put it
here and the other thing to remember
make sure that you quit the ok we can
push back
make sure you quit that app otherwise
everything will launch too big and
that's difficult to read okay so that's
sort of a lot of the things we've added
so far for for drawing for new ways to
do drawing you better ways to work with
CG context little ways network with
images and paths and so on so now we're
going to talk about a layer which is a
new thing we're adding for tiger to help
out it's really designed to sort of help
optimize the traditional case of where
you want to do off-screen rendering but
you still want it to be hardware
accelerated and the past what we've done
is said oh if you want to do some sort
of off-screen rendering just create a
bit of I'm context draw into that and
then take the result and composite to
your destination that's a good way to do
caching and it's very powerful and very
straightforward thing to do the one
problem though is that the data always
lives on your side of the of the bus
essentially right it's all sort of in
your address space so any time out
drawing you do there which you then try
to draw on to say the video card will
have to be uploaded that's not efficient
but and by design by the bitmap kind of
design it's sort of limited to that so
digi layers are sort of an optimized
version of sort of a better way to work
with sort of do this type of you know
bitmap context type caching now the the
key thing too is that in addition to
just sort of being a convenient way to
do to sort of do off-screen rendering
will when you create it you specify a
reference to another context and what
we'll do is we'll set up set up the
layer that you that we create for you so
that sort of matches the the destination
context for example it will match the
color space so that means you don't
really need to know about the color
spaces of the context you're working
with for example of screen color space
and so on instead we'll sort of create
that set that up for you so when you
draw into the lair what will happen is
that it will already be matched to that
color space so when it's actually
finally used you don't have to a
specialist separate match so it makes
sure that things stay highly efficient
and you can sort of think of it it's
really in many ways sort of like you
know as this example shows just like a
rubber stamp on some sense you can take
the layer you draw into it and then you
can sort of use that over and over and
over here
sample we created a layer or drawn the
butterfly into it and now we can just
use that and what's great is if the
butterfly sorry if the layer is
hardware-accelerated so that you it's
already up on the card when that stamp
happens as all none of that comes back
over to the CPU and so those all done in
the GPU so that's very very efficient
and very high-powered for those types of
cases where you need to do sort of
off-screen caching and you don't want to
have things sort of live on your side of
the the the bus does it work so like I
mentioned we sort of imagine that one
common use probably the majority will be
cashing so you would use a layer instead
of a bitmap context for off-screen
caching another one which may be a
little bit less likely is sort of a
buffering where you might be in the
middle of the screen update on this side
so you can sort of go ahead and start
drawing into a layer so that the next
time you get a chance to composite to
the destination you you you're ready to
go that might be a maybe a little bit
less common use so the short demo will
go back to my little app so here we have
our app again and here's a PDF file or a
nice PDF file but it's got a lot of data
and it's a slow thing to render now if I
wanted to animate that I would be fired
because this is not really animation
this is sort of chunky blob eNOS so
instead of course obviously what you
want to do in Panther we would have said
oh well to solve this problem create a
bitmap context very good solves the
problem for Panther the problem though
is of course that's not hardware
accelerated for tiger you can create a
layer and what this does is will draw
the PDF file into the layer and then the
layer is now hardware accelerated so
what we do animation it's going to be
completely smooth it's all however
accelerated everything looks beautiful
now the thing to remember of course is
that it's still in some sense bits so if
you think about it it's not the original
high resolution PDF file that if you
scale up you'll get you know perfectly
beautiful results at any scale factor
you will start seeing you know perhaps
some sort of image
artifact is you if you were to scale up
the this particular layer because it is
this but in many cases for something
like this that doesn't make any
difference you don't care about that so
much you just simply want to have
high-quality fast efficient rendering
their Safari still okay so that's that
demo so that's a lot of the of the api's
for both drawing content for high
efficiency off-screen rendering and now
we wanted to also added some stuff for
PDF support as you know many the PDF is
the metadata file format in court and
Mac os10 it's our high fidelity
rendering of drawn content it's a great
format it works really well for us the
problem is that sometimes people have
trouble sort of both in the creation
slide they want to do more things and on
the sort of input side where they want
to really look and introspect more into
the pef itself so that it's an API to
help do that so let's go through those
the first is links an acre is very
comparable to HTML again it works just
for the PDF context that is able to
record this information and you can do
sort of the basic stuff you can do with
links and anchors you can specify a URL
and say well when somebody clicks on
that point in the PDF page 17 I want you
to open up this URL in safari so you can
do that and then of course the anchor
type model where you can say well if I
click here on page 47 go back to page
eight I'll go forward to page 75 I think
you can sort of do forward and backward
references and it's very simple API it's
very easy to use but it gives you a lot
of power to sort of preserve that and of
course the far i will be using this and
tiger to preserve links and anchors when
they actually go to print from safari
in addition for creation a lot of times
people have wanted to create encrypted
PDF this follows the PDF specification
put a published by Adobe works with a
PDF context and at creation time you
specify a password or more than one
password and the permissions associated
with the passwords so you can sort of
say well for this password don't let
them print the file or don't let them
copy anything from the file for the WWDC
build we support 40-bit encryption and
of course we expect to extend that up to
i think 128 which is part of the people
to standard in PDF for tiger okay so
that sort of output so new things you do
without but for input the what we've
added for Panther was something really
powerful it lets you sort of introspect
about the document structure a PDF file
anything about it was a giant tree of of
objects and you can use the API is that
available in Panther to sort of walk
through that tree and look at all the
content the one problem is that the real
meat of the file the actual content
stream for the page would said you know
draw the circle here and put this text
here and draw this circle here and put
this text over here it was just sort of
the big black box you could look at the
content you could print it out but you
couldn't actually interpret all the
pieces of it unless you wrote a park to
yourself a little bit left some I don't
know a little bit less attractive for
most people and in fact the it's not
something you can just sort of read
either and sort of figure out exactly
what it what it element so for Tiger
what we've added is a new way to take
the content stream itself and parse
through it and call your functions back
for each operator your is interested in
so that way you can really take for this
page look at all the pieces of the PDF
file and get essentially sort of blow up
in the whole file so you can look at
every bit in the file itself this good
glad you like it this is it's very easy
to use the idea here is that that for
each operator you're interested in you
supply a callback function you don't you
can do it for none you can do it for all
of them and we'll just run through it
through the content stream itself and
call you back here's an example again a
very simple example the idea here is you
create an operator
able where you specify both the callback
function and the operator you're
interested in in this case we're
imagining that someone wants to look at
the BMC operator the begin marked
content operator well that's why they
just want to do that you create the
content stream of course from the page
that's that's new as well where you get
the content stream out of the page and
create a CG PDF scanner that takes a
content stream that you're interested in
the operator table with your callbacks
and an optional info parameter and you
call CG PDF scanner scan that will parse
through all the PDF do all the mucky
work inside of parsing and so on and
call back your function so each time it
sees a BMC operator called you or the
arguments on the stack for the BMC off
for the sort of the BMC op that you're
looking at you can do what you might
want to do maybe you're counting the
number that times they appear so like
that when that function returns in CG
PDF scanner scan returns the full
context cream has been parsed so now
you're done you release the scanner you
clean up after yourself and you now have
the information you want so it's
actually very simple used and require a
lot of work and you can do really
powerful things with it for example this
is what we use inside of courts GD to do
both all of our PDF rendering and the
text extraction zone that's part of
previews so we use this in two very
different ways but again it's very
powerful for with just very simple API
in addition for an addition for Panther
we start excusing for tiger what am I
saying we we now have a new thing called
PDF kit the special session devoted to
it is a little bit higher level it's
sort of a way to it's actually what
preview uses now to sort of do all of
its PDF rendering and it lets you sort
of do things like you know do to up
drawing to sort of one up drawing do
text selection so on all in your own
application so it's sort of a cover for
a lot of complicated functions the
preview used to only be able to do but
now you'll be able to do with PDF kit so
if that is something you want to do if
you want to bring PDF into your own
application a lot of people do select
text selection searching all that type
of thing within your own application and
it's definitely worth attending the
session right after this session in a
different room okay so this is a lot of
stuff that we've added
for tiger we've added some both easy to
use and some advanced pass AP is of
course some new colors may see color
space api's lots of image support
various ways to work with images
floating point images of course as well
a floating point bitmap context a better
way to get an image data in and out of
quartz 2d new layer support for a
high-efficiency high quality off-screen
rendering and blue and and also a bunch
of functions for PDF input and output so
that's sort of the new API is in Tiger
the new things you can work with and now
John Burke is going to come up here and
talk about courts to the going extreme
and what what that means for you hi
everybody I'm John berkey i'm here to
talk to you about courts Chiddy and
quartz extreme so before I start I want
to sort of give an idea what we would be
talking about I want to talk about the
architecture enough that you get an idea
of what we're doing so you sort of know
what to watch for when you're doing your
apps and then we're going to do some
demos so you'll see that it's real and a
lot of interesting different ways and
then finally we'll be talking about
rules of the road we're ending with that
because there's a lot of very specific
things we need you to do so that your
apps really really Rock so let's get to
it so of course extreme for Tiger the
big change is that 2d is now hardware
accelerated the thing for us that was
important was that we need to maintain
courses very very high quality but we
wanted to speed things up a lot we use
the GP when we can we use the CPU when
we need to so the big thing that we did
to start with before we began this
project was we analyze a lot of
applications what we did was we made
sure that the things that the
applications do we worked on first and
what you'll find is you get your apps
going is that basically all the things
that your typical application does are
all in hardware and on GPU so you'll see
a lot of speed
and that's where it will be calling out
as we talk about specific operation and
then the less common operations things
like stroke lines with dashes things
like that that are less common will be
accelerating later if at all because
those things will be coming up very very
rarely and importantly why say
celebrated that I'm talking about on
hardware the other thing we're doing
though which items really exciting is
you'll find that software performance is
actually quite a bit higher and tiger as
well and the reason for that is that as
we've optimized the pipeline for
hardware we found a lot of opportunities
to continue to optimize the software
pipeline to so I'm very excited about
that and wheres noting about that all
the numbers you'll have on screen here
will be tiger software versus hardware
you'll find that even we saw four
numbers as I'm saying they're quite a
bit higher so so here's our course
extreme architectural diagram and I want
to just point out a couple things just
to sort of set the idea of what we're
talking about we remember that the three
big things we do in the course extreme
engine are we deliver a few d.video and
3d to the screen we use the GPU and then
we see this word surfaces so we use
Hardware surfaces to deliver video and
3d so focusing in on 2d there's just a
few main concepts so first of all the
application as you know renders into the
window we call the backing store the
bitmap for the window and then it says
flush so when we say flush what we mean
is we talked to quartz extreme the
course compositor and it flushes the
content to this to the screen so what
that does is it actually talks to OpenGL
and OpenGL execute the flesh in behalf
of quartz extreme so focusing in men
this is the key point here is that
course 2d up until Tiger was using
software was running with the CPU so we
were still very limited by the CPUs
performance our CPUs are really really
great today we all know
but still compared to the GPUs are more
limited then the great thing of our
courts extreme was that we use the DMA
engine on the graphics cards to do an
asynchronous VMA pull of the bits across
to the GPU and that's been great because
we get a synchronous behavior and the
GPU is really good at this so we get a
lot of speed as you know it's quartz
extreme so that was awesome but there
was more to do the first thing that
Peter alluded to in his presentation is
that GPUs have become this other animal
GPUs are now these amazing things that
can do these core image effects no blurs
all these different things and 2d has
been isolated from that part of the
capability because we've been using it
as a blitar to get our content to the
video screen the other thing is is that
window back towards are very big so even
if the CPU was capable of writing those
pixels into this window backing store
very very quickly we're still to shovel
those bits across the bus to the screen
so there's still a lot of work to be
done and then last thing is that the
window actually is a synchronization
bottleneck because while the CPU is
shoveling the bits into the window
remember where DMA flushing from that
same bucket to the screen and so so we
don't have tearing we they have to wait
for each other that's what we talked
about we talked about over flushing so
we'll see that in each of these cases we
have solutions so as I've been saying
it's all about bison BAM list we're
software rendering here and this is a
great number the g5 really really rock
but it's 5 gigabytes per second and as
we know there are other things for the
CPU to do on the system so and also i'm
using these big blocks because we're
talking about a lot of data window back
stores are huge and we using to talk to
render is a lot to do here we are with
the hardware DMA flushing it's 2.1
gigabytes per second but still we're
shoveling big black says he knows a lot
to do here and finally the bottleneck in
the window vacuum store here's the key
point look at that 30 gigabytes per
second and climbing the charts are all
you know point in the same direction
here you've seen
in previous presentations so this is our
key we want to focus on that that's our
deliverer so here's the punch line then
this is our plan right we used want to
stop using the cpu and we want to move
towards using the GPU GPU as peter
showed in this presentation has a lot of
pipes it's very very wide it's very good
at this and importantly by using the two
together we can keep the GPU as busy as
possible the GPU is a single purpose
piece of equipment for doing graphics so
by keeping the CP is that pipeline we
can keep it really busy so what does
this mean it means the backing store is
no longer in DRAM so where does it go
goes to the graphics chip OpenGL we work
with OpenGL to deliver things to the
hardware and this gets pretty
interesting here so there we go I'm not
going to clicking OpenGL commands are
very small hence the small purple boxes
that's the key point here for this part
is that we're delivering a lot less
fights across the bus and this is a very
optimal stream the OpenGL guys have
worked very hard as you know our gaming
engines today deliver millions of
polygons so what we're talking about
here we use in a very tight stream and
then uploads this is important we'll be
focusing on issues this issue a lot as
we go forward in the presentation
uploads are very infrequent and we will
be working together to ensure that
they're much smaller than window backing
stores very key point right the aqua
artwork and some would fill a quarter of
a screen but we fill the whole vacuum
store with it so that's the point as we
upload the artwork cross and reuse it we
have less to do there then finally this
is great back to the flushing bottleneck
with the window backing store on the GPU
the flushes are occurring all on
hardware running to hardware with the
small commands and then all the flushes
are on hardware so that whole flushing
biomech
goes poof so that's really really great
as the key to our performance so how
does it look like looks pretty good it's
pretty amazing to me I work with this
stuff every day but I'm always astounded
by how much performance you can get from
the system what's amazing is this is
still CPU limited as fast as I can
shovel stuff with the present day she'd
be used i can still travel faster in the
future and what you'll see then as you
test it yourselves as you'll see that
performance will scale more in hardware
as you use your cpu gets faster so we
have numbers where the Lions performance
scaled between my powerbook and the g5
number that's here is scalable like
three or 4x when I went to the g5 and
only scale by 2x this sort of the CPU
normal number so I think it's really
interesting I want to call it a couple
things this is about still right here
the point here is is that with the big
rectangle we're not limited by so rate
still so we're really really fast that's
important because this is what we do
first and we render a window you know we
fill it with stuff so that's important
text rendering is very important there's
a lot of discussion on the net about
that and I just want to call it out
we're really working hard on this we
actually spend a lot of man might sign
making this great and we're basically
almost breaking 5 million here so that's
really great then Lions this is
something we'll be talking about today
in detail we have a new API to really
make that easier for you to rather live
a lot of line performance on your
machine so sort of defocusing here
stepping back one step here's PDF
rendering which is a user of the core
primitives and what you see here is is
that there's two documents here in two
different ways of running the same
documents on the top of the screen is a
core graphics benchmark and we're
getting above 2 x 2.2 x with that we're
getting the same values below they're
about twice as fast a little bit less
but i'm quite a bit slower when it's in
preview that's okay because preview is
an application it has to do an
patience and stuff you know some of us
are speed freaks but there's a lot of
really great value that's delivered by
the higher and apps so it's not always
about that also wanted to point out that
on the run remember so anyway
application resizing this is a very
interesting slide for those who are
focusing in on the numbers so see the
red this is something that we're using
as a reminder for you when we go into
the rules of road section so we have
lots of speed here but it's sort of
burying and sometimes the blue line is
the hard work number is actually slower
so this is a thing if some of these
applications aren't cashing their image
rest still we all work together at Apple
but we're all you know still pitching in
we're not shipping Tiger yet so we
thought it was great to show you some of
these cases where our apps are slower to
and to call this out and I'll have a
demo to explain this to so again you see
the number on the bottom text edit is
crazy fasted its frames per second they
just have to turn off being sink and
your monitors to even take this
measurement because it's just a big
flickery craziness I'll show you that in
a minute spree neat so the key is as we
move forward you'll start to experience
more and more of this performance
because as I showed you the benchmarks
the poor primitives you're actually
using or are fast so this is great I
think this means we can deliver and this
is my experience in bringing it apps up
and looking around the system and what
their wild nekton we should experience
2x across the board with GUI performance
for tiger
so uh yeah kind of demo machine not that
one okay so for the first one I'm just
going to show you a little finder
performance that's pretty fast it looks
good and then here's a little make sure
yeah good there's a little even faster
so that's all good there's a lot of
light little demos just starting to give
you a flavor for what we're talking
about your Safari so it's getting pretty
fast I'll show you here with it off
you'll see that it's still fast it's a
little slower it's about this number is
about 1.5 x and so we're pretty happy
with that for where we are here's
another who's the one of the flickery
crazy ones and they make sure I'm set
here
so that number you can actually some of
the guys in the team taught me just you
can measure by the lot of terrorists you
see what the frame rate is but it's
looking at around 200 here that's really
great this is an interesting one okay
actually I do that right okay so here's
the Java app which is cool because they
just told me they were doing this little
demo and so we grabbed it to see what it
looked like it's kind of cute what it
does the tries to render as many balls
and keep the frame rate at 30 so you can
see with hardware with this great g5 we
can do 430 two balls per second it's a
new benchmark number balls per second
great so but now when we turn on
hardware this is actually with no pain
co changes from them they're still I'm
working with them to crank up their pipe
a little bit more I still got beam stink
off so that's why this will tarry but
anyway we'll keep going so it should
flow it up there about 3 X 2 what's
offer was that's really great is there
quite there's a quite a deep pipeline
here to get the code from job all the
way down that's pretty sweet
so here's our friend the wireframe this
is an important demo to us because this
is something Joseph Mars been using for
quite a few years to talk about
performance in quick draw and Mac OS 9
and part of our message to you is we
take this very seriously when you tell
us that you need better performance in
these areas so you can switch over to
court so this is a big deal for us so
you can see already with course software
performance things are looking great 1.3
million on a g5 at me I'm amazing and
then you know hardware is crazy and
what's really neat about this for you GL
people like me this is immediate mode so
I'm actually using the slowest version
of OpenGL primitive rendering here so
when we switch to things like vertex
buffer objects people learning about
this week we'll be able to crank this
number up even higher and it's already
in the stratosphere so that's pretty
great the cards are very very fast ok so
this is sort of back to kind of pseudo
this is going to get us into the rules
of the road so pay attention to this one
in sort of a different way make sure I
do this right yeah ok so software
hardware really really great that
numbers like 20 X and then now so what
about this this is strange behavior this
is hardware with no caching look at that
frame rate whoo so this is our first
example of how we need to be compliant
with our API the good news is there's
not a new message here just we actually
need you to be compliant now to your app
rocks so here's a scaling performance
g5s do great that's three times however
my powerbook that's nice so here's
hardware whoo so
all right back to slides OOP where's the
clicker okay so caching this is the
important thing this is sort of a
typical command stream you just saw go
by contain some commands and contain
some resources to be uploaded and we're
just showing the cute little picture of
a resource being cached so now we light
it up because we're going to reuse the
resource and see it was all little
blocks that's the technical term and
what we want to focus on is making lots
of little box so we want our resources
to be cash we want to reuse our
resources and here's another one so you
know the point is is this is the kind of
behavior you'll see a lot of the things
you'll be doing there'll be a nice clean
command pipe and then the idea is that
rarely you'll be also doing resource
uploads so using be rammed of room this
is actually I think a nice simple story
for you everything is automatic all you
need to do is use the retain release
semantics that same core foundation and
if you do that what you're doing is
volunteering to be a part of our caching
strategy importantly what you're not
doing is volunteering to have all of
your images put on the card all the time
well we'll do for you behind the scenes
is dynamically take care of that to make
sure that when i put putting too much up
there or too little so that takes a way
that worried that you have a Vaux I got
to retain just the right amount so that
my app doesn't freak out what you need
to do is just retain all your image rest
all your pattern rights law your color
dress all the things you're using and
again that's how we know because we can
see that and we can see the hardware you
want and what we'll do is take care of
the rest so in a typical picture whose
finder we have images and patterns text
Lions rectangle these are the primitives
that I gave you the numbers on in
typical case that will be everything on
the screen will take care of everything
there refer that as a visual working set
the idea here is that you know you have
this set of resource you'll be using and
those will sort of stay cash for you so
rules the road this is the big one cash
your rest this isn't exactly how it
would look for you but you get the point
right here in the first top part we have
a loop and every time through the loop
we do the cheesy thing bad thing which
is make an image ref render it and then
release it don't do that instead some
port in your code where you're loading
your images whatever you doing create
your images hold on to them that's your
cash your cash is to hold rest your cash
is not the whole bit if you want to hold
bit that's fine just pull the rest with
them and then as many times you want to
render the image render the image just
don't do what you did up above there so
layers great new strategies their hard
work silver to bitmap and bitmap context
they're very easy to use the key point
is again they're dynamic will make as
many of them Hardware cell rate as we
can based on the card you have what
you're asking us to do otherwise and
they're really easy to use I really like
how the guys did this because they're
actually easier to use them the old
bitmap context anyway so these are great
importantly right bitmap context always
have to go across the bus and layers
will be in hard or whenever possible so
here's one of our new performance AP is
the key point here is you know you see
the top here good and bad this looks
like nice clean code just that by
iterating a loop like that and giving us
one line at a time we have to create a
lot of temporary storage to handle
anything you might give us with the
second one here you tell us how big your
storage is and you give it to us all at
once so in actually actuality with the
hard to render work it basically doesn't
get touched until we're ready to talk to
GL that's very powerful here
Rex case this API is existed for a while
but we're encouraging to use it more
same thing right storage all at once
tell us the number it travels all the
way through the system don't catch your
caches this is another thing that we
found in working with the operating
system people will find this a lot of
caches around for different reasons
rendering has changed a lot in last few
years so some things are faster than
they used to be this is the key point
check your caches we've all got them and
try turning them off and see if they
still help it might not anymore so
that's what I'm talking about here also
very important this is an awesome
session the shark technology we have is
where the best technologies has come out
of Apple and less five years this might
sound silly but really really really
measure your app before you think you
know what it's doing it will look very
different with hardware acceleration
number one and number two as you know
the scale of applications has really
grown West ten years so we need to
measure our stuff and then simplify the
scenario that you're measuring until
you're sure you understand it and you
know what to do that where you're
working on the right stuff so this one
is important to me I use quick draw for
a long part of my career and I really
appreciated Joseph Mara and Mike
Frankovitch working hard to make this
run great on 10 but here's the deal
quick draw it's as we're saying here too
fragile and undocumented it's just got a
lot of funny stuff about it from the
past if now's the time we're all have to
say goodbye this is the key point for us
we're faced with this so if quick draw
is rendered in a window we're going to
turn off hardware acceleration so that's
how or what we're ended up with so more
good news it's really easy to turn on
you've seen a bunch of demos here you
have the stuff in your computer kamandi
it's right here and all you see that
it's listed there also importantly only
some of
our products support quartz extreme with
2d these are the one the key of this
fragment program stuff we have a very
modern new graphics pipeline and we're
taking advantage of these new features
so we actually do need them so final
thoughts first core imaging it's a great
new technology it works with us very
well we supporters a real image this is
the link was one of those top secret
sessions so I wanted to make sure you
saw where it was when it was actually
very soon now this is really great
though you can make an image using their
api's and well it'll be clicked with our
clipping and everything so it's really
great it's right in and then here's the
rest so the whole system is now hardware
accelerated and we really this enables
us to do a lot of really amazing things
so this would be a lot of great things
in the future here's our mailing list we
love discussion
and we've got documentation from a
quartz game amazing so we have two
different documents a lot of people
working with Artemis and this is where
you can find them they're actually on
your computers and like to bring up
Travis she's going to lead QA