WWDC2001 Session 105

Transcript

Kind: captions
Language: en
hi everybody I hope I think they were
running a little bit late next door
hopefully you guys made it over from the
other other session so let's see if I
can get this thing going
okay that's me so here's the block
diagram you've all seen probably many
more times than then you need to right
now but what we're gonna be focusing on
here is is the graphics layer and what
I'm going to be doing in this session is
really bringing you through an overview
of all the various technologies we have
on OS 10 for in the graphics layers for
you to use in your applications from 2d
3d multimedia then I'm going to show you
a few demos on the system here and then
as Travis said have a pretty extensive
pointers to other sessions throughout
the week that I highly recommend you go
to if you're interested particularly in
graphics so again the the the bubble
bullets for the graphics layers we have
quartz for a 2d OpenGL for 3d and
QuickTime 4 for multimedia and video now
what we did when we came up with Mac OS
10 was want to take the best
technologies in each of these areas and
adopt them and not necessarily reinvent
the wheel or come up with new and
different technologies so you can see
what we've done as I go through the
presentation is really look for best
practices in the industry and employ
those in OS 10 as building blocks for
you guys to do your applications so
let's start with quartz some of these
slides you may have seen before in the
keynote if any of you work at big
companies you know how this works but I
did them first so so anyway PostScript
and PDF are really the industry standard
2d imaging model and if you think about
it it's pretty amazing basically all the
pages that you've probably seen in
publications
over the last 10 20 years since the
basic underpinnings of PostScript were
figured out can be described in this
language and it's really evolved only
very little over time
to add things like device-independent
color other font formats and things like
that so it's really a credit to the
designers of this imaging model that
it's withstood the test of time it's
even gone through a language change I
mean started out as being bound to you
know very much Java like vm oriented
environment with the PostScript language
interpreter into a file format like PDF
which is just declarative and then as we
have with courts will be showing you
just a 2dc library which implements the
same imaging model so I thought it'd be
worth a few minutes just to look at the
various components of what make up the
PostScript imaging model because it may
be new to some of you they're basically
three fundamental primitive data types
that PostScript and PDF can draw there's
outline fonts sampled images in vector
line art outline fonts or mathematical
descriptions of fonts if you think about
back to nineteen eighty five eighty four
when post grip was first being conceived
the idea of device-independent fonts was
sort of unheard of I mean fonts were
things that you know are in the ROM of
your CGA card and you know they're
bitmaps of a certain size or they're in
that thing you stick in your HP inkjet
they come in a cartridge in Iram the
whole idea of device-independent fonts
was really something that was pioneered
with PostScript and widely adopted and I
think that you know not a lot of people
remember those days but it was pretty
stunning innovation and overtime
different font formats have come out of
course there was the original type one
that came along with PostScript and of
course Apple innovated with TrueType
which was a more programmatic way to do
hinting and then open type which cut is
kind of a repackaging of open type and
true type into a more standard container
the other thing that's important about
the imaging formats is they're all very
orthogonal and in other words outline
fonts can be transformed and rotated as
concealed images as can the vector line
art so just because in some systems for
example you know text color was say a
different thing then
a color you could draw a circle with and
the PostScript designers were very
orthogonal about the imaging model and
how they put it together and whatever
you could do to type images or line art
you could do across the board so the
second type is sampled images sampled
image data is really just a rectangular
array of image samples that are then
reproduced on the device by some
rendering paths which could involve
resampling the image some image
processing half toning that sort of
thing to come out on the device now it's
pretty amazing back in the early days
you know PostScript was primarily one
bit 1 bit devices was all it drove but
still the designers had in mind that why
not express the images as as deep as
deep data even though they may be
halftone down down the stream in a
device independent format so they really
thought that through even though that's
that's a lot of sort of circa 80s
graphic systems tripped up because they
wanted efficient bitmaps but PostScript
kind of did it right with the
algorithmic model for sampled images
vector line art is kind of the last
thing I have on the slide here these are
kind of the past a bit Bezier cubic and
quadratic paths that you can you can
draw with the PostScript imaging model
again they can be transformed and filled
in fact they use pretty much the same
path description logic that the outline
font system uses to describe arbitrary
vector art so those are the kind of
three primitive types and together as I
said before they can describe
practically any page that's well
certainly been printed in the last 20
years and maybe forever it's pretty
amazing and so where does courts fit
into this and you've heard the name
courts 2d well of course is our
implementation of the imaging model of
PostScript so taking those same concepts
of the three basic primitive types and
putting them into a lightweight C
language library that you can call in
your applications and we can call within
the system to do rendering on your
behalf as well one thing that's
important when coming up with a 2d
graphics library is to think about meta
file format or a stored structured
graphics format the original designers
of quick-draw we're very
because they came up with a pic file
which was a way to basically pickle any
function call you might make two quick
draw through the bottlenecks and record
them into a file and play them back
again and it's a very powerful concept
and Windows GDI has a GDI meta file
which is a very similar thing and most
designers of 2d systems think about well
what's going to be a distorted
representation of the graphics and so we
kind of arrived at quartz 2d kind of
working backwards we knew we wanted PDF
graphics streams to be the persistent
format and the record and playback
format that the 2d library could use but
then we work backward from that to
figure out what what should the API be
and of course we should draw a Bezier
path so they can be represented exactly
the way PDF expresses them so that's
really a little kind of motivation for
you of how we ended up with with quartz
2d we really started with the idea of a
PDF record and playback and then arrived
at the API from that the other thing for
courts 2d is we have very fast
anti-aliasing one of the major
motivations for building quartz was the
user interface and some of the ideas
that we wanted to do with aqua and we
knew it had to be very fast and very
high quality and so quartz we spent a
lot of time with the an tal griffin's
and it's really sub-pixel accurate in
terms of text positioning and and fixed
point sub pixel coordinates an exact
coverage per pixel so it's pretty
sophisticated what's going on there then
we have Apple type technology that Apple
actually already had so we didn't need
to rebuild anything here Apple has a
pretty extensive type machinery called
the Apple type system and it supports
TrueType as well as type 1 and other
pluggable font formats and that we just
built right in to courts and courts
calls the Apple type solution whenever
it has to do any any character handling
and in fact that piece of code can
stream out data when we need to embed a
font we just tell them hey we use these
three glyphs from this type from this
font
and the type system will subset the
glyphs and stream us out in a format
that we can just stick right into the
PDF file
so it's a really good way to get a lot
of value out of these technologies that
we had at Apple that were that had been
around before we even did courts on the
type 1 scaler I mentioned we do have
type 1 built in in fact we worked with
Adobe on getting the type 1 scalar so in
terms of ATM you don't really need ATM
as a rasterizer I mean there's font
management things in atm as a product
but in terms of the original reason why
ATM existed which is to bring type 1
support to the Mac with we've clean that
up with OS 10 and it's now just built in
as a standard format no extra software
required colorsync is another technology
we had at Apple and used primarily in
high-end workflows and it's a it's a
great API for wiring together
transformations on colors and and
sending image data through it and so we
just use that directly in courts 2d
behind without necessarily writing any
code if we're asked to draw an L a B
image rotated 45 degrees we'll set up a
the color worlds behind your back and
match appropriately through the color
worlds to the screen or however you've
set up the destination profile for your
rendering context there's going to be a
lot of talk about how this exactly works
the matching between the color sync
implementation in quartz 2d and the PDF
model versus this straight ICC model
which is in color sync and there's a in
the color sync talk they're going to
explicitly talk about color sync in
quartz working together so if you're
interested in that topic that's a good
session the other thing that we did as
you've no doubt heard is we've bundled a
bunch of fonts about 50 megabytes worth
of fonts
they're both decorative and classic
designs as far as Romans Broman
typefaces go and for Japanese we've got
six really high-quality Japanese fonts
these fonts are
a range again from kind of more classic
designs to more modern designs but
additionally they come with a huge
amount of glyphs I think it something
like twelve thousand glyphs in each of
these fonts so it's a really complete
font and what's more there's no
resolution limit or embedding limit in
terms of what you can do with these
fonts on OS 10 and that's something
that's I think sort of been holding back
desktop publishing a little bit in the
Japanese market because it's very
expensive to to have high quality fonts
and the outlines are very protected and
if any of you have dealt with you know
laser writers with these kind of fonts
you know what I'm talking about
but these fonts are there's no such
limitation or no copy protection or
anything like that and together with our
streaming we get out of the Apple type
software what we're able to do is embed
only the glyphs necessary from the fonts
that are shown into the PDF file so that
PDF file can travel anywhere any
platform and still have those great
fonts embedded in it so let me show off
some of the fonts here so we have
Baskerville a bunch of sizes you'll
notice it's not just the regular italic
bold kind of set we've actually got some
of the semi bold and intermediate
weights like that so obviously much more
much more expressive than quick-draw
style bits we've got regular bold light
and condensed various flavors of
American typewriter which is a nice
fixed advanced font some of the classic
faces here caslen dido copperplate
some more decorative faces
we've got marker felt which is kind of a
nice kind of presentation font that we
know which is a font designed by hermann
zapf the font designer in the spirit of
his own calligraphy his own handwriting
and that's really a beautiful font and
I'll actually show that interactive
during the demo Optima Optima is a great
fund it's been around for a while
one of the things that is great about
optima and OS 10 is the anti-aliasing
really brings through some of the subtle
characteristics of this font for example
the be in bold has a very slight bow on
the curve and to really represent that
faithfully requires a pretty
sophisticated anti-aliasing and you know
more than 616 levels of gray at some
some approaches do it really benefits
from the the real sophisticated
anti-aliasing approach we take and a
normal hinting renderer would you know
flatten that thing thing away the first
thing that first time it saw it so
that's it's great to be able to preserve
some of those subtle subtle details of
of the typefaces and then here are the
Japanese fonts as I said I hope I'm not
offending anyone with this I don't speak
Japanese but I'm told it says this is a
beautiful font I trust them on that but
you can see we have some of the modern
faces and the more classic serif faces
too at the bottom so that's a little bit
about 2d one a very important aspect of
2d though is printing and one of the
prime reasons we did the courts 2d
effort and worked with PDF was we really
wanted to retool the printing
architecture and when I put this next
slide up all the guys who are on the
printing team in the audience say if
only it were that simple but believe it
or not this is the essence of what's
going on when you print on Mac OS 10 we
use quartz 2d heavily even for carbon
applications which may be calling quick
draw to draw their pages well what we
what I've depicted here is Internet
Explorer on the Left which is a carbon
app running against the carbon runtime
calling quick draw to draw its pages it
goes into its printing loop and what
happens is we've taken a bunch of code
and expertise that we had in the laser
writer 8 driver which knew how to
translate if you will from the
quick-draw imaging model into the
PostScript imaging model
well since the image model PostScript in
PDF and quartz are all sort of the same
we
retooled that code to essentially
implement a set of quick-draw
bottlenecks which call courts 2d and so
when you go into your printing loop in
in carbon what you're actually getting
back is a graph port that's backed by
these special bottlenecks that we've
implemented that call courts and so that
flows through courts and then courts of
course knows how to record that into a
PDF file and so what we do in that first
half of the schooling phase of the
printing architecture on OS 10 is really
record a faithful representation of what
the application told us to draw or told
quick-draw to draw or if you're calling
courts 2d directly what you told courts
2d to draw and so that's really you
obviously need a really great packaging
format an imaging model to be able to
basically describe what any application
wants to draw and that's again one of
the great advantages of using PDF so we
don't that's we don't take any steps
like in Distiller which is a product
that exists you know for the purposes of
creating PDF and there's a lot of great
bells and whistles like down sampling
your images you know recompressing the
JPEG converting CMYK to RGB none of that
is really essential in this in this past
year because what we what the only thing
that we're trying to do is faithfully
represent what the application drew so
it's like a high fidelity application
independent device independent
representation and the pages that's what
we use PDF for there and then that file
flows through the printing system and it
arrives at back-end piece of software
that manages the actual IO to the
printer and what that piece of software
does is call courts 2d is again - this
time parse and replay the PDF file and
convert it to a device specific format
for the printer so it'll convert PDF to
PostScript in the case of PostScript
printers it will convert a PDF to raster
data in the case of inkjet printers and
so the inkjet printers that you see on
Mac os10
are taking advantage of the high
fidelity rendering of quartz 2d in their
output which is a great feature and in
fact we have a bunch of
thanks to any of you from Canon Epson or
HP to help get help us get this done and
to our printing team as well for OS 10 I
think we we did really well getting 50
printer drivers for inkjet printers in
the box and that's only the beginning of
what we're going to be doing with these
vendors and others the interesting thing
for you guys as application developers
is you don't really have to treat inkjet
devices as sort of second-class citizens
anymore in terms of high fidelity
graphics and PostScript quality
rendering and yeah you can rotate text
on them and if you go to the talk on
Thursday morning that I'll point you at
you'll hear all about the current
printing graph port for carbon
applications and in fact it interprets a
lot of the special pic comments that the
laser writer 8 code did so you can
really get some of those effects across
the line on inkjet printers which is
really pretty nice so anyway a plug for
ink Jets they're great so that's about
it for courts let me briefly show you
some slides on on OpenGL some of these
may look familiar
so OpenGL is industry standard 3d
technology it's been around for I don't
know 15 years or so started life as the
GL graphics library on the SGI machines
and i think early 90's became a standard
it's been through about two or three
revisions as OpenGL and it's very much a
vibrant standard there's all kinds of
extensions being proposed very
frequently it's got a very clean model
of how to add functionality how to probe
for functionality to be able to take
advantage of specific card features and
it's a it's a real nice nice programming
model and a lot of our game developers
have used it to develop some great games
over time this is a picture from Star
Trek and if you go down on the show
floor you'll see the games all running
on OS 10 it's really pretty amazing I
don't know I its calculations probably
off but not by an order of magnitude but
I figured there's got to be something
about like a hundred gigaflops on that
table of
of quake games that people are playing
at now if anyone has a better number
shout it out but you know you figure
you've got g4 CPUs in there and you've
got NV eleven and videos GeForce 2 cards
in there I think it's a pretty stunning
supercomputer display there but OpenGL
is really not just for games again you
may have seen the slide before it's also
for a lot of applications that kind of
grew up around GL and UNIX and
environments like that which are
prevalent in the high-end modeling
animation scientific engineering world
Maya is a great example of such an
application and we're real exciting and
working very closely with the maya guys
to get their port of for OS 10 working
really well but there's a whole stream
of other developers who are kind of in
this camp as well and hopefully some of
you out there who come from this
heritage of high-end workstation
graphics with either you know custom
vertical solutions or even broad
solutions so we're really excited to be
working with with any you developers out
there we have a great OpenGL team and
great developers support so get involved
with Apple if you or you know someone
who got a great solution to bring to the
platform now that we have UNIX and
OpenGL all working together the
architecture on OS 10 for a graphic
acceleration we have spent a lot of time
making sure that the graphics
acceleration architecture on ten
aggressively virtualizes the resources
of the of the graphics cards so it's if
you're playing a full screen game for
example essentially every byte of video
memory can be owned by that game when
you're running in a windowed environment
the video memory is obviously shared and
arbitrated among the among the
applications but none of them have to
explicitly get in the game of managing
their memory so really it's almost sort
of a vm system that works in the video
memory of course there aren't page
tables so the analogy is not completely
accurate but it really we
take our our role as an OS vendor very
seriously when we looked at how to
support 3d graphics acceleration and
it's really a nice nice architecture so
we're we're really happy to have great
drivers for both Nvidia and ATI cards
out there and work with you on any
additional cards if you're at one of the
other graphics houses QuickTime just a
few slides on QuickTime Mac OS 10 comes
with QuickTime 5 there's a few new
features in QuickTime that are important
actually first let me back up and just
mention on this slide by my comment
about quick times longevity so QuickTime
has also been around for a long time you
know early 90s or so and the amazing
thing about QuickTime is is that it has
really survived the test of time I mean
the original architecture we think back
the early 90s based on the component
manager and pluggable codecs and dynamic
codec chain building all that stuff was
really pretty groundbreaking and I think
has paid off over the history of
QuickTime because the basic architecture
is still the same the object orientation
even though that was you know pre you
know pre you know heavy duty use of C++
and object-oriented and and the
component manager was what they made it
go with but it's really pretty amazing
that there's been such innovation in
video codec technology video streaming
technology and the basic architecture is
still sound and so I you know I think
it's it's a great thing to point out
about QuickTime and I don't always get
an opportunity to talk about QuickTime
so I thought I would give that plug and
and thanks to the team for for caring
for that architecture over the years so
the stuff that's in a QuickTime 5 that's
built in that's new there's a new user
interface there's a flash for codec a
cubic vr which is actually pretty
exciting because the cubic vr not only
lets you look up and down but he uses
the six phases of the of the cube and
it's very amenable to hardware
acceleration and so you can a lot of the
cards now are starting to implement
cubic environment mapping
so that I think there's some very
interesting opportunities with cubic VR
and OpenGL working together
DLS music synthesizer we saw the demo of
that earlier MPEG 1 streaming the new DV
codec which is much higher performance
and higher fidelity in terms of getting
DV to the screen very important for
applications like iMovie and editing
apps that you might be working on it
also leverages the the sound
architecture on OS 10 which takes
advantage of some of the real low
latency we're able to get inside the
kernel in OS 10 and additional tricks to
really get the sound performance pretty
amazingly low latency and high
throughput on OS 10 I think games will
take a lot of advantage of that as well
so that's the technologies that you as
application developers can make use of
on OS 10 and one important point about
them is they're all optimized for g4 and
multi processing so we think it's our
goal that you should see a nice you know
sort of performance increase as you move
up the product line so the g3 is we
spend a lot of time tuning for g3 s
which is probably the most important
work but we also spend a lot of time
tuning for g4 and also have multi
processing we kick off extra threads at
certain certain places throughout the
system when when it's appropriate so you
should see a nice you know performance
signal as as you move through the
product line and I would encourage you
to take a similar approach with your
apps because it's nice to be able to
point people at you know higher and
configs in terms of a way to get around
performance issues obviously makes us
sell a lot of high-end hardware but also
makes your app shine too because you can
employ these tools to make users more
productive so definitely take advantage
of those things so one other topic which
I wanted to cover in these slides was a
kind of fourth area
that we innovated in in OS 10 and that
comes down to the windowing system and
what do you do when you've got all these
different graphic environments wanting
to share the screen you've got demands
for very high fidelity anti-aliased
icons with you know very subtle edges
and what we did was decide to rethink
how the windowing system actually worked
and approach it from a slightly
different angle and so back in 1984
there was a paper by Tom Porter and Tom
Duff from Lucasfilm who then went on to
form Pixar where they really laid out
the beginnings of digital image
compositing and introduced the Alpha
Channel among other concepts in this
paper and it's really been a touchstone
paper for any of you that are in
computer graphics or have got go to
cigarete proceedings because it's really
a very elegant formulation of a
compositing algebra and how to how to
put together images that may at that
time come from different batch offline
processes you had one program that could
draw spheres another program could draw
fractal terrain and you wanted to put
them together I didn't want to put all
the code together so they worked out a
system where you could preserve the
anti-aliasing with the Alpha Channel and
you could recompose it after the fact
different scene elements and save a huge
amount of time in movie production and
so you know fast forward 15 years we're
trying to apply these exact same
principles when we designed the
windowing system on OS 10 we've got
content that's rendered we've got
dynamic content underneath it we don't
want to redraw every one every time
anything happens so we really look back
on this initial work and implemented a
system which is really I think a great
thing and I think this is the way
windowing systems are going to be done
from now on that provides more of a mix
or analogy for the display rather than a
simple switcher in the switcher analogy
each pixels owned by one application or
another I think time has proven both in
the making of video switching equipment
and windowing systems that the switcher
is really too primitive and doesn't give
people designing experiences with
such a system enough flexibility and
obviously you know everyone has a in
television studios they all use mixers
film production as well the the idea of
a switch as far as a way to arbitrate
real estate is really just kind of an
old concept so we're really excited
about this I think it's gonna you know
when avi talks about laying the
foundation for the future I think this
is really one of the one of the things I
turn to as re-examining some fundamental
assumptions in the OS of you know how
are we going to put together images for
the user to make a compelling user
experience moving forward and I really
think using the compositor analogy is
going to be important there so what we
call this piece of technology is the
quartz compositor it's really the
windowing system last year I think we
call it the lightweight windowing system
but it's a client-server architecture
it's responsible for presenting for all
the final presentation blitz that happen
in the system in order to mix the
content onto the display and it works
with the acceleration layers of OpenGL
QuickTime and everything working
together to share the screen some of the
features of the quartz compositor full
double buffering double buffering is
important not only from the standpoint
of not needing to wake up an app to
redraw the display but also to have
access to the the rendered content of
any window on the system at any time so
we can fade it in fade it out put a drop
shadow over it whatever per pixel alpha
channel important to have per pixel
control of the opacity of a window so
that for example in the menus the text
is actually opaque but the rest of the
menu is transparent and obviously quartz
2d plays a role in that too because you
need a rendering system that can
understand how to draw destination alpha
into the into the the back buffer for
the window when you draw with quick-draw
you can only get opaque pixels so for
another feature that we have is an
overall per window fade control so
that's one big opacity knob on the whole
window so that's used for fading and
getting out we have some per window
transform and warp capabilities sheets
genie dock animation those kind of
things and the important thing is that
this really integrates across whether
you're doing 2d 3d video certain things
are currently assumed opaque like an
accelerated 3d surface someday we'll get
those transparent too but by and large
the model holds together if the user
picks up something that's transparent
and drags it across the screen you know
the illusion is not broken he can just
slide it over every every piece of
content on the display so I think with
that I'm going to switch over to the
demo demo machine here
I get them on one please yeah great so
let me just bring up a very basic OpenGL
application here to get some pixels
moving this is sky fly which you played
with the developer CD at all this is on
your CD as a small application in OpenGL
it's a simple terrain model with a
flying airplane
I changed the code a little bit to run
in a window it runs full screen ads as
it as it comes out of the box but it's
pretty straightforward so you can see
the window look of aqua is very clean
we've got drop shadows the window
content goes all the way to the edge of
the windows that was really important to
the look they were after with aqua and
so to set off the windows all's we
really have is the drop shadow and so
you can see what's happening is the drop
shadow is actually dynamically
compositing I mean it's not a very
in-your-face effect but it carries the
illusion that even with animated
accelerated content underneath we can
still set off that Finder window without
having to just chop the pixels with a
hard hand off between the accelerated
content and the and the Finder window
again down in the dock we you can see if
I drag it will gol window down
underneath the dock you can see the
compositor kicking in to do all the
blending on the dock you can see that
all the icons have a high degree of
anti-aliasing very you know high
production value if you will
graphics so to really carry that
anti-aliasing above arbitrary content
it's essential to have some kind of
compositing going on in the windowing
system otherwise you're just going to
hard clip it and it's not going to look
right so my opinion is as soon as you go
to anti aliased icons that are going to
be piece parts of the user experience
you got to do composited windowing
system or it's just not going to hold
together let me now show you a little
more in detail with that with this pixie
application which is on the developer CD
and I just have to configure it here to
refresh continuously
this is a nice little tool you can use
it fruit for you know sniffing at your
pixels and seeing what's going on but if
I hold it down here you can see what's
going on in the display is actually you
know there's a lot of you know half
colored pixels in order to get the the
side of the Microsoft icon they are
working well and that's all getting
composited on the fly as the OpenGL
application is playing and you know it
obviously it's a lot of work for us to
do that and design a system such that
that could be possible but it's really
important you know to not break the
illusion if you're going to present the
user with really nice anti-aliased icons
you really don't want them falling apart
when they go over 3d 3d content so again
just another minor example if I go up
there to the corner of the window you
can see the corners the window have a
little anti-aliasing to make them look
real crisp and you can see how the
accelerated content is is bleeding
through bleeding through there as well
so I have one other other thing to show
I'm kind of ad-libbing a little bit on
my demos because some of those were
shown earlier too so here's a piece of
PDF clip art this is actually just an
EPS file that we distilled and I just
dragged it into a text editor here and I
can add a text label so that makes the
font a little bigger
so again for for the purposes of drag
and drop what we do is we do live
dragging of text which includes graphics
so if the user wants to move this to
another application he's going to move
it over that over that 3d content and
you really want everything to just
continue to work and you can see how the
the app is still animating under there
we lose a little bit of framerate but
the illusion holds together and and the
user just thinks he's dragging this this
text clipping over a piece of acetate
over the whole screen and so it's
important you know and this kind of a
theme I guess from working at Apple is
that you just want things to appear as
they should and not it doesn't matter
technically how difficult they might be
but you really want to sort of delight
the user with with the interact
interactivity and the production values
of the system so that's enough of a
compositor let me show you actually
while I have opengl going here this one
does have a little bit of audio this is
a kind of a teaser for a session that'll
be coming up I think it might be
tomorrow Wednesday but we'll see in a
second that shows how to combine some of
these technologies together so for
example this is actually using OpenGL 2
and QuickTime together to put a movie on
a surface and I think Jeff's getting
some outrageous frame rate like 350
frames a second but there's probably not
that many frames of video but it's an
apple one of the Apple ads that
and you can see not only are each of
these building blocks that we have
pretty compelling but you start to put
them together and I think there's really
some pretty interesting interesting
opportunities like to start off with Liz
polyester Brian sure for you so I'll
give you a pointer to that one in this
when I talk about sessions later let me
bring up Internet Explorer here and hit
on a few things about the printing
pathway so I'm basically going to do the
first half of that demo where I showed
you the the printing pathway recording
to PDF so Internet Explorer is a carbon
application drawing it's content with
quick-draw here's news-dot-com see
what's going on today bring up the print
panel now this is a system-wide print
panel so that's another advantage from
OS 10 another difference from OS 9 where
the print UI you didn't know what you
were gonna get you you know you made the
call to put up the print dialog and you
know it's kind of up to the driver from
there we have standard print dialogues
that we bring up the drivers plug into
them and can add capabilities but it's
no longer kind of a guessing game in
terms of what applications are going to
do with the print record we have got a
nice property list where we store all
the information about the job so
hopefully that'll make your lives easier
particularly you know when QA against a
whole bunch of different print drivers
but I can hit preview here and that's
going to basically run the first half of
the of the printing process and bring up
in the preview application here a PDF
rendering of the page and it's basically
the content that that the application
drew through quick-draw calling courts
2d and then saved alphas a PDF file and
if I were to actually print this then
this exact PDF file would go through the
back end and get converted to PostScript
or a raster bits or whatever
and just to show you it's real PDF so
let me try it dragging it on to Acrobat
here and there's Acrobat showing the
same file so anything anything we create
can can obviously be read by Acrobat
because it's just PDF a little bit more
on the 2d here's a a TextEdit document
that has a bunch of different fonts in
it these are the fonts that we ship
these are all interactive just scroll
through some of these so you can look at
them there's that Fino gill sans I don't
think I had that on the slide Helvetica
neue a bunch of faces of Helvetica if
you can't find one in there you need I
don't know Optima let me show you let me
show you what I was talking about with
that B if I go on with prank on the
slider here just a scale if that B you
can see how those subtle curves are in
there in the font design and because of
the anti-aliasing we can carry those
those features down to very low point
signs
again the Japanese fonts let me pick
oops
didn't mean the launch QuickTime let me
pick one of those and again zoom in on
that I think I got my slider set up to
500 yeah I can type in something even
bigger but you know no resolution limit
you know pretty dynamic interactive
scaling so hopefully this will this will
be a great thing for not only Japanese
users but everyone and you know as was
mentioned in the other sessions these
fonts exist on every system every users
system so you know it might be likely if
you're writing say example for example a
mail application and someone gets a
message in Japanese they may actually be
showing Japanese content in your
application so I think you also need to
be aware of those possibilities that
someone may be buying your app not in
that particular market but may be
dealing with language content from other
other places and whether that means
linguistically obviously you may not
have not have all the dictionaries and
stuff necessary to edit in that content
but at least to display and manipulate
letter forms and things like that I
think that's a good opportunity to for
that help help us share content
worldwide oh wait actually before I kill
that I wanted to show you what's that
Fino up here so this application is the
the simple cocoa TextEdit which the
source is on the CD it's I don't know
three thousand lines of code something
like that but basically all the work is
done by the cocoa tech system in terms
of the interaction with the with the
user and the selection and insertion
point and all that stuff at the lower
levels it's the the Apple type system
framework and then even below that is
quartz doing the actual rendering now if
you're a carbon application you can call
the Apple type solution that Suey API is
in order to do some of this there's also
a higher level object
more or less moral equivalent of the
text object which is called ml te which
is a multilingual text edit version that
removes the 32k limit among introducing
a bunch of international features so I
think no matter what framework you're
coming from there's a lot of a lot of
great text technology and something I
just wanted to show off with these fonts
or you know it's if you wrote your own
text edit handling code you know you
might be confused you might have bugs or
redraw bugs with a font like this
because you can see the swashes are
overlapping characters are running into
each other being over painted you know
the italic angle is not quite the same
throughout the whole font so obviously
you know the if at all possible you want
to use one of our platform services in
order to interact with this kind of type
because it really is fairly tricky to
get it all right so for example he's got
a ligature when he designed the font for
his own his own name there's the PF
ligature and that forms automatically
with with the cocoa text system because
they've got automatic ligature formation
turned on by default
I think ll is another ligature so you
can see it's a real beautiful font I
think I think that having those in the
system is a lot of opportunity for
everyone I don't know what the line is
on that screen but I don't see it I
don't see it they were there but
hopefully it's a projector okay
so what's next let's do my final final
thing that I'm going to talk about
before I go back to the slides which is
and yet another image capture demo but
this one is a little bit different I'm
going to use a SanDisk USB flash card
reader so this is just a mass storage
device I have on here some pictures that
I took on my Hawaiian vacation I don't
have kids so you get to see my travel
photos you could see my lovely wife
however so I'm just gonna plug that in
and what's going to happen is os10 is
going to recognize the the flash card
and you see it mounted the volume and
then actually what happens is image
capture kicks in and you might ask
yourself well how does that happen
that's just amount of volume showing up
on the desktop and what actually goes on
there is the image image capture
software is notified anytime there's any
amount any volume amount occurs on the
on the platform and because of things
like flash card readers and stuff like
that we wanted to plug in at that level
and also allow users to work with their
digital images through there of course
you can have a camera driver which would
directly recognize the camera but we
wanted a file system solution as well so
what we came up with is a way to
intercept that that mount point get
notified and we actually look out on the
disk for a particular file structure
which is laid out which actually turns
out to be a standard thing that all the
camera vendors agreed on because they
didn't want to get in each other's hair
and start writing files other cameras
couldn't read so if you look through
I'll actually give you a brief look at
the at the file structure here in the
mounted volume but so what image capture
has done is recognized that Oh removable
medium is das formatted has a slash DCIM
camera named folder in it and decides to
kick in the image capture software and
we've built a bunch of heuristics in
there and we've built a bunch of
heuristics in know about it'll recognize
insertion of photos seed not photo city
what's it called picture CD
other formats so if you guys are
developing a plug-and-play disk layout
that holds images if you tell us about
it we can make image capture kick off on
those mounts as well so anyway the the
usual thing to do of course there's
automatic tasks you can you can set up
to to download images you can have them
run on Apple script which you saw the
demo of you can also have the user can
configure it to run any application
which can handle a multiple open of
images so for example even if your a
word processor for example if if you can
accept an apple event to open a
collection of images then the user could
set up image capture to call your
application and maybe you make a new
document with those images laid out or
something like that so another thing to
keep in mind very simple way that you
can take advantage of image capture in
your in your in your code now there's
gonna be whole session on image capture
where they'll go into the API if you
want to like take a picture for example
or do camera specific things but there's
a lot of opportunity here so let me go
ahead and download my Hawaii pictures
here and by default everything goes into
the pictures directory of the user but
if you noticed on the on the in which
capture panel there was a place where I
could you know set a different folder
for example so those are my small
selection of my hawaii pictures i only
golfed one day but i spent a lot of it
in the sand trap so and then of course
are all of our favorite application
which is the slideshow which I'm going
to talk about a little bit while these
images show up so what's going on here
there's my lovely wife Nancy hi Nancy
when when we started looking at what was
going on with graphics cards and what
you could do with OpenGL this is
actually looks perhaps a lot easier than
it is I mean what's going on here is
there's about six megabytes worth of
textures that are getting cross-faded
there's a front buffer and a back
for and probably about 12 12 to 16
megabytes worth of video memory to do
this well we do use less memory and
downsampled textures if we're on a lower
lower config probably about 2 gigabytes
per second of video memory bandwidth are
necessary to draw this much textures
with all this blending going on so and
of course there's all other sorts of
complications if you programmed with
opengl there's a power of 2 texture
limit so the textures have to be diced
or rescaled so there's a lot of you know
kind of trickiness to kind of getting
this going but the end result is you
know something people would expect I
mean you know my parents would love to
see these pictures of you know us on our
Hawaii vacation and this would be a
great way to show it to them so again
this is an example of you know kind of
OpenGL not being just for games
obviously it's great for games but the
innovation that's being driven so hard
there in terms of performance lets you
do some of these things in real time I
mean this is not at all a scripted a
scripted slideshow at all it's just
reading the raw JPEGs it's picking a
random place to zoom in it alternates
between zoomin and zoom-out and it's all
very dynamic in fact it's great I run my
my PowerBook you can actually boot a
power book with the lid closed I don't
know if you know about this if you have
an external keyboard you can boot it up
and that way all the video memory on the
power book will you be used on the TV
Jack and you can run the screensaver on
your TV and put on some music and
entertain friends for 4 days and days so
pretty cool but I wanted to give you
some insight into all the technology
that that's actually behind that and I
think we're at a point now in the
evolution of graphic systems where there
is a lot of technology that's untapped
that you know we can we can try to tie
it tie together some of these things
that are traditionally offline batch
processes and do them in real time for
the user so that's kind of the brief
brief demo there let me go back to the
slides
okay so I've gone through all of the
technology here that we've put for you
in Mac OS 10 from the API levels with 2d
3d QuickTime for video
tying it all together with the
compositor and kind of putting back up
kind of the the the textbooks there that
because an important point for us is we
really are trying to adopt best
practices in the industry and look at
what has worked before and deploy that
in in a integrated fashion for all of
you to develop your applications on so
that's really the the extent of my
slides what I've got for you next is a
bunch of pictures of the of the roadmap
CD hopefully the colors are the same I
took them from the web and I'm going to
kind of give you my picks of sessions to
go to throughout the throughout the week
so if you want to follow along in circle
you can tomorrow first thing in Hall J
there's a session on PDF quartz and Mac
os10 this is kind of an introductory
session if you're not really familiar
with PDF and hey you know how is PDF
different than Acrobat and how we use it
in OS 10 as a graphics file format
versus how it cools like Acrobat add
much more data to it to become more of a
document format you should go to that
session and and understand understand
that then after that in I guess this
hall a1 which is this room there's going
to be a session called 2d graphics using
quartz which is going to be down with
with the quartz API looking at actual
sample code and how the how the quartz
2d system is put together in particular
and what the function calls look like
tomorrow afternoon f right after lunch
there's going to be a graphics for games
a session which is going to talk about
primarily how to use opengl for games
opengl we think is a great solution for
not just 3d games but 2d sprite engines
as well in fact I do
and hearing if anyone has a sprite
engine project they're working on
because I think it would be really a
great way to bring that class of apps to
another level of performance after that
there's a session on drawing Unicode
text with atsui
so if you're a carbon developer and you
want to really get into the layout and
the Unicode to glyph processing that
happens in the system that's the session
you want to be at you can also learn how
to draw from carbon apps using atsui to
quartz graphics context as well so if
you're looking to get features from
quartz and be a carbon app that's one
approach is to go to that session there
and learn what they what they are
offering let's see on Wednesday in the
morning there's the QuickTime overview
just get the State of the Union on a
QuickTime later in the morning there's
image capture framework you'll hear
about the tools they have for if you
want to write an application to to
acquire images or if you a little bit
about if you're if you're a device
vendor and you want to play with the
game how to how to get involved with
Apple on that that's a good session to
go to the color sync session at 2:00
o'clock after lunch on Wednesday is
going to be a combination of just color
sync for those of you that are familiar
with color sync and also color sync and
quartz working together and how that
whole integration works in OS 10 so
that's a really interesting session if
you're interested in color and some of
the integration work that's been going
on in OS 10 then in the big hall 3:30
Wednesday is text on Mac OS 10 this
talks at a slightly higher level about
all of the text facilities on OS 10
who's internally using it in terms of
layers of the stack and how impaired
inspire gets done what it needs to get
done a little bit on the high-level
object I mentioned earlier the MLT II
just kind of the whole infrastructure
top to bottom for text on OS 10
in the red there we've got the OpenGL
sessions the first session is one called
OpenGL high performance 2d and that
talks about some things like the the
QuickTime movie view or the slideshow
application Jeff Stahl has some examples
of how to actually dice up big images to
put them on texture maps and move them
around and rotate them so if you if
you're interested in doing some two 2d
graphics where you've got a lot of
pixels you want a lot of interactivity
and you want to you use opengl for that
that's a really interesting interesting
talk
later that afternoon there's opengl GM
geometry and modeling which is again
more the traditional 3d uses of OpenGL
an interesting session a bunch of
QuickTime tracks going on as well in
parallel tomorrow Wednesday afternoon
Thursday first thing we've got OpenGL
optimization which is going to be you
know really kind of an expert session if
you're familiar with OpenGL and you
really want to tune the last last little
bit out of here out of your app go to
this session and learn about all the
other little tricks and techniques and
how to package your vertex data so it's
optimally passed through the system and
things like that and OpenGL advanced
rendering also some advanced techniques
for OpenGL experts on Thursday morning
font management on OS 10
so rather than text processing this
talks about fonts and how they live in
the system what directories they live in
how you can enumerate all the fonts all
the font management API is we have on OS
10 so that's a good session to go to
we've got a printing printing session at
3:30 which is a great session to go to
to learn about all the printing api's
from the carbon standpoint as well as
some of the objects from the cocoa
standpoint brings talks about the whole
architecture in a fair amount of detail
there's also a parallel with added
session over in the Civic Center on Java
graphics which is pretty interesting if
you want to hear how the Java guys are
doing job
Toodee using a combination of courts and
some OpenGL actually very interesting
session there as well if you're
interested in Java feedback forums start
on Thursday afternoon we've got the
OpenGL feedback forum they encourage you
after you've gone to the sessions to
bring any feedback to that on Friday we
have a session 9 a.m. in Hall a2 called
graphics and imaging tips and techniques
tips and tricks this is not the session
you go to learn how to put the doc on
the left-hand side of your screen these
are actual programming tricks of how to
there's some debugging optimization
demonstration there's also some
discussion of how the printing graph
port works on OS 10 a bunch of topics
that none of which we're big enough for
a whole session but we kind of put them
together into kind of bag of tricks here
that we thought people should know about
while we had all of you here for the
conference
then there's the feedback forum on
graphics at 2 p.m. in in j1 so come
there and give us feedback on what what
we should be doing what we could do be
do we do better
you