WWDC2004 Session 218

Transcript

Kind: captions
Language: en
good afternoon is everybody ready to go
home
okay well and make yourselves
comfortable so hello yet again in case
you don't know I'm Matt your aunt's i
work in DTS and I can't stay off the
stage I can't help it I've been on stage
every day this week I think that might
be a record actually so um I kind of
adopted this talk at the last minute
some of you may have been expecting to
see George Hemsky who's our resident
graphics guru but he had a small
incident come up tuesday night so so
Gerard won't be joining us I'm going to
do the best I can to sound as
authoritative as he would on this topic
so what are we to be talking about we're
going to talk about the changes that
we've made in Java 14 to update 1 you've
got Developer Preview three available to
you as of this week I'm talking about
the architectural changes we made
specifically to on-screen Java 2d
drawing and we're going to talk about
improvements not only in correctness and
hopefully eliminating a lot of the bugs
that some of you have reported but as
well as performance improvements that
we've seen as a result of these changes
we talk very briefly about courts 2d and
it's and its relationship of Java and
how that story has changed with the
announcements that have come in with
Tiger this week and we're going to talk
a little bit about the third party aap
is that are new to mac OS 10 as of the
last time we talked here last year a
specifically Java advanced imaging Java
3d and jo dl which we which we
affectionately know it's joggle so Java
2d on Mac OS 10 like I said earlier we
have a bunch of improvements and some
new features that we've made available
to you with gp3 among them is a graphics
context management and I'm going to be
saying context a lot in this talk and
and within the context of this
presentation by context I mean that just
a standard Java graphics object java.awt
graphics object and talk about state
management the state of a Java graphics
object and how we've worked around a lot
of problems that we have with that in
the past and talk about how these
changes affect and improve
multi-threaded drawing and update one
we're going to talk about the
intelligent screen update mechanism that
we've introduced with BP three I believe
that's new with TP three okay and we're
going to talk about some improvements to
full screen mode drawing and XOR drawing
as well so let's go right into it and
talk about context management like I
said it's a new mechanism and update one
and it's completely independent of the
cocoa drawing context that we use
beneath as you know the AWT in 142 in
Java 14 is built on top of the cocoa app
kit now in the past we used to share our
graphics context with cocoa and created
a number of problems or issues that we
needed to be delicate with and the most
problematic one was that sometimes cocoa
would prevent Java from drawing all
together so if there was some painting
things that needed to be sent through
the contest Coca would som sometimes
deny us the shared context and it would
cause us to either skip frames or paint
methods all together or at least delay
the visual appearance of painting on the
screen the new system improves
performance in addition to correctness
and the perceived the perceived
quickness by the user before not only
could the context have been denied by
Coco but it could have been modified by
Coco as well and we didn't necessarily
know what we were getting back and we
needed time to reconcile those states
and make sure that the state that you
expected your Java graphics2d object to
be in was the same as the states that we
were using behind the scenes so now
because the contexts are independent we
always know the state of your Java
context and we can start drawing that
much sooner and when I say state changes
I mean things like color clip
anti-aliasing fonts all the set methods
that you would call on a graphics 2d
object all of those things we know at
all times now what those states are from
what you're setting in your Java code
before we had to make sure whatever we
got back from Coco was synced with what
your Java code was expecting and now
that we have an independent contest we
no longer have to worry about that and
that obviously is less execution time
and more time devoted to painting
operations here's a visualization of
what I'm talking about it might sound a
little confusing what you see here is
that we're having a little fun with
you see Java and cocoa basically
fighting over the same context and that
checkered blue and orange there is
basically kind of illustrating the
unknown states at a cot that a given
shared context would be in in the
original 14 to release and whenever Java
got it we needed to make sure it wasn't
in cocoa state we need to make sure it
was in our state so now in 14 to update
1 because we have our own context
exclusively available the job available
to you we no longer have to worry about
any of that reconciliation cocoa can do
what it wants and Java can do what it
wants and this particularly is good for
multi-threaded drawing the improved
continence management that I was just
talking about basically allows us to
draw correctly from multiple threads we
don't have to worry about locking or
contention or anything like that it
allows us to cash graphics2d objects now
that we don't have to worry about state
validity or anything like that you can
safely cash a graphics 2d objects where
in the past it was really really not a
good idea and not only just makes things
more correct but it makes it very very
much faster and this is a huge benefit
particularly to old applets from the 10
and 11 days that maybe had been doing a
lot of multi-threaded work from inside
the browser and here's another
illustration of that you can see this
huge heavyweight master lock that's
basically blocking or barring the
entrance into all these context from all
the Express and in the past you had
cocoa the main cocoa drawing thread and
all of your Java threads if you add
multi-threaded drawing trying to go
through the same locks through the same
area now it's what update one it's just
the Java threats because we have our own
context and because we have our
management down much better it's a lot
less work to go through from multiple
threads and draw to the screen and I'd
like to invite Caroline up to show us
just what we're talking about
alright thanks Matt so this demo is
going to show you the huge improvements
we've made in multi-threaded drawing
support in 142 update 1 first I'll show
you a demo of 142 doing some what they
try to drawing or trying to the first
square is actually the normal retaining
mechanism not multi-threaded where you
draw you make your paint method and the
events get posted to the AWT thread and
the second square let me open this up
again so you can see it again the second
square is using a separate thread and
calling get graphics on each repaint the
third and fourth squares are each using
to drawing threads but they're not
working very well so if we go to 142
update one on demo to on demo to please
you can see that it looks a lot better
it's considerably faster in all cases so
these bottom two squares the third one
is using two threads once drawing in
black and once drawing in yellow and
it's using one cache graphics object to
draw into and the fourth square the
green and black one there's a green
thread and a black thread basically each
drawing up green or black lines and are
drawn into suit two separate cash
graphics objects so if you need to use
multi-threaded drawing it's now a real
option for you in using 142 update one
all right I'll hand it back to Matt
thank you Karen
you sure those were both running the
same hardware okay you're sure okay so
we just showed you that multi-threaded
drawing works a lot better an update one
but we want to mention a few things just
for general advice even though
multi-threaded wrong should work
correctly and and perform well as you
saw up on the screen we still recommend
that you draw on the AWT thread using
the standard paint mechanism and repaint
mechanisms whenever you can whenever
it's reasonable cash graphics objects
are also much safer but we really prefer
you still use the graphics request
method whenever you can and make sure
you have a valid graphics object the
real time to consider multi-threaded
drawing is if you have an event heavy
application if you have a lot of user
interaction and you find these user
events starting to basically clog up the
event threat and really start to affect
your painting performance and you know
it's kind of the old aided you know
never pre optimized try to use the
normal pain mechanism first and if you
have a really heavy interactive
application and you notice that your
painting is suffering based on user
interaction then then would be the time
to start thinking about using multiple
threads cash in context things like that
and all the things that we've been
talking about up until now the
multi-threaded drawing the context
management is this is all referring to
on-screen drawing doing off-screen
drawing into things like a buffered
image from a separate thread is
absolutely fine same as it always was
and so if you have kind of a scene
graphing situation where you're painting
into an image off screen and then
throwing that onto the screen at the
last minute you can do that as you
always have before it should be just as
correct and as fast as it always was so
the next thing we want to talk about is
intelligent screen updates and this is
kind of a marketing term you may know it
more commonly as flushing so the thing
that we were talking about when we were
coming up with this mechanism is that
hardware is getting faster software
optimizations are always happening but
displays are still fix that a risk as
are a relatively pretty slow rate your
application and usually draw it you know
100 100 100 frames per second or more
and you know the display is chugging
along at you know six
70 hurt and so it seems like a waste of
time trying to push all this stuff up to
this up to the screen when it's not
going to refresh any way physically so
what update one does now is it only
updates the screen when it's necessary
when we know it's possible you know you
can see the thing on the screen here
says 100 Java updates is more than 60
screen updates there's no point
naturally performing that flush
operation if you're not going to see it
now this might sound a little scary but
we've done a lot of work to make sure
it's correct and we're not skipping any
paints or any dropping any frames or
anything like that there shouldn't be
any visible difference and well there
should be something of a visible
difference and that's that drawing
should actually be faster as a result
much much to the tune of the the state
management in the state reconciliation I
was talking about earlier we're spending
less time waiting for screen update so
we're wasting less time doing that when
it's redundant anyway and that gives us
more time to process your next paint
event also full screen mode this is a
place where you know we've gotten a lot
of i'll call it feedback on we've
reimplemented it to you know primarily
for correctness and in doing so we also
did a lot in the speed department and
actually it wasn't that complicated a
riorca texture we basically changed from
a blip offer strategy which is basically
copying the bits directly from the back
buffer into the front buffer which which
is a particularly expensive operation we
change that into a flip up for strategy
where you have a front and back buffer
and then just flip those from one of the
other back on the screen the former
front buffer is now your back buffer
where you can do your next frame for
withdrawing and I think Caroline is
going to show us what that looks like
alright so I've got one more another
before-and-after demo here we have a
full screen application running in 142
and what this is trying to do is it's
trying to maintain 30 frames per second
but since we're only getting ten frames
per second there's only going to be one
ball the more it continues to add balls
and tell until it reaches 30 frames per
second so one ball but if we move to 14
to update one on demo two please
you can see it keeps on putting involved
until it reaches 30 frames per second
how high are we going to get wow this is
a pretty fabulous improvement full
screen is now a really good option for
you if you if you want to use it on Mac
os10 all right back to men thank you
very much Caroline I don't know if you
had seen had any of you use puzzle
pirates before when you had seen Scots
deploying demo earlier puzzle pirates
did not used to do too well in full
screen and obviously Scott was demoing
it in gp3 the other day but yeah one
ball at ten frames per second 600 balls
at 30 that's 1800 x I think and that's
all software changes so drug did a good
job there so we also want to talk about
X or drawing last but not least well
maybe it is least we did a lot of work
to improve the correctness of X are
drawing and this has been a perpetual
thorn on our side and the main reason
being is that quartz which is what our
java 2d drawing mechanisms built on top
of doesn't support x or natively and if
you've been to any of the court sessions
you may have heard this from the courts
engineers themselves so we've had to
implement it for you on our own and the
correctness should be there now but
obviously there's going to be a cost
because we've implemented it all on our
own so the performance may not be what
you expect and it probably never will be
because courts does not simply doesn't
support it natively and so what that
means is that you should probably if you
feel like you need to use X or drawings
for some reason you should probably talk
to us to see what you really need to do
and see if we can offer some kind of
alternatives for you things like alpha
composites if you need to do blending
operations and the other thing that
bearing and Tom mentioned already on
Wednesday is that if you're doing X or
to erase things you know you're doing a
double draw to erase some text for
example anti-alias text is not going to
be erased by X or drawing because of the
anti-aliasing operations those are not
taken into account when you do an XOR
operation
so even if that if that was your real
reason for using X or it's not going to
work anyway and if you need to do an
erase operation the best thing to do is
to keep your static scenes that be
before the before modification seen in
something like an off-screen
bufferedimage and then push that to the
screen when you need to do your race or
your undo so what are all the
performance gains as a result of all
these things we're talking about well I
mentioned the context management changes
we're no longer waiting for Coco's
permission to draw we don't have to do
any guesswork on the state from when
Coca may or may not have modified it and
we've got safe context cashing for
multithreaded wrong if you want to do
that the intelligent update mechanisms
it means we spend less time waiting for
the screen and more time actually
executing your pink coat and the full
screen drawing change from a flip them
from a lit buffer to a flip fluffer
tongue twister that changes really as
you saw quite an improvement in
performance and here's just a quick
graph of some of our internal benchmarks
as well as the standard ones like the
Java to deed MO and the swing mark
benchmark in aqua you know we've seen
anywhere between twenty and fifty
percent improvement just with these
software changes in DP three and I think
Caroline is going to give us one more
demo of the overall changes all right
this is the last before and after demo
this is really similar to what you may
have seen in the Java State of the Union
except in Java State of the Union we
compared our selves to windows and here
since we've been talking about the
graphics changes we're comparing
ourselves to ourselves so this is a java
2d demo running in 142 and the important
point to notice is that we're getting
around 115 frames per second alright so
in 142 update one on demo to getting
more than twice back around 200-250
sometimes so these are significant
improvements I think you're really going
to like them all right back over to miss
Thank You Caroline
so I think you can see doing a lot of
work to speed things up for you and also
at you know it's hard to demonstrate
correctness the multi-threaded drawing
demo did a good job at that but
hopefully you guys will see this in your
applications and we'd like to hear your
feedback if it is or if it is not either
way you want to make sure we're doing
the right thing before we release this
update so let's talk a little bit about
tiger I don't have a demo for you but as
you probably heard courts 2d is going to
be open to yell so is it hardware
accelerated using OpenGL in 10.4 and
because we're built on top of course to
do our 2d drawing this is just going to
be faster java graphics for free without
any work that you need to do or even
months of the java team needs to do and
i have again on there because some of
you from the 13 days may remember that
we had an OpenGL pipeline of our own in
Java and you might wonder why that went
away well that's because the gentleman
who were who was working on that for
Java is now working on court and if
you're wondering why hardware
acceleration went away from Java it's
because it went to the whole system
instead and thank you for waiting and
you should have it very soon now and the
difference between the tiger hardware
acceleration and what you may have been
used to in 13 is that it really does
look like quartz in 13 the hardware
acceleration looked like OpenGL and then
Tiger it's going to look like quartz
it's going to have the anti-aliasing and
all the things that you expect from
ports drawing so we've been talking
about what we've done i wanted to talk a
little bit about what you guys can do
this is kind of a repeat of things that
we said last year but they're still very
important tips so i wanted to put them
on here the big thing with images in
particular is that you need to make sure
that you are using an image format
that's compatible to the system and when
i say when you're using images i'm
talking about creating new images that
you're going to draw into the paint onto
the screen later using methods like
toolkit create image graphics
configuration create compatible image
that lines actually a lot longer you
need to do about three or four other
static getters before you get to a
graphics configuration but static images
meaning things that you've loaded from a
JPEG file using image IO dot read new
imageicon so on and so forth those
things will be fine we will optimize
though
for you once they're decoded as far as
new images that you're going to do
drawing operations in to make sure you
use these compatible in JP is things
like particularly the index color
formats that are very popular on windows
because they're low memory those that
image format is not compatible on Mac OS
10 and we're going to need to do pixel
conversion for every operation that you
do and that brings me into that third
bullet there that says if you must use a
non-native image please avoid using
doing direct pixel manipulation because
every time you have to go get the data
buffer and get the raster will need to
convert those pixels from whatever the
store from whatever the image format is
to something that courts can actually
read we'll need to do that every single
time if you use general Java drawing
methods like you know drawstring draw
line we can fake that and not worry
about doing the conversion every single
time and obviously that on the fly
conversion is going to be expensive so
if you can please it will please use the
compatible image formats and try to
avoid using something non-native so
let's talk about some of the new things
it's not exactly new anymore we
announced that I believe in December or
so but we haven't talked about on stage
yet so I wanted to mention the Java AI
is now available on Mac OS 10 you saw a
demo of this Monday night with the Mars
rover the gentleman from the the JPL
labs in in Pasadena and you know
basically bringing high-performance
image processing a number of new formats
that are not available on the standard
java.awt you can you can process him you
can do image processing we're a network
we did implement the native media lib
code from Sun and brought it over to Mac
OS 10 so you can get the performance
that you would expect from j'ai and it's
going to be included with tire it's
currently a download for you and more
importantly for your users if you're
writing in j'ai for Panther it's
available as a download the entire group
will be pre-installed and you won't have
to worry about it and to the same the
same tune Java 3d is also was made
available at the same time it's a higher
level 3d model it's not really a direct
3d programming API it's intended for
higher level seen graphing and things
like that it is platform independent so
you can write applets and Apple
occasions in 3d we implemented the we
implemented Java 3d with opengl and core
audio for the audio and like to have AI
it'll be available as in Tiger and
currently available for panther as a
separate download and there's one other
thing to talk about which is Jo GL I'd
like to bring Ken Russell up to talk to
you about that good afternoon so my name
is Ken Russell I'm at Sun Microsystems
and one of the authors of the Geo GL
package that you may or may not have
used an Gerard and I worked a bit on the
jo GL ports OS 10 so what is this
package its binding a Java binding for
the OpenGL 3d API that works on as many
platforms as we could make it work on in
particular OS 10 works on linux that
works on solaris works on yeah windows
and the intent here is to let you write
portable fast 3d applications in java
the the binding is open source it's part
of sun's java gaming initiative you can
get the source code in java net it's one
of the core gaming api projects he's got
bindings to jo al the open audio library
we've got an input device a binding
called j input which lets you get inputs
from you know joysticks and keypads and
stuff like that some of the features of
this opengl binding versus some of the
others that are out there are first of
all that it supports all the stuff
that's in opengl one dot five so you use
this you get all the latest stuff all
the way to sender extensions you can do
vertex programs fragment programs or
shader objects all this stuff it's got
portable support for p buffers so you
can do hardware so accelerated
off-screen rendering in a portable
fashion get the results on screen very
quickly and where necessary you can
actually dive down to the vendor
specific extensions so it's like if you
actually had to use the wiggle api's for
something on Windows you can sort of
test to see what platform you're on and
then dive down into those AP is if
necessary we are working on
standardizing this binding and the real
reason for this is so that we can get
some amount of
sort of compatibility guarantee for you
the developer on all the platforms that
you're going to be deploying on and the
another intent here is to try to get the
mobile device space and the desktop
device space where j2me and se have a
bit of a rift we're trying to make the
OpenGL bindings for these two platforms
as similar as possible so that you'll be
able to effectively write a desktop app
and maybe when the cell phone market and
hardware acceleration down there catches
up you'll actually be able to deploy
basically the same app on a mobile
device which would be really really cool
so as I mentioned before Jo GL supports
multiple platforms and I'd like to point
out here are two things first of all
even if you're not using the awt and
that sort of framework and you're using
the cocoa Java bindings for Mac OS 10 jo
GL actually supports that configuration
explicitly so you can instantiate a
little mac OS 10 GL impul object and
then treat it in exactly the same way as
you would in a and a WT based joga lap
and it works and we've got unfortunately
we don't have a demo today showing this
off but there has been demos developed
in the past which which have have
stressed this out another thing to
mention is again the full screen support
that was just shown that's working great
with Jo GL & OS 10 so you can write full
screen fully hardware accelerated games
and apps and we're going to actually
show one today that's pretty cool and I
see a point here on the slide that I did
not know which is that this works well
with the new OpenGL profiler on oh wait
is this shark vs ok so well it's working
with the OpenGL profiler I was 10 so it
looks like you can get fairly detailed
information about what's going on in
your apps they're all are also a couple
of pipelines available for the Jo GL
binding and these let you get debugging
and tracing information and this is sort
of a Java specific feature but it's it's
extremely useful when you're trying to
debug something that's going wrong in
your app it'll basically stopped at the
point of failure at the actual OpenGL
call that made the error state in the
state machine and give you a back trace
which is pretty helpful so I'd like to
show you is a really cool demo done by a
couple of crazy guys over in Germany
call
by tonic software I assume that this is
a play on words of Bionic vs Bionic but
so if we could go to the the demo2
machine oh wait aha rongsheng okay but
we're going to see here is a full screen
app full screen little game that you
guys may have seen before this is place
to do okay now what are you looking at
here this is not a new tension written
in Java hello Susan devil said looking
at here or not we knew and you let me
kill some guys here maybe they'll
they'll shut up mute on the keyboard huh
ah misty okay thank you alright so what
we're looking at here is it's not a pure
Java new sort of engine that is written
to use the quake to file format this is
a port to the java programming language
of the original see source for quake to
woo a lot of that can we kill the bad
guys yeah I got to take care of some
business first okay all right yeah okay
so this is a port a straight
transliteration of the sea sources for
quake two that were available by the GPL
on its website to the java programming
language all right they use jo GL at the
bottom layer to get to the graphics
subsystem and they did whatever data
structure conversion was necessary from
c to java to be able to use the pack
file format the original data files from
the game in this port this is running at
over eighty five percent of the speed of
the original see sources
all right at this point there is no
reason to not write your next game in
the Java language this binary runs on
multiple platforms it runs on OS 10 it
runs on linux it runs on Windows it runs
on solaris okay no porting necessary and
now you can do even more cool stuff
because you're in Java you can ship
bytecodes around the net and it's all
safe you can load new stuff in new
functionality download it to the client
you don't have to worry about you know
oh gee did I just open up the door for
the next after virus for example because
you got a security manager in place and
the axis is all safe to the underlying
hardware features so this is actually an
open source project you can go to the by
tonic software website and the demo is
called Jake 24 you know quake 2 to Java
so what you can do is just google for
Jake to and say I think the first hit
their their websites out there you can
get the binaries you can get the source
code and I would encourage you to send
them feedback because I think that this
is just awesome stuff they did what a
lot of us have been talking about doing
for a long time which is G what would
happen if we had the huge commercial
apps side by side C and Java how fast
would it be this is showing that it's
pretty darn fast I think that's pretty
much it ok well thanks again I kind of
hijacked Ken with that opengl profiler
bullet at the last minute we decided he
was going to say this slide so yeah he
didn't know about that i was looking at
talking to some of the OpenGL guys as
well as Gerard about the fact that you
can use the OpenGL profiler to profile a
joggle application now it's not going to
give you what I mean it it is a
sea-based profiler but because joggle or
Jo GL is a rapper it's pretty
straightforward as to what it's talking
about and you get a list of functions
named ABC functions but you know the
names are very similar and it's usually
pretty easy to tell what's going on in
your joggle app versus what the OpenGL
profiler is reporting and again we demo
dick too that's the by tonic de as in
Germany link if you want to check it out
and yeah it's pretty cool we
then we've been playing it quite a bit
so in conclusion we're going to let you
guys out early today because it's friday
but we're working on it we've heard a
lot of your feedback and we know that
we've we've had a lot of work to do and
we've done a lot of it we've done a lot
to improve correctness and drawings
we've done lots of prefer to improve
drawing performance we've done a lot
with system integration especially been
working very closely with the courts
team to make sure that everything in
Tiger is working smoothly not only for
courts but for java as well and we've
got these new AP is available to you
java 3d java AI and of course Jo GL we
want to make sure that you do use your
images wisely use this use the Java API
is to create a compatible format and try
not to manipulate pixels directly if you
if you don't know what kind of format
you're using or what you need to use and
check out joggle and write your games in
Java you really can do it now there's
really nothing stopping you so for more
information obviously we gave you the by
tonic link earlier we've got that we've
got a dmg of course with every session
there's the reference library is
available to you for all the Java items
we should have a tech note coming out
for image performance tips and the kind
of things that I talked very much in
brief today and there's a great article
which is not new at all about painting
and awt and swing and what the expected
behavior should be on a standard java to
platform and these are the kind of rules
that that were guidelines that you
should keep in mind when you start to do
a lot of heavy drawing in Java this is a
really priceless article and it's very
similar to some of the stuff I talked
about today it took a lot of my
information from this from that article
the Jo GL homepage is off java.net and
of course you can go to Java suncom for
anything else you're interested in
that's not listed