WWDC2013 Session 215

Transcript

X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
>> Hello, welcome to
Optimizing Drawing and Scrolling
on Mac OS X [applause].
My name is Corbin Dunn and I'm
an AppKit Software Engineer.
I'm going to be giving this talk
with my colleague, Raleigh Ledet
and let's just jump
right into it.
So what we're going to
be talking about today,
we have four major subjects.
We're going to be talking about
Optimizing AppKit Drawing,
Layer-Backed View Drawing
and utilizing Core Animation
in your views and
how to make it fast.
Raleigh is going to come on
stage during the second half
and talk about Responsive
Scrolling and what you can do
to opt into that and make
it fast on Mavericks.
And then finally,
Raleigh is going to talk
about Magnification
in NSScrollView.
So let's talk about
Optimizing AppKit Drawing
and the best practices that
you can do in your application.
So you're probably already
doing this inside your drawRect.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Inside of your drawRect
implementation, you're looking
at the dirtyRect
and just filling
and pointing your model objects
inside the area that's actually
dirty that you really
need to draw.
The interesting thing about this
is you're probably already doing
setNeedsDisplayInRect on just
that small little rect
that's really dirty
that you need to redraw.
You're hopefully not doing
setNeedsDisplay:YES an entire
view and invalidating
everything
which is not good
for performance.
Of course, if you
have a big red view,
you do a setNeedsDisplayInRect
on this little orange rect,
you do another
setNeedsDisplayInRect
on another rect and
the same run loop pass.
What's going to happen is
your dirtyRect is going
to be the union of
those two Rects.
So what you can do
as a developer
to do better drawing is
inside of your drawRect,
utilize getRectsBeingDrawn:count
which is some older API
that we've had for a while.
You can enumerate all the
dirtyRects and then just fill
in those Rects that are
actually really dirty.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Or, what you can also do here is
just pull in your model objects
that exist inside those
dirtyRects instead of pulling
in everything inside
of your view bounds
or the visibleRect
or the dirtyRect.
This is much more performant.
Another thing you can do is
you can use needsToDrawRect
which is some API where
you can say, "Hey,
I already have this rect, I need
to draw it or fill in my model.
Do I really need to draw it?"
If you do, call needsToDrawRect,
it'll say yes or no
and then you can actually do
your logic based on that answer.
What types of things
should you be doing inside
of your drawRect, we should
only be doing drawing inside
of your drawRect.
You don't want to be doing
network calls, you don't want
to be doing image allocation or
loading, you probably don't want
to be doing file access,
you definitely don't want
to be doing layout or adding and
removing subviews which are kind
of be a recursive loop.
If you add a view inside
drawing, it's going to need
to draw again and
that's not good.
Now, other things that you can
do that are performant, well,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
hiding a view may
be actually better
and faster than adding the view.
So, you can utilize setHidden
to make the views be hidden.
An exception to this, which I'll
discuss a little bit later is
if you're using layer-backed
views
and I'll discuss the
details on why in a bit.
So let's talk a little
bit more about images
and utilizing loading of
images and what you should do.
It's really good to cache images
so if you're using -imageNamed
to load an image, you probably
want to retain it into an ivar.
It's quite possible that AppKit
may not retain that image
and it may be cached, the cache
may go away so if you want
to draw it again and
again, retain it.
Now, if you have a big
image that you want to load,
it's probably a good idea to
not load it inside drawRect
and to asynchronously load
it in the background thread.
So you can use an
NSOperationQueue,
add an operation with block to
do some work on the background.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
You can actually do
your heavy lifting
such as initWithContentsOfURL
to allocate your image.
You can then kind of
pre-cache and warm the image
by calling
CGImageForProposedRect context:
hints and pass in the actual
size which you're going
to be drawing and this will kind
of warm the image up to get be--
to get to be ready
to be drawing.
Then once it's warmed
up, you can kick it back
over to the main queue
and add another operation
that actually updates your
image property of your view
and actually marks the
area need to be redrawn
which will then be
happening on the main thread.
As I mentioned before, you
only want to do drawing
and don't do any layout in
the background and inside
of drawRect so where
should you do layout?
You should probably do
layout in the layout method
or utilize viewWillDraw,
that's the location to add
and remove subviews, mark areas
as being dirty and whatnot.
Definitely, do not
do inside drawRect
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
because it's bad
for performance.
Of course, if your view
is opaque, then you want
to override isOpaque on
NSView and say yes.
If your view isn't, then
you have to still say, no.
But if it is opaque,
and you tell AppKit that,
we can do more
performant operations
by knowing this information.
Another thing that you can do
to make faster drawing is
override wantsDefaultClipping.
By default, wantsDefaultClipping
says yes,
meaning your view
is going to clip
to whatever bounds it
is actually drawing to.
Now, if you constrain
all your drawing
to getRectsBeingDrawn,
you can override
-wantsDefaultClipping and say no
and we won't do that
clipping for you
and it might be faster
for drawing.
There are some methods
in AppKit
which are a little
bit heavyweight
and called frequently
and it's very good
to avoid overriding these
methods if possible.
The class methods are
the GState methods.
So if you're overriding or
calling GState, allocateGState,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
releaseGState and in particular,
setUpGState and renewGState,
we'd frequently see people
overriding these methods,
in particular, renewGstate
to do things to know
when their view's
global position
within the window changes.
So for example, the views
frame hasn't really changed
but its position inside the
window globally does change
and some people are
using this as a hook
to know when that happens.
Instead, it is much better
to use notifications to find
out when your view's
position changes.
Use the
NSViewFrameDidChangeNotification
or the
NSViewBoundsDidChangeNotification.
Raleigh is going to come
up in a little bit and talk
about this a little bit more
with the context of ScrollView.
So, that was discussing
typical AppKit drawing and how
to do some performant things
for a traditional drawing.
Let's talk about layer-backed
view drawing and some
of the best practices for Core
Animation and what you can do.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So last year, we gave a
talk on layer-backed views
which I highly recommend going,
digging up, and watching.
It discusses a lot of
things that are great
to get performant animations
using Core Animation.
I'm going to cover a couple of
the properties here very quickly
to just reiterate how
important they are
to get good fast animations.
I'm going to talk about the
layerContentsRedrawPolicy
and also updateLayer
and wantsUpdateLayer.
So in Lion, we introduced
some new API called
the layerContentsRedrawPolicy.
It has a whole set
of different values.
The one that's the
most important is
that NSViewLayerContentsRedraw
OnSetNeedsDisplay.
So when you set this property on
your view, what it means is you
as a developer using
a layer-backed view,
whenever your content changes on
the view or your frame changes,
you are responsible for
calling setNeedsDisplay.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
We do not automatically call
a setNeedsDisplay on the view
when the frame changes.
And what this does
is it allows us
to utilize Core Animation
animations
to smoothly animate your view.
So prefer to use this
property if you can.
The frame change default
is the default value
and so we require you to set the
OnSetNeedsDisplay version of it
to opt in and please do so
and should get some fast
animations doing this.
So how do we do drawing when
we have a layer-backed view?
But since we added
updateLayer in Mac OS 10.8,
here is the typical flow
path that we do for drawing.
You have a Core Animation
layer and it needs to draw
so it was dirty in some way.
The next thing that happens
is it ask NSView, hey,
do you want to use updateLayer
and depending on what the answer
to wantsUpdateLayer is, we
do two different code paths.
So let's say you say no to
wantsUpdateLayer, which is more
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
of a traditional drawing
in drawRect-based code path.
What happens is Core Animation
creates a CGContextRef the size
of your view.
That can be thought of as
an image the entire size
of your view then
uses a delegate method
drawLayer:inContext
in order for AppKit
to call your drawRect
implementation.
Then whatever you draw and setup
your views bounds is captured
into a layer.contents
as an image.
Now, you may want to
use wantsUpdateLayer
and I'll describe
why in just a moment.
The way that works is we use a
different delegate method called
displayLayer and then we call
instead of AppKit updateLayer
which is a method
that you can override,
you can set whatever
properties you want
in the layer including
the layer contents
to provide your representation
of the view.
So let's take a look
at doing that.
So here is an example
using wantsUpdateLayer.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
You override wantsUpdateLayer,
you say yes.
Now, instead of getting
a drawRect,
you get an update layer call
and you can set layer properties
to represent how your view
should be represented.
So for here, we don't have
to create the backing store,
we don't have to create
that image that backed
your view. Instead, you
can just set properties
such as the background
color, the border color,
the border width and you can
represent your user interface
this way without
having to use memory
for the actual backing store.
This is a very efficient
way to update layers
without using a lot of memory.
Speaking of properties
that people can set
on Core Animation
layers, you probably want
to avoid properties
that are expensive
so if you're using the
cornerRadius property, the mask,
filters, backgroundFilters,
those are all expensive
properties
which might make your layer
rendering a little slow.
So if possible, try to
do your UI in another way
that can avoid these
properties in order
to get better performance.
As I mentioned before,
if your view is opaque
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and you're saying
yes from isOpaque,
that value is directly
assigned to a layer.
And so, opaque layers are much
faster to composite together
so prefer to say yes
from isOpaque and realize
that property is
transferred over to the layer.
Now, let's say that you are
drawing a big view inside
of your drawRect
of document view.
So you're drawing this big
electric bug picture here.
What you're probably doing
is you're probably clipping it
so you have just a tiny little
area that you want the user
to see so you have it
inside of this NSClipView.
The ClipView in itself is
inside of the NSScrollView
so you can scroll around
to whatever portion
of that view you want to see.
So how do we do this when
you're using layer-backed views
in AppKit?
We have a special layer called
the Tile Layer that we use
in AppKit to take your big view
and just chop it up into lots
of little individual tiles.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Now, your view still has
just the visible area
because the VlipView is
clipping to what you just see.
So what we can do is we can
make intelligent decisions here
where only the tiles
that intersect
in that visible area are things
that are going to
actually be drawn.
Everything outside of it
doesn't have to be drawn.
Add a little asterisk next to
this because Raleigh is going
to cover some details where we
might do something differently.
So why is this important
to know?
Well, all those tiles
inside of your visible area,
each one is going to get its
individual drawRect in order
to fill in its contents.
So if you're doing the things
that I recommended at the start,
we're properly watching
for the dirtyRect
and getRectsBeingDrawn, you'll
only fill in your model area
and only draw in the areas
which you are drawing
to an actual individual tile so
it's very important that you do
that and it's important
to be aware of
that we might be calling
this more than once
for one particular visible area.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Of course, we're actually
much more intelligent
than the picture I
was showing before.
If your view is very
skinny and tall,
our tiles might be really
wide and not very tall
and so we're creating tile
sizes that dynamically change
to be the most efficient
for your application
and your view size.
It's important to realize this
because our actual
implementation
and how we do the tiling
may change over time
to make it more efficient.
So another thing you can do
to get performant drawing
in a Core Animation
layer-backed view is to try
and reduce your layer count.
So let's take it a
typical layer-backed view
and see how the hierarchy works.
So you have your
top most view here
and you do setWantsLayer:YES.
Implicitly, all your
subviews are going
to get their own backing layers
so your big parent
one has a layer,
all the subviews each have
their own backing layer.
What could be the
problem with this?
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Well, as I mentioned before,
if you're using drawRect,
each of those views might have
their own backing store, again,
their own little image contents
and they might be compositing
on top of each other which
might be a waste of memory.
You also might have a
high composition cost
where if you're not
animating that view around,
why have a bunch of layers there
when they're not really needed.
And unlike just regular AppKit,
hidden layers actually might
have a composition cost where
if they're in the layer tree,
they still have to be processed.
So if you a layer-backed view,
it's probably more performant
to actually remove the
view from the hierarchy
as opposed to hiding it.
And what I mean here having a
couple you are hiding is fine,
but if you're actually
hiding hundreds of views,
that's probably not a good
idea and you probably want
to remove them from
the hierarchy instead.
So, another way to reduce
your subview count--
or sorry, your sublayer count is
to use some new Mac OS 10.9 API
called CanDrawSubviewsIntoLayer
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and the setter,
setCanDrawSubviewsIntoLayer.
Let's take a look
at how this works
and why you would
want to use it.
So in the same example here,
you have setWantsLayer:YES
on your top most view.
In addition,
you do
setCanDrawSubviewsIntoLayer:
YES.
And what's going
to happen is just
that top most subview is
going to get the actual layer.
All those other subviews will
no longer have their individual
layers instead, they're
all drawn
with their drawRect
implementation
into the parent layer.
The interesting thing here is
even though the parent layer--
parent view has a layer, if it
says yes to wantsUpdateLayer,
it's not going to get an
update layer call, that view
and its layer and all the
children must utilize drawRect
to do their drawing.
But this is a great way to
reduce your layer count.
So, one interesting thing here
is that what if you did want
to view a subview that has
its own layer because you want
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
to animate that button around,
you can opt in one individual
subview really easily
by just doing setWantsLayer:YES,
and that view
which normally would
have been drawn
to parent layer will
now get its own layer
and you can animate
it around smoothly.
So why would you want
to reduce these layers?
Let's get a direct example
where it might be more
applicable for what you can do.
So here's a view base table view
and you want to reduce a lot
of these subviews
into a single layer.
You might have made your layer--
or you might have made your
ScrollView layer-backed
so that you can actually
get fast mode--
fast smooth scrolling and also,
you can do cool row animations.
But the row animations
themselves are just
on the individual row views.
And so what you can do is you
can collapse these row views,
the subviews layers
into one single layer.
So what you can do for each
of those table row views do
setCanDrawSubviewsintheLayer:
YES for each of them and all
those individual subviews,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
the image, the text and
whatnot will be drawn instead
of having individual layers into
just one layer of the row view.
One interesting caveat
to note here is
that for text fonts moving
to work such as the title
of that word there, it must
be drawn into an opaque area
so the row view or
something else that was drawn
in that layer must have filled
with some opaque color in order
for fonts moving to work just
something to be aware of.
So that was just
discussing layer-backed views
in Core Animation, I'm now
going to bring up Raleigh Ledet
to talk about Responsive
Scrolling.
>> So you've seen the demos
already for responsive scrolling
and for an overview, I want to
give you another demo of that
so let's take an example.
So, I'm going to go ahead
and turn off responsive
scrolling globally real quick
and I'm going to run my
little test app here.
And this test app does lots of
horrible things during drawRect
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
so your drawRect performance
is very poor and you can see
that when we try and scroll the
scrolling performance is painful
in this app.
So we're going to go ahead and
turn responsive scrolling back
on and we'll go ahead
and run the same app
and I'll just [inaudible] so
you can see it, there we go.
And now, when we do scrolling,
we have nice smooth 60 frames
per second scrolling and--
[applause].
Thank you.
And, you know, that's--
that was the point
of responsive scrolling, we want
60 frames per second buttery
smooth scrolling, I could just
keep listing bullet points
of describing this all day long.
We're excited about it.
And I'm going to give you
a quick brief overview
of how this works and you've
already seen this but--
say you have your document
view which is obviously inside
of a clip view and only a
small portion of it is visible
and traditionally,
that's all that was drawn.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
When we get responsive
scrolling,
we're going to ask your
document view to draw portions
that aren't visible and
we call that the overdraw.
And now that once we
have this overdraw,
on a background thread,
we can go ahead
and change what the user sees
on screen very, very quickly
to any portion that we
have that's already drawn
in the overdraw.
And in a nutshell, that's
all that we're really doing
with responsive scrolling.
Under the hood, there's a
lot going on and I'm going
to cover some of those details.
Particular, I'm going to
talk about the overdraw model
and exactly how it works.
The event model, there's lots
of big changes going on there.
Some API that we have to
help you adapt to the changes
that we have and what you
need to do in your application
so that you can make sure
that your application adopts
in to responsive scrolling.
So let's kick off with overdraw.
The main thing about overdraw
is still main thread driven.
Your drawrect calls, they're
always going to be called
in the main thread so
you don't have to worry
about doing any additional
locking
that your app wasn't
already doing,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
you can access the
view hierarchy
and you're data model
just like you always did.
Of course, drawRect is
now going to be called
with nonvisible Rects.
So as Corbin mentioned
earlier, it's really important
that you respect
those dirtyRects
and you only do drawing and
you do it just in the areas
that we're asking you to draw.
When your app is idle, that's
when one figures a good time
for us to go ahead and ask
you to generate some overdraw.
We're only going to ask you
though for a little portion
of overdraw that way
your drawRect can be fast
and if the user tries
to interact
with the application
while you're in the middle
of generating overdraw, that
overdraw drawing will be quick
and the user will not see any
lag between trying to interact
with your application while
you're generating overdraw.
So last just for a little
bit, that will get drawn,
we have some overdraw,
that's great.
The application is still idle
so we'll say, "Hey, great,
let's draw a little bit more
and we'll draw some more
and this is going to go around
all the access that you have
for your scroll view."
This one is only
doing vertically
so we'll just go up and down.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So we have a little bit more.
If your app is still idle,
we just continue this process
until AppKit has decided
that we have enough overdraw
to be responsive for what
the user is likely to do.
Now, we don't want to
draw your entire document,
that would be a huge backing
store and that would take a lot
of memory and it would even
take a lot of power just
to have you draw that
whole document view.
So we don't want to do that.
AppKit plays a very
careful balancing act
between how much overdraw we
have so that it's responsive
for the user and not
using too much memory
and not using too much power
to be able to accomplish this.
And we even go a step
further and where possible.
We'll make sure that
those backing stores
that we're creating to hold
the overdraw are purgeable
by the Kernel.
So if there's memory
pressure on the system,
that memory can be
freed by the Kernel
without even waking
your application
and when your application
becomes active again,
we'll notice that and we'll ask
for those areas to be redrawn.
And this is great and in
general, you won't have
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
to do anything but sometimes
it's not quite enough.
You need to be able
to watch what's going
on with overdraw
and react to that.
One example is if you are
adding your own subviews
that you only want the
subviews for your documents
that are visible to be there.
This is a common technique.
Table view does this for
a view-based table views.
Now, you need to make
sure with overdraw
that those subviews exist
in the overdraw area is well
so that they are
always going to be ready
for when the user
scrolls to those.
So we have new API that you
can adopt and you can play
around in the overdraw
world then make sure
that your content is
going to be available.
And let's give you
an example of that.
In your document view, you would
override PrepareContentInRect
and we'll go ahead and pass
in the rect during idle when--
whenever we want to
prepare some new content.
And the rect that
actually gets passed
in is not just a little
sliver that we're going
to eventually ask you to draw.
It includes the entire overdraw
area that we want currently
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
which is always going to at
least include your visible rect
and in this case, the new
little section of purple here.
Then you go ahead and you
prepare you content as needed.
In this example, this is what
we're going to add our subview
to the view hierarchy because
this is of course called
on the main thread always.
So, adding a few
to the hierarchy
at this point is perfectly safe.
Now, when you override
and PrepareContentInRect,
you're not just reacting to
the rects that we're provided
in you to do overdraw.
You can actually be
an active participant
in deciding how much
overdraw is being used.
And in this example of the
subviews there, we don't want
to clip half of it
off in the overdraw,
it's just a little bit.
So when we tell super how
much overdraw you've prepared,
we want to extend it just a
little bit and cover the edge
of that subview and that will
be a little bit more efficient.
So you can be an active
participant in this.
After you return from
this, little bit later,
we'll go ahead ask that
new section to be drawn
and we'll have that in the
overdraw ready to be scrolled
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
to for the user at
a moment's notice.
Of course, if you have a
still idle, we'll go ahead
and start asking
for more overdraw
and as you've seen before, we'll
just continue this process.
Continue to call,
PrepareContentInRect
and to just further
drive the point home.
Now, the rect thats getting
passed includes that overdraw
that we drew earlier, the
visible rect and the new section
of overdraw that we want
so you can make sure the whole
area is properly prepared.
Again, when I've talked about
being an active participant
under very specialized
circumstances you might know,
when is the better
time to end overdraw?
AppKit plays is very
careful balancing act,
there are some situations
out there where you know
that pulling in a
certain amount of content
for you makes more sense
than AppKit would
otherwise want to do.
And in that case, if the rect
you return as super as the same
as the previous time we
called PrepareContentInRect,
then we'll of course-- we won't
ask for that area to be drawn
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and we're going to stop
asking for overdraw
on idle unless the overdraw
gets blown away for some reason
and then we'll come back on
idle and rebuild it from scratch
where you can terminate it
again through the same mechanism
that once you return the same
rect twice in a row, we go ahead
and stop asking for it.
Of course, now that you have
content that isn't even visible,
if that content becomes dirty,
please setNeedsDisplayInRect
and just those portions
that have been dirtied
and we will go ahead and have
those redrawn when appropriate
and that way when we
scroll to that section,
it's always the most up to
date content for the user.
There's some special
circumstances where you need
to tell AppKit that you should
totally get rid of any--
of all that prepared
overdraw that we have.
And you can set the
PreparedContentInRect
in this case to the
visible rect.
For example, the user might
have changed something,
choose a different group
of items which are going
to totally change the
content that you're showing
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
in the scroll view and the
original prepared content there
is completely not even
appropriate anymore.
So, you want to go ahead and
tell AppKit to just drop it all
and we'll drop down
to the visible rect
and the next pass
to the run loop.
We'll draw the content
for the visible rect
and that's what the user
will see, and during idle,
we'll build it back up with your
new content that's appropriate.
So, all of this with overdraws
are done on the main thread,
drawRect is being called
your nonvisibleRect,
make sure your drawRects
are only doing drawing
and they're as fast as possible.
In general, let AppKit balance
the amount of overdraw along
with memory empower usage and
purgeable memory and handle all
of that complicated matter.
Only if you have specialized
needs then you can go ahead
and cut that short a
little bit perhaps.
In other circumstances like when
you're adding your own views
and then make sure that the
prepared content is there
in the view hierarchy,
then go ahead
and use the prepared content
rect API that we have.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So that's overdraw.
Let's move-- switch gears and
talk about the event model.
A lot of exciting things here
as already been pointed out.
But let's back off a
little bit and talk
about the traditional way
that scrolling is handled.
You got a scroll
wheel event comes
in through your scrolling
device.
It gets put into an event queue
and your main run loop is
running on the main thread
of your application
handling event sources
and other run loop sources,
pulls the scroll wheel event
out of the queue, we hitTest
that scroll wheel event,
they get passed to
your document--
your some subview of
your document view.
And then we call
scroll wheel on that,
that goes up the responder chain
finally gets to scroll wheel
and in that scroll view,
then that scroll view
moves your content,
withdraw the little section of
content that became available
and we'd let the run loop go
back to handling event sources
and the next scroll
wheel comes in
and we do the whole
process all over again
for every single scroll
wheel event that comes in.
Well with Responsive scrolling,
we break that cycle and once
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
that first event comes in
as soon as it gets to--
in a scroll view, and
the scroll view sets
up concurrent tracking thread
and it will now handle the
events on the background thread.
The scroll wheel events from
the track pad are now going
to be routed to this private
event queue so you won't be able
to see them at all and they will
be processed by the scroll wheel
and now your main thread
is allowed to just run
as it normally would
processing other events
and other run loop sources.
Let's dig in a little bit more
with the concurrent
tracking thread
and inspect a little bit further
on exactly what its doing.
It's pulling events from the
event queue and it figures
out where we want to
move the scrolling.
And so we can change that
do the user very quickly
on a background thread, anywhere
that we have the overdraw.
So let's say it lands
right there,
course your main thread
run loop is still running,
it's still doing its
thing respecting timers
that are firing.
But if you would ask
your view hierarchy
at this point what the visible
rect is, it's still going
to say is that blue
square up there.
Though what the users sees on
screen is the red dashed area.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So, that's an important thing to
realize as what the users sees
on screen can be different
than what your main
thread is reporting.
Now, the concurrent tracking
thread whenever it updates the
screen, it goes ahead
and it issues a synchronization
request on the main thread.
This gets run in
the main thread,
it talks it in a scroll view and
it actually does the scroll too
and all of your view properties
are going to be updated
and their visible rect will
be the most up to date version
of the visible rect that
we possible can have.
And normally and traditionally,
this would cause drawRects
to occur but hopefully here,
since we already have overdraw,
this is just updating
the view frames changes,
this can happen very,
very, quickly.
And if-- and once the
synchronization occurs,
it'll match with what we have
on the screen and every--
everything will be in sync.
And if this happens in
exact same display refresh
that the concurrent tracking
thread move things then
everything appears
to be in sync.
The other thing that the
synchronization request does is
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
if your app-- if your main
thread is otherwise idle,
it'll ask for some prefetch.
So we'll do some more overdraw.
In this case, the
concurrent tracking thread
and the synchronizer know
what direction the user is
scrolling in.
So we ask for prefetch
in that direction.
So unlike idle which is
trying to get a general case,
this is more specific and
we can get drawing ahead
of where the user is
going but it does bring
up an interesting situation.
What happens when the user
tries to scroll to area
that you haven't been
able to catch up to yet.
Well, in that case,
we have to back off
and respect what the main
thread can keep up with.
We don't want to show blank
content that could be drawing
and confusing to the user.
So we end up-- what's we run
out of overdraw, we have to slow
down to whatever the
main thread can do
and if the main thread
catches back up,
because maybe it was
just a long processing
of some timer information,
once the main thread
can catch back up,
if the user is still
scrolling, we can get back
into responsive scrolling.
But it's a situation
we want to avoid.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So, my point here is
it's not a silver bullet.
Responsive scrolling works
really great to make sure
that it's responsive right away
when the users starts scrolling
and that there are
any little hiccups
that your main thread might
have in responding to timers
or network request or anything
like that won't interfere
with user's experience
but at some point,
if your app can't keep up and
the user is scrolling fast
and they're scrolling
far, we'll have to drop
down to whatever your
main thread can handle.
So quick overview,
event tracking is now
done concurrently.
Once we get that first
scroll wheel event,
you won't see the other ones
until the gesture
is fully completed.
What's on the screen may not
match what the main thread is
reporting as the visibleRect
and it's not a silver bullet.
So make sure you drawRects
are as fast as possible
and you're only doing
drawing in those drawRects.
Let's go ahead and move on to
some API and how you can play
with this brand new world.
Overriding scroll wheel
is obviously isn't going
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
to work the way that it used
to because scroll will use
to see every single scroll wheel
event in your document view
or in your subclass and
that's not the case anymore.
A better way of watching
scrolling changes is
to watch the clip
views bound changes.
And the clip view of
course is the contentView
of the scrollView and you'll
need to tell the clip view
to pout-- to post its
bounds change notifications
so setPostBoundsChanged
Notifications: YES,
and then once the clip view
is posting its bounds change
notifications, you
can ask to observe
that on the notification center
with the NSViewBoundsDidChange
Notification on the clip view.
Now, this is better in general
then overriding scroll wheel
because now you'll be informed
anytime the scroll wheel--
the scroll view scrolls
which could be in response
to a scroll wheel event,
it might be in response
to the user moving the scroll
bar, it might be in response
to keyboard access or it could
be even you've programatically
moved the clip view bounds.
And this way, you'll catch all
of those and you can respond
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
to them appropriately
in your application all
in one central case-- all in
one central place without having
to have codes sprinkled all over
your application to handle that.
So this is the way that we've
been suggesting that you watch
for your scroll view
bounds changes instead
of overriding scroll wheel.
However, there's a case
where you really want to know
that the user initiated
scrolling.
Well, until now,
we've implemented some new
live scroll notifications
and you can get exactly
that information.
You can find out when the user
starts to do a live scroll,
you'll get a whole bunch of Dids
as the user is scrolling every
time we move the content.
And then when the last
scrolling completely ends,
you'll get a DidEndLiveScroll.
So this worked great
with the gesture capable
scrolling devices.
And what we do here
with these devices is
as the user scrolls
multiple times consecutively,
you will only see one bracket
of will start and did end.
So even though the user
scrolled three or four,
five times in a row
will coalesce those all
into one will start, a whole
bunch of dids and did end.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Not only that, we will
actually extend this
across the animations.
So if the user goes into
rubber banding for example
and there are no more
events coming in,
there's that little bit of a
rubber band animation going on.
We don't call dead-end until
the animation completes.
And every frame of the
animation we issued dids.
So you've got a will start,
some dids, the animation starts,
you'll get a few more dids.
When it ends, you'll get a--
in a scroll view
did end live scroll
and the scroll is
completely over at that point.
We take it a step further
and says this is user-initiated
scrolling, we can--
we know when they're
tracking the scroll bar.
Now, this is checking the
scroll bar still driven
on the main thread in this case
but we know when it starts,
we know every frame
that happens in between
and we know when it stops.
So we'll go ahead and
report those notifications
in that case.
We'll even do this for
keyboard things, page up
and page down for example.
Those are animations generally.
Well, we know when
the animation starts
and we knows when--
we know when it ends.
And then all of these
cases, the--
these notifications are being
issued on the main thread.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So that's an important
thing to point out.
Your notifications-- these live
scroll notifications are always
issued on the main thread.
But this device is a
little interesting.
If you look at the devices
that actually have a physical
scroll wheel or something
that has like a scroll ball
like the Mighty Mouse, it does--
we don't know when it starts,
we don't know when it ends.
All we know is that a
scroll wheel event happened.
Another one might happen
soon, it might not.
So in that case, we issued just
a did live scroll notification
and of course, that's
going to happen
on the main thread just
like all the others.
So, if you're looking for
this and you're looking
for user-initiated scrolling,
it's important to realize
that some devices
won't be bracketed
with the will start
and a did end.
If you have some floating
subviews in your content,
like let's look at
this table view here.
It's a floating group row.
What's going on here during
scrolling is that your--
generally, you have a subview
in your content, it scrolls
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and you change the
frame of this view.
So to the user, it appears
that it's floating
above your content.
It hasn't moved.
Though in reality, it has moved
within your view hierarchy
or for moving the content
on the background thread
that can get out of sync.
And it's a lot of code
that you shouldn't have
to write for yourself anyway.
So in a scroll view, we have
this new API addFloatingSubviews
for access, you tell us which
access you want this view
to be floating on and we'll go
ahead and put in a special place
in the view hierarchy
and scrolling will be
handled automatically in sync.
So if you're scrolling ob the
access is not floating on,
it'll scroll that content in
sync with the clip views content
and when it's floating, it's
just handled automatically
for you without you having
to update those frames.
So, it's less code for you
to write, a lot easier to do,
and this is what table
view does automatically
for responsive scrolling
on 10.9.
So that was a whole-- there's
a whole lot there going
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
on with responsive scrolling.
In order to adopt in
to responses scrolling,
you have to be linked
on 10.8 or later.
Your window that your
scroll view was in,
its alpha must be 1, it
has to be completely opaque
and your document must
not be an OpenGL context.
These are the absolute
basics that we need
to start adopting your scroll
views into responsive scrolling.
We have a few more.
But from that-- this point on,
we try to make it as automatic
as possible, we want you to--
have to do as a little
work as possible.
But as you know,
scrolling is the cooperation
of a scroll view, the clip
view, and your document view.
If anyone of these views for
whatever reason can adopt
into responsive scrolling,
then as a collection,
these three views won't
do responsive scrolling.
I'm going to talk about some
more things that you need to do
to adopt in the responses
going in the next few slides.
But from this point on, you
can use this explicit API
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
isCompatibleWithResponsive
Scrolling
to bypass all those checks.
When you return yes,
we won't even check
and we'll just adopt you
in the responsive
scrolling from this point on.
Obviously, you can override
scrollWheel or lockFocus,
scrollWheel in particular
as we've covered,
there's a huge behavior
change here.
So if you notice that,
your scroll view subclass,
your clip view subclass,
your document view is overriding
scroll wheel then we're going
to adopt you in to
responsive scrolling.
So please as I've pointed out
earlier, use other API and try
and just remove your scroll
wheel override all together.
Or if you absolutely have
to have it and you're OK
with just peaking at the one
event because you need to decide
if it's a scroll or not period,
before you have been passed it
to super, then go ahead
and override scrollWheel
and in the class that
you override scrollWheel,
override is compatible
with responsive scrolling.
Remember, it's a class
method and return yes.
lockFocus turns out with the way
that we do overdraw
just isn't applicable.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So, please stop overriding
lockFocus.
But, you can-- like I said,
with the previous slide,
this is something
that you can override
with the
isCompatibleWithResponsive
Scrolling and you can find
out what happens in
your application.
From this point on,
we have two models
that we've been talking about,
the traditional drawing model
and the layer-backed
model that we have
and Corbin covered
the various cases.
Responses scrolling
support both.
If you're using traditional
drawing then copiesOnScroll
which is a property actually on
the clip view, but there's also
on the scroll view and it
brought it over to the clip view
and there's little check you
can check in interface builder.
That should be yes and
isOpaque must return yes
for the document view.
The isOpaque one
is really important
because as Corbin
covered earlier,
in order to get fonts moving
right, your checks needs to be
on an opaque-- needs to
have opaque pixels in order
for fonts moving
to work correctly.
And if your isOpaque
is returning no
which is the default case, then
we're not sure what's going
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
on there and we won't
be able to adopt you
in to responsive scrolling.
However, if you know that your
view has some transparent parts
but those transparent
parts don't have any text,
then go ahead and override
the isCompatibleWithResponsive
Scrolling and you can have
a non-opaque document view.
Or you can just go layer-backed.
We don't even check
the copiesOnScroll
and isOpaque if you layer-back.
But when you go layer-back,
you need to make sure
that your layer-back at least
at the end of scroll view layer,
in a scroll view level, or any
of the scroll views ancestors.
You can do that by calling
setWantsLayer on the scroll view
or you have a scroll
view subclass,
you can just return
yes from wantsLayer.
An interface builder in
the Core Animation section,
there's a nice little check box
and that check box is the same
thing as saying setWantsLayer
and so you would do that on a
scroll view or its ancestor.
And also, as Corbin mentioned,
I want to reiterate the
canDrawSubviewsIntoLayer.
There are different
performance characteristics
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
when you go layer-back,
when you have--
you have to composite all those
different layers together.
So, depending on your
application and the amount
of memory that all these
layers use and what--
and exactly what you're
animating, you can decide
at what is the appropriate
level that you want
to collapse these all down
into one layer and the--
Corbin's example that he gave
was great, you might want
to do it for a view-based
table view
on the row view for example.
And that way, you can animate
each individual row smoothly
and quickly, but this will be a
lot less layers for us to have
in the view hierarchy
to composite together
for each frame.
We have some support in Xcode
for helping you debug
some of this stuff.
When you're running
application at this new menu
which has some neat--
some neat options,
in particular Show
Responsive Scrolling Status.
If your scroll view is not
opted into responsive scrolling,
it'll look like this and
well, we don't want that.
What we want is this.
We want green.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So, if everything turns out--
everything is laid out
appropriately, and we can opt
in your view, all of the views
into the responsive scrolling,
your scroll view or subclass,
your NSClipView subclass,
and the document view, it'll
be green and this is a way
of being able to provide
that feedback to you
through the debugger that your
scroll view has completely opted
in to responsive scrolling.
So in summary, we
really want to make this
as automatic as possible.
You can explicitly opt in but
do that as a last resort please,
but there are cases where
that's the only way to do it.
Think carefully about
layer-backing
if you're not already
layer backed,
if you want to go layer-backed,
there's some difference
performance characteristics.
So, but for responsive
scrolling,
we'll support both layer-backed
and traditional drawing.
And you can use Xcode to verify.
So that comes out to be handy.
So, thats responsive
scrolling and we'll go ahead
and move on to magnification.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Specifically, in magnification
as relates to NSScrollView.
And 10.8, we introduced
magnification support natively
into NSScrollView
and all you have
to do is set the
allowsMagnification property
which you can also do
an interface builder.
There's a nice little check
box and you could check that.
And your scroll view will
response to the pitch gesture
and we'll do zooming
and magnification
for you automatically.
When it comes to responsiveness,
if your scroll view is opted
into responsive scrolling,
magnification is still main
thread driven at the moment.
But you'll likely to have
overdraw and we can use that.
So during the magnification
gesture, we use that overdraw
and we scale your
existing content.
So if you have content like
this and you zoom into it,
during the overdraw, it
will scale that content
and it will look
kind of like this.
But when the gesture
completes, we'll go ahead
and redraw the visible
rect when the gesture ends
and we'll get the nice
crisp content back in there.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
But during that whole gesture,
it was nice and responsive
to the user, we weren't
even calling drawRect,
so that was great.
If you're going the opposite
direction for example
and you get to here and we run
out of overdraw, we don't want
to draw blank content
on the sides there.
So in this case, if we run out
of overdraw, we have to pause
and wait for new drawing to come
in since this is all
main thread driven
and so the user will see a
lag in the responsiveness
of your magnification here.
So make sure you
drawRects are as fast
as possible but they'll pop in.
During the middle
of the gesture,
if new content comes in,
that content will be drawn
at the appropriate scale because
your drawRect will be called.
Likewise, if you dirty
any content in the middle
of a magnification gesture,
we will redraw that content
at the appropriate scale factors
but that will require
a drawRect right
in the middle of your gestures.
So you want to try
and prevent that.
So when it comes
to magnification,
your drawRect speed is crucial
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
to getting responsive
feedback to the user.
We do have some live
magnification notifications,
these were introduced with 10.8
when we introduced
magnifications NSScrollView,
the WillStartLiveMagnification
Notification
and
DidEndLiveMagnifynNotification.
These are great that you can
perhaps stop doing some things
in the main thread, turn off
some timers or pause some things
so that you can devote as much
as your resources as possible
to being responsive to
what the user is the doing
with the magnification gesture.
If you're overriding clip
view, one of the reasons--
main reasons people
override clip view is
to center the content
in your clip view
when your content is smaller
than the size of your clip view.
And to do that, you override
constrainScrollPoint.
And with magnification,
if you zoom pass the
minimum amount here,
when the user removes their
fingers, we want that to animate
to the center, but
this is what happens.
In constrainScrollPoint,
you're given a point
and all you have is whatever
the current bounds are
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
of the clip view.
You don't know where we're
going to, what the new size
of the clip view is going to be.
So you can't give us an
appropriately constrained scroll
point and your content isn't
centered like you want it to be.
So we've deprecated
constrainScrollPoint
and we've replaced it
with constrainBoundsRect.
We'll go ahead and
pass in a complete rect
so that you have what we want
the proposed new size to be
and you can constrain
that hopefully,
all you're changing
is the origin.
But now, as the user
pinches and go
and pass the minimum
allowed size so that we need
to bounce back, when they
let go, we'll animate
to the appropriate
centered position.
So, for those of you that
are overriding clip views,
please add constrainBoundsRect
and override
that in your application.
So in conclusion, we covered
a lot in this session.
We talked about optimizing
your AppKit drawing
and layer-backed drawing
and optimizing that and some
of the performance
characteristics
with layer-backed drawing.
These are still very important
with responsive scrolling
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
as you've seen, it can get
over a lot of rough edges and--
that the main thread might
be doing and be responsive
to the user and that is
a tremendous advantage
for your application as you've
seen in that demo application,
it can make a huge difference.
However, it's not
a silver bullet,
so make sure your drawing is as
fast as possible at all times.
So all-- everything that
we said at the beginning
of this talk is very important.
And we've also covered
magnification
and how we've made
some changes there
to make it a little bit more
responsive even though it's
still main thread driven.
For more information, there's--
you can see our App
Frameworks Evangelist Jake
and we have the documentation
in particular the Core Animation
Programming Guide has got some
really nice information there
for when you're making
your views layer-backed
and doing animations.
The Developer Forums and I
didn't get a chance to put
on the slide but make sure
you read the release notes,
we cover a lot of details and
everything I've talked here
in the release notes as it
relates to responsive scrolling.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
The Best Practices
for Cocoa Animation
which actually just occurred
right before this one,
but if you haven't seen
that one yet, please go back
and watch the video, it's
another great session that you--
that relates to responsive
scrolling
and animating your views.
And that's it for
responsive scrolling.
Thank you guys for coming out.
I hope you enjoy the
rest of the show.
[Applause]