WWDC2015 Session 217

Transcript

X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
[Applause]
>> RALEIGH LEDET: Good morning.
Welcome to Session 217:
Adopting New Trackpad Features.
My name is Raleigh Ledet, I'm an
AppKit engineer and I'm master
of using the new
Force Touch trackpads,
so this is what we're
going to be talking about,
the new Force Touch trackpads.
They're very, very cool
pieces of technology.
What makes them different
than our previous trackpads is
that there isn't a physical
button for you to press on.
Instead, we have these
four force sensors
so we can measure how much
force the user is applying
to the trackpad and
then we marry
that with our Taptic Engine.
And when we've determined
that the user has pressed
enough force on the trackpad
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
that the user has pressed
enough force on the trackpad
to issue a mouse down, we go
ahead and use the Taptic Engine
and we yank the trackpad
sideways just a little bit,
and the user gets the feeling
that they have actually
pressed down on a button.
And so, the trackpad
moves sideways,
but your brain makes you think
that you have actually
pressed down on a button.
It's a really awesome
sensation and it's really neat
on how your mind is tricked
into believing you've
pressed a button.
We have some of these
downstairs in the lab,
if you haven't experienced
one of these yet,
please come down and try it out.
Another quick look at the
Taptic Engine going there --
pretty awesome.
So, to recap, you get a
little bit of pressure
and you get a click, but
allows us to recognize
when you apply more pressure to
the trackpad and we can sense
that as what we call
a Force click,
and you'll get a little
bit more haptic feedback
from that as well.
So, you can do lots
of interesting things
with the Force click.
In fact, let me show you some
of the interesting things
you can do with that now.
I have up here, you can see as
I'm moving the cursor around,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
I have up here, you can see as
I'm moving the cursor around,
you can see the force that
I'm applying to the trackpad.
If I go down to the next level,
you get down to Force click.
And so, everybody was
able to hear that?
I've added -- you know, normally
your computer doesn't play a
sound when you do Force
click; I've added that in
because you can't actually feel
the haptic feedback since you're
over there and I'm over here.
So, you'll be able to watch
as I do this demo the force
that I'm applying
to the trackpad.
One of the things that you
can do is renaming files
in the Finder; sometimes
it's kind of tricky,
you have to click on it, and
you have to do it a second time
in such a way that --
and I missed it again.
You got to do the right timing
so you don't double-click
and open it.
But with the Force Touch,
you just Force click on it,
it goes right immediately
into editing the filename.
It's a lot easier to do; that's
just one of the small things
that you can do with
Force click.
I'm going to open up
TextEdit real quick.
Let me grab the window.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
You have seen this example
as well, you can Force click
and you can get Quick
Look to come up.
It's kind of neat, you can
actually sit here, you can play
around with the animation if
you adjust your force levels.
That's another feature
that we have.
Lastly to show you some
things real quick, you know,
I could change how fast
by varying the force,
how fast the photos come
in, or changing the amount
that the indicator is showing,
you can control the
amount of pressure here.
And of course, no pressure
demo would be complete
without having a drawing field
that you can draw on as well.
Lastly, don't have the
sound hooked up for this,
but on this control
when it gets --
when you rotate the
photo back to zero,
you're feeling a little
bit of haptic feedback
on the trackpad for that.
So that's pretty neat.
Let's switch back to the slides.
We've got a lot to cover.
We're going to talk
about the APIs,
that that application was using,
the various APIs that we use
in the system as well,
so that you can go ahead
and add all sorts
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and add all sorts
of new features using
the Force Touch trackpad
in your application.
I'm going to teach you today how
to become a master using the
Force Touch trackpad on OS X.
To become a master in
the Force Touch trackpad,
even though we're going
to do it in one day,
you've got to start off small,
you've got to start
off learning everything
that a squire would know about
using the Force Touch trackpad,
and this is all about
using the high-level APIs
and the built-in tools that we
have in some of our controls.
The we're going to move
on, and we're going
to learn everything a
knight needs to know
about using the Force
Touch trackpad.
This is all about the flow of
the force through the system,
the event stream, we'll talk
about customizing Spring Loading
in your application and
then taking it all the way
to doing some Alignment Feedback
and providing some additional
haptic feedback to the user.
Then finally, we're going to cap
it off with becoming a master
of using the Force Touch
trackpad and this is
about controlling the force.
Configuring the trackpad
so it is doing the
appropriate haptic feedbacks
for the given situation
for your controls
and manually playing haptic
feedbacks where appropriate.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and manually playing haptic
feedbacks where appropriate.
So let's dig in.
In being a squire we're going
to talk about Table Row Actions,
it's a nice, neat new
feature, you have seen
that in the Mail
application demo
that Craig did during
the keynote.
We'll talk about
spring-loaded controls,
some things that are built in,
and the accelerator controls.
We're going to do that by
looking at some case studies.
Here is Mail, for example, and
if you do a two-finger swipe
on a row in Mail, you get
some more Table Row Actions
that could come up, and
you can click on them,
or you can do a smooth swipe,
a little bit longer swipe
and it will actually
activate the default action
and so it is one complete
gesture for the user.
It is a really nice way
of adding some additional
functionality
to your application.
I use it every day
in Mail myself.
The API for it is
incredibly simple.
In your tableView delegate,
just implement tableView
rowActionsForRow edge,
and we will tell you what edge
it is, it's either the leading
or the trailing, so we
handle right to left
and left to right for you.
And then you just return back
an array of Table Row Actions.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
And then you just return back
an array of Table Row Actions.
Create a new NSTable Row Action,
you init with a style, a title,
and a handler, the handler's
what's going to get called back
if that item is selected,
either by clicking on it
or if they did a complete
swipe for the default one,
it goes ahead and the
handler is called.
We have a couple of styles:
Regular and Destructive.
Regular is the one with
the blue highlighting,
and that's what you
should use for most styles
of your Table Row Actions.
Destructive is the
one that's red.
Don't choose these
because of their color;
it is more than just red.
The destructive items, since by
nature they're destructive we
actually make it a
little bit harder
for the user to do a full swipe.
They have to swipe
a further distance
on the trackpad before a
destructive action is triggered
by default.
This is so that they don't
accidentally trigger it.
Use regular for almost
everything
and reserve destructive items
just for destructive, don't try
and get the red color, there is
some important semantics there
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and get the red color, there is
some important semantics there
that you need to be aware of.
And that's all there
is to this API.
It's that simple.
This works on a new Force Touch
trackpad and it also works
on our legacy trackpads
and the Magic Mouse,
so you can easily add
these to your application
and you can open up some
great new possibilities.
Let's look at Spring
Loading in the Finder.
So I have this image of
Lola that I downloaded,
and I want to move her
to my Documents folder.
Some I'm going to start dragging
and go back in my history
by Force clicking, I'm even
going to change it to icon mode
so I can find the
Documents folder easier,
and then finally go ahead and
drop it in my Documents folder.
You might have known about
spring loading before
where you could hover over a
folder and after a timeout,
the folder would go
ahead and spring load.
We have added spring
loading to a lot more places
and applications to
bring windows forwards,
and as you saw the
buttons were spring loaded,
and you can bypass
the hover timeout
by just doing a Force click,
and it becomes much more
intuitive and easier to use.
To implement Spring Loading
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
To implement Spring Loading
in those toolbar buttons all
Finder did was set the Spring
Loading property of
NSButton to true.
Really what they did was
just check the box in IB.
It is that easy.
You can do that for
segmented control as well;
when I change the icon layout
from list view to icon view,
that's on NSSegmentedControl,
and again it's just a
springLoaded property
and you can set it
with a check box in IB.
It's that simple to turn Spring
Loading on in your buttons
and your segmented controls.
You have to opt in for
this, but for places
that are doing navigation
during drag
and drop it is really
useful and I suggest
that you go and turn it on.
There is another example of
using force; this is QuickTime.
I want you to pay attention to
the fast-forward button here.
I'm going to use the Force
Touch trackpad and I'm going
to apply different
pressure to the button,
and you can see I can
go up to 5x, 10x, 30x.
You can back off.
As you're moving
through your movie file,
you can control how fast
you're moving forward,
so you slow down when you
get close to that area
that you're looking for
without overshooting.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
that you're looking for
without overshooting.
The way QuickTime does this
is we have a new button type,
so the fast-forward buttons are
really literally just NSButtons
with a custom image, and
they set the button type
to either AcceleratorButton or
MultiLevelAcceleratorButton.
For AcceleratorButton -- you
can set it right here in IB --
as the force in the
trackpad changes,
the button will continually
send its action message,
so as the force changes, you
get a new action message;
when the force changes again,
you get a new action message,
and the range of the
doubleValue is going to be 1
when the user clicks
the button up to 2
as the user presses the
maximum amount of force
on the trackpad that we accept.
You can see the pressure
change between 1 and 2
and you can adjust
however you --
whatever you need
to do with that.
In QuickTime's case, that
controls the acceleration.
You will finally
get a value of 0.
A final action message
with a value of 0
when the user ends
tracking of the button.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
when the user ends
tracking of the button.
When they release
the mouse button up,
you'll get a last action
message with a range of zero.
Now, what QuickTime is
actually using here is the
MultiLevelAcceleratorButton.
You can set that
right here in IB.
The MultiLevelAcceleratorButton
is discrete;
whereas the AcceleratorButton
is a smooth range,
a continuous range
between 1 and 2,
the AcceleratorButton
is integer levels,
it's a discrete integer
levels, and you can set
that with the
maxAcceleratorLevel.
Our range is between 1 and 5
so you can set how many levels
that you want to have
in your acceleration;
by default it is 2.
QuickTime sets it to 5.
Then again, you look
at the doubleValue.
Now the range is going to be
0 to 5, it's 1 when you click,
as the user goes up
through the levels,
it'll go up to whatever
you set the max level to,
and you'll get a final
message action of 0
when the user finishes tracking.
And this is what
QuickTime is doing.
Here you can see how QuickTime
is just mapping 1, 2, 3, 4,
and 5 to your various speeds,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and 5 to your various speeds,
and for example 3 is
10x fast-forwarding.
Another example of Accelerator
Controls: here is Maps,
and Maps has this
nice zoom buttons,
and these are actually
implemented
as NSSegmentedControls.
NSSegmentedControls
has a new tracking mode
which is MomentaryAccelerator,
and this works exactly
like the accelerator
NSButtonType does --
real easy to set
that in IB as well.
But instead of asking
for the doubleValue
when the segmented control
action messages are fired,
you want to ask for
the doubleValue
for the selected segment.
It is the same range as
we discussed earlier;
it's 1 to 2 with a final action
message with a value of 0
when tracking has ended.
Segmented controls don't
have a multilevel option,
they only support
the continuous mode.
Here is an example of Photos.
For moving through your
photos in Photos by applying,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
For moving through your
photos in Photos by applying,
varying the pressure,
I can control how fast
the photos move across.
This is something that we call a
continuous accelerator control.
If you have an accelerator
control settings
on either NSButton or
NSSegmentedControl,
in the NSControl section you can
set the continuous flag to true
or just check the box in IB,
and you'll get what we call a
continuous accelerator control.
Continuous accelerator
controls are different
because you don't worry about
their doubleValues so much.
You just want to move to
the next slide as soon
as the action happens.
The doubleValue doesn't matter.
They come back, come
in on a heartbeat,
and the force changes the
frequency of that heartbeat.
That's the difference.
To drive the point home a little
bit, let's compare the two.
In accelerated control,
ou get your action message
whenever the pressure happens
to change, so there may be a
little bit of delay in there,
it may come close together,
and you just change how
fast you're fast-forwarding
in the movie.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
For a continuous control
it comes in on a heartbeat
for a continuous acceleration,
and then you just
do your action.
The frequency, they might
come together closely
if the frequency is high because
the user is applying a lot
of force.
It is a great way for doing
something like sliding photos
where the animation speed is
constant, but when do you need
to bring in the next photo?
That's the next time
the action message fires
and it gives the user
a lot of control.
That's everything you need
to know to be a squire.
Congratulations.
You are all now squires in
using the Force Touch trackpads.
We have talked about Table
Row Actions, a very easy API
to implement, spring-loaded
NSButtons
and accelerator NS Buttons
and segmented controls,
they're real easy to turn
on, our high-level API,
you get a lot of bang for your
buck here in using these APIs
and we hope that you turn
them on in your applications.
Let's move forward
to being a knight.
Being a knight is all
about understanding how the
force flows through the system.
We're going to talk about
the force event stream,
and then I'm also going to talk
about the spring-loading
protocol
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
about the spring-loading
protocol
so that you can use the same
API that NSButton is doing
to provide spring loading
in your custom applications.
Then finally we will talk about
some Alignment Feedback API
to help you do snap-to
guides and things like that.
Let's talk about
the event stream.
We have a ton of events already
on OS X; you get the mouse downs
and ups, your gestures
for magnify and rotate.
And now we're introducing
another one:
NSEventTypePressure.
Or actually, we introduced
it in 10.10.3
when we introduced
the new MacBooks,
so you can get
NSEventTypePressures there
as well.
The are pressure gesture.
And along with the new
event type, of course,
we have the event
mask to go along
with it: NSEventMaskPressure.
And what this means is that
during your tracking loops,
you can just add
event mask pressure
to your tracking loop mask,
and you can get the
pressure events coming
in as you're tracking
the mouse as well.
It is really easy to use.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
It is really easy to use.
If you prefer to use
the responder approach
where you're overriding
mouse down and mouse drag
and mouse up, we also have
a new responder method
pressureChangeWithEvent, so
you can get them that way
as well if you prefer.
Let's dig into the properties
of the pressure gesture.
It is a gesture; unlike
a mouse event sequence
which has individual types
for mouse down, mouse drag,
and mouse up, there is just
the single type for pressure,
it has a phase, and it goes
through a cycle of Began,
Changed, and Ended
when the gesture ends.
And we have a stage.
A stage is how we determine
when a Force click happens.
So when you do a mouse down with
a trackpad you'll get a Began
phase pressure gesture and
it will have a stage of 1.
This is the standard
click level stage.
The user presses harder to
get to Force click level
on the trackpad, the
stage will change to 2.
That's how you know the user's
accomplished Force click
and you can just immediately do
your Force click action and go
into rename on the
Finder, for example.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
into rename on the
Finder, for example.
Then as the user releases
the force from the trackpad,
it goes back to stage 1
as it gets to click level,
and eventually it gets to
stage 0 when the gesture ends
because the mouse button
down, it is no longer down,
so you get a final event of
stage 0 with a phase of Ended.
Now, of course, no pressure
gesture would be complete
without actually
having a pressure value,
so we have a pressure
property as well,
it is within the
range of 0 to 1.
It is important to note here
that the pressure
property is the pressure
of the current stage.
So as you can see in the chart,
as you enter stage 1 the
pressure starts to go up
and reaches 1 as you approach
stage 2, and once you cross
over to stage 2, the pressure
drops immediately back
down to 0, and it
goes back up again
as you increase the pressure
while you're in stage 2.
The pressure is of
the current stage.
That's really important.
Now I'm showing a linear
mapping between the force
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
on the trackpad to what the
user is doing to the values
in the pressure event,
but I don't want you
to read too much into that.
We like to think of every
click on the trackpad
as a new adventure,
and we look at a lot
of different variables -- is
the user using their thumb,
what firmness setting do they
have in their preferences,
how are they interacting
with the trackpad --
so we dynamically change
these curves on the fly
to give the user the
best possible experience,
and we normalize the
input from the trackpad
into the pressure
range of 0 to 1
and that's what you should
be using in your application.
As you notice when I was
clicking around earlier
in the demo, just clicking
around in TextEdit,
I wasn't getting the animation
of the popover until I started
to really get close to
having enough pressure
to reach Force click.
You don't want to
have a whole bunch
of distracting animations
occurring
when the user's just
clicking around.
Doing this animation as they're
approaching the transition
to stage 2 can be useful.
This is what the stage
transition property is
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
This is what the stage
transition property is
useful for.
As you see here, it is in
the range of 0 to 1 just
like pressure, but it stays at 0
for a much longer period of time
until you start to
approach the next stage,
stage 2 in this case.
I want you to get to approach
stage 2, it will then shoot
up to 1, you can use this range
to control your animation,
and it won't interfere
with just clicking
around in your interface.
You can get that
animation and as soon
as you reach stage 2 you can pop
that animation to completion,
and you can see the
stageTransition value drops back
to 0 once you reach stage 2,
and we don't have a stage 3
in this case to transition
to, so it just stays at 0
for the remainder of the time.
I mentioned earlier that
mouse events are going
on at the same time as
pressure gestures are.
The trackpad is still emulating
a mouse like it always has.
It is also issuing
pressure gestures.
I'm going to look at how
those flow in the system
at the same time in parallel.
This is what we're
going to cover.
This is an example of the user
putting force on the trackpad,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
This is an example of the user
putting force on the trackpad,
applying a click,
going all the way
to the Force click threshold,
and then releasing
pressure off of that.
That's their input.
Down here we're going to show
you the events as they're coming
into your application.
So the user starts off,
they're applying light pressure,
and these are all mouse moves.
We haven't reached a
click threshold yet.
We haven't even started
a pressure gesture yet.
These are just mouse moves,
there are no pressure
gestures coming in.
You reach the click
threshold, a mouse down occurs,
and you also get a pressure
began event with a stage of 1
and the pressure value is
going to be 0 at this point.
We don't guarantee
if the pressure began event
occurs first or if the mouse
down occurs first; they
can swap sometimes.
The easiest way to handle that
is just look for the mouse down
and if you also want pressure
events, then start looking
for them after the
mouse down occurs.
As the user starts to apply
more force to the trackpad,
you see the pressure
rises up towards 1
as we approach the
Force click threshold.
If the cursor is moving,
these are going to come
through as mouse dragged events,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
through as mouse dragged events,
you have the mouse
dragged events
and the pressure events --
we're still in stage 1 --
we reach the Force
click threshold,
you now get a pressure
gesture with a stage 2
and the pressure value
drops all the way back to 0.
The user continues to apply
more force on the trackpad,
the pressure starts to
increase again, as they start
to release the pressure
from the trackpad it starts
to go back down toward 0.
Now they have released it back
to the Force click threshold,
that pressure is at 0.
We're still at stage
2 at this point.
As the user releases
a bit more pressure,
we're still at stage 2.
We have actually exaggerated
it a little bit on this graph.
But it is very difficult for
somebody to hold pressure
at a constant rate
on the trackpad.
If they're right at the Force
click level they'll be going
above it and slightly
below it and we don't want
to be triggering Force click on
and off, on and off, on and off;
that's not what the
user's trying to do.
We require that you drop below
the Force click threshold a
little bit further before we
finally unlatch from stage 2,
and we give you finally
an event of stage 1.
And during that small section
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
And during that small section
of time the pressure is
going to be 0 in the event.
You're obviously below
the Force click threshold
where the pressure
would start to go up.
Then now that you're in stage
1, the user continues to release
and the pressure jumps up and
starts to come back down again.
You notice it didn't jump
all the way back up to 1,
because that's part
of the little gap.
One of the things to note
here is that this is one
of the reasons why you
shouldn't try to combine stage 1
and stage 2 to get a
larger dynamic range,
we're going to talk more
about that when we get
to the master section.
Whenever there's stage 2 occurs,
automatically we also provide
haptic feedback, so don't try
and combine the pressure
into two stages.
Either look at stage 1 and use
the pressure or look at stage 2
and use the pressure if
that's where you need
to do your animation or the
stage transition property.
We reached the click
threshold point for stage 1,
so your pressure is now back
down to 0, and just like we did
with the Force click
threshold, the user actually has
to release even more pressure
from the trackpad before we
finally release from stage 1.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
from the trackpad before we
finally release from stage 1.
We're going to do a
pressure with a phase ended
at stage 0 and the mouse up.
Again, whether the
mouse up occurs first
or the pressure ended
event occurs first,
that is not guaranteed;
the easiest way to deal
with this is look for the
mouse up, track your pressure
and your mouse movements
at the same time
until the mouse up occurs.
Just ignore any pressure
events that might occur before
or after that sequence.
This will work with mice as
well as our older trackpads
and the new Force Touch
trackpads as well.
Finally, if the user is
moving around with very,
very light pressure after the
mouse up, these are mouse moves,
the pressure gesture has ended
and we're starting the cycle
over again as we did
in the beginning.
It gets real important to
know sometimes on the mouse
down if there is going
to be pressure associated
with this mouse down.
Is this coming from a device
that's emulating the mouse
and also issuing
pressure gesture events.
The way we have to
help you do this is
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
The way we have to
help you do this is
by using the associatedEventMask
property
on the mouse down events.
You can see this on the
mouse dragged events as well.
It is real easy to use.
You ask the mouse down event
for the associatedEventMask,
you find out if it contains
the EventMaskPressure,
if you're using Cocoa
you're just ending it
with the NSEventMaskPressure
because you're checking
the bit field.
If it does contain
the EventMaskPressure,
you know pressure events
are going to be coming,
and you can set it up so that
you have varying brush widths
for example in your drawing.
And that's great.
If it doesn't contain
the EventMaskPressure,
then it is coming from a mouse
or an older style trackpad
for example, and at
that point you want
to choose a default brush size,
usually you choose the maximum
brush size at that point
or maybe you want to just
choose half a brush size
and you use this constant value
for the entire mouse
dragging sequence.
That's the new
EventTypePressure,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
That's the new
EventTypePressure,
it's got a lot of new
properties, it's got a phase,
because it is a gesture,
pressure which is
within the range of 0 to
1 for the current stage.
Of course you have
stage, you can easily see
when the user goes to Force
click, stage transitions
for animating those transitions,
the associated event mask
so you know when your
mouse events are going
to have pressure
associated with them,
and of course the
pressure change
with event responder method
if you're using the
responder methods.
Let's talk a little bit
about spring loading
and how NSButton
implemented spring loading;
you can use that exact same
API in your custom controls
to add spring loading
to your application.
There is an
NSSpringLoadingDestination
protocol; it's very similar to
NSDraggingDestination protocol
if you've ever implemented that.
In your destination, in your
destination view you need
to implement either
springLoadingEntered
or springLoadingUpdated.
To give you an example
of how these work,
NSButton implements
springLoadingEntered
and not springLoadingUpdated,
because a button is
either enabled or disabled,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
because a button is
either enabled or disabled,
so as soon as you
enter the button,
it can return the spring-loading
options and it is not going
to change for the lifetime
that the drag is
occurring in the button.
NSSegmentedControl,
on the other hand,
implements springLoadingUpdated.
Each segment may be enabled
or disabled independently
of each other, it's one view,
so with springLoadingUpdated,
segmented control can watch
the drag and find out exactly
where in the control the drag is
and dynamically change the
spring-loading options.
As I mentioned, you need to --
if you implement one
of these methods,
you need to implement
spring-loading options.
You can implement both if
you want, you don't have
to implement just one of
these, you need to implement
at least one of these though.
For your spring-loading
options, it is pretty obvious,
you return if spring
loading should be enabled
or disabled, that's pretty easy.
We also have a couple of
other interesting options,
continuous activation --
generally a spring-loading
action is discrete,
spring loading occurred,
the button fires its
action, it is over.
There is also a continuous
version
which we'll cover
a little bit more
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
which we'll cover
a little bit more
in the next couple of slides.
I want to move on to no hover.
As I mentioned earlier, spring
loading can be triggered either
with a hover, which
works great for people
that don't have Force
Touch trackpads
or you can Force click.
If you want to do a Force
click on a canvas for example
that has a large area, and the
user is just dragging the item
across the canvas, they're just
trying to get across the canvas,
and they lift the
finger up to come back
down to continue the
drag, or lift the mouse
up to continue the drag,
that amount of time
might be just long enough
that the cursor stays still that
spring loading is activated.
So this would be a
false activation,
that wasn't what the
user was trying to do.
If you have a situation
where you're getting a lot
of false activations because
of hover, then you might want
to consider the no hover
spring-loading option
which will still allow users
with the Force Touch trackpad
to do a Force click
to get spring loading
in that area of your view.
Use it sparingly, make sure
that you really think about it
because we don't want
to leave out any users
that don't have a
Force Touch trackpad,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
that don't have a
Force Touch trackpad,
but if you're getting
more false activations,
this is a good option
to use sometimes.
Along with springLoadingEntered
and Updated,
of course we have
springLoadingExited --
this lets you know when the
drag has exited your view --
and there's also draggingEnded.
Spring loading is part of
the drag and drop operations,
so if the user has
dragged over your view
and your spring-loading
destination
and you implement draggingEnded,
when the dragging does
completed, when the user lets go
of the mouse, cancels the drag,
you will get back
the draggingEnded.
Whether this drag, whether
the user completed the drag
in your application
or in another process,
it doesn't matter.
Everybody that's implemented
draggingUpdated will get
their callback.
Since this is part of dragging,
the NSDraggingDestination also
has a draggingEnded function.
It is the exact same
function we have here.
If you are both a Spring
Loading destination
and an NSDraggingDestination,
you only need
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and an NSDraggingDestination,
you only need
to have one implementation
of draggingEnded,
it applies to both, so you'll
need to do any cleanup you need
to do for both Spring Loading
and dragging destination
if you're both a Spring
Loading destination
and a dragging destination
at the same time.
Now we get to the really fun
stuff about spring loading.
So you're required to implement
springLoadingActivated.
This is where we
tell your destination
that the user has spring loaded.
We have a Boolean value,
which is normally yes.
As I mentioned, spring
loading is a discrete action,
it has occurred, and
NSButton just fires its action
and everybody is happy.
But if you have that
continuous bit set, then as soon
as the user Force clicks,
we'll send a
springLoadingActivated yes,
you can start a timer,
you can add continuously
on that timer firing your action
message, and when you release
from Force click it will respond
with a springLoadingActivated no
and that you know to turn
off your timer at that point.
If the user is using hover,
you will get the
springLoadingActivated
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
you will get the
springLoadingActivated
at the hover timeout with
a yes, and you'll get a no
when they move out
of the control.
Again to compare this to
the normal discrete action,
usually spring loading
from Force click occurs
on the release of Force click,
so they move all the way
down into Force click, it's
when they release Force
click you will get a
springLoadingActivated yes, if
you're not using continuous;
that's the one that
we generally suggest
that you use unless you need
to continuously fire your action
message during a spring load.
Lastly, we have
springLoadingHighlightChanged.
We like to give the user
feedback on what's going on.
We have three forms
of highlighting: None,
Standard, and Emphasized.
When you get a
springLoadingHighlightChanged
message, you need to ask
the draggingInfo for what
that springLoadingHighlight
is, set your view
that needs display, and
then you update and you draw
with the correct highlighting.
What's important here is that
you don't try to infer any kind
of behavior that the user
is doing with this yet.
That's what
springLoadingActivated is for.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
That's what
springLoadingActivated is for.
Sort of like when you select
an item from a menu we blink
that menu item to let
the user know and confirm
that they have selected that
item, we'll do something similar
to that in spring loading,
and we do that by
changing the highlighting,
and so all you have
to do is just draw
with the new highlighting
style whenever requested
and you'll get a consistent
look in your application
to match the rest of the system.
Use springLoadingActivated
to know
when to fire the
action messages.
That's Spring Loading
Destination.
We have Alignment
Feedback, you can see this
in Interface Builder when
you get two items together,
we snap them together, draw that
nice little alignment guide.
We have some new API
to help you do this.
The drawing is up to you, but
our new API helps you decide
when to do the snapping.
To give you an example of that,
let's look at a tracking loop,
a typical tracking loop.
Let's zoom in on that.
You get your mouse down event,
you figure out what your event
mask is, which events you want
to track, you ask for the next
event, you move your item,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
to track, you ask for the next
event, you move your item,
update your data model,
set your needs display,
is this a mouse up, no, and
you just continue the cycle,
you draw whenever drawRect is
called and you drag your item
across the screen until
the mouse up occurs.
We have an
NSAlignmentFeedbackFilter object
to help you out with this.
You get your mouse down,
the first thing you do,
you get the input event mask
from the Alignment
Feedback filter.
This is the events that the
Alignment Feedback filter needs
to know about, you just or them
or union them in the Swift case
with your event mask for when
you call nextEventMatchingMask.
Once you get the next event,
the very first thing you do,
you give that event to the
Alignment Feedback filter,
it's going to return right away,
it just updates some
internal state.
You move -- excuse me, if you're
using a pan gesture recognizer,
you can also update with the
pan gesture recognizer --
it works very similar
to a tracking loop,
everything applies except for
this one message difference.
Once Alignment Feedback
filter returns
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Once Alignment Feedback
filter returns
from processing the input, you
move the item in your data model
as you normally would, and
then you prepare alignment.
Let's dig in to prepareAlignment
a bit and you're going
to use the Alignment Feedback
filter to help you do that.
We have the object in the
data model previously,
we got the event, and we decided
the user moved it to here,
this is the default location
that if we don't
do any snapping,
this is where the object
is going to end up.
We want to know, should we
snap it here, should we snap it
down to this line, or should
we snap both at the same time
and get it down in the corner.
Use the Alignment Feedback
filter object to help
to decide this,
alignmentFeedbackToken
ForHorizontalMovementInView
(without space), previousX,
alignedX, and defaultX,
these coordinates are in
the window coordinates space
so it works really well
regardless of your zoom level
that you might have
in your view.
If snapping should
occur, we will go ahead
and return you an
Alignment Feedback token.
If you don't get an
Alignment Feedback token,
don't do any alignment.
If you get an Alignment
Feedback token,
in your data model change
the X position in this case
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
in your data model change
the X position in this case
to the aligned position
and hang on to
that Alignment Feedback token.
You can then check for
vertical movement as well,
except you send your Y
values instead of your X
and again you may or may not
get an Alignment Feedback token.
If you've gotten two Alignment
Feedback tokens you'll have
aligned in both axes and
so your object will be
down here in the corner.
Then sometimes -- this
happens more rarely,
but sometimes you only
want to snap to a position
if it is both aligned on
the X axis and the Y axis,
and it's either aligned to both
axes or aligned to neither,
in that case we have Alignment
Feedback token for movement
and you pass in points
instead of an individual X
or Y coordinate, and it
works the same way, you may
or may not get an
Alignment Feedback token.
You iterate over the various
items that are being dragged
that they can snap to, you get
back your Alignment Feedback
tokens and you change your data
model if snapping should happen,
and now you have a
collection of tokens.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and now you have a
collection of tokens.
You want to perform
haptic feedback
to the user using the
new Force Touch trackpad.
We'll use the Alignment Feedback
filter to help us do that;
we'll just ask the
Alignment Feedback filter
to perform feedback
at a performance time,
just use the default for the
performance time for now,
we'll cover performance times
a little bit more later.
Then you just pass the
array a feedback token.
You can even pass an empty
array if nothing was aligned
and Alignment Feedback
filter is robust with that,
it knows to just do nothing.
Then you set that your
view needs display
and you redraw whenever you're
asked; if you have the tokens,
then you know when
you're redrawing
to also draw alignment guides if
that's appropriate for your UI.
The feedback will then be
performed synchronously
with the screen change and
the user will have a nice
synchronous -- sees the
alignment guide pop up
and they feel the
haptic feedback
from the trackpad
at the same time.
If the user is not using a Force
Touch trackpad the Alignment
Feedback filter knows how
to work with that as well,
and so you just have to write it
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and so you just have to write it
as if there is a
Force Touch trackpad;
if there's not, it
just works great.
So that's all there really is
to using Alignment
Feedback filters.
They're pretty simple
APIs, real easy to add it
to your application and
existing tracking loops,
provided a consistent
feel across app,
we look at the velocity of
the cursor, for example,
so that we don't do snapping
if the user is moving quickly,
because we don't want
to do Alignment Feedback
either at that point.
That's not what the
user is trying to do,
we look at the modifier values,
and so this will
provide a consistent feel
across all applications that are
using the system-wide Alignment
Feedback API.
And you can use this for more
than just dragging a item;
if you're doing a resize for
example or a size to fit,
that's another good place to
use Alignment Feedback filter.
It could be used in a
lot of different places.
So that's everything you
need to know to be a knight.
Congratulations.
You're all knights in using
the Force Touch trackpad.
Let's move on to
becoming masters.
This is all about
controlling the force.
This is about configuring
the trackpad
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
This is about configuring
the trackpad
so that it works appropriately
for your custom situations
and manually providing
haptic feedback.
Let's go back to
that drawing example.
You start to do a
drawing and you press
on the Force Touch trackpad,
you get a Force click
and that's not really
appropriate in your drawing.
And as I mentioned earlier,
you don't want to try
to combine the pressure
values from stage 1
with with the pressure values
from stage 2, it's not going
to be even during the
release, and the user's going
to get this haptic feedback
in the middle of the drawing,
that's not a good experience.
So we want to configure
the trackpad
to not provide Force click
actuations at all in that case.
And this is what we can do
with the pressure
configuration object.
We initialize one with
a pressureBehavior,
check out the header file
and the documentation,
check out NSEvent.h,
there's a lot of description
in the header file
about each of these,
I'm not going to cover them all.
The default one is
DeepClick, that's what happens
by default in the system.
In this case we want
PrimaryGeneric;
PrimaryGeneric is a
one-stage gesture,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
PrimaryGeneric is a
one-stage gesture,
so the user won't get
a Force click actuation
when they're drawing,
and it provides the largest
dynamic range of user input
of their force on the trackpad,
mapping that back out to you
and to pressure between 0 and 1.
It's the best one to use
for drawing and for a lot
of other situations, and
then check the header files
for the descriptions of
the other behaviors to see
which one is appropriate
for your situation.
Once you have a pressure
configuration,
you just call set.
The trackpad is now configured
in this new configuration
and everything is great
-- with some caveats.
You can only set the trackpad
into a different configuration
during a mouse drag,
so on mouse down, you
check the mouse location,
if it is supposed to be changed
for this specific
mouse location,
create a pressure
configuration, you call the set,
and the trackpad is
going to be changed.
You need to realize that
you're racing the user here.
The user might move the cursor
over your view, go immediately
to a Force click,
release the mouse,
and you didn't even
get the mouse down yet,
perhaps you're being paged
in from virtual memory
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
perhaps you're being paged
in from virtual memory
and your app is not responsive.
You can try to set the pressure,
the trackpad configuration
at this point but the user is
already completed their gesture,
and it won't take effect.
When you do set the
trackpad configuration it is
automatically reset back
to the system default
when the user ends the gesture
as well, so you don't have
to worry about unsetting it.
But this isn't ideal for most
situations; it is really useful
when you need to decide at
the very last minute based
on the mouse location what
the configuration should be.
Instead, just set the pressure
configuration on NSView.
Create a pressure
configuration ahead of time,
set it as the pressure
configuration property
on the NSView and the
system will go ahead and set
up the trackpad before
mouse down even occurs.
In fact, the system can
configure the trackpad
to this configuration even
if your app isn't
being responsive yet.
Now the user can go ahead and
interact with your application,
you didn't even get
the mouse down yet,
but they didn't feel the
Force click because it is
in the PrimaryGeneric
behavior and the events you get
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
in the PrimaryGeneric
behavior and the events you get
in from the application,
from the system at that point
when you finally
do get your events
if you're not being responsive,
don't have Force click in them,
they don't go to stage 2.
Let's talk a little bit
about haptic feedback.
Haptic feedback should
be used sparingly.
This is for subtle interactions.
We just want the trackpad
to just feel right, right?
So when the user is
trying to align something,
they get that haptic
feedback and it feels great,
we're not trying to massage
the user's finger here,
we just want this to
be subtle interaction.
In fact, if the user
goes back to one
of our older trackpads
we want them
to maybe not even realize
why something is wrong
but it just doesn't
quite feel right.
That's the point of using the
haptic feedback is doing it
appropriately when the user
is trying to do something
to give them that little
bit of subtle feedback,
oh yes, this is just right.
This is how it should
have always been,
and I didn't know that.
Subtle interactions.
You just ask the
NSHapticFeedbackManager
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
You just ask the
NSHapticFeedbackManager
for the defaultPerformer, always
ask for the defaultPerformer
because it can change
based on the input device
and the user's preferences.
You ask to perform
the feedback pattern
at a specific performance
time, we have three --
three patterns: Generic,
Alignment, and LevelChange.
Alignment can be used for
a lot of different things,
even if you're just rotating
a photo to align a horizon,
for example, you can go ahead
and use alignment for that.
LevelChange is what NSButton
uses in the multilevel mode;
it will provide haptic
feedback as the levels change.
If those two don't sufficiently
describe what you're trying
to do, then go ahead
and just use generic.
You want your haptic feedback
to perform synchronously
with what's going on
the screen so that
by default that's
the DrawCompleted.
If you're using Cocoa
drawing or core animation,
you just use a DrawCompleted
performance time
and this will just automatically
synchronize them for you
so that you can determine your
haptic feedbacks during event
processing and your drawing can
just concentrate on drawing.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
If you're using Medal
or OpenGL directly,
then you can just use Now and
as things update on the screen,
you will go ahead and need to
make sure that you line them
up so that they occur
simultaneously.
That's all there is.
You're now all masters in
using the Force Touch trackpad.
I can't wait to see what
you do with these things
in your application, we have
covered Table Row Actions,
accelerator buttons,
Spring Loading, we've talked
about how the force
flows through the system,
doing Alignment Feedback, and
finally controlling the trackpad
and configuring it for
your specific needs.
There's a lot that
you can do here.
That little app that I
did earlier in my demo,
it's called Force Touch
Catalog, you can download
that and check that out.
We also have an alignment
guide sample application,
so that's really
great to use as well.
I suggest that you also
check out the What's New
in Web Development
in WebKit and Safari
so you can learn how
Safari is exposing pressure
in the web environment.
We have a lab coming up
right after this, the Cocoa
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
We have a lab coming up
right after this, the Cocoa
and Force Touch and Gesture
lab; I will of course be there.
I'm real excited to hear your
ideas on how to use Force Touch
in your applications or
even to show me anything
that you have already done,
I'm really excited to see that
and to talk about it with you.
Please come find me in the
lab right after this session.
Thank you very much.
Enjoy the rest of the show.
[Applause]