Transcript
[ Crowd Noises ]
[ Applause ]
>> Hi, everyone.
I'm Donna Tom, and I'm a TextKit
engineer.
And today, my colleagues AppKit
engineers Jeff Nadeau and Taylor
Kelly will be joining me and
we're all really excited to talk
to you about advanced Touch Bar
concepts.
Now, hopefully most of you got a
chance to watch or attend the
Fundamental session earlier
today.
If you're following along at
home, go watch that now.
If you're here, please stay.
It's a really great talk, I do
recommend you go and watch it
later if you've got the time.
In it you've learned about how
to create your own NSTouchBars
and how to use their input
capabilities to enhance the user
experience in your app.
And you can easily adopt Touch
Bar by using just what you
learned in that fundamental
session.
But you're probably here because
you want something more to give
your app that extra edge.
Maybe you'd like to customize
the standard bars offered by
Touch Controls like the Mail
app's email Autocomplete item.
Or maybe you'd like to add a
Scrubber to your app so the user
can quickly scroll through and
select content like the Calendar
date selector.
Or maybe, you'd even like to add
your own completely custom
control like the system color
picker.
Now, in this session, we are
going to show you ways to build
all of these things.
We'll dive deep into techniques
and best practices for
customizing the standard
components as well as building
your own custom controls giving
you the power and flexibility to
build rich experiences for your
users.
So, first up, is to customizing
TextBars.
We're going to start by going
over the standard items, then
we'll talk about how to disable
the standard items, and how to
add your own custom items.
And finally we'll learn about
the Candidate List item and
we'll take a look at how to
build an email Autocomplete item
similar to what you'd see in the
Mail app.
So, let's get started with
standard items.
Now, we built a lot of great
functionality into the standard
TextBars.
If you're using a TextView or a
TextField, guess what?
These components support
NSTouchBar right out of the box.
Now, when a TextView or a
TextField becomes the first
responder on your app, it'll
automatically get a bar
populated with items that are
driven by the properties and
content of the control.
So, by default, here's what you
get.
QuickType suggestions, which can
be expanded and collapsed using
that little angle bracket on the
left.
Top bar shows what it looks like
when it's collapsed and the
bottom bar shows what it looks
like when it's expanded.
You'll also get a character
picker, for all those great
emojis.
And if rich text is enabled,
you'll get a color picker,
formatting controls, and
alignment and list controls.
And you get all of this for free
when you use NSTextView or
NSTextField.
You don't have to write a single
line of code.
But, how does all of that
actually work?
Well, NSTextView and NSTouchBar
are very tightly integrated in
order to be provide all of that
great functionality.
Since the items in the bar may
need to change depending on the
contents of the entered text,
NSTextView actually makes itself
the delegate of its own
NSTouchBar instance.
And if you're using an
NSTextField, remember that an
instance of NSTextField serves
as the field editor for the
currently active TextField.
And so when you place items in
the TextField's bar, that's not
the same instance of NSTouchBar
that contains the standard
items.
The field editor's bar will be
the one that contains the
standard items.
And while we're on the subject
of the field editor, remember
that the field editor is shared
among all text fields in a
window, so if you make changes
to that field editor's bar,
those changes are going to
affect all the text fields in
the window.
And this is an important concept
to keep in mind when working
with text fields.
So, now that we understand a
little bit more about how these
components work together, let's
talk about how to disable the
standard items.
It's actually pretty simple.
All you have to do is set the
corresponding property on your
textView or your textField.
And when you set these
properties, the textView will
automatically get notified that
its bar items need updating, so
there's nothing else you have to
do.
QuickType and the character
picker are each controlled by
their own property.
But the rest of the standard
items are text styling controls
and these will be present if
rich text is enabled on your
text field, and if it's not,
then you won't get those items.
Now, before you go and disable
QuickType on all your text
controls, please consider your
users.
I know some of you might not
like to use the QuickType item,
but I'd like to emphasize that
your users might not share your
preferences.
In fact, your users can disable
QuickType suggestions for
themselves
either through the System
Preferences pane as shown here,
or by customizing their Touch
Bar.
So, please, consider leaving
QuickType enabled and allowing
your users to make this decision
for themselves.
So, now that we know how to
disable the standard items,
let's talk about how to add your
own custom items.
So, here are a couple examples
of items that have been added
alongside the standard items in
a TextBar.
The bar on the top might look
familiar if you've seen the
Fundamentals talk.
It has the Send Mail button
placed all the way on the left.
The bottom bar has a Select All
button placed just to the right
of the character picker.
So, how do I go about adding
buttons like these to my
TextBar?
Well, you might be tempted to
put something like this in
ViewDidLoad.
Grab your textField's Touch bar,
append your new button to the
defaultItemIdentifiers and
that's it, right?
No. Please don't do this.
You're going to have a bad time
guys.
And why? It's because the text
view needs to be able to reset
its bar items based on the
current configuration.
And so here's what's going to
happen.
You're going to just have a bar
that looks something like this
after you add your new button,
which is just as you would have
expect.
But, if you the user disables
rich text while your app is
running, that text view will get
notified and it's going to need
to reset its bar items.
And it's going to do this by
removing all the existing items
and then just recreating the
ones it needs based on the
current configuration.
But remember, that the TextView
is the delegate of its own
NSTouchBar instance.
So, it's going to recreate these
items by calling its own
implementation of
makeItemForIdentifier.
And so it will recreate the
QuickType item, it'll recreate
the character picker, but it's
not going to know how to
recreate your custom item.
So, it just won't get recreated.
Now to avoid this you're going
to need an approach that works
with the TextView and there are
a couple of different ways to do
that.
The first approach is to use a
higher-level responder to
provide your custom item.
And this is a good approach to
use when it makes sense for your
item to appear in more than one
context.
And our Send Mail bar from
earlier is a good example of it.
So, here, we have a Send Mail
button and ideally we'd like
this button to be present in the
bar, when the Compose Mail
window is open.
It doesn't matter which TextView
or which TextField within that
window has a first responder.
We want this button to be
present for any of them.
And so, this is a good situation
in which to use a higher-level
responder like a ViewController
to provide your custom item.
And here's what that might look
like.
Here we have our ViewController.
We're going to override
makeTouchBar.
We'll create our custom Touch
Bar item and set its view to the
Send button.
And then, we'll create our Touch
Bar, we'll configure it with our
custom item, and then we'll
return it.
Now, note the use of the
otherItemsProxy here.
And this is going to nest the
bar items from the field editor
so that they appear alongside
our custom item.
And that's a basic example of
how to use a higher-level
responder to provide a custom
item along the standard items.
So, now onto the second
approach, subclassing
NSTextView.
Now this is a good approach to
use when you have an item where
you only want it to appear in
the bar when your TextView is
the first responder.
And to see what I mean, let's
take a look at the second
example from earlier.
Here we have a Select All button
that selects all of the text in
the TextView.
Now in this case, we only want
our button to appear when our
TextView is first responder
because it wouldn't really make
a whole lot of sense for a
Select All button to be there if
nobody's editing any text.
And so this situation is a good
one to subclass NSTextView and
put your button in that same
NSTouchBar instance as the
standard items.
And so there's two methods
you're going to want to override
in your TextView subclass.
One is
updateTouchBarItemIdentifiers
and you want to override this
method because this is called
each time the state of the
TextView's bar needs changed.
And this method is where you'll
actually add the text bar to
display your custom item, to the
defaultItemIdentifiers array.
And the second method to
override is the delegate method
makeItemForIdentifier.
And you'll want to override this
method because this is what the
TextView uses to actually create
its bar items.
And so this method is where
you'll create and you're
configure your custom item.
Now, with both of these
approaches you're going to want
to make sure that you call the
superclass implementation before
you do any of the work
associated with your custom
item.
And by using this approach
you'll ensure that your custom
item will be recreated along
with the standard items whenever
the TextView needs to reset its
bar.
And that wraps up the two
approaches for adding custom
items.
Let's shift gears and learn
about the Candidate List item.
The Candidate List item is a
pretty nifty control for giving
your users a list of contextual
suggestions to choose from.
And we'll refer to these
suggestions as candidates.
And this item is also tightly
integrated with NSTextView.
So, every TextView has a
reference to the Candidate List
in the bar, accessible via the
candidateListTouchBarItem
property.
Now once again, remember the
Candidate List you see with a
TextField is actually in the bar
associated with the window's
field editor.
Candidate data generally comes
from these three sources.
In the QuickType case, the data
for the Candidates is supplied
by NSSpellChecker.
But the control can also be
configured to show Candidates
from other sources, like the
system input method, which
allows you to type Chinese,
Japanese, Korean, or Indic
characters using a Latin
Keyboard.
But what most of you are
probably interested in is the
ability to provide your own
completely custom Candidates.
And there are two different
approaches your can take to do
this.
The first approach is to use a
delegate method.
The benefit of using this
approach is that it's the
simplest one since the TextField
does most of the work.
So, to use this approach, you'll
implement a delegate method
candidatesForSelectedRange in
your TextField or your TextView.
And you'll get the original text
control, the field editor if
applicable, and the range of the
text, which you can then use to
determine your Candidates.
So, you'll simply return an
array containing your custom
Candidates, and they'll be
displayed in the standard
Candidate List.
Now since this approach uses the
standard list, there's nothing
else you have to do.
The TextView will do the rest.
The second approach is to use a
higher-level responder.
And the benefit of this approach
is it allows you to use model
objects rather than strings for
your Candidates.
So, this is similar to the
higher-level responder approach
that we looked at a little bit
earlier for adding custom items
to your bar.
Here, you'll use a view
controller or some other object
to create your own instance of
NSCandidateListTouchBarItem and
provide that instead of using
the standard one.
Now, this approach is a little
bit more complex than the
delegation one, but again it's
really useful if you'd like to
use model objects for your
candidates instead of strings.
And we'll go over this approach
in greater detail a little bit
later.
But for now, let's take a look
at implementing a simple version
of the email Autocomplete item
that we've been teasing you with
since the beginning of this
talk.
We're going to use the
delegation approach for this.
But we have a slight problem
here.
Now, since our text field is
intended for email addresses, it
really wouldn't make a whole lot
of sense for it to have emojis
in it, or rich text.
But if we turn those properties
off on the bar, look at what
happens when we collapse our
Candidate List.
You've kind of got this weird
empty bar just sitting there,
and that doesn't really look
good, so we don't want that to
happen.
And we're going to prevent that
from happening by turning off
the collapsing functionality
using the allowsCollapsing
property.
And when we set this property
that little angle bracket on the
left goes away and you can no
longer collapse the list.
But now, we have another
problem.
We set that allowsCollapsing
property on the field editor
Candidate List item and the
field editor is shared among all
the text fields in the window.
And so, if we set the
allowsCollapsing property on
this field editor's Candidate
List it's going to affect all of
these text fields, which we
might not want.
And so we're going to solve this
problem by providing our own
field editor for our email
TextField and then that way,
when we modify its properties,
it won't affect the shared
editor.
And so, to create the custom
field editor, we'll subclass
NSTextFieldCell.
In our subclass, we're going to
override fieldEditor for
controlView.
And then we'll create a
fieldEditor and return it.
And that's all there is to it.
We're not adding any custom
behavior.
We're just providing our own
instance of the stock NSTextView
to use as the field editor
whenever someone wants to edit
the contents of our Text Field.
And don't forget to set your
TextField to use your new
subclass, either through
Interface Builder or
Programatically.
So, now, we can implement our
textField delegate method.
Here, we'll turn off the
collapsing behavior of the field
editor and return to our array
of custom Candidates.
And that's a simple
implementation of email
Autocomplete with the Candidate
List item.
Now earlier we touched on a
second more complex approach for
returning custom Candidates by
providing your own Candidate
List item from a higher-level
responder.
Now if you need finer grain
control over the candidate list
item, this is the way to go.
And if you want to use this
approach, here's what you'll
have to do.
First, you want to disable
QuickType for your text field or
view.
Since you're providing your own
Candidate List, there's no need
for the standard one.
Second, you want to implement
the NSTouchBar delegate method
makeItemForIdentifier.
And this method is where you
will create your custom
Candidate List item.
And if you're using model
objects as your candidates, this
is also where you're going to
map your model objects into
string representations that can
be displayed in the Candidate
List item.
And so to do this, you're going
to use this block property
attributedStringForCandidate.
Third, if you're using the model
object candidates, you need,
you're going to need to
implement the delegate method
endSelectingCandidateAt index
and this is where you're going
to provide a mapping in the
other direction from user's
candidate selection to an
instance of your model object.
And finally, you want to update
the candidates that are
displayed when someone types
into your text field or view.
And so you can do that by
overriding controlTextDidChange
and then calling setCandidates
for selected range and string on
your custom Candidate List item.
And that wraps up customizing
Text Bars.
So, to recap, we covered two
approaches for adding custom
items in text bars and two
approaches for providing custom
candidates in the candidate bar.
I encourage you all to try out
customizing your text bars using
these approaches that we
recommended today.
Now, next I'd like to invite
Jeff Nadeau up to talk about
NSScrubber, a control especially
created for NS Touch Bar.
Jeff?
[ Applause ]
>> All right, thank you.
So, as Donna just said,
NSScrubber is our very first
Cocoa control that we've
designed especially for the
Touch Bar environment.
And, the central theme of this
control is taking your app's
content and bringing it forward
conveniently right at your
fingertips.
Now just some examples, we've
been showing you this nice
Calendar timeline as an example
of NSScrubber, but in fact we
use it in several places across
the operating system.
And two more of them are
Safari's tab picker, and also
Keynote's slide navigator, which
appears while you're giving a
presentation in Keynote.
And just putting these side by
side, you can tell that this
control is very versatile.
It's capable of expressing a
wide variety of visual and
interactive designs.
But before we dive into any of
the API, we should probably
answer the question, what is
NSScrubber?
It's a collection-like control
that is all about arranging a
list of your app's content and
then providing touch
gesture-based highlighting and
selection of that content using
both tap and pan gestures.
And if you used NSCollectionView
before, this API is going to
feel very familiar.
However, it has been streamlined
and adapted for the unique Touch
Bar environment.
Like NSCollectionView,
NSScrubber has a compositional
interface, which means that it
delegates a lot of its
responsibilities to other
objects.
And the best way to see these is
to actually pull the control
apart and take a look at each
piece in turn.
First, we have the control view
itself, NSScrubber, which
manages all of the user
interaction with the control.
It also provides a couple of
nice cosmetic properties like
backgrounds and other such
things.
Next, we also have the selection
decoration.
These are the views, which
appear to indicate that a
specific piece of content is
selected.
We have a dedicated layout
object, which abstractly defines
how the contents of the
controller are arranged.
And then, of course, we've got
your content, which is provided
using a data source protocol.
Now, we're going to look at each
of these pieces in turn;
however, we should first look at
what we can configure on the
control view itself.
And the first and most important
thing is the mode of
interaction, which is controlled
appropriately enough with the
mode property.
NSScrubber has two primary modes
of interaction.
The first, called "fixed",
offers a touch-based selection
that directly selects and
highlights the item underneath
your touch.
The second, "free", is kind of
like a free wheel, which you can
freely scroll the way that you
would any other touch driven
scroll view.
Once you've chosen between these
two modes, there's also the
continuous property.
This is a familiar name from
several other Cocoa controls and
we typically use it to mean that
a control continuously in
response to user interaction is
updating its value or sending
its action.
Now, when continuous is false,
in an NSScrubber you can see
that tapping on an item and
panning across highlights an
item but the selection stays put
until we finish our gesture and
we commit a new selection.
But if we turn it to true, we
can see the effect in something
like Safari.
Where as we pan across tabs,
they are immediately selected as
your touch passes over each
item.
This also goes for free style
scrubbers, for example, the
Keynote slide navigator we just
looked at.
We can scroll through here, look
at what, you know, navigate our
content and then we can tap
items to select them.
But if we set continuous to
true, just the act of scrolling
is going to be fluidly and
continuously updating the
selection within the content.
And so this gives us a very
stark distinction between
continuous being false, where
selection is a very deliberate
thing, which might be
appropriate if it's kind of a
heavyweight action, and
continuous being true where
navigating and selecting viewer
content is very fast and fluid,
and may be ideal for lightweight
interactions.
Once we've decided what we want
or how we want to select our
items, we need to decide what we
want that selection to look
like.
And that's controlled with the
selectionBackgroundStyle, and
OverlayStyle properties.
We provide two built-in styles.
We have the outlineOverlay
style, which is that nice bold
white outline that you've seen
in several of our examples.
And then we also have a
roundedBackground style, which
looks great behind text items.
Now if you're particularly
observant, you may have noticed
that the NSScrubber selection
style declaration is not
enclosed enumeration.
It's actually a class.
And so you can subclass this and
define your own selection styles
if you'd like.
One other fun cosmetic option
for selections is this
floatSelectionViews property.
When it's false, changing
selection looks pretty much like
you'd expect.
The old selection disappears and
then the new one appears
immediately.
But if we set this to true, we
actually get a nice, smoothly
floating and gliding selection,
which is kind of a very cool
effect.
There's one other trick that you
may have noticed in several of
our examples.
Some of these controls, when you
scroll, flick, or pan through
them, always glide smoothly so
that some item always lands
neatly in one place, in this
case the center.
And that's controlled with the
itemAlignment property.
Now by default this is none,
which means that we won't do any
kind of adjustment on your
scroll, but if you set it to
leading, trailing, or center,
after a scroll event, flicking,
panning, momentum, anything like
that, will always adjust it so
that some item appears neatly
aligned with either a leading,
trailing, or center of the
control.
Finally, NSScrubber as a couple
of nice cosmetic options for the
area behind your content,
including a background color, or
if you want to draw anything you
like, you can place it in view
behind your content.
And we actually provide a named
color on NSColor called the
scrubberTexturedBackground
color, which works great in the
backgroundColor property to
provide this cool textured
vertically hashed appearance
which you may have noticed in
apps like Calendar.
Now that seems like a lot of
properties, but if we pull it
all together, we can fully
configure an NSScrubber in only
a couple lines of code.
So, we create our control, and
then we assign it a layout, a
delegate, and a dataSource.
We won't dwell on this, but I
promise we'll look at each of
them in turn in just a moment.
We decide how we want to
interact with the control.
In this case we want one that
scrolls freely, but has a
continuous selection as we
scroll.
And then we also want the
itemAlignment to be center so
that after we scroll, some item
always lands neatly in the
middle.
Then we'll choose what we want
our selection to look like, in
this case we're going to apply
both of the built-in effects.
And we'll turn on that nice
floating selection so that as
the selection changes, it floats
neatly between the views.
And then finally we'll apply
that cool textured background
color.
And with just these couple of
lines of code, we've produced
the blueprint for exactly
Calendar's scrubber that we've
been looking at this entire
time.
Now that we've configured the
control the way that we'd like,
we need to layout the items.
And that's done with the
NSScrubberLayout class.
This defines the arrangement of
all of the content within the
control and it does this
abstractly using these
NSScrubberLayoutAttributes
objects, which are essentially
value types that bundle up in
itemIndex, and where it should
be laid out within the
coordinate space of its content.
We provide a couple of built in
layouts that you can just pull
off the shelf and use if you'd
like to, you know, just really
quickly get started and that
includes a flow layout, which is
comparable to NSCollectionView
flow layout.
But if you want to define your
own, it's really not very hard.
There's only three key methods
that you really need to
implement.
The first, scrubberContentSize
defines the entire size of
everything in the control, and
this defines how far we can
scroll within all of the
content.
There is
layoutAttributesForItems in
rectangle, which provides a set
of the attributes for every item
that falls within some specific
rectangle.
And that might be the currently
visible rectangle or it might be
one that the user is about to
scroll to and you want to get
ready and lay everything out.
And then finally,
layoutAttributesForItem at
index, so we can determine the
layout for some specific item if
we need to.
The other half of the layout
life cycle is invalidation.
InvalidateLayout signals to the
control that your layout has
changed in some way and it needs
to refresh new fresh layout
information from the NSScrubber
layout.
Now, if your layout depends on
certain information like the
selection highlight or visible
rectangle like maybe you want to
make the selection be twice as
large as every other item, you
can optionally request that your
automatically invalidated when
those things change.
And this prevents you from
having to track that information
yourself and manually call
invalidate layout.
Finally, after your layouts been
invalidated, before we do any
further layout passes, we'll
call the prepare method on your
layout object.
The base implementation doesn't
do anything but this is a great
opportunity for your subclass to
run some calculations, do some
measurements, and prepare some
caches so that subsequent
outcalls are really fast.
And that's all you need to do to
put together a layout.
Now that we've configured our
control and we want, we know how
we want to arrange all of our
content, we need to get our
content into the control in the
first place.
Content in NSScrubber is
represented using simple views,
subclasses of the
NSScrubberItemView class, and
that's comparable to
NSTableCellView.
Like NSTableCell, or like
NSTable and NSCollectionView,
NSScrubber provides a reuse
queue so that you can
efficiently recycle views as
they cycle out of the control,
rather than allocating a brand
new one every time it's
requested.
The dataSource protocol for
NSScrubber is really super
simple.
There's only two methods that
you have to implement.
The numberOfItems in the entire
control, and then you just need
to be able to prepare a view
that represents a specific item.
To aid you in this task, we have
two built-in NSScrubberItemView
subclasses.
We have a TextItem and all you
need to do is provide a string
that you'd like to present and
we'll present it in a way that
is consistent with many similar
controls across the operating
system.
And then we also have an image
view class, which provides
aspect fill presentation of your
image and then allows you to
specify an alignment beyond
that.
Now, if neither of these are
exactly what you're looking for,
you can alway subclass
NSScrubberItemView and just do
whatever drawing you'd like.
When you do this, you get access
to a couple of nice properties.
You get isSelected and
isHighlighted, which you can use
to alter your drawing based on
that state.
And your drawing will be
automatically invalidated when
these change so you don't need
to observe them yourself.
And you can also override the
applyLayoutAttributes method,
which allows you to inspect and
interpret the attributes, which
were produced by the layout
object.
And that's how you provide
content.
Finally, we assume that the user
is going to use this, this
control and so we want to
respond to that.
And that's done using a delegate
protocol.
You can be informed when the
selection changes, the highlight
changes, and if the range of the
visible items in the control
changes due to either scrolling
or layout.
You can also be informed when
interaction begins, ends, or is
cancelled.
And this might be useful for
example, to create an undo
grouping around a single
interaction with the control.
So, as the user is rapidly
scrubbing through, you can
bundle all of those changes into
a single undo grouping.
And that's all you need to do to
put together your own
NSScrubber.
I hope you've, you've come up
with some great ideas for
content in your application you
can surface with this control.
But, we've -- between the
Fundamentals talk and this one,
we spent a lot of time
explaining how to use AppKit's
built-in controls and classes to
really build up your Touch Bar
experience.
But if you want to, to really
unleash your creativity and
build something truly custom,
you're going to want to stay put
for the next section where
Taylor is going to walk you
through doing -- doing
completely custom controls for
Touch Bar.
Thanks.
[ Applause ]
>> Here they go.
Hello. So, when it comes to
building custom controls for the
Touch Bar there are four areas
that you want to keep in mind.
The first is handling the touch
events that are coming in.
Second is styling your view so
it looks and feels at home in
the Touch Bar.
Third is making sure it's the
right size and position amongst
all the other controls.
And finally it's tying
everything together by applying
a little bit of animation.
Let's get started with event
handling.
Now the obvious thing here is
that we have direct touches on
the Mac.
And I really want to qualify the
difference between direct and
indirect touches.
Those of you who are might be
iList developers are already
familiar with this distinction
where you have direct touches,
where you're directly
manipulating content on the
iPhone or iPad versus the
indirect touches on the Siri
Remote's touch surface.
Where you're remotely
interacting content that's
actually on the TV.
On the new MacBook Pros, we have
both of these in a single
device.
We've already had indirect
touches on the gesture-rich
trackpads, where you can get
access to individual touches on
that trackpad.
Now, with the Touch Bar, we
support having direct touches
where you can directly
manipulate content on the
screen.
So, the existing NSTouch class
has now been extended to add a
TouchType to describe whether it
is this direct or indirect
touch.
For direct touches, you can now
get the location of that touch
provided some view.
And providing that view is
extremely important because it
provides the coordinate space
for what that point will be
relative to.
As an example, when the user
taps down on the slider, we want
to get the location of that
touch, relative to the slider as
a whole, and we can take that
offset and simply divide it by
the overall width to get our new
value.
It's pretty straightforward.
Now, one very important
difference with NSTouch and
UITouch is that while UITouch is
this long lived object whose
location will update over time,
NSTouch is essentially a
value-type snapshot of that
touch on -- at a certain point
in time in that gesture.
And you can use the identify
property to tie a sequence of
touches together when forming a
gesture.
So, as events are coming in
through the Touch Bar, each of
these events will contain a
collection of one or more
touches.
And each of these touches are
unique instances, so again the
way you tie them together is by
looking at that identity
property.
So, even when the user touches
down with multiple fingers, we
can distinguish which is the one
we're tracking by comparing its
identity to the one that we're
tracking.
We can make this a little bit
more concrete with some code.
So, we want to keep a variable
to hold onto that opaque
identity object.
When we first get some touches
on our view, we go through
those, pick up the first one,
and save its identity as the one
we're tracking.
Later, when additional touches
come in, we go through the ones
that have changed, and compare
each of their identity to the
one we're tracking to find our
new touch.
We can now use that new location
to update any events in our
view.
Now, where do these events and
touches come from?
Well, one place are these
existing NSResponder methods
touchesBegan, Moved, Ended, and
Cancelled, which will be called
back over the lifecycle of our
view.
We can opt into receiving them
by setting the allowedTouchTypes
to include direct or indirect
touches, but once you link on
the Sierra SDK, you'll be
automatically opted in to
receiving direct touches.
One really important method here
is toucheisCancelled.
While there's not really a
notion of cancelling mouse
tracking events, this is
extremely important for the
Touch Bar because it's so
context sensitive.
As the user switches active
applications, or active windows,
the content in the Touch Bar
will reflect that and if the
user's interacting with any
views at that moment, we'll send
touchesCancelled so it can clean
up any event handling, but
separate that from the user
actually lifting their finger.
However, for the most part, you
actually don't have to worry
about this, and you can just use
gestureRecognizers.
These have gained all these same
methods to for subclasses to
implement so they can handle
touches, as well as the same
opt-in method with the only
difference here being your
explicitly have to opt-in,
there's no automatic link check.
And pretty much you can use all
the built in GestureRecognizers,
which now work in the Touch Bar.
For instance, the color picker
simply uses a combination of
press and pan recognizers to
implement all of its event
handling.
It had to do no extra work.
But it's important to remember
even for these you still do have
to opt them in to receiving
direct touches when you do use
them in the Touch Bar.
Now, one really interesting
context when it comes to event
handling, are inside
Press-and-Hold popovers.
In the Fundamentals section you
heard how you can set the
Press-and-Hold Touch Bar of your
NSTouchBar to have it when the
user long presses on that
button, for it to immediately
present some very simple lists
like UI.
You can put your own custom
views in here and as the user
continues that gesture and pans
over your view, you'll get call
to touchesBegan, Moved, and
Cancelled as they exit the view,
and finally once they lift their
finger, the Touch Bar will be
dismissed.
However, because of this very
unique event handling model,
GestureRecognizers are not
supported in this context, so
you have to use those
NSResponder methods.
However, again, outside of
Press-and-Hold popovers it's so
much simpler just to use
GestureRecognizers.
One very interesting aspect of
the Touch Bar is around event
modality.
Since the Touch Bar is this
input device akin to a keyboard
it should always be responsive.
Just because the user's dragging
something around on the screen
doesn't mean your keyboard stops
working.
So, even where you previously
had a modal event-tracking loop,
the Touch Bar will still be
responsive and still be able to
issue state changes.
So, you might want to go back
and actually look at those
event-tracking loops to make
sure they can handle the type of
state changes that could occur.
In addition, the Touch Bar
itself is capable of multitouch.
So, while the user's interacting
with one control, they could
begin and end interacting with
another.
And again, you want to make sure
you can handle those state
changes.
However, once you do that,
you're afforded a lot of really
interesting interaction
opportunities by combining all
the input devices that exist on
this device.
You can create either really
advanced work flows for your
users or these kind of
delightful experiences that the
user's might discover by playing
around.
For instance, in the
customization UI, we've added
individual pan recognizers to
each item, so that users can tab
down with multiple fingers and
reorder multiple items at once.
Similarly, the color picker has
Press-and-Pan recognizers on
each individual slider, so you
could be editing multiple
components at once, or while
your'e editing that color, you
can save it to your favorites
all at the same time.
And that's just using
Multi-Touch.
Once you combine it in the
keyboard or trackpad, you can
really take things to the next
level, such as being able to
edit the color of your text as
you're entering it, or editing
properties of your cursor as
you're moving around.
So, I really encourage you to
think about how you can easily
use gestureRecognizers to add
this event handling support for
your app while also
simul-handing these simultaneous
state changes.
Next, let's take a look at
styling and appearance.
And so, obviously the styling
inside of the Touch Bar is
extremely different from the
[inaudible] on the main display.
I'm going to cover a number of
these areas, but I want to start
with something that you might
not have even noticed.
This is something that you can
only see by taking your MacBook
Pro into different environments.
So, our standard stock photos
look something like this with
nice even lighting, if you use
it in a bright studio
environment, or maybe even take
it outside, it might look
something closer to this.
If you're like me you might
primarily use it in the dark,
and it's pretty cool how the
blacks on that OLED display
basically disappear.
And finally, if you get pretty
tired as it gets later on in the
night, you might turn on the
keyboard backlight and get that
nice glow.
And so, while physically the
MacBook Pro looks very different
in all these environments, the
Touch Bar content is actually
adjusting as well.
The bezel colors of the
different controls is adjusting
to that ambient brightness, and
even the glyphs within the
controls is reacting to that
white point change of the
keyboard backlight.
And so if you implementing your
own custom controls, you want to
make sure that you can do the
same.
And thankfully this is pretty
easy by using system colors.
These are existing named colors
that semantically describe how
they should be used, and once
you go to draw with them, that's
when they'll be dynamically
resolved against their current
context to determine what color
should be drawn with, taking
into consideration if it's being
used in the Touch Bar versus
Aqua, as well as whether the
white point or brightness has
changed.
In addition, new in High Sierra
is this expressive palette of
colors that you can use to match
system UI and have been
specifically designed for the
out glow appearance and for the
Touch Bar.
So, one important thing to keep
in mind is how you draw with
these colors in order to take
advantage of all that context
sensitivity.
So, if you're using layers to
draw your views, you might be
tempted to init with frame,
ViewDidLoad or immediate
response to event handling,
immediately update the
properties of your layers.
And you don't want to do this
because when you get that CG
color to set it on the layer,
that will immediately resolve
those RGB values against
whatever the current context
happens to be.
So, as the whitepoint or
brightness changes over time,
your layers will stay exactly as
they were at this moment.
So, what should we do instead?
Well, we can override this
method called updateLayer and
well, update our layers there.
This will get called
automatically for you when your
view is displayed, such as the
first time it's shown on screen
or windows white point or
brightness changes.
And if you have a dynamic color
that you actually want to set,
it's important that you hold
onto that color as an NSColor
because again this maintains
that semantic nature of the
color as well as the context
sensitivity.
Any time that color changes, you
just set needsDisplay to true
and we'll recall this method for
you.
This same exact approach can be
used if you're using
DrawRectBasedViews, where here
the only difference is that
you're setting that color
against the current context.
So, that's how you can use these
custom colors and your own
custom drawing.
However, a number of our
controls support customization
of their colors just out of the
box.
You can now set the bezel color
of your NS buttons, the selected
segment color or even the fill
color of your sliders.
You can use our built in system
colors or even your own customer
colors if you have some overall
app theme.
What's really cool here is that
the text and image effects
within these controls is
actually adjusting depending on
the lightness of the colors
you're setting.
You actually don't have to do
any work to get that.
One thing you might have seen
across the system are these
default button blue colors.
And it's important to note that
this not achieved by setting the
systemBlue color as the
bezelColor.
There's actually a subtly
different color that's used for
these, and you can get that
keyEquivalent to the return key,
just like you would have done on
the main display.
One thing you might have noticed
in these past few slides as well
is that the Esc key has been
replaced by this Done button.
And this is pretty easy to
achieve by setting the
escapeKeyReplacement
ItemIdentifier on your presented
NSTouchBar and whatever control
that references will be placed
in that Esc key region.
However, it's important to
consider if an when you should
be using this.
It should really only be used
when the user has entered into
some modal context and they can
use that control to exit that
context.
Any actions they take in this
context should be undoable, so
if I edit my photo here and tap
Done, I can still undo that
change afterwards.
And finally, you don't want to
be adding explicit constraints
to this button to try and make
it be the standard size of the
Esc key.
We'll automatically apply
metrics to make it be that
standard size as well as even
adjust the padding around the
text to fit the most characters
possible.
So, for instance, we found that
the longest translation of Done
is [foreign language] and even
that is able to fit in with
these adjusted metrics.
However, as a last resort we
will allow this control to grow
in case there are some
extenuating circumstances.
Now another really important
aspect of the styling in the
Touch Bar is when it comes to
the font.
Now you might have noticed that
the font has changed within the
Touch Bar on the main display in
Aqua, we use a standard font of
San Francisco UI size 13.
And in Touch Bar we use
something actually closer to the
Watch, San Francisco Condensed
size 15.
So, the family and the size have
both changed.
And those of you who have
followed the font Mac font
transition perhaps all the way
from Chicago to Charcoal, to
Lucida to Helvetica to San
Francisco, know that you should
not be hardcoding your fonts in
your application, and instead
you can use systemFont of size
0, which will give you back this
dynamic font that when used will
be resolved against its current
context, just like those system
colors, and pick the right
family and size.
In addition, there's a weight
variance that you can get
thinner or bolder fonts, and
another really interesting
aspect of font you might have
noticed are these monospaced
digits used in things like the
AVKit Player or even the
component value of the sliders.
And even as the value changes,
those numbers aren't jiggling as
the metrics adjust.
They're monospaced.
And so you can get this by using
the monospacedDigitSystemFont of
size weight to get this same
effect.
Now, there are actually a ton
more interesting typographic
features when it comes to San
Francisco, and there are some
great talks in the past few
years that I'd recommend
checking out if you're
interested in learning more.
However, the other important
type of glyphs in the Touch Bar
are, of course, images.
And, since the Touch Bar is this
P3 Retina display, all you need
to provide are 2x assets for the
images used there.
However, they should be
specifically designed for the
Touch Bar.
You shouldn't just take your
toolbar icons and plop them in.
In addition, you should be using
template rendering to take
advantage of those white point
and brightness adjustments I
mentioned earlier.
To illustrate this, let's say
we're making a custom light
button.
We've specifically designed it
nicely for the Touch Bar, and
we're using it alongside a bunch
of other standard icons.
For the most part it looks
pretty good, except for later
that night, we take it home and
we turn on our computer
backlight and it looks something
like this.
All the other icons adjust and
our icon basically sticks out
like a sore thumb.
So, let's not do this.
Instead what we can use, is
provide an image that
communicates its shape purely
via the alpha channel and uses
template rendering so that
AppKit can take care of all of
the different styling that might
occur for that image such as the
white or even the blue styling
when the whitepoint is adjusted.
Now, before you have your
designers go off and create a
bunch of awesome icon
specifically for the Touch Bar
to take advantage of template
rendering, do be sure to check
out all of the standard icons
that the system already
provides.
There are really a lot of them
here and the Human Interface
Guidelines goes over all of
them.
And what's really cool is that
these are all public constants
provided by NSImage so you can
very easily drop them into your
application with really not a
lot of work.
So, those are some aspects of
styling an appearance, what's
really cool is that for the most
part, you can just use system
standard techniques and get a
lot of this behavior for free.
Next, let's talk about layout.
In the Fundamentals section, you
saw how the standard item layout
is to simply stack the items
along inside the Touch Bar.
You can use flexible spaces to
give your items a little bit of
breathing room, or you can use
the principal item to center a
single or group of items.
Now, layout in NSTouchBar works
very similarly to NSStackView,
where the items themselves
provide their sizing information
and NSTouchBar positions them
based on that size.
And when it comes to describing
that size, it works exactly like
it does outside of the Touch
Bar.
For instance, you do override
intrinsicContentSize and
calculate the size you want your
control to be and just return
that.
In case any state changes, such
as the button's title changing,
and when you want to note that
the intrinsicContentSize has
changed, you can call
invalidateIntrinsicContentSize
and NSTouchBar will relay
everything out.
Another approach that you might
have used in the past was to add
an explicit widthConstraint to
your control and then update
that constant over time.
This again works, the same way,
and NSTouchBar will again lay
everything out.
When you're building custom
controls, you also might have
flexibly sized controls similar
to NSSlider or NSScrubber, and
here the only difference is that
that simply doesn't have a width
constraint, or it has an
intrinsic content width of
noIntrinsicMetric.
NSTouch Bar will take all of the
available space within the Touch
Bar and divide it up amongst all
these flexible controls.
In this example, the slider is
the only control in there so it
takes up the entire application
region.
Now, if we want to restrict it
to a minimum and maximum size,
we can simply add inequality
constraints just like we did
outside the Touch Bar.
NSTouchBar will follow that
right along.
What's really great about having
this flexibility is, as the user
customizes in additional items,
they're still some flexibility
around how big that control can
be, and the user can really make
the most out of their Touch Bar.
So, that's how you can customize
the sizing and layout of
individual items.
But you also might want to
coordinate the layout across
groups of items.
One new thing in High Sierra is
that prefersEqualWidths property
on GroupTouchBarItem where you
can have all the items within
that group prefer to be equal
width.
What's cool is that this works
with user customization.
So, not only can users customize
items in and out of this equal
sizing group, but the equal
sizing only effects the items
that are actually present in the
Touch Bar, it's not relative to
all the possible items that the
user could add.
This also works really nicely
with localization.
In case one of these buttons
becomes extremely long when
localized, we'll choose to break
that equal width rather than
overflow the Touch Bar and then
cause items to be hidden.
However, sometimes that overflow
might just happen.
And that's where visibility
priority comes into play.
Here you see some start window
UI that looks pretty nice in
English, but when we initially
run it in German, it looks
something like this.
There's no clipping, however the
New Collection button has become
missing.
And the reason is that all the
items in the Touch Bar can't fit
in the space that we've allotted
and so NSTouchBar has overflowed
that New Collection button and
hidden it.
In this example, we might
instead want to hide that
Bookmarks button instead of the
New Collection button and we can
do that by setting its
visibilityPriority to be lower
than all of the rest.
And NSTouchBar will detach the
lower ones first.
And this time we'll get
something like this, which is a
bit better but really it's not a
great user experience to hide
any items.
So, in this case, we could
probably do a bit better.
And using the new Compression
behavior in High Sierra, we can.
So, in this example, we can note
that the titles of the different
buttons actually fully
communicate what effect they'll
have.
And so instead of dropping any
items, we can drop those hidden
images instead.
We can set the prioritize
compression options of this
group to be hideImages and now
when we run in German, it looks
like this.
What's really cool is that this
happens atomically.
We don't end up with half the
buttons having images and half
of them not, it's an all or
nothing thing.
And there's really a lot of
flexibility here.
We can instead think that our
icons are descriptive enough and
opt to hide the text instead in
which case we get this.
And you can even go so far as if
you have your own custom ways
that your custom controls can
compress, you can add additional
options to these prioritize
options to describe that your
custom control should compress
in those different ways.
So, these are a few ways that
you can use different group
Touch Bar properties to control
the group layout of your items.
However, the container views
that you're used to using
outside the Touch Bar, can also
be used within.
There was a really great talk
earlier today "Choosing the
right Cocoa Container View" that
discussed some of the
differences between these as
used outside the Touch Bar, but
I'd like to highlight a few of
the ways that you can use them
within.
You can use NSStackView to have
really precise control of the
spacing or sizing of your items,
such as what Calendar's doing
here where they've completely
eliminated the spacing between
these two buttons.
You can also put that StackView
into a ScrollView to enable
scrollability of these small
list of items right inside the
Touch Bar.
Jeff talked earlier about how
you can use NSScrubber when you
have these unbounded list of
items where you want to maintain
a selection over time as well as
benefit from some of the other
aspects that Scrubber can
provide.
Finally, you can go all the way
to using NSCollectionView right
in the Touch Bar to really have
precise control of the layout,
or completely custom
interactions, such as what the
favorites list in the color
picker does.
So, that's layout.
What's really cool is that for
the most part, layout works
exactly like it does outside the
Touch Bay, so any concepts
you're used to are using, you
can use within.
Now finally, let's talk about
how we can use layout to really
tie all this together.
Before we do that, I really want
to reiterate that the Touch Bar
is an input device and not this
extra display.
It's not the right venue for you
to be showing off your different
animation skills with animations
that might be distracting from
what the user's trying to focus
on the main display.
And any animations that you do
add should always be relative to
user input.
And because that user input can
change at any time, your
animation should be
interruptible and updatable.
So, one obvious thing we might
want to animate are the sizes of
our items as they're changing.
We saw earlier how we can add an
explicit width constraint to
control that size, and what's
really great is that to actually
animate that size change, all we
need to do is use the animator
proxy on that constraint and
update its constant instead.
And we get all the animation
benefits that you'd expect, such
as simultaneous animations and
even interruptibility just with
this one change.
If instead we opted to use
intrinsicContentSize to express
the size of our items,
and then
invalidateIntrinsicContentSize
to note it changed, what we can
do here is we can wrap that in
an animation group, set
allowsImplicitAnimation to true,
and then call layoutIfNeeded.
And again we get all the same
animation benefits that we saw
in the other approach.
Something else we might not want
to animate is whether an item is
flexible or not.
So, we can see here as the item
takes up all the available space
animatedly, and the same exact
technique of using that
animation group can be used here
with the only difference being
the same as before, where when
the item's considered flexible,
it has a intrinsicContentSize of
noIntrinsicMetric for its width.
Now we can tie all this together
and start to build something
similar to the color picker
where as you change the active
item, it grows to take up all
the available space and the
other items shrink down.
So, we can start by defining
this custom AccordionView that
has an active state as well as
defines an intrinsicContentSize.
When it's inactive, it has this
small fixed size.
However, once we set it to be
active, we want it to have that
flexible size that takes up all
the available space.
And again, we can use
noIntrinsicMetric to achieve
that.
Now we stack a few of these
items together in our Touch Bar,
and we add pressRecognizers to
each individual one.
When the user taps down on one
of our accordion views, we can
handle that press and then take
the oldActiveAccordionView and
set it to be inactive, so to
have that small fixed size, get
the newActiveAccordionView and
activate it.
And finally using that
allowsImplicitAnimations simply
layout that change.
And we get exactly what you'd
expect.
It's pretty straightforward.
So, that's how you can animate
different item size, but you
also might want to animate the
content within the items, such
as the content within the
different scrubber.
Now, if you're using constraints
or overriding layout and setting
your frames or updating your
properties there, what's really
great is that there's no extra
work needed to actually achieve
this.
So, as we resize this custom
item animatedly, you'll see that
the Scrubber's items flexibly
resize down, fluidly resize down
and even the images within them
nicely resize up.
And what's really cool is that
there is no extra work needed on
my part or by NSScrubber to
achieve this effect.
Because NSScrubber overrides
layout to achieve all its layout
and view properties, all of this
came completely for free.
The other type of content that
you often use in your Touch Bar
are buttons.
Now as your state changes, you
might in the past have updated
the properties of these buttons,
but you might want to achieve
something similar to what
FaceTime does where it actually
animates those property changes.
New in High Sierra is the
ability to use the animator
proxy on these buttons and
update your properties there,
and again you'll get these
animations completely for free.
If you focus in on that Remind
Me Later button, you'll note
that as the image position
changes, so does the overall
layout of the Touch Bar and we
can combine all these approaches
of setting the imagePosition
against the animator and then
using allowsImplicitAnimations
to apply both of these
animations all at the same time.
Now, when it comes to building
your own custom controls, you
might also have completely
custom animations that happen
alongside item sizes, or maybe
even independently, such as the
fluid knob resize happens when
you activate these sliders.
Now there's a few different
approaches you could actually
use here, such as using the
animator proxy on your custom
view to animate built-in view
properties or your own custom
properties.
You can also take explicit
CAAnimations and add them to the
content of that view, or if
you're able to express this as a
state change in your layout
override, where you update your
view properties there, you can
again take advantage of that
allowsImplicitAnimation call and
actually do no extra work as
part of animating that change.
There was a talk from a few
years ago that actually went
into a lot of detail about these
three different approaches and
their pros and cons "Best
Practices for Cocoa Animation."
I'd recommend checking that out
if you want to know more.
So, we talked about a pretty
wide variety of content today.
And I hope you can take
inspiration from some of these
areas and actually go back and
create some really rich Touch
Bar content for your
applications to really create a
nice user experience, taking
things like custom candidates in
the Candidate List item,
creating your own unique
Scrubber layout, or combining
gestureRecognizers in really
unique ways to really make your
UI pretty interesting.
For more information, you can
see this website where we have
links to documentation and the
Human Interface Guidelines.
And most of the related sessions
we have are in the past, but
there's actually one tomorrow at
the same time "Building Visually
Rich User Experiences" that will
actually go into a few tips and
tricks of using Core Animation
to achieve some of these really
interesting effects that equally
apply when used in the Touch
Bar.
That's it for our talk and I
hope you guys have a great rest
of your WWDC.
Thank you.
[ Applause ]