WWDC2014 Session 207

Transcript

X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
>> Welcome, welcome to Session
207: Accessibility on OS X.
My name is Patti Hoa.
I'm a software engineer on
the OS X Accessibility team.
Today my colleague Chris Dolan
and I are extremely excited
to tell you about this brand
new API was have installed
for you that's designed
to make your life simpler.
But before I talk about
that, I want to just touch
up on the importance of
accessibility, and what it means
for your application to
be accessible to everyone.
Then I'll show you some examples
of how to use this new API
to make some UI's accessible,
and then I'll also show you
some tools that you can use
to verify your changes.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
to verify your changes.
Then in the latter half of this
session, Chris will come up
and give you a live demo of
making a game accessible.
So, at Apple, we really care
about providing the best
platform for all users to have
that great and wonderful
experience
when using your application.
And when I say all
users, I mean users
with various ranges
of abilities.
That includes those who may be
visually challenged, and may...
(excuse me) ...those who
may be visually challenged
and have trouble seeing
the colors and the graphics
on the application: those who
are deaf and hard-of-hearing,
who may not be able to
hear the sound feedback
in the application; those who
may have mobility limitation,
and may not be able to use the
physical mouse or keyboard.
Well to achieve the
goal of making sure
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Well to achieve the
goal of making sure
that everyone could
have equal access,
at Apple we work really
hard to provide tons
of accessibility solutions
built right into OS X,
and that includes products
such as: VoiceOver,
designed for the visually
impaired user; Switch Control,
designed for mobility impaired
users who may only be able
to use a single switch
to operate the computer;
Closed Captioning for the deaf
and hard of hearing; and tons
and tons of other great
accessibility features built
right into OS X.
But all of these features
are simply tools for the user
to use your application.
After all, it's your application
that they're looking at, whether
or not it's to read an
article or to watch a movie.
It is your application that
they're interacting with,
whether it's to play a game
or to write a term paper.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
whether it's to play a game
or to write a term paper.
It's your application
at the end of the day
that they're looking
forward to use every day.
So how do you make sure
that your application is
accessible to everyone?
Well we've built a
foundation for you
to build your application.
And we have this what we
call "Accessibility API"
that you can implement to
make sure your application's
accessible to everyone.
So, I want to spend the rest
of this session telling you
about this API, but the first
thing I want to tell you is
to give you the big picture
of how this Accessibility
API works in the system.
So first there's
your application,
and then we have what we
call assistive softwares,
such as VoiceOver and Switch
Control, that help users
to perceive, understand, and
interact with your application.
And that happens through a
channel, on what we call...
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
And that happens through a
channel, on what we call...
Accessibility API.
And so, that's what I want
to talk to you about today:
it's this Accessibility API
which, for OS X Yosemite,
we created from the ground
up -- this brand new API.
I'm excited to tell
you about this.
So, when we were first designing
this API we had a few things
in mind...
Actually, just one goal,
and that is to simplify --
to make it easy for you to use.
And to show you just how
easy it is to use this API,
I'll walk you through
a few examples.
So what you see on the screen
first is a window that contains,
like, a media player controller.
So what we're going to do is
focus on the UIs in the middle.
What I see here is: a rewind,
play, and forward button.
Well how did I come
to that conclusion?
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Well how did I come
to that conclusion?
Because visually you can
see three distinct icons,
and each of those icons
represents something.
Well if you're not sighted, and
you depend on assistive software
like VoiceOver to help you
perceive what's on the screen...
Well, this is what a VoiceOver
user would probably see.
Just a bunch of buttons.
They wouldn't know
what to do with them.
Even worse, if you
implemented them
as custom views,
they'll see nothing.
Not terribly accessible is it?
Well today I'm going
to show you how
to make these UIs accessible.
So what we'll do first is
we'll focus on just one button
in the middle: the Play button.
If you say you're using
AppKit control you're in luck,
because you get most of
that accessibility for free.
By default, AppKit
will already fill
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
By default, AppKit
will already fill
in all the accessibility
information for that piece
of UI knowing that
it's a button.
However, AppKit is not psychic.
It's not going to be able
to know what the function
of that button will do.
That's something only you,
the developer, will know...
and so that's where you come
in: to help label this button.
So, there's two ways
you could do it.
You could go into Xcode
Identity Inspector.
Under the Accessibility
Identity section,
you could just add a
description: "Play."
So this, you could do
that in your zip file.
Or, if you'd like to do
this programmatically,
you could simply go into
your initialization code
and in your button cells simply
set the accessibility label
property to be the localized
string that you want it to be.
And with just that alone, now
a VoiceOver user could be able
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
And with just that alone, now
a VoiceOver user could be able
to be perceived this
as a Play button.
What makes it so easy?
Well with the new API --
which is a method-based API --
we've converted all the
attributes into properties.
So just like any objective-C
property, you could now get
and set the property
directly on the object itself.
So you no longer
have to subclass.
So whenever possible,
use AppKit Control.
But for any reason you can't
use AppKit Control and you have
to do your own stuff, like
creating a custom view for each
of the UIs, well in this case,
obviously VoiceOver
will not be able
to see anything on the screen.
So now you're wondering, "Great,
is there a lot more
I have to do?"
Well the good thing
is: it's just as easy
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Well the good thing
is: it's just as easy
if you use this brand new API.
You could continue to use
the accessibility property I
just mentioned.
I gave you an example
of tweaking one
of the properties --
the label property.
There's tons of properties
you could use.
But one of the challenges
for the developers
is knowing exactly
which property you
have to implement.
So now we've introduced an
accessibility protocol that's
designed to take that
guesswork out of the equation.
The protocols will guide you
through that whole
implementation process
so you know exactly which
property you have to implement,
and we have protocols
that should cover most
of the commonly used
types of UIs out there.
So how do you go about
implementing this?
Well first you have
to pick a protocol.
So, we have a list of
protocols you could choose from.
So you should pick one that
corresponds to the look and feel
of your UI; the way or
the function of that UI.
So in our case we have the
example I was showing you
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So in our case we have the
example I was showing you
earlier with the Play button.
It looks and behaves
like a button.
So we're going to pick
the button protocol.
So, in your interface
file simply say,
adhere to the
NSAccessibilityButton.
So now you add that;
you compile.
The good thing is,
now Xcode's compiler
will tell you...
will give you some warning
and tell you what you
need to implement.
So in this case, you have
to implement the
accessibilityPerformPress
and accessibilityLabel methods.
So let's go ahead and do that.
So in your implementation
file, simply --
for the accessibilityLabel
method --
just return a localized
string of the word "play".
For the
accessibilityPerformPress,
we simply need to perform the
action of pressing that button.
So what's next?
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
There is no next step.
That's all.
That's just how easy it is
to implement accessibility
even for a custom view.
So what makes it so easy?
Well it's because AppKit's doing
a lot of work on your behalf.
By simply using this
protocol, AppKit will infer
that these elements are
accessibility elements.
And depending on which
protocol you use,
AppKit will auto-populate
information
like what the role is.
So if, for example, if I use the
NSAccessibility button protocol,
AppKit will automatically infer
that the role for
this is "button."
So as you can see how easy it
is to use, I also want to kind
of give you a little bit more
detail about these protocols.
First of all there's
the base protocol,
which is what we call the
NSAccessibilityElement protocol.
It has a few required
methods like the frame
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
It has a few required
methods like the frame
and the accessibility parent.
Those are basic information
that all accessibility
elements should have.
And then you build
on top of that.
We have all the other protocols,
like the button protocol
I just talked about,
which has two required
methods, the accessibilityLabel
and the
accessibilityPerformPress.
And as you saw earlier,
Xcode compiler already warned
us about those methods.
So that's just how easy it is
to implement a custom view.
By the way, the protocols
are optional.
So you don't have to
implement if you don't want to,
but it's really, really handy.
So let's go through another
example that's a little bit
more complicated.
On the screen you see six
different image shapes.
The way the developer may have
implemented is simply a custom
view where they're
doing their own drawing.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
view where they're
doing their own drawing.
So if you have something
like this,
someone like a VoiceOver user
will still need to be able
to know that there's
six distinct UIs sitting
on the screen.
So you have to give an
accessibility representation
for each of the images that's
being displayed on the screen.
So how do you do that for UIs
that you're rendering yourself
and you don't have
an NSView backing?
Well, for those of
you who are familiar
with the old sample code,
we have something called
FauxUIElement.
And now with the new API we
have a formalized version
of that called
NSAccessibilityElement.
You can use this class
to support all those UIs
that do not have a
backing in NSView.
So let's see how you'd go
about implementing this.
So for the example
earlier with the image --
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So for the example
earlier with the image --
the six shape images -- for
each of those images you want
to create an
NSAccessibilityElement instance
for that.
So you create one of those
instances, and then you set
up some of the accessibility
properties for that,
such as who the parent
is, what the role is...
Well, in this case
we're just going
to say these are image role.
And then give it a unique label,
and then set up what
the frame is.
So when that custom view --
containing custom
view is being asked
for the accessibilityChildren,
then this is
where you return your array
of NSAccessibilityElements.
And one thing I do
want to note is:
you should keep these
accessibility elements alive
as long as they are on-screen.
That's because assistive
software
like VoiceOver could be
asking for information
about these objects at any
time through IPC calls.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
about these objects at any
time through IPC calls.
So as long as they're on the
screen, cache them around.
Okay so that was pretty simple,
even for something like that,
where you are rendering
your own UI.
So there might be a few
of you in the audience
who may have already implemented
accessibility with the old API
in your application
and you're wondering,
"well how does this
apply to me?"
Well, if you already
added accessibility
into your application, there
is no extra work for you.
Your old API code will
continue to work just fine
because the new API is
binary and source compatible,
so you don't have to worry.
But if you do like to
try out the new API,
and we do encourage you to do
so, you could continue to do so.
The old code will continue to
work alongside the new code.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So I just told you why you
should make your application
accessible, and I ran
through a few examples of how
to use this brand new API
to make some UIs accessible.
You saw how quickly
and easily we were able
to solve the three
examples I've shown you.
And it really is fast, and it
will shave some time for you
to develop in making sure
your app is accessible.
So with some extra time
you may have on your hands,
I really hope you
could take some
of that time to do some testing.
Because after all, you
know, the code you add?
You want to make sure that it
will be usable by everyone.
So how do you go about making
sure that your app is accessible
after you make those changes?
Well, we have two
tools you could use.
The first is Accessibility
Inspector.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
This is a tool that displays all
of the accessibility information
for any piece of UI that's
currently under the mouse.
And with the newest
version of the Inspector,
we will now display the
accessibility properties
of that UI.
So, this is really
helpful, so you know exactly
which properties you
may need to tweak.
The second tool we have
is called VoiceOver.
VoiceOver is a screen
reader designed
for the visually challenged
users to be able to explore
and interact with the
content of the screen.
And the users will get feedback
through speech or braille.
So how do you go
about using this?
First you've got to turn it on.
If you just remember
the shortcut:
command F5, that
will turn it on.
Or you can go into the
Accessibility Pref Pane.
There's an Enable
VoiceOver checkbox.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
There's an Enable
VoiceOver checkbox.
Simply check that and you
can turn VoiceOver on.
So the very first time
you turn VoiceOver on,
it's going to prompt
you to see if you want
to go through a tutorial.
So if you just click on
the Learn More button,
than it will run through
a tutorial with you.
So I really encourage
you to try that.
Run through the tutorial.
Now, VoiceOver is a
very powerful tool,
so it has hundreds of commands.
Well, today I'm going to show
you just five simple commands
you can use to be proficient
in testing with VoiceOver.
The first of that is how you
navigate from one piece of UI
to another: and that's
"control, option, right arrow."
So just remember that once
you have VoiceOver turned on,
any time you press control and
option, any other key you press
at the same time will
automatically turn
into a VoiceOver command.
So we have "control, option,
right arrow" to go to next,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So we have "control, option,
right arrow" to go to next,
and if you want to reverse,
you do "control,
option, left arrow."
And if you want to press a
button or activate something,
simply do "control,
option, space."
This is very similar to pressing
a space bar on something
that has a keyboard focus.
And then we have
two more commands.
One is Enter Group -- which
is "control, option, shift,
down arrow" -- and
Leave Group: "control,
option, shift, up arrow."
So what is "group?"
Well VoiceOver has this
concept of "group" and it goes
like this: If you
have, on the screen...
what I have on the
screen is just some window
with lots of UIs.
Well, to be able to navigate
quickly to a certain piece of UI
at any moment, VoiceOver has
this concept of grouping.
On the screen you see
three different groups:
one is the toolbar; the other
one is the table of folders;
and also the collection
view of images.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and also the collection
view of images.
To be able to interact...
or, you know, enter
and exit these groups,
like I said earlier you could
just press the Enter Group
or Leave Group command.
So with just these five
VoiceOver commands alone,
you should be pretty good
in testing with VoiceOver.
So now that I have shown
you the tools and run you
through some examples using
the API to implement some
of the UI examples
I've shown you earlier,
I would like to bring
up Chris Dolan to come
up to give you a live demo
of making the Tic Tac
Toe game accessible.
[ Applause ]
>> Thanks.
Thanks Patti.
Hi, my name is Chris, and
I'm a software engineer
on the OS X Accessibility Team.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
on the OS X Accessibility Team.
And I'm really excited to
show you just how simple
and just how easy it is to
add accessibility to your app
with our brand new API.
So to do this, I've written
a little test app here,
and it's a very simple
game of Tic Tac Toe.
So in this case the board
is full, the game is over,
and it says the O
player has won.
I can come in and
restart the game,
place some pieces
on the board...
I can even turn on the AI
opponent and play against that.
This is a very simple
application,
and I want to use this kind
of as our launching point
to show you how you would
implement all the different
techniques, that Patti just
told you, in this application.
But before I do that, I
want to give you an idea
of what this application looks
like to someone who's
using VoiceOver.
So, they may have just
heard from social media
that this is the great new app
to have, or that you wrote it
and they want to check it out,
and this is their first
experience with it.
I want you to keep that in mind.
So I'm going to use the keyboard
command to turn on VoiceOver.
So here, two things
have happened
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So here, two things
have happened
when I turn VoiceOver on.
>> You are currently
on a text element.
>> The first is that the
caption panel has shown
up in the bottom corner.
And all this does is it has all
of the text that's
currently being spoken.
So this is really, really
helpful if I'm collaborating
with someone who's
using VoiceOver,
and they often have their speech
cranked up really, really fast.
So here I can read all of
the text that's being spoken,
and it helps me when
I'm collaborating.
The second thing you'll see is
that there's a black box drawn
around the X wins label.
And we call this the
VoiceOver cursor.
You can kind of think of it as
an alternate keyboard focus.
So just like I can press space
on a keyboard-focused element
to do something with that
element, when I press one
of the many VoiceOver commands
(keyboard commands) it normally
acts on the currently
VoiceOver-focused element.
So now what I'd like
to do is turn on a mode
in VoiceOver called
Trackpad Commander.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
in VoiceOver called
Trackpad Commander.
And all it allows me to
do is just explore the UI
with my finger.
Just like VoiceOver on iOS, I
can move around the interface
and VoiceOver will
speak the descriptions
of all the items
underneath my finger.
I'm going to turn that on.
>> Trackpad Commander on.
>> And now I'm just gonna
explore around the interface,
and you'll see a blue circle
wherever my finger is.
>> Close button.
Minimize button.
Dimmed, zoom button.
>> Great. So the
toolbar is looking good.
But you'll notice anywhere
where there's no elements
there's this sound being
played [beeping].
It says this is empty
space [beeping].
>> Tic Tac Toe.
>> Okay. Title is good.
[Beeping] empty space here.
>> X wins.
>> Okay the label shows up fine.
[Beeping] more empty space.
Ah, we've hit our first
problem [beeping].
There's nothing being
reported here
for accessibility
for this checkbox.
So this is one of the
problems we'll have to fix.
>> Difficulty.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
>> Difficulty.
>> Difficulty's fine.
>> 2.
>> As well as the level.
>> Button, button.
>> Hm.
>> 2. Button, button.
Delete. 1.41%.
>> So we saw there's a
problem with the buttons.
It just says "button."
So if I'm not a sighted user,
I don't understand what's going
on with this button; what
does it do when I click it?
[Beeping] More empty space.
And again, the Restart
Game button's the same way;
there's absolutely
no information
for accessibility there.
What about the grid?
Well I had the same
problem here.
The grid isn't even being
reported to accessibility.
>> VoiceOver off.
>> So as you can see,
the user experience
for this application is quite
poor when using VoiceOver.
A lot of the major functionality
for this app just
doesn't show up.
I can't play a piece.
I can't restart the game.
I can't play against the AI
because I can't click
that checkbox.
So what I'd like to do
is take you through each
of these individual
elements and show you how
to make them accessible.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So here's my Xcode project.
And before we even
talk about the code,
I just want to orient
you to, kind of,
the layout of this project.
So over here...
this is a classic MVC style app.
I have all of my models,
and these are the files
that encapsulate, you know,
the Tic Tac Toe board...
and the game rules...
and what it means
to win the game.
We're not going to touch
any of the model files here.
This is the game state.
We don't need that.
The next folder is
all of the views.
So we have a lot of custom
UI in this application
and we need views to back them.
These are all of the files
that we use to do that.
This is where we're going
to be spending a majority
of our time in the code.
We also have the
controller section.
The AppDelegate is
currently empty.
And the Tic Tac Toe view
controller is the thing
that hooks up all
the UI interactions
with the monologue jacks.
The last thing I want to point
out is the main menu nib.
This is the one and only
nib for this entire project.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
This is the one and only
nib for this entire project.
And this is actually where
we're going to start.
So the first elements we're
going to work on are the plus
and minus buttons here
for the difficulty.
If you remember, this is one of
the items that we could land on,
but there was no
contextual information
about what these buttons did.
So I'd like to show you how
to add accessibility to that
in two different ways.
The first is actually
quite simple.
I'm just going to select that
element in Interface Builder.
And then up in the top right
I can open up the panel.
And the third icon from the
left is the Identity Inspector.
And in here is the Accessibility
Identity information.
So this is where I'm going
to add the description
for this button.
So what does this button do?
Well, when I click it, it
decreases the difficulty.
So I'm just going to
type that in here.
Great. And that's
all I had to do.
Very simple.
So what about if I
decide that I want
to do the same thing,
but in code?
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
to do the same thing,
but in code?
Well it turns out
that's really simple too.
So I can go over to
my view controller.
And in my awakeFromNib, here I
can pull out the button cell.
And I have an outlet
to that button.
And then I can use one of the
new accessibility properties
to just set the accessibility
label.
And remember, you always want
to localize these strings.
Okay. And this is the
increase difficulty button.
So I'm just going
to add that in.
Okay. Just like that.
That's all I need to do.
And with that, now when
a VoiceOver user lands
on these plus and minus
buttons, they'll be read
as the increase difficulty or
the decrease difficulty buttons.
So why did we get that
accessibility for free?
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So why did we get that
accessibility for free?
Why did they even show
up in the first place?
Well, this custom class is
actually a subclass of NSButton.
So we've done a lot of
work behind the scenes
to bake accessibility in to all
of the standard AppKit controls.
So whenever possible, try to
use those because we've done
as much work as possible
for you.
The one thing we needed to
do is just add a description
which is quite simple.
So now that we've worked on
the "plus" and "minus" buttons,
let's move on to the
Restart Game button.
You might recall that this
was one of the items that...
nothing was reported
there for accessibility.
Let's take a look at why.
So I have a class
for this button.
And the first thing I
immediately notice is
that it's an NSView subclass.
I'm doing a lot of extra
work behind the scenes
to make this view act
and look like a button.
This includes all
of my own keyboard
and mouse event handling, as
well as all of my own drawing.
I'm doing a ton of
extra work here.
Well, how do I add
accessibility to this element?
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Well, how do I add
accessibility to this element?
It turns out it's
also quite simple.
All I need to do is pick
the accessibility protocol
that best represents
this object.
The protocol for this is the
NSAccessibilityButton protocol.
Now that I've conformed
to that protocol,
I can move into my
implemenntation file.
And the first thing I see
is I get two new warnings.
So this is really,
really helpful for me,
because now I know exactly
what I need to implement
to add accessibility
to this custom control.
The two methods here
are accessibilityLabel
and accessibilityPerformPress.
So let's take those
one at a time.
I have a section here
for Accessibility.
The first one is
accessibilityLabel.
Great. Now what do I
want to be returned here?
I happen to know that this
button is only ever used
to draw the Restart Game button.
So the label for this is just
going to be "restart game."
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So the label for this is just
going to be "restart game."
That's it.
So now I can go back up
to the top of my file.
I notice that warning
has gone away.
So I only have one more
method to implement.
And that's
accessibilityPerformPress.
Let's do that.
"performPress."
So what should happen here?
Well, when I use the mouse to
interact with this button --
when I click it something
happens, right?
That same something
should happen
when an accessibility
client says, "hey,
I want to activate this item."
So in my case, I happen to have
a method called pressDown...
or performPress rather.
And performPress
is what gets called
when the mouse handling
logic says "yes,
this button has been pressed."
So I want to call that
same, exact method.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
And now you notice
that accessibilityPerformPress
return is a Boolean.
This is a yes-or-no,
"did I handle this?"
event. And I'm always handling
it, so yes, I return "YES."
And with that I can go back
up to the top of my file
and I see all of my
warnings have gone away.
So that's it.
The protocol has guided
me through what I needed
to do to add accessibility.
I implemented that
and I'm finished.
So now that this lower
right section's finished,
I want to work on the Artificial
Intelligence checkbox.
If you recall again,
this was another element
that wasn't visible to
accessibility at all.
So I have my header file
here for the checkbox.
And again I see the same
situation I had with the button.
It's an NSView subclass.
So I do a lot of work to make
it look and act like a checkbox.
So in this case, I need
to pick a protocol.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So in this case, I need
to pick a protocol.
And the protocol for this is
the NSAccessibilityCheckBox
protocol.
And now when I go into my
implementation file I get a few
new warnings.
So this tells me
what I need to do
to make the checkbox accessible.
So the three warnings I have
here are accessibilityValue.
accessibilityLabel.
And accessibilityPerformPress.
So I'm going to take
those one at a time.
So the first one, we've
seen this one before,
is accessibilityLabel, so
I'm going to do that first.
So what should be the
label for this checkbox?
Well this is a control
that I designed
and I thought I might
reuse in other places.
So I actually have an iVar
around that keeps
the checkbox text.
This is the string that actually
gets rendered in and drawn
in the drawInRect method.
So I can just return
the checkbox text here.
Great. So one of my
warnings has gone away.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Great. So one of my
warnings has gone away.
The next one I want to work on
is accessibilityPerformPress.
So just like the
button, when I interact
with this control using the
mouse, something happens
that changes the checked state.
I need that same
something to happen
when the accessibility
clients say "hey,
I want to interact
with this element."
So that's the
accessibilityPerformPress.
And I have a method
called toggleCheckedState.
And this is what gets called
from the mouse handling logic.
And just like with the
button, I return "YES,"
I did in fact handle
this action.
So now if I go back
up to the top I see
that I still have one more
method I need to implement.
And this is accessibilityValue.
And at this point
I ask myself, well,
"what is the value
for a checkbox?
What am I supposed to return?"
We've worked really, really
hard to try to answer all
of these questions in one place.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
of these questions in one place.
We tried to make
NSAccessibilityProtocol file the
one-stop-shop for this.
This is where all the protocols
are implemented or defined.
So in another tab I have
this protocol's file.
And there is the
NSAccessibilityCheckbox protocol
right here.
And I see under the required
methods the AccessibilityValue
returns an NSNumber.
Okay, so that answers
my first question.
And it says I return "yes" if
checked, or "no" if unchecked.
Great. So now that I
have this information,
I can come down here and
implement that method.
So here I have another
iVar around,
which is "am I checked or not.?"
I can use that to
return "yes" or "no."
Oh return.
All right.
Okay. So now I can go back
up to the top of my file.
I see all the warnings
have gone away.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
I see all the warnings
have gone away.
So this is all I need to do
to implement an accessible
CheckBox.
So now the entire
right-hand side
of our application has added
some basic accessibility to it.
So let's work on the next
part, which is the grid,
and this is perhaps the
most difficult part.
Again, I'm going to
back into my project,
and I have a view
representing the board --
it's the board view.
And here I notice:
it's, again, an NSView.
I'm doing all of my
own drawing here.
And in fact this class draws
the board and all of the pieces.
So I need to make
this accessible.
And I need to pick
a protocol now.
So what protocol do I need?
Well the way that
I like to think
about the Tic Tac Toe board
is it's really a collection
of squares, right?
Every time I interact
with the board,
it's not so much a board --
I want to play this square;
I want to put my piece there;
I want to get three in a row
to win; I want to
block my opponent
from getting three in a row.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
from getting three in a row.
And so this board is
really a collection to me.
And the way that that's...
the protocol that you
can use to represent
that is the NSAccessibilityGroup
protocol.
It's a collection of things.
Great. So now I go to
my implementation file.
And I notice I have no
warnings showing up.
Okay. I have no warnings.
So what does this mean?
Well I have no required
methods that I need to implement
for the accessibilityGroup
protocol.
But just like with what we
saw with the other elements,
I can still add accessibility
to it.
So what's the first
thing I want to do?
Well, the board view
-- when I land on it --
it's a group and I want
some label to describe it.
So just like with all the
other controls I'm going
to add a label here.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
to add a label here.
An accessibilityLabel
is very simple.
So what am I going
to return here?
Well just, it's the
Tic Tac Toe board.
So when the VoiceOver
cursor lands on this,
it says this is the "Tic
Tac Toe board" group.
Okay, so I have a group
that contains things.
Well what are these things?
They're individual squares.
But each of those squares
doesn't have a backing view
to them.
So I need to create some object
to represent it to
accessibility.
And the way that
we've done this is
with the NSAccessibilityElement.
Patti showed you in the slides
how you'd create an instance
of this inline.
And what I've chosen to do
here is just create a subclass
of it in another file.
So I have a subclass here
called the TicTacToeSquare
AccessibilityElement.
And it's a subclass of
NSAccessibilityElement.
It has one initializer,
which takes a row,
a column, and a delegate.
And it has three properties: a
row, a column, and a delegate.
So the delegate method
-- or the delegate --
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So the delegate method
-- or the delegate --
is just a class that can
answer questions about "hey,
what pieces are at this
location on the board?
Is the game over?
Is it still in progress?"
So it can answer a
lot of the questions
that this piece is
going to need to ask.
So I have this element
representing it.
Well, I'm going to
pick a protocol now
to use to implement it.
So what best encapsulates
what a square does?
Well when I click on
it, something happens.
I place my piece there.
I put my piece there.
I play it, right?
And elements where you click
them to interact with them
and they do something
are buttons.
So I can conform
to the
accessibilityButtonProtocol
here.
And now when I move into
my implementation file,
I'll get a bunch of
warnings for the items
that I need to implement.
So the four warnings I get here
are accessibilityParent, Label,
PerformPress, and Frame
need to be implemented.
So I'm going to take
these one at a time.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
For the frame and for the
parent, I'm just going
to return whatever my
superclass' implementation is.
So at some point these will need
to be set, so keep that in mind.
The accessibilityLabel is a
little bit more interesting.
And this is the description
for a particular square.
So what I'm going to do is
ask my delegate for "hey,
what's this square at
my row and column?"
And this is actually an instance
of the model object representing
a square on the board.
And then I can ask the board,
using one of its class methods,
"what's the description
for this square?"
So this is going to
return something like,
"this is an X square
at the top right."
Or an "O square in the center."
Or an "empty square
at the bottom left."
And so when I land
on this element,
that's what I want
to be spoken out.
And then the last method I need
to implement is
accessibilityPerformPress.
So when this item is
pressed, what should it do?
Well, I want to play my
piece there; I want to play
that square; I want
to make my mark there.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
that square; I want
to make my mark there.
And my delegate knows how
to play a square at
a row and column.
So I'm just going to ask it to
do that and again, return "yes,
I handled this event."
So with that I can go back up
to the top of my file and I see
that all the warnings
have gone away.
So that's great.
I have, now, a class that
represents an individual square
for accessibility that
doesn't have a backing view.
So now I need to go
back to my group.
I need to create
instances of this class
to represent each square.
So I'm back in my board view.
And I want to say, "this group
has children of these elements."
And the way to do
that with our API is
to override the
accessibilityChildren method.
So I'm going to walk through
-- it looks like a lot of code;
I promise it's not -- I'm
going to walk through,
line-by-line, what this does.
The first thing is,
I'm overriding the
accessibilityChildren array.
So, I'm the accessibility
ChildrenGetter
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So, I'm the accessibility
ChildrenGetter
and returning an array from it.
I have iVar around.
So if I've already
created my children,
I don't need to create
them again.
And if I haven't, I
create an array to hold all
of the accessibility Children.
And then I set up two
variables that I'm going
to be using during
my computations.
Now what I need to do
is create an instance
of that accessibility element
for every square on the board.
So I'm going to iterate over all
the rows and all the columns...
and then instantiate one
of those accessibility
elements for the square.
Now remember that the parent
and the frame were just returned
as whatever the super-classes'
implementation was.
So I need to make sure
that they're set somewhere,
and that's what I'm
going to do it here.
So I can set the accessibility
parent using a property.
And I just set it to
myself because the parent
of a square is the board.
And then I need to
set the frame.
Typically the accessibility
frame has always been given
in screen coordinates.
And It's important for us
because that's what we use
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
And It's important for us
because that's what we use
to draw the VoiceOver cursor
around a particular element.
And we were thinking about this
with the new API and saying,
you know, it'd be really
cool if we could just say,
here's my frame relative
to my parent.
And then I don't have to do
any work, when the window moves
or my parent moves around,
to recalculate my
frame in screen space.
So we've added this nice
convenience property
to NSAccessibilityElement
and it's called
accessibiltyFrame InParentSpace.
It does exactly that.
I can calculate the actual rect
that I'm using during
the -- for drawing.
That's given in my parent space.
And I can set that as
the accessibilityFrame
InParentSpace.
And now whenever
something moves,
we have enough information
on our end to recalculate
where the frame is,
and you don't have
to do any of that bookkeeping.
So now that we've created
this accessibility element
for each square, we
want to make sure
that we add them
all to the array.
And then as Patti said,
you want to make sure
that you cache these elements.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
that you cache these elements.
So at any point in your
applications lifetime
where this item is
being drawn onscreen,
the accessibility element
representing it has to be there
for accessibility
to be able to query.
And then lastly, remember
we overrode the getter here
so we need to make sure we
actually return these children.
So with that I can go back
up to the top of my file...
and I finally just made this
really complicated UI accessible
with not too much extra work.
I see that all of my warnings
have gone away (well we didn't
have any warnings
here, for example).
But this has now added a basic
level accessibility to my app.
And so let's take a look at
what that looks like now.
And the way I'm going to
verify that my changes worked,
is by using one of the
tools Patti mentioned.
It's called Accessibility
Inspector.
And you can launch it
from the Xcode menu
under Open Developer Tool
and Accessibility Inspector.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
under Open Developer Tool
and Accessibility Inspector.
Again, what this does is it
shows you the accessibility
information for the
item under the mouse.
So here, there's three
sections that are showing up.
The very top is the
accessibility Hierarchy.
So if this is analogous to...
it's kind of like
the view Hierarchy.
If your accessibility
elements aren't
in the accessibility
Hierarchy, they're not going
to be available to
accessibility clients.
Much like a view that's not
in the view Hierarchy
won't be shown.
The next section
is the Attributes.
So these are all the properties
that we have now added
onto accessibility elements,
and you can see each
of their values.
And there's also some
more complicated ones
like Parameterized Attributes.
So the very first
thing that we worked
on was the "plus and minus" one.
And remember this
was to increase
and decrease the difficulty.
And we needed to just
add a label there.
It was a standard
AppKit control.
And what I can see
is that the access...
down here is showing up
as decreased difficulty.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
down here is showing up
as decreased difficulty.
So this is good.
And we know that this
element is now labeled.
It's going to be spoken when
it's navigated with VoiceOver.
Same thing with the
Increase difficulty button.
The label is there,
which is good.
What about the Restart
Game button?
Well remember this wasn't even
available to accessibility,
and now we see that
it's in the Hierarchy
as an accessibility button.
And it also has a label
of "Restart Game."
So this is great.
it's going to be read out
as a Restart Game button.
Similarly with a checkbox,
it's now in the view
Hierarchy (which is awesome)
so it will show up to
accessibility clients.
It says it's a checkbox.
It has a label of AI opponent.
And now the grid.
So the grid shows up as a group.
And one of the accessibility
properties is the
accessibility Children.
And we see that it has nine
items and they're all buttons.
So if you take a look at
just one button, it says it's
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So if you take a look at
just one button, it says it's
in the Hierarchy, as well
as it has a label for
"it's the empty top left."
The next one says "empty top."
"Empty top right."
And you'll notice here also
there's an action section
on the bottom saying
PerformPress.
And it says, this is what this
element supports for actions.
So that's actually really great.
Okay so now that
I've audited my app
with Accessibility
Inspector, I'm fairly certain
that the accessibility I've
implemented is correct.
So now I'm going to
turn on VoiceOver;
I'm going to audit
it using VoiceOver.
>> VoiceOver on.
Tic Tac Toe.
Window. AI Opponent.
Unchecked.
Checkbox has keyboard focus.
>> Great. So we can now
land on this element.
>> Check. AI Opponent.
Checkbox.
>> I can activate it.
>> Difficult.
Decrease difficulty, button.
Press. Press.
Press. Decrease difficulty,
button.
>> The buttons are labeled,
and they support the actions.
>>2. Increase difficulty.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
>>2. Increase difficulty.
Restart Game, button.
Press. Restart Game, button.
>> I can press to
restart the game
if I had the game in progress.
>> Tic Tac Toe board ,group.
>> I also land on
the board group.
>> Interact with Tic
Tac Toe board group.
9 items. Empty top left, button.
>> And I can move
into this group
to see the different pieces.
>> Empty top, button.
>> Now if I place a piece there
using the Activate option.
>> Press X top, button.
Empty top left, button.
Press X top left, button.
VoiceOver off.
>> I can play a full
game of Tic Tac Toe now.
So this is really exciting,
and I hope you've seen
that with just a few lines of
code we can really walk you
through what you need to
do to add accessibility
to something like this.
And I hope you really
see that the level
of accessibility
is quite drastic.
Before, there was hardly any,
and now there's almost
full support.
So there's a couple more
things that you can do here
to make this app
even more accessible.
I'm just going to point out one.
Suppose for example that
I'm playing this game
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Suppose for example that
I'm playing this game
with a user who's
using VoiceOver
and they're on the checkbox.
And focus is on the checkbox.
They can activate the item
using "command, option, space."
But I can also go in and click
that checkbox with my mouse.
So how does a VoiceOver user
know if I've changed the state
of something out
from underneath them?
Well we've solved
this problem using
Accessibility Notifications.
And there's always been
some questions about,
"which ones do I use?
When do I use them?"
And we really tried to answer
that with the new
protocols file.
So again, this is
your one-stop-shop,
and you can really
go back there.
Here in the checkbox protocol
declaration, there's a comment
which tells me about the
notifications I need to post.
So in this case it says,
"the AccessibilityValueChanged
Notification needs
to be posted whenever
accessibilityValue changes."
Recall that's the checked
or unchecked state.
So if at any point it
changes for any reason,
I need to send this
notification.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
I need to send this
notification.
And it's really easy to do.
So I'm going to go back
to my checkbox view.
And if you recall, we had
the toggleCheckedState
and this is what got activated
when you used the
AccessibilityActivate action.
And here's where I need
to post that notification.
So I can do that with
NSAccessibility PostNotificaton.
The element this notification
should be associated
with is the checkbox itself.
And then the notification
type is an NSAccessibility
ValueChangedNotification.
That's it.
So again, some of the more
advanced accessibility items now
don't actually take that
much more code either.
And so now if you were
to run this app again,
and you were collaborating
with the VoiceOver user,
and you clicked that button
they would now be announced --
ah, the new state would
now be announced to them.
So that's the end of the demo.
And I hope what you've seen
is that we've tried to make it
as easy as possible for you
to add accessibility
with our new API.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
I should also mention that
we have a sample app --
you may have seen
this in the past --
we call it Accessibility
UIExamples.
And what it is, is a collection
of all different
types of elements.
So here there's a
button, text, images.
We also have more complicated
UI like layout areas,
collapsible outline
views, table views.
And what we've done
is implemented each
of these using different
methods.
So for example, subclassing
from standard AppKit controls,
making it a subclass of NSView,
but also OpenGL CALayers --
we have examples
for all of those.
And for this release we've
gone through, we've ripped
out the accessibility
that we had,
and added it right back
in with the new API.
And what we found was a
drastic code savings: many,
many fewer lines of code.
And what we hope to do
is have this available
as a reference for you.
Say "hey, I want to learn
how to use this protocol,"
you have an example
in a concrete implementation
that we've done.
This is available, along
with the Tic Tac Toe app,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
This is available, along
with the Tic Tac Toe app,
on developer.apple.com today.
Please go check it out.
Download it.
We also have, in this particular
example, a nice readme file
that has kind of a
developer quickstart
on how to use our API.
So with that, in summary,
we really took this release
to think about "how
could we make it easier
to add accessibility
to your application?"
What we came away
with was a brand new,
really easy-to-use API.
And it's really easy
to use because all
of the accessibility
attributes are now properties
on the objects.
So they're really easy to set.
We also have the accessibility
protocols, which guide you
through the items that you
need to implement in order
to make your custom
UI accessible.
And then lastly we formalize
the faux UI element class
into NSAccessibility Element.
And it's there for you to be
an accessibility backing object
when there's no UI view for
it, or NSView backing it.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
We'd like to encourage you
to test with VoiceOver.
It ships with every Mac and
it's a great way for you
to see what our customers
are going to experience using
that assistive technology
with your application.
If you'd like to just kind
of debug a little bit more,
we have Accessibility
Inspector there to really get
in at what exactly is going on
with your accessibility element.
And lastly, we wanted
to support you
in this transition
to the new API.
And we really want to
encourage you to use it.
And so we've built up these
really great developer examples
that we've now released, and we
hope you can use as a reference
for your implementation.
If you'd like more
information about our team,
or accessibility, or frameworks
in general, please feel free
to reach out to our App
Frameworks Evangelist,
Jake Behrens.
His email is behrens@apple.com.
And if you have more technical
questions, you can take a look
at the documentation
on developer.apple.com,
as well as ask questions
on the dev forums.
If you liked this session, and
you're interested in finding
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
If you liked this session, and
you're interested in finding
out how accessibility works
on our other platforms,
there's an "Accessibility on
iOS" session in this room,
right after this presentation.
And then a Web Accessibility
presentation
on Friday at 9:00 am.
Thank you so much for
coming to our session.
We really appreciate
you being here.
And from all of us at Apple,
we hope you have a
fantastic rest of WWDC.
Thank you very much.