Transcript
[ Music ]
[ Applause ]
>> Good morning.
My name is Susumu Harada.
I'm an engineer on the
Accessibility Team.
And I'm here today with
my colleague Patti.
And we're really excited to
introduce to you a new tool
that will make it easier than
ever for you to find, diagnose
and fix accessibility
issues in your apps.
So, some of you may be
new to accessibility.
So let me quickly go
over what it means
to make your app accessible
and some of the issues
that you may encounter
as you develop your apps.
So, accessibility
is about making sure
So, accessibility
is about making sure
that everyone can enjoy
the products and services
and the great apps
that you all create,
regardless of their abilities.
Now, some users may
have difficulty
with their vision
or their hearing.
And some may experience
physical and motor challenges,
or cognitive and
learning difficulties.
So, to support this
wide diversity of users,
we at Apple have worked
hard to make a number
of assistive technologies
that help augment various users'
abilities, such as VoiceOver
for people with visual
impairments, or Switch control
for people with motor
difficulties.
And we also made sure that these
are available on all devices
across all of our
platforms, from macOS
and iOS to watchOS and tvOS.
And while these assistive
technologies are great,
enabling tools, the heart
of our users' experience are
the great apps that you create.
So to better get a sense
for how your apps interact
with our assistive technologies
and the kinds of issues
that you may run into, let's
take a look at a quick example.
So here I have a simple app that
I created called Math Learner.
And it just presents a user with
an equation at the top to solve
and four answer buttons
on the bottom.
And the user can tap
on an answer button
and you get a feedback whether
or not the answer is correct.
It's pretty straightforward.
And it looks like its
functioning properly.
But how might a user who is
blind use this application?
So, obviously without any
assistive technology they won't
be able to see the screen.
So, it's pretty useless.
But fortunately, they can
use our screen reader called
VoiceOver to have
the information
on the screen read out to them.
And they can use various
gestures to navigate around
and interact with different
controls on the screen.
and interact with different
controls on the screen.
So, if this app is
fully accessible,
when the user touches on the
screen showing the equation,
VoiceOver will read
out: "5 plus 3 equals."
Great. So what does the
assistive technology
like VoiceOver need
from your app in order
to present the interface
in a meaningful way?
So, there are several key
pieces of information.
But first it needs to know
where all the important
elements are on the screen.
And for each of those elements
it needs to know their traits
or their types such as what is a
button, heading or static text.
And finally, it also needs
to know a descriptive label
that uniquely identifies that
element so it can be described
to the user in an
understandable way.
So with these key pieces
of information VoiceOver
and other assistive technologies
can present the interface
to the user in a
very usable manner.
And all the accessibility
information is provided
And all the accessibility
information is provided
by the app through
our accessibility API.
And we at Apple have
worked very hard
to build accessibility support
right into our UI frameworks
so that if you build your apps
using standard controls all
of this accessibility
information will be filled
in for you automatically
by the framer.
And you don't have
to do any extra work.
But of course, your apps
will ultimately contain
custom controls.
And when they do, you
may need to provide some
of this accessibility
information yourself.
And if you don't, you might end
up with an inaccessible app.
So let's see what the impact
of that might be like.
So, if I had implemented
that equation view
at the top using complete custom
drawing code like CA Layers
or OpenGL, so they don't have
any accessibility information
by default.
So unless I explicitly
provide them,
they might be completely
invisible to VoiceOver
and other assistive
technologies.
And with these four
buttons at the bottom,
And with these four
buttons at the bottom,
if I used custom views
that act like buttons,
but forgot to indicate
their accessibility trait
as being a button,
then VoiceOver
and their users will
have no idea
that they can press on these.
And finally, for the
Help button here,
if I implemented it using
custom drawing code,
and forgot to supply the
accessibility to the label,
VoiceOver will have no idea
how to describe it to the user.
And the user will have no
idea what that button is for.
So you can see how if you're
missing some key accessibility
information the user
experience for someone relying
on assistive technology
could be pretty bad.
So obviously it's very important
to make sure your app is
fully accessible, to make sure
that it has a positive
user experience.
But there's also
another added benefit
to making sure your
app is accessible.
And that is a fully
accessible app --
with a fully accessible app you
can write automated UI tests
for it using Xcode's
UI Testing Support.
for it using Xcode's
UI Testing Support.
And for more information on
how to do that, I encourage you
to check out UI Testing
With Xcode session
from last year's WWDC.
Great. So, how can you make sure
that your app is
providing all the necessary
accessibility information?
Well, one method that
we've been recommending is
to use the actual assistive
technologies like VoiceOver
and Switch control
with your apps.
Now, this is still
the ideal method,
since you can mirror the
actual user experience
and also catch usability
issues beyond whether or not
that something is accessible.
But it does require that
you be familiar with how
to use these technologies.
And especially if your app
supports multiple platforms,
you might find it quite
challenging to have
to repeatedly deploy your
app on a specific device,
run through it using each of
the assistive technologies
and repeat the process
across each of our platforms.
And all those platforms also
have a number accessibility
And all those platforms also
have a number accessibility
settings that the
user can configure
such the Dynamic Type font size.
And you as a developer might
find it a little tedious to have
to go and switch back and forth
between changing the
accessibility setting,
going back to your app,
verifying its effect on your app
and repeating the process.
So, we've heard all
your feedback.
And today we're very excited
to announce a new, an easier,
and more streamlined alternative
to auditing your apps using the
new Accessibility Inspector.
And in the new Accessibility
Inspector,
with just a single
press of a button,
it can automatically
audit your app
for common accessibility issues.
It's super easy.
And it even suggests to you
how to fix those issues.
And in the Interactive
Inspection mode you can really
easily, point at any
element on your app
and get very detailed
accessibility information
and get very detailed
accessibility information
about them.
And this works whether your
app is running on a simulator
or on a connected device.
And we also offer a live
preview feature for settings,
which lets you really easily go
and change an accessibility
setting
and immediately see
its effect on your app
without having to
leave your app.
And we're really
excited to announce
that this Accessibility
Inspector now supports all
of our platforms.
This means that with a single
Accessibility Inspector app
on your desktop you can audit
and inspect your apps running
on macOS and iOS,
watchOS and even tvOS.
And you can do this simply
by launching your app
on the simulator or on a device
and choosing the respective
target from the inspector.
It's that easy.
And we hope that with this
feature it'll make it easier
And we hope that with this
feature it'll make it easier
than ever for you to test
an audit for your accessibly
across all of our platforms.
So, to give you a demo of
all these exciting features
and the new accessibly
inspector,
I'd like to hand
it over to Patti.
[ Applause ]
>> Hi. I'm Patty Hoa.
And I'm super excited to
be the one to tell you all
about our new Accessibility
Inspector.
So, we've been working
really hard
to incorporate all
your feedback.
And so I can't wait to
show you what we have.
So, to note, Accessibility
Inspector is now part
of the developer package.
That means you can
simply go into Xcode
under the Open Developer
Tools section in the minibar
and launch the Accessibility
Inspector.
So here is a screenshot of
our new Accessibly Inspector.
I want to bring your attention
to the top section first.
I want to bring your attention
to the top section first.
On the left we have what
we call the Target Chooser.
That allows you to inspect all
the different Apple devices
that we have, including
your iPhone, your iPad,
your desktop machine,
your watch,
your TV and the simulators.
In the middle we have what we
call the Inspection Pointer.
Now, when you have this enabled,
then the tool can
now inspect what is
under your finger
on your device.
Or if you're in a simulator
or the desktop, it is what is
under your mouse cursor.
Then on the right we
have these three buttons.
These are three major
features of the tool.
And this is what I'll
be spending the rest
of the session telling you
more details and give you demos
of these three features.
So what are these
three features?
First, we have Inspection.
The Inspection feature
allows you to debug
The Inspection feature
allows you to debug
and analyze the accessibility
state of your app.
Then we have the Audit
feature to help you
to find accessibility
issues in your app.
And last we have the Settings
feature that helps you
to quickly test how accessibly
settings affect your app.
So the first one I want to talk
about is the Audit feature.
So, ultimately your goal is
to make sure your app is
accessible to everyone.
And to do that you need
to first audit your app.
Now, earlier Susumu
mentioned it would be awesome
if you could actually run with
VoiceOver or Switch control
and audit your app that way.
Or you could also hire an
accessibility consultant
to do the evaluation
on your behalf.
But today, I'm going to show
you a more cost effective way
to get that solution.
And that is simply use our
Audit feature of the tool.
And that is simply use our
Audit feature of the tool.
And so, in our Audit tab we
have this Run Audit button.
And the best way to
show you and tell you
about this feature is
to give you a demo.
So I have here on the
screen a running version
of the iOS version of the
Math Learner app running
in the simulator.
And so you could play a game.
We have equation.
We could pick one
of these choices.
Not that one.
This one. There.
So that's how you play the game.
Okay. So now I want to launch
the Accessibility Inspector.
So, I simply go to Xcode.
And on the top, under
Open Developer Tool,
we're going to launch the
Accessibility Inspector.
we're going to launch the
Accessibility Inspector.
So the first thing you
want to tell the tool is
which device you
want to inspect.
So in our Target Chooser, I'm
going to pick the Simulator.
And then we'll click
on this Audit tab here.
And here's that Run Audit
button I told you about.
So I'm going to go
ahead and click that.
In a split second we've
just audited this app.
And here are the
issues it found.
You can now select that.
As you cycle through
these issues,
you'll note that there's a
corresponding red highlight
over the area where
the problem resides.
And of course, if you're
inspecting an actual device
like iPhone or iPad, this is
probably the view you have.
So, now if you want to
know where the issue is,
So, now if you want to
know where the issue is,
you will have to take your
eye focus away from the tool
and look on your iPad or iPhone.
So what we have here is
a little nice features,
just give you a little
preview, a screenshot,
of that same element that
has the issue right here
in the tools.
So you could keep your eye
focused on the tool itself.
All right.
So we have these issues.
So what's the next thing
you'd probably want to do?
Go fix it.
So let's pick one of
these issues here.
Element has no description.
And click on the disclosure
triangle here to expand
to get more information.
All right.
So element has no description.
What does that correspond
to the code?
Do you know which
piece of code to write?
Well, if you went online,
read all documentation
about the accessibility API
that might give you some hint
on how to fix this issue.
But today I'm going to show you
a faster way to get that answer.
But today I'm going to show you
a faster way to get that answer.
Simply click on this
question mark here.
And the tool will tell
you which API to fix up.
It says: Consider setting
the accessibility label.
So, at this point, if you
know how the API works,
you would just go right into
your code and change it.
But if you want more information
about how this API operates,
you could click on
this hyperlink,
which then will take you to --
eventually take you to
documentation like this
where you could find more
information about the API.
All right?
Since now I know which API
to override, let's go right
into the code and fix it up.
So, what I'm fixing
is those button views
with the dots in there.
And the way it's implemented,
it's simply a custom UI view.
So now I'm simply going
to override the accessibility
label here.
And I'm going to return a string
that describes those dots.
And of course, you should
always return localized string.
Okay? So now I have my
method written here.
I'm going to save it,
run and rerun that.
Okay, let's go back to Addition.
And let's clear out
the old issues we have.
And rerun the Audit.
With that simple
change in the code,
we've just fixed up
a bunch of issues.
Isn't that great?
[ Applause ]
All right.
So, note that this same powerful
audit feature can be utilized
by other teams of your
member, like QAs who may want
to do an assessment
on your behalf.
And of course, then they will
want to report those bugs.
So, to speed up the process
in reporting those bugs,
they can now just,
after they run Audit,
they could simply copy and
paste it into the bug report.
And of course, every good bug
report should have a screenshot.
So we have here this
Quick Look button here.
If I click on that, you should
get a nice big screenshot
of your entire screen,
plus where the problem
resides in this blue outline.
So again, you could
just copy that
and paste it into
your bug report.
And now you're ready
to send off that bug.
And now you're ready
to send off that bug.
[ Applause ]
All right.
So what you've just seen is how
the Audit feature can help you
auto-detect problems
in your app.
It could give you
suggested solution.
And even speed up the process
of reporting those bugs.
As powerful and as
great as this feature is
in helping you detect
issues in your app,
similar to the compile
warnings in Xcode,
having no warning does not
mean your app doesn't have
any problem.
So that's why it's
important for you to do some
of that manual auditing
yourself.
And to show you why
this is important,
And to show you why
this is important,
let me give you an example.
So, suppose you have a button
and its function
is to send a mail.
Of course, the button comes
with a bunch of properties.
One thing we tend to
like to do when we want
to create a new button
is we simply take one
of these existing button
and we duplicate it.
And of course, that
means all the properties
of that button gets transferred.
And we go and tweak
some of the properties
like the actions and the icons.
That's fine.
That's all good.
Except if you forget to
update some of the property,
that could be a problem.
Especially in this case,
it's the accessibility
label property.
For assistive clients
like VoiceOver,
the accessibility label
property is super important.
Because that's the only
piece of information it has
about how this button functions.
So, if an assistive client
like VoiceOver were to land
on this button, and you haven't
updated the accessibility label,
it still says Send Mail.
But they're going to
press a button thinking
that they're sending a mail.
In fact, they've just
deleted that mail.
So that's not a great outcome.
And so for that, we're
here to help you go
through that manual
auditing using the tool.
And that's why I want to
talk about Inspection.
So let me give you some
highlights of Inspection
and show you how that helps you
through the manual
auditing process.
So, suppose we have
a button here
that has eight dots in there.
In the Inspection tab on the top
section there's the Quick Look.
And on the left side we have
the place where we describe,
or give you a summary of how
this element's being described.
or give you a summary of how
this element's being described.
So in this case the eight-dot
button, we'll just say 8.
And to the right there's
the Navigation button
that allows you to
easily navigate
to the neighboring UI
elements from this element,
from this button here.
Then below we have
the basic section
that contains properties that's
used to describe this element.
So for example, we have
Label, which corresponds
to Accessibility Label.
And returns the value 8.
And we have Trait.
In this case, it's a
button, so it returns Button.
And as this button
is interactable,
you'll look into the
Action section and see
that there's Activate
Action for this.
And so you could simply click
on the Perform button to verify
that this action works.
So that's a highlight
of Inspection.
Let me go ahead and
give you a demo of that.
Let me go ahead and
give you a demo of that.
All right.
So we're back here on our app.
And now we're going to go
into the Inspection tab.
So remember earlier I mentioned
there's this Inspection Pointer.
Right now it's enabled, which
means when I take my mouse
over the items in the simulator,
the tool will start inspecting
that element under the cursor.
And you can see that
green highlight.
So on the simulator in desktop,
if you simply do a single click
on that item, it will turn
off the Inspection Pointer,
which then allows you
to now move your mouse
around without affecting
what is being inspected.
So right now we're
inspecting the addition.
And you see a description
for that.
Now, I could click on the
Go To Next Element button.
Now, I could click on the
Go To Next Element button.
And you'll see the highlight
goes down to these buttons here.
Click again and we
go to the next one.
And also you get a description.
This is the four-dot.
Note that we completely
bypass something important:
The equation.
So simply doing this
navigation you could kind
of find issues along the way.
In this case we might have a
problem with this equation.
Let's keep going.
So, we're on the
eight dots, nine dots.
And down to this
Help button here.
Except the description
doesn't look so great, does it?
IMG 0151. Hmm.
Doesn't sound right.
So, again, that may be
another potential area
where you need to fix it up.
So, if you could simply use
this Quick Look section,
navigate through all the
UIs in your app and verify
that all the description
sounds good,
then you're on the right track.
And of course, if you have
any UI that is interactable,
like these buttons,
you will simply go
to the Action Section and, in
this case I'm going to click
on the Perform button
to activate that button
to verify the action works.
All right.
So what I've just shown you is
how you could use the Inspection
to help you along with your
manual auditing of your app.
Now the Inspection is also great
to help you do some debugging
and analysis of the
accessibility state of your app.
An example of that is
understanding how the
accessibility hierarchy works,
which becomes quite important,
especially on an OS X app
where the corresponding
view hierarchy tend
to be more complex.
So to show you how this
feature works, let me go ahead
and launch the OS X
version of the same app.
and launch the OS X
version of the same app.
All right?
So I have here the OS X
version of the same app.
And let's do something a
little bit more complex,
multiplication.
And so then when you
play, 3 times 6 --
>> Sorry, try again.
>> -- that, how about this?
>> Incorrect.
Try again.
>> -- 6. 18.
>> Correct.
>> Okay, so you play
the same game.
And you know, this app works.
We have the same version
app running on iOS and OS X.
All right.
So the first thing you
want to do is tell the tool
that you're inspecting
this new app.
So, we're going to go
and select my MacBook Pro
because we're inspecting the
desktop itself right now.
And in this case we
could also select
which app we want to inspect.
So I'm going to pick
the Math Learner.
And again, we could run Audit
and see what issues we find.
And it found a bunch of issues.
I want to bring your attention
to these four issues here.
It says: "Potentially
inaccessible element."
So the way these answer
choices were implemented,
they were simply custom NSview.
That's drawing those
text via CA text layers.
Now, by default, CA layers
have no accessibility support.
Which means if you create them,
clients like VoiceOver is not
going to be able to see them.
And so the tool is
telling you that right now,
that these are not accessible.
So to understand
how to fix this up,
it helps to know how the
accessibility hierarchy works.
So let's go to the Inspection.
And again, I'm going to
turn on Inspection Pointer.
And so if I bring my
mouse over and, again,
it's going to inspect
whatever's under my mouse.
What I'm interested
in to inspect right now
is the entire window.
And the reason is
because I want to know
who are the children
of this window.
So, the way I could I look
at that is I could look
under the hierarchy
section here.
And then here are all the
children of this window.
So, I'm on the equation.
You can see that red highlight.
On the image.
Some status text.
And a bunch of stuff in
the Tool Bar section.
So right now, those four crucial
answer choice buttons do not
exist as children of the window.
And so we know we need
to go and fix it up.
But today I'm not going to
show you how to fix it up.
Instead, I'm going to refer you
to our sample code
accessibility UI example.
It's something to download
from developer.apple.com.
This sample app contains
a whole library
with different UI scenarios
and how you could add
accessibly support for those.
So I encourage you to do that.
And once you fix it up, I expect
that those four answer choice
buttons should then appear
as children of this window.
So you could verify
it in this hierarchy.
So, what I've shown you
so far is how you can use
the Inspection feature
to help you do some of
that manual auditing.
And also how you could
use Inspection feature
to do some more debugging
and analysis
of the accessibility
state of your app.
The next thing I want to
talk about is Settings.
So, on every OS platform
we have many,
many accessibility settings.
That comes built right in.
So for example, on the
iPhone, in the Settings app
under General Accessibility
you're going to find many,
many wonderful accessibility
features built right in.
many wonderful accessibility
features built right in.
And settings like Toggle.
So let's say if you want to
know how Dynamic Font affects
your app.
You could go in, adjust the font
size to something really large.
Then go back to your app.
And then visually verify how a
large font looks in your app.
Then you could go back
to the Settings app.
Readjust to something
really small.
Then see how a small
font appears in your app.
Now, you could see that this
could become quite tedious
if you have to constantly
switch back and forth
between the Settings
app and your app.
And how do you know
which accessibility settings
are important to test?
Well, our tool will
answer that for you.
So, here's our Settings tab.
Our Settings tab
now present you some
of the most common
accessibility settings
that we want you to test.
In addition, since it's
built right into the tool,
you no longer have to
switch back and forth
between the Settings
app and your app.
[ Applause ]
And here's a demo to
show you that can work.
So in the Settings tab, now
you can invert the colors.
And immediately see the
effect it has on your app.
And we could do the same
thing with the font size.
Adjust it.
And see it directly.
So, what you've just seen is
how cool the Settings feature is
to make it easier for you.
So, today I've shown you
our three major features.
The first is the Audit
feature, that help you
to autodetect accessibility
issues in your app,
give you suggestions
on how to fix it.
And make it faster for you
to report those issues.
Then we saw how great
the Inspection feature is
to guide you through that
process of manual auditing
and allow you to do
debugging analysis
of the accessibility
state of your app.
And last, you just saw
the Settings feature
that will make it
much easier for you
to test how accessibility
settings changes affect
your app.
And that is the quick look
of Accessibility Inspector.
[ Applause ]
>> Great. Thank you Patti.
So we really look
forward to having all
So we really look
forward to having all
of you try the new
Accessibility Inspector.
And we hope that you now
feel motivated to keep users
of all abilities in mind when
designing your next great app.
And to help you do that,
we introduced you today the
new Accessibility Inspector
that we hope will make it easier
than ever for you to find,
diagnose and fix accessibility
issues in your app.
And it integrates seamlessly
with your development workflow
so that you can easily
inspect and audit your apps,
whether it's running on a
simulator or on a device.
And we're really excited
to support all platforms
from macOS, iOS, to
watchOS and tvOS.
So we hope that with this
great new tool you will go out
and create wonderful
apps that can be enjoyed
by all users of all abilities.
So you can find more information
about this talk along
with the video and the
transcript at this URL.
with the video and the
transcript at this URL.
And we'd love to
hear your feedback.
So we'd love to hear from you
about how we can better
support your workflow.
And any issues you might
encounter while using this
Inspector to audit your apps.
So you can send us
enhancements and bug requests --
or bug reports, sorry,
not requests,
bug reports at
bugreport.apple.com.
And also participate
in our developer forum
at forums.developer.apple.com.
And in case you missed
it yesterday,
we highly encourage you to
go back and watch the videos
of these great sessions,
including the "What's New
in Accessibility" talk and the
"Inclusive App Design" talk.
And this evening we have
the Accessible Technology
and Inclusive Design
get-together
in Buena Vista Park.
Not the real park, but the
one on the first floor.
So we'd love to see you there.
So thank you very much.
And enjoy the rest
of the conference.
[ Applause ]