Transcript
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
>> WIL TURNER: Good morning.
And welcome to UI
testing in Xcode.
My name is Wil Turner.
With me is Brooke Callahan.
We both work on the
Xcode developer tools.
And I am extremely excited
today because we are sharing
with you a huge expansion
of the testing technology
in the Xcode developer tools.
That is UI testing.
With UI testing you can find
user interface elements,
interact with them, and validate
the UI properties in state.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
interact with them, and validate
the UI properties in state.
Along with UI testing, we've
introduced UI recording
which will allow you
to really rapidly set
up UI testing for your projects.
Finally, we've updated
the test reports in Xcode,
which show the pass and fail and
results of your test outcomes,
to accommodate new data
we have with UI testing.
So I want to talk about
the core technologies
for UI testing, the
first is XCTest.
The second is accessibility.
So XCTest is Xcode's
testing framework.
In it, you create subclasses
for your test cases,
you implement test methods, and
you use assertions to validate
that your expected
outcomes are holding true.
XCtest is integrated with Xcode,
which means you get everything
from the ID in terms of
code completion, debugging,
and the ability to run
your tests directly
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and the ability to run
your tests directly
from your source code, and
see the results right there.
You also get continuous
integration via Xcode Server
and Xcodebuild.
Finally, XCTest supports
both Swift and Objective-C,
so you can choose the
native coding language
that you're most
comfortable with.
XCTest was introduced in Xcode
5 as a unit testing framework.
In Xcode 6, we expanded it to
support performance testing.
This allows you to catch
regressions in your code,
and ensure that it continues
to perform optimally,
release to release.
Now in Xcode 7, we've introduced
UI testing which you can use
for both correctness
and performance testing.
So that's XCTest.
Now let's take a look
at Accessibility.
Accessibility is the
technology on our platforms,
that gives disabled people
the same great experience
on our devices and
with our applications
that all other users receive.
To make Accessibility
work, it offers a rich set
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
To make Accessibility
work, it offers a rich set
of semantic data about the UI
that technologies like Voice
Over can use to guide users
through the application.
UI testing uses that, and
Accessibility is integrated
with the UI kit, and app kit,
so, when you use controls
from those frameworks, you get
a lot of accessibility support
for free, right out of the box.
It also provides
APIs that allow you
to fine tune the accessibility
data that is exposed.
The key about this is
that with UI testing,
your tests will interact
with the application
just the way a user does.
UI testing has a
few requirements
that you should understand.
The first is it depends
on new features in the OS.
For iOS you need iOS 9 and
for OS X you need OS 10.11.
UI testing protects
your privacy.
And to do so it means your
iOS devices need to be enabled
for development and connected
to a trusted host running Xcode.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
for development and connected
to a trusted host running Xcode.
On OS X you'll need
to grant permission
to a special Xcode helper app,
and you will be prompted to do
so the first time
you run UI tests.
Let's take a look at what
you need to get started
with UI test in your project.
First of all there's a
new Xcode target type.
Traditionally, unit tests were
a specific target type in Xcode,
and now UI tests are
target type as well.
We've also introduced a
large set of set of new APIs
for UI test, and of
course UI recording,
which will really get
you started quickly.
So the Xcode testing targets
support the special requirements
that UI tests have.
This includes executing
in a separate process
from your application
that you are testing.
And it also handles the
permission to use Accessibility
in the privacy protection.
These targets have new templates
for both Cocoa and Cocoa Touch
and the assistant for
these will set everything
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and the assistant for
these will set everything
up the way you need
it to get started.
There's a target to be tested
setting for UI test bundles
that identifies the application
that you are testing.
In the new APIs there
are three key classes.
The first is applications.
The second is elements.
And the third is element query.
We will take a deep dive
into these APIs a little bit
later in the presentation.
UI Recording lets you interact
with your application
hands-on your device,
the simulator, or OS X Mac.
While you are doing so, it
generates the code necessary
to recreate those interactions.
You can do this to create new
tests or expand existing tests.
So let's take a look at
what this is all about.
Brooke, let's see a quick demo.
[Applause]
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
[Applause]
>> BROOKE CALLAHAN: Thanks, Wil.
So, without further ado.
The project I'm going
to be using
for the demo today is
the lister application.
This is an example project
that you can download
from developer.Apple.com.
So, let's get started, now it's
got my target configured just
the way I want, but the one
part I want to point out is
that the target to be tested
is the lister application.
This is the one application
that my tests will be
able to interact with.
So now I've got my
new test class here.
And there's a little stub test
method here and a setup function
which is going to
call, which is going
to launch the application
before my test method is called.
And it's going to do that for
all the test methods I add
to this class.
So Let's add a new test
from the lister app.
I'm going to put the keyboard
cursor in the method and click
on the record button
down the debug bar.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
on the record button
down the debug bar.
Now, Xcode is launching
my application.
And here it is.
The lister application allows
me to manage a series of lists.
So probably the most common
thing people do is add
and remove items
from their list.
So I am going to
click here, add item.
It looks like we have a lot
of health some things
in this grocery list.
So I'm going to add cookies.
You can see that, as I'm typing,
the source header is updating.
If I press delete, it also
removes what I removed
from the text field.
Next thing I'm going to do
is tap on the cookies item
to mark it as completed.
And another thing I can do
in this application is
remove items from the list.
So I'll click on edit,
and then delete cookies,
and the delete confirmation
button, and finally done.
Great. Now I have a
simple test that adds
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Great. Now I have a
simple test that adds
and removes an item
from the list.
So I'll click stop.
Let's see that in action.
It added the cookies
and it is removing it.
And we are done.
[Applause]
Thanks. As the test
interacts with UI elements,
we get implicit validation
that those UI elements exist.
But What we don't
get validation of,
are things like state
changes in the application.
For example, when we tapped
on that cookies button,
we don't get any
validation that the state
of the button actually changed.
And later on in the
test, when it taps
on the delete confirmation
button,
we know that the delete
confirmation button was tapped.
but running the test
doesn't validate
that the cookies row
actually is removed.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
that the cookies row
actually is removed.
So to get validation
of those for my test,
I want to add some
explicit assertions.
First thing I am going
to do is add an assertion
that the cookies button here
actually changes its state.
To do that, I will
add a new constant.
Call this let cookies button.
I'm going to change the test
to tap on that constant.
Now, to add the assertion I
need some state, some properties
of the element to assert on.
And cookies button
is XUI element
and XUI elements have
a value property.
What I'm going to do, is I'm
going to set a breakpoint here,
and run the test to that point.
So here we have the
test in the state
where it's added
the cookies row.
And it has yet to
tap on that button.
So I'm going to go
in the debugger,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So I'm going to go
in the debugger,
and print out the value
of this cookies button.
Little typo.
-- there we go.
I can see that the value of the
cookies button is the string
with the number zero in it.
Next I'm going to
step over that line.
So now it's tap the
cookies button.
I will print out the value
of it again and now I can see
that the value of the button is
the string with the number one.
Great. So now I have all
the information that I need
to assert that the value of the
button changes when it's tapped.
So I'm going to add an assertion
using XCT assert equal.
I'll assert that the value
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
I'll assert that the value
after the tap is a string
with the number one.
The value is in any object.
So I am going to need to
assert that this is a string.
So I'll use as string.
And assert that it's
number one after the tap.
And I'll assert that
it is the number string
with the number zero
before the tap.
And lastly, after the delete
confirmation button is tapped I
want to assert that that
cookies row goes away.
I'll call XCT assert
equal and assert
that the button no
longer exists.
Now iIf I run this test again we
should see it does all the same
thing but also passing
these assertions.
Great, so now I have
just added my first test.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Great, so now I have
just added my first test.
Back to you, Wil.
[Applause]
>> WIL TURNER: That
was pretty awesome.
You can see just how easily
Brooke took an existing
application and used it,
just like he would as a user,
and in a handful of
minutes he created a test.
He could expand that
test using XCT assert
to do some additional validation
and he just added reliability
to his project with
minimal effort.
That's pretty exciting.
So you can see we added
UI testing target,
very straightforward just like
all our other other targets,
with an assistant and templates.
Used recording, interacted with
the app, and it creates the code
that uses the elements
and synthesizes the events
and then the additional
validation with XCT assert.
Let's take a look at
this UI testing API.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
I mentioned earlier that
there are three classes.
They are XCUIApplication,
XCUIElement, and
XCUIElementQuery.
How do they work?
Let's start with a very
simple, something even simpler
than Brooke's example.
Line-by-line going through here
first I am instantiating my
application object.
This is a proxy for
my application.
Then I launch it.
That brings it up for me.
And then I use element and
query to find the add button.
And then I synthesize the
event that taps on it.
Finally, I add in an
assertion just like Brooke did
to make sure that the UI
had the expected state
at the end of the test.
So I mentioned that
UIApplication is a proxy
for the tested application.
It is independent of the
lifecycle of the launch
and termination lifecycle
of your app.
Because Your tests are
running in a separate process.
You get explicit control
over when the app is launched
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
You get explicit control
over when the app is launched
and when it is terminated.
When you launch we will
always spawn a new process.
So that's something
we do to kind
of not give you completely Clean
Slate because you have a lot
of state in your application
that it's up to you to control.
But by launching a
process clean each time,
we help you minimize the number
of variables that you have
to deal with in your test.
So if it's already running,
calling launch will terminate
the previous existing instance.
Application is also the starting
point for finding elements.
Let's talk about elements.
XCUIElement, just like
Application, is a proxy object,
but this time for user
interface elements
in the tested application.
Elements have types.
Types are like button, or
cell, or window, and so forth.
They have what we call
identifiers, strings of data
that we get from the
accessibility system,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
that we get from the
accessibility system,
hich are an identifier or
label or title, so forth.
Most of the time you'll find
elements using a combination
of the type and the label.
For example, if you have
a bullton which is add,
you'll look for the button
type with the identifier add.
Elements form a hierarchy
in your application.
The application is
the root of a tree.
If you think back to your
computer science days you'll
remember tree data structures.
With the lister application we
have this very simple tree here
with the application
at the top and we have
like the nav bar
and the add button.
Groceries label.
So on and so forth.
So each of these is an element
that you can reference
in your tests.
These are used by queries.
This hierarchy, as well
as type and identifier,
to find your elements.
Elements, when you
work with them
in your test, must be unique.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
in your test, must be unique.
So what does that mean?
Well, every one of these
elements is backed by a query.
The query has to resolve
to a single instance.
Otherwise when we go to
synthesize the events
that you tell us to tap but
there's multiple buttons
that match that, it
is not deterministic.
We can't really know what
you mean for us to do.
Similarly, if you ask us for
a property of the element,
we don't know which one
did you really mean?
Maybe there wasn't one at all.
So it's important to the
queries for the elements
to resolve to exactly one match.
If they don't, when you access
the element we'll raise a
failure for that.
There is one exception to this,
which is an API called the
exists property on XCUIElement.
This allows you to test for the
existence of the element safely.
You can use this as
Brooke did to verify
that the element had
been removed from the UI
and you can also use
this to handle cases
where you have conditional UI
that might display
in some context.
For example, if you are
saving a file to a location
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
For example, if you are
saving a file to a location
where another file already
exists, you might get some kind
of confirmation sheet.
That wouldn't be
present all the time.
Just the case where
there's these conflicts.
You can use an exist
check for those cases.
Elements are where event
synthesis APIs live.
Event synthesis is how we
simulate user interaction,
and do this at the lowest level
of the system so everything goes
through the same channels it
would when the user interacts.
The APIs for event synthesis
are platform-specific
with some exceptions.
We have button click,
for example in OS X
and the corresponding
is button tap on iOS.
We have type text which is
the same on both platforms
and takes a string of text.
XCUIElementQuery is our API
for specifying elements.
Queries resolve to collections
of accessible elements.
They can only find elements that
are visible to Accessibility.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
They can only find elements that
are visible to Accessibility.
And they will resolve
to a set of these.
That means you can
get the number
of matches using
the count property.
And you can also
specify distinct elements
in that set using
subscripting with an identifier
or using an element
at index API.
We will look at these
more in a moment.
So how do queries work?
So I mentioned that
element hierarchy.
So the relationships in
that hierarchy are one side
of how queries work.
The other side is by filtering.
Filtering is taking
one set and reducing
that set according
to certain criteria.
With our lister example again
here is how we might express
certain relationships, the
first of which is descendants.
In this case I'm
showing the view.
All the gold cells
are its descendents.
On the other hand children is a
more restrictive relationship.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
On the other hand children is a
more restrictive relationship.
It's just the elements that
are directly below the element
you're querying.
So, the table's children
are just the cells.
The final relationship
we use is containment.
This happens to be very
useful when we have cases
where the elements, like the
cells, they don't have a lot
of data that unique
them from each other,
but they contain
elements which are unique.
For example, the first cell
contains the groceries label.
Filtering lets us take
things like the element type,
or its identifiers,
to create queries
that filter a previous query.
We can also do this with
predicates which will allow us
to go beyond the identifiers
to look at the values
or do partial matching
such as "begins with"
and that sort of thing.
We combine relationships and
the filtering in the APIs
and the first of which is
descendantsMatchingType.
You can figure out
what that does.
It find the descendants
that match a certain type.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
It find the descendants
that match a certain type.
This turns out to be the
most common query you'll use.
Some examples of this are,
I can find all the buttons
in an application
by calling app,
descendentsMatchingType button.
Similarly I can find
all the cells in a table
by telling the table to give me
descendentsMatchingType cell,
or, another example,
with menu items.
This is such a common query
that we provide convenience API
for every one of these types.
DescendentsMatchingType
buttons, becomes just buttons.
DescendentsMatchingType
cell becomes just cells,
and so on and so forth.
These convenience APIs help
make your tests very expressive
yet concise.
[Applause]
ChildrenMatchingType is
the other combination
of relationships in filtering.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
of relationships in filtering.
So in that, that allows
us to differentiate
between descendants, all the
descendants that match a type,
and just those that
match, that are children.
Again the all buttons
example, is app.buttons,
if I want to find just the
buttons that are children
of my nav bar, I
can take my nav bar
and say childrenMatchingType
button.
It is not as common a
query, but there are cases
where it becomes very
useful to differentiate.
Finally, containing type.
This allows us to find elements
by describing their descendants.
So in the example
we have the cells
that are each somewhat
anonymous.
They don't have any
identifying characteristics,
but they do contain
unique labels.
Labels are also known
as static texts.
So here I can form a cell
query which takes the cells
and finds the one that is
containing the type static text
with the identifier groceries.
That will find the
first cell for me.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
There's also predicate
variant for this API.
So those are our three APIs
that combine relationships
and filtering.
DescendentsMatchingType,
childrenMatchingType,
containingType, and of course
all the convenience APIs
for descendentsMatchingType.
The other powerful
thing about query is
that they can be
chained together.
So we can take the output of
one query and make it the input
of the next, just like you
would pipe commands together
on a Unix command line.
This is ver powerful,
and lets you build
up very complex queries,
again in a way
that is concise and expressive.
So here we have our
application and I want
to find just the
labels in the table.
So I start with the app.
And then I get the tables.
And I ask for static
texts and I'm done.
I have those three labels.
So queries are sometimes
the end unto themselves.
You want to get the count of
the query and maybe assert
that you have the right
number of items in there.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
that you have the right
number of items in there.
But often the goal of a
query is to find an element.
All of our elements
are backed by a query.
To get an element from a
query we provide several
different choices.
The first is subscripting,
which allows us to take a query
and then subscript
using an identifier.
That would give me
back an element
which is the groceries label.
It can also do this
with element and index.
If I have a set, maybe
the rows in the table
and I want to iterate over them.
I could one at a time call
element and index on these.
If I have a query which
I know resolves uniquely,
hat is to a single thing, maybe
I only have one navigation bar
in my application, I can
use the element property
to create a new element
backed by that query.
So when are queries evaluated?
So they are not actually
evaluated just
when you create them.
They are evaluated on-demand
or as they are needed.
This means that with an element,
the query will be evaluated
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
when you synthesize events
or read property values.
You can create the element
but until you use it,
the query won't be evaluated.
Similarly if you create a query
directly it will be evaluated
when you get the number of
matches or if you call one
of the APIs that
returns all the matches.
It will have to be
evaluated at that point
and we will reevaluate
queries when the UI changes.
So you are always working
with the most current view
of the application rather
than data from ten seconds ago
or two minutes ago, depending
on the length of your test.
So in this way you
can think of queries
and elements being
somewhat similar to URLs.
They are, with a URL
you can create a URL
but it doesn't fetch a
resource immediately.
It is not until you actually
go to create your URL request
or session that the
actual URL gets resolved.
And so even if the
URL was invalid,
no error would be
raised until that point.
Similarly, queries and elements,
they are just specifications
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Similarly, queries and elements,
they are just specifications
for accessible elements
in the tested application.
So creating them doesn't do
anything until you need them
and at that point
they are resolved.
So that's the API.
There's three classes:
The application
for launching your
application; elements,
which like the application
are proxy objects for elements
in your app; and
finally queries,
which are more complex
ways to specify elements.
So now I want to
talk a little bit
about Accessibility
and UI testing.
I mentioned earlier
that accessibility data is
what makes UI testing possible.
So given that, it is not
hard to see how the quality
of the accessibility data
really impacts your testing.
In fact, the better
the accessibility data
for your application, the
easier it is to write tests
and the more reliable
those tests are over time.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and the more reliable
those tests are over time.
So you get a double benefit
when you improve
the accessibility
in your application.
You've not only made it
easier for your own testing
but you've improved
the experience for all
of our disabled users.
I would really encourage you to
keep that in mind when you work
with UI testing and
accessibility.
Sometimes up's need
to do some debugging.
It's an element may not be
accessible, may not be showing
up for you even when
you are using recording.
That could be because of
the custom view subclass
that may not be accessible
by default.
Or it's actually not a view.
It's a graphics object in a
lower level graphics subsystem
such as a layer and
that sort of thing.
In other cases, the
element is visible
but has poor accessibility data.
All those table cells
I was looking
at in the containment query,
partly that might have gone away
if the cells themselves had
better accessibility data.
When this happens, there's
a couple tools I would point
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
When this happens, there's
a couple tools I would point
you at.
The first of which is
UI recording itself
because UI recording will
give you the closest view
into how the testing
system sees the elements.
But beyond that there's also
great accessibility inspectors
on our platform.
They'll let you see the
raw accessibility data
as it is exposed
by the application.
When you need to go
and improve that data,
your first stop should
be Interface Builder.
Interface Builder has a
great accessibility inspector
which allows you to enable
or disable accessibility,
set values for the various
accessibility attributes,
and configure the traits
which have a direct impact
on how the element is expressed
as a type in UI testing.
There's also API if you're
working with elements
that you can't access
through Interface Builder.
You can use APIs
in NSAccessibility
and UIAccessibility to directly
control how the element is
expressed to Accessibility.
So with that in mind,
let's see another demo.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So with that in mind,
let's see another demo.
This time Brooke
is going to take us
through more complex test
cases and also a little bit
of accessibility debugging.
Brooke?
>> BROOKE CALLAHAN: Thanks, Wil.
[Applause]
>> BROOKE CALLAHAN: So in the
last demo we saw adding a test
that can add and remove
an item from the list.
So while I'm here I would
like to add some more tests
around this area
of user interface.
First I'm going to add a test
that adds multiple
items to the same list.
With this multiple items
with the same name to a list.
So to do that I'm going
to copy this code here
from the last test.
And I'll call this new
test, add to cookies.
I'll just paste that code in.
Great. Now I have a test
that is going to tap
on the groceries label.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
on the groceries label.
Next it is going to add the
new cookies item to the list,
and then it is going to
tap on that cookies button
in that item, and verify
that it actually gets
to that tapped state.
So to add the second
item I'm just going
to copy this code here.
And then to verify that the new
button exists and gets tapped,
I'm going to copy that section.
So let's run this test
and see how that works.
Now let's add the first one.
Second one.
Ahh, failed an assertion here.
So on this line here, where
we are getting the value
of the cookies button for
the second time around,
we are actually failing
the assertion.
Looks like it's failing because
multiple matches are found.
I think I have an idea
of what went wrong here.
The way that the cookies
button constant is specified,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
The way that the cookies
button constant is specified,
it is just looking for all
the buttons in the table,
and giving me one
that's called cookies.
By this point in the test,
there are two buttons
called cookies in the table.
It will find both of them
and there is not going
to be one value that
the element can return
because there's these
two matches.
Now, I only know that because
I just saw the test running.
Normally you are not going to be
watching your test as it runs.
So we thought we should
provide a way for you
to see what the test looks
like when it last ran
and we've added this information
to the test reports in Xcode.
So if I go to the report
navigator and click
on the most recent test report,
you can see the test
add to cookies test.
If I expand this item, you
can see all of the activities
that happen during the test.
And down here I see this last
find the cookies button item.
There's my failure.
Multiple matches found.
There's also a quick look button
here, where I can look on that
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
There's also a quick look button
here, where I can look on that
and it will show
me the exact state
of the application
when that happened.
Just like we know
happened, yeah,
there's two rows called cookies.
One of them is tapped and
one of them is not tapped.
[Applause]
>> BROOKE CALLAHAN: I am going
to close that and if I want
to see the complete
assertion failure,
I can go to the log section
and on this line I can
expand the log for that test.
And here we can see the complete
assertion failure showing the
accessibility hierarchy
of both those buttons.
And here on the side I can see
that yeah, there's one button
that is unchecked and
the other one is checked,
just like I expect.
Let's go back and fix that test.
The easiest way for
me to fix this test is
to use recording again.
I'll set a breakpoint on this
line right before the assertion
failure, and run the
test at that point.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
failure, and run the
test at that point.
All right.
So now I've got the application
in exactly the state I need it
to be to get the value
of that unchecked button.
All I'm going to do is I'm
going to click on recording,
tap on that button,
and stop recording.
Now I have a way to refer
to that second button.
I'm going to clean this
up by using the table
constant in both.
I'm going to call this
cookies button two,
a new constant in my test.
And now to fix the test
I'm going to change each
of the next three lines
to use that new reference.
And when I run the test again,
we should see that it works.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Great!
[Applause]
All right.
Now I've got one test that
can add and remove an item
from the list and another
test that can add two items
to the same list, and
verify they both exist.
Now I want to build a test
that will remove all
the items from the list.
Once again I'm going
to use recording.
I am going to tap on groceries,
edit, and then I'm going
to remove the apples
row from the test.
All right?
I'll tap stop recording.
So all this much of the test
is pretty much how I want,
but what I want to do is, I
want to make my test agnostic
to the data of the application.
I want it to remove all these
items, but I don't want it
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
I want it to remove all these
items, but I don't want it
to actually refer to the
items by their labels.
Because that is going to
get kind of wordy here.
So first thing I'm going to
do, you see these tokens here,
the tokens provide multiple ways
to get to the same UI element.
In this case, for that
first delete apples button,
I can get to this by calling
table.button delete apples.
I can be more specific and say
it's the delete apples button
in the cell called apples.
So I'm going to use that.
And I'm going to double click
on it to convert it to text.
So now I've got two rows here
where we are getting the
elements the same way.
What I would like to do is
get the cell simply by index.
I will add a constant called let
cell, table., and set this equal
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
I will add a constant called let
cell, table., and set this equal
to table.cells elements index.
Since the apples row was index
one, I'm going to use that.
I can simply replace these with
the reference to that constant.
All right.
I'm almost done.
The next part I need to change
here is how I'm getting the
button out of the cell because
the other rows are not going
to have a delete apples button.
They will have a delete
oranges button or delete bread.
What I need is a way
to find the button
where the label starts
with the word delete.
To do that I'm going
to use a predicate.
So here I'm using
matching predicate
to find the button whose label
begins with the word delete.
All right.
So the last thing I
need to change here,
I want to add an
assertion just like before.
I want to verify that
that cell goes away
after we've removed it.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
after we've removed it.
I am going to use
XCTAssertEqual,
and assert that the cells
exist property returns false
after we tap that delete
confirmation button.
I'm going to run the test now.
It is removing the apples
row but failed the assertion.
I have an idea what
might be going on here.
To show you what is going
on I'll use the debugger.
I'll set a breakpoint here
and run the test again,
to that same breakpoint.
All right.
Now in the debugger, I'm going
to call debug description
on the cell.
So the debug description
has a lot of information
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So the debug description
has a lot of information
about how the cell
is actually resolved.
So I can see here
that when I call it,
this cell result first
finds the application
and then it finds the
table in the application.
Then all the cells.
And then the element and
index in those cells.
I can see that it's actually
finding the oranges row.
It looks like what might
be happening here is
when we call the exist property
on this element called cell,
it is actually re-resolving
itself.
Even though we just
removed the apples row
from the table there's now
a new cell at index one.
So that is no problem.
It just means I need to
use a different way to find
out whether or not
we've removed the row.
So I'm going to add
an assertion,
assert that the number of cells
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
in the table goes
down at this point.
So I'm going to add a new
constant called "count"
and I'll set this equal to the
count of the cells in the table.
And then I'll simply assert that
that's equal to count minus 1.
Now, the last thing I need
to do, I said this was going
to be a remove all items test.
I'm going to change it to add
a wild loop and simply do this
over and over again as
long as there's more
than one cell in the table.
Let's run that.
So it's removing the
apples row, and oranges.
All right.
[Applause]
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
[Applause]
>> BROOKE CALLAHAN: Last thing
I would like to do is add a test
to use that color row that
we saw in the edit UI.
And once again I'm going to
use the recorder for this.
I'll tap on groceries and edit.
Now, the color UI here lets me
change the color for a list.
It looks like right
now this list is green.
So I'll try changing it
to red or how about blue?
Okay, so when I tap on
those buttons it looks
like it's not actually
recording what I want.
It looks like it's
recording a tap
on the static text called
color, which you see over here.
So I think what might
be going on here is
that these UI elements
simply might not be visible
to Accessibility.
So I'm going to stop recording,
and I can actually use the
accessibility inspector
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and I can actually use the
accessibility inspector
to tell me what's going on.
I'll right click on Xcode and
go to open developer tool,
accessibility inspector.
And the accessibility inspector
provides a lot of information,
but all I want to use it for
is ths shortcut, command S7.
It will highlight the UI
element that's underneath the
mouse cursor.
If I put the mouse cursor
over the word bread here,
and press command F7.
it highlights bread.
If I put it over the
delete bread button,
you can see that
it highlights that.
Now, let's see what happens
if I put the mouse cursor
over this yellow color button.
Aha! So it highlighted
the whole row.
So that pretty much confirms
that this UI element is simply
not visible to accessibility.
Luckily I can actually
change this,
and fix this problem
using the story board.
I'll open the story board now.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
I'll open the story board now.
So here I've got the same
buttons in my story board
and if I open the
inspector, here I can see
that these buttons actually have
a class of color tappable view.
I'm familiar with this class.
I know it's actually
not UI button.
It's custom view.
If I go down to the
accessibility part
of the inspector, I can see
it's not enabled for them.
I have gone through and
added labels for them.
To fix this all I need to do
is select all of the buttons.
And then I'll check the enabled
for accessibility check box.
Since they behave like buttons,
I am going to give
these the button trait.
All right?
Now let's run the
application again.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
And now I'm going to record
this test one more time.
Tap on groceries and edit.
Then red. And orange.
Yellow, green, blue, and gray.
Great. Now it actually
recorded all those.
I'll stop recording and
let's run it and see
if it can play it back as well.
Great. I've fixed
accessibility in my application.
And I've also made
it more testable.
If I was going to
complete this test,
I would probably add
some assertions to verify
that the state of
these buttons changes.
For now, I'll hand
the stage back to Wil.
[Applause]
>> WIL TURNER: So that's
really quite awesome,
especially just how easy
it was to make this view
that previously a Voice Over
user would have had no luck
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
that previously a Voice Over
user would have had no luck
with at all and just
with a few quick changes
in interface builder
Brooke was able to make
that both accessible
and UI testable.
So in the demo, some sort
of more advanced UI testing.
You saw he had cases where
he dealt with a conflict
and a query, and how to
correct those queries,
and how to debug it, and also
how he can loop over elements
and validate them, and how you
can use the exists property
and also highlighted how
queries are reevaluated.
They use the criteria that
you created them with.
That specification is
what gets reevaluated.
We used element at index one.
That pointed to the apples
label the first time through.
As soon as that was
gone and the UI changed,
now that was pointing to the
oranges label.And then finally,
how to improve accessibility,
and the rewards are truly
fantastic for doing that.
Brooke also gave you a
peek at the test reports.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Brooke also gave you a
peek at the test reports.
We have done some
work in Xcode 7
to overhaul them for UI testing.
To recap are, the test reports
are where you see the results
for all the tests that ran.
It shows the pass or the
fail, the failure message,
performance metrics
for performance tests
are shown in the reports.
You get the same UI in
XCode and in Xcode server.
It's a consistent
experience regardless
of whether you are looking
at integrations or the run
that you just did on
your local computer.
On Xcode server you also
get per-device results,
because you can have the devices
integrating multiple devices
at the same time.
The additions for UI testing,
we collected additional
data during UI testing,
and this includes screen shots.
You saw how it helped Brooke
debug the conflict in his query
by pulling up the screen shot
at the time that query failed.
Also, we organized the API calls
into these nested activities
that help you understand
how the API call is working.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
that help you understand
how the API call is working.
So let's take a look
at an example of this.
This is not the list draft.
It is a different application.
But I'm going to show you
how there are several steps
in the nested activities
breakdown.
This example we're
typing into a test field
so the high level you
call the API type test.
But internally that breaks down.
In the first step where we wait
for the application to idle.
We're actually observing
the main run loop
of the tested application.
Because We don't want
to send events to it,
when it's busy processing.
We want it to be
responsive as possible.
Once it's idled we capture the
data we need from Accessibility
and we resolve the query,
make sure it matches
exactly one thing.
Next step, we synthesize
the actual events
that insert the text
into that field.
Finally, we wait one more time
for the app to idle afterwards.
Because, again, we want to
hand off control in a way
where things are reliable
and deterministic.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
where things are reliable
and deterministic.
The quick looks are
provided for screen shots
that are captured
during critical steps.
You can see here is the state
right after I type that text in.
Make sure everything looked
just the way you expected.
So UI testing is obviously
a huge expansion of the kind
of testing you can do
for your applications.
So when do you use it, right?
We've got unit testing already.
UI testing is a complement to
UI testing, not a replacement.
You should continue to use unit
testing for your model objects,
and your controller logic,
because unit testing will more
precisely pinpoint failures
when they happen in your code.
UI testing allows you to
cover much broader ranges
of functionality, but tracking
down the failures can
be more challenging.
It's a balance in your project,
of finding the right blend
between unit testing
and UI testing.
Some great candidates
for UI testing.
Well, think about your app.
You have customers,
you show it to them,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
You have customers,
you show it to them,
and you have little demo
sequences that you show to them,
and you walk them
through, "well,
here's how you do
this with the app."
Demo sequences are great
candidates for UI testing.
You'll know nightly
build after nightly build
that those demos are
going to work for you.
Second, beyond that,
common workflows.
What the app is used for,
if it's an editing app,
how you edit the documents.
Any custom views you
have in the application.
And finally document-based
workflows, opening and saving.
These are all great
things to automate.
They are really hard to
capture with unit testing.
When they go wrong they
have a huge impact on users.
So UI testing.
New in XCode 7.
Opens up a world
of possibilities
for how you can test
your applications.
In UI testing you
find and interact
with user interface elements
and you synthesize events
that drives them, just
the way a user does.
You can validate the UI
properties and state.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
You can validate the UI
properties and state.
And UI recording lets you
create these tests super quick,
super easy.
Finally, we've overhauled
the test reports
to make you better able to
understand how your tests work
and collect additional
data about them.
So that's UI testing.
[Applause]
>> WIL TURNER: So for more
information we've got great
documentation on XCTest.
You can get to it through Xcode
itself and on our website.
And Accessibility also
has great documentation,
and I encourage you
to check out.
Developer Forums are great
places to raise questions
and trade tips with other users
about how you're using things.
Our evangelist Stefan
Lesser is a great contact
to get you started.
There's some related sessions
if you want to dial back in time
and watch the accessibility
session from yesterday morning,
probably watch the video
on your lunch if you want,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
probably watch the video
on your lunch if you want,
and continuous integration and
code coverage, new technology
in Xcode server, you can
see tomorrow afternoon --
tomorrow morning.
Have great conference,
everybody.
[Applause]