Transcript
[ Music ]
[ Applause ]
>> Good afternoon, everyone.
My name is Zoltan.
Later my colleague, Eric,
will join me on stage.
We're both engineers
with the Xcode team.
Software development
these days is a lot
like conducting an orchestra.
You and I maintain suites
of tests, and when one
of them is not performing well,
we need to quickly understand
the issue and correct it.
Okay. Maybe you're a
testing maestro with hundreds
of thousands of tests.
Well, Xcode has tools
and techniques
to conduct even the
largest test suite,
and today we're going
to show you how.
First, we're going to
introduce some concepts
that haven't been addressed in
previous sessions, and then,
we'll introduce some
new features in Xcode,
in Xcode Server, in particular,
the configurable
integration user.
And then, we'll conclude with
some new features in xcodebuild.
Let's get started.
So, as a brief recap of testing,
you can think of testing
So, as a brief recap of testing,
you can think of testing
as these four characters.
There's XC test.
That's the framework
for your tests
for both Objective-C
and for Swift.
And your tests are compiled
to bundles before they're run.
Xcode, that's the IDE
for authoring your tests,
and you can also run individual
test while you're developing.
Xcode is also where
you review reports
from both local test runs
and from Xcode Server.
And talking about Xcode Server,
that's the continuous
integration solution
for your tests.
You set up bots to
periodically run your tests,
and Xcode Server will
generate reports for you.
And if something goes
wrong, it'll notify you.
There's a great way to keep
track of your project over time.
There's a great way to keep
track of your project over time.
So, Xcode Server is built
on top of xcodebuild,
and you can use xcodebuild
on the command line, too.
You can run tests and see
results in the console.
Xcodebuild is the
building block for custom,
continuous integration
systems, as we'll see later.
So, you can learn more
about these four characters
in our previous sessions.
But, today, we want to
look at some concepts
that we haven't covered before.
We want to take you
behind the scenes
and show you exactly how
your tests are running.
And to do that, it helps
to think of a timeline.
After your tests are
compiled, they must be hosted.
That gets the test
started in the first place.
And then, once the
tests are running,
you can see detailed progress
in your tests using a
technique known as observation.
So, we're going to look
at these two concepts,
hosting and observation.
Let's look at hosting first.
So, the hosting story,
it's different depending
on whether you have
unit tests or UI tests.
So, for unit tests, your test
bundle is loaded directly
into your application.
And, in this case, we
call your application the
host application.
But for your UI tests, your test
bundle is going to get loaded
into a UI test runner which is
separate from your application.
And in this case, we
refer to your application
as the target application.
Now, this has some implications.
For your unit tests, you
will have direct access
to your applications
data structures and API,
to your applications
data structures and API,
but for your UI tests, you have
to access your application using
accessibility and send events
and see your application as a
user would from the outside.
For your unit tests, all
your tests are going to run
in the same launch of
the host application.
So, you should be
careful to clean
up between test invocations.
But for your UI tests,
your tests can terminate
and relaunch the application.
So, that's great if you want
to test how your
application starts up.
Okay. That's hosting.
That's how your test gets
started in the first place,
and once your tests are running,
then you can use a
technique known as observation
to see detailed progress
in your tests.
So, let's zoom in
on that timeline.
So, let's zoom in
on that timeline.
Here you can see two test cases,
one running after the other.
And these test cases
belong to a test suite.
Test suite corresponds to a
test class you've written.
And you'll have multiple test
suites in your test bundle.
And maybe you're interested
in doing some setup work
before any of these tests run.
Or perhaps doing
some tear down work
after all the tests
have finished.
Or maybe you're interested
in doing some custom
logging while the tests run.
Well, you can do that with the
XCTestObservation Protocol.
You write an object that
conforms to this protocol,
and after registering it with
a shared observation center,
your object will
receive call backs.
So, for instance, before
any of the tests start,
you'll get this BundleWillStart
call back.
you'll get this BundleWillStart
call back.
And then, before
the suite starts,
you'll get a SuiteWillStart
call back.
Then, for every test
case that runs,
you'll get this
testCaseWillStart call back
and testCaseDidFinish call back.
If something goes awry,
you'll get this testCaseDidFail
call back.
And then, as the
tests are wrapping up,
you'll get this
testSuiteDidFinish call back,
and your final chance
to do any work is
in this testBundleWillFinish
call back.
So, here's an example.
This object conforms to
XCTestObservation Protocol.
In the object initializer,
I'm going to register
with a shared observation
center.
Then, as the tests progress,
I'm going to log
events, such as this one.
I'm going to log
events, such as this one.
When things go wrong with the
test, I'm going to log this.
And then, once the show is
over, I'll log that, too.
So, an ideal place to
set up this observer is
in your test bundles info.plist.
So, you do that using
Xcode's info.plist editor.
You add this NS principal
class entry,
and this is a test specific
instantiation for your observer.
It's independent of the
class load initializer.
Okay. That's hosting
and observation.
They're useful concepts to help
understand exactly how your
tests are running
behind the scenes
and to help you diagnose
issues with your tests.
Let's look at some new
features in Xcode 8.
Crashes are a frequent
source of failures in tests,
Crashes are a frequent
source of failures in tests,
and the crashes can be in
both your host application
and the target application.
So, normally, Xcode will
relaunch your host application
to complete your test suite.
But it's up to you to gather
the diagnostics necessary
to resolve the crash.
I'm please to say that
this year, Xcode is going
to help with this issue.
We will now gather the crash
logs for you in the test report.
So, this is both for UI
and unit tests, for local
and server runs of your tests.
The crash logs will be
collected in the test report.
And you can see the
textual representation
of the crash there, or you
can choose to see the crash
in the context of your source.
And I'd like to show
you that now.
Eric and I have been
moonlighting.
We've been working on a tvOS
application in our spare time.
So, I'll run the application
by control clicking
on the Run button.
It's an application
defined nearby concerts
with some strangely
test related band names.
I can move in the
simulator using the keys
on the keyboard here.
So, we're adding a new
feature to this application
to support users who have
location services disabled.
Let me show you what it's
like for those users.
I'll go to the settings
menu, and in privacy,
I'll disable location services.
So, now, back in Xcode, we've
added a new ViewController
to support those users.
They can enter in a zip code.
Let me just add one more
test for this ViewController.
So, I want these tests to
run in a scheme on their own.
So, I'm going to go
to the scheme menu,
and I'll duplicate
this existing scheme.
I'll call this something
meaningful.
And, in the settings
for the test action,
I'm going to disable
the debugger.
So, that's a technique to
avoid interrupting the tests.
It allows all the tests
to run to completion
without breaking
into the debugger.
And it's similar to how your
tests run on Xcode Server.
And it's similar to how your
tests run on Xcode Server.
So, I'll disable tests
that are not related
to location services.
I'll share the scheme.
And then, I'll run the
test with Command U.
So, here's a new
ViewController for users
who have location
services disabled.
They can choose to
enter a zip code.
So, right now, the tests
are entering a zip code,
and that's an unexpected crash.
So, right now, Xcode has
gathered the crash log,
and it's reported
the test failure.
and it's reported
the test failure.
And, let's have a look at that.
So, here's the failed
test, and I can click
to jump to the test report.
I'll disclose the
test transcript.
Here's the events as the tests
were entering in the zip code,
and here at the bottom
is the crash.
Now, I can click to see
the textual representation
of this crash, but in this case,
I want to show you the crash
in the context of the source.
So, I'll click on
this arrow here.
So, here's the exact line
that the crash occurred on.
And you can see in the
top left, the stack frames
in the debug navigator.
Now, I can see the crash
in the context of my source
and diagnose the issue.
So, I happen to know
that I'm referring
to the wrong ViewController
here.
Instead of parent, this should
be presenting ViewController.
Instead of parent, this should
be presenting ViewController.
So, I'll make that change,
and now, I'll rerun the test
by going to the test navigator
and clicking on this icon here.
So, earlier, I showed
you the test report
for a local test run, but that
report would look exactly the
same if it came from
Xcode Server.
So, now the tests are
reentering a zip code.
Great. So, these
tests have passed.
We would now check in this
tests and build confidence
that we're supporting users who
have location services disabled.
But, for now, let's
go back to slides.
So, you've seen how Xcode will
now gather crash logs for you
So, you've seen how Xcode will
now gather crash logs for you
and capture them
in the test report.
And you can choose to see
the textual representation
of those crashes, or
you can see the crashes
in the context of your source.
So, that's a great way
to diagnose the issue
and to make the fix right in
the context of your source.
So, we have some new
features in Xcode Server,
and to show you more,
please welcome my colleague,
Eric Dudiak, on stage.
[ Applause ]
>> Good afternoon.
I'm Eric Dudiak, and I'm
going to talk to you a bit
about Xcode Server and what
we have new in Xcode 8.
So, let's go over a
little bit of an overview
of what we're going
to talk about today
in Xcode 8 with Xcode Server.
So, we have custom
environment variables
that you can now
set per integration.
We have advanced
trigger editing workflow
that we've improved in Xcode 8.
We also have some enhancements
to issue tracking and blame
to make sure you get
notified of issues
as they come up on your bot.
And we'll see how that plays
with upgrade integrations,
a new feature we have for you.
And finally, we'll talk
about the configurable
integration user
which is new in Xcode
8, as well.
So, let's jump right
in, and let's talk
about custom environment
variables.
Now, this is a little
bit of a cheat.
It's actually new in Xcode
7.3, and it allows you
to configure the exact
environment that's passed
to xcodebuild on your bot.
And it controls how your
integrations are run there,
so you can configure any number
of settings that you might need
for your bot to run on
your server different
than you might have locally.
This is a great way to customize
how your server runs your
integrations, and it's a great
way to do that without having
to create a lot of extra
schemes in your project.
Now, on to what's
actually new in Xcode 8.
So, we've significantly improved
the trigger editing experience.
So, we have two types of
triggers, scripts and emails.
First, let's talk a little
bit about trigger scripts.
First, let's talk a little
bit about trigger scripts.
These run either
before your integration
or after your integration, and
they're normal shell scripts.
So, we've improved the editor
by giving you a lot more space
to see the scripts, and you
can actually see exactly what's
in them, so you have a lot
more visibility into that.
Along those lines, we also
now let you name triggers.
This is great if
you work on a team.
Your other teammates can see
exactly what each trigger is
supposed to be doing, and if you
have particularly long running
scripts that are part of
your integration triggers,
you will see that
called out in the status
as Xcode's integrating.
So, when you see a project
integrating into status UI,
you can actually see which
script it's currently running.
Finally, if you have a script
that you come up with later
that you actually really wish
was running before all your
other scripts, you can now
reorder triggers in this UI.
Simply add a new script
trigger, drag it to the top,
and it will become the
first to run on your bot.
Now, let's talk a little bit
about the other type
of trigger we have.
We have email notifications.
Historically, in Xcode Server,
this was always a one
email per integration setup
this was always a one
email per integration setup
which can leave you
with a lot of spam.
So, in order to help you and
reduce the amount of email
in your inbox, we've split
these types of triggers
up into two different types.
We still have the report
triggers or report emails
that come out every time
you run an integration.
Of alternatively, you
can now schedule them
to run just once every
day or once every week.
This way, no matter how
often your bot runs,
your email isn't flooded,
or your inbox isn't flooded
with a lot of emails.
And we think this is great
for managers who want to check
up on the health of
a bot continuously
but don't want all their inbox
to be filled with emails.
Additionally, we also, now,
let you configure certain
fields on the email.
So, you can configure
your cc field as well
as your reply to fields.
This lets you have a lot more
control over exactly the types
of emails you're sending out.
Now, I hinted a little bit
earlier that we actually, now,
have two different
types of email triggers.
The other is issues.
So, new issues come up.
We will now send you an email
to the people of interest
We will now send you an email
to the people of interest
for that particular issue.
We'll go over a little
bit of that in a minute.
But, if you do have more
committers that are part
of the code that
you're integrating
than you necessarily
want to email,
we do let you filter
recipients right here
to make sure you're emailing
just the people you want to be.
So, if we take a look at that.
If you have multiple
repositories, you can choose
to only send emails
to committers
from certain repositories,
or if you know exactly
which domains all the
email addresses will be
from that you want
to send emails to,
you can add those right here.
So, let's take a minute and
talk a little bit about issues
or build issues anyways.
And, nobody is perfect and
writes perfect code every time.
That's the whole reason we
have continuous integration.
That's why we have unit tests.
We have unit tests
because we know
that they will inevitably fail.
That's why we write
them in the first place.
We also, occasionally, commit
code that just doesn't build,
and some of us don't
check it before we commit,
and that's exactly
the kind of thing
and that's exactly
the kind of thing
that continuous integration
is great at catching.
And, in these types of cases,
Xcode will send you an email
and notify you that
you broke your build.
But those aren't the only
issues that can come up.
Sometimes, even if you write
absolutely perfect code,
things can change around you.
That can be when you install a
new Xcode, you get a whole bunch
of other new features, too.
You get new SDKs that might
have new deprecations.
You might get new issues
from language improvements.
And, we like to make
sure that every Xcode
that we ship is smarter than
the Xcode we shipped before.
So, you might see issues
that we didn't find before
that have always been
there that Xcode now tracks
such as static analysis issues.
So, the important
thing when you see one
of these emails is
we really want
to make sure it's
actionable, and let you know
that you're receiving this
email, for the issue emails,
when it is because
something happened
that you can do something about.
So, the first type is if
you introduced an issue.
And that's because you broke it.
So, this will call you out,
as you see right in the email.
You'll see an email like this
that says you introduced
an issue.
that says you introduced
an issue.
And we know this because
the issue showed up on
or near a line that you
just recently modified.
We also might know if,
between the two integrations
where the issue showed up, you
were the only person committing.
In that case, that's
pretty much a fair guess
that it's you that broke it.
Now, that's not always the case.
Take, for example, a
application that's built
on top of a framework.
A change in the framework
might cause breakage
in the application
without anyone committing
to the application.
So, in this case, you'll see
an email a little bit more
like this, and it'll be a little
bit less condemning of your work
and will simply say
that you might be able
to help fix an issue on the bot.
And we know this because you
commit frequently to the area
where the issue showed up.
So, we, essentially, assign
ownership to various areas.
And, keep in mind, that this
is a bit fuzzier matching
than when we're directly
blaming someone.
So, expect this email to go out
to a bit of a wider audience.
It casts a very wide net
when trying to figure
out who might be of interest
for a particular issue.
Now, that's great for issues
that come up in your code,
but sometimes, you can
actually get issues that come
up regardless of code changes.
One of the easiest ways
to see that is actually
by reconfiguring your bot.
So, Xcode 8 or Xcode Server in
Xcode 8 will now track changes
to your bot configuration
and actually call those out.
Whenever possible, we will
attribute any new issues
that come up to changes
in the configuration,
such as if you enable testing
or enable static analysis.
Those types of issues may have
been in your code forever,
but we can actually attribute
them specifically to changes
in the configuration
of the bot rather
than a change in your code.
We also make sure to include
this information in emails
for the next integration,
so that when you do see one
of these emails, you know that
some of these changes might be
because you're picking up
a configuration change.
Let's go back to something
we were talking about earlier
of installing a new
Xcode and getting a bunch
of new features in the process.
Well, when that happens,
Xcode Server
on Xcode 8 will actually
reintegrate your entire project,
on Xcode 8 will actually
reintegrate your entire project,
and we call this an
upgrade integration.
We take the exact same revision
of the previous integration
of all your repositories and
simply rerun all your tests,
rebuild everything, rerun
the static analyzer,
and when this happens, we
know that the issues that come
up in your project at this
point, any new issues,
are specifically
because of the upgrade
since we took the exact
same commits as before.
This saves you a ton of time
trying to track down changes
in your source code
that just aren't there
because the issues came up
from changes around your code.
So, that's some of the new
features and issues and blame,
but I want to talk a little bit
about my favorite new feature
in Xcode Server and
Xcode 8 which is our new,
configurable integration user.
So, we now give you full control
of the macOS user that's
running your integrations.
And this gives you a lot
of improved visibility
into how your integrations
are being run,
and more to the point,
it allows you
to configure exactly how
your integrations run.
Historically, in Xcode Server,
there's a hidden macOS user
that was running all of your
integrations in the background.
that was running all of your
integrations in the background.
You wouldn't have
access to the password,
and you wouldn't be
able to log in as them.
This meant that you were
basically getting a stock user
no matter what, and you
couldn't make any changes.
Now, you'll own and
manage this user yourself.
You will be given the password.
This user will be a
completely normal macOS user.
It'll be any user on the system.
Can be anyone you want.
We suggest a new user.
But it will available
at login window,
and it's fast user switching.
So, you can log in to that user.
You will notice you're
running as that user
through a menu extra that
shows the Xcode Server
integration icon.
We'll see that in just a minute.
Let's go over how you
would set up this user.
So, here I've opened the server
app to the Xcode service pane,
and in order to enable
the Xcode Server,
I need to go ahead
and choose an Xcode.
So, I'll choose a new Xcode
8 that I have installed,
and I'll be presented with
this dialog, asking me to set
up which integration user I
want to use, and I'm going
to go ahead and create
a new user just
for running integrations.
I'll go ahead and give it a name
and a password, normal
user stuff.
And I'm going to leave
it as a basic user.
And when I push "create user,"
Xcode's going to go ahead
and do a little bit of work
in the background while
it gets everything ready
to run integrations, and once
that user's mostly set up,
we're going to be
asked to login as them.
Now, we are asked to log in
as them because we're going
to go ahead and run a little
bit of the setup assistant.
Remember, it is a real,
normal macOS user.
So I can log in, I can sign
into a test iCloud
account, for example.
I can stage any data that I want
to have for my integrations.
Anything like that.
When Xcode's ready
to run integrations,
you will see a notification
like the one you see
in the top right here, and in
the top right in the menu bar,
you'll notice that there is
a little hammer indicating
that this is the Xcode
Server integration user.
With that all done, I can
go ahead and switch back,
and I'll be back in server app,
and I'll have the indication
that the user is ready
to go, logged in,
and integrations
can begin running.
So, now, we've seen how
to set up that user.
Let's take a look at some of
the things that we can do now
that we have access to
the integration user.
And for that, let's go
to the demo machine.
All right.
Here we have that same
project we saw earlier,
and we saw an issue
where it was crashing
because it didn't have
access to location.
And we had to go in
manually to the simulator
and turn off location
access to test it locally.
But, as we're developing
our application,
we might not want to do that.
We probably want to leave
location services turned
on on our Apple TVs
and on our simulators,
so that we're running,
more or less, the way most
of our users will see it and
how we want to see the app.
But, now that we've fixed
the issue we saw earlier,
we definitely don't want
it to come up again,
and we want to be
notified if it does.
So, it just so happens that
this machine I'm actually
on right now, happens
to be my server machine.
So, I don't have to go
anywhere to go to my server.
I can actually just go to
the fast user switching menu,
and I see that have a
configurable, continue,
integration user
called Xcode Server.
And I'm just going to
go ahead and select that
and enter my super-secret
password that's only four
and enter my super-secret
password that's only four
characters long.
And we will log in as this user.
So, I've gone ahead and
changed the desktop background
so that I know that it's
the build service user.
And, here we see the hammer
icon of Xcode that indicates
that this is Xcode Server.
We see it's on and
waiting for integrations.
So, it's all configured,
but I want to go ahead
and go into Xcode.
So, I can actually configure any
settings I want in Xcode locally
on this user, and they will be
picked up in my integrations.
So, I'm going to use
a neat little trick.
If I go to the devices
menu, I see all the devices
and simulators that are plugged
into this particular machine.
Now, being generous at
Apple, we give you one
of every device we've ever
made, but if that's not enough
for you, for absolutely free,
you can get more simulators just
by clicking this
Plus [+] button.
Now, the Apple TV 1080p
simulator's the built in one
that I got when I
installed Xcode.
I've gone ahead and created
another one called Apple TV no
location, and I've
actually got it booted here.
And if I go in, I can double
check, and it is, indeed,
And if I go in, I can double
check, and it is, indeed,
has location services
turned off.
So, I go to settings,
general, privacy.
Location services is turned off.
So, this simulator is
all configured for use,
and my other one's still usable
for when I want location
services turned on.
Now, let's do a quick
switch back,
and just log back
into my normal user.
Normally, I'd be
walking across the room
to my actual work machine,
so this is a little faster.
And here we see all
the UI tests.
Now, I already have a bot that's
integrating this normally,
but I want to go
ahead and create a bot
that will integrate
just the case
where we don't have
location data.
So, I'm going to go
product, create bot,
and use the Harmony no
location scheme on this server.
Going to go ahead and
give it location, sorry,
access to my repository so
it can check out the project.
I'm going to disable
the archive action
because this will be the
same as my normal bot.
So, I don't care about that,
but I do want to run tests.
And, let's leave
static analysis on.
And I want to be notified
as soon as I break this,
but probably once a
day is good enough.
but probably once a
day is good enough.
So, I'll know within 24 hours
if we ever break this again,
our UI test will catch
it on our server.
So, we'll just run it every
day at 1:00 AM, and here I get
to select the devices.
By default, every integration
runs on all devices of that OS.
So, this is a tvOS
project, so it'll run
on all tvOS devices
and simulators.
Instead, let's do specific TV
devices, and I'm just going
to select the Apple
TV no location.
This is that environment
variable view we talked
about earlier.
I don't have any
environment variables I need.
I'm going to go ahead and
make sure that I get emailed
when a new issue comes up
in this particular case,
and go ahead and create the bot.
So, that is going to go ahead
and kick off an integration
which we see running here.
It's checking out building.
We've already seen the
UI test run a little bit,
so let's actually
look at an integration
that I prebaked a
little bit earlier.
And if we go to the test,
we can see that it did run all
our tests including this zip
code test.
And we can see, if I look at
some of the screenshots from it,
it is, indeed, running without
location because it's opening
that ViewController
that we saw earlier.
that ViewController
that we saw earlier.
So, we saw how you were able
to get significantly
improved visibility
into how your user runs.
You can log in as them and see
everything that's happening.
We saw how you could
customize different settings
such as simulators to run
exactly the operations
that you want to run
on your integrations.
We saw this was a
completely normal macOS user
that I was able to switch
to in fast user switching
and that I had the password to.
And, we got to see
that menu extra
that shows you a little bit
of your integration status
in that integration user.
Now, with this great new
power comes a little bit
of responsibility, and so
we have some best practices
for you.
First, we highly recommend
you dedicate a new user.
You want this to be as similar
to your customer's experience
as possible, so you don't
want all your settings
to be causing impact
on your bots.
We also recommend you avoid
administrator accounts.
Keep in mind that anyone
who can create or edit bots
on your serve will have
access to this account
through triggers, so
this goes the same
for any private or
customer data.
Try to avoid storing
that in this user.
Try to avoid storing
that in this user.
If you want to stay logged in
as this user in the background,
use fast user switching.
Integrations can continue to run
in the background just
as they always have.
However, if you do want
to run the integrations
as the front most user,
make sure that you
turn off screen lock.
Just like iOS screen
lock blocks testing,
so does macOS screen lock.
And finally, make
sure to customize this
for whatever needs you have.
That includes, as we saw
earlier, any simulators.
If you have any specific
networking configuration
that you need for
your integrations,
any stage user data or settings
that you want for UI tests,
go ahead and configure those.
And finally, if you have
any advanced provisioning,
such as if your administrator
gives you a provisioning
profile, however you
would configure on Xcode,
you can actually configure
that in your configurable
integrations user, now,
and make sure that code
signing will work for you.
So, with that, I'm going to
bring Zoltan back to talk
about some new features
in xcodebuild.
[ Applause ]
>> Thank you, Eric.
So, you've seen new features
in Xcode and Xcode Server,
and now we have new features
to show you in xcodebuild.
xcodebuild has the test action
for custom continuous
integration systems.
You give it a workspace, a
scheme, and a destination,
and xcodebuild will
dutifully build your sources.
It will install built products
onto devices as needed.
It will run your tests,
and then report results
to you on the command line.
So, this year, we're introducing
two new options for this action.
With the only testing option,
you can effectively
constrain the set of tests
that the action will run.
So, you can specify TestCases or
TestSuites or TestBundles to run
to the exclusion of other tests.
And you can also use
the skip-testing option
to specify TestCases to exclude
from the tests while
running everything else.
And the big news this year is
that we're introducing two
new actions to xcodebuild.
So, we're effectively splitting
the test action in two.
These actions are already
available in the Xcode IDE,
but we're going to bring
them to xcodebuild, now.
And so, we'll look
at each in turn.
Build for testing is just
the building portions
of the test action.
So, you give it a workspace,
a scheme, and a destination
as before, and it will
build your sources,
perhaps making certain
symbols visible,
and then it will output the
built products to derive data.
It also produces
this xctestrun file.
It's a kind of manifest
for everything your
test needs to run.
for everything your
test needs to run.
We'll come back to
that in a minute.
Test without building then, is
the second part to the story.
You give it a workspace,
a scheme,
and a destination, as before.
And, xcodebuild will find the
build products in derived data.
It will install those
onto devices as needed,
and then run your tests and
report results just as before.
But the cool thing
is, you no longer need
to provide a workspace.
Instead, you can just
provide this xctestrun file,
and Xcode will run your
tests from binaries alone.
So, it will ingest
the xctestrun file.
It will find binary products
relative to that file,
and then it will run the
tests and report results
in exactly the same
way as before
in exactly the same
way as before
but without having any
access to your sources.
So, this is ideal
for distributed testing
environments.
You would build your tests
on machines up to optimized
for building, and then
move those built products
onto machines optimized
for testing.
And then, in parallel, you
can gather the test reports.
So, this is all made possible
by the xctestrun file,
that manifest for your tests
that specifies which tests
to run and which to skip
and which test machines.
It provides environment
variables
and command line
arguments to your tests,
and you can read
more about the format
in our man pages
using this command.
[ Applause ]
So, we covered a
lot of ground today.
We first reviewed testing in
terms of these four characters,
and then we looked at some
new concepts in Xcode.
They were great for diagnosing
issues with your tests,
and then we introduced
new features in Xcode,
in Xcode Server,
and in xcodebuild.
I hope these insights and
these new features help you
to develop test suites which
perform well, and, after all,
that will help you
deliver great applications.
You can read more about
this session at this URL.
Thank you.
[ Applause ]