WWDC2014 Session 415

Transcript

X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
>> Good afternoon everyone.
Welcome to our session:
Continuous Integration
with Xcode 6.
Thanks for being
here this afternoon.
My name is Brent Shank.
I'm a software engineer
on the Xcode team,
and today we're going to
talk about my favorite topic
which is, of course,
continuous integration.
So just a show of hands, how
many people are using some kind
of continuous integration
system out there?
All right.
Awesome! Okay, next question.
How many people are
using Xcode Server?
Okay. Okay.
Good. Well, we hope next year,
when we ask that question,
it's going to be everyone.
So if you're not familiar
with continuous integration,
we're going to teach
you all about it today.
We're going to talk to you
about our continuous
integration product,
which we call Xcode Server.
We're going to tell you
about all the great new things
we built into Xcode Server.
You probably want
to see that slide.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
You probably want
to see that slide.
So we have lots of great new
features in Xcode Server,
in Xcode 6, and we're going
to give you lots of demos,
because it really shows
well on the screen.
So, first, if you're
not familiar
with continuous integration,
what is it all about?
Well, fundamentally it's all
about improving software
quality,
and there are a lot
of ways it does that.
A continuous integration system
that catches problems
quickly and automatically.
So you don't have to worry
that when you make code changes
at the end of the
day and you leave
and you introduce the build
error or warning, you don't have
to worry about that not being
caught until the next morning,
because a system like
this is going to catch it
for you quickly and
automatically,
and for me that really
enhances collaboration
because you don't have to worry
about blocking your team
members, and let's face it,
when you're writing code,
you're taking a lot of risks.
You know, you're doing
interesting and creative things.
You're going to introduce
problems and a system
like this helps catch
those problems quickly.
Another key thing
is that a system
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Another key thing
is that a system
like this helps broaden
your test coverage.
Because you have your
code running on a server,
the server can support lots of
different testing configurations
that would be a big
pain to do on a client.
So another great thing about a
server product that's building
your code is that it
can gather histories.
So build history and test
history for your product
over time, and this is
actually really interesting.
Because at the beginning of
a product development cycle,
as I'm sure you know,
there's a lot of change.
There's a lot of churn,
and you're probably
introducing a lot of issues.
You're introducing warnings
and static analysis issues,
and you probably
don't have a whole lot
of tests at the beginning.
Towards the end of that
cycle, you don't want
to have any issues, and you want
a lot of tests, and a system
like this will help gather that
history and show it to you.
So it's a great indication
of where you are kind
of in your product development
cycle, and of course a system
like this makes it
really easy for you
to distribute builds
to your team.
So hopefully you have testers.
I hope everyone does,
and if you do,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
I hope everyone does,
and if you do,
a system like this makes it easy
to distribute those
kind of builds.
So terminology, bear
with me here.
So there are three terms that
we're going to talk about over
and over again in
relation to Xcode Server.
Term number one is a scheme.
So I won't quiz anyone, but
I'm sure if you've used Xcode
for a while, you've seen this
term, and it's really simple.
All a scheme is it's
just a recipe
for building your project.
It contains the targets that
you want built, and the target
in turn contains the source
files and the resources
and everything else that
make up your product.
So why is this important
for Xcode Server?
It's important because a
scheme needs to be shared.
This is not a big deal.
Xcode makes it really
easy to share schemes,
but when you first
create a scheme,
or when Xcode creates
a scheme for you,
that scheme isn't shared.
We'll show you how to do it.
It's not a big deal; just
something to keep in mind.
So term number two is bot.
And this is our friend the bot,
and it's just a thing that we --
the word we use to
refer to the thing
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
the word we use to
refer to the thing
that builds your
software on a schedule.
And when the bot runs, we
say that it integrates.
It makes an integration.
So these are the
three key terms:
scheme, bot and integration.
So let's take a look,
in a little more depth,
at our friend the bot.
So what defines a bot?
So fundamentally there are
two key pieces of information.
One is the project you
want the bot to build.
That's pretty basic.
The second is the
SCM information,
or the source control
information.
So where does your project live?
So if your project lives
in a Git repository
or a Subversion repository
and if Xcode can check
out your project, then you can
create a bot for your project.
That's really simple.
The next part about a
bot is the schedule.
So when do you want
the bot to integrate?
There are a lot of
possible options here.
You can choose to schedule
your bot periodically.
So, every hour or once a day or
even once a week if you want.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So, every hour or once a day or
even once a week if you want.
There's also on-commit bots,
and that's a very popular option
because it means that when ever
anyone commits source control
to your repository, the bot
will fire off an integration,
and of course there's also
the manual scheduling too.
So, how to build it.
So this is where the shared
scheme comes into place.
The shared scheme tells the
bot how to build your project,
and you can also configure
three other options.
So do you want the bot
to run static analysis?
Do you want it to run tests,
and do you want it
to build archives?
Finally, a bot wouldn't
be a whole lot of good
if it didn't tell you
when something went wrong.
So we have a great
system for notifications,
and you can set these up to
fire before a bot integrates
and after a bot integrates
so that you
and your team know the
results of each integration.
So like I said,
we call our continuous
integration product Xcode
Server, and there are two parts
of Xcode Server, actually.
There's the Xcode part,
the part that lives
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
There's the Xcode part,
the part that lives
in the Xcode application, and
then there's the part that lives
in the OS X Server application.
This is important to know.
If you try to set this
up with just Xcode,
you're not going
to get very far.
You also need to
get OS X Server.
It's easy to get.
There's a new preview
of that available
on the developer website this
week, and when it gets released
to the public, it's also
available on the App Store.
So there are two parts to it.
Some of its features is that
it integrates very easily
with the Apple Developer
Program,
and this is really important
because in order to run any
of your code on devices, we need
to do all their provisioning
and certificate stuff,
and that needs
to know your Apple
Developer information
in order to do all of that.
That's part of the product.
We make it really easy
to get started with that,
and of course Xcode Server
builds any iOS and Mac projects.
If Xcode can build it,
Xcode Server can build it.
And one of the really great
features about Xcode Server is
that it can run your tests on
multiple devices and simulators.
So think about it.
Before we had this product,
if you wanted to test
on a couple iPhone's
and a couple iPads
and a couple iPod Touches, you
would have to plug each one
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and a couple iPod Touches, you
would have to plug each one
of these into your
development system separately
and run the tests on them,
and it was a big pain.
But now you can plug all those
devices into an Xcode Server.
You can configure
your bot to run tests
on all those devices,
and it's great.
You have this really broad
test coverage I was talking
about before.
Of course, Xcode Server can
produce IPA and PKG files
so you can more easily
distribute these products,
and there's great
integration in Xcode.
It looks pretty good too.
So, like I said, OS X Server is
the other part of this equation.
Setup is really easy.
It's got great features
like hosted repositories,
and this is where you configure
the devices that you want
to make available for
testing your software.
We'll show you a demo of
that in just a minute.
So last year in Xcode 5,
we introduced Xcode Server
for the first time,
and we were really,
really proud of its feature set.
When we took a look at the
product and we took a look
at what features we wanted
to build in this release
and future releases, it
was pretty clear to us
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and future releases, it
was pretty clear to us
that there were parts
of the foundation
that we really needed
to improve.
So we're happy to say we
have a shiny, new foundation
for Xcode Server and Xcode 6.
It's much better in all ways.
It's faster.
It's more reliable.
It's super scalable, and it's
allowed us to add support
for all these great features
like support for Swift,
support for performance testing,
a new feature we
call issue tracking
and probably the most
requested feature
from the first version
was triggers.
[Applause]
Thank you.
Just wait.
It's like I said, we have
a great new foundation.
Super reliable, and it's
really scalable too.
So I decided to do a
little test a few weeks ago.
I took an old Mac Pro that
was sitting in my office.
It's a five-year-old Mac Pro.
It wasn't doing anything.
It was a nice footrest for
me, and I set up Xcode Server
on Xcode 6 on it, and I
decided to add a few bots.
So I started with a small app.
I started with the WWDC app that
you have today in your devices.
I created a bot for that.
That went pretty well.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
That went pretty well.
Then I decided to
just keep going.
So I added a bot for Xcode.
That went really well.
I added a bot for Game Center.
That went pretty well.
Again, these are bots all
running on this one old server.
I added a bot for the
maps project, for UIKit,
and that was all going well so
I threw WebKit in there too.
This is a very old
piece of hardware.
It's building bots for all
these projects, both small,
medium and really big,
and it's doing just great.
So this is something
that just wasn't possible
with the first version
of Xcode Server.
Triggers. Okay, so we heard you.
You asked for this
loud and clear,
and we have a really
great story for triggers.
So we support triggers, both
before an integration starts
and after an integration
runs, and there are lots
of interesting reasons for
supporting both of these.
There are a number of conditions
you can specify that determine
when a trigger runs, and there
are two types of triggers.
There's a shell script
trigger and an email trigger.
So what's really cool about
the shell script triggers is
that you can do anything
you want to do.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
that you can do anything
you want to do.
Right? It's just a shell
script, and we provide a number
of environment variables
that you can leverage
on those shell scripts, and
we'll see a few of those
in the demo in just
a little bit.
The email reports that we
provide, you can configure them
to show you just the minimum
amount of information,
which is just like the
build errors and warnings,
or you can customize those
and make them really rich
and add all the issue
tracking that we do.
So speaking of issue
tracking, what is this?
Well, what's an issue?
An issue is either a build
warning, a build error,
a static analysis issue
or a test failure,
and issues are important.
You don't want to ship
software with issues,
and in fact we have a
policy, a really good policy,
on the Xcode team, which is
if there's a build warning
or static analysis issue in
the source space, that's a P1,
and that's what we work on.
It's important.
We can't ship with these
things, and you shouldn't ship
with them either, and by having
a great issue tracking service
in Xcode Server, it makes
it really easy to know
when those issues
get introduced.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
when those issues
get introduced.
And we can do really cool
things, like we can track
that issue all throughout
its lifecycle.
So if the issue still exists,
we mark that as an
unresolved issue,
and then when an issue
finally gets resolved,
we service that in
the reports too,
and because Xcode Server tracks
all of this history over time,
it's really great, easy to see
when an issue was introduced
and even who introduced it,
where we can be pretty
smart about that.
So there's a lot of stuff there,
but to really bring
the point home,
I think it's better
to show it in a demo.
So I'd like to bring up Matt
number 1, who's going to talk
about all these things.
[ Applause ]
>> All right.
Thank you so much, Brent and
as you heard, we've got a lot
of really great new features
in Xcode Server and Xcode 6.
But in case you've never used
it before, we're going to start
from the very beginning,
which begins in Server.
So we have OS X Server
installed on this machine,
which you can get from
developer.apple.com.
You need the preview to
work with Xcode 6 beta
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
You need the preview to
work with Xcode 6 beta
that you all got this week,
and we've already set
it up on this machine.
You can see our server
is called Bobert,
and it's ready to
start integrating.
So what we're going to do
is set up Xcode Server.
So we select Xcode
on the left-hand side
in the Services list, and the
first thing that we ask you
to do is choose which version of
Xcode you're going to be using.
So you do need Xcode installed
on your server machine in order
for Xcode Server to work,
because that's what actually
does all the building.
But we give you the choice
of which Xcode you use
in case you want to try
out a new version of Xcode
or in case you need to support
multiple versions of the iOS SDK
or something like that.
So we're going to select the
Xcode that you all have a copy
of right now, and
when I choose that,
it's going to start
setting up our database.
It's going to generate client
SSL certificates, and it's going
to start all of our
processes so that we're ready
to start doing integrations
with Xcode Server,
and after a few seconds, that
sets up and it's ready to go.
Now, if you're doing
Mac development only,
that's all you need to do.
You can stop right there.
All set. Good to go.
But because we're going
to be testing on devices
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
But because we're going
to be testing on devices
and everything that we do on
iOS needs to be code signed,
we also need a way for Xcode
Server to be able to do
that code signing, and
for that we're going
to join an Apple
Developer connection team.
So to do that, next
to the developer team's
label there's an Add button.
So we'll choose that, and then
you sign in with your Apple ID.
Once you've signed in, that's
going to give you the option
to become a member of
your developer team.
So when I click Add,
what it's actually going
to do is add your
server as a new person
on your Apple Developer team.
It gets its own set of
certificates, private keys,
signing identities, and it's
also going to download a copy
of all the provisioning
profiles that work with the apps
that you have registered for
app IDs on your developer team,
and this can take a few
seconds because it does go out
and make a bunch of
certificate sign and requests
and then pull those all in.
But once it finishes, you'll
see that you are now a member
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
But once it finishes, you'll
see that you are now a member
of the developer team
that you signed up for.
Now, since we're going
to test on devices,
I do have an iPad
mini right here.
You can all see it.
It's real, and what I'm going
to do is plug it in right
into my server machine,
and after just a second,
in the Devices list
that shows up.
So you can see it's ready
to use, and it shows
which teams it's enabled for.
If this device had
never been used before,
you also get an option
right here to choose Use
for Development, and that goes
out to the portal and adds it
to your developer team.
So really easy to do.
From this screen you also
have the option of configuring
who can use your server.
So by default, I'll click
this Edit button here.
You can see that bots
can be created by anyone
who can log in, and anyone can
view those bots once they've
been created, and
we want to lock it
down just a little bit more.
So I'm going to choose only
some users can create bots.
Administrators is fine for me,
but I'll turn view only
access for everyone back on.
So that's pretty good for me.
I think that's a good set of
access controls for my server.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
I think that's a good set of
access controls for my server.
From OS X Server you can also
set up hosted repositories.
If you'd like Xcode Server
to host your repositories
for you, you can do that here.
You can also set that up
directly from within Xcode.
But that's all we need
from Server for right now,
so I'll close Server,
and we'll head back
over into Xcode itself.
So once you've launched Xcode,
the first thing you're going
to need to do is set up an
account for your server,
and that happens
in the preferences window
on the Accounts tab.
You can click the plus button
in the bottom left-hand side
and choose Add Server, and
you can see my server, Bobert,
is going to show up right here.
So I'll select Bobert,
choose Next and log
in with my user name
and super-secret
password, and there we go.
All set. One of the new features
that we've added in Xcode 6,
which we think is pretty
useful, is down at the bottom
of the Accounts window, we do
show you which version of OS X,
OS X Server and Xcode your
server machine is using.
So if you've got
multiple servers
and something starts
going a little funky,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and something starts
going a little funky,
you can look right here
and it will tell you what your
server machine is running.
So that's not a bad place
to start when you're trying
to diagnose a problem.
So that's all set,
and we're going
to close the accounts
window, and we're going
to open our project, and it's
a project called CoffeeBoard.
So on our team from time
to time we make mistakes.
We're only human.
I don't make that many mistakes,
but some people on our team do,
and when that happens,
if a mistake happens
that causes someone
to spend more
than half a day fixing
a problem,
we think retribution
is in order.
So we've written an app that
keeps track of when that happens
and assigns debts of coffee
and ice cream to the members
of our team, and we
call it CoffeeBoard.
So that's this app right here.
The other Matt's going to show
it to you a little bit later,
but you can see it's got
a lot of tests all set up.
It is written in Swift,
which is pretty cool.
So we're going to create
a bot for that project.
I'll switch over into the report
navigator, and I'll choose
from the Product
menu Create Bot.
Now, the first choice you get is
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Now, the first choice you get is
which scheme your project
is going to integrate with.
I'll use the iOS one.
You get to choose what the
name of your project --
what the name of your
bot is going to be.
So I'll choose CoffeeBoard.
You also choose which server
your bot gets created on
and whether or not it integrates
immediately after creation.
So I'll leave that set.
Now, one of the issues
that some people ran
into in the first
iteration of Xcode Server was
that you would go through the
process of creating a bot,
it would all look
like it was all fine
and then your bot would go
to run its first integration,
and it wouldn't be able
to check out your project,
which wasn't fun because that
wasn't all that easy to fix.
So one of the new features we've
added in Xcode 6 is the concept
of source control preflighting.
So what's going to happen when
I click Next is it's going
to scan your entire workspace,
find all of the source
control working copies
that it knows about, gather up
the credentials that Xcode knows
about that goes with those
repositories, package them
up in a nice, secure,
little bundle and send them
up to the server so that the
server can determine right now
if there's going to be
a problem when it's time
to check out the repository.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
to check out the repository.
So I'll click Next.
Yeah. It's a cool feature.
[ Applause ]
So I'll click Next.
It tells us that all of the
repositories that it found.
The CoffeeBoard repository
it couldn't authenticate,
and that's because I
was recently a victim
of identity theft so
I changed my password.
So what I can do now is
correct this issue right away.
So I'll click the Change button.
Enter my username
and my new password.
Click Next, and this time
it says my credentials
are verified.
So it knows that when it
goes to check out my bot
for the first time,
it's going to work,
which is a really
comforting feeling.
So now we have an option
to choose how frequently we want
our bot to run, which is great.
By default it's going to be a
periodic bot that runs hourly
on the hour, but
because we want this
to be a little bit more
alive for our demo,
we're going to choose On-Commit
so that every time we
make a change to the bot,
it's going to kick
off a new integration.
You also can choose which
actions your bot will run,
when it integrates and also
how frequently it cleans.
By default we always
do incremental builds,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
By default we always
do incremental builds,
but because I really
like squeaky clean bots,
I'm going to choose Always.
Now, depending on which
scheme you select,
if your scheme can build targets
that have products that run
on iOS or on Mac,
you have a choice
of where your targets
will run for testing.
So since we're doing an iOS
target, we are going to use iOS.
But we do have an
option of specifically
which devices our
test will run on.
So I'll drill down into
that and choose the iPad
that I connected earlier on.
And then you get to the option
to configure your triggers,
and I'm going to talk more
about that in just a minute.
But we will show you that by
default, we create a trigger
that will send an
email notification
when something goes wrong,
and you can choose if you'd
like that email to be sent
to all of your committers,
and you can also build another
list of email addresses if you'd
like to send it to more people.
You can choose what goes
into that email message,
whether you want Issue and
analysis issue summaries,
the Commit messages, if
you want to hide those
because they might contain
sensitive information.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
because they might contain
sensitive information.
Or you can turn both off if
you just want people to know
that an integration
is completed,
and you also can choose when
that email will get sent,
whether it's on success,
test failures, build errors
or any combination of the three.
So we'll click Create.
Our bot will get created,
and after a few seconds
it will start integrating,
and you can see another
great new feature we have
in Xcode 6 is live
progress information right
down to the source file that's
being compiled by your bot.
So you can really
keep close tabs
on exactly what's happening
while it's integrating,
which is really cool.
So this will take a
few seconds to compile,
and once that's done it will
start running tests on the iPad
that we have connected
right here.
I know you can't see it,
but it's unlocked the iPad.
The app is launching,
and the tests have run,
and you can see we
have the results
for our integration
that has completed.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
for our integration
that has completed.
You can see that because
multiple people have committed
to the repository
for this integration,
we have a nice breakdown
of the commits,
and we have these nice little
circles which indicate relative
to each committer how much
of that integration
they own, so to speak.
It's the number of files
that have been changed,
and if I click on a committer, I
can see which commits were made
by those committers,
which is pretty cool.
But because a single
integration isn't that exciting,
we're going to switch
over to our other machine
which has a bot that's
been baking for a few days.
So you can see this bot has
been running for quite a while.
I've gotten taller,
if you've noticed.
This bot has been
running for quite a while,
and it's got a much more
interesting history,
which is a great way for us to
show off another new feature
that we're really excited about,
which is our issue
tracking feature.
So if I select integration 4,
you can see that it shows we
have no errors, one warning,
one analysis issue and six
tests ran successfully.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
one analysis issue and six
tests ran successfully.
But these warnings and issues
are new in this integration,
and you can see that Xcode
Server's smart enough
to notice that, because it
puts a plus 1 under each
of those circles so that we know
that these were just recently
introduced, and you can see here
in the issue summary it
also says New Issues:
1 warning, 1 analysis issue.
Well, I came along and
fixed that warning.
So in integration 5, you can see
that it says Resolved Issues,
Warning: Unused variable,
and it knows,
based on the file location of
these issues and the message
that the issues generated, that
this is exactly the same issue,
and we can safely say that
this issue has been resolved.
So that's a really
great way to keep an eye
on the health of
your source space.
Really, really cool.
We also have here logs
just like you would see
for local projects integrated
in Xcode, and you can even drill
down into them and see
the same kind of view
that you would expect
from looking at these
from a local product,
which is really cool.
But now I want to show you
probably what was the most
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
But now I want to show you
probably what was the most
requested feature
from the first version
of Xcode Server,
which is triggers.
Triggers are really cool,
and we're really excited
to show you what they can do.
So we'll start by
talking about how we work
with them on our team.
We have a friend.
His name is Jeeves, and he sits
in our messages chat room
all day long answering useful
questions for us like,
what's the weather like?
For instance, you can see
in Cupertino it's 77
degrees and partly cloudy.
Of course, here in
the beautiful city
of San Francisco it is a
good 12 degrees colder,
which is to be expected.
But Jeeves is really helpful.
We really like having him
around, and it's very helpful
to our team to be able to
get that kind of information
on the fly, but we want
to supercharge this
with information from Xcode
Server, and the best way to do
that is with triggers.
So we'll switch over into
Xcode, and I'll put that up
so you can see it side-by-side.
I'm going to edit the bot, and
most of these settings I'm going
to leave the same, because
we're not really changing much
about the bot, and you'll
see as I click Next,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
about the bot, and you'll
see as I click Next,
it will revalidate those
credentials that I set
up earlier to make sure nothing
is broken in the meantime.
We'll leave all of that
information the same,
but when we get to the triggers
screen, I'll choose Add Trigger
and select Run Script.
Now, I've already got some
triggers written down here.
This is a ruby script.
They're shell scripts
by default,
but you can put a little
shebang at the front here
and run using whatever
interpreter you'd like,
and all this does is it hits
a http endpoint that we expose
on Jeeves, and it uses things
like the integration number
and the bot name
to build a message
that will show in our chat room.
So I'll select all of that
and paste it into a trigger,
and I'm going to do the same
for a post-integration trigger,
and these are just a
little bit different.
You can see I actually use
the failure count, error count
and warning count so that we can
provide even more information
in our chat room, and for
post-integration triggers you
get a choice of under what
conditions your trigger will
run, whether it's
only on success,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
run, whether it's
only on success,
only when something goes wrong
or any combination of those.
So I've created the triggers.
I'll choose Update, and that
will kick off a new integration
of our bot.
So we'll pull up
the status here,
and you can see we get a
message in our chat room
that integration 14 on
CoffeeBoard is now starting,
and that message ran after the
source code checkout happened.
So that way if you'd
like to, for instance,
inject versioning information
into your built product,
you can do that right from
a pre-integration trigger,
which we think is really cool.
You can see the integration
completed, and we got
yet another information
message in our chat room,
and because we wanted to take
it just one little step further,
we added another little feature
where we can say, Jeeves --
oh, not weather -- Jeeves,
integrate CoffeeBoard,
and you can see it's kicking off
another integration in Xcode,
which we think is pretty cool.
[ Applause ]
So that's just one
of the many things
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So that's just one
of the many things
that you can do with triggers.
We're sure you have ideas about
how you want to integrate them
into your own workflow.
We have a lab right
after this session.
Come see us.
Come talk to us, and we
can help you make sure
that you can make triggers
do whatever you would like.
Thanks so much.
Brent, back to you.
>> So, of course, the big
news this week, Swift support.
So how does Xcode
Server deal with Swift?
Well, it deals with it just
like it deals with Objective-C.
It does complete feature parity
with Objective-C projects.
So whether you have
a pure Swift project,
whether you have a mixed
Swift and Objective-C project,
Xcode Server can integrate bots
of those projects no problem.
All the features work the same.
This is true of testing support.
So whether you write unit tests
in Objective-C or you write them
in Swift, all the testing
supports just works.
The same goes for
performance testing,
and Xcode Server has always
had great Assistant Editor
integration in Xcode.
It still does, and of course
that works just as well
for Swift code as it does
for Objective-C code.
Let's talk about
performance testing.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Let's talk about
performance testing.
This is another great new
thing we released this week,
and if you haven't heard
any of the talks yet,
I highly recommend you
go back and watch them.
There was a session this
morning about testing,
and performance testing
is pretty great,
and we've made it really simple.
The great thing about it, in
the context of Xcode Server,
is that when you write
a performance test,
you can have Xcode Server run
that test on multiple devices.
So this is really great.
So you may be surprised to find
that your code runs a lot slower
on one device or on
one operating system
than you expected it
to, and with Server,
it makes it really
easy to see this.
The key thing about
performance testing is
that they work great
out of the box.
You can just write your
code in a measure block.
You don't have to
set a baseline,
but if you set a baseline,
you get a lot more value
out of them, and the thing
with baselines is you have
to set them on a per
test, per device basis.
So that look something
like this.
We have this report.
You can see all the tests here.
You can see I have a
couple of different devices
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
You can see I have a
couple of different devices
that these tests are running on,
and in order to get more value
out of these, I need to set
a baseline for each test,
for each device, and this
is the UI that does that.
We'll see more of
that in a minute.
So in case this is new for you,
this is what a performance
test looks like.
It's really simple.
This is what it looks
like in Objective-C.
Any code in the measure
block here is going
to get executed 10 times.
It's that easy, and in Swift
we have also a measure block.
It's actually a closure, but
it's still called measure block.
That's the API.
It's the same thing.
So to show you all this in
action, I'm going to bring
up my officemate, Matt number 2,
who has the loudest
keyboard I have ever heard,
to show us a demo.
Come on, Matt.
[ Applause ]
>> Thank you, Brent.
So I'm really excited
to be here today
to share the new Xcode Server
and Xcode 6 with you guys,
with support for Swift, our
brand-new programming language,
and the awesome new
performance testing features.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and the awesome new
performance testing features.
All right.
So we have the iOS
simulator open
with our CoffeeBoard
application.
I've got a list here with
me, Matt, Brent and some
of our friends and our balances.
So it looks like I've
got a negative balance.
I guess I owe some
people some coffee.
Here we see a list with exactly
what I owe to each person
or what they owe to me.
I think I remember Brent causing
the issue earlier in the week
that cost me some time.
So I forgot to add
that transaction.
I can do that right here, and
it will get added to the list.
Now, I'm worried that as we
keep adding transactions --
we have some back-and-forth
on our team --
I'm worried that as this
grows, that condensing these
down into a nice list
for the UI is going
to get more time-consuming.
So I want to test the
performance of that
and make sure it stays good as I
keep adding to the application.
And the performance
testing feature
in Xcode 6 makes it
really easy to do that.
So I'm going to quit
the simulator
and switch back to Xcode.
So here I've got an
XCTestCase subclass with a bunch
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So here I've got an
XCTestCase subclass with a bunch
of tests making sure that the
function I have for condensing
down these transactions
works correctly,
but I think we all know there's
more to a great application
than just working correctly.
Customers expect
great performance.
They want your apps to be
fast and get out of their way,
and a performance testing
feature will help you make sure
that that's the case.
So I'm going to add a
new performance test
to our application, and then
I'm going to go ahead and commit
that to my repository so
that Xcode Server can start
integrating that.
We'll push that up, and
while it's doing that,
I'll take a closer look at what
this test is actually doing.
So it's just like
any other unit test.
It's just a method on
your XCTestCase subclass.
It starts with the word test.
Takes no arguments.
Returns no value.
The first thing we do is
create a list of transactions
that we're going to merge,
and then I don't want
to waste any time
measuring that,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
to waste any time
measuring that,
so I do it outside
the measure block,
and that's the next step here.
We call this measureBlock
function,
and pass it at a closure,
and whatever code's inside
there we'll run multiple times
and measure the time,
each iteration,
and average those together
and report that to you.
So I think our integration
has finished completing.
So I'm going to go take
a look at that now.
We have this lovely test
report that's new in Xcode 6.
It breaks down exactly
what happened in each test.
I can see our performance
test is here on the list,
and we've got this
new column for time
that reports the average time
that each iteration
of that test took.
It looks like this took
.15 seconds on average.
That seems pretty good.
If I click that time, then I
can see a breakdown by device.
I've only got the one
device plugged in right now,
but I could have several,
and they would each have
their own time measurements.
Now, Xcode's telling
me something here.
It's telling me that
we have no baseline
for the time performance metric,
and that's because when you
first add a performance test,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and that's because when you
first add a performance test,
Xcode's not going to
make any assumptions
about what's good
performance and what's not,
because every test is unique,
and every device is unique,
and we don't want to
make any presumptions.
So we'll report the time
for you, but we won't try
to failure test based
on how it performs.
If you have any assertions in
your test, we'll fail based
on that, but that's it.
But when I keep working on this
application, I want to know
if this performance gets worse.
I want Xcode Server to
notify me when that happens,
by failing my test, and so to
do that I can set a baseline
for this test, and to
do that I click the time
for this particular device,
and I can hit the
Set Baseline button.
What happens when I
click that button is
that Xcode updates its project
file with entries to say
for this particular test and for
this particular kind of device,
like iPhone 5S, iPhone 4S, iPad,
that this is the
performance standard,
that if we significantly
regress and get slower,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
that if we significantly
regress and get slower,
then this number, that
should fail our test.
The interesting thing
about this getting added
to the project files is
that they get committed
with source control.
So I'm going to go ahead and do
that now because I want our bot
to be able to use them.
You can see these are
just property lists.
You could edit them
by hand if you want,
but we've got a an
[inaudible] on that test report
for managing those baselines.
So I'm going to push that up
and let Xcode Server
start integrating that,
and while it's doing that,
we can look at the
test report here.
We have a lot of
filtering options here.
Like, we can show
only failed tests,
which I don't have any of,
which is great, and if we want,
we can just focus on
the performance test,
like the one I just added.
If there's a particular test
I'm interested in looking at,
I can search by name and filter
to just tests that
match that name.
Or if I'm interested in what
the code is for a specific test,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Or if I'm interested in what
the code is for a specific test,
I can click this little
button here and jump right
to the code for that test.
Now let's see what's
happening with their bot.
We can see exactly which test
it's running and there it goes,
and now let's see what
happens when a test runs
when we have a baseline set.
So now I can see that this time
our test ran a little bit worse,
a little bit longer than
the baseline that I set,
but our test still
passes because it's
within a acceptable range.
We understand that there's
going to be fluctuations
between each test
run, and we don't want
to fail overzealously.
So let's see what happens
when I actually do introduce
a significant regression.
So I'm going to switch
back to the code
that handles merging
these transactions.
Now, what I have here
is clearly working.
My tests are all passing.
The performance tests are
succeeding, but I'm worried
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
The performance tests are
succeeding, but I'm worried
that later on when I come
back to look at this,
I'm not really going to
understand what this was doing.
I think I have an idea for how
I could rewrite this function
in a way that makes
more sense to me
that I'd understand
better later.
Now, I'm just going to
be a little reckless here
and just commit it right away
because I know Xcode
Server has my back,
and if I break the application
or if make it perform
significantly worse,
Xcode Server is going to
tell me and my entire team.
Now I'll push that up and
let that start integrating.
Now, I'm really excited
about this performance
testing feature,
but they can make your
test workflow slower.
So Xcode Server gives
you a great way
to keep your speedy workflow
but still take advantage
of these performance tests.
Now, they'll run slower because
we're running several iterations
of what is ideally
significant work.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
of what is ideally
significant work.
Otherwise, it wouldn't
need testing.
So you can set up one scheme
that Xcode Server runs
that runs all of your tests
including your performance tests
and another one to run locally
that just runs your
quicker ones.
So you get the best
of both worlds.
I think I saw Jeeves tell me
that my integration was done.
Oh no, it looks like I
have some failing tests.
I guess that new merge function
wasn't as good as the old one.
Looks like it is
performing 137% worse.
That's pretty bad.
I don't want to ship that
when we ship our application,
but I'm a little busy
to fix it right now.
I'm busy giving a
WWDC presentation.
So I'm going to trust that our
notifications let my team know
and that someone else will
take care of fixing that.
All right.
I'm going to go ahead and hide
Xcode, and I have one more thing
that I want to show you guys.
So last year when we
introduced Xcode Server,
we gave a great way to give
high visibility to what all
of your bots were doing
inside your office,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
of your bots were doing
inside your office,
and we called it Big
Screen, and we have a new,
beautifully updated
version of Big Screen
that I'd like to show you now.
So this is the new Big Screen.
Along the left side you can
see we have a list of bots.
These are all the bots
configured on our server.
We got a little punny on
our team, and we can see
that below each bot name, we
have icons indicating what kinds
of issues we might
have on those bots.
So CoffeeBoard clearly
has test failures
and a performance regression.
Oh, but it looks like someone
started a new integration
on there.
I hope somebody came along
and fixed my mistake.
Now, when we focus on each bot,
we show the counts of errors,
warnings, analysis issues,
tests or test failures
if some are failing, and
we even show the change
in those numbers form
the previous integration.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Along the bottom you can
see the list of committers,
and you can see even --
if we have issues that
we've attributed to them,
how many they might have.
Looks like CoffeeBoard
finished integrating.
Our tests are now
passing, which is great.
It looks like somebody came
along and fixed my mistake.
It looks like it
was the other Matt.
So applause for him.
[ Applause ]
That's the new Big Screen, and
that's all I've got to show you.
I'll hand it back to Brent.
Thank you everyone.
[ Applause ]
>> Thanks, Matt.
Those are great demos
by two great guys,
both with the same name.
So in summary, we've got
a great new foundation
in this version of Xcode Server.
Got tons of bug fixes
and other improvements.
Please keep sending
us your feedback.
We are listening, and we've
got great, great new features,
like in-flight progress
notification.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
like in-flight progress
notification.
So you can see exactly
what your bot is doing
when it's integrating.
We have issue tracking.
So Xcode Server can tell you
when an issue was introduced,
if it still exists, who
introduced the issue and,
finally, when it was resolved.
This gives you a great sense
of the overall health
of your project.
We have a great new test
result UI including support
for performance testing.
We have a really easy way
to set baselines for tests
for individual devices,
and of course we have
awesome support for triggers.
So we've got a lab
right after this.
Here's more information,
if you need it.
Come to the lab, and maybe
you can try to out trigger us
and write something
even more awesome
than our friend Jeeves,
and so that's it.
Thank you.
Have a great week.
[ Applause ]