Transcript
[ Music ]
[ Applause ]
>> Hi, welcome to Getting to
Know Swift Package Manager.
I'm Rick Ballard.
With me today is Boris Buegling.
And we're very excited to tell
you all about Swift's Package
Manager, or as we call it
sometimes SwiftPM.
We're going to focus today on
the open source project.
And not on Apple's other
developer tools.
But we've got plenty to tell you
about here today.
The Swift Package Manager makes
it easier to develop and
distribute source code in the
Swift ecosystem.
Today we're going to talk about
its goals, some of its design,
and a little bit about where we
can go from here.
We'll start by telling you a
little bit about why we decided
to create a new Package Manager
as part of the Swift project.
We'll show you little bit about
how to use it.
And then dive into its design
and features.
We'll tell you a little bit
about where we can go in the
future and close out by
describing SwiftPM's open source
process and how you can get
involved if you're interested.
I'm sure most of you are
familiar with package managers
they're a great way to share and
reuse code.
But why did we decide to create
a new one for Swift?
First of all, Swift is a
cross-platform language.
So we wanted a great
cross-platform tool for building
your Swift code.
This makes it easy to configure
your code in a consistent way
and run it on all of Swift's
supported platforms.
SwiftPM includes its own
complete build system, allowing
you to configure your software,
build it, test it, and even run
it from one tool.
We also wanted to make it as
easy as possible for you to
share your Swift libraries with
anyone wherever they are by
providing a canonical package
manager in the Swift project, we
hope to define a common standard
for the way that you can
distribute your libraries.
This makes it easy to grow the
Swift ecosystem and make Swift
better for everyone.
Many of you may have great ideas
for features that you'd like to
add.
But we'd like to be careful
about what is added to the core
libraries so we can maintain a
careful and curated API.
A great package manager makes it
easy to distribute these ideas
as packages instead of having to
put them into the core library.
The best ideas can organically
gain traction with the community
over time and become
increasingly standardized.
Finally, by building a package
manager alongside Swift, we're
able to take advantage of
Swift's power and philosophy.
SwiftPM is itself written in
Swift and it's even a Swift
package.
Beyond that, we have the
opportunity to work closely with
the Swift language and core
library projects to build great
package manager features that
will help your Swift code sing.
SwiftPM is part of the Swift
open source project, and has its
own presence on Swift.org, and
on GitHub.
The Swift Package Manager
section of Swift.org is a great
place to go to get started.
When you're ready to try it out
you can find it included in
every Swift toolchain, also
downloadable from Swift.org.
And of course, it's included in
every release of Xcode alongside
the Swift tools.
So, to start telling you a
little bit about how to use it,
I'd like to invite Boris
Buegling up to show you the
basics.
[ Applause ]
>> Thanks, Rick.
Let's take a look at how to use
SwiftPM.
SwiftPM consists of four command
line tools, and at the top level
Swift Command.
Swift Build, to build your
package.
Swift Run to run its executable
products.
Swift Test to run tests.
And Swift Package to run various
non-build operations on the
package.
Packages are stored in git
repositories.
And diversions are represented
by git tags.
Next, I'm going to show you a
demo of how easy it is to create
your own first Swift Package.
We start in terminal, and we
create a new directory, called
helloworld.
This will also be the name of
our package.
Switch to that directory and we
will run Swift Package init with
the type executable.
With this, SwiftPM creates a
basic package and structure for
us.
Let's open finder to look at it
a little bit more closely.
We have the Package.swift
manifest file, which describes
the structure of the package.
We get a basic README.
You have the Sources directory
with a subfolder for our
helloworld target.
And the main.swift file for our
executable.
You also get a test directory,
where we could later put some
unit tests.
Let's switch back to terminal.
And we will type swift run to
build and run this package.
This compiles the package, links
executable, and we see
helloworld is the output.
Next, I'm going to switch to
another terminal window, where
I've prepared a more complex
package.
We will use this in the
following to discuss the basic
concepts of SwiftPM.
But, first, let's also just run
it to see what it does.
So, you can see, it outputs
randomly generated playing cards
to the terminal.
Now, we can switch back to the
slides to talk about SwiftPM's
basic concepts.
A package consists of three
major parts; dependencies,
targets and products.
And we'll look into each of
these in more detail in the
following.
Dependencies are those Swift
packages that you can use when
developing your features.
Each dependency provides one or
more products such as libraries
that your package can use.
Let's take a look at how
dependencies look in the package
of Swift manifest file.
Each dependency has a source
location and it is versioned.
Targets are the basic building
blocks of packages.
A target describes how to build
a set of source files into
either a module or a test suite.
Targets can depend on other
targets of the same package and
on products exported from other
packages, declared as
dependencies.
Products are executable to
libraries and products are
assembled from the build
artifacts of one or more target.
Packages provide libraries for
other packages by defining
products.
By default, you do not have to
declare the type of library
explicitly, but SwiftPM will
choose it for you based on its
use.
If needed, you can explicitly
declare a library a static or
dynamic.
Let's take a look how our
targets are configured in the
manifest.
In our example, we have three
targets.
The first is called libdealer,
and it contains the
implementation of our main
functionality.
And it has one dependency, the
deck of playing cards product
which comes from the dependency
we declared earlier.
Second target, dealer depends on
that to provide the command line
tool that we just run earlier.
And finally, we also have a test
target that depends on the two
other targets and this is where
we can unit test our
functionality.
In our example package, we have
also configured two products.
The first is a library product
corresponding to the libdealer
target.
And this provides our
implementation as a library for
external consumption.
Second, we have an executable
target depending on the dealer
target which provides an
executable for command line use.
To complete this section, I will
show you how we can use a
package to add a new feature to
the example.
This, we switch to a new
terminal window and we open up
the package.swift manifest file.
We want to add a new dependency.
In this case, this is actually
SwiftPM itself.
As Rick told you, it is itself,
its own Swift Package.
It doesn't provide a stable API,
though, that is why we're
depending on an exact version
number.
We also want to depend on one of
its products in the libdealer
target.
It is called utility.
And among other things, it has a
class called terminal
controller, which provides us
with the possibility to color
the terminal output.
Note however, that this is no
official Apple API, we're just
using it for the demo.
Let's switch back to the
terminal.
I already changed the code to
make use of the new dependency
before this demo.
So, we can just run it to see
the result.
And as you can see, we have the
same output, but now it's a
little bit more fun with some
colors.
I want to show you one last demo
which is how SwiftPM can run
tests.
For this, we're using the Swift
Neo package.
A networking library that Apple
open source earlier in the
spring.
We will run Swift Test with a
parallel option.
This allows us to run tests in
parallel.
So, you get your test results
faster.
And we also pass the filter
option.
This allows you to run a subset
of tests to you can iterate on a
single future.
This will now again compile our
package and run the tests in
just a few seconds.
And as you can see, we get a
nice progress bar and the tests
finish really fast, because we
were running them in parallel.
Let's switch back to the slides
again.
Next, I'm going to talk to you
about the design of the Swift
Package Manager.
SwiftPM follows Swift's
philosophy.
It is fast, safe, and
expressive.
It is safe due to its isolated
build environment and the fact
the builds cannot run arbitrary
commands.
It is fast due to using a build
engine that is scalable to large
dependency graphs.
And it's expressive due to using
the Swift language for the
package manifest.
And this also allows you to use
a programming language you're
already familiar with.
For the rest of the section, I
will take you on a journey
through the different steps you
will encounter when creating
your own Swift packages.
We will start with
configuration.
As we saw earlier, SwiftPM's
manifest is based on Swift.
Using Swift makes it easy to
understand because there is no
new language to learn.
We follow Swift's API design
guidelines to make it even more
familiar.
And, it allows us to take
advantage of existing tooling
written for Swift, but when
writing your own manifest, you
should still prefer declarative
syntax and avoid side effects.
Because SwiftPM makes no
guarantees about when or how
often your source code is being
evaluated.
On the left-hand side here, you
see an example that is not
really declarative.
We cannot see the name that is
being generated and it's used in
a couple of times across the
package.
In contrast, on the right-hand
side, we have a fully
declarative manifest by using
string constants.
It is easy to understand and see
what the targets are.
So, as you can see, not using
declarative syntax also makes
your manifest harder to
understand for you and your
users.
Source files organized on disks
in folders named after each
target in the package.
This makes it easy to get
started and allows packages to
adopt a common structure, so
that you can navigate them
quickly.
Package Managers and other build
tools often have attention
between what is explicitly
configured by the user and the
conventions that are imposed by
the Package Manager.
As I told you earlier, the
source file is automatically
picked up from convention base
locations on disk, so that you
can very easily add or remove
source files without having to
edit the package manifest.
Products and targets, however,
are worth explicitly configuring
to make it easier to understand
the package and what it defines
without needing to cross
reference with the layout on
disk.
It also makes it easy for
clients to see what a package
provides just by looking at the
manifest.
SwiftPM also supports building
source code for other program
manager languages, such as C,
C++, and Objective-C.
This allows integration with
existing code.
Note however that we do not
support mixing those languages
with Swift in the same target.
Next, we're going to look at
dependencies and versioning.
To make sure your packages can
benefit from bug fixes without
constant churn, Swift packages
should adhere to semantic
versioning.
This is a commonly used standard
which assign specific semantic
meaning to each of a version
number's components.
The major version signifies
breaking changes which required
clients to update their code.
Examples for the changes could
be deleting an existing type,
deleting a message, or changing
its signature.
But they also include backwards
incompatible bug fixes, or major
changes to the behavior of
existing API.
The minor version should be
updated if functionality is
added in a backwards compatible
manner.
Examples for this is adding a
new method or type.
And finally, the patch version
should be increased when you're
making backwards compatible bug
fixes.
This allows clients to benefit
from bug fixes without risking
breaking the source code.
SwiftPM needs to determine the
exact versions of all packages
in the package graph before it
is ready to build.
We do this with a process called
dependency resolution.
As part of this, SwiftPM looks
at all the requirements
specified for packages and finds
the latest version that is
compatible with all of them.
Let's take a closer look at what
SwiftPM is doing in the process
using the demo I showed you
before.
The dealer package from the demo
has two direct dependencies.
One, is SwiftPM itself, and the
other one is deck of playing
cards.
SwiftPM will resolve the
versions of these direct
dependencies.
For the first one, this is
straightforward because we
specified an exact version,
beginning exactly that tag.
For the second one, we're using
the from syntax.
That means we're getting updates
to the minor or patch
components.
In this case, we're ending up
with a tag 3.1.4.
The whole process is recursive.
So next SwiftPM will look at the
transitive dependencies of all
diary points.
So, PM has no further
dependencies, so there's nothing
to do there.
But a deck of playing cards
depends on the Fisher-Yates and
playing card packages.
Next, SwiftPM has to resolve the
versions of these packages
again.
For the Fisher-Yates package,
this works the same way as
before because we are also using
the from syntax.
In this case, we're ending up
with a tag, 2.2.5.
For the playing card package,
we're using up to the next minor
syntax.
This means, we're getting only
updates to the patch component.
You might want to use that
syntax if you want to be more
conservative as a dependency and
only take bug fixes.
In this case, we're ending up
with a tag 3.0.2.
Finally, when looking at a
target, SwiftPM has to match up
its required products with the
packages that we resolved.
For this, we're looking at the
dealer target from the demo, and
as you can see, the utility
product is provided by the
SwiftPM package.
And the rest of the packages
provide the other products.
After dependency resolution, the
resolves are recorded in a file
called package.resolved.
the purpose of this file is so
that you can share the resolve
versions with other members of
your team, or your continuous
integration infrastructure, so
that you get dependable build
results, and you can
deliberately choose when you
want to update a dependency.
You do so by running Swift
Package Update, when you're
ready to update.
Note also, that this is your
top-level package that contains
package.resolved.
If one of the transitive
dependencies contains a
package.resolve file, it will be
ignored for dependency
resolution.
Next, let's take a look at
building your package.
SwiftPM uses llbuild as its
underlying build engine.
llbuild is a set of libraries
for building build systems.
And it's built around a general
purpose and reusable build
engine.
This provides us with ability to
do fast as well as correct
incremental builds.
And is also used by Xcode's new
build system.
It is also the part of the Swift
open source project.
Developing software in isolation
with all dependencies explicitly
declared ensures that even
packages with complex
requirements can be reliably
built and used in different
environments.
Instead of installing packages
globally into the system,
SwiftPM only allows you to use
packages that you explicitly
depend on.
We also leverage build
sandboxing so that nothing can
write to arbitrary locations on
the file system during the
build.
SwiftPM does not allow executing
arbitrary commands, or shell
scripts, as part of the build.
This allows us to fully
understand your build graph and
all of its inputs and outputs to
do fast, as well as correct,
incremental builds.
Because we have a view of all
your dependencies.
As I showed you in the demo
before, SwiftPM also supports
testing.
This is based on the XCTest
framework that you're already
familiar with.
We support parallel testing, so
that you can get your test
results faster.
And we support test filtering so
that you can run a subset of
tests and iterate on a single
feature.
As we're evolving SwiftPM, we're
thinking about workflow
features, especially so that you
can do all of your development
on the command line.
One such feature is edit mode,
which allows overwriting all
transitive occurrences of a
specific package, with a locally
checked out copy so that
temporary edits can be made, and
changes to transitive
dependencies can be tested
without having to forward all
packages in the graph upfront.
Branch dependencies allow
depending on packages without
strict versioning requirements.
This is useful when you're
developing multiple packages
together.
This is a development only
feature, so you have to change
to certain version dependencies
before you publish a tag.
Local packages allow you to use
packages directly from the file
system, instead of from a git
repository.
This is useful to make it
possible to bring up multiple
packages during the initial
creation.
So, last topic, I want to talk
to you about adopting new
versions of SwiftPM and the
Swift language.
Each new version of Swift can
bring a new version of the
package.swift manifest API.
The previous API is still
available, so that you can take
advantage of new source tools
without having to update your
package or losing access to
existing packages.
New API can be adopted
independently of changing to a
new Swift language version for
your packages' source code.
To specify which version of the
API is being used, we're using
the Swift Tools Version command
at the top of the package.swift
manifest file.
This specifies the minimum
required version of the Swift
tools that is needed to process
the given manifest.
Each package can also declare
which versions of the Swift
language it uses for compiling
its source code.
This is a list, so you can
choose to support multiple
versions of Swift with the same
version of your package, by
using compiler directives.
A package graph can be a
mix-and-match of packages with
different language versions.
I told you a lot about how
SwiftPM works today, next I'd
like to invite Rick back up to
the stage to tell you where we
can go from here.
[ Applause ]
>> Thanks, Boris.
So, Boris has shown you what you
can do today, but there's a lot
more potential out there.
SwiftPM is still a young project
with lots of room to grow.
Swift uses an open evolution
process which means that anyone,
including you, can contribute
your ideas.
If you're looking for some
inspiration, we'd like to share
some of our ideas, but none of
this is a plan of record.
We're sharing these ideas so
that you can see the potential
of the Swift Package Manager.
And we welcome you to provide
your feedback, comments, and
your own ideas as we evolve is
this product.
The ideas I'm going to cover
today break down into four
different themes.
Letting the Swift Package
Manager integrate with other
tools that may want to sit on
top of it.
Helping you publish new versions
of your package and deploy their
products.
Supporting more complex packages
than SwiftPM is able to build
today.
And finally, some forward
looking thoughts on package
discovery and trust.
While the SwiftPM command line
experience is important, we want
to make sure that SwiftPM can
integrate with other tools, such
as development environments,
automation, and more.
We've already laid the
groundwork for this with
SwiftPM's library-based
architecture.
SwiftPM doesn't have a stable
API today, but for tools that
are willing to keep up with the
SwiftPM changes, it's available
for adoption and additions
today.
If you're looking to build
support for SwiftPM into your
developer tools, we welcome your
contributions and discussion.
We want to make SwiftPM part of
a thriving ecosystem of
developer tools.
One thing we've seen requested
recently on the Swift forums is
a way for people to edit their
package.swift manifest from
automated tools, instead of
making their users always edit
the source code directly.
We think that it's possible for
SwiftPM to support this,
probably by using libSyntax.
libSyntax is a library being
developed in the Swift open
source project that makes it
easier for you to understand and
manipulate Swift syntax from
other tools.
Boris told you earlier that you
should prefer declarative syntax
for your package.swift manifest,
and this is another reason why.
That will make it much easier
for SwiftPM to understand your
manifest, so that it can make
automatic changes, such as
adding new dependencies or
targets as shown here.
So, there's a lot of room for
SwiftPM also to add new
functionality to help you
publish new versions of your
packages and deploy their
products.
Today, when you want to publish
a new version of your package,
you tag it manually with git.
And if you want to inspect your
published tags, you use git
directly for that as well.
We could add new functionality
to automate this process and
perform additional housekeeping,
validation, and other auxiliary
tasks you might want as part of
a streamlined publishing
workflow.
One especially useful feature we
could add here would be
assistance with maintaining
correct semantic versioning.
We could have SwiftPM analyze
the API differences in the new
version of your package and
detect when you've made a change
that is not compatible at
compile time, so that it can
suggest updating the major
version of your package.
Another thing we could do is
make it easier to deploy the
products of your packages from
SwiftPM.
You may want to customize the
linkage with libraries, or the
product layout for your specific
deployment environment, whether
local or on a server.
Or, maybe you want to include
version information about what
packages were built into the
product.
Or, you otherwise want to use
the context that SwiftPM has
about your packages somewhere in
your product.
SwiftPM could add new commands
to support all of these needs.
There's a lot that you can build
with SwiftPM today, but we also
want to be able to support more
complex packages with more
sophisticated needs.
The biggest gap we have right
now, is probably support for
resources.
If you have any images, data
files, or other assets, SwiftPM
currently provides no way to
bundle these up you're your
products.
The foundation core library,
actually just added API this
spring for doing resources in a
cross-platform manner so SwiftPM
could adopt this API if we want
to build this feature.
We know that some users also
want support for specifying
compiler flags, linker flags,
and other properties that
SwiftPM doesn't support today.
It would be really great for us
to add a robust build settings
model, potentially including
things like conditional
settings, or fine-grain control
over what parts of the package
get which setting values.
Boris also talked to you earlier
about SwiftPM build isolation,
and why it's important.
We don't let you run arbitrary
shell scripts.
But many users may want some
level of customization for their
build, either because they want
to support custom languages, or
processors, they want to run
their own documentation
generator implementor, or they
have other steps that they need
to bring to the build process.
We think that SwiftPM could
support this safely, possibly
even through real tools packages
that bring new tools into your
build process.
The important thing we need to
make sure here, if we do such a
feature, is that any new tool
that's brought into the build
process have to correctly
declare their input and output
dependencies, so SwiftPM can
continue to maintain correct
incremental, and parallelizable
builds.
Finally, I want to talk a little
bit about some forward-looking
thoughts on package discovery,
trust, and management.
Git itself supports, the
protocols that get supports,
provides security mechanisms
like TLS to make sure that
you're actually talking to the
remote repository that you think
you are.
But a malicious actor could
still compromise remote
repository and put malicious
content in.
This is actually something
anytime you're using third-party
code, you should be aware of
these sorts of risks.
But the Swift Package Manager
provides a great opportunity for
us to build security features to
make sure that you're actually
getting the package content that
you expected.
SwiftPM also prevents your
package.swift manifest
evaluation in your build from
escaping and writing things out
into your file system or
accessing the network.
We're using macOS' sandboxing
technology for this today.
And it's great.
But we'd like to bring this kind
of security to other platforms
as well.
Many users may want to be able
to fork their packages easily,
either because they want to make
a private customization to one
of the packages in their graph.
Or, even because they just want
to override the origin URL of
where they're getting each of
those packages from, so that
they can point at a private
mirror that they control and not
depend on the original package
always being there.
Ultimately, I'd like some day
for us to have a real index for
Swift packages.
In addition to providing a
standardized namespace and
making it easier to discover new
packages, we could even support
things like quality metrics for
a package.
Like what is its automated test
coverage?
Or ways for you to evaluate the
trustworthiness of a new package
that you're considering
adopting.
So, I've gone over a lot here.
But these are just some of the
possibilities.
Ultimately for those of you, who
are interested, we're interested
in your feedback, ideas, and
contributions to help make Swift
Package Manager the best tool
that it can be for the developer
community.
So, to talk about how you can do
that if you'd like to, I'd like
to talk about Swift's open
source process.
As I said earlier, the Package
Manager is part of the Swift
open source project.
And Swift.org is also a great
place to go if you want to learn
about the community and the
process.
SwiftPM uses the Swift language
evolution process which means
anyone can submit a proposal for
major new features or changes to
the Swift Package Manager.
Before you go off and draft a
whole formal proposal though, I
recommend that you swing by the
Package Manager section of the
Swift forums and socialize your
idea with the community.
You may get a lot of feedback
that helps make your idea even
better.
If you're interested in dipping
your toe in the water with a
smaller contribution the Swift
bug tracker at bugs.swift.org
has plenty of ideas.
In particular, you may want to
look for bugs tagged with this
starter bug tag.
And since, as I said, SwiftPM is
written in Swift, you may find
it's actually pretty easy to
dive in and take a look.
Of course, if you find bugs when
you're using SwiftPM, we
encourage you to go file them on
bugs.swift.org as well, where
you can track how we handle
them.
SwiftPM gets to take advantage
of the same great continuous
integration infrastructure that
the Swift project has.
Which means that poll requests
can be automatically built and
had their tests run before their
merged.
Because the SwiftPM code base
itself has great test coverage,
we found that this
infrastructure is really useful
for us.
When you're ready to try out the
latest changes, you can download
the Trunk Snapshot Toolchains
that are updated on a regular
basis available on Swift.org.
We've been really happy with the
growth of the SwiftPM community
so far.
We've had over 180 people
contribute, either with bug
features or new features.
And the Swift Package ecosystem
is growing at a healthy rate as
well, often with cross-platform
packages, and many public
packages available on GitHub.
What this means is that you can
focus on what makes your product
special and let package
dependencies handle the rest.
There are a couple things that I
recommend you try SwiftPM out
for today, even though it has a
lot of room to grow in the
future.
Those two things are command
line utilities and libraries and
for developing with Swift on the
server.
The server-side Swift community
has been making extensive use of
the Swift package manager.
And server-side Swift has been
growing well itself with many
frameworks now available for
doing web and backend
development.
If you would like to take a look
at this approach, I think you'll
find that Swift is a great
language to do this kind of
cross-platform development on.
But you could also go ahead and
use SwiftPM for creating
command-line utilities and
libraries today, whatever makes
sense for you.
Ultimately getting started is as
easy as opening a terminal
window and running Swift package
init.
So, the next time you're
thinking about trying something
new, I encourage you to give it
a try.
And if you're interested in
contributing, swing by the Swift
forums and start a conversation.
If you'd like to come chat with
us, you can find us in the labs
tomorrow at 3 p.m. Ultimately,
I'm really excited about where
we can go from here and what
this developer community can do
together.
Your contributions will help us
design a package manager that
will be great for the whole
Swift community.
Thank you.
Enjoy the rest of WWDC.
[ Applause ]