WWDC2014 Session 217

Transcript

X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
>> Good morning everybody.
Thanks for coming.
Welcome to Creating Extensions
for iOS and OS X, Part Two.
I'm Damien.
I'm from Core OS.
And we apparently have a user
facing feature this year.
[ Cheers and Applause ]
Glad to see everybody's
excited about Extensions.
Let's dive right in.
So before we talk about
what an extension is,
let's talk a little bit
about what an app is
and why Extensions
are different.
So for an app, it's still in
iOS 8, even with Extensions,
is the most important
experience to the user.
On iOS, the app owns
the entire screen.
On OS X, the cursor focus
goes to the front-most app.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
On OS X, the cursor focus
goes to the front-most app.
This is still the number
one priority for our users.
And on iOS, apps are
completely managed
by the user via the App Store.
And OS X the user can drag them
to the trash, move them around.
Apps are entirely
managed by the user.
So what about Extensions?
So Extensions are
important to the user.
But they're not more
important than the current app.
They are there to augment
the app's experience.
And rather than directly
managing them,
Extensions come and
go with apps.
So the user isn't
going to be purchasing
or installing individual
Extensions.
They're going to be getting
those Extensions with the apps
that they download
from the store.
So Extensions are
built separately.
They are a separate
target in XCode.
So we have this new
section in XCode
for app extension
template targets.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So when you want to
create an extension,
you're getting a new
piece of code in your app
that is actually
a separate bundle
and admits a separate
executable.
And with that executable comes
a distinct set of entitlements.
And when we have a separate
executable with its distinct
of entitlements on disk, that
allows the operating system
to know that these two things,
the app and the extension,
are different entities.
And with that, you're
getting a different process.
So an extension is running
in an isolated address space
from your app and it executes
completely independently.
So this means that the system,
even if your app is not running,
can still fire up
your extension,
even if your app is suspended.
That doesn't affect any of your
Extensions running for the user.
And because of this, the
system can optimize each
experience separately.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
experience separately.
So we can schedule them, and
even if the app goes away,
your extension still
sticks around.
So here's an example
of an extension
and what the user might do
to invoke your extension
and its UI.
Here we just have a
little network thing.
So it's a new social
networking app.
And it's bundling a
social sharing extension.
So we see it in the far right
there with the purple icon.
And I go to this site and I want
to share this with my friends
on my new social network.
So the user will see an
icon representing my app.
But what that means is that I'm
going to be using the extension
for social sharing
vended by that app.
So in previous releases of iOS
we've been able to do this,
but the Facebook and the
Twitter experiences, et cetera,
have all been provided by Apple.
Now we're letting
anyone who wants
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Now we're letting
anyone who wants
to provide a social sharing
extension not only provide it
but define their own experience.
So in this case, when
the user elects to share,
they're getting a window that
is designed by the developer
of this social networking app
with whatever experience --
whatever experience that app
developer wants to provide.
So that's a high level
of how Extensions are
architected on our system.
Let's talk a little bit
about how we actually extend
the experience of your app
and project it into other
parts of the system.
So as we saw yesterday
in Part One,
Extensions are a very
focused experience.
And the operating
system takes care
of seamlessly merging the
experience that you provide
in your extension
with the current app
or just whatever the
current system UI is.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So you guys are going
to want to provide code
to make the experience your own.
And that's going to almost
certainly mean that you're going
to want to re-use at least
some of the code from your app.
So we hope you've been using the
Model View Controller paradigm.
Features like Extensions
are exactly why we
so desperately encourage
you guys
to adopt this design
pattern year after year,
because chances are with your
extension, you're going to want
to share at least the data
model and probably some
of the View Controller
layers, too.
You probably don't want to
share the view of your app
because your app is
designed around taking
up the whole screen whereas
your extension will not.
But that's only one-third
of your code.
You don't want to have
to rewrite the entire
two-thirds of it.
You want to be able to
share code between the two.
And if only computer science
had come up with a way
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
to allow the same code to exist
in two processes without having
to duplicate those code pages.
I think we might have
a solution to this.
And I think that
solution's called Frameworks.
So, on iOS, in iOS 8, we're now
allowing you to ship Frameworks
within your app bundle.
[ Applause ]
Don't get too excited.
So this is obviously new to iOS,
but you've been able to do this
on OS X since the dawn of time.
But it's a brand new
capability to iOS.
And what we're doing with it
is we're also encrypting the
frameworks that you
ship in your app bundle
with the same encryption
that your apps are encrypted
with to give you that
same level of protection
against piracy and
-- well, piracy.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
against piracy and
-- well, piracy.
[Laughter] So it's
critical to note
that these frameworks are not
general code sharing mechanisms.
So you're not going
to have many apps
on the system referencing
the same copy of a framework.
These frameworks are here to
facilitate sharing between
and an app and all
of its Extensions.
So the code that that framework
contains can be used by the app
or any of the Extensions
it bundles,
but nothing else on the system.
So when you use -- when
you bundle a framework
with your app, there
are some implications
for your minimum
deployment target.
If your app links the framework,
we're going to change its
minimum deployment target
to iOS 8 because
as I said before,
we're encrypting
these frameworks.
And previous versions of the
operating system don't know
about how to decrypt
them when you use them.
However, if you're
just shipping your app
and then there's a few
Extensions you have
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and then there's a few
Extensions you have
that all link in a framework,
your app's minimum deployment
target doesn't change at all.
Your extension functionality
obviously won't be available
to previous versions of iOS,
but the app will still be able
to run without any problems.
So that's a very brief overview
of embedded frameworks on iOS.
To learn more, I encourage you
to go to the session tomorrow
at 3:15 in Presidio,
Building Modern Frameworks,
where it'll cover API usage,
how to write a good framework,
how to bundle it,
all that good stuff.
But for now, you've
got Frameworks.
And it seems like you
guys are excited about it.
So, another topic for code
sharing is API availability.
If you're writing one
of these frameworks,
or if you're bundling
one of these frameworks,
what code are you
going to be using
between your app
and your extension?
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
between your app
and your extension?
So, the vast majority of our
API is going to be available
for Extensions to use.
But there are going
to be some exceptions.
And these exceptions are
going to be marked explicitly
with a new kind of
unavailability macro.
And here's an example
of what that looks like.
So, UIApplication --
or with UIApplication,
the shared application method
has been explicitly marked
as unavailable.
And the UIKit folks have
helpfully provided an error
message that says: Use a view
controller based solution
where appropriate.
I'm in Core OS, no
idea what that means.
But I'm sure you
guys have an idea.
Now, if you're dropping down
to Super [inaudible] API
that has underscores,
there's going
to be a corresponding
availability macro there
that is evaluated the
same way by the tools
and will generate the
same errors at build time.
So that's unavailability.
Like I said, the vast majority
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Like I said, the vast majority
of cases you probably won't
need to worry about it.
But if you do trip across
an API use in your extension
that is not allowed, you'll
know as soon as possible
at build time before you
can even run the extension.
So that's how you share code
with your extension
and your app.
But at run time you're
going to want
to create a consistent
experience, too.
And a consistent experience
often means sharing the same set
of resources and making sure
that changes the user has made
to their data in your
extension are reflected
in the app the next time
they see it, or vice versa.
So we have a few data
sharing solutions
for you for Extensions.
So the first thing
to understand is
that an extension has
a separate container
from its app by default.
It's just like how two apps
are segregated from each other.
They don't have access
to each other's data.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
They don't have access
to each other's data.
But for any extension
you bundle,
you can opt to share
specific bits of data
that will help you provide
that consistent experience
that all of our users expect.
And you can do this with what's
called a shared container.
So this is based on a new
concept called an App Group.
And this provides a shared
storage area for kind
of just general data sharing
between your app your extension.
So if you have some custom
data models or custom databases
that your app and your
extension need to use,
this is a good place to put
those files and access them.
However, you have to remember
that there's nothing stopping
your extension and your app
from running at the same time.
Your app may be in a
long-running background task.
It may not have been
suspended yet.
And it might be writing some
data to that shared container.
And then the user might
bring up your extension.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
And then the user might
bring up your extension.
So what this means is
that you have to make sure
to safely coordinate
all the reads
and writes for those files.
You have to make sure that
you only write to the file
when there are no
readers contending for it.
And that you're only
reading when the file is
in a consistent state.
Otherwise, you're going
to blow up your file,
or it's just going
to get corrupted.
So we introduce some sort of
synchronization scheme here
so that if the extension
writes, it holds off the app.
And if the app writes, it
holds off the extension.
And they're both dealing with a
consistent view of the universe.
And here are a few of
those technologies.
The first one is
NSFileCoordination,
which kind of provides a general
inner process synchronization
strategy for your
extension and --
for your extension and your app.
If you're using CoreData, you
can get a lot of this for free.
There are certain
CoreData backing stores
that are documented
as being usable
for multiple processes
at the same time.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
for multiple processes
at the same time.
And sqLite also makes
some of these guarantees.
So if you're using one of
those two technologies,
chances are you're already set.
But if you have something more
custom, you're going to want
to use NSFileCoordination.
But if you're talking about more
structured data that is, say,
vended by Apple's APIs
like NSUserDefaults,
you can end up sharing
much more easily.
So, an app and its
extension both have different
defaults domains.
But you can set up
a shared domain
with the initWithSuiteName API.
And you create this suite, and
the API manages access to it
for your app and your extension
so that you don't have to worry
about making sure that
you're not writing
to that backing store
when someone else is
trying to read from it.
The API coordinates
all of that for you.
And this is a really
easy way for, say,
the user to modify
some setting in the app
which then the extension can
pick up on its next invocation.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
which then the extension can
pick up on its next invocation.
There's a similar
story for keychains.
So the shared keychain
is based on an app group.
And the app group can
encompass a few apps
as well as your Extensions.
So by default, as I
said, the extension
and the app have a
different keychain.
But you can set up a shared
keychain with an access group.
There's an issue right
now in the first beta
where this is actually
team ID-based.
So if you're familiar with that
where you pre-fix your bundle
identifier with a team ID
in order to facilitate sharing,
that's how you do it
in the first seed.
We're going to fix this up for
App Groups by the time we ship.
So in terms of privacy, it's
a slightly different story.
When I say privacy, I
mean the dialog that comes
up when an app is trying
to access certain sensitive user
data like photos or contacts.
You'll get a dialog saying:
"Would you like to allow
this app to have access
to this piece of data?"
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
to this piece of data?"
And the user can
either decline or allow.
So when the user allows
your app to have access
to a certain piece of
data, it's actually going
to cover the entirety
of your app bundle,
including Extensions.
So that the user, once they have
approved your app for access
to your photos, they don't
have to then go and approve all
of your Extensions for
access to the same data.
Now, there are going to be
some Extensions to this --
or sorry -- some
exceptions to this rule.
But in general, the rule is
that approval covers the
entirety of your app.
So let's go over best
practices a little bit.
As we said at the beginning,
the front-most app is
the most important thing.
So it's very important that
your exceptions be lean.
These things do not
have the full run
of the system like the app does.
The app's still there.
It's still using the lion's
share of system resources.
The Extensions are very
focused, very purpose-built.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
The Extensions are very
focused, very purpose-built.
And it's important that they get
in, do their job, and get out.
And be stateless.
You might know that we
kill apps aggressively
or suspend them on iOS.
We're doubling down on
that with Extensions.
And we're going to
be more aggressive.
And because of this, there
is no general multi-tasking
functionality available
to Extensions.
So no VoIP, no long-running
background tasks,
that kind of thing.
We do support a short task
completion before we suspend
or kill the extension.
But this is meant to give you
time to flush any dirty state
to disk so that the user
doesn't lose their data and so
that the app can pick that up.
And finally, be awesome.
Make these things seamless.
Make them useful to our users.
And as a user of your products,
I can't wait to be surprised
and delighted by what
you guys turn out.
With that, I'd like to
turn it over to Aki.
[ Applause ]
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
>> Thank you.
Thank you Damien.
Hi, I'm Aki Inoue
from Cocoa Group.
Today, I'm discussing
an exciting new feature,
Action Extensions
for iOS and OS X.
Yesterday in Part One, Ian, Matt
and Guy introduced Extensions
and two Extensions point
along with key technologies
such as remote view
controller and activation rules.
Damien explained foundation of
technologies for Extensions.
Action is an extension
point that utilizes
and integrates all
these best technologies.
Let's get started.
So, Action is a screenshot
[inaudible] system to services,
an inter-application
collaboration technology
available since the
first release of OS X.
Action operates with user
selection inside the host
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Action operates with user
selection inside the host
application that is it
extends the system framework,
not just the single
application or service.
Earlier in this conference,
you have seen one
of Action Extension's
Markup operating inside many
applications on Yosemite.
In fact, this extension
is available
to image attachments
throughout the system.
Let's just dive into
the details.
Just like any other Extensions,
Actions are described
by info.plist.
First, inside NSExtension
Dictionary, it's identified
by NSExtensionPoint
Identifier.com.
apple.ui-services.
Next, NSExtensionPrincipalClass
specifies the main controller
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Next, NSExtensionPrincipalClass
specifies the main controller
[inaudible] for your extension.
Just like Guy and Matt
described yesterday,
it's typically a subclass of
the ViewController option.
Now, the principal class
is the main controller
for your extension as
well as the [inaudible]
to the extension APIs.
With a new View Controller
property called Extension
Context, you can get
NSExtention Context Object,
which represent connection
to the host application.
Through that, you can access
an array of NSExtention items.
Each item representing a logical
data unit coming back from --
coming from the host
application.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
And, you can attach [inaudible]
data to extension item.
Movie, image, voice,
so and so forth.
The data is contained inside an
object called NSItemProvider.
Okay let's back to
the Info.plist.
Inside NSExtensionAttributes
Dictionary,
which specifies extension
point-specific attributes,
you declare NSExtensionservice
RoleType,
which [inaudible]
editor or viewer.
In this example, we are
using NSExtensionservice
RoleTypeEditor.
That means it's only
presented to the context
where the document is editable.
Next, as share Extensions you
can specify activation rules
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Next, as share Extensions you
can specify activation rules
with NSExtensionActivationRules.
It can be NS [inaudible] for
define your string or the list
of supported data types.
Now, let's look at code samples.
First, you can get
extension context
through the extension
context property.
Next, you can get
extension item through item,
inputItems property,
which returns the array
of extension items.
Finally, you can get attached
data using attachments property
and it's containing the
NSItemProvider objects.
Once you have NSItemProvider,
you can load data
from the host application.
It's lazy and secure.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Using the
LoadItemForTypeIdentifier
options completionHandler
and specifying a UTI
type of your choice.
In this case, we are using a
custom data type, MyDocumentUTI.
And you specify a representation
of the data you want to receive.
In this case, we
are using NSData
and it contains the custom
data format, NSDocument UTI.
Once you have that
data, you can set
up your user interface
according to the data coming
from the host application.
Next, with Action, unlike
shared Extensions, it's possible
that extension time
return some modified data.
You've seen Markup changing and
marking up your image in place
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
You've seen Markup changing and
marking up your image in place
and return it to
the mail document.
Your extension can
perform the similar thing.
When user click a Down button or
similar action, you can perform
and return the modified data
back to their host application.
The process is reverse of
loading the data coming
from the host application.
First, you want to
instantiate NSItemProvider
with your data and data type.
Next, you can create
NSExtensionItem
and attach your item provider.
Finally, you can send
completeRequestReturningItems
completionHandler to
your extension item
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
completionHandler to
your extension item
with an array of
extension items.
This actually concludes
your extension life cycle.
So typically you want
to return your data
and conclude your NUI
operation at this point.
Unlike [inaudible] application,
if your viewer type,
you don't have to
return any data back
to the host application.
So if your viewer application
just like share Extensions,
you can invoke
completeRequestReturningItems
completionHandler
with new argument.
So far, I have covered how
to write Action Extensions.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Now, let me explain how you
can integrate Extensions
into your own applications.
On iOS UIActivityViewController
drives both social and share
and Action Extensions.
The [inaudible] controlled by
the View Controller can invoke
and manage Extensions for you.
UI API-wise there's
not much difference
for UIActivityViewController
from the previous release.
You still instantiate
the View Controller
with the selected
items from the user.
And View Controller takes over
and present the [inaudible]
and voila.
One enhancement we have
for iOS 8 is the support
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
One enhancement we have
for iOS 8 is the support
for editor actions.
We added a new property called
completionWithItemsHandler
to UIViewController, or
UIViewActivityViewController.
With this method, if the invoked
action is editor and wants
to return value you
could receive the result
in returnItems array.
And you can replace your
document content based
on the returned items.
NSTextView.
Well, if anybody know in the
audience, I just can't help talk
about text every year
in one way or the other.
On OS X, NSTextView
plays the central role
in presenting Extensions
to the users.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
in presenting Extensions
to the users.
By hovering us on
top of extension --
by hovering extension
on top of text selection
or image attachments,
it presents [inaudible]
with manual listing old
Extensions available
for the context.
Now I have written a simple
little extension for OS X.
And I'd like to show how
to simply present your
Extensions to the user.
I have a text edited document
that contains both
text and image.
By hovering mouse cursor
on top of text selection,
you can reveal the list
of extension supporting
the text data.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
of extension supporting
the text data.
Similarly, by hovering
on top of attachments,
you can show the list of
Extensions supporting images.
In this case, in addition to
the sample extension I wrote,
Party Crasher, it lists Markup
in the text edit application.
So you can see the Markup
is an extension that works,
not just in mail but any other
text view inside the system.
So, let me explain this little
extension called Party Crasher.
Just like many of you, I
like hanging out with people.
And anytime I see
pictures like this, I feel,
well, kind of left out.
[ Laughter ]
So, with this extension, I can
add my picture to any images
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So, with this extension, I can
add my picture to any images
in the system instantly.
[ Laughter and Applause ]
I like this picture,
so let's keep it.
And the change is
saved in the document.
Also, there is another way
to invoke Action
Extensions on OS X.
I have modified this
Text Edit Application.
By the way, Text Edit
Application is a core sample
application for Cocoa
Application.
So you can modify
any way you want
and learn the neat
stuff about Cocoa.
With this Text Edit Application,
a version Text Edit Application,
you can show toolbar.
And as you can see
in the toolbar,
extension can register
themselves as toolbar item.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
extension can register
themselves as toolbar item.
And it can be shown with any
toolbar inside the system.
And the action is applied to
the current selected object.
It can be text or
image, so on so forth.
So, let's invoke the Party
Crasher for text selection.
It appends a message:
Exception [inaudible]
to any text selection
in the system.
Okay. As you can see, Action
Extension is so powerful,
you can write simple
extension like this.
Or you can scale up to
[inaudible] as Markup.
The sky is just the limit.
I have demonstrated
-- with the demo,
I have demonstrated Extensions
can support multiple data types.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
I have demonstrated Extensions
can support multiple data types.
It is common that
you want to support
as many data types as you want.
More data types you support,
it's easier for users
to work with your Extensions.
Let's take a look at how you
can change your behavior based
on the input data coming
from the host application.
In this case, first
we want to check
if it's a text data type coming
from the host application.
We use the kUTTypeText UTI type.
And it contains many
text data format
with hasItemConforming
ToTypeIdentifer method
for itemProvider you can check
if the itemProvider
contains the text data.
Once you make sure, you
can load the data using
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Once you make sure, you
can load the data using
loadItemForTypeIdentifier
options completionHandler using
the data representation
type of your choice.
Since itemProvider has simple
data coordination facility,
it can map some data
to the target object
data type you specify.
In this case, we are
using NSAttributedString.
Once you have your data,
set up your user
interface and code into it.
Similarly, if your
text data check fails,
you can check for image data.
After checking if it's
image, you can load the image
and as I mentioned, itemProvider
can [inaudible] data type
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and as I mentioned, itemProvider
can [inaudible] data type
into UIImage or NSImage for you.
And if you have image, configure
your user interface for image.
For OS X, it's common that
your extension's user interface
doesn't cover the entire
host application's window.
Just like Markup.
Typically you want to overlay
your user interface on top
of the original representation
of the host application.
It's easy to accomplish.
We have additional property
for NSItemProvider on OS X.
It's called sourceFrame.
It contains the screen
coordinate
for the original representation
inside the host application.
If you get [inaudible]
rect information,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
If you get [inaudible]
rect information,
and if it's not empty, you
can adjust the frame according
to your user interface needs.
You might want to have
some additional frame
around the original
representation just like Markup
or Party Crasher does.
Then set up two NSViewController
properties,
preferredScreenOrigin
and preferredContentSize.
As you've seen with Mark's
presentation yesterday,
the preferred content
size is shared
between the [inaudible]
and Action Extension.
By setting up these properties,
the View Controller system takes
over and configures your
extension user interface
overlaid on top of the
original representation
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
overlaid on top of the
original representation
in the host application.
Now, I'd like to bring my
colleague Ian Baird who's going
to be discussing
really, really cool way
to integrate interactive
Web and actions.
Ian.
[ Applause ]
>> Thank you Aki.
So, Aki showed you how to
create a custom action.
And he showed it on OS X.
And today I'm going
to show you how
to build a Safari
custom action or how
to take your custom action
and enhance it for the Web.
We think you guys are really
going to love this feature.
So first, you can see our
custom action in the bottom row
of the activity view controller
as I showed you yesterday
in the Part One of this talk.
And it's called TinySketch.
It can't do everything that
its brother Markup can do,
but it can still pack a punch.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
but it can still pack a punch.
So first, diving in,
Safari Custom Actions is
rich new functionality
in Safari on iOS.
It gives you access to the DOM.
And there are two types
of Custom Actions.
A view controller-based
custom action,
the kind I'm going
to show you today.
And then there's
a no view action
like the Bing Translate action
that Craig showed
you in the Keynote.
How do these work?
We use some JavaScript.
Your extension will
provide JavaScript along
in its resources.
And Safari is going
to find this.
So, here's what this JavaScript
is going to look like.
We're going to expect you
to provide a pre-processing
JavaScript object.
And this object is going to
have two methods, a run method
and a finalize method.
We're going to start by
calling the run method
and give you access to the DOM.
And what you're going
to be expected to do
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
And what you're going
to be expected to do
in this run method is to call
the action arguments completion
function, passing the
data you want to provide
to the custom action
when you're done.
And when the action
has completed,
when the computation inside
of your custom action extension
has finished, we're again going
to call your JavaScript.
And this time we're going to
invoke the finalize method
and pass the action arguments.
And as you can see in this
case, we're just calling alert
and showing the message,
which was provided.
One of the most important
things for you
to remember is this
global variable:
ExtensionPreprocessing
JavaScript.
This is going to be an
instance of your object
that provides these
methods to Safari.
And I think this should
be burned into your mind,
because if you don't provide
this, we're not going to be able
to invoke your JavaScript.
And nothing's going to show
up inside of your action.
So, it's really easy to
talk about these things,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So, it's really easy to
talk about these things,
but it's far better to show it.
So I'm going to show you
a demo of TinySketch.
Moving right in.
Let's build and run
and see what we've got.
So what I've done here is
I've built a very small test
application called TinySketch
which contains my
TinySketch custom action.
This is a good way
to get started.
You might not want
to dive immediately
into creating an extension for
Safari because there are a lot
of moving parts, and it's
a lot better to build it
up in pieces when you can.
As you can see, we don't
have anything here yet.
And we don't actually
have any data.
And our Action button
is not wired up.
Let's go ahead and do that.
Going back to Xcode.
TinySketch Support is our
new support framework,
as Damien was talking about,
embedded frameworks earlier.
We're going to go
into our data model.
And because our application
and our extension may want
to use the same data
model, we're actually going
to provide the accessor to
the resources here, like this.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
to provide the accessor to
the resources here, like this.
This is going to iterate
through the image URLs
in my custom Extensions
documents folder.
We'll add support for it there.
And then we'll go back to the
app and we'll actually wire
up support for it
in viewDidLoad.
Get the pictures
of the cute kids.
Fix up the indentation.
And we'll build and run again.
And this time you'll see
some pictures of my children.
They're usually enjoying a
day at the park or at school
or doing something else.
Hit Reset on this.
I forgot to reset it before.
We can tap on any of these
images and in they go.
You can see right there.
Tap on any of them.
It's really cool.
All right.
Tap the Custom Action
button, and nothing comes up.
Well, that's because we
haven't wired it up yet.
As Aki was telling you
earlier, we need to set
up the UIActivity
View Controller
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
up the UIActivity
View Controller
to provide data to
the extension.
So let's go do that.
Going to go into
our Share Method.
And we're going to
add some more code.
Setting up the UIActivity
View Controller.
As you can see, as Aki
talked about earlier,
we're creating an activity item
using our selected image URL.
And now we're going
to add some code
where we're just configuring
the activity item as we go,
as Aki talked about
earlier -- oops.
We're adding a little bit
more code to actually deal
with receiving the
data items right here.
We're going to take
a look at make sure
that we were successful.
We're going to count the
number of items and we're going
to grab the item
provider, which is provided
to us by the custom action.
And then we're going to take
the image and save it back
into our image collection.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
into our image collection.
And then we're going to
redisplay that image inside
of the collection view.
And now finally, now that
we have the Activity View
Controller completely
configured,
we're going to present it.
So, let's build and run.
This time we ought to be
able to tap on the picture
of the cute kids, at least
I think they're cute.
Tap on the Action item.
And now we can see our
custom action right here.
If we tap it, up it comes.
It doesn't do much yet, because
while we're providing the data
to it, we haven't actually
added any code to consume
that data inside of the action.
Let's go do that now.
Going into our TinySketch
action,
which is our target
containing the custom action.
And we're going to go
down to viewDidLoad,
which is where we should
consume the extension item.
To add something to
uphold the input item
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
To add something to
uphold the input item
from the extension
context, and we're going
to grab the first
image item provider.
And we're going to make sure
that it has an item conforming
to our type identifier
of a type image.
And then we're actually
going to load that image.
And with the magic
of NSItemProvider,
we're going to -- it's going
to be coerced from a URL
into an item auto-magically
for you.
We actually take a
look at the signature
of these block parameters
and perform all the coercions
behind the scenes for you.
I thought that was cool.
[ Applause ]
And then we're going
to take that item,
we're going to assign it
to our image properties
so that we can hold onto it.
And then we're also going
to show it in our imageView.
Once we're done with
it, I'm scrolling past,
we're going to need to
be able to take this data
and send it back to the
host testing application.
So let's set that up.
We're going to create
an extension item.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
We're going to check that
we're not editing for the Web.
That will come later.
And we're going to attach
the newly-created image,
the flattened image, the image
containing all the adornments,
into the extension item
using NSItemProvider.
This time we're just, we're
not going to supply a URL,
but we're going to
supply an image.
And then as Aki talked
about earlier,
we're going to finish
the request
by calling
completeRequestReturningItems
and passing the extensionItem.
Let's build and run.
And let's edit.
Pulling it up and
annotating the image.
And you can see we actually have
data flowing across the wire.
We can dismiss, and
the data goes back.
That wasn't all that interesting
because I didn't
actually make any edits.
So I'll make an edit this time.
Put an appropriate label in
here, in the speech bubble.
Having fun at the park.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Having fun at the park.
Hit Done. And you'll notice now
our image has been replaced.
The data has made it back
from our custom extension
to our test application.
And we have round-tripped
a custom action on iOS.
Now, while this is cool,
I'm really here to show you
about how this works, can
work, with Safari and the Web.
So let's go ahead and
enhance this custom action
to actually pull its
data out of a live DOM
and then replace the data
in that DOM when it's done.
Let's do that now.
Going back, and stop.
And now you may have seen
some of my notes earlier
about adding Web support.
We're going to scroll back up.
And this time we're going
to add an else condition
to receive a URL from Safari.
You notice we're
going to have an else.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
You notice we're
going to have an else.
And if we have an
itemConformingToTypeIdentifier
KUTTypePropertyList, this is
important because we're going
to be sending a dictionary.
We're going to unpack the image
URL out of this dictionary.
Also set up our little
editing for Web flag to Yes.
And then we're setting up
an NSURLSessionDownloadTask
to actually perform
the download.
If the download succeeds,
we unpack the image data
and render into a UIImage.
Again, stashing the image
away in our property
and assigning it
to the imageView.
If we're unable to
download, we'll get an NSLog.
And on the other
side of it, in Done,
when we're done editing
the image,
we need to pack it back
into a property list.
And I've got some
code to do that here.
You'll see that we're going
to take the flattened image
and write out a temporary
file here.
And then we're going to actually
create a JPEG string of that
and pack that into a Web
dictionary basically creating a
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and pack that into a Web
dictionary basically creating a
data URL that Safari can
then swap back into the DOM.
So, everything's wired up there.
Let's go into our JavaScript.
As I talked about earlier, we
are expected to provide the run
and finalized functions.
And let's do that now.
Run, we're going to iterate
over all the images in the DOM.
And we're going to send them
over in the imageUrls
key to the custom action.
And we're also going
to send the base URI.
Our custom action is only going
to consume the first image,
but you know, this is
written for growth.
And then finally, when
the data comes back
from the custom action, we're
going to replace the image
by taking that encoded
data URL and shoving it
into the image using the --
or replacing the
image source property.
Save. Build and run.
Up comes our application.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Up comes our application.
We know this works.
Let's home out.
And let's to go Safari.
I happen to have a Web
server running in local host
with a picture, again,
of my kids just hanging
out at the park.
Let's see if we can
edit this image.
Hitting Annotate Image.
Up slides the custom action
with the data from Safari.
Now let's edit and provide
an appropriate speech bubble.
Hit Done. And now you can see
there's a live replacement
of the data right there in
Safari using the JavaScript
that I just showed you.
[ Applause ]
So, Safari Custom
Actions, as you just saw,
provide the flexibility
of the Web married
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
provide the flexibility
of the Web married
to the power of Extensions.
You can transform Web data
really, really easily.
And you can float a native
interface right over the top
of it, which allows you to
create these rich work flows
that were unimaginable
in previous releases.
So as Damien talked about,
Extensions are secure
by default.
They don't share data
unless you tell them to.
And they have a one-to-one
correspondence
with the hosting app.
That way you have your
own address space in which
to make your mistakes
and your action.
And the last thing we
really want to encourage you
to do is just to have
fun with this feature.
And don't think of it
as sticking new things
onto things like barnacles.
Think of it as constructing
these awesome new workflows
that you were really unable
to explore before now.
For more information,
you can talk to the man
in plaid, Jake Behrens.
Or I'd urge you to read our
excellent documentation,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Or I'd urge you to read our
excellent documentation,
The App Extension Programing
Guide at developer.apple.com.
If you've never been to
the Apple Developer Forums,
you can interact with
employees like myself,
Aki and Damien there, and
provide peer-to-peer support.
Thank you for coming to
learn about Extensions.
And we can't wait to see what
you guys do with the feature.
Thank you.