WWDC2015 Session 709

Transcript

X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
>> VIPUL PRAKASH: Hello.
Welcome to session 709.
I'm Vipul Prakash
and I'll be joined
by my colleagues Dave
Salim and Jason Douglas.
We'll give you an in-depth tour
of the new Search APIs in iOS 9.
As you saw in the Keynote, deep
links, the content in your apps,
they can now appear as tightly
integrated search results
in Spotlight.
This is tremendously exciting.
On iOS users spent most of
their time inside of apps,
this is where all the great
content is, this is really
where people live their
digital lives these days.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
where people live their
digital lives these days.
We have been interested
in connecting all
of this great content
through Search.
We didn't want to create an ad
hoc one-size-fits-all course
of indexing scheme going
inside of your apps
to grab whatever it can
and stuffs it in Spotlight.
That's an approach to indexing
that worked well for the web
because it is uniform but
it is the wrong approach
for app search.
What we wanted to do was
create an experience where you
as developers choose
exactly what is in the index,
how results appear in search
and where the user goes
to when they tap on the results.
I think we have been
able to achieve this
with search APIs keeping
them simple and lightweight.
The goal of search APIs
is to bring the content
in your apps more deeply
in a search context.
In addition to results appearing
in Spotlight they
also appear in Safari.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
in Spotlight they
also appear in Safari.
This works in a way similar to
how Spotlight suggestions work
which are produced in iOS 8.
As the user types a query
into the Safari address bar,
you have the results from the
Internet like Maps, Wikipedia,
App Store and the user can tap
on the results to jump directly
to the content they're
looking for.
In iOS 9 users will be
able to jump directly
to content inside of your apps.
This is another vector
of distribution
and exposure we provide.
Public deep links inside of
your app, will now appear
in search results for users that
don't have your app installed.
This is super exciting.
This provides a whole
new level of exposure
for your app potentially
to hundreds
of millions of iOS users.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
[Applause]
>> VIPUL PRAKASH: Let's
look at how this works.
Let's look at examples.
We invited the folks from
Airbnb to look at the APIs
and they were able to do an
integration within a few hours.
What they did, they took
reservations that users make
and the communications that
they have on the platform
and index them to
the search APIs.
Now when I look for the
reservation I made in Napa,
I put that in Spotlight,
the reservation comes
up as the first result.
It has the dates --
it is really cool.
[Applause]
>> VIPUL PRAKASH: It has
the dates, number of guests,
descriptions, and when I
tap on it it takes me inside
of the rich experience
in the app.
Another result that you will see
here, this is result number 2,
which is a message from the
host of the reservation.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
which is a message from the
host of the reservation.
As you can see, I can read
the message, it is very useful
and there is also a
call glyph on the right,
when they indexed this,
they are added data
to include the phone
number of the host
and Spotlight picked this
up, rendered a call glyph
and when it happens, it
does what you expect.
It makes the call.
So this is a really beautifully
integrated integration
to achieve search APIs.
Let's look at other examples.
Here is LinkedIn, a
professional networking app.
What's happening
in this example,
my first degree network,
all of the communications
that I have had on LinkedIn have
been added to the search index.
Now they're available in the
same way as built-in Contacts
and built-in messages
that are available.
Now search APIs, are not
limited to just content.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Now search APIs, are not
limited to just content.
One really handy use case
is to make navigation points
and features inside
of apps searchable.
As an example, our very
own Health app does this.
It indexes all the deep
links of all the sessions.
If I want to see how many steps
I have taken, I type steps
in Spotlight, get this
deep link, tap on it,
go straight to the steps screen.
[Applause]
>> VIPUL PRAKASH: I
think this is very cool.
One of the most popular
users of Spotlight Search is
to find and launch apps.
We believe in iOS
9 going straight
to deep navigation points
inside of apps is going
to be another very
popular use case.
In all these examples
the results came
from your own device index,
the deep links were put
in the device index and
they never crossed the
device boundary.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
device boundary.
What if the user does not
have your app installed?
We maintain a massive
index of deep links
in Apple's cloud servers.
We can serve results
seamlessly from this index.
We have come up with a novel
way of populating this index.
Developers, you can tag content
which is public as public
and once enough users engage
with it, it is then promoted
to the cloud index where now
it is available to all users
of your app and if there
are web links associated
with the deep links then it is
available to all users of iOS.
Now indexing and crawling apps
is nonsensical in some ways
but on the web, it is not.
If your app mirrors its content
on a website what
we can do is go out
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
on a website what
we can do is go out
and proactively go index your
website, find the deep links
and bring them in a cloud index
when they're seamlessly
available in search.
This logic of seamlessness of on
device search and cloud search,
it is very important to
the App Search story.
There is a lot of content
in apps that's private
and then content that's public,
it is not both at the same time.
What we can do with this
approach is present a seamless
search interface to the user
where the results are coming
from the device index
or the cloud index.
Let's look at examples of this.
I searched for "northern lights"
a music festival, I'm presented
with a result from the cloud
index from the EventBrite app,
I don't have this app installed,
this result has a web link.
I tap on it, it takes me to
Safari and opens the space.
You notice that EventBrite
users, the smart app banner
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
You notice that EventBrite
users, the smart app banner
which encourages me
to download the app
and experience this
content in a richer way,
which is what I do now.
Imagine if this was your
app, wouldn't it be awesome
to get your content in
front of all iOS users
in this very contextual way when
they're looking for something
that your app provides
to promote app discovery?
Here is an example where I
searched for "sprained ankle".
I don't have an app for
medical conditions on my device.
I'm offered multiple results
by App Search based on the apps
that users have previously
found interesting
or useful for this query.
I can tap on this,
install the app
and continue my exploration
inside of the app.
So how does this work?
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
App Search has three APIs
that are distinct but related.
They're designed for
enabling different use cases
of App Search.
The first one is NSUserActivity.
This is an extension
of the Handoff API
that we released in iOS 8.
When we looked at the
pattern of information access
and search one of the things
we found is that users want
to search for things
they have seen in the past.
NSUserActivity makes
searching this information
and indexing it very easy.
It sort of works like how a web
browser maintains your browsing
history and the user
activity maintains the form
of app history and
makes it searchable.
The second API is
called CoreSpotlight.
This is a very powerful way
of comprehensively indexing
any content inside of your app.
All built in apps in iOS
9, Mail, Notes, Calendar,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
All built in apps in iOS
9, Mail, Notes, Calendar,
they use this CoreSpotlight.
It gives you low level access
to the index on the device,
it sort of works like a
database API you can add
and remove items,
and it's the best way
to index any private
content that you have inside
of your app for the user.
Third API is web markup.
This is for apps that mirror
their content on a website.
You can simply markup your
web content with deep links,
annotation information,
we go out, crawl it,
put it in the cloud index and
it is then seamlessly available
to users through iOS 9 search.
Here is the Agenda for
the rest of the session.
We'll do a deep dive
into each of these APIs,
show you how they work,
and we'll look at how best
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
show you how they work,
and we'll look at how best
to adopt these APIs
in your apps.
To talk about NSUserActivity
and CoreSpotlight,
I would like to ask Dave Salim.
[Applause]
>> DAVE SALIM: I'm Dave Salim,
a developer on App Search.
We just heard about some
exciting new features
to make the content in
your apps searchable
and more discoverable.
Now we'll take a look at some
of the APIs you can adopt
to enable these features.
We'll start off by taking
a look at NSUserActivity.
A little background,
NSUserActivity was introduced
in iOS 8 to support Handoff.
Handoff is the amazing
feature that allows you
to start an activity on one
device and pick up right
where you left off on another.
NSUserActivity is a lightweight
way to capture application state
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
NSUserActivity is a lightweight
way to capture application state
which can be restored later.
Now new in iOS 9 is we're adding
the ability for NSUserActivities
to be indexed and searchable
from iOS search and Safari.
You will now be able
to associate metadata
with your activities so you
can describe your activities.
And that information
can be used to index
and provide rich search results.
When users engage with
your search results,
your app will be launched
and passed the activity
with that result.
You can continue restoring the
state bringing the user right
back to its content.
Let's take a look at an example
of how NSUserActivities
are added to the index.
Here we have a recipe app and
as a user browses a recipe
in this app, we create an
NSUserActivity capturing its
state, providing it
the information needed
to get back to this recipe.
We then describe that
activity with information
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
We then describe that
activity with information
from the recipe, the title,
description, even a thumbnail.
Then we inform the system
that this activity represents
the user's current state
and that activity is added
to the on-device index.
So as a user moves about the
app we follow the same pattern
of creating activities,
describing them,
and informing the system
that these represent
the users current state
and these activities will be
added to the on-device index.
Let's take a look at some of
the new and existing properties
on NSUserActivity used
to support App Search.
First, there is now properties
to designate an activity's
capabilities, there is one
for Handoff, by default Handoff
is enabled, if you choose not
to enable Handoff
for your users,
you can easily set
this to false.
There is one for search
which allows on-device
indexing of the activity.
There is one for Public Indexing
which we'll talk
about in a minute.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
which we'll talk
about in a minute.
Next, how do you
describe your activity?
First you give your
activity a title.
This title is used both for
indexing and for display
in the search results.
You can also provide keywords
to describe your activity,
these will also be indexed.
To fully more describe your
activity you can use a new class
we're introducing
in CoreSpotlight named
CSSearchableItemAttributeSet
which makes it easy to
describe the activity
by setting properties
and you can set this
on the contentAttributes
property of NSUserActivity.
If you prefer for your
activity to only be shown
until a certain date, you can
provide an expiration date.
Also if your activity's content
can be restored on the web
or you support Safari's
universal links,
set the webpage URL property
and the system will
appropriately launch your
activity in Safari
or in your app.
Now let's take a look at
how we create an activity.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Here we're instantiating an
instance of an NSUserActivity
and we're providing
it an activity type.
This is a unique
string that you provide,
this is the same string
you provide to the system
to let it know what activities
your app is capable of handling.
We recommend a reverse DNS-style
notation for these types
of strings to keep them unique.
Next, our activity
needs a title.
Again, the title is used
for indexing and used
as a display title in
the search results.
Next, we're setting a dictionary
on the userInfo property
containing the information we
need to recreate the state.
Next we're enabling on-device
indexing of the activity
by setting eligibleForSearch
to true,
and with this activity it
represents what the user's
current state is and we inform
the system by becomeCurrent
on the activity and the
activity will be added
to the on-device index.
What does this look
like for your users?
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
What does this look
like for your users?
Now users can be
returned search results
for activities indexed
from your apps.
Here we can see how the title
property maps to the title
in the search result and if you
provide more information using
the contentAttributes set we
can provide richer results.
So now what happens
when a user taps on one
of your search results?
A UI application delegate
method in your app is called,
the same one used for Handoff.
Here we see continueUserActivity
restorationHandler.
And we pass that activity
for the search result.
We check its activity
type and we can continue
to restore the state using
the information populated
in the user info.
So now users can tap on results
for activities from your app
and you can restore them right
back to the proper state.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and you can restore them right
back to the proper state.
NSUserActivity in App
Search is a great way
to let your users get back
to the content they
previously visited.
We wanted to take
it a step further.
So when we were designing this
feature we took a look at a lot
of apps in the App
Store and noticed a lot
of apps have content that's
available to many other users.
So we thought wouldn't it
be great for discoverability
if we could take the activities
representing public content
and make them searchable by all
users of the app not just ones
that previously visited
that content?
That's what we're doing
with NSUserActivity
and Public Indexing.
So how does this work?
As a user navigates
the public content
in your apps you create
NSUserActivities capturing
that state and you
designate them as public.
As users engage with
the search results
for these activities
the activity is shared
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
for these activities
the activity is shared
with our cloud index and
made searchable by all users
of the app, not just ones that
previously visited the content.
If your activity can
be restored on the web
and you set the webpageURL
property you can massively
increase its audience by being
searchable in iOS Safari.
So let's see how we designate
an activity as public.
We can start off with the
activity we previously saw
and we can set one
additional property.
Here we're setting
eligibleForPublicIndexing
to true and we just designated
this activity as public.
Even though we have set
this property to true,
that doesn't mean
that the contents
of the activity are immediately
shared with our cloud index.
For an activity to be shared
it first needs to appear
in search results and be
engaged with by many users.
We track these engagements using
a zero-knowledge proof method
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
We track these engagements using
a zero-knowledge proof method
which keeps the activities'
contents private
until an engagement
threshold is met.
So here in more detail as a user
engages with the search result
for an activity designated
as public a one-way hash
of the activity is sent
to our cloud index.
Not until the cloud index has
seen the same hash many times
from users is the content
of the activity shared
with our cloud index.
We took extra measure
in defining the feature this way
first to respect user privacy
and second to prevent
any user specific data
from accidentally being
shared with our cloud index.
In addition, the properties on
NSUserActivity eligibleForSearch
and eligibleForPublicIndexing
are false by default
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and for everyone here,
only designate public
on your activities that
represent content and features
that are popular among
many of your users.
Now let's take a look at some
additional benefits you get
for adopting NSUserActivity.
The first is Handoff.
So by adopting the API you've
just added support for Handoff.
If you choose, you can allow
your users to start an activity
in your app on one device
and continue it on another.
The next two are related
to new features from Siri.
As shown in the keynote, swiping
left of home presents the user
with Siri suggestions.
Along with suggested
Contacts and suggested apps
and nearby places, activities
from your app can be suggested.
Here we see a suggestion for
an upcoming calendar event
and a suggestion to
resume a podcast.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and a suggestion to
resume a podcast.
Next is the Siri
smart reminders.
Now users can ask Siri to
remind them of specific content
from your apps and when
they do the activity
from your app is
embedded in the reminder.
It is an easy way
for users to get back
to the content they want
to be reminded about.
NSUserActivity in App Search
makes it easy for your users
to get back to the content
they previously visited
and it can make the
public content
in your apps more discoverable.
Next let's talk about
the next API you can use
to make the content in
your apps searchable,
that's CoreSpotlight.
CoreSpotlight is a
brand-new framework on iOS 9.
It provides a database-like API
so you can add, update,
delete items.
The items you indexed using
CoreSpotlight will be searchable
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
The items you indexed using
CoreSpotlight will be searchable
in iOS search and it is used by
our own built-in apps, Messages,
Mail, Calendar, Notes,
you can get an idea
of what you can index
with CoreSpotlight.
Let's take a look at the API.
First we start off with
a CSSearchableItem.
This will represent the
unique item we wish to index.
Next we associate a
CSSearchableItemAttributeSet
which is an easy way to describe
the item by setting properties.
Then we pass the CSSearchItem
to the CSSearchableIndex
and the item is added
to the on-device index.
Let's see how we go
about creating an item.
First we start off
by creating the
CSSearchableItemAttributeSet.
Here we have one for an image.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Next we set a few properties
on the attribute set
to describe the item
we wish to index.
Here we're setting a
title and a description.
We'll see how the title
and the description map
to the search result in a bit.
Next we create the
CSSearchableItem
and we supply a unique
identifier,
this is an identifier
that the app can use
to reference the actual
data that we wish to index.
We can supply a domain
identifier.
This is a way of
grouping items together.
In our example here
we're indexing a photo.
We may want to group all
of our photos together
under a domain identifier
representing an album.
And we provide an attribute
set which we created
above describing the item.
Next, we index the item
calling indexSearchableItems
on the CSSearchableIndex
passing it an array
of items we wish to index.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
of items we wish to index.
What does this look
like for users?
Users can now go into iOS search
and be returned search
results things in your app
that have been indexed
using CoreSpotlight.
Here is how some of the
attributes, properties we set
on the attribute set map
to the search result.
What happens when a user taps
on one of your search results,
a UI application delegate
method in your app is called,
the same one used for Handoff
and here we see again
ContinuousUserActivity
RestorationHandler and this time
you know you're being launched
by a CoreSpotlight item
because the activity type is set
to CSSearchableItemActionType
and the unique identifier
we used
when we initialized the
searchable item is accessible
for the value
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
for the value
in CSSearchableItem
ActivityIdentifier
in the userInfo.
So now users can tap on search
results and you can return them
to the right context
for that item.
Now let's talk about
maintaining your index.
The index supports
updating items.
Over time the title of
your item may change,
the description may change, the
user updates, certain fields
of your -- representing your
item, you can update them
by calling the same method
you used to index items
by creating a CSSearchableItem
with a unique identifier
that you wish to update
associating an updated
CSSearchableItemAttributeSet
and indexing the item
calling indexSearchableItems.
Another common maintenance
routine is deleting items.
It is good practice
to delete any items
that are no longer
relevant from the index.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
that are no longer
relevant from the index.
We support multiple
ways to delete items.
First, you can delete
by identifier
so you can pass an
array of identifiers
to delete searchable
items with identifiers.
Those items will be
deleted from the index.
You can delete by
domain identifier.
You saw earlier when
creating our item,
you can use a domain identifier
to group items together.
You can pass the
identifier and the group
of items will be deleted
when calling
deleteSearchableItems
WithDomainIdentifiers.
Last, you can delete all
the items from your index
by calling
deleteAllSearchableItems
WithCompletion handler.
Now let's take a look at a
demo to see how easy it is
to add CoreSpotlight
to your apps.
Our demo app is a photo
viewer displaying a collection
of photos.
I'll show you how easy it is to
index the items by their title
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and we'll look at
the user experience
of searching and restoring.
Here we have Xcode and
the iPhone simulator.
Let's launch our app
first to get an idea
of what it looks like.
Here we have a collection
of photos,
each photo has a title below it.
There are some nice
sunsets in here.
Okay. Let's see how easy it is
to index these photos
using their title.
You go in Spotlight, and we
have a method, index items
and we'll call this when
the data model initializes
and we'll enumerate over all
of the photos in the data store
and we'll add each photo
using CoreSpotlight.
We'll start off by creating
the attribute set describing
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
We'll start off by creating
the attribute set describing
the photo.
Here we're creating a
CSSearchableItemAttributeSet
for an image.
Then we'll take the title
from the data model and set it
on the Title property
of the attribute set.
We'll also include a
description based on that title.
Next we'll create
the item itself.
Here we're passing in a unique
identifier which represents
that we can use to reference
the image in our data store.
Then we'll call
indexSearchableItems passing
in the item into the
CSSearchableIndex.
Now let's build and run this app
and here we are back in our app
and this time when our app
initialized it called index
items and enumerated all of
the photos in the collection
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
items and enumerated all of
the photos in the collection
and added each photo
to the on-device index.
Now we can go into iOS search
and I remember seeing some
nice sunsets in there.
We can see results from
our app for these sunsets
with the Title of
"sunset" in them.
Tapping on one of
the results allows us
to restore the app right back
to the proper context
for that item.
[Applause]
>> DAVE SALIM: That's
how easy it is
to index items using
CoreSpotlight.
So some additional features
in CoreSpotlight is
support for batching.
Your app may choose to index a
large number of items and wish
to break it up in batches.
The index supports saving
the State of where you are
in your batch process
and retrieving it
so that you can resume
where you left off.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
so that you can resume
where you left off.
There is also support for
a delegate and extension
so that the index can
communicate with your app
in the foreground, background,
or when it is not
currently running.
There is support for
data protection classes
so you can choose the
appropriate security policy
for the information
that you're indexing.
CoreSpotlight is an easy
to adopt database-like API
which you can use to
index any features
and content from your apps.
So now I'll hand it over
to Jason Douglas talk
about the third API, web markup.
[Applause]
>> JASON DOUGLAS: Hello.
My name is Jason Douglas.
I'm from the Siri team.
We'll talk about web markup.
You have seen two amazing ways
to make your app
contents searchable both
in NSUserActivity and
CoreSpotlight, the third set
of those APIs is web markup.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
of those APIs is web markup.
And this is for content
from the app
which is also available
on a website.
The key to making this
connection is a mobile
deep link.
A deep link is a lot
like a URL but rather
than to a webpage
it links to content,
specific content directly
inside of your app.
You saw this diagram earlier
where we're building
both an on-device index
and a cloud index
from public items
that are sourced
from the device.
Well, to call crawl content
from websites Apple now has
a web crawler called AppleBot
which is going out and
crawling app websites
for quite some time now.
And for a lot of this content,
you didn't have to do anything.
If your website supports
mobile deep links, we were able
to go ahead, discover that
content and make it searchable
from iOS search and Safari.
This content when
pulled from the website,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
This content when
pulled from the website,
we surface it directly in search
and Safari like you see here.
And when a user taps on one
of the results they're
deep linked directly
into your app seeing the content
that they saw from
search results.
You will notice there is a
link to get back to your app
which you get for free.
You don't have to
do anything special
in your app to get
this behavior.
Users simply have to tap on
that link in the top left corner
and they're taken
back to searching.
This makes it seamless
to go back and forth.
There are four simple steps
to making your apps content
searchable via the web markup.
The first step is making
sure Apple can discover
and index your website.
The second step is ensuring that
your website contains markup
for mobile deep links.
Next you want to also be sure
that your app can handle opening
those deep links and finally,
this is optional, but
highly recommended,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
this is optional, but
highly recommended,
is adding mark-up
for structured data.
We'll talk more about
what this is.
This allows your results to
look rich and engaging to users.
So the first step, making your
website discoverable by Apple.
We have been going out crawling
the web for a while looking
for sites with deep
links, the easiest way
to make sure we can find the
app's website is to use it
as either your support
or marketing URL
when you submit the
site to iTunes Connect.
When you submit your app there
is two text boxes for this.
It is very easy.
We use them as a starting
point for going out
and finding mobile deep links.
The next thing is making sure
that your website has
markup for deep links.
The most popular way, you have
probably seen this before,
are something called
Smart App Banners,
they have been around
for a while.
This is a simple
tag specifying the app
and what the deep
link URL should be.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and what the deep
link URL should be.
Users when they visit your app's
website in Safari see a banner.
If the app is not installed
it allows them to go directly
to the App Store to install it
or if it is already installed
you can click the view button
to view that content
directly inside of your app.
If you want to learn more
about Smart App Banners check
out the Safari Web Content Guide
which has more information.
In iOS 9 there is a new
feature called Universal Links
which is an even
better alternative
to Smart App Banners.
There are a few advantages
over the custom URL schemes
that you may have used in
the past for deep links.
The first of the features
is they're unique.
Multiple apps can't
claim the same custom --
they can claim the
same custom URL scheme
but with universal
links they can't collide
because we use the app website
as a way to uniquely identify.
It is secure, just because
you say that an app belongs
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
It is secure, just because
you say that an app belongs
to a certain website we
don't take it for granted.
A signed certificate is
uploaded to the website
and when the user installs your
app we check the certificate
to make sure that they're
truly linked together.
They're also flexible so users
can control the behavior whether
they prefer being within the
website or within the app,
they can easily toggle
this behavior
and choose the default
they want the most.
This means that you can
also avoid using things
like JavaScript to
try to ask the user
which they would prefer.
Another great advantage of not
using Javascript is it's one
of the types of deep
link which we have a bit
of a hard time understanding.
Using something like Universal
Links ensures we're able
to parse your deep
links correctly.
The final advantage,
they're universal.
A Universal Link is a single URL
used both for your web content
as well as your app's content.
You don't need to have two
separate URLs, one for web
and one for the deep
link content.
If using Universal Links
we'll still need to know
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
If using Universal Links
we'll still need to know
about your deep links
via your web markup,
so you want to use
Smart App Banners
or an alternative
supported standard.
This also ensures that if a user
doesn't have the app installed
they're still able to go
discover your app, and find it,
install it since Universal Links
is a feature that's enabled once
your app is installed.
To learn more, I highly
recommend that you check
out the session Seamless
Linking to Your App
which will be tomorrow.
So the next step or the other
alternatives we also support
for deep linking, one of
those is Twitter cards
which also uses  tags
and Facebook app links.
In addition to just supporting
smart app banners we support
these as well and
we'll add support
for other popular standards
that may come along later.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Finally, making sure that
your app can open deep links.
In the past and you may
be familiar with this,
you may have already
done this for your app,
is making sure you've
implemented openURL.
Here is a simple
example of using openURL.
In this case in our
example we have a URL
which can view profiles and
it simply takes a profile ID.
We simply parse the URL
with NSURL components
and pass the profile ID
from the query parameters
to the view controller.
This is a very simple
example of how easy it is
to support deep linking
for your app.
So now I want to talk
about structured data
which I alluded to earlier.
Structured data is something in
addition to the mobile deep link
which lets us understand
and parse information
about your results.
Parsing freeform web
text is not an easy task
and this helps us
out significantly.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
It also ensures that
your results can stand
out to the user rather than
just having text you can have an
image, ratings, and
other key things
that are called out
in the result.
It can also help
improve your ranking
because we know this result is
rich and engaging to the user,
we may tend to surface it
more than other results.
So one standard for
adding this type
of structured data
is open graph.
Open graph you can
specify an image, an audio,
or even a video file
with your result.
You simply place this
markup on your webpage
and we'll then associate it
with the deep link we also
find on the same page.
Another very common format
for structured data is
schema.org that's been
around for a while and we're
adding support for it rapidly.
A common schema.org
is AggregateRatings
for ratings and reviews.
This example we're showing
adding the AggregateRating
markup with the schema.org micro
data and we support RDFA as well
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
markup with the schema.org micro
data and we support RDFA as well
as JSON-LD depending on
what you're comfortable
with implementing on your
website we'll support that
and parse that to
be all-inclusive.
Now let's dissect an example
of how this works
in the real world.
Here is an Airbnb
result and the first
and most obvious thing is
the title which we parsed
with standard HTML  tags
and then we go a step further.
There is a price
per night shown.
We're able to do this because
on this webpage the Offer schema
was added and it is
associated with a hotel.
We're able to parse that price
and price currency and show
that directly in the result.
We're also able to show number
of reviews via AggregateRating
from schema.org and
a rating value
which we can render as stars.
Finally, the image has been
provided using open graph's
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Finally, the image has been
provided using open graph's
image tag.
So now to talk about
some of the schemas
which we support
from schema.org.
The first is AggregateRatings,
another is offers, prices,
or if you have a consumer
shopping site, you offer things
for sale and as we saw
with hotels maybe, booking,
price range in case you
don't have a fixed price
but rather a range of prices,
interactionCount for things
like likes, views, comments,
also interactionCount will
be superseded in schema.org
by the new Action set of schemas
which we'll support that as well
if you're thinking of migrating
from interactionCount
to Actions.
Organization for things
such as businesses,
you can also have
addresses associated
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
you can also have
addresses associated
with these contact information,
like telephone numbers
and we'll see a bit later how
that can help make your
result more richer.
Recipes, this one
is self-explanatory.
SearchActions, in case your
website has a landing page
for searches by marking it
up in a special way we're also
then able to treat your result
in a little bit differently,
we know it is a landing
page for search.
Finally, ImageObjects.
This is for nothing
else but images.
So now I want to also
talk about Actions.
Actions are a way
using structured data
to allow your results
to become actionable.
There's a set of actions that
we'll be supporting on launch,
the first of which is dialing a
phone number, getting directions
to an address and playing
an audio or video file.
For the first of these
examples dialing a phone number.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
For the first of these
examples dialing a phone number.
As we saw before with the
Organization schema you can add
something like a
telephone as a property.
When we see this telephone
number we can parse it
and put a call button
directly in the result.
This makes your result
actionable now.
Another is using an
AudioObject schema
with AudioObject you can
specify a URL to an audio file
and the users will play it from
the result, they don't have
to deep link into the app
and play it from there.
Finally, getting directions.
Providing a postal address
we can then allow the user
to navigate directly
without having
to deep link into the app.
This has the benefit of
making your result much more
potentially interactive
to the user.
If a user sees a list
of results and sees
that they can get
directly to the action
that they want they may be more
likely to click on your result.
So we have shown a lot of
different schemas, markups,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So we have shown a lot of
different schemas, markups,
we'll add testing
tools later this year
for testing the markup
on your website.
This will be available on the
developer site later this year.
We'll be having testing tools
for both the deep link markup
and the structured data markup.
Finally, I just wanted to
call out a few more resources
as you're adding both
either mobile deep links
or structured data to the
website, do check out some
of these resources,
provide a lot of information
and again we'll hopefully
help get you started.
Thanks.
[Applause]
>> JASON DOUGLAS: Now I
hand it back to Vipul.
>> VIPUL PRAKASH: So those
are three simple, powerful,
and flexible ways of making
your app searchable on iOS 9.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and flexible ways of making
your app searchable on iOS 9.
Which one should you use?
It is a function of the
nature of your content as well
as the kind of search experience
that you're trying to create.
CoreSpotlight is for
private data which is indexed
on the device and you
can use CoreSpotlight
to comprehensively
index data in your app.
If you're building or you
have a social networking app
for instance and you wanted
to index all of the messages
that the user has
sent and received,
CoreSpotlight is the
right tool for the job.
Web markup is for apps that
mirror their content on the web.
If you do that, you should
absolutely adopt that markup,
because it creates tremendous
opportunity for your app
and content to be
discovered by users
that don't have your app yet.
Use NSUserActivity for both
public and private content
as well as for indexing
navigation points inside
of your app.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Now there is another flavor of
NSUserActivity that Dave touched
on which is Public Indexing.
This is really powerful.
If your app has a large
number of items, millions
or even hundreds of millions
you can use Public Indexing
to essentially move those
items and have them indexed
in Apple's cloud index.
These APIs while distinct,
they're really designed
to work together.
In the same app for the
same content you can adopt
multiple APIs.
The only thing to remember
is for items that are indexed
from multiple places, you
want to connect these items
by giving them the same ID.
If you're using all three APIs,
the obvious ID is the URL.
You would set CSSearchableItems
UniqueIdentifier to the URL
and NSUserActivity related to
the unique identifier as well
as the webpage URL to the URL.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
as the webpage URL to the URL.
if you use NSUserActivity
and CoreSpotlight together,
which is a very common
pattern, you just want to ensure
that they have the same ID.
Now this help search
deduplicate results
but it also helps
items rank stronger.
Let's talk a bit about ranking.
The magic of search really is
to provide the most relevant
result based on a few keystrokes
of input from the user.
We have done a lot of work here.
Relevance and ranking,
is a complex function
of matching the query
to the items.
Most of this is completely
transparent
to you developers
and to our users.
However, an important component
of ranking is interaction
with the content and results.
What you can do is help ensure
that search can capture all
of these interactions.
The three types of
interactions that we care about,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
The three types of
interactions that we care about,
one is URL popularity,
this is applicable
to content that's indexed via
web markup and this is based
on any quality indications
you add through schemas
like ratings, likes, reviews, as
well as the position of the URL
in the web's link graph.
The URL improved this
position based on popularity
and this happens automatically.
The second interaction
is the frequency of views
that your content
receives inside of the app.
Really the only way to capture
this is through NSUserActivity
which is why we encourage that
you always adopt NSUserActivity.
The third and most
important form
of interaction is engagement
with search results.
This is when a search result
is presented to the user,
whether they tap on the
result, or if they're satisfied
by the description in the result
which we call silent engagement
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
by the description in the result
which we call silent engagement
and we capture through
session analytics.
A quick checklist of what
to do to optimize relevance.
First, continue to
do what you're doing,
make great apps and
great content.
Really, the search
system is designed to find
and bubble up the best content.
You can help by ensuring
that we can capture all
of the interactions,
using NSUserActivity,
use NSUserActivity with Public
Indexing where applicable
because now we can
capture the engagements
with your content
across the user base.
Adopt schema mark-up
for interactions counts
and quality indications.
Finally follow user
interface guidelines I'll talk
about in a moment.
The flip side of bubbling
up great content is suppressing
content that is not relevant.
We have done a lot of
work on this as well.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
We have done a lot of
work on this as well.
If malicious or a poorly
considered app is spamming the
index, we will downrank
their results
or suppress them entirely.
The most important sort
of measure here is
engagements-to-shown ratios,
the amount of times your
results are shown in search
and how many times are engaged.
Low engagements will
be suppressed
and high engagement ratios
will help your content rank
up higher in search.
One of the best ways to
get high engagements is
to follow a few simple
user guidelines.
One of the things
you will notice
in iOS 9 is search results even
from built-in apps are very
descriptive and rich and this is
because we found that richer
results get more engagements.
A few things here,
provide a thumbnail,
provide a well-structured
readable description, ratings
and actions where applicable.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and actions where applicable.
The image in the thumbnail
is really powerful,
it gives the user a
sense of what they'll see
when they tap on the result.
Provide a good thumbnail.
Provide key information that
the user is looking for.
For instance, if the user is
looking for a booking number,
a reference number, include
it in the description
so that the search task can be
completed right there in search.
Now another powerful provision
is support for keywords.
In all the APIs you can
provide a number of keywords
that will then trigger
search results.
Category keywords tend
to be very, very useful.
Users may search type
"ticket" when searching
for a specific ticket
rather than typing the name,
they may type "recipe"
when looking for protein.
This creates a magical,
intelligent experience
for users.
Other types of keywords
that are useful are synonyms
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Other types of keywords
that are useful are synonyms
and abbreviations, users want
to type as little as possible,
so instead of searching "San
Francisco Giants" they may type
"sf giants" and you can provide
this recall through keywords.
Just as important, is
behavior on launch.
This is what happens when
the user taps on the result.
You want to take them
straight to the content
that they're looking for without
interstitials, without pop-ups,
without screens,
without spending a lot
of time loading that screen.
This is, in fact, so
important that we measure
and estimate the time from tap
to the content shown and use it
as a factor in ranking.
Here is two examples.
One is right.
One is wrong.
Let me show you what to do.
[Pause]
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
[Laughter]
>> VIPUL PRAKASH: I'm sure none
of you have apps that do that!
[Applause]
>> VIPUL PRAKASH: It is
frustrating for the user
and you want to find ways
of completely avoiding it.
Let's talk about
what to do next.
You can, of course, index your
app content comprehensively,
these are certain things that
we have found to be very useful
in the search context.
Any content that the user
has viewed in the past
that you can capture with
NSUserActivity and content
that the user may have
created or curated by starring
or creating lists inside of
the app, people want to go back
to things that they
have done in the past.
Navigation points and
features are very, very useful.
New messages, new content
arriving that the user may want
to look for inside of your
app, you would also want
to look for this in search.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
to look for this in search.
You can get creative.
You can proactively index items
that the user may
want to look at.
Say I bought a ticket to a music
festival, I may be interested
in other music festivals
in the area.
You can use your app
analytics to figure
out interesting content
and put it in the index.
Really the APIs are
flexible and allow you
to create novel search
experiences.
The possibilities here
are really limitless.
So summary, just
let's take a look
of things we covered
in the session.
We introduced fast on-device
and cloud architecture that used
for indexing deep links
and for App Search.
We showed you three APIs
and deep linking methods
which you can adopt.
We showed you how great
content, descriptive results,
and fast content-to-result
interaction can provide engaging
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and fast content-to-result
interaction can provide engaging
search experience.
Provisions built in the APIs
that are taking your content
and showing it to iOS
users around the world.
We have some more information.
If you're looking for
details, you can head
to our search API documentation
portal, we have forums
for more interactive help.
I'd recommend going to the
session on seamless linking
that Jason introduced.
We also have two labs, one
that starts immediately
after I stop talking.
You should come and join us
there and another one on Friday
and we would love to see some
integrations with search APIs
on Friday that you
made during WWDC.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
on Friday that you
made during WWDC.
That's it for App Search.
[Applause]