WWDC2016 Session 223

Transcript

X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
[ Music ]
[ Applause ]
>> Hello everyone.
Welcome. My name is
Vipul Ved Prakash.
I work on the Siri and Search at
Apple, and today I'll be joined
by my colleague John
Hornkvist, and we are going
to show you what's
new in Search APIs
and how you can make
the most of them.
As you know, Spotlight has been
becoming a more powerful search
tool in recent releases of iOS.
In iOS 8 we introduced
ability to search
through internet sources, like
App Store, Maps and Wikipedia.
And then in iOS 9 we
significantly increased the
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
And then in iOS 9 we
significantly increased the
scope and introduced
sources like music,
currency conversions,
live sports scores,
web, and much, much more.
Perhaps most importantly,
we introduced simple
and powerful ways in which we
could make your apps content
searchable through Spotlight.
So in iOS 10, today we
are announcing extensions
of these Search APIs that will
provide even deeper search
of your app's content.
We've also made Spotlight easier
to use and easier to find.
So let me start by showing
you some examples of things
that we've done to Spotlight.
Spotlight is now present
in Notification Center,
which makes it a lot
faster to get to.
Let's look at an example here.
User receives a message
with a question in it.
Very typical thing that happens.
And they want to run
a search to answer it.
Instead of having to quit
messages and go to Home screen,
pull down Spotlight, they can
simply drag Spotlight on top
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
pull down Spotlight, they can
simply drag Spotlight on top
of messages, run their
search and find the answer.
It's super convenient.
We've also introduced Spotlight
on Lock screen and added support
to quickly preview
results with 3D Touch.
That's see what this looks like.
Suppose you're having an
argument with a friend
on where Steph Curry was born.
You can go to the Lock screen,
run your search,
see the result, and now you can
use 3D Touch to preview these --
and find the one you
were looking for,
which is Wikipedia in this case.
Really, really convenient.
And, of course, when you're in
Lock screen, the only results
that you present are public
results from the internet.
Now, another neat
feature that we've added
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Now, another neat
feature that we've added
to Spotlight is query
suggestions, which appear
in the quick type
area above the keyword
when you type in the query.
Let's look at an example.
Here the user searches
for banjo, and we know,
based on past queries,
that two common completions
for banjo are banjo
chords and banjo tuner.
So if the result that they
are looking for is not already
on the screen, they can pick one
of these and get to the answer.
This will make a lot
of queries a lot faster
to execute in Spotlight.
Now, we've done many
more improvements.
We've improved spell correction
for the app launch use case.
We've improved support
for Japanese
and Chinese scripts on device.
And we've made relevant,
in general, much better.
And all these changes,
of course,
apply to content that's
inserted while Search APIs.
So now let's turn our
attention to Search APIs.
Now, as many of you
may remember,
we introduced three
Search APIs in iOS 9.
There's CoreSpotlight,
which allows you
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
There's CoreSpotlight,
which allows you
to insert user specific or
user created content right
on Spotlight's device index.
We also introduced
NSUserActivity,
a searchable version of this,
that provides an easy way
to index everything that the
user has seen inside of your app
so that they can search for
it and get back to it easily.
And finally, we introduced
a Universal Links index,
which is an index on the
server side consisting of links
that we have discovered
by crawling app websites.
And we designed these
APIs to work in concert,
so whether the results
are coming from the device
or the server, they
blend together
to provide a seamless experience
of searching your app.
I'm happy to announce
that over 50,000 apps
in the App Store have
an app that Search APIs.
It's really incredible.
[ Applause ]
I think something really
neat to see is how the users
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
I think something really
neat to see is how the users
who Search APIs have improved
day to day interactions
that our customers are having
with your app and
iOS as a whole.
I have some of my
favorite examples
that I'd like to show you.
This is actually
a recent example.
I'm planning a trip to
Maui, Hawaii, this weekend,
and I've been planning this trip
over the last month on and off.
Now, various pieces
of information
like hotel reservations and
tickets, they are somewhere
on my device, and typically
the way I find them is I go
to the app or the website,
or dig through the
e-mail to find them.
But with Search APIs, this is
what my experience looks like.
The apps that I've used to
make these reservations,
have added salient
pieces of information
to Spotlight's index
using CoreSpotlight.
So I can see my flight ticket, I
can see the hotel that I booked
through Hotel Tonight,
and a couple dinner reservations
I made on Open Table.
I can find more information
by tapping on these results,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
I can find more information
by tapping on these results,
like what is my check-in time
or address of one of
these restaurants.
And you'll also notice
that I have some results
here from Pinterest.
I've been using the
Pinterest app
to explore the things I
can do when I'm in Maui.
Now, Pinterest uses
NSUserActivity
to index everything that I have
looked at inside of the app,
and I can tap on one
of these results.
And continue my exploration.
So this is a really
fantastic experience
for doing travel search and
to find all the information
that I know exists on my device.
And what we've seen is, in
general, indexing content
that your users want to get back
to works really, really well.
Another example I want to
share is around contacts.
Contacts is one of the most
popular search use cases
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Contacts is one of the most
popular search use cases
in Spotlight.
Here I'm searching
for buddy Allen.
I see his contacts, which
is a native iPhone contact,
but I also see his Skype profile
and a couple of conversations
that I'm having with him
in Yahoo Mail and WhatsApp,
and I can easily get back
to these conversations.
And this is so powerful because
it allows me to get this sort
of centralized view of people
that I know, and allows me
to have conversations with them
in the channel of their choice.
Now, what we have seen is
that users are naturally going
to Spotlight for all the things
that Spotlight supports
natively,
and if you have content in these
categories, it's a great way
to broaden that search
experience,
as well as drive
engagements to your app.
Now, finally, let's
look at an example
of Universal Links index.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
of Universal Links index.
wikiHow is a high quality how-to
resource that has hundreds
of thousands of articles
on a variety of subjects.
Here I'm searching for a
critical first aid question,
how to perform CPR, and
Spotlight presents a bevy
of results from web, YouTube, as
well as a section from wikiHow.
Now, wikiHow uses
Universal Links
and they have a popular app.
So we have indexed their
articles on the server
and we present them for
appropriate queries.
It's pretty comprehensive
results that by clicking on one
of these results, I go
to this very clearly
illustrated article.
The index of wikiHow has really
added a whole new capability
to Spotlight, which is what
makes Search APIs so powerful.
With that, let's start
looking at what's new
in Search APIs in iOS 10.
The first feature we are adding
is called Continue Search
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
The first feature we are adding
is called Continue Search
in App.
And for a lot of queries
users want to look
at a larger result set, or
you may not have been able
to add the right result to
Spotlight for various reasons.
So for now apps have an option
to present a search in App Punch
Out on the top right corner
of their section,
as you can see here.
And tapping on this takes
the user into your app
with the search query.
So you can continue the search
query inside of the app.
It's super cool.
The second feature we are
introducing is CoreSpotlight
Search API.
It's a slightly overloaded name,
but there what's happening is
that you're adding
all these items
to Spotlight's search
index by CoreSpotlight API.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
to Spotlight's search
index by CoreSpotlight API.
Wouldn't it be awesome if
you could use this index
to power search inside
your own app?
I think it would be.
And this is exactly what
CoreSpotlight Search API allows
you to do.
The third feature we
are introducing is a way
to estimate popularity of
deep links using differential
privacy, the technology that
was mentioned in the keynote.
We will discuss this more later
when we talk about ranking.
And finally, by popular
request we've added a feature
to our web markup preview tool
that lets you visually inspect
your schematized results before
you put them in a
Universal Links index.
And in addition to these we
made many, many enhancements
to Search APIs based
on your feedback.
And now I will invite John to
do an introduction of these APIs
and do deeper dives
into these new features.
[ Applause ]
>> Let's start talking about
how to leverage the Search APIs.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
In the next half hour I'm
going to cover some things
that you have to do for your
application to work great
with Spotlight, some
things you'd want to do
to give users a great
experience, and some new APIs
that we've added to
make it easier for you
to accomplish what you need.
So roughly in the order that
you need to implement this,
I'm going to talk about
getting your content
into the index available
to Spotlight
and keeping it up to date.
Presentation and
user experience,
launching into your app,
whether for restoring content
or for search continuation,
and then I'll talk
about the new Search API
before Vipul comes back
to give you an overview ranking.
So we have three technologies
that together cover
most use cases.
CoreSpotlight, for all that
you have on the device.
NSUserActivity for app history.
And Universal Links
with web markup
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
And Universal Links
with web markup
for public online content.
Of course, you can use
all these three together.
As an example, consider
a recipe application.
It provides an interface to
a vast collection of recipes.
There are hosts on the website,
so deep links are a great fit.
The app may also have
a favorites feature,
and for that you would
use CoreSpotlight.
Your users will want
to get back to content
that they've looked at,
so for the best experience
you use app history
through NSUserActivity as well.
All right.
For those of you who
haven't gone off to lunch,
instead of conducting Search
APIs, let's dive in and talk
about indexing content, and
we'll start with CoreSpotlight.
CoreSpotlight is an API
for indexing on iOS.
It's on device and
supports file protection.
So you can index the
user's private content.
Your app is in charge and
you decide what you want
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Your app is in charge and
you decide what you want
to add to the index.
You can index all that your app
has to offer, be it favorites
and bookmarks, messages and
e-mails, documents, images,
music, videos, game
levels, jump-off points
in your app, and much more.
So there are two
basic operations.
First you need to add
items to the index.
To do this you create a
CSSearchableItemAttributeSet.
This contains the metadata and
other content for your items.
Then you set at least
one attribute.
In this case we're
setting the display name.
You create a searchable
item using the AttributeSet.
You use the uniqueIdentifier,
which Spotlight will use
to identify this item
for any future operations
and which is also used when we
launch your application later.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and which is also used when we
launch your application later.
And the domainIdentifier,
which allows you to set
and shared property
across many items
which can later use
for deletion.
And then you ask CoreSpotlight
to add the item to the index.
When your callback is called,
the item is then safely
committed to storage
or an error will be passed
back to the callback log.
For deleting content
there are three APIs.
You can delete a specific
item by its identifier.
For example, if the
user deletes a document.
You can also delete groups of
items by the domainIdentifier
that I mentioned earlier.
This is great if the user
signs out of an account,
ends a subscription, or
something of that sort.
Finally, you can delete all
content for your application.
This is useful if
you have something
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
This is useful if
you have something
like an incompatible
version change and you need
to clear the index and
start from scratch.
And this is also
called by the system
if your app gets uninstalled.
Now let's go over
some best practices
and some more advanced
scenarios.
We'll cover registering
as an index delegate,
using CoreSpotlight client state
to handle progressive indexing,
some performance considerations
and creating a CoreSpotlight
extension.
You want to register
as an index delegate
because this lets
Spotlight initiate indexing
when your app is first
launched on the system,
perhaps after restoring
a backup,
when the user installs
your app, and sometimes
for disaster recovery.
It also lets Spotlight
reach out and request
that you reindex
a particular item.
This is commonly because
you've set an expiration date
and Spotlight wants to check
that the item has truly expired.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So to be an index
delegate you need
to implement the
CSSearchableIndexDelegate
protocol.
This has two required methods.
ReindexAllSearchableItems,
which is called when you need
to add everything to the index.
If you track indexing
of individual items
in your own database, you want
to clear the indexing state
when this call is received,
unless you're using
client state,
as we'll discuss in a moment.
The second call is reindex
items with identifiers.
When this is called,
you should look up items
that Spotlight is requesting
and add them to the index
or delete them as appropriate.
For both methods you call the
acknowledgmentHandler only
when you're completely done
and receive the last callback
for any work you
issued to CoreSpotlight.
This ensures that we know that
your content is fully indexed
and that we don't have to
call you in the future.
If you don't call this,
we might call you again,
and if you call it early,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and if you call it early,
you might not get a
chance to finish indexing.
For some applications we found
that it's more convenient
to use client state than to try
to manage the indexing callbacks
with your own database
transactions.
The client state provides
an asynchronous way
of keeping your content
in Spotlight in sync.
Because it's asynchronous you
essentially have an eventual
consistency model, but you
can keep Spotlight up to date
with your own database
without any redundant work.
The client state is an
opaque token that is stored
in the Spotlight index that
you update as you index
and then fetch back when
your app is launched again.
Typically, the easiest way to
do this is to put annotations
in your own database, for
example, the sequence number
that you then as a client
state with Spotlight.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
that you then as a client
state with Spotlight.
After relaunch you would
check this client state
and if it doesn't match
with your expectations,
you would index any
item in your database
with a sequence number higher
than the one you fetched
back to Spotlight.
Another approach is to use
the sequence number as a way
of knowing where to start
the journal playback.
This can be really, really
good for keeping your content
up to date, asynchronously,
and for making sure
that you don't use
too much power.
So to work with client
state you need
to create a named
index instance.
The name lets us know what
client state you want.
You can only get the client
state for your own application,
but you may have more than one
database that you're indexing
and then you would
use a separate name
for each of those databases.
With the index you
begin an index batch.
You'd add searchable
items as usual.
The completionHandler is not
particularly important here
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
The completionHandler is not
particularly important here
because you'll get a completion
when you finish the batch.
You compute your picked state
and you pass that to Spotlight
when you finish the batch.
And here you do need
to pay attention
to the completionHandler.
When your app next launches,
you fetch the client state,
and you figure out what
operations you need to run
to bring Spotlight in
sync with your state.
Because the client state
is kept with the batch
into the index, it
ensures integrity.
You can replay exactly
operations that you need
to bring them both into sync.
So when your app next starts,
you create an index
instance using the same name,
you fetch the client state.
And this is an asynchronous
call.
So in the callback you
deal with any errors
and check whether the state you
got back is what you desired.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
If not, you call the method
to bring them up to date.
Now let's talk about some
performance considerations.
Spotlight is really, really fast
so you want to minimize overhead
on your side to make sure
your app can keep up.
You want to optimize any
access to files or databases,
and pay careful attention
to your memory use.
Do notice that each call to
CoreSpotlight has a cost,
so pass batches of items instead
of single items when possible.
And just making batches as small
as ten items will still
reduce the IPC overhead
by an order of magnitude.
Since your app will be
performing indexing while the
user is using it, make sure that
you don't block the main thread.
And finally, to avoid
interfering with UI run
on a background thread.
All right.
Let us talk about
CoreSpotlight extensions.
The extension can index
when your app isn't running.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
The extension can index
when your app isn't running.
This lets you catch up after
a disaster recovers in backup,
or when your icons expire.
Spotlight can reach out to your
extension instead of reaching
out to your application,
which is great
because your application
might not be running.
The interface to the
extension is the same
as for the index delegate.
So if you can factor your code
so that the index delegate
is separate, it's really easy
to implement the
extension as well.
To make content available
to your extension you
can use share.groups.
To find out more about this,
take a look at last
year's session
on App Extension Best Practices.
Next let's look at
keeping content up to date.
As I mentioned, you can
use expiration dates
to keep stale content from
accumulating in the index.
CoreSpotlight will call your
app around the expiration date
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and you can update
the expiration time
or update the data for
the item if you need to.
If you need to get new
content into Spotlight,
you can use background fetch.
This will allow your app to
launch in the background,
letting you take care of adding
content and getting it indexed.
Finally, if you have more of
a push model with irregular
or infrequent updates, you can
use silent remote notifications
to let your server tell you
that updating is necessary.
To find out more about
using background fetch
and silent remote notification
take a look at the What's New
with Multitasking
session from WWDC 2013.
That's it for indexing
with CoreSpotlight.
Next let's talk about app
history with NSUserActivity.
NSUserActivity was introduced
for Handoff in iOS 8.
It lets you create
a representation
of your application's current
state that can be passed
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
of your application's current
state that can be passed
to another device,
and since iOS 9 stored
in the Spotlight
index for app history.
You submit the activities as
the user is browsing in your app
for content that the
user may remember
and want to get back to.
So the question you
want to ask yourself
about when the user activity
should be indexed is simply will
the user want to
get back to this.
If the same item might be
indexed with CoreSpotlight,
then the answer is
almost always yes.
The 2014 session on Adopting
Handoff has great information
on how to use NSUserActivity.
To make NSUserActivity available
for search you need to mark it
as searchable and add index
of the metadata using
CSSearchableItemAttributeSet.
You mark it as searchable
by setting the eligibleForSearch
property to true.
You can also mark NSUserActivity
as eligible for public indexing
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
You can also mark NSUserActivity
as eligible for public indexing
to make it a candidate
for the online index,
as Vipul will get
into a bit later.
This done, it will show up as
a result in Spotlight Search
and your application can
revisit the user activity
when the user selects
it in Spotlight.
Your users just have to
remember a single keyword
from what they saw to be
able to get right back
to the content in
your application.
All right.
So, as I said, you can use
CoreSpotlight and NSUserActivity
for the very same content.
The difference is
that NSUserActivity reflects
what the user has done
in your application.
CoreSpotlight is about
what your app has.
So if you use both, you
can relate NSUserActivity
to the CSSearchableItem for the
same content to help ranking
and avoid duplication
of the results.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and avoid duplication
of the results.
This is done by setting the
relatedUniqueIdentifier property
in the AttributeSet to
the uniqueIdentifier
of the CSSearchableItem that
you want to relate it to.
This also ties the lifetime
of the NSUserActivity
to the CSSearchableItem,
protecting you
against leaving data
on the device
after the user has
deleted private content.
However, not all data is
private and managed by the user.
For example, for the recipe
application that we talked
about earlier, you may want
to relate NSUserActivity
to a possible CSSearchableItem,
an item that doesn't exist yet,
something that the user might
make a favorite in the future.
If you were to use the
relatedUniqueIdentifier,
CoreSpotlight would immediately
delete the NSUserActivity
as you try to add it because the
related item doesn't exist yet.
To solve this, we're
adding a new property,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
To solve this, we're
adding a new property,
weakRelatedUniqueIdentifier
which lets you bind weakly
to the CSSearchableItem.
It can exist before the
CSSearchableItem and will remain
when the CSSearchableItem
is deleted.
As for the
relatedUniqueIdentifier,
you simply have to set the
property in the AttributeSet
for the NSUserActivity.
Now, the downside is that when
the CSSearchableItem is deleted,
the NSUserActivity remains.
So if you have concerns
about the lifetime
of the searchable item
or the NSUserActivity,
you do need to delete
NSUserActivity yourself.
Fortunately, in iOS 10
we're making this possible.
We're adding domainIdentifiers
to NSUserActivities.
It's part of the
CSSearchableItemAttributeSet
and it allows you to
delete NSUserActivities
by the domainIdentifier,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
just like you can
for CSSearchableItem.
If you use both
CSSearchableItems
and NSUserActivities,
it's a good idea
to use the same
domainIdentifier.
So that's indexing with
NSUserActivity in iOS 10.
Next let's talk about
Universal Links and web markup.
The content driving your
app may live on the web,
not locally inside the app.
If this content is public,
you can use web markup
to make a searchable
for Spotlight via the
Universal Link index.
This is perfect for content
hosted on the website
and available in your
app, and a great solution
when your content is too
large to fit on the device.
Because your content has
a presence on the web,
results can be displayed to
users that don't have your app,
which lets you reach new
users and drive app installs.
Finally, these results can
be shown in both Spotlight
and Safari, which makes
your content available
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and Safari, which makes
your content available
to even more users.
So to implement indexing
with Universal Links you
need to allow indexing.
Allow Applebot to call your
website [inaudible] text,
and let Apple Note
into the site.
Specify the call URL
when submitting the
app to the App Store.
For deep links we strongly
recommend Universal Links.
For this you need to
implement dual authentication
for the website and the app.
Implement the
continueUserActivity method
in your app delegate to ensure
that your app handles deep links
when the user selects
the results.
Markup your content with
schema.org or Open Graph
to provide a rich display for
attributes in your content.
And use the Search
API validation tool
to test deep links, markup,
title, description, and more.
Take a look at the Introducing
Search APIs Session from 2015
and the Developer documentation
on Universal Links.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and the Developer documentation
on Universal Links.
So these are the schemas
that we support today,
and we plan some
more in the future.
Pay attention to interaction
count and aggregate rating.
These are very useful for
ranking results for your app.
Apple provides a test tool at
search.developer.apple.com,
that you should consult
if you implement the
deep links for your app.
It now displays a visual
representation of your result,
including supported
schema [inaudible] markup.
The information from the
validation tool can help you
visualize pieces of information
that the Applebot web
crawler has indexed,
including the title,
description, markup and URL.
Now you've seen how to index
content using our three APIs,
and there's a good
chance that you'll want
to use them all together
even for the same content.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So the most important
thing to remember
when using multiple APIs is
to link items representing
the same content together
across the APIs.
By setting NSUserActivity's
relatedUniqueIdentifier
to CoreSpotlight's
uniqueIdentifier,
CoreSpotlight's content URL, and
NSUserActivity's webpage URL,
to the URL of the webpage, you
tell Spotlight that all records
in the index represent
the same item,
which allows search
to de-duplicate.
It also provides strength
to the ranking of items.
All right.
Now we've covered three ways of
getting content into the index;
CoreSpotlight for content,
NSUserActivity for app history,
and Universal Links
with web markup
for public content
available through the web.
Next let's talk about how
to present this information
to the user.
To get a great presentation
in Spotlight you want
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
To get a great presentation
in Spotlight you want
to set a good thumbnail.
By default Spotlight
will use your app icon
which makes it hard to
distinguish results at a glance.
This doesn't matter much
if you're just going
to get a single result,
but if you get multiple,
it makes a big difference.
Just as important
as the thumbnail,
and perhaps more, is the title.
A good title is not
just great visually,
it's also what users most
frequently search on.
After the thumbnail and
the title you'll want
to set other fields that are
suitable for your content.
A description is great when
available, as is rating,
rating description, date
attributes for items
that are time bound, such as
travel reservations, dates,
reminders, events, and so forth.
For documents, general
metadata, such as file size
and page count, are
helpful as well.
If you set the right content
type for your content,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Spotlight can also do a
better job of displaying it.
Let's look at some examples.
The Hotel Tonight app makes
good use of the thumbnail,
showing an easily identifiable
landmark, as well as a title
and informative description.
You can get the same with web
markup by setting the og:image,
og:title and og:description.
Open Table uses the title
description, the rating,
with rating description
attributes,
to let the user get
great information
to choose the right result
before jumping into the app.
In making the same attributes
available in web markup,
you ensure that the user will
get a consistent experience
whether they're getting the
results from the web crawler
or from the local device index.
A great user experience is not
just about the presentation,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
A great user experience is not
just about the presentation,
but also about what data you
make available for search.
Setting attributes that
the user can understand
and remember makes your
content quickly accessible.
Conversely, setting misleading
attributes in metadata
or stuffing content and keywords
with dictionary words will
cause your results to show
up frequently, but
rarely be selected,
which will annoy the user
and have a strong negative
affect on your ranking.
Another aspect of a great user
experience is being able to get
to the salient part
of the result
in as few steps as possible.
Enabling quick actions,
like directions and calling,
has significant value
for your users.
Finally, when a user
selects an item,
you want to launch directly
to it as quickly as you can
and without interstitials
or multistep builds
that impede the user.
Let's look at some examples.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Redfin's app provides
attractive looking results,
and by setting the latitude
and longitude attributes
and enabling navigation, the app
lets the user punch out directly
to Maps to go look
at a property.
Similarly, here is a great
use of the call action result.
You can get the same by setting
the phone number's property
and support phone call
properties for Spotlight.
On seeing the result, a user
that already is familiar
with the location
can immediately call
and reserve a table.
For web markup you can
accomplish the same
by using the postal address
and telephone schemas.
Next let's talk about
launching your application.
For both CoreSpotlight and app
history we use NSUserActivity
to restore the state.
Your app delegate
will get called
with continueUserActivity.
You examine the NSUserActivity's
activity type
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
You examine the NSUserActivity's
activity type
and the user info
dictionary if necessary.
If you're being launched
because the user selected the
CSSearchableItem in Spotlight,
the activity type will be
CSSearchableItemActionType,
and you retrieve the identifier
from the user info dictionary
by using CSSearchable
ItemActivityIdentifier.
The 2014 session Introducing
Handoff goes into further detail
on how to launch
NSUserActivity's review
and activity type.
For Universal Links, once
again, we use NSUserActivity.
So your app delegate
will get called
with continueUserActivity.
As usual, you examine the
NSUserActivity's activity type,
which will be
NSUserActivityTypeBrowsingWeb.
You parse the URL
and take the user
to the part indicated
by the URL.
That's launching.
Now let's look at a new feature.
In iOS 10 we've added a
feature to allow the user
to continue search
right in your app.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
to continue search
right in your app.
Conduct human results
for Spotlight
to display will show an
affordance, and you should opt
for Spotlight Search
continuation
so the user can go
directly to the app.
If you already support
search, it's trivial to adopt.
It lets you leverage your
customized search interface
and has been widely adopted
by our internal apps.
It's another great way of
getting increased engagement
with your application.
The user is taken
directly from Spotlight
into the search experience
that you already have.
Now
[ Applause ]
Thank you.
To support this in your
own application you need
to add a key to your
info.plist which declares
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
to add a key to your
info.plist which declares
that you support the feature.
Your app delegate will get
called with a new activity type,
CSQueryContinuationActionType,
and the new query string will
be passed in the user info
and a CSSearchQueryString key.
At this point you can
invoke your own search UI
with the same query string,
letting the user continue
the search in your app.
To avoid confusion, it's
usually a good idea to make sure
that your search results
are somewhat consistent
with Spotlight.
Since Spotlight is based on
prefix search, we recommend
that you use similar
search rules.
And if you can't support prefix
search, consider taking the user
to a completion interface.
Another way of ensuring
consistent with Spotlight is
to use CoreSpotlight's
own Search API,
which is the approach taken by
many of our own internal apps.
CoreSpotlight Search API
makes it easy for you
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
CoreSpotlight Search API
makes it easy for you
to implement search
of your own data.
It uses the data that you
already provided CoreSpotlight
for Spotlight Search.
It writes consistency
with the rest of the US,
and is already used by many
of our own applications,
including Mail, Messages
and Notes.
It avoids overhead
so you don't have
to maintain an additional
search index,
as it's the same index
used by Spotlight.
The index provides full
metadata and content search.
If all your content
is on the device,
CoreSpotlight can be a
complete solution for search.
If you have a mixture of
on-device and online content,
you can combine queries
and merger cells,
getting responsiveness from
CoreSpotlight on the device
and completeness from
your online index.
Mail uses CoreSpotlight
not just for search,
but also to create
search suggestions
and to make complex queries.
This is a great way to
take advantage of the power
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
This is a great way to
take advantage of the power
of CoreSpotlight Search
while keeping the user
interface simple.
A CoreSpotlight Search
API has created data
that you've given Spotlight,
but your data is protected
from other applications.
The query engine is
fast and scalable
so you get excellent
responsiveness in your app.
The query syntax allows
complex, full inquiries,
as well as range searches,
numerical and date searches,
and a set of powerful
text message features.
For those of you
that are familiar
with the metadata
framework on Mac OS,
this syntax will
feel very familiar.
Here's an overview of the most
commonly used search operations.
As you can see, CoreSpotlight
supports a full range
of search comparators, as well
as the Boolean operators
AND, OR and NOT.
In addition, CoreSpotlight lets
you customize string matching
to suit your needs.
If you want case insensitive
search, you add the c flag.
If you want to ignore
diacritics, such as umlauts,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
if they are not important
in the current language,
you use the d flag.
And if you want to match on
words within a field instead
of anchoring your searches at
the beginning of the field,
you specify the w flag.
This, by the way, is
implied for text content.
And if you want multiple words
to be dealt with individually,
then you pass the t flag and the
query string will be tokenized.
So let's look at an example.
We're implementing
a search function
that takes the user
query as input.
We make sure to cancel any
currently running query
so that we don't have multiple
queries running concurrently,
as this will slow
down the new query.
Because we're dealing with
user input, we make sure
to escape the query string.
We use the double star syntax to
create a query that will match
on either content or metadata.
The escape user input is
tasked as a search string,
and we had the cdw
and t operators,
which creates a case
insensitive, locale aware,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
which creates a case
insensitive, locale aware,
diacritics insensitive, word
matching, tokenized search.
With a query string we
create a query object.
We request the display
name to be fetched,
and this will be available
in the AttributeSet
of the searchable
items returned.
We set the foundItemsHandler,
which will receive batches
of searchable items if there
are no results for your query.
And a completionHandler
will get called once,
either with an error or when the
query has finished successfully.
In our completionHandler
we can opt it
to display finish any
processing, and so forth.
And all that remains
is to start the query.
CoreSpotlight will call
the handlers with results
and then call the
completionHandler.
So let's put this into practice.
We've built an application.
There is a simple
picture gathering.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
It already supports
indexing in CoreSpotlight
and it already supports Search.
So the first thing we're
going to do is to add support
for Continue Search in App.
This is very simple.
We go to the app's info.plist
and we enable
CoreSpotlightContinuation.
That done, we go
to the app delegate
and in our user activity
continuationHandler we add
support for Continue
Search in App.
As you can see, the
activity type is a
CSQueryContinuationActionType
and we get the search query
by inspecting the user info
for the CSSearchQueryString.
We can then activate
our view controller
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
We can then activate
our view controller
with the search query.
So let's see what
this looks like.
All right.
Here we have our picture
gathering, and we can pull
down Spotlight and search for
the word "snow," which happens
to be popular in this gallery.
So we get two results and a
search in app continuation.
So click search in app and
I'm taken into my application.
[ Applause ]
Now you might observe that I
had two results in Spotlight,
but only one result in my app,
and this is because I'm not
using CoreSpotlight Search.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and this is because I'm not
using CoreSpotlight Search.
The search is a simple
prefix search.
So let's fix that.
In our view controller
we have a search method.
I'm going to remove the
simple search implementation
that we already had and start
implementing CoreSpotlight.
First we want to add a variable.
A query object that will keep
the current search query.
We want to cancel the
currently running query,
escape the query
string and create --
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
escape the query
string and create --
create a query object and, of
course, create a query string.
Oops.
All right.
So now we have a query string,
a query, and then we need
to set the foundItemsHandler.
The foundItemsHandler creates
displayables and appends those
to our list of results.
Now, since the query's
getting CSSearchableItems back,
I need to implement an adapter
for taking my CSSearchableItems
and creating something that
I can actually display.
So let's look at what
that would look like.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So let's look at what
that would look like.
So I have what I call
a lazy picture object.
What this does is
it gets the data
that our query made
available to us and uses
that whenever possible.
When that's not enough,
it goes to our database
and gets the picture
object for this identifier,
stores that away for further
use, and then returns that,
and that lets it return
all the other properties
that are not available
from the database.
By doing this lazily we ensure
that we can display our results
without having to go
back to our database,
which is great for performance.
Let's implement our
completionHandler.
In the completionHandler
we sort the results,
jump over to the main queue to
make the results displayable,
and then we call our
table view update.
With that, all that remains
is to call the start.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
And let's see what
that looks like.
All right.
Now since this is using
CoreSpotlight Search
and I've made the query very
forgiving, I should be able
to type any word that I see
here, for example, river,
and I find anything
that has river either
in the metadata or
in the content.
All right.
That is how easy it is
to implement search
with CoreSpotlight.
And with that, I'd like
to invite Vipul back
to talk about ranking.
[ Applause ]
>> Thanks, John.
That was a fantastic
overview of Search APIs.
As you are thinking about
implementing these APIs
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
As you are thinking about
implementing these APIs
in your app, it helps to know
how Spotlight ranks result
and how you can positively
influence ranking, so let's look
at ranking in Spotlight.
Now, Spotlight's goal's
prime directive is
to present the best results
for your query in order,
and this order is determined
with a set of factors.
The two most important
factors are engagement ratio
and content popularity.
Let's talk about
engagement ratio first.
Engagement ratio really is a
measure of how often results
from your apps are being
selected by the user
when they are presented,
and Spotlight maintains
three flavors
of these engagement ratios.
One is maintained on the
device and two on the server.
On the device the engagement
ratio measures essentially
users' personalized
interactions with your app.
The server will maintain an
engagement ratio per query,
which is based on every query
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
which is based on every query
for which your results have
been shown in the past,
and using this Spotlight can
uprank or downrank results
for a particular query.
And then finally there's a
global engagement ratio that's
based on all interaction of all
users that have happened in past
with your app, and this is used
when there's no query level
engagement ratio available.
A couple important
things to remember here.
One, you don't get
penalized if the results
for a particular query have
not been shown to the user
and they're sort of under
the fold, under the keyboard.
The best practice here really
is to use keywords, titles,
descriptions that really
clearly describe the content
that you're indexing
and will take the user
to the same content.
The second important thing
is content popularity.
In general, items
that are more popular,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
In general, items
that are more popular,
tend to be ranked higher.
And Spotlight can
understand the popularity
of items in three ways.
If you use Universal Links, they
will automatically go and look
at the structure
of the web and pull
out a reputation for your link.
You don't have to
do anything there.
All you have to do is
implement Universal Links.
If an item has an
associated NSUserActivity,
Spotlight can track how often
the user is viewing this item
on the device and then
use that in ranking.
Now, if you use both
Universal Links
and public NSUserActivities,
which are eligible
for public indexing, iOS 10
can now estimate how often the
entire iOS population has
viewed a link in your app,
and this is used in ranking.
And this is done for
the new provision
of differential privacy.
Let me show you how that works.
So in this example Yelp has
adopted NSUserActivities
that are eligible
for public indexing,
as well as Universal Links.
When a user encounters
the deep link from Yelp,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
When a user encounters
the deep link from Yelp,
iOS computer hash adds some
noise to it and takes a fragment
and then sends this
fragment to Apple servers.
And by itself this
fragment is useless.
It doesn't contain
any information.
But once thousands of users
have reported lots of fragments,
Apple servers are
able to recover hashes
that have been viewed thousands
of times without knowing
which users reported
which hash fragments.
So this really provides a
way to determine something
like aggregate behavior of
users without doing anything
about individual
behavior of any user.
Once popular deep links
are discovered in this way,
they start getting ranked
higher in Spotlight.
So this is very cool.
So let's have a quick look at
best practices for ranking.
John showed many best practices
for indexing your app's
content well with CoreSpotlight.
You should follow these because
they will provide a consistent
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
You should follow these because
they will provide a consistent
search experience for your app's
content, and then users will go
to Spotlight to search
for stuff in your app,
which will then impact
engagement ratios.
If you have app content,
allow Apple to index them
by using Universal Links and
this will expose your app
to users that don't yet have it.
So you know content
popularity is important,
so link both Universal Links
and CoreSpotlight items
to NSUserActivities.
We introduced
weakRelatedUniqueIdentifier this
year to help you do
that more easily.
And also, I'll repeat this
because absolutely important
that the title, description
and keywords
of items you're indexing
are related to the deep link
that that item would go to.
Pay attention to
your presentation.
We know that well-presented
links have higher
engagement rates.
And finally, when appropriate,
implement Continue Search in App
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
And finally, when appropriate,
implement Continue Search in App
as it allows the user
to complete a search
task inside of your app.
So in conclusion, key takeaways,
I think Spotlight is the new
universal search for iOS,
and in iOS 10 we made it more
functional and more accessible.
And users are increasingly
expecting well behaved apps
to be searchable via Spotlight.
So if you haven't adopted
these APIs, we would recommend
that you consider doing so for
the general release of iOS 10.
And if you've already
implemented Search APIs,
then we would recommend
adopting new provisions
that have been introduced today.
Some related sessions,
an excellent session
on Proactive Suggestions
tomorrow that a lot
of you should go to
because what you're doing
in search applies
pretty much directly
to Proactive Suggestions.
We also had a fantastic session
yesterday on SiriKit APIs.
You can find this on
the conference website.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
You can find this on
the conference website.
And we recommend the session
on privacy from Wednesday
that covers differential
privacy in more detail.
And like all other talks,
this will be posted online
at the first URL, and we
also post all Search API
documentation on
search.developer.apple.com.
And that's all I have.
Enjoy the rest of the WWDC.
[ Applause ]