Transcript
>> Hello. My name is Stewart
Montgomery and in this session
we'll discuss how your app can
use new APIs and iOS 11 to
filter out unwanted SMS and MMS
messages and help users avoid an
increasing nuisance.
Before we dive in, let's take a
look at what happens when we
receive an unwanted SMS message
in iOS 10.3.
Here we see what is pretty
obviously an unwanted, spam SMS.
Messages like this are annoying
for users since they play a tone
or vibrate just like a normal
message and distract from
whatever you're doing.
And if I launch messages it's
right there at the top, mixed in
with my real messages and
cluttering up the list.
Unfortunately some iPhone users
get a lot of these messages and
they would love to have a way to
filter them out.
With iOS 11 we are introducing
APIs to allow your app to
analyze the sender and content
of any SMS or MMS message from
an unknown sender and attempt to
filter out those which are
unsolicited.
Let me show you how it works.
Here on my iPhone running iOS 11
I'll launch the New Messages
app.
And since I've already installed
a Message Filter app extension
and enabled it in settings I now
see a second tab called SMS
Junk.
And if I get a message which the
app believes is junk it'll only
appear under that tab.
Ah, there's a new message now.
Let's see what it is.
This is the same message as
before, but now it no longer
appears in the regular list with
my known contacts and doesn't
distract me with a sound or
notification.
And if I tap to read it I can
see from the label at the bottom
that it was marked as junk by an
app called Filter It.
Now there are a few reasons why
we decided to add this
functionality.
Unwanted messages, which include
any unsolicited or spam messages
a user receives, have been an
increasing nuisance for users in
the recent years.
But beyond the annoyance, what's
more concerning is that often
these messages are phishing
attempts and include links which
may harm users.
So of course we want to prevent
these messages from being
delivered whenever possible.
Now there's an important
difference worth noting between
iMessage and SMS and MMS
messages.
For iMessage we offer the report
as Junk Service on-device since
those messages are encrypted end
to end and delivered over the
iMessage network.
But we don't have the ability to
do this for SMS or MMSs since
they are delivered directly from
a wireless carrier to a user's
device.
So the filtering of these
messages must happen locally
rather than on a centralized
server, and that's where these
new APIs come in.
Finally, we've heard that many
of you have developed expertise
in analyzing messages and
detecting which ones are
unwanted and we're excited to
invite apps to help with this
task.
So for the remainder of this
session I'll cover a few areas
in detail.
First I'll walk through the
details of what we call Message
Filter Extensions and show how
they work.
Next I'll talk about some
important considerations around
privacy since these extensions
come with some special rules.
Then I'll talk about how an
extension can check with a
network-backed service, which
some apps might find useful and
I'll show a couple of demos
along the way through creating
one of these extensions in
Xcode.
So let's get started.
The way this works is using
something we call Message Filter
Extensions.
So let's talk about those
in-depth.
As the name implies, this is a
new app extension type, which
your app can include.
The APIs for it are in a new
framework in iOS-11 called
Identity Look Up.
Now once a user has installed an
app with one of these
extensions, to begin using it
they must first enable the
extension in Messages Settings.
Only one extension can be
enabled at a time, or if the
user wants to disable the
feature they can choose None.
And once it's enabled that
extension is invoked every time
an SMS or MMS message is
received from an unknown sender.
And there is some other criteria
used when deciding when to send
a message to the extension,
which I'll discuss in a few
minutes.
Let's walk through a diagram
showing the overall flow.
When a message is received by
the phone it starts in the
Messages app and if it is an SMS
or MMS and it's from a sender
which isn't in the recipient's
contacts, then the extension,
which the user selected in
Settings will be launched and
will be passed the messages
sender and body via an object
called IL Message Filter Query
Request, which is part of the
identity look-up framework.
When the extension receives this
it begins examining the message,
looking at the sender or the
body of the message, or both,
and it might check against a
known list of bad phone numbers
or it could look for a
suspicious looking web link in
the body, whatever is
appropriate.
Ultimately the extension has to
form a response using an object
called IL Message Filter Query
Response, describing whether to
allow or filter the message, and
it sends this back to the
Messages app.
And once it receives a response,
Messages will either alert the
user normally or suppress the
notification and move the
message thread to the Junk tab.
So that's a basic overview of
how it works and before we go on
I'd like to touch on a very
important topic, and that's user
privacy.
We thought a lot about how to
maintain the strong level of
privacy Apple customers expect,
but also allow them to enable
this if they want to confront
this persistent problem of
unwanted messages.
So there are a few special rules
that extensions must comply with
when using this API.
The first rule is that a message
recipient's phone number is
never sent to an extension, only
the sender's phone number or
email address is included, since
that's all that should be
required to make a decision
about whether to filter a
message.
Another key rule is that a
message filter extension can
never export the contents of
messages outside its container,
and these extensions have some
additional restrictions because
of this.
They cannot write to files
shared with their containing app
and they cannot perform network
operations.
The reason for this is that,
although some messages may be
unwanted junk, others may be
legitimate and be sent from
someone who's just not on the
recipient's contacts yet.
So then it's imperative that all
the messages be kept private and
never exported in any way that
reveals more about the recipient
than what is contained in the
message itself.
Although they can't perform
networking themselves, it is
possible for these extensions to
defer a request to their server
indirectly, and when an
extension requests to defer iOS
will make a web request on the
extension's behalf in a secure
way.
We'll see an example of that
later.
The main thing to keep in mind
is that your extension should
never export messages outside of
its container to maintain user
privacy.
Now there is some specific
criteria that the Messages app
uses to decide whether or not to
send a given message to an
extension.
First, this feature is only used
with SMS and MMS messages and
never with iMessage.
As I mentioned earlier, unwanted
iMessages are handled using a
different mechanism, so this
only applies to SMS and MMS.
As I've mentioned a few times,
only unknown senders, or those
which are not in the recipient's
contacts are actually sent to
the extension for analysis.
If a sender is in Contacts we
assume that that recipient knows
the sender and wants to receive
messages from them.
That also means that if a
message is ever miscategorized
as junk then the user can add
that sender to their contacts to
ensure they aren't filtered out
in the future.
Also, if a user is exchanging
messages with someone who is not
in their Contacts and they reply
to that thread multiple times,
then we will stop sending any
subsequent messages in that
thread to the extension.
Or, if the user replies multiple
times to a thread, which is
already marked as junk, that
thread will be restored to the
non-junk tab.
Responding multiple times is
interpreted as a signal from the
recipient that they really do
want to be communicating with
the sender.
So all of this criteria doesn't
directly affect the API, but it
is something you, as a developer
of one of these extensions,
should be aware of when testing
or troubleshooting your app.
So now I'd like to show a demo
in Xcode of how you can create a
message filter extension.
Here we have an app I've made
called Filter It and I'd like to
add a message filter extension.
The first thing I need to do is
add a new target.
And I'll choose the new message
filter template for iOS and I'll
give it a name.
I've got a new file added to my
project now called
messagefiltereextension.swift.
Now let's take a look at that.
The first thing we see is a
method called Handle Query
Request with Context and this is
called on our extension so that
it can examine the incoming
message and return a response
using the completion handler.
Now the template is structured
to first attempt an offline
check using this method, called
Offline Action for Query Request
and it returns an action which
is either allow, filter, or
none.
So for this demo what we need to
do is customize this offline
action helper method.
Let's look at what it does
currently.
Right now it always returns
none, but I'll replace that with
some simple logic to always
filter if the message contains
the word junk.
In a real extension this could
do something more sophisticated,
but this works for now and
that's all it takes to create a
simple, offline only, message
filter extension.
Now although some apps may be
able to do most or all of their
checking offline, other apps may
find it useful to check with a
network server whether to filter
messages or not.
So next I'd like to talk about
network deferral.
The best way to show how network
deferral works is with another
diagram.
As before, when a message is
received it starts in the
Messages app and is sent to the
chosen extension, but this time
the extension chooses to defer
this request to its network
server, whose URL is specified
in its info.plist.
So it tells Messages to defer
and Messages then makes a JSON
request to that server URL.
The server then examines the
Messages contents inside the
JSON request and can respond in
any format it wants and that
response is handed right back to
the extension.
And from here, the extension
reads the response from the
server and finally returns an IL
message filter query response
back to Messages.
There are a few restrictions to
be aware of when using network
deferral.
First, the deferred network
requests contain no personally
identifiable information about
the recipient of the message.
The network URL is hardcoded
statically into the extension's
info.plist file under a key
called IL Message Filter
Extension Network URL.
So it cannot vary between
requests or for individual
users.
All URLs must be secure https
and the server must be
configured such that it doesn't
require any app transport
security, or ATS, overrides
since there's no way to
configure them.
Also, this feature requires that
both your app and server use the
associated domains, or Apple App
Site Association Feature, which
is something you may be familiar
with if you've adopted other iOS
features like App Links or
Shared Web Credentials.
And for more information, see
the session; Seamless Linking to
Your App from WWDC2015.
And the last restriction to be
aware of is that any cookies
that the webserver attempt to
set will be ignored to maintain
privacy.
The request made to the network
server is formatted using JSON
and includes the same things
which are in the IL Message
Filter Query Request Object
including the message sender,
which is a phone number or email
address, and the message body.
The request also includes the
version of your app, which is
the CF Bundle Version Key from
the app's info.plist.
This may be useful in cases you
have -- in case you have
released several versions of
your app with different
capabilities and need to format
the response to insure that
particular version of the app
can understand it.
And we include the version of
the JSON request format itself,
which is currently one.
Now, unlike the request format,
the response format is entirely
up to your app to define and
doesn't necessarily have to be
JSON.
The response body is passed back
to your extension to parse, so
there are not requirements about
its format.
And here's a quick look at the
JSON request format, and you can
see it includes all of the info
I mentioned.
Now, let's go back to my Filter
It app and add network deferral
capability to the extension.
If we take another look at the
Handle Query Request method from
earlier we can now see that,
after the offline check is
performed, if the action it
returned was none and it falls
into this case of the switch
statement, we assume that this
query request could not be
handled using only an offline
check and actually needs to
consult our network server to
get an answer.
To do this our code calls the
defer query requests to network
method on our extension context
and this causes a network
request to be made on our
extension's behalf, which will
call this completion block
asynchronously when it
completes.
Inside the completion block, if
there was a response from the
network and there wasn't an
error, we use another helper
method called Action for Network
Response to translate it into an
action.
Let's jump to that method and
see what it does.
Just like with the offline check
helper method this method
defaults to returning none, but
let's customize that to parse
the response from our server.
I'll assume the server returns
JSON, although it doesn't have
to, and I'll use the new
Foundation Decoding APIs in
Swift 4 to decode the response.
I'll paste some code I've
already written to do this, but
let's briefly walk through it.
I first define a struct, which
describes the JSON format that
my server returns, then I create
a JSON decoder instance and I
use it to decode the data from
the network as an instance of
the struct.
Finally I return the action,
which was decoded, and store it
in the struct.
And if there were any errors I
handle them below.
And return the default response
of none.
And just like that we've added
network deferral support to our
message filter app extension.
So it now supports both offline
and network checking of incoming
messages.
So that's how your app can help
filter unwanted messages using
the new message filter
extensions and identity look up
framework in iOS 11.
We've tried to strike a balance
between user privacy and solving
a very pressing need our users
have.
And the result is a powerful new
API your app can use, but it is
subject to some special rules
you need to be aware of.
So please, download the new STK,
check out the new identity look
up framework, and try making a
message filter extension filter
today.
For more information see this
link for the Sessions page on
the official WWDC site.
We do have a few related
sessions and this year's
conference to mention.
For more information about
privacy practices on our
platforms see the Privacy and
Your Apps session happening in
the Executive Ballroom on
Tuesday at 11:20.
And for more info about the new
foundation and coding APIs I
showed in my demo, see the new
-- see the What's New in
Foundation session in Hall Two
on Wednesday at 11.
Also, WWDC2015 had an excellent
session called Seamless Linking
to Your App, which walked
through the associated domains
feature which message filter
extensions require when using
network deferral.
So check out that session in the
archive for details about how to
implement it in your app and on
the server.
Thanks so much for watching.