WWDC2014 Session 231

Transcript

X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
>> Hi. Welcome to the
Advanced CloudKit session.
[ Cheering & Applause ]
Thank you.
My name is Jacob Farkas.
I'm a Software Engineer
at Apple and one
of the designers of
the CloudKit API.
And my colleagues and I
have put a lot of hard work
into the CloudKit API.
I'm really excited to talk
to you guys about it today.
So, let's jump right into it.
So, CloudKit is this thing
that we've built on top
of our iCloud servers.
We've built a lot of iCloud
services, and what we're doing
with CloudKit is exposing those
database servers that we use
at Apple to all of
you developers.
So, we're actually using this.
This is something that we
use in the new iCloud Photos
and iCloud Drive feature
that we're introducing,
and we're building all of
that on top of CloudKit.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and we're building all of
that on top of CloudKit.
If all of this is unfamiliar to
you, you might want to go back
and check out the
Introducing CloudKit session.
It was given on Tuesday and
there should be videos online.
So, we're going to
jump right into this.
What we're going to learn today,
we're going to over the CloudKit
private database, which is a way
for you store private user
information in iCloud.
We're going to talk about
modeling your data in CloudKit.
We're going to talk about
advanced record manipulation
and different ways of saving
records to the server.
We're going to go over how to
handle notifications reliably,
if you miss a push, what to do.
And finally, we're going to
go over the iCloud Dashboard
which is a web-based interface
for managing your
CloudKit application.
So, let's start off by looking
at the CloudKit API
really quick.
We designed the CloudKit API
to be highly asynchronous.
Everything has a callback,
nothing is synchronous.
And we did this because all
of these requests are
going out over the network.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
of these requests are
going out over the network.
When you get the network
involved, anything can happen,
you know, the server might
not be responding quickly,
there might be a bad
network connection.
We don't want to block the UI
and cause a bad user experience.
And to help do this, we've used
NSOperation almost everywhere
in our API.
I say almost everywhere
because if you went
to the introductory session,
you remember seeing what we
called the Convenience API.
And the Convenience API is our
way of helping you get started
with CloudKit really
quickly and simply.
All of these APIs are, you
know, single calls that work
on one record at a time.
So, in this case, we're
fetching one record
and it's asynchronous.
We get that one record back
in our completionHandler.
But, as you use CloudKit
more, you might find
that you need some
additional functionality.
And that's where the
NSOperation-based API comes in.
So, what we've got here is
the CKFetchRecordsOperation.
And this is an NSOperation
that does the equivalent
of that convenience
API we just saw.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
of that convenience
API we just saw.
We give you a lot
more functionality
when you use the
NSOperation-based API.
So, you can see here that our
initializer takes an array
of record IDs.
You can fetch a whole batch
of records all at once.
We also give you more
feedback on what's happening.
We've got a completion
block for each record,
and we also give you progress
as we download those
records from the server.
And finally, these operations
give you more knobs and levers
to tweak what is returned
and what the operation does.
In this case, we've got
a desiredKeys property
that lets you specify
what key should come back
on the records that you fetch.
So, since all of this
is built on NSOperation,
let's do a quick overview
of what the NSOperation
class looks like.
What we've got in NSOperation
is a completionBlock
and a cancel call so that
you can manage the life cycle
of your operation.
We've got a couple of variables
that tell you some state
about the operation.
And NSOperations
have dependencies,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
And NSOperations
have dependencies,
so you can link two
NSOperations together.
Once you have an NSOperation,
you're going to want to start
that operation, and you
do that by adding it
to an NSOperationQueue.
When you have an
NSOperationQueue,
you can also manage the life
cycle of that operation queue.
You can suspend it
and resume it,
and you can cancel
the operations in it.
So, if you go to look at
our NSOperation-based API,
you might just see this
big list of a bunch
of different operations.
It's kind of overwhelming
and confusing.
The best way to think
about this API is to think
about the objects that
you want to deal with.
If you remember from
the introductory talk,
CloudKits-based objects
are records and zones
and subscriptions, and you'll
see that up here we have a fetch
and a modify operation
for all of those items.
One of the really cool
things about NSOperation is
that is has dependencies.
So, you can set a dependency
between two operations
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So, you can set a dependency
between two operations
and the second operation
won't fire
until the first operation
is completed.
This is really handy
with CloudKit if you want
to do something like fetch a
record, add a property to it,
and save that record
back to the server.
You can make the
FetchRecordsOperation,
make the modify operation at the
same time, set up a dependency,
and when the fetch completes,
you can put that data
in the modify operation, and
it'll start automatically.
These dependencies also
work between queues.
So, even though CloudKit has
its own internal operation queue
that you can use to run
operations, you're welcome
to create your own
NSOperation queue,
and you can then
manage its life cycle.
You can stop operations or
suspend them or cancel them.
One tip with using
NSOperation though is
that the NSOperation-based class
has a completionBlock on it.
This completionBlock ends
up firing asynchronously
with dependencies.
So, if you've set
up dependencies
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So, if you've set
up dependencies
and you're using the
NSOperation's completion block,
they might happen
at the same time,
and that data you
were trying to funnel
into the next operation won't
get there in time, you know,
that operation has
already started.
So, what we've done with
the CloudKit API is we have
CloudKit-specific
completion blocks.
And if we look at what we had
for that fetchRecords operation,
we have a
fetchRecordsCompletionBlock.
You'll see this pattern on
all of our NSOperations.
And these completion
blocks hand back all
of the information
you needed to know
about the operation
that just ran.
In this case, we've got the
errors for that operation,
and we've also got the
records that were fetched.
And finally, NSOperations can
have their own priorities.
So, you can set background
operations and have them run
at a really low priority
and keep your UI responsive.
So, when we were
designing CloudKit,
one of the things we noticed is
that there's two general
classes of applications.
There's one class of application
that stores a whole bunch
of data up on the server, and
when you use the application,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
of data up on the server, and
when you use the application,
it's just presenting
the view of that data.
It downloads it on demand,
displays it to the user,
and then, you know,
tosses it out because it's,
the real copy's on the server.
But there's another
class of application
that has just a little
bit of data, but you want
that same data on
all of your clients.
So, if you remember from
the introductory talk,
we talked about what we
called Big Data, Little Phone.
That's that case where we have
a lot of data on the server.
It's not all going to fit on
one phone, and you download it
and view it on demand.
So, you can see these clients
are downloading records,
viewing them, and the truth
lives up on the server.
However, there's another
class of application,
where it's a small
amount of data.
It lives on one client, but you
want it on all your clients.
An example of this
is an application
that manages your receipts.
So, every time you buy
something, you take a picture
of the receipt, and you want
that information on your phone,
in your iPad, in your Mac,
and you want them all to be
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
in your iPad, in your Mac,
and you want them all to be
up to date with the
same receipts.
So, what we've done
to help you solve
that is we've got something
in CloudKit that does that.
If you remember, we have, every
application gets a container.
In every container is a
public and private database.
And just as a refresher,
that private database is
actually one private database
for every user.
Inside of those databases,
we then subdivide
them down into zones.
So, both the public and the
private database have a default
zone in them.
But we've also given you
what we call custom zones,
and these custom zones allow us
to give you some extra features
that we can't provide
in the default zones.
You can create these custom
zones and use the new features.
Let's go over a couple of them.
The first feature
is atomic commits.
So, CloudKit has
relationships between records.
And if you start using CloudKit
and using these relationships,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
And if you start using CloudKit
and using these relationships,
you're going to build up an
object graph, and you're going
to realize you want
consistency in your data.
If, you know, you might have
an object graph that you want
to commit all at once.
And if some of those
things don't get committed,
the data on the server
doesn't make sense.
On the public database,
we can't guarantee
that because there might
be thousands or millions
of users hitting
the same database.
So, there's no way to lock the
database while we, you know,
commit your very
special records.
But in the private database,
you only have one user.
It's the current user's account.
And in that case, we can provide
you things like atomic commits.
So, with atomic commits,
these batch operations
in the NSOperation API will
succeed or fail as a whole.
So, if any record had a problem,
you will get back a
CKErrorPartialFailure.
Inside of that partial
failure error, you're going
to see a user, the
userInfo dictionary is going
to have this
CKPartialErrorsByItemID key.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
to have this
CKPartialErrorsByItemID key.
And that's going to be a
dictionary of record IDs
to errors for each
of those records.
And some of those
errors are going
to be the real failures
that you care about.
These are the reasons that
the atomic commit failed.
But, you know, the rest
of the records failed
because it was atomic commit,
and we need to let you know
that they failed,
so you're also going
to see
CKErrorBatchRequestFailed.
That's just a way of
saying that something else
in this batch failed,
and, you know,
this wasn't the real problem, it
was that other record in here.
Another great feature
that we give you
with custom zones
are delta downloads.
So, delta downloads are a
way for that second class
of application to be
possible in CloudKit.
You can download only the
changes that were made
in that zone, and you can
cache them all locally.
So, what does that look like?
Well, we've got our Mac here,
and we've got our iPhone,
and we've got a custom zone.
So, let's step through a
delta download really quick.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So, let's step through a
delta download really quick.
We've got an orange
record and a green record
over on the Mac here, and we
want them on the phone as well.
So, we're going to first
upload those both into CloudKit
in our custom zone, and
then our iPhone is going
to perform a delta
download to get up to date.
So, these delta downloads
are based
on what we call change tokens.
And a change token is a
way of tracking the state
of the server the last
time you talked to it.
So, this phone has never
talked to the server,
and what it's going to do is
send up a nil change token.
And that's a way of saying,
"I've never talked to you,
just send me all the
records in the zone."
So, the server is the going
to take that nil change token,
send down an, the orange
record and green record.
The phone is going to save
them into its local database.
And then the server is going to
send down a new change token.
In this case, it's change
token A, and that means that,
you know, the records you
have are all from state A.
If the phone came back
again with change tag A,
the server will go, "Well,
I don't have anything
for you, A is good enough.
There's nothing to download."
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
There's nothing to download."
But let's say the Mac
comes along and it creates
that purple record, and it
deletes the green record.
And it uploads those
changes to the server.
The server is going to track
the changes, so it'll note
that that green record
was deleted.
It'll note that there's
a new purple record.
And now, when the iPhone
comes back with change tag A,
the server goes, "All right,
well, we're are at B now.
That's farther than A.
Here's a delete of
the green record
and here's a new purple record
that happened while, you know,
since the last time we talked."
Then it sends back
that new change token
and everyone is up to date.
So, you can use this
delta download
to implement an offline
cache in your application.
If you want to do
that, there's a couple
of steps your app
should to take.
This is kind of an outline
of a basic state machine
for every time you talk
to the server and you want
to do a delta download.
The first thing you're going
to want to do in your app is
to track the local changes.
You're going to want
to make a change table,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
You're going to want
to make a change table,
and every time the user makes
an edit in your application,
you want to write that down.
You're going to want to do this
because you might be offline
when the user makes the changes,
and you're going to need
to replay all those
changes back to the server
when you can talk to it.
Then, you're going to need
to send all those
changes up to the server.
You want to do that before
you fetch the changes
because someone else might
have changed the record
in the meantime, and you need
to resolve these conflicts.
So, we'll go over conflict
handling in just a little bit,
but just keep in mind
that that's an important
step in this process here.
Finally, you're ready to
do the delta download.
This is the point
where you're going
to call
CKFetchRecordChangesOperation.
The server is going to send
you back updates and deletes
and modifies and adds of
records, and you're going
to save those into
your local database.
Finally, the server is going
to send you back a
new change token.
And that's the change
token you want to save
so that the next time
you talk to the server,
you can get only the records
that have changed
and not everything.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
that have changed
and not everything.
So, one of the other features
we give you with custom zones
and delta downloads
are zone subscriptions.
In the case of that state
machine I just talked about,
you could pull every, you
know, 10 minutes or 5 minutes
or whatever and hope that
there are changes up there.
But wouldn't it be great if
the server just told you,
"Hey, I've got changes.
It's time for delta download."
Well, we give you that.
We give you what we
call zone subscriptions.
These look like query
subscriptions,
but they're a little
bit different.
What they do is every
time something changes
in a custom zone, you'll
get a push notification.
When you see that
push notification,
you know you should go do a
delta download with the server,
and you'll get new records.
So, a couple of notes on
designing custom zones
and when you should use them.
These custom zones are meant
to compartmentalize your data.
Because of that, there's a
couple of restrictions on them.
The first is that you can't
move records between zones.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
The first is that you can't
move records between zones.
You can pick these
records up and copy them
and make a new copy in the new
zone, but you can't move them.
You also can't make
any cross-zone delete
self relationships.
So, you have to think of
these zones as self-contained.
If you have records that need to
go across zones, you might want
to rethink your model.
And finally, these zones
determine the level of updates.
If you're using a zone
subscription, you're going
to get a push every single
time something has changed
in that zone.
If you have a lot of data on
the server and you only care
about getting updates for
one part of that data,
you might want to make that a
zone so that you can ignore the,
you know, really busy
stuff going on over here
and just download the
things you care about.
So, let's talk about some
advanced record operations.
When you're using CKRecord,
any changes you make
to a CKRecord object locally get
tracked, and then when we talk
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
to a CKRecord object locally get
tracked, and then when we talk
to the server, we're only
going to send the changes
that you made to that CKRecord.
This is the default behavior,
and it works great most
of the time, but you might
want some additional control
over what we're sending
to the server.
And we give you that
with save policies.
So, these save policies,
we have three of them.
They're
CKRecordSaveIfServerUnchanged.
We have CKRecordSaveChangedKeys,
and we have CKRecordSaveAllKeys.
So, let's look at the
differences between those.
The biggest difference is
what they determine for a,
what I'm going to
call a locked save.
And a locked save is
a way of making sure
that you don't overwrite
data on the server
that someone else
has already written.
When you perform a locked save,
it says that if the record has
changed since-on the server
since the time you fetched it,
the server will give you an
error saying your record is
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
the server will give you an
error saying your record is
out of date, you need to resolve
this conflict and try again.
So, the only thing that performs
a locked save here is CKRecord
save policy
SaveIfServerUnchanged.
The other two actually force
these changes to the server.
So, SaveChangedKeys
and SaveAllKeys always overwrite
the values on the server.
These policies also determine
how much data we're sending
to the server.
In the case of
SaveIfServerUnchanged
and SaveChangedKeys, we're only
going to send up the, sorry,
we're only going to
send up the values
on the record that have changed.
In the case of SaveAllKeys,
we're going to send the
entire record, all the values,
whether you've changed
them or not.
So, back to locked updates.
We've got this thing
called locking,
and if you don't use it,
here's what could go wrong
with an unlocked update.
Right now, we've got
this contact card.
We've got the same name,
first name, last name,
picture on two clients,
and in iCloud.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
picture on two clients,
and in iCloud.
Everything is good right now.
But let's say the iPhone
changes that contact completely.
So it modifies the picture,
it modifies the first name,
and it modifies the last name.
Well, it's going to perform a
SaveChangedKeys to the server,
which is going to send up
the things that have changed,
and that's going to overwrite
whatever is on the server.
Next, before the Mac gets a
chance to download that record,
the user edited the first name
of that record and
just changed it.
And that Mac then sends- does a
SaveChangedKeys with the server,
and it sent up just the
property that was modified.
And now, we've got a problem
here, oops, we've got a problem.
This contact record isn't
what either the clients meant
to have.
So, to work around this,
we have locked updates.
And locked updates are
a way of making sure
that the server knows that
you're updating the same values
that are already on the server.
And we do this with
a change tag.
So, the change tag is a
property on the record
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So, the change tag is a
property on the record
that necessarily changes anytime
a value in the record changes.
In this case, we've got change
tag A both on the server
and locally, and so we know that
they both have the same values.
If we go and we add a new
property on the local record,
when we save that to the server
using the default save policy,
it's going to send up
both the change tag
and the property that changed.
The servers then,
before it does anything,
it's going to check
the two change tags,
and if they're equal, it'll
apply the change that you made.
If on the other hand we have
another client that, oh, sorry,
and then the server is going
to send back a new change tag
because we did change
a property.
We return this new record to
you with the updated change tag
in the completion block.
So, if you take the record that
we gave you at the end of a save
and you do all your subsequent
modifications on that,
you'll be using the
updated change token,
and you shouldn't have any
conflicts with the server.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and you shouldn't have any
conflicts with the server.
Now, let's say we had
another client that, you know,
hadn't seen that change and
he was still at version A.
And he decides to add
a different property
on the record.
Well, he's going to save it to
the server, he's going to send
up the change tag and
the modified property,
and the server is going
to realize that, hey,
those two values
aren't the same.
And the client is going to
get back an error saying,
you know, you have a conflict.
So, how do we handle
these conflicts?
Well, if you run into this
case of a locked save failing,
you're going to get an error
that CKErrorServerRecordChanged.
And because you guys all
went to the introductory talk
and you learned about how
great error handling is
and how important it is,
you've got some fantastic error
handling here, and it's going
to check for
CKErrorServerRecordChanged.
When you see that, you know that
you're out of date, you're going
to need to get the new
record from the server.
You're going to need to
take your local changes,
apply them to that record
and try to save again.
And we've already
anticipated that.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
And we've already
anticipated that.
So, what we're going to do
is help you out a little bit,
and we're going to return
those records to you already
because we know that's the
next logical thing to do.
So, inside of the userInfo
dictionary of this error,
you're going to find three
copies of the record.
You're going to find the record
that you attempted to save
to the server, the one
that encountered the error.
You're going to find
the original copy
of the local record, which
is the copy we downloaded
from the server before
you made any changes.
And finally, we're going to
give you the server record back.
This is the copy of the
record with the most
up to date change tag.
You're going to want to figure
out what makes sense out of all
of those copies of
values and keys.
You're going to apply them
all to the server version
of the record, and then you're
going to retry that save.
So, you might be wondering
at this point, you know,
what type of save
policy should you use?
And the simple answer to that is
that you should just always
use CKRecordSaveIfUnchanged,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
that you should just always
use CKRecordSaveIfUnchanged,
and it's the default.
It's the default for a reason,
and that's because
it's the safest.
If you remember that example
of an unlocked update gone bad,
you could end up with really
mixed-up, corrupt data
on the server if you're
not using locking.
However, there is
always a time and place
for using unlocked updates.
And the biggest case for these
are highly contentious updates.
If you're doing anything
in the public database,
you might have hundreds
or thousands or millions
of clients accessing
it at the same time.
And if every one of those
clients is trying to save if,
save the same record and
you're locking on that,
most of those clients are going
to be hitting locking errors,
they're going to hit
conflicts, and they're going
to be retrying really
frequently.
So, if you know this is going to
be a really contentious update,
you can structure your client
to do unlocked updates as long
as you know you're always
writing consistent properties.
On the case of that unlocked
update that failed, you know,
you could make sure you
always write both the first
and last name, and you
know it'll be consistent.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and last name, and you
know it'll be consistent.
The other reason to use an
unlocked update in the case
of SaveAllKeys would
be if you want
to force something
to the server.
The client might say, you know,
the copy on the server is bad,
but I know I have a good
copy of the record here.
I want to just force that all
on top of the server's copy.
There are some catches
to using SaveAllKeys,
and one of the problems is
that any property on the server
that doesn't exist in the
local record that you're trying
to save isn't going
to get removed unless
you explicitly remove
that property on the record.
All those words are
really confusing,
and it's really hard to explain.
So instead, I'm going
to try and explain
that with the picture here.
So, we've got our
contact record again.
What we're going to do is
add a couple of properties.
We're going to change
the first name,
we're going to add
a new property,
and we're going to
delete the hobby.
We're now going to do a
SaveAllKeys to the server.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
What that's going to do is send
up all of these properties,
even the ones we didn't change
locally, and they're going
to update the values
on the server.
But what you'll notice here is
that the server had
an additional property
that we didn't have in our
original copy of the record.
This hometown property
wasn't removed.
We do this to help you with
versioning of your app.
You might release a version 2
of your app that adds hometowns
but version 1 didn't ever
know about the hometown.
And if version 1 was
using SaveAllKeys,
it's going to overwrite these
properties on the server
that it never knew about, and
that's kind of a bad thing.
You can't have backwards
compatible code easily that way.
So, what we do instead is we
still allow you to remove that,
but you need to explicitly
tell us that you want
that property deleted
on the server.
So, even though there's
no hometown property
on this record, we can still
call CKRecordSetValueForKey nil
or we can call
removeObjectForKey.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
or we can call
removeObjectForKey.
We're going to remove
the hometown,
and now when we save
it up to the server,
that delete will also
go up with the save.
Finally, one of the really
neat things about CKRecord is
that we allow you to
have partial records.
So, you might have a really
big record on the server,
and you only care about one
or two properties of that.
Well, with the desiredKeys
property, we allow you
to fetch just a certain
subset of the properties
that are on the server.
And we expose this desiredKeys
property on any operation
that fetches a record,
so you're going to see it
on CKFetchRecordsOperation,
CKQueryOperation,
and
CKFetchRecordChangesOperation.
And the really neat thing
about this is you can then
take these partial records
and save them back
up to the server.
You don't need to work
with entirely full records.
You can, you know, if you
want to update just one value
on a record, fetch that
one value from the record,
update it, and do a
save of just that record
that has that one value.
If you're doing this
with locking enabled,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
If you're doing this
with locking enabled,
you know it's safe because
if the record is changed
on the server since then,
you're going to get a conflict.
So now, let's talk a little bit
about modeling your
data for CloudKit.
If we go back to that
example of a Receipts app,
let's say we have a shoebox
that holds all our receipts
and that's going to be
one object in the cloud,
and we're going to
have an object
for each one of our receipts.
And the initial way you might
design this is you create a
receipts array inside
of your shoebox,
and every time you add a
new receipt, you're going
to add new entry to that
array and that entry is going
to be a reference to the
record you just created.
We call these forward
references.
These are references from
a parent to its child.
And the problem with
these is that you end
up getting bottlenecked
on that receipts array.
If this was a public database,
you might have hundreds
of clients trying to
save this record at once.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
And every time a client
tries to add a new record,
they need to update that
array on the shoebox,
and even though the clients
might be adding different
records, different receipts,
they're all going to run
into conflicts on the shoebox
app, or on the shoebox record.
So, we recommend that you try
and avoid forward references
in your design, and instead, use
what we call back references.
So, instead of having
the shoebox point
at the receipt that's in
it, have the receipt point
at the shoebox that it's in.
The great thing about this
is it's very scalable.
You can have millions of clients
adding receipts all at once,
and there's no bottleneck.
They can create their receipt,
point at who owns them,
and none of them are going
to run into conflicts.
When you need to go get
everything that's in a shoebox,
rather than fetching that
shoebox, getting the array
of records in it, and
fetching all those records,
you can instead just
use a query.
That query is going to select
all of the records that have
that shoebox as an owner.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
that shoebox as an owner.
One other neat feature
we give you
with the relationships
are cascading deletes.
So, cascading deletes
are a way for you
to make your object graph
get cleaned up automatically,
and you do this by
marking your references
with a
CKReferenceActionDeleteSelf,
and that means that
when the record
that you referenced is deleted,
you will also be deleted.
In this case, the
green record has
that reference action
pointed in the orange record.
When the orange record
gets deleted,
the green record is going
to get automatically
deleted by the server.
These deletes also cascade.
So, if we had a whole
tree of objects hang off
of that one orange record,
those deletes are going
to cascade all the way down
and clean up our graph for us.
But one thing to keep in mind
with these delete
self references is
that they're not
reference counted.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
that they're not
reference counted.
That means that if you have
multiple references coming
out of one record, the first one
of those records that's deleted
will cause you to get deleted.
And finally, these delete
self references give you one
additional feature.
If you're doing a batch save and
you've got a jumble of records
in this graph, and they
have delete self references
between them, CloudKit is going
to do an automatic
topological sort for you.
We're going to upload
the records in order
so that all the records that
are referenced will be up there
by the time the record
reference in them is uploaded.
This is really great
in the public database
where you don't have atomic
commits but you still want
to be able to upload data that
looks consistent at any point
when a client downloads it.
So, while developing CloudKit
internally, we've had a couple
of our clients ask us,
you know, why do I need
to use these reference objects?
I have to, you know, alloc and
omit and it's so much work.
You know, I've got a record
ID, why can't I just put it
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
You know, I've got a record
ID, why can't I just put it
in the string and put
that in my record?
You can, so, you know, you come
up with this clever
little RecordIDToString,
but then you look at
the recordID class
and you realize it's
actually two properties,
and you can't just store the
description of the record
on the server so you
get a little smarter
and you store the recordID
and the name and the zoneID.
But then, you look at zoneID
and that's also composite.
We need to know what, where
zone is and whose account it is.
So, you get a little more clever
and you make this
RecordIDToString
and ZoneIDToString, and now
you call that everywhere.
The problem is now you're
not forwards compatible.
If anything ever gets added to
references, all of your records
on the server are already going
to have this hard coded format,
and you're going to have
to go through a lot of work
when you query records.
Instead, just use references.
It's going to make
your code really clean.
I mean that shoebox
could be a recordID,
that shoebox could be a record
itself, it could be a reference.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
that shoebox could be a record
itself, it could be a reference.
All of those will just work as
long as you use the CKReference.
A couple last notes on
modeling your data in CloudKit.
One of them is that CloudKit
is a transport mechanism.
What we've tried to do here
is give you an easy way
to access the CloudKit
servers, but we're not meant
to be a local persistence layer.
We want that to be up to
you, and you need to figure
out the best way to
store your objects.
And to that extent, we recommend
that you don't subclass CKItems.
You should take those items when
you receive them from the server
and translate them into your own
model objects, and you can do
that on the way out as well.
When you need to save one of
your model objects, translate it
into a CKRecord and upload
all of that to the server.
So, if you remember from
the introductory talk,
we have these things that
we call subscriptions.
They're persistent queries
that run on the server,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
They're persistent queries
that run on the server,
and every time something
changes that matches that query,
you're going to get
a push notification.
And these push notifications
are sent via the Apple Push
Notification Service, just the
same way that it would work
if you built your own server.
But there are some drawbacks
with Apple Push Notification
Service.
One of them is that they
can't make any guarantees
on delivery of these pushes.
They're kind of meant to
be these ephemeral, little,
you know, here's a
push, here's a push.
If one of them gets dropped
because of a bad network
or anything else going
on, there's no guarantee.
So, in practice,
they're really good
about delivering these pushes.
And one of the reasons is
because the server
will store pushes
for you if you're offline.
So, if you put your
phone in Airplane Mode
and you get a push, as
soon as you come back
out of Airplane Mode, the
server will have that saved
and still deliver
the push on to you.
But the problem with this is
the server only stores one push
per client.
So, if you received a bunch of
pushes while you're offline,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So, if you received a bunch of
pushes while you're offline,
you're only going to
get the most recent one.
What does that look like?
Well, we've got our
CloudKit server,
we've got the APS server, and
we've got the iPhone up here.
And we send a push, and we're
online, everything goes through,
we get our badge,
everyone's happy.
But then, we get on an airplane.
Our phone is in Airplane Mode;
we have no network connection.
And when that push gets sent,
the APS server helpfully
stores it for us.
If we were going to come
back online right now,
we would get our subscription
push and we'd be happy.
But this is a long plane flight
and while we're on the plane,
we got a zone update push,
that new zone subscription
that I was talking about.
When that gets to the APS
server, it's going to drop
that previous subscription push,
and now when your phone comes
back online, all you're going
to receive is the zone update.
The problem with this
is that you never heard
about that subscription
that fired.
So, how do we solve this?
Well, we've created a
notification collection
in CloudKit.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
in CloudKit.
Every time the server sends
a push to the APS server,
it also makes a copy
of that push
in the notification collection.
So, you can see we have that
same problem where we're
in Airplane Mode, we dropped
our subscription push,
our phone comes online,
and it gets the push,
and because this is a
well-behaved client,
it knows every time
it receives a push,
it should go check the
notification collection to find
out about anything
it might have missed.
So, how does that work?
Well, this notification
collection is a lot
like the delta downloads
I talked about earlier.
It's all change token-based.
You hand a change
token to the server,
and the server hands back
all the notifications
that have changed
since then along
with an updated change token.
So, since this is our
first time talking
to the notification
collection, we're going to send
up a nil change token.
We're going to get back
both of those pushes,
one of them which we missed.
And then we're also going to
get an updated change token
for the current state.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
One thing with subscriptions is
that you might want to use those
to change some UI
in your application.
For example in our Receipts app,
we might want a subscription
for any receipt that
was over 100 dollars.
If that happens, we want to mark
that receipt in a
different color.
So we're going to get this
push for a subscription,
and now that receipt is blue
because it was a big
expensive receipt.
But, you know, this is going
to happen on all your clients,
so you're going to have you iPad
showing that receipt in blue,
you're going to have your Mac
showing that receipt in blue,
and once this acknowledges
it, you want that UI state
to go away on all your clients.
Well, the way we do that is we
let you mark a subscription,
sorry, we let you mark
a notification as red.
And when you do that,
you're going to get a push
for the updated subscription,
updated notification
on every client.
So, in this case,
we mark our receipt.
We're going to go mark
that subscription as red
on the server, and now
there's a new entry.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
on the server, and now
there's a new entry.
And if we switch over to our
Mac which is still showing it
in blue, it's going
to get a push,
it's going to check the
notification collection,
it's going to get that
subscription notification,
and it can tear down
its UI and now all
of your clients are
in the same state.
So, with the notification
collection,
keep in mind every
time you get a push,
you should check the
notification collection.
You never know what
you might have missed.
And this isn't just for
Airplane Mode or bad networks.
This can happen if you get
multiple pushes in a row.
If there's a lot of changes
that happen all at once,
the CloudKit server
might coalesce them,
or the push server might
coalesce those pushes.
And of course, because a lot
of these are mobile devices,
they're iPhones, they're
going to be moving in
and out of network states.
They might be on Wi-Fi, or they
might be on a bad cell network.
You never know what
your network is like.
Just assume that there
might have been more pushes
that you didn't hear about.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
that you didn't hear about.
So, now we're going to go
over the iCloud Dashboard.
And the iCloud Dashboard
is our web-based interface
for managing your
application in CloudKit.
The iCloud Dashboard
lets you view your data.
It lets you manage the schema
that we're creating for you.
It lets you control what's
indexed, and it helps you moving
from development to production.
I'm going to explain all of
those in detail in a bit.
Let's start with viewing data.
So, if you remember the view
of our containers up here,
we've got a public database
and a private database.
And when you're viewing your
data in the iCloud Dashboard,
all of that data in the public
database is of course public.
So, in the dashboard, you'll
be able to see every record
in your public database.
But if you remember, the private
database is again one database
for each individual user.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
And in this case, we're
only logged in as one user.
We're logged in as
our developer account.
So, all we're going to see in
the iCloud Dashboard is the data
in the private database
for our developer account.
This is really important.
You know, you might use a
different account for testing
and a different account
for viewing data
in the iCloud Dashboard.
If you do that, you
won't be able
to view your private
data in the dashboard.
So, what does the
dashboard look like?
Well, we log in, and the
first thing we're going
to see here are our
different record types.
So, you can see we're
using the Party app.
We've got parties,
and over here,
we see the different schema
for those values
that's been created.
When you're developing
a CloudKit application,
you're talking at the
development environment,
and the server
in the development environment
creates a just-in-time schema
for you.
So, we did this because we
wanted you to be able to develop
as rapidly as possible.
We didn't want you have to go
to a, this dashboard and plan
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
We didn't want you have to go
to a, this dashboard and plan
out what you were going
to do in your application
and choose all the data types.
That's not as much fun as
just writing some code.
So, we let you dive right
in, you write some code,
and the things that you upload
to CloudKit as you upload them,
those values get
locked in into a schema.
So, you can see here
that we've got a couple
of different values, and we've
got their, couple different keys
and their values but we
notice that we made a mistake
when we're developing this app.
We uploaded a date
value or a date key,
but we use a string value.
And what we really want
there is a date value.
The iCloud Dashboard is
going to let us fix that up.
So we can go to this
and we can delete it.
We can remove it
from our schema,
and now we can recompile
our app, use a date value,
upload that record again,
and when the server sees
that record, it will
create a new schema entry
for the date value using an
actual date instead of a string.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So next up, you can
view all the records
in your public database
using the iCloud Dashboard.
So, you can see here,
we already have a couple
of different parties
in the public database.
What you can also do is create
records in the public database.
So, you can tap on this Plus
button and fill out a new entry.
We're going to make a new
party for coffee on Friday,
and we can save that and you'll
see that that actually got saved
into the public database.
Any clients that are fetching
records are going to see
that change in the
public database.
We can also search for records.
This is just like CKQuery.
So we can click on
that magnifying glass,
and we can type any
string, and we can filter
down to any party
that mentions WWDC.
Additionally, this gives
us all the functionality
that we have in CKQuery.
We can build compound queries
right in the dashboard.
So, if you click on the Plus
button, let's choose location,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So, if you click on the Plus
button, let's choose location,
and let's filter down
to everything that's
within 5 kilometers
of Moscone Center.
I happen to have
those memorized.
So, that filtered everything
down just to the two parties
that are in this area.
Finally, in the public database,
by default every record
can be read by every user,
and it can be created
by any user.
Once you create a record, that
record can only be updated
or deleted by the
user that created it.
But we understand that that
doesn't provide all the
functionality that you might
need to make an application
in the public database.
So, what we've given you
are what we call roles.
These roles let you
choose sets of users
that have different
permissions for record types
in the public database.
So, one example of that might
be that I want to restrict it
so that only I can create
parties in the public database.
So, what I'm going
to do in order to do
that is create a new role.
I'm going to call this, you
know, PartyAdmin, and in that,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
I'm going to call this, you
know, PartyAdmin, and in that,
I need to choose a record type
and give it special permissions.
So I'm going to choose parties,
and I'm going to give it Create,
Read, and Write permissions
for those parties.
Now, what I need to do is go to
the record type and restrict it
so that no one else
can create a party.
So, I'm going to find my
party, my party record type,
and you can see up there
that parties can be created
by anyone who's authenticated.
We don't want that, we
want only party admins
to create this record.
So, we're going to
uncheck that value.
Now, we need to assign
people to that role
so that there's actually users
out there that can do this.
If we go to the user records,
we can see everyone who's
used this app so far.
If they have marked themselves
as discoverable, you're going
to be able to see their
first and last name.
If they haven't,
you're just going to see
that recordID up there.
So, I'm going to click on
my user, and then I'm going
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So, I'm going to click on
my user, and then I'm going
to add myself as a party admin.
And now, I'm the only user
in the public database
that can create a party.
[ Applause ]
So, one of the last things the
iCloud Dashboard helps you do is
move from development
into production.
So, as I talked about earlier,
we wanted the development
environment to be as quick
and easy to use as possible.
We wanted you to hit the
ground running, we wanted you
to just open up Xcode,
start typing some code,
save your record to the server,
and see immediate results.
And that all works great because
the server does just-in-time
schemas, but that's not very
efficient if we're working
in a public database that
might have millions of users.
We need to prepare some
things before we go
into the environment that all of
our customers are going to hit.
So, the way this works
is we've got our records,
we're developing
our app right now,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
we're developing
our app right now,
and we just created a record
for the very first time,
and we uploaded it in the
development environment.
The server is going to see that
and since this is the first
time it's seen a party,
it's going to automatically
create a new party record type.
And then for each of
these keys and values,
it's going to create an entry
in a schema, and it's going
to create an index on
every one of those values.
This index is what lets you
query for any value in a record.
So, while you're in development,
you can just run a query
and search on any value
for any key in a record.
And let's say we're going ahead
and we're developing this app
and we decide we want to
add an additional property,
so we decide we want to
have a background property
for the color of the party.
So, this party's
background is blue.
All we have to do is
upload that record.
It's going to send all those
other properties to the index,
and the server is
going to notice, "Oh,
there's a new property in here."
It's going to create a new
schema entry, build a new index,
and index that property.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
When you're ready to release
your app, you're going to want
to make sure you've run
everything in development first
so that you've built up
this just-in-time schema.
You're then going to use
the iCloud Dashboard to move
that schema from
development into production,
and you're going to want to
check all of these fields
to make sure that
they're the right values,
make sure that every key
that your app will ever
use exists in the schema.
And when you do that, you're
going to move it into production
and lock that schema in.
Additionally, all of these
indexes take up a decent amount
of space because
it has to make it
so that you can query
those records.
If you know you're never
going to query a value
like we know we're never
going to query for records
with just a blue background,
you can drop that index,
and this will help free up some
space in your app's database
or in your user's database.
So, finally, a couple
CloudKit tips and tricks.
So, we already went over this
in the intro talk
about error handling.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
in the intro talk
about error handling.
Please handle your
errors in CloudKit.
This is really important.
I know error handling is
hard and it's not fun to do,
but CloudKit is all based
on network communication.
And when you're talking over the
network, anything can go wrong.
You know, the network
can get dropped.
Because we have other
people talking
to the server, we
can get conflicts.
We can get errors.
All kinds of things can
happen, and as Paul said
in the first talk, this is the
difference between a working app
and a not working app.
It's not the difference between
a good app and a great app.
If you don't handle errors,
your app just isn't
going to work right.
In CloudKit also, we've tried
to avoid any sort of magic.
We don't want to try and handle
these errors for you and figure
out what might be best.
We want to just tell
you what happened.
We want you to be able to figure
out what you need to do next,
and we want to give you all
the information to do that.
So, keep in mind when you're
handling your errors here
that your operations
can have partial errors.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
that your operations
can have partial errors.
These partial errors when you're
using the NSOperation API,
you might be sending
up a batch of records.
And if you're saving them
to the public database,
it could be the case
that just one
of those records had a conflict
but the rest saved just fine.
If that happens, you're
going to get a partial error,
and inside of that, you're
going to find one error
for the record that failed.
If this was in a custom
zone, you might see
that as an atomic update error.
So, you might see that one
of those records failed,
and the rest got the
batch error in there.
And finally, we want
you to make sure
that you retry any
server busy errors.
It could be the case
that the, you know,
a bunch of people are going at
the servers at the same time
and the servers can't handle it
and they need clients
to back off.
It could also be that
your client is misbehaving
and hitting the server
too frequently,
and the server is saying,
"Slow down, buddy."
This is our way of
saying, "You know,
we need a little more time.
This request was OK, but
please try it again".
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
This request was OK, but
please try it again".
And anytime we give you
a server busy error,
we're going to hand back
a CKErrorRetryAfterKey.
This is a number of seconds
that we'd like you to wait
and retry your request.
So, limits, in the
keynote, it was mentioned
that CloudKit is
free with limits.
How do those limits work?
Well, anything stored in the
public database is counted
against your app's quota.
We give every app a
quota, and that's just
for the public database.
Anything you put in the
private database is counted
against the user's account.
So, every iCloud account gets
5 gigabytes of free storage,
but users might have bought
more, or they might have filled
up all that space with
emails or backups or photos.
So, every user is going
to have a different amount
of storage in there.
And because this is
kind of the user space
and it's shared space,
we want you to, you know,
remember to be nice
to your users.
It's technically free space for
you because it's theirs but,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
It's technically free space for
you because it's theirs but,
you know, don't fill it
up with unnecessary stuff.
So again, with the
limits, how much do you get
for your app container?
Well, what we're concerned
with here is customers
having a great experience,
and these limits that we've
specified here are really
to try and prevent abuse.
We don't want to
prevent legitimate use.
We just don't want
anyone abusing CloudKit.
And the numbers that we
give you here also scale
with the number of
users you have.
If you go on
developer.apple.com,
you can get the full breakdown
of the different values,
but as an example, if you had
10 million users of your app,
we're going to give you a
petabyte of asset storage,
10 terabytes of database
storage,
some pretty high transfer
limits, and this is all
for your public database
in your application.
So, finally, a note
on efficiency.
One thing about CloudKit
is, again,
it's a transport mechanism.
We are only there to talk to
the iCloud servers for you.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
We're not storing any records.
We're not caching records.
Anytime we give you back
a record, it's something
that we got from the server.
We're trying to be as
transparent about that
as possible and not do any
caching shenanigans here.
We're always just giving
you what the server gave us.
In terms of efficiency
though, you might remember
from the save policies
that we're only going
to save the values that
have changed on the records.
So, we try and be smart about
what we send over the wire.
When we're sending up assets,
those can be potentially
really big blobs of data,
and that data is
transferred efficiently.
So, if the server already
has a copy of a file,
we won't re-upload it.
If we already have
that file locally,
we don't need to re-download it.
So, in summary, we're really
excited about CloudKit here.
We've built a lot of
really great features
and it's something that
we actually use at Apple.
We've built iCloud Drive, we've
built iCloud Photos on top
of this, and we want you guys to
start using the same technology
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
of this, and we want you guys to
start using the same technology
that we're using at Apple.
I'm really excited to see
what's going to happen.
I can't wait to see some
apps that use CloudKit,
and good luck with
using CloudKit.
[ Applause ]
So, if you need any, if
you need to contact anyone,
Dave DeLong is our Evangelist.
He's the guy with the bowtie.
If you need documentation,
it's on developer.apple.com.
We had a Introducing CloudKit
session earlier in the week.
There's a couple more
related sessions.
Thank you.
[ Applause ]