Transcript
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
[Silence]
>> Nick Colet: My name is
Nick Colet [assumed spelling]
[applause] I'm a
[laughter] -- thank you.
I'm a software engineer on the
Core Data team here at Apple,
and I'm excited to take you
through what's new in Core Data
in iOS 7 and Mac OS X 10.9.
We've done a lot of great work
in the Core Data framework
over the last year.
We've made a significant number
of improvements to key areas
to improve performance,
reliability and stability,
but today, we want to take a
moment to focus on just a few
of them, beginning with iCloud.
Now, we've made a
number of improvements
to our iCloud integration.
We've been iterating on
it for almost 12 months
since our WWDC session
last year,
and we took into account a lot
of your feedback that we've seen
on developer forums, in
radars and around the web.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
And so, the improvements
that we've made require us
to revisit a few of the topics
that we discussed last year,
beginning with improvements
we've made
to the fallback store, as
well as changes to how you set
up the store asynchronously;
and what happens
with iCloud transition events
like when the account changes.
We're also going to
talk about some new API
that we're introducing today.
And finally, I'll
introduce you to some changes
that have happened across the
organization and developer tools
and frameworks to change how
you live and work on iCloud.
They'll make it easier to use
and provide a lot
more transparency
than was previously
available to you.
Then, Melissa Turner
will come up
and take you through some demos.
And finally, Ben Trumbull will
come up and give you an overview
of several enhancements that
we've made to the SQLite store.
Now, before I dive into
the meat of the discussion,
I'd like to take a moment
to frame it by talking
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
about our goals and
objectives related to iCloud.
You see, last year after iOS
6 and Mac OS X 10.8 shipped,
we took a look at our
iCloud integration and tried
to identify several key
areas where we could focus
on providing valuable
improvements toward integration,
the first of which is speed.
This year, we've made a
number of improvements
that will change not only how
efficient our integration is
in terms of its memory
usage, its processing usage
and battery consumption,
but also how well it deals
with iCloud under the hood.
We've made a number
of improvements
to downloading files so
that changes appear faster
and peers can communicate
their changes faster than ever.
We also strove to
improve simplicity.
You see, after we
shipped last year,
we felt the developers
still had to write a number
of complex pieces of code
to integrate successfully
with our iCloud integration
and we found
that this hampered your ability
to build effective applications
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
that worked on top
of our integration.
And finally, we really want
to focus on consistency.
As you may have noticed, we
didn't have a clear paradigm
that you could reuse over
and over again, when working
with our integration, and this
made it exceptionally difficult,
or at least more difficult
than we had originally planned
to handle transition events
like when the account changes.
So with that in mind, let's take
a look at the fallback store.
As you may remember, the
fallback store is a concept
that we introduced last
year at WWDC as a way
of providing a local
persistent store
that your application
could use in the event
that the iCloud store was
unavailable for a certain period
of time, and your application
would simply add the store
to the coordinator
to make it available
to your managed object context
so that your user could
write changes to disk while
in the background we set up
the iCloud store for you.
And this would produce
a pairing, if you will,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
of a local persistent store file
and an iCloud persistent
store file
that were tied to
the same account.
Now, this worked out
rather well until,
you ended up with
multiple accounts,
and this is when things
became a bit cumbersome
because developers would
end up spending a lot
of their time writing codes
and manage these different
persistent store files instead
of focusing on building
our applications.
And so for iOS 7
and Mac OS X 10.9,
we've completely
eliminated the need for you
to manage the fallback store.
We've subsumed that
responsibility
into the framework and it
will be entirely managed
by Core Data.
This also means [applause]
[laughter] -- thank you.
This also means that your
application will only ever use
one persistent store
file per account.
Now, we say "per account"
because we still have
to provide a different
actual persistent store file
for each iCloud account
that's in use on a system,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and this requires
that we change a bit
of our guidance from
previous years.
You see, we had previously
advised
that you could keep
the iCloud store inside
of the iCloud container
in a .nosync folder.
This had a rather
convenient advantage
of automatically deleting
the store file for you
when the account went away.
However, because we
now take responsibility
for managing the fallback store
and the fallback store must
exist in local storage,
we now require that you store
the persistent store file
somewhere in local storage,
usually the application sandbox.
However, if your
application isn't sandboxed,
you can keep it wherever you
normally keep your persistent
store files.
Additionally, and because this
is now being taken over entirely
by Core Data, we want to
provide a bit more transparency
than we used to about events
that are happening related
to the fallback store,
and for this reason,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
we'll log some new
events to console
so that you can see them
during your development.
You may be familiar with
our Core Data logging.
All of our messages start
with CoreData: Ubiquity:
and when we're using
the fallback store,
you'll see Using
Local Storage 1.
You'll see Using Local
Storage 0 if we've been able
to transition off
the fallback store
and your application is
fully connected to iCloud.
Now, it's important that you
understand the expectations
associated with each
of these log messages.
Using Local Storage 1 means
that changes that you make
to the persistent store
are being persisted vocally
and they won't be available
to other peers over iCloud
until you see Using
Local Storage 0.
This is when you can expect
to start seeing import
notifications.
And when you should expect
seeing your changes uploaded
and made available
to other peers.
So that's the fallback store,
and now I'd like to tell you
about some of the changes
we've made to how you set
up the store asynchronously.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
You may be familiar
with this line of code.
It simply adds persistent
store to an instance
of NSPersistentStoreCoordinator;
and when you call it,
you expect to get a store
file back fairly quickly.
In most situations,
especially those without iCloud,
we only have to do a
small amount of network --
of local I/O to bring the
persistent store file up,
so it returns immediately.
However, when it is connected
to iCloud, we sometimes have
to do a significant
amount of network I/O
to make sure the
persistent store is
in a state that's consistent
and ready to be used
by your application, and in iOS
6, we did this synchronously.
That meant that whenever you
called add persistent store
and you didn't have a
persistent store file on disk,
we would go out to the
network and discover the state
of the contents of
the iCloud container.
In iOS 7 and Mac OS X 10.9,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
we no longer do this work
synchronously within the call
to addPersistentStore,
so addPersistentStore will
always return immediately.
And this is a huge
benefit to your application
because it eliminates
the need for you
to call addPersistentStore
on a background thread.
Now, have we done this?
Well, we've done it by
using the fallback store.
We will transparently pass
the fallback store back
to your application and so that
you can begin working with it
and your user can begin
making changes immediately.
But, because we manage the
fallback store now, we were able
to take the integration
one step further
than we felt developers
could on their own.
We now store individual
transaction records
for each change that's
made to the fallback store
and this has implications for
both performance and bandwidth.
You see, last year, if your
application took responsibility
for migrating contents
from the fallback store
to the iCloud store after the
iCloud store was available,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
you would have to do a
lot of extra work to scope
that migration and ensure
that it didn't consume
too much memory
or bandwidth over the wire.
However, now because
we keep our record
of each individual transaction,
all that work is done for you.
And this means that once the
iCloud store is available,
we can transport those changes
into the container immediately
and make them available
to other peers.
So there's no additional work
for you to do the migrate data
from the fallback store
to the iCloud store.
But we also wanted to give
you a better way to tie
in with this transition
and that requires
that we introduce some new API.
Today, we're introducing
NSPersistentStoreCoordinator
Storage Will Change notification
as a way that we can tell you
that an event has occurred where
we need to change the contents
of the persistent
store file on disk.
When you receive it,
you can optionally call
NSManagedObjectContext
save if there are changes
in your managed object context
that you wish to persist,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and then you can call
NSManagedObjectContext reset
and prepare your
UI for new data.
When we get control back from
your notification observer,
we'll remove the store from
the coordinator for you
and then post
NSPersistentStoreCoordinator
Storage Did Change notification
when the store is
ready to come up.
Now, it's important
that you realize
that this transition
is very fast.
It's almost as fast as if
you called addPersistentStore
on a store file without iCloud
options, so you shouldn't expect
to notice any interruption
in what your user sees
in the application.
You can then start working
with the storage just
as you normally would, and call
NSManagedObjectContext save
to make changes to
the iCloud store.
Now, once you've
handle this transition,
your application will be
fully up and running on iCloud
and you can expect that at
that point changes will be
available to other peers.
Now, I need to mention that
this is a special case scenario.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
You will only see these
notifications during our
asynchronous setup process
if there is no persistent
store file on disk,
or there are has been a
change to the iCloud account
or the contents of the container
since we last launched.
Normally, you'll just start
seeing NSPersistentStore Did
Import Ubiquitous Content
Changes notification
which you can merge into
your managed object context
by using merged changes
from Context Did Save.
This is important to
realize because the goal
of our integration this year
for asynchronous setup was
to eliminate any work
that you had to do
by your application at all.
However, we're providing this
new transition paradigm as a way
of enabling you to have a
consistent implement once use
case that you only
have to pay attention
to for our integration.
And so with that, let's
take a look at changes
for how you handle changes
in the iCloud account.
Normally in iOS 6
and Mac OS X 10.8,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
when the iCloud account changes,
the only way to notice this is
by subscribing to the NS
UbiquityIdentityToken Did
Change notification.
When that happens, you have
to remove the persistent store
from the coordinator and then
replace it with a new one
to talk to the new account.
In code, this looks a
little bit like this.
You call NSManagedObjectContext
reset
and then remove persistent
store,
and then you call
NSPersistentStoreCoordinator
addPersistentStore to add the
new file to the coordinator
for use of the new account;
and this would work just fine
in iOS 7 and Mac OS X 10.9.
However, because we
control the fallback store
and because we have
a new integration
for asynchronous setup,
we thought that we might
as well extend that to
account changes, as well.
And so now, you'd no longer
have to subscribe to any
of the iCloud notifications
at all.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
You simply implement your will
change handlers and respond
to NSPersistentStoreCoordinator
Stores Will Change
and will notify you
automatically when we need
to change the persistent
store file
because there's new
account on the system.
Of course, you can then call
NSManagedObjectContext save
and NSManagedObjectContext
reset.
Now once you've done that,
we'll remove the store
from the coordinator just
as with the asynchronous
setup process
and then we'll send you
NSPersistentStoreCoordinator
Storage Did Change
notification, again,
just like asynchronous setup
and you can begin working
with your application
as you normally would.
Now, let's talk about this
in a little bit more detail.
When you receive
NSPersistentStoreCoordinator
Stores Will Change notification,
the persistent store is
still available to use,
and so unlike what we advised
you of last year where you had
to immediately drop the
persistent store and wipe
out your managed object
context, you can still write
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
to the managed object context
and those changes will
be persistent locally
to be imported to the account
if it every comes back.
This means that although your
user's changes won't make it
to iCloud immediately, if
they ever sign in again,
they'll be there and waiting.
Another thing you should know is
that once you receive
NSPersistentStoreCoordinator
Storage Did Change notification,
the store file may no longer be
available to you to use on disk.
This is because Core Data
automatically manages all
of the stores associated
with the account.
You provide us a single store
URL inside the application's
local sandbox and we then
create an opaque container
with an entry inside of it for
each account on the system,
including the local
account, which is our term
for what happens when there is
no iCloud account on the system.
This is a special store
that's managed by Core Data
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
so that you don't have
to do anything special
because your user doesn't
have an iCloud account.
How does this work?
Well, you give us a store URL
somewhere inside the application
samples, and we chop
that up and put
in a special root directory
called Core Data Ubiquity
Support, followed by a
directory tree that allows us
to uniquely identify each store
and tie it into an account.
Now for simplicity's sake,
we've kept the persistent
store file name the same
as the one you pass into us
so that you can identify
the store files inside
of this container
if you need to.
Finally, all of these
store files will be managed
by Core Data and that means
that we could remove
them at any time.
Each store will be removed
once its account has gone away
because we can rebuild
the file from the cloud.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So we want to free up as
much disk space as possible
for your application to use and
not have old store files lying
around that could take
up additional resources.
And so that's our
changes to the account.
So now let's talk
about some new API
that we're introducing
this year.
I'm sure you recognized
this line of code.
This calls NSFileManager's
URLForUbiquitous
ContainerIdentifier
method, and it allows you
to get the URL inside
of the iCloud container
where your application can
store files and write data.
However as you may know, when
you change iCloud accounts,
this method can take a
little while to return,
and so we couldn't provide
you a truly seamless
and fast integration without
a eliminating the need
for you to call it.
Normally, you take the value
from this method and pass it
to us and as an option
called NSPersistentStore
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
UbiquitousContentURLKey,
and this is how we know
where to keep all of your
data in the iCloud account.
So in iOS 7 and Mac OS X
10.9, you'll no longer need
to pass a value for that at all,
and we'll call URLForUbiquitous
ContainerIdentifier
automatically under
the hood for you.
Now as you may remember,
our advice in past
years has always been
that you should keep our
transaction log content inside
of a special subdirectory
in the container,
and you may be wondering
how it is that we'll know
where to find this if you're
not passing a URL into us.
Well,
NSPersistentStoreUbiquitous
ContentURLKey now takes a string
subpath, as well.
This is optional, but if your
application already exists
in iCloud and you have
a special subdirectory
where you've been keeping
all of our content,
you need to pass this
value in so we can find it.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Otherwise, we'll
create our own value
by default called
Core Data Support.
This is a special directory we
create in your iCloud account
and store all of our
transaction log content in.
One of our main goals
this year is
to provide a very simple
integration; and that means
that we really need to consider
all of the cases of applications
that are going to
be in use with it.
So this year, we're
introducing NSPersistentStore
UbiquitousContainerIdentifier
key as a way for applications
that have multiple iCloud
container identifiers
in their entitlements.plist to
tell us which one they want us
to use when we call
URLForUbiquitous
ContainerIdentifier.
So how does this work?
Well, if you're application has
multiple container identifiers
in its entitlements.plist,
by default,
NSFileManagerURLFor
UbiquitousContainerIdentifier
will select the first
one if pass No.
However, if you want
to use something
that isn't the first one, you
need to pass us this option
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
so that we know which
one to use,
and this value will
be passed directly
to URLForUbiquitous
ContainerIdentifier
under the hood.
We're also trying to help you
manage iCloud content and for
that reason we're introducing
NSPersistentStore Rebuild
From Ubiquitous Content option.
This is an option
that allows you
to remove the persistent
store file on disk
and examine what happens
when we rebuild it
from the iCloud content.
It's important that you know
addPersistentStore will always
return an empty store
when you pass this option,
that's because we need to switch
over to the fallback store
while we -- excuse me --
while we rebuild
the iCloud store
from the transaction log
content in the cloud.
So how does this work?
Well, you just pass it --
pass us a [inaudible]
that evaluates TS inside
the options dictionary,
and we go off and
replace the store file
with one that's freshly
built from the cloud.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
We're also introducing
a new option
to help you create
backups or local copies
of the iCloud persistent store
called NSPersistentStore Remove
Ubiquitous Metadata Option.
This removes all associated
metadata from the iCloud store;
that means, anything
that we write
into the metadata dictionary as
well as the store file itself,
and it's critical if you
want to use the migration API
to create backups
or local copies
at a persistent store
you wish to open
without the iCloud options.
Finally, we're adding
a class method
to NSPersistentStoreCoordinator
which removes all
of the iCloud content and the
persistent store files on disk.
This is our way of providing
you with a clean slate.
If you call this method, we
will go into the iCloud account
and delete all of the
content associated
with a given persistent store.
You pass us the store URL
and the options dictionary
that you normally pass so that
we can correctly identify the
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
store and its content
in the iCloud account.
Now, I can't over
state this enough.
This is our clean slate.
If you call this method, none
of the data will remain on disk
or in the iCloud account that's
associated with a given store,
but it is the easiest way
for you to start over.
And because of that, we've
made this method synchronous.
There is a significant amount
of network I/O that it has to do
under the hood when it talks
to iCloud, and because of this,
it may take a little
while to run.
However, you can't work
with a persistent store
until it's finished.
And so, it will be synchronous
and once it returns
successfully,
you can work with a store again.
How does this work?
Well, you'll have some content
and a persistent store file
on disk and we'll erase the
persistent store file first
and then nuke the
iCloud content, as well.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
And we have some special
integration under the hood
to make sure that this is
robust and reliable as possible.
It actually only requires
one I/O operation with iCloud
to propagate the delete
to all of your devices
and that makes it very
fast and very robust,
but we still recommend
that you only do it
when you have a good
network connection
such as an Ethernet
connection from a Mac.
It works just fine on
iOS devices, however,
if the connection isn't great
or iCloud isn't available
on that device, you won't
see the change propagate
for quite some time.
Now as you've already seen,
we're introducing a new
notification this year called
NSPersistentStoreCoordinator
Storage Will Change
notification, and
this only applies
to iCloud persistent stores.
You won't see it with any
other type of persistent store
on the system as it's our
way of creating a consistent
and reusable paradigm
for handling transition
events related to iCloud.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
The user info dictionary
will contain instances
of NSPersistentStore that
identify the iCloud store that's
about to change, and you
can subscribe to it just
like any other notification.
We recommend that you scope
this using the persistent store
coordinator that your
application uses to talk
to iCloud as the object when you
subscribe to the notification.
As well, you should subscribe
to NSPersistentStoreCoordinator
Storage Did Change method --
or, sorry -- Storage Did
Change notification and scope
that by the persistent
store coordinator, as well.
And this is because
the implementation
of your Did Change handler will
probably be slightly different
for the iCloud store
than it will
for the other persistent
stores on your system.
So let's take a look at
those notification handlers.
Now, I mentioned earlier that
our goal for this year was
to provide a very
simple integration
that significantly reduced
the amount of complex code
that developers have to write.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
And so, with the Will
Change notification,
we've tried to make it as easy
possible to get up and running
and all you have to
do is optionally call
NSManagedObjectContext Save
and then NSManagedObjectContext
reset.
This will completely prepare
your application for use
with the new iCloud store.
Of course, you may have some
custom code you need to run
to prepare your user
interface for this.
Now as I mentioned,
this transition will
happen fairly quickly
and so you should not need to
block your UI while you wait
for the store file
to be swapped out.
However, you may wish to
prevent the user from trying
to write new data to disk
such as blocking a Save button
which you can then enable inside
of the Did Change handler.
And as you can see, this
is only a handful of code.
In fact, most of the
sample applications
that we use internally
have completely changed
by reducing a ton of code to
just a few lines to integrate
with our iCloud integration.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
And so that's all
for the new API.
Let's talk a little bit
about how things have changed
for living on iCloud and
iOS 7 and Mac OS X 10.9.
There have been a number
of substantial improvements
to the developer tools and
frameworks that are available
to you as well as the underlying
infrastructure related
to iCloud.
Perhaps one of the most
significant is Xcode's new
iCloud Debugging pane.
This is a pane that
automatically runs inside
of the debugger when your
application uses iCloud,
and you can see the amount
of storage space you consumed
as well as the status of a given
iCloud account whether it's
idle, uploading,
downloading, or actively working
to set up the account.
But I think the real gem here
is the file transfer graph.
This shows all of the
activity on the iCloud account,
both uploads and downloads,
and so you can see any activity
that's generated on your behalf
by the Core Data
integration under the hood.
You can also see a list of
every single file inside the
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
container, including
our hidden directories
and special files that's in use
by the Core Data integration;
and as you can see, the
status of each file is printed
in the column to the right.
This will let you know whether
or not a file is what
we call current on disk,
meaning that it has been
downloaded and is available
to the Core Data
integration to use.
And so if you're looking through
our transaction log directories,
you can actually see whether
or not files from other peers
that were changed recently
have been downloaded
and are available
to us to import.
So this can be a big help
when you're wondering why you're
not seeing NSPersistentStore
Ubiquitous Content Changes
notification without having
to enable a ton of logging.
The iOS simulator also
now fully supports iCloud.
You can use the same
account with your application
that you're using on your Mac
or a completely different one,
and this allows you to
test iCloud integration
without having to change
settings on your Mac.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
As well, you can also test sync
between a Mac and an iOS device
without actually connecting
to any devices, and to me,
this is a huge advantage
because it allows you
to evaluate your cross-platform
integration without having
to worry about carrying
devices with you.
It fully supports
iCloud document storage
and the iCloud key
value store, as well,
you'll also receive support
for pushing notifications now
in the simulator [applause].
[Laughter] Thank you.
But it wouldn't be
a Core Data session
if I didn't tell
you about logging.
Now, all of this has existed
in our iCode integration
for the last couple of years.
But, in iOS 7 and Mac OS X 10.9,
we've made it possible for you
to enable the logging by
setting a user default inside
your application.
Of course, you can still pass it
as a launch argument
using com.apple.coredata.
ubiquity.logLevel.
Number 3 is the highest,
so if you turn that up,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
expect to get a lot of logs.
Of course, we also have
com.apple.CoreData.SQLDebug
which allows us to
see what activity
and what SQL we're generating
when you use the iCloud store.
And I have to mention that
if you're having problems
with iCloud related to your
local managed object context
that is the one that's in
use by your persistent store,
this is a critical tool
for us to help you debug
that because we need to see what
activity your store is at --
or, sorry, your managed object
context is actually attempting
to do in relation
to the iCloud store.
Of course, you can pass
both of these arguments
as a launch argument
through application inside
of Xcode scheme editor.
There have also been
improvements
to the underlying logging
system related to iCloud itself.
On OS X, we're introducing a new
tool this year called ubcontrol,
and ubcontrol interface
is directly
with the underlying
daemons that talk to iCloud.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
It allows you to do things
like enable debug logging
and you do this by
calling ubcontrol-k7.
Now, 7 is the highest
logging level that they allow
and it's what we need to debug
any iCloud issues related
to the actual file
transfer on the system.
So if you see a bug, enable
this, reproduce the issue
and then follow radar.
Of course, on iOS, we still have
the iCloud Debug Provisioning
profile and this is available
from developer.apple.com
as a download.
You can download and unzip
this file on your Mac
and then email it yourself.
Finally, on iOS, open the email.
Follow the onscreen instructions
and then reboot the device
once the profile is installed.
It's critical that
you reboot your device
because this is what
restarts all of the daemons
on your iOS device
that talk to iCloud.
Without it, they won't pick
up the new logging level.
Once you've reproduced
the issue,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
you should sync your
device directly with iTunes
and then gather the logs
from Library/Logs/CrashReporter
/MobileDevice
and then the device name
and diagnostic logs,
and you can file a
bug with those in it
and we'll take a look.
Now, I want to talk for a
minute just about filing bugs.
We received a lot of bug reports
from you over the last year
and this has been critical
in not only helping us
scope our integration
but also decide what areas
of improvement we can provide
to you that will
have the most value.
So when you file bugs, please,
please, please include all
of the logs that you
can, and if possible,
your local persistent
store files
because this helps us get all
the information that we need
to identify the issue
that you're running into.
So with that, I'd like to
bring Melissa Turner up
and she'll take you through some
demos of a few other changes
that we've made this year.
[ Applause ]
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
>> Melissa Turner:
I'm Melissa Turner.
I'm one of the Core
Data engineers,
and I totally thought I was
going to go out talking on stage
at WWDC this year until
a couple of weeks ago
when my manager came to me
and said, "Hey, Melissa,
you totally are the
one who's going
to see all the weird
esoteric bugs
that nobody else
is going to hit.
I want you to do a demo,
take one for the team."
So here we are.
We'll do it this way.
If my demo fails gloriously
on stage, I'll have found bugs
that we will then not be
shipping to you, guys.
OK. So here we have
a little application;
it's a note-taking application.
I'm going to add a note.
I need to add another note.
It's pretty simple.
Not much to see here,
but as important
as what you're seeing is,
what's more important is
what you're not seeing.
And what you're not seeing
here is, well, a hang as I wait
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
for addPersistentStore to return
with my ubiquitized store.
You can see that as I
was sitting there waiting
and as I was talking to
you, as I was working
with that responsive UI,
our iCloud integration off
in the background was going
off and fetching information
about our lab times today
-- for the next week.
So, you can see as Nick said,
this is the asynchronous
store setup.
We've made things a lot better,
a lot smoother, a lot easier
for you, guys --
one line of code.
So, because I'm me and
this is a demo on stage,
there was a good potential
something was going to go wrong
and I was going to
have to come in here
and try debugging whatever
had gone wrong on stage.
So this is, as Nick said,
the iCloud Debugging pane
and it comes up whenever
you are trying
to debug an application
that uses iCloud.
The top left-hand side, you
can see some information
about your application and
about your iCloud status.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
If you bring up this pane when
you're running an application
and all you see is the
little cloud on the left
and a thing saying,
"iCloud not enabled," well,
that would be why you're
not seeing any of the data
that you're expecting
to see from the cloud
because you haven't enabled
iCloud on your computer.
There's a storage meter,
it tells you how much
of your quota is there.
If you're not seeing data and
that meter is pegged red, well,
probably you're not seeing it
because the client
couldn't write it.
Below that, we have the
transfer activity gauge.
You can see some
green stripes there.
This coincides with
those notes I created
that were then pushed
up into the cloud.
Green stripes are
data being pushed;
blue stripes are
data being pulled.
Because this is a Mac and
it's [inaudible] up here
and it's been sitting on
stage connected for a while,
we don't see any blue stripes.
All of the data that
would've been downloaded
for this application was
downloaded well before
we launched.
And below that, we have
the documents in data view.
This is showing you a listing
of all the files that are
in your ubiquity container,
and you really don't need
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
to pay attention to the
stuff that starts with a dot,
that's our private stuff; you're
not going to be able to figure
out very much from there.
But under that, you'll
see some directories
that have an accountname.uuid
and these are your
peer containers.
And digging notes inside
those can be interesting
because that's where you'll
see the Core Data receipts.
And if you've got data coming,
you're debugging an application
on two pairs and you're
expecting data from one to show
up in the other, this is where
you'll find out whether or not
that data has actually
made it through the cloud.
If you've got five receipts
on one file and only three --
on one machine and only
three on the other, well,
that's some transactions
that have not
yet made it across the network.
Transaction logs appear in
these lines and you can see
over on the right-hand side the
status whether data is current,
whether we know it exists
but it's still in the cloud
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
because it's a large file
and we haven't managed
to download it yet, whether
it's current but waiting
to be pushed to the cloud.
So, and here's that using local
storage that Nick was talking
about -- logs that
Nick was talking about.
We're not using it in this
case; we're using the cloud.
So, on top of that,
Nick also talked
about account transitions,
about how Core Data has taken
over managing the iCloud
state transitions for you.
And, well, how to demo that?
Well, that's pretty easy.
You need to go off.
I'm going to go into
system preferences.
I'm going to hit
the Sign Out button.
And voila!
Automatically, we have noticed
that the account was signed out,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
sent you a notification
saying --
oh, by the way, that
file is going away,
and my application
received that,
and basically refreshed its UI
and you can see all of the data
that was in the cloud
is now gone.
That's because once you
remove an account, well,
you no longer have
that iCloud data.
It's off in the cloud,
not on your local system.
So our local store, because we
had not created any data before
we logged into cloud is empty.
So, well let's signing out
and you're probably
saying, "Well, sign out?
Yeah, that's easy.
How about sign in?
Can you do that?"
Well, we can do that, too.
And to show you that, I'm
going to actually switch
over to my other computer
here, and actually [inaudible]
that demo with the First
Launch Experience demo.
So, for those of you who
are extremely perceptive,
you might notice that up here
in the corner I'm not connected
to Wi-Fi, and those
of you who are
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
in the front rows
can probably see
that I'm waving Ethernet
cable at you.
This laptop is not
connected to the network
and has not been
connected to the network
since I airdropped
my project on to it.
So this is your canonical
first launch situation.
The application has never seen
the network, but there is data
in the cloud out there.
So how does this work?
Well, we need to run my
application and it comes up
and as we expect, it's empty.
We're not attached to the cloud,
no reason there should
be data there.
Create an offline note.
OK? It's a note.
And then I'm going to come
into system preferences,
but first I'm going to find
my Ethernet cable and plug
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
in my Ethernet cable and log in.
[ Pause ]
I don't care.
I don't care [laughter].
Do show me documents and data.
Let's see.
And it looks like something
has gone wrong, of course,
because something
would have to go wrong.
Oh well. You could see
that we actually --
oh, I see what's happened.
We're blocking waiting for
the initial sync to happen.
This is the log you'll see
when Core Data tries to go out
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and connect to the iCloud
account, will basically have
to block as we sync data in.
And eventually because,
you know,
you guys are probably
hammering the network
and that took a little while,
we come back and we say, "OK.
Now we're not using the
local storage anymore.
But you can see that offline
note I created is still there.
So, yes, this is what
we've been working on.
This is how we've been trying to
make your developer experience
with iCloud and your customer's
experience while using iCloud a
lot easier.
You've seen asynchronous
setup, it worked.
You've seen account transitions
to and fro and you've seen,
you know, how you might
go about debugging issues
that you're seeing with
a Core Data application.
And at this point I'm going
to bring my manager, Ben,
on stage to talk to
you about the changes
that we've been making to
the Core Data SQLite store.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
[ Applause ]
>> Ben Trumbull: Great.
Thank you very much, Melissa.
Good morning, everyone.
So I'd like to talk to
you a little bit today
about the changes we've made
to some interest structure
items underneath it covers here
with the Core Data and the
SQLite store, and there's one
in particularly, which is we've
changed the default journaling
mode that Core Data is
using with the SQLite store.
And this change is going
to be active for every app
that rebuilds against
iOS 7 and Mac OS X 10.9.
Now just to sort of recap,
we've been using the original
SQLite journaling mode,
the rollback journaling for
years now -- since about 2004,
and we're switching over
to the write-ahead logging
journal mode.
So this is going to be something
that apps need to accommodate.
So the difference is that
in rollback journaling,
SQLite copies the original pages
out of the main database file
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
into a file on the side just
in case something goes wrong
with the transaction, and then
it updates the main database
file directly in place.
And when it's done, in
order to do the commit,
it just deletes the
journal file.
For write-ahead logging,
it actually leaves the
main database file alone
and it just keeps appending
transactions to this WAL file,
and it uses the shared
memory file to keep track
of which page it --
the most recent copies
of each individual pages.
And then once you've aggregated
enough [inaudible] together,
a series of transactions,
several megabytes of data,
it'll perform a checkpoint
operation automatically
to merge the WAL file back
into the main database file.
So, to give you a little
diagram of what exactly is going
on here, generally, as you're
going along with the SQLite
in the rollback mode, you'll
only see the main database file.
But, when you start
a transaction,
you'll end up with this
intermediary-journal file
floating alongside of it.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
And then when you commit the
transaction, it'll go through
and basically delete
the journal file.
In WAL mode, you're going
to end up having all three
of these files around almost
all of the time, and this is one
of the behavioral changes
that you'll need to sort
of accommodate yourself.
So what ends up happening is
when you begin the transaction,
it'll start appending these
transactions to the WAL file.
And unlike delete mode,
when you come through
and you commit the transaction,
it actually does almost
no additional work,
so the transaction stays
appended to the WAL file.
And then, once you've aggregated
several megabytes of data
in the WAL file,
it'll come through
and then perform a
checkpoint operation.
And the checkpoint operation
will merge the WAL file back
into the main database file.
And then you may end up
sporadically in a place
where there is no WAL file, the
WAL file is being truncated,
but that's going to be a fairly
transient state of affairs,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and generally you're
going to see in WAL mode
that you have all
three of these files.
So, the reasons why
we're doing this is
that on the consumer grade
hardware we've experienced,
there have been some
difficulties
in getting the level of
reliability that we really want
to have across kernel
panics, and power loss,
and some very severe
failure modes,
and updating the
main database file
in place has been somewhat
difficult to manage.
So under WAL mode --
because that's not happening,
we're able to do a more
expensive sync operation that's
basically being managed
by the WAL
and the checkpoint
aggregating several megabytes
of work together.
We also see some very
substantial performance
improvements because
we're doing fewer syncs.
There isn't any actual
syncing going on in WAL mode
with the individual
commits, right?
So it's just appending these
things to the WAL file.
And this has actually been
a pretty big difference
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
on [inaudible] devices and iOS.
And finally, then we see
some improved concurrency.
Under WAL mode, it's possible
to have multiple readers going
concurrently with the writer.
So, I'm going to talk
about this in a second,
but it's actually much better
than the standard
reader/writer lock model
that rollback journaling uses.
And we've adopted this
internally at Apple
across the system and it's
seen some great benefits,
so that's part of the
motivation of rolling this out.
So WAL databases support --
or write going concurrently
with the readers as opposed
to blocking out readers
the way it does
in the traditional journaling.
Now for Core Data itself,
NSPersistentStoreCoordinator
still serializes each individual
fetch and save.
So to maximize concurrency using
WAL journaling within Core Data,
you're going to need to
have two Core Data stacks.
So for leveraging concurrency,
you have this SQLite file
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
that you're sharing between
multiple coordinators;
and you'll set up an
additional coordinator
and managed object context
for each of these types
of operations and they
can both be pointed
at the same SQLite store.
And with this kind of setup you
have, you can basically have one
of these coordinators doing
reads while another one is
doing writes.
Now when you're using
multiple coordinators,
you can't really pass
any objects between them,
so there is some friction there,
and this includes object IDs.
So if you want to
pass references
between these two stacks,
you'll need to use the
URI representation.
Nevertheless, this
is actually excellent
for doing background imports
where you have a very segregated
stack that's sort of siloed away
from the main UI and
everything else you're doing.
You can do importing, all kinds
of background batch changes,
and stuff like that, and
it allows the UI thread
to remain unblocked and to do
a lot of aggressive reading
or faulting and fulfill the UI
and the main thread while you're
doing a fairly substantial write
operation in the background.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So this is particularly
useful for responsiveness
around launch time
where you might want
to be importing changes
from some other data source
around the time the user
starts up the app again.
Now, there are some sharp edges
using the WAL database file,
but we don't recommend
this for any kind of file
that you're going to be
moving around yourself.
So if you're using file
system operations now
on the database file, then
WAL mode is probably not going
to be very compatible
with the code you have
because there are going
to be two additional files
that your code probably
isn't prepared for.
And we don't recommend
this for read-only files,
so there's been some
issues there
and this is primarily
useful for library files
that you're going to be writing.
And finally, we don't
recommend this obviously
for a document format
that you already have
if you have a standalone SQLite
file as part of your document.
Now, you can put a wall
of database inside a document
package, and that's fine,
but a lot of people
we've seen have sort
of free standing SQLite files
as their documents and some
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
of them is running
to some challenges
where the journal file itself
is also outside of any kind
of wrapper package, so that can
be a little bit of a friction.
We definitely recommend that
if you're going to use SQLite
for a document file, you put
it inside a package wrapper.
And something else to keep
in mind is the WAL journaling
really only goes back
as far as MAC OS X 10.7.
So, if you need to communicate
these files persistently
with older systems, you'll need
to use the rollback journaling.
So to get back to
rollback journaling,
the SQL command is
fragmentjournalmode=delete,
and to set this in Core Data,
you just set the
options dictionary
that you passed
addPersistentStore with type,
and you just pass in this
pragma on the journal mode
to set it to the delete mode.
And this will get you back
to the same behavior you
had in previous releases.
Now, some general caveats is you
really shouldn't be using any
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
file system routines directly
against any open SQLite database
file and we've seen a lot
of people try to try
to like delete files
or maybe they're trying
to make backup copies
or something like that.
And this really messes
up file locks and a bunch
of other things that
are going under the hood
and at its core using
NSFileManager
or at the POSIX file routines,
it's going to be
bypassing the SQLite API.
So if the files open, we
really strongly discourage this
and you have gotten
away with this
in the old classic
delete mode for journaling
but it's not going to work out
very well under WAL journaling.
Also for network file
systems, we've seen a bunch
of people try and
they keep trying.
But at the end of the day
-- at the end of the day,
the caching that's going on
in the kernel is not going
to be coherent between multiple
different physical machines.
Right? So it's OK to
have an SQLite file
in a network home
directory that's being used
by a single machine,
but sharing these files
across multiple different
machines simultaneously is not
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
going to end very well.
And finally, a lot of
people try to change
for performance reasons
the synchronous pragma,
and especially under WAL mode,
we've customized this behavior
for Apple's build of SQLite,
so please just don't
change it at all.
And if you want to continue
doing whatever it is you're
doing with the rollback
journaling, that's fine.
And as part of some
of this other work,
there are general
infrastructure changes going
on with the SQLite store,
in particular to integrate
with the power and I/O
changes, the throttling changes
that have been talked
about earlier in the week
to improve battery
life on Mac OS X.
So, we've done some extra
work in Core Data to integrate
with I/O throttling and improve
the amount of battery time
that SQLite is using if you have
multiple contending connections.
And finally, there's also a
guarded file descriptor concept
that's being used.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
We're not going to go
into this very much,
but you should be aware that
it exists and SQLite is going
to be making efforts to
prevent accidentally corrupting
databases by smashing
the file descriptor
by closing a file
descriptor that doesn't belong
to you and things like that.
So that's available on
Mac OS X 10.9 and iOS.
And so that's most
of the infrastructure
we've done this release,
and for more information,
you can contact Dave DeLong,
or send your email to
the Cocoa feedback group,
and we have documentation
in all the usual places
and we have some
related sessions.
Tim Isted is going to be doing a
Core Data performance talk this
afternoon and there
will also be a --
hidden gems in Cocoa and
Cocoa Touch on Friday.
Thank you very much.
[Applause]
[Silence]