WWDC2010 Session 118

Transcript

>> Good afternoon.
Welcome to Session 118, Mastering Core Data.
Please turn off your WiFi devices they make me nervous.
Much better.
I'm Miguel Sanchez, I'll be doing the first half of the
talk, and then pass it on to Adam Swift to wrap it up.
So Core Data has been around on the desktop since
10.4 Tiger, on iPhone OS since 3.0 last year,
and obviously on the iPad earlier this spring.
The purpose of this session is to help you
become more proficient with our technology.
We are, this is not an introductory session.
We assume a basic level of knowledge with the framework.
You could have attained that by reading the
documentation, or worked through an example.
You certainly don't have to be a super expert.
But we will not be covering the
most basic concepts of Core Data.
This is our to do list for the session.
And let's jump right in.
So let's start out by talking about
some tips and tricks for modeling.
Quick recap.
The modeling, the model is how you describe
your data to us, and we help you manage it.
So the better job you do at describing
that data, the more we're able to help you.
The vocabulary you'll be using to describe your
model are the entities, which more or less correspond
to the tables in your
store, attributes, and relationships.
And you want to make sure that you design your model, not
around an ideal representation of what your data could be,
but more around a practical implementation of how you're
going to be accessing that data with your application.
Once you give us a model the data will be
presented to you as instances of NSManagedObject.
Now you don't need to generate any code.
You can just use the standard class that we provide.
But we actually do want to encourage you
to start using ManagedObjects subclasses.
Why? First, by moving away from the KVC access patterns,
you gain a lot more support from the compiler to tell you
when you're accessing something incorrectly.
It's better to use a more direct set name
accessor method, rather than to set value for key,
and you might mistype name or something like that.
This improves your code readability.
And it's also faster at execution time, because
KVC doesn't have to do the dereferencing
of the method, of the state that you want to access.
Now when I say, please generate subclasses,
I don't mean write a lot of code.
All you have to do is the modeling tool
allows you to generate the stub for the class.
And it's really just a declaration of the properties, so
that the compiler can check them when you're accessing them.
Here we have three properties up on the screen, a first
name, a simple string property, to one relationship manager,
and a to-many relationship direct reports.
Remember that to-many relationships are
represented as sets within Core Data.
You don't even have to write the code for
the implementation for these properties.
If you use the @dynamic compiler directive, we inject
the code at run time, the actual implementations
of these methods for you, so you have; it's all
there for you, you don't have to write the accessors.
Another thing we've seen people have questions around in the
Core Data forums is, once you do the initial implementation,
the initial generation of your classes, what happens
when you have additional properties
that you want to add to your entity?
Do you need to regenerate your whole
classes and stomp over what you had before?
The answer is no.
Please be aware that the modeling tool
has this menu items, design data model,
and then the 4 items where if you select
a particular property in your model,
we generate either the implementation
or the header declaration
for that property, and leave it on the pasteboard.
Then you can go into your specific
file that you've pregenerated
and paste the appropriate piece of code that you have.
So you don't have to stomp, you don't
have to stomp over your preexisting file.
A couple of more tips for working with Managed Objects.
Remember that you are inheriting
from NSManagedObject and NSObject.
So be careful not to use property names that might
conflict with properties that you're inheriting
from these classes, such as Description and Deleted.
Those are the two most common ones
that we see people have conflicts with.
And also remember that, because these are KVC compliant
classes, it's not enough to avoid the property deleted,
but also isDeleted, getDeleted, and setDeleted.
So all the KVC resolutions for a particular name.
So the initial generation of your object is pretty
straight forward, you declare the properties.
But now let's start writing some more interesting code.
A property, a type of property that's
available to you with Core Data are transients.
Transient properties are declared in the model.
You get the benefit of change tracking at memory,
during memory while Core Data is managing the objects.
But their state is not persisted to the Store File.
So you're independent from the source schema.
You're able to add transient properties
without incurring a potential migration step,
because you wouldn't be changing the store schema.
Here's two examples, I mean I walked you through 2 slides
showing what people mostly do with transient properties.
The first is the simplest.
You're simply computing a full name
value from a first name and a last name.
So the first name and the last
name are stored in the database,
but your application requires the
display of this, it's a full name.
So you would write an accessor like the following:
you would see if you've already precomputed that data
by getting the primitive value of full name.
And if you don't have that value yet, you simply concatenate
the first name and last name, and you return that.
So this is a transient property.
You now have access to the full name, even
though full name does not exist in your database.
A more interesting use of transient properties, and
we've got some of this questions in the lab yesterday is,
to reference external resources, which you are
building yourself within your core data instance.
So let's say that you have a particular type that
Core Data doesn't handle, some document object.
You can declare a transient property document object that
depends on a persisted property, persisted document path.
So you are using Core Data to store the string
value of the path to access this resource.
And then you do whatever you need to do
to fetch the actual object from that path,
and you return that in your Core Data object.
So you're using a persisted property, so as a locator for
something that you're going to go out into the file system
and go load yourself in however mechanism you choose to use.
Another type of property that we have
available are transformable attributes.
You can see the list of types that
we support for the attributes
in Core Data from integers though strings, booleans.
If there's a particular type there that you don't
see that you want to use in your attributes.
You can always pick Transformable at the very bottom there.
A Transformable attribute is one
that is handled by Core Data.
We are doing the storing of your data.
But we're converting it into an instance of NSData,
where it's basically binary data
that we're putting in the store file.
But that data to you behaves as
whatever type you declare it to be.
So here, for example, we're declaring a property that's a
NSColor, and we don't have NSColor support in Core Data.
So you do the declaration of your property.
Color is declared as a Transformable property.
And we do the right thing behind the scenes of
archiving and unarchiving the color data for you.
But from your code you're dealing with colors.
When using Transformables we are using a custom,
I'm sorry, a default class of NSValueTransformer.
When would you want to use your own subclass of a valid
transformer and put it into your transformable class?
Core Data doesn't do any encryption of data.
So if you want to archive something out in binary
format, and you want to add an encryption step,
Subclassing ValueTransformer might
be a good way to put that code.
You might not want the default behavior
that we use with keyed archiving.
You might just want a straight write that bytes
data directly to the string that you're writing out to.
And if you do that, make sure that you take into account
endianess between your, the byte ordering of your data,
because your data might be stored in one
platform and fetched in a different platform.
So make sure that if you do, are
writing your own transformer
that you are taking into account the byte ordering issues.
The last thing I want to talk about
Transformable Attributes is
if you do Subclass NSValueTransformer this is the
convention that Core Data uses when calling those 2 methods
that you, the primary methods in ValueTransformer.
When we call Transform Value, we
are going from your type to NSData.
And we're using the reverse transformation
to go from data to your type.
So that's; people get this backwards some times.
There's an example on how we do
this, the photo locations example,
it's associated with this session
if you go to the attendee website.
The last thing I want to touch on in this section
is adapting the model to your access patterns.
So let's say that you have a problem of,
you want to search on titles of books.
So your initial inclination might be to define an entity
book with a simple title property that's a string.
And you want to search on titles.
So searching means contains, right, in database terms.
And you'll also want to strip out the
diacritical marks and case sensitivity,
so you do Contains with the brackets,
DC that's the diacritical stripping.
It turns out that Contains can
be pretty heavy, heavy weight.
This is fully ICU compliant, localization aware,
basically regXmatching at the database level.
So it's a pretty hefty operation.
Not only that, you're doing it each time
you're searching for each row in your database.
So if you have a large dataset you might
not get the performance that you want.
Here's the, a secret for you guys.
Apple, we ourselves on the phone specifically, on
mobile devices, we don't really do full text searching,
full searching in some of the searchers that you're doing.
We're doing prefix matching.
So this is actually very good for most
of the functionalities that you want.
So one trick that you can do is;
first, well actually two tricks here.
You can take care of the normalization of the
data, right when you're setting the title.
So you add a secondary attribute, normalize
title, where you do all of the stripping
of the diacritical marks and standardizing on a case.
And then you put that, you put your normalized title into
your book instance, and then you also index that property.
You also index the normalized title property.
And now you can write a much more efficient predicate
by searching on any normalized title that's greater than
or equal to the prefix that your matching
on, or less than the subsequent prefix.
For example, if your user is typing S T A R, star,
you can use the following predicate to much quicker go
and find taking advantage of the index,
and you get back a much quicker result
than if you're doing a regular contains predicate.
So we also show you how to do this with the direct
property example, also associated with the website.
One final step here, you'll notice that in the previous
slide we're now doing prefix matching on the title,
but we're only matching on the first, on
the beginning of the title, basically.
So what if you want to do the match
on any word on the title?
One thing you can do is split all the words in
your title into a separate title words entity.
And it's one-to-many relationship.
Do the prenormalization.
So you're storing the title words already normalized.
You're still using the same predicate as before.
But now the predicate is searching on title words.
So you're matching on any word in the title.
And whenever you find a match on the title word,
you can just use the relationship back
to point to the book that it belongs to.
Okay? So, this has been an example of how, what I meant by,
see determine what the actual usage
pattern of your application is.
Where is the bottleneck?
Where do you need the fastest turn around
time, and adapt a model to do that.
Let's move on to talk about the Managed Object Life Cycle.
Remember Managed Objects are the instances
of your data that we're providing for you.
They don't exist in a vacuum.
They always exist with regards to
the context that is managing them.
So anytime we talk about the life cycle of an object,
it's always something that's happening
with regards to that context.
So we're inserting a context where, I'm sorry, we're
inserting an object, we're inserting it into the context.
If we're fetching an object, we're
fetching it from the store into the context.
As we're updating our data, setting state on these objects,
the context is maintaining track of what's going on,
so that if we ask it to undo a change,
the context is doing that undo.
When we decide to clean up, there can be at a very high
level two kinds of clean up; either a direct deletion
that you are asking us to perform, because you're
messaging the context, please delete this object.
Or there's more memory level deletion where
we realize that there's no more references
to your instances, so we turn them back into false.
A thinner shell of those objects, that
doesn't use the full memory footprint.
So what are your options for hooking
into this lifecycle of an object?
You have three high level ones.
First is, if you're dealing with per instance
actions your best bet is probably to override methods
from NSManageObject, and I'll be talking about those.
If you want to react to graph level changes, you probably
want to register with the Manage Object Context and listen
for the notifications that it's posting
and react to those notifications.
And thirdly, you remember that as you're asking the context
to perform certain things, some
of those methods return errors.
So make sure that you're inspecting those return values
and reacting to whatever the context is telling you.
So let's go one by one in these.
If you're going to override methods in NSManagedObject,
let's say that you want to add
additional initialization code.
The awake methods, the awake set of methods that you
see on the screen here is our good place to do that.
Let's walk through each one of them independently.
awakeFromInsert.
awakeFromInsert will be called once
during the lifetime of your object
when you do the initial insertion
of that object into a context.
This is a good place to set baseline state for your object.
There will be times, remember that when you're creating
the Managed Object Model we have text fields for each one
of the properties that allow you to
set default values for your properties.
There will be times when this is not enough for you.
For example, if you are creating employees,
when you're doing the model you can't tell
what the next employee ID is going to be.
So the awakeFromInsert is a good place to put that
code where you're initializing the employee ID.
This is the awakeFromInsert from Managed Object;
your own subclass that represents an employee, right?
So this is something you can't put in a
model, so this is where you would put.
awakeFromFetch is very similar to awakeFromInsert, except
that it's called each time that your data is fetched
from the database and an object is created in a context.
What do you want to put here if
you ever override this method?
This is another good place to initialize
transient properties.
Some slides back I showed you the full name method.
That's sort of like an on demand
initialization of your full name property.
But you could also choose to set that
value up once you know that you have all
of your database state and you're awaking from a fetch.
Finally, during the lifetime of your objects
there will be situations that require us
to revert state to your object from a snapshot.
But specifically if you're asking us
to refresh the object or to do an undo,
so there will be times when the object will be
kind of reawakened with state that it had before.
You will be notified of this with
the awakeFromSnapshotEvents method.
This is a good place to put code that
might reset some of your cache data,
so that you can compute it on demand again later on.
So now we move onto the objects where,
I'm sorry, to the mechanisms that you have
to react to states changes for the whole graph.
NSManagedObjectContextObjectsDidChangeNotification
is how the context tells you that,
of what's going on with your edits.
It's post it's notification that informs
you, and a user info dictionary, you'll,
give you the list of inserted objects,
updated objects, deleted objects.
Please note that we're not telling you these are the
changes that have already been saved to your store.
We're only telling you this is what's going
to happen the next time you do a save.
Right? We've processed the changes in memory and this is
what we know about, but you still have to do the save.
When do we; when will you be getting this notification?
When is it posted?
It's posted if you explicitly tell the
context to process all the pending changes.
It will also be posted right before a save.
But it's also posted, very frequently, at the end
of the event loop and anytime we're doing a fetch.
So you're getting this notification
throughout the lifetime of your application.
Other notifications that are posted
by the ManagedObjectContext,
which are interesting for you to hook into.
When we actually do a save you will get the
NSManagedObjectContextWillSaveNotification
and DidSaveNotification.
Some of you have asked on the forums
about time stamping your objects.
So you want to put a timestamp on the
object right when it was last changed.
This is a good place to do that.
If you register to receive the
WillSaveNotification, you know that your graph
of objects that's being communicated
to you is about to be saved.
You can set the timestamp for each one
of those objects, and then we save it.
This is also a good place to set up relationships.
If you remember in the previous slides in the awake methods,
those were meant mostly for single instance manipulation.
So by this point more of the objects are set up,
so you can set up relationships between them.
Once the save does happen, you
will get the DidSaveNotification.
This is a good place to put your code that needs to notify
others in your application that the save has happened.
Let's say that you're managing
multiple context, multiple peer context.
Once the save happens and you want another
secondary context to know about that save,
you can start that messaging from
replying to this notification.
How would you do that?
Let's say that you have more than
one context in your application.
You do a save in one context and you want to communicate
all of those edits that just happened onto another context.
We have a method,
mergeChangesFromContextDidSaveNotification,
which the notification you're sending, by the way,
is the one you got in the DidSaveNotification.
You simply get that, take that notification, and then
you send this message to the other contexts that you want
to have the exact same changes that were
just saved, and we do the merging for you.
Now while we're in the topic of saving, don't forget
that the save method returns a boolean,
and it also has an error parameter.
So what kinds of things can go wrong when a save happens?
The first is validation errors.
Remember that as you were defining your model, you're able
to declare certain boundary conditions about the maximum
and minimum value of your properties,
or optionality of certain values or not.
So if there's any validation issues that we detect
while we're doing the save, we will fail the save
and communicate this back to you in the error parameter.
If there's more than one validation issue, they
will be chained in the NSDetailedErrorsKey.
So be sure to inspect the User Info
Dictionary inside of each error
and see if there's more than one validation problem.
The second type of issue that can cause the
save to fail are optimistic locking failures.
This is a mechanism that we use
to detect multi-writer conflicts.
When we're doing a fetch from the store we
keep around a snapshot of the last value
that we saw for a specific property in the store.
So if anybody else changes that value underneath us in the
store, now that's any, that somebody else could be you.
It could be another thread in your application.
It could be another peer context that you're managing.
It could be another application that's
still dealing with the same store.
Whatever it is, there's a change
in the store, you tell us to save,
and we detect that somebody's changed the value underneath
us, so we will raise an error and we will notify you.
The default policy that Core Data has
is to raise an error when this happens.
If this is not what you want to do, you can change
the merge policy on the Managed Object Context.
And you have here the set of policies that you can set.
If you want us to try to merge the changes from
the store and from changes that you have in memory,
you can use one of the, the second or the third policy.
And with each one of those you're
telling us who do we give priority
to when we detect a conflict on
a property-by-property level.
Do we take state from the store and stomp your memory state,
or do we state from the object and stomp your store state?
Or do you just want us to take the whole object
from memory and override whatever was in the store?
Or do you want us to take whatever was in the
store and override whatever was in memory?
So, you get to pick whatever policy that you want.
But please be aware that our default policy
is just to say this save doesn't work,
because somebody changed the data underneath.
Cleaning up.
Like I said at the intro to this section, there
is two types of clean up, one is deletion.
You're telling the context delete object, or delete.
Remember that the delete doesn't happen
until you do the subsequent save.
So you can tell a context, please delete this
object, but it's just marked for deletion,
it's not actually deleted until you do the save.
When you do the save you will be notified that the save
happened with the WillSave and DidSave notifications.
By the time you're getting those notifications, we've
already done the delete propagation in your graph.
So if you, you can't access the relationships
in your objects within those notifications.
So if you want to keep around; if you want
to kind of take notes about what's going
to be deleted for you to do secondary processing.
For example, if you're managing your own resources,
like that document path example I
showed with the transient properties.
A good way to plug that code is
in the prepareForDeletion method.
This is something you would override
in your NSManagedObject Subclass.
So this is when we're telling you, hey,
we're going to eventually delete this object.
This is where you would say, oh, this object is going away
and I'm managing an external reference
for this object myself.
So let me just keep track of this
path that will eventually be deleted.
So that when I'm notified that the deletion actually
happened, I will go ahead and remove the data.
The second kind of clean up that can
happen is kind of memory level cleanup.
You're not telling us to delete
objects; we're simply cleaning up,
because we don't see that you have referenced this to them.
So please don't override the dealloc method.
We don't quite guarantee at what
point that's going to be called.
The equivalent for you Core Data developers
should be they will turn into fault.
That's where you, that's the equivalent
of dealloc for you guys.
This is where you want to clear
out your caches or any dependencies
that you've registered for for a key value serving.
Now turning something back into a fault happens when
either Core Data detects that we don't have any references
and nobody has references to those objects, or
you're explicitly telling us to refresh an object
by calling the following two methods
on the ManagedObjectContext.
You're telling us to refresh the
object, or you're telling us
to reset the context and turn everything back into a fault.
Please don't call the refreshObject method and
tell us to ignore the merging of the changes
when you have a dirty object, because the
consistency of your graph will get out of sync.
The final thing I want to talk to you
about is multithreading with Core Data.
Most of the time most likely you will be considering
introducing multithreading into your application
to improve the UI responsiveness of your application.
So you want to push; you want to make Core Data
applications, operations asynchronous by pushing them
into a background thread, so that your UI is
free to continue to interact with the user.
Be aware that there's always pitfalls when
working with multithreaded applications.
It doesn't come for free.
So just because you spawn off numerous threads,
all of these have a little bit of a cost
as you're doing the context switching back and forth.
Make sure that all of, even though you have multiple
threads executing, they're not all contending
with the same resource, which defeats the purpose.
And you're also introducing a little bit of complexity
into your application, specifically with the debugging.
So this is not a free solution.
But you do decide to go down this path, so the
golden rule that we want you to always remember is
to give each managed object, each
thread its own Managed Object Context.
And I quote thread here, because you
could be using Grand Central Dispatch.
Basically each concurrent unit of execution
to get its own Managed Object Context.
Managed Objects are not thread safe.
You can't pass them around threads
and expect them to work properly.
What is thread safe is the objectIDs
that each one of those have.
So let me illustrate this.
You have a UI thread interacting with
the user and the background fetching.
The background thread is doing
the fetching of three objects.
You're ready.
You've warmed up the application.
You don't just pass those instances
over into the main thread.
You actually take the object IDs
from this object that you fetched.
You pass that across the thread boundary.
And then you use the method on the context, such as
objectWithID where we construct a local copy of that object.
Now fear not, you're not doing a fetch from scratch here.
Because the background thread was already warming up the
role caches that Core Data uses to create the objects.
So creating an object in the first
context is actually very, very quick.
You are taking advantage of the work that
you're doing with the background fetching.
If your background threads are inserting new
objects, please remember to first save them
to the store before you pass the object ID across
the thread boundary and ask us to fetch it.
When you create an object, we pass a temporary ID.
You can't pass a temporary ID to another context
and expect it to find it unless it's been saved.
So once you do the save the ID becomes permanent
and we can get it from the other context.
If you're doing this with Grand Central
Dispatch, the pattern is the same.
Let's say that you have a serial queue.
You know that blocks inside of a serial
queue will execute serially by definition.
So all of your blocks can potentially share
the same context, instance of context 1 here.
But, and this is a very important but, just because we have
serial queues doesn't mean that you don't have concurrency.
Right? You might have more than serial
queue executing at the same time.
So blocks within different serial queues
could potentially execute concurrently.
So make sure that if you have more than one serial queue,
the blocks in that second queue are
using a different instance of a context
from the blocks in the first serial queue.
And that's how you maintain the golden rule.
Of course if you're using a concurrent
queue, you know that by definition block 4
and block 6 could potentially execute concurrently,
so you take care of giving them different
instances of a context to manipulate.
The last thing I want to talk about
in this section is what happens
when you're doing edits to your data in multiple threads?
Well, what happens is that you
better know what you're doing.
I mean, a lot of this has to do with doing, defining
a work, a good workflow in your application.
Okay? This isn't so much Core Data's problem, it's
what does it mean for somebody to be editing an object
that was deleted in the background, or vise versa, right?
So a lot of that work is you guy's work.
Well once you figure out what it means in your
application, the two mechanisms that Core Data gives you,
and we've seen these methods before is, first if you want to
refresh the state of an object, turn it back into a fault,
you do the refreshObject with mergeChanges method.
Or if you did a lot of processing in a background thread,
in a background context, and now you want to push all
of those changes over into another
context in another thread,
remember that I mentioned the
mergeChangesFromContextDidSaveNotification.
So you're passing all of the objects.
Here, this is the one place where
objects are crossing the thread boundary.
But because this is being handled by Core Data,
we take care of doing the right thing
behind the scene so that nothing goes wrong.
So this is what happens with multi-party edits and deletes.
And now I'll pass it onto Adam to conclude the session.
[ Applause ]
>> Adam: Thank you Miguel.
And now I'd like to dig a little
deeper into fetching and performance.
So you know how critical performance is to providing
a great user experience for your application.
You want your user interface to stay responsive,
even as you scale to dealing with a lot of data.
And the two key strategies for achieving these performance
goals are limiting memory usage by only fetching the data
that you actually are going to show in your user interface,
and amortizing your data base I/O by fetching in batches.
So keep in mind fetching is performing disk I/O.
So you will want to avoid the extremes of fetching
too much data all at once, and on the other hand,
frequently fetching a little bit of
data and repeatedly calling out to I/O.
You want to find that right middle balance where you're
fetching your data in objects and reasonable batches.
And we can do that by leveraging
the strength of the database to do
as much work as possible at the database layer.
So we can use predicates and sort descriptors
to work across your entire data set
at the database level and keep memory and I/O under control.
So I want to walk you through a few examples
of how you can use predicates to do the work
at the database level, and keep
your memory and I/O needs low.
Let's start with an example of how you can avoid
fetching objects from a to-many relationship,
when all you really want to know is how
many objects are in the relationship.
You can use the account expression to avoid fetching those
objects, when all you want to know is how many there are.
For example, if you have a list of music playlists and
you want to find all the playlists without any songs,
you can use the @count expression, and a predicate
like this to look up those playlists without any songs.
And you won't be fetching any of the song data back,
you're just fetching the playlist that match that query.
If you want to work with the attribute value from
objects related through a to-many relationship,
you can use a SUBQUERY expression to access the attributes
owned by the objects in the to-many relationships.
And this gives you a powerful way to
test those attributes without, again,
fetching of any of the objects from the relationship.
So in this example we want to fetch all the
artists with songs longer than 10 minutes.
And we do that with a SUBQUERY expression that takes the
songs, the name of the relationship as its first argument,
and then tests if the song length
is greater than 10 minutes.
If the results of that SUBQUERY, if the songs returned
by that SUBQUERY, affect the count of the songs
that are returned by that SUBQUERY are greater than zero,
then we're going to return that artist in the fetch.
You can also work with and fetch attribute values directly.
So if you're only interested in fetching back
unique attributes from one of your entities,
you can fetch back those unique
attributes as read only dictionaries,
only fetching back the attribute
value without anything else.
You're evaluating that work in the database, and
only fetching back the results you're interested in.
So let's look at an example where we want
to fetch all of the unique album names.
So we tell the request we want it to return distinct
results, we want the results returned as dictionaries,
and all we want is the unique names from our album entity.
And when you execute this fetch, you'll get back an
array of dictionaries with all of those names in it.
You can go even further working with attribute
values directly in the database by calculating
and evaluating aggregate data on those
attributes and returning dictionaries
without fetching all of those objects into memory.
This is a powerful technique for performing
a lot of work at the database level.
So let's look at an example where we want to calculate
the total length of all of the songs in our music library.
So the first thing we need to do is create an expression
that represents the function we want to evaluate.
And in this case we want to take the
sum of the length of all of our songs.
Then we need to wrap that expression in an expression
description that tells our fetch how to encode
that information back in the dictionaries
that we're going to be returning.
And in this case we want the results
back as a double with the name totalTime.
The last thing we need to do is we need to configure our
fetch request to perform the fetch on the song entity,
only search for the property that we've constructed here,
which is the function to calculate the sum of the length,
and return those results as dictionaries.
Again, we're doing an incredible
amount of work at the database level,
and only fetching back the single
result we're interested in.
Sometimes you just want to know how many
objects are going to be returned by a fetch.
Either to display that number on screen, or to make space
for the objects that you're going to be fetching back later.
And you can take any fetch request and ask the context
for the count for that fetch request to get that value.
So in this case I'm showing a table that lists playlist
names, and we can use countForFetchRequest to look
up the number of songs for each playlist.
But then we can go a little bit further with
working in the database by using a sort descriptor
and setting a fetch limit to fetch the first three songs.
So now alongside the number of songs that
we've got, we can show our users a preview
of the first three songs in each playlist.
It's kind of improves the user experience, but
you're only fetching back a little bit more data,
even though you've got a lot of data under the hood.
So now let's take a closer look at what you're
doing when your fetching managed objects.
There are a lot of options available
to you on a fetch request
for how you're fetching objects and
what you're actually getting back.
In the case that you're fetching objects
that you want to use in your working set,
and you want to access the attribute data right away,
you want to fetch back fully faulted managed objects
with all of their attribute values pre-populated.
But you're not going to have to
fetch back all of the relationships,
even though they're fully faulted managed objects,
the relationships are still represented as faults.
So you're not paying the memory cost
for traversing too many relationships.
To get back fully faulted managed objects,
you need to tell your fetch request
that you want to ReturnObjectsAsFaults: NO.
So why would you want to fetch back faults?
Well faults are a very useful tool.
They're a very lightweight placeholder for managed objects.
And their attributes are fetched on demand.
And when you turn a fault into a managed object,
and it fetches its attribute values on demand,
it doesn't change its pointer address, so you're
still working with the same object in memory,
so you can keep it in the array that you
had before; where it used to be lightweight,
now it contains all of the information
from the managed object.
And there's also a middle ground called partial faults,
where you can fetch faults, but specify that you want
to prefetch or you want to specify
that faults should include some subset
of the properties from your managed objects.
So if we wanted to show a listing of song titles, but
we didn't want to fetch another heavier weight attribute
from the song entity, we could tell the fetch request that
we want properties to fetch to include only the title.
And the smallest representation for a
Managed Object is the Managed Object ID.
These things are really small.
Each Managed Object ID is only 16 bytes.
So it's actually possible to work with
a large set of Managed Objects ID's
that represent the Managed Objects
without taking up a lot of space.
As Miguel mentioned before, the Managed Object IDs
are also thread safe, so it's a great way to pass
that information between different threads.
They're also perfectly suited for using in predicates.
So any time you've got a predicate where you would
be supplying a Managed Object in the predicate,
you can supply a Managed Object ID,
and Core Data handles it just fine.
To get back Managed Object ID's you need to tell your fetch
request that you want the Managed Object ID result type.
And then you want to tell the request
not to include the property values.
And you might be thinking, wait a minute, I'm fetching
back Object ID's, they don't have property values.
But by default, the fetch request assumes that
if you're fetching back Managed Object ID's,
then you're probably going to want to use those
Managed Object values some time in the future.
So even though the Managed Object ID doesn't store
the property values, the property values are fetched
into the row cache, so if you later look up the
Managed Object for, or create a Managed Object
for that Managed Object ID, it doesn't need to
do a fetch from the database to get those values.
But if you really want to work with a large number
of Managed Object ID's and minimize the amount
of memory you're using, you want to make sure to
tell the request not to include the property values.
So back to talking about fully faulted managed objects.
If you're not just working with the
attribute values in your working set,
if you're not just displaying the attribute values in your
working set, but you also want to display related values
on screen right now, then you want to prefetch
that relationship, so that as you're displaying,
all of the managed objects in your working set, you're
not having to execute an individual fetch to fault
in those relationships, which, getting
back to amortizing a database I/O,
is incurring a round trip to do a fetch every single time.
So you want to take advantage of prefetching to get
those related values ready for your working set of data.
And I'll show you an example of how to do this.
If you want to show a list of playlist songs, and
alongside the song show the album name for the song.
You can tell the request that you
want to set the relationship keypads
for prefetching to include the album relationship.
So I've talked about a number of techniques you can use
to keep your memory usage low and amortize your I/O.
But what about the times where you
can't control the access pattern?
What about when you're trying to work with some sort of API
that it takes the entire array of
objects that you want to fetch?
How can you batch; how can you fetch your objects in
batches, when you're handing over the entire array?
You might think that you're either handing over an
array of faults, in which case they'll be fetched one
at a time and hit that frequent fetching pattern.
Or you're fetching everything all
at once and handing it over.
In which case you're doing that big
upfront fetch that you wanted to avoid.
Well the fetch request can do this for you automatically.
You can set the batch size, and when you execute your fetch
request, it will return an array subclass that's configured
to automatically fetch your objects
in batches as they're accessed.
And the way you do that is you tell your request,
set the fetch batch size to the size you want.
So I hope you've gotten some good ideas about
things you can do to improve your performance,
your fetching performance with Core Data.
But before you dig in and start making changes
to your code, I want you to use the tools
that are available to you to focus your efforts.
The Core Data instruments and instruments can pinpoint
exactly where in your code you're hitting those fetching
and faulting hotspots, so you can use your efforts
in the spots where you need to put the time.
And I also encourage you to absolutely take a look at the
header files and class documentation for NSFetchRequest
and its expression the Predicate Programming Guide.
And make use of the developer forums as well.
There's also, just searching on the net, you can come
up with all kinds of great information and resources.
So I'm going to wrap up this session today
by looking at the topic of migration.
First of all, why do you need to bother with migration?
Well, think back to the beginning of this session where
Miguel was describing that the data model is your contract
with Core Data that describes how your data
will be saved, and structured, and accessed.
So any time you go to make a change to your data model,
that's going to change how that data is saved and accessed.
So if you want access to your old
data, you need to adapt that old data
to a new structure, and you do that with migration.
In Leopard we introduced versioning and migration using
a custom mapping model that you could hand construct
with flexible logic to translate objects from your old
data model to objects in your new data model, in memory,
by fetching data from your old store with the old
data model, transforming them with the mapping model,
and then saving them to the new
store with the new data model.
In Snow Leopard and in iPhone OS 3
we introduced lightweight migration.
And lightweight migration works by looking at your old data
model and your new data model, analyzing the differences,
and inferring a mapping model automatically to
translate data from your old format to the new one.
As an enormous huge benefit to
this, lightweight migration is able
to perform this migration entirely in
the database using nothing but SQL.
So what kind of changes are supported
with lightweight migration?
Well you can add, or remove, or rename just about anything:
attributes, relationships, entities, all supported.
You can also change the numerical type of
attributes, so you can change an int to a float.
You can promote a relationship from a to-many, or from,
excuse me, you can promote a relationship from a to-one
to a to-many, and preserve the
related objects in the new data model.
You can't go the other direction, however, because
from a to-many to a to-one, there's no way to infer
which objects should be saved and which ones to let go of.
You can even make changes to the
entity inheritance hierarchy.
So you can add a child entity, or a new parent, or you can
even take two peer entities, create a common new parent,
and move properties up from each of the child
entities into the new parent, and into migration.
All the data from those entities will be preserved.
So what do you have to do to take
advantage of lightweight migration?
First, you need to make sure you keep the old data models.
We need this for two reasons.
We need the old data model, so that we can compare
the old model to the new one to infer the changes.
Second, we can't read the old data
without the old data model.
So before you go to make any changes, go to
the Design menu in Xcode, choose Data Model,
add Model Version, and start making changes on the new one.
The second thing you need to do is set the
options when you load your persistent store.
Set the migration options when
you load your persistent store.
That's the migrate persistent stores automatically
option and the infer mapping model automatically option.
Now if you've skipped over step
one you'll see an error like this,
Cocoa error 134130, "Can't find model for source store."
And that means Core Data couldn't find your
source model, so we can't do the migration.
So I said you can rename just about anything.
And I meant it, but you have to give us a hint.
You have to give us a hint in your data model, so
that we can tell when you're renaming something,
as opposed to when you've deleted
one attribute and added a new one.
So I'll show you an example of how this works.
You need to set the renaming identifier, and
we'll do that here to change a song's name,
and our old model to its title in our new model.
So you can see I've got the Xcode data modeling design
tool here, and I'm looking at the song title attribute.
And this is version 2 of our data model.
And I've highlighted where the naming
identifier appears in the inspector.
So all we need to do to preserve the data that used to be
stored as the song name in our new model as the song title,
is put name in as the renaming identifier.
A couple of tips to keep in mind when you're dealing with
lightweight migration, changing a transient attribute
to a persistent attribute is the same to
lightweight migration is adding a new one.
A transient doesn't exist in the persistent store.
So all of the same rules apply as
when you're creating a new attribute,
that it needs to be optional or have a default value.
Or for a new relationship it must be optional.
There's a lot more information available about migration
in the Core Data Model Versioning
and Data Migration Programming Guide.
Covers both lightweight migration and
the custom mapping style of migration.
Before I let you go there's one
more thing I wanted to highlight.
This is a technique that's incredibly useful
for adding back some of that custom flexibility
that you might miss, but doing it in lightweight migration.
And the way it works is with a post-processing
step that you use after migration.
So after you; the way it works is you open your store with
the migration options, check the metadata for a custom key
that you've chosen, like DonePostProcessing.
If the key isn't set, then you do your post-processing to
populate derived attributes, or insert or delete objects
that you want present or removed from
your second, your new data model.
And then set the store metadata, so that
you don't wind up post-processing again.
Then save the changes in metadata and you're good to go.
Now I'll show you a code sample to see,
so you can see exactly how this works.
First we open the store with the migration options enabled.
Then we check the store metadata for
our custom key, DonePostProcessing.
And we check to see if the value for
DonePostProcessing is less than 2.
If it's less than 2, then it's time for us
to update our normalized titles for books.
So we go ahead and do that work to
populate the derived attributes.
And then we make a copy of the metadata to update
our custom key, but preserve the other keys
in the metadata, and set it back on the store.
And finally we save.
A really useful technique for adding back
some of that flexibility that you get
with custom migrations in a lightweight migration form.
So I hope this session has given you some
ideas on how, the many different ways
that you can use Core Data to mature your application.
And I want to stress that you want to invest the time
to come up with a good initial model for your data.
And then Core Data will help you out,
as you need to adopt your application
with evolving access patterns and
incremental changes over time.
And if you do find yourself wanting
for some feature or encountering a bug,
please use the bugreport.apple.com
website to report those to us.
We read them.
And the more information you can provide to help
us understand and reproduce the problem you see,
or feature you'd like, the better
chance it has to be dealt with quickly.
For more information please contact Michael Jurewitz,
our Developer Tools Evangelist, jurewitz@apple.com.
And take a look at the Core Data documentation.
There's great programming guides, examples, and
tutorials, and they're always being updated.
And also take a look at the Apple Developer Forums.
And if you want even more focus on performance on
iPhone applications, come to tomorrow's session at 4:30
where Melissa will talk about Optimizing Core
Data Performance on iPhone OS in Presidio.