Transcript
>> Gregor Purdy: My name is Gregor and we're going to
talk about Building a Server-Driven User Experience today.
And I'm going to start out with kind of a scenario, maybe
not all of you will be in this scenario but hopefully,
a lot of you are, where the assumption is you've got
already some sort of a web property, maybe a store,
maybe something else, where you've got great content
already there and somebody is probably going to say
at some point "Hey, I know, let's build a mobile app.
Won't that be great."
And a whole conversation is going to ensue
about this, lots of ideas will be thrown around
and here's a little translation key I put
together for you about some of the things
that will probably come up in that conversation.
[Laughter] So we're going to iterate the design,
change requirements, right, dedicated team,
if you're dedicated you're small, tight deadlines
and you're probably paying for your own soda.
But it is going to be great and what we're going to
talk about today is some things that we've been looking
at from some development we've been doing internally for
both server side and client side for accomplishing this.
And so let's start with this idea.
Let's say that we have, in this case,
it's a store, it's a flower store,
and we really want to follow this vision, right.
Expand our profit by taking our existing content that's
web content and repurposing it in this new context
in a mobile application and-- but let's look
also at kind of the reality where we're at now.
Presumably, if you already have a web property, it'll
look something like this but not exactly like this.
You may have some web services internally.
You may even call internally proxies or otherwise out to
an external web service you're getting from some vendor
and you have a web application in the middle
that what we really want to do is say reuse all
that lower-level technology and instead of having a
web server, a web application in the middle serving
up HTML amount to your customers, let's replace that with
something very similar that's doing a content web service
for driving your mobile application and we'll do this in
order to reduce cost inside the market by leveraging assets
that we already have and changing
the fewest things possible.
This again assumes you have a services-oriented
architecture or you're in the middle of moving to one
or if you're building something, you know
that you're going to build it that way.
So what will be some of the requirements
at a high level for such an application?
Well, first, we want to have fresh content that's
true on the web, true in the mobile space as well.
We really want to drive people to come back on an
ongoing basis to the application both for direct revenue
for selling something or making money based upon
their interaction with the application directly
but also if you're using iAds or some
other way of serving ads indirect revenue
by people coming back to the app and doing these things.
And so we want to be able to do immediate
content updates wherever possible
where immediate might have different
definitions depending on each type of content.
We'll cover that in a minute.
We want to be agile, right.
We want to be able to have this
application that's out in the world.
It's in the hands of thousands, hopefully, millions of
people and we want to be able to have new content types
that we can push out to that application without in every
case having to rewrite the application or rewrite part
of it and push and update to the App Store.
We'll talk a little bit today about
some of the techniques for doing that.
If we don't rev the app, that's better for everybody.
I mean revving app is actually fairly easy, right.
The App Store is great, the updates are pretty easy but it's
better for everyone if you don't have to take that update
for every little change that you might make.
So what we're going to talk about in order today then
is we'll talk about service-oriented content delivery,
I'll cover with service orchestration, and
then we'll have Scott come up and he'll talk
about designing the client side in a flexible way.
We're talking about general enough to represent
different data types that we've talked about a second ago
and what we've been sort of referring to as remote
controlled but native UI and then we'll have David come up
and talk about some core frameworks that you can use to
manage the data both for efficient client-server in rations
but also just managing remote data and
how you might cache in on your system.
I'll come up at the end for a couple other
points that we'd like to make at the end.
This is the picture I showed a second ago just for context
but there are four main elements that I'd like to share
with you today on engineering the solution.
They're kind of the design considerations
that are going to come up.
We're going to talk about what are the kinds of content
and what comes up when you're talking about your content,
what are your architectural options for
actually putting this thing together.
There's a couple of different ways of doing it.
We'll talk about how the application can
respond to context of the customer using it
and we'll also talk about anticipating change.
And there's a little bit of overlap in some of these.
There are a couple of things that
come up a couple of different times.
So let's talk first about this content, right.
So you might have user-generated content, this
might be your existing web property, for example,
might be something where people are submitting,
it might be video, it might be audio,
it might even be stories, it could be anything.
Or you could be talking about user-generated
content also on your new mobile application.
In any case, you might have that.
You might also have curated content, right.
This could be content.
Maybe you're primary business is generating some content
and it's highly curated and it's great and it's wonderful
and people want to come to your app
specifically to get your content from your brand.
It could also be content in this category.
Maybe you have a third party vendor that you acquire some
content from and maybe you mash that up with other content
that you get from other parties or that you generate
internally, but more of a curated experience.
But in any case, commonly, but not always,
this might be accessed by a web service,
whether that's a web service provided by a vendor or
you loading in this content into your own database
and vending it out to your own web services.
This would be internal or external
as I said at the very beginning.
And having this content vary by
context, right, so the obvious content--
context would be the country the person is in
that's your customer or could be the language
that they're using that they set on their device.
These are the two most obvious elements of context.
This would come up with in translations, right, so we've
got English today and now we want to add Spanish and French
and German translations of the same content.
It might also be not necessarily
translations of the same content
but just different bodies of content in this situation.
And finally, in many real applications,
you know, we don't want people to just come
to the app and kind of browse around that stuff.
I mean, you can do that if you've got an ad-driven model.
But we might want them to do other things like for example,
manage their accounts, some kind of account with you
that content would be used to drive that experience as
well and you might want different translations overtime
and sensitivity to context for those as well.
Purchases, right, great direct revenue, in-app purchases
or purchases if you're doing sort of e-commerce style ad.
So, with all these kinds of content that we
want to be able to vend out wherever we got it
from and how we're storing it internally.
There's a couple of main ways to be able to
deliver that content in a mobile application.
The first one and actually, this was
actually a pretty good one and we've talked
about it before in previous WWDCs UIWebView.
UIWebView is great for fast time to market.
You can do almost an entire app this way.
You do need to be careful to still be sensitive to the
UI standards and don't need to do really unusual things
that people won't be able to understand
when they interact with your application
but you can do some great things with the UIWebView.
It's really a fancy shell around existing web content
that you may already have and whether that's a big chunk
of your application or maybe it's a small piece
that you get to a certain part of your application
and you got some content that you have that
you're not going to repurpose and rework it.
You want to just be able to drop that into the application.
You can use the UIWebView and do some great things.
Amazing HTML, we've talked-- other people have talked
about this in sessions at WWDC this year, right?
JavaScript, CSS and HTML5 gives you a lot of
options for doing great stuff in this environment.
But the main thing that we're going to talk
about today is building a native app, right?
And this is a great way to get a better user
experience, kind of native user experience without having
to work really hard at it within
the HTML, CSS and JavaScript.
Very easy to meet the customers expectations about what
it means to navigate in an application and feel right
to them given the other applications
that they've interacted with.
And obviously, stateful interactions are much
easier when you're doing a native application.
It's a natural thing to do, maintaining
state within your Cocoa Touch application.
Another great reason to use a native application is you can
leverage all these great frameworks that we have in a system
for things like coordinating calls to multiple web services.
There's some security limitations to
doing this if you're going to do it all
within a UIWebView making cross-site scripting sort of ways
of getting content from multiple places and mashing it up.
But when you're in the Objective-C world, it's very natural
and easy to go make three or four web services calls
as you're pulling content to be
able to show to the customer.
Obviously, custom rendering and animation.
There's a lot of this, a lot of sessions here at WWDC this
year that talk about custom views and custom rendering
and all those sort of things and Core Animation.
And also in-app purchases using the camera, you can--
the list goes on and on of things that you can
do in a native application that are either harder
or impossible to try to just leverage a UIWebView.
And so, we'll spend the rest of this time talking about
elements related to building a native application.
What I'd like to do now though is talk about in that
application that we're driving from the server side,
how can we respond to the context of the user?
We want to select content that is relevant
to that customer and an obvious way
to do this is to use the device country and language.
Let's get that piece of information over when
we're making our web services call to the back-end
and let the server side decide given that
information, what's the best content to give them.
If it has French content in there and a French language
setting then let's go ahead and give that to them,
but if we don't have French content, we can
still try to return them something, right?
We can return in English or we can
decide to say sorry, I don't have French.
That decision can be made on the server side.
You can use Core Location, right.
You can send that information over when you're making
your web services call to the back-end and use that.
If you've got a content that's highly relevant to certain
locations, given cities or even very small locations
where you can use that to drive different content
that you return, that's a great thing that you can do
to select content and be context sensitive.
Device type is another great thing, right.
So small device, big device, fast device, slow device,
these kinds of decisions you can take that information
of knowing what the device is that the customer is on,
send that to the server side in your web services call
and then use that to be intelligent about what
kind of content or the format of the content
that you're going to return back for display.
Another one that may come up as your
application ages through time is the app version.
Version 2 might have different capabilities than
version 1 and sending that piece of information
across in the web services call can keep you out of
trouble and make it easier to evolve overtime as well.
So that you can return, you do not return that
special piece of content that doesn't work
as well in 1.0 when a 1.0 client is calling you.
A great way to get started down this path is
to design your application in the beginning
to only have one baked-in URL into the application.
Don't bake in a bunch of different
endpoints that you're going
to make different web services calls for different things.
Bake-in one, one call that suffices to get
all the configuration and startup content
that your application needs to get going and all the other
information it needs to know to be able to write URLs
to other web services calls that
it might need to do to do its work.
This is also great just internally when you're doing
your development process, when you kind of switch
between you know, pointing at your development servers
and your QA servers and your production service.
If it's one URL in your app and from that
URL you get everything else that you need.
It's very easy to make that so you can kind of
switch between environments as you're developing.
Language fallback, I kind of already
talked about this a little bit, right.
So this is the idea of having somebody's
telling us they're wanting French
but we don't have French so we fallback to English.
The nice thing about building your application this way
where the clients doesn't really know about this is that if
on the server side and back in your content
repository you do eventually get French content.
The client never changes, right.
You don't have to build a new client
that's now French aware.
It just gets French content because all along
it's been passing along that piece of context.
It's just been getting back not
French content until you had some.
And again, it's the same thing with
adding new translations overtime.
All of this is really gear towards
what we said at the beginning.
Let's try not to rev the app.
It's not that it's illegal to rev the app but let's try not
to rev the app every time there's some
little thing that we want to change.
And have the least amount of baked-in content as possible.
And with all this content that we're talking about
and we're talking about there's different layers
in your architecture where the content might be cached.
So having web services at the back-end,
they're talking to databases.
They're probably going to cache some of the content.
You're coordinating content web service,
you kind of stick in front of everything.
It might cache things if you're trying to be
able to be smart about utilization of resources
and you might even have a content delivery network that
sits in front of everything that kind of keep large amounts
of traffic off of hitting your servers for repeat traffic.
And so, at each one of these layers, there's an opportunity
for content to be cached or not depending on a policy
that you control and that really
depends on how you want to set that.
It depends on how fresh you need that content to be.
Some application's fresh content
might be less than a day old.
Other types of content stock data or something like that.
If it's more than a few minutes old, it's old.
And it may have an application that the answer
isn't the same for all of your contents.
So having a strategy where you can control at all these
layers that in particular are also on the app side.
How long does each of these layers hold on
that content and how is that controlled.
One way to control this is to have the server side
with all of our other answers today, for the most part,
have the server side be able to tell the client, here's a
piece of data, please don't hold on to this for any longer
than 10 minutes or 1 hour or whatever
you think makes most sense.
And again, by controlling that from the server side, you
discover a week from now or a month from now, "Oh no,
we added a new piece of data today that really
doesn't make sense if we cache it for that long.
Let's change how long we cache it."
From that point forward, if you can just change it on the
server like normal web technology, you just rev the server
and all your clients don't have to
change to get that new cache site
and they just honor the settings
that are coming up from the server.
A couple of ways of doing this,
obviously, you can use HTTP headers.
That's a very natural thing to do
if you're using HTTP web services.
It might be something accustomed to, it
kind of depends on the situation you're in.
If you have already some internal mechanism for managing
cache ability in your services-oriented architecture
that doesn't use HTTP headers, you might want
to just reflect that up to the client as well.
A content delivery network might use the standard
HTTP headers or it might have its own custom headers
that use this to capture time to live data.
Anytime you're talking about caching,
especially at this many different layers,
each one adds to the underlying layers, right.
So you have 10 minutes here, 10
minutes there, 10 minutes there.
Sooner or later, you're talking about half an hour or more.
Then you think about what are the implications if some
of your content got out that shouldn't have gotten out.
Somebody put a bad version content out and had a swear word
in it or something that was going to damage your brain.
What would you do?
A couple of approaches to this.
One, there's joy in having low cache timeouts, right.
Because if you make them low enough, eventually you get
so low that even by the time somebody found a problem
and did all the processes that you might have
done to manage that emergency to fix that content,
that might already have been 30 minutes and if you cache
this on the 30 minutes, there's nothing else to do.
But if you are not in that situation and you
want to have content that is cached for longer,
you might need a mechanism where you
can force all those intermediate layers
to drop the content that they already have on the floor.
So there's a couple different ways of dealing with that
just by setting low TTLs or by having emergency process.
And one final thing it is not exactly
caching but one final consideration to think
about in the space though is you know,
real world servers need to be taken
out for maintenance or have problems, things go wrong.
And what will you do in your app
when it can not talk to the server?
You know, a lot of people just, you know, they
put up little boxes that's connected to server,
that is common, there is nothing wrong with that.
But if you took the server down for maintenance on
purpose or something like that, you might actually want
to have a way where you can substitute
if you're in maintenance mode,
substitute at the load balancer
level or something like that.
A piece of content that your application will
notice that might do some special behavior other
than just putting up an alert, I can't talk to the server.
If you want to communicate more intentionally
to your customer, hey, we'll be right back.
We're doing something great, whatever.
And that's really the core stuff that I want to talk about
in the server side is and I want to let Scott come up
and get into some of the more client side stuff and
then after him David and then I'll come in at the end.
Scott.
>> Scott Lopatin: Thanks.
[ Applause ]
Good morning.
Thanks for coming.
My name is Scott from Apple Store Engineering.
So we have our HTTP servers repurposed
delivering dynamic content
to our iPhone application, our native UI application.
How do we use Cocoa Touch to have
changing content rendered beautifully
on the iPhone device without having to rev the app?
The challenge here of course is, you know,
back in when you have your web application,
changing the web application once, all your customers
immediately get those updates the next time they log
in or go to your website.
The client is a little bit harder because you're
not guaranteed that everyone is going to go
and download the app on the App Stores as easy as it is.
We also want to leverage native Cocoa Touch UI to get
all the openness that app provides at the same time.
So I am going to talk today about some solutions that
we've come up with, little tricks that we can use
for who knows what is coming in the
future, at least as best as possible.
I'll talk about using property lists instead of HTML
to get dynamic content or changing data structures,
data that describes itself to provide the server
to redirect your app in any different way.
We'll talk about handling dynamic data when actually the
keys and the values of that data could change overtime.
I will show some examples of URL path
generation, how you can lead a user down a path
or like an assistant type flow dynamically.
And I'll round out with some flexible categories
that you might want to use when dealing with content
that changes overtime and some further optimization.
So let's get started.
So here's our flower store on the app, coming from the
App Store and typically, if this was a webpage you know,
the content would be coming from your web services
as HTML over HTTP connections and all that.
But what we find is, you know, using property lists and
having a web server's output text property lists is,
you have many advantages, so you can
serialize into all your NSDictionaries,
NSArrays, base types of strings and Booleans.
It also can be generated from many different
platforms and several different languages.
And then the client, there's one line to decode that into
these objects, which then you can do whatever you want with.
So here is an example of let us say changing a specific
item, let's say, on this UI of this flower pot store.
So, let's say you know, we have some content that you
definitely know might want to change in the future
and you want to control it so you can actually
send back hints or like description to Cocoa Touch
to tell it exactly how to lay out these buttons
and actually what view controller to go to,
what URL to load once the user taps on
that link or this button or whatever.
Well, remember this is not HTML and you can come
up with any key or value you want and plan for it
on the client so we can add things like device.
So, only when this is running on an iPhone I want
to provide a Call Us button otherwise people
can't call from an iPod touch or an iPad.
That's one specific element in your app but what
if you want to control the whole view controller,
which view controller the app goes
to based on the server response.
By doing something very simple and routing all of your
calls through one central kind of dispatch location,
your app can handle whatever the server decides
in the future where you want your app to go.
So here's an example.
We get a data type back and we add let's say a product
and this means that we're going to go to a product page
but in other instances, maybe you have a list
of products come back in its search results.
Your app can pick the proper view controller based
on these responses no matter what request is sent.
So this is a case where we have a specific set of keys and
values so far like we know how to interpret the responses
but what if we have cases where you might not know
all the possible keys and values that might come back.
We can support this too by doing
some Cocoa Touch trickery here.
We can also provide ways to write less code so a bunch
of keys and values come back and you might not want
to have instance variables for all these.
So how do we do this?
So here's an example.
In the US, let's say we have on this the flower store.
We have a way for people to enter their address.
You have address, city, state, ZIP, but maybe
you're rolling out to different regions.
You may have province or different address formats, how
do you support this in different countries without having
to rev your app and reference these keys by city or state?
One way you can load in this address
object but then you have attribute objects
which actually store the keys and values.
Then you can iterate over the attributes and list in
whatever order the server provides, let's say for the form,
and use these values but, still you might
want to reference some of these properties
in the code like first name might not change.
You know, people have first names everywhere
so you could use dynamic properties
and then dynamic properties let
you tie into when they're called.
Use methodSignatureForSelector to add
a method to the class as it's called.
There are lots of examples online of this.
And then using the dynamic property name,
look up in an object all the places--
the value for that based on the attribute list you have.
URL path generation, so there's other points maybe in your
application where you want to lead the user through a set
of kind of assisted flows where they pick one option and
then go to another screen and pick some more options.
What if these options-- what if the
number of screens changes overtime?
You can bake-in one beginning URL and then add or remove
steps by providing and sending back the URLs to follow.
So, let's say we have first screen with a few
options and Next button to go to the second screen
and then the final screen with
like a Done button, let's say.
So going through the screens, maybe these data changes,
maybe there are four steps tomorrow, maybe there's one step.
By baking-in, by returning the actual follow on path URLs
from the responses, your application can change remotely
without having to send a new version down the line.
As long as it knows what to do with all
these keys and values they come across.
It's also helpful sometimes you know you might have
certain, let's say like a flower here might have any number
of different information, let's say like bright red,
loves lots of water so these might
change, these might grow and shrink.
You can add some great UIKit categories to make this
kind of information flow nicely on any different screen
of your application without having to rev the app
because you can send in just the list of your views
so you have like a UIImageView, a bunch of labels.
Let's just return back what we're going
to display by just iterating over these
and making them the right size below each other.
So, now there are some further
optimizations I'll talk about.
For mostly, for mobile device like this is not your HTML
and your web servers used to having clients and there's lots
of optimizations that happen there but on a mobile device,
you might have a very slow connection you might want
to optimize these for whichever circumstance you're in.
One is like, so if you are-- you can-- on the server
side, you can provide different interfaces to your data.
Really easy in Java and other languages
but have like different versions
of your data come back only for
the specific place they're needed.
So if you have only displaying certain data on one page,
you might not run and return the whole object every time.
So having light and heavy versions of
your data objects might be helpful.
Also Gregor mentioned caching, cache control.
You can use HTTP cache control headers.
You can also define whatever you want in the property lists.
So here, we can actually put an expire flag
and tell us when our content has to change.
So this is useful when you cache these responses.
You might want to flip around to different screens on the
phone but still load the same property list without having
to refresh the data, saves a lot of network
traffic that makes the app feel really fast.
And then of course there are times when you have like,
let's say, a state list HTTP server and you want to,
not have to write any code on the client side but
have some way to know which user you are working with.
Remember, even though, it is a native application, you are
using HttpRequest and you can leverage things like cookies
which gets sent in every request to NSURLConnection,
you know, and the request handle this without any code
and they will automatically get sent back and forth with
every request without, you know, having to do anything.
But even still, with all this, there are some
cases where, you know, updates may be required
and new features could be added where, you know,
sometimes you don't know what you don't know.
So with that, I am going to hand it over to
David to talk about Core Frameworks for Data.
[ Applause ]
>> David den Boer: My name is David den Boer,
I am the Engineering Manager for Apple Retail
and my team is responsible for creating web and mobile
applications that are used in our retail stores worldwide.
Today, I am going to talk to you
about Core Frameworks for Data.
Using remote data in your application,
storing it, fetching it, and optimizing.
So, one of the considerations we want to look
at is the remote data types available to us.
There are many different data types
using common client-server interactions
and we are going to have a quick look at those.
We are also going to look at parsing data.
How fast is it to parse those different data types?
This will help us chose which kind of
data we want to send back and forth.
Next, we will look at client-side storage.
We're going to talk about, we've got our data and we want
to sometimes store that on the client for a very long time.
What are our options there and given that we want to store
stuff on the client, maybe Core Data is our best option.
So, what are the benefits of using Core Data for that?
And lastly, one of things I want
to talk about is given all of this,
can we build solution that automatically
handles client-sever interaction for us
and uses Core Data in the back-end with our data store.
So, let us take a look at the remote data types.
In a common client-server interaction, there are
several different mode data types you can use.
I am going to talk about a couple of
the standard ones that everyone knows.
So, looking at JSON, we've got in our flower application,
common flower attributes that we want to send down
and we want to store those flowers
because they do not change very often.
So, given a list of 500 flowers, that takes
388k to transfer that over the wire using JSON.
With XML, the same data takes 473k
to transfer over the wire.
So, using property list, which is a common format on Mac OS
X and iOS, in the ASCII property list format in specific,
we got 402k and lastly, we have also got
binary property list as an option available
and the same data with binary plist is 405.
So now, we've got the data on our client, one of the
biggest design considerations that you have to look
at is how fast is that data going to parse.
So looking at the same data again
with JSON and an open source parser,
we're at 416 milliseconds to parse that 500 element list.
With XML and NSXML parser, you
got a whopping 812 milliseconds.
That's almost twice as slow as JSON.
Now at property list, in an ASCII property list, you got
140 milliseconds, 1/3 the time it takes for JSON and lastly,
with binary property list, you're at a blazing fast 19
milliseconds to parse that same data on the iOS device.
Based on the data that we have seen,
maybe we are going go to property list
as our data type that we are going send back and forth.
So, taking a closer look the property
list, we have a small data size.
Compared to JSON and XML, property list is equal,
little less than XML, little more than JSON.
So, it is a reasonably small data size.
Property list, we get very fast parsing.
Obviously, with ASCII property list
we're at a very fast parsing speed
but with binary property list we're orders
or magnitude faster than XML and JSON.
Property list is also very easy to create.
If you've got WebObjects on your server side as
we do in Apple Retail, it's one line of code.
And because Core Foundation is open source, there are
many libraries available for other application servers
that allow you to create property list
and binary property list on the server.
Property list is also very easy to parse.
With Cocoa, as Scottie said, it's
one line of code and there it is.
propertyListFromData, mutabilityOption,
format, errorDescription.
So with property list we've got one line of code to parse
and again, with WebObjects, one line of code to create.
So we've got a very good solution there
for client-server interoperability.
So, now that we've got our data, we've got our of
our flowers and because these flowers aren't changing
on us very often, we want to store those on the client.
So, what are the options we have available to us to store?
So let's take a look at NSDictionary.
We want to keep this data in an
NSDictionary and use it on our app.
So, the pros of NSDictionary, obviously, it's very simple.
With a property list you actually get an array
of dictionaries for each one of those flowers.
But the cons of course is that there is no real persistence
for NSDictionaries and if you have very complex data,
it can become unmanageable doing all of these
value for key, value for key, value for key.
So we'll look at using data objects or just plain NSObjects
with attributes for each one of your flower attributes.
It's very simple as well and it's extensible.
You can add custom methods to your data objects very easily.
Of course, the con again is that there is
no persistence without writing it yourself.
So, one of the other options of
course that we have on iOS is SQLite.
Benefit of SQLite of course is that it's SQL.
You can do SQL fetching.
You can use qualifiers to fetch.
You can do updating with SQL in a batch manner
and of course, you get persistence with SQLite.
Of course, with SQLite, unless you are using a very good
third party library, it's very complex to implement.
Lastly, we'll look at Core Data.
With Core Data you've got a simple and powerful
API available to you that is well supported.
It has been around since Mac OS 10.4 and it has been
on the iPhone OS since iPhone OS 3.0 or iOS 3.0.
It's very extensible.
Your managed objects, you can create
classes for those and extend them
and provide validation and of course, you get persistence.
Now, a con of Core Data, if I can call it that,
is that it's not a pure database in the back-end.
The back-end store is abstracted way from you.
So you don't actually have access
to write SQL commands against it.
So, why would we want to use Core Data given all that?
Well, for starters you get persistence and we are
going to use persistence for data that rarely changes.
On our case, we've got 500 flowers and these
are our common products that we are selling
on our store and they don't change very often.
Those are the number one products that we're always selling.
So, we'd use that.
We're going to persist it.
We don't want to have to load that from the server
every single time that somebody looks for flowers.
We get efficient fetching and saving.
So, if we are storing all this data inside of our
application, the users can do a search inside your app
and rather than hit your server for the search,
we are going to hit your direct Core Data store.
It will be very fast.
With Core Data, you get change tracking and undo.
So if you are creating an order, you can create your
order in Core Data and if the user wants to remove an item
or undo something, you can do that very easily.
And Core Data because it's an object graph, you
get object validation and relationship maintenance.
So when you've got one object related to another and
you are adding and removing from those relationships,
Core Data is handling all of that for you.
Core Data supports key-value coding and key-value observing
so you can use key-value observing
in creating your remote UIs.
When data changes, you can have your UI changed.
And lastly, coordinates very fast on iOS.
So, let's take a close look at the Core Data
architecture, a very simple look at Core Data architecture.
So we start off with an NSManagedObjectModel and
this is where you are going to define your entities,
the attributes of those entities
and the relationships between them.
From your NSObject model, you're going to
give that to a PersistentStoreCoordinator
and your PersistentStoreCoordinator
is going to talk to your store,
and that can be in the back-end,
a flat file or a SQLite database.
But more often than not, you're just going to
be working with your NSManagedObjectContext.
This is where you create data, you fetch
data, you update your data and delete it.
And of course when you're going to fetch something,
you're going to use an NSFetchRequest and you can use
that NSFetchRequest to give it simple qualifiers or
complex qualifiers if you want to perform fetches
against your data store and retrieve
one or many objects back.
Given that Core Data architecture, how can we build on that
to produce a client/server data store,
and why would we want to do that?
Well, number one, some data changes rarely.
The data you want to persist, in our case
the flowers, doesn't change very often.
So we want to store that.
It's very easy to develop.
With the Core Data API, your developers don't
have to know anything about HTTP fetching.
They don't have to know anything about caching.
It just all happens in Core Data.
They perform a fetch and if you build it right,
that fetch can automatically fetch on the server.
So what can we do?
We can do automatic fetching from the server.
We can propagate our deletes to the server, so
when something is deleted from our Core Data
that can automatically happen on the server side as well.
We can do automatic saving to the server.
So, how are we going to do this?
How do we build this client/server
data store with Core Data?
So we start with a great foundation
obviously, and we start with Core Data.
And in some cases, you're going to need to
subclass the classes that are in Core Data.
So in our case, to do what we really want to do,
we need to subclass NSManagedObjectContext
with my myManagedObjectContext.
And then we need to add categories.
So we're going to add a category to NSFetchRequest
that allows us to do fetches on a server,
and we're going to add categories to
NSEntityDescription which allows us
to add extra attributes or helpers onto our entities.
And then, we actually have to subclass our NSManagedObjects
as well to provide one little bit of extra data needed
to let us know whether or not we're going
to be doing operation on the server.
Also need to update your model.
So in your ManagedObjectModel, you need
to then add attributes onto each one
of your entities in the userinfo dictionary.
It all starts at the entity level.
When you're making your entities in your
ManagedObjectModel you need to do a few extra things.
You need to decide whether or not your entities
are going to have a server-side counterpart.
So in your ManagedObjectModel, you might have entities
that are client only and some other client server.
And you need to know whether or not your entity
supports what I call the four server-side operations,
fetch, insert, update and delete.
So, how do we do these on our entities?
Well, they need helpers.
Your entities need to know whether or
not what operations they can perform,
what route they have to take to perform these operations.
So, in the userinfo dictionary, in our
ManagedObjectModel, we had a key called operations.
And in our case, what we've done is we put a bit
string in there and that bit string is just 4 bits,
ones or zeros corresponding to whether or not it
fetch, it supports fetch, insert, update, and delete.
We also need to add a route.
So given the example where your web services is on
the back-end or supporting our REST web services,
you need a route for that entity on your REST service and
we can put that in our userinfo dictionary for that entity.
So, if you have a flower entity and your
REST service to fetch flowers just happen
to be flower, you would put that in the route.
We then need to build a category on
NSEntityDescription with methods to get the route for this,
and whether or not we shouldProcessInserts,
updates, or deletes.
So next, we need to look at the managed objects.
So we need to create an NSManagedObjectContext
subclass with an override of save.
So when you're saving data to your local Core Data store,
you need to know whether or not
you're going to save on a sever.
So we override NSManagedObjectContext
save in order to do some extra work there.
And then we also override executeFetchRequest:error
because that is where we look at whether
or not this entity supports client
server fetching, and if it does,
we will then fetch from the server
rather than fetch from the client.
And we're also going to create a category on
NSManagedObjectContext to get a local instance
of this object in persistent store, and our NSManagedObject,
we're going to create a category which adds toDictionary.
We want to-- in order to do clients/server
operations, you need to serialize your NSManagedObjects
so that you can pass that up to the server.
So in our case, using plist as our
client server transport mechanism,
we want to serialize our NSManagedObject to a dictionary.
So we add a toDictionary method, and we
add a method for localInstanceInContext
which is basically just calls our
localInstanceOfObject method on this entity
or on this object with the past in context.
So, let's take just a close look at one
of the categories that we need to create.
So here is our localInstanceOfObject method.
We're adding this to NSManagedObject.
It's going to return an NSManagedObject,
and one of the things we need to do is get--
check whether or not this object is
registered in the current context.
And if it is, just return that.
You're done.
You got your local instance.
If it isn't, we're going to create another copy of this
object in the different context and then set the attributes
in relationships on this object on
to the new object and then return it.
So now that we built our entities and they
support client/server operations, how do we fetch?
How do we fetch from the server?
So by default, all of our fetches are client side.
To do server side operations, we need to
add an endpoint finding to NSFetchRequest.
So this is where the subclass of
NSManagedObject comes into play.
And what we do there is we just-- we create a
simple subclass that becomes our base class for all
of our managed objects and we added an attributed
called endpoint, and it's a transient attribute.
So that in our FetchRequest, we can then
just add, we can set that endpoint attribute.
So in this case, we want to find some stores.
We want to find the flower stores
that are nearby where you are.
So we are going to pass in the latitude and
longitude from Core Data or from Core Location
and our endpoint nearby stores on our server side, and the
endpoint specifies the method on the server to execute.
Obviously, everything is done with a base URL that is
passed down like Scottie told in our one config there.
And then we need to serialize that NSFetchRequest, and in
our NSFetchRequest, we're going to include the expressions.
So latitude equals latitude, longitude equals
longitude, and endpoint equals nearbyStores.
We're also going to serialize the sort orderings
so that when you send that fetch request to server,
the server can send it back ordered the way you wanted.
And we're going to serialize fetch limits.
So if you only want the first five,
the server can give you the first five.
So an example of what a fetch request looks like serialized
that is we can have an array of predicates and the--
or we can have array of expressions inside the predicate,
and that expression includes keep all your pairs
and qualifier operation to perform on your server.
So in our case, we're going to look for
flowerID where the flowerID equals 1231.
So, given our Core Data stack and that we're doing
client/server Core Data, how are we going to support things
like data that is persistent or transient?
Well, we do that with multiple persistent stores.
So one of your persistent stores where you're
actually keeping your data all the time,
like our flower database, is SQLite.
But if you're using client/server transactions
where you don't want to store that data
for a long period of time, you use in-memory persistence.
So you create another persistent store that is in memory.
You only need one Managed Object Model, the same
one, and you have them talking to both stores
and you can transfer your objects back and forth using
that category method we created localInstanceOfObject.
And to WebObjects people, they'll
know what that method does.
[ Laughter ]
So here we go, we've created now two persistent
store coordinators, two persistent stores.
One of them is in memory and one of
them is talking to the SQLite database.
So, I talked to you now about Core frameworks for data
and Gregor is going to come back up and talk to you
about a few lessons learned in
creating our flower application.
Gregor.
[ Applause ]
>> Gregor Purdy: Thank you, David.
Alright, so I'm still Gregor.
What I want to talk about here at the
end just for a few minutes is a couple
of things beyond the stuff we talked about at the beginning.
We talked a little bit about caching which comes up a
lot of times when you talk about-- dealing with scale.
And I want to talk about just four other elements of
dealing with scale beyond caching, call them the Four "M's".
The first one is Measure, we'll cover that
in a second, Model, Monitor, and Message.
Now, I'm making a few assumptions in covering
these things that we're all planning for sort
of large scale traffic on our systems and things like that.
It may not be true for everybody.
But even in a small scenario, some of these
things can be just as important when it comes
to preparing yourself to deal with
troubleshooting of issues.
You know, we're talking about computers.
Computers are made by people.
So, they have problems and you are going to want
to have some things to help you troubleshoot,
and so some of these are still
relevant even in a smaller scenario.
So, let's talk first about measuring, right?
So, this is the first duty for those of
us who are concerned about performance.
We can't do anything about performance unless we have
some measurements to help us understand what's going on
or what could be going on, and you can't do that unless
you do the work in the beginning to capture stats, right?
So, we're talking about possibly
stats captured on the client side.
You can definitely use that kind of information,
if not, in your deployed application certainly
when you're doing-- running in a development environment.
You can be logging various things in the client
side and they can be telling you in the log files
and the console what is-- is the experience
that it's getting talking to your services.
Is it getting timeouts?
What is the latency for various calls and things like that?
That's a great place to start just to help you see what's
going on each individual developer who might be running
into some kind of a problem talking to the
service while things are still in flux a lot.
But I really mean this even more on the server side, right?
Imagine we got a big data center,
we've got our web services back there.
There're multiple instances.
There're load bouncers, there's all kinds of things
going on in capturing stats on your web server side
where it is keeping track on its own of each call that
it's receiving from the client, and what was the latency
of that call and its internal processing
to get that response ready.
You can capture that data internally and have a way to
vend it out if somebody has access to your internal systems
like you, the developers can hit a different web
service call on that you don't expose to the world
to be able to fetch back what those timers are.
You can do this in a wide range of sophistications where
I've seen it done effectively actually with just duct tape
and bailing wire kind of versions of this data.
Even in production, it's way better
than the alternative of having nothing.
You can do really sophisticated things like in production
having your various timers and gauges and cache measurements
and things like that exposed via JMX if
you use JMX elsewhere within your company.
You can do things like getting that data off
of the host that's running the web service
and off into some data grid cluster.
They do interesting things with in terms of
capturing it down to a database or running it
through a charting package and interesting things like that.
There's a whole wide variety of things and the
main point of this is don't do nothing, right?
Do something in this space.
Get yourself started so that you're prepared for that.
The second one is modeling.
OK, great, so we've got these measurements.
Now, we've measured them on development or QA hardware and
we maybe had to generate some artificial traffic to do that.
But let's create a model.
You know, presumably, we hope, the business people
created some estimates and we think we're going
to do these many hits per month at our peak month.
And that's what we're sizing for, make sure we can do that.
And you may not be accurate but it's the best thing you got
to start with, and as engineers, we can always put a couple
of doublings in there or rounding up or something like that.
But start there, but also figure out, you know,
don't just look at modeling to the average, right?
OK, well, we've got a monthly number, how do we
turn that into a number reasonably of an estimate
of what our peak hour of the peak day looks like if we
got some spikiness to our traffic that we're expecting?
If you got a website that has similar traffic
patterns to what you're expecting in the mobile space,
what you might do is look at your web stats and look at the
relationship between kind of the average number for a month
and the peak day of the month and what is the
multiplier there in terms of request volume.
And then either within the peak day, you might look
at kind of what's the average hour within the peak day
versus the peak hour of the peak day, and that can at least
get you started of being able to back out of business number
into something that you can use as a rough
volume metric number to measure your stats on.
You also need to think about your traffic
from a pattern perspective, right?
So what do we expect?
Which pages do we expect people to use within the app more?
Everybody hits the first page, right?
Unless your app has some persistence on which page they
were last on and bringing them back in there, well,
you can still make some estimates of what we're thinking of
50 percent of our hits are going to be the first in the--
in the second page in that flow and,
you know, 10 percent are going to be off
on our orderings area and things like that.
Use that data to estimate given the traffic
that you think you're going to have a peak.
Which web service calls are you going to make
and what volume at kind of that peak hour?
And that can allow you to back
into-- are they fast enough, right?
Do I need, you know, 3 instances or 20 of this
service to be able to take all that traffic?
Then what are the implications for the other
services that our web service calls, right?
It can our ordering system handle that volume, hopefully.
And what does that mean for your SLAs, right?
It's nice if we're doing all these
measuring that we talked about a minute ago.
That's all flowing into some nice fancy system.
It's nice if you can express SLAs in there.
Even if they're not used for alarming, still having those
and thinking about the world in terms of SLAs can keep you
out of trouble because it keeps you focused on measuring
things and, you know, watching the chart and saying, "Well,
you know an alarm didn't go off, but
I saw that went out of SLA, came back.
Well, let's go look at that before
it becomes a real problem."
I've kind of cheated a little bit
by talking about monitoring mixed
in with the last couple, but do something for monitoring.
And again, this is the one that I've seen
done a lot with duct tape and bailing wire.
I've seen it done, you know, just hitting
the service through the load balancer
and just doing a random sample of
what's going on on those services.
You're just baking them out to some flat files.
It's way better than nothing.
Hook them up to something more sophisticated
if you have access to that or can find one.
But, when you're looking at doing your monitoring what's--
it's important to know are you in SLA or out of SLA.
That's very good.
But another thing that's really important
to know when you're looking at charts is--
and especially if you're a person who's making the charts
possible, arrange it so that you have some history shown.
So show, you know, yesterday along with today or last
week along with this week or something like that,
so that people can really see what the story is.
Because a lot of times, there's a
deeper story in the numbers, right?
You might see things like across the week you have one day
that for whatever reason it's not that the numbers go high,
it's that they go more erratic for some reason.
There's something going on Tuesdays, right?
And you won't see that necessarily
unless you compare things.
You know, look at, "Oh yeah, that repeats every week."
To know that you're trending just
very, very slightly up, up, up, up, up.
That can be hard to see unless you show
that context of what it was last week.
So, if you're able to please do show history to save
you a lot of trouble in finding more subtle bugs
that can bite you later if left to fester.
And finally, the first duty for
troubleshooting is message, right?
So, both on the client side when you're in development
mode, there's logging that you can do at console
and that's great, and you should to those things.
Make sure they're compiled out when you do production build.
But on the server side as well, you know, if you're
doing orders or doing things involve session or talking
to external systems where there's identifiers going back
and forth to identify things that you're doing with them,
make sure that when you do your logging, you
don't just say I had trouble talking to system X.
Provide the session identifier.
Provide URL that it was trying talk to.
Provide order ID plus session identifier.
Any of these bread crumbs that you
can leave for yourself will help.
You know, you may have first line
support, some people have that.
But they may call you if you don't have
this data in there for them to go look
at the other systems and see how things are going.
So, leave yourself clues on how to resolve problems.
And that's really the extra sort of tailing lessons
learned that I wanted to leave you guys with today.
So we've talked about service oriented content delivery,
how to leverage existing content that you have.
We've talked about designing a flexible client
that allows you to adapt over time at least
in certain circumstances without having to rev the client.
We've talked about some techniques for managing data
flowing back and forth between you and web services
and also efficiency considerations and
then-- in there-- There's more information.
You can contact Mark.
Here is his contact information, he can get you routed.
There're a couple related sessions.
There's Mastering Core Data today.
There's also Networking Apps for iPhone OS
or iOS, and Crafting Custom Cocoa Views,
which you've seen that we've talked
about a little bit here, on Friday.