Transcript
>> Hello, everyone.
Welcome to Adopting Multitasking on iPhone OS, Part Two.
My name is Charles Srisuwananukorn, and I'm a
software engineer on the iPhone Performance Team.
Joining me later in the talk will be David
Myszewski, who manages the iPhone Performance Team.
Now, let's recap a little bit what we learned in Part One.
Multitasking on iOS 4 doesn't mean that applications
need to run all of the time in the background.
In fact, we believe that most applications don't
need to run all of the time in the background.
Most applications only need to switch quickly between
each other, which we covered extensively in Part One.
But, we recognize that there are some applications
that do benefit from executing in the background,
and that's what we're going to focus on in this session.
We're going to survey several new services in iOS 4
that you can use in your apps to run in the background
and implement features that just
weren't possible before in iPhone OS 3.
In particular, we'll look at the task completion service,
which allows your application to request some extra time
to run in the background as it enters the
background to complete a task on behalf of a user.
And then, we'll look at the background audio services,
which your applications can use to play audible content
to the user in the background, and
respond to the remote control events.
And then, we'll look at a few location based services.
The first is a navigation service
for turn-by-turn navigation apps.
And, a couple of location tracking
services, so your applications can respond
to location changes while in the background.
And finally, we'll look at some Voice over IP services
in iOS 4 that your applications can use to make
and receive phone calls while in the background.
Now, let's get started with task completion.
As I said earlier, task completion allows
you to request some extra time to run
in the background to complete a task on behalf of a user.
So, when the user taps the Home button, for example, if
your application was in the middle of uploading some photos
or videos, or applying an image filter to some photos, or
even downloading a digital asset like a magazine or a book,
you can continue running in the background before
being suspended, so the user doesn't have to wait.
In iPhone OS 3, your application would have had to
remain in the foreground while it finished these tasks,
and so the user would have had to wait for your application
to finish these tasks before using the
device or using another application.
With task completion in iOS 4, you can do this in
the background and the user doesn't have to wait.
But, if we let applications use the task completion API
to run arbitrarily or indefinitely in the background,
that would be really bad for battery life.
So, the amount of time you get to run in the background with
this API is limited, to prevent excessive battery drain.
So, let's review the states that your application
goes through as it enters the background.
On the left there, we have the state diagram
that we saw in Part One earlier this morning.
When your application is in the foreground, it's in the
active state, and then when the user taps the Home button
to dismiss your application, it transitions from
the active state into the inactive state briefly,
and then into the background running state for a short
period of time, where it can prepare for suspension
by saving out the application state
and reducing its memory usage.
And then, after that short period of time in the background
running state, it transitions into the suspended state,
where it's not actually executing,
but it is still resident in memory.
But, what if your application was in the middle of
doing something; for example, uploading some photos?
Well, with the task completion API, you
can ask the system to keep the application
in the background running state
longer than it normally would.
And so, you have some time while you're running there in
the background running state to finish that photo upload.
And then, once you've finished the photo upload,
you tell the system that this task is complete,
and your application will transition from the
background running state there into the suspended state,
and then it stays in the suspended state
until it comes back to the foreground.
To use the task completion API, all you have to do
is bracket the operation that you'd like to finish
in the background, with calls to begin background
task with expiration handler, and end background task.
And, we'll talk a little bit more about
what the expiration handler is later.
But, in this code example here, assume the upload
photos method implements the photo uploading.
Well, before we call upload photos, you simply
call beginBackgroundTaskWithExpirationHandler,
which will return to you a handle for the background
task, and this tells the system that you're
about to start an operation that
shouldn't be interrupted in the background.
And, the system will try its best not to suspend your
application while that background task handle is active.
And then, once upload photos finishes, you simply call
endBackgroundTask, passing the background task handle,
and then the system will suspend your application.
But, as I said earlier, you only have a certain
amount of time to finish this task in the background.
And so, in our photo upload example, say the photo upload
is taking a little bit longer than it normally does.
And, we run up against this timeout
that we have for task completion.
Well, if you don't call endBackgroundTask
with passing it that background task handle,
the system will then terminate your
application and generate a crash log.
And of course, when that happens,
your application transitions
from the background running state
into the not running state.
To help prevent this ungraceful termination, you can
register what we call an expiration handler with the system.
And, the expiration handler is just a block of code
that's executed immediately before you hit the timeout.
And, when you get this call back, you can then
pause that long running task or the background task,
and then call endBackgroundTask
to let your application suspend.
Now, you should be aware that this expiration
handler may actually be called on a separate thread;
on a different thread from the one
that's executing the background task.
So, you may have some synchronization
issues that you need to be aware of.
And so, again, because you're about to be suspended,
you should prepare for suspend by saving state
and reducing memory usage, as you saw in Part One.
And then again, just pause the background
task; in this case, the photo upload.
And then, call endBackgroundTask
to let your application suspend.
Now, here's the example we saw
earlier, in a little bit more detail.
Again, we have the upload photos method, but imagine
now that this upload photos operation can be paused.
And, when we pause the upload photos operation, it
remembers that it was uploading; it had uploaded,
say three out of ten photos, and then it
returns from this upload photos method.
And, here is the expiration handler,
and beginBackgroundTaskWithExpirationHandler
takes a block as its first parameter.
And, if you haven't seen blocks before, the syntax may
look a little bit odd, but for the purposes of this talk,
it's similar to just passing a function pointer
to begin background task with expiration handler,
except we can define that function in line,
as we do here between the curly braces.
Now, there's actually a lot more to blocks than
I've just said, and if you'd like to learn more,
I encourage you to come to the Introducing
Blocks and GCV on iPhone OS talk later this week.
Now, back to the example here.
This block here, these two lines here are the
expiration handler for our upload photos example.
And, when the upload photos process takes too long
and the system is about to timeout the operation
and terminate the application, it calls back the expiration
handler, which executes prepareForSuspend and pauseUpload.
PrepareForSuspend is a helper function I have here that, say
just saves the application state and reduces memory usage,
because we won't get another chance to save out
our state after our application has been suspended.
And then pauseUpload there is the method that
will actually pause the photo upload, and again,
this expiration handler may run on a separate
thread from the one that's running upload photos.
And so, pauseUpload there handles all of the
synchronization and messages back to the upload thread,
and then the upload thread returns from upload photos,
and we call endBackgroundTask,
and then our application suspends.
Now, there are a few things you should
be aware of with task completion.
When your application is running using the
task completion API's in the background,
the system will try to prioritize all foreground
activity above any background task completion activity.
In particular, your application will be running
at a lower CPU priority, and your network
and file I/O will be prioritized lower than the
foreground applications network and file I/O.
And again, as we saw in Part One, the GPU and real-time
threads are off limits to background applications,
and so if you try to use the GPU, the system
will actually terminate your application.
That said, you should still try to minimize your
resource usage as much as you can in the background,
and of course, this involves CPU and memory, as always.
But, it's also important to end the
background task as soon as possible.
So, as you saw in the upload photos example, we called
endBackgroundTask as soon as endUploadPhotos returned,
whether or not we were about to
hit the background task timeout.
And then, it's also important to make the background
task resumable in case we do hit that timeout
and the expiration handler is called,
and we suspend the application.
And finally, it's also important to avoid that
ungraceful termination due to timeout by making sure
to end the background task in your expiration handler,
or as a consequence of calling the expiration handler.
Now, I'd like to hand it over to Dave Myszewski, who's
going to tell us a little bit about background audio.
[ Applause ]
>> So, I'm David Myszewski.
I manage one of the iPhone Performance Teams.
And, we'll talk a little bit about
background audio, and in particular,
in this section we'll focus on media playback applications.
There are many different ways in which you can play
audio in the background, and this one will focus on kind
of the long term, canonical media playback
apps, something like Pandora that you saw.
So, this type of app will play audible content to the
user, while streaming the audio data over the network,
and of course, continue playing audio in the background.
And, one of the great things about our Multitasking
UI in this whole Multitasking solution is
that we integrate really well with the remote controls.
So, as you see in the screenshot there, when the user
swipes to the left, they have these media playback controls
that are available to them, and these controls are available
to the application that's currently playing
audio, or the last app that's playing audio.
And, all of these remote controls handle a few different
types of controlling the audio, of media controls.
There's the multitasking UI; there's,
of course, the lock screen controls.
And, of course, integration with the headset, so we
can double-path to move to the next track, for example.
So, we'll give you a very brief
overview about the audio system.
The main thing is that the audio system
kind of acts like an air traffic controller
to make sure that we have the best user experience.
And, we do things like prioritize audio, so that
the right audio, the audio that the user expects,
is playing at any given point in time, that
some audio mixes and ducks when necessary;
we'll describe an example of that in a moment.
And, of course, integrating well
with headsets and external speakers.
So, today, like I said, we'll focus on the audio
services for a media player style application.
So, to adopt background audio, it's fairly simple.
The main thing is there's an info plist key.
And, the info plist key, you just choose
the app Plays Audio Entry; and then,
as long as your application is playing audio with
certain audio categories, it will continue playing
when the user puts the application into the background.
So, to give you an idea about what the application
life cycle looks for one of these applications,
when the application is in the front,
it's of course in the active state.
And then, when the user hits the Home button,
the application transitions from the active state
through the inactive state and
in the background running state.
Now, because the developer had chosen to add the audio
key list entry, the UI background modes, to the plist,
the application continues playing in the
background, and you'll note that there's a Play icon
in the upper right corner, denoting that there
is an application that's actively playing audio.
Now, as long as the application is playing
audio, the application will continue to run,
so when the user presses the Home button, it'll continue to
run until the point at which, say the user pauses the audio.
When the user pauses the audio, then there's
no longer a need for the application to run,
so when the music is paused, we transition the
application from the background running state
to background suspended, to preserve system resources.
So, how does all of this work?
Well, every application can have an AV audio session
of singleton object, and some sort of delegate.
Now, you need to set the correct
category on the AV audio session,
which in this case is AVAudioSessionCategoryPlayback.
This is what you would use for a long
term media playback sort of application.
And, this particular category has
a certain number of behaviors.
It silences other audio, so if, for example, the user was
playing the iPod app, and then they started using your app,
you wouldn't want the iPod app to continue playing;
you want your app to be the one that's playing.
So, this category says; well, we should silence
the iPod audio so that your music can play.
It also allows you to play the audio behind
the locked screen, which is really desirable.
When the user locks, if your app's the
front most, you want it to continue playing.
We ignore the ringer switch, and
of course, for multitasking,
we continue playing the audio in the background.
So, all of these behaviors are
defined by the audio category,
and audio categories help us get
the right user experience here.
If you're a background audio app, you need
to handle audio interruptions appropriately,
so that we get the correct kind of
behavior for the user experience.
So, audio interruptions can happen in a few
different places; we'll describe one in a moment.
But, during this interruption, the audio
system will silence your application.
And, in that case, say you're running in the foreground.
You'd want to, of course, update your UI appropriately
so that when they go to the app, the UI looks correctly.
And, of course, you may want to resume
the audio after the interruption,
if the audio system says that your application should.
So, how does this all work?
Well, again we have your audio session
delegate and the AV audio session objects.
If there's an incoming phone call and your application
is playing audio in the background, so, it's Erwin.
So, Erwin's our good friend; we're going
to accept the call and talk to him.
When we hit the Accept button, then the AV audio session
will send a beginInterruption event to your application,
and this denotes that your application was interrupted.
And, you should do a few things here, of course.
If you were busy downloading a stream from the
network; well, your audio isn't going to play,
so you should stop downloading the stream.
You should update the UI.
Maybe your app was in the foreground, and so
you should toggle the Play/Pause button, et.
cetera, just like you would; or,
the Play Time, things like that.
And, of course, if you had some
sort of, maybe 3D visualization,
you'd want to stop the visualization when this happens.
So, then your audio will be interrupted, and
then let's say the user hits the End Call button.
So, the user is no longer talking to the person.
Your audio isn't playing.
Now, the AV audio session will send you an endInterruption method; an endInterruption with flags.
And, this will tell you that the interruption is over and
it's maybe time for your application to continue playing.
In this particular method, there's a flag; the
AV audio session interruption flag should resume,
and that's a flag that's set by the audio system
that tells you whether or not you should resume.
So, as an example, for the phone call
case you would want to resume your audio.
But, if the iPod were the one who interrupted you,
then when the user stops playing the
iPod, you wouldn't want to resume.
So, as long as you implement the right behavior with
this flag, your application will work really well
with the audio system, and everything will
work just like the user really expected it to.
So, we have these nice media playback controls, and
we give you some API's to take advantage of these.
As I said before, that the last app that
was playing the audio receives the events.
So, if the user paused your app, you
know, five hours ago and decides; boy,
I want to hit the Play button,
I want to listen to some audio.
The last app wins, so the user will
start playing your application's audio.
All of these events are routed through the responder
chain; we'll show you which method to implement here.
And, one of the things about audio behavior is, like we
showed before, once the user pauses your application,
the application remains in the suspended state
until the user either relaunches your application
or uses the media controls to play your application.
So, how does this work?
So, the user paused their application a few hours
ago, but the app is still alive in the background.
So then, when the user; so, you're
in the background suspended state
and then the user presses the Play
button in one of those media controls.
Then, we resume your application for just a little bit
of time to allow you to handle this incoming event.
So, we'll move your application
to the background running state.
Then, let's say that you saw that;
oh, this was a play event.
If it was a play event, you would start playing audio.
And of course, if you start playing audio,
because you're a background audio app,
you will continue running and playing
audio in the background.
So, that's how the app life cycle works for
a media playback app with media controls.
So, how do you implement this?
Well, it's fairly easy.
You have a UI application method called
beginReceivingRemoteControlEvents.
This tells the system that your audio application
is interested in receiving these events that happen
from the multitasking UI, the lock
screen controls and the headsets.
And then, somewhere in the responder chain, you
implement this method; remoteControlReceivedWithEvent.
The documentation for this is in UIEvent, but
there are several different event sub-types
that you can handle appropriately, so one to
toggle Play and Pause, one to go to the next track,
one to go to the previous track, and
there are about a dozen of these.
And, you should implement any of these that are specific
to your application, that make sense for your application.
It's also important to use these sub-types for exactly
the kind of behavior that the users would expect.
So, Play/Pause should play/pause; you
shouldn't overload it with a different meaning.
That way, everything behaves as correctly;
Play button really plays, et cetera.
So, this is the method that will be called to handle
the event, if say your application is unsuspended.
So, for the background audio solution, we make sure
that you, as developers, have access to all of the tools
that you need in order to play background audio.
So, if you, for example, are an
application that synthesizes audio;
those applications have some really hard real-time deadlines
that have to be made, so we make sure that we allow you
to play those kinds of apps, because we
want those kinds of apps to be available.
And so, as part of this, it's really
important to minimize your resource usage,
just as it is for any other background application.
You know, we'll make sure that you get the CPU,
the file I/O, the networking that you need.
And then, on the other side, if you can minimize your
resource consumption, then it'll be a better experience
for all of the users because this is one system, and
so you should reduce your memory usage and CPU usage,
since we have a lot of shared resources
that we need to have.
And, a lot of the audio apps are really good about this.
One important bit; if you're an application
that's streaming data to the network,
it's really important to request
and download data in chunks.
So, if I have, say a three-minute audio track,
it's probably a good idea to try to download,
say all three megabytes of that audio track.
And, the reason is, particularly over 3G,
it's very expensive to send and receive data.
The radios are some of the most
power hungry parts of the system.
So, as an example, let's say that I downloaded
maybe, I don't know, 30 seconds of the music.
So, that's denoted by the orange box.
Then, the red box shows how the 3G radio behaves.
So, the cell towers tell the 3G radio to stay in a high
powered state shortly after the transmission is done.
So, there's some period of time after which the last
byte is sent or received for which the 3G radio will stay
on high power, because the cell towers said
that we needed to stay in the high power state.
And then, at a certain period of time, the cell
towers will tell us to go into a lower power state.
So, what that means is that if we're trying to
send and receive, say a few bytes at a time,
then our 3G radio power will be pretty
high for a pretty long period of time.
So, what you really want to do is just request data in bulk.
Get all of the data that you can, and that way the 3G
radios kind of are, you know, are in the right power states.
So, you want to send and receive data in bulk, and
that's true for anything; it's not just background audio.
Send and receive data in bulk will be better for power.
Now, the last thing about background audio is, of
course, we provide background audio recording as well.
And, this works conceptually just the
same way the background playback does.
There are a few quick differences.
One is, of course, you need to use a
different audio category; the record category,
which tells us that you're recording
audio in the background.
And, on your behalf, if your application
is recording in the background,
the system will create a double-height status bar that's
read, so that the user knows that there's some application
out there recording audio; so, for privacy reasons.
And then also, we provide great functionality to tap the
status bar, so that the user can return right to your app,
and then maybe stop the recording or make a new recording.
So, these are a couple of the capabilities
that we provide for background recording.
And, so that's Background Audio, and Charles
is going to show you a quick example here.
>> Charles Srisuwananukorn: So, I'd like to demo
now how to create a background audio application.
But, since we only have a limited amount
of time, I'm going to cheat a little bit
and start with some example code from developer.apple.com.
So, let me switch over to the device here.
And, here I have a slightly modified version of
the AV Touch sample from developer.apple.com.
And, you'll notice that if I tap Play, [Music] the audio
starts playing and it shows the current track name,
the play time of the track, and it has this nice level meter
visualization and actually uses GL to draw the level meter.
But, if I tap the Home button now, to send it to the
background, the audio actually fades out and stops.
So, let's look at how we can modify AV Touch so
that its audio continues to play in the background
and it behaves well as a background audio application.
So, I'm going to switch over to Xcode
now, and the first thing I'm going
to do is edit the info plist for the AV Touch application.
And so, I'm going to add a row down here, but now I'm
going to zoom in so that you can actually read that.
And, I'm going to select the required
background modes key in the info plist.
This is actually an array, so I have to drop down the
disclosure triangle there, and the first item that I'm going
to add, or the only item I'm going to add for this
background audio application, is the App Plays audio value.
And now, let me zoom up.
Now, with this key, AV Touch can actually play audio in the
background, but if I build and deploy this to the device,
I'll find that when I tap Home, the audio still stops.
And if I take a look at the crash logs in
Xcode, or in the console in the Xcode organizer,
it'll tell me that my application was using the
GPU in the background, for that GL level meter.
So, what I need to do is stop the level
meter when AV Touch enters the background,
and then resume the level meter when
AV Touch returns to the foreground.
And, I'm going to do that in the Awake
from NIB of this AV Touch controller class,
which is called when the AV Touch
controller is loaded from the NIB.
And then, the first thing I'm going to do is register
for the UI application, didEnterBackgroundNotification,
which was sent whenever AV Touch goes into the background,
and I'm going to do that by grabbing the default
notification center and adding in observer
on the default notification center that
calls this pause level meter helper function.
So, this will actually pause the level meter when it goes
into the background, and so it can continue playing audio
without terminating due to background GPU access.
But, I also need to resume the level meter when it comes
back to the foreground, so to do that I'm going to register
for the UI application, WillEnterForegroundNotification,
which as its name suggests,
will be sent when the application
comes back to the foreground.
And again, I do that by adding an observer to
the default notification center and this time,
I'm going to call the resume level meter helper function.
Now, I'm not going to show you
what these two helper functions do,
because it actually touches quite a
few files and we don't have the time.
But, it essentially stops the update timer
from my GL drawing, and if you'd like to see
that in more detail, please come to the labs.
Now I've paused the GPU access while in the background.
The next thing I want to do is respond to
interruptions appropriately, as Dave mentioned earlier.
And, I'm going to do that down here where we see
that the AV Touch controller is already implementing
the beginInterruption and endInterruption methods.
The beginInterruption method is okay,
but the endInterruption method really
should be endInterruptionWithFlags.
So, I'm going to replace this method
with endInterruptionWithFlags,
and then in endInterruptionWithFlags, I check whether
the ShouldResume flag is set on the flags parameter sent
to this delegate method, and if it's
set, then I call Start Playback.
Now, the last thing I'd like to do as a background audio
application is respond to the remote control events.
And, as we said earlier, the remote control events
are sent through the normal UI responder chain,
so I'm going to edit the AV Touch view
controller, which is the view controller
for that main view that you saw in AV Touch.
And, the first thing I'm going to do
is override canBecomeFirstResponder,
so that we can receive those remote control
events in this class, and I'm going to return Yes.
And then, I need to tell the system
that my application is interested
in these remote control events, and
handle the remote control events.
So, I'm going to do that in the
viewDidAppear for this view controller,
and I'm going to call beginReceivingRemoteControlEvents,
which tells the system to route remote control events
to my application if I was the
last application to play audio.
And then, I'm going to call becomeFirstResponder, which
makes the AV Touch view controller the first responder
in the responder chain, so when those events
come to my application or come to AV Touch,
the AV Touch view controller is the
first class to handle these events.
Now, of course, I need to actually process the events,
and I'm going to do that by overriding the
remoteControlReceivedWithEvent method from UI responder,
and then switching on the event sub-type.
And in this case, the only event that I'm
handling is the Toggle Play/Pause event,
but I could also handle the previous
Track and Fast Forward Track if needed.
So, I'm going to go ahead and build and run that.
So, now when I press Play, [Music] you'll notice that again,
it has the track name and the play time, the level meter,
but now in the upper right corner by the battery
meter, you see a little Play icon indicating
that there's an application playing audio in the system.
If I now tap Home, the audio continues playing.
And now, if I double-tap to bring up the
multitasking UI, and slide over to the remote controls
in the multitasking UI, I can actually pause it from here.
And in addition, I can lock the device,
bring up the controls on the lock screen,
resume playing from the lock screen.
[Music] And also, one last thing.
If you look at the remote controls in the
multitasking UI again, you notice that the icon next
to the remote controls actually
changed to the icon for AV Touch.
If I now tap the icon there next to the remote
controls, it returns me back to AV Touch.
The track was looping there, it didn't stop playing; sorry.
>>David Myszewski: So, that's Background Audio.
So now, we have a few more categories that
we want to cover, starting with Navigation.
So, Navigation works a lot like
Background Audio, as you'll see.
So, we're going to create a very simple example application
here, which is kind of a turn-by-turn directions app.
And, that app would want to keep the users informed
of their current location at a very granular level,
so that we don't miss any turns, and
speak out turn-by-turn directions.
So, how do we do this?
Well, this type of application would want
to register for both background location
and background audio in the info key list.
So, there are entries for both of these.
As Charles mentioned, this is an array
and you would simply add both location
and audio to your UI background modes array.
And, the application life cycle
works just like Background Audio.
When the user has the application in the
foreground, they're in the active state.
And then, when the user presses the
Home button, the application transitions
through the foreground inactive state
to the background running state,
and as long as the application was tracking your location,
the application remains running in the background.
So, to put this on the timeline view,
if the user is updating your location,
and then the user presses the Home
button, the application will continue
to run until, say you reached your destination.
Now, when you reach your destination, the application
might stop location services, as we recommend.
And then, at that point, because you're not updating the
location constantly, then the application will be suspended.
So, the application transitions from
background running to background suspended.
So, how does all of this work?
Well, we have the CL Location Manager,
just like we did in iPhone OS 3.
And, you may have a delegate object here that's going to
call this particular method on the CL Location Manager
to say that we really want the
best accuracy here for navigation.
This will be the finest grain controls
that we have for location updates.
Then, the CL Location Manager will provide updates
after you make this call to start updating the location.
So, this tells us; great, let's turn on the GPS.
Give me quick location updates.
And, the CL Location Manager will send you these updates
to say that you've moved from one particular location
or to one particular location from another, and so, you'll
get many of these as your location changes over time.
Now, for Background Audio support, so
if we want to speak these directions,
we want to do three things in the background.
The first thing is we need to, as before, set
the appropriate audio session category, so again,
we're going to use the AV audio session category playback.
But, this time we're going to set a couple of properties
on the audio session so that we get the intended behavior.
So, in particular, we're going to set the AV audio
session property OverrideCategoryMixWithOthers.
That will say that we should mix with other sounds, so
you'll want this in an application that speaks location,
because you may have other sounds that are going on at the
system at the same time, and so you want to mix with others.
Then, there's also the property to mix with
others, or that other mixable audio should duck.
The economical example her is sort of, you're listening
to your iPod app because you're moving down the road.
You know, it's heavy metal; you
probably don't hear directions over that.
So, what you really want is for that other mixable
audio, i.e., the iPod app, to duck a little bit,
so that you can hear your directions,
because that's what you want to hear.
You don't want to miss your turn.
So, I think this property is important for a navigation app
and it'll allow the user to hear want you want them to hear
for that brief period of time, as the iPod
app quiets down so that the user can hear it.
For Navigation, we have the similar best practices.
You should minimize CPU usage.
Hopefully, it shouldn't need a whole lot of CPU
as you're updating from location to location.
And, as we said before, for power reasons it's
great if you can turn off location updates
after you reach the destination, because
GPS is a fairly high power operation.
So, once you're there, then turn off location updates
and then if the user wants to navigate to another place,
they can return to your app and
choose another place to go to.
And, that's really all there is to Navigation.
But, there are some other important location
categories, and Charles will describe all of those.
[ Applause ]
>> Charles Srisuwananukorn: So now, I'd like to talk
about a couple of new location tracking services in iOS 4
that allow your application to
track the device as it moves around.
To motivate these services though, imagine we're
implementing a location aware, Capture the Flag app.
And, the first thing you do in this application
is set two coordinates as the two flags,
and then set a couple of regions around each flag.
And, to capture the opponent's flag, you
enter the region around your opponent's flag
and then return to the region around your own flag.
So, that's how that works, basically.
But in addition, it can also display a map of all
of the players that are currently playing the game,
and approximately where they are in the city.
To implement this, we'll use two new services in Core
Location; one called Significant Location Changes,
and the other one called Region Monitoring.
Significant Location Changes will notify your
application when the device has moved cell towers,
and we use cell towers as a proxy for when
the device moves a significant distance.
The Region Monitoring API, on the other hand, will
notify your application when the device enters
or exits certain geographic regions of interest that
you've previously registered with Core Location.
Both services use less power than the standard location
services, because they are based on cell positioning.
Both services will resume applications
that have been suspended.
Both will also relaunch applications that have
been terminated for, say low memory conditions.
But, there are also a couple of
caveats that you should be aware of.
If the device is sleeping, significant location change
notifications may actually be coalesced or delayed,
in order to preserve battery life, but
region monitoring notifications won't be.
And also, since both services are based on cell positioning,
they're only available on devices with cell radios.
But, in particular, region monitoring
is only available on iPhone 4,
whereas significant location changes is
available on both iPhone 4 and iPhone 3GS.
So, let's talk some more about the
Significant Location Changes API.
So, as I said earlier, the Significant Location
Changes API is meant to send you a notification
when a device has moved a significant distance.
And, it does this by calculating an approximate position
for the device whenever the device changes cell towers,
and then sends your application that approximate position.
The accuracy of that position, of course,
will be similar to the cell positioning.
And, also again, you should be aware
that these notifications may be coalesced while the
device sleeps to prevent that excessive battery drain.
So, our Capture the Flag application can then
use the significant location changes service
to implement the player map that we saw earlier.
And so, whenever a player enters a game, the first
thing our Capture the Flag application does is start
up the significant location changes service.
Then, when the player moves cell towers, this Core Location
will notify your application with that approximate,
current location, and our application can
then upload that location to a server.
And then, if they move cell towers again, the same
thing happens, and every time they move the cell towers,
we can upload a new location to the server.
Now to set this up, our application first creates a
CL location manager and a location manager delegate
and associates the two together, and then
calls startMonitoringSignificantLocationChanges
on the location manager.
Once we've done this, whenever the
device changes cell towers again,
the location manager will call back the delegates;
locationManager:didUpdateToLocation:fromLocation:
delegate method.
Now, this is the same method that's called
when you used the standard location services,
as we saw earlier in the Navigation example.
But, the position that it passes you is the
approximate position based on cell positioning.
When you receive this notification, you
might actually be in the background,
so you don't have much time to
run and process the notification.
But, you can also use the Task
Completion API to continue running
in the background while you upload
that location to the server.
Now, let's talk more about region monitoring.
Region Monitoring will send a notification to your app,
again, when it enters or exits a region of interest.
And, the big benefit of region monitoring is that your
application can be suspended while it's moving around,
and it's not crossing any of these
boundaries between the regions of interest.
Again, this service is also based on cell positioning, so
it also uses less power than the standard location services,
but your application is also limited
by the number of regions
that you can register, and it is only supported on iPhone 4.
So, now assume that this red pin over here, it
represents one of the flags in our Capture the Flag game,
and we've registered a region of interest around
the flag, represented by those red circles.
Now, and then assume that the Capture the Flag application
suspended, because there was nothing else to do.
Now when the device moves, core
location won't notify your application,
because it hasn't actually crossed into this region yet.
It's not until the device actually enters the region that it
then sends a notification saying we've entered the region,
and your application has a chance
to process this notification.
So, to use region monitoring, again we create a CL location
manager and a delegate and associate the two together,
and then call startMonitoringForRegion, passing
it a description of a geographic region.
Now, whenever the device enters or exits one of
these regions, the location manager will call back;
the locationManager:didEnter or
didExitRegion:, depending on what happened,
and then your application can process those appropriately.
In the case of the Capture the Flag
application, we may want to notify the user,
saying; hey, you've entered your own base.
Congratulations, you've won!
But, your application may be in the background again, and so
it won't be able to actually display its own UI to the user,
and instead, it can also use the local
notifications service to display an alert to the user,
very similar to a push notification
alert, that the user can then use
to bring the Capture the Flag app back to the foreground.
Now, let's review what states the application
goes through as each of these events are received.
Now, assume that the application is suspended again;
the Capture the Flag application is suspended again.
And then, we either enter one of these regions
or we get a Significant Location Changes event.
So, there we are in the suspended state, and the system
will unsuspend our application to handle the event,
which transitions us from the suspended
state into the background running state.
And then, once the event is handled, we can then transition
back from background running into the suspended state
and we can be suspended until another
interesting location event happens.
But, as I said earlier, your application
may also be relaunched.
So, assume that the Capture the Flag app
had been terminated, again due to low memory
or for some other reason, so that
it's in the not running state.
Then the system will relaunch your application when
one of these events happens, which transitions your app
from the not running state into
the background running state.
And then, it'll deliver the event to
your application so you can handle it.
And then, once you're done handling the event,
the application will then become suspended again,
so we transition from background running to suspended,
and then it's suspended until,
again, one of these events happens.
But, your application may not want to start
location services every time your app is launched,
and so to help you detect when you've been launched
in the background for one of these location updates,
the iPhone iOS 4 will pass the, or will set
the UIApplicationLaunchOptionsLocationKey,
in the options dictionary, pass to your
application, didFinishLaunchingWithOptions.
So, you can then check this key and if
it's set, create a CL location manager
and start monitoring those significant location
changes again, or start up the location services.
So, a few best practices you should be aware of would
be services; you should use significant location changes
and region monitoring only if possible,
and only use the standard location services
if you need the extra precision or the extra accuracy.
You could even do a somewhat hybrid approach
and fire up the standard location services
when you receive a significant location changed
event, so that you'll only use the high power GPS
when the device has actually moved a significant distance.
And, even though these services use less power
than the standard location services,
they still use some amount of power.
So, it's best to stop using those
services as soon as they're not needed,
and you can do that by calling Stop Significant
Location Changes on the location manager,
or Stop Monitoring Region when the
region's no longer interesting.
Now, the last multitasking service or set
of multitasking services I'd like to talk
about today are those geared toward
VoIP, or Voice over IP applications.
A VoIP application is an application that makes and
receives phone calls using an Internet connection.
It notifies users of incoming calls, whether
it's in the foreground or in the background.
It can receive calls in the background.
And, it can stay on a call as it enters the background.
And, the first thing you do as a VoIP application
is set the appropriate background modes in Xcode,
and for a VoIP application, you want the App provides Voice
over IP services, which enables you to call the VoIP API's,
and the App Plays Audio background mode, which allows you
to play the call's audio while it's in the background.
And, almost every VoIP application
will probably need both of these keys.
[ Silence ]
So, there are a few things that every
VoIP application will want to do.
First is respond to incoming calls quickly.
If your VoIP application receives an incoming call,
the incoming call notification should be delivered
to your application as quickly as possible.
In the same vein, it also should maintain a
persistent signaling connection to the VoIP server,
so that it doesn't have to reconnect and re-authenticate
that signaling connection when
it receives the incoming call.
It should notify the user of the incoming call,
whether it's in the foreground or in the background.
And, if the user accepts the incoming call, it should
implement the appropriate audio behaviors for a VoIP call.
And finally, it should integrate well with cellular calls,
which usually means simply putting the VoIP
call on hold while you're on a cellular call.
So, let's look now at how the system helps
you first respond to those incoming calls,
or receive those incoming call notifications quickly.
So, if the device receives an incoming call
notification intended for your VoIP app,
it's much better if your VoIP app is already running, so we
don't have to relaunch the application and incur that delay
of relaunching the application
before delivering the notification.
And so, what the system does is on device
boot, it will actually launch your application,
and it will also relaunch your application if it's been
terminated for, again, low memory or for some other reason.
But, after it's done launching,
it can actually be suspended again
until something interesting happens
for your VoIP application.
For example, if your application receives an incoming
call, then the system will unsuspend your application,
and as long as the user or the
application is on that incoming call,
your application runs until the call completes.
Once the call completes though, it can then suspend
again until some other event happens; for example,
a network keep-alive, and we'll talk more
about what network keep-alives are in a second.
But, they are a way for your application to
maintain this persistent network connection.
And so, then when one of these events
happens, it unsuspends your application again
to send the keep-alive, and then re-suspends it.
Now, let's take a look at how to maintain
that persistent signaling connection so that
when the incoming call notification
comes in to your application,
you don't have to reconnect the connection
and re-authenticate the connection.
So, the first thing you do is you create a CF stream, and to
create the CF stream, you can use any of the CF stream API's
that we have, and in this case, I'm going to use
the CF stream, Create Pair With Socket To Host,
which will return a read stream
and a write stream to this host.
Then, I call this prepare stream method, passing
it both the read stream and the write stream.
The prepare stream method then does all of the things you
would normally do with a stream, like setting the delegate,
scheduling in the run loop, and
calling open on those streams.
But, most importantly for our VoIP
app, it sets the NS stream,
Network Service Type VoIP Property,
for the NS stream network service type.
And, this tells the system, or iOS 4, that it should wake up
or unsuspend your application for
any incoming data on this CF stream.
So, in that way, you can actually be unsuspended
whenever there's incoming data on the stream.
A couple of other notes about this
CF stream, or VoIP CF stream.
As I said earlier, you can use any of the
CF stream API's that create CF streams.
In addition, you can use the ones that wrap POSIX or
VSD sockets, which is a common question on the forums.
And, you should be aware that the CF
stream API only supports TCP streams,
but this is OK because you only need the
VoIP property on your signaling channel.
If the call's audio channel uses UDP, for example,
just by playing audio your application remains
unsuspended and you can continue to play the audio.
But, even though your application is now unsuspended
whenever there's incoming data on this channel,
it can still be subject to certain timeouts,
like NAT timeouts anywhere along the route,
or it might even be using some protocol that requires
the client to check in with the VoIP server periodically,
and if it doesn't, then the server
will close down the connection.
So, to help your application maintain these connections,
you can register a keep-alive handler with the system
by calling setKeepAliveTimeout:handler:
on the UI application class.
And, so that KeepAliveTimeout:handler takes
as its first parameter a time interval,
and a block as its second parameter.
It will then unsuspend your application on the interval that
you specify, and execute that block that you've passed in.
And so, using this you can actually
send periodic keep-alive packets
to prevent these NAT timeouts or protocol level timeouts.
And, you should also note that the minimum
keep-alive timeout for this mechanism is ten minutes.
So, now we've set up this persistent signaling
connection and we can receive incoming call notifications,
and we don't have to reconnect to the VoIP
server when we get an incoming call notification.
But, how do we tell the user that they have an incoming
call, because your VoIP application may again be
in the background, and so it can't
present any of its own UI?
Well, again we have the local notification
service, which your VoIP app can use
to present a push notification-like
alert in front of the user,
and you do this by calling presentLocalNotificationNow,
which will actually display the alert that you see there.
Note that you can also dismiss local notifications if the
user receives multiple calls before looking at the device
to prevent these alerts from stacking up.
So, we've alerted the user that they have an incoming call.
Imagine now that the user clicks
Accept, and we have to connect the call
and start playing the call's audio
and take input for the call.
To do so, we have to implement the appropriate audio
behaviors, and so the first thing we have to do
to implement the appropriate audio behaviors is set
the right audio category on our AV audio session.
And for VoIP apps, the right audio category is the
AV audio session category, Play and Record category.
This category is similar to the Playback category that we
saw earlier, but it allows simultaneous access to both input
and output, so you can take an input for the call at
the same time as playing the output from the other side.
Like Playback, it silences other
audio, so if you were playing iPod
when you started the call, it will actually silence iPod.
And, it also enables output to both the
receiver or the earpiece, and the speaker,
so you can implement a speakerphone
in your VoIP app, if you like.
But, we would also like other audio to resume after a VoIP
call ends, because this is the way that phone calls behave.
So, if the user was playing the iPod before receiving
the VoIP call, and then we connect the VoIP call,
when we end the VoIP call, iPod should resume playing again.
And, to do that,
you set the
AVAaudioSessionSetActiveFlags_NotifyOthersOnDeactivation
flag when you deactivate the session after the call ends.
And, what this does is it just tells other
applications that were playing audio,
that had their audio interrupted,
that they need to resume their audio.
And, more specifically what it does is it sets that
AVAudioSessionInterruptionFlags_ShouldResume flag
when it calls those other applications
and interruption with flags methods.
So finally, VoIP applications should
integrate with cellular calls, and again,
usually that means just putting the VoIP
call on hold while on a cellular call.
And so, in iOS 4, we provide the CoreTelephony framework,
and in the CoreTelephony framework we have an API
called setCallEventHandler: on the CTCallCenter object.
And, setCallEventHandler: takes a block that is executed
whenever the user receives an incoming cellular call,
or ends the current cellular call.
And so, your application can then use this to put the VoIP
app's call on hold whenever the user is on a cellular call.
So, a couple of best practices for VoIP apps.
VoIP apps are also audio applications.
So, everything that we've said about audio
applications also applies to VoIP apps.
In particular, we try to ensure that VoIP apps have all of
the CPU and all of the network and file I/O that they need
to play the call smoothly, which means you should try
to minimize your resource usage as much as possible
by minimizing your CPU, and to
avoid using large amounts of memory.
And, more specifically to VoIP
applications, you should try to use as long
of a keep-alive interval as you
can to maximize battery life.
And, we've found empirically that
29 minutes is a pretty good tradeoff
between timing out our sockets and battery life.
Multitasking, again, on iOS 4 doesn't mean that
applications run all of the time, but, you know,
there are some applications that do
benefit from executing in the background,
and for those applications we provide some
new services to run in the background.
We have the task completion API, which allows your
application to request extra time to finish up a task
on behalf of the user while in the background.
The background audio API, to play audible content to the
user and respond to remote control events in the background.
The navigation service, to implement
turn-by-turn navigation applications.
A couple of location tracking services;
significant location changes and region monitoring,
to monitor the device's location in
the background in an efficient manner.
And finally, the VoIP services,
which allow your application to make
and receive phone calls using an Internet connection.
And again, if you missed part one earlier today,
we'll be repeating part one on Friday at 9:00AM.
And, we've covered a lot of ground today, and so we've only
scratched the surface what each of these services can do.
If you'd like to learn more about each of those
services, please come to the related sessions here.
In particular, we didn't cover push notifications at all,
so if you'd like to learn more about push notifications,
please come to the Implementing
Local and Push Notifications session.