Transcript
>> Hello and welcome to the
Enabling Your App for CarPlay
session.
My name is Albert and I'm an
engineer on the CarPlay team.
Today we'll be going through an
overview of how apps behave in
CarPlay, how to get your app
integrated on the CarPlay
platform and best practices and
pitfalls to avoid.
Let's look at how apps run in
CarPlay.
When CarPlay is active some
information will be available
and accessible from the CarPlay
screen.
If an app has posted now playing
metadata and playback controls
for audio content this
information will also be
accessible from the Now Playing
app on the CarPlay Home screen.
Audio will be played through a
car speaker system and some
SiriKit intents will be
supported in CarPlay.
If we user triggers of SiriKit
intent that is not supported
while in CarPlay Siri will
notify the user that it is not
available in the car.
Even though your app may not
support CarPlay users may use
your app while in CarPlay before
they begin driving.
To ensure a great user
experience for all apps in
CarPlay make sure that your app
doesn't play audio unless
requested since your app could
override currently playing audio
the user intended to play.
In some instances, your app will
want to interact with other
audio streams appropriately.
For spoken audio apps such as
Audiobooks use this protocol so
your audio will pause when other
audio streams are active.
For navigation apps where
guidance instructions are given
intermittently use these two
protocols, the latter of which
will pause audio streams marked
as spoken audio before
initiating your audio session.
Because guidance voices will mix
with other audio streams set
your audio session to mixable so
that it can interact with other
audio streams.
For apps that have deeper
CarPlay integration the app icon
will appear on the CarPlay Home
screen.
Apps that are launched in
CarPlay will launch in the
foreground on both the CarPlay
and iPhone screen.
For apps that are not integrated
with CarPlay only the iPhone
screen will have the app in the
foreground and CarPlay will only
display the Home screen.
There are certain categories of
apps that can support deeper
CarPlay integration.
These are audio apps, messaging
and VoIP apps and automaker
apps.
Audio apps deliver music, news,
podcasts and other audio content
using a consistent design that's
optimized for use in the car.
Messaging and VoIP apps work
using Siri and can be updated to
appear in CarPlay.
Automaker apps provide vehicle
specific controls and displays
to keep drivers connected
without leaving CarPlay.
If you think your audio,
automaker, messaging or VoIP
calling app has the potential to
be supported by CarPlay tell us
about it at the CarPlay
developer portal.
There you'll be able to request
an entitlement to get your app
running in CarPlay.
Keep in mind that certain assets
and images will need to have
CarPlay specific assets and to
provide 2X and 3X sized images
since the CarPlay screen has
varying resolutions and sizes
depending on a vehicle's
display.
For more guidance on icons and
designing your app for CarPlay
check out the iOS Human
Interface Guidelines and the
CarPlay Human Interface
Guidelines.
While you can test the CarPlay
app on iOS simulator in Xcode it
is best to test on an actual
device connected to a real
vehicle head unit for realistic
results.
For apps that will play audio
the simulator has some
limitations on playback state
which does not reflect on what a
user will actually experience.
To fully debug your app with
LLDB support Xcode now supports
wireless debugging so that an
iPhone can be connected to a
vehicle or aftermarket head unit
while debugging your app.
For more information, please see
the debugging with Xcode 9
session which will be available
this week in WWDC.
If your app uses data protection
for user data or content keep in
mind that apps in CarPlay will
likely run while the iPhone is
pass code locked.
If your app has pass code
protected data using the
following types for files,
keychains or SQL like databases
shown here your data may be
unavailable while in CarPlay and
your app may behave
unexpectedly.
Let's take a closer look at
getting your app integrated with
CarPlay.
We will highlight the three
categories of apps that support
CarPlay integration, audio apps,
messaging and VoIP apps, and
automaker apps.
Let's get started with audio
apps.
Audio apps in CarPlay present
content in an interface that's
optimized for the platform
providing the user an easily
navigable way to access their
content.
Audio app developers provide a
data source, as well as a
navigation hierarchy for the
CarPlay platform to render
onscreen.
In addition, now playing
information will be presented
providing the user with relevant
metadata, as well as adjustable
commands for the user to handle
playback speeds, tracks and
other actions audio apps may
support.
After receiving the app
entitlement audio apps in
CarPlay must at a minimum
implement the MP playable
content APIs, which include the
data source and delegate in
order for CarPlay to fetch
content items and initialize
playback.
Audio apps must also respond to
MPRemoteCommandCenter events
which let the user perform
commands on your content, such
as play, pause or skip tracks.
Finally, audio apps must set and
update the
MPNowPlayingInfoCenter
dictionary which contain
metadata on the item that is
currently playing, such as item
title, artist name or duration
time.
Let's take a closer look at how
data is structured for audio
apps.
CarPlay will ask for content
items at a specific index path
using an NSIndexPath.
And this is different than how
NSIndexPath objects are used in
UITableView.
Let's go through an example
shown here where the left most
content items are the root items
and can be represented in the
user interface as individual
tabs or a root table view.
Here's the index paths for each
content item represented in the
hierarchy.
When CarPlay asks for a content
item for a particular index path
we, the audio app, would
traverse the hierarchy for the
requested content item.
In this example, we are asking
for the first child of the first
content which is the running
playlist content item.
CarPlay will also ask for the
number of child items of a
particular index.
In this example, we are asking
for the number of child items
for the third child item of the
second route content item.
In this example, we will return
two.
Some vehicles may enforce
limited content to show onscreen
depending on whether the vehicle
is in motion.
This can limit the number of
rows shown onscreen, as well as
container depth when drilling
into content.
Your app can take these changes
into account using MP Playable
Content Manager.
To check if content limits are
enforced implement the Delegate
Callback Content Manager Did
Update and check if the content
limits are enforced.
From there your app can check
the context properties to
determine how many items are
permitted in the table view and
the maximum depth in the
hierarchy your app is permitted
to show.
If your app benefits from
displaying data using tabs
rather than exclusively table
views you can add tabs by adding
UI browsable content support
section browsing in your app's
info.plist file.
We recommend having at most four
tabs with short titles due to
space constraints and for
vehicles that have a narrow
screen size, as well as
accounting for the Now Playing
button shown while content is
playing.
Tab image assets will be
rendered as template images on
the CarPlay screen.
Now let's talk about the Now
Playing screen in CarPlay.
Some controls and metadata here
are identical to what is shown
in Control Center.
In general, what controls and
data you see in Control Center
should also be presented here in
the CarPlay Now Playing screen.
This Now Playing screen can be
accessed either by the user
tapping Now Playing on the top
right of your app's navigation
interface or through the Now
Playing app on the CarPlay Home
screen.
Your app's name will appear on
the top right of the Now Playing
screen when it is active.
Setting metadata for CarPlay's
Now Playing screen is the same
as Control Center and other
sources.
Set a dictionary of information
to MPNowPlayingInfoCenter and
fill out as much of the entries
as appropriate.
In addition, respond to playback
commands so that the user can
choose to adjust content such as
play, pause, skip tracks or
shuffle and repeat of playback
queue.
These commands here are the ones
that are supported on the
CarPlay Now Playing screen.
New in iOS 11 is the ability to
adjust and display the playback
rate in the Now Playing screen.
To display playback rate in your
app's Now Playing screen in
CarPlay add MPNowPlayingInfo
PropertyDefaultPlaybackRate to
MP Now Playing Info Center and
respond to the change playback
rate command with an array of
supported rates in
MPRemoteCommandCenter.
Here is an example of how you
would implement playback rate
adjustments and code.
In this sample, the default
playback rate of the currently
playing item is 1.0 and the
supported playback rates here
are 0.5, 1.0, 1.5, and 2.0.
When the user wants to adjust
the playback speed increment the
playback rate forward cycling
back to the beginning if the
audio is playing at the fastest
supported rate.
In this code sample, playback
rate will increment to 1.5 then
2.0 and then back to 0.5.
Some audio apps may respond to
multiple commands in MP Remote
Command Center.
Depending on what commands are
enabled CarPlay's Now Playing
screen will combine certain
relevant commands into a single
button such as an ellipsis or a
menu button replacing the
previous track button.
For best practices, make sure
that content is actually ready
to be played or be displayed
before calling the completion
handlers in MP playable content
data source and MP playable
content delegate.
For apps that do not opt into
tabs and only use table views
the root table view must return
at least one item.
CarPlay would show a loading
activity indicator and will time
out after period of time if an
app has not responded with
content.
If your app has some initial
setup such as login credentials
populate the first row with an
item informing the user of the
app's current state onscreen.
That's it for enabling audio
apps and I'll hand it off to
Chris Whitney to discuss
messaging and VoIP apps in
CarPlay.
>> Hi, I'm Chris Whitney from
the CarPlay engineering team.
Albert talked about audio apps
now I will discuss a few more
categories of apps that can
integrate with CarPlay.
The first category I will cover
is messaging and voice over
Internet protocol calling apps.
In CarPlay Siri provides the
interface for messaging and VoIP
calling apps.
After the app icon is selected
on the Home screen, Siri offers
interactions for reading
messages, starting a new message
or starting an audio call.
Apps using SiriKit can already
handle these actions, but the
Siri request must contain the
app's name.
With a few additional changes
messaging and VoIP apps can have
a CarPlay experience that
includes a Home screen icon,
notifications, and a dedicated
Siri interface.
Here are the requirements for an
app to support messaging or VoIP
calling in CarPlay.
Messaging apps must implement
SirKit messaging intents and
sign their app with an
entitlement that is specific to
CarPlay messaging.
Similarly, VoIP calling apps
must implement the SiriKit
calling intents and sign their
app with an entitlement that is
specific to CarPlay VoIP
calling.
VoIP calling apps must also
implement CallKit to inform
CarPlay about the status of
calls.
A single app can support both
messaging and VoIP calling in
CarPlay.
Let's look into the SiriKit
requirements in more detail.
For messaging apps, Siri sends
messages using
INSendMessageIntent.
Siri reads new incoming messages
and searches for unread messages
using INSearchforMessagesIntent.
And once done reading Siri marks
messages as read by calling
INSendMessageAttributeIntent.
All three intents are required
to support a complete messaging
workflow in CarPlay.
For VoIP calling apps, Siri
starts audio calls using
INStartAudiCallIntent and
performs searches, such as for
missed calls using
INSearchCallHistoryIntent.
Both intents are required to
support a complete calling
workflow in CarPlay.
For details on SirKit see the
session's What's New in SiriKit
and Making Great SiriKit
Experiences.
Any app that has already
implemented CallKit gets a
CarPlay interface for incoming
and active calls.
But if your VoIP calling app
hasn't implemented CallKit yet
that will be a requirement for
CarPlay integration.
As a general overview, the app
must report incoming calls to
CallKit and handle the start,
answer and end call actions.
Additional actions, such as
mute, grouping, holding and
keypad tones should be handled
if supported by the app's VoIP
platform.
For more details on CallKit
watch the enhancing VoIP apps
with CallKit video from WWDC
2016.
Apps can show notifications in
CarPlay for incoming messages
and missed calls.
There's some additional setup
required to show notifications
in CarPlay.
The CarPlay option must be
included when requesting
authorization for notifications
and the messages must be
separated into a notification
category created with the Allow
in CarPlay option.
This notifications category must
be used exclusively for messages
and it should not include
notifications for other app
features.
The notification category must
specify a SiriKit intent to
handle selection of the
notification in CarPlay.
Only messaging or VoIP calling
apps that meet the CarPlay
requirements will be allowed to
request authorization for
showing notifications.
Also, users can use the Settings
op to disable showing your app's
notifications in CarPlay.
Let's look at the code for
setting up notifications in
CarPlay.
First, include the CarPlay
option when requesting
authorization for notifications.
Gracefully disable notifications
related features if the user
declines notification
authorization.
Next, create a new message
category that contains only
notifications for messages.
Notifications for other app
features, such as news or games
should not be included in this
notification category.
Assign an identifier to this
category and make sure that any
local or remote notifications
for messages have the same
category identifier.
Set up an intent identifier to
configure the Siri interaction
when the notification is
selected in CarPlay.
Usually that will be
INSearchForMessages
IntentIdentifier which
configures Siri to search for
and then read the message that
matches this notification.
There are a few best practices
to follow when building
messaging and VoIP calling apps.
Notifications should only
include information like the
sender and group name in the
title and subtitle.
The contents of a message should
not appear in CarPlay.
After reading a message, Siri
will use
INSentMessageAttributeIntent to
mark content as read.
The read status should be
reflected in the app and
subsequent search for messages
requests.
Since users may drive in and out
of cellular coverage areas apps
should use notifications to
communicate any missed calls or
message delivery failures.
This may require setting up
additional notification
categories as we have described
earlier.
The final category of apps that
can integrate into CarPlay are
automaker apps.
Automaker apps are the only
category of apps that can
display a custom user interface
in CarPlay.
In this example, the automaker
app provides controls for
adjusting the climate settings,
selecting radio stations, and
summoning roadside assistance.
Providing these controls in an
automaker app allows the user to
adjust the vehicle's climate and
radio without leaving CarPlay.
Automaker apps can take
advantage of the iPhone's
Internet connectivity and
powerful development platform to
create additional vehicle
features.
For example, this automaker app
could enable Siri driven voice
control of the climate and radio
settings.
Automaker apps must be created
by the car's manufacturer to
provide information and control
features on a connected vehicle.
These apps must be signed with a
specific CarPlay protocols
entitlement.
They must display a user
interface in CarPlay.
And they must only appear on
supported vehicles.
CarPlay determines if an
automaker app is supported on a
given vehicle by matching the
external accessory protocols
declared by the vehicle with the
CarPlay protocols entitlement
sign in to the app.
Thus, it is important to
consider potential automaker
apps during vehicle development
in order to have the appropriate
set of protocols declared by the
vehicle.
Here's an example of how CarPlay
matches automaker apps to
vehicles.
This vehicle declares two
protocols, a performance
protocol for collecting vehicle
statistics and a protocol for
climate control.
As shown here, protocol names
should be declared in reverse
DNS format.
The automaker wants to take
advantage of the vehicle's
performance statistics in a new
app called Track Laps that's
designed for use on racetracks.
The app includes the performance
protocol and its entitlements
and since both the app and the
vehicle support the performance
protocol the Track Laps app will
appear in CarPlay.
Automaker apps can also support
multiple protocols.
This automaker decides to follow
up on the success of Track Laps
with a new app called Comfort
Cruise that conveniently
configures the radio and climate
control with the driver's
favorite settings for commuting.
The app's entitlements include
both the climate protocol and a
new radio protocol.
This vehicle doesn't support the
radio protocol, but since both
the app and the vehicle support
the climate protocol the Comfort
Cruise app will appear in
CarPlay.
After launching automaker apps
should query the supported
external accessory protocols and
only display a user interface
for the supported features.
In this example, the Comfort
Cruise app should only show
climate controls since the radio
protocol is not supported on
this vehicle.
The automaker wants to create
another app called Charge Up
that shows their electric
vehicle charging stations.
To limit this app to only their
electric vehicles the automaker
declares a new electric protocol
in the app's entitlements.
Since this vehicle does not
support the electric protocol
the Charge Up app will not
appear in CarPlay on this
vehicle.
Building an automaker app
requires communicating with the
vehicle.
Automaker apps must communicate
directly from the iPhone to the
vehicle using the External
Accessory framework.
Directly communicating using
External Accessory improves the
responsiveness of the app and
avoids relying on connectivity
to Internet services.
To use the External Accessory
framework the app should define
a custom protocol name as shown
previously, implement the named
protocol in the vehicle
software, in the app observe
accessory connection events, and
upon connection initialize an EA
session to create networking
streams of the vehicle.
For more information see
External Accessory programming
topics.
Automaker apps can use UIKit to
display a user interface on the
CarPlay screen.
To create an interface in
CarPlay the app should observe
the UIScreen connection events
and when the screen connects
verify that the connected
screen's traits contain the
CarPlay idiom.
Besides CarPlay, iOS also
supports other external
displays, such as an airplane
display.
Only show CarPlay content on
screens with the CarPlay idiom.
The app creates a UIWindow for
the CarPlay screen and sets the
root view controller to a custom
UI view controller subclass.
Let's see how this looks when
all put together.
Here's the code for displaying a
view controller on the CarPlay
screen.
Usually this code would be
placed either in your UI
application delegate or in a new
class you've made for this code.
First, we'll add a new class
property to store the UIWindow
that we will create for the
CarPlay screen.
Next, we'll define an instance
method called updateCarWindow.
To check if a CarPlay screen is
connected we look for a UIScreen
that has the CarPlay idiom.
If no CarPlay idiom screens are
available then release any
previously created UIWindow
since CarPlay may have
disconnected.
If a CarPlay screen is connected
initialize a new UIWindow for
that screen and set up a custom
rootViewController that contains
the content that should appear
in CarPlay.
Here we are using a UI view
controller subclass called
carViewController.
That class will need to be
implemented.
Your app may launch after
CarPlay connected.
To handle that scenario call
updateCarWindow early in your
application's lifecycle, such as
an application did finish
launching with options.
Since CarPlay can also connect
after your app is launched
listen for the
UIScreenDidConnect and
UIScreenDidDisconnect
notifications and call
UpdateCarWindow in response to
those events.
There are a few differences in
UIKit behavior on the CarPlay
screen compared to the iPhone's
main screen.
UIButtonTypeSystem displays with
the default CarPlay style and is
the recommended way to create
buttons since it provides a
style and behavior consistent
with other CarPlay apps.
In CarPlay,
UITableViewController may limit
the length of tables and for
vehicles with a hardware
navigation device, such as the
[inaudible], the corresponding
focus movements are handled
through UIFocusEnvironment and
the related focus classes.
Note that not all system user
interface elements provided by
UIKit are available to automaker
apps.
There's a selection of SiriKit
intents relevant to automaker
apps.
The commands intents are
available to all apps.
They don't require a CarPlay
connection nor participating in
the CarPlay app program.
But an automaker app is a great
place to implement these
intents, which include locking
the vehicle, checking the fuel
level and sounding the horn.
There's also a set of intents
that require a CarPlay
connection and can only be
implemented by automaker apps.
The CarPlay intents include
climate control, defroster and
seat heater settings.
Also included are intents for
tuning the radio and selecting
an audio source.
There are a few best practices
for automaker apps.
The first is to make your app
useful even when CarPlay is not
available.
For example, when disconnected
from CarPlay an automaker app
can use Internet services to
remotely communicate with the
vehicle.
Consider backwards and forwards
compatibility in your automaker
app.
App updates should continue to
support older vehicles and new
features should not break
existing versions of the app.
Simplify the user interface,
most CarPlay apps should only
contain fundamental interface
elements, such as buttons,
labels, tables, navigation bars
and tabs.
Finally, if your automaker app
supports vehicles with a
hardware navigation device such
as a [inaudible] determine how
focus should move between the UI
elements.
We hope that you've learned how
to integrate an audio,
messaging, VoIP calling or
automaker app into CarPlay.
For more information about
CarPlay, see
developer.apple.com.
You can also learn more in
related sessions.
See the Debugging with Xcode 9
session for more information
about debugging apps during a
wired CarPlay connection.
For more information about
SiriKit, see the sessions What's
New in SiriKit and Making Great
SiriKit Experiences.
To learn how wireless CarPlay
works watch the Developing
Wireless CarPlay Systems video.
Videos from WWDC 2016 are
available for both CallKit and
CarPlay systems development
topics.