Transcript
[ Music ]
>> Hello, and welcome
to our first session
on developing CarPlay Systems.
My name is Emily,
and I'm responsible
for Car Experience
engineering at Apple.
CarPlay was designed from
the ground up as a smarter,
safer way to use your
iPhone in the car.
CarPlay takes the
things you want to do
with your iPhone while
driving, and puts them right
on your car's built-in display.
CarPlay brings the
familiar elements
and interaction patterns
of iPhone into your car.
Users already know how
to do things in CarPlay
because they know how
to use their iPhone.
CarPlay is about more
than just showing iOS
on the car's display.
CarPlay is thoughtfully
integrated
CarPlay is thoughtfully
integrated
with your car's built-in system
to deliver the best
possible user experience.
CarPlay allows you to
design for the future.
With CarPlay, you can
build a car today,
and when Apple introduces
new versions of iOS,
users will immediately
benefit from new features.
Your car will remain fresh
for years into the future.
Today we'll give you an overview
of how the CarPlay
feature works,
and discuss the most
important requirements
for you to consider.
We'll explain the user
experience that we want
to enable with CarPlay
and the design guidelines
that we've provided.
At the end, we'll talk about
CarPlay supported apps.
IPhone user interface is
designed to work perfectly
when held in the hand, when
you can touch the phone.
CarPlay translates that
user interface to the car,
making it appropriate in that
setting, and safer to use.
It works with what you
already have in your vehicle.
It works with what you
already have in your vehicle.
It's comfortable to use
in that environment.
Now let's talk about how
CarPlay works in detail.
Let's start with the native
user interface of the vehicle.
When an iPhone is plugged in,
the CarPlay session begins.
When a CarPlay session
is brought up,
information is sent
both directions via
communication protocols.
Video can be sent
from the iPhone
to the car as well as audio.
Audio can also be sent from the
car's microphone to the iPhone.
Once a CarPlay session
is established,
different combinations of
these data streams are used.
When a CarPlay Home
screen is first shown,
and no audio is playing, the
communication protocols are used
for setup and location
information.
An H264 video stream showing
the Home screen is sent
An H264 video stream showing
the Home screen is sent
from the iPhone to the car.
If we then play audio
in CarPlay,
the communication protocols
will additionally be used
to send information
about the playing track.
The video stream here is
encoding the now playing screen,
and audio for the song
is sent to the car.
If the CarPlay UI is exited,
and the car's screen is shown,
the CarPlay session
remains active.
The communication protocols
will continue to be used
for now playing and
location information.
The video stream is
no longer needed,
but CarPlay audio still plays,
even though CarPlay
video is not shown.
The screen and audio are
controlled independently.
Video from the native system
can be shown while CarPlay audio
is playing.
If the CarPlay interface
were shown,
audio from the native system,
like FM radio, could play.
audio from the native system,
like FM radio, could play.
Let's see what happens if Siri
is then launched using the
push-to-talk steering
wheel button.
The communication protocols will
be used to tell the head unit
that iOS voice recognition
is underway.
The CarPlay UI is
sent to the screen,
and the Siri audio chime plays
on the vehicle's speakers.
In this case, audio is also sent
from the car's microphones
to the iPhone.
These examples gave you
a high-level overview
of how CarPlay works.
Now let's look in more detail
at what's happening
in the head unit.
For wired CarPlay,
data goes over USB.
All data is wrapped in IP.
Except for the iAP2
communication protocol.
Information from vehicle sensors
is sent over iAP2 as is audio,
telephony, and turn-by-turn
metadata.
telephony, and turn-by-turn
metadata.
The communication plug-in is
source code provided by Apple,
and that receives the incoming
video and audio streams as well
as an additional
communication stream.
The audio and video is sent to
the head unit's infrastructure,
connected to the speakers,
microphone, and display.
Control events from the
touchscreen, knob, buttons,
or touchpad are sent
to the plug-in.
Now in the case of
wireless CarPlay,
much of the system
looks the same.
Both Wi-Fi and Bluetooth
are needed.
For wireless, all
data is wrapped in IP.
Connections to the vehicle's
subsystems are the same.
Now that you have a
good understanding
of how CarPlay works, we'll go
over the most important
vehicle requirements.
We will talk through
each of these in detail.
We want to make sure you
understand the requirements
We want to make sure you
understand the requirements
before you begin to design a
system so that you can be sure
that it's capable of
supporting CarPlay.
First, the vehicle
needs a great display.
A high-quality display
is critical
to the CarPlay experience.
Several standard display
resolutions are supported,
and those resolutions
are shown here.
Most are approximately 16 by 9.
Extra-wide screens
are also supported.
Today, all supported resolutions
are landscape orientation.
We have support for screens
with varying pixel densities.
Asset sizes in CarPlay have
been adjusted depending
on the resolution of the screen.
So that assets will
appear the same size on low
and high resolution screens.
Without this, icons on the 720
screens would be much smaller.
If you do have a display with
a slightly different resolution
than what was listed here,
please contact Apple to see
if it can be supported.
We require that aftermarket
screens be no smaller
than six inches
since aftermarket
systems could be installed
into a wide variety of vehicles.
There is no minimum for
systems built by automakers.
The automaker has the
responsibility to make sure
that CarPlay is appropriate on
the selected vehicle screen.
I should mention several
additional screen requirements.
Twenty-four bit color
depth is required,
and a 60 hertz refresh rate
is strongly recommended
for a responsive user interface.
Make sure that your H264 decoder
supports the right profile.
This is a hard requirement.
A high-quality display in
the vehicle is critical
to the CarPlay experience.
Let's talk about the car's
audio system and microphones.
CarPlay audio is separated
into two streams main
audio and alternate audio.
Main audio is bi-directional,
and it is used for music
and other media plus
phone calls and Siri.
Alternate audio is
used for notifications,
and it must always be mixed by
the head unit with main audio.
If the head unit is playing
audio, like FM radio,
the head unit must
mix alternate audio
from iPhone with that audio.
LPCM is used for wired CarPlay.
Wireless CarPlay
requires compressed audio.
AAC-LC is used for media, and
you have a choice between OPUS
and AAC-ELD for other audio.
CarPlay supports the
various user input devices
in your vehicles.
The CarPlay UI can be
controlled by a touchscreen
on the head unit, by knobs
and buttons, or by a touchpad.
on the head unit, by knobs
and buttons, or by a touchpad.
The Siri button on the steering
wheel is also a critical user
input device.
Many vehicles have touchscreens,
and let's talk about
those first.
When the CarPlay UI is active,
and the user touches the screen,
the head unit sends
the iPhone the x,
y coordinates of that touch.
The head unit sends all
touches to the iPhone
when CarPlay is displayed
on the screen.
CarPlay is controlled
with single touches only.
We support two types
of touchscreens,
what we call high-fidelity
and low-fidelity.
High-fidelity touch
systems have low latency.
So swipe gestures that track
the user's finger are allowed.
A high-performance system
is required, however,
one that has less than 140
milliseconds touch latency.
This is how list scrolling
using a swipe gesture
This is how list scrolling
using a swipe gesture
that tracks the touch
input works
with a high-fidelity
touch system.
Low-fidelity touchscreens
may have higher latency
and are often resistive screens.
Taps control the UI, and
a swipe will not track the
user's finger.
This is how list scrolling
with single taps works
with a low-fidelity
touch system.
Some vehicles use knob,
button, and touchpad controls.
Sometimes these are the
main user input devices,
and sometimes they are
secondary to a touchscreen.
All combinations are
supported by CarPlay.
Knobs are supported by the
head unit, sending rotate,
select, and back events.
This is the minimum set
of supported features
This is the minimum set
of supported features
for a knob that's used
as a primary interface.
Rotate, select, and back.
Some knobs also support
a tilt or nudge feature.
This capability is
supported by the CarPlay UI.
Vehicle touchpads report
x, y coordinates as well
as select and a back button.
If a knob or touchpad
controller is primary,
back and select are required.
Secondary touchpads can also
support character recognition.
Buttons for various functions
such as next, previous,
and telephony functions
are optional.
The CarPlay UI will
automatically change
to match the user inputs
available in the vehicle.
to match the user inputs
available in the vehicle.
All combinations are possible.
If the main controller
interface is touchscreen,
then combinations
with a knob, touchpad,
or knob plus touchpad
are also possible.
A hard button for
Siri is required.
Siri is really important
to the CarPlay experience.
The user needs a quick way
to always access Siri even
if the CarPlay UI isn't showing.
Usually this requirement
is satisfied
with a push-to-talk button
on the steering wheel.
For the Siri button, we require
that the head unit send iPhone
all button up/down events
so that interaction during the
Siri conversation is possible.
This wraps up our
discussion on user inputs.
Sensor information is
important to CarPlay.
CarPlay needs location
information from the vehicle
so that the map's
app works well.
Speed and GNSS information
can be provided.
Every vehicle needs
to know its speed.
Speed information is used
by iPhone for dead reckoning
and is critical if the
vehicle does not have a GPS
or GLONASS receiver.
GNSS information includes
latitude and longitude
if the vehicle has
that capability.
Satellite location information
from the vehicle is used
in addition to the
iPhone's sensors
to determine the
location of the user.
This data from the
vehicle may be better
since a larger antenna can
be integrated into the car.
This is the recommended
configuration
since it will result in the
best maps user experience.
Sending satellite
location information
to iPhone is required
if the vehicle has the
technology on board.
GNSS information
is also required
for any system supporting
wireless CarPlay
since it's more likely that the
phone will be in a pocket, bag,
or somewhere where the phone
itself has poor reception.
or somewhere where the phone
itself has poor reception.
What are the requirements
for connecting an
iPhone to the head unit?
Apple CarPlay supports both
wired and wireless connections.
We're excited about the
addition of wireless CarPlay
since it offers a very
convenient experience
for customers.
Users can leave their
phones in pockets and bags,
which is especially
nice on a short drive.
Let's first talk about
wired connections.
USB is used here,
and all audio, video,
and control information
is sent over USB.
Your head unit will need
to support a USB role swap.
The head unit will switch from
a USB host to a USB device.
High throughput is a
requirement, and you'll want
to make sure your USB
infrastructure can support the
required data rates.
If you have multiple USB
ports in the vehicle,
and not all of them
support CarPlay,
you'll need to properly
label the ports.
you'll need to properly
label the ports.
Wireless CarPlay is
supported in iOS 9.
Both Bluetooth and
Wi-Fi are required.
Bluetooth is used for discovery
and the initial connection.
Once the Wi-Fi credentials
are sent to the iPhone
over Bluetooth, Wi-Fi
is connected,
Bluetooth is disconnected,
and all subsequent CarPlay
communication is over Wi-Fi.
During the CarPlay
session, all audio,
including phone call audio and
video, is transmitted over Wi-Fi
as well as the control protocol.
CarPlay requires a
Wi-Fi certified AP,
and 5 gigahertz is recommended.
Note that high throughput,
25 megabits per second,
is required.
The Bluetooth subsystem needs
to support the communication
protocol.
to support the communication
protocol.
Wireless CarPlay offers a
convenient comfortable user
experience, and we're excited
to see your vehicles
that support it.
Now let's hear from Donald about
the CarPlay Design Guidelines.
>> Hi. My name is Donald, and
I work on design at Apple.
Now that you've seen
how CarPlay works,
let's look at some
design guidelines
that will help you provide
the best CarPlay experience
in your cars.
One of the most important
things to keep in mind is
that CarPlay is really designed
to use the entire display.
CarPlay is instantly
familiar to iPhone users,
and they feel comfortable
with it
because it's a consistent
environment with its own icons,
visual language, and system
of internal navigation.
Every interaction with
CarPlay is easier and faster
when users can trust
that everything
on the screen is
part of CarPlay.
If a user has to discern which
elements are part of CarPlay
and which elements are part
of your car's user interface,
and which elements are part
of your car's user interface,
then every interaction
becomes more challenging.
Also, your car's native user
interface may include icons
that resemble or have
similar functions
to icons within CarPlay.
And it can be confusing
if these icons are
on the display at the same time.
Instead, use the entire
display to show CarPlay.
Let's take a look at what should
happen when CarPlay starts.
CarPlay always starts the moment
iPhone is plugged into your car.
Users should never have to
think about how to start it.
When the user connects
their iPhone to your car
for the first time, the
car's display should switch
to show CarPlay right away.
This provides feedback
that CarPlay is started
and creates a strong association
between the act of connecting
and the familiar
appearance of CarPlay.
and the familiar
appearance of CarPlay.
Don't show notifications
to announce state changes
related to CarPlay.
The needed appearance of CarPlay
in the display is the most
effective visual feedback.
Now let's consider
what should happen
after the initial
CarPlay connection.
When users connect their iPhone
a second time, it should feel
like a continuation of
their previous session.
So if CarPlay was
showing on the display
when the user left the car,
CarPlay should be shown again
when iPhone is connected.
And if CarPlay media
was being played
when the user left the car,
CarPlay media should resume
when iPhone is reconnected
as long
as the user did not select
a different audio source
between their CarPlay
connections.
In other words, we want
to always respect the last
selection the user made
in the car.
Let's see how last
user mode should work.
Notice that I'm using CarPlay,
and it's showing on
the car's display.
Now if I disconnect, CarPlay
ends, and the car falls back
to its native user interface.
If I plug in again, CarPlay
should appear on the display
because that was the last
thing I was doing in the car.
So if CarPlay was the
last thing on the display
when it was disconnected,
or the car was turned off,
CarPlay should be shown when
iPhone is connected again.
If CarPlay media was playing,
CarPlay media should play again.
Let's see a different example.
This time, CarPlay is active
with the car's display showing
the native user interface.
Now if I disconnect
and then plug in again,
CarPlay does not
appear on the display
because the native user
interface was the last thing I
was using in the car.
So if CarPlay was not the last
thing shown, it should remain
in the car's user interface.
But even if CarPlay was not
the last thing on the screen,
But even if CarPlay was not
the last thing on the screen,
if CarPlay media was playing,
CarPlay media should play again.
CarPlay is available the moment
the user plugs in their iPhone,
but they can switch
between the CarPlay display
and the native user
interface at any time.
As we saw earlier,
the user returns
to the car's user interface
by selecting your brand's icon
in the CarPlay Home screen.
They can switch back to CarPlay
by using the Apple
CarPlay button.
The Apple CarPlay button is
the most important indication
that CarPlay is available.
As soon as CarPlay starts,
the Apple CarPlay
button must become active
in your top level menu.
And the Apple CarPlay must only
appear when iPhone is connected
and CarPlay is active.
Be sure to hide the
Apple CarPlay button
when iPhone is disconnected.
And they also show the
Apple CarPlay button
And they also show the
Apple CarPlay button
in an enabled state
when CarPlay is active
and in a disabled state
when CarPlay is inactive.
You can optionally replace an
existing button the top level
menu with the Apple CarPlay
button such as a phone button.
Just remember that the Apple
CarPlay button must appear
on your car's Home screen
whenever iPhone is connected.
When iPhone is disconnected,
the Apple CarPlay button must
disappear or be disabled.
Let's talk about the
Apple CarPlay button
that should appear
in your interface.
There are a variety of approved
Apple CarPlay logos available
to use as buttons in your
car's native user interface.
Choose the one that makes the
most sense with your design.
There's a black logo and a
white logo as well as white
and black variance
with backgrounds.
And, finally, there's
a color background.
Let's talk about how
media sources are handled
when iPhone is disconnected.
Let's say that CarPlay
is active,
and the user is listening
to CarPlay audio playback.
When iPhone is disconnected,
it's a strong indication
that they're about
to leave the car.
Your system should
gracefully handle CarPlay media
being disconnected.
When iPhone is disconnected,
the user should hear silence
and audio should not fall
back to another audio source.
If the iPhone is disconnected
while CarPlay is displayed
on the car's display, you
should gracefully fall back
to the last screen displayed
before entering CarPlay.
Don't show a CarPlay unavailable
or device disconnected screen.
So, again, when CarPlay
media is playing,
and the user disconnects iPhone,
they should just hear silence.
The hardware buttons in
your car should also be used
to support CarPlay.
Siri is an integral
part of CarPlay
and requires a dedicated
tactile button.
Because it's so important,
the button should be
on the car steering wheel.
Users expected Siri to
work at any time, whether
or not CarPlay is currently
visible on the car's display.
They also expect the
Siri button to work
like the Home button on iPhone.
Pressing and holding it for 600
milliseconds should activate
Siri, and the activation
delay must not be greater
than one second.
The media buttons in your
car should be connected
to CarPlay media playback
so users can play, pause,
move to the next track
or previous track,
and fast forward and rewind.
Finally, if your car has
shortcut buttons for phone,
media, or navigation,
they should also be mapped
to the respective
apps within CarPlay.
>> Now let's talk about
apps that work in CarPlay.
>> Now let's talk about
apps that work in CarPlay.
Apps supported by
CarPlay will appear
on the CarPlay Home
screen in the vehicle.
Three categories of
apps are supported.
Audio, those specific
to particular vehicles,
and messaging apps.
If a user has an audio app
that's been updated for CarPlay,
that app will appear
in any CarPlay car
the user connects to.
The user interface for
these apps is fixed
with the structure
provided by iOS
and the content provided
by the app itself.
You'll notice that
audio apps look a lot
like the built-in music app.
Users choose what to listen
to via a list-based UI,
and the Now Playing screen
shows the appropriate content
and controls.
The app needs to have a
special entitlement in order
to be compatible with CarPlay.
We've made some improvements
in iOS 10
to make the audio apps
experience even better.
to make the audio apps
experience even better.
New in iOS 10 is support for
tabs, which make it easier
to navigate list hierarchies.
In the Now Playing screen,
you can now see the app name
and additional playback
control buttons.
In the list views, you can
tell which item is playing,
which are available
for streaming,
explicit or live stream.
Some apps are designed
to play audio as soon
as they're launched,
and we've added support
for these apps in iOS 10.
All of these combined make for
a much improved experience.
Check out the new tabs, the
Cloud icon for streaming apps,
and the way tend to note
what's currently playing.
In the now playing
screen, here's the app name
and the additional
playback controls.
Now let's talk about that
next category, apps written
to work in specific vehicles.
Apps published by
automakers are written
to control vehicle functions
or give information
about the vehicle.
They only appear in vehicles
specific to that automaker.
A CarPlay entitlement is
required for these apps as well.
New in iOS 10 is Siri
support for automaker apps
that control vehicle functions.
Siri now has built-in support
for changing radio stations,
changing climate
controls, and more.
The automaker app
communicates with the vehicle
to perform these functions
after Siri tells the app what
the user would like to do.
Let me explain how
automaker apps work only
in the cars that support them.
An app icon is enabled in
the CarPlay Home screen only
if the iPhone is
connected to a head unit
that supports the same
protocol name as the app.
A protocol name supported
by the head unit
and the app need to match.
iAP external accessory
is used for app
to head unit communication.
Head units can declare
multiple protocol names in order
to support current
and future apps
with different groups
of target cars.
Head units should
use a protocol name
to match all cars
from that brand.
For example, com.brand.
They should also use
protocol names to match cars
with specific features,
for example,
electric vehicles
or sports models.
A protocol name for the specific
model should also be included
so that apps that target just
that model can be
published in the future.
The head unit needs
to use iAP2 to declare
at least one supported
external accessory protocol.
Likely more than one.
Start and stop external protocol
session must also be supported,
and external accessory protocol
CarPlay must be set to true.
and external accessory protocol
CarPlay must be set to true.
If the car supports
wireless and wired CarPlay,
these iAP2 messages
should be implemented
on all supported transports.
Most automaker apps will
communicate with the car
to perform tasks such as
changing the climate controls.
These apps need to design
and implement a communication
protocol as well.
So how does the app
support this feature?
The app's Entitlements.plist
needs to be modified
to include the protocol
names that match those
on the target head
units, as shown here.
With these additions plus
support for the protocol
to communicate with the car,
and a CarPlay entitlement,
your app will be
ready for CarPlay.
We're looking forward to
seeing some great CarPlay apps
from automakers.
We have a new category of
apps supported in CarPlay
We have a new category of
apps supported in CarPlay
and iOS10, messaging apps.
When a messaging app has been
updated to support CarPlay,
that app will appear on
the CarPlay Home screen.
Messages can be received
and sent,
all driven by Siri interactions.
We're excited to extend CarPlay
capability to messaging apps,
making that experience
smarter and safer.
We hope this has been a
useful overview of CarPlay.
CarPlay is a great way to
keep the user experience
in vehicles fresh long into
the future, and we're delighted
to get a chance to talk
to you about the features
that are new in iOS 10.
For more information, check
out this session's WWDC URL.
We also have a follow-on
CarPlay deep dive video session
that may interest you.
Thank you for watching.