Transcript
[ Music ]
[ Applause ]
>> Hello everyone.
I'm Ross. I'm an engineer on the
RealityKit team at Apple and
welcome to Building Apps with
RealityKit.
So this session is intended as a
follow up.
So you might want to check out
Intro to RealityKit and Reality
Composer Session.
Today we're going into the
applied usage of RealityKit.
And we'll walk you through
building an augmented reality
app and how you can leverage
many of the frameworks key
features.
Before we dive in let's do a
quick recap of what RealityKit
is.
So RealityKit is Apple's new
framework for building AR Apps.
Trying to make it as simple and
intuitive as possible for any
developer.
It's been built from the ground
up to work with AR.
And it allows you to seamlessly
blend rendered content with a
real world environment.
Its Swift API is simple yet
powerful and lets you do a lot
with only a few lines of code.
All right, let's build an app.
The app we're going to be
building today is called "Memory
Cards."
And as the name cleverly
implies, it's a card matching
game.
To play you tap on a card to
select it and its image is
revealed.
Select another card to reveal it
and if the pair is a match the
cards are removed.
If they don't match both cards
are hidden and you're free to
select another pair.
This simple game will allow us
to highlight and discuss a lot of
useful RealityKit features.
So we're going to build Memory
Cards in four stages.
First we're going to put
together a prototype that will
place our content in AR and
allow for some simple
interaction.
Next we'll add some polish with
some fancy art assets, improve
performance and AR rendering
tricks.
Then we'll leverage the entity
component system to track custom
state information.
And finally we'll show you how
to use RealityKit's built in
network support by adding
multiplayer to our game.
All right.
Let's get started with our
prototype.
So as mentioned in the intro
session there are four objects
that you'll need to use in every
RealityKit app -- an ARView, the
Scene, Anchors, and Entities.
ARView is your window into the
world of AR.
And in your entry point for
RealityKit.
It's a view and it goes into
your apps View hierarchy.
The scene holds up with a
virtual content that makes up
your app and is owned by ARView.
In RealityKit, Anchors describe
how objects relate to the real
world.
And you'll need them to be able
to place your virtual content.
To place an anchor you add it to
the scene and when the
appropriate target is found the
anchor is automatically placed
in the world.
For Memory Cards we'll use an
anchor to place our game board
on a horizontal surface.
And entities are used to
represent your virtual content
and are the building blocks that
make up your scene.
In Memory Cards each of the
cards will be represented by an
entity.
Since your game has 16 cards
we'll need 16 entities.
So now that we've covered the
important elements that we'll
need in our app let's talk about
how we'll create our anchor so
we can place our virtual game
board in the real world.
RealityKit anchoring is built on
top of and integrated with
ARkit.
But then you leverage it's full
feature set.
To define Anchor content you
create an anchor entity.
To clear the type of anchoring
you'd like to use and then add
the anchor entity to your scene.
Once the appropriate target is
detected by ARkit, the anchor
entity will automatically track
it and making your virtual
content feel like it's been
placed in the real world as your
device moves.
Also as mentioned in the intro
session, RealityKit supports all
of the anchor types available
with ARkit, allowing me to
anchor things like planes,
faces, images, objects, and new
for ARkit 3, body anchors.
For Memory Cards we just need a
single anchor to place our game
board.
We like to place it on a
horizontal surface with enough
space to fit all the cards,
about 20 centimeters squared.
And that anchor will represent
our game board in the world and
we'll place our content around
it.
Let's write some code to do
this.
So here we have the view
controller for our Memory Cards
prototype.
We have an ARView in place in
the View hierarchy and we're
ready to find our anchor and
create out virtual content,
which we'll be doing in the
viewDidLoad method.
To define our anchor we create
an anchor entity and using this
convenience initializer we can
specify the type of plane we'd
like it to anchor to.
In this case the horizontal
plane.
And we can optionally pass in
the minimum area we require, 20
centimeters squared.
RealityKit's units are in meters
so I get 20 centimeters by 20
centimeters, we pass in 0.2 by
0.2.
Then we just add our anchor to
the scene and as soon as
RealityKit finds a horizontal
plane of at least 20 centimeters
squared, any virtual content we
attach to this anchor will
appear in AR.
Now that we have our anchor set
up let's add some virtual
content.
First we need to load in the
model.
RealityKit natively supports
assets in USDZ in the new
Reality File formats.
Loading can be done
synchronously or asynchronously.
For now we'll start with
synchronous loading but we'll
touch more on this later.
When you load a USDZ or reality
file asset, RealityKit
automatically imports its entity
hierarchy, and meshes used by
the asset, its materials, and
any animations it may have as
well.
Let's load in our basic card
assets.
To load a model you just need to
go entities load model method
with the name of the asset.
There's not need to specify the
file extension if the asset is
in your app bundle.
If RealityKit is able to load
the asset, it will automatically
create a ready to use entity for
you to use however you want.
Here we're loading on the eight
base card models and storing
them in an array.
So Memory Card has 16 total
cards.
There are eight card types, each
with a different image.
And then there are two instances
of each type to create our eight
matching pairs.
So we've got our eight type
models loaded, but how do we get
the other eight?
Now we could call entities load
model method again, but any
setup we perform on our cards
will need to be done for each of
these as well.
RealityKit provides and easier
solution with entity cloning.
To create a clone you call
entities clone method.
And cloning creates an identical
copy of the original entity.
It references all the same
assets used by the original, and
cloning can also be done
recursively, which will clone
all of entities children as
well, which really comes in
handy when you're composing
complex scenes with deep
hierarchies.
It's also important to note that
clone entities are copies of the
originals, not instances.
So if you make a change to the
original entity, say by removing
a child, that change will not be
reflective in any of its all
ready existing clones.
Let's use cloning to create our
card pairs.
So here we're cloning the two
instances of each card type we
need and storing them in a
separate array, which we'll use
to build our game board.
We could use the original card
templates and then just clone
one new instance, but by keeping
the card template separate we
can easily refer back to them to
recreate the game board later
without having to load them all
again from scratch.
Now that you have all 16 cards
ready to go we need to place
them in our game board.
The anchor will be at the center
of the play area and we'll
arrange the cards in a
four-by-four grid around them.
Then all we need to do is add
each card to the anchor and
they'll be displayed in our ARC.
So to do this we calculate the
position of each card using its
index and the array and then set
it on it's position property.
This will position it relative
to its parent, the anchor, when
we add each card to it.
And that's all we need to do to
get our cards laid out and
rendering in AR.
So we got our cards placing in a
surface in AR, but we can't do
anything with them yet, so let's
add some interaction.
We want the cards to flip when
we tap on them.
But to be able to do that we
need to translate that tap on
our devices screen into the
world of AR so we can figure out
what we're actually tapping on.
And RealityKit provides a
solution to this with hit
testing.
Hit testing works by turning a
2D point that was tapped on your
device's screen into a ray in
our virtual scene.
That ray is then cast into the
scene and RealityKit finds all
of the objects that are
intersected by the ray.
Any entities that were
intersected by the ray are
returned and you now know what
objects lie under the tap.
ARView provides the methods for
hit testing.
Entity at point returns the
entity closest to the camera for
the given point and entities at
point returns all the entities
that are intersected by the ray
caster that given point.
And we can easily integrate this
into our Memory Cards app.
Here we have a method responding
to a tap gesture recognizer.
We get the tap location in the
ARView and then pass it into
entity at, which we're using
because we only want the entity
closest to our device.
If there's an entity under that
tap location, dis call, we'll
return it, and we could then
perform our interaction on it.
However, there's still one more
thing we need to do for hit
testing to work.
For entities to be hit testable
they need a collision type
shape.
Collision shape is simplified
geometry, typically a box.
They're easy to find and they
allow for efficient intersection
and collision calculations.
And importantly without a
collision shape, entities are
not hit testable.
So let's add them to our cards.
So this is the same code we saw
before for leading our models
and creating our card templates.
We'll make a small change here
to add entities generate
collision shapes method.
This will automatically generate
simple box collision shapes for
an entity using the entities
visual bounds.
As you can see, much like with
cloning, this can be performed
recursively creating collision
shapes for all the children of
an entity as well.
And speaking of cloning,
collision shapes are included in
the data that gets copied when
an entity is cloned.
Since we're all ready cloning
the card templates to build our
game board, they'll be
automatically included with
these collision shapes as well.
Now that we can figure out what
entities we're interacting with,
let's add an animation that will
play when we tap on a card.
RealityKit supports two kinds of
animation.
The first is transform
animation.
It let's you animate the
position, rotation, and scale of
entities in code.
And the second is asset
animation, which plays the
animations that are baked into
your assets that are loaded in
from USDZs or reality files.
And RealityKit also provides
completion handlers for use with
both kinds of animation, letting
you know when the animation
finishes.
Once our card assets don't have
a baked in animation, so to flip
them we're going to use a
transform animation on our card
entities.
Transform animations can make
use of a variety of timing
functions that can control the
speed of which the animation is
played.
There's linear, which plays the
animation at a constant rate
with instantaneous acceleration
and deceleration.
There's the ease in, which
gradually ramps up the animation
speed over its duration.
Ease out is the opposite of ease
in, which slows the animation
down until its complete.
And the ease in and ease out
combines both, ramping the
animations to be up until the
halfway point and then slowing
it down until its complete.
And there's also a cubic Bezier
option for customization of your
timing function.
Let's create an animation for
our card flip.
So we're going to start by
copying our card's current
transform.
This ensures that we preserve
the current scale and
translation of the entity, which
right here we don't want to
change.
Next we'll set the transform to
a 180-degree rotation around the
x-axis using a quaternion.
This will flip the card so its
image is facing up.
Now we can start the animation
itself.
Transform animation is applied
using entities "move to" method.
RealityKit will smooth the
animated entity between its
current transform and the one
you provide in this method.
Using the requested timing
function, in this case "ease in
and out" over the given length
of time.
This gives us the animation we
need to flip the card face up.
And entities move method return
in animation with playback
controller that allows you to
pause, resume, or stop the
animation, or to receive
notification when the animation
completes like we're doing here.
Then to flip the card back to
being face down, we just need to
make one small tweak setting the
rotation of our transform back
to zero.
When we call move this
transform, the card will flip
back to being face down.
So now we can show the app with
interaction enabled, using hit
testing to detect what objects
the user is selecting and
transform animation to hide the
cards.
Now that we've got the basics
working, let's polish things up
a bit with some detail work.
We've got our simple card models
loaded and placed in AR, but 2D
images on cards is not very
exciting.
Thankfully our art team has also
created a bunch of high quality
models, so let's add them into
the mix to help our app be a bit
more three-dimensional.
We could load these new assets
the same way we did before with
the synchronous entity.load
method.
However, these models are a lot
more detailed than the simple
card assets.
And the larger assets will take
longer to load.
Loading them will be quick, but
the app will be blocked while
they're loading.
And if you're loading a lot of
assets that can add up to
noticeable delays where nothing
is happening.
Is there anything we can do to
make this better?
Thankfully as we mentioned it
earlier, RealityKit offers both
synchronous and asynchronous
loading.
We can load models
asynchronously by calling
entities load model async
method.
With asynchronous loading,
assets are loaded in the
background and this unlocks the
app and allows it to continue
uninterrupted, which makes your
app responsive and allows ARkit
to continue observing the world.
When asynchronous loading is
complete you'll receive a call
back and then it can use your
assets just as you would with
synchronous loading.
And additionally you can also
combine load requests and
execute them simultaneously and
receive notification when all
assets have finished loading.
So you don't need to load your
content piecemeal.
Let's write some code to load
our new models asynchronously.
So we'll start by loading a
single model asynchronously.
Entities load model async method
takes on the name of the asset
just like its synchronous
counterpart.
It returns a load request to
receive notification when the
model has finished loading and
can be used.
We call the sync method with a
closure that will be executed
when the asset is ready.
The load request uses API
introduced with a new Swift
framework Combine.
So I recommend checking out the
session Introducing Combine and
Advances in Foundation.
And that's all you need to do to
be able to load your content
asynchronously.
No you could load all of your
assets individually like this,
but you can also combine
multiple load requests into one.
We can do this by simply
appending another load request
on to the first one.
We then call collect to combine
our to load requests.
And then we call sink.
And our closure will be executed
when both models have finished
loading, packaged into the
models of array parameter for
easy processing.
And we can extend this as far as
we want.
Here we're collecting all eight
of our detailed card models into
a single load request.
Our sink closure will be
executed when all eight have
finished loading.
Combine and load requests makes
managing your content much
easier.
Let's show how synchronous
loading compares with
asynchronous loading.
On the left our app is using
synchronous loading and on the
right asynchronous.
As soon as the synchronous app
starts loading everything comes
to a stop as the app is blocked
until loading is complete.
By contrast the asynchronous app
continues to be responsive
allowing it to respond to user
input, update the camera, and
you continue to observe the
world in front of the camera.
Loading takes about the same
amount of time for both, however
the device using async loading
is able to place its content
first because its able to
continue observing the world
during loading.
While the device using
asynchronous loading is not
making it a much better fit for
use with AR.
So now we have out detailed
models loading in without
freezing the app.
However an astute observer may
notice that we can see the new
models while the cards are faced
down.
We could simply hide the models
while the cards are face down,
but nothing would stop the user
from bending over and looking at
the underside of the cards and
seeing which ones match up
without having to flip them.
So RealityKit provides a super
useful way to fix this,
occlusion materials.
Occlusion materials are
invisible, but when applied to
geometry in a scene they hide
virtual content behind them
revealing the video pass
through.
Here you can see we've added
some occlusion geometry to our
scene, along with some
additional highlighting to show
the shape of an otherwise
invisible object.
As in animates up and down parts
of the robot it intersects are
hidden and lets the real world
behind it show through.
Occlusion materials are great
for simulating real world
objects that would block virtual
objects from being seen like a
table or wall.
So let's make use of it in our
app.
Let's start by adding an
occlusion plane under our game
board.
We'll create a plane mesh half a
meter wide and half a meter
deep.
We'll then create our occlusion
material and then create a model
entity using it and our plane
mesh.
We'll position it slightly below
the game board so that it
doesn't intersect the bottom of
our models.
And then we just add it to our
anchor to place the occlusion
plane in our scene.
Here we can see the occlusion
plane in action.
At first glance this seems to do
a great job of solving the
problem, however when we start
to move the device down we can
see the edge of the occlusion
plane and see our virtual
content rendering inside the
table.
So this plane works great when
we're above the game board but
we need occlusion that will work
for an angle in this case.
Solution is using occlusion
box instead of an occlusion
plane.
So we'll generate a box, make
use of the same occlusion
material, and then create a
model entity using the box in
place of the plane.
Generated geometric objects have
their origins in the center so
we need to bump it down by
half a size and then just a
little bit more so that the top
of the box will rest just below
the bottom of the game board.
And then once again we just add
it to our anchor to place the
occlusion box in our scene.
Initially it looks the same as
the occlusion plane until we
start to move our device down
and we can see, or rather can't
see our virtual content.
Our occlusion box is preventing
content from rendering inside
the table simulating the real
world object in our virtual
scene.
So now we have an app where we
can place our virtual cards in
the real world and interact with
them.
We're loading assets
asynchronously and we're
simulating our virtual content
being hidden by real world
objects using occlusion
geometry.
Now I'd like to invite my
colleague Courtland up to talk
about tracking game state.
Courtland.
[ Applause ]
>> Thank you Ross.
Hello. I'm Courtland, an
engineer on the RealityKit team
at Apple.
We've just seen how to prototype
a functioning AR game, add
interaction, and integrate final
artwork.
Now I'd like to show you how to
best track your state with
custom components and entities
and finally how to add
mutli-player.
Let's start with tracking state.
As we covered in the intro
session, RealityKit uses the
entity component design pattern
to build objects within the
virtual world.
An entity itself is comprised of
pieces called "components."
These components define specific
behaviors and data that can be
added to individual entities.
Using entities and components
allows for reuse of code and is
flexible to use.
Let's take a look at how we can
apply components.
We're using RealityKit's model
entity to represent our cards.
It provides us with a set of
components, which are useful for
representing most common virtual
objects.
We've made use of the model
component for visual appearance
and the collision component for
hit testing.
Model entity also contains a
physics component, allowing the
entity to move and interact with
other objects in a physically
realistic way, though we haven't
made use of it here.
RealityKit enable you to
customize an entity.
By using an entity component
design you can include the
behaviors you want, exclude the
ones you don't need, and add new
behaviors of your own.
Let's customize this entity for
our cards.
Model entity has most of what we
need.
We'll remove physics since we're
not using it.
We want to store whether the
card is hidden or revealed and
the kind of card which we'll use
to determine if two cards are
matching.
To do this we need to create a
new card component type with
these properties, which we'll
add to the entity.
So what exactly is a component?
A RealityKit component is a
Swift struct that contains your
properties.
It conforms to the component
protocol, which allows us to
attach it to an entity.
It's also a good idea to conform
to Codable, as we'll see in the
multi-player segment.
So let's make one.
We start by declaring our struct
called CardComponent.
We adopt the component in
Codable protocols.
Next we'll add our two
properties -- a boolean called
"revealed" to represent whether
the card's contents are hidden
or revealed and a string "kind,"
which we can use to match the
two cards.
That's all for the type.
We'll start with one of the card
models Ross loaded earlier.
For demonstration purposes I'll
show how to remove a physics
body component.
The model entity provides a
physics body property, which
makes this easy.
We just assign nil and we're
done.
Now we'll load the card
component by assigning to the
components array indexed by the
type of component.
This will add the component, if
it doesn't all ready exist, on
the entity.
Changing the kind property is
similar.
Indexed into the entities
components array, and assigned
to the kind property.
Since not every entity contains
a card component, the accesser
returns an optional value.
For common configuration of
components used throughout your
game, we can take it a step
further and create our own
custom entity.
RealityKit comes with a number
of entity configurations such as
directional light and model
entity.
And you can make your own very
easily.
We're using cards everywhere in
our game, so this makes an ideal
candidate to turn into an
entity.
We'll get compile times, static
typing and code completion for
these objects.
This is also where we can add
methods to encapsulate
functionality.
This is especially useful when
you're changing multiple
components at the same time such
as setting the state of the card
to revealed at the same time as
flipping it over.
Creating new entity is just a
few steps.
First we needed a new class to
represent our entity.
Then we added any RealityKit
components we need.
Next we'll add any custom
components.
We solved the physics body
property on model entity
earlier, which provide a
convenient syntax for accessing
the physical body component.
We'll do the same for our card.
Last we extend it with methods
reveal and hide.
So we create our card entity
class derive from entity.
Next include the RealityKit
components.
We're adding the HasModel and
HasCollision protocols.
These protocols give us access
to the model and collision
components via properties and
any methods that they provide
such as generate collision
shapes.
Last we add a card property,
which returns a CardComponent.
Since all cards need a card
component we'll make it
non-optional.
The getter will retrieve our
CardComponent from the
components array.
We use the nil coalescing
operator to return a default
value if we didn't all ready
have a card.
The setter copies new value into
the components array.
Now that we have our class we
extend it with methods.
I've included the reveal method
to show us how easy it is to
coordinate multiple updates.
We first update the card's state
to indicate that it's revealed.
We'll do this immediately rather
than at the end of the animation
in case you tap it again while
it's still animating.
Next we use the same code that
Ross showed us earlier to
animate the card flipping over.
And that's it.
Hide method would do the
opposite.
Set reveal to false and turn the
card to be facing down.
Let's see what this is like to
use now.
We'll go back to the onTap
handler.
We call the same ARView entity
at method, but this time cast
its result to a card entity.
Though we only have cards in our
game, your app might have other
types of entities.
Casting to card entity lets us
perform actions specific to the
card.
With a new card entity and
component, we can just ask the
card if it's revealed.
If it is currently revealed
we'll hide it.
Otherwise we'll reveal it.
With hide and reveal
encapsulated as methods on the
entity, we don't need to be
concerned with the specifics of
hiding and revealing cards here.
We've cleaned things up
considerably by adding a custom
component in entity and made one
important functional change
here.
We now know that a card is
revealed and can tap on it again
to hide it in case it wasn't the
card you remembered.
Now that we've modeled our state
as components and entities, it's
time to take the game to the
next level.
What's something easy we can
add?
Just add multiplayer right?
AR games are fun and playing
with friends makes them even
more fun.
Multiplayer can turn a simple
card matching game into a real
competition, however making a
multiplayer AR game comes with a
few additional challenges that
we don't encounter with non-AR
games.
When they place virtual objects
in the real world, we want them
to be in the same location for
everyone playing the game.
And because everyone is in the
same place, we want updates
between devices to be fast, to
maintain the shared illusion of
reality.
To enable this we built
RealityKit from the ground up to
support multiplayer AR.
Combined with ARKit's
Collaborative Session, also
introduced this year, this gives
you the tools you need to add
multi users support to your AR
experiences.
So let's take a look at what
multiplayer means for our game.
Both players will play
simultaneously and can turn over
cards at any time.
Everyone can see the reveal of
cards.
So you can gain an advantage if
you're paying attention to what
the other player is doing.
We'll also add a small white
circle to indicate to the player
which card they turned over.
Let's take a look at multiplayer
in RealityKit.
RealityKit provides automatic
scene synchronization.
Changes made automatically
update on all devices.
Leverages, device discovery, and
connection for multi peer
connectivity.
This makes it simple to find and
connect to nearby devices
without maintaining servers or
even connecting to the same
Wi-Fi network.
It provides an easy to use
ownership model allowing you to
control which peers are allowed
to change which entities.
And provides low latency even on
busy Wi-Fi networks, which is
essential for convincing AR
experiences.
Let's go through the steps for
adopting multiplayer for our
game.
First we need to designate a
host.
We'll place the game board.
We'll ask the user to select
this at the game's main menu.
Then we connect the sessions so
that they can communicate with
each other.
We'll also enable ARkit's
Collaborative Session, which
enables our peers to create a
shared map of the environment.
And we'll create a synchronized
anchor, which attaches our game
board to a specific world
location and coordinated between
peers.
And last, we use ownership to
make changes to the game board
such as flipping over the cards
and removing them.
We've asked the user to choose
"host" or "join" from the main
menu.
Now we need to establish a
connection.
We use the multi peer
connectivity framework to get
connected.
We won't go into all of the
detail of multi peer
connectivity here, but if you
want to know more check out the
2014 session, Cross Platform
Nearby Networking.
First create an MCPeerID and
MCSession.
Be sure to enable encryption as
it's required for use with
RealityKit.
We'll check the user's role
before proceeding.
If they choose host we'll
advertise the session using
MCNearbyServiceAdvertiser.
This broadcast that we have an
available session.
The client creates an
MCNearbyServiceBrowser to start
looking for sessions.
And now that we have an
MCSession we need to instruct
RealityKit to use it for
synchronization.
We do this by creating a multi
peer connectivity service, which
is a RealityKit class that wraps
the MCSession and makes it
usable for scene
synchronization.
We'll create this and assign it
to the synchronization service
property on the scene.
Now we have two devices
connected and synching their
scene.
However, they don't yet know
where they are in the physical
world relative to each other.
Let's take advantage of the
collaborative session introduced
in ARKit 3, which is natively
supported by RealityKit.
Collaborative session lets us
build the world more quickly and
lets multiple users share the
same in world experience.
Since we want collaborative
mapping we need to turn it on.
We'll do this at the end of
viewDidLoad.
We create a new world tracking
configuration.
Set is collaboration enabled to
true, instruct AR Session to run
our configuration.
Now that we have collaborative
session enabled we can create a
synchronized anchor.
Previously we created an anchor
entity requesting any horizontal
plane that's at least 20
centimeters on a side.
With multiple players we want to
make sure the board is in a good
place for two players to walk
around.
To ensure this we'll ask the
host to place the board.
And since we want the same world
position on both peers, we'll
ask ARkit to synchronize this
anchor.
How do we do this?
On the host, we'll use the same
onTap handler as before.
We want to pick a spot that's
good for everybody.
So we'll raycast into the real
world.
This is similar to the hit test
Ross showed us earlier, however
this one is run against an
estimation of the real world
rather than virtual objects.
Our game board needs to go on a
flat surface, so we're asking
for a horizontal plane.
If it finds a horizontal plane
we'll take the world transform
and use it to create an AR
anchor.
This is an ARkit anchor, which
we add to the AR session
provided by the view.
And this creates a synchronized
anchor whose real world position
will be coordinated by ARkit.
Using that AR anchor we'll
create a RealityKit anchor
entity.
This is our bridge between ARkit
and RealityKit and allows us to
attach our card.
And then we have the game board,
as Ross showed us earlier.
Only the host needs to build the
randomized board.
Everyone else receives it
automatically via network
synchronization.
The models we loaded aren't
synchronized as part of the
scene because that would be
loads of data.
Remember when Ross showed us how
to load the card templates?
Make sure we still do this on
all appearance.
And that's enough to get our
game running.
On the left we can see that the
host has placed the board and
the client can see it.
It's in the same real world
location.
Watch as the host flips over one
of the objects and then another.
OK, not a match.
Notice how it's automatically
reflected on the client and how
smooth the animation is.
No network program here
required.
Now watch as the client flips
over one of the cards.
it flips over in their screen
but not on the hosts.
To explain what's happening I
need to introduce you to
ownership.
What is ownership?
It's the right to modify an
entity.
In a shared session an entity
has one owner at a time, which
defaults to whomever created the
entity.
In our case that's the host.
Ownership is transferable, which
is how you can allow other
players to make changes.
And ownership transfer is
configurable so you can decide
which entities can be
transferred and at what times.
To illustrate ownership let's
look at what's happening in our
game.
The host creates the cards,
which means it owns them.
And the cards automatically show
up on the client.
When the owner of an entity
makes the change, such as
flipping it over, this change is
sent to the client and reflected
in their scene.
Since RealityKit has been built
from the ground up for
multiplayer, we're only
synchronizing the instruction to
animate, rather than to
transform every frame.
This is why it looks so smooth
on the client.
Now the client flips over our
card.
Since the client does not own
the entity, the change will not
get sent to peers.
While it's still allowed to make
the changes locally, they're
likely to get overwritten the
next time the owner sends an
update.
This presents a dilemma because
we do want to make changes to
the card.
So let's go back a step before
the client had turned over the
robot.
So I mentioned earlier,
ownership can be transferred.
Any peer can request ownership
of an entity.
Let's have the client request
ownership of a yellow robot
before making its changes.
It sends a request to the host
asking for ownership.
The entities owner decides
whether to allow the transfer.
It may decline if a different
peer requested ownership or if
the object's transfer mode was
changed.
Since from the default state the
host accepts the request and
ownership is transferred to the
client.
The client is now free to make
changes to the card.
We can reveal the card and the
changes will be reflected on the
host.
And we've made this simple to
use.
The client starts by requesting
ownership of the entity.
If we all ready own the object
that's fine.
The request will be granted
without doing any additional
work.
When the ownership request
returns we'll be informed
whether the ownership was
granted.
If it was, we'll call reveal on
the card to flip it over.
However, if the request was
denied, for example someone else
turned it over, we'll give the
user the opportunity to select a
different card.
And that's all we need to flip
over the card and see it on both
peers.
But we want to go one step
further.
While the card is revealed, we
want to keep ownership of it.
We want to deny any requests to
take ownership while we have it
revealed.
We'll go back to our reveal
method.
Here's where we previously set
the revealed property to true.
Because we adopted the codable
protocol on card component, this
automatically updates on other
peers.
There's nothing else to do.
And after we change the cards
revealed property to true, set
the ownership transfer mode to
manual.
This automatically denies any
requests to transfer the entity.
When we flip the object back to
hidden we'll want to start
accepting requests for
ownership.
This is what the hide method
might look like.
After changing revealed to
false, we changed the ownership
transfer mode to auto accept.
This will instruct RealityKit to
accept transfer requests for
this entity automatically.
One thing that's worth pointing
out -- the host isn't special
when it comes to ownership
because it placed the board, it
initially owned all the cards,
but otherwise behaves like any
other peer.
Once an object is transferred,
if the host wants to turn it
over it needs to request it too.
You can use the ownership
transfer mode to change this
behavior to suit your app.
And there's one last detail for
our app.
When you play with two or more
players you'll notice that the
board can get a little
confusing.
It could be hard to know what
piece you've selected and which
ones were selected by others.
We'd like to add a transparent
disk to display our selection.
As we've seen, everything gets
shared.
So that will show for everyone
and defeat its purpose.
Fortunately, RealityKit supports
local-only entities.
This is ideal for displaying
selection indicators or hidden
information like your hand in
poker.
And we do this by removing the
synchronization component and
otherwise handling the entity
like any other.
If the entity has children, they
will be unshared as well.
This can be useful to make an
entire tree of entities local
only.
While I'm not showing it here,
I've created a selection entity
class that represents our
selection.
It's initializer adds a white
slightly transparent disk.
Let's take a look at adding it
to our reveal method from
earlier.
We create the entity and
position it slightly above the
card.
Then we assign nil to the
synchronization component.
This instructs RealityKit that
we do not want to share this
entity.
And we add the child normally to
enter the hierarchy.
Now that we've added it in
reveal, we need to remove it
when we hide the card.
Here's the hide method from
earlier.
After changing other properties,
we'll iterate over the children
with the for loop.
We can use the where clause to
look for entities of type
selection entity.
Once we find it we'll remove it
from its parent.
Since we know there can only be
one selection entity, we'll
break out of the loop.
And that's it.
We're handling ownership
correctly and adding or removing
a local entity for selection.
Our game now works correctly
across two devices.
And while we've demonstrated
using two devices for
simplicity, we can actually
support more devices without any
additional code.
Let's look back at what we've
learned.
Today we've shown you how
RealityKit makes building AR
Apps easy and fast.
We've covered how to place
content with anchoring, load
assets both synchronous and
asynchronously, integrate
interaction in hit testing,
create custom components in
entities, and how to add
multiplayer to augmented reality
experiences.
I hope that we've given you a
better understanding of what
RealityKit is and what it
provides.
We can't wait to see what you do
with it.
For more information and access
to this session video check out
this session webpage.
Also check out Introducing
RealityKit and Reality Composer
for more information on
RealityKit and don't miss
Introducing ARkit 3 to learn
more about collaborative session
and other new features.
And be sure to catch us in the
labs today at noon and tomorrow
at 3:00 PM.
Thanks everyone and please enjoy
the rest of the conference.
[ Applause ]