WWDC2015 Session 606

Transcript

X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
[Applause]
>> Thank you.
So good afternoon, and welcome
to the enhancements
to SceneKit session.
My name is Thomas Goossens.
If you don't already know what
SceneKit is, here is an overview
of the different
frameworks on iOS 10.
SceneKit and SpriteKit
are the high-level APIs.
SpriteKit for 2D
and SceneKit for 3D.
You can use them with other game
technologies like Model I/O,
GameplayKit, and GameController,
and give you access
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
GameplayKit, and GameController,
and give you access
to the low-level APIs,
which are OpenGL and Metal.
SceneKit was introduced
on OS X on Mountain Lion
and last year on iOS in iOS 8.
Last year, we added a few
new features to SceneKit
to make it ready for casual
games, like particle systems,
physics, force fields, and the
integration with SpriteKit,
for example, for all
the 2D game overlays.
This year, the two
main additions
to SceneKit are the
transition to Metal
and the new Scene Editor.
In this presentation
we will start
by showing you the
new Scene Editor,
then explain why we are
transitioning to Metal
and what it means for
you in terms of adoption.
And we will finish by presenting
a few new features that we added
to the engine this year.
All right.
The new Scene Editor.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
The new Scene Editor.
It is available in Xcode 7,
so we can run it in El Capitan
or Yosemite, and it is
available in the force field.
To open it, simply open a 3D
file with SceneKit, for example,
DAE, OBJ, olympic, or you can
create using template chooser.
In that case, it will create
a new empty SceneKit document
with the extension .scn,
and an scn file is simply an
scn scene instance archived
with an archiver.
That means that you can produce
your own SceneKit document
programmatically if you
want, and this is very handy
for the production flow.
Then the main goal of
the Scene Editor is
to let you build the
levels of your game.
You will see that it's really
easy to combine multiple assets
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
You will see that it's really
easy to combine multiple assets
from different files
into a single scene
by keeping references
to the original assets.
The Scene Editor also includes
some visual editors for some
of the key features of
SceneKit, for example,
to let you configure particle
systems live in a 3D scene,
configure your physics, inspect
and configure the physics shapes
of your objects, and preview
a simulation for example,
sensing for force fields, and
it includes an action editor
to let you build
action sequences
with a graphic user interface.
Scene Editor also includes
some more advanced features,
for example, if you want
to editor your shared
modifiers live in your scene,
and some baking tools,
for example,
to bake ambient occlusion,
and for this please refer
to the Model IO session for
more information about this.
Okay. But nothing better
than a demo to show you this,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
so let me bring Amaury on stage
for presentation of Xcode 7.
Thank you.
[Applause]
>> AMAURY BALLIET: Hello.
My name is Amaury, and
I am a software engineer
with the SceneKit team.
We dramatically improved the
SceneKit editor in Xcode 7,
and with its new
editing capabilities,
it's now really easy to
build scenes from the ground
up in a visual and
interactive manner.
Later on in the session, I will
show you how we used SceneKit
in editor to build Red
Panda, the puzzle game,
but right now we will start
with something more simple,
a small game of physical
scale, which consists
of building a tower of blocks
that will eventually collapse
as we remove blocks from it.
So what you see here is a
SceneKit Scene Editor showing an
empty scene.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
On the right side, you have
the usually utility area,
with its inspectors
and libraries.
In the object library, you can
find many SceneKit objects,
such as modes, lights,
cameras, parametric geometries,
as well as physics fields.
Now, for our wooden block, we
use a simple box geometric.
To customize its appearance,
I can drop and imagine it.
Okay. So now just like
in Interface Builder,
inspectors as well can
edit all the properties
of your 3D objects.
For instance, in the attributes
inspector, I can change the size
of my geometry to
be 2 by 1 by 6.
And to center it in the scene,
I can use the node inspector
and set its position to zero.
But actually, there is a
more simple way to do that.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
But actually, there is a
more simple way to do that.
Using the 3D manipulator,
I can freely move the
object in the scene.
Okay. So now that our wooden
block is ready, we will use it
as a template to
build other blocks.
Using the 3D manipulator,
I can easily make copies
by holding the alt key.
I can repeat the
operation, make a selection,
duplicate that selection,
rotate it, make a new selection,
repeat the operation, and one
last time to finish the tower.
Okay. We are now ready to use
the physics engine that's built
into SceneKit to make the
scene more interesting.
I will simply select
all the objects,
head to the physics inspector,
and add a dynamic
body to each block.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and add a dynamic
body to each block.
We are now ready to
run the simulation.
As you can see, our blocks
are now subject to gravity
and fall indefinitely.
To prevent that, we will
add a floor to the scene,
so I open the objects library,
put a floor, position it,
and I need a static body
so that it participates
in the simulation but
is not allowed to move.
If we run the simulation
again, nothing happens.
But as we remove a few
blocks from the tower,
we get a fair idea of what
the game could look like.
[Applause]
One additional thing I would
like to show you
is a scene review.
You can review the scene review
in the lower left
corner of the editor.
The scene review is where
you can see all the nodes
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
The scene review is where
you can see all the nodes
in your scene, and it is where
you can edit the node hierarchy.
For instance, we have our
floor, the many wooden boxes,
but we also have a spotlight.
I placed this light
in the scene right
after I created the project.
It is currently hidden and has
no effect, so let's show it.
To further improve the lighting,
and to avoid these
completely dark areas,
I will simply add an
ambient light to the scene.
And that's it.
Just through the editor,
we built an entire scene
with 3D objects, textures,
lighting, and dynamic bodies
that could make a nice game
after we implement gestures.
So the SceneKit Scene Editor
really allows you to build more
of your game up by
writing less code.
And there's actually much
more things that you can do
with Editor, and to
cover that, let's switch
to the Red Panda demo, and
let's see the demo again.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
to the Red Panda demo, and
let's see the demo again.
So Red Panda is a puzzle
game in the 3D world,
where you can manipulate
the camera with pan gestures
and control the character
on his path.
The goal of this game is
to collect these flowers
and here is the first one.
On his path, Red Panda will
also have to collect items.
See how the world is rich and
animated with many details?
We have lots of particle
systems and animated vegetation.
Now, Red Panda can go
everywhere in the level.
For instance, he
can climb hills.
But on his path Red Panda will
also have to avoid obstacles.
Notice the sound became
louder as we pushed.
It's because we used one of
SceneKit new API's 3D audio.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
It's because we used one of
SceneKit new API's 3D audio.
Many of the features
are in play here.
We have the physics
engine for navigation
and collision detection.
Lots of particle systems.
And shadow modifiers and actions
help to bring the scene to life.
So we just collected
our seventh flower,
and by moving the camera, we
can see where objects hide.
We have a quick tour
under the arch,
and the level is completed.
[Music]
Thank you, Thomas.
[Applause]
So we are now going to see how
we built this game using the
SceneKit Scene Editor.
So this is a starting
point for our game.
A quick and easy way
to immediately improve your
scene is to add a skybox.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
to immediately improve your
scene is to add a skybox.
A skybox is simply a
collection of six images
that represent the environment.
After we add a skybox to
the scene, we have a context
in which we can build the level.
So if we now take a look at
the resources in our project,
we can see that our
artist provided us
with many building
blocks, and by that I mean
that we have separate files for
the bamboos, for the flowers
to collect, for the
pearls to collect,
as well as the many
structural elements.
So now how do you use these
external building blocks
in your scene?
Well, you go to the
scene, and if you drag
and drop an external
building block in the scene,
you can see that the
reference node has been created
as indicated by this
little arrow icon.
It is very nice and convenient
to have these external resources
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
It is very nice and convenient
to have these external resources
as external files because that
way you can design the level
and still have a chance
to save the assets,
and you can also
reuse these assets
to build other levels
for your game.
So as you can see, I've already
included other reference nodes
for other industrial elements.
So this is how you
typically build a level.
You would add your
building blocks one
after the other and iterate.
You would do the same thing
for the flowers to collect.
The pearls.
And finally, the vegetation.
So here you can see
how far we went
with just a simple operation.
So as you remember,
our scene is rich
and animated with many details.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
We have a lot of
particle systems,
but we also have
animated vegetation.
Many were animated with skimming
which is a fairly Complicated
animation technique that has
to be done in a special tool.
But for we use shadow modifiers.
Actually, there is one last
element to animate here
in the small waterfall.
So to do that, I will open
the scene for our waterfall,
and the SceneKit's
editor now allows you
to add modifiers right into this
scene for immediate feedback
and live compilation issues.
You can add and edit
each of the modifiers
in the assistant editor.
So here I select the
waterfall and choose
to add a geometry modifier.
I add the modifier, and you
can immediately see its effect.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
You can play with your
shadow modifier and try
to tweak the final rendering.
You can try to find what
the final modifiers are.
Once you are happy
with your scene,
you close the assistant editor
and go back to your main level.
As you can see, our
mountain is now animated.
This is because we
used reference nodes,
and when you edit assets,
they are automatically
reflected in the scene.
One last thing we
want to do here is
to add an enemy in the scene.
So this is a scene
for our enemy.
As you can see, it is static.
It has no behavior.
To add the behavior,
we will use actions.
The Scene Editor now
allows you to add
and edit actions
right in the scene.
So this is the action editor.
I will select my enemy.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
I will select my enemy.
Choose a row detection.
Add it. Set it to
rotate along the Y axis.
And make it repeat forever.
So this is a simple behavior
that we have for our enemy,
but the action editor allows you
to build some complex scenarios,
and for more information
about that, I invite you
to watch the What's New
in SpriteKit session.
We can now go back to our level,
add a reference to our enemy,
and our scene is finished.
So just -- we just saw
how the editor allows you
to build an entire level
from external building blocks
and how you can bring
life to the scene
with shadow modifiers
and actions.
And with that, I hand over
to Thomas, who will talk
about the architecture
of the game.
[Applause]
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
[Applause]
>> THOMAS GOOSSENS: So
yeah, we are very happy
to give you this nice
demo as a sample code.
It's available on the
developer website.
You will see it's very
simple and straightforward.
It's about 1,000 items of
code for both iOS and OS X,
it includes everything like
the sound, the controls,
everything you just saw.
So Amaury showed
you what was done
in Xcode 7 to build this demo.
I will quickly present
the rest, what was done
on the inside and the outside.
The first thing we did
was to define a direction
and a simple game
play for our game.
So we started with
our main character.
Then we defined the
shape of our level,
including where the
collectible items should be
and where the enemy should be.
And we defined the final
appearance we wanted
to achieve using SceneKit.
Then the next step was to scope
the 3D models in the 3D tool,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Then the next step was to scope
the 3D models in the 3D tool,
to paint the different
textures, configure,
and animate our character.
And the final polish was done
by adding some particles,
vegetation, and fog,
and we finished the game
by adding some 2D
overlays on top of that.
Then the first challenge
on the code side was
to manage the collisions.
For example, we don't want our
character to cross the walls.
To do this with collisions,
we work with collision meshes.
A collision mesh is a simplified
version of a rounded mesh
that has a similar shape so that
the collision feels consistent
with what you see on the screen.
It's very important to
work with collision meshes
and not the rendered mesh
when you deal with physics
and collisions for
performance reasons.
And as for the ground, we
want our character to be able
to progress everywhere
in the level
but always stick on the ground.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
but always stick on the ground.
So we need to know
what is the elevation
at any point in the 3D world.
For that, there are
several solutions.
What we did here was to again
work with the collision meshes.
And we do array intersection,
vertical array and meshes,
this I was go us an
intersection point,
and this is where we
can place our character.
Then whe we move the
character with the touchscreen
of the keyboard, we simply
move it on the X and Z plane.
Then we do a new intersection
where the character is.
This gives us a new intersection
point, and we use that point
to adjust the Y position
of our character.
We redo that at every frame,
and this is how simple it is
to animate our character
in the game.
Then animations.
There are several animated
elements in the game.
The character is animated
when it is idle or working.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
The character is animated
when it is idle or working.
The grass and the
bamboos are animated
to simulate wind effect.
And as a nice side effect, you
can see that the shadows casted
by the bamboos are also
automatically animated.
And all these animations
are important
because they contribute
making your game more live.
And technically, the character
and the bamboos are
animated using skinning.
So the artist already built
the skeleton and the animations
in the 3D tool and
exported that to a DAE file,
so they are absolutely no good
to do here for the developer.
And the grass was animated
with a shadow modifier
directly in Xcode 7.
Related to animations,
particle effects.
So there are several particle
effects in the game, and again,
they were all done entirely
in Xcode 7, no code here.
The pearl and the flower
with some sparkles emitted.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
The pearl and the flower
with some sparkles emitted.
And the enemy throws some fire.
That was done with a --
modulated with a color gradient.
The particle was done with
four images of dots more
or less blurred with random
sizes and an initial velocity
to simulate the wind effect.
Now, one note about
our materials
and how we achieve this
great-looking materials
in the game.
So here is our looks
like in the game,
but if you just take its
geometry and render it
with flat colors, it
would look like this,
which is a little
bit too simplistic.
So the first thing we did was
to set a diffused texture,
and this gives us a base
color of our objects.
Then we added a normal map
that added very fine details
to our surface without
adding many polygons.
So this is quite cheap.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So this is quite cheap.
Then we added a light map using
the self-illumination property,
which is new in this release.
And the light map
adds very fine details
about global illumination
to our objects.
You can see in the
corner of the stone,
you can see some
occlusion, for example,
that is baked in the light map.
Then the last thing we did
was a reflective cube map
to reflect an environment
and, in particular, the sky.
So the reflective cube
map looks like this.
We attenuate its effect
using an attenuateer
to attenuate the
reflection on the surfaces
that are facing the
point of view.
Now, one note about
the light map.
It only baked the
global illumination
and not the main direct light
and not the main shadows
because we want the
main lighting
to continue to be fully dynamic.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
to continue to be fully dynamic.
That way if we want, we can
animate the position of the sun
if needed, but most of all,
a character can progress
in the game, and its shadows
are live and automatically,
you have nothing to do here.
So that's about it
for this demo,
and I hand over to
Sebastien to talk
about the transition to Metal.
Thank you.
[Applause]
>> SEBASTIEN METROT:
Thank you, Thomas.
So I am Sebastien, and I
am working on SceneKit,
and I am going to talk to you
about the transition to Metal
as well as a bunch
of new features
that we have added this year.
So SceneKit is transitioning
to Metal.
Metal, as you know,
is very low-level
and low-overhead graphic API
that brings better performances
and in much more modern
API to the platforms,
and most importantly for you
SceneKit users, it just works.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
It also brings some new
features to SceneKit,
like compute shaders so
you can just use a Metal --
or the geometry source or Metal
texture as material property.
It also allows us to
do automatic batching.
So if you have the geometry used
by multiple objects
using the same material,
we can dramatically reduce
the number of [Inaudible].
I will show you that
with a demo.
So what you see is a simple
SceneKit application running
on the OpenGL ES editor.
It is displaying a planet
with orbiting asteroids,
and on the OpenGL
is eating up much
of the CPU just displaying
the asteroids.
As you can see, we cannot
add many more asteroids
without maxing CPU and
reducing the frame rate.
But if we move to the
same application running
on the Metal renderer, we
can add many more asteroids,
as you will see in a moment.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
as you will see in a moment.
Yes. Of course, the CPU
starts to be eaten too,
but we can raise the number
of asteroids dramatically.
I think it's quite awesome.
[Applause]
Thank you.
So how do you adopt Metal with
your SceneKit applications?
Well, first is the default
on iOS 9, so you just have
to recompile your
application with the new SDK,
and it will use Metal on all
the hardware that supports it.
It's also backward compatible,
so if you have an
existing application
that you didn't change
and didn't recompile,
it will work on OpenGL
by default.
And you can opt for using the
OpenGL renderer, if you want to,
for example, if you
have custom shaders
that are OpenGL specific.
Choosing the renderer
is really easy,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
Choosing the renderer
is really easy,
if you have a built-in
application
with built-in renderer, you just
have to use this small button
to change the default.
You can also do it
programmatically.
So if you created your
scene view at runtime,
you will give it the
right option, use Metal
or OpenGL, as you want.
And what's new also is
that we have added support
for Metal shaders.
So how do you deal with shaders
in SceneKit with the new API?
The first way to use custom
shaders with SceneKit is
to use Metal shader modifiers.
Shadow modifyers used
to be only in GSL.
Now we support both languages.
If you add on GLSL modifier, it
will work as it is on OpenGL,
and it will be translated
to Metal.
It works in most situations.
If you write your
shadow modifier
with Metal shadow
language, it will not work,
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
with Metal shadow
language, it will not work,
it will be ignored, but
it will work as Metal.
The second way to use shaders in
SceneKit is to use SCNPrograms.
In this case, there is
no automatic translation,
so you will do it by hand.
Will you have to
translate all your code.
It's supported -- it
supports both GLSL
and Metal shading languages,
so you can support both
in the same application.
You can compile them offline.
You just have to provide the
function names for the shader
for the fragment
and vertex shader,
or you can compile
it at runtime.
In this case, you just have to
give the source code in addition
to the function names.
I will show you now
how you do it
by creating a very small shader
that would work in SceneKit.
The first operation that
you will have to do is
to include our SceneKit header
so that you can have access
to some semantics and, for
example, the default metrics.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
to some semantics and, for
example, the default metrics.
Here we define our custom
vertex structure that is
in the correct data
in SceneKit thanks
to the semantic definition.
We can also define the use of
a metrics that is, for example,
the model view projection.
And we can define our own
structure with our own data
for the algorithm we
are trying to create.
Then we just have to
create the vertex function
by giving the arguments for the
structure that we have defined.
Really easy.
Then in the program, we just
define the same structures,
we have to use the same
types and the same layout
for the structure as we used
in the Metal definition.
We created a program,
give the vertex
and fragment function names.
And then we can give some
data by using the structure
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
And then we can give some
data by using the structure
and wrapping it around
time stamps.
Very simple.
The last way to use shaders in
SceneKit is to use techniques,
techniques were introduced
last year.
It's a great way to
do post-processing
for all your applications on the
role scene, and in this case,
you just have to give the
correct function names
for the Metal shaders in your
pass definition, and that's it.
It will work just like
the OpenGL version.
That was it for Metal.
Let's move on to
some new features
that we have added
for you this year.
The first one is the
integration with Model I/O,
so we have added the support for
Model I/O, both for importing
and exporting of your
models and scenes.
As you can see, not all the file
formats support all the data
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
As you can see, not all the file
formats support all the data
that we are supporting in
SceneKit, so choose the one
that is most suited
for your application.
We have also added scene
transition with one line
of code, very easy, you
can create transitions
between scenes, and
with great animations.
Really easy to use.
We have added also
debug options,
so you can visualize your
boxes, your physics shapes,
everything at one time
for debugging purposes.
We have also added
blend modes to material
so you can choose how
transparency works
for each node -- for each
material in your application.
You can create new
effects with that.
And we have also added an
audio engine that works
for everything in SceneKit.
We can create 3D sounds,
attach them to nodes,
and they will move with
the nodes in the scene.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
and they will move with
the nodes in the scene.
It also works with
ambience and music,
so you can digital position
audio and loop the sound.
The listener is placed on the
camera node, but you can move it
to any node in the scene.
It's really easy.
It's based on the AVAudioEngine,
so you just have to annotate
from AV audio node, and you can
do your own audio processing
if you want to if you have
an existing audio engine.
And last, we have added also
a way to created sound action
so you can integrate sound
into your action sequences.
That's really easy to do.
And here is how you
use the sound API.
First you have to instantiate an
audio sound from the file fame.
Then you create a player and
attach the player to any node
in the scene, and it will start
to play right away and stop
when the sound is finished.
If you want to do
ambience or music,
you have to just flip a switch,
like digital positional
audio and start looping.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
like digital positional
audio and start looping.
And last, it's really easy to
create a play song selection.
It's just one line, and you
can run the action or add it
to an existing action sequence.
This API was used in the
sample code for all the sound,
and we used the collision
mesh as --
that Thomas showed you
sooner to detect kind
of ground the character
is working on.
For example [scratching sound]
when he walks on the grass.
Or when he walks on a rock.
It was really easy to do.
So that's about it.
And I give you back to Thomas
for the wrap-up of this session.
Thank you.
[Applause]
>> THOMAS GOOSSENS: Okay.
So that was an overview of what
we added to SceneKit this year.
So a great new Scene Editor
that will save you tons of lines
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
So a great new Scene Editor
that will save you tons of lines
of code and do more for
your game insight tools,
which is usually a good idea.
A smooth transition to Metal
which should boost the
performance of your game
with nothing to do on your
side most of the time.
And some new features that were
presented, and we covered some
of them in this session,
but check the documentation
for a list.
And an amazing demo app
that we are really proud
to share the code with you.
Search for fox on the developer
website, and you should find it.
Now for more information,
please check the resources
and documentation
we have online,
and for general inquiries,
contact our Evangelist,
Allan Schaffer.
Some related sessions,
Model I/O,
great new frameworks
introduced this year,
and check the What's New
in SpriteKit session.
They have a really great
demo of the scene editors
that go much deeper
into details.
X-TIMESTAMP-MAP=MPEGTS:181083,LOCAL:00:00:00.000
that go much deeper
into details.
So you definitely
want to see that one.
And that's about it.
Thanks, and enjoy
the conference.
Thank you.
[Applause]