Transcript
[ Music ]
[ Applause ]
>> Good morning.
Good morning and welcome to
Advances in SceneKit Rendering.
My name is Amaury and
I'm delighted to be here
to present you how we brought
SceneKit to the next level
with state of the art graphics.
So we have a lot to cover today.
So I will start with a quick
intro on SceneKit before we dive
into this new rendering
advances.
Next Jean-Baptiste and
Sebastien will join me on stage
to present a cool demo,
explain how we built it,
and present all the new features
such as great new
camera effects.
such as great new
camera effects.
And finally, Nick will join us
to present a base to Model I/O.
So in a nutshell.
As you know, SceneKit
is a high level API
under the GameKit umbrella
and it focuses on 3D graphics.
It plays nicely with
[inaudible] and it's built
on top of Metal [inaudible].
And use SceneKit in any
situation where you need
to disperse with the
graphics on screen.
And when you start to think
about it they are used
in a great deal of places.
For instance, we just
introduced Swift Playgrounds
where SceneKit makes scenes
more visual and helps kids
in the first steps in
learning how to program.
In Xcode we use SceneKit
to create an innovative
and extremely useful interface
that helps you develop
your apps view hierarchy.
In iBooks and iBooks Author
people can create rich books
In iBooks and iBooks Author
people can create rich books
with enhanced illustrations
which are interactive.
And of course SceneKit
can be used for games.
Last, but not least, thank you.
You guys found of so use cases
for SceneKit and 3D graphics.
You published thousands
SceneKit-based applications
to the Store.
So thank you.
[ Applause ]
Now, as you know, SceneKit
is tightly integrated
with the system.
It works seamlessly with
all the Apple technologies
and it takes the
most of macOS and iOS
where it's been available
for a few years now.
And sense we last talked at
WWDC we also introduced SceneKit
on tvOS.
All we had to do for the
[inaudible] sample code was
to add two ports for game
controllers and it was ready
to be played on the big screen.
So it's absolutely fantastic
to see how the same game
So it's absolutely fantastic
to see how the same game
and code can run on
macOS, iOS, and tvOS.
And this year we
are closing the loop
with SceneKit coming to watchOS.
[ Applause ]
Thank you.
[ Applause ]
So SceneKit on watchOS is a
great opportunity to start
to think about new
interactions and a way
to present content
on your wrists.
Now, as you might imagine,
there's a lot to say
about the [inaudible]
for the Apple Watch.
And we won't have time
to cover this today.
But we have a [inaudible]
sessions, "Game Technologies
for Apple Watch," on Friday,
where you will learn more
about what's available, how to
play with SceneKit, SpritKit
and other technologies.
And if you are new to SceneKit
and want to learn more,
you can always go online to
check previous WWDC sessions
where we explained basic,
but also really advanced,
features of SceneKit.
Okay. So now let's dive into
this new rendering capabilities.
Well, this year SceneKit puts
physically [inaudible] rendering
in the hands of everyone.
It means that developers,
you guys,
get to have stunning graphics
for the arts and games.
It's the biggest leap forward in
SceneKits rendering capabilities
since its introduction.
We rely on the latest
advances in 3D graphics
and leverage modern technologies
to provide accurate rendering
and physically based shading.
Now, physically based
shading has several
[inaudible] requirements.
So we start with
linear rendering.
So what you see here is a smooth
radiance that goes from zero
to one and, as you can see,
everything looks perfect.
That is until you comprise
it so that it can be stored
in an eight disk optic
set image, for instance.
And, as you can see,
bending occurs.
And, as you can see,
bending occurs.
And that's new because our eye
is more sensible to variations
in the dark effluence.
Now by applying gamma encoding
we can assign more values
to these darker turns.
So, for instance,
here's an illustration
which shows the difference
between storing pixel
data used ordinarily
or using gamma encoding.
Now the thing is that when
shading is [inaudible] all the
lighting information
and equations are
expressed in a linear space.
So in a non-linear pipeline
what you end up with is color
that [inaudible] that is rich
with gamma encoded
[inaudible] texture
and then it's processed
using many [inaudible]
and the resulting
enrichment is written
to a texture or framebuffer.
And, as you might imagine,
that's not correct.
For the final image to be
correct all the operations need
to happen in enough space.
So, as an illustration,
here is a scene
So, as an illustration,
here is a scene
with lighting occurring
in gamma space.
And here is the same scene
with shading in linear space.
And if you compare them, you
will see how light fall-offs
and edges appear harsher
in linear rendering.
Now linear rendering
is essential
for physically based rendering
but it actually applies to any
of the SceneKit lighting model
because it just makes
the [inaudible] white.
Now, as you know, color is a
big [inaudible] thing this year
at WWDC so in addition to
gamma corrections [inaudible]
management automatically.
So what does that mean?
It means that the color
profile that is assigned
to a texture will
now be [inaudible].
Any operation that happens
between the image is loaded
from disk to the moment
it's handed to the system
so that it can be
displayed on screen,
we will respect the
integrity of the color data.
So a SceneKit-based application
will pull this wonders
of the color accuracy
of a professional
[inaudible] application.
Now, as you know, [inaudible]
just stop raw data that happen
to be stored as colors.
And SceneKit knows that
and so it won't color
match such images.
Now to help you with that
there's a great new features
in Xcode 8 asset catalogs
and they are texture sets.
In a texture set one can specify
whether an image holds color
data or raw data and then Xcode
can automatically convert these
images to CPU and GPU
efficient texture formats.
But to learn more about
that we have a session right
after lunch, "Working
with Wide Color,"
where the Metal team gets
into [inaudible] details.
Now, in addition to textures,
color management also
applies to color objects.
So color components are no
longer assumed to be sRGB.
And so if you are creating
colors programmatically,
it's now really important that
you use the right initializer.
So here is an illustration
with two color objects,
one Display P3 and
the other sRGB,
that were created using
the same components.
When working with color
pickers pay attention
to the color space
that you choose.
Above the menu we let you choose
from [inaudible] color spaces,
including device
independent ones,
such as [inaudible]
and [inaudible].
And there's also a handy
option to display values
as [inaudible] rather
than integers
so that they can be
easily copy-pasted to code.
And speaking of which,
as you know,
shader modifiers are a
great feature in SceneKit
that will allow you to
customize our rendering.
Now, as I said, this year
shading appears in linear space.
Now, as I said, this year
shading appears in linear space.
So you must be sure
to convert your colors
to the linear extended sRGB
color space before these
components are used
for color [inaudible].
Now a few notes about
backward compatibility now.
Linear rendering and color
management are automatically
enabled whenever you link your
app against the new [inaudible].
There's no performance
cost in enabling them
but they will dramatically
change the look
of [inaudible] scenes.
So, for instance,
here's last year's demo
which did not use
linear rendering.
And here's what happens
if you just recombine it.
Now, of course, textures,
lighting,
and shadow modifiers
can be reworked
with linear rendering in mind.
But if you want to deploy your
application to older versions
of the system, or want to
update a linear rendering
in color management
for some reason,
we have found a way to do that.
You can [inaudible]
level by specifying a key
You can [inaudible]
level by specifying a key
in your apps Info.plist file.
And then there is
wide gamut content.
So, as you know, with
wide gamut color spaces
such as extended sRGB
[inaudible] exists
and they are really important
when working for
modern hardware.
The new iPad Pro and iMac
with Retina Display
have wide gamut displays
that SceneKit can
[inaudible] automatically.
All you have to do is to
bring your wide gamut content,
so textures or colors,
and SceneKit will enter
that transparently.
Now wide gamut textures
and framebuffer will require
more memory to hold that data
and that will lead to an
increased bandwidth usage.
So may you experience
any performance issue,
we offer a way to upload again
at the [inaudible] app level.
we offer a way to upload again
at the [inaudible] app level.
Now let me mention the color
gamut showcase sample code
that we built in collaboration
with the Cocoa and
Cocoa Touch team.
It's a synching-based
application that will allow you
to see the out of
gamut color components
and it's also really useful
because on the wide gamut
display you will be able
to see what this display brings
because you can simulate
a non-wide gamut display.
So to learn now about working
with wide colors and how
to convert color components
from between color spaces,
again we have a great
session this afternoon.
And so that works for accurate
rendering which is a requirement
of physically based rendering.
Now what is physically
base rendering and why?
Well, [inaudible] scenes
with detailed models.
And that is definitely true.
But shading is what
makes objects tangible.
But shading is what
makes objects tangible.
So all you see here
on the screen used
to be a soup of polygons.
And shading is a process
of finding the right color
for each detail on the screen.
So all the highlights,
shadows, and the sense of depth,
it comes from shading.
Shading is that magical
operation
that can bring a scene to life.
Now how does it work?
Well, first there is light
which is emitted from a source.
And when light hits an object it
interacts with matter according
to properties of the surface
and then light is reflected
to find your eye or a
camera in this case.
Now this interaction between
light and matter is something
that is really complex.
And over the years many
mathematical models were
developed to try to
best describe it.
Physically based rendering is an
approximation of light transport
that relies on such mathematical
models and they take
that relies on such mathematical
models and they take
into account the physical
properties of light and matter.
But, as you know, SceneKit is
a high-level API and we want
to allow anyone to benefit
from this new lighting model.
So we will expose a
super easy-to-use API
so that you can use
physically based rendering
that artists love.
So at the end of this session
you will be able to get
from this rendering,
which is standard,
to a physically based one.
Okay. So in SceneKit we
will export physically based
rendering from two angles.
First, physically based
materials and then,
physically based lights.
So first, physically
based materials.
Here is a description of
a point on the surface
with normally indicating its
orientation is for the space.
And when light hits that point,
it's split into two terms,
diffuse reflection and
specular reflection.
diffuse reflection and
specular reflection.
Now diffuse reflection
corresponds to light
that goes underneath the surface
and is scattered so many times
and in so many directions
that it appears uniform.
The color of the diffuse
reflection is albedo
or the base color of the object.
So when designing the interface
for physically based material
in SceneKit we will want
to use an albedo map.
Now specular reflection
does not follow that way.
Specular reflection
is just made of lights
that bounces off the
surface and so it's
of the color of the
incoming ray.
So here is what we
call a cube map.
It's a collection of six spaces
that represent the environment
around the location in 3D space.
And when we place a
perfectly specular object
in such an environment we will
see that acts like a mirror.
Now let's take a more realistic
example with a plastic ball.
As you can see, it's
not a perfect mirror.
At the center is the reflection
is dim but as you move closer
At the center is the reflection
is dim but as you move closer
to the edge it gets brighter.
And actually for raising
angles all light is reflected.
Now not all materials have
the same reflectivity amount.
What you see on the
top is a curve
which represents the
reflected values in function
of the incident angle
from zero to 90 degrees.
And you will see that these
reflectivity values stays almost
constant from zero to 45 degrees
and actually we can
use this value
to reconstruct the whole curve.
Now gold is an interesting
example
because it has different
reflectivity values of the red,
green, and blue components.
The one last thing to note here
is that metals, such as aluminum
and gold, have high reflectivity
values whereas non-metals
or dielectrics have low
reflectivity values.
And this difference
in reflectivity is
actually essential
in reflectivity is
actually essential
for the final look
of the object.
So in SceneKit we want
to expose a metalness map
which will indicate which
parts of the object is metallic
and which part is not.
So in addition to reflectivity
to different type
reflectivity values,
also note that metals will
absorb all light beneath the
surface where dielectric
will scatter with light.
So the visual effect of this is
that metals have a wide
specular reflection
and no diffuse reflection and
dielectric will have a lot
of diffusion and specular
reflection will almost be seen
only at raising angles.
So in SceneKit we will reuse
the diffuse Metal property
to store the reflectivity
values of metals
and the albedo of dielectrics.
And for the reflectivity values
of dieletrics we just use
a global low constant.
of dieletrics we just use
a global low constant.
So we just reuse the
diffuse Metal property
that we brought from
[inaudible].
Now one last aspect
I would like to talk
about is the surface roughness.
So, as you know, no
surface is perfectly smooth.
As a microscopic level
you always have tiny bumps
and cracks that will affect
the specular reflection.
So the rougher the
microsurface is,
the blurrier the reflection
will be because reflected rays
of light are no longer aligned.
So again in SceneKit we would
want to expose the roughness map
which will indicate which
parts of the surface is rough
and which part is smooth.
And this one is a
[inaudible] image.
So we just saw how we can divide
three fundamental properties
and each of them has a
clear meaning and is derived
and each of them has a
clear meaning and is derived
from [inaudible]
properties of the surface.
Now creating a physically
based material
in SceneKit is straightforward.
You first create a material,
then set its lighting model
to the new physically
based lighting model,
and finally you provide
your maps.
So let's take an example.
We start with a mine cart
and only a diffuse map.
We will then add
a roughness map.
So, for instance,
take a look at coal.
Coal is rough so there
is no [inaudible].
And finally we will
add a metalness map.
So, for instance, take a
look at rails and wheels.
Let's take another example.
We have a fire truck.
Again, we start with
a diffuse map.
Now we will add a metalness map.
And finally a roughness map.
So for instance,
take a look at tires.
Now one thing I would
like to mention.
Now one thing I would
like to mention.
For the metalness, roughness,
and ambient occlusion maps,
please use grayscale images.
Having different channels
for the red, green,
and blue would just
be a waste of memory.
And even more if you
add another function in.
Now, furthermore, if you
want to use the same value
over the whole surface, you
can use the color object,
or even better, for these
metal properties we know
[inaudible] numbers.
So we just saw how we can
create a really simple
and high-level API to create
a wide variety of materials.
Here is the same object
and on one axis we changed
the roughness values
and on the other axis we
changed the roughness value.
Now remember how we said that
we would export physically
based rendering.
Let's now have a look at
physically based lights.
Well, in SceneKit lights can
be split into three categories.
I will start with image
based lighting, or IBL,
I will start with image
based lighting, or IBL,
then cover light probes,
and finally point lights.
So image based lighting.
As I said, you can use a cube
map to describe the environment
around a location in 3D space.
So when shading a point
on the surface we can
consider the finish here
above the end point
according to its normal
and the right lighting
information on the color
that is [inaudible]
in this cube map.
So for instance,
here is an object
which is lit only using
image based lighting.
There is no light in that scene.
And you can see how changing
to cube map dramatically
affects the look of the object.
Using image based
lighting all the objects
in your scene will
have a coherent look
and will work nicely together.
By using image based lighting
in SceneKit is really
straightforward.
We added a lighting environment
property of the scene.
And you can simply set a
cube map to its contents.
And you can simply set a
cube map to its contents.
And what's great is
that it works perfectly
with the background property.
So for instance, if you take an
object and set the same image
to the background and
lighting environ properties,
you will be able to display
an object in its context.
Now cube map, it can show
the distant environment
and the aesthetic.
So when shading a point on
the surface it's possible
that this environment is not
visible because you're in a cave
or there's another
object between them.
And that can be taken
into account
with image based lighting.
So it does not work very
well for occluded objects.
Luckily we have a solution
for that: Light probes.
Light probes are local lights
that are faced towards the scene
and they capture the local
diffused contribution.
So when shading a point
on a surface we can find the
four closest light probes
and interpolate lighting
from these probes.
So as I said, light probes,
they are local lights
and so they can account
for occlusion.
And they are implemented
in such a way
that they are really
lightweight and efficient.
You can have dozens of
light probes in the scene.
And we actually recommend that.
Because the more
probes you have,
the finer the [inaudible]
will be
and the better local lighting
information you will have.
So creating a light
probe is easy.
You create the light and
then change its type.
That can be done
either programmatically
or within the Xcode
SceneKit scene editor.
Now just like cube maps,
light probes capture
static lighting information.
And this information
will be baked
into the probe easily using the
Xcode scene editor of this API.
So we just saw how using IBL
or light probes you can have
indirect lighting in the scene.
But of course if you want direct
lighting, you still have access
to all the other kind of lights.
So omnidirectional, directional,
and spot lights work
with physically based rendering.
And actually we have [inaudible]
so that you can be
a better configure.
For instance, we added
the light's intensity.
A light's intensity is expressed
in lumens which a default
of watt 1000 which
is in the order
of magnitude of a light bulb.
We also added a light's
temperature which is expressed
in Kelvin and from which
we can divide for color.
And one great new feature,
we added a new kind
of lights, IES lights.
So IES lights, or
photometric lights,
can account for any
attenuation shape.
So while the spot light
or omnidirectional light has a
really symmetrical attenuation
curve, IES lights can better
accumulate the behavior
of a theater world light.
And, for instance, it can
account for [inaudible].
It can account for shadows.
For example, due to
the frame of the light.
Now creating photometric lights
in SceneKit is really easy.
Again, you create the light.
Then you change its type.
And finally you provide the
URL to put them into profile
which can, for instance,
be downloaded
from the website
of a manufacturer.
So as a quick recap, we
just saw how simple it is
to create a physically
based material in SceneKit
and all these properties derive
from here where properties
on the surface so they are
really easy to understand
and how we can work with
lights in the context
of physically based material.
So with that please
welcome Jean-Baptiste
So with that please
welcome Jean-Baptiste
and Sebastien for great demos.
[ Applause ]
>> So thank you, Amaury,
for this great presentation
of the new rendering
capabilities of SceneKit.
So let's see them in
action, the [inaudible].
So, as you will see,
almost everything
that has been presented
by Amaury is actually
very [inaudible] available
in the [inaudible].
You will be able
to tweak properties
and see the result in real time.
So I have a very simple
scene ordered here
with just one light
on this truck.
I go to the Materials inspector.
As you can see we have just
two materials of the subject.
One for the body and one for
the accessories, et cetera.
So I'm going to select
those two.
We are continue using the Blinn
lighting model so we'll switch
to the physically
based lighting model.
to the physically
based lighting model.
Now I've set the two
materials as metallic.
And, as you can see,
there is an issue
because we don't see the
reflection of the environment.
So we can go to the Scene
inspector and we have
to set the lighting
environment for project.
So for that I will use the cube
map, for example, this cube map
of a parking as the
lighting environment.
So. Shortly I'm focusing on
those three main properties
of the physically
based lighting model.
So let's now move to
the roughness value.
The roughness is indicate
how smooth the surface is.
So you will see that the
rougher the surface is,
So you will see that the
rougher the surface is,
the blurrier the
surface will be.
So if I move the value of
the roughness closer to one,
I have a blurry reflection.
And then almost no reflection
at all when we reach one.
So if I move back to zero, I
have a very smooth surface and,
as you can see, everything is,
the whole environment is
reflected in the metal.
So now I'm using just
one constant value
for the roughness.
And I would like to be able
to use, to specify a value
for each part of the object.
For that I just have
to use a roughness map.
So let's use a roughness
map for the body.
And a roughness map
for the accessories.
And a roughness map
for the accessories.
So we have the same kind of
issue with the metalness.
So we want to be able
to specify which part
of the object is
metallic or not.
So for that we [inaudible].
So let's set the metalness
map for the accessories.
A different map.
So, as you can see,
the body parts
of the object is nonmetallic
while the front radiator grill
is completely metallic.
The final touch is
to add the albedo.
And we will be done.
So that's it.
So we have a full [inaudible]
rendering of this fire truck.
I can now switch
to the [inaudible]
I can now switch
to the [inaudible]
and change the cube map.
For example, this cube map
of the lighting environment
with trees.
I can set it in the background.
So that's it.
So, as you've seen,
it's very simple
to use the new SceneKit
scene detail.
And, you know, now to
demonstrate this kind
of rendering in action we've
built a cool demo that I'm going
to show you now while
Sebastien is presenting it.
>> Hello.
[ Applause ]
Thank you.
[ Applause ]
So I'm delighted to present
you our new furry friends
for this year.
Bub. Bub is a badger.
He rides in the mining cart.
And he tries to catch gems
and boosters for speed.
So everything you
see is rendered
with the new SceneKit's
renderer.
All the materials
are physically based.
All the materials
are physically based.
All the lights, too.
We also used the usual
properties of SceneKits
such as action, animations, and
everything you used to have.
It's a Swift application that
runs on macOS, iOS, and tvOS.
It's fully built with Swift,
about 700 lines of code.
We placed light probes
along the track to take
into account the
change of local light.
And pay attention to the
light that changes when we go
in the caves or in the tunnels.
We have also added new
effects such as motion blur
which you can see when Bub
catches a speed bonus just
like this.
We have a new HGI camera
which is why the light changes
and when there is a bright light
or when the environment changes.
and when there is a bright light
or when the environment changes.
We also use IDL's for
the light environment.
Again some new, some,
we love the motion blur.
You can also see bloom when
there are bright lights.
And all the materials, as
you see, are completely PBR
so we have free reflections
for the crystals,
and for all the bonuses,
and the gems.
Once again, you will
see the light change.
It's tone mapping
doing the work.
Thank you.
[ Applause ]
So, let's go to the slides now.
I will tell you a bit
more about this demo.
And the first thing
that we're very glad
to tell you this year is that as
usual the demo is a simple code.
Yes. Thank you.
[ Applause ]
You can download the code on
all the assets from the website,
from the developer
website, and play with it,
inspect the scene code,
see how we build it.
And it's 700 lines
of Swift code.
We think it's pretty
simple to understand
and we hope you really like
what you see and learn a couple
of things from the demo.
So this year we had to
decide if, for the demo,
we discussed it with
our artists.
And we produced some
drafts to take
into account the
design idea we had.
And once we agreed
on the design ideas
and the workflow the artists
started to model the world.
And as it's an interactive
process we really needed tools
to be able to ingest the
models as they were built
and to start programming
right away,
without waiting for
the final assets.
So we have a custom
tool written in SceneKit
that uses the full
power of SceneKit
in a common line application.
To involve the tools from the
DAE files they convert the units
to meters and they also place
light probes automatically
in the scene because there
are more than 200 light probes
and we don't want, we
didn't want it to placed
in by hand each time
the scene changes.
We have used image
based lighting.
So we have a cube map
for the background image,
another cube map for the
lighting environment.
We used the lighting environment
to add the reflections.
And it's, as you've seen,
great for outdoor scenes.
We have also used light probes.
You can see these light probes
as they were displayed in Xcode
and we've highlighted them.
You see that only
from this point
of view there are
already many light probes
so you can imagine how many
there are for the whole scene.
So the custom tools placed them
in the environment
and compute them.
You can also do it by hand
in Xcode but, of course,
the more light probes you
have, the more tedious it gets.
It's essential for the inside
but it's also adds a nice touch
to the view in the outside
to detect small changes
in the scene.
We have added light
maps for the inside
because it overrides
the environment,
the lighting environment, which
is very important for the caves
as the light is very
different in the caves
so we have the probes and the
light maps that change the light
and the mood of this
part of the scene.
Of course we use
normal maps as usual
to add details to the models.
We also use baked
ambient occlusion maps
for a very much better
lighting view and rendering.
We have one big point
light to simulate the sun.
It's very high in the sky
in the scene and we use it
to create dynamic shadows and
to improve global lighting.
As I said, all the
materials you see
in the demo are 100%
physically based materials.
So we get the nice water palms
reflecting the environment
as well as the crystals.
Talking about crystals,
this is very simple material
that we built.
It has no texture map so
it's very simple to create.
It's fully metallic and
has no roughness at all.
And just a diffuse color.
So it's a very nice
way to create a gem
that reflects the
environment almost for free.
On the other side of the
spectrum you can see this tower
which is one object
with metallic parts
and nonmetallic parts.
We used, of course,
metalness and roughness maps,
texture maps to create that.
And, as you see, we
still have diffuse color
on the normal map to add detail.
So basically the demo used
all the new capabilities
of SceneKit.
Physically based shading,
all the SceneKit API
for materials, lights.
We used Xcode integration
and also new custom tools
we built for the work flow.
and also new custom tools
we built for the work flow.
And we think it's
a great showcase
for this year's capabilities
and a great sample code
for you to learn new things.
And we hope you will
really like it.
Thank you.
[ Applause ]
So, as you've seen, we've
quite upgraded what's happening
with materials and
light this year.
But we also had to change how
the camera behaves because now
that we have great
materials and light,
we also needed a
much better camera.
And now that we have light
that are realistic we needed
to have an HDR camera
or High Dynamic Range
because the usual camera used
to have Low Dynamic Range
which is 8-bits per components.
Now we have float per components
so we can have very small,
very unbright light such as a
candle or a light bulb going to,
for example, the sun which
is very, very bright light.
So this creates a very high
dynamic range that we need
So this creates a very high
dynamic range that we need
to remap to the dynamic
range of the screen.
And for that we used
tone mapping.
Tone mapping is the action of
remapping part of the rendering
to a smaller capability device.
So we need to enable
the HDR camera.
It's not automatically
set by default.
You can set that in
the API or in Xcode.
And you can configure
the tone mapping.
You can change the gray
point, the white point,
and the range you
want to expose.
And you can also frost
the exposure offset.
So, for example, you can have
this nice look of the scene,
but you can create a low key
one with underexposed rendering
or overexpose it, well, but
just by changing the offset.
It's very simple.
We have added also very
nice effects thanks
to the new HDR camera.
The first one is bloom.
Bloom is a way to simulate being
blinded by very bright lights
from the scene and reflections.
And it will be created
by bleeding the ejection
And it will be created
by bleeding the ejection
on the light on the
surrounding pixels.
So you can see in this example
it's a very nice effects
and we can see how it looks
in action with a reflection
on the roof of the tower.
I think it's a very nice way
to see how the light bleeds
on the surrounding pixels.
And it adds a very nice
touch to the rendering.
Next we have added motion blur
as you've seen in the demo.
So it smoothens the
camera movements.
And the thing is when
you just add motion blur
to the whole scene
this is what you get
so sometimes we don't
want to blur everything.
For example, we wanted
the badger and the cart
to be sharp and crisp.
So we have a new
API that enables us
to exclude some objects
from the motion blur
and the result gives you a nice,
crisp look for the subjects.
We have added a couple
of variations
from real life camera
lenses this year.
The first one is vignetting.
Vignetting is a way to, is an
aberration in real life lenses
that creates shades on
the corner of images.
So you can change it from
this image to this one.
And you can also
change environmentals
to change the filtering going
from the center of the image
to the border of the image.
Another aberration we have
simulated this year is
color fringe.
Color fringe is a defraction
of lights that happens
in real lenses, in the
glass of real lenses.
So it creates a magenta
and sienna shadow
of the lights in the rendering.
And we go from this
look to this look.
This is a very exaggerated one.
You can go more subtle
to get a nice look.
We have also added a very
nice way to change the mood
of your scene with
color correction.
So you can change
the saturation,
go for an almost
black and white look
or overblow the colors
if you want to.
And you can also change
the contrast of the scene.
So you can have the normal
look or a desaturated one,
or oversaturated image,
and change the contrast.
And the last one we have, it's
a really very great effect.
It's color grading.
Color grading enables us to
completely remap the colors
of the scene to completely
different colors.
So we use a strip
of a square image
to create the 3D color cube
that we use as a lookup table
to remap the original
colors in new ones.
For example, in this case would
remap normal color that you see
on the upper side
to a sepia tone.
So we get this looking like
that, like on, in sepia.
And it's very simple to use
and we think it's great.
It's a very nice look.
So that's all we have
for HDR camera this year.
We think it's a very
nice upgrade for cameras.
We can't wait to see
what you do with that.
We can't wait to see
what you do with that.
We got brand new effects that
are cumulative so you can use,
you don't have to
choose in between,
for example, bloom
or motion blur.
You can use all at
the same time.
Of course it has a cost but you
can really create a very nice
image and very cool
looking scenes.
So now I hand over
to Nick to tell you
about Model I/O improvements
for this year.
Thank you very much.
[ Applause ]
>> All right.
Hi, everybody.
So I'd just like to start
out by covering a little
bit that's improved on input
and output of models
and SceneKit.
So this year SceneKit
can import models
in their native authored
format, i.e.,
not necessarily just triangles
as before, but in the topology
of quadrilaterals or
arbitrary polygons
that the authors originally
created their content in.
SceneKit, if necessary, will
automatically triangulate
SceneKit, if necessary, will
automatically triangulate
for you in order to
perform rendering.
And the thing is if you want
to use our new tessellation
facilities you're going to want
to have accurate
tessellation for good shapes.
So you'll need to opt-in,
using the preserved
original topology flag.
That flag corresponds to
the same flag in Model I/O
and you bring in the
assets and you specify this.
It will preserve holes, and
creases, and all the things
that are important
for an accurate rendition
of the object.
Now this year we have improved
our subdivision algorithms
to the new system and
OpenSubdiv 3 from Pixar.
You can see in this example here
that previously we would
have imported as triangles
and when you do the tessellation
that box which we want
to smoothly subdivide comes
out a little bit lumpy.
Now if you bring it in
preserving its topology,
you can see that the quads go
to a uniformly round surface
and it looks very nice.
So this kind of facility
is great
So this kind of facility
is great
for having lightweight objects
that can scale the resolution
to your scene, and
so on and so forth.
Now the other aspect of
input and output that I want
to emphasize is that last year
we introduced physically based
materials and things
to Model I/O.
They bridge naturally onto
all of the SceneKit stuff.
So if you have a high
dynamic range camera specified
in Model I/O, it
will come across
and without losing
any attributes.
So on to Model I/O.
Yeah. Quick refresher.
As it says on the [inaudible],
it's for input and output
of models onto our
frameworks and systems.
Need this so obviously to
bring your data from your apps
where you've created
things, translate objects
between frameworks,
such as SceneKit
and MetalKit, and so on.
And we provide support for a
number of standard file formats.
Now file formats are the
method by which things come
Now file formats are the
method by which things come
from your art program
into your tools.
And historically the formats
that we had have been
quite narrowly specialized.
For example, they might
just bring in a model.
Or they just might
bring in bulk data.
Now really exciting thing that
I'm bringing to you this year,
we're bringing to you, is in
conjunction with our friends
at Pixar we're introducing
support
for Universal Scene Description.
Now Universal Scene Description
is a new open standard.
And the thing that's really
interesting and exciting
about it is it's not only
a file system and a format
that can be either easy to
read in ASCII or efficient
for loading in binary
but it also includes a
scene composition engine.
That really distinguishes it
from any other format
that's come before.
It embodies years of practical
production experience.
Pixar uses this for their films.
And "Finding Dory,"
coming out tomorrow,
is rendered entirely
from USD files.
Now USD has data types that
are specialized for scenes.
And it introduces, once
again unique to this format
as an open format,
is file layering
to enable concurrent workflows.
Now concurrent workflows is
kind of an awesome thing.
Here is a representation
that might get
in Universal Scene Description
for a typical scene in a film.
We have a shot layer, the
layer, the shot is layered
from components,
background, characters.
The characters themselves might
be made out of many components.
Now you can see there's
layers in that image
of a shot layer there.
That's because not only can you
just create the scene with all
of these things composed,
but you can make variations.
And so the scene description
will know that this is
like take three, maybe
the characters come
in a little bit faster
or a little bit slower.
in a little bit faster
or a little bit slower.
And you can have all of
those variations embodied
in one file and for review.
Now another really unique aspect
to Universal Scene Description
is, as far as I know,
it's the only open
source file format
that allows the specification
of classes
in variations of objects.
Now, you can imagine that
you might have some sort
of a situation where you
have lots of monsters,
and they all want to go
to university and stuff,
and there's like books.
Now in a traditional
workflow you're probably going
to find yourself creating
your books and your program,
slaving out millions and
millions of different files
for every little book, and then
placing them on your bookcase,
and getting it out for
rendering like that.
Now that is tedious.
In games you have things
like teams of characters,
maybe they all differ in
like hairstyle and shirt.
And you might have to
bake those all out.
Now Universal Scene Description
allows you to specify
in a single file
classes of objects.
in a single file
classes of objects.
So the class represented
here obviously is a book.
So the file can represent
many different geometrical
interpretations of books.
Like you obviously
got a wide one,
and a tall one, and a thick one.
And when you instantiate
your book
into the bookcase you can tell
Universal Scene Description,
"I want this book and I want it
to be this wide and that tall."
And it will provide the
information that you need
to instantiate that
into your runtime,
or your shot, or whatever.
The variations that you can have
in a single file can
vary along many axes.
In this case I'm changing
some shading properties.
So previously I had
all those books.
I can make them whatever
color I want as well.
And the magic of that is I place
the book and when I finally ask,
for the purposes of rendering,
"What color is the book
for the purposes of rendering,
"What color is the book
on the shelf in this place?"
it will work out,
according to all of the logic
about scene composition that
file and the engine embodies,
the way that it should
be represented.
Now beyond that you can
also represent a simple,
in a single file,
different capabilities.
So what I'm showing here is
that on the very low end,
like say for a wearable device,
I might have a low poly version.
The same file can have one
that's suitable for use
in the highest rendering
capability that you've got.
Now we've integrated Universal
Scene Description across all
of our systems and frameworks.
So at a very nuts
and bolts level.
If you import a Universal
Scene Description file
into Model I/O -- I don't expect
you to be able to read that --
you're going to be
able to get a hierarchy
of familiar Model I/O objects
with all the properties
that were in the Universal
Scene Description file exactly
represented so that you can
use our tools that are provided
represented so that you can
use our tools that are provided
in Model I/O, such as
placing light probes
and evaluating them
towards optimal positions.
However, beyond that, let's say
that you're working on a project
and your art team just gave
you a folder full of stuff.
You can just open that window
in the binder with all the stuff
that you just got and Finder
will prepare thumbnails for you
so you can see what's there.
And Quick Look works
with it as well.
So you can select one of these
things, whack the spacebar,
and it'll pop up and
you can tumble it.
Now, of course Quick Look
shows you one thing at a time.
If you want to hold
things up for comparison
or maybe your USD file has
multiple cameras or something
in it that you want to inspect
individually, you can bring
that up in preview and Universal
Scene Description is working
great there.
And if you're bringing
Universal Scene Description file
into Xcode, it imports via
Model I/O into SceneKit
into Xcode, it imports via
Model I/O into SceneKit
with an exact representation
of what was in that file
so that you can inspect it
in the hierarchy browser,
you can look at the properties,
you can move things
around, you can add cameras.
You make edit scenes,
send it back out to USD.
And then you can send it back
to your artists and say, "Hey,
you know, I've got
some edits for you.
Can you, do you know the rep?"
So finally, it's
incorporated into SceneKit.
And so friends at Pixar supplied
up with Mr. Ray from
"Finding Dory."
And this is just stock
out-of-the-box SceneKit
with the new physically
based shading
that you just heard all about.
And we're just playing the
movie asset with three seconds
of animation, and it
looks really, really nice.
So plugins are the thing that
you're going to need in order
to incorporate Universal Scene
Description into your workflows.
So that will enable the motion
of your assets between people,
your content creation programs,
the apps that you make.
your content creation programs,
the apps that you make.
Now the plugins, and the
open source information,
and all availability,
and schedules, et cetera,
are available on the openusd.org
website which I encourage you
to go visit to find out
how you can use this
in your pipelines and processes.
So that's Universal
Scene Description.
[ Applause ]
So a quick summary.
SceneKit is available
across our entire ecosystem
on every platform.
It's kind of an amazing thing.
We have physically based
rendering for any state
of the art looks and state
of the art representation,
just a beautiful look.
And HDR cameras and
effects give you control
over how things are represented
and how they look,
really high quality.
And we've got support for
Universal Scene Description
which we're really happy to
get behind and think it's going
to make a big difference
in workflows
to make a big difference
in workflows
in coming days and months.
More information on this
session which was 609,
is available on the site.
There's related sessions:
Visual Debugging with Xcode,
Wide Color, Game
Technologies and Apple Watch"
that you can attend
today and tomorrow.
And thank you very much.
[ Applause ]