Transcript
[ Applause ]
>> Morning.
Welcome to Advances in TVMLKit.
I'm Trevor.
I work in Localization, and
later, I'll be joined by my
colleagues, Parry and Jeremy,
from tvOS.
Today, we are really excited to
talk to you about three new
advances that we have made.
TVMLKit is Apple's developer
framework for building native
applications for Apple TV using
JavaScript and a markup language
that we call TVML.
I am going to get things started
by talking about right-to-left
languages.
Then Parry will show us how to
take advantage of new
optimizations in your templates,
to take performance to the next
level.
Finally, Jeremy will show us
some awesome new features that
you can use in the Web Inspector
to make debugging your
applications easier than ever.
So let's dive right in with
right-to-left language support.
Arabic and Hebrew are
newly-selectable languages in
tvOS 11.
With over 400 million speakers
worldwide, this is a huge
opportunity for you developers
to reach new audiences with your
app.
Now, what makes Arabic and
Hebrew different compared to
other languages you might be
supporting already is that they
are written and read from right
to left.
Let's take a look at some
examples, so we can really
understand what that means for
your applications.
What you see here is our product
template in a left-to-right
language, and it looks really
normal to most of you in the
audience.
The text is aligned on the left,
the content is flowing from the
left side of the screen to the
right side of the screen.
You see the focus is on the
first button, which is the
left-most button.
So, when you take this template,
and you load it in a
right-to-left language, for
example, Hebrew, you see the
content is mirrored.
It is now flowing from the right
side of the screen to the left
side of the screen.
Our text is aligned to the
right.
Our focus is still on the first
button, but now you see it is on
the other side.
Let's look at another example.
Here, we have the catalogue
template, and again, it looks
very normal and natural if you
are a reader of a left-to-right
language.
So when you look at this same
example in a right-to-left
language, you see that the
content is mirrored again.
It is flowing from right to
left.
The first image is still the
desert, but it is on the other
side of the screen.
So that is how these apps look
in right-to-left languages, and
as you can see from these
examples, we've built the
support into TVMLKit.
So when you're using the default
templates, you get the support
for free in your apps.
Now, supporting a right-to-left
language works the same way as
supporting any other languages
in your app.
You need to go into X code in
your project settings, and in
the localizations, just add
Arabic or Hebrew, which are
supported languages.
This is really key to make sure
that TVMLKit knows when to
render these views at
right-to-left at run time.
And for more coverage about
everything that it takes to
support localizations in your
apps, including localizing
content and dealing with
formatters for dates and times,
I recommend checking out this
year's talk, Localizing With X
Code 9, which goes into much
greater detail.
So the default templates work
out of the box, but what if you
have custom views, and you want
to customize the behavior beyond
those templates.
Well in tvOS 11 we have
introduced three specific areas
where you can customize layout,
text alignment and images.
Let's get started with layout.
Today, you might be customizing
your layout using tv-align, and
tv-position, and you are
probably using values of left
and right.
Well, for right-to-left
languages, left becomes left and
right becomes left, or something
like that, so you need to take
care to use the right direction.
And for that, we've introduced
leading and trailing.
Leading and trailing will
automatically resolve to the
corresponding values of left and
right at run time, based on the
running language of your
application.
So, for example, if you have a
style defined where you're
setting a tv-position to right
or tv-align to left, just
replace those values with
trailing and leading.
This is going to ensure that in
your left-to-right language,
everything looks exactly the
same, but in a right-to-left
language, content is flowing
correctly.
For values of margin and
padding, we've introduced a
brand-new media query in tvOS 11
called Layout Direction.
You might already be using media
queries today for customizing
your appearance for light and
dark.
These work just the same way.
Any styles that you define
inside the media query will be
active for that media query.
So, here in our example, we have
two styles defined, one in our
LTR layout direction, left to
right, and one in our RTL layout
direction.
And we take care to ensure that
any horizontal values are
flipped according to the side
that they need to be on.
So here, in our example, our
margin of 12 needs to be on the
correct side of the element.
So that's layout.
What about text direction?
So for alignment, you're used to
using things like left, center
and right.
And when you look at the text on
the screen, two kind of jump out
to you as a more natural
presentation for a left-to-right
language, left alignment is
natural, and for a right-to-left
language, right alignment is
natural.
So, for that, we got with our
marketing team and thought
really deeply about what kind of
term could we use here, and we
came up with a new value for
text alignment of natural.
Natural does just what you'd
expect it to do.
It aligns a text naturally based
on the UI language of your app.
Easy stuff.
When it comes to images in your
apps, the majority of the images
you have are universal.
They are appropriate to show for
any layout direction.
But in some cases, an image has
an implied directionality, like
this Chevron at the end of a
list item.
In this case, it's really
important that the arrow is
pointing in the correct
direction, otherwise it would
look really weird and probably
broken.
So there's two ways that you can
support these in your apps.
The first is for resource
images.
Images that are coming from
inside your application bundle
can be specified in asset
catalogs, which provide a really
convenient way for customizing
if the image is fixed for all
layout directions, can be
graphically mirrored at runtime,
like our chevron, or if we
needed a dedicated asset for
each language.
For images that are coming from
your server, we have a brand new
attribute that you can set on
your image elements, called
Source Set.
Source Set allows you to
identify a given URL for a given
layout direction.
What's great about Source Set is
that you can also use these for
customizing your appearances for
light and dark.
So here URL one will be loaded
when my layout direction is LTR,
and URL2 will be loaded when my
layout direction is RTL.
So this is a really concise way
for you to just define a single
image, and have the correct
thing happen at runtime.
So I'd like to see if we can
take all of these pieces and put
them together with a little
sample app that I've been
working on.
So I've been working on an app
that lets me showcase the WWDC
sessions from last year's
conference, and I'd love to show
it to you now.
So here you can see I have a
grid of the past sessions, and
at the top, I have a banner,
where I have a featured session.
So before I want to support a
right-to-left language in my
app, I can do a few things
without even adding a
localization.
In the edit scheme, in our run
options, I can change the
application language from the
system language to a
right-to-left pseudolanguage.
This lets me simulate what it's
going to be like in a
right-to-left language without
having to change anything about
my project.
So now, when I build and run,
you see our grid is now flowing
from right to left.
So because we're using a grid,
we get that support for free
from our template.
But because my banner was
custom, that I added with my own
styles, looks like I still have
some work to do.
So let's see if I can apply the
new API in tvOS 11 to fix these
problems?
The first thing that I'm going
to do is check my style in my
TVML and replace all of my
hard-coded references of left to
leading, which is the left side
of the screen in my development
language.
Similarly, I'll replace my
values of right with trailing.
Now, when I build and run my
app, hopefully we will have some
progress.
So great, looks like our text
has now flipped to the other
side, and our play button is on
the right side of the screen,
but it seems like it's a little
bit too close to the edge.
If I look at my style, my banner
play button lockup has a custom
margin that is applied to the
right side of the element.
We need to make sure that margin
is on the other side when we are
running in right to left.
To save time, I added a snippet
earlier that defines the two
media queries for my Banner Play
Button Lockup for LTR and RTL.
I'm going to move my definition
of the margin into the LTR, and
in my RTL definition, I'm going
to swap the horizontal position
of that 60.
Now, when I build and run, looks
like our play button has the
right padding.
So we are nearly there, but
looking at this app now, we can
kind of see that the text is on
the other side.
It is a little hard to read with
that image right underneath it.
So my designer gave me an
alternate asset that I can use
in my right-to-left languages,
so I can add that to my project,
since I'm using Asset Catalogs.
Here, in my Asset Catalogue, I
have a single image defined for
my banner, and then in the
attribute inspector, I can
change the direction from fixed
to both.
This now gives me a well where I
can drop the asset that should
be used in our right-to-left
language.
So I'll drop that in place and
build and run my app one last
time.
And there you have it.
With just a few small changes,
our app is now ready to support
right to left.
[ Applause ]
So as you saw, with just a few
small changes to our app,
because the default templates
give us the support for free, we
can have an app that fully
supports a right-to-left
language, with very few changes
needed from us.
We also have a really powerful
tool in the right-to-left
pseudolanguage, that allows us
to simulate the effects of a
right-to-left layout without
having to read a right-to-left
language.
Now, one last note, if you're
using custom views in your
application, we have a system of
constraints in auto layout
engine that is a really powerful
tool to take care of layout for
right to left languages.
We also have an API effective
layout direction, which will
return to you if you are in a
left-to-right or a right-to-left
layout direction at run time,
which is really useful for doing
things like frame math in an
animation.
For more in-depth coverage about
all the things, and nuances for
supporting right-to-left
languages, I recommend checking
out last year's talks,
Internationalization Best
Practices, and What's New In
International User Interfaces.
I can't wait to see your apps
supporting right-to-left
languages later this fall.
Now, here is Parry, with
template optimizations.
Thank you.
[ Applause ]
>> Thank you, Trevor.
Hi! I'm Parry.
I'm going to walk you through
some of the enhancements we made
to our templates that is going
to improve the performance of
your app, both in terms of time
and memory.
If you worked with TVMLKit apps
before, you might have noticed
that the performance of a
template degrades as you try to
add more content into it.
Let me illustrate with an
example.
So I'm working with Trevor on
our sample WWDC app, and I'm
trying to enhance it to add all
the sessions from the past,
instead of just the last year's
sessions.
We are keeping a simple design
with a grid to showcase all the
sessions.
Each session being represented
by a lockup composed of an image
and a title.
Now, you can imagine that there
are thousands of these sessions
to show.
So in a typical scenario like
this, you don't want to show all
your content up front, because
A, it is going to take a long
time to load, and B, it might
not even be possible, because
your servers might not support
that.
But instead, you paginate your
data.
You start off with something
small, let's say about 500
items, and then you keep adding
to it, as the users scroll
through the end of your content.
Now, we did a bit of performance
analysis for such a scenario,
and this is what it looks like.
What you see in this graph is
the time it took to build the
template with a certain number
of items, and what you see on
the x axis is the number of
items in the template, and what
you see on the y is the time it
took to reach there.
And as you can see, it's an
exponential curve.
Meaning that it takes longer to
add the same number of items to
the template as the size of the
template grows.
But why is that?
Well, one of the major factors
is the size of the document
object model back in these
templates.
The problem is two-fold.
On one hand, you have the
parsing overhead.
First, in your JavaScript, when
you parse your data into the
DOM, so when the DOM size
increases it takes longer to add
more stuff to it, and then when
TVMLKit has to parse a DOM in
order to calculate out
information, like cell sizes,
line spacing, even scrolling
offsets.
All the things that make your
TVMLKit app look beautiful.
But as you can imagine, as the
size of the DOM increases, it
takes longer to perform these
operations.
And on the other hand, you have
an increasing memory pressure
because of an increasing DOM
size, which slows down your app
as a whole.
So to solve these problems, in
tvOS 11, we've come up with a
brand new paradigm of defining
templates using prototype and
data binding which is going to
significantly reduce the size of
your DOM, hence improving your
app's performance, and on top of
that, we've added APIs to
support pagination.
Let's look at these in more
detail.
In order to understand what a
prototype is, I'm going to walk
you through a typical template
building process.
You start off with the data from
your servers, and an empty shell
of a template.
In our case, it's a grid.
And in your JavaScript, you go
through each one of these
objects, and you can word them
into their corresponding TVML
markup.
In our case, it's a lockup.
Once you've done that, you parse
that into a DOM, and that is
when TVMLKit starts producing
the user interface for it.
But if you look closely, you'll
see that all of these lockups,
they look very similar.
In fact, the only thing that is
different between them are the
values that are coming from the
data.
Once you remove these values,
what you end up with is an
identical looking lockup that
has all the information that
TVMLKit needs to compute that
layout up front.
So it turns out, you don't need
all of these lockups, you just
need one of them.
And that's what we call a
prototype.
So a prototype is a TVML schema
for your data objects that
TVMLKit can use to compute the
layout up front, and then at
runtime, it can combine it with
the data to author the DOM for
you, but only the parts that it
needs when it needs it.
So the DOM size remains concise,
and independent of your data set
size.
Now, you can imagine a template
with just a prototype would be
incomplete, because you also
need to provide data.
But on top of that, you also
need to provide a link between
your template and your data.
This is how TVMLKit can make
sense of what your data is, and
complete the template for you.
And you specify that using data
bindings.
There are three ways that you
can bind data to your templates
in tvOS 11.
First, you can bind a source and
attribute of an element.
In this example, the source
attribute of the image element,
which is bound to the URL
property of the data.
You can also bind text content
of an element, like in this
case, binding the text content
of the title element, with the
title property in the data.
And finally, you can bind items
of a section using the items
binding.
With this, you can bind the
trial DOM elements of the
section with an array of
objects.
So in a nutshell, data binding
provides the association between
a TVML property and a data
property.
And you can intuitively specify
it directly in TVML using a
binding attribute.
Now, when it comes to providing
data to TVMLKit, you can do that
in your JavaScript using a new
property you've introduced on a
DOM element called Data Item.
Here is an example.
You parse your JSON into
JavaScript Objects, then you
attach it to the section element
using the data item property.
Now, let's move on to
pagination.
In tvOS 11 we have introduced a
new event called Needs More that
you can use to conveniently
implement pagination.
This event gets invoked as the
users scroll toward the end of
your content, and you get it on
list, shelf, grid, and even
stack template, so it's very
versatile, and you can use it in
almost any template.
However, if you're using
pagination on a data bound
template, you need to create
observable objects for your
data.
This is how TVMLKit can observe
the changes that you make in the
data, and push those updates to
the user interface.
Let's take an example of how to
create these observable objects.
So you start off just the same.
You parse your JSON into
JavaScript objects, but then you
go through each one of these
objects and map them into a data
item class.
This is something that they've
also introduced in tvOS 11 that
has an observer pattern built
in.
You construct the data item
class with an optional type, in
case you want to have multiple
prototypes in your template, and
a required identifier that
TVMLKit uses to push those
updates to the user interface
efficiently.
And once you have all of these
objects mapped, you want to wrap
them up in another data item,
and that is what you attach to
the element.
Let's take an example of how to
handle Needs More.
Now, Needs More is just like any
other event.
So you listen for it by using
the Add Event Listener method on
a DOM element, and you provide a
function that would have kind of
a similar implementation to what
you would have when creating the
template.
So you fetch the data from your
servers, map those objects into
data item, but finally, instead
of attaching those data items
directly to the element, you
append to the existing elements.
And once you've done that, you
call Touch Property Path Method
on the data item class to push
your updates to the user
interface.
Now, let's put all of these
pieces together in a demo.
So as Trevor was doing his demo,
I actually went in and
implemented pagination in the
sample app.
So why don't we start from
there?
So I am going to take you
directly to the part where I
construct the stack document,
and the event I'm handling is
directly on the stack element
itself.
And as you can see, what I'm
doing is fetching the next batch
of assets that I want to push in
the template, creating
JavaScript objects from the JSON
that is returned, and then
recalling populateGrid to add it
to the template.
Let's see what populateGrid
does.
Now, at this point, I haven't
converted my templates using
prototype and data binding.
So what you see here is very
familiar.
You would probably already know
what is happening here.
I start off with an empty grid
and I map all of my objects into
the corresponding TVML markup,
which is a lockup in this case,
and finally I parse all of those
lockups directly into the DOM.
Now, before I go ahead and
convert it into a data bound
template, let's run this app,
just to get a feel of what
pagination looks like in
TVMLKit.
So as you can see, it starts off
just the same.
But notice as I scroll toward
the end of this grid, the index
bar on the right side jumps
back, and that is because we are
adding all those items as we
approach toward the end.
So great, our pagination works.
But let's go back and complete
this example by converting our
templates to using prototype and
data binding.
Now, as you might have imagined,
all the changes that I need to
make in order to convert it into
a data bound template are in
this one function, populateGrid.
And the first thing I want to do
is, instead of adding a simple
grid, I want to add prototypes
and a bound grid, like this.
The next thing I want to do is,
instead of mapping all of these
objects into their markup, I
want to map them into observable
objects using data item class.
Just like that.
And finally, instead of creating
all of these markups into the
DOM itself, I want to append my
created data items into the
existing ones.
Like so. So now that we have all
of these pieces in, let's re-run
this app and check it out.
Now, right out of the box, one
thing that I have noticed is
that it loads faster, even with
such a small subset of content
that I'm loading up front, it
takes less time to load.
And as I scroll toward the
bottom, it has flawless
performance with buttery smooth
scrolling.
It is quite addictive, I could
scroll all day actually.
[ Applause ]
So let's do a quick recap of
what you saw.
What you saw is how to implement
your templates using a better
paradigm of prototypes and data
binding.
That is going to reduce your DOM
size, and make your apps
perform, and how to implement
pagination conveniently using
Needs More event.
Now, there is one more thing
that I want to recreate before I
go.
Remember this graph that I
showed you earlier, about how
long it takes to build a
template to a certain number of
items?
After re-implementing our
example with prototypes and data
binding, we re-ran this test,
and we found there were more
than 50% reduction in the time
it took to build these
templates.
So we were really happy with it.
So I encourage you to check out
these APIs, and try them out in
your apps, and see what you can
gain from them.
I am going to hand it over to
Jeremy now, to talk all about
Web Inspector.
Thank you.
[ Applause ]
>> Thank you Parry.
Hi everyone, my name is Jeremy.
And today I am here to tell you
how you can achieve happiness
when you do TVMLKit development
with the help of Web Inspector.
Now, we have seen amazing demos
from Parry and Trevor, trying to
build up a simple WWDC app.
And at this point in time, it is
fairly capable.
It supports RTL, so it can be
localized immediately to
right-to-left languages.
And it uses data binding and
prototypes, so you could scroll
forever, and not stop.
Now, as luck would have it, just
a few days before we are ready
to ship, the designer comes over
and hands us this mock-up.
He gives you a thumbs up and
says, "this should be easy to
fix."
Now, we all know that is not
true.
And I empathize with that, and I
can hear all the inward groaning
in all your bellies.
So let's try to articulate this
feeling that we have by looking
at the development cycle,
especially when it comes to
updating user interface.
The very first thing you have to
do is that you have to formulate
speculative fixes, guess well as
to what it involves to fix the
UI.
Is the margin one pixel to the
right, or to the left?
Once you have formulated all
your fixes, you have to go
through this tedious Build and
Run cycle.
You build and run in X code,
wait for the app to start, look
at the changes, is this right or
not?
And if it is wrong, you go
through that over and over again
until it is right.
Consequently, because of that,
there is a huge loss of context.
You forgot where you were and
what you're trying to fix,
simply because there is this
huge time gap between the
changes you make, and seeing
them on the screen.
So how amazing would it be if
there was something that could
help us fix all these problems?
Seasoned web developers out
there already use Web Inspector,
with all these amazing features.
It is about visual debugging,
local and session storage
introspection, even performance
analysis for JavaScript.
And today, tvOS only supports
this small subset of features in
TVMLKit for Web Inspector.
So today I am really pleased to
announce that in tvOS 11 we are
adding support for the remaining
few bits of Web Inspector.
[ Applause ]
>> So to get us started, let's
talk about visual debugging.
Your TVML app is a document that
is represented as a DOM, and to
visualize that, Web Inspector
has the Elements tab.
It shows a tree of all the nodes
that TVML uses to render UI on
the screen.
So the very first thing we did
is we wanted to eliminate all
guesswork as to the things you
actually have to fix, and that
is through the usage of
visualization.
If you move your mouse slowly
over the nodes in your tree, we
would highlight the
corresponding views on the
screen itself.
We even give you information
about its dimensions, and its
elements that it is associated
with.
So now that you know the exact
node you have to fix, you can
edit the individual XML
structure of that node itself.
Any changes you make will
validate before rendering UI for
it.
Now, if you don't want to deal
with XML, and you just want to
reorder nodes on the screen, you
can also do that.
It is as simple as dragging the
nodes between the areas that you
want it to go and then the
screen updates itself.
For developers that want to see
their applications in a
different light, light or dark,
you can change the attributes on
the template itself.
In fact, you can change any
attributes on any nodes you
want.
And finally, all these changes
that you have made, you are able
to copy it out and paste it in
any file you want.
Now that we've seen how we can
change the layout itself, let's
look at the properties we use to
render UI on the screen.
Web Inspector allows you now to
look at the individual rules
that are attached to individual
nodes, so you can see every
single rule that is used to
render them on the screen.
These are arranged in a cascade
ordering, rules that matter the
most are right at the top, and
things that it overrides is
loaded right at the bottom.
Media queries are able to be
visualized as well, so you'll be
able to see the individual rules
that correspond to specific
media queries, so for example,
you have rules for LTR, you will
be able to see them grouped that
way too.
Finally, the Web Inspector
allows you to look at the
default rules that ship with the
framework itself, so that
totally takes guesswork out of
figuring out what is the default
rules that you have to change in
order to make your UI look like
the beauty it is.
And of course, with all these
rules that are displayed, we
will also show you the coalesced
version of it.
These are the exact styles that
the framework uses to render UI
on the screen.
In Web Inspector, there is a
reload button, and now, by
hitting the reload button, you
are able to restart the entire
JavaScript context without going
through Build and Run over and
over again.
So with that, I'd love to show
you a demo of all these features
that I've talked about.
So I have gone ahead and gotten
the project ready, and updated
the UI, but the last bit to do
is to check if it accurately
supports RTL.
There we go.
And once again, we are going to
use the thing that Trevor taught
us, by going to the scheme, and
changing the system language to
right-to-left pseudolanguage.
Let's build and run.
All right.
As we can see, everything looks
kind of right, except the banner
right up there, which we haven't
optimized for RTL.
So instead of going through that
Build and Run cycle that Trevor
was doing, let's try to do it in
Web Inspector.
The way to do this is that you
start Safari, there is a develop
menu that I've already enabled,
and you go into Simulator, and
inspect the app that you want to
look at.
So Web Inspector starts up, and
let's get going.
So we want to change specific
user interface elements on the
screen, and we can see that the
title, the subtitle, and the
button needs some work.
With Web Inspector, you can
easily drill down to the exact
nodes that it is related to by
moving the mouse, and you can
see that things start getting
highlighted.
So let's go quickly to the title
and change it.
Now, Web Inspector shows you all
the rules that correspond to
this node, and all we need to do
is hunt for this pesky bottom
left that should be actually
bottom viewing.
And as I type, observe what
happens on the simulator screen
itself.
Boom, it updates automatically
without needing to build and
run.
So it's so quick, so for those
that missed it, let's try to do
it again for the subtitle
itself, and it's the same steps.
Locate the element you want, and
change the style associated with
it.
There we go.
Now, let's quickly update the
Play button as well, so it's on
the right side of the screen.
For that, we are going to change
all right to trailing.
Trailing. And of course, like
Trevor did as well, we need a
special media created for its
margins, because it is different
when in RTL language.
So I am going to copy margin
here.
Paste it over.
And change all that.
And there we have it.
Now, since all these changes
were made to the author style
sheet itself, which is contained
in the style attribute, the
style tag, we are going to copy
this out.
In order to verify, we are going
to paste it into our TVML
document.
So here we go.
And then instead of building and
run, we're just going to hit
Reload.
And there we have it.
Everything works correctly
[applause].
Let's go back to the slides.
[ Applause ]
So what we are seeing is a very
quick way to figure out user
interface issues and fix them.
What we've done is we have
looked for the particular nodes
that it has affected, edited
them in real time, and finally,
verification is just done by
copying the TVML properties out
and restarting your JavaScript
context without doing build and
run over and over again.
So that is the visual debugging.
I would like to now move on to
network analysis.
Web Inspector now supports
looking at all network requests
that are sent from TVMLKit
application.
We will show you individual
request timing information.
So how long a DNS query takes.
How long a response takes to get
sent over the wire.
You will inspect the individual
request properties that you set
on your request, so you can
verify if these are the exact
things we are sending out.
And finally, response and
request headers is something you
can look at as well.
You may have scripts already
today that you use to develop
local and session storage.
There is no need to do that
anymore, because Web Inspector
provides a nice user interface
to look at every single key
value pass that is in your local
and session storage, and because
you can look at them, it also
means you can modify them.
So you can copy them, delete,
and move them around.
With that, that is TVMLKit and
Web Inspector support, and that
is also how you achieve
happiness.
[ Applause ]
So what we've seen today,
TVMLKit supports RTL right out
of the box for all default
templates.
It is a little bit of work if
you have custom styles, but it
is easy to do.
Use data binding and prototypes
if you want buttery smooth
scrolling, like Parry has said,
and finally, use Web Inspector
if you want to reduce the amount
of development time it takes to
fix bugs.
For more information, please
visit the following URL.
We have sample code and
documentation that you should
really look at.
There are amazing sessions, as
well, especially What's New in
tvOS tomorrow.
Thank you for attending WWDC 17,
and have a great rest of the
conference.
[ Applause ]