WWDC2004 Session 701

Transcript

Kind: captions
Language: en
good morning and welcome to session 701
an introduction to digital media my name
is Stephen tell and I'm a product
manager on the QuickTime product
marketing team and we thought this would
be a great way to sort of kick off the
entire quick time and digital media
track by having sort of an intro course
that explains you know just about
everything you need to know about
digital media from be interlacing 23 to
pull downs everything you need to do to
take that tape content you have and
digitize it and make it look really good
and so we have a special guest today we
have hege Van Dyke from discrete who
knows just about everything about
quicktime he's even an old quicktime guy
from apple and i'm here will be able to
take you guys through for about an hour
okay great thank you hi everybody first
thing I'd like to say is thank you all
for coming appreciate coming down here
and talking about one of my favorite
subjects which is digital media and also
thanks to the QuickTime team for
allowing me the opportunity to speak so
let's get right into it if we can switch
to the machine please i want to talk a
little bit about what we're doing here
today get this started okay so what I
want to talk about today sort of digital
media basics real simple stuff is how do
we go from this which is our camera you
know taking pictures of stuff very you
know and I well what I will say used to
be analog format but you know capturing
pictures and assembling them putting
them together in very creative ways this
is what a lot of you are specialists at
is really in producing this media so
that's the next real level about it and
then ultimately how do we get that what
do we do with it and where's it going
creasing lean more and more I could play
the video to catch up here but we're
starting to see that we need to take our
media you know they used to say that
editing touches every frame well now
encoding really touches every frame and
your media needs to be distributed in a
variety of way so how do we sort of sort
out all of the issues and logistics with
our media as we take it from the camera
where we collect those images sometimes
in random haphazard any sort of format
way produce them in exciting ways using
great creative tools like this and
ultimately get them down into formats
that we might have in our pocket anytime
anywhere that's the basic premise of my
discussion today and again hopefully
we'll have some fun and we will take
questions at the end of people are
interested we'll have some time for that
as well so okay I'd like to switch to
the slides please and we find my little
switcher here okay we can that we've
done that we've done that and now we're
here okay so capture create output
capture management deliver produced
deliver how many folks have been to a
trade show and seen these words
somewhere in the last 10 15 18 years
oftentimes them when we go to shows we
see words like this on big banners above
the convention center drawing us all in
what does all this mean these are really
terms that are used to describe the
media workflow so in order what I'm
trying to do today is give us a language
that we can use to discuss digital media
and hopefully that some of these basics
will help bring value to you as you
attend some of the other sessions here
at WWDC 04 so again having these sort of
catch phrases capture create output
their stages in the production workflow
we use these terms to describe that what
I'm going to be doing is just using that
as a frame of reference for our
discussion today and kind of breaking up
the media production process into those
three categories capture creation and
outputs and just to kind of screw things
up a little bit i'm going to start in
the middle with the create parts so
create how do we create media well we're
already on the desktop in the center
thing and the way that we create media
is using the very powerful QuickTime
format so what is quicktime there's
obviously one of the one of the reasons
why I took this talk I'll have to admit
this is that how often have you been on
the receiving end or been the customer
who said you know what kind of content
are you using and you either said or
heard I have a quick time okay some of
you've got to have heard that before
well that sort of
the things it's like I'm speaking here
in the United States there's not quite
enough information in the QuickTime
quicktime is so much more than that that
again having other terms or a little
deeper understanding of what it is will
help us with this discussion so what is
quicktime quicktime is system software
so it's a core set of services that
allow applications to treat media in a
consistent fashion okay quicktime is a
time-based a universal time base for
multiple multiple synchronous event okay
so what that means is that my computer
over here I can play a movie my computer
over there I can play a movie that movie
has audio and video maybe few other
kinds of tracks associated with it but
they play back and think and then I can
play them in different places now that's
kind of a minor point is that we all
take for granted but at one time we had
a real time hard time making things
synchronized up the fact that quicktime
sort of handles this is a master for a
time-based is again something that we
might take for granted a little bit but
the one that we really want to talk
about today the thing that we're all
focused on and the element that we all
deal with is the QuickTime file format
that's really again what we're talking
about today so a QuickTime movie as it's
known is a container for your media
that's the most important thing to
remember is that again quick time is the
services and this timing references but
it's a container for media and what I
mean by that is it's a rapper that lets
your video frame move through the
production process and sometimes it's
transcoded but that's the central ideas
quicktime is a container it's not a
codec in and of itself it has codecs and
we'll get to that in just a second
rather it's a container that wraps
around your media and kind of protect it
as it goes and is used by different
applications in different ways it is
traffic
so again quicktime is a container of
multiple tracks we're all familiar with
an audio and video track the QuickTime
has many other exciting traffic types
including MIDI tracks texture text
tracks sprite tracks we're always trying
to get people to use these other track
types because again quicktime is such a
technically robust and truthful
foundation for media we have all these
track types so we're just not used to
using them so it's always good to be
aware of that and again obviously it's
identified by the dot move file
extension we're all used to what movies
are let's talk about this a little bit
what is a QuickTime movie I realize this
is really obvious stuff but again it's
good that we all sort of repeat it has
the same perspective on it so video is
framed sighs how tall and how wide it is
that the actual frame is the reference
there okay it's the frame rate how many
frames are repeated one after another
it's a codec or a compressor
decompressor we'll talk about in a
second and all that turns data rate so
let's talk about these things first of
all frame size measured in high spy with
okay so DV the frame size here that I
started with is 720 x 480 so that's
pretty big size I think the resolution
I'm here is 1024 x 768 pixel size okay a
computer screen is made up of pixels
those are called pixel elements that's
what that short for a pixel so a
computer display shows video by turning
on basically a pixel and what they uses
there's a value so already with digital
video we have a resolution component how
many colors do I see if we had a
black-and-white monitor with the pixel
on it pixel has two states that's white
or black offer on and get a black and
white image with a single pixel now if I
take that fixed line I make it eight bit
I multiply it get an 8-bit words that
I'm using for that one pixel for a
single color I get 256 shades of gray
for that one color and color is made up
on
computer of red green and blue so we add
those three colors together at 256
levels of gray for each of them put it
all together and we get a few million
colors that we're used to in our display
okay but the main thing to realize is
that again computer's display in pixels
so the 720 x 480 that's the measurement
that we're talking about then also
switching to the audio here audio has
the same thing audio has a sample size
so much like the pixels at eight bit
pixel that I just talked about which is
determining the video resolution audio
also has a sample size and that
determines the dynamics for audio video
quality is the difference in loud and
softness and that again is controlled by
your sample rate that should have been
first so sample size rather so what I'm
talking about is 16-bit audio 8-bit
audio determining that dynamic there and
then sample rate or frame rate a picture
so we have a single frame and the movie
is made up of multiple frames that
playback in order ok and those frames
playback at a specific rate so as
pictures repeat again you get frame rate
we have some common frame rates that the
video works at my video camera here is
ntsc that shoots at 29.97 or 30 frames
per second in Europe they have a
different electrical standard which
relates to a different video standard in
the video over there moves at 25 frames
per second to get that tendon camera but
what we're talking about right now as a
container is just that video has a ray
so these these repeat after each other
in a certain rate okay so the last thing
is we have our here we have a codec so
it's interesting a codec is a concept
that quicktime brought us it means a
compressor decompressor okay before when
we'd have a video frame we'd have to do
with it its uncompressed form so if i
just take a picture that picture is
about a meg or the size of an old floppy
disk it's going to bring a floppy disk
today but i realized some of us wouldn't
know what to do with it so i didn't
bother but anyway floppy disk is one meg
so at uncompressed video 720 x 480 one
meg if i run that at 30 frames per
second that 30 Meg's a second okay so
that's starting to be a large amount of
data now quicktime introduced this
revolutionary concept when it came out
1992 called a codec or compressor
decompressor what what a codex does for
you is it take that data and shrinks it
down often 10 times that original five
it also decompresses it but because that
codec is real time or you don't notice
that it's there then if you store your
media in a particular codec it
essentially is ten times smaller now ok
now we always have a trade-off with
using codecs the trade-off is image
quality we want to preserve image
quality and so for that reason there's
really two types of codecs two general
categories for codecs and how they're
used one of those is for the production
of media high data rate codecs where I
want to take high-quality images and
combine them together what I want to do
is use a high data rate codec for that
then there's distribution where I want
to take my media and I want to play it
in as many places I as I can and get
that media exposure those are
distribution codex so there's two
different times and again the beauty of
QuickTime is quick time allows you to
convert between those codecs very easily
and efficiency efficiently because again
it's all within the QuickTime architects
doesn't let you to do that so the last
thing about codec is again data rate or
video size which is analogous to video
quality again the amount of data that we
can allocate for particularly for a
particular movie would be considered its
data rate that's measured by per second
so data rates are usually talked about
in megabytes per second for production
or megabits per second much smaller unit
when we're talking about distribution
and again with audio so audio has a
similar set of attributes again sample
size determines its resolution sample
rate really determined how
sounds and the data rate is very much
tied into that we use less codex it's
really less codecs for audio do some
more the Codex and audio either
uncompressed for production then we have
distribution codecs for audio all right
now what I'd like to do i believe is
show you just a quick demonstration of
what i mean they're going to hide this
and bring that in question so you'll
have to excuse me because my little
frame count got off there a little bit
but just to demonstrate this a little
bit so if I step through my movie here
up there I got it back so what I'm doing
is I'm just stepping through my movie
obviously okay here so we have frames
over time what that's doing is causing
motion to happen so I'm now on frame 16
now if I take this movie and go here
hoping to jump straight to perspective
you but if I take this and then I pull
out and I look to the side of it you'll
see that I have a series of frames over
time there so again let me go ahead now
and play this just show you that and you
have a series of movie so still image
frame determine your frame rate and then
over time again multiple frames so this
is going to turn for us and show us that
we have multiple frames so that's the
basics of a movie again frame size frame
rates kodak or format and data rain
so ones do is bring up to stop this hi
there what I want to do is just quickly
bring up quick time and if i go to
export this as though i have quicktime
pro let's just take a look at the Codex
available to us inside quick time so
here i'm exporting inside quicktime pro
i have video and audio settings and if i
bring up the quick the standard
compression dialogue to see that inside
here quicktime has a lot of different
codecs associated with them and so what
are all of these things well these
codecs are different ways of compressing
media for different purposes so for
instance if i'm doing DV i can select DB
pro or dvd BBC pro ntsc that's a DV
codec two runs at three point five
megabytes per second to production kodak
so i could take my media preserve the
quality assemble it what's really
important about production codex is
often there what we say a fully kia
they're all I frame or they're fully
editable so as we start to get into
distribution codecs what we do is we
start to pull tricks that manipulate the
image resolution and all the data isn't
necessarily there in a codec that is
used for distribution for production
codecs what we do is usually try and
preserve all of the image quality called
an iframe and again iframe formats are
what are used for production so again
jpg or motion jpeg is a real popular
format for production because every
frame is fully editable and as we get
into DV DV is a slightly compressed
format but again it's always editable
other choices that you see inside here I
do have component video so again
production codex i also have a few other
ones in here i have mpeg-4 video and
also have sorensen video so these are
other codecs that are used for
distribution again much lower data rate
codex
the two types of codex okay I think we
can switch back to the other machine a
please okay so that starts to get us
into the create part of it how we create
stuff again QuickTime movies now let's
go back out into the real world and talk
about how we capture stuff and the
format's that we deal with when
capturing video so i have a DV camera it
shoots at 720 x 480 again if this was a
pal camera it's also shoot set again 30
frames per second you've heard the term
SD or standard resolution standard def
standard definition what that means is
that it's a four by three aspect ratio
very common standard definition and
again the size of that is 720 x 480 you
can think of SD is like our television
again our television is a 4 by 3 aspect
ratio 640 by 480 the reason why this is
720 x 480 is because the pixels are not
quite square again now i'm on the edge
of going less than a little more than
video basic so i'm going to stay a
little bit away from pixel sizes but
again just realize that you have SD 720
x 480 that's the size that we're used to
now there's a new frame rate in town and
there's been a lot of buzz over a format
called HDTV everyone says you know come
in come in soon HDTV so HDTV is a much
larger size it's taking our SD sigh 720
x 480 and basically multiplying it by 4
so there's a variety of HD sizes the two
main ones that were concerned about are
the one called 720 which is 1280 by 720
in size and then 1080 which is again a
huge frame which is 1920 x 1080 so a
huge video frame now it's also
interested about HD is that it's also
higher resolution so again it can be up
to 10 bits per pixel which means that we
can't even display it on our computer
screens all the color information at
once because our computer screens are
only a bit
so dealing with HD has a whole bunch of
interesting attributes to it it's very
big frame size it has a lot of color
information to capture and what's also
interesting about HD is that we're back
into a situation where with HD and HD
production when you're building a system
around that often your video quality is
constrained by your data rate and a data
rate for an HD signal is up around 160
megabytes per second so again a DV
camera shoots a steady data rate of
three point five megabytes per second
and HD at 160 so again a lot more data
what's really interesting about HD
production is we get more and more into
it that we start to learn a lot more
about HD HD is you know technically a
lot of resolution there but what we're
starting to learn a lot is that there's
a lot of issues associated with HD
capturing so you want to go out and
shoot an HD movie all of a sudden you
need a better makeup person or a lot
better makeup you need good life all of
the details that film and DV cameras was
so forgiving about because we're in the
initial stages of HD production we're
really suffering from that we have to
blow a lot of smoke on stage to create a
lot of depth the images tend to look
very flat and again over time I'm sure
that it will work with lenses and ways
of softening that up so it's a little
more useful but today HD is this huge
frame size very hard to struggle with
but once you get it obviously the
quality is four times that now we also
have frame rates that when you make a
movie on your computer the frame rate
can be any frame rate that you want
again the idea of starting with the idea
of a QuickTime movie as a frame size and
a frame rate is just to let you know
that those sizes are arbitrary and that
you can make them whatever you need to
but not in the real world sizes are
dictated to you by the hardware so again
my camera shoots full size that size
happens to be 640 by 480 runs at 2997
now film and HDTV
run at 24 frames per second so that's a
little bit different a little bit slower
and actually the difference between 24
frames per second and 30 frames per
second is really enough that that
matters about how the background looks
that there's actually aesthetic
considerations made to film shooting at
24 frames per second and how you can pan
a subject on a camera and the way that
that looks versus video which runs at 30
frames per second actually it's
interleaved I'll get to that in just a
second so it's actually showing you 60
images per second so video is much
higher resolution than the 24 frame for
film that's why again video is very
popular for sports any sort of content
where you need a lot of fast to action
again video resolution at 30 frames per
second allows you to capture that ok so
again out in the real world we have
different formats we have SD we have HD
we bring those in so what we need to do
at a certain point is capture that video
now I used to say that this was an
analog versus digital scenario and that
it was real easy that our cameras were
always analog and that we've just be
digitizing that more and more though
it's really portable versus not that
increasingly our cameras digitized on
site and already have converted that so
this is a digital camera this DV camera
it's a mini DV camera it's very old I
props at about 14 times from places like
this you can see I need a new one but
the main thing again at so at 20 720 x
480 this is three and a half megabytes
per second so the video comes in when I
capture this ok it's digitize right on
the camera so all I need to do when I
capture this is I take as a firewire
output what I'm doing is I'm moving that
digital information between my camera
and my computer so you can think of this
as a hard drive again I can push the
video back but I'm just taking visual
information in
digital format and pushing it back and
forth with my camera so DV is very easy
to use because the signal is already
digital if you if you have an analog
format we need to digitize that and
again there's a variety of products to
do that we have the i/o products from
Asia again very high-end cards that lets
you do component input if you have say
producing on digital beta digital
betacam has a digital output called an
SGI SBI Jack and again that SGI Jack you
can bring into a deck length black magic
card again a naja box and bring that in
but at some point you need to take the
video from that outside world and bring
it in and again that's where you capture
and again what we're talking about there
is that you have to be aware of the
frame size the frame rate the codec and
the data rate of all those different
formats okay so we talked a little bit
about capturing again variety of ways of
capturing so once you've captured that
video what do you do with it and again
we showed the video of the different
content there's a variety of
applications what mainly happens in the
production phase is that you're taking
those QuickTime movies you're combining
them all together either editing which
is a temporal issue so you're you're
combining the creative story where
you're compositing them together the
main point about production though the
creation of content from a technical
perspective is that you end up mastering
that format at the same size again a
common misconception is that I'm just
going to output that video once i'm done
to a small resolution and be done with
it and the point really of having the
discussion today is just to let everyone
know that really increasingly you're
going to go back to that content and
want to deliver it in multiple formats
so we must make a master version of that
content that again can be used in many
different places so the best way to do
that is to produce it at full size and
then to take that master
file and start to convert it off to
different formats also at some point
down the road you're going to come back
and want to make another version of it a
DVD another CD or something like that so
again having a full res master is really
important so when you're producing your
content again DVI output a 720 x 480
movie 30 frames per second using the DV
codec so at a high again a distribution
data rate our production data rate
excuse me so once we've got in our video
we've created the master now what we
want to do is start to distribute that
video what formats do we take to
communicate our message out to multiple
formats there's really a few different
sizes there and again a few different
things that happen with that video as we
go through the production process the
first thing is we want to make a DVD out
of it that's the most popular kind of
format or very accessible format for
making video well DVDs are interesting
if we take our little formula here frame
size frame rate codec data rate the
frame size of a DVD is set for us it's
full size 720x480 so it's great about a
DVD and why one of the reasons why it
looks good is we've preserved that full
frame size of our media now mpeg-2 is
the codec so that is actually the
compression scheme when we talk about
mpeg-2 and peg to is is a compression
sphere and that has a data rate
associated with it and the data rate
that we most commonly use for DVDs is
5.7 megabits again a DVD has a data rate
choice of up to 10 megabits but I work
with a lot of compression project
products and what happens is a lot of
times people have a short 20 minute
movie they want to go ahead and make a
DVD of that and we're all concerned
about quality we want to get the highest
possible quality from our media so if I
have less
than an hour and 15 minutes to put on my
disc and I'm compressing that to DVD
what I'll often do is turn the data rate
quality up or what I would think is to
turn the video quality all the way up to
eight or nine megabits per second go
ahead and encode that media and then go
ahead and play it well there's a little
bit of a problem with that and that what
we've just what we've determined you
know sort of extensive testing has found
that if you have a real high data rate
like that you're going to have an
inconsistent playback experience between
the different DVD players so
increasingly when I talk with people
about making DVD content I really urge
them to try the data rate of 5.7
megabits and to try and use filtering or
other processes to into him to preserve
that video quality as much as possible
because what we found is that a 5.7
megabit DVD really plays well in all
places now as we move from DVD to other
formats for instance web video the
choices open up a lot more again with
web video what we're trying to do is to
get the video to play smoothly that's
where the data ray comes in but the
sizes are rather arbitrary and really
when we're talking about compressing for
video we're looking at those same trade
off so I have uncompressed video that's
full size the first trade-off that I can
make is the frame size to reduce that
data and again common web sizes is 320 x
240 what we're trying to do with web
video is really get a theatrical
experience whether or not we realize
that we all want a theatrical experience
from the web so we're really trying to
get as full size codec as we can my work
experience has been rather interesting
I've been doing digital video for 18
years at this point and what we started
off with was a technology called quick
hon where everyone complains that they
were little postage stamp movies and
what we did was we worked really hard to
with Hardware assist get that to be full
screen full motion we achieve that in
about 19
94 the variety of products to do that
then what happened was about nineteen
ninety-six the internet became more and
more prominent and we started to get
digital video happening on the web and
there was once a time when Peter hottie
who was the QuickTime architect asking
me to make him a 60k movie no he wanted
a 30k movie 30k bit movie and I said
well Peter that's you know I'm going to
be 60 x 90 and that looked too good he
says I want to movie that played and oh
I made the movie he used it for the demo
but what was interesting about that was
that we've now we went to a software
world and then for after 1996 what
happened was slowly we've been trying to
get that frame to size to grow so it's
in stretching that frame sizes in codecs
developments and increasing the data
rate can we actually stretch that size
out to be full size and we've almost
achieved that but there was an
announcement yesterday very significant
announcement for anyone making digital
media which is that Apple is going to
include a new h.264 codec in the next
version of the OS in the tiger operating
system now what's really exciting about
that is that right now we can do a
fairly good job of stretching mpeg-4 to
full screen full motion it's pretty
difficult to get it but if we increase
the video by a factor of four that's
going to look fantastic so we really
look to mpeg-4 to deliver a software
version of a codec that is full screen
full motion now the interesting thing
for me is just about the time right now
that we're getting that together there's
a new thing on the horizon and that is
everyone's driving around in their car
with their Wi-Fi we've been Hong Kong
they're watching video on these portable
formats so again it's kind of
interesting about this is we work really
hard to make it full screen full motion
on software and they invent a new
platform for us so this is a Klee a this
is a pj 37
and what this is is a 200 megahertz CPU
so again what they've done is they took
my beige g3 or so and they shrank it but
again the video quality on this is going
to increase better and better than what
we're going to see more and more of is
video in these other portable formats
and again what's the great thing about
mpeg-4 and the upcoming h.264 codec is
that it's going to increase the video
look here by a factor of four so that's
going to be very exciting and again just
to get back to the size thing that often
times when you're dealing with phones
cell phones or palm OS devices such as
this that again you're going down in
size to 160 x 120 so you're able to
compress that down a whole bunch so
let's go ahead and switch to my machine
for a second please go back to
and so what this is just a picture of
relative frame sizes for you so again
the big white box is our 1920 x 1080
frame size the next box is our twelve
eighty by seven ten eighty by seven
forty seven twenty frames I excuse me
here's our 720 x 480 here's our 320 x
240 standard website and again here's
our 160 x 120 our cell phone side so
again one of the ways we're reducing
data is by stretching that frame size
down we can do the same thing with frame
rate but frame rate is a little more
it's worth little less forgiving with
frame rate because that has to do with
temporal motion or motion over time and
again what you want to be aware of about
frame rate is that we have with really
good eyes as far as seeing stuff like
seeing deteriorated motion and motion
resolution so what we really want to do
is preserve motion wherever possible you
have to really look at your content to
see how how fast it's moving to
determine whether or not you can switch
the frame rate I kind of go back and
forth on this and I'm currently in a
personal phase where i try and keep my
frame rates it's matching as much as
possible so i will make my frame rates
2997 or 24 you can you can hit 215 or
seven at lower bitrate but again the
movie gets choppy but the nice thing
about that is again with quicktime as
the technology that choppiness will at
least be consistent okay you won't get
snappy video you'll get choppy video so
that's what I'm an expert in choppy
video okay so great we can go off of
their back to okay so actually I want to
do a little bit of a demo now and just
talking a little bit about distribution
there's going to be some seminars we can
go actually back to my machine there's
going to be some sessions during the
week about
about video distribution and video
codecs and preprocessing so what I
wanted to do is talk a little bit about
again how we take our media and
distribute it so you just cancel and
close this cancel this
I'm going to be using cleaner as an
application to sort of talk for a minute
about the process of distribution and
how to get the most out of it the reason
why I want to use cleaner here is
because we're going to be talking about
pre-processing when you spend your time
creating your content you don't really
want to think about color values things
like that you really want to have a time
when you're when you're being creative
and assembling your media but at a
certain level you have to come back and
take a look at that media in terms of
distribution and go okay now I need to
distribute this I need to shrink this
thing down many hundreds of times
smaller than its original how do I do
that and that's where compression
applications offer you these
pre-processing so the first thing again
video when it's shot shot in the field
so we talked about this before where you
actually have alternating fields the
reason for that is that when we started
remember that a computers computer
display has a pixel on it single pixel
is going to shoot that on play always on
when when you see that display on your
computer well the television works a
little bit different a little more
arcane if you will television works by
scanning every line it draws one line in
the thirtieth of a second then it goes
back and draws another line in the
thirtieth of a second that's called
interlaced video again field that's made
up of these alternating fields they're
scanned even and odd depending on your
on your video format so DV is even odd
even odd even odd again that alternating
back and forth happens at the point of
the camera so all of your video
resolution is caught in that alternating
field and when you digitize that video
those alternating fields are also
captured and if you produce your video
and output it at full resolution those
alternating fields are maintained
through that process when you're doing
video so what happens is when you get to
the computer screen to view that video
all of a sudden you have a problem
because the computer video doesn't
display that way it's always on
and if you're going to show that frame
with both of those you're going to see a
difference there the interlace
difference in the weather thought that
motion is and if you're not careful with
how you process that video all of that
difference in motion is going to be
interpreted by the computer which
doesn't know what it is as noise it's
going to create it giving you a very
blotchy out outcome so again one of the
things that we have to do we take video
in and start to D interlace it or reduce
that data down by taking away one of
those fields and thus capturing that
motion smooth and we're going to use
cleaner to see that in just a second so
again video alternating and D emulation
is really critical to data reduction but
it's also critical to getting a good
look for your web video because again
that those motion artifacts will be
interpreted by as noise for you another
thing is that if you're using film film
is shot at 24 frames per second there's
a problem with film though if I edit my
film on video video runs at 29 97 so
it's a different frame rate so there's a
difference in the frame rate that film
shot at 24 in the video at 2997 so how
do we deal with that well there's a very
standard process called teleson which is
the process of of moving 24 frames per
second video up to 30 so it's not like
the video case where every field is to
here is interlaced but basically habit
of 60th of a field what's happening with
telus in each video or video that
originated on film is we get to
progressive frame so so progressive
frames the whole frame is captured we
get two of those and then we get three
interlaced frames to make up that
difference and it's that difference of
five over for overtime in the telus any
process that makes up that 2997 so we've
taken five frames mapped four frames map
them to five frames and added three
interlaced frames and to progressive
frames that's where we hear that term
three to pull up or pull
again it can be 23 or 32 depending on
where you start but the main thing to
realize is that there you have to
progressive frames and three interlaced
frames now when you're in that situation
and you apply at the interlaced filter
what you're doing is only you're only
doing the appropriate thing for those
three frames because it's mixed up
progressive and interlaced frames the
interlacing all of that is going to
cause a deterioration on those two
progressive frames so what we need to do
when we go back through web video is we
need to do a process called intelli cine
which is to remove that three to pull
down back to that 24 frames per second
so one of the really what I consider the
pinnacle of web quality is the Apple
trailer site the team in cupertino does
a fabulous job encoding those and if you
look at them they're all 24 frames per
second and again the key to that was
that those all those movies have gone
through a nutella cine filter to restore
that original frame rate never know
where your media is going to come from
today so again restoring that original
29 original 24 from your 2997 is
important if you're doing stuff in pal
it's just a one percent raise and lower
so it's a little bit different they
don't go through quite the intricate
process the telus tinny process in tell
city it's really only relevant to 2997
video ok so again as I was talking about
the next thing is how to how to process
the video so that it looks good we're
taking uncompressed videos into this big
and we need to run it through a hole
that's too small so we want to do every
single thing we can to preserve the
quality of that video so again throwing
off the frame rate making the frame size
a lot smaller well that helps there goes
half the data the interlacing that video
takes away half that resolution actually
helps us reduce that data but we're left
with a variety of other techniques to
preserve that video and that's where
you're going to see some sessions on
pre-processing because the key to good
looking web video is pre-processing and
what I'm really talking about now here
is web video
we talk about DVD we're not going to do
a lot of processing to our DVD because
it's already full size 720 x 480 the
number one thing that I see people
running into with a DVD production is
they need to go from there interlace
format to a progressive format they're
going to be playing the DVD that will
always be played back on a computer the
only problem with that is that when you
take your twin at your full size
interlaced video and UD interlace it to
make it progressive you've eliminated
half the resolution vertically on that
and you've softened that video so the
one thing that you need to be aware of
when you're making DVDs when moving from
recive to influence moving from in
relation to progressive as your videos
can go soft now there is some tricks
that you can do with the green channel
and balancing that stuff out but just
remember that again video goes soft when
you're going down too much smaller data
rates what you need to do is anything
that you can do to to what we call this
media mastering which is really a
process of applying filters to preserve
that data rate wherever possible so I
want to walk through that now a little
bit with cleaner and talk about what
some of these pre processes and
principles are and how you use them so
I'm going to use my bubbles movie for
this the reason why I'm going to use
this movies because it has a whole lot
of emotion in it so you see this is a TV
movie it runs at again it's 3.5
megabytes megabits megabytes per second
because it's on production codec and
again this is just a little scuba trip
that I had in them going down here and
you're seeing a lot of these bubbles
come up so just quickly how to use
cleaner i have my quicktime movie is
here in my batch window go ahead and
remove this one because we're not going
to use it i'm just going to bring up my
settings dialog and apply settings for
this and what's nice about cleaners that
cleaner presents to you all of the
parameters for quicktime movie and
preprocessing right here in its settings
window you'll see that i have the format
at the very top here it says quick time
and again I've my file suffix quick
so how this cleaner works is basically I
have these tabs that are going to list
to me all of the output parameters and
the pre processing parameters that I can
apply to my video so again looking at
the image size here the image size 320 x
240 let's see right here that again I
have this deinterlace what's nice about
this I'll see how this looks on this
particular screen what I need to make it
a little bigger that's okay okay so what
I have here we find a little bit better
frame cross
so this is a before and after view of my
movie the form before I actually
compress it so let's look at that before
view and again I've intentionally picked
this piece of media because it's got a
lot of motion so you see these sort of
horizontal lines in my video in this
before section what those are is because
what we're seeing is a frame of video of
a bunch of bubbles rushing up at us so
the whole thing is moving and again all
those interlace lines what those are is
the difference in movement between that
60th of a frame so this is where I'm
talking about de-interlacing your video
in order to get the best possible look
because again if I were just to compress
this without deinterlacing it look what
I'm feeding the encoder it's a mess it's
never going to get both above this high
level noise to get me any sort of
picture of an image at all so again
deinterlacing as the first step to
creating a delivery for your media an
output platform it's extremely critical
because with all that otherwise you're
going to get all that noise in there now
cleaner is kind of interesting cleaner
has a feature called adaptive there you
see now if I go to the output also
conversely notice how much that cleans
up particularly this group right here in
the center see how much just pulling
away half of that field really clues
that video up to get that started there
so again be interlacing probably the
most important filter when when pokémon
making web video because again you're
able to decrease that video but you're
really preserving the size the look of
this video adaptive deinterlacing if I
have a Batman shot everything's at an
angle has lots of diagonal lines in my
video and I were to take a picture of
that and in video and I were to be
interlace it because I had diagonal
lines in there and I take away that half
every other line what I'm doing is
starting to introduce jaggies into my
output little jagged lines there well
with an adaptive deinterlace your only
be interlaced
pixels that move so you're only going
after these artifacts to get rid of that
motion stuff again prepare your video
for a distribution kodak so again
adaptive deinterlace very important
you'll notice that cleaner has a sharpen
filter in there that sharpen filter is
just to brighten things up again because
codecs inherently doll our content so
again having that sharpened codec in
there is very helpful and what we have
is a concept of noise reduction so with
all of your video when you go down to
lower data rates you're going to be
using noise reduction noise reduction
filters are very very cool what they do
is they'll take the edges to preserve
those edges and they'll apply blur to
everything in between so the areas of
less details that our eyes are much more
forgiving for again can see this noise
reduction filter it helps scale the
video down very well so again how to use
noise reduction is really keep it on
mild or moderate and on how much motion
you have and we'll do that ok the next
filters just to talk about for image
quality we have a few of them that you
need to be aware of first thing is gamma
that there's a difference in gamma
between the mac and windows platforms i
believe like gamma taco X right here
so what I can there's a difference in
gamma between the mac and windows so a
mask computer is darker than a Windows
display so if you're always producing
your media on one platform on a Mac and
you give it to your you play it for your
windows friends they might say that it
is a little bit dark for them the way
that we handle that when producing
content for the web is to add a gamma
filter for that what a gamma filter does
it just brightens up those mid-tones you
can see here I'm just going to turn
these off for a moment we go through
this just update that so if I add just a
little bit i'll turn the gamma filter
off and again it's just going to take
that and brighten that up just a little
not not too much or the other way and
again by raising the gamma we're able to
make a movie that plays well in in a lot
of different places now also with
different encoding application jewellers
you always have a brightness and
contrast filter now color correction is
one of the main things that we do when
we're compositing media together because
when have you ever had two pieces of
video that were shot at the same place
in the same time with the same line
pretty much never so one of the things
that I work a lot with people who
produce visual effects is really in how
to balance video with brightness and
contrast in order to balance the
different elements together well the
same thing happens or a similar thing
happens when you digitize your video and
now I want to output it on to a variety
of different formats in order to keep
that video looking good you really want
to take and think of it in terms of
brightness so you have black and you
have white as the two pillars of
brightness here and so what you can do
with this brightness and contrast
filters is you can look at your video
and you can reset what the black and the
white level is so that when you're going
into the process of encoding it you've
given yourself really strong color
values defined what the black and white
level is and if you keep strong color
values you're going to help reduce
compression artifacts that's what use a
brightness and contrast filter for so
brightness is really used for finding
the black level in your image
you notice my image doesn't have a lot
of black in it so it makes it a little
bit hard to see but again you can tell
that by bringing down that the
brightness filter or just playing with
the brightness filter in your compressor
that you're able to again find a value
that looks rich conversely contrast is
what you use to adjust your white level
so again you can move through enable
your contrast filter and bring that up
just a little bit you want to be careful
because when you when you stretch your
contrast too much you're actually
reducing the range of pixels there so
you're making your putting less color
information in your video but again
you're resetting that black and those
white levels so again very good to be
aware of your brightness and contrast
cleaner has two other very important
filters in it that are used for making
the video really look the level of
quality that we're talking about on the
Apple trailer site and that is the
concept of black and white restore so
what these filters do in here for you is
again they'll take and they'll actually
regenerate those colors for black and
white for you so this is significant
because again it's going to really
ground your York it's going to give your
content good color values so you might
say well there's not a lot of black in
this video why would I use a black
restore video on content like this well
if we notice if we bring up the amount
of black restore in the video image
you'll see that this blue that's all
over this this content really does have
elements of black that there's a
darkness associated with this whole
background inside this video clip and
that I can sharpen that up or increase
the clarity if you will of that kind of
stuff using a black restore filter so
how you actually use these filters is
you'll notice they have a smoothness
control so if I just turn on the black
restore and then turn this way up you're
going to see that at a certain level
it's going to start to get very blotchy
so you notice in here starting to get so
black is being added and just
regenerated into all these areas but if
i use a smoothness control here what I'm
able to do is just get a little bit of
softness
and get a little bit of an edge and nice
lift to that and really get those colors
back so to get good contrast in that
video I can go ahead with a lot of
smoothness and softness and bring up
that with that black hooker same is true
for the white restore so again same
principles apply that all colors have an
element of white in them and that by
regenerating this black end these white
colors were able to really preserve
those color values so that when you go
ahead and compress that video you get
the highest qualify output so we call
this media mastering the other thing is
to realize that when you do audio again
for audio formats audio is an
interesting case for a long time
starting with quick times 3 we had audio
codec called the cue design music codec
so cute design was it was a very good
codecs for reducing the data rate of our
video but cute design did have an
unfortunate artifact associated with it
it would produce a phase shift at high
at upper frequencies basically it would
make sure your audio sounders so you get
a swishing sound with the cue design
codec now that was followed up by the
very popular mp3 codec and mp3 you know
it's available i believe quick time for
came in and again mp3 very popular very
portable format and mmm that's my mp3
impression so again mp3 is very good at
planning they're just missing high end
so mp3 doesn't have a lot of high end
the other thing about mp3 that you
should realize is that mp3 has really
severe characteristics to it that change
or your lower mid range so again if
you're if you're in coding stuff to mp3
it's going to go through a tam bral
shipper the quality of the sound will
change a bit it's going to get a little
lower mid-range e if you're if you have
a jazz band with a saxophone the
saxophone is going to be reduced in the
ride cymbal is going to come up a little
bit so you just want to be used to the
fact that that mp3 while it's very small
and portable does have like the cue
design
I'm some baggage associated with it
that's where we have now of course what
I always have to show here is the mpeg-4
audio or the AAC audio codecs this codec
was pioneered or created by Dolby same
folks that brought us that ac3 codec
that is the standard for compression and
DVDs the ac3 audio codec or the mpeg-4
audio codec as it's known is the best
thing since sliced bread for video on
the web this codec is the greatest
what's nice about this is you get a full
frequency there you get a full frequency
dynamic so you get all of the
frequencies associated with the video so
no little high end stuff and you can do
it at significantly lower data rates so
again mpeg-4 really operates well for
voice at 96 to 64k bit whereas with an
mp3 we really would be up in the 164
range 128 range so you're able to reduce
that a lot more you get a lot less
artifacts now mpeg-4 audio is a
compressed format and with you know four
thousand dollars worth of speakers you
will be able to hear a phase shift with
it go ahead and say that but if you're
in your car if you're in on a PC speaker
that can't reproduce that range for
ninety-eight percent of the uses that
you will use it it sounds like a CD and
and and and basically this whole concept
that I said initially about a Kodak you
have the technology that allows you
shrink something and then decompress it
in real time so that size is no longer
the original size it's smaller now
because you're using these codecs and
again for your audio and just and
distributing your audio and pay for
audio codecs are really the key to that
the few other things you can do with
your audio such as normalize it that's
just to bring the volume up so you don't
introduce distortion and things like
that okay now great so let's
[Music]
let's go back to back to the
presentation please thank you okay so
I'm starting to wrap up here and I'm
really wanted to finish again where I
started a little bit which is this
concept of video what is digital video
well digital video is quicktime movie or
other formats but it has a frame size
than a frame rate these frame rates are
really a lot of the sizes controlled by
the codec and ultimately that Kodak has
a data rate associated with it and that
data rate really is what determines the
quality of your video great can we
switch back to the demo machine please
so I wanted to show you that again what
we've talked a little bit about here
we've introduced this idea of this great
new h.264 codec and what's going to be
happening there and really want to talk
a little bit about some of the trends
here in media so one of the exciting
things that we have here inside cleaner
we have the ability to go out using the
kinoma exporter to the palm OS again
palm OS devices are extremely good at
playing video I have my little create
hear that go ahead and play this and I
think more and more what people are
seeing is that our videos on our cell
phones so we can take pictures with our
cell phone now so it's just a matter of
time before our cell phones will be able
to actually display video moving
pictures okay so we've got the single
frame now we need to get multiple frames
there's also really exciting component
to this technology that well there's to
a certain two other components first of
all there's a thing called Wi-Fi 802
dot11 wireless technology that will be
able to allow me to view this video
anywhere and increasingly we're seeing
more and more you probably have it at
your house but increasingly you know
down at starbuck you can go to places on
just last week it was announced in
Washington DC they're actually deploying
Wi-Fi in a whole neighborhood under city
blocks so more and more we're going to
be able to take media wherever we want
now if you capture a video on your cell
phone realize that the resolution is
this size it's going to be 160 x 120 so
we're going to have an interesting
period
time when we try and take that video and
scale it up to full size but again we're
going to have very exciting codecs like
a chat h.264 that will help us make that
transition so again what's very exciting
about that is what we're happening you
know where this media is going and
increasingly what we'll see is that
you'll produce your video in these
high-end format perhaps HD SD very high
data rate a lot of creative stuff
happening with that you get support
where you need to distribute that media
and oftentimes you're going to
distribute it in multiple formats you'll
make a DVD of it you'll make a web movie
of it you'll make a cell phone version
of it and again the idea here is that
we're distributing this in multiple
formats so you're going to make a master
version of that and you're going to then
from that derive all of these other
versions at these lower data rates much
smaller sizes okay we can go back to the
to the demo machine are to the
presentation machine please so again
we've hit some basics today and just
sort of discussed what quicktime was
quick time is a container for your media
that's extremely important that it's not
just a codec it has codec attributes
associated with it this is the reason
why I have to draw that contrast as
there's some other formats in the world
where you don't have this level of
technical complexity and dis amount of
tools and really rich features to use
where the codec is the name of the
format so I shouldn't have to say what
they are right thank you okay good so
again quick time and to cook up really
codec base so it really gives you that
frame size that frame rate codec and
data rate and that again data rates
determine the plaza how you produce it
how that image quality look and that we
have different codecs for distribution
than we have for production and again
the codecs for distribution are very
small the codecs for production are very
high we have all these frame rates
different sizes of videos that we
collect from all over the world we bring
those all into our we digitize those on
our computers assemble those and then
upload them it's still more smaller
sizes in different format so again lots
happening with the media I think that
we're going to now take some questions
people are interested in that
and again what we do is we have first of
all who to contact if you have any
questions about the topics that we
talked about today or just quicktime and
quicktime technologies in general a few
very important names you should know
first up is Guillermo Ortiz he's a
developer relations person for quick
time and again wonderful key person to
know for any of your questions you have
regarding the QuickTime and the media
track here in addition my colleague
Stephen Stephen is available he's a
product manager for QuickTime available
for questions and follow-up and then
myself I'm Hagee Van Dyke I'm available
at here at discrete calm and my role is
I'm a applications analyst so I'm really
a person who works with all phases of
media production and delivery and
actually product development delivery I
work for a company called the screen at
a certain level we're known for a
product called cleaner which is a kind
of transcoding tool designed for
compressing video from one format to
another but discreet also has a real
legacy and in history in high-end media
production we have a product called via
the flame which is an SGI based of
real-time independent resolution system
designed to give you very high quality
motion effect and again what's
interesting about the flame it does deal
with still image formats so again large
stuff and we're going to take some
questions I believe