WWDC2004 Session 221

Transcript

Kind: captions
Language: en
hello everyone and while
so this year's image capture show or
actually by just looking at this slide I
said well we guess I guess we have a
problem in showing real pretty cool demo
later on because this side says we are
referring we have to refrain from taking
pictures and I guess we haven't set up
everything here but I guess we just file
it through so let's talk about image
capture we will start with a short
overview what image captured us and the
easiest way truly to describe image
capture is really if the framework that
allows you to work with still image
capturing devices the camera or scanner
and the nice thing really is we allow a
vise range of devices and we support
multiple communication protocols it's
funny like couple of years ago when we
did the first image captured session we
were like forcing you're at least asking
you as device manufacturer well wouldn't
it be great if we had a few maybe just a
handful of protocols that you can use
and the really by just using a handful
of protocols this is in a handful of
driver or device modules and be able to
talk to as many devices as possible the
industry trend has really shown that
most of the new devices are really mass
storage or PTP picture transfer
protocols based which is really really
great and so later on in the session we
will show you a couple of new AP is that
we are going to add four Tigers to make
the usage of new devices new device
classes even easier let's start with a
short overview of the image capture
framework and it's pieces at the bottom
level we have the hardware the device
that you connect and this has actually a
device module that is responsible for
talking during the communication to that
device
this device module is a user client
background application for you there's
no need to add any kernel driver salsa
and it talks to the image capture
extension image capture back on that the
image capture extension can handle
multiple devices at the same time and
can actually handle multiple clients at
the same time so in theory you could use
iPhoto you could use the image capture
application file maker or whatever your
application for example application and
then at the same time access the single
device this is bit different from if
you're familiar with the train model
we're all we have a layering stack
that's very simple or very similar to
what we have but they all run in the
same process we have different processes
and really the image capture background
application as a bottleneck will hear
your life the device requests and really
make it easy and straightforward for
client application to talk to like one
or multiple devices at the same time now
if we look at this structure here
actually everything inside image capture
is made up of very small building blocks
first building block is the notion of an
ica object 98 object just has a type and
subtype and may contain a reference to
another icy object and it may contain
properties or actually a collection of
properties metadata in the dictionary
which leads us to the second bullying
clock that's the properties as a
properties here again we have a type and
subtype and these properties contain
actually the real data with real data we
mean the kind of image data and
available thumbnails its image data or
if the device support
like mp3 files or the movies then of
course we have the music or the movie
data and the last thing we added for I
guess Jaguar was the addition of
dictionaries when we came up with the
initial design of image capture we were
thinking well we could put all the
metadata all the properties in two
separate objects I say properties it
turned out it's kind of cumbersome to
get all the metadata collected so it was
much easier and you will see that in a
little bit much easier to use CF
dictionary refs or or NS dictionaries
that you can pass round and they will
contain all the metadata for example for
a given image so all the excess data
will be in one of these dictionaries if
you look at these three building blocks
they form kind of a unit and so once
again the object has basically type and
subtype the properties has the entire
image and Sam Dale and the dictionary
contains the this case access metadata
that allows you to work with images in a
very convenient way out of these you can
build up your tree and the top-level
object that you see here the device list
is actually always available even if you
do not have any device connected what
was new with Panther was the addition of
an dictionary to that device list object
that dictionary of course does not
contain any metadata because there's no
image for the device list object but
what it contains is a list a reference
to all devices that are connected so to
cause gives me the device list and then
give me the dictionary for that device
list if you do these two calls
you will have information of what kind
of devices are connected and again by
just using these building blocks that's
what we do inside the image capture
framework so whenever for example I
photo wants to talk to a device first
thing as all other clients have to do
first thing is really gives me the
device list and then query the device
list are the object for example gets the
dictionary get out of that dictionary
the number of devices for each device
you can get the dictionary and that
dictionary actually will contain either
flattened out or as a tree both variants
are in there they're all the images all
the files that are on that device now
you can use that on a single machine but
what if you have to machine well of
course you can have the same thing
running on two machine but with Tiger we
were introducing file or a device
sharing for image capture devices it
could by just setting up the devices
share on one side and listen on the
other side you can take the standard
camera or scanner and there's hook it up
to one device but use it on the second
device the detection is fully automatic
using round able and you just pick and
choose but need more cooler thing on
Panther was the image capture web server
I guess this was a like best kept secret
for for Panther because a lot of people
were later on and review ously we're
seeing well I didn't know about that
feature it's really cool feature the
image capture web server is the Faceless
background application a regular client
for the image capture fright of the
inscription framework and what it does
is it takes the attached camera and
publish that over the next
so what you could do you can connect at
home you can connect the camera and from
work you could actually look at the
images on that camera or if you have a
camera that supports PTP and allows you
to take pictures you can switch to a
mode where it actually allows you to
take pictures over the internet and it
has a monitoring mode so that every so
in so many seconds it will automatically
take a picture and display it and that's
a really convenient way to get to your
data now that was really the kind of
quick waters image capture and put up
the image capture component one thing
that we are really handling and the
image capture framework is devices that
are connected via USB and firewire we
handled it pretty well because like out
of the box just fresh installs megawatt
spin you connect the camera and whoops
the camera shows up the iPhoto launches
or image capture application launches
and allows you to download the images so
this hard plugging is done through an
icy notification framework completely
invisible to the users but what is
happening really in the device gets
connected at i/o hit level we learn oh
there's a new device we do a matching
based on information that we have inside
the device modules and we then launch
actually the appropriate device module
well that's all create if you have USB
and firewire devices but what about
devices that are none hot pluggable well
up to now that was the problem and for
Tiger we want to have a solution for
that so we really see that there are
lots of new devices for example cell
phones coming out that have Bluetooth
and the buildin camera
what about these devices or PDAs some of
the PDAs has a built-in camera others
have just attachment so you can plug in
the camera and in fairly decent
resolution camera nowadays and one thing
that has probably a very bright future
is the the area of wireless cameras so
by end of the year or early next year
you will see a lot of new devices
showing up that have wireless support
directly built-in so these devices well
they may use rendezvous or may use like
worst-case just met it feels very type
in IP address but there will be away and
there needs to be a way to use these
devices as well and of course the same
thing for Scotty based devices good old
skazhi non hot pluggable I mean you have
to really make the connection upfront
before you turn on the machine and those
devices it would be nice if we had a
simple way for the user to use those as
well so we in order to support these non
hot pluggable devices we have to
introduce new API and it's enough to
just introduce like an icy a load device
module and unload device module for the
load and unload you can imagine then we
probably don't want everyone to write
code to do that so the idea is to add a
device browser to the image capture
application so you would go to the
device menu image capture and then you
could bring up a prouder to select the
non hot Blackwell device that you want
to use and ideally another three
dependent on request so if you would
want to have something like a common
panel common UI for that we were
thinking about implementing that as well
that would just allow you again to do
the device browsing and in handle the
connection so if we look at the ica load
device module you see we have same
structure of course it's just parameter
plug and an optional callback proc if
that callback progress mill is the
synchronous call otherwise asynchronous
call and then the ICA loaded device
modules the parameter clock has decide
the header that's in all parameter
blocks we have it has fields that allows
you to specify a device module via URL
and we have no s type that specifies the
transport type so that could be
Bluetooth it could be USB firewire
whatever and then a set of parameters
that are specific to the transport type
ideally you wouldn't have to worry about
the device module URL so you could just
pass in now you specify the transport
type and then you specify a Bluetooth
address that you get wire any iokit call
the unload device module is a lot
simpler because all you what you have to
do is really just pass in the ica object
that you want to unload so let me show
you very quickly how we do that in image
caption we please switch to demo 5
so I will launch again a smaller helper
app that shows you really what's going
on on the image capture side it has
entries for all image capture components
and we'll put out a check mark next to
the name whenever one of these
components is running so let me launch
image capture you see image capture with
was launched get the check box but also
the very first entry the image capture
extension image capture extension is
running now but we do not have any
device module well there's no device
connected so let me go to the device
menu and we see we have a connect item
in here and what will happen and this is
really not the final you I but that's
just a temporary you I to show you what
direction we are going is it will list
in this case the Bluetooth devices that
I have right next to this machine and
that's a PDA and the phone down here it
shows you similar information that you
will get from the regular Bluetooth
control panel so let me select the phone
and let's do a connect see what's
happening the phone works recognized and
yeah we do see the items on the phone
and we can download those and by the way
nice is saying we added 4014 tiger
whenever you want to specify a download
folder you can actually drag it up here
and that should be short cut to the
Proverbs or very easy to get to the
download folder and here we could
specify preview to open it and then just
try to download and you see here is the
image that we have from the phone
okay so that was the like manual way of
loading the device module let me quit
image capture and shoot you another
small application that will actually
hopefully make it to the SDK that's an
application that allows you to just test
all the image capture api's and you see
on the left you see all the image
capture aprs we have and you see down
here the new one the IC unload device
module and appear you see what it takes
a parameter there's only one input
parameter that's the icy object and if
we open the tree so where do we see on
the left side here at the right side
here we see all the objects that we have
on the device we have up front the
device list we have the device itself
and then we have three images we could
now select the device object copy that
and just paste that in here and then
execute the command either synchronous
or asynchronous way and then you see the
device is gone and the device module as
well image capture extension will quit
in just a bit can we switch back to the
slide please
so that was just a quick demo loading
unloading device modules and really just
a very first preliminary UI for doing
that so if we look at this cell phone if
you want to really use it as a digital
wallet it's really nice way to say hey
that's my cat that's my kid that's my
dog and the cell phone I said we'll
probably have a way to take pictures or
download pictures are there something
missing so what about uploading well
again wouldn't it be nice if he could
upload to a device and if you look at
the API said that we have for image
capture used to have is set property
data or the set property data what you
would think that this is the right way
to do the upload unfortunately that's
not just enough because we just don't
want to set the actual image data we
probably need a little bit more
information and so we will have a new
API to handle the uploading device
uploading and it's I say upload file and
it's really just more than just the file
copy well it's a file copy call for non
image files but for image files it also
allows you to scale the image
and the parameter block for this looks
like that we specify a parent object and
the file of a stress and a couple of
flags there's one thing about the parent
object that's just the kind of suggested
parrot object so really depending on the
device you may not be able to really
upload to the specified folder so it's
always good I mean you could try it it's
always good to just upload to the device
itself and then normally the device
takes care of the positioning is within
the device file structure so let me just
give you a quick demo of the uploading
switch back to the same applies
so
we will connect to the device
and now you have two ways of doing the
the upload one way is to drag it to the
icon or whenever you are in the download
mode just drag it in here and then you
get a sheet popping up that allows you
to specify the option you want to scale
it yes or no yeah in this case we want
to scale it and see actually it suggests
the size it knows that this device has
the screen size 120 x 160 but actually
the device also supports this as native
size also to 88 53 52 so it could choose
either one let's stick with the first
one and do the upload now really
depending on the speed of the device you
see we got new entry in here and it was
scaled to 107 by 160 the image the
original image was larger what about
this and you see it was for 29 x 640
and again show that using the RT api
test the upload file or we just specify
again a parent object and in this case
or we want to do is even upload the text
file and if that works then we should
see it takes one the theory we should
should see it on the right side it
should pop up when waiting for that but
far too long okay can we switch back to
the slide please
another area that was introduced new
feature was introduced with panther with
the fast user switching for image
capture there were really no problem if
you had a scanner or a mass storage
device connected message device was very
simple like all your firewire hard
drives you would just expect that it's
good work if your user a and switch user
D and then both users should have access
to the device for a scanner it was also
not a problem because scanner we from
the beginning had an icee a scanner open
session in closed session and there was
only one client allowed for a session so
that simply means if you are user a you
are working with the scanner you have a
session open you switch to user B and
there was no way that user B could get
the scanner if you are user a and you
are done with your session then user B
will have em easy access to the device
four cameras that are non mass storage
for PDP camera we do not have the
concept of a session so what's happening
is really user a uses the camera this
which to use a be now the question is
what should happen should user A or B
half half the camera well instead of
just like unplugging the device and
reconnecting the device which actually
would give user bxs we want to have a
better solution and what we are going to
do is introduce a session handling for
cameras so we won't have an open and
closed session and these calls are
really similar to the ica the scanner
open and closed session but they are
different because they are optional well
we do not want to break accessing apps
so the current iphoto should just work
the current image capture app for my car
and your f it should just work and the
other thing which much different is
really the number of clients well for
cameras there's no no need to limit that
down to a single client so multiple
clients can still use the device
so these optional calls are just to make
fatties of switching a lot easier what's
a cup just then for example whenever the
future iphoto is done with importing or
you're not at the import tab then there
is no need to plug the device from a
second user flow IPA open session just
take the device object and returns a
session ID I see a closed session just
takes the session ID that you got on the
open call these are the main changes we
have an API was for for tiger let's
quickly look at this kind of support
scan aside before we go to some real
camera related demo so on the scanner
support since the Jaguar we were
supporting native scanner modules and
also clean data sources for tiger sorry
for Panther we added support for
transparency units and also add it very
simple to use UI that allows you to do
some image enhancement is automatically
or with a manual correction and for
tiger we are going to support something
that was also requested by a lot of
users the support of a document feeder
and let me just show you that on demo 6
so this is the standard image capture
you I if you have a scanner connected
you see in this case we are using the
train bridge which means we are using
Twain a train data source using in this
case the absinthe in a data source that
is just rerouted through train bridge to
image capture and since this device here
supports as a Twain capability a
document feeder we will modify the UI
and add this extra checkbox here so you
can select the document feeder then the
main view will change because when you
have are in the document feeder mode you
are probably not in this workflow where
do it priests can first then select your
scan rectangle and then do the final
scan what you want to do in the document
feeder mode is really scan the entire
page and do it quickly and do it often
so again this is probably not the final
you lie because the one thing that you
could not select right now is let's say
do you have an 84 or us letter us letter
or you is legal so we need some way to
allow the user to choose between
different paper format right now in this
demo we will just scan the entire
document no put the scanners working
what was probably in the sleep hopefully
for exact
light is flashing and yeah it is
scanning the first page so this is not a
professional 24 pages or 60 pages per
minute scanner this is just a low-cost
easy to use kind of home type scanner
that you would use in order to 24 for
scanning and multiple document see
what's happening the first page of scans
in the background preview is launched
and as soon as a page comes in now it
will be added to preview and yeah each
page will be displayed actually want one
nice thing you could do in preview since
these pages come in one at a time what
you can do in preview you could set the
preferences for images to say well
create only one you don't want to create
the particle windows disk only create a
single window and then all the pages
that come in will be added to the same
window then for example in preview could
do a print and print to PDF for the a
PDF document of whatever you have okay
so i guess we're stand with default
pages yeah also new just one minor
remark we in the past we were just
generating like a critical name based on
scan and then date and time in this case
you can really specify a prefix and we
will just add a page number to it well i
think it's a little bit user friendlier
than before okay back to the slides
please
so looking at the device developers what
do we have new for them or first of all
something handling we were introducing
an API I see a copy objects and nail in
Panther and the initial support was for
I see a sample format with tiger we are
going to extend that and really allow
you to download the family it's directly
in jpeg or tiff format and for the
developer of a device module this means
well not that much of a change because
you can directly give us the native
thumbnail that you have or crawl back to
the old way just returned and I see it's
anthea so if you are not going to change
that's fine because we will do the
conversion between I see a thumbnail to
jpg or tif so the user will get whatever
some news format you see 11 invitation
we had in the past was with isolated Sam
Hales was really the 128 x 128 pixel
limitation and we want to loosen that
little bit and allow really as described
in this female 15 minutes a body allow
larger thumbnails and up to a maximum of
160 I one funny
one thing that we got a lot of questions
about because I get the documentation
was lacking a little bit on that whereas
what's up was the info.plist device
info.plist well for a device module that
we have on image capture we need since
this is a regular bundle we need then
folded penis that has all the things
that are required for info topi lists of
bundle executable I can name and these
things but we needed in the past also
information that would allow us to do
the matching of the hot black event with
this device module and this information
this matching information was kept in
the info depillis we in addition needed
the device input of Peters which was in
the resource path of the device module
and that divides info the plist was
containing information about the current
profile or device icon for example for
Tiger we are going to change that we
will have all that information just
combined in a single p list that's the
device in philippi list you still need
the info to penis but no no image
capture related entry in there and it
will be the same location as the current
divided for the plist and really combine
into the entries of of two plus some new
entrance we will for example specify in
that info the penis we will specify the
physical transport or media information
for example just this device support
jpeg tiff movie avi whatever format and
it sizes and you already saw why we are
going to do that you saw on the upload
we have this proverb that was giving us
science information
so image capture uses really this for
the upload panel and allows the user to
choose the kind of native slice it has
information with the screen size of
devices or sizes that that this device
produce and since we are also explaining
that to add the device capabilities we
will have the information for example is
this device able to handle uploads of
JPEGs is this device able to handle an
erase or take picture let me just show
you the current format on on number 5 so
if we dig into the device modules so
they are in system library image capture
devices and here we have a whole set of
of devices so if we show the package
content go into the resources here we
have the device input appeal it and now
this has some USB related information
and some device related information and
now you can go in and look at all these
entries or instead of using the property
list editor we can also use a small tool
that we are also going to add to the SDK
the device in for the plist viewer and
this allows you to dislike double click
on for example the PTP module and what
is happening is just collecting all that
information actually it looks up the
icon and displays that as well and you
can pick whatever device you have
and go to the device and you see okay
this one supports jpg and these
resolutions this one supports download
and erase it does not support capture
for example and we see the protocol
level it's there all PDP cameras these
are all the defenders so see all main
vendors from Canon Kodak nikon have and
Sony they have PDP cameras that we do
support and this is just a convenient
tool to get to that information and as I
said we are using these sizes heavily in
the future okay back to the slides
please
and finally I come to the main demo part
of this session and the idea was to just
give you kind of ideas of new
applications that it could do with edge
capture first thing the high dynamic
range images for those of you who
attended session 207 on Wednesday they
learned everything about how dynamic
images you probably will ever need to
know Luke and Gabriel are doing it as a
great job and introducing that and to
just keep it very short one of the
problems you will have and get rid of
showing that if you look at the apples
garage and take a picture of that scene
you see with the short exposure you get
pretty much good detail in the back
everything in the front you have no idea
what's that if you have a too long
exposure you see all the cars these are
not being w snow okay and using the
secret sauce of tone compression you can
take multiple exposures and do it on
compression and you really get a result
that's like that so what is the best of
all worlds up to this point it was kind
of complicated to take these high
dynamic range images because you had to
go out put your camera on a tripod and
then take an image change the exposure
time take an image change exposure time
it is that a couple of times and then
run that through an algorithm well if
you have a camera and that allows you to
do that automatically you could just use
image capture calls to really do one
quick high dynamic image rather than
Grange image capture I really wish I
could say that he could go out and take
any camera and you could do it but now
we have one
and really hope that next year more
vendors will open the protocol 3 support
more of these kind of things so the
Canon d70 that's the one we are going to
use now and oh sorry nikon are correct
okay what i want to do is create a scene
here for a camera that's kind of
difficult to capture and we have a
bright light the box there's something
in the box and the camera are just
pointing to that so let me turn the
camera on and you see these 70 is
recognized and what we're going to do
now is we take the first image this is
automatic exposure and we see underneath
the image it's the exposure time so what
we do is we take out of the metadata we
display the exposure time so basically
verify that whatever we set the exposure
time to before we do the take picture
these times are really in the image we
have to still have it here and now we
take a couple of exposures and then take
theta Gabriel's secret sauce and combine
that and it's already done because the
progress bar is it stopped moving and we
will be able to open this that's the
current
time
and we capture the scene now if you look
at this it's actually pretty amazing
because what we will see up here we can
actually see the inside of this land
desk lamp but we can also see inside the
box it can actually make it a little bit
brighter and yeah we can really see
what's inside the box and also see
what's inside the land nice thing these
are really floating point images so the
information is really not frozen to this
or whatever you see now what we suit
will always have the slider to go either
into the dark or in the lighter side
so just give you an idea how these
images look without this processing well
that's what you normally see as soon as
you have the Tigers visible if you
really try guys coming heavily and yeah
you can see the Tigers but you have no
details in the in the desk lamp or if
you have the detail on the desk lamp the
tiger is done
so this is a pretty nice way to capture
high dynamic range images and really
it's it's simple to do the only thing
you have to do is click one button and
it happens automatically okay so the the
next demo is timeless and here i guess i
have to thank all of you because you
were all really contributing to that
thimble without knowing though so let me
just show you what did it yesterday
so we took of time that off from
yesterday afternoon till yesterday late
night but the campus event see this
group down there they are quite busy
the clouds come in getting darker
I Fallon show begins even the moon come
back
in the back you see scent of the airport
or eppinger coming in
ok I guess but now leave the last one
scar all right
who's right
okay so ever so how did we do that well
the small application sitting on top of
image capture that just has two sliders
very easy to use you can specify the
time if you want to capture this case
whatever 12 hours and how far do you
want to decompress it so if you want to
play 12 hours within 60 seconds you take
this setting otherwise yeah you can
change it like say an old a 24 hours and
want to play it back in like two minutes
and then just hit start and it sort of
an he takes the images creates the movie
and two hours later you're that also is
obviously that you're done okay if you
have a little bit flat time so last
thing I want to show you is just for fun
a small application that actually allows
you to take 3d images and the idea is
very simple if you have a camera that
flips and so basically the distance of
the eye and we flip the camera and just
doing that we could would mount it on a
tripod I just don't want to do that
because II already have those images
captured connected device and then
bring up you I that we'll just look at
the images on the device and the nice
thing since there's a sensor in the
device what really showing whether you
wrote rotated this way or that way we
can please slot the images and actually
pre slot them in a way that whenever I
change one image here you see that the
one on the right side changes as well
because they are kind of grouped by date
so we know it's the left and right side
and then we can very easily crew bit and
yeah just select whatever you want and
then you could say process and then what
should happen is wait two large images
and what should happen is it will
display like that and you can do some
fine adjustment and using one of the Red
Queen classes you can look at it and you
have your 3d image also very simple to
do and without any any extra effect okay
go back to the slides
we see we did the demo and we are now up
to QA
you