WWDC2004 Session 615

Transcript

Kind: captions
Language: en
hello everybody this is the second of
our 2 X and sessions X an in-depth
basically i'm greg vaughn and i'm going
to be talking of a lot of the same
material that was in the overview
session how many of you were here for
the overview session okay good i'm going
to cover some of the same material but
from a slightly different perspective i
won't have any product prices up here
stand a little code samples and
hopefully some more technical
information going to start actually
going over some of the same stuff with
the different types of file systems just
to separate them out and make it clear
sort of in terms of file systems what
makes a sandpile system different from
other types i'm going to talk
specifically about x am and sort of the
communication protocols and how it's
really working i'll go into a bit of
depth on the x an admin will show a demo
of setting up the sand some of the other
features of the admin i'll talk about
how the volumes work in terms of when
you're writing apps there's some
different characteristics both from
local volumes and network volumes you
need to be aware of and then finally
I'll talk specifically about some
developer api's that can be used in the
applications to make them work better
with x and b more x and we're so
starting out with the different file
system pipes Tom mentioned
direct-attached storage which is
basically you know your traditional
local hard drive just some new fancy
names for it the way in which a local
hard drive works is you've got your file
system the drives presenting just an
array of blocks the job of the file
system is to organize that data using
the catalog present a higher level API
up to applications so in a typical
diagram you've got basically at the high
level API you're dealing in terms of
files the application is going to open a
file write to a file use
biol offset the file system is going to
translate that to the blocks on the desk
this is the technology that's been
around for decades but of course a
block-level device can't be shared by
multiple computers because there's no
way to keep the catalog in sync between
the multiple computers so this early
limitation got solved by the network
attached storage or the file server
network attached storage basically is
just taking the exact same file server
and put it into a box so the underlying
technology is exactly the same the idea
here is you're going to take the
high-level call ship it across the
network to the server and the server
will perform the action so you've got
the same diagram you're making the call
at the file system layer the network
file system directly mirrors that call
and on the server when it makes the call
you're doing the offsets to the but disk
this will allow you basically to have
the volume integrity but at the price of
bundling all the data over the network
to the server and all funneling it
through the single machine the other
slight thing about that in dealing with
file servers is when you scale them up
you tend to have problems with the
different types of requests especially
metadata intensive requests doing large
directory listings and so forth cause
the server to load catalog blocks off
which can interfere with the streaming
of data off the hard drive and
conversely those metadata requests can
be locked behind the large iOS and when
you're dealing with heavy file servers
you see the big Layton sees and like
opening up a little directory listing
and that those are really hard to
overcome those sort of scalability
limitations and file servers raid solves
the other part of the problem you're
you're trying to overcome the
limitations of disk speed by combining
the multiple drives in addition to the
performance you're giving the
reliability of redundancy between the
draw
I've important thing about the raid is
that it's happening sort of behind the
scenes of the filesystem the file system
isn't aware of that the raid system is
presenting the single Drive to the file
system and the fight and the raids
mapping the block offset internally to
the multiple drive and you've got the
different raid scheme I always get them
confused specially raid 0 and raid 1 but
raid 0 you got the striping mirroring
provides the redundancy and then with
raid 5 you've got both the performance
of writing to multiple drives plus the
redundant data so that if anyone drive
fails it can be rebuilt easily without
losing access to your data in addition
to the underlying raid you've got
software raid which happens at the
driver level to distribute do your data
out to the multiple drive and then once
you've done that then the raid box will
further distribute it out to the
individual disks but the point of all
this is that the raid system is all
happening down at the block line device
the file system is still your
traditional file system be it HFS or ufs
that is just dealing with what it sees
as a single array of blocks and it's
maintaining the same catalog data as it
always would so it's just doing the same
translation down to the disk offset and
then down at the raid level that's being
distributed out to the multiple disc so
the problem with that is you've still
faced with the file server if you want
to distribute your dear data even though
you've sped up the hard drive all the
data is flowing through the one file
server and that is your big limitation
so how the fan file system really works
to overcome this is by separating out
the notion of the catalog from the data
there's no real reason why the catalog
needs to live with the data once you've
decided a one particular place to store
your catalog you can have a special
purpose server that is going to just
deal with the catalog and do part of the
job of a traditional file system
basically update the catalog and figure
out where on the drive the data is going
to actually live that way the client
file system can talk to the metadata
controller to get the catalog
information but then do the i/o directly
to the raid devices so here's your
typical XM setup diagrams a little
different than Tom's diagrams but it's
got all the same components got your
client system I'm only showing one here
a couple raid boxes and your controller
got everything hooked up with the fibre
channel and you've got the IP network
between the client and the controller in
this particular case you've got the
normal raid configuration for the
example because you've got the two
controllers we've have the two virtual
disks / raid 2 total of four lumps that
we're going to group together to be our
extra X and volume so the first thing
you do is you select one of the lungs
that's going to store your catalog data
you can decide to either dedicate that
to store in the catalog data or you can
choose to store other data alongside it
the point is though you don't want the
other day to be stored with it to be
high performance data because again
you'll start to run into the same
limitations yeah but with a file server
of the data competing with the catalog
information so even though you might
want to store other files there they
should be less accessed files then
you're more performance critic
one so in this example we've chosen one
one to store our metadata basically in
setting up your volume all you're really
doing is configuring the controller the
controller is the only machine that has
a real notion of what comprises the
volume so you tell the controller
basically what all the lungs are that
compose the volume and I'll go into a
bit of additional information you can
give it in terms of how to construct the
volume out of these ones but first to go
over our our same old example you've got
the client application making the file
level call at this stage rather than
shipping the whole call off to a file
server as it would in the normal network
attached storage case it's just going to
take the make a request to find where
the data for this file live but keeping
the actual data on the client system the
controller is going to read the catalog
information out of its private metadata
storage on 11 and then reply with the
disc offset back to the client at this
point once the client has dedicated
storage on a particular lon it can talk
directly to that one and stream the data
across without worrying about corrupting
block offset the other thing is because
you've got a collection of lunge you're
not only telling the client the offset
that you're telling it which lund though
the actual data lives on but other than
that the client has no notion of the
actual sort of file system layout on
these loans how they're divided into
files where they how the catalog data is
structured or any of that all it knows
is this is the data that this is the
area where it wants to write the data
the application is given it
the other thing you can do as Tom said
is group the lungs into storage pools
which has an effect very similar to
striping and software raid the big
difference here is even though you're
getting the same performance effect of
being able to talk to to lunds rather
than being handled by a driver on the
client system it's the metadata
controller that's going to control the
access to these two lungs so in this
particular case the clients going to ask
for that same file offset and the
controller is going to tell it that part
of this file lives on 13 and part of it
lives on 14 and the client is just smart
enough to know oh well if I'm writing to
do different ones I can stream the data
out simultaneously so you get the same
performance effect that you would and
software striving the other main job of
the controller is handle the file
locking the with the X and because
you've got the the controller handling
the catalog data you don't need to worry
about file system level corruption but
still within individual files you need
to worry about data corruption that's
the same as any network file server and
the way that hat is handled is at the
application level you need to make
locking calls this works it's the exact
same locking calls you use for NFS or
anything else but it's the controller
that needs to keep track of all those
locks and actually arbitrate between the
different clients so it has that role as
well and then other to that as I said
the clients are just writing data out
wherever the controller asset to and to
the client applications they just see it
as one big volume and they don't really
know about the lungs behind it
the other thing as mentioned before is
to failover basically how this is going
to work is when the clients notice that
a controller is down they actually get
together and vote for a backup
controller and you can configure it such
that it fails over in it in a commune Oh
predictable way the backup controller
comes up it knows where to read its
catalog data from reads out the catalog
data the journal file system so it's
able to look at the journal and
reconstruct the last few transactions
very quickly come up and start running
we're still doing some of the
performance benchmarks but as Tom said
it should just be a few seconds I think
15 seconds would be sort of the outer
limit the important thing to note is
during this time 15 seconds can be a
long time in terms of video streaming
but the clients don't always need to
talk to the metadata controller once
they've asked the metadata controller
and gotten the offsets for their files
they're dealing with directly with the
raid box so if they're streaming a file
off the raid box and the metadata
controller fails it's possible the new
one will be up again before they even
notice it's gone before they have a need
to actually talk to the metadata
controller again in which case you'll
have uninterruptedly the other thing the
clients need to do is once the
controller comes back up the new
controller of course doesn't know about
the locks the clients have taken out so
when the clients see that the new
controllers up they need to go and tell
it about all the locks they have in the
controller will rebuild the lock table
so in terms of volume configuration we
talked about the various ways that you
can group your lunge together to build
up your whole volume the first thing
you're going to need to do is to pick
which lun your metadata is going to be
stored on it's going to be both the
catalog information and the journal
technically you can configure those to
be on different lungs but usually
there's no reason to do so so in our
admin software generally they'll both
always be stored on the same line the
other thing you're going to do is decide
whether you want to store other data
files along with the catalog data
basically the catalog data doesn't take
much room so you can either partition
your raid in such a way that you have a
very small LAN and make it exclusive for
the catalog data or as we have a larger
lund you may choose to store other files
alongside it the most important thing
about the lung being used for the
metadata storage is that it be safe
storage basically if you lose your
catalog pretty much it's hard to
reconstruct your file system so it's
very important that this be mirrored
storage whereas the streaming
performance isn't so important because
we aren't talking about huge amounts of
data here the more important thing is
the random i/o performance catalog data
tends to be read in a very random order
so it's very important to have a very
responsive drive the key part of the
performance for the whole fan system in
addition to the streaming performance is
the responsiveness of the metadata
controller I mean and all the questions
about the requirement for an IP network
basically what we're trying to do is to
make that round trip time to the
metadata controller as fast as possible
so the other part of that is making sure
the storage the metadata controllers
talking to is very responsive and like I
said it's not very big ten million files
will probably only use up about 10 gig
of space for a catalog so you don't need
a lot of storage there but
high-performance storage
very important so once you've got your
metadata controller the question for the
rest of the storage is how are you going
to group it into storage pools certainly
if you take all your lungs and combine
them into storage pools the ideas the
client could talk to all the lungs at
once and theoretically get you know very
high performance there's a few
limitations to that one is you want to
make sure all the lungs have the same
characteristics because it's effectively
like software striping when you've
combined all these ones together you're
really going to get sort of the smallest
lund and the slowest lund will be the
gating factor for the whole storage pool
so you really want to take all your
identical lungs and group them into
storage pools but there's also another
aspect of storage pools and that is even
though everything's combined into a
volume once the clients done talking to
the metadata controller it's only
talking to the lungs on the storage pool
it's dealing with at the time and that
won't affect people at all if they're
talking to different raid lawns on
different rate boxes so there can be
benefits in segregating your data out so
that for instance an in gestation is
talking specifically to one lung and if
nobody else is talking to that lung even
though they may be talking to storage
pools using other loans they won't
affect the performance of that one
person that's something that only when
you're configuring your your ex n system
you know only you know how the data will
be used so that's a bit of a there's
some compromises they're involved in
setting these up and especially if you
have lots of lungs there's lots of
different ways you can do that and the
other side effect of that is that you
are going to end up then with some
storage pools that are faster and other
storage pools that are slower so as was
mentioned in the first part the problem
is because
the applications normally don't know at
all where the data goes which storage
pool gets stored on if you are setting
things up in this very particular way
you need to have more control over that
the default is that the controller will
every time you create a new file it'll
just go round robin around the storage
pools and create a file on each one
affinities are the ways you force the
files to be stored on a particular
storage pool the most common way to do
that the ones supported in our
administration app is you create folders
and basically say this folder items in
this folder will always go to this
particular storage pool the only
exception to that is if a storage pool
full fills up you'll still be able to
write into that folder it'll just then
randomly go on to another storage pool
and start putting the files there at the
finder level you'll just see the one
volume and how much room is available in
the admin you'll actually be able to
look at each storage pool and see how
much room is available on each storage
pool in addition to the administration
stuff of mapping folders there's both a
command-line tool that all talk about
for refining signing affinities
particular files and then finally an
application API where the application
itself can choose so now talk a bit
about our administration software and
how that works and how you set up ass
and using the administration software
we've tried certainly as we do in all
our products to to consolidate this and
sort of make it easy and understandable
as possible although certainly SN is a
complicated thing and there's lots of
different aspects there's always
trade-offs then in terms of giving
access to functionality versus making it
sort of easy and you know
straightforward to use so here's just a
few slides shot of what a setup looks
like the first set in the setup is to
you got to define all the
genes that are going to be part of your
sand both the clients and the
controllers it'll find these machines
over rendezvous it actually detects
athletes machine what lungs it looked up
to so it can decide which machines are
actually on the same fibre channel
network and then it'll come up it allow
you to select these machines and say yes
these are the machines that I want to be
part of my XM system you then enter the
serial numbers for those machines
because you bought a separate X sandbox
for each one you'll have a separate
serial number to enter each of those
machines and then you choose whether you
want them to be clients or controllers
and for the controllers you decide their
failover priority you can actually make
them all controllers if a machine is a
backup controller Justin standby even if
it's normally used as a client editing
system there's really no there's no
problem with that unless it actually
becomes the controller just because it's
set as a backup controller there won't
be any performance degradation and the
license allows X once you've installed X
down on system you can either make it a
client or a controller it's your choice
once you've done that you need to
configure the storage basically this is
you decide how many which volumes you
want what the storage pools in each
volume are and then what lungs are part
of each of those storage pools and then
once you've done that you basically
select the volume and tell the
controller to start up on that volume
and soon as the controller start up the
volume saleable to be mounted on client
so at this point oh actually no few
other things that does in addition to
setting up the volumes you can set up
certain administrator notifications you
can set up email or page or
notifications if storage pools fill up
or you have certain failures
and users exceed their quotas you also
can mount and unmount volumes on each of
the clients you can see you know whether
clients currently have the volume
mounted you control you know when they
mount and unmount the volumes you can do
this all from the one centralized place
you can set the user quotas or group
quotas you can view logs on the various
system and you can create the folders
with affinities as I said before so now
we'll have a demo of the various admin
functionality all right good morning
everyone you all got your ex and
developer preview CDs and hopefully you
installed it in try to play with it but
unfortunately without us and it's not
terribly interesting now I have a sand
here set up for for your enjoyment and
I'm going to make the light to blink and
I'm going to make everything happy so
the first thing to do when setting up
your xam system is to determine which
computer is going to be your meta data
controller so let's go ahead and set
this guy so we entered the chill number
before because I don't think you want to
watch me type in the serial number on
all these computers we set the role to
be controller I mean if we had multiple
controllers we could choose the failover
priority also you want since you want to
be on a private meta data network if you
have a dual nic machine or multiple
ethernet cards you could choose which
interface to access the fan easily right
there and here are some information
about the computer to help to choose
which machine you want to be hit
controller next you move on to your
lunge all you need to do with your lungs
is give them a name all the information
before is defined by your rate admin
configurations so you can just rename
that there and here's really where the
fun part is you take your storage and
create your storage you need to find
your first volume so we click on create
volume and we'll name this WWDC volume
and you can change things like the log
size and the max number of connections
you want to access to the fan now the
block allocation size is an important
field to pay attention
too and it goes in power of 2 from for
k2 512k and that is a performance doing
parameter that depending on your typical
I Oh sighs you may need to tweak and if
you need to know more about any of these
values and you'll see some more coming
up we have help buttons in all of the
sheets and it will bring up contextual
help for every single field so we
created that and we create our for
storage pool simply the same way and
let's make that cool one and let's say
we want it to be an exclusive metadata
and journaling storage pool so other
data won't interfere with that traffic
striped breath is another important
performance tuning value if you have
multiple loves in your storage pool this
is how many bytes that will write to
each lawn before moving on to the next
so we don't need to change this here
because the metadata pool will only have
11 so now that we've done that we bring
up our little drawer with lunge and I
have a pre-configured LUN this is just
by taming it that's all you have to do
to configure it drag drop there's your
metadata then actually we can come back
here and call this MV so you know it's a
metadata now we create another storage
pool for all of our data coming here
will call this video data and we want to
ensure that no journaling and metadata
spills over to interfere with our high
definition video or SD or whatever we
happen to have on here and we can change
our threat breath 2 128 blocks and this
size here 512k is how many bytes it
right and the size here also depends on
the block allocation size you'd find in
your volume and you could change
multipath method and permissions and
other stuff and the help will tell you
all about that so let's skip this disc
here because it's not the same drag all
that in there you go we've configured a
6.8 4 terabytes actually 7.29 terabytes
and in about a minute and that's all you
need to do
okay so we have a stamp set up before
with a file so we're going to pervert
and you can see here we have the meta
data pool we have a small audio pool
because we don't need as much bandwidth
for audio and we have our SD video are
high def and our post-production so
let's move over here and here you can
see all of our storage pools and you can
see a snapshot of the currently running
volume you could each of these will fill
up to show you how full each storage
pool is to know when you need to grow
your storage in the logs tab you can get
all the relevant logs on all of the
controllers all the machines on the sand
and even filter for certain things and
the clients have you can mount and
unmount you can mount them all at the
same time if you really feel so inclined
or unmount at the same time and over in
affinities we could set up all the
affinities and in quotas you can create
quotas elite quarters and it's really
simple just go ahead and you dragging
users and this is all ldap integrated so
if you have a directory server that
you'll see all the records there you can
drag in stuff here set the quoted 10
megabyte softcore I probably 10
gigabytes off quota 20 gig hard quota
and give them 24 hours and then go ahead
and hit save and it would send it out
and if you actually had some data in
here the quota status would show how
full how close they are to their soft or
hard quota or if they're even above
their soft quota and that's x 10
[Applause]
so basically we've shown you what the
admin does if you're familiar with Mac
os10 server you'll notice that it looks
awfully familiar that's because
basically leverage the same technology
as we did for the server admin the main
difference is in the server admin its
main goal was to connect to a server and
administer that one machine even though
you could mr. multiple machine from the
admin each was considered to be a sort
of separate unit in the UI the X an
admin sort of treats the whole Sam as a
particular entity so you saw that when
you're administering it your mystery in
the entire Sam at the same time
basically the server admin agent is
going to run on each of the xsan
machines both controllers and clients
the X and admin is going to take care of
replicating your configuration files
around between the machines so it's
particularly important between the
primary controller and backup
controllers that they have the same
configuration the backup controller
thought the volumes were arranged
differently that wouldn't be a good idea
so it'll make sure that the
configurations are all the same it'll be
able to monitor his status of the
machine so you can quickly look up and
see which machine are are currently
active which ones have volume is mounted
and it'll contact machines as necessary
to perform its functions it's sort of
behind the scenes it establishes
connections in addition to the admin app
we do provide a set of command line
tools as i said we try to keep the admin
app streamlined so in certain cases
there's additional functionality
available in the command line tools that
we don't actually surface through the
admin all the tools live in one place
inside library file systems there's an X
and folder and inside there there's a
binary folder that's also where config
files with another thing
the tools we'll all be documented
however if you look at the documentation
on the concurrent CD they aren't there
there are actually man files though on
the install that you can look at but
we'll try and come up with some better
documentation for these here's an
example of a few of these cv admin is
sort of the main tools the one we used a
lot when we were still developing the
user interface because it does a lot of
the same functionalities one of those
sort of interactive command-line admin
tools you can start and stop the
controller and do a lot of the various
functions see the affinity is the one
that you can use if you want finer
control over the affinities the admin
only allows you to set folders and that
everything in that folder will have a
particular affinity if you want to set
affinities on particular file or see
what the affinity on a file is currently
you can use the CD affinity tool dvfs
check is the normal FS check style
utility for the X and volume so if you
bring up disk utility you'll be able to
click on the volume and do a normal
verifier repair and it will call this
tool behind the scene but if you want to
have scripts or whatever to run it
there's a tool is available the final
one is the defrag tool that was
mentioned so the defrag tool can be used
to defragment of your file data it does
have one particular extra utility that
can be useful sometimes during dataflow
stuff you might for instance ingest a
file into one storage pool because it
has particular storage Christ particular
performance criteria but then later on
you might want to access it using
different raid Lunz so that you can
ingest new files s NFS defrag can be
used to migrate the storage for a file
from one storage pool to another without
affecting where it appears in the in the
volume basically when you look at the
file structure of the volume the file
still appears unchanged
but the actual backing store for that
file has moved to a different rate set
we mentioned the cross-platform set up
with the store next file system just
wanted to quickly go through and sort of
show how easy that is there's two
scenarios adding the store next clients
to the to the X and system all you you
can set up your XM system normally
you're going to get a license for your
store next client that will actually get
installed on the x and controller and
then you're going to just set up your
store next clients the way you would
normally do for a store next system
there's basically some information you
just need to enter into a couple config
files the trickier one is when you adn't
want to add an X an client to a store
next file system our admin software
basically is written to administer the
entire X an environment and so it
doesn't really understand a single X and
client connecting to some other type of
fan file system so in this case you're
going to need to administer the fan
manually luckily it's fairly easy to do
the main thing is you have to add your
serial number manually to the config
file and then there's just a couple
other files mainly the controller
addresses to tell how to contact the
controller so that's all quite
straightforward and it'll be fully
documented in the documentation so now I
want to talk a bit about sort of how
these volumes appear to applications the
first thing that is important is that it
is a shared volume pretty much though in
terms of writing and testing
applications it's going to be just like
a network file system in that way the
only issue you may run into is you do
find sometimes there are certain apps
that because of performance
considerations aren't used to running on
network volumes I mean if you've got
something that basically ingest
high-definition video there aren't that
many file servers that are able to
handle that bandwidth and so it may not
be used to running on a shared file
system so it is important to to make
that the applications are doing file
locking you can also be managed sort of
at the user level but it's better if the
application itself you know does the
coordination to make sure another coffee
isn't going to stomp on your data the
file system supports the normal calls
that would be done through back OS 10
you have both the file open flags the
shared lock and exclusive law as well as
the eff set lock f control for doing
bite range locking this is commonly
referred to as you know POSIX locks and
BSD locks also the open deny modes in
carbon get translated into these the
same way as they would for a NFS volume
the other thing to be aware of of course
is that these volumes are very large I
mean x-rayed volumes are already quite
large but the sand volumes are going to
be built up even larger I mean multi
terabyte volumes as you saw it's it's
really easy to build up these big
volumes because there's a tendency to
try and consolidate all your storage
even if you've got a bunch of different
rate boxes into one big volume so in
writing software that's the important
consideration that as well as having
very big file if you have these huge
volume you're going to have you know
possibly many millions of files on this
volume and so if you're writing backup
software and so forth you need to be
aware that things tend to get grouped
into larger volumes and they may have
otherwise the last point is that we did
mention that xn has its own file system
format it unlike raid where it's down at
the disc level and your formatting it
just like you would you know as an HFS
or whatever this is an ex an file system
format not an HSS volume it looks a lot
more like an NFS or aufs volume its case
sensitive and it's single for file
system so you can use carbon but it'll
do Apple double the same way as it would
on an abscess volume and basically in
terms of capabilities it's shared just
like an NFS volume is
it's just that the performance is very
different from an NFS volume so speaking
of performance the the main point of all
this is to have the fast file i/o but
even when you're contacting the server
the if things are set up properly the
server should be much more responsive
than it would be if it's a normal file
server you you're limiting the i/o to
the small io is a small request for the
metadata information you're not sending
i/o across so if you do have an IP
network or you set up your IP network
properly it should be very responsive
and the controller itself should be very
responsive in addition it's not having
to load in large file so it's able to
use its cache more for just cashing the
catalog information so hopefully it
won't have to go out to disk as often as
a normal file server would so it should
be much more responsive for a normal
file server but it'll still be less
responsive than a local file system when
it comes to these metadata operations
you're having to make a call across the
network you're calling a computer that
is potentially serving lots of thought
lots of clients the metadata controller
does end up being the bottleneck when
you're scaling up to large numbers of
clients so it's important certainly if
you're trying to support 64 clients that
have that be a very fast machine and in
terms of the application you need to be
aware that the metadata operations are
going to actually be slower than the i/o
operations there are certain ways you
can tune your app to to deal with that
I'm going to talk a little bit more
about prevention expense but the other
thing is to just track I mean catalog
informations tend to be the ones that
have to go out to the metadata
controller so it's good to minimize
those in terms of the i/o you want but
the file i/o is going to be going
directly to the raid so in that case it
shouldn't be any different than if you
add a locally mounted raid volume
so now I'm going to talk about few few
api's you can use certainly it's
expected that you'll hat start to have
server clusters that we'll be using as
the ex am and so server app may want to
take advantage of some of these features
distributing computing apps and then
certainly multimedia apses is a very
strong focus so three api's i'm going to
highlight i'll mention a couple other
minor ones but the extent pre-loading
and the affinities and then the
bandwidth reservation that Tom mentioned
earlier the api's all use a similar
mechanism their specific 2x and volumes
they're going to be accessed through
sysctl but basically we have some sample
code that sort of helps you call it
because the actual glue code is is the
bit growth so and the other thing to
note is that the API is still in flux so
we provide some sample code on the on
the CD but if you compiled using that
sample code you would need to recompile
before this final shipping version comes
out so it's there just to try out the
api's and see how they work but we'll be
feeding final api's closer to the ship
date and then the last thing is because
these are x and specific api's you
should use data to determine whether
this is an X and volume you're talking
to here's some easy code basically just
going to call SATA FS on this on the
file and the FF type name will be unique
to X and we actually have a constant in
the header that you can compare against
so here's an example of a typical sort
of lovely sysctl call you've got your
structure that you're going to pass down
into the kernel and it's going to get
filled out and then pass back up this is
a easy call in that it's just getting
some version information the other thing
about this API is you always need to
open file descriptor to make the call
obviously in version information you
don't really care what's not particular
doing individual file so a common thing
to do is to just open up the root
directory of the file system and make a
call on that but this particular call
will return the same information no
matter what file descriptor it's called
against as long as that's for a file on
an x and volume so the load extents call
the key here is when you open a file and
start reading and writing it the file
system is going to react to your calls
as you make them in my example you saw
there's the right file system call the
file system needs to go out ask the
metadata controller you know where the
file is before it can start writing the
data out but because you have that
latency and talking to the metadata
controller you you may have a hiccup in
terms of the reading and writing the
load extents call can tell this system
up front that you're going to be reading
a riot II in these offsets for this
particular file and tell it to go ahead
and get all that information up front
but when you're actually doing the i/o
you don't have any of the Layton sees of
talking to the controller so the
affinities the thing here is often you
don't want the layout on the file system
to necessarily reflect where things are
stored in storage pools common example
of this is you might have a project
folder that project folder could contain
audio files and video files but as far
as the users concerned they want all
these files group to
they're in a single folder but as far as
the system is concerned you may want to
store the audio files on a different
storage pool than the video files the
most efficient way because configuring
that all sort of by hand could be a very
complicated thing applications can take
advantage of this because they know what
types of files they're saving out and
what the characteristics of those files
are going to be so we're going to have a
demo of this all right so I'm going to
demo affinity steering open up my demo
app sorry I don't have a nice icon so
we'll create a file called my file and
we're going to put it in the post
production storage pool so I'm gonna go
ahead and start that and you can see
it's writing at reasonable speeds you'll
get a initial burst of speed as it fills
the raid cash but then it will level out
and if you can see the lights over here
you should see one lun being pegged how
if we start up a second file say this is
our video file video file dot MPEG and
we store that saying hi def we start
that up we should be pegging two
different lines or two different storage
pools and two months and it should be
going much faster which it is now if you
go ahead and look in our volume it was
stored it in project files they're both
sitting right next to each other and
that is affinity steering
so the one of the points on that demo
was you know basically we wanted to show
the difference between the storage pools
but we only have one fiber channel
connected up to the system and didn't
really tune it so don't take those
performance numbers and it's typical
performance numbers but we just wanted
to show the ways in which an application
can can talk to the different storage
basically how it's going to do that is
first it needs to find out what storage
pools are available early on the we
called storage pool Strike groups so
that's still reflected in the API get SG
info it's going to give you information
about the stripe group so theoretically
an application might be able to do some
intelligent figuring out of which stripe
group it wants to use the other thing
that's probably more common is to do
what we did in the demo app which is
basically just present a pop-up to the
to the user to choose which storage fool
they want in the API it's also going to
give you an 8-byte key for that storage
pool that's what you actually use in the
set affinity call so normally you would
open a file and create the file using
open set the affinity on the file and
then start riding it the other thing you
can do what we did actually in this app
was called alec extent space which will
pre-allocate the space for the file load
the extents into the client and then
allow you to start writing out and that
gives you the the highest performance
riding
so the next thing I want to talk about
is bandwidth reservation basically I
mean Tom described this pretty well for
the people that weren't here i'll try my
little big thing basically the idea here
is that when you're doing a critical
operation you can't necessarily control
what other people are going to be
accessing the sand so if you've got your
in gestation and it's really critical
that you get your high-definition video
you know streamed on to there without
any hiccup you don't want somebody else
coming up and just starting to stream
some other file on or off that same
storage pool and mess up the bandwidth
so basically this is a way for
applications to to guarantee that
they're going to get a particular amount
of bandwidth if somebody else launches
something somebody else launches
something where they don't care about
the performance it'll just get scaled
back if somebody launches an application
that's also demanding the critical
performance they'll get an error saying
look this is already in use this amount
of bandwidth has already been reserved
so there isn't enough left for you to do
your application this is basically used
for streaming especially real time
streaming and it's / storage pool
because as I said earlier if somebody's
writing to one storage pool it doesn't
affect the i/o to another storage pool
anyway so the you're reserving bandwidth
on a particular storage pool and people
reading arriving other storage pools
won't be affected by the reservation so
we'll have a demo of this alright so
we're going to be the same application
and say we have a video file and an
audio file and we're going off and
running those the same storage pool
high-def now say we want 120 megabytes
per second and unfortunately we're
sharing it at about 80 or 90 mega second
over the single fiber channel and go
ahead and attempt to reserve bandwidth
here and that will jump up while the
other one goes down and this is being
written to the exact same storage pool
and they're still sitting right next to
each other but one is getting more
bandwidth than the other as much as we
you had configured it to meet and there
you go that is balanced reservation so
bandwidth reservation is the one feature
that only works with applications
support it because it's the application
has to tell the system you know what
file it is that they want to reserve
bandwidth for and and how much bandwidth
needs to be reserved the other thing
about it is that it requires additional
configuration that we don't support in
the in the admin API it's pretty simple
basically the system when you configure
the volume it isn't able to determine
what the throughput to your varied
storage pools is it's a very hard thing
to determine you know programmatically
there's a lot of variables involved so
you just need to run a simple test run
an app like the one we just had find out
what the throughput to your storage pool
is and just enter that field into the
configuration file and then it'll 0
basically how much is able to be
reserved off of that another important
thing you can add is to tell it how much
you don't want to be able to be reserved
as you saw once somebody makes the
reservation the rest of the performance
is going to drop way down to to allow
the person to have that bandwidth it's
critical that you don't have everybody
else dropped to zero and have one person
who reserved the entire bandwidth
because that can lead to dead locks and
other problems so at a minimum the
system leaves 1 megabyte per second free
that other people can at least do very
slow I owe to that storage pool but
under certain circumstances you may
actually want to increase that so
there's another field to determine the
non reservable part of the bandwidth
so the call is basically set real time I
oh the idea is that you're going to put
the storage pool into real-time mode
that means that basically once a client
has loaded the extents normally it's
just doing the file i/o as I said it
usually doesn't even care whether the
metadata controllers still around it's
doing it file i/o it's happy but once
the you put the storage pool into
real-time mode it goes out and tells all
the clients that are using the storage
pool that they now need to make requests
for i/o so each client will then ask the
bandwidth ask the controller say
basically I want to do I owe to this
storage pool the controller will give it
a token allowing it to do a particular
amount of i/o for a certain time slice
depending on how many clients are asking
for i 0 it'll partial partial out
different amounts of i/o to the
different clients and this is will
balance that sort of as you know
dynamically as time goes on the
important thing is that the person who's
reserving the bandwidth is when they
make the call they're going to specify a
file descriptor that's going to be used
for the performance critical operation
and that file descriptor it's it will
not be limited you'll be able to make
reads and writes freely to that file
descriptor there's actually another call
you can make if you have multiple file
descriptors you can make another call to
enable multiple file descriptors to be
ungated
that's basically my session you know you
saw how xn can allow you to configure
your lunge together in a much more
flexible way and and get the performance
to all of them out to various clients
the important points I want to make our
it that the it is a shared file system
that's a very important thing that
applications need to be aware of and
that there are these api's available to
to add additional value to applications
if you're writing our next Sam system
and then I think we will have Q&A oh
well more information basically these
are the documents that are available on
the on the CD and then I think tom is
going to come back up for Q&A or Eric
[Applause]