---
title: WWDC2004 Session 632
framework: wwdc
role: article
path: wwdc/wwdc2004-632
---

# WWDC2004 Session 632

## Transcript

Kind: captions Language: en good morning so I'm standing between you and lunch I believe so we'll try and make this either an opportunity for you to get those juices going to to look forward to lunch or and certainly try and try and make this an interesting session for you as mentioned I'm the director of engineering for a pro video applications that includes final cut Final Cut Express final cut pro pro video codecs and some other other projects within our pro applications division I'm going to be talking today about the xsan product and how it fits into the video workflow and although i will be talking it from the perspective of final cut pro it actually is very applicable to anyone in the applications domain who's dealing with midi media in a Nexian environment it's also I think you'll find it appropriate for anyone who's dealing with the kind of the creative professional marketplace in general there's a number of different tools that fall into this category and xn is an excellent tool in this space but it provides some interesting challenges as you as you're dealing with media and we'll talk about a lot about that today certainly folks are dealing with site administration I think we'll find this interesting as well because setting up this environment is actually the biggest challenge that will be will be talking about so as we get into the talk first I will talk a little bit of that kind of set the context around how video editing you know kind of the view of video editing from the editors view from the kind of creative professionals view and what their expectations are and then we're going to get into how this applies to X an environment and how shared storage and workgroup computing kind of changes that model and impacts that model in a pretty significant way and then we'll finish up with a little bit of thinking about where we go from there so first to get into video editing today I think one thing to understand in the marketplace right now is that the focus is around as much performance and real-time capability is you can possibly get out of the hardware but certainly what drives the market in a big way they're looking for the fastest hardware they can get the fastest storage they can get the looking for as many streams of video as many effects as much real-time capabilities that can possibly get out of the system and that's driven around a number of different kind of workflows both from the kind of media that people are dealing with and the kind of people that are working with the media so that's everything from the you know individual independent you know production house people or video in event videographers who are producing corporate videos to the independent film makers to small work groups of people who may be working on short form commercials or movie trailers or doing music videos all the way up to broadcast television you know how two shows on you know HDTV and you know movies that you go to see at your local movie theater all of that is expected out of the today's existing tools and hardware and it's a huge range of expectations and it drives you know in a large number of different kinds of media that people have to work from if you're dealing with film and you probably you're obviously not working with film directly in the computer but taking it into this on top of offline format but many of you know and for news gathering and for other things they're dealing with db25 your standard you know camcorders if you will all the way up to your news production high-definition uncompressed standard definition you know for TV on converse high definition as well as other types of media like audio and stills one thing that's really interesting to consider as you're looking at the performance issues around dealing with its media is the kind of you know data rates that you're dealing with for this kind of stuff for your conventional db25 or your you know standard mini DV camcorders you know it's in the range of 2.8 to 3.6 megabytes per second megabits per second of information and as you go up the you know the through the different types of media from dbg / 50 and dvcpro HD the data rates of course go up appreciably when you get to uncompress things get really complicated because you're dealing with it's just monstrous amounts of information the good news is compared to things like DV the various TV formats is these are uncompressed so you you know the kind of the trade-off there is computational complexity if you will you know having to compress and rican for decompress and recompress that information is you're dealing with it as opposed to moving just large quantities of bits for all this in a data and I want to remind you again that you know people today are looking to get as many streams of videos and effects as they possibly can through the pipeline and today you know with final cut pro for example we deal with a number of streams of video of various types you know at a real-time level without any additional hardware and a dual processor g5 so people expect that if they're going through the creative process that they're going to be dealing with lots of streams of video and they're going to be doing lots of things no expect you'll time Behavior all of the video of course I think it's worth mentioning and kind of in contrast that there's also a lot of streams of audio to be dealt with but it's really interesting as you look at the you know in upwards of 100 megabytes per second or megabytes per second for you know uncompressed HD you're only talking 192 kilobits per second for you know for audio but usually people are dealing with many many streams of audio at a time but there's a very large range of expected data rates for the kind of media that you're dealing with in a typical project and this is a project in Final Cut that and I think it again is worth noting that typically there's a number of streams of video that particular dealing with if you look at something like a commercial today a commercial you know actually has lots of streams of video going on at once there's usually mom you know most various motion graphics pieces there's multiple camera you know attacks that are coming in and their criminal cross fading or moving across the screen there's a lot of stuff people are doing and you may not typically think of you know in the normal process how many streams of media that people are dealing with but it's quite a lot one thing that's very very common in the case of audio is that people are dealing with usually you know upwards of twenty four tracks of audio certainly when it all gets done it all comes down to one stream of video and maybe a you know a stereo pair of sound but when they're going through the creative process there's usually lots of different layers of the stuff that is getting composited together at various levels and as they're you know working with it they expect to see it listen to it in real time so it all comes down to really the way today's you know model of creating video is it's very much a single system workflow that you know people have really optimized their process around very high performance single system solutions and while they may be in an environment with many different machines you know today most of the media is tends to be moved around you know via the network or you know by taking that little firewire drive and going from one system and doing the sneakernet thing and the downside to that and that model is that you end up getting a huge amount of media that's duplicated all around the enterprise it's not very efficiently used but a lot very effectively use the upside is you've got a huge amount of power that you get out of that single system because you can make a lot of assumptions about being able to use every bit of that band with the other downside to having all that media duplicated around the environment is that you know dealing with backups because it's all distributed around your your your enterprise or infrastructure and dealing with media security which tends to be a really big issues in you know for example and for folks who are doing maybe trailers or you know trying to do you know unreleased movies they're trying to keep you know pretty tight control over that media before it gets out and gets leaked out and put on the on the internet so doing that when all that media is being sneaker netted around anyway and a firewire drives makes it even easier to you know how thats get sneaker netted right outside the front door and there's certainly a number of other challenges that the single system workflow you know has it just really makes the process as you're working as a group very very difficult and that's where x gain comes in xsan really provides an opportunity for multiple clients to work together to share data over what's the really most important part of this thing is a very high speed interconnect the difference today when you're dealing with you know a network file system solution and video is it just isn't working all we talked about the number of streams the video that you know people expect to work with you know over a network infrastructure you know typically you just can't get that type of flow the sand environment really you know provides an architecture in the design that I'm going to makes it possible for you know media rich environments to be able to share information because you get pretty much the benefit of you know local speed interconnect because of the fiber and the way that the file system and sand is designed relative to how something like a network file system is designed certainly you know as we talked about them you know and the negatives there's a lot of of a single system environment as we move to Juneau shared storage is a lot of benefit here being able to get to the point where you can actually share media assets you know within an organization is a big deal if you're dealing in broadcast for example you know you may have a lot of stock material that you deal with it gets brother and that you really want to be able to use over and over again if you have to copy you know literally terabytes of data around to each one of your workstations in order to be able to make that stuff useful it makes it you know very ineffective and a horrid use of your storage so the other advantage of xane sword shared storage environment is the ability to consolidate that storage typically you have a lot of material that you're you're sharing amongst you know troops with people not just what you're working on right now by the large amount of you know stock material or material that's potentially coming in from various sources and again avoiding having to have the large capacity storage on each of those workstations is a really big plus a considerable amount of money the other big big benefit is as you look at the economics of you know a video environment that one of the biggest you know most expensive pieces these days is getting to be the acquisition devices themselves the decks particularly easy as you start talking about high definition or or film you know those you know pieces of equipment there somewhere on the order of 50 200,000 to potentially up to a million dollars in order to be able to acquire that media into digital form it's not the kind of thing you're going to stick on every one of the workstations within your environment so being able to get to the point where you get stored shares shared storage pardon me you get the also the benefit of being able to you know reduce the number of equipment costs that you have for acquisition because you can centralize that get the material into a place where people can share it and avoid having to I have lots of redundant redundancy as far as that kind of equipment is concerned obviously the benefits you know being able to now have that information there means that you can actually start putting together workflows you know within the workgroup means you can get materials it can go from editor editor or the creative professional for the Deaf different parts of the creative process because they have the ability now to be able to all get to the same media as I talked about a little bit earlier the issues of single system workflows is that the security is a real challenge that sneakernet that you know approach of moving media around is you know a big problem you know in in areas that are dealing with potentially unreleased material movies or other things and being able to get to the point where because you now have central control of your media and being able to start applying the kinds of permission capabilities that that a UNIX system and a shared storage system and you know enable means that you also can start to control the security of your information being able to make sure that you can say who it is that can actually see the material who is it that's allowed to change it and you know basically get to the point where you have a lot more manageability of your assets and of course probably the most important part of this whole thing is finally being able to get point word because all your materials together it becomes actually feasible to back it all up however with all those good things there's a number of things to consider when you're dealing with putting together a fan environment one is performance remember we talked about single system performance or what people expect today in the creative process they're getting every single ounce of power that they possibly can get out of that as soon as you trying to distribute that now one of the challenges is trying to rein in and control the ability to continue to get that kind of performance the other issue is dealing with redundancy shared storage is great because you can all share it but you now have the opportunity for single point of failure and in that in that situation you need to be thinking about what do you do to make sure that you don't put your enterprise at risk because you now have a single point of failure those individual workstations are great if one went down you still have the other five of them that you head up and running but again you know this is one of these how do you manage the trade-offs of being able to get to share but being able to get that sharing and make sure that you don't put yourself a you know enterprise at risk in the process security while is a really important thing to get in there it's a really hard problem to solve and we're going to talk a little bit about what some of the issues are there but you know I want to prepare you that getting and putting together in a kind of the security infrastructure that you need in terms of managing multiple users is very difficult and there's so I think a lot of challenges today in terms of dealing with collaboration it's a real plus but you got to think it through ahead of times and can understand what it is you want to try and accomplish here so let's start with performance first the goal in the San environment really is to get back trying to make sure that you can get to that single system performance that's what your editors are going to expect let's require your creative professionals and your enterprise is going to expect because you want to be able to take advantage of the creativity that being able to do all these things in real time enable the challenge is you've got to be thinking about now that you are spreading potentially the performance load or the expectations with a large number of clients within your environment let's say you're putting together a you know five system fan and you're all going to you know some sort of central storage you know you map that against five individual systems that had their own local storage remember there's a one common element in that which is the storage which is now getting hit by five times as many people as it would be if you have them all completely isolated and so you need to make sure you understand what each of the clients within an environment are actually doing what kind of media are they actually working with and how many streams the video or media you know are they working are they typically dealing with in the projects that they're doing like I said you know if you're talking about something like you know commercials commercials are good tend to be a good example where lots of streams of video is you know see these things on TV TV it's all about you know transitioning between lots of different shots there's a lot of motion going a lot of activity if you look at something like you know more of a conventional film you tend not to get quite a large number of streams that you deal with because you're dealing with a lot more long-form material but you have to look at the kinds of things that you're doing and factor it in that into the process and as you go you need to kind of go back to that graph that I showed earlier in the slide and and no map the kinds of things that you're doing against the size of the streams of the number of streams to understand what exactly you're doing in terms of the perform the data flow now one of the things that you should also consider if you're dealing with single trying to get the single system performance is that in your fiber interest sure for something like ass and your switch is going to be your friend or it needs to be your friend it's really that one of the most important components you know in the infrastructure and the reason is its job is to help isolate traffic from each of those individual systems in the sand and the individual components of your storage and the more parallel ISM and the more isolation that you get between those components the more throughput you'll end up being able to get through the process through through the whole network basically the more things bottleneck the more things end up going through they want through one pipe the left you know total throughput and throw to you know total capability that you get in your environment the ability for your switch to isolate is really really important so as you're looking at that you know part of your expenditure one you'll find out quite shockingly that the switch is probably going to be one of the most expensive pieces of equipment you buy it's going to be probably more expensive than your storage it's going to be more expensive than any one of your computers it's certainly going to more be more expensive than the software and you know it is going to be something that you you know should be prepared that it's going to be a very costly piece and you need to think ahead about how many clients I'm am I going to have in the environment how am I going to put my storage together and how many different strands of fiber am I going to be running ultimately in order to be able to get the maximum amount of data through this to the system you've got to do the math meaning if I talked about just a moment ago you've got to look back on what are the clients in your environment doing what kind of media they working with and how many streams of media are they dealing with you gotta count audio you've got to count you know the video and you got to kind of there and do all that calculation and unfortunately it's one of those things you've got to do really upfront figure out multiply out you know total data flow within your organization and figure out how much data is going to be driven through you know the strains of fiber you know back to the storage and where the bottlenecks are going to be always remember that and this is one of the the typical things that gets forgotten is that the editors are always going to be where we're dealing with multiple streams and don't assume one workstation one stream of data the reality is that they're dealing with many streams because that's part of the creative process another thing to factor into an environment like this is typically in video you're dealing with pretty large files you know they're you know certainly many many megabytes if not gigabytes in size you know typically depending on them in the media that you're dealing with and from a perspective of impact you're dealing with a reasonably small number of files but a large amount of data the data is kind of coming in you know if you can think of a fire hose and I just a huge amount of informations coming in but you're not you're not dealing with a large number files you're just dealing with a large amount of data what you have to be careful of in the environments like there is that typical computer usage workflow let's say email or web files or you know you know you know Microsoft Word documents these are very small files and usually if you look on your your hard drive there's lots of them and the impact between those two different usage models is pretty significant that you know the fan is actually extremely well tuned to be able to deal with large file access lots of data coming over the fiber but not lots of little bits bits of data coming over the fiber that's not a very effective use of that particular resource and it gets worse when you may start mixing that data and so when you factor in your planning and you're looking at your infrastructure you know one of the things you might you know think of doing is you put in the same environment is cool share everything well that may not be the wisest thing for you to do because as you start looking at all those little file things that are going on with your network you could end up actually completely taking all the performance out of you're out of your infrastructure and doing that and if it's appropriate for you to do that or let's say the small files you're dealing with their photos because those are certainly reasonably small files and you can have certainly a lot of those what you mean start thinking about doing is trying to isolate that kind of material so that those large files and small files are actually kept in separate places so you can get again more parallel ISM you know through the process if you move on to kind of the next step of you know first understanding just basic capacities you one of us start thinking about how you actually can you know what things you can do to enhance your storage capability within the stand so certainly a one opportunity is being able to add more storage so more storage does not necessarily mean more disk space though that's certainly one of the things you can get out of more storage but it's really adding more storage in terms of getting more parallel ways to be able to get to data so it's basically more heads or you know more rotating platters if you will that can all be accessed in parallel so in the case of what I'm referring to your in terms of adding storage the goal here are an outing storage is adding more parallel ISM more ways to be able to get to that data simultaneously rather than adding more bikes that you can possibly put on the disk that's enough happens to do benefit one of the other aspects of that is how you actually orient your storage one of the opportunities is looking at spreading the load across multiple basically multiple rotating services by striping your data so this is a standard raid behavior but you want to think about again if you take all the you know the capacity that you're looking for how can you possibly get as much of it in parallel across multiple devices simultaneously and you should look at striping the information across these multiple devices is one way of being able to get there another way of approaching that is actually looking at segregating your data so if you have lots of different media within your environment that you dealing with there's a couple of you know capabilities that exist within X and to really help you there in terms of of being able to isolate that media in different ways one is the use of affinities affinities are a technique for being able to effectively take a portion of the file system and tell it to put it on specific storage or look at and I look at the opportunity of actually using multiple volumes as a way and what kind of a more coarse grained way of being able to you know force material into different storage pools and the kinds of things you want to be thinking about in terms of segregation is again keeping kind of the big stuff separate from the little stuff is it kind of an easy way to look at it because if you can take something like video which is again usually a less number of files but usually a large amount of data to get streamed over time and separate that from potentially smaller files that are going to be accessed more frequently one example of that is certainly audio and video they have different substantially different data rates much smaller much lower data rates as compared to the much larger and much much bigger amounts of data certainly still is another example if you're doing an environment where you're creating you know video that has a lot of skill material look at keeping the fill material separate from your video that can help quite a lot another thing to consider is not only the kinds of material but how the material is used so if you have material that is like you're you know stock photos or stock video that you want to have on hand that can people can use for creative purposes but they're not used very frequently particularly you may have a huge amount of it and use people tend to use just little bits of the time look at keeping your stock material isolated away from your current active projects current active projects tend to get a lot of change a lot of activity of you know against them versus potentially stock material that may you know have a much lower rate of change and being able to isolate that particular and slower media or better yet away from the material that's getting accessed at high rate can have a big impact another thing to look at is actually separating your projects themselves the actual you know different projects that are going on within your enterprise from each other you may have you know potentially fiber you know six major projects that go on at a time and one of the things you may want to consider is organizing your storage and organizing your infrastructure so that each of those projects are isolated from each other not so much from a security perspective but again from where it sits on the storage so you can get more parallel ISM you can keep it all on one volume and use affinities as a way to be able to do this kind of thing basically allowing people to be able to share material and be able to see the material as appropriate but being able to use the infrastructure of the sand to be able to tell it to where to put the material actually on your physical storage one I certainly recommend is you know thinking about you know where you're doing ingest within your facility as opposed to you know the kind of again the less likely to change material static material or potentially rendered material ingest is one of those cases where it's really important that you don't drop you know frames or other things you know through the process you're trying to get this what's coming in probably from that $50,000 deck or you know a million-dollar tell us any device and it's pretty critical that you you know want to shorten that process for being able to get that material you know into digital form as quickly as possible again looking at a fitting these is a way to isolate that so you can manage potentially bandwidth and to be able to again make sure that you effectively get isolation through the fiber from much of the other activity that's going in in the infrastructure you actually want to take a moment and kind of draw out the map of your of your fiber the users and your storage and think about in the different usage cases which lines of fiber actually get lit up if you will during the process and you'll find that you know as you go through that process that you're going to find places that are you know our bottlenecks that yet lit up every time you do things and those are places that are opportunities you know to look for more parallel ism where you can potentially move either workflows people and data around to be able to get more again more parallel ism through through your data flow one of the thing to kind of focus on your specifically people who and are looking at developing software is that you know within that context it actually is really important to make sure you enable your users to be able to have fine-grained control over the different types of data that may be dealing with and where that it gets put it's the flip side of being able to manage the storage it's being able to allow the softer to be able to segregate the data so that you can manage your storage and I'll show you an example of that in a little bit with final cut but you know for those of you you know who are doing media applications who are starting to think about this you know you know look at ways to provide this kind of fine-grained control for your users more opportunities for enhancing storage capability are thinking about now where your be kind of the other side of the data flow I'm assuming most of you have been to some of the other exeunt sessions and in are aware that the you know infrastructure for dealing with a fan is both fibre channel for the data access and an ethernet path for the metadata the metadata is the basically file system operations aspect of of the of the file system that that communication path occurs over an Ethernet channel as opposed to the fibre channel fibre channel is really designed for again that big packet large data when you're doing lots of file creation and file deletion that's one of the cases where there's a large amount of traffic that tends to go back and forth between the client and the metadata control of them basically the server aspect in the X an environment that basically is the arbiter of how the information is ultimately laid out in the disk when you're dealing with video again big files but little numbers of them you don't tend to do lots of metadata access usually it's a I need to find where the file is there's a real quick message to the metadata controller metadata controller says it's in this blocks on the on the storage and then from that point forward the data flow typically goes through the fibre channel when you're dealing with small files and you're doing lots of creations and lots of deletions there's a huge amount of traffic that goes back and forth between the client and the metadata controller and if you're doing a lot of that particularly over a large number of clients this is another opportunity for a bottleneck so as we've talked about for fiber you also need to be thinking about switches for your Ethernet you need to be looking at isolating that particular path of the Ethernet traffic so that the metadata you know traffic is not you know interfering or getting overrun by potentially you know browsing the web or getting email off of your mail server or you know connecting through a network attached storage device with some variety so it becomes really important to think not only of your fiber infrastructure and where the data flow is but potentially what your Ethernet traffic is within the environment excuse me the other thing to look at on your metadata controller is you're doing more of these types of things is ram helps a lot metadata controller tend to the cast this kind of information it is something that an Ram will help live and multiple metadata controllers to can help spread some of the load one thing to think of at large is look at keeping much of your caching activity on your love on the local system on the local client cash a cash for is typically a files that are created by an application purely for the purpose of speeding things up its job is to basically the pre-render material or two do something so that it doesn't have to redo that operation many many times it's one of those pieces of information that typically can be thrown away because the computer well ultimately will will recreate them so if by keeping them on the local system you're not losing any important information in your workflow and you're keeping the amount of traffic between the local machine and the rest of the infrastructure down to a minimum the cash is really intended to be kept fairly local so if you have potentially render files or various cache files the logs those are the kinds of things that really are encouraged if possible income cases you can't but if possible to try and keep these on your local system that can only take a load off of the rescue infrastructure the other thing you should consider particularly in the case and I can speak for final cut is that you should really consider keeping the project files themselves locally that's not the media the media ends up getting can have kept on the shared storage but the projects that you're actually working on is kept local it's one of those things that tends to get accessed a lot it's an example of lots of small file reads and writes and it's one of those things that again isolation you know can help a lot with really the whole goal when you're looking at enhancing the storage capability within your environment is finding a way to spread the load of across as much of your infrastructure as you can and avoid single choke points and that's where drawing it out as tedious as that maybe it can be really really helpful in figuring out where those single points of acts are access are we doing it within your fiber infrastructure or within your network infrastructure and I want to take a little commercial break here and talk about redundancy it's one of those things you want to think about here you know when you're dealing with the sand that you've now gone from lots and lots of individual seats that are fully isolated to you know the ability now to share lots of material it also means you have an opportunity for single points of failure so you should be thinking about you know that from the get-go you probably don't want to take the chance of taking down your whole interest sure because a component fails as something happens and there's some ways to protect yourselves from that certainly reported raid in a raid configuration doing mirroring is one of the you know the things that you should look at there's you know looking at failover method metadata controllers within the the fan environment you can even use and particularly in small work herbs it might make sense to actually use even one of the clients as a potential failover in case it's one of those things that you hope will never happen but if it does at least the system will fail gracefully and you won't take your whole infrastructure down another thing to look at is multi passing multipathing is basically using those two strands of fiber that are coming out the back of the computer you know as potentially multiple paths to be able to get to your data in case one of them fails and I believe one of the other sessions talked a little bit about how you know you can use multi passing with within the the axion environment certainly one of the most important things in any situation is back up and so back up is you know something you should factor in when you're doing your site planning and the other data consider of course is is power protection this is a great place to be putting battery backups or other other things so again just one of the things you want to think about is you're going through the process of setting up the environment that you consider now that you have another opportunity for a single point of failure and look at all the places that you can protect yourself moving on to security and collaboration dealing with multiple users in a media environment is a real plus but there's some real challenges here you know it is a lot lot lot harder than dealing with a basic file system server setup I think you know from the get yo should understand that as you're starting to think about how it works here you've got to think things through a little bit more and the reason is it's more difficult is because in a typical file server setup your you talking about you know someone being able to log in to a file server to get to their files and just you know save and retrieve their files as you start dealing in in a multi-user media environment you're not actually having to think about the collaboration aspects of that and how you actually can share material across multiple people if you're dealing with that in a fire in a network file system environment then you're probably dealing with many of the same issues but you typically don't tend to deal with that in that environment to deal with a shared media environment certainly with Tiger being it now so the supporting you know acl's excess control is and actually that will be an incredible benefit unfortunately we don't have that right this moment so until then one of the real things that you have to kind of factor into the whole process is planning make sure before you even put the first piece of fiber to a computer that you've thought this out because it's one of those things that six months into it you're going to go oh my gosh i completely forgot i didn't think about this and what you almost are stuck doing is carrying it all apart and starting over again so it's really important to actually take the time upfront to actually do the planning there's a lot of parts to deal with here when you're dealing with shared media and we'll talk about what some of those things are and we talked a bit about performance well one of the they you know the big challenges is that you know security and performance don't always mix real well and as you'll see we'll get into this they tend to collide in a couple of important ways that you can plan for and work around but that's because it if you do it up front with planning and the other thing to think about is when you're dealing with multiple user projects where you're going to be sharing material that again you need to think it out a little bit ahead of time so as you're planning for multiple users one of the things that you know I consider to be an absolute must is you need to use a centralized directory server of some type I'm not going to say which one you've got to use but the fact is you need to make sure that you centrally manage your users in this environment because as you're now dealing with all the way UNIX permissions in this missus in this system files are going to have you IDs applied to them and if you don't think about what the things are that are going to be assigned to those different you know files what user ideas and what group IDs and as soon as you start to deal with sharing those files you know six months down the road after you've you know finally realized gee maybe I need to worry about this you could find yourself in a lot of trouble because there wasn't something that you thought up front and then you don't have a centralized way of managing so it's really critical that you manage the users within some environment and some centralized way and you need to just get over the fact that you are going to become an expert at UNIX permissions this is one of those things that you know in this particular situation it's it's something you can't avoid it's a going to be very critical in terms again how you lay out your file system how you deal with the multiple users and you're going to have to really understand you know at a time how you want to you know put things together you might want to look at putting together you know a fight point in kind of checklist if you will and look at each of the workstations within your environment and look at for example whether you're you know how your media workstations are going to be configured in the environment how your normal office workstations are going to be configured so in the North kind of normal office workstation environment I wouldn't expect any big surprises this was should work as it normally does using home directories or whatever on your network attached storage or whatever solution you have those particular things will probably work pretty much as you're used to for media work stations on the other hand deciding whether or not there's going to be a dedicated workstation per user or whether you're going to allow them to move around within the environment if something got to think about because in a media environment there's a lot more than just a few preferences that go from place to place in these setups particularly today and I certainly think over time this will get easier but at the moment anyway it's a really complicated problem because as you move if you're going to move users from workstation to work station usually the project its performance characteristics it's expected asians in terms of that real-time behavior what kind of devices graphics etc you know can be very much tied to the workstation that that that project is being done on and if you move around you may find yourself having to redo an awful lot the other thing is is there's a lot of other material that tends to go with the projects scratch files and render files and things like that and as you move around from station to station you'll find you won't will end up having to effectively recreate that you know through the process and that can have effectively slow down you know the kind of usage model that people may have you want to think about where where the home directory is going to be for a given user in that environment because the home directory in a media environment is going to get hit a lot that's where those scratch files tend to be put that's usually where a lot of other cached information is kept within the within the system unless you manually go and override and set that up for each workstation and user they're going to end up having to you know one it's going to end up putting a lot of load back on your your your infrastructure and two you're going to end up finding out that your performance is just not what you expect it to be things are not working the way you expect one of the thing to think about to at least right at the moment is that the home directories tend to be when you're dealing with an environment where you've set up a centralized no file server for for dealing with multiple users the home directories are actually connected over the network they're not connected over the fiber and this can be very confusing and again I think this is one of these things that over time it will change but you'll get Network beer get performance characteristics of home directory access that's network based as opposed to fiber-based again placement of where all of the material that has to go with an individual user can really have an impact on performance so you want to think about you know how you're going to be doing that you also want to be thinking about how media plays into this environment if you have globally shared like stock material permission set up in that particular situation is reasonably straightforward usually can keep all that stuff together and usually stock material is something that can be shared again you know by all the users within within an environment but when you get to per project media this is where your permissions issues tend to get more challenging and you want to think about when you're setting up projects within your fan environment you know how you set up your directory tree how you set up the permissions within that directory tree usually when you create a new project for example you want to make sure you create a UNIX group for that project so that you have the ability of being able to manage which users actually have access to the material within that project and you need to do that from the get-go you also want to make sure that the directory tree within the project is set up appropriately with the permissions not only for the whoever the owner of that particular project is but that the group permissions are correctly and properly set and that the media within that project within that directory tree is mapped as appropriate based on the performance affinities that we talked about earlier and this is where it gets really complicated because you basically have this one tree of information that you're dealing with in terms of how your projects organize and a totally different structure that represents how your media is actually stored across your storage and these are you know multiple dimensions that you have to factor in at once because it's so complicated he really encouraged you to actually draw a map of the projects and how they're organized over your storage again not only in terms of their organized how the user sees it but more importantly how they're organized in terms of the storage so that you get the performance expectations that you have when you get to this level this is again where our dick tends to get kind of hard merging the needs of sharing material and the needs of being able to get performance you know can be you know rather complicated again you're dealing with multiple dimensions of of hierarchy here that need to be fettered in and it's one of the things that can really help you over time is making sure that from the get-go as you're planning that you can think about putting together a consistent structure for all your projects and I'll show you in a moment you know a very simplified example of this but you really want to look at organizing your projects by performance types by the kind of material you have so they have the ability later to go back in and apply the affinities that you may want to apply so you can make a performance adjustment you may find on getting more and more users the performance was going down I need to add storage but if you didn't organize the material well to begin with you're going to find you have to back it all off put it back on and restructure and that will take you down potentially for a few days so one thing you might want to think about is you're dealing with storage again is kind of this two dimensions to this is how your material is structured where you put your you know shared material at your stock media and within the various projects how you structure it so for example within each project I would you know encourage you to isolate your media from the metadata basically the project stuff the little file so you've got all your media isolated you know from all the little things that tend to go with your project and thats things maybe like scripts or notes that you have or other small files project files for example that may make up your project that you're doing those are the small things and keep the media separate from that and even within the media looking at separating the media out by type certainly looking at you know breaking out audio and video separately the advantage of that is is that you'll find again the requirements for audio and the requirements for video in terms of speed in terms of the amount of data that's moving are pretty different and you may be able to organize your storage appropriately to be able to give you much better performance the good news is that if you look at organizing your projects this way from the get-go you can reorganize your storage separately from this because the users don't need to know how your storage is organized and they just care that if I go down and look in these particular folders I'm going to find the stuff that I need to find so think about it in advance so that gives you the flexibility of being able to get your storage organized in an effective way I've been talking a lot about scratch data through a number of these things so this is an example from Final Cut Final Cut allows you for example to segregate a bunch of the ancillary files that you know tend to be involved in a project you may typically think all i'm doing a video project i've got certainly my final cut pro document and then i've got a bunch of video in a bunch of audio well the reality is there's usually and this is not you know just you know for final cut but I think for many you know sophisticated media applications a lot of other stuff that is generated part of as part of the process in order to get performance and in order to get the kind of you know you visual expectations that people have for final cut you have the ability of segregating for example where your you know your audio from your video and where it's placed as we talked about from you know a project organization standpoint that can be helpful for affinities you can in this case also manage where some of the render files the files that are used to you know maybe you know render out you know complicated transitions or effects those are scratch files that are not necessarily critically important you know in terms of day-to-day but it's important more from the you know getting the kind of performance that you expect and there's places where it puts those and puts those things on the system and final cut allows you to kind us again you can segregate where that stuff is kept you want to keep that stuff locally you may want to put it on the separate volume or within the sand and there's other caches that in the case of Final Cut are used where various you know thumbnails for pictures and other things that use little things that that are used just to make the UI and other things you know perform a little bitter many times most of the stuff can begin except on the local local system you you know don't lose anything by losing this material other than kind of the UI responsiveness typically in a given session it's the kind of thing you probably don't want to put on the sand because it just generates a lot of unnecessary traffic as you're thinking about multiple person projects independently of the security issues and other things we've talked about you want to look at other things to consider as well to make it possible for collaboration given that I think in the case of media this is a pretty new world in terms of able to get root you know we kind of you know if you will you know completely seamless that we know flow of collaboration between multiple users that actually needs to be a little thought right now you know from the standpoint of how you lay out your projects and work in order to make it you know work more effectively as far as so for final cut for example you may want to look if you've got a really big project that you're dealing with multiple people that you actually find ways to break it into sub projects in the film industry they usually do 20-minute real for example you know you don't do the whole you know multi-node movie it's actually done in 20 minutes it's client in a little segment you can actually break up your project into this and it makes it a lot easier to work with and in the case of Final Cut it's actually pretty easy to create a master project that brings all the pieces together you want to think about and make you know watch out for that you know multiple uses shouldn't be modifying the actual thing in the same files from a developer perspective this is something that you serve you know certainly we're used to in terms of you know CBS and you know other kinds of tools and facilities there's not a lot of that kind of you know facilities right now in the media space there are some media management tools that are out there but most tools that exist today you know don't handle concurrent modification merging and other kinds of things that you might expect from you know source code or you know software development and you know multiple person projects the multiple personal project model has been very hard to to get to without something like this and where you know this logged media is actually can be shared in a real way so this is an area where there's weakness you know in this particular an area what I think there's also a lot of opportunity but you want to think about you know how the structure and organize things to avoid these kinds of collision usually for media files this isn't a problem something like Final Cut Pro or logic or you know the various you know dawes and other you know digital audio workstations other things most of them are in a non-destructive editors the media itself tends to get modified rarely if ever it's usually what gets modified is the project files and you know corresponding render files and things that are done so the good news is you probably don't have to worry too much about me people modifying the same media files usually but you need to think about it I mentioned early on it it's really important to me to consider making sure that you use the central directory service in terms of dealing with multiple users I want to reiterate that again it's one of those things that if you don't think about that up front it will bite you I you'll come back later and you go yep you're right you need to think about that the other is making sure that you can you know do that organization up front to enable the storage tuning later down the road two really important things that require you know some forethought so now we're wrapping up here the looking you know kind of forward from this point one thing that's really interesting is that with the introduction of X and and with the just the dynamics of the market certainly I think Apple and final cut and xn are itself are really changing the economics here this is something that used to take hundreds of thousands of dollars of infrastructure and cost in terms of the workstations other things that evolved to be able to put together a shared media storage environment but something like a fan environment together and the economics at this point are changing in a very dramatic way and we've seen this you know certainly apples been through this kind of economic transition in a number of different areas before you know desktop publishing certainly video production you know in general and now as we looked in terms of the shared media space I think we're going to see the similar you know kinds of change the economics are really going to change the market we're going to see a lot more of this going on there's some really interesting interesting technologies coming in tiger that actually they're really going to help here I think spotlight for example is going to be a major major major plus here as you now look at now having storage media environments for metadata can be attached to on the media and it can be searched and found very quickly and effectively means that sharing masters of information actually becomes very feasible and you know very practical and it's been 11 of their other real challenges looking at things such as you know karate and core video just so many implications of their the kinds of things will see in the production space are going to have another big impact as well because they're going to be more and more people involved in the creative processes these tools become more and more approachable you don't have to have expensive hardware to be able to do a lot of these things the challenge I think as we move forward is how to get media management you know more as part of this integral process I think there's a real developer opportunity there and I think you know something that is an obviously big hole that needs to be filled in terms of now how do you manage the media through the workflow and there are some good solutions out there we're bringing down to the scale at this level and to be more approachable I think it's going to be one of the interesting opportunities I think the other thing to factor here is that is just the actual workflow process itself I think is going to be an interesting part that this enables that you know now having vast amounts of information available to many people in effectively real time at the desktops means that you know managing how the material gets through the creative process through multiple people and gets trucked through there which is beyond just media management but actually workflow process management is going to be another big developer opportunity another opportunity within the space hopefully there are people out here who are thinking about those kinds of problems and certainly love to hear about the cool things that your guys are doing