WWDC2001 Session 406

Transcript

Kind: captions Language: en oops so I'm crystal crime the engineering manager for the streaming server I've been doing that for about three years I guess I'm going to talk to you kind of at a high level about the QuickTime streaming server so some of the things I'll tell talk about our kind of explain what the server's do what they are I'll give you some status on our latest version and I'm also going to give you an overview of some of the functionality and some of the typical configurations of the server which is something that's in a documentation but it might be nice for me to get up and actually talk about it a little bit I'll give a real high-level overview of some of the streaming protocols which is important because it's actually possible to write tools around these protocols that will interact and interoperate completely with the streaming server and the streaming client I'll also mention a couple of ways that you can extend the streaming server go through a few of development opportunities and I'm just going to talk about our general future direction so what are the QuickTime streaming server and the Darwin streaming server says three servers in one but actually it's more than three but the three major things we do our video on demand which basically allows you to serve stored content off of a hard drive it's a lot like a web server serving static HTML except the server streams movies instead of serving up HTML we do the server does live stream reflecting and what that means is that you've got a broadcaster or some kind of an input source sending media to the server and then the server reflects that out to massive number of clients thousands of clients we also do something we call simulated live using our playlist broadcaster future and what that allows you to do is to have stored content on the server and have that broadcast kind of in a delayed fashion as if it were a live broadcast it's really good for doing delayed broadcasts obviously and also for doing things like setting up internet radio stations you can actually create playlists of songs or have videos and play that out to the server on this if it were a live stream the server is completely standards-based when we decided to get into the streaming server business we kind of took a look around and decided that we didn't need another a third proprietary format so we decided to go completely with standards and we actually believe in it so much that Apple's one of the founding members of the internet streaming media Alliance which is an organization whose Charter is basically to make sure that mpeg-4 streaming and mpeg-4 in general works in an interoperable fashion between different vendors the server scales really well it on a single processor machine we can do over 2,000 concurrent off of disc movies or off of disk connections or streams and we can do well over 3,000 when it's reflected from a live broadcast that's not a single processor machine there also ways to kind of tie machines together and do even larger larger broadcasts which is was a real a feature is the relay feature allows you to set up a collection of machines so that they one one server can or one relay can relay out to other relays which allows you to do instead of maybe two thousand client you can have ten machines and do twenty thousand clients and we of course support UDP UDP is for those of you don't know is kind of the sister the brother to TCP except that it's not reliable and that's the normal way to stream media using RTP which is a standard protocol we also can tunnel over HTTP and the reason we do that is to allow extremes to get through firewalls that are not configured to allow RTP traffic to come through so basically a request will look just like an HTTP HTTP request to a firewall or to a proxy and everything I'm all the media can get through that in trouble that way that is people love us for doing that the QuickTime streaming server and the Darwins streaming server are also open sourced so a point of confusion for a lot of people has been that people are continually asking what's quicktime streaming server and Darwyn streaming server what's different about these two and the answer especially as of the latest version is that there's absolutely no difference whatsoever we simply use QuickTime streaming server as the brand name for the Mac OS 10 version of the server and we use Darwin streaming server to refer to any other platform that the server runs on and to the open source project so we build obviously for Mac OS 10 and we've been we ship a binary for that we also build binaries under the darwin name for linux solaris freebsd and windows NT 2000 and we actually build these things and treat them as a first-class citizen and in terms of being equal to Mac OS 10 we do test those those versions of the server we do put effort into making sure that they perform well so you can download the binaries and it's you can be assured that it's at least as good as Mac OS 10 in addition to that because we're open sourced developers have ported to Darwin streaming server 2 to a wide variety of platforms basically every major platform you can think of so it's been ported to Mac OS 9 by us and people in Brazil it's been ported to SGI's I Rick's it's been ported to to Hewlett Packard's hp-ux Compaq ported it to their tru64 operating system it's been ported to basically every flavor of Linux in the world and so it's available on almost everything I'm not I think I heard something about Abby OS version but I'm not actually sure if that's out or around right now and because it's open source we also make the source code available to the public in kind of two forms you can download the source code for any GM version just as a tarball so you've always got a copy of the GM version of the source code in addition to that we actually do our development in a public CVS repository so if you were to pop over to our CVS repository right now and do an update or a checkout from there you'd actually get code that's in progress by the Apple engineers and various the darwin developers and some it's a great way to help us out you can download that stuff and if you see above fix it for us submit a bug and we'd be happy to roll into the into the CVS repository for you so where we at today probably all know that we just released QuickTime streaming server 3 Oh this week it's available for download on the web as an installer and that can install on Mac OS 10 desktop or Mac OS 10 server but it's actually already included with Mac OS 10 servers one of the standard services on that package run that product and of course we've got all of the the darwin binaries up there for download as well for all the various platforms that Apple support that Apple actually builds on and so what's new in 3m kind of give you a high-level overview of each of these I'm gonna go any detail into these in a second skip protection is a new feature it's actually not a feature it's more of a collection of quality of service enhancements that we've added to the server and it's kind of an ongoing process for us to always improve quality we've added a web-based administration to the server so if you're familiar with streaming server 2 we had for Mac OS 10 a little cocoa based app that did some limited administration it looked nice its aqua fide but for the other platforms configuration basically meant that you needed to get over to the server telling it in modify some text files stop the server or send a signal to the server and hope you got everything right hope you didn't screw up the config file and we decided that was too complicated so we decided to do a UI for all platforms we've strengthened the authentication and by that I mean dedication that happens when a client tries to connect to the server and access a movie that's been protected and we've also in conjunction with QuickTime 5 foot 10 5 added some api's that allow you to do live broadcasts and to do basically encoding from an in from an input source broadcast that out to a server so we support that we also and we support some of the new features they added such as announced support I'll get into that in a second and broadcasting via reliable transport we've also done a couple of API enhancements that I'll talk about so skip protection if you guys were at the keynote you probably saw avi do the demo where he had two movies up side-by-side he pulled the plug which was intended to simulate packet loss the the old 2o version of the stream stopped and the three overs and just kept on going there are a couple of things that we did that allowed that to happen so the first thing that we do is we now do intelligent retransmission of lost packets basically the client is continually telling the server which packets that gotten lost and the server makes some decisions based upon whether the packets whether there's enough bandwidth whether the package is really important or whether the client will even be able to use it it may be too late to even retransmit so it does some intelligent retransmission then the and in conjunction with that we actually do congestion avoidance you know in our algorithms we don't just blindly treat retransmit things and the reason we do that is that if the server's being notified of packet loss and we just start retransmitting more data and it turns out that the packet loss is caused because there's not enough bandwidth or there's a congested link somewhere on the net all we're gonna do is evaporate the problem um so we try to be really smart about what we actually retransmitted we in conjunction with this we do something we call over buffering and what that is is basically sending the media and faster than real real time and the reason we do that is that typically you've got a three second buffer in the client in what over buffering allows is for us to stretch that buffer out and it gives us a lot more time to perform retransmit so instead of performing retransmits up in this little 3 second buffer we may have up to 25 seconds to do a retransmit which gives us a lot of time to recover from just temporary conditions on the internet and things like that and so over buffering is cool but you don't want to just like send things as fast as you can we actually need to know how fast of a connection each client has so we have something we have we built some technologies into the server that actually can detect the available bandwidth for all of each of the clients that are connected and we use that when we're doing the over buffering in addition to that we've improved our thinning algorithms and what thinning is is there are situations where there just isn't enough bandwidth and somebody's asked to watch a movie that maybe requires 40 4k but their modem is only running at 40 K so what sending what thinning is is the mechanism where we actually select packets based upon their priority and just don't send those and the idea there is to and to decrease the bandwidth of that stream so that the client gets at least some semblance of the usable stream in the past with - oh the shifts when we were doing thinning were pretty abrupt you'd be watching you'd see a movie would look like crap we would sin it would immediately go to a slideshow if that didn't work we'd go down to audio and basically we've just got to smooth that progression a little bit and skip protection is like I said it's not a feature it's just it's just our umbrella term for quality of service enhancements to the server and the things we've added for three are just the beginning we plan to do quality of service enhancements for every release of the server moving forward an obvious goal is to get the broadcast quality at some point then even beyond that better than broadcast quality there are some things we think we can do with interactivity and just really high high quality video high resolution video that we're gonna be looking into over time it's gonna take a long time obviously next new feature is our web-based administration and one of our goals with kind of from the beginning with the servers that we wanted to make it extensible for developers and we wanted it to be easy to extend so we kind of took that philosophy to heart when we did the web-based administration and the way this works is we implemented it as basically a collection of HTML that contains some tags in it and some Perl scripts that know how to parse that those HTML files and read those tags make a request off to the streaming server for data and kind of reform at the HTML so that it looks like normal HTML on the web browser we built kind of a mini web server based on Perl which just a very simple little web server room for actually talking to the web browsers the web browsers never talk directly to the straining server and as I said it's extensible just by replacing some add in simple cases just adding some additional HTML with whatever tags in more complex cases if you've got some complicated data data that you need to parse and you might have to write a little pearl CGI that does the parsing for you and of course it's secure we the little mini webserver actually does SSL you know I probably have this upside down don't you okay and here the screenshot I'm actually gonna demo I was gonna show you just a screenshot but I decided that a demo might be better just gonna quickly show you a couple of the features in the web based admin so this is kind of the initial screen it's basically just a snapshot of what's happening with the server hopefully you guys can all see that in the back if I if I had a lot of I could check my CPU CPU load here I can see how many bytes the ministered I can see how much bandwidth this winged out it's kind of a convenient thing here are some of the general settings here you navigate just by moving through these tabs and compute the logs in the admin there's an error log lots of errors and another thing we added so is UI for the playlist broadcaster if you guys are familiar with that you'll actually like this in the past you actually needed setting up the playlist broadcaster on all platforms really consisted of modifying multiple text files and people inevitably got it wrong and so we added UI here basically you can create a new playlist here all the movies listed down here that are in this playlist you can tell it how you'd like it to play sequential weighted random you can tell it to turn logging on things like that and then turning this on or off is just a matter of clicking the little play button here and the broadcast automatically gets started up and automatically starts talking to the server and you've automatically got your internet radio station or your broadcast happening I guess you guys hated the text files on announce support so like I mentioned could time 5 has added they've made the broadcaster api's public so in the past we had there was one broadcaster with Sorenson and the way that worked is that you would set up your broadcast plug your camera in he'd have the Sorenson broadcaster generate an SDP file which is a file that had basically a description of the session you would have to take that file get access to the server that you're gonna broadcast to in a lot of cases he actually had to go in and modify the SDP file which meant that you needed to understand the SDP format and after that you would start the broadcaster up and typically it wouldn't work the first time you look at your FTP file figure out what's going on basically it was a mess and we didn't think it was easy enough so we decided we needed to automate that so we've added some support to the broadcaster api's to use the standard RTSP and to use the announce method in our TSP and what that means is that all you need to do now is type in an IP address on the broadcaster application specify the name of the broadcast and press press start at that point what happens is the broadcaster will send all of that information that was originally in the SDP over to the server server will say first of all or it'll ask are you okay to do this and actually authenticate that broadcast or been the reason for that is you don't want to have anybody being able to broadcast to your server what once it decides that the authentication is OK it'll let the broadcaster know things are cool go ahead and send me over all of the information that I need and the broadcaster will start broadcasting and you're done so basically it's type in a couple of fields push a button you're done in addition to the announced support for broadcasters there was a problem with the original broadcaster broadcast only on UDP and if I mentioned UDP is a non reliable transport which means that packets can get lost so that's a minor problem when you're a client watching a stream in one pack one or two packets get lost and then everything is fine after that it's a major problem when a broadcaster is sending data to a to a reflector and it loses packets because then every single client loses those packets so we decided we needed to have a reliable transport and we decided to just go with TCP rather than doing anything real complex the problem the second problem with using TCP is that the buffer sizes are typically too small so the reliable broad with reliable broadcast the broadcaster is actually able to tell the server how large it would like its buffer so it can tell the server I'd like a 1-minute buffer and what that allows is for situations where TCP gets into congestion problems or maybe a little little flow control problem it gives us give us the broadcast or plenty of time or the TCP stack plenty of time to get caught up and make sure that basically all the packets get through and everybody's happy which means that all of your clients will receive much better quality stream in the case of packet loss between the broadcaster and the server so those were those are the new features I'm going to talk about some of just some of the typical configurations that you can put the server into and the thing to the thing to notice that these are not modes that this River goes into the server can be configured to do any one of these or all of them at the same time these are just different ways the server can be operating so we've got video on-demand live streaming simulated live streaming and we have a more advanced the advanced configuration is real a streaming slow go into those more detail actually have pictures there for you guys to look at so probably the most typical form of streaming is stored media streaming or video on demand and this is actually as I mentioned just light or it's analogous to web servers serving static HTML basically all you need to do is take some hinted movie files drop them into a directory on your server the client will make a request to the server just using a URL it looks like an HTTP URL except it says RTSP server will go out try to find that movie and then just begin streaming it to the server and as I mentioned these movies needs need to be hinted and what hinting does is it takes a standard QuickTime movie and it adds some additional track for each track in the movie it adds an additional track and that track basically contains information that the server uses to determine how to which block which blobs of media to put into which packets and what time to send them out so and one of the big advantages of that is that it allows the server to be completely agnostic about media formats that's why we're able to stream media on a linux server or freebsd server where quicktime api's don't exist in the codecs don't exist next configuration is live streaming which adds a couple of pieces to the puzzle first you need a camera or some some form of an input source and what you do is hook that camera up to an encoder broadcaster machine and the job of the broadcaster or the encoder is to obviously encode the incoming stream and then to packetize it into RTP format and send it off to a server or a client so in this picture though you'll see that it sends it off to a server and there's a really good reason for that you technically can send things directly from the broadcaster but decoding and encoding can take quite a bit of CPU and so it's not really not a good idea if you want to scale to any more than a few clients so what happens in this case is that the encoder encodes one stream and broadcasts one tree stream to the server and then the server which has hardly any load on it because it's not doing any encoding can broadcast out to thousands of clients simulated live this is the playlist broadcaster I just showed you and the way that works is kind of a combination of live streaming not really and video-on-demand and the way it works is you put your hinted movie files onto the hard drive of the server and you create a playlist which can be either a single movie or unlimited number of movies and then there's a little tiny process that runs for each playlist on the server machine itself and it basically goes through and parses those files reads the hint tracks and then streams those things and local loopback to the to the streaming server which then broadcasts that out to all of your clients so what this looks like from a client perspective is a live feed even though it's really not there they're not gonna be able to thumb back and forth in the movie they're all going to see the same thing at the same time and it's really useful for doing things like delayed broadcasts let's say you've got an event that you pre-recorded you can drop that on the movie or drop that on the server create a playlist with that individual file and it started up but whatever time you want to start it and then everybody can tune in and then as I mentioned it's really it's also really good for creating things like internet radio stations if you go to yeah actually go up to Apple's put time TV site and look at Warner records for example they've actually created kind of a mini mtv using the playlist broadcaster it just shows video after video throughout the day and more complex and probably doesn't affect most of you but it can certainly affect some of you is the configuration we call relays streaming and in this case and relays dimming is used for live or simulated live and it's used for allowing you to do events that are much bigger than what one server can handle so basically we've added just another layer between the original it's the fervor connected to the encoder and the claims so you've got the live stream coming into your relay server and instead of that server broadcasting out the clients it broadcasts out to other servers which means that you can have instead of one machine doing 3000 you could have maybe 10 machines doing thirty thousand concurrent streams so it allows you to do really large events this is effectively what companies like Akamai and digital island are doing ferd for for their live events they I don't think they're using our exact our code but they've got something that looks really similar to this so I'm gonna go over the kind of the three major protocols we use really briefly here first one is RTSP and it stands for our real-time streaming protocol we second one is RTP and real-time streaming protocol actually is kind of the control mechanism for the client from the client to this river lets the client tell the server what to do real-time protocol or real-time transport protocol is the protocol that the media is actually sent on and that operates under UDP typically it can be on TCP we do both actually with HTTP tunneling and rtcp is real-time control protocol is actually used by the used you know the way it works is that the client uses that protocol to tell the server how things are going are we losing packets is there Network jitter happening things like that and I've got a little picture here that'll either make it easier to see your heart ride around so on the top we've got RT SP and RTSP is based on HTTP it actually looks almost identical the difference is that instead of HTTP in the header it'll say RTS T and instead of methods like get and put it'll have methods like play and pause and describe and set up RTP runs on a different port RTSP typically is on port 55 54 RTP happens on a different port from the server and that's the stream that the media actually gets gun and RTP basically is the way the package look it's pretty simple it's just got a little packet or a little header on the top of the packet that is an RTP format and then a blob of medium and the client knows how to decode that media once it gets that repackage or D packetize it our tcp as I mentioned is a status channel from the client back to the server and that section that's what the server actually uses that for doing all of the quality of service and features that I talked about with skip protection well that's how we get notified the package for getting lost and which packets are getting lost oops here's an example session and I think you guys can read that so kind of walk through what happens when somebody presses the play button on a movie so the first thing that happens is the client will send an RTS P describe request to the server and that contains just a URL basically and the URL looks a lot like an HTTP URL it'll have the server name slash and then a path to the movie and the server responds to a describe request with information about that movie and the information that it responds with are things like the number of tracks what types of media each of the tracks are and things may be a bit rate of each track the language of the tracks things like that so once the clients got that information it can then make a decision about which tracks it actually would like to listen to for for example you might have a movie that's got a single video track and three language tracks Spanish English and French the client will then send a setup request for each of the tracks that it actually wants to listen to so in this case it's going to send back a set up for the video track and a set up for one of the audio tracks that it would like to hear server will respond with OK's to both of those and then the client sends a play request which tells the server okay now start streaming me the things that I asked you to set up at that point media just gets sent out over RTP everybody's happy nothing more happens until the stream either ends or the client it sends either a teardown request if the movie wants to be completely shut down or somebody presses the pause button and they just want to the movie pausing the movie is different than shutting down the movie because the server actually maintains the state of the movie so that if you press play again it's going to know what to do so that's kind of it for the protocols there are a lot of ways to extend streaming server and I think I don't think I think it's pretty clear that the QuickTime streaming server is easily the most modern most extensible streaming server in the world there are other servers out there they're not typically extensible other than maybe some little module api's we have a lot of ways to extend kind of the most obvious way is that the source code is all available and you can take the source code and do whatever you want with it we have source for the server we also have a proxy server that we have source for all of the source codes available for the the admin that we added for 3oh and the only the only issue here is that Apple's got a license we call the Apple public source license and with that license says is if you are going to make modifications to the to the core code of products that are open sourced by Apple we just ask that you send those changes back so that we can share them with the rest of the open-source community other than that you're free to do whatever you'd like and a lot of times you might make changes that have no meaning to us we're not gonna roll that into the server but you still need to post that for the public can see it there's a way to get around that with the streaming server though we recognize that there are companies that are gonna want to build things that have some some really strong intellectual property value or are just proprietary they don't want other people to see it the way to do that is to build an 8 module that attaches into the server and the reason it doesn't fall under the apsl is that you're not modifying any of our source code you're actually just writing brand-new source code that belongs to you the plug-in API that we use for these is really similar to the Apache plug-in API if you're aware of that it uses concept of roles and callbacks in different during different states in the server there are two ways to build modules you can build them as Dinan dynamic modules and those are basically standalone little shared libraries that you drop into directory the server auto discovers the asks them to register for their roles and the way they go or you can compile in compile the module directly into the server and then it just becomes a part of the single server binary and in fact most of the server features are implemented as modules so things like the ability to reflect live streams as a module error logging access logging authentication the module that lets feed web-based admin work those are all done as modules that are just built into the server we did we we think we did a really good job of making sure that that API or is it that that architecture was really high performance because we use it them another way you can extend the server new with three oh and maybe it's not an extension of the server but just kind of build the ability to build some products around the servers to use the admin protocol now the admin protocol is basically a way to get and set attributes in the server using simple HTTP requests and the requests will basically have a URL to the server and then maybe some parameters at the end that tell the admin the admin module whether you want to get a value set a value to lead a value things like that and this is what the web the web-based admin uses this primarily so the things that can be built around this are like I'd like to see somebody write some really cool monitoring applications they don't have to be written in it in Perl or anything like that they can be written in any language because it is just straight HTTP maybe some advanced configuration or advanced administration applications things like that the kind of the last way to extend the server I want to talk about is by extending the web-based admin so the read there are a couple of really good reasons for doing this first one might be that maybe you don't like our admin maybe you want to repurpose it for your for your corporation or you want to repurpose it and sell some to sell a product to somebody you can just take the existing admin and modify it probably a more common reason to extend it would be that you write a module for the server and you want to have some user interface for your module so what you can do is just create some HTML files with the special tags possibly write a Perl CGI drop it into a directory and it'll automatically appear in the user interface along with the rest of you it'll be along with all of the UI that we've written it basically becomes a first-class citizen in the admin you might might show up if there's an additional tab in our admin or something like that and all of these some ways of extending the server there's a session immediately following this and we'll have some streaming server engineers up to show you how to do each of these extensions see a little bit more of the web admin and a little bit about the API as well as how the admin protocol can work so development opportunities so we have we don't have a gigantic team at Apple working on the streaming server with so we can only do so much we think we think we've done a really great reference River we think it's a really high performance server but there are like hundreds of features that we're just never gonna be able to get to because we don't have enough people to do it we do have on the other hand 500 darwin developers that are that are building things around this thing so we really want to get developers out there building things around this possibly building the server and repurposing it as your own product we actually really promote that as long as you follow the Apple public software license there's actually absolutely no issue with doing that so a couple of ways you can a couple of development opportunities might be to port to some other platforms most of the major platforms are done but if you're doing maybe a network appliance server like NetApp does or Novell I think has something Tomi you can take the server put it over to your device and where you go and you can and resell it there's no issue with that at all from Apple there are also a bunch of plugins we'd like to be see see written so we have some really basic authentication in the server I'd like to see a module that does something keishon with a stronger back-end authority maybe or with with a database of user accounts it's also possible to write modules that use a database as the means of storage for all of your content I think that you know that's kind of a vertical market but there's definitely not there and of course you can do additional logging and reporting modules and maybe a cool feature would be a cool product would be to kind of build an entire broadcast system for internet broadcasting similar to what TV stations use they've got software that manages all of the scheduling and which content to serve and add insertion and things like that and might be interesting to see somebody write some modules into the server that do that and reaper and kind of go after that vertical market and as I mentioned monitoring tools via the admin protocol admin protocols news so I hope to see a bunch of those coming out and of course web-based administration extensions possibly do a web-based admin that's purposed for some corporation and sell it into the corporation things like that so our future directions are actually pretty simple we plan to continue working towards standards and to support standards we actually work with the IETF to extend the protocols when we think that they've got some limitations a good example of that is that we recently proposed an internet draft to the IETF that makes it much makes it possible to write streaming caching servers that work much better with with streaming media nobody else has done this and it was something that was desperately needed basically the streaming caches were at the mercy of the network they didn't have any information coming to them about packets that were being sent to them or ways to get the media sent to them in a reliable fashion and so we worked on an extension with the IETF to do that as I mentioned we're also Apple is also a founding member of the internet streaming media Alliance which is basically an mpeg-4 alliance and we are big on making sure that mpeg-4 works with all of the different vendors and I think you're gonna see a lot of and for coming out of Apple over the next year and of course we plan to continuously improve quality and of course performance sometimes they get lucky we get a really fast machine and that helps us automatically but we're continually looking at the software and looking for ways to make the software perform better and of course we'll always be looking at adding new features and looking for other leading edge technologies to add into the server so there are a few places that you can go to get more information about the streaming server the first place maybe or maybe the second place or third place I'm not sure it depends on you but we have a mailing list for developers and that list is actually it's a really high-quality list I want a lot of mailing lists some of them are not high quality this one is good it's the traffic is probably between 5 to 20 messages a day they're all very focused and very development related it's a great way to have contact with the engineers because all of the engineers and and some of the marketing people are actually on this mailing list and monitoring and replying to message to questions so I highly recommend you go to that the best way to get to it is to just go to lists Apple comm and search for the word stream you'll find a developer's list and you'll also see a users list which is interesting too if you want to find out what users are doing or if you want to become a user and figure out how to use the server better all of the open source information the Darwin versions of the server are all available on Apple's open source website at WWF in source code comm and for things that are more client related or authoring related you should go to Apple's the developer site for QuickTime that's developer.apple.com slash QuickTime and then even though you may not ever write any networking code it's always it's actually a good idea to understand what the protocols do and how they work and so go up to the ietf site and check out the RFC's for each of the proto for the major protocols we use our TSP is 2326 RTP is RFC 1889 and SDP is and here are the sessions that are remaining at WABC that are related to the streaming server so immediately following this session in this room you don't even have to get out of your chairs is a session on how to customize and extend the streaming server and we'll have some streaming server engineers up for that there's a feedback forum tonight at 3:30 in room j-1 and that's going to be that feedback form is actually intended for both QuickTime client and server issues it's just kind of a uber QuickTime and feedback set for them recommend that you go to that and of course there's a beer best at Apple tonight we'll have engineers in either the we'll be either in the server area or the QuickTime area for me it depends on how close the beer is if you need to contact anybody with developer related questions you should contact Jeff Lowe he's the quicktime technology Manafort manager for Apple Worldwide Developer Relations his email is Jeff flow at apple.com he also handles them so he handles both quicktime River and client you