WWDC2003 Session 110

Transcript

Kind: captions Language: en I'd like to welcome you to session 110 overview of Mac OS 10 networking api's my name is Tom wire I'm a consulting engineer in our field engineering group and I have the fortune of introducing this session there's a lot of things in Makka left n that have changed over the years a lot of new features have come into the operating system and there's a lot of material in this session as you saw in the keynote we have new VPN functionality new enhanced support for ipv6 and a number of other key features so with that I'd like to welcome up I'd like to welcome Vincent lube a manager of core OS networking good morning so this is what we're going to go through today so we're going to start with a brief overview of the Mac os10 networking a little bit the architecture what the new features and then we are going to dwell in to a networking networking API they have a lot of networking API so it will be a relatively high level just more a list of features and maybe hopefully you will have some some ID which you know depending of your knees which API to use and basically we have system level API and then also framework api's from a clear goal and Becky will do the second part talking about the frameworks so we have ongoing goals and they haven't changed since last year and the year before I think so the ease of use we'd like to have the networking be as transparent as possible you know so one one great example is horrible the other zeroconf fo performance also go important we have the same kernel the fun zone on a laptop or an also on an excel so performance and scalability is an ongoing goal for us extensibility that a common feature from you from a lot of developers and also standard compliant whenever we can speak to stand out we would like to do it so the curve of the networking is in in the in the corner that Darwin layer and basically that where we r us that we talk start that is not used very much these days but the tip also the tcp/ip stack and all of that would cost services and a lot of time ourselves the applications environment so my quest and the stack is based on on freebsd last year web data to suggest the 3.7 i remember something like that no fog and so with that you get the so it's a unix-like protocol stack cbse so you have a lot of code that runs on other universes that are very easy to bottle my question the CCP stack includes architects and ipv6 when introduced at last you've already so we still have superb horrible talk fully most access suite of protocol based on PPP the seconds with the firewall and also not email the cool stuff we have we have added two unique dynamic configuration based on on the framework with a plug-in architecture so that the part with we're trying to push the the system so that the configuration reconfigure automatically and of course we have on table and that was the major breakthrough of last year so what's your incentive so more only we are aware application and services so she went to the three oxidation yesterday I used to a lot more applications and that you are like I'd like to sorry and more Webb also more ipv6 aware application and services one of the so for example AFP the one that you are an inventor runs over ipv6 we are introducing also an ipsec VPN I got a tow to the twin x four link layer of application even better support for mobility and WebKit and all the safari framework and a lot more so we're going to go through a little bit so a CO 2 dot when X is a link layer application that means that that's an open user authentication that allows that that runs before really the your link can can go up so it's it's really the come to be the prevalence a way to secure wireless LAN but also it's also use the internal switch environment for Ethernet and in infancy we support the wide range of application method a 22 the 20 x is still a young protocol so not clear which of those of application protocol will be the equivalent but we support you know all that list that has the one and that are used by fiscal microsoft and other vendors the apple VPN so it's a client and server and so we use the IP check for the provider ready the sexual sexual layer and we use also l2tp for interoperability vsvp is in fact it's it's running PPP over over unity and that provides interoperability with windows fiscal and things like that that's really the the really delicious standard VPN photocall out there and so we quicker a yes for the encryption for those who know the cryptographic algorithm its exploitation but we still support cripple death and other an algorithm and we have also added support for secure ID as very popular ways to provide the user notification so having a lot of deployed in your company but you have also other photos of an S chart also v2 and things like that so we are the different level of networking API so that just shortly and we are going to go through that in the drive to the station so first going to start by them the bottom if you like that sort of the system level API it's more or less what you find in Dow in the open source implementation so i'll start with the vsd socket so that the fundamental api for networking everything else is it mostly every other networking api is based on bsd socket and so that's the one you use to get the most out of the system that that's the rocks s to the to the colonel is done yes okay so when will you forget for performance so she 17 government so if you're writing a server application with the US insurance two of the most of the of the system you have a very fine control of what what's happening you have access to all the low level low level control complete ID sig.png RV and things like that next up about that and also bsd so it brings unique compatibility and possibility so on windows that winter collectively closed so it also need it's a good a good way to ensure profitability with the beardy tail gate comes the resolve a library so it provides name and address resolution and in fact they are there are 22 level of api's so some of them works with other or lymph system not only the dns and it goes on and with the socket API in center they been updated to bind line to the latest and greatest and we are also full and tv6 support that means that you can resolve an IP ipv6 address using the the resumption but also you can resolve aerobic different and each and we fix so it's the the full the full set so idealistic so we implement the standard API out of forgetting an extension of the basic socket API something that we turn off that was the nice nice functionality was mapped address that we are able to use when you wanna serve where you work for multiple we're using just just one socket a v6 circuit and because this accelerated a larger-than-life addresses with the option of maps address you could have also christoval so before before connection on the circuit for security reasons we visible that by default it's off the disabled bodies off by default because some security reason implication they are some implementation out there that in fact use gonna lead the way to eternal vitro increases and sometime it like that firewalls and people are not very happy with that so in answer it was value for the field circuit option if you know what you're doing you can use it my question does internal different physics but they are following generation that do and so with the DNS over ipv6 inventor and so the idea here is for application last year we are the Food Association on on ipv6 and how to do code for advice family independent it's it's really easy to write the program that's probably what we saw andy sixx that's why those aps are after that and we encourage you to do that and so the one of the web stuff about diseases only who works over ipv6 so I thing any a very easy way to find out services and test stuff on the network I don't need to ever with ipv6 you don't need to ever and infrastructure we fix as automatically configure our addresses link-local addresses on all the interfaces so you can find services it works perfectly with my GMA will be on the system ipsec all right IP level security from within the camel the bottom of the struck by we support a bunch of the number of API to set those security policy and Association database the world those are the kind of the filter of the tells you what traffic to to anchor it using IPSec that the PSD so cal ipi successfully hit the library function and also you can either just forget control so we thought you already a few jaguar f you application of us application using a decade gap so the new thing is that in intention we also using a piece XL VPN so there is something that is something to remember is that this is a single Shadley system white crystals so and it's managed by a hacker the internet key exchange the one the protocol that exchange the key is managed by a single diamond callicoon and so the single resource so please if you use IPSec make sure you don't override another question answers or taking some other application and with added also not possible in intent well thats something that's truly in these days with a lucky revenant and so we support that that it's not just ordered it still a ITF class but that's what will you that's the best that's the way to have a nut with a TS exact so most likely to be interoperable of the expansionist order so we have an ID firewall that works so with ipv6 inventor and again just like with ipsec and BR cool it's a single system-wide resource so with ipsec you you can design filter actions to do on certain type of packets but just remember a single word resource and just like so far about you've certainly notice that the built-in firewall will not will not chemicals on if detects there are other words on the system so the big denied and also overwrite other people hold on you can ask the user if you want to flush those and there are two ways to control the firewall there's a comment line IPA value and also just straight forget option PSN grv circuit so PSN dear mr. gates are so getting that allows to you to write users based protocols that means that they they really that facility we have in the kernel using this type of circuit 2 to send the role ethernet frames up to two years of space and we use that with a 2 2 dot when x exam internet protocol in fact and so that that's a great way to stay out of the corner and if it's also use a total can be used to implement all the protocol stacks that are being specific for an application like decnet or ipx on orders there are other solutions so that you also can be used instead of peer can be Aristocats also to implement user space port o call those are all set to the digest circuits that worked with the firewall and MVPs the Berkeley packet filter so we even though the state of the internal API for the top generic extension we really like to if possible that you use us intimate report volume in your space if possible so that joint can be sure that there's no problem option compatibility in the future so let's talk about the problem I mean the network expansion so today that they are the nke are tied to the colonel implementation so we cannot really say that we have an API 3d will provide hooks wear extensions can can really get to the internal mechanisms that Hydra is going to break it broke in the past every time we will never speak for example or want to to improve performance we have to think about it is going to break a third-party kernel extensions so if you came to the decision about the DePiano yesterday morning you certainly aware that we are working on on your new API that will I isolate the extension from the implemented internal implementation of the camera and it's not it really is underway and we certainly liked you or your occupation and feedback because we are going to do certainly some drastic changes in their system configuration so that the condition for the our mobility solution and it's it's based on a diamond and a couple set of API so we are set of epi to manage assurances that's the one that I use for example by the network's accounts panel so if those are system wide local catherine 444 big holes and we have also NEPA to get the status about the state of some parameters so that that's the basis for the dynamic behavior so there are modifications when the desert changes in the state of some data or key or the questions and they are used today i also buy a lot of ugly applications for the whole ISPs once will provide their own dialogue they deduce that and also application that how they can dynamically react to a networking change using the status and is used in enforceable by nayland safari and also the basis for the dynamic demand so what an application they use usually the presidency I just talked about are more used by four to manage the configuration that we are napi called the SD network capability api that allows to to check whether set of maybe nel addresses of Paracelsus love inventor can are reachable which ability doesn't it's not a guarantee that the for example the distant host is you can really connect two distant holes but what it says is that from the standpoint of the of the system locally on the machine there's an IP address a router if there are maybe there's also dns server configure and so it's this recipe is so unified a set of information you know it gives you a synthetic view of it so when to use AC network capability so you know it just to tell you is the you have a chance to connect to the server maybe this server is so it will not tell you there's a fire wall of your problem with nut or things like that you know things outside of the control of the system but the visit that you can tell you the the system is properly configure so the when they were changing the in connectivity that's you have the chance to to clean up and maybe to provide some feedback to the user for example CF circuit stream use that on only three circuits dr. Vickie going to talk about the CF cerca di you to be later on but our own framework skews the vcp is Veggie p.m. so something new in center that works in fact with the reachability API is the net SD network connection API the this is the VI that enabled dine on demand from within the application so and it's similar to the TPP remote access epi we add on unlike last seven eight nine so the ID so today what you what we are is really something that we call audible dialogue traffic so if you enable that Katherine's in the MVP the BBB every time it sees that some some some data is to be sent in it will guide the modem and that's because of a lot of fun plans because it's the system dials at the most inopportune time so what we want to do is instead with with having this with this new API to enable application they refer to put to be in control and they can decide or maybe from the user so I need to get on the network emoji of me and the Freak work to die only to die yes no and so far it's great for many clients and and also that the same API that is used not only for application but also for diagnosis different set of parameters the internet connector on internet connection better using TCP I so we would like to you to adopt this applies or put together this particular sample code just to show you a roasting polity so so for example this is how to to tell if you want to call her to access that will double the double decker what you would do you would you would call SD network HIV key create with name so when you pass the 77 m and then you use it if I a call back with these for this reference you get microvessels and specify your call back from Anderson even that happens for the data change in the ability of the video is at apple.com your code back will be called and you just schedule it with your run loop so it was great for our application above the BFD they are editing and you just run the random and then this is the H ability call back when you get another basically the really the major result is the flag that tells you to set of likes you get but the ones actually interesting here is the weather it's whichever normal and they are all sub slags and well give tells you if the connection is required and now in other if the server away so if you need to start the connection you just have to use the connection API and the stop connection epi huge usually when you are not a dialer you just want to use the default the most certainly the most current connection the the use of it with internet connect so you just tell the do so just copy the use of the current restaurant and you tell the system to to create a reference again because that's the kind of the hook the endpoint that you will use and again it it's a can even best mechanisms so you use scheduling the run loop to to be ready to receive a run and then use you call the start and you're ready system should we don't need and again when they have to an event on the on the connection if you mean call back and hear of the the main event to which you are very similar to do cannot access epi and it just really disconnected connecting connected disconnecting and case of error are you getting the Reaper and when you're done you close the connection so the one point in the DCP I the the connection as our last contact that means that the idea that the when the the connect when your occupation doesn't doesn't need to to use the system certainly you would like to the the modern to disconnect so you just release the connection if there were some other applications using the connection it will still hold the reference on the connection so it's also on the last reference that the connection will be calm down and and with that I'd like to oh no one more thing last but not least that was my butt is the is the dns service discovery API saw those after the Holloway p I there was a session yesterday about Oh something is wrong here the decision not gonna programming that it that's the hollywood p is one or two so the name service discovery I saw the foundation for Hollywood and urs you have set of API to do register name services also an API to browse our services on the who is really something for end user so that's the idea that the end user will peak intrusion service it's not so it's not so great for for background connections because really ladies that the user choose browse something and just remember the name and then when it's time to connect to that service you have an ETA to resolve and the service into an address on it cause you can use in subsequent search on a conference call we have a3p BHT ph levels depending what level your application what can type of epi your application is using so we have the dns service discovery API it's a system level API that's the foundations of the other one in CF network there's the CF net services so if you're on loop you know it's sudden it would be the natural way to use and in cocoa there's an SMS services that works for code objective-c and now with that I'd like to introduce the bqe which is going to talk about high marks [Applause] thanks Vincent so visit has taken you through the basic feature set of what's available direct from the Darwin system what's available at the Berkeley sockets layer and what I'm going to do now is talk about what we've built on top of that into the framework layers above it so I was thinking about how to prepare this talk and it seemed with the introduction of Safari 10 the best way to look at those api's was to look at what Safari has built upon and that's how we're going to start out after that I'm going to talk briefly about some of the other networking technologies on the system and finally I'm going to give you some tips and hints on how you can transition off some of the older mac OS 9 based api's so let's talk for a moment about the Safari technologies when we started writing Safari we of course built on the technologies we already had on built on top of the existing api's but we also discovered a lot of places where those api's were lacking and so we developed many new AP is at the same time and now we've made those api's available to you that's part of the Safari SDK as you've already heard from the keynote those api's are available on any Panther system and on any Jaguar system provided safari 10 has been installed and I do not have enough time to go into those api's in great detail so I'm going to point you to some other sessions as we go including session 403 which took place on Tuesday but when you get the dvds you can take a look at it Don Melton gave a pretty good overview of all of those API is in greater detail than I'm going to have time for so what goes into Safari it's on Mac OS 10 so you know the core of the networking has to be bsd sockets but there's a lot of stuff between sockets and what ultimately is the safari application and the first layer in that stack is core foundation a core foundation provides a couple different important things it's not really a networking library per se but it provides a couple basic abstraction for us first and foremost the run loop the run loop is how all of the different interests events inside an application get funneled into a single callback mechanism on our system so if you look at NS run loop or carbon events that's how user events ultimately are transmitted and it provides our basic ation including mechanism CF URLs basic abstraction for URL pure data type and those two kind of come together with CF socket which provides a way to take a socket a file descriptor and connect it into the run loop so rather than having to manage a select loop yourself or a file descriptor set yourself you can use CF socket to get that input managed in the same way that all the other user events are managed CF socket can handle literally any socket it works with most file descriptors too but we we only promise it with every true socket that includes UDP sockets as well as server side except sockets built on top of CF socket is CF stream CF stream is just a basic stream app abstraction you get a stream you open it you read it eventually you close it if you're unlucky it errored out somewhere along the way it's hooked into the run loop so that it can signal the client your application whenever bytes are available or something interesting has happened however because it's a stream based API it only really works with TCP sockets it doesn't make much sense to talk about a stream for an accept socket or a UDP socket so that's core foundation you take a step up some core foundations we get to CF Network CF network is the library on our system that provides the basic protocol implementations this is where our HTTP stack lives this is where the ftp stack lives we don't CF network itself does not actually provide the ssl implementation but it provides all the hooks into SSL and TLS so if you've got a upstream you want to make it secure CF network provides the hooks to run SSL over the socket stream and finally rendezvous this is a Vincent mentioned that there's anything pardon me Vincent mentioned see us services which is the API that we that CF network provides for accessing rendezvous the basic goal besides CF network is to provide you with those protocol implementations and hook them into the run loop however our goal with CF network was not to be the most convenient API there are higher level ap is that provide that convenience at this level we're trying to give you full control what that means is that all the protocol details are fully exposed so there's a lot of power implied with that there's nothing you can do with HTTP for instance that you cannot do through CF network but on the other hand you better know HTTP or the CF Network api's aren't going to make a lot of sense to you other ways we give you full control at the CF network layer you have complete control over precisely when the reads and writes are done again this can be a pain if you're not actually interested in the details but if you need that control this is the layer to get it at likewise the threading policy we do not dictate a threading policy for you at this level at higher levels you will see some api's will actually take care of spawning background threads for you for instance but not at this level couple things new and CF network with panther we've added DNS host resolution this is asynchronous host resolution so you can start a request to look up the address of yahoo com or anything and you'll get a call back later on telling you when when that address is available we've also added ftp support inside TF network and for more information about CF network come to the session excuse me session 1 12 tomorrow afternoon at 5pm yeah ok last talk of the conference but what do you want ok so what do we have above CF network well I mentioned some convenience ap is that do not expose as many of the details to you that's what these new foundation URL API s are all about it's a new set of API is in Panther it's a full-featured set of URL loading api's and here the point is for the API to be as convenient and abstract as possible so the policy is to make most of the choices for you does that mean that you can't modify those choices if you want an unusual setup no of course you can but the policy is going to be to start out by making the most common decision for you and then you kind of have to undo those decisions whereas if CF Network the policy was that you have to apply each and every policy decision yourself the API is asynchronous building on the run loop of course it's basically callback based if you're familiar with the cocoa API paradigms we do it via delegate and at this level we add extensibility via subclassing so if you got to have the Gopher scheme implementation we don't provide it for you but there's a class you can subclass to implement it yourself and get it plugged into the entire URL loading system and again it's part of the foundation library so it's built right into the core and available to you at that layer and for more information about these go to session for 18 this afternoon where we'll be talking about these AP is and much greater depth than i'll have time to go into here but just to give you a flavor for the API here are the basic abstractions you start out by creating an nsurl request this encapsulate everything all the information needed to configure a URL download so the URL itself if you have a custom user agent and you're doing HTTP you can set it there proxy settings all of that goes into the request once you've got a request you build a connection from the request the connection represents the ongoing download the connection has a delegate that's the nsurlconnection delegate and you use that to to get information about how your connection how the download is proceeding finally what you're going to get back from the delegate is the data of course but also an nsurlresponse-- object that encapsulates everything that's not actually part of the data so that includes things like the mime type like the HTTP headers that came back and then I just want to highlight briefly the other api's or the other features that are built into the foundation URL api's I've listed each and every class here but I'm just going to talk about the major bullet points to explain the features caching caching happens automatically with this API there's both an on disk and in-memory cache available and used cookie storage and management all handled automatically all shared for for a given user across all applications unless you there are preferences to change whether your particular cookie is stored but by default they'll be stored for the so that all applications can access them authentication and credential management we've got a bunch of classes built in so that you can know exactly who's asking for a password whether there's a password already available on the system for your youth and give you everything you need so you can prompt the user and get that password if necessary and finally the extensibility I mentioned so I there's a class available nsurl protocol that you can subclass if you want to add your own particular scheme that we do not provide so that takes us all the way through the URL loading AP is the last piece of course is WebKit WebKit is new and Panther it's brand new framework public framework and this is the layer that actually does all the rendering and navigation so it's what actually draws to the screen and interprets the user events is the user clicks or move throughout the document it's a fully plugged pluggable architecture so that we you can support rendering new kinds of document types we of course handle HTML text and the image types out of the box but in fact we provide sample code in the SDK that shows how you would extend WebKit to be able to display PDF it's an objective C API fully integrated with the app kit classes so in fact the central class webview is a subclass of NS view the SDK also provides a bunch of example code that shows how you can take that web view and use it from a pure C or C++ API and in fully with carbon HIVs and carbon events and for more information about that there's a session tomorrow sometime I'm not sure when but session 420 they're going to go into the WebKit ap is in a great much greater detail and that's the full stack that's what goes into the networking portion of Safari and I'm going to touch briefly on these three other networking api's web services NS stream and launch services web services is a framework we introduced in Jaguar it provides a P is for issuing an xml RPC or a soap request in Jaguar the API was very focused on the client so if you were a client and wanted to issue such a request and receive a response from the server we cover it all up we had api's for that but starting in Panther we've added support for server-side operations so if you're on the other side of the equation and need to interpret an incoming request we can now help you do that NS dream is new in foundation in Panther it's actually two different classes and its input stream and NS output stream their toll free bridge to CF reads tree mmcf right stream which have existed for a while and the reason why that's important is because NS input and output stream is designed for sub classing so we received a lot of requests saying hey I don't want to talk to a socket or I need to interpose a data filter between the actual bytes coming in from from the network before you start processing it how do I do that we can't really do that with the capi but with NS stream we now have a way to do that the subclass and its input stream or output streams insert the filter or extra data processing you need and then thanks to toll-free bridging you can take that object and pass it directly into the AP is that consumed read and write streams and this was touched on briefly at session 4 10 yesterday the cocoa update where they talked about everything that was new and foundation and app kit and finally launch services well most of you are familiar with launch services as the API you use if you have a file and you want to open it exactly the same way finder does so you're probably thinking what the heck that's not a networking API well the fact is that launch services is good at finding helper applications of all kinds not just for files so the launch services API is can be used to open it an arbitrary URL as well or to find out what application would handle a particular mime type in fact internet config which is an older API that you may be more familiar with calls through the launch services to handle these kinds of services and so I mentioned that of course to get to this next point we have some older api's some legacy api's but we're kind of interested in getting you to transition off of to some of the newer ones and internet config is one of those are we taking these api's away from you no of course not they're still on the system they'll continue to work the way they always have worked but we're no longer extending these api is the way we're extending some of the newer AP is that I've just talked about these are the 4 i'm going to address in particular URL access manager and a few RL handles internet config n OT so URL access manager clients tend to use it in one of two fashions either they have a one-off well controlled transaction or they're doing some kind of longer-lived transaction if you're in that one off case where you you know exactly the details of what you want to do you know you're performing an HTTP POST to a server you control you set up as part of your business CS network is probably the best way for you to go with something like that gives you full control of the details at that at that kind of level you know the extra abstraction almost gets in your way because you know exactly what you're trying to do on the other hand if you want an API that's simpler kind of one-shot tire and forget nsurlconnection is the way to go and then if you're using URL access in the other way where you're performing some very long-lived complex transaction where you don't want to handle the details nsurl upload an nsurl download are the two new glasses in the URL 80 loading api's that are available for that kind of functionality nsurl handle in foundation it went thought up excuse me the new URL loading API is really superseded nsurl handle so use nsurlconnection instead we've added a compatibility shim that connects the two engines so if you have some investment in a URL handle subclass that will continue to work as it always has and not only that anyone using the new API the URL connection API will automatically see and talk to that subclass as well works in the other direction so if you have so if you have have a URL handle based API you'll actually see and call through to the new foundation loading engine as well internet config so if you're looking for a helper app you're just trying to figure out who should i use to open this URL or who should i use to handle this mime type call launch services directly instead the other thing that we see an internet config is access to these system preferences things like the proxy setting to get those talk directly to system configuration instead finally open transport we've been telling you for at least three years now that open transport is not your high performance native networking solution on the system the SD sockets is so as you're writing new code we encourage you to use sockets directly also when you're choosing your networking code maybe sockets is not the right layer for you and we encourage you to think about whether one of the higher level AP is that's going to provide connection to the run loop for you well maybe serve your needs better so that's all the material I had for today I want to continue to repeat this roadmap I've talked about most of these already though I did not talk about 411 internet technologies which talked a lot about the Safari technologies in the web and kind of a general sense that took place yesterday Vincent mentioned the rendezvous talk also yesterday kernel extensions CF network in depth is tomorrow and then the feedback forum so you don't like all that all of what I just said come tomorrow morning at ten-thirty some contact information there and with that I'd like to ask Tom wire back up on the stage for the QA oh sorry should mention the documentation there's a load of documentation for the different networking available go go to ADC home documentation networking and pretty much every API have talked about here everything that Vincent talked about has documentation under that heading