---
title: WWDC2001 Session 311
framework: wwdc
role: article
path: wwdc/wwdc2001-311
---

# WWDC2001 Session 311

## Transcript

Kind: captions Language: en hello my name is rich Murphy and I'm the manager of the platform security group this is session 311 web enabling your application during this session we're going to be talking about URL access giving an overview of the functionality in the library discussing the api's talking about URL access on Mac OS 10 and also talking about the nsurl api's and cocoa web programming during this session you're going to learn what you can do with URL access basically how it applies to your problems how you can incorporate URL access into your carbon applications the limitations of URL access and future plans for the for the AP is what is ural access your all access is an API that allows you to easily access data on the Internet by the youth the use of URLs supports HTTP HTTP FTP and file type URLs has downloading and uploading capabilities and it's a Carbon API that's available on both Mac OS 9 and Mac OS 10 this slide indicates how we see URL access actually being used for the most part if you're building a carbon application you need to use URL access URL access is available on both platforms and will provide all the basic URL functionality or URL download and upload functionality that you need it's based on the the Mac OS itself which provides all the networking functionality underneath actual socket access or file access and that sort of thing on the cocoa cocoa side we have the CF api's core foundation or CF provides an additional URL accessing type api's and they're also based on the Mac OS 10 kernel and the networking capabilities of that here's some standard URL access features we have an authentication dialog that pops up whenever you need to actually authenticate or log in to a remote server whether you're doing FTP type access or whether you're going to an authenticated HTTP or HTTPS site there's a progress dialog built into the the library that will come up and show you how much time is left it includes file decoding capability it links into doing deeb in hexing and also working with Apple single decoding additionally in Mac OS 9 it currently will expand dots it files using the Aladdin just stuff and expand your capabilities it's scriptable using AppleScript has proxy and firewall support and it also supports the keychain for storing your authentication information for HTTP we have download capability we have the capability of doing post and put although it requires you to do a little bit of additional work with the api's it auto redirects and you have full access to the server responds by taking a look at properties it'll also do proxy authentication for FTP URL access will all you also provide upload of files and directories or download of files and directories and it gives you the capability of getting directory listings from the FTP server for HTTP we also include SSL version 3.0 capability with 128-bit encryption there are some functional gotchas when you're using URL access basically it doesn't bother to parse the response of HTML this can be a problem when you're doing redirection if you have a file not found on the remote side or if when you have login failures and that sort of thing there proxy limitations for FTP uploading when you're trying to send a file up to an FTP server or when you're doing FTP directory downloading there's a couple third-party applications that are actually using URL access both in 9 and 10 I cab is currently using it in order to download HTTP sites and adobe uses it an acrobat to implement the web capture facility there's also a fair number of companies that are using the scripting capabilities to help manipulate files get them up to servers and back down from servers as well so the next thing we're going to do is go over the api's there's synchronous api's they include a system event procedure that allows you to trap system events and react to those there's various flags that you can specify to URL access which will help modify its functionality there's properties that can be set and that sort of thing and interrogated in order to perform various operations like HTTP puts and posts or to get back information from an HTTP server there's the asynchronous API which allows you to start URL access off doing a transfer and in those cases you want to be checking for events then there's various states that URL access goes through that you can interrogate to see how it's working first of the synchronous API is is URL down simple download this is really the easiest way to download a URL if you're writing an application that basically just needs to grab something off of a server this is really the function to use all you have to do is provide a source URL a destination file spec or handle and then flags indicating how you want URL access to actually behave if you're using this as an application that needs to be interactive you need to be able to you need to use this in a thread because it is a synchronous API it will block until the transfer actually occurs here's an example of using URL simple download notice here we've created a little function called get URL which takes a source URL as an argument in a file spec in order to download the actual file to a file on the Mac OS file system we set some URL open flags for passing into the actual function call telling it to replace the exist an existing file if it's there display the progress as you're doing the download and expand the file as it comes down we call URL simple download passing in the URL passing in the destination file spec passing a nil to the memory argument because we're not going to download it to memory passing in the fought the flags and in this particular case we aren't providing an event proc and so therefore we also don't need to provide a context it makes the call and then it simply returns along with its status it will have downloaded all the data into the file and that sort of thing or return the near status system event procedure can be used as a parameter to the synchronous calls this will allow any of the dialogues that are displayed to become moveable modal any update events will be pass through to the the procedure that you passed in if you don't have one of these the dialogues will become non moveable modal the flags for aural access control optional aspects of the transfer as we saw a little bit earlier it had the replace existing flag or the display progress flag and that sort of thing these flags can be used in both the synchronous and the asynchronous calls and various calls will accept different Flags check the documentation online for that here are some sample Flags that we have the replace existing flag as I said earlier will replace a file on the download or the upload if it if it's already there displaying progress will happen if you pass in the que URL display progress flag that'll put up a little dialog display off will actually invoke the URL access authentication window to get your login and password expand file will invoke Aladdin expander stuff expander bin hex on the way up will actually bin hex the file and then there's the no auto redirect flag in other words come back with an error if you if you're being told to redirect there's also on the synchronous side there's a little bit more complex call if you want to called URL download if you call URL access through you or I'll download it takes a few more steps in order to actually invoke the call but it allows you a lot more creativity or a lot more flexibility in what you can actually get out of the calls you have to create a URL reference using URL new reference and URL dispose reference and once again this should be used in the threads and since the synchronous call but once you have that reference you can set various properties and do additional manipulation of how URL access will actually be used so as I said before if you use URL download you can get in set properties before and after the download you can set things like the user password if you want to or things like that and you can also after the call is made interrogate the URL reference in order to find out what the status was coming back from the server a little bit better or in a little bit more detail if you've actually threaded it you can monitor the state of the download or you can't abort the download as it as it's progressing if it's taking too long here's an example of using URL download in this particular case we've made another function get URL same as before taking a URL and passing in the file spec destination you'll notice the call to URL new reference which actually allocates a URL ref which can later then be used in URL download you notice the call structure doesn't take a URL as the first thing it takes a URL ref in has the arguments shown here after the call to URL download we do a URL dispose reference on the synchronous upload side we have URL simple upload and then also URL upload and they they pair up the same way the URL download calls pair up URL simple upload is easy to use URL up URL simple upload is very easy to use URL upload is a little bit more difficult it gives you a little bit more flexibility in this particular case you give the destination URL where you want to go and a source file spec is provided one of the limitations of this is you can't upload from memory different Flags apply to the upload side and right now it's FTP only for properties you can get the URL host or the URL actually passed in or you can get the resources it points to meaning on the way back you can check to see what the size was of the resource that was downloaded or the resource name on the way down which might have been specified by use of the URL itself or you could have specified it through the file spec also the mind mime type that had ended up mapping to on the download you can get the status string check and you see how things went or you can get the total number of items for instance if you've downloaded your directory this would tell you how many files actually got brought down in in the download for HTTP specific properties you can get the request method the response header body or a redirected URL properties are received using URL get property if you specify the property that you're asking for and give it a variable that you expect to have it come back in in order to set properties you call URL set property any of these properties may set as may change as a result of the transfer itself it may be that the username and password might change as a result of actually redoing the authentication from the dialog coming up that sort of thing or it may be that other properties are set as a result of the transfer from the HTTP server returning certain information these are all the properties that are that you can set when before making the call once you have the URL reference you can set any one of these properties in the URL ref here's an example of using the properties in order to do an HTTP POST operation notice before we said that you could do a put or a post but it requires a little bit more work in this particular case we're going to end up doing a URL download in order to actually post data up to the server we make the new reference we fill out the request method property with a post request we also set the property of the HTTP request body with the actual data we want to send up to the server then we just call URL download which should have the full complement of arguments sorry about that after the call we dispose the reference the asynchronous API is need a URL reference in order to do an asynchronous call you need to create a URL reference the one call that you can make for this is URL open URL open will either do an upload or a download depending on what you've requested you retrieve the data with URL but get buffer in URL release buffer you monitor the state of the transaction using URL get current state various reasons why you might want to use the asynchronous api's if you're basically just filtering something you want to send it on through it's a good reason if you wanted to display your own progress rather than having URL access provide the progress or if you don't want to include multiple threads in your application here's an example of an asynchronous download using URL open so in this particular case we created a URL reference we also set up a notify proc with the notify upp equals new URL notify upp with the function my notifier which we'll just show you in the next slide the URL open call includes the URL ref the flags the notify proc pointer and then the events mask that we want to take a look at and also a pointer to the user context in this particular case we're just setting up a little done flag we have all of that in a do-while loop here and checking to see if we're done or not and in order to give you oral access a little bit of time to work it calls URL idle here's the notification routine that we'd actually registered my notifier you notice that it's switching based on the events coming in for a database available event which case we just process the data or in the particular cases of a completed event or an error occurred event we simply set done to true and we break out of the routine here's the process data call that we gets called from the notify proc gets the current state gets the current buffer does something with the buffer and then releases it events are always reported to the notification routine you can register for a subset or you can actually handle all events coming in if you need all the events use que URL all buffer events mask otherwise you actually or to gather the appropriate flags that you want to look at in this particular case the completed event mask in the URL error occurred event mask here's a sample of all the events that you can possibly be waiting for resource found property changed and so on here's the case of an asynchronous download where we're actually going to pull so we set up to do the download call URL open URL open and then we go into a do-while loop basically what we do is we just check the state by calling your I'll get current state saving that into current state and then check it against the various states that would be completed the URL completed state and the error occurred event the URL get current state function itself reports the state of the transfer most states have some sort of an event counterpart you can't find out about property change events by using States so you can't tell about for instance the HTTP properties changing or something like that there is no way to actually signify that in the state checking States is useful in both the asynchronous in the synchronous API so that if you've used one of the synchronous calls and gone off into another thread you can get the current state of a transfer here's some sample states that are available the DAO connecting state download state data available state error occurred and completed state so in review there is a synchronous API using URL simple download URL download URL simple upload and URL upload and then the asynchronous API where you'd use URL open various flags and properties and events as we've seen help you manipulate how the transfer actually works some more programming gotchas URL access currently is highly threaded you have to call URL idle in order to give URL access a chance to work the set property function sometimes expects C strings and sometimes Pascal strings one of the things we're working on in URL access is updating some of the documentation as we've started to use it we've had a transition period over the last couple of months we have some new people working on it we're finding that we need to actually get a little bit more documentation so expect that to be updated really soon I hope as far as what's what properties are expected as C or Pascal strings you also cannot can't call URL dispose reference from a notification routine if you're using Mac OS less than 9.0 for the request body Kanak can't contain null bytes the URL reference itself isn't reusable and if you're using URL open the authentication dialog isn't currently supported for Mac OS 9 vs Mac OS 10 there was some capability of having plug-ins for URL access in Mac OS 9 we no longer support that for Makka oo s 10 we've fixed some of the ways some of the properties work in Mac OS 10 so they work a little bit more correctly and currently the file protocol is not available on Mac OS 10 if you want to develop applications on Mac OS 9 here's the URLs if you want to work with Carbon Lib 1.0 2 or later there's a URL for that I think these are all available at the end on one master URL at the very end so you'll be able to get all those if you want to develop on Mac OS 10 you install Mac OS 10 and currently link it with the system library frameworks Carbon framework that's we're all part of carbon and the latest documentation is that the URL provided here for future plans for Mac OS 10 we're working currently on a new URL access that will be a little bit more targeted directly towards Mac OS 10 we've had some problems where some things aren't working quite properly using some of the old ways of working with 9 and this is giving us some performance problems and some stability problems so that's really give in kind of an area focus over the last month or 2 months actually there will be a lot less threads in the library it'll hopefully be a lot more stable it appears to be so far and it'll include streaming upload and download capability will actually be supporting connection reuse intend so that if you have multiple instances of going to an HTTP server they'll actually be able to pickup the same connection and secure transport will start to use the security framework root certificates this has been a problem for people with their own self-signed cert sand that sort of stuff trying to access secure servers we've had problems where people need to update for new new root certs that are out in the environment so in summary URL access allows you to access Internet data by URLs it is a carbon API has fairly easy to use synchronous Abe API is in pretty flexible asynchronous api's so now next in order to talk about web access from Cocoa Becky we'll rich will be out here to talk about the core frameworks thank you thanks very much Richard so I'm going to talk a little bit about how you can do the same kinds of things as URL access provides two-carbon from the cocoa world and I'm gonna start by talking about CF URL CF URL is just a simple CF type from core foundation that's used to represent URLs I need to correct one thing that Richard said core foundation is not part of the cocoa API so it's part of the Carbon api's and it's at a low level in the system where it can be leveraged from both so CF URL is not a cocoa specific type now for those of you that are working in cocoa you'll find that CF URL is bridge to nsurl so a CF URL is an NS us URL and an nsurl is a CF URL what's the F URL does for you is it parses the URL string and gives you access to all the different component pieces it also handles your internationalization issues things like the percent-encoding x' right things like the % encoding the encoding is in particular and when you're working with the CF URL API is you should be aware that it unless the encoding is explicitly specified in an argument we're assuming utf-8 however CF URL is a pure data class in other words it does nothing but represent the string there's no support in there for doing the actual network i/o necessary to get the data so just one more note on using CF URL you'll find that a lot of the core foundation AP is and increasingly the Carbon API is as well are taking CF URLs to represent files this is trickier than it sounds because in part of the encoding issues involved with high bit characters so whenever you create a file URL we recommend that you use the functions that are named with the word file and I've given some of those up there and then for further compatibility with the Carbon world there are conversion routines which will take a CF URL and convert it to the matching FS ref one catch there of course a CF URL can represent files that don't exist FS refs can't so in that case the conversion will fail and will return an error okay so from that I'm going to go on to talk about cocoa these are the topics we're going to cover we're going to start out by showing how you can use URL access as Richard described it from cocoa then we're going to look at the specifics of using nsurl from there we're going to look at nsurl handle which is the class that actually does the download work inside of cocoa and then finally we're going to talk about how you can extend nsurl handle to do your own downloads to support different schemes and so on so the first thing is you can in fact use URL access from cocoa more or less as you would from a carbon application there are two things that you have to be very much aware of though the first one is that you cannot use any of the UI switches in URL access that's because URL access brings up Carbon panels and carbon panels and cocoa panels don't coexist very well the second thing you have to be aware of is that URL access uses thread manager threads so when Richard talked about URL axis being highly threaded those were all created via the thread manager api's well the thread manager uses cooperative threads whereas people you people program in cocoa are more used to P threads which are not cooperative when you use URL access you have to be prepared to work in the co-operative threading world of the thread manager so what does that mean the first thing it means is that you must call URL access from a thread manager thread URL access is not prepared to be called from a pre-emptive thread the good news is the main thread counts the bad news is if you call it from another cooperative thread the main thread has to be playing nice that means the main thread must be prepared to yield to the other cooperative threads the other thing that means is if you are using the asynchronous api's URL open you must remember to call URL idle for the same reasons that Richard described if you don't call URL idle you won't be you won't be giving time to the other cooperative threads the easiest way to do this from cocoa is on the main thread post a notification to yourself with the posting style post when idle when you receive the notification you know that there are no events in the queue so take the opportunity to call URL Idol and give up time to the other threads okay so now we're going to do the same thing that Richard did with the URL access api's we're going to talk about the synchronous ways you use nsurl and then the asynchronous ones synchronous one is very simple you just call them you just send the URL the message resource data using cache pass a flag yes or no depending on whether you want nsurl nsurl itself to hold that data for future reference the asynchronous api's are a little more complex you send the message load resource data notifying client and then you pass whatever object you wish to use as the client using cache yes or no what that does is it starts an asynchronous download in the background and as events come in on that download your client is notified with the methods listed below those methods are provided in an informal informal protocol so you implement only the ones you're interested in and the others get ignored and you can see the events there you get note you get told when resource data comes in when the download finishes if the download was cancelled and if the download fails it wrong button there we go so what's really going on I said just a little bit ago that nsurl is a CF URL and a CF URL doesn't do any downloading so here how am i doing downloads with nsurl what's happening in the background is that NS URL is creating a connection to a new object an nsurl handle and it's that handle object that's actually managing the work of the download so one of the side effects well one of the reasons for this is so that nsurl handle itself can provide a more complex API and when you need to do more sophisticated URL work you work directly with the nsurl handle you can obtain a handle by a by passing the message URL handle using cash yes or no once you've got the handle you can manipulate it for more direct control you can also use it to flush the cache so if you find once you once you've used the data as much times as you need to you can then flush the cache in empty memory and for the full details I'm going to point you towards nsurl handle dot H okay so here's the good the bad and the ugly and as URL currently well nsurl handle currently supports only HTTP and file unlike URL access you're not going to find proxy or firewall support there nor are you going to find HTTP however the good news is we intend to do all of those and the architecture itself is designed to be extensible so that you can easily add add new subclasses to nsurl handle and thereby provide new functionality so how do I do that well what you do is you simply subclass nsurl handle once yourself once you have written the subclass you register that subclass with NS URL and once it's registered nsurl will automatically pick it up and as as its messaged for its data it will start using your subclass so start by creating a subclass implement all the methods inside of nsurl handle h that are that are marked to be overridden and then call nsurl handle register URL handle class to register your subclass here's a list of the methods to override I'm not going to go through them in great detail right now but it gives you an idea of the scope of the problem there may be ten methods up there from your subclass you should message the superclass occasionally to inform it basically of the state of the download and the two methods you need to use are the ones here background load did fail with reasons so when you're back when your engine detects a failure you need to message your superclass to pass that information on and did load bytes load completed which you use to tell the tell the superclass that new bytes have arrived and whether or not the download is now finished that's all I have for you today about nsurl and cocoa I'm going to ask Richard back up here on the stage and Thomas as well tom wire is or DTS contact excuse me worldwide developer relations contact so if you have questions about either URL access or the cocoa side the NS URL and nsurl handle he's the guy to contact you