WWDC2003 Session 420

Transcript

Kind: captions Language: en good morning thanks for coming out nine o'clock on friday morning especially after the big bash last night there's a one-point avoid I'd like to just mention before we start the network's down they are apparently over in moscone they cut through the OC 3 so no network today we have a fewer backup strategies and I hope their work but please bear weather's if further a few glitches during the demos so there's been several sessions on the web technologies the conflict so far and you've probably seen over views about WebKit and Safari and how supplier uses webkit what I'm going to do in this session is drill down a bit into the WebKit API and show you a few specific features and some code demos to explain how to use WebKit as you will know to follow it was a great success and we're going to bring that success to you through a web kit so what is WebKit WebKit is a framework that's installed on your system when you install Safari one point 0 and it's also included with panther it allows you to embed web content HTML images plugin content like quicktime java flash and it entirely encapsulate the open source engine now i want to drill down into a demo in pretty quickly here and like I said no network connection so we'll see how this goes now good downloads tend to flow up the management change so some of the other sessions have shown you this demo already the Achilles browser but I want to use this demo to illustrate some point so I'm going to build the code looks browser see how many of you saw this demo before the coatless browser demo okay join us it again yeah okay okay no well I'll try move quickly and show you a few different things in this demo so I'm going to load interface builder and show you the new palette the WebKit palette and this palette has a single class on at the web view and this little rectangle is deceptively simple that can go really is your high level view into the WebKit it's where the web content will be shown what I'm going to do is hook this up to the text field using the normal actually let's skip the target action and go directly to the the binding technology so i'll create a controller for the web view and i'm going to bind the text field to the main claims uo l and what this will allow me to do is keep the web view and the text field in sync as I navigate around a page now once that you haven't seen before is is showing the favorite icon for a page so into father's address bar sometimes we show an icon for a URL that's called a fav icon so I'm going to hook up a fav icon for this this demo its place that in the window and I'll bind that to the the icon for the page and then let's add some back forward buttons and then will bind the back and forward button to the web controller so that when we know we can go back they'll be enabled and disabled as appropriate but that's pretty much it let me set the resize flags for these things and then we'll test the interface and hopefully we'll have some degree of net connection so I can show you some of the capabilities of WebKit let's go ahead and text this or interface now what I'd like to do is attempt to access this local groups cubic this local web server that we just set up five minutes before the presentation and let's hope that it's good learning so I'm going to cut and paste the URL directly out of Safari into there we go woohoo what you got to set the resize flags on the text field so let me go ahead and just do that so so that's that's a quick overview of the high-level capabilities of the webview now I'd like to ask somebody to come up on stage we're going to attempt to use a modem to connect to the world wide web so you may not see the speed of what you're used to with Safari but at least we'll get some external connection to the web so these slide please could you switch to the slide please slice q spikes thank you so I'm going to recap the design goals a few of the design goals of the webkit some of these you may have heard before first and most importantly we wanted to support a world-class web browser and we think we've done that with Safari secondly and very importantly for this discussion we wanted to support embedding so that's what the web gets all about and you get the same benefit that society has in terms of high performance small footprint fast loading faster and drinks and excellent standard support that Apple is really committed to continuing to enhance and extend very importantly to a simple and flexible API now you've seen how simple it is but often simple means limited and inflexible we think the WebKit at the API even though the surface level is very very simple it provides you with enough hooks to really extend and customize as appropriate in your applications so a milligram focus on next design goal embedding for this discussion this is a high-level overview of the architecture of WebKit and it illustrates how WebKit entirely encapsulates the open-source internals of WebKit HTML we depend on a few key system frameworks the app kit for native cocoa widget core graphics for hardware accelerated renderings and foundations for fast URL loading now even though Apple focuses on the web confirms of its API we are committed to expanding and enhancing HTML now late above this is a see API that facilitates using the WebKit classes and technology in carbon applications so as you as you already know probably the WebKit is based on K HTML and the WebKit is an umbrella framework and embedded inside the umbrella framework are two you sub frame works webcor and javascriptcore as you know probably at this point they're open source and they include apples work on cage tml now really important point is that we're not supporting the web core API which you can see if you download the web Corps table we're not supporting that is a public API you will change in the future so instead use the WebKit API if you want to embed this technology in your applications so I'm going to focus on these key concepts when I talk about the website now now these this isn't the only set of functionality provided in the WebKit but if you understand these four basic ideas you can really start to use the WebKit views that's how things actually get rendered on screen web frames which is how we represent an individual frame how you get data into the WebKit and finally the delegates which is our mechanisms for extending the API so if the next few slides please keep their diagram in mind this I'm going to come back to this so you don't have to understand it all right now but this is where we're going to go for the next few slides these are the classes and the relationships between the classes that we're going to talk about and to talk about the classes I'd like to use an HTML document a really simple document on the left side of the screen on the right side of the screen I'm going to talk about the classes that represent the various different aspects of that HTML document so first up web view which exceed already and the code looks browser demo it is the rectangle that will show you the content which represented by your web page now web page is a little bit more details than simple documents web pages connects with me compound documents as represented by a frameset so in this case we have a really simple frame set with two subframes we represent a flame with a class called web frame now there's not a one-to-one correspondence between a web frame and an HTML claim or an i claim we use a web frame even for frameless documents to represent what we call the mainframe so web view will always have at least one the webview always have at least one web frame the mainframe in this particular example there are three frames the mainframe and web frame representing a dot HTML and web frame representing V dot HTML web frames in turn have two pieces what we call the web data source and the web frame view the data source holds the data the actual bite that represent the the document or the image whatever else is being shown in a web frame and the web frame view is the piece that does the rendering there's a little bit more detail so web documents can actually be various different data types very different mime types can be represented in the frame in this case an HTML document and a JPEG so for each different mind types that can be shown in a web frame we have mime type specific classes these are actually protocols that are implemented on private classes in WebKit so the web data source has a web document representation that's mimetypes surface specific and the web frame view has a mime type specific view the web document view so if you understand these core classes you really understand WebKit again this is a summary of the classes and the relationship to each other you see we have the webview an underneath the webview we have a hierarchy of web frames to represent the frame set on a page or the ice creams on a page or even just the main frame of a document so how do you get data into the WebKit well we use the new foundation URL loading mechanism and there was a great talk on that yesterday I hope you had a chance to see that and the two key classes that show up in the WebKit API of the NF u of l request and the nsurlresponse-- the request represents a response to go get data and the response is to reply once you receive that data and the session on disability it happened yesterday but I encourage you to go back and look at it on DVD so how do you initiate a load you initiate the load on a web frame typically what you'll do is get the mainframe off the web view and tell that to load data and you can load data in one of three ways you can use a request typically if you're going to go and get a resource out of the of the web you use the URL request you can specify string data you can actually create a little HTML page and place that right into a web view alternatively you can use earnest data and you could even for examples provide image data world image data to the web frame specify the mime type and will correctly decode that and do our best to render that image and of course all aspects of loading the frame level to be monitored so not only can web pages be compound documents but a single web page typically has many sub resources images CSS JavaScript even we even consider subframes resources in a certain sense now these resources are loaded automatically for you you don't have to do anything to have WebKit load resources for you it happens behind the scenes but you can if you wish to monitor the load of these sub resources and even change the loading of these resources you can redirect URLs two completely different sites you can change the caching mechanisms you have complete flexibility about how these sub resources are loaded if you want to so I'd like to actually jump down into some code before we go back to another demo and show you how easy it is to create a web view that typically you would do this with interface builder you wouldn't have to write this code but if you want to this is how you would create a web view you simply Alex and then it was where name like any other NS few and then add that to your view hierarchy depending upon how your application is set up then to low data you typically get the main claim off the web view and create a request and then load that request to simple as that alternatively in this example has created a little HTML fragment and the WebKit engine the Cape smell engine is really smart enough to understand fragments of HTML so in this case it's really malformed there's not even H an HTML tag but the WebKit engine will still go ahead and render this correctly so one more code example Before we jump to a demo this is a little snippet that shows you how to extract the HTML source of the data sources representation once a document is loaded so in this case I simply get the mainframe I get the data source of the mainframe I get its representation which again is mimetypes specific and I ask that if it's capable of providing document sauce because if it's an image a JPEG for example it won't be able to provide move document source I get the document sauce and then they log it to the console so the demo I'd like to show you next highlights one specific aspect of the WebKit which i think is really interesting and allows for some creative opportunities hopefully it will inspire you to think about different ways to use web kit in your applications so one thing we get asked a lot is how do I interact with the document if I load a document in WebKit in my application code how can i change the document how can i modify the Dom well the way you do that is to execute javascript in the documents context and a typical approach to this is in the content that you may display a WebKit define a JavaScript function and then from your application code invoke that JavaScript function and like I said you can use this to modify the Dom of the document in pretty creative ways so this is a little code example that shows an HTML page with a JavaScript snippet that has a function to find an element on a page in this case and image element using the standard Dom API get element by ID and then it multiplies the image width now by default when you multiply one dimension of an image both the width and the height of that image will change so this effectively scales down the image how do you invoke that from your application code this is how with this little piece of objective-c I create a string it represents the JavaScript I'd like to execute and then I simply tell the webview execute this string and now we'll execute the JavaScript in the context of the document let's go ahead and do a demo of this my am I actually live on the net okay before we do this let me try and show you another piece of the demo I'd like to I would have liked to have shown you earlier had the net been up and that with me if it doesn't work run a mow down and it's been a long time since I've used the modem so who knows how well the speed will be like it's been it back by the way oh we know we're not okay so this is a site that I don't know if you've been to Ranger calm and is it going to load I don't think it's going to load okay so give it a few more seconds loading loading loading nope we're not going to get to see anything press so let me exit interface builder okay how about we force quit interface builder time to move on let's quit bye-bye okay so let me bring out the plug X in Xcode and let me launch the the nib file for this project and you can see it's very similar to the previous demos the single window with a webview and what I'm going to do is load some static content into this window so we don't need the web and then we're going to interact with it from our application so let me go ahead and just show you the content that I'm going to display here this context kind of interesting it's three divs and they're layered and one of the divs has a transparent image in it so you can see this nice transparency gives you a sense of some of the advanced features of our engine now what I'd like to do is add a couple buttons to this window and when I click those buttons I'd like to zoom the picture of the spider now this spider happens to be the mascot of via safari team before we add the nice comfort we used it internally for a long time so let me go ahead and add those two buttons in interface builder I'm going to shrink this window a little bit and I'm going to add a button to a zoom in don't need the air capital N and a button to zoom out then we'll go ahead and connect these zoom in and zoom out and what I'm connecting these two is just a simple custom class that's been instantiated it's called Zuma now let's light these methods that we've disconnected zoom in and zoom out so I'm going to go back to let me just save that I'm going to go back to Xcode and bring up some source code now this is a pretty simple class and hello ok let's try bring that up in the next little window so the first thing this class does is and it's awake from nib method is it loads the template that we're going to display for that HTML document and this just uses the normal cocoa bundle and bundle and string functions so then we go ahead and invoke the mainframe load HTML string to load that template let me just show you the document really quickly so we're going to load this is simple HTML that has some styling has some script zoom in and zoom out method and this is the image element that we're going to reference back up in our JavaScript when we change the width and height so to add the zoom in and zoom out actions I'm going to scroll down to this code and just enable it you can see it's pretty strips pretty simple I have the webview string by evaluating JavaScript from strings zoom image in and I pass in the name of the image elements that I want to find and invoke it so let's go ahead and build and once it's linked we'll run it I guess ask you a quick first there we have it the back to take the resize flags again but I can zoom in zoom out and of course you can do use the same technique to modify any aspect of the page it's not just limited to this particular feature so what I'd like to do now is to let me just clean up a little bit and then go back to the to the slide so we've covered the important classes of the webkit the web view the web frame the data source the frame view the next really big set of important functionality of the delegates and unlike most of the cocoa delegates where there's one delegate per class we actually have four delegates the class / web view I should say and this is how we control most aspects of the WebKit each galaga is can implement methods that are defined by an informal protocol and so those of you that aren't familiar with informal protocols this is really just an interface with methods that may be optionally implemented on your class and again i emphasize optional because you don't have to implement any delegates to make WebKit work it's only if you want to customize behavior and as i mentioned all of the delegates are managed by the webview so going back to our previous diagram these four delegates fit in to the webview like this and I'll come back to this diagram again this is where we're going those four delegates are the web same load delegate it monitors the progress of a document loads a document level the web resource load delegate and it will let you look at each individual image and CSS file and jobs which file as it loads the web policy delegate which gives you a great amount of flexibility about how you want to manage linked reversal the user clicks on the link what happens in X the policy delegate will let you handle them on the web UI delegate the web UI delegate is responsible for managing all of the chrome related to a window and also the window itself as well as handling JavaScript interaction for things like status and alerts so when you load a document over the web it's actually a relatively complicated process more detailed process I should say and at the simple level it described by these four states you start loading a document we have this notion of a document being committed and I'll come back to that in a second the document finishing loading so it's being done and then finally the document going away being closed now what's this committed thing well the web tends to be elephone so typically if you initiate a load of a document and for whatever reason the server doesn't respond we don't get in data back that document isn't committed it's really a provisional load until you reach this committed stage it's only once we receive some data back from the server that the load becomes really valid and you can progress to the next stages so this notion of provisional committed really is very important during the loading stages of a document things get even more detailed though before you can get to the committee stage often a server may redirect you to an entirely different site so you enter one URL on the server redirects you to an entirely different location and will give the flame low delegate an indication when this happened and this can happen one or more times you can get redirected to several servers before you get to the final page that's going to be shown the jet mode get more detail so provides good user feedback during a page load what you often like to do is show perhaps the page title or the same wood icon so we'll give you Delta callbacks when we receive that information and then finally even after a document is loaded the user might want to navigate to a link within a page an anchor point within a page so we'll tell you when that happens and finally the last notification that the same load delegate receives is if a client redirect happens for example is it might be a meadow redirect on a page and that you might be redirected to another site and Kylie or perhaps JavaScript might change the location of the page so all of these events are sent to the web claim load delegate now the web resource load delegate is again the delegate that receives messages about loads of images CSS JavaScript and it has these four set of methods that sent to the resource load delegate when we send the initial request and at this point you have an opportunity to look at the request modified in whatever way you choose including canceling the request entirely you can add headers you can change the URL you can do whatever you want then you receive a call back telling you that the server's responded and again you have a chance to look at the response and for those of you that are familiar with the nsurlresponse-- and request API is this should be very familiar to you then after the response you get a number of callbacks for each data packet that we will see from the server and in fact you may receive multiple of those depending upon how the server responds and then finally when the resource is finished loading you get a done message the policy delegate this delegates a little different than the other delegates the other delegates mostly just go back to tell you about things happening the policy delegate actually asks you a question what should I do when navigation happens tell me tell me how I should respond and the response can be asynchronous so you can put up modal dialogues wait to use a response and then tell well WebKit how to continue its each of the methods that we send to the policy delegate there are three possible choices go ahead and use it as you normally would the default policy ignore it can stop whatever you're doing drop this load on the floor or download download download the URL to a disk so the policy delegate is consulted at least three times during the document load initially its asked should I open this URL in a new window and if so the UI delegate comes into play remember it's the thing responsible for managing windows should I go ahead and handle this request you know this gives the policy delegate a chance to look at the request details potentially presents the user with a dialog like this is an HTTP request HTTPS request do you really want to do that and then finally based on the mime type that we receive and this happens after we send the request we get the response back with the mime type what should I do with the mime type and this may be the location for example that you hand off a request to launch services to handle opening the request externally do i delegate the i delegate is probably that the fattest delegate in terms of the total number of methods and it's responsible for all the chrome so in safari this is the thing that open up opens up a new window it manages the status bar at the bottom of the supply window managers the toolbar at the top of the window new potential javascript alert and literally allows you to modify the context menus so if you ctrl-click or for those people with two mice right click on an element the UI delegate is referenced so Vaculik diagram this is what we've covered so far the classes the core classes of WebKit and the delegates of web view the web chrome load delegate loads or let you monitor the load at a document level the web resource load delegate lets you monitor loading / resource where policy delegate which lets you intervene to manage navigation actions and the web UI delegate would select your marriage you I chrome so I'd like to go back to a demo oh we back exciting oh it's modem it's just modem ah ok well let me let me go let me be step on the wild side and try ok just in Safari I want to share this page because I think it's really cool i'll do it into folly but remember that left angle in safari is the same rectangle that you would see in WebKit so what you can do in Safari you can do in WebKit and this demo highlights some of the amazing things that you can do with modern web standards this is just simple HTML a JavaScript pretty pretty cool huh minimize windows expand them close them so I'm ticket think about this when you're thinking about web get applications it's not just the web browser and you can use the power of HTML CSS and JavaScript in marvelous and creative ways that we can't even anticipate yet I think that they'll probably be 20 browsers on virgin cracker in the next few months but we think supplies pretty good i'm sure they're going to be other good glasses out there but think about other ways to use WebKit ok back to the demo so what I'd like to do is go back to the image Zuma demo and add a context menu so I had buttons at the top of the screen to zoom in and zoom out what about using a context menu to zoom in and out on that image but only want a mouse over an image let's do that so I'm gonna bring up the source again for the image Zoomer and the first thing I need to do is set up a delegate and i'm going to add that code up in the wake from dave method that's as simple as that i'm setting self as the UI delegate and then the one method that i care about for this demo is the method that lets the UI delegate specify the context menu so in this case if you're familiar with some oats familiar with cocoa you'll be familiar with this code here and the expected response of this method webview context menu items for element default menu items is an array that contains the menu items that are going to be shown when you context click on an item so in this case I only care about image URL keys so if the click is over an image URL I'm going to go ahead and construct 2 menu items zoom in and zoom out and return that away otherwise otherwise I would turn nil so we'll get no context menus unless the user right clicks on an image we go ahead and build this se rule files yes build and then once its link we'll go ahead and run it again so zoom in zoom out buttons but now if I hold down the control key I the context menu zoom in I get the same behavior umount simple so this is an interesting demo but it's somewhat contrived you know would you ever really want to do this probably not but the same technique can be used in an application that's actually on the SDK which is useful actually as an application in its own right i'm going to show you that application now and the entire source code for this application is available on the sdk let's go ahead and hydro this and this is a browsing application that lets you browse pictures and i often use this when i'm trolling for icons using my my applications so i'm going to open up the applications directory and on the left side of the screen we see a list of the applications kind of like a finder and i can drill down into let's take a look at I photo I can drill down into a directory and see all of the images for that particular directory so it's kind of like an image browser and you can see that calendar often may be considered but this is all dynamic and the pages constructed on the fly based upon scanning the file system it's really useful for looking good images so if I can even zoom in quick back and forth and you get a lot of features which I'm not going to talk about today like back and forward in history these things are relevant for applications other than web browsers so if I go back you're like I really shut up to a nice little back forward okay let me show you some pictures that I took this weekend at auckland zoo and this this den will actually makes use of custom protocol believe it or not the URLs for these pages on HTTP or file URLs that were custom protocol and that protocol is interpreted to generate on-the-fly HTML content so as I click on a link this is a actually what I called picture browser protocol that lets me generate the rapper HTML for this particular content so I encourage you to take a look at that application for tips and techniques about how to use WebKit so there's one more demo I'd like to show you and this is again a demo that is available on the SDK the full source for it's available on the SDK and this is more of a traditional browser really tries to show you the capabilities from a browser perspective you fly off list it's called the mini browser that's the name on the SDK let me go ahead and run it for you now and I guess I'm still on the modem so speedy supply isn't so speedy on a modem but it's still pretty good on the modem so I have a lot of the features that I'd normally expect I can you know make text bigger if there is any text on this page isn't too much text smaller now one thing you'll notice though is that I don't get a lot of sleep oh this is actually good a slow link I can show a progress feedback better because take so long to load so if I go to another page take a look at the title bar there this is loading loading loading loading and we've got the title but I'd like to buy I'd like to provide a bit more feedback maybe a counter that shows me the number of resources that are loading no one is 40 year and then that would progress as I load additional resources but at least it would give me you know some idea that the page is still loading that's a key pretty good at the modem okay so let me go ahead and add those features so what I'm going to do to enable these features is extend the document for the mini browser to be a resource load delegate remember the resource load delegate is the thing that gets notified as resources awhile use over the net and I think they'll become code down here look at that okay let me cut and paste this code again and i'm going to set up the document as the resource load delegate and now there's actually more than just one method here to do this enable this code and i'll just start talking through the code we don't need that that will compile so the first method is identify us for initial request now why do we need to identify it why can't we just use request well requests can actually change they communicate they can be redirected and what you often want to do is identify the notion of a resource independent of those redirects or modification of the request so the identifier will remain constant throughout a resource load even if the request changes so you can use this kind of as your own tag to track a particular resource load and in this case which can allocate an NS number it can be an arbitrary object whatever you want WebKit doesn't care about the identifier just lets you specify it so then you get notified that we're about to send the request will send requests and at this point you can modify the request change it in whatever way you choose and there's a lot of flexibility here change your caching policy you can add headers you can disable cookie management whatever you want you can just really modify the request so what we're going to do is just update the resource status and I'll show you this method in the second but it really just modifies the title of the window and then we return the same request so we let WebKit go ahead and handle it as it normally would next method did fail loading with Ella it sent to you if that's an l a-- and you feel free to know do whatever you want with that ever and the object it's not just a code it's a fully fledged NHL object it has a localized description you can present that in a dialog box what we're going to do is just count increment a counter to indicate that the resource failed and again we're going to update the title bar with that of the resource cutters and then finally you get this message did finish loading some data source when the load complete let me just show you the update resource status method all it's going to do is it's a failed count it's going to construct the strings and tell you about those others if there isn't a fail counts it's going to just construct the string that search loaded X of X to X of n let's go ahead and build this which build button should i click this 10 this one ok sable so many options to build to the build there we go ok I'm still running ok so now tickle isn't title bar loaded 21 and 41 to making the number of resources on somebody pages it just go to a couple let's cue we shall courage so on the other thing that is interesting that you might have noticed that the total changed that's because the number of concurrent loads actually you know may change over time because the webkit loads images as it encounters them as it passes a page ok so that's the mini blazer demo so the last topic I'd like to cover in the many minutes is the capi and the WebKit is a objective-c total framework no doubt about that but we've gone to some effort to make the same technology available via carbon applications the way we've done this is to create an H I've you that actually really backed by a web view so you can use the web technology in your carbon applications but if you really want to drill down into some of the advanced features you will have to write get to see you objective c plus plus this is a little C snippet code snippet of how you would construct a web view using the capi we simply created a chai web you create and then the next few lines of code will just modify the frame of the webview to correspond to the content view of the main view of your window and then finally you add the web view as a sub view of the root of your window so I would like to do now is to show you whipped a demo of a few of these carbon applications now in the XDK we have full source code for the carbon web browser which is a common application and it isn't just a simple carbon application we've actually done a lot of code to provide adaptive to carbon events so you can use this as a model for how to interact with WebKit via carbon events I encourage you to take a look at the source code for that but as you can see it it's a full full full out full featured browser this particular demo also includes history and md source but the source actually gets logged to a console not in a separate window then the other demo that I'd like to show you is the carbon downloader a misuse is actually the foundation ap is to download a URL to the desktop as you can see so I encourage you to look at both of these examples so that's it for the demos and one final note about the XDK it does work on 10.2 in fact the XDK is designed to be installed on 10-2 so you must first install safari one point 0 and then the sdk on top of that and I gather there are some problems installing the sdk on Panther so what you can do is just copy the headers from know that it ok ok so I'll Panther you don't have to do anything you get you get the sbk built in and as I mentioned there are several examples that are quite comprehensive in the SDK and they'll be more to come and if there's anything in particular you like us to see or like us to write for you as examples you know please let us let us know it also includes conceptual and reference documentation that's pretty comprehensive and it's available now actually as of monday i believe for download from a DC for more information you can look at the ADC website there's actually a number of topical tech notes on there one of interest is how do I detect whether or not the XDK is installed so for example if I want to build an application to depend on this technology how can I know that my clients actually have web kit installed so there's actually little toe temple to show you how to do that which is I think pretty helpful the WebKit reference subject to see the website reference to see and the URL loading materials also up there sample code and of course you know what really makes the WebKit great is the HTML CSS and JavaScript technology that you can really show in WebKit so I encourage you to look at these great books for more information about those technologies and contact john glenn see if any questions so I think that's it for the slides so Q&A