WWDC2004 Session 414

Transcript

Kind: captions Language: en good morning this is section 414 and we'll get a slides in a second my name is Richard Williamson and this is taking advantage of new web kit features and I'm going to be telling you about some of the new features and WebKit until we're introducing this year just over a year ago now we introduce Safari and it's a great web browser millions of people are using it and we're pretty proud of it but the technology behind Safari is not just about web browsing already last year some of the system applications in OS 10 starting to take advantage of the technology behind Safari inicia mail dot ask is going one step further not only will it use the supply technology to display content you'll be able to compose rich HTML messages using the Safari technology and we introduced Wanda last picture the calculator well actually a bunch of apps in dashboard and the technology behind dashboard is the same technology behind Safari and that technologies WebKit so in this session I'm going to talk about the new features in WebKit but also I'm going to talk about what it means to use this technology not just the browsing but going beyond browsing so quick recap what is web kidding it's a framework that ship to know is 10 and it allows you to embed web content obviously HTML images plugging data and it's highly scriptable and this last topic is something which i'm going to focus on quite a bit in this session behind me is a high level architecture diagram of WebKit the web framework actually encapsulates the open-source k HTML and kts engines and it's layered on top of basket for UI controls core graphics for rendering and foundations for nsurl loading and laid above the web frameworks it's a capi to allow carbon applications to access this technology now even though it's layered on an open source engine we encourage you to use the WebKit API you can look at the source code in the implementation but please implement against the WebKit API okay what's new well when awful lot is new this year and I'm going to cover some new API areas and also some new features in web clip when you are you will graduate this year support for what we call web archives and web archive is simply a way of representing both the page and its associated resources in one package lots of new scripting support and some of you may have seen some of this already in some of the dashboard sessions dashboard relies heavily on new scripting technology that we have editing API is this is a big new area in WebKit and the API to introducing a pretty comprehensive here to support the Dom the document object model in objective-c and also to support editing in a very cocoa style fashion there's a session on Friday that will drill down into quite some depth in the editing API it was mixed named in the brochure and the real name should be website HTML editing api's it's friday at ten-thirty when i strongly encourage you to go attend that session dragging support we had a lot of requests from you all to override the default dragging behavior of the webview you can now do that with the new API plugins we've always supported plugins Netscape style plugins in WebKit but it so it's been difficult to write a plug-in for WebKit so this year we're including everything necessary to write an ex-gay plugin in the in the OS we've also introduced a new form of plugins cocoa plugins that make it vastly simpler to write a plug-in for web kit and it's a great session later this afternoon section 413 that will go into some detail about both the club the cocoa plugin api and the netscape api and as many many other tweaks we don't really have enough time to describe them all but a lot of these tweaks were inspired by you all there's some activity on the web kit SDK mailing list and we went through that list and we gleaned a lot of information and there's also a lot of features that you always requested we pay attention so if it's a feature you'd like please let us know so in terms of the new API is I'm going to focus on just two areas in this presentation where've archives and scripting but before I drill down into that I'm going to describe some of the new features regarded to WebKit we now have comprehensive support for CSS 2.1 and we also have support for a lot of advanced nucleus s3 features css3 features I'm still waking up yeah so there's a session yesterday they hire gave 419 we went into a lot of detail about some of the new CSS features that we have and even though that session is passed I encourage you to go and review the slides and the DVD DVD material for that presentation so I talked about dragging support at the web framework level we also added dragging support at the JavaScript level what this means is that your pagers can interact with drag operations so you can actually have an element participate an element on a web page participate in a drag operation this is a very nice feature and the implementation of this follows the Microsoft's dragging support standards de facto standards like an egg supply 1.2 introduced lifenet that's the ability to interact between Java and JavaScript bridging those two languages we also introduced XML HTTP support in Safari 1.2 this is the ability to initiate an HTTP request from JavaScript very useful for example if you want to dynamically add stuff data from a web service to the webpage many other applications there's a great session that's also been given already on Monday but I referring to that in the DVD material to discuss these last three things dragging support life connect and xml httprequest and finally many many compatibility fixes particular very glamorous topic but we spend a lot of time trying to make WebKit work with the existing web content and believe me there's a lot of crazy content out there so it takes a lot of effort sometimes to figure out what the problem is and before you even fix it so in addition to pushing the support in Safari we've added some features that go beyond the standards and many of these features weren't inspired by Safari RSS and dashboard there are things that we wanted to do that you couldn't quite do with existing standards one of those things is image compositing what this will allow you to do is on the image tag on the HTML page specifies composite operator and for those of you who don't know what composite operators are it's the ability to take advantage of the alpha channel in an image when you blit that image into a surface so all the power of the composite operations that are provided by CG core graphics are now available to you in web pages we added a few CSS extensions to support the Flex box model this is used in Safari RSS they're also a few other things that we've added that you can support but you can use in your web pages a couple of new input elements but also in Safari RSS the slider widget so you can actually have a slider that will provide range values and the search field both very nice features and finally the canvas element so we found that we wanted to dynamically draw into content on a web page you can't do that with anything that's out there that's expanded so we introduce this new element that will support that but I'm going to after talking about the new API in depth I'm going to drill down some more on the canvas element show you some examples of them so let's get into it talk about the web archive API well a web page is a lot more than just the initial HTML page that you use to lay out your your document it's also the associated resource data images JavaScript and CSS so web archive is all of those things collectively put together we represent resources by a new class web resource and the entire thing by new class the web archive and the on disk format of a web archive is a single file a single binary file which makes it very convenient to move these things around there's also an in-memory format the pasteboard format that is used to facilitate editing so for example if I have to web views now make us watching in one web view and drag that to another web view look editable the paper representation will be a web archive so this facilitates editing so you typically don't create a web archive directly instead you have an existing web page that you create a web archive farm or alternatively an existing selection that you create a web archive phone and once you have a web archive it's really simple to take that archive and load it into a webview or use it to replace a current election what is the code looks like to do this very straightforward for those of you that are familiar with WebKit this will look very familiar we get to mainframe off the webview we get the data source associated with the frame and then we asked you to create a web archive that's all it takes to create a web archives on the selection you get the selected down range and you ask that give me a web archive now that second method that selected Dom range is a new method that we're introducing this part of the Dom API and again I encourage you to attend Friday's session to learn more about that new Dom API so once I have a wet once I have a web archive how do I get it into the page the existing methods to get content into a web you look like this load request that takes an NF URL request and we'll go and retrieve the data and put it into the web view well load HTML string which will actually take an HTML snippet and stick it into away view the API looks like this very much what you would expect to load a web archive low dark eyes and you patch in the archives so alternatively you can replace the current selection like this replace selection with all kinds okay you really drive this point home let me do a quick demo so there's a new way of saving a page in safari that takes advantage of the web archive feature I can add a sailor page as well source which I'll do here go ahead and save that in my documents folder well I can say leave the web archive you want to let's now go ahead and disconnect from the net empty safaris cash go ahead and load these papers again that looks pretty crappy let's go ahead and load web walk eyes much more like what you'd expect so this is a great way this is a great way in your applications if you want to create content that convey efficiently be packaged and loaded you can create web archives and include them in your application bundles back to the slides please ok how about some stuff that can really help me build a good widget guess what we're going to talk about now the first little bit of homework there's always homework I'm going to talk about scripting in quite some depth what do I mean by scripting we've had JavaScript a long time we all know about JavaScript on web pages what I'm going to focus on in this discussion is scripting in the context of binding javascript to native code so that I can message from JavaScript infinitive code and in the other direction message javascript from native code specifically what i mean by native code is a chunk of code like this this is some Jessa vetoed that wants to call into JavaScript it wants to invoke this function set image in JavaScript well alternatively I could have a hunk of JavaScript code like this who wants to call into an objective-c instance in this case an address book object I'm going to get some address book records so when we talk about scripting it's really how do we bind native code to javascript in Objective C it's also we've also introduced support for binding JavaScript to see you don't have to implement an objective c you could do this in situ and this is really useful for plugging the particular existing netscape plugins that aren't implemented in cocoa I'm going to talk about the objective-c to JavaScript bridging in this session the plugin session this afternoon is going to describe the more detail the sea bindings to javascript so first let's talk about going calling javascript from objective-c so JavaScript objects in objective-c are represented by what we call a web Script object and a web Script object is an objective c class that has has methods to allow you to call functions and get in set properties on the corresponding javascript instance now how do you get to the JavaScript interpreter what is the root how do you get that hook into the JavaScript interpreter well as a JavaScript window object and the JavaScript window objects in Objective C corresponds to the window object that is the root object on your web pages and there's a new method on web view windows script object let's get to that initial hook into a web page it's an example of how you could use it very simply to get the location hof property from a web page so in this case I'm asking the webview give me the windows script object and then give me your location property which is an object JavaScript web Script object again and then after that object give me the excess property it's a simple event there's an alternative method to do the same thing I can take a little chunk of JavaScript and evaluate that on a particular web Script object in this case I'm evaluating location.href on the window web Script object these two mechanisms are equivalent okay how do I invoke that JavaScript function from objective-c well first I get the window Script object again then i create an array that represents the arguments that i'm going to pass into javascript in this case it's just a single element in the array the name of an image that i'm going to pass to add image and then i invoke the javascript function by clone call web scripts method with the name of the function add image and the arguments again alternatively I can use a snippet of JavaScript to do the same thing so in this case add image the again these two mechanisms of equivalent depending upon your usage plans you can use one over the other okay so what kinds of things can attach to and from JavaScript well you can pass any object to JavaScript any objective c object n extremes and NX numbers are treated specially an extreme gets converted to a string in Java Script an NS number is converted to another in JavaScript and NSA is also handled specially and I'll talk a little bit more about that in a moment web script objects again web Script object represents the JavaScript object in objective-c when you pass one of these back to JavaScript it becomes unwrapped and you have a reference to the original JavaScript object all other kinds of objective-c instances are represented by proxy in JavaScript so here's a summary of the type matching between JavaScript and rejected see when calling javascript farm objective c NS string string and it's numbers map to a number now an nsarray is a special case arrays in Java Script strange flexible which strange beef you can have spa surveys you can have associative arrays and it's all the same object and its array is really a much simpler the straightforward arrays but there isn't a one-to-one correspondence between the JavaScript array and an nsarray so the array you get in javascript is we don't wait unlimited you can't use it like an associative array but you can access the elements positionally a web Script object of course as I've said gets unwrapped in in JavaScript when become to the original JavaScript object and any other objective c instance is mapped to a proxy in javascript and i'll talk a little bit more about boxes next first let's do demo and i'll do a quick demo to show you how to write a web site application to interact between Coco components Coco widgets and a web page ok so let's build from scratch I've got a fresh Xcode project here first let's build a nip file we're going to use this and I've actually built this already so I'm really just going to take the components from the existing project and copy them in to this new project so let's get a nib file copy that into the project and open up this nip foil so you can see here I've got three controls that are going to specify three different images and when I click on one of these controls it's going to set the content of the webview to be the appropriate image for that button and you'll see that the controller is just going to send although the radio matrix is just going to send a message to a controller object to pick an appropriate picture type so let's go ahead and hide this and pull in the code to do that here we go so this could very straightforward it has the single method well as two methods I'll talk about this one first picture type clicked and this looks very much like the example I showed you on the slide we get the windows script objects which corresponds to the window object of the web page then I go ahead and call set image with arguments when I'm constructing to set the appropriate URL for that particular image let me back up a second and show you what we do in our wake from nib method what I'd like to do is load content into that web view that represents the page that I'd like to show so I do that by extracting a resource out of the bundle that is a string it's officially just txt file I can put that to a string and then load it into the web view so let's take a look a beer the webpage that we're going to use here add it as a resource very straightforward page it has a single javascript function it's going to get the image element by ID and then it's going to set the source of the image element and this source in the URL okay so the final thing I need to do is add some images picture of fire let's add that and the picture of the grand canyon and a picture of Napa Valley go ahead and add those and then let's build this project and run it okay so here's my application now when i click on nasa what actually happened there is i sent a message from this objective c control to my objective c controller instance which in turn sent a message to javascript to load the napper image click on the fire button it's sending the file URL to javascript and grand canyon go ahead back to the slide okay what about the other direction I have some JavaScript code to watch the claw into objective-c well this is web proxies come in if you have an objective-c object that you want to have a peer in JavaScript will represent that object by proxy and will automatically reflect the methods and the properties of the objective-c instance in objective in JavaScript and this is an up we have an opt-in policy for the methods that we expose to JavaScript what that means is you have to tell us I want this particular method or this particular property to appear in JavaScript and the kinds of things that you can expose the JavaScript are any objective c object you can expose methods that takes Kayla kilometers scalars are in some floats of course and other kinds of scalar type you can also expose scalar its variables what you can't do is expose structures or pointers to arbitrary memory you can expose a pointer to an object but no other kind of pointy types ok so again let's say how i have this example class in address book class and it has these methods get a shared incidents get a composite name the first and the last name and a way to position the cursor in the address book position carded index and i want to expose this to javascript i do i do it well i get the instant the shared instance and then i simply set the objective-c object is a property on the window object get the window Script object and call set values the key with the name address book and that name is the name that will appear in your JavaScript code for this particular property so from JavaScript if I wanted to write something to retrieve the first name in my dress book this is how I do it windows autograph book gets me the window object and I call position card index to position the first record and then I called composite name simple as then now for those the view that were paying attention that last slide had something that was a little too strange that under by the name at the end of that method name position card index on tuba what's that all about kind ugly well by default we have to mass objective-c names to JavaScript names and objective-c names have colin's after each or before each parameter so a colon gets mapped to an on the bar but that's really ugly so if you want to override the default name of an objective-c method that gets mapped to JavaScript you can implement this method on your class the class that exported to JavaScript web script name for Celestra so in this case we take it's lighter in and we return the name that will be exposed to JavaScript so I'm going to use position carded index with no underbar so now my JavaScript code would look like this position guarded index okay so in practice you typically want to expose functionality into javascript very early on in the construction of your page so you might have javascript code in the head section of your page and that JavaScript code needs to access functionality native functionality so when do you actually publish your objects into JavaScript well you can do that really early on we've added a new web frame load delegate method which is Windows Script object available and this method will be called before we do any processing on the page so you can insert your functionality into the JavaScript interpreter before we evaluate any JavaScript on the page so in this case I'm going to insert the address book instance into the JavaScript namespace very early on okay summarize type matching in this direction a javascript string is mapped to an error screen a JavaScript number is mapped to an NS number all other kinds of JavaScript object objects including JavaScript arrays a map to web script objects and JavaScript boxes or an objective-c proxy in JavaScript when it's past back to objective-c is represented by the original objective c instance ok let's do a demo in this direction and what I'm going to show you is a demo to dynamically dynamically construct a web page using the records from my address book to close this project and again we'll build it from scratch to show you how simple this is and I built it already so it's not really from scratch but so first thing will copy the nib file again for this project into Xcode a nip nip file is really straightforward it's just a single window with a webview now one thing that we've done here is we've set up the frame load delegate of the webview to be the controller instance so this is the incidence that will receive that message window Script object available let's go ahead and go into Xcode and take a look at the source code well she first let me show you the the address book class that will be exposing so it's a pretty simple object with a shed instance it implements two methods two weeks or two or three methods that we care about composite name this will return the name of one of my contacts in my grass book the way to position the cursor in the address book and a way to get the number of cards I'm going we're actually using the address book framework here to download my entire dress book into a simple NS array that's how we're going to get the data there are a few things that are relevant to this demo web script name for selector this is how we map the name position card at index Cohen to position carded index and this method is the message that we implement to expose the message this case yes I want these methods available in JavaScript and you have to implement this method if you didn't implement this method and people if you returned no this method every single method of your objective c object would be exposed including things like the owlet and it doesn't make a lot of sense to call the Alex from JavaScript receptor controller instance source code this looks pretty similar to the previous demo I'm going to extract some content HTML content out of the pack out of the application bundle and I'm going to set that content as the HTML of the web frame in my wake from their method and now very simply in the frame load delegate method I'm going to set the address book as the address book property of the window object so I need to do one additional thing here and that is as the address book framework oh one additional thing too I have to copy over the source for the page and let's take a look at that so this is a pretty simple page initially it has an empty body there's no content in the page we're going to dynamically dynamically construct the entire page and that's going to be done in get contact onload handler and they simply accesses the address book instance that we've exported to JavaScript gets a number of cards and then iterates over the cards getting the name of each record in the address book and then it creates a div and adds back to the document go ahead and build and run this there we go let's pull up the address book and make sure the visa records match oops phone numbers in there so all the records in address book match the dynamically constructed webpage so it's really simple to do this stuff if you have native functionality and you want to get into JavaScript it's very straightforward and I think there's going to be a lot of synergy between the cocoa framework and this technology so so for example core data is going to really allow a lot of interesting applications that use web pages of front-end to relational databases one last topic on scripting and that's the Dom well we've introduced a significant new set of objective c api is to expose the dom in objective-c that just the week quick recap the dumb is the document object model of a web page and there's a set of standardized api from the w3c that define the Dom API and we have a mapping of that API in objective-c now if i get a Dom object image like to see what does that mean in terms of scripting well all Dom objects inherit from web Script object so that means if you have a dollar object it's inherently scriptable so well let me just have another call out to the sectional Friday again that's going to be a great session talks about the Dom API in detail but let me show you a quick example of how you would do this there's a new method on web frame to get the domdocument that's the document object to represent your page and you can evaluate web script on it just as you can any other web Script object in this case I'm going to call get element by ID to get the context there or the element with the ID context of this is equivalent to the objective-c method get element by ID get element by ID is a new method in the Dom API we're introducing these two mechanisms are equivalent so why would you use one over the other well it's much more efficient to use the objective c api directly so for example if you're doing things that iterate over a tree it's much more efficient to use the get a 1 by DS you're going to scan the entire document for an element soon on the other hand if you're doing stuff dynamically with JavaScript the first method is appropriate so there's a lot of flexibility in the scripting API we think it's going to give you a lot of new capabilities that are pretty significant okay finally last feature that we're going to discuss in some detail the canvas element and when we were putting together the dashboard we came to the conclusion pretty quickly there were some things we wanted to do we just couldn't do in the dashboard with existing HTML technology so we and one of those things was doing procedural renderings into a gadget so we introduced the canvas element to support that and what it allows you to do is to draw into an XML element on a page and specifically you draw into a page using a context and we've created a 3d context that allows you to do 2d drawing operations into a canvas element on the canvas element is treated very much like the image element it's a replace by default to replaced inline element which means it flows just like an image on a page and because the context is backed by core graphics you have all the power of core graphics in terms of the rendering capabilities into a canvas which means your toddler accelerated if you have the appropriate hardware and you can really do really fantastic vector graphics using the canvas element and you can do really interesting effects with image compositing so specifically the drawing operations that we expose all the same glowing operations that you can do with a CG context you can construct arbitrary path including curves you can fill in store closed paths you can composite images you have control over the transparency levels and the compositing operation at the global level so you can actually set a composite operator for drawing other than just image compositing this is an example of how you specify a canvas element on a web page it looks very much like the image tag you specify width and a height a composite operator this is the operator that will be used to composite the resulting image into your web page and this by the way the same compute name that we've added to the image element okay so how do I actually do the drawings in to use a canvas element well here's an example in this case I'm going to draw a line from the top left to the lower right of the canvas the first thing we do is we get the canvas element using the familiar get element by ID and then we ask that element give me a context in this case a 2d context now I begin by going click on text it's going to fill a context with transparency which talk to pass you know 00 to the lower left I mean lower right check the stroke color and struck out that pretty straightforward so I think this stuff is best illustrated with some demos let me show you some demos of the canvas let me increase the font size of it okay so here's a really simple web page that specifies a canvas element not too exciting yet let's open up safari and load the page now let me turn the network back on before I forget to do that so what does this look like well next to a rectangle not too exciting so we're just starting so let's go ahead and draw some lines into the canvas so what I'm going to do is add a little script in here and this is actually going to draw two lines like this through the through the page we're going to do it in the onload handler of the body element okay let's go ahead and save save this is something different two lines in canvas whoo okay let's crank this up a bit show something a bit more wizzy okay this looks more interesting let's hide the other applications so what I have here is a rectangle with some shapes but we're drawing dynamically frame by frame using in canvas and we can do things of course you know change the colors and because the canvas itself is an element on the page like any other element we can do CSS tricks like positioning so I'm going to dynamically move the canvas as I render each frame you need to do things like change the colors there's one other thing i want to show you on this page which is kind of interesting and that's the circle up there this is another canvas but if the canvas used to implement a widget special a new kind of widget on the webpage and what this widget does is specify the objects and the angle of the shadow in each shape so you'll notice there's a blurred shadow and its shape as I move this control to see that shadow change so this is implemented using standard JavaScript event handling and the canvas to render the widget pretty nifty so of course we can change the blue raiders with the new slider input element I can do things like change the background color so this is pretty cool let me show you one other thing that I think is pretty cool let's stop this thing moving there's a new debug menu in JavaScript that I call dashboard mode and what it does it takes away all the stuff disavow edge and you just have the pure HTML here with no background so putting if you let me just show you how powerful this really is let's pull up another window load some some interesting content here incredibles this trailer i love this trailer okay and you can see that it's doing everything you'd expect this is a web page being rendered transparently over quicktime movie pretty amazing okay just a few more slides so it's not just about web browsing this web key technology is really powerful and now with this ability to bind all the cocos flame web technology into javascript there's a lot of new capabilities that we're providing to you you can build visually compelling applications really quickly and you can leverage your existing web design skills classic prototyping this is an interpreted environment so you don't have to go through a compile phase it's great simple deployment these are just web pages so I said we'd build a gadget so let me do that now in the remaining time that we have this is actually a really simple gadget it's the clock most of the clock that is included in the in the dashboard so the first thing we're going to do is construct a web page to close this crease the font size save this okay not very interesting I know I'm just creating the face of the clock it's a simply an image we need to put an HTML element around this I've been kind of sloppy I'm not specifying the doctype okay let's load this into Safari very sloppy HTML at this point but that's the clock face that's the beginning of the web page of the gadget i should say okay now what I'd like to do is add some JavaScript code to load in the images that I want to use for the clock hands so what I'd like to do is have just three images that are used for the hours minutes and seconds hand and use the canvas element to apply transforms to the image images as I composite them into the canvas because it's just Fiji so you have access to all the transform capabilities that you'd expect so this code creates three image objects the minute hand second hand and our hand and it goes ahead and loads them and I have this little function that counts out the images as they loaded because in this case the images are all local but they could be remote so it may take some time to load the images and I don't want to start compositing until I have all of the images okay what next let's add a load handler and the canvas element so what i've done here is i've added a canvas element that's going to be positioned using CSS right on top of the image element initially it's just transparent but it's positioned over the image gentlemen this is standard CSS you can see i'm specifying absolute positioning 00 so let's take a look at what this looks like now it's fine that's exactly the same there's nothing new ok let's add some code to draw the hands I'll first I guess we have to add one more thing we have to add the code that will figure out the appropriate angle to composite the hand so we have to know where to position the second hand the minute hands in the hour hand we do this using the standard JavaScript data object some simple math we compute the angles for the second minute and our hand never go ahead and draw the hands float this again still nothing new ok let's get to the final piece actually drawing the hands resize this window a little bit okay so what we did with the onload handler is we added a javascript function that's going to be called every second and this JavaScript function will go ahead and compute the angles for each of the hands and then call a draw hands method that will do the actual work of compositing the images and the draw hands method is implemented like this the first thing we do is make sure we have all of the images available the image count has to be three we clear the context and then we call say well what's a Volvo save save the graphic space of the context that's used to enter into the canvas so after calling say followed by a restore if you've done any transform operations in between a save and restore they'll be restored who's appropriate so what we actually are doing here is we're translating to the center of the clock face actually just a little bit above the center of the clock face because that's the center of the circle then we go ahead and call draw hand which is my job function up here too compositor that image now we've translated ready to the center but now I need to rotate is appropriate to composite the image hand so here's the rotate of the angle that we previously computed and finally finally we get to draw the image using raw image on the context and then we restore the angle because the next hand is going to be painted a different rotation so that's it let's go ahead and load it now oh yeah we have hands working but they're not fighting the center what's going on with that well by default the style of the body element on a web page has a margin that isn't 00 so let's go ahead and add some style to adjust the margin of the body element we're going to set the margins to do we live that looks much better then we have the clock widget let's go ahead and turn on dashboard mood group just like it appears in dashboard so you have it so I want to say a few final words about the availability of these technologies of course it's going to be about everything we've talked about today is going to be available in to 52.1 tiger all of the web kit features that we've talked about today are also going to be available on Panther in 10.3 and there's already a developer preview of that release that you can download today just one final point there's lots of documentation and sample code available on the connector apple com site encourage you to check that out