WWDC2003 Session 311

Transcript

Kind: captions Language: en thank you and good morning it's good to see that there are so many people coming to this session because automated testing has been one of my most most considered ideas to promote from echo s10 so of course you know why are we here because there's a great need for testing automation our applications have become more complex our our needs for for debugging we want to reduce our our engineering support requirements and what we want to show you today is two things one a solution from Apple that's Apple script based functional testing based upon our section 508 accessibility API and to a solution is being provided for us from the redstone redstone software folks called eggplant which was user centric testing from a scripted scripted base and without further ado let me introduce John Comiskey from the senior software engineer from the apple script group thank you Godfrey thank you all for coming good morning my name is John Comiskey i'm going to engineer in the applescript group and i want to talk to you today about GUI scripting first a little bit about where GUI scripting came from section 508 the federal law that mandates accessibility features for hardware and software purchased by the federal government and if you're interested in that kind of thing you can look at the US government site about the law itself Apple responded to this being good citizens by creating an accessibility API it's in Jaguar right now you cannot look at the header files which is how I learned about it you can also go to the documentation on the developer website and learn about it that way if you search for accessibility you'll get some hit on documentation for carbon and for Coco and for how it should appear to your users if you're working in the accessibility space if you're building hardware or software to make the computer more usable by people with disabilities when when applescript saw this we realized that this doesn't just have to be an API it could be available through scripting as well so we created GUI scripting it's a sweet inside the system events program if you target your event system events they'll be handled there and if you want to look at the dictionary you can open up the dictionary for system events and look in the process is sweet because GUI scripting has been implemented as an extension to the processes sweet and there's some interesting examples on the applescript website and you can see the URL for that there so what is GUI scripting it's an object model of the current state of the UI at this particular instant instant you can access the things that you can see on the screen through a containment hierarchy of objects every process on the machine is now also a UI element in addition to being a UI element each process is a container for other UI elements every process contains one or more UI elements usually in one menu bar and one or more windows and then containment for UI elements is recursive inside the windows there's buttons sliders all sorts of things and you can get all of these by walking down the tree to the object that you're interested in every UI element has a set of attributes buttons have titles various UI elements might be enabled or disabled you can get at these you can get at these attributes as a collection as elements of the object as elements of the UI element itself certain ones that are particularly useful we've called them out and given them property names so as you can get in them directly one less step in your hierarchy to get at those on each UI element also contains a collection of action those are the things that that UI element can do sew buttons can be pressed sliders can be slid and an individual UI element will tell you these are the actions that I think I can do and if you want to do if you want to do one of those things there's a perform verb and you apply to perform verb to the action object and the action applies itself to the two its container which is a UI on it so what can you do with GUI scripting you can access the attributes and actions of all the standard controls on your screen so you can find out how big a button is where it is on the screen whether a menu item is enabled or disabled you can get at all of that information and you can use this information to test the user interface of your program whether you're whether your application is scriptable or not you don't need other kinds of scripting to do this and this allows you to control the behavior of otherwise unscripted applications whoever would do that what it can't do Glee scripting can't get at any non standard controls if your application has a lot of GUI widgets on the screen that you designed and built yourself we probably won't know anything about them and we won't be able to get at them they'll just be dead spots on the screen that we can't touch the accessibility API does have some features that allow you to take your non standard controls and make them behave enough like standard controls that the accessibility API can see them and work with them I would strongly suggest you do that if you depend heavily on custom controls and not just for GUI scripting but for accessibility we bought into accessibility you guys should too and the way you do that is to take your custom controls and make them behave in a little less custom fashion for accessibility purposes another thing that you can't do with with GUI scripting is you cannot directly test the object model of your application everything comes in through the user interface you're actually clicking buttons and sliding sliders so there'll be parts of your code that don't get touched coming in this way it's also true that there'll be parts of your code that do get touch coming in this way that don't normally through object model scripting object model scripting usually come letely bypasses the view and the controller layer and go straight to the model objects if you use GUI scripting you'll invoke both of you and the controller before reaching the model objects so it gives you more testing in some parts your program less testing in others and last thing most important left is the reason I'm here is to tell you that this does not replace real object model script ability in your application this is not your ticket to blow off scripting this is this is at best a stopgap and we'll get to some of the reasons away so how I use it first thing you have to do is you have to turn on the accessibility API which is off by default in Panther and it is not scriptable to turn it on so somebody is going to have to turn on the accessibility API by hand um before they can use this next thing you've got to do is identify which UI element it is you want to act on and this can be used or it can be hard depending upon how complex the screen is and how deep the hierarchy goes then you want to identify what action it is you want to perform on that UI element and then lastly performing action on the element so if you need to enable the accessibility API you should launch the system preferences select the universal access pain and down at the bottom second check box from the bottom is enable access for assistive devices that's the one you need to turn on or none of this is going to work to find the UI element that you want to work on their three choices of how to do that you can write an applescript that digs its way through the hierarchy layer by layer until you find what it is you're looking for this can be hard because there's things on the screen that you can't see group might exist and the thing you're going after is inside the group unless you probe what's inside each group it's going to be hard for you to find it there's a couple of tools that make this a little easier one comes from Apple called the UI element inspector and you can download that from the applescript GUI site there's another one from the folks at prefab called you I browser and they're both very good at finding things the one thing that you I browser can do the UI on an inspector can't is it can actually spit out some apple script code for you so you don't have to write it yourself then you want to identify what action it is you're going to perform that's one step down in the hierarchy so if you've written a script to find the element that you're interested in then you just ask that element tell me what your what your supported actions are in the UI element inspector and you I browser if you're pointing at that UI element the actions will be listed right there for you and again the UI browser can just spit out the code you need to perform one of those actions then you want to apply the action and you do that by applying the perform verb to the action object and that you do that the same way whether you were using scripts or the UI element inspector to find it and again you I browser does it for you the it's a the accessibility API and GUI scripting support keystrokes it's always better to set the value of a text object rather than to send it keystrokes you're going to get the results you're looking for with less hassle that way but there's some times when you have to send keystrokes particularly if you're typing into a field that's actually a non-standard control you can still get data into it by sending keystrokes keystrokes are always directed to the object that has the focus and if you want to change the focus an applescript activate set frontmost quick and select are all things that can change the focus for you and you'll have to experiment to see what you need in your situation to get to focus where it needs to be the keystroke verb takes Mac Roman characters and turns them back into virtual key codes this works pretty well for the English language it does not work pretty well for other languages where we know that the problem we're working on it on the accessibility API group has the same problem that I have and we're trying to figure out a more robust way of handling this but for what you've got now works well in english not so well in other languages to get around that to give you an escape hatch we also have a key adverb where you can send your own raw virtual key codes finding out which one to send is hard but once you find out which one it is you can send it and now I'm going to show you how to do this I've got some samples that I've written up since UI elements is off by default it's a good idea to test to see if it's on first the number one source of calls and emails about this when the people that have been working with it so far is that it's been off when they tried to use it so i put a test for that in each of these scripts and if it's not on I take that there is enough script ability and in a system preferences to take you right to the point where you want to turn it on and so we're going to do that and this is going to ask me for a password and that was it so now since we've checked enable access for assistive devices the accessibility API is on and gooey scripting will be able to do its bit ok hide your phone there okay now um sometimes right we're going to lab and the other people are working to lab like to take the doc and stick it someplace else I like it on the bottom there's a menu item that will move it for me but on that thing itself is not scriptable but we can get at it with GUI scripting you notice that I click the menu then I click the menu item within that and then I click a hierarchal menu item within that gooey scripting is very deliberative you have to you frequently have to traverse the path to something before you can use it the Edit menu is particularly sticky in this fashion because most of the things in the Edit menu are disabled until a selection is made so you frequently have to step through all the intervening steps to get to your your item if it's in the menu bar and it moved the doc we did this in the first place because we had a customer that had a dialogue that was blocking a workflow and he just needed to dismiss that dialogue so one of the things that you're likely to want to do pretty soon at the beginning is dismiss a dialogue and again we make our same we make our st. check at the beginning to make sure everything is on and because of because of the way that text edit works I have to expend a keystroke in order to get the document to be dirty and so what I'm going to do is create a text edit document put some text in it try to quit the application and it will come up and say I have stuff to save and then we'll use glue a scripting to dismiss that dialogue and I got an error that's script editor because I don't have text editor awning already know you ski while I go out to the finder and get my applications and launch text edit okay this might work a little bit better this time there put some text in there send a keystroke its Sun okay I'm sorry I went too fast textedit did close and quit without you guys getting to see the dialogue I'm sorry about that might try that again but we'll go on to the third one instead and here's a scripted that I've written that set the preference in this particular instance the sound and it just toggles the sound between two different values but it shows how a slider can be manipulated it shows a preference that's not otherwise scriptable can be accessed no yes okay I don't hear a beep is that the sound from the demo machine isn't on that true okay okay all right now we should hear it change and be louder and now it changes again and it's softer okay thanks just just for fun just for fun let's see if we can get to TextEdit to work the way we thought it would click the script editor relaunch the script will try this one more time just to see if we can get it to work nope sorry it happens too quick and I seem to have an error as well but that saw that's GUI scripting you can use it to test your application we expect that this is not going to be a good end user solution testers that work for you and you pay them to run the same test over and over again and change the tests every time you change the code they'll put up with this but end users are not going to take this as a solution for their problems so we'd like to encourage you to learn about going scripting use it for your own purposes internally as a test tool but for your end users you should provide them with object model scripting and now we can go back to the slides I'd like to introduce Doug Simon's from eggplant software or from red stone software who's going to talk to you about eggplant much more professional tool for testing your application [Applause] hi it's good to see you all here a lot of people I'm Doug Simon with redstone software and we're going to tell you about eggplant which as you've heard is our software test automation tool so begin with why would you test I'm sure that all of you have a lot of reasons or you wouldn't all be here at this session but here's a few ideas that I had the mac OS environment is about quality software mac OS users expect a good experience with your software if you deliver you know crummy software that doesn't work well you're not going to have a good experience in the the market clearly also there are some advantages to you if you're a software works well you're going to have less for support calls and your support personnel can spend time answering questions about your programs functionality rather than fighting fires with bugs so the bottom line i think this equation speaks for itself better software means happier customers and if your customers are happy then chances are you're happy too so why don't people test well of course we all do test and developers when we add new feature to a product we're going to try it out and make sure that it works a lot of development environment or ganization have a big emphasis on unit and functional tests at a lower level but testing is a complex process if you've added a new feature to an application chances are it works well on its own but maybe it's going to have some difficulty interacting with other features of your application that have been there already so those kinds of interactions both within your program and with other programs are important to test as well so you might want to go hire some people to do this testing for you and that's great but of course it's a costly thing to hire people and uh the process of testing is there's not the most fun always it's a kind of a repetitive and mind-numbing task often and you might rather have those intelligent people doing something more interesting so you might want to invest in some test automation software to free up some of your testers time make them more productive unfortunately a lot of the software solutions that are out there have been traditionally hard to learn and use there's a deep learning curve involved and so it can be difficult to get going and really make use of this let's look at a few different testing approaches first of all obviously is manual testing almost everyone does a certain amount of manual testing and this is good but it can be very labor-intensive and besides that people have a tendency to be a little inconsistent if you even if you've written up test scripts that your testers are following to you know step your software through various processes different testers are likely to follow those instructions in slightly different ways and if you don't have instructions then who knows what sort of results you'll get from one test run to the next so one obvious solution is to automate that process and in the past sometimes that's been done through hardware based solutions Apple computer at one time had a hardware testing product and our own parent company gresham had a hardware based testing solution also and the hardware test testing works great when it works and the problem is that it can be very inflexible and platform specific if the system being tested changes in some way or the young you're working with different hardware that you're connecting to it can be rather fragile sometimes and prone to problems so obviously a more flexible solution would be to to use software and software solutions can be great but as I mentioned before they're often hard to learn and also require integration most often into the software that's being tested so that's an additional barrier to implementing and using those we found in talking to people that quite often the expensive software solutions the companies invested in ended up collecting dust in the corner as shelfwear so as I said that our parent company one time had a hardware testing products and about a year and a half ago we started looking at what other alternatives might exist we abandoned the hardware product which was pc based and investigated what was going to take two to come up with a better solution so i'd like to share with you some of the design principles and goals that went into creating eggplant first of all we wanted it to be easy to learn and easy to use because we really didn't want to create another piece of shelf where we wanted people to use our software and get some benefit from it we also would like it to be powerful and elegant well those are great marketing buzz words but what we mean is that we wanted to focus on the essential functionality that testers needed and we also wanted to be flexible so there could be integrated into an overall testing environment we also wanted to create eggplant to be independent we wanted to to minimize the amount of integration required in fact no integration required with your software so that you don't have to know ahead of time what you're going to be using to test with and we can test with any software that's out there we also thought it was important to be non intrusive to have a minimal impact on the performance and behavior of the system where your software is running cross-platform is another important goal because software exists in a multi-platform world even if you're creating software for mac OS can only chances are that your your program these days is going to be working with files from other systems importing or exporting and you need to be able to have some way to to work with a cross-platform testing environment and finally we felt it was very important to test from the user perspective as we said there are a lot of functional and low-level unit type tests available but the bottom line is you want to create software that is going to satisfy your users if you have a wonderful computing engine that does wonderful calculations but the the user interface is is crummy or simply doesn't work your users are not going to get any value out of that so how does eggplant work well first of all we provide remote control of another computer how many of you have heard of or used a product called VNC and that's quite a few so VNC is an open source project that's been out there for a number of years that allows one computer to remotely control another it does this by displaying the the screen of the other computer and allowing you to control the keyboard and the mouse so eggplant leverages this technology we've built a VNC client into eggplant that allows us to talk to any VNC server out there and as an open source project VNC is available for essentially any platform that you wish to test so we're able to connect not only to Mac OS 10 but Mac OS 9 unix linux solaris and any flavor of Windows then we added in scripting we have a powerful scripting language that allows you to automate any task on any of those systems so the bottom line is that eggplant can drive any software on any system through the user interface just like a user would so I'd like to invite my colleague Jonathan Gillespie up now to give you a quick demo of how a gland works so there thanks Doug so for our first demonstration what I'd really like to show you guys is exactly how it plant works so so that you can see an example of how you'd use it to test your own applications so as Doug mentioned eggplant works by connecting to a remote computer so I'm going to be connecting to the demo machine over here just like so and I have an IP right I'm going to go ahead and make this display a little smaller so we can so we can go ahead and see it easily over on this machine ok so there we go that's our that's our remote machine that we're going to test now to show you how to go about testing one of your applications we need an application to test so we've gone ahead and created a really simple application this application is so simple we decided to name it very simple so although very simple very simple I want to go ahead and show you what it does do so when you double-click and launch this application it brings up a login panel and you go ahead and type in your login name in this case it's spread with the password of Wilma and then very simple goes ahead and brings up a list of some pictures some interesting pictures so we'll go ahead and select a picture from the list and click the preview button and as soon as it does that it it launches that image up in the preview application and that's about all that very simple does it's just a real simple application to show you something we might want to test so I'm going to go ahead and quit out a preview and quit out of very simple so now that you know what very simple that is I want to go ahead and build a test to test that make test to make sure that our application works correctly I do that by creating an eggplant test suite a test suite is simply a collection of all of the resources that you need to test an application you can have multiple scripts within a sweet so we'll go ahead and create the first one of those now just like that come on okay right from this point we could actually just start scripting right here in our script window using eggplant scripting language sense talk but instead I'd like to use eggplants script generation capabilities to show you just how easy that can be so we go into what we call capture mode you'll notice that in capture mode we're no longer interacting with the remote computer directly but instead we have a selection rectangle here which we can use to identify elements of the remote screen that we want to deal with so in this case we're going to highlight in on the very simple icon and tell eggplant that we'd like to double click on it by choosing double clicks from the toolbar sorry eggplant brings up a save dialog and we give it a name for our image and click and click Save when we click Save three things happen the first is eggplant saves that image in our very simple or in our very simple test suite the second thing it does is it actually generates that command into our script double-click on the very simple icon and the third thing that it does is actually execute that command against the remote machine when it does that eggplant goes and searches through the remote screen until it finds the very simple icon and double clicks on it and you can see here it's gone ahead and launched a very simple application the next thing we want to do in our script of course is click in the name field so we can enter our username and password we identify that field by the name label but of course we don't want to click directly on the name we want to click to the right of it we do that by setting eggplants in the images hotspot you can set the hot spot anywhere on or relative to the to the actual image so we'll set it right there in the field and take click again eggplant saves the image rights that into the script and you can see now that the name field is selected now we'll go ahead and type the username in we have a type text command so we can type in Fred and you can also spent ten special characters like the tab key to go to the password field and then we'll type in the password now we'll go ahead and click on the submit button just to log in and up comes our list of pictures we're going to go ahead and just pick an item from the list we'll go ahead and pick the chess item again and click on that and finally we'll go click on the preview button you'll notice these aqua pulsing buttons like preview in the submit button are pulsing on the screen since eggplant works by finding images on the remote screen we we have a special color matching mode that allows us to locate those images there we go it's message the preview application and brought up our image we want to go ahead and make sure that that image actually came up on screen because that's what eggplants all about is validating the results of your application so we'll go ahead and use the wait for command to do that the wait for command will wait up to the maximum period of time for and for the image to come up on the screen in this case that'll be eight seconds as soon as it plant sees that image up on the screen we know we've successfully completed the test so we want to go ahead and quit out of preview and quit at it very simple I'm going to use the type command feature to send a command queue to preview and that quits out and then I want to quit out a very simple and let's go ahead and do that through the menus there we go so as you can see we've now created our very first demonstration script that tests are very simple application we have all our script commands here we have all the images that eggplant uses right over here so we can go ahead and run the script and watch eggplant step through login highlight that see the image and quit out I think anytime eggplant runs a script it records the results over in the results tab in our suite so we can click on the most recent run of that script and you can see here that eggplant has stored not only every action that it performed but it also identifies where on the screen at found that image and it also identifies at what time it occurred at this point some of you are probably wondering exactly what it looks like if a script fails so we can cause a failure by going in and getting rid of the password so we can't login and we'll go ahead and run that oh sorry eggplant fails here because it can't continue forward through the rest of the script so you can see over here in our results in our results tab that the last run isn't highlighted in red and if we select that we see that eggplant automatically has taken a full screen shot of what the screen was like at that point and it's highlighted the image that it was looking for by using this you can quickly see that we weren't able to find the chess item because the authentication failed so that's a pretty basic look at the easy features of eggplant so I'm going to pass it back to Doug thanks Jonathan so now you've seen something about how eggplant work just take a quick look at some of the technologies that are behind this and the benefits that those give to you first of all we have a powerful scripting language called sense talk penstock is a very easy to learn understandable language it's actually modeled after hyper talk the language in hypercard which probably a lot of you have seen before we also have added a number of extensions and set stock is a very full-featured object-oriented language so it gives you the power that you need to do whatever it is that you need to get done there we also use VNC as we mentioned before which stands for virtual network computing and VNC as i said is an open protocol an open source project that allows us to see the other the screen of the other computer and control the keyboard in the mouth this gives us separation from the application and system that's under test as you saw in the demo we were testing not just a very simple application but actually the interaction between very simple and the preview application from our point of view there's really no difference all we know is that there's a screen out there and we can interact with it this gives us a lot of flexibility and we not only test interactions among different software on the same system but even across a system from one computer to another and finally of course this separation allows us to control the system just like a user would we also use cocoa I'm sure a lot of you are using cocoa and we think this gives us some advantages obviously it gives us a clean intuitive aqua interface but also more importantly allows us to be responsive to our customers needs we use the XP extreme programming approach and we find it's very important to to be able to be responsive to the requests of our customers let's take a look at dock at eggplant as a tool we see eggplant as just one component of your overall development process we know that there are a lot of other tools out there that you use and that you need to be able to integrate with so eggplant is able to interact with other tools that can be integrated into your build process of eggplant script can be called from the command line or an entire suite actually of script as you'll see you shortly and we also are able to integrate with any reporting tools you might have all of the files in an eggplant sweet our standard files are the images we capture or save as tiff images and all of our report logs that we generate are in plain text files that you can load into a spreadsheet or access with another reporting tool that you might have we're also able to interact with UNIX command line tools as I said we can be called from the command line and we can also call other command line tools and the same with applescript now here's an interesting point I don't know if any of you how many of you are aware of the Apple compatibility labs if any of you have used those not too many hands going up Apple has this great facility they've got a number of different computers dozens really of all different makes and models of well all Macintoshes and and running various versions of the mac OS and these are available to developers if you go into their labs in cupertino you can bring your software en and test it out to make sure that it works on all the different platforms unfortunately it requires a trip to Cupertino to do that up until now we've been working closely with Apple to ensure that it's possible for eggplant users to access the labs remotely from the convenience of your own office and [Applause] [Music] and we thought you might appreciate that so anyway I'd like to bring Johnson back up for another demo of some of eggplants more advanced features thanks conduct okay so on our previous example we saw some of the basic features of eggplant but now I'd like to take this opportunity to show you some of the more advanced features so I've gone ahead and built off of the demo that I that I did just a moment ago and created another test suite which I call not so very simple you'll see that not so very simple actually has two scripts now let's go ahead and look at the login script I've actually gone ahead and just broken out the functionality that deals with having the user login as you can see in here you can pass parameters can you guys see that okay as you can see you can pass in parameters in this case we pass in the name and the password into this script and go through the process of validating the user so let's go ahead and look at our main controller script which is view images you can see we've added quite a few little extensions in here the first example is we've created a couple of lists here that we can use to iterate over all of the various logins that we want to validate and that's what this repeat loop here is for it goes through the basic test we've already implemented but now we want to try it with a whole variety of cases here's the login line you can see where we're passing the parameters to the other to the login script I just showed you and then we go forwards through the rest of the script you might also notice that that is within a try-catch loop whenever eggplant has detects an error has a problem it raises a typical exception which many of you may be familiar with and you have a choice you can either choose to let that exception raise up to the top and eggplant report that as an error or you can catch it and try to resolve resolve that here in the in the first example I gave we had a problem when we tried to login and the password was invalid what I've done up what I've done here is set a catch which will catch and see if if the error was because there was the no password panel came up and if so we go ahead and dismiss the password panel and continue iterating through the list and continue iterating through the list we go ahead and log that that passwords that passwords not valid but will gracefully continue so that we try all of the example so let's go ahead and bring up our connection again and try running our more sophisticated script so here you can see it doing the same thing I have animation turned on so you can see as it's calling the loops well there you see an example of where eggplant doesn't work let me go ahead and quit out and run that again they're go this time I decided to try something new that may be all hadn't seen so I brought up a picture of a g5 because I thought that'd be fun and actually I think it blew through it while I was talking but you might have noticed I'll run it again real quickly it actually brought up in the on the second log in there that's actually an invalid login right coming up here oh well apparently that is a valid Logan okay one final feature that I'd like to show you guys and Doug alluded it short alluded to it a second ago we have a nice little schedules tab in our suite the schedules tab is useful for building up a batch of scripts that you'd like to run so you can drag some scripts in there just like that the schedules tab has some nice features you can you can make scripts dependent on the successful completion of a previous script you can enable and disable which scripts you'd like to see running at a at a current point in time you can set up a repeat loop so that you can repeat that over that batch a number of times another neat feature that are that the schedule tab has here is we can actually choose which computer we'd like to connect to to run that so I've so i'm going to pick these two computers and we'll start running new computers are actually located downstairs in one of the lounges it's actually the ADC compatibility lounge that they've brought to this facility but they could just as easily be any computer computers anywhere on the internet that we're accessible so you can see it running through there that that logins not valid and it cleans it up so I'd like to point out that although what we're doing here is testing the same script on two different machines you could just as easily switch your connection in the middle of a single script and in this way you could test scripting you could test something that runs it moulds at multiple points on different machines and hopefully this has given you a good example of how you might be able to use eggplant to help test your application alright thanks again Jonathan so I have just a couple more slides here we as you saw eggplant is very useful for testing your software but because we are automating process on a remote system it doesn't have to necessarily just be used for testing software it could be used for some other things as well and here's a few ideas that we thought of and that some of our customers are playing around it one is documentation we know that you all like to document your software but of course chances are you've got a lot of screen shots than in your documentation and as your software evolves you're going to need to replace those screenshots with up-to-date images using eggplant you could create a script to walk through your application and at the appropriate point where you want to capture a screenshot you can tell eggplant to do that we can capture the full screen or any region of the screen that you want to capture we think that could also be useful in training particularly we're working on a feature right now that will should be out in the next version of eggplant to allow you to create a QuickTime movie of any part of your script as it's walking through this could be great for creating a demonstration movie of your software or variety of other purposes and last of all first a system and administration is kind of a natural if you've got an eggplant machine that can connect to any number of different systems across your network could be very useful for scripting and automating any tasks on those machines so finally I thought that John Comiskey was and I have a slide in there at the end with a quote from Matt Newberg I guess that came out I'm his final presentation there but we thought gee maybe we should put in a quote form at newburgh too so here you go here's some some good advice from Matt Newberg who saw eggplant here macworld in January and was rather enamored with what he saw so now I'd like to invite Godfrey back up to do the wrap up thank you very much so in summary what have we got you know testing testing is very important we know it's very important to all of you this is just you know the beginnings of Mac os10 automated testing the redstone product is excellent is in use by quite a large number of our developers already applescript gooey Apple steps GUI scripting capability gives you a certain level of functional testing and it's limited in scope we understand that we know we need to do more there but i'm sure that lots of people can create point tests for functional testing using it and logging out put logging errors and all kinds of other things eggplant user centric scriptable cross-platform broad scope of capabilities we've only shown a little bit of it here they have and they have lots more things in store for it we're to learn more the Apple script GUI scripting is the UI element inspector these are the URLs to get to these components GUI browser and of course just go to the developer site and search for accessibility you'll get reference to a lot of documentation eggplant by redstone software's this is their company address and contact information they have eggplant CDs available here at WWDC that we're giving them out of their booth during the course of the of the conference downstairs if anybody has not received one please meet meet with the the gentleman from redstone they'll be here after the session and out in the halls and they'll be able to get some more for you and also the access to our ad C test labs if you have interests in more automated testing I really in requests that you come contact me my email address is right there we're looking for your input what sets your needs what would you like to see we're looking for where we can improve in this direction and of course excuse me of course red stone software and Jim Armstrong will be up in our Q&A panel tease their sales engineer and can answer a lot of your questions about access and finding information about their products and we're very close to the end of the show but we thought we'd give you a couple of things to go to both at the same time this afternoon unfortunately but number one how to write a modern carbon app take advantage of the latest techniques and concepts from echo as ten carbon applications and a feedback forum for the Apple Developer connection and give us feedback about the conference about all the services we provide through the year you