WWDC2004 Session 424

Transcript

Kind: captions Language: en [Applause] one morning I'm guy Fullerton manager the high-level toolbox ex Elise node hello my name is Mike anchor and I'm an engineer in the cocoa group and just to make things more confusing we're going to change sides now so hopefully everybody got a chance to see the voice-over session yesterday and hopefully you got a really good feel about how important accessibility is to Mac os10 and how powerful it can be so in this session we're going to build on yesterday's session and describe some of the implementation details you need to go through with your applications to make them accessible we're going to give you a brief architectural overview of how accessibility works we're going to talk about how to accessorize two types of widgets we're going to tell you how to add stuff to a standard widget augment it with a few extra things you might want to augment it with and then we're going to tell you how to accessorize a custom widget that derives from one of our standard ones and finally we're going to have a long section on troubleshooting and giving you tips and tricks for accessorizing your application so I think Mike's going to start out with the conceptual overview okay so before we get into the details on what you need to do to make your applications work with accessibility I'm going to give you a little bit of background on the accessibility API so when we talk about the accessibility API s there are really three things to consider first there's the accessibility api's themselves and in this slide they're shown with the green box and these are the AP is that we introduced back in Jaguar and that assistive apps use to examine the user interface and interact with other applications on the system the second part of the story are the assistive apps themselves and those of you who went to session for 24 yesterday got to see a prime example of an assistive app which is voiceover and the third part of the story are the cocoa apps from the carbon app which the assistive apps you know examine and that's what we're going to be focusing on today specifically what you need to do to make sure that your application works properly with the accessibility API now one thing to bear in mind even though we talk a lot about voiceover as being a prime example as an assistive app nothing in this presentation is specific to voiceover the things you do here are going to enable your application to work with all kinds of different assistive apps that have been developed for the macintosh OS now this slide is an abstract representation of an assistive app trying to examine the user interface of either a Coco a4a carbon app and the question marks show one of the key design problems that we had to make a decision on when developing these api how are the applications going to represent their user interface to the assistive app now an obvious choice would be to use the natural way each application at implemented objects so in a Coco app that's NS windows and its control and for a carbon offsets window rest and control rep but the problem with doing it this way is it puts an extra burden on the assistive apps you have to do twice as much code you have to have one set of code that knows the cocoa objects and then you have to have another set of code that understands the carbon objects and the situation only gets worse if we introduce other frameworks into the mix so in this slide we outline our solution and the basic way we represent the user interface is through something called a UI element everything's a UI element windows or UI elements buttons or UI elements even the application itself is the UI element so both cocoa laps and carbon apps are going to return to the accessibility api's their UI represented as a bunch of UI elements so to summarize a UI element is an abstract object we use to represent a part of the user interface and the applications represent themselves as a whole hierarchy of UI elements assistive apps will look at a UI element and they can in it which has attributes and the sisters apps can use these attributes to get information about the UI element for example its title or its value or could use the children attribute in order to start traversing the hierarchy of UI elements and UI elements also support actions and these are what assistive asked use to actually drive applications for instance to press a button or to increment a slider so to sort of bring these concepts alive guy is going to demonstrate one of our tools called accessibility inspector and do a live demonstration of inspecting one text edit yeah so if I could have the second computer please perfect thank you all right so what I have running here is text edit and I've got a document open that has just a little bit of text in it and I'm going to run the accessibility inspector application which is part of the developers tools install and when i run the accessibility inspector you get this floating window that describes whatever the mouse is over now by default the font size is really small and I can see it's fine but something tells me you won't be able to read it unless I make it bigger so that should be ok so now as I move my mouse around the screen and point at different things the accessibility inspector updates with information about each of the elements that I roll over so if i stop on this text area for example you can see the hierarchy that contains that text area it's inside the application textedit inside the window document inside a scroll area inside a text area and then it's a link as you can see in the attributes section there's a number of attributes that supports it's gotta roll ax link has a role description which looks like it's a bug to tell you the truth it's got a sub role you've got a parent various other pieces of information about it and of course the attributes that an element supports varies based on the element so if i go and hover over a push button you can see a different set of attributes and even some actions in this case one cool thing that the accessibility inspector can do is lock on to a UI element so if i press f5 you see the text in the inspector go red and you see another floating window appear and now when i move them out the text in the inspector no longer updates so I'm locked on that push button which lets me do a couple really cool things the first thing is I can potentially manipulate other parts of the UI with my mouth without disrupting what the inspectors showing but the second cool thing I can do is potentially go up and down the containment hierarchy in the second window that shows up there's a go to menu and I can choose other things from within that go to menu I can go to the buttons parent which is the Preferences window as a whole now this is really important if you need to get to an element in your application that doesn't really have much of a visible UI widget that you can point at but you can point to some child of that so if you can point to a child of it you can continually traverse up the parent hierarchy until you get to the element you want not only can you go up but you can also go down so now that I'm looking at the Preferences window element I bring up the go to menu and I can see that there's a list of children so I can go to any of these groups and from within those groups going to the children of the group and so I can find all kinds of different things out about elements so let's go ahead and get back to the button real quick I want to show you another cool thing you can do so also in this secondary window there's a pop up with all of the bring this up a little bit all of the attributes that the button support so you can choose one and it changes the little value in the text area here to represent the value for that attribute now this doesn't show it really well and see if I can find one here if I lock on to a text field indeed I am in a text field and I choose the value actually when I pop up this menu you can see that there's a w next to the ax value attribute that means the value is writable so now when I choose value from the pop-up you can see that the text is now writable inside the secondary window and I can change that to a tee press set value and you see that actually updates in the interface another cool thing you can do with the secondary window is closed the highlight checkbox this is an easy way that you can test to make sure the size and position you're returning for an element is correct so I'm I've locked on to this text field and when I click highlight you get that pink window that shows what your highlighting are the bounds of the thing you're highlighting and in fact highlight sort of stays on once you've clicked it you can click it again to turn it off but the cool part about that is you can hit f5 again to unlock go point at some other object and it now highlights that other object so it's really good for testing those kinds of problems bounds related problems let's go back to the button again because there's one more thing I want to show you don't need to highlight anymore all right so i hovered over the button and it's revert to default setting and you see that I'd already changed the window with 275 so I've already varied from the default another thing in the secondary window is a set of actions that an element supports buttons support a press action because you can press button and the cool thing about the palette is I can perform that action and when I perform that action you see the revert button actually flash and then the window with gets updated back to 75 so accessibility inspector is a really powerful tool probably probably the most common tool you'll use when you're rising your application to test that you're doing things correctly so I'd like to go back to slides now ok so that's a brief overview of one of the tools so now it's time to dive down into the nitty-gritty of how you can sort of implement this in your own application but some of you are probably wondering how hard is it going to be so we have this fancy little chart here that can illustrate that for you okay colors and jokes aside the basic principle is the more standard stuff you use the far easier it is to become accessible the more custom stuff you use the more work you have to spend making it accessible and that's just you know regardless of whether your cocoa app or a carbon app for example if you're a modern Coco app or a modern carbon of that app that's based on HIV you and compositing it's really really easy to accessorize that application but most apps can't use just the standard system widgets right most apps have a few custom things and so those apps you know they'll need to do a little bit more work to accessorize those custom widgets and that's not too tough either the real problem case is when you're at the far other end of the spectrum and you're a completely custom app doing all kinds of old stuff let's say you're on a really old class library and you ported your application to 10 and made it run but it's not using the very modern technologies of either framework well that's going to be really challenging to accessorize because you're pretty much going to need to add accessibility information to everything in your interface except for the few standard widgets you use so the take home is really maybe this is the time not just to accessorize but to adopt other Apple technology like a chive you in your application sure it's going to take a little bit of effort to adopt a chive you but that effort is going to pay off in two ways you're going to get a bunch of other benefits from a chive you and of course it's going to make your accessibility implementation a lot easier as well you should seriously consider that when you're accessorizing your app okay so I showed you some attributes in the inspector but let's talk a little bit more about some of them so fundamentally attributes provide information to an accessible application about the elements in your app and an assistive application will want to make four main requests of your app with respect to its attributes given an element and assistive app needs to know what attributes is that element support it might want to get the values of certain attributes of course it'll want to see a fine attributes edible and if it's edible it's going to want to set the attribute now all of the attributes that we sell the standard attributes can be seen in the H I services header I put on the slide but when you open that header don't be scared by the sheer quantity of attributes in there there's a lot that dozens and dozens and dozens don't make don't let that make you think it's really hard to accessorize most elements only support a small handful of attributes as I was showing in the inspector you know push buttons have about ten various other things might have about fifteen so it's really not that tough so don't don't feel daunted when you open up the header so I want to talk about some of the very most common attributes that you're going to have to use and you're going to come across the most fundamental attribute is the role every element must have a role a role describes an element's basic purpose and in fact a role is what an assistive application uses to determine how it should present this interface to the user in some other way voiceover for example when it sees a button it knows the buttons that certain properties associated with it and it knows it can communicate with the user in a certain very specific way about the button however an assistive application can't use that role as sort of a spoken description to the user about what the the element is because mainly the role strings are prefixed with ax and they're just there to be sort of switched on at runtime by the application so if an assistive application wants to present a human readable representation of that rolls purpose it uses the role description attribute which is a localized string that can be spoken to the user or shown to the user in some fashion now another really common attribute is the title attribute the it's generally obvious when to use this you know the don't apply button up there its title is don't apply the check box is in the other window the titles right next to those are pretty easy but the thing to realize is if the title attribute is only for visible text don't try to put a title in place for a push button that just has a picture on it but doesn't make sense the assistive applications assumed that if there is a title attribute there is visible text associated with that element if you have an icon button there are other ways to provide information to an assistive app and we'll cover that a little bit later another really common attribute is the value attribute again this is self-explanatory for the most part but there's a really cool aspect and that is that unlike a lot of other attributes the value attribute can be of just about any type of course the type depends on the role of the element you're talking about the check boxes on the furthest back window there on the screen they are going to have a value that's a number and it's going to be either 0 or 1 the based on whether the checkbox is checked by contrast the text field on the terminal info window its value is the text inside the field and that's going to be a CF string ref generally possibly some slightly different representation but again it's two completely different data types that that depend on the role of the element you're talking about now as mike said elements are hierarchically organized within an accessible application you've got a application that has as children probably a menu bar several windows and each of those windows are going to have child elements their various views and buttons and things like that within the window so every element has a parent so that's not technically true there is one element that doesn't have a parent but that's that's something that we write so you'll never need to worry about than here in your applications for your purposes just remember that every element has a parent the children attribute however are not necessarily supported by every element children attributes on the children after be only needs to be supported for elements that obviously contain other elements a window is going to have children is going to be the views within the window but a push button for example doesn't have children it doesn't make sense because the text in the button is already represented as its title so mike is now going to take you through the easiest case of accessorizing an application okay so this guy said if you stick to the standard widgets you're going to get most of your accessibility for free and in fact all the attributes guy was just talking about are things that the framework you know either cocoa or carbon are going to provide for you and there's lots of other attributes like the position and size which you didn't go over that the frameworks are going to provide for you but there are some things that we can't figure out and the example guy gave of a graphical button the button that just has a picture in it is is one case where we cannot figure out what that button mean and that's what we're going to discuss in the next few slides a handful of attributes that you're going to need to implement because there's it's not enough information for us to provide them at the framework level so the first is for lack of a better name I'm going to call these instance attributes and that's because you have to go in by hand and for each instance of these kind of widgets you're going to have to provide this sort of information for us and the first one I want to talk about is the description now the description is a string that describes what a widget does but it doesn't describe what the widget is now let me reiterate that it describes what the widget does what the widget is is covered by the role and so for instance a button is known to be a button because of the role attribute in F this button happens to cause the text to be left aligned that's the information we want to be provided by the description attribute so if we take a look at the DVD controller we can see that a lot of the buttons there don't have any text and so in this case the assistive app is going to rely on you to provide a description of what they do otherwise all it's going to know is that this is a button and then the users you know can will be able to press it but they'll have no way to figure out in advance of pressing it what the heck is going to do now even buttons that have titles are sometimes hard to figure out take a look at the calculator all the buttons there have a title but a good percentage of those titles aren't particularly informative so for instance you know the MC button or the period button you know if you don't provide a description it's going to be pretty mysterious as to what these buttons do so in case all this is a little bit confusing all these attributes let's just go over an example let's take the MC button in the calculator that button is going to have a role the role is going to be a X button and that's provided by the framework the button is also going to have a role description and that's going to be button and this is a human reasonable string it's going to be localized and again we provide that in the framework it's going to have a title and the title is going to be a two character string the letters MC and once again that's provided by the framework and then there'll be some other attributes like its position in size that are also provided by the framework but that's it and we're going to rely on you guys to go in and give that but in the description which should be a string something like memory clear the next instance attribute I'm going to talk about is the title UI element attribute now recall that we were just we mentioned before that the title attribute is a string that a widget actually displays and take a look at the the lock button the string next to it isn't part of it so the lock button will not have a title but yes there's this piece of static text next to it the decided user can figure out serves as the title for that lock in order to make this this relationship available for assistive applications you need to implement the title UI element attribute and its value will be a reference to another element that serves as a title in this case the piece of static text next to it another common case our text fields on the screen I have a text field that contains the text Apple and this text field has no title at first you might think oh its title should be Apple when you think about controls the value is the part of the control that changes as you manipulate it and when it comes to a text field the part that changes as you use it is the string inside so that's the value not the title now just string the string next to it is a separate widget and that serves as its title but the only way in assistive apps going to know that is if you once again go in and set the title UI element attribute for that text field to point to that piece of static text there's a related attribute called the serves as title for UI elements attribute and this is the back pointer and this could be useful if an assistive app allows the user to say mouse over a piece of text and it wants to decide well what what purpose does this text serve in the user interface if it has they serve this title for UI elements attribute then I'll know that this piece of text is the title for this list of UI elements and this value the value for this attribute is an array because a single piece of text often serves as a title for multiple widgets another very important instant ass instance attribute is the linked UI elements attribute the common model and user interfaces is to allow the user to make a selection from a list or a table or a radio group and then another part of the user interface reconfigures itself to reflect that selection now just because you make a selection in one spot and other widgets changed there's no way for us the frameworks to sort of detect this relationship and that's why we rely on you to give the list or the table a link UI elements attribute that shows us that the selection in this object affects these other widgets now the link UI elements attribute is an array because often this collection affects the whole bunch of other UI element and it might look like in this slide that that's the case here but it really isn't in this case the array would contain just one element the tab group now if the tab group weren't there then it would contain the whole host of elements you see on the screen and I think in most cases you'll find that the the linkage will be to a single sort of grouping UI element in the last instance attribute I'm going to talk about is specific to sliders another common practice is to create a slider and then create separate either pieces of text or graphics that annotate the various tick marks and once again these are not part of the slider so there's no way for the frameworks to detect this relationship so we rely on you to provide a label UI elements attribute which is an array of the elements that annotate the tick marks and that's only part of the story because the other thing you want to know is well what what value does each annotation correspond to well in order to discover that you need to give each of those elements a label value attribute which contains the corresponding slider value setting alright so now we've talked about some of these instance attributes and the next question is well how do you go about implementing them and before tiger the answer was you had to had to actually subclass the widget in order to add these attribute and that's that's pretty cumbersome and now that we're trying to encourage developers to add all these attributes it's not a really nice story to say well you have to you have to create a subclass for every whit in your in your user interface just in order to provide description so in Tiger both cocoa and carbon have introduced ati's that allow you to take a particular instance of a widget and associate a new attribute with it and that's what we're going to talk about in the next few slides first I'm going to discuss how it's done in Cocoa guys going to cover how it's done in carbon and then we're going to give you a sneak preview of how this can be done in ib interface builder without writing any code but that doesn't quite work yet in your tiger seed so that's sort of a future direction but the slides were going to present next on how to do it in code work today with your tiger seed so I'm going to start by just giving you the API in Cocoa that performs this it's called accessibility set override value and you passed it the attribute and the value you want it to have and that's pretty simple well it's not quite that simple the first thing is even though this is a method on nsobject you can't use it on any old NSG object you can only use it on objects that implement the accessibility protocol and the second the second point goes back to it another thing I alluded to earlier one of the big advantages of the whole scheme of UI elements is that we squish some extraneous detail out of the out of the natural representation and we don't expose it to accessibility so for instance in a cocoa app there's a button on the screen actually consists of a button object and inside is another object called a button cell and the real meat of the button is implemented in the button cell and that's all this exposed to accessibility so if you set the attribute on the outer button not the button cell it's going to have no useful effect because the button set of the button is not exposed to accessibility only the button cell and if that sounds a little confusing I would like to refer you to the app kit release notes and there's a long section that's written about how you know these sorts of God shoes and another thing that might cause some confusion is the fact that there's a couple of these methods that both have the word set and attribute in their in their name now there's an entirely different concept in accessibility and that is allowing assistive apps are allowed to set the values of certain attributes so for instance that assistive apps might want to move a window and it does this by setting the value of the position attribute but that's an entirely different thing than giving the window a position attribute the window already has a position attribute and the giving it of it the the method I'm describing here allows you to add attributes to thing and it's something you do whereas the the method accessibility set value for attributes that's where the assistive apps come in and they set the attribute values for the attributes where you allow it so here's a bit of example code on how you might accomplish this in a Coco app we're going to give a description to a back arrow button that otherwise has no text associated with it the awake from nib method is called when the files owner on the files owner when this widget is loaded so that's a natural place to try to do this and notice the first thing we do is we extract the proper thing we express the button fell out of the button and then we make our call accessibility set override value we pass it a string back in the name of the attribute we want to set and that's all there is to it and now guys going to explain how this works in carbon so before I dive into this new AP we added for Tiger I want to cover one other detail and that's the notion of what is exactly accessible in carbon as you can see this api's first parameter is an H I object graph H object refs are the key to accessibility on carbon nehi object ref and controls windows menu there are all H I objects each of those constructs is inherently accessible in fact controls windows and menus already have a bunch of built-in accessibility infrastructure in them that's how we're going to provide all the default default attributes so you'll see some of our accessibility API take H object and essentially what you can do is you can take any window wrap or control ref men us cast it into an H I object and just call these api so the equivalent API to what Mike just showed is called H I object set auxiliary accessibility attribute well I actually got that out first thing you need to pass in is the H I object and on the identifiers that represents the element you wish to add the attribute to then you give it a name and you give it the data you want to associate so again same sort of example there's a back button in your interface but it just has an icon on it no text you want to give it a description here's how you do it let's say you happen to create your window from a nib you can call a chive you find by ID to look up your back button now you've got a reference to your back button then you call this new API pass the back button and zero see how I have casted the back button which is normally an HIV ref into an object ref pass that back button and an identifier of 0 which says hey Owen will associate some data with the element that represents the whole button you give it the attribute you want to add and you give it the data you want to add which is the string back and once you've done that the accessibility infrastructure and carbon will take care of the rest for you automatically it's really pretty cool so now I'd like to bring up Aaron Haney to give you a brief demo of interface builder and how it can make your lives a lot easier or we'll be able to make your lives a lot easier thanks guys ok so this is very similar to the demo I did yesterday during the voice-over session but i'm going to show a few additional things what we have here in xcode is a small hello world style app that just has a couple of text fields in a button and right now it doesn't have anything other than the default accessibility information in all of the UI elements as guy in like say this is you know a lot of the standard widgets will give you a lot of stuff for free but it doesn't quite give you everything so let me just show you real quick what happens if you look at this with an accessibility application interface is an active press control option and six to learn more then we'll have application window accessibility demo on to in a text field now you can see it's just saying at a text field and it texts few and down here okay that's not very descriptive it tells you what kind of control it is but it doesn't say anything else about it let me turn this off before it gets too chatty so to fix this we can open the nib and you'll notice in the info window there's an extra item accessibility so let's select these to edit text field now this is a case where we don't want to enter a title for this you ilm in itself what we want to do is link it to another UI element and have it used the title of the one that it's linked to so what we can do is just ctrl click and drag and create a connection and select title UI elements and connect that up now let's do the same thing for the last name field and while I'm here I'm going to take a look at this button which is an image button a fairly common case as I mentioned a title is for a case where there's actual text on the control and in this case there is no actual text so a title is inappropriate what we want to do is add a description so we'll go to the description field here and type in some text now the fact that it's a button comes out of the role description attribute so you don't want to put button in the field because then it would read ok button button so we just want to put what you know just the description of the control itself not what kind of control it is so let's try this again spoke on interface with an active press control option x6 to interface builder applications window accessibility demo on to first name and a text field and there it it last name in a text field okay okay so now it's just a little bit of work in interface builder we've added accessibility to this small hello world application now a couple of other things that I want to show you is a an accessibility inspector guy already mentioned that in the locked view let's lock on one of these text fields and we'll highlight it just so you know which one I'm locked on okay the go to menu can be used to transverse the parent-child hierarchy oops I shouldn't move that while it was locked I can hit refresh refresh where's the Refresh right below the pop-up okay you can also go to any linked elements such as the title i own and now you can see it jump over to the first name field and from now on whenever I update the text in that static text field it'll automatically get picked up as far as accessibility concerns the title of these edit text fields will be the same as the title of these static text fields next month so that's it now we want to emphasize that their support that's on the CD DVD is incomplete this is just to show you the direction that we're moving in we advise you to not use it as it is right now we hope to get that addressed as soon as possible anyway this is where we're headed we want it to be as easy and convenient as possible so that you can add accessibility information from all sorts of places including interface builder and that's the demo thanks guy thanks Aaron okay so now we've talked about the easy case what you have to do with standard widgets and we anticipate that's going to be the vast majority of the work that's needed by applications to support accessibility but now we're going to bump things up a notch and we're going to talk about what you need to do for a custom views and in the example we're going to be working through we're going to be talking about if you created your own button class and instead of starting from NS button or a button ref you start from just a plain view and guy and I are each going to show how you do this in carbon and cocoa and how you to give a title to this widget and a title attribute and how you'd implement a press action now the first some of you might be thinking okay we just showed you how to add attributes can't I just add a title that way and the answer is yes that will work just fine and if you only have a couple instances of your custom button in your app that might be good enough but maybe you're providing this class for other people to use and wouldn't it be nice if the title attribute came along for free instead of making everyone implemented or another thing to think about is maybe it's some other attribute like the value in your widgets a slider the methods we just showed you take a name of an attribute and a constant value and it's not going to really work so well for the value of a slider I mean I guess you could every time the slider changes value reset that attribute but that's not really very elegant so we're going to show you how to get some code to run in order to implement attributes and and then we're also going to show you how to do this for action now in cocoa there's a protocol or a suite of methods that you implement on an object to support accessibility and there's four of those methods relating to attributes and they correspond to the four things that assistive apps can do with attributes one of them is to get a list of the attributes supported by a UI element the second is to get the value of a particular attribute the third is to test if a particular attributes edible and the fourth is to finally set that value and this code on this screen shows how you implement the first of those methods accessibility attribute name now here I've chosen to cache the attribute names in a local static variable just so i don't have to compute them over and over again and the first thing we do is check if the cache is full because if it is we don't have to do anything else that will get returned but if we haven't computed the list of attribute names yet the first thing we do is go to the super class and get the standard list of attributes then we create a new array by adding the attribute we want to add the title attribute to the end of the list and finally we tuck that away in the cache and that's all we need to do all right this is the code that support returning the value for an attribute the method name is accessibility attribute value and the first thing we do is check if the attribute being queried is the attribute of interest in this case the title attribute and if it is we return our notion of the title otherwise we let the superclass handle it and the last met that i'm going to show you is the method that handles returning whether or not an attribute can be set and once again we test is this the title attribute and if so we return no because we don't allow users to change the title of our button and if it's not we let the superclass handle it now those of you who have been paying attention might realize I said there's four methods well the fourth method involves actually setting the attribute and since we don't allow the title ever to be set the code for this method would always just defer to the superclass so there's really no point in implementing it so from carbon a very similar protocol to what Mike described in carbon accessibilities is achieved through I think its nine different carbon events you can install those however you would normally install carbon events in this example here I happen to be doing at an H object subclass registration time you don't have to do it that way if you use create custom control or create custom window or something like that after you create the control of the window you can call install event handler and install your event handlers afterwards it's all going to work just the same so three of the attributes sorry three of the carbon event that we have up here are the get all attribute names is named attributes edible and get named attribute carbon events again there's also an event that's used for setting there's some events that we'll talk about later use for actions there's also some events to use for hit testing we're not really going to cover that today but there's lots and lots of documentation in carbon intensity H on exactly how you need to handle these events what parameters you need to look for parameters you take out so let's dive into the example ok so the same example Mike was talking about you've got a custom view that has a title and you want to provide that attribute because it's not being vided for you automatically by the by the view system because it doesn't know your customs you have the title so here's what your handler forget all attribute names might look like the carbon event comes in you need to call next event handler to let the inherited a chai view implementation pre-populated with a bunch of the standard attributes like the size and position and stuff like that all the stuff you want the system to provide for you then you extract the mutable array of attribute names from the event and you add your attribute to that array any return and that lets the assistant accessibility system know that hey there's this additional attribute that's been added in the next event I want to talk about is the is named attributes edible carbon iment when this carbon event comes in sets the name out of the event and sorry push the attribute name out of the event and see if it's the attribute support if so in this case obviously the title is not settable so we just stuff an event parameter any of the events indicating that no it's not settle and then you return let me jump back here is something I left out of this code example obviously if it's not one of the attributes you support call next event handler or return event not handle there and that'll let the inherited implementation provide all the right information for it so when an assistive application wants to query a value again very simple extract the attribute name out of the carbon event if it's the one you support stuff an event parameter in the event with the value in this case we're stuffing the title in presumably the title is a CF string rep and the carbon event manager is going to do all the right retain release semantics for CF types and I'm going to talk a little bit about abstraction ok so we've shown how to add an attribute to your custom view next we're going to get into how to add an action but first a little background on what actions are first actions are very simple things they're about approximately what you could accomplish with a mouse click and there are generic things like ticker press now often developers think oh I've got a print button I need a print action or I have a cancel button i need a cancel action and that's not the case if the button it should support the press action the fact that this button prints can be discovered by looking at its title or description so don't fall into the trap of thinking you need a lot of action in fact there's less than 10 actions defined and I don't anticipate that list growing very much the assistance the accessibility API support three basic operations with actions first you can ask a UI element for a list of all the actions and support and in most cases it's the empty list or just one thing then you can ask just to get a use human readable string for a particular action that fits the description and finally you can try to perform that action now Coco and its accessibility protocol has three methods that correspond to those three operations and this slide shows an implementation of the first returning a list of action name and it's very similar to what you do with an attribute first you call the superclass to get a list of inherited action and then we slap our action on the end of the list and return the augmented list this is the method it handles returning the description for a particular action as with attributes the first thing we do is check is this is this action the one of interest because if it's not we'll let the super class handle it but if it is we're going to call the NS accessibility action description function and this is something new and Tiger before before tiger you were if you will do to return an action description you sort of had to look around and say well how did we describe the press section and you might use accessibility inspector and discover you know the buttons had a press action and and then you also had to maybe guess how might they localize that so we're sort of taken the mystery out of that process and giving you a function that's going to return what our standard widgets do for action description and I refer you to the applicant release notes where the functions described in detail along with some related ones for role description and the final method in this action suite is for actually performing the action and first we again we check is this the press action that we care about if so we do the appropriate thing otherwise the superclass handles it and now guys going to show what you do in carbon so three of those nine carbon events are pretty much identical to the application methods for performing actions the first one is get all action names it's a lot like get all attribute name so the fact you handle it the same way you call the next event handler to let the inherited implementation add any name you extract the mutable array of names and you add your new action to it get named action description is also simple but there's one kind of weird catch here there is a mutable string ref in this event you need to extract that string rep and replace it with your text replace the text in the string with your text and return so that's how you provide an action description I've left out the code here just for simplicity of where we are looking to see if the action that we support and if not of course it's going to call next event handler now this is the only tricky part of the accessibility carbon event and that has to do with performing named action but before I kind of dive into this let me give you a little bit of background for why this is complex when an assistive application is telling your app to do a press operation that assistive application is blocking waiting from a response from our messaging protocol to say whether or not the app is alive right your app might be crashed or tongue or in gdb or something and so the assistive application needs to know if a timeout has elapsed and you know it's got to forego the attempt to press however that need often contradicts what actually happens in an application when you perform a press type operation on a button a lot of buttons will bring up some kind of modal interface or might cause a menu to be tracked which means that your accessibility event handler is not going to return right away right it's not going to return until the users done dealing with the dialogue which could potentially take minutes depending on what the dialogues for which means that assistive application who just sent the press requests over is going to get a timeout and it's going to think the press failed and it's going to communicate to that that to the user so now the user is going to be really confused because they thought something didn't work but now there's this new interface that's showing up and it's just a big mess so the way we solve that mess is by a parameter in the perform named action carbon event there is a parameter indicating whether or not this event was sent to your element in a cube state now by default and just mainly for compatibility purposes the event gets sent to your element initially in an uncute state we dispatch it directly that's just the way we've been doing it since our initial accessibility implementation so if you see that happen well let's go to the other case let's say it hasn't been queued or sorry it has been queued that's the case you want to handle if it has been queued you can go ahead and carry out your action in this particular situation we've got a button we just call HIV simulate click which will cause the button to flash and out all the right carbon events so your application can respond to it and it's safe to do it now because the events been queued you already know the assistive app has received an appropriate return value saying aai was successful but if it wasn't cute you need to return the event for excessive sorry event defer accessibility event error from your handler and that essentially is a request to please cue this event and send it to me at some time later because I know I'm going to put up some potentially blocking kind of in that's really the only trick to it okay so so far we've been talking about fairly easy cases right you've got an existing view like a push button you just want to add a description to it or then you've got an HIV or a nephew subclass and you need to provide some accessibility information above and beyond but the frameworks offer for you but now the hardest case is you've got some class it's not derived from NSU or a chive you and you have to provide all the accessibility information for it now we're not going to go into tons of detail mainly because we've already shown you how to do this in the other example there's just some things you need to realize the first is that if you don't inherit behavior from our standard views and objects you have to provide all the accessibility information for your objects that means all the attributes you have to tell sighs position everything else you saw in the accessibility inspector you have to provide all that you also need to provide handlers for hit testing and focus testing parts of the accessibility protocol this allows assistive applications to figure out what's under the mouth in fact that's the key for how accessibility inspector works it finds the mouth position and calls an accessibility API to try to find out the element at that Mouse position and so if your custom elements don't support hit testing and focus testing there's no way accessibility browsers going to work with your app another thing you have a responsibility for is sending notifications there's a handful of notifications it's in the HR services headers so you can take a look there for what needs to get sent out but essentially it's for things like my objects about to be destroyed or there was some major state change with my object like a value change well if you are implementing an element from scratch you need to make sure you send out those notifications appropriately and we have full carbon and Coco api's to do that there we go all right so now let's talk about some things you need to think about when you're accessorizing your application said it before we've said it actually number times in this session if you can switch over to the standard controls you'll be much better off because we're going to give you a lot of support for free so again look at your applications interface see if maybe you're using some custom objects that we've provided some modern equivalents for and recent operating systems try to switch over to those it's going to make your applications interface better in a lot of ways obviously it's going to meet the latest aqua guidelines because hey we're doing it and it's going to give you accessibility support for free now a very subtle point is don't invent new stuff don't try to come up with new rules don't think you need to come up with new roles and the main reason is that assistive apps won't know how to deal with your new roles assistive apps are designed around fairly finite set of rules that we've already described you know there's groups there's list and things like that and if you invent a new role you're essentially asking every assistive application out there to rev just to support your new widget so really what you have to do is kind of turn your widget around and think about what your widget is a lot like in terms of the standard widgets the best example I can give for this is the crayon picker in the color picker panel as we were going to accessorizing various parts of the system you know we got to the color picker and we said oh wow cramps what are these things I don't know do we need a new role for that is it a crayon cran role in a cran box I don't know but we took a good hard look at this and said no we don't need any role because really when you click on one cran it selects that cran and it selects all unselect all the other crayons in the box so really what you're talking about is a radio button inside a radio group so that's how we accessorized it every kranz role is a radio button and then the group around all those crayons is a radio group and suddenly assistive applications can just deal with that for free the way they've already been dealing with radio groups okay so you found some part of your application and you used you decided to use one of the standard rules to represent it now here's some of your responsibilities after you do that take a look with accessibility browser it sorry accessibility inspector and find some element in the system that has that same role and make sure you support all the same attributes assistive applications rely on the fact that a given role implies certain attributes for example and actions in fact as well for example an assistive application see the button it knows it's got a press action right it needs to be able to assume that kind of thing so make sure you support the same attributes and actions as the standard roles now if you run into a situation where your element yeah pretty much fits into a standard role but there's enough subtle difference that you kind of want to call it out but you don't want to force all the assistive applications to rev to support you you know you'd like to work for free with voiceover well you can do that through a mechanism called sub rolls several sub roll is just another attribute on an element the best example of this are windows there's a bunch of different types of windows on the system we've got document windows floating windows system windows sheets all these kinds of things sheets was a bad example cuz that actually has a different role but all those other types of windows all use the window role but we found that in some cases assistive applications need to tell the difference between a dialog window and a document window so we do that with a sub role the sub role says oh it's a document oh it's a dialogue and now an assistive application can if it wants look at that several and make a better decision about what to do with your element but at the same time it can keep working just like a window normally could in that app so you don't force anybody to rest another thing we found is people are very tempted to come up with a special role because they look at a elements purpose in sort of in the sense of their overall application for example you might have a toolbar and that toolbar might have a print button on it and you realize that hey printing is very fundamental to my application so I really want this to be a print roll we need to avoid that temptation mainly because the notion is this button print button is already conveyed by something else we've given you a couple of ways that that might happen obviously that button might have a title that says print so the user already knows it print if it doesn't have a title you probably put a description on it that says print so the user already has these ways for determining a buttons purpose so don't be tempted to give it a new role just because it has some unique purpose in your application again just like with attributes you need to use the standard action there's a small handful of them and they support just the most basic mouse and interaction sorry bait yeah it's just the most basic mouth interaction if you're tempted to give an element a new action see if maybe you can't achieve the same thing by giving it an attribute that's edible best example of this is the Select ability of text in an edit field right we could have introduced some complex set of actions so that you can set selection here then send selection there now we don't bother with that that gets really complex instead there's just a rideable attribute that is the selected text range so the user can set that that way now a lot of applications these days are inherently mouse based and that means they're not going to work real well with accessibility right it to be to be a true accessible application you need to be able to unplug the mouse from your computer and still use all parts of your application so an interface this entirely drag-and-drop based really isn't going to cut it so as you're looking at your application in your examining its functionality see if there's anything that's drag-and-drop base that doesn't already have a keyboard driven alternative if it does have a keyboard driven alternative you don't really have any extra work to do but if it doesn't you need to consider providing an alternative some good examples of how we did that for windows normally if you drag the window you click with the mouse you move the mouse to a different position and let go well that's not going to work at it on the mouth plugged in so to achieve that through the accessibility API all windows have a writable position attribute another good example but the finder uses when you manipulate files in the finder a lot of people click with the mouse drag a file from one volume to another in order to move it or copy it but find your ad is something a few releases ago that allows you to copy using the normal command key equivalents in the menubar copy a file navigate to some other place in the hierarchy and paste it and that's a great way to offer an alternative interface to doing what was otherwise a completely Mouse based interaction and the most cool part about this is you're not just helping out users of voiceover you're helping out everybody I mean I use the copy paste functionality and find out all the time so really what you're doing is you're just expanding your feature set in a way that's going to help everyone so as we implement our interfaces a lot of us use grouping views and in fact some of these grouping views that we put in our windows there they're forced on us by well for me you know I put various views in your windows and Mike puts Coco views in your windows and so the real implementation hierarchy might look something like this I've got a window inside the window there's a frame view inside that frame view there's a content view of course there's other views for implementation details and there's a button and in cocoa there's even a button cell inside that well you don't want all that complexity to be seen by a user of an assistive application like voiceover mainly because a sighted user doesn't see that complexity either when they glance at the interface so for accessibility purposes you want your interface to look like this there's a window and there's a button in it and the way you achieve this the notion of ignored elements there's a carbon API called H I objects that accessibility ignored impasse nehi object which means any control you really only want to use it on controls and you can say hey ignore this control you know I just use it for grouping purposes and I think in cocoa the method name is set except for their accessibility is ignored accessibility is ignored you can just call that with true to mark an object as ignored now it's similar but suddenly different point is the concept of suppressing an element some applications that we've played with bring up parts of their interface just at a certain time a good example might be a download status dialogue in a nap when the files done downloading and the app wants to get rid of that window some apps just move it off screen you know they make sure they move it to some crazy coordinate 10,000 20,000 so they know it's not visible to the user well the problem with that technique is accessibility still knows about that window it still looks logically visible from our API perspective so both frameworks report that to the user and so that's really odd that a user of voiceover can get this window that's off in la-la land you want to avoid those situations entirely and it doesn't just happen with windows some applications move their controls away to some crazy coordinate just so that they're not visible to the user so if you are one of these applications or you see your code doing this make sure to logically hide the object logically hide your windows logically hide your controls so so some one thing sometimes people new to accessibility get these two concepts mixed up ignored and hidden so when something's ignored even though it's not there from an accessibility point of view its children sort of pass through if you hide a view it's gone along with all its descendants so those are really different concepts and usually if something's ignored it's always ignored and whereas as your UI is your UIs dynamics whether something's hidden or not might change over time and so try not to get those two things confused so you'll also find other cases where you need to provide accessibility information for an element that doesn't have an object to back it best example this is the scrollbar if you use the accessibility inspector and you look at a scrollbar sure there's a scrollbar element there but there's also some button elements in the scroll bar to represent the up down and left right buttons in fact there's buttons to represent the page up and page down regions and then there's an indicator element to represent the thumb well there's no actual view in our view system to represent those sub components it's all part of the the overarching scroll bar view so if you have a similar sort of situation in your application the way to deal with that is by creating objects sort of dummy objects to represent them as accessibility objects in carbon you create an H object subclass potentially to do that when you do this you're fundamentally creating a new accessible object from scratch so again you are responsible for providing all the attributes and actions for that object but there's some shortcuts you can use to sort of get around that requirement generally you want your the instance this sort of done the object to provide things like the parent and the role but oftentimes you can either by routing carbon events or calling methods on the parent you can delegate some of the responsibility to the parent code to at least keep your code kind of self-contained another cool thing about this is since they're dummy objects you don't really need to create them until they're needed right don't instantiate them at view construction time because you don't necessarily know that accessibility is on and even if accessibilities on maybe a user's never going to go to that scroll bar or whatever so create the instance is only when they're needed right maybe the first time you're the parent object is hit tested or the parent object is asked for its children attribute or something like that on the other important point is make sure the parent does the hit testing and focus testing otherwise excessive sorry assistive applications won't be able to find those those dummy objects now mike is going to go into some details on troubleshooting some common problems with the excessive accessories ation of your application okay so yeah I have a few slides and I'm going to talk about some common problems that you might run into accessorizing your app so the first one pretty simple make sure accessibility is on you need to go to system press and there's a checkbox that says enable access for assistive devices and you need to make sure that's on now this is not the same as turning on spoken interface or voiceover which is an assistive app that requires the access for assistive devices to be on so it's up to you whether or not you want to enable voice over the next point is really specific to Coco apps and this is a pretty good habit to run into if you're if you're implementing accessibility on your cocoa lap and that is to run with NS accessibility dbol debug log level of one and I usually do this with a command-line argument although you could write it to your system press and always run that way and I'll neither confirm nor deny what values other than one will do but if you specify one what's going to happen is where's the cocoa app run it's going to see you informative error messages to the console concerning accessibility now perhaps what's more important than the error messages themselves is the fact that you're going to get a clue that something's going wrong one thing that both carbon and cocoa do is in when handling requests from assistive applications is we eat any errors or any exceptions that are raised during the process because we don't want an assistive app to be able to go in and crash some other application and because we just eat these exceptions you may have no clue that something's going wrong and your apps your ass maze just sort of mysteriously seem to not work with accessibility so if you turn on this option if you're you know if your apps not working and you turn ons option it's likely you're going to see all kinds of spewing informing you of what might be going wrong now if we return to the example guy gave of a scrollbar and has it with it five children the up and down arrows and the page up in the page down region and the and the thumb and say you've gone through the trouble of implementing something like that and then you go over and use the accessibility inspector and none of the little pieces of your scrollbar show up and that's pretty pretty frustrating so I'm going to give you some tips on how i would go about sort of debugging that first off it could be that there's something going wrong some exceptions being thrown and you just don't realize it so if you use the NS accessibility debug log level of one you know look at that console output see if some exceptions being thrown and that's and that's the root of your problem alright the next thing that might be going wrong is in accessibility inspector you're using the mouse to sort of mouse over the widgets you're interested in looking at and so that involves hit testing now in in the frameworks we can do hit testing we can figure out if this points over a window or what even what view it's over but the fact that there's a thumb in your scroll bar we have no idea that that's going on and how accessibility works is we're going to tell your scroll your scroll bar view look we think the mouse is in you and now this is your chance to refine that search and tell us about some children and if you don't do that your children are not going to show up so that's a common mistake so the next thing you might want to look at is if the children aren't showing up is actually go to the parent object maybe you can get your mouse over that in accessibility inspector and look at the array of children and if you're you know it's the thumb and the up and the down arrow aren't even in that array well then you've got even more fundamental problem perhaps there's some exception being thrown that's keeping your children attributes from returning the proper value and if none of that is the problem and one other thing you might want to look at is the is the coordinates you're using for your hit testing remember that the accessibility API is all deal in terms of screen coordinates now the framework end we each have a slight twist on it in carbon app you're going to get a coordinate in top left relative screen coordinates and in cocoa it's going to be bottom left relative but in either case its global coordinate so if you're doing your computations in sort of view relative coordinates that could easily be what's going wrong and then the final point of something guy already touched on and that's the issue of something being ignored if if the element you're returning is ignored then it's not going to work properly and as an example in Co Co NS views have no visible you know a plain generic view has nothing to show for itself on the screen no way for the user to interact with it so by default views are ignored and if your if your object descends from NFU you need to go in there and have the accessibility is ignore method you need to override it to return no all right here's a couple more problems that sometimes arise it's important that your that the hierarchy be sort of self consistent and that means that you know if the UI element says that some other UI element is its parent if I go to that parent and I look in the children list I better find that child in the list and if that's not the case then you might get sort of weird behavior you know the the assistive at might work in an inconsistent way because depending on what path it takes through the hierarchy it may discover the child or it may not so that's a that's a common thing to look at and we actually have a tool that makes these checks for you and then the last point is once again something guy mentioned before is sometimes people user interfaces are dynamic and you hide things maybe you move away off screen or maybe you obscure it with some you know some something that draws will opaque white over it well if you do that the underlying elements are still going to be available to assistive apps and assistive app you know they may press a button that no user could normally press and this may exercise code taz your app that hasn't been tested rather dangerous thing to be doing so you want to hide things in a sort of proper way either rip them out of the view hierarchy completely or use the appropriate method from your framework actually hide the view in our last major section is on performance I have a couple slides that discuss you know how should how should doing all this affect the performance of your application in the bottom line is it really shouldn't affect the performance of your application unless accessibility is armed and that's the principle that both carbon and cocoa have used and implementing this stuff that we don't we don't want anyone to pay any price for this unless accessibility is turned on and so you to help you out with that you don't have to really worry about making our calls we're pretty careful if you if you want to set an attribute or you want to post a notification you know go ahead and do it because we're going to bail out really quickly if accessibility is not on and sometimes or even smarter when it comes to posting notification if we were actually going to check is anyone even listening for notifications on this object so there's no one is you know we're going to do nothing now when it comes to your code if you want to optimize it this way you're going to need a way to test if accessibility as an R is on and you might want to use it in fact the only way to do this is to test if the accessibility API is turned on using ax API enables and this is a call from the accessibility API that you can make and you should check it out before you do any expensive operation another thing to note is you shouldn't have to worry about accessibility being turned on while your apps running if you make this check at the beginning of at launch that's fine I know if you get in there and you play around it kind of seems the work that you can turn it on while in apps running and things work but that's not something you can rely on and it's and so again and that's that's sort of a warning the assistive app developers to you know if you accept bility gets turned on users are expected to quit and restart any running application and a final tip is something off that guy mentioned is you know sort of be lazy about creating any data structures associated with accessibility so if we go back to the example of the scroll bar with all the children you know I would create the children when I'm asked for them and and only keep them around long enough to satisfy that request and then throw them out there's no reason to have these things around and then they'll get out of sync if your scroll bar changes state so it's much simpler just to creative as needed and now we have a couple slides talking about the resources available to you and the first one I want to plug is our accessibility dev mailing list and this is a public mailing list that you should subscribe to you can ask all your questions on it if you come up with some with use of some weird widget and you don't know how to accessorize it you know post an email this list is trolled by guy and myself actively and some other engineers and we know we don't promise that we have all the answers accessibility has been a the whole story's been evolving since Jaguar and more often than we like to admit you know people come up with weird you is that we hadn't thought about how is this going to work with accessibility but if you post to this list we can usually steer you in the right direction and finally there's three documents relating to accessibility one of them is for people writing assistive apps and even if you're just accessorizing your app it might be useful to peruse that one and then there's a cocoa specific document in a carbon specific document for accessorizing your application and that will open it up for Q&A