---
title: WWDC2001 Session 707
framework: wwdc
role: article
path: wwdc/wwdc2001-707
---

# WWDC2001 Session 707

## Transcript

Kind: captions Language: en so the Darwin documentation project was launched a few months ago when we launched Darwin a couple of years ago we made a major leap into open source with the documentation project we made another major leap because we recognize that the documentation is as important as the source code and just like the source code it's got to be open there have to be resources and tools out there for you to contribute and you have to see what we at Apple are doing so this presentation will give you the lowdown on everything that's going on it's going to cover the what's changing in documentation so we've learned a lot from open source it's changing how we're doing documentation you may learn a few things about why documentation hasn't always worked the way you wanted in the past and why it's getting better now I'm gonna go over with heterodox how to and the man pages the areas of documentation that the project covers and then we're gonna get into some detail on what documentation do we have related to Darwin what documentation are we working on and where are some great areas for you to contribute if if you happen to have some knowledge one big question we get is well what is the Darwin documentation project cover Darwin documentation covers what Darwin covers and probably most people in this room are aware of what that is but here's a quick summary obviously that's the mock BSD kernel kernel extensions system framework command line environment network services now one of the things that people notice is that our apple writers the people contributing to the darwin documentation project are primarily focusing on drivers and some basic kernel stuff some people in the community want some other things we'll get into some opportunities there for people in the community to contribute and such but I can say right up front that our focus is definitely drivers and and the core kernel information that you need right now so as I mentioned you've probably been the recipient of problems in the traditional documentation process meaning once you get the documentation it's not complete or it's really late it comes out way out to the product why does that happen well here's the traditional model for documentation and it starts with the engineer the engineer has the information that you need and they may write that down they may write comments in the header file they may meet with an it writer and tell them what that information is the writer gets that information they put it together into a nice document they get it reviewed they send it to the editor the editor goes through it for consistency grammar make sure that they're communicating clearly works of them on those things then it goes to the production department the production department goes through every page of the documentation they make sure it's paginating correctly they make the art look good they make sure that we're following the right template it then goes out to the vendors and distribution where it gets printed up into a book it goes out into the bookstores finally it gets to you this whole process is taken anywhere from six to nine months now you open the book and let's say you find a bug or a feature missing that you want to document it and you communicate that back to the engineer six to nine months later if you're lucky when the book comes out the information that you requested is in there or maybe it's not maybe we didn't have time to get to it so this is the source and even though this is a circle this is kind of a waterfall that I'm communicating here this is the source of many problems and many of the reasons that the documentation doesn't meet your expectations we have to follow this sometimes there are some books for which this is the best model large conceptual books like the Iook at fundamentals book that we'll talk about later which need to discuss the whole architecture from top to bottom provide lots of illustrations get lots and lots of review to make sure that we're communicating it exactly right they got to go through this process but you know you read that book once and the rest of the time you're reading the rest of the documentation and it doesn't need to go through that process so great we have a broken slide okay we get together the tech pubs managers get together about once a year in an off-site and talk about what is going right what's going wrong or what we could do better and what we can learn from our industry and a year or two ago is my job to present we could learn from the rest of our industry and we sat there and scratch their heads for a while and realized not a whole lot at this point where Apple is actually in a position of being really leading-edge in the industry in terms of our public lean methodology and what we're doing but we knew that there was a lot more we could do and after some research what I discovered was that there was an area of the software industry that we had a lot to learn from and that was the open source area and they bring a new model to documentation because the open source projects when they created documentation projects it was engineers who created those projects engineers come from a code background they believe that things should be buildable in a source control repository that anyone should be able to contribute and it should be difficult these are all brand new concepts to us in publications so the model will see what happens these slides here the the model in open source is that the people at each of those portions of the cycle that need to be involved get involved the rest don't get involved so we have the production team they provide tools for everyone else but they aren't necessarily required at each step to get the documentation out we have the content providers so that's and here's a major difference in the open source world content providers maybe the writers but they may not be frequently there the developers on the project or even subject area experts who just know a lot about it and have a lot to communicate finally you have the audience which is right now in Darwin primarily developers this will also be a course system in and users over time one of the things to notice is that you know developers on the content providing side developers on the audience side so these people you need the tools to talk to themselves with the open source tools and using repositories and such we cut down on the waterfall model we instead of having necessarily an editor in the process it will maybe just go to the tool and then go to the audience or in the ideal case and heterodox which we'll talk about in a moment is an example of this in the ideal case the production people provide the tool and get out of the picture so in the case of header doc we give you hetero doc the developers have hetero doc it goes directly to the audience when they release their their product or their header the audience can comment on that the developer can re release it right away or even better and this is what's happened with it Darwin we had IO graphics lib dot H in the i/o kit we hadn't gotten around to getting header doc into that yet Tori Lyons Darwin developer noticed this and he put together the header doc and sent it to us so in this new model things are so different from the old one that the audience is now providing the documentation to the developers this is very exciting to us and this is where we're focusing all of our energies so let's talk about what it called the ideal model there and that's hetero doc hetero doc treats the documentation as source code the documentation resides in the header files in a tag format hetero doc currently covers c and c++ projects is very similar to Java doc tags and I'd like to have Matt Morris come up and show us a header doc file and we could good so this is a heterodox file and notice in addition to regular comments we have right at the top a header wide comment this is so that you can provide some documentation related to the entire framework or header that you're dealing with but then most of the comments are on a per function or type def or data type basis so if we go down to i/o FD create cursor it's great shared cursor there you can see this is a so this is a function declaration and the header doc related to it and the thing to notice I'll provide some detail over here first thing to notice is that it has a special comment format and that is slash star bearing some people have asked us why we did that why we didn't either use the javadoc comment format or just a normal comment which would just be slash star the reason we went this way was because first of all well we designed hetero doc to be very similar to Java doc it's not Java doc and it has to use different tags because of C and C++ so we didn't want to confuse any Java doc processing tools and have them try to grab this comment and do something with it second though we didn't want to look like a regular comment because sometimes as you can see here the heterodox bloons up the size of the header we were concerned that some engineers would not want that to be the case in their published headers actually so far people don't really seem to have an issue with it but just in case we wanted them to be able to run a tool before they published a header which would strip out the header doc files but leave the normal comments so they could leave in comments like don't pass an OLE to this and such and not have those stripped out of the file so that's why we have this comment format there's only one required tag for a header doc comment and that's the first tag which explains what it is so in this case it's a function I Oh FB create shared cursor there are a variety of different tags depending on what you might be identifying the rest of the tags are optional a great one to have is an abstract the abstract will get pulled out in some of our tools and used in tables of contents so people can see quickly what it is they're going to jump to then of course the meat of the comment is the discussion this is typically one to three paragraphs of your information about what this function does then for one or more of the parameters however many you choose you can provide parameter specific documentation finally if you have a return code you can choose to identify what that return code is about the only other requirement for a hetero comment is that it's placement must be directly above the thing that it's talking about there can be whitespace and all that but it has to be above so that when our Perl tools go through and process this they can connect the two together and make use of them so now I'd like to have Matt walk through the actual process of creating some header doc here ok we can build header doc either directly in project builder and I'll show you how to do that but more commonly especially for Darwin developers you'd be building it from the command line because you don't have project builder to use the command is quite simple its header doc to HTML and that's in user bin on your system you can give it an output directory to drop the documentation into - oh is for the output directory and then you tell it what to process and what to process can be either a single header a group of headers or a folder and if it's a folder it'll recurse through the folder finding all headers and processing them in this case before we run this let's take a look we've got Iowa graphics live dot H as we were talking about but we have a number of other headers here too there 8 headers once in a subfolder so we process it it's gone through the 8 headers its emits a comment for each one that visits here you notice that it's skipping one because it didn't find anything of interest to document there were no header doc comments in that header and I know because I counted it's gone through 8 headers and it's produced over 50 pages of documentation so let's look at that documentation so we produce this folder dock and here are the seven folders of documentation that one per header that we've generated and we can look at i/o graphics Lib documentation produces a frame set of documentation so you can open the index.html to get there and as you can see you've got a TOC on the left and that app header comment that Ron pointed out has become the opening page of your documentation for each function you have a link and you can see how those tags have been converted into HTML the function tag let's look at the one that Ron pointed out that create shared cursor we have a the title that was the @ function tag abstract is here the declaration is pulled directly from the header this is the discussion block and then each parameter is listed and described in this table and the result code follows so you have a long page of each of the functions in the TOC you'll see grouping headings depending on the type of API enumerations and pound defines and so you can print out this documentation which would be hard to do with a frame set there's a link at the bottom that will open up one continuous page of documentation and then you can print that out it has all the functions all the enumerations and so on okay now that's nice that we've created a folder of documentation for every header but be even more convenient if there is a way to gather this all together in a more easily navigatable arrangement and so we have another script gather header docx it takes as its input the the folder to go visit it visits every island of documentation that header doc created and it creates the TOC and also adjust some links so that you can get from the island back to the TOC and from the TOC to the island so let's look at what that looks like okay okay so here's our documentation notice that now we have this master TOC a lot of the names of the files are configurable through a configuration file that I'll talk about later so if we open that you see the the header the links to the each of the pools of documentation and we've added a top button here to get back to the TOC now if one of these headers let's say it's a event source if it described more than just capi if it had a C++ class you end up with more than one frame set of documentation this is the the frame set for the header itself and in fact there's only one thing other than the class of interest in it and that's that pound to fine but there's also a class I event source so we can go visit that and notice that this is much like the the C frame set except now you have member functions and member data and within each of those headings you have the access type public private and so on and your class documentation is there for you okay so I told you that you can build this documentation directly in project builder I'll show you how that's done if you've been to any of the project builder talks you know that their their build phases for your targets so if you go to the targets tab and look at this is just an empty project that contains my header files and look at the build phases you'll see down here I've added two more these are shell script build phases and all that does is run a script and here I define it and this is pretty much exactly what I ran on the command line only difference is I'm using some symbolic constants that project builder knows about to tell it where to find the source and where to put that the documentation so I'm going to look for the source in the source headers in the project source and I'm going to put the documentation in the resources folder inside the app wrapper and after I do that I'm going to run the second shell script gather header dock which will go in to that folder of documentation and create the master TOC so let's do that now if you look up here in the build pain after it does the normal build phases you'll see running by the header dot comments I think we're about there okay that was header doc going by there's gather header doc so if we go back to the files display and look in the products now we have our application header doc demo app and you can go in there and we have created this folder of documentation and here's our master TOC and as you saw in some of the project builder talks we're trying to integrate the documentation with project builders so in a later release you'll be able to see your documentation right there so one last thing I'd like to show is the documentation for header doc it is installed in on your system it's in developer documentation developer tools and it comes with a header doc if you download it from our website as well header doc has reference to tell you pretty much what I've just told you about how to use it and what to expect from it how to run it it also talks about the configuration file I mentioned that lets you set some of the names of the files and then it starts discussing how to tag your source and at the bottom here there's a reference for all the tags that you can use in C and C++ so if you're wondering about function tags well here are the tags that you can use at function at Bram at result and for every type of API it gives an example of a declaration that's been tagged and for those of you who would like to contribute to the header doc effort and want to play with Perl it's a fairly nicely structured script I think that has a parser that generates clap objects it's an object-oriented perl script and this is the class hierarchy of the objects that are produced the source code itself has a lot of comments and there's an active mailing list that you can sign up for to to join the effort so I hope you use it thank you thanks Matt and Matt is the primary author of heterodox if we could go to slides on both screens things so our goals for header doc all C and C++ API in Darwin should be using heterodox if you own or are maintaining or contributing to a C or C++ project start using header doc today if you're not already we will then have a universal approach to reference across the system and and it'll be the best for everyone we really want to support objective-c of course we we have at each step added functionality ahead of duck based on what we needed at that moment as cocoa becomes a reality on 10 we're very interested in adding Objective C a number of our people in the Darwin community have expressed interest in doing this Matt's a really busy guy right now and he's actually doing the code also to put the the editing the documentation viewing and editing into project builder so we're not gonna get Objective C in for a little while and if you're someone who knows or would like to learn perl and is interested in that you could really help us out join the mailing list you can go to list a pecan or to the document or to the Darwins site and find out about the mailing list and finally matt showed how you can do this in Mac OS 10 and project builder we want to make a push button so that you don't have to go and and add in your own build phase stuff that will happen in the near future next we have as an area that we cover in the Darwin documentation project or how to documents so these are different from header doc in that who provides them as tends to be different in header dockets almost always the the engineer who created the code who provides the header doc with how to documents anyone who has the knowledge can and is encouraged to provide a how to document these are typically self-contained small articles from one to ten pages there's no particular restriction you can make it as long as you want that's the standard they solve a particular problem or tell you how to get your job done in a particular aspect of the system and we'll get into some examples later what we have and some of the ones that would be useful to add and they're authored in docbook xml so a lot of people have heard about xml maybe they've used a bit for their system but they haven't necessarily seen how it's how it was intended to be used originally and we can go into that and how we use it so docbook xml is the industry standard xml definition for technical publications so the idea with XML is that each industry would come up with its own set of tags specific to that industry the medical industry is doing this for billing etc well probably the most advanced set of XML tags in the world is a doc look for exactly the kind of stuff that we do XML is structured information which we'll get to in a minute and there is if you are interested in trying to venture into XML and learn what this is all about there's a lot of information in the Darwin documentation how-to in the how-to site and the the only problem there is that the tools for XML the the editing tools for XML aren't quite there yet if you remember with HTML it took two three years before they were good HTML editors and XML is much harder to create an editor force so there are lots of editors in progress but there are there aren't any that are gonna solve all your problems and we're more interested in getting the information if you have a how-to in you if you have information that other Darwyn developers or system administrators or users should know about we just want the information so if if you don't want to get into XML or if you if you venture into it and find it's a little murky just go ahead and email to me your document or to the Darwin developer list we will turn it into XML for you and add it to the project so let's get into some of the detail of XML and how it's different from HTML which you may be familiar with HTML has formatting based tags so your tag will say this word is italic this word is bold this word is an h1 but it won't say why it's italic and won't say why it's bold so processing tools do anything with it they can't change the formatting based on the output or do anything with it XML is exactly the opposite it only tells about the content of the tags it does not say what to do with it so for instance in a Darwin documentation how to you have a title of the document the title is clearly named title you have a first name of the author you have the email address of the author so absolutely nothing is said about how to process this that comes later other tools will come along take a look at these and process them and we'll show you how that works this is a very important difference it takes people a little while to get used to the effect is to turn your document into a mini database the other very important aspects of XML and really in my opinion the reason that XML is taking off in the world are that the tags are hierarchical and closed now HTML has a little of both of these the problem is it's not strict about it so it doesn't help very much so for instance a paragraph tag in HTML does not have to be closed some browsers make you close list items some don't and so it you don't know how your document is going to work in different browsers until you just try it in XML this never happens is an absolute rule tags must be closed so to take a reasonably sophisticated little block of XML from a how to document the revision history so there's a revision history section each revision gets a comment and such this is has an opening tag rev history and a closing tag Rev history everything else will be inside there each revision has an opening revision tag and a closing revision tag then for each piece of information about that revision it has an opening clag and a closing tag so we have the revision number we have the date of the revision and finally any comment we want to create for this revision so this is a great example of how a tool goes through this and figures out what to do with it the writer doesn't say anything but here's the information that I'm providing to you one of the things people find when they hit doc book is doc book is a very mature set of XML tags and it tries to do every things so you can create facts with docbook you can create books you can create articles you can create lots of other things and that that means there are so many tags that people get overwhelmed and they don't know what to do the trick is ignore them you only need in order to author a doc a Darwin documentation how to document to know about 10 tags now to be honest at the top and bottom of the document there are a few more tags in that like the Reb history there you just want to copy those from the template that we give you on the site and and don't worry just fill in the information you don't need to remember the tags or anything the only tags you actually need to know when you're providing information are about about 10 this is a good sample of them so how do what a section is a title a paragraph a couple different kinds of Lists how to List code and URLs that's about all you need to know so it really you just grab our template that we have on the site that gives you all the tags you need to know you don't need to worry about any of the others I'd like to have Matt come up again and show us a demo of this okay so Ron talked about the nested nature of XML here's a document this is the Darwin doc how to document a document that describes how to create these how to documents and and as he mentioned here's that revision history block and other standard blocks like that abstract something that more like what you would be doing is these section blocks this is the fact section we'll see this once we generate the HTML question answer it has a title and a paragraph and a section indicator okay just as with heterodox you can build the stuff in project builder but it's more likely that you'll be doing this on the command line so the command that we use this XSLT proc which is a command that applies an XSL file to the XML file the XSL file provides the formatting actually it's more than that it it's a transformations file that takes the XML that is just defined in terms of semantics you know this is a title or this is an email address and applies format to it beyond that it can do other transformations and we'll see that the one that we're using is doc book and we're going to process the Darwin how-to and redirect the output to a file and it'll be HTML by the time this is done okay and we can open that how to file okay so this is that XML now transformed into HTML and things like the title are just applied formats applied to them that things like the email address the transformation is to create a mail to link as well as to format it in a specific way and then our input XML document did not have a table of contents this was extracted by XSLT proc and the XSL Transformation file by looking at the structure of the document and extracting the upper and lower level headings and making them into links also we've got URLs that are now hot and here's this question-and-answer section that we were looking at the titles the questions and the formatted answers history section 1 distress all right yeah so just wanted to stress here here are the revision history tags that the writer provided here is the output as you can see this the the XSL tool has done something completely different with them it's created a sophisticated table if you've ever done your own table editing and you know HTML you know it's quite a pain in this case the writer didn't even know is going to be a table and there you have it so that's what a great power of using this process okay thank you so we go back to slides on both please so our goals for how to documents well the we started the heterodox project quite a while ago and that's going very well all of i/o kit is in heterodox and many other projects are begin to do that many other companies throughout the software industry even totally unrelated to Darwin are beginning to use header dot because they I mean yeah yeah heterodox because they find it to be a really valuable tool how to documents we just got going in the last couple of months and our goal is to document all common tasks so anyone coming to Darwin to help develop it administrate it whatever has a place to go and find a quick explanation of the various things they need to know so a great example of this is and it's up there now how to build Darwin we will have next week going up how to be a Darwin developer we also and this is really important to us want to cover all the common driver families so we'd like to have a how-to for each driver family you can go to that gives you the lowdown on what you should be sub classing and what you need to expect when you're working with that finally very important to the community and important Apple to is that there are a lot of topics that our writers at Apple are not necessarily the best people to provide information for and we're all really excited to see that the Darwin is up on Intel that a huge number of people have downloaded it for Intel but to be absolutely honest those of us sitting in Apple offices are not necessarily the best to provide you information on how to use it on Intel so this is a great opportunity for people in the community to provide information for those Darwin users who need it and we'd love to see some of this so if there's anything you know like that that'd be great to contribute finally we have our last section of documentation that the Darwin documentation project covers that's Man pages as if you've done any development whatsoever on a UNIX system you know man pages are critical for Darwin development if you've done any development on our system you know man pages are maybe not in the state that we would like to be them in right now so this is a major focus for the next couple of months for us and what we're working on is of course we want to have a complete set of accurate useful man pages and how we're getting there is a few years ago it made sense at the time to take the man pages and put them into a documentation project that a writer would maintain and we we ended up separating them in the code from the projects that they were documenting this turns out maybe not to at least at this point have been such a great idea and the result is we have some stagnant man pages that need a lot of updating and the engineers have kind of gotten out of the habit of keeping their own man page up-to-date so we need to explode this project get the man pages back next to their commands where they belong get the engineers updating them we need to clean off a lot of old bugs we really need to get back into staying in sync with our upstream main pages so we need in particular to be working closely with the BSc people get the bsd freebsd main pages in there and get that cleaned up and that's so again the next whole month's a major focus for us and we'll be seeing that that's something we primarily we will be providing some goals and guidelines for people doing their own commands on how to keep up a man page but the rest of that's primarily something we need to do at Apple on Mac OS 10 we're gonna provide Mac HTML display the man pages in project builder along with the rest of the documentation we're also considering and this is something unlike we'd like to get input on maybe during the Q&A or if you send email or anything we're considering doing like open BSD and some of the others and putting up on the Darwins site of full HTML version of the man pages that you can search and navigate through and where that goes on our priority list depends on how much people want it we're not totally sure so now we get to what might be really interesting for some of you what actual documentation do we have at this point first of all you should be aware that at this time there's there's two sort of places to go when you're dealing with Darwen related information one of course is the Darwin site the other though is the ABC Mac os10 website where we have a whole kernel page of information that you may or may not be aware of this includes pointers to Iook at header doc I get tutorials conceptual docs etc in fact as of this week is a great thing to go and check out we have the first part of our i/o kit fundamentals explaining the whole system we have brand new information about USB and hid up on the website and that's a great place to check it out we have how-to documents so the Darwin developed how to be a Darwin developer we'll get up next week building Darwin is there now some general ones like setting your Intel screen resolution and then of course the Darwin documentation how to which can contains quite a lot of information on all the stuff we're talking about today tools you can use what you need to know etc we are working on we have tutorials now kernel and i/o kit and packaging and such tutorials we're doing a revision of these to include versioning information these should be up in the next week or two what we're working on well as you can see we're starting to bridge the gap between Darwin and Mac os10 in terms of this information so we're no longer thinking of i/o kit documentation and such as separate from Darwin its documenting Darwin technology's an important point there though is this doesn't mean that everyone has to be a Darwin developer or go to the Darwin site to get our information this is how we're developing the doc this is how we're making it our processes available letting people have more information about what we're doing when we distribute IO kit documentation or anything else like that it will go to all the standard places it will go to the standard developer CDs and up on the website and all the normal places etc so the Darwin process is just an opportunity for us to let you get closer to the documentation see what we're doing and contribute if you want to we're doing the aisle kit fundamentals book that I've mentioned a couple times really critical book go and check out the first part of it we're gonna be getting this up as rapidly as we can to give you the whole picture of the i/o kit and written by Terry Donahue is the same person who did the system overview book for Mac OS 10 that you saw a year or two ago so we know it's gonna be really good and in addition to more cookbooks of course we're continually working on more header overage this is what we like to call open source opportunities you might also call it it's not there yet so this is all stuff we'll get to someday but we would love it if people who know as much or more than we do in the community could contribute in these areas so please if you're someone who has information and kernel programming or debugging architecture VFS kernel extension network kernel extensions make a note of it think about doing a how to contact us you know email the Darwin developer list email me and of course heterodox coverage in your own projects and if you see as Tory did something heterodox missing and something that you know a lot about and you could spend an hour and provide it and help everyone else out a lot that would be really great so if you don't know the stuff on that list well at but you'd like to contribute join the Darwin developer list we don't have a special documentation list at this point because we want everyone to feel included so join the Darwin development list ask what people need see what people are talking about see if there's something that you can contribute to write a how-to document to document and in particular as I said if you need to just send me the HTML the RTF whatever it is I'll turn it into XML for you so on any of issue related to the Darwin documentation project you can contact me or send email to the darn developers list for header doc and such you can contact Matt even better a send email to the hetero doc developers mailing list which you can also join you