---
title: WWDC2001 Session 139
framework: wwdc
role: article
path: wwdc/wwdc2001-139
---

# WWDC2001 Session 139

## Transcript

Kind: captions Language: en thank you and welcome to session 139 leveraging BSD Services in Mac OS 10 hope you all had a good lunch we will try and keep you awake this afternoon and as most of you know my wrapped around Darwin kernel Mac OS 10 it's our own customized version of BSD and with that what that means for us developers you now have a whole library of BSD tools and applications available to you and you can also explore the power of our BSD to bring your BSD applications to our platform so without further ado I'd like to introduce the director of core OS engineering Brant Hallie [Applause] good afternoon I'm impressed here it is Friday afternoon after lunch right before a three-day weekend and you're in here so you must be the real hardcore guys so we're gonna spend some time this afternoon and talk a little bit about what BSD is in Mac OS 10 what kind of role it plays within the system how you can actually get involved in programming with it on 10 and a bit about the BSD community because it's more than just the technology by now if you've been to a number of our sessions you've seen the architectural diagram for your where Darwin and core OS fit into the architecture of Mac OS 10 there's a lot of Technology down on in in the kernel and and we're going to focus today on just the BSD portion of this a little later today there'll be some discussion on the bsd kernel pieces and the mach kernel right after this session first we want to talk a little bit about bsd from the kernel perspective understand that you know as far as darwin kernel architecture is concerned that bsd if you will suckin on that if you layer the even at the kernel level bsd sits on top of our kernel architecture it has the filesystem technology the networking layer and basically provides the POSIX if you will functionality layer for the system IO kit within the kernel space is our answer for dealing with io and drivers and and basically abstractions for devices on the system and underneath that sits mock which is fundamentally responsible for abstracting the processor dealing with low-level tasks and thread abstractions memory management and again dealing with the processor and our processors in the case of an MP system for BSD need to realize that you know as being just a sub component of Mac OS 10 that really BSD is an operating system in and of itself it's it is a kernel environment that sits within our within our kernel kernel space of the system it's also a set of user land libraries and services that are available to the applications and it's also an application environment in and of itself the command line if you will from the kernel perspective it's worth noting that we there's some history to how BSD actually came to play in Mac OS 10 and we'll talk a bit more about that later but for what we ship it's based on the BSD 4 4 4 4 environment as well as being integrated with mock and I ok it provides the personality api's for the system mock as it sits on the bottom is very much of an abstraction Lee are dealing with processor management memory management it doesn't actually provide any if you will policy to the system bsd kernel however does it very much represents the OS policy of the system it is responsible for the process model that defines how each and every application space runs it also provides the basic security policy for the system whether it be the concept of individual users or even as is abstracted down to the file system to access the files and other services of the OS from a process model perspective a you know there's kind of two ways of looking at this within Mac OS 10 again at the very lowest level Mach is responsible for the abstraction of the processor and the memory management and it has an abstraction or model called a task which is responsible for providing that container to ten to a memory basically for an address spacer a chunk of memory that represents an application the bsd process sits on top of those very low level primitives and provides a considerable amount of additional state to the system it's responsible for much of the OS resource management so for things like file descriptors and you know network services and things like that and high level memory abstractions as well as of course all the network resources the bsd process is responsible for that and when a bsd process you know terminates for some reason either because the application quits or because there's some type of fault condition it's the responsibility of BSD to actually reclaim all the resources that are associated with it with a given application in addition there's other ancillary services that the process provides things like environment environment variables and signal delivery ways of being able to provide some type of a low-level interrupt dispatch if you will at the application level BSD also provides the rest of the security policy and that is certainly the concept of users where inherent to the system at the very lowest level is this concept of multiple users on the system you know as you normally as you set up Mac OS 10 you set up your you know yourself is the default user much of that is kind of hidden away as the system will automatically log in but it's possible to set up Mac doors tend to support multiple users on the system each with their own set of privileges and capabilities and its responsibilities of BSB BSD to enforce that there's also this concept in BSD of a super user and this is you know affectionately known as root and root has certain particular capabilities and in the system to be able to do things that administrator or access certain resources that are beyond other the capability of other users in the system for Mac OS 10 we've worked from a user experience perspective to try and abstract away the concept of root it's still there if you're a smart you know familiar with and of how UNIX works and BSD works it's possible to actually to enable root in Mac OS 10 and use it just like you would in any other BSD system but it's something we try to abstract away for your conventional user because we don't want them to have to become UNIX system administrators but it's important to note that there is a certain set of capabilities and facilities that the super user or root user of the system is able to do and that when you do things like installations or other things and you go through the process of authentication you're effectively enabling that level of access to the system other aspect of security pasal policy involves the file system again the concept of users in the system is actually reflected down into the file system each and every file system file on the system has an owner a group that it's it's part of and permission access to to those files that they're associated with it and those but those same types of capabilities can be applied to other kinds of capabilities in the system bsd is also kind of the environment with which the fart within which the file system sits it's based on a VFS architecture the standard bsd file system architecture and which just supports a number of different file system plug-in types and things like that an earlier in the in the week there's a session on file systems that went into this and more depth but it's important to note that the file system environment is a subset if you will or whether a subsystem of the BSD environment the same is true for networking the capabilities of networking and Mac os10 are built on the BSD sockets api's and if you're you know interested in learning more about either of these particular subsystems that bsd at the kernel environment there's some really great books out there by mcusic and a couple others that actually go into some of this architecture Stevens is a good one for networking and there's again both of these can are based off a standard BSD architecture before I get on to the user environment I do want to make sure people are reminded that immediately following this session there is a session on the darwin kernel itself to get more into the low levels of the system well they'll talk a lot more about Mach and the bsd kernel and that's here in this room moving on to the bsd user environment and that's really what's kind of you know important from the standpoint of most people who are going to be writing code against bsd the user environment is could be considered to be app in other peer of cocoa and carbon and classic if you will it's another application environment available to the system and it has a number of services and facilities that are part of it certainly it includes the command line and shell and you know those kinds of things that are what do you would expect to be as part of a standard you know unix and environment but it is also where most of the you know network client tools things like ssh things like various sets of file tools and other command-line tools that you would expect is part of unix all those things exist within the bsd user environment in addition a lot of our high-level network services are actually services that exist within bsd they're actually bsd applications if you will things like net info dns services bar bind the network time protocol demons and the system log and the apache server which is used for personal file sharing and on our server product apache is also used for the other high end web serving all of those facilities are actually bsd you know demons or commands and utilities that exist on the system and run within the bsd environment there's a number of different kinds of applications that you would expect to run here that are part of either the system or the developer tools things like Emacs and perl and samba those kinds of things all of those are the kind of applications you would expect in this kind of environment the way we abstract the api's and Mac OS 10 for BSD is through something we call the system framework you probably as you've been to a number of the various sessions in Mac OS 10 we try and abstract api's through the concept of frameworks and the system framework is the one that represents the API set that obstructs BSD and the other aspects of the kernel they include things like the POSIX style of api's again things like pthreads and other things like that all exist there your standard math library see libraries all of that those are the kinds of things that are part of the system framework it's worth talking a bit about how POSIX actually fits into the the plan here for for Apple from our perspective it's important to try and make sure that the API is you know are are compliant so as as people are working with the system we find places where we need new api's or things like that we actually will use POSIX API s as a reference point that's usually the basis as Steve said at the keynote on Monday the focus around staying within standards is a good thing however even though compliance is a goal for our perspective certification is not something that we're trying to do so you know as you're looking for things there's going to be probably a lot of little details where we may not necessarily be you know completely you know completely POSIX because that isn't a goal for Apple but again where there are API is when we introduce things like pthreads like there's also the POSIX shared memory facilities those are as we've introduced those into the system's will tend to reference you know the POSIX API as our starting point it's a worth noting that the BSD API said is is is in fact a first-class citizen in Mac OS 10 it's a just like carbon and cocoa it is another application environment it can a coexist in fact however with carbon and cocoa and in fact Java because the way that it sits is that all of the application environment selves are actually built on these api's even though it's a peer of carbon and cocoa if you will and the ability of writing kind of bsd and unix types of applications the fact is this is the abstraction layer that actually represents the operating system and iOS personality of the system and all of these services are what the other application environments use so for things like cocoa in particular you have pretty direct access to the BSD and POSIX and system framework api's given that when should you actually use these things well certainly if you're porting an application from a POSIX environment a bsd the linux or whatever that's a pretty obvious case where you might actually want to use the system framework and those API is pretty directly if a goal of your application is actually to be able to have that kind of portability obviously this is yet another place where it makes sense for you to be able to use these particular api's when you're dealing in situations when carbon and cocoa don't give you a low enough level access to a particular service this also makes sense this again these particular application environments are actually built on the bsd api set and the system framework they're there and available however you should always try and use the highest level of destruction you can for your applications many times there's some dependencies that exist in those higher level abstractions for example if you're using in carbon the dependency on the the way the files and things like that and the dependency ins and support for things like resource forks are completely supported by the carbon environment access to resource Forks and things like that is way below the the abstraction layer that something like the BSD api's will support for you you'll actually have to go in there and fiddle bits at that the level so carbon actually helps you a lot in those abstractions and so you should really try and use the highest level of abstraction you can but that isn't always possible if you're porting a UNIX application to Mac OS 10 one thing you should realize is that built into the system of course there are isn't built in x windows support so if you're porting some type of GUI app over you really should not depend on it the good news is there's actually a couple of third-party solutions out there if you're trying to port your app over I believe tenon has one they even want a design award I believe and there's work going on the X 386 community there's a basically on Darwin itself you can run Darwin without the rest of Mac os10 just with an X Windows environment so it's actually possible to actually port apps like that over to the system obviously an X Windows user interface is not exactly the kind of user experience that most of our users are more likely to want to use so this is probably just a stepping stone approach that you would take to porting an app to Mac OS 10 if you're going to be wanting to really make it 10 friendly take advantage of aqua and the other kinds of capabilities exist you should probably be using cocoa to wrap your your app so we effectively replace the the GUI or the user interface with cocoa Carbon can also work however I found that cocoa is actually probably the fastest and easiest way to get there there's a number of little little hints if you will about porting your app over one thing for example is we basically discourage the use of common variables and such our implementation for dynamic libraries is very very different we also do not support the use of the the c c++ pre pre compiler at least is for the purposes of porting are a pre compiler for c++ is very different than than the other environments and you should really try and avoid its use again dynamic libraries are very different under ten so use of things like DL open is not recommended that won't work you should actually look at the deliberate ease going to try and try and create analogous solutions to pull the plug see if plugins and other kinds of services that exist those dynamic library services are not the same on 10 one thing to note is that new make in the build environment is the default build environment for BSD apps there is BSD make on the system as well you need to explicitly use it if that is how your at what your app depends on also use of auto comp which is a very common portability solution that's available for many BSD tools and applications usually works and in fact a lot of the pro various ports collections and things like that or had been modified to run under and build under Mac OS 10 already however if you find that it doesn't we actually include as part of the developer tools and using Olympics a set of config recipes if you will that will usually allow your app to be able to be ported over completely and one other thing to note is that in terms of how well the system is laid out and when you're building your application remember it is built on a bsd basically uh if you know as a bsd system if you compare this with how for example linux files are laid out you're going to find that the filesystem layout is a little bit different between these two environments and our environment certainly reference is much more off of the bsd environment and you should try and if you're going to be modifying the build you want to lean more towards a bsd you know set of build variables and settings in order to be able to get your system to work all of these directories that are mentioned here certainly exists on the ten system but you probably noticed they're invisible to the end-user this is certainly something that we're not trying to encourage people to use in terms of the layout some tools and and commands that you get are dependent on some of these file system layouts but from a perspective of applications that you package up we've actually would actually try and encourage you to package them up as part of a GUI app and in other places but these standard install places exist I kind of just show a little bit more about how you might actually go about porting an application to ten I'd like to bring Eric Payton up and New York you know one of the things that I'm interested in seeing is you know you're being able to take a nap or I kind of off the net yep and build it for ten how do you go about doing that sure not a problem so we see do we have the right yet we did about a week and a half ago two weeks ago Brett called me and said Eric go to the ports collection and do something cool and I said okay and I went up to the ports collection which is something I'm not intimately familiar with but I've played around in there a couple times and I had a mission I've been playing around and I have a very large collection of mp3 files and they were spread over multiple different directories multiple different discs and I wanted to get them all on my nice big server so I needed to find ones that were duplicated all over the place and I knew that I could do this with you know a little bit of shell scripting but that's not really the point here the point here is to go get a tool to help me do that bring it down make it maybe put a GUI on it or something nice like that so what I did is I went up into the utilities directory up on the port's collection up on FreeBSD org I believe or something and I grabbed same file what same file does is in multiple different ways it looks at a collection of files and determines which ones match and one of the interesting things about same file is that it does it from standard in and vs. a lot of the other ones we have to like enumerate all your stuff on your command line and I thought it'd be an interesting technical challenge to bring this down right a little make make sure that it builds and do something interesting with it so I downloaded it just you know I'm me web or ie or whatever I used to came to remember and I brought it down and it's unpackage it now this is one of the the well-behaved packages in the fact that I can just whoops once I get in the directory I can just configure it and it works out of the box like Brett said in user live exec the config files are quite often needed to to bring across a package that hasn't been touched in in a while something that hasn't been updated recently with the more recent config files but this one works out of the box and I type make and if you notice I have an executable that I built 25 seconds ago or whatever it was and that's just pulling down directly off of the ports ago this is pulling directly off the ports collection no work on my part no messing with make files nothing just everything works yeah a little bit we'll talk a little bit about what the ports collection is because you know certainly one of the advantages of the bsd and being part of the bsd community there's they're actually thousands of interesting little tools and utilities that exist out there within this thing called the ports collection yeah and the interesting thing like I said was that same file uses standard in to receive its output and that's what you do in this case I've got a little little command line fine here that will find all the files from here on down and pipe it into same file and you notice here we get if the font wasn't so big it might actually look like lines but it's pretty darn ugly this is not a Mac user yeah this is not the user experience that we're looking to provide for a Macintosh application so like I expect better than that yeah so we're gonna try to do something a little bit better than this you'll notice here before we move on you'll notice the first column if you look along this size here site here is the size of the file that it found the next column is the first file the next column is the second file and there's some extraneous stuff on the end which doesn't really mean much unless you read the readme and start playing around with the different option and that are available to it but that's not really what I'm looking for here I just want to find how big are my files and which ones match up and are the exact same size so let me get this right this tool basically takes two big clumps of file directories full of files and points out which ones happen to be the same ones regardless of what they're called yep or anything yep irregardless of filename and that kind of good stuff and like I said this could easily been done with shell scripting but not nearly as interesting so I have already let's go ahead and hide that I've already created a project but the first thing we want to do when you're coming at something from a Coco's perspective a lot of lot of times you just want to make a UI first so I created a project which provided the main menu dot nib which if any of you had been to any of the cocoa sessions throughout the week you would have probably seen that multiple times by now and I've already dragged my files over and I'll show you all the code in a few minutes but let's just start off with something simple here let's create a little UI a lot of window make it nice and big so that we can see these long paths all over the place we will add a table so we can see the output and you got to love the blue lines and what we really cared about was in that output we had three columns that mattered we had file size the first file name and the second file name and that provides us with some interesting stuff so if we had ended up with output that looks something like this in a table view that we could sort that we can mess around with that be a little closer to a reasonable UI so there we've got some nice big fat columns so we can see all of our paths and we can for example add a field that will let us put in a path that we can type in or how about even better a lot a little button so that we can select a directory to start from and I'll throw another button up here that allows us to search for duplicates that's clean this up a little bit nothing really major we're just playing around I'll call this half two files so the thing with with certainly with programming and KOCO here is basically what you're spending all the time doing is laying out the UI yeah this is the fun part yeah spend a lot of time doing the fun stuff laying it out make it all nice and pretty and interface builder if you haven't if you did if you missed the interface builder tutorials and sessions this week you missed a lot of really good stuff it is one really cool tool so here we've got a UI that I would consider somewhat usable for us UNIX people something that we can work off of let's go ahead and stop right here and start hooking stuff up and figure out what's going on yeah we want the window to be visible when we launch the application I've already created a lot of the code and we'll walk through that together here in a second but let's start off by creating our controller object which I've already imported into we have a same file controller object I've already imported the code which we'll look at in a minute let's go ahead and hook up the portions of that object that matter to us at this time once again like I said if you missed the interface builder talks I really recommend that you spend some time playing around with this tool it's pretty cool so we've got a table for output we have our three columns a size column a column for our first file and a column for our second file we have a text field that is that contains our path we have our our button that begins and ends our tool we can go ahead and stay here everything's hooked up coming out of the controller we know what all the output parts of the UI are gonna be let's hook up what's going into the controller the things that make stuff happen so there's two major portions here that matter and of course the window is so big you can't see what you're doing here hold on this is an easy way to do it we want to tell the system to start my tool we want to tell the system let me select a directory will save our little nib in our project here and for now this is all we really need to do so let's go ahead and go off and open up our project that I previously had created and let's walk through what the code actually does to actually get this to it to output information to the GUI so at this point this is all pretty standard cocoa stuff yeah nothing really unusual nothing unusual at all however it's not hooked up to BSD there there's no tools involved or anything like that first thing we want to do let's go ahead and add our well it's already here I'll show you what it is I've already included the the same file binary as a resource of my project now this is the one you built that was the command yet this is the one that I built four minutes ago or whatever it's a command-line tool the reason that I'm putting in the resources directory of my project is so that I don't have to have separate install instructions for my user to get the same file UNIX binary installed on their system in slash user user local bin or anything like that if I include it as part of my project they don't ever have to worry about it it just becomes cohesive whole it all works together nothing else has to ship and they don't have to install files in two locations that they can't delete them from or don't even know that it's happening in the first place so basically if you're gonna wrap something like a UNIX tool unless it's something that already exists in the system you can bundle put it inside your application bundle and make it you know just another resource if you will of your application most definitely and we really suggest that you do that to avoid situations where you're installing fragmented portions of your package all over the hard disk it it's not very Mac OS ish to do that it's something that we're looking to to see improvement from you now before I began that our earlier we were talking about my different classes and I had the same file class and the same file class is is the portion of my code that is the controller object for what's going on in my interface it consists of two major parts it consists of two methods a method for beginning my func my my search and a method for finding finding files for the user to do that so they press that button and it fills in a path so that stuff can happen let's start off with the selector so the user runs the application the first thing they're gonna do is they're going to go off and select a directory and so what I've done here is I've added in a simple little method here five lines of code creating an open panel telling them that they can can't choose files but they can only choose directories I don't want multiple selection at this time it would be an easy enhancement to add with some more UI later and then I tell my my system to begin a sheet for a directory you'll notice here it's modal for the window and I've got a selector that happens at the end so basically this just pops up the sheet life isn't possible sheet lets them do what they want to do and everything works up above the other the only other method in this class at all is the toggle tool method which will start or stop your execution if it's not executing go ahead and look at the path that we had set up in the in the UI the one that user just selected the one the user selected or the one that the user had typed in let's create an array of our arguments the first argument is the path to our same file resource that's our binary that we had included that's the straight UNIX binary nothing special there then we add in some arguments you'll notice here that I have if verbose check state so if the verbose checkbox is checked add an object well I never added a verbose checkbox and to show you a little bit about how easy some of this stuff can be what I'll do is I'll come back here and I'll add a little checkbox on to my UI verbose or you could name it something much more concise than that if you wanted to and we'll hook up the verbose check there and what this would do if you go back to the code that we're looking at all it's going to do is it's going to add an object into my array of - V and we'll talk about why this array looks the way that it looks in a minute when we get to the process what is really happen so basically like most UNIX tools there's a half a bazillion little options and stuff and it's very easy in the UI to be able to add checkboxes things like that most definitely gonna hide that kind of stuff away from having people lefting to build command lines yeah it's a lot easier to start once you have all this set up and you'll see it's not very much code once you have it set up it's very easy to add command line switches can turn into GUI switches very rapidly no very little work whatsoever so you'll notice here that I've at the beginning of this code segment I've got a current process equal process out luck init with controller self arguments and what that is is for every time that we run this we want to have a process run something's going to launch this UNIX tool get the output and all that we feed it in with the arguments at the end I just called them Arg B but they you could name them whatever you'd like down after that you'll notice I have an append output this is just a little log so that you can see that it's begun begun running and in this block here you will notice that what we do is we take the path that the user had entered or selected use the NS File Manager class which will a BSD program would probably become a very good friend of yours it's a it's a general-purpose file manager class we retrieve all of the paths at our the sub paths from that path we even do some nice stuff like we expand the Tilda in path if you have BSD users they very well might want to use tilde for their home directory and not have to go selecting all over the place with a with a file browser because some people type faster than they click usually the tilde is something expanded by the shell not yeah we have no shell performing any of the actions here so there's there's no possibility for expansion there we if I can interrupt one thing that's worth noting is there's a huge number of facilities within the foundation portion of cocoa that actually wrap a lot of and do a lot of the the kind of the services that would you expect to be able to talk to background UNIX apps things like expanding till these things like breaking up command lines into specific individual arguments yeah breaking up command lines finding your home directory finding the current user all that kind of stuff are all portions of foundation in this specific case is a is a method on NS string but a lot of them are are in different portions of the foundation classes and they're very very useful to somebody who's trying to leverage BSD you'll notice here what we have is to our process so we get all of our paths we enumerate through our paths and we pass our process on standard in the string that we had just expanded and then when we're finished with all once we've spun through all of them we pass it on we tell the process ok I'm done writing go ahead and go off and do your thing the this closed writing is I guess you could say an analogous to hitting ctrl D at the end of messing around a cat I'm done messing around go deuce do something and that's all it is in our same file class itself there's there's nothing else to it really except for handling of the output we we've abstracted into the superclass so we've got a class that controls them pushing buttons but what happens where would you know where where does all this go on I don't see any work going on here so you'll notice that I've got another class process and a class simple tool simple tool is the parent class for the same file and the reason that it's abstracted out was because the simple tool class is an abstraction that I've used in multiple different places and it made it very quick and easy I just literally dumped this class in object-oriented programming let's use it again and what it does is allow it it's the thing that handles the output and passes the output back to the UI now there's a very common set of abstractions when you're trying to wrap a UNIX tool which is passing a set of arguments and Kanyon you know information into the tool and then grabbing the output of the tool and processing in some way that's appropriate for the UI so you'll notice here that the only thing that this thing really does is have the process has started ok let's create some strings to stuff the data in and the process is finished let's create some arrays to stick the output in but we still don't have the glue ok we can get the output we can tell them what the input is but what's in the middle and that's what my process class is for so cocoa provides in its foundation class a couple fundamental tools that can be used by bsd programmers NS pipe and s file handle and s task and there's a bunch more I expected you know all of you will go home into your homework and start playing around with them in the case here if you remember earlier we we created our array of output or our array of paths and everything and put them or our array of control and put it into an arguments that we passed to init with controller for the process class and here you can see exactly what we do we just retain those arguments and we create a right to pipe the right pipe you see there it's NS pipe I pertain creates a pipe that we can use to pipe to standard in if you look down into the next major function you'll see when they click on start when they click on toggle tool and it begins we gather all the arguments we tell it to start the process we create an NS task the one of the main purposes of NS task is to wrap this kind of work for you so that you don't have to do interesting neat but boring things like fork exec and sitting around and waiting for output and as task does it all for you and provides all these tools in an object-oriented manner so that you can access them you can set the standard output you can set the standard error you can set the standard input in our case like I said earlier same file presented a interesting case because it needed standard in vs. most things was just passed everything on the command are a lot of tools that pass everything on the command line so we've got a pipe for four standard in and we've got the same pipe for standard output and standard error we tell it the launch path you'll notice here the arguments is object at index zero which is somewhat hard-coded in this case but if you remember back when I created the array the first thing that I passed was the path to the same file binary so my launch path is the entire path to the same file binary the little bitty UNIX tool you're building up the command line building up the command line right here the next thing I do is set the arguments and I want everything else except the path to the to the UNIX command line so I subarray with range and I start at one and I just go to the to the end so this one line right here will easily allow you to just keep on adding on arguments - B - D - for 1000 or whatever the arguments need to be to that UNIX tool you can add them on without having to change your process class in any way whatsoever you then need to register with the Notification Center and if you go in and look at NS file handle NS pipe and NS tasks you'll notice there there are some standardized themes throughout all of them but one of them is the fact that you can register with the Notification Center to see when something's done to see when a task terminates and that kind of stuff here you'll see that I'm registering for NS file handle read completion notification when the file handle has received some output let me know we're basically wanting to know when all the output from the cool sure women when the tool start spitting stuff at me I want to know what it is the next thing I want to do is I want to tell it to go and that consists of two parts in this in this situation we tell it to read in background and notify that tells us your job is to wait for background notification and then tell me and then we tell it much pretty simple stop process is even simpler we tell it okay remove my notification for reading and let's terminate the task the only other interesting thing in here is what happens when I get data when I told it that I wanted to register for notification I gave it a selector to a method notify me when I get data by calling my get data method and you notice here all it does is it takes all this stuff and then it passes back to the controller that I registered at the beginning append output so we can get the output this class does all the work the only other I guess this is another small but interesting thing you'll notice here the insert string method this is the method that goes to the right pipe and and does standard in for all intents and purposes to our little UNIX tool and all it does is it writes the data in utf-8 encoding and then the pass is a /n newline on so that it knows that all the writing is done and then when we told it to closed writing at the very last thing control D as I explained earlier we let's close our file we're done we're done piping stuff to standard in go off and run and do stuff 99% of what you just talked about is basically standard for talking to any UNIX application there's nothing unusual about this particular tool that we're running it's basically there's a hint a couple of small methods that you use to be able to pass information to the background tool there's a small number of methods to be able to read it in and parse it and some basic stuff for wrapping the management of that particular task but the stuff that you showed me so far as doesn't have anything really at all to do with your particular know this is very tool agnostic if you if you know the we ship some applications that are based on the same style of concept network utility for example uses multiple different UNIX tools to do the same type of work and this is this is the easiest fastest and most productive way to do this kind of thing so now we know we have the controller that gets the information from the user and tells that tells the process to run the process runs and can tell the controller the output the only other interesting thing left is back in here we have a tool so simple tool which was our parent class knows how to put output into a table view and for this specific case it's a little hard coded because it's a demo basically all it's doing though is just taking the output and formatting it up for the UI that's it takes the output it creates a row or you know an array of rows those rows contain a raise of output the size of the file the first file name of the second file name and this is a table view delegate which if you look back here in the UI we had our table view and our table view needs to be hooked up to a delegate and a datasource now what that does if you if you're familiar with a bi programming a datasource provides data to an object that requests a table views browsers those kind of I guess you could call heavier weight cocoa objects quite a few of them have data sources delegation is a concept we won't go into here but it's a it's a way of having an object perform actions on the behalf of other objects our table view needs to get the data from something and so we've set it up as is that it will get the data from our it just knows to ask your it knows to ask my class for the data when I'm ready to have it draw it only matters there's only two methods that that truthfully matter and it's the table view wants to know how many rows do I have so it can start setting up its scroll bars know how much amount of the amount of room it needs to allocate and so on and then what value do I put into this row for this column and you notice here if table column is the size column then I want from my row array the very first indexed object I want my size you can use table column identifier to get much better granularity than this in in slightly less hackish terms but this will do just as well okay I'm in management I like to see the end result show me end result so let's build our project here and he's a management you all notice that built with no warnings no errors so let's go ahead and select a directory and this should all work let's select a directory that we know has some duplicates otherwise this would be very boring you notice here I had to select directory I filled in the path I could have typed that path in as well but gotta show that the Sheep works let's see if this works ooh there it goes and let's not a waited long enough so you'll notice here what it did is it went off and it grabbed all that output is essentially the exact same output with some typos errors it looks like stuffed in there probably parsing it's that's bug fixing this is feature complete but not bugged finished you'll notice here that it went off in it noticed that I have a same file binary inside my app which is what I requested and if you look here in the second column there's also a same file binary in both the root of my app which I copied there earlier on accident and a same file binary in the same file directory of what I downloaded you'll notice that they're all exactly the same and in this case they're all the exact same name and everything but if we would have renamed one of them it would have shown it up anyway and done its magic if we had turned on the verbose flag in this case we don't have any place for the verbose output to really go we could have dumped it to standard out but if the verbose output doesn't really belong inside of a table view the next steps for what you could do in here are things like adding in flags for size the - s flag I believes the same file then you put in 10,000 that means I don't care about anything underneath 10,000 bytes so all it would have shown were these first two rows that would have literally been four more minutes worth of work but my time is over Thank You work I think the important key here is there's as you see was a bit of work around kind of setting up the standard structure for being able to talk to a background BSD application in terms of being able to manage sending data to it and getting it back but cocoa does an enormous amount of the work here those are being able value to parse the arguments and frac you know tear them apart and basically send them to you I at this particular point it's UI polish and that's really is the fun part but the neat thing is being able to take you know something like a BSD background application and even just starting with this as a starting point being able to support just about anything you can imagine that runs there and this is of a particularly valuable and important feature for Mac OS 10 of being able to leverage these kinds of capabilities and tools in the background of your application and being able to provide them forward to our end users in such a way that doesn't require them to learn you know something like Emacs or the command line so we can move on back to the slides thank you so it should be kind of obvious you know why it is we did BSD in here it is it provides an enormous amount of power there's a lot of leverage that's possible from being able to have a system you know some capability like this as part of Mac OS 10 there's a little bit of history though that comes with with some of this one you know certainly you know picking bsd over maybe another solution is that one this is pretty tried and true technology it's been around for a long time there's also a lot of history here for that comes from our acquisition of next part of it is the technology that was used as part of next step which is actually based on BS 3 bsd 4.3 and that evolved along through open step until an internal project it next called argon which was the process that orion modernized that particular component the system they recently moved the bsd subsystem to bsd for four so there's a bit of history here in terms of how they the evolution has occurred and as part of our rhapsody ins of course part of as mac OS 10 bsd that's part of the system is based on basically some net bia sources that are part of the BSD for for derive source base and then most of our user user land and library space is either based on FreeBSD or it was in something that we're working actively to sync up with current and active versions BSD also benefits a lot from much of the other technology available and that it is inherently for very scalable you may or may not be aware that the servers that are run Yahoo are all based on FreeBSD he's a pretty robust and powerful technology and that's certainly something that we consider to be a valuable you know set of features and capabilities we want to have as part of 10 also BSD is something that's incredibly well known there's a lot of books and knowledge and inherent community knowledge behind the technology and that there's a very important part of the decision process for us and obviously the fact that it's open source is another key aspect the open source part of how we deal with the operating system technology in 10 is a very important part of how we do business it's not just AG this is a great way for us to get technology but we're actively working with the community at large to evolve it and the nice thing about the BSD technology is there's our great open source community an active developer community already in place that we can partner with and given that I'd like to invite Jordan Hubbard who's the founder and core team one of the core team members of FreeBSD to talk a little bit about FreeBSD Swick yeah so as Brett said they incorporated quite a bit of freebsd technology into Darwin because several years ago and the project was first launched but I don't think that collaboration certainly ends there and there are a lot of interesting things that we are doing at FreeBSD and have done and I think that there's quite a bit of room for collaboration in both directions so I'll talk about it a little bit from the FreeBSD perspective so one of the nicest things I find from a developer about FreeBSD is the source tree organization and that may seem like kind of a small thing but it's actually of tremendous value for a lot of reasons one thing it gives us is a common taxonomy it gives us a place for everything to go it gives its place for new things to be incorporated into the tree without having to argue for four months about it first and it also means that there's a one-to-one correspondence between where binaries are for example bin LS you know it's going to be an user source bin LS so it's very easy to find stuff and it also segregates all of the code that's for example under the GPL or as export restricted into concise areas of the tree where you know exactly where it is and you can firewall it off if you're making a product that requires the BSD license for example or you're exporting it to Croatia and then there's issues behind that the source tree also handles it encodes all of the bootstrapping issues so that if a binary depends on a certain library it is smart enough to go boil that library first before building the binaries in question and that removes a lot of the burden on the developer of figuring out exactly how things are tied together and that that's it's very important we also have a distinction between vendor supplied software and stuff that we maintain ourselves we use CVS very aggressively and one of the things we do for example is when a new version of GCC comes out which is a vendor supplied bit of software we import it on a vendor branch and we have all of our own changes on the project branch so we can keep the two change sets distinct and so we know exactly what the new people have changed whoops my microphone fell off I'll just hold it we know what the new people have changed versus what we've had to change and and this makes ongoing maintenance a lot easier also we provide a series of targets in the source tree like world and kernel and update which remove a lot of the the hair of knowing exactly what order to build things in again so it gives us a common reference point if we have developers who are complaining that they're seeing some strange and almost behavior we can say well when did you last build world and we know exactly what that means we know that it's they've gone and they've updated all of their binaries and include files their libraries and so forth from a common reference point it also makes it easier for newbie developers people who are maybe not so familiar with the project that may be very skilled at C or whatever but they don't want to have to learn the ins and outs of our build system they just want to grab the source Street type make world watch the whole thing work and be done with it so these are all things I hope that well eventually in some way shape or form migrate over to Darwin as I say we also using CVS very aggressively have come up with a number of interesting ways of distributing the bits we have something called CB sup which understands CVS natively that means that when you run it it pulls over just the deltas that have been added to the repository since the last time you ran it and that means that it preserves your own local changes so if you have a local branch in your own CVS repository it's not just going to come and splat a file on top of it it's gonna interleave the deltas together and that's that's very valuable it also uses our sink and some other advanced protocols so it's very fast and very efficient and if you run it on a daily basis it takes maybe 5-10 minutes to run and you're completely synched up it also understands how to check things out so you can use CVS up to checkout a branch of the repository so if you don't want the repository bits but you want a certain branch of FreeBSD you can ask CVS up just to give you that and keep it up to date we also support some esoteric methods of getting the bits like CTM which actually bundles the patches together and check sums them and sends them through mail so you know some of our developers in Estonia in places like that who don't have good hard wired connections or who have very infrequent connectivity can sync up that way and then of course we support the classic and on CB s methods and we also have a CBS web interface so that you can see very colorfully what's changed at any particular point we also support multiple branches of development as I said earlier we have stable branches which are sort of more tried and tested branches that we aim at the the end users and we have current which is the bleeding spurting severed artery edge of development which is guaranteed to hurt you and so we aim the developers more at that but we do supply both so this means that we don't have to freeze out developers from doing active and in some cases experimental development that development can always occur at whatever pace it wants to but the the people the yahoos of the world don't have to suffer from that they get their changes back ported from the current branch at a much more slow rate of so slow pace and they also get things tested you don't just bring things immediately over into the stable branches there's a there's a methodology that requires a certain amount of testing and integration on the flip side it's also a real pain in the butt to back port stuff so and CVS isn't tremendously good at holding your hand in that so it's that's an ongoing project cost that we incur nor is it very easy to synchronize things across repositories so we have had a lot of initiatives that have kind of fallen flat to share code say between net bsd and freebsd or previously and open BSD because they have stuff in their repository we have stuff in our repository and it's just it's just a lot of pain to manually bring those changes back and forth so we're looking at ways and perhaps automating that somehow putting pointers in CVS for example that's whenever you make a change to this particular subtree it gets bundled up and sent over to a neighbor repository and automatically checked in or something like that there's a lot of low-hanging fruit I think also in the freebsd source tree that the darwin community can benefit from we've added a lot of creature comforts to our user land you know just for example our ftp client does file name completion so whenever I'm on my OS 10 box I'm always walking tab and kind of browning it but it doesn't work so things like that would certainly make a positive difference we have a lot of really interesting libraries that front end things like fetching bit so Richard FTP or HTTP that make writing clients a lot easier and then obviously we have some of the large-scale subsystems and whatnot OpenSSH OpenSSL get updated I think on a much more periodic basis we have a lot of security work that's ongoing we've added POSIX ACLs for giving you much more granular access control than simply the one big route and obviously a lot of ongoing auditing work as well and I would like to see those changes getting into Darwin and on a more frequent basis as I'm sure the Darwin people would as well finally it's also even though we do use a very different driver model Darwin using i/o kit and that's using new bus I think that obviously having working drivers to look at is a tremendous benefit to Darwin developers and I hope that they will be we'll continue to use them so getting away from the bundled apps we also have the ports collection which you've heard referenced a couple times already I started it in August 20th 1994 I just wanted to kind of prove a concept which was that you could create an expert system as it were for porting software that is to say I found whenever I went into a new box the first things I do is I bring over Emacs I bring over bash I'd bring over a couple of tools I was familiar with and that involved of course remembering where they were where the reference repositories on the net were FTP unpacking them configuring them building them installing them in the whole nine yards and it occurred to me that you know there was kind of a very common process running across all these different types where and maybe I could encode the smarts for doing that in addition to any patches that I might need to make into some sort of taxonomy and this became the port's collection I think by the end of the year it was 200 ports or so and today it's over 50-100 ports in 52 different categories there are languages localized versions of software German Vietnamese you name it it's just a huge huge collection of stuff and it grows at a rate of about 50 percent a year so far and there's there's really no end in sight unfortunately there have been some Forks of the ports collection the net bsd and the open BSD folks have taken them made some slight tweaks and the open packages effort was launched by daemon News in an effort to kind of bring all that back together again and create one common standard for the port's collection and so that they've been doing very good work there and I believe they've gotten Darwin to sign on as well and so Darwin will be substantially leveraging a lot of that effort as I said what the port's collection does is it automates the fetching the configuring the building installation and all of the dependency checking so if you build some little gnome client for example it goes off and installs reams and reams of stuff goes off and gets libraries gets a support infrastructure a tremendous amount of stuff basically drags all of norming with it and you don't have to know about all of that and it's a tremendous time saver it also is the way that we build all of our packages because you've got the dependency information already encoded in the port's collection it's very easy to pass that onto the package system which was actually written after after the port's collection was done and it gives us a great way of automating the package building we have a cluster of 16 or so machines which do nothing but 24 hours a day build all 50 100 packages and put them off into an FTP site and you can you can access this very easily just by doing package add - our name of package and it will go off and figure out what version of the operating system you're running what the closest FTP site is and suck it down along with all of the dependencies so if you don't want to build the stuff from ports and go through all the compilation time you don't have to do that we also use the packages to do more than just store the results of Port building they're also a great way of encapsulating other binaries for example the Linux Netscape ports and whatnot are really just encapsulated binaries that go off and make sure that all the dependencies and stuff have been properly resolved so they're useful for a wide range of things so like I say it's really great that it saves a lot of time and makes it really easy to go build really complicated pieces of software without having to know anything about the dependencies and or where the tar balls are stored or what particular weird configuration arguments you need to use it's all and pretty much hands-off what sucks about it is all of this is encoded in Berkeley make and if I had it to do over again I probably would have put all the meta information in XML or something and written a tool which just extracted the information and built the make files on the fly or something so I am actively looking now at some way of doing a imports next generation effort or something which which makes it a lot easier to build documentation trees web browser Bowl pages and whatnot which describe the ports collection and that's something I hope maybe we can do hand in hand with Darwin so I think our community is at least as important as our bits obviously we've been around for nine years and we wouldn't have been around this long if we hadn't had some ways of keeping the whole thing coherent which is now a democratically elected body that is chosen by the majority of committers and committers are people who have right access to the CVS repository so if you're a committer you can stand for election and every two years we choose a majority it's a pretty good system and I think we should try it in this country sometime anyway we have 267 CVS committers now who are segregated in three different categories the docks people who just work on docks people who work on the bundled sources and the people who work on the port's collection so you don't need to have be a cross-disciplinary person you can choose an area to specialize on and committers are approved by core so we're not quite as it's not quite as difficult as it is to become a Darwin developer for example but you do need to have the unanimous approval of Corps before you make it in so there's some level of quality control we also make allowances for sort of collective versus individual ownership in in the large picture FreeBSD as a project owns the sources and nobody can really put an exclusive lock but we do have kind of a non exclusive lock model where somebody says hey I'm really maintaining this everybody should should lockstep their changes through me because I understand it better than anybody else and so we have maintainer bits that allow for that and this works very well actually because it lets people feel a certain degree of ownership over it but without destroying sort of the collective ownership of the sources which which keeps the project together and I think it's really important to to get developers hearts and minds if you want to keep them interested over the long term and what I mean by that is you can't just get a pile of bits thrown over the wall periodically from some developer who doesn't buy off on your vision or your overall strategy because that pile of bits will just hit the ground with a dull thud and start to smell immediately we tried that with with bsd OS and some other groups where we're code was sermon over the wall but we didn't get the developers that came along with those bits and that was a critical mistake in all cases and in such cases those projects essentially died off so if I could say something to the Darwin community would be it's not just about the bits it's about communicating your vision and really getting people to buy into it and that's what gives you longevity so our identity as it were is also determined largely by the users and this was kind of a shock to us we started off as a bunch of hobbyists with a computing problem and we did it really for our own own edification and and ego gratification and whatnot but what we found is that we have pretty much been driven over the last five or six years by what the users want because they all very loudly for it and we have some pride in our work and so we have really become extremely user driven the yahoos of the world pretty much tell us what to do they ask nicely because they've learned their lesson but they do essentially drive our direction and so our vision is essentially nothing more than listen to what the users ask for and try and fulfill that as best possible and still try to have fun obviously and people will still work on things that interest them the most but if I had to point to the overall direction it would be listen to the users and I think that's that's what's going to happen with Darwin as well the needs of OS 10 will dictate Darwin's direction if they follow a parallel line of evolution so where I think that FreeBSD project can work with Apple and an ongoing basis is one I think that Apple is helping us to shatter the desktop myth which is that BSD or FreeBSD are only good for servers because they run Yahoo's and hot males and companies like that that it's no good as a desktop OS and we've certainly known that that's not true but Apple is helping to really drive the point home with a very large hammer and we appreciate that it's a great foundation for a desktop operating system and I think that having the same OS run on your server and your desktop also means that interoperability is more or less assured you're gonna have the right tools for doing network communications between the two it's going to be much easier to administer a heterogeneous collection of servers and desktops and I really think that that there's a strong case to be made for that for one operating system being both and also paradoxically I think Darwin has more freedom to innovate now than we do we have been around for nine years we have a very stodgy and and somewhat conservative user base and as I said they tend to drive our direction so we can't make any sudden u-turns or or swerve in in in violent directions without pissing a lot of people off whereas Darwin is still comparatively young and they're well firewalled in some respects from OS 10 by this nice GUI desktop and I think that Darwin has already done quite a bit of interesting stuff with IO kit and and even the stuff pulled across your Mach that addresses some of the old problems in very new ways and that's something that I think we can learn from Darwin can learn a lot from similar methodologies and they can learn a lot from our technologies but I think we can learn the most from Darwin's innovation and I look forward to working with them in the future thank you Thank You Jordan so kind of in closing here again you really need to realize that BSD is an amazingly powerful part of Mac OS 10 we've done a really good job of hiding it but from you as a developer it's there it's something you can take advantage of and really should however you should be careful about exposing the you end-user to the unit user user experience it is a pretty different world underneath the covers there this is a very powerful tool for you to use as a developer it's a powerful tool to help leverage new technology into Mac OS 10 but it's up to you as the developer is to be very cognizant of keeping it Macintosh in terms of the ease of use and the kinds of things that you would expect on a Mac and there's a very interesting juxtaposition there are goals it's also important for you to realize that many of the BSD parts of the system are actually optional in terms of how they're installed so if you're writing an application on Mac OS 10 and you're depending on bits and pieces of BSD make sure that you include the pieces that you need to and that you know what's actually installed kind of in the base system install the BSD package that exists and install installer may actually you know may not be installed by the user and in some environments may they may decide not to so be very very careful about making assumptions about what pieces are actually in the base install of the system and there are things you can do like packaging your tools as part of your application to help protect you from that and we're working on additional things for you know long term for being able to do things like package management and stuff like that to help out but it's something you should be very conscious of make sure you leverage Darwin I think this is an important part of Mac OS 10 and I mean Darwin from the open-source perspective this is a pretty new paradigm for for the Apple community to use and we something that we consider to be a real advantage that Mac os10 brings to the table take advantage of it also leverage free of the freebsd community there were very much part of our extended community you know Jordan you know and and his team really is you know brings a lot of experience and stuff to this aspect of the system and we're really happy to be partnering with them and being part of that community and would it very much encourage you to leverage some of the knowledge and technology and that exists there become part of the community we have a vast email lists and other services and capabilities that exist in under the open-source website and as part of them that developer mailing lists and stuff that you can really gain an awful lot by being part of the Darwin community and barded the bsd community so you know help join and be part of it to get more information on this we certainly have a lot of information around the open-source aspects of the system that are on our open-source apple.com website there's also if you're interested in learning a lot about unix and and the bsd pieces a handful of books out there that are pretty good one of you know the things that i would recommend if you're understand learning about the bsd internals is getting the design implementation of the bsd operating system by kurt mcusic he's certainly one of the real the key Keefe if you will defending fathers of bsd and there's a lot that can be learned there's also a number of our riley books on all the various technologies that exist in the covers as well as a freebsd in it and SBA's net bsd organizations again they're part of the extended community if you also understood and learning more about the ports collection just this week I know one of the vendors actually has available at the Mac tech booth a four CD set of ports that have already been imported to Darwin and Mac OS 10 so there's already a lot of community activity there encourage you to go grab a copy of it and lastly later this afternoon if you're still hanging around we have a feedback forum at five o'clock we'd certainly like to see you be part of you