---
title: WWDC2001 Session 103
framework: wwdc
role: article
path: wwdc/wwdc2001-103
---

# WWDC2001 Session 103

## Transcript

Kind: captions Language: en it's been it's been a great year for Darwin and opensource starting with Mac os10 shipping on March 24th and we have great contributions and signed up for Darwin open source and we'll be sharing a lot of accolades in the numbers with you as well throughout the week so without further ado I'd like to kick it off by introducing the director of Korres engineering Brett Haley so good afternoon the first session after lunch hopefully you guys won't fall asleep on me off we have a lot of technical material in this session so I will try and get over it the focus of this session is to you know basically give an overview of what Darwin is all about and I'm sure you've seen this picture a numerous times that's just this morning and our focus again really is on that bottom layer of the system and there's a huge amount of technology that exists there as we talk about it today I think it's important to note that you know we will use some terms quite interchangeably Darwin is the core OS of Mac OS 10 we talk a lot about Darwin we talk a lot about Pakistan they are the same thing where they are not two different piles of Technology they are one in the same and from that perspective it's important to note that the Darwin is also open source so as we talk about the open source aspects of Darwin and the technology it's the same pile of code the same technology that ships as part of Mac os10 again they are one in the same so how in the world do you describe an arm you know the core OS or the operating system of Mac OS 10 interestingly enough it's technology that tends not to be very visible if we're really doing our job well you don't really see us so you might consider that you know we're certainly kind of the foundation of the system we're you know kind of deal with the plumbing and the piping and the you know wiring if you will that holds Mac os10 together again if we're doing our job well you don't see the core OS you as the developers will interact with very you know at a very high degree of interactivity but the developers should rarely if ever be aware that the core OS is there the architecture of darwin is a very unusual in that we've you know taken a basically best of class technologies and pop-rock them together to bring in the base of mac OS ten together and there's a number of different technologies that we'll talk about in this session this technology comes from a number of different areas academia the open source community and certainly some significant history at Apple and next and there's probably thousands of person years worth of development and testing and usage in much of this codebase but we've taken all that technology and we've again kind of brought it together in a very unusual combination but tuned it toward the customer base and developer base that's unique to mac OS 10 so you've certainly heard a lot of focus around things like UNIX or certainly the heritage Mac OS we've actually brought aspects of this this technologies together to support both of these environments and to really focus on our customer base directly and from that perspective there's an awful lot of untapped potential that exists in the system we'll talk a lot about more that what more of that during the week of course but in terms of providing that modern operating system base this is it this is the basis that you know really enables all those capabilities and because of the power that exists there I think we're enabling you to create applications that you couldn't do on previous versions of Mac OS from a feature perspective you could consider that Darwin or the core OS is responsible for the preemption aspects of the system the you know memory protection the basically it's responsible for managing and coordinating the multiple address spaces that represent the system each of the applications in the system exists in a separate address space it supports the application environments provides all that plumbing and wiring and stuff that allows the application environments to exist on the system and to be able to do there export their particular brand if you will of memory management and such up to the application the core OS is inherently device independent and what I mean by that I mean that seems kind of an obvious statement but that we don't actually have any particular preference to particularly any I don't know an i/o or device architecture we realize that you know technology in the i/o space is evolving at a fairly furious rate and we've tried to make sure that we have a design that will enable us to move forward and be able to take advantage of the fact that we make the whole widget and to be able to you know enable new hardware and new technology together new software technology it's a very flexible base if you as we talk about the pieces of the core OS you'll see that they were designed to support a number of different kinds of categories of application spaces and Hardware spaces and and such and we've really kind of tried to leverage that flexibility of the system to give us a base which we expect to last for the next 10 20 years the course is also designed to be scalable and you can interpret this in a couple of key ways it's scalable from the hardware's perspective in that we you know support certainly various classes of you know processor architectures g3 g4 but also be able to support multiple processors in a system so support for symmetric multiprocessing from that level of scalability as well as scalability from the i/o perspective being able to support you know the storage network devices and large you know very large capacity devices on the system or high bandwidth networking with Gigabit Ethernet for example but it's also scalable in the kind of customer set that we've tried to support the same base Darwyn core OS is used for our client operating system Mac OS 10 and Mac OS 10 server 10 point oh that was announced this morning so the same operating system core is used to drive both the server and the client environment and that the system itself basically is tuned you know by those environments to kind of provide the appropriate level of service and capability and being an OS and things like networking and all that there's more buds worse than you can imagine that's part of the system so we're gonna spend a little bit in actually do a tour of the chorus and the various components and that are part of it and you know first of all this kind of gives you a bit of an exploded view of that Darwin layer of the system and there's a number of pieces that make it up we're gonna start first by spending some time and talking about mark mark is kind of the actual foundation layer of the system it's designed in fact as an OS foundation and abstraction layer if you will and it's responsible for managing the processor and memory resources of the system it's basically the piece of the system that actually schedules the processor and give an application or thread in the system for being able to run and it's important to note that at this very very basic level of the system the threading is that threads are considered to be you know an essential building block of Mac OS 10 that scheduling and the various services that go with it or actually supported all the way down to the very base of the system it's not a tacked on the top concept mark is also responsible for handling the memory protection the memory management basically if you will they both the processor and memory resources of the system all the VM capabilities and such because mark keeps everything abstract it as far as how memory is concerned and the various address spaces that exist in the system it also has to provide some very key and basic services so that there's a way for the user level applications and the the kernel to be able to communicate back and forth together and that's done through something called IPC inter-process communication RPC remote procedure calls that is a fundamental again key building block of MOC and it's the way MOC uses to be able to communicate with any other portion of the system Marc is inherently policy neutral it's again it's designed to be a very much a an abstraction layer to the processor and memory architecture of the system and it doesn't actually have any concept of things like file systems or i/o or networking or anything security policies things like that that's not mocks job it really is designed to be a very low level abstraction layer the mock we use in Mac OS 10 is based on for those of you that follow this technology is based on a mock 300 plus a bunch of stuff that has been added to it by the various players that have been participating in its development it's derived from mock that was developed at CMU avi tavini and of course was one of the original team members of the of the mock team at CMU but it was taken on by the OSF reach Research Institute a number of years and in fact Apple funded a lot of that development work at the Research Institute and and when we took it in and incorporated as part of Mac OS 10 of course we've done considerable amount of work to enhance the the 300 base beyond even those starting points it as fare very much from its original design point for MOC was to actually be able to support things like SMP and some of the work that was done in 3 oh and later also has architectural support for being able to deal with various mechanisms and in facilities for supporting real-time and we'll talk about that a little bit more and certainly more in the kernel session later this week the wall is talking to us the markers also has built into it because it's responsible for managing the processor resources the capability of supporting multiple scheduling policies for example built into to the market ships with 10 is the ability to schedule certainly either kind of a real-time scheduling or fixed priority based scheduling or time sharing based policy scheduling and there's actually the ability to support other types of scheduling policies within the system Marc also tries to abstract the mechanisms for being able to deal with the other half of the virtual memory system so VM certainly if you look at the management of memory there's the managing the actual you know parceling out of the address space that's available for a given application but there's also the other aspect of it which is what to do when you try and access more memory than you actually have physical RAM installed in your system for and MOC actually interacts with a number of external interfaces to be able to support being able to store to be able to do handle the backing store for the OS either on the file system or and certainly it's capable of other kinds of backage towards backing store solutions Marc is also again it's kind of repeating here but it's very much designed to be very modular MOC was intended to have OS personalities put on top of it again it's policy neutral so it's you know it's interface layer it's abstraction layers of stuff that it assumes that there's some other OS and other RS technologies that sit on it and that's very much a key part of its design space now this is all fine and interesting this is all down at the very base of the system but if you're writing an application for Mac OS 10 what does MOC mean to you well it's important to note that all process models of the system in other words the things that actually kind of handle the scheduling and the kind of an environment which your application rests within are all fundamentally built on top of Mark Priven primitives those mark primitives are tasks and threads and we're whether you're using carbon or whether you classic or you're using Coco they all of those application environments are fundamentally built on these particular abstractions all memory management is also built on top of mock primitives so at the very lowest level again there's management of Hewell VM objects and the handling of virtual memory and the paging services and at your application you're probably dealing with things more like malloc or you know other kinds of services get pointer or new pointer whether depending on the type of application that you're using but fundamentally when he gets right down to actually managing the memory at the OS level Mach is responsible and again as I mentioned before with Mach built on this concept of inter process communication that all kernel to user and user to kernel and user to user communication and then what I mean by user to user is user space from a more process architecture processor architecture perspective well all that communication that exists in the system between applications and between applications in the OS are done built on top of these mock primitives basically inter-process communication RPC or what you'll end up hearing a lot in mention and number of the documentation is mock ports now to get into this a little bit more and give you a sense of exactly how the system actually uses some of these facilities I want to invite Joe Sokol up on stage and what we're going to do is show you that as right you're running the applications on your system that there's actually the ability to look at the system much more from a if you will a systems perspective to see the kind of the impact of your application that at the OS level okay so many of you are probably familiar with that's not the one with the applicant with top running on UNIX which is capable of showing you a lot of stuff going on in the system what we're going to do here today is just kind of take a look at a simple little carbon app because that's kind of the runtime environment that isn't quite so used to running on top of Unix and to show you that in fact just because it's carbon right it actually is making use of a lot of these mock services so you're going to see the fact that they're threads they're ports being allocated on your behalf and we're going to take a little look at some characteristics of performance or lack of performance that would like you to avoid okay so you've got W so the little app is called this white next event loop and the first thing we're going to take a look at is what it means to have an app that is basically not only compute BAM but basically running through it take a slightly different view here a lot of system calls ok so the weight next event in the carbon runtime is actually implemented with a you know a few mock calls and a few BSD calls but when you amplify that if you're just sitting in the loop doing a wait next event with a zero timeout basically you get really really busy on the system now this has basically three bad aspects to it the first one is kind of obvious in a time-sharing system our little test app here his priority is diving way down and it's going to stay down because he's never blocking so you might not like the responses responsiveness you get in this app over time secondly one that you might not think about so much as you're keeping a number of pages hot all the time because you're constantly executing them so those are pages that the VM system might have been able to you know steal and actually give to a foreground application that really needs them so it's continuing to put pressure and competing for for memory and then the third thing is all of our power management is actually driven and triggered off of our Idol loop right which is an actual mock thread that goes that gets run when there's nothing else to run so that is where we make the decisions to put the system into nap or doze mode you know to reduce the power consumption not the deep sleep but these other modes that allow us to cut back on on power consumption and if the application itself is basically implementing an idle loop you know that does not allow us to enter into those modes another thing that we can look at with top is some of the other resource utilization so again you can see that this is a very simple dumb you know hello world style Carbon application and he's managed you know we've managed to allocate him two threads and 61 ports so these are you know things that are happening as part of the carbon runtime model so another thing we can show is let's go and cut down on the amount of when we flip back here real quick sorry this little bit easier to see in terms of the CPU utilization and the number of system calls we've now simply made the wait next event wait for one tick so instead of it just being strictly pulled even just going to one tick wait really reduces the load on the system if we go to something that's a little more approaching infinity then of course it goes real prescient so even if you can't construct the app to be purely an event-driven model if you have an event loop if you can at least make the times that you're blocking for long that really helps in terms of relieving pressure in the system alright so now we want to show the a memory leak so excuse me so again top can be used to see things like memory leaks going on in the system you'll noticed under under the art private which is the resident private column that all of a sudden we have a little plus sign sticking out there on the right and what that's indicating is that that actually is a growing size there so you can see the size going up slowly this is basically a fork a leak once a second now if we were to be leaking every time around that event loop we don't really we don't really enforce any kind of real upper limits on the amount of memory that an applicant obtained so it is probably a good idea on your part to watch your app and make sure that it doesn't have any leaks because over time you know maybe the app stays up for a long period of time of course that would just keep going a larger you know going longer and lot growing larger and larger over that period of time to the point where you might run out of paging space on the system so an app can approach three gigabytes and size thank you back to slides here so I think it's important to realize that even though you may be thinking of your application in terms of the carbon API space or if you're writing a new app in Cocoa that there actually is a lot going in on in the system under the covers and that there are actually tools that you can use to be able to tell what the actual resource limit usage in the in the affects if you will of your application are on the rest of the system if you're interested in learning more about the Darwin kernel there's a session later this week on Friday that I encourage you to go to over in the civic center and this is this particular session will not only cover the mock aspect of of the system but will also talk a bit quite a bit about the bsd kernel as well moving on in our tour now into the i/o kit space io kit is an object-oriented framework that's basically designed to try and ease the process for developing implementing drivers on Mac OS 10 it has a lot of kind of native or inherent capabilities that are part of that framework support for things like plug and play and and dynamic device management and of course again those things sound like you know mom and apple pie kinds of things but it's very important to note that again from the very basis of the system that the ir system is intended to be able to support devices that can come come and go as far as the system's current concern that the i/o system itself has no inherent policy in terms of the you know kind of the arrival or that or the departure of given devices on the system that policy is actually managed up how much higher you know in the system space for example whether the the file system has a dependency on a you know volume that happens to exist on a particular disk but IO kid itself supports the ability for devices to hit to come and go in the system and is a very inherent capability another important aspect of i/o kit that's that's really there from the core is support for power management is that expected that all devices in parts of the our system participate and are involved in the process of managing power on this and we'll talk a little bit about more of that in a minute in a minute and as well I okay it is intended to be very modular and extensible we don't know what kind of devices may appear five 10 years from now and we've tried to make sure that the basic architecture for our kit can be very flexible as new i/o and our device technology come on on the scene in addition to that basic capability of course il kit does support and provide the abstractions for all the common you know class of devices that exist in the system things like scuzzy and 18 USB and stuff like that and those are all done through another aspect of the framework called families and we'll talk about that in a minute as well again iokit is a framework it's actually an object-oriented framework c++ based and it provides basically the common kernel services and facilities for being able to support IO in the system and it also has the facilities to be able to enable the communication from the kernel to the user space so that the actual IO can get up to where the applications can take advantage of it in many cases that's done through all the layers of abstraction but it the inherent capability for being able to do that is built into I okay I okay if in effect models the physical world if you will is you'll you know think about your computer there's a motherboard and there's chips and there's potentially a PCI bridge and potentially PCI cards in a graphics card or a scuzzy card that that's in there and I okay it actually is responsible for being able to develop that model if you will of all those devices in the system and then once it has that model being able to actually connect up the appropriate driver to that given Hardware so that the rest of the system can actually talk to it the protocol specifics for any particular class of devices and something we call families a family basically is in a if you will a domain appropriate abstraction for a particular device a good example of this is if you would compare something like scuzzy and audio they're pretty very you know very separate very very radically different approach you know use if you will on i/o you're probably not as interested in setting the volume of your scuzzy discs you know for example and so the families actually provide the appropriate API set and capabilities for that particular kind of device space the actual device specifics themselves exist in drivers so that's the the part of the system that directly talks to the hardware and they in and of themselves then communicate with the families and then the families with the rest of the system as I mentioned power management is a very key part of macros Tanner you guys were at the keynote this morning and saw obvious demo and it's it's really quite amazing to see something like the titanium you know effectively be a week you know before you can open it I've heard a number of you know refrigerator door kinds of stories of you know trying to make sure is it is it really off or or not but it's because of the the i/o the architecture for power management that that actually becomes possible it's integral to the i/o kid architecture and it's based on this concept of power planes much like I mentioned that i/o kit tries to model the physical world from the power management perspective there's another model that models kind of the power distribution in the system and they may not correspond in a one-to-one basis depending on how the hardware is designed the interdependencies at a power level actually may be different but Iook it's responsible for trying to keep that abstraction in that that set of relationships together it's important to note that for Mac OS 10 the way we were able to get the speed and performance at of wake up is that because of the full architecture of both yoke it as well as the rest of the system that this is the kernel itself is multi-threaded the system itself is multi-threaded and that the wake up process of waking up the system is actually done as much in parallel as possible and as a result you're able to actually enable the parts of the system that are necessary to get that application up and going as quickly as possible as long as it doesn't have any inherent dependency on you know the disk or a networking device which may take a little bit longer to come up to speed you're able to get up and use and interact with your system in a UI level very ver quickly if you're familiar certainly with mac os9 in waking up sometimes it can take a pretty substantially long period of time and part of the reason for that is the fact that the system is very seriously dependant on making sure that all those components actually are fully awake before you can access any part of it in addition things like Apple talk for example have to go through the process of renegotiating if you happen to have Apple talk enabled on nine have to go through the process for any renegotiating node numbers and such like that which end up having to be done before the system can fully wake up on ten again because these things can be done in parallel for those parts of the system that aren't dependent on on having to use Apple talk for example if you have that enabled you don't have to wait for that particular renegotiation to occur before you can use the system from the application perspective you probably won't deal with Ione kit much directly most of the devices are actually handled through other layers of abstraction certainly if you're interested in discs chances are the way you're accessing those discs or through file systems or networking through either you know your ot interfaces from carbon or form from sockets or other network services that may exist or if you're interested in input events most likely you're dealing with the actual event system so again for most application developers you probably don't ever directly and get involved with the specific devices however we do have situations where there are specific user clients that that may exist when there is an application need to be able to access a particular kind of device USB and the human input device manager is a good example there are a lot of sessions for i/o kit this week so if you're interested in various aspects of it be it firewire or USB or the storage drivers or the basic architecture I encourage you to go to that in fact immediately after this session and if there's another overview because I couldn't possibly get into enough detail on i/o kit here but I encourage you to get to go to that session if you're interested in you know working either at the device level or are interested in devices from an application perspective moving on now in our tour to the bsd portion of the system and this is where that power of eunuchs comes into play so there's two aspects that we look at for four bsd and it helps us kind of manage the different kind of views and responsibilities that exist the first is the bsd from the kernel perspective and the bsd in the kernel is one it's based on bsd for four with a lot of intense integration that exists between it and mark and it an i/o kit so if you look at the kernel environment for mac OS 10 it's basically the bsd kernel io kit and mock are the three key pieces that make up the kernel environment the bsd kernel is what is actually provides the OS personality if you will api's and services for the system much like mock is completely agnostic as far as the policy is concerned vsts responsibility is actually to manage the policy of the OS so it actually provides the application process model they support for things like signals and tracking of file descriptors and things like that when your application you know dies or gets torn down its BS Dee's job to make sure that all those rate resources are all reclaimed and ran and handled it also is responsible for the basic security policy of the system so if you will that's the concept of multiple users on the system or even when you take it down to the fastest the mauville being able to manage access to individual files or or ask pieces of data that basic security policy is a very inherent part of the kind of the bsd environment and is an enforced again down at the kernel level it's also important to note that both the file systems architecture and the networking architecture for Mac os10 basically fit inside the BSD in kernel space they are both based on the bsd implementations and we'll talk more about both file systems and networking in a few minutes the system framework is basically how we try to describe or abstract the set of interfaces that bsd provides and it's basically the system calls and if you will POSIX level interfaces and bsd system api is if you will for the system if you're accessing things are like porting a bsd application this is actually the api is that you're mostly what most likely will write to the application environments themselves are tend to be written on top of this api set the system framework is also responsible for other kinds of services as well things like pthreads the math library the basic c libraries that basically the very low level set of api's and services that you expect to be able to support an OS environment the user environment is kind of another if you will almost a fourth application environment that exists on the system you've probably seen all the other architectural diagrams and they refer to classic and in carbon and cocoa or in potentially even java it's important to note that in terms of the actual system architecture there's actually one more and it's the bsd user environment it's where things like the shells and command line scripts and things like that actually run it's where a lot of the network administration client tools actually execute the things that are kind of common command-line tools and facilities that come from the bsd community it's also a huge number of file you know various file tools for creating directories and getting copying files and that kind of thing as well as tools for being able to kill other processes or top like i show you we've showed you a few minutes ago all these things are available and exist within the bsd user environment it's also where some very key and important kind of system services or network administrative services exist in the system things like net info there are dns services bind network time protocol server the system log and things like apache and it may be important to note here that if you for example are running you know the client environment and you happen to turn on personal web sharing that it's actually apache that ends up getting launched and run on the system really the single most used web server on the planet but all that stuff exists kind of behind the scenes and it helps support the system from a developer's perspective it's important to note that you really want to be kind of aware of how BSD influences your life certainly again it contributes to the process model the system the support for things like signals and management of file descriptors and other important OS resources are all managed by the BSD Process Model but there's also things like environment variables and things like that that are also available the kind of the standard things you would expect in it from a UNIX process it's also responsible for the security policy again and remember those will impact the way your application behaves and from that perspective you should be aware that things that you maybe have been used to doing under Mac OS 9 you know that the security policy of the system actually plays a significant role here in immaculate in whether or not you can access certain files and places that you can write into the system and things like that it's also important to step back a bit from that from the application perspective and look at BSD from the developer perspective there are a ton of tools that exist in this environment now certainly we'd try and discourage the BSD kind of UNIX user experience for our typical end users but from a developer perspective this is an incredibly powerful part of the system as you saw with top you know there's ways to actually look at the system kind of from behind the scenes this particular setup we had was actually two systems we telnet it into the system that was running the application to run top so we could observe it without impacting the UI in any particular way kind of being able to you know put instrument probes if you will on the system but there's a huge number of other tools that are available there that might help building your applications or you know just managing your development resources and in the environment that you work in importing UNIX applications to Mac OS 10 it's a you know important to note that there's a lot of options that are available here later this week there'll be a session that'll actually show you a couple of ways that you can approach this if you happen to have your favorite Phoenix favorite UNIX tools or other that you you might want to use on Mac os10 porting them over to 10 is actually fairly straightforward but and in fact you can actually use a kind of a stepping stone approach to getting your applications on a Mac OS 10 if you happen to have one that even has a GUI there I believe there's a third-party X Windows server for example that's available so if you want to pour it over an X base application and get that up and running very quickly and then spend the time to actually wrap it in something like cocoa there to replace it with a more aqua friendly you know more user friendly experience that ends up being a very easy thing to do under Mac OS 10 it's also interesting to know that it's also very possible in something we actually do a lot ourselves being able to use UNIX tools from within a GUI application many of our control panels for example or actually you know execute command line tools in the background they actually you know are able to launch and interact with a UNIX command line environment again that's completely hidden from the user Apache as a pro you know it's an excellent example of that if you go out and get an Apache manual I've had to do full of system administration there's a pretty thick books out there for administrating the command the the various configuration files and stuff your user should never have to deal with that and it's possible to actually wrap that kind of power in something that's very easy for them to use there's a number of sessions also that are related to the BSD aspect of the system leveraging bsd services latter part of the week will actually go through in demo porting some UNIX tools and wrapping them in a in a cocoa application but there's also a couple of other important sessions as well threading on Mac OS 10 and I will talk about the P threads and some of the other threading options that you have available in the system and as well support for directory services and such on the on Mac OS 10 moving on now into file systems oops sorry about that filesystems of course is a lot more complex than just this little blob on the diagram is actually a lot a very very broad range of file systems that are supported within system and this is all based on basically the the BSD file system architecture it's important to note the file systems implementation is part of the bsd kernel environment and it's based on the VFS design there we go out and get your favorite unix bsd unix tex well actually as a great reference for being able to learn about the VFS architecture but it's basically a stackable virtual file system model which we've extended both from the basic POSIX interfaces beyond that to be able to support more rich file systems such as HTTP plus it's apart of file system type in fact on Mac OS 10 we support a huge number of file system types today and there are third parties that are also been making file system services available recently I read that the Andrew file system for example has been ported now to Mac OS 10 we've also extended those file system interfaces to be able to take advantage of Unicode so you'll find that there are ut8 utf-8 interfaces available so that you can be able to take advantage of localized file names and such but it's also really important to note that from a way we've designed Mac OS 10 much like how we've approached the i/o system that we have no particular file system affinity at the OS level in that we don't particularly care whether or not a Fitch HFS+ whether it's ufs we are intentionally trying to design the architecture of the system so that it can be flexible across a number of different file system architectures as you move up the application chain more of the those high-level services and capabilities are exported through your applications the file system is responsible for enforcing the file system security policy of the system again unlike Mac OS 9 in this respect there's a concept of security does existence your ability to write given files renamed even execute privileges all exist in are enforced by the file system there's a concept of users and groups that is throughout the system and the access control is all managed by this aspect of the system the application environments themselves tend to provide the abstracted level of interfaces that you'll tend to use if you're writing a carbon app you're using the file manager and things like resource Forks and things like that are you know fairly fluidly part of that particular environment if you're writing in cocoa you're going to be using something like NS file and where the use of resource Forks is not the the typical way of writing an application in this space or if you're porting a bsd application you'll end up using the POSIX + + api's and what I mean by deposit + + is this the POSIX + the extensions that we've provided for Mac OS 10 now to talk a little bit more give you a better idea what the implications are file systems in the system it's Joe to come back up again and again much like Mark where there's a lot going on behind the scenes on your application the same is true for file systems okay so we're gonna talk a little bit about a tool called FS usage let me get the top out of the way here that basically is something that you're probably not familiar with I don't believe it shows up on other UNIX system it's something that we developed here but it will give you a comprehensive list of all the files and directories being touched and the size of the iOS that are going on the amount of time that you're waiting for that IO to complete there's also on the far left there you can see a current clock with millisecond precision and then the name of the the task that's actually causing the iOS to occur so it's both comprehensive both in kind of a global manner if you're looking at the basically allowing to look at all of the tasks that are running in the system or you can focus it in on a particular task but I find it more interesting to look at it with more the global method just because there are lots of things that are happening on the system that you trigger when the application does something that you might not be aware of because a lot of the services are tied in indirectly to some of these higher-level calls it's useful for exposing redundancies of access pretty obvious and as we as I alluded to earlier it can show you where your app is blocking or waiting for i/o so if the app feels sluggish you you can use this tool to determine whether or not it's it's due to lots of wait time for iOS and I'll even show you where you're waiting for synchronous page ins to complete so where the apps trying to page fault some of its code and/or its data and then one other interesting aspect which you can I would suggest reading or reading the man page so you can see how to turn this on but there's a little backdoor trap that basically enables the display of the higher-level carbon calls so for you Carbon developers out there who are curious as to what system calls what UNIX system calls we turn a lot of these higher Carbon file manager calls into FS usage will show you that it will actually show you the encompassing or the calls being encompassed by these various higher level calls so we do something like open the menu and then maybe bring that up you get an idea of all the stuff that's going on but this is a real useful tool for you know kind of getting a good idea of what's actually happening in the system you know from some of those higher level calls that you're making it may turn into just a whole lot of other stuff going on so it's very educational thank you so again it's worth noting here that for much of the things that are going on in your applications that down deep underneath that application there's a lot of OS you know facilities in play to make those things possible and there are a lot of very very powerful and useful tools on Mac OS 10 that you can take advantage of to be able to see the system in this kind of from this kind of perspective if you're interested in learning more about the file system there's a whole session on that as well on Wednesday in the morning encourage you to go to that Clark is always quite entertaining moving on in our tour now to networking for networking onto Mac OS 10 again networking is part of the BSD kernel environment and it's based on the as you put this morning there and a network with a reference stack if you will for for IP and in the internet world we're based on the four-four tcp/ip stack which we've actually synced up with FreeBSD 3-2 and this is a sockets API based environment it comes with a quite a lot of powerful capabilities built into the networking stack certainly support for things like multihoming and routing and firewall and network address translation or nap mechanisms these are all pretty basic services that exist in our networking stack another very key one however is this concept of Auto configuration and again I'll be touched on that the keynote this morning but it's a very important part of our architecture and a key part of foundation that we're working on as well as we move forward with Mac OS 10 for the networking environment the way that you actually go about extending that is through something we call n keys or network kernel extensions um there's a basically a model that we've developed for being able to extend or enhance the the capabilities of our networking stack much like you can create drivers a forgiving piece of hardware or you can write a fire file system kernel extension for being able to support new in exotic file systems there's also ways for you to be able to extend the networking aspect of Mac OS 10 one important part of the architecture for our networking is actually how we support classic and we very part key part of that architecture of the system is to be able to support so that the virtual environments above us for example like classic or maybe something like a virtual PC like environment can actually share the the core networking services in configuration and setup so it's not necessary to have multiple IP addresses for each virtual environment that exists on the system we also support access of the networking through an open transport layer that exists for porting carbon applications things like Internet Explorer and things like that all use the open transport services that we provide as carbon as a way to get their app over quickly onto Mac OS 10 we also support PPP and PPP over ethernet as key services that exist in Mac OS 10 and pppoe is something that's like new for for Mac OS the ability to be able to inherently be able to connect to your DSL or you know cable modem provider also built into Mac OS 10 is the support for DHCP both the client and server so this is actually used in different situations depending on how Mac OS 10 is set up certainly as a full server environment you can administrate and manage DHCP for things like netboot and stuff but it's also used for being able to do ad hoc networking ad hoc networking is the ability for you know a small group of Macs to be able to just to set up either either through a crossover cable or over airport and without having to have all the infrastructure full internet environment but so that you can transfer files back and forth for example and those actually take advantage of things like DHCP in order to make that work mobility is a very key part of the direction of our networking architecture networking under Mac Westin is already incredibly rich and powerful through the multihoming and other aspects when one area that has not been historically been well known in UNIX networking is the ability to deal with mobility and that's something that we consider to be very critical so we've actually spent quite a bit of time developing a very flexible architecture for being able to handle mobility be able to support things like automatic network configuration being able to deal both configuration and reconfiguration based on the you know link level detection that exists so if you have a portable and you unplug that cable out of the back um it'll automatically notice the fact that you're even removed the cable and if you have an airport um it'll automatically yes and silently convert over completely to be able to use the airport if you're using DHCP configuration and all of this support is very dynamic in addition we also support multiple configurations and this is stuff that Ivey demoed this morning so I won't go into a lot of detail around this but they certainly the ability to be able to support multiple kinds of you know interface and configuration sets but another key aspect of it is to be able the ability to support application level notification our mail program is into the first example of this use where an application can actually be aware of the ability of a network a network connectivity and accessibility to the network so if you actually have had a chance to play with mail if you you know remove the cable out of the back of your system or lose connectivity to the Internet mail will automatically go offline it detects the fact that you know or have a connection and does the right thing and when you plug it back in again it'll automatically be involved in the process of reconfiguring itself and reconnecting it to the appropriate servers and this is the kind of behavior we'd like to see as we move on and move forward with Mac OS 10 there are a number of sessions for networking this week networking overview exists tomorrow and a number of other networking sessions tomorrow both the kernel and extensions for networking as well as the configuration and mobility session as well tomorrow and if then there's a feedback form later this week so take a little side trip here and talk about how the core OS and g4 architecture actually played together this is certainly kind of outside of the architectural space of thinking of the core us but it's an important part in an impact sexually or designed space for Mac OS 10 in terms of how we get some incredible performance on on the system first and foremost of course is support for the velocity engine for all the ultimate unit of the processor and there's a number of libraries that are available on Mac OS 10 there's a V V DSP library which is responsible for basically signal processing and fast Fourier transforms and convolutions and things like that and that library as it was existed on Mac OS 9 1 but is also available on Mac OS 10 new for Mac OS 10 however as a new library called V Blas the vector basic linear algebra subroutines which basically are handled support for things like a large matrix manipulation basically this is this kind of thing is used for things like mp3 coding and decoding MPEG speech recognition and image processing those are common places where this kind of library would be used and this particular service is only available in Mac OS 10 it's important to note that from this library on a 500 megahertz g4 this is where we get some of the really amazing performance numbers that are possible because of the g4 is 2.2 gigaflops performance again this kind of stuff is used in in things like iTunes and stuff like that there's also a number of other vector libraries that are available math the math lib which is basically basic math library that's been altivec tuned vector operations and basic ops which is kind of an extension to the basic instruction set if you will for the velocity engine and bignum which is a sports so basically large you know 1024 bit multiplies that kind of thing the other aspect of the g4 worth noting and how it plays with with Mac OS 10 and the core OS is in multi processors that Mac OS 10 inherently supports and some true SMP and this is really differentiated from Mac OS 9 whereas if you have an application and you happen to have a name you know MP box that unless you have specifically written your application to use the MP API you won't actually benefit Blainey from any significant an MP system on Mac OS 10 that's very different in that the system itself takes advantage of the processors that are available and scales automatically each thread in the system is capable of running on a separate processor again mock at the very lowest levels of the system is where the part of the system responsible for handling the scheduling and processor management and will automatically make sure that each thread on the system runs on whatever processor resources are available it means if you have a single application with multiple threads each of those threads might actually run on a different processor and that's really important to note because that introduced really tends to be a great way of finding situations where you may not necessarily be doing your multi-threading in the best possible way and doing the locking and things that need to occur I encourage if you have a multi-threaded application that you find a way to test and run it on an MP system if you're using tend to want to be able to take advantage of that there's a number of ways through the different application environments to take advantage of it certainly the MP API is four-carbon and interesting enough classic as well as anise thread for cocoa and the P threads package for BSD applications so there's a number of abstractions that you can use to be able to take advantage of multiple threads it's important also to note kind of how the system itself deals with MP on the system again mark is you know design very much from the beginning to be take advantage of the multiple processors and to be able to deal with things like threads UNIX is was is not quite as evolved in the same way it's from its heritage is actually designed to to run more on uniprocessor based systems we've done a bit quite a bit of work to be able to take advantage of MP on Mac OS 10 and we'll be doing a lot more as time progresses but it's important that if you're actually doing work on Mac OS 10 that you know that the BSD environment and how it manages itself across multiple processors is a key part of how the system is designed I encourage you to go to the session later this week where there's more discussion on this particular issue particularly if you're doing file systems and networking you'll find that there's this concept of funnels that are involved for managing what parts of the system actually are involved and run on which processors since we're talking about things in the kernel there's this concept that I've used number time now of kernel extensions and basically these are modules of code if you will code fragments that are allow if you will a plug-in model for the kernel allows things like drivers and network kernel extensions and in file system plugins to actually be load and and managed as part of the as part of Mac os10 really important thing to note that a Mac os10 kernel extension is not the same as a mac os9 extension we have very explicit usage models for these particular kinds of extensions if you're doing a driver or you need to be able to create a new file system this is the place that would you use a Mac os10 extension many things that were used that were are extensions in a Mac OS 9 are not appropriate to be done as a kernel extension you really should only do them if you have to it's basically is the plug-in model for the kernel and we try and keep the structure of these things much like an application bundle in the sense that we have bundles that have property lists and the binaries for the actual thing that gets loaded and there's tools and stuff for loading and unloading these things in the system as well as some common api's and things that are supported for this the other API is however beyond the basic loading and initialize and finalize capabilities are all actual api's that are very domain-specific that's why I said this is really not a general mechanism for extending Mac OS 10 it's really only used for things like drivers and faucets and plugins and networking services when you're developing inside the kernel should you be one of these people writing external a kernel extension it's really important to note that the rules change a bit one you tend to have to deal with things through a to system debugging environment to machine debugging that the language features that you have available are much much more limited we don't support Objective C in the kernel that C++ that exists there is actually a very strict subset of the full C++ standard so they're things like multiple inheritance and exceptions and things like that are not the kind of thing that we really encourage or in some cases even support obviously and when you're down at the kernel level there is no direct user interaction some cases you're at interrupt level you certainly aren't going to be able to talk to the user at that point you don't have access to things like aqua there are interesting logging tools and other things that you can use so that you can get information to you while you're writing your applications but it's definitely not a user interaction solution resources in the kernel are considerably more costly here when you're dealing with an application if it allocates memory even if you run out of the available physical memory the OS itself will look you know through VM and through backing store will do its best to try and support your application if you're in the kernel and you use lots of memory you're stealing away actual physical RAM from the rest of the system and it's very expensive so if you're writing in this particular space you should always use the least amount that you possibly can failures in the kernel are fatal obviously one of the reasons we want to discourage development of things like kernel extensions is this is one area where you can introduce instability in the system if you have a driver that goes wrong while you can just as easily crash the system that way as you can under Mac OS 9 so that's why there are very explicit cases where we support an extension of the kernel are we in discourage all other of extensions from this level moving on to the open-source aspect of darwin i think it's important to note again that virtually all of the components that are available in the core OS are open-source a couple of very rare extension exceptions things like some some of the drivers that are licensed from third parties things like that and that we actually very actively are involved in actually feeding code back to our upstream providers so it's not a we've just taken code from the community and have integrated in and they're just are running with it we actually work for things like BSD and Apache and other things that if we find bug fixes or the things that we actually feed those fixes back to the BSD community their apache community or other providers we are actually very actively trying to make sure that there's a community involvement here an open-source excuse me and that it's important that this community is very cooperative for example things like security fixes that come in some of the patches that were part of some recent software updates were actually fixes that came in from the external community that we rolled into the software updates or even platform fixes we ended up getting a from one of the bug reports that was submitted to our support group actually included a bug fix right right in as part of the submission so it's really important to note from a perspective of open source this is something we take very seriously it's not an experiment it's very much a part of how we do business for in the core OS and that our developers are involved in the mailing list in the community to to actively make this a two-way two-way process and that you as developers can actively participate in this process it's important to note again that our repositories and things like that actually many of them are outside apple's firewall that we actually have people outside of Apple that do not are not under a payroll don't work for us to actually have commit access to these repositories that people can actually are involved in the process of developing and involving the darwin and darwin codebase excuse me there's a couple of sessions later this week that I encourage you to participate in one is the open source at Apple on Wednesday talk a bit more about Darwin from an open source perspective and other open source projects as well as a birds of a feather session tomorrow evening and finally I'd like to give you a handful of things to think about as you kind of go from this point hopefully off to all these other sessions one is that you know it's our intention even though we have all this power on the under the hood that the user experience of Unix is something that we really want to hide the goal is not to make Emacs the the ideal user interface right from our perspective this is really an optional environment for power users and developers this should not be the typical behavior of Mac OS 10 but this is something that you as developers should take advantage of but always keep in mind you know give you some examples of how we've actually evolved the system in this way is that the concept of root under Mac OS 10 is something that we've really tried to move away from it's possible to actually set up your system so that you have a root user but you have to go to some work and you have to really know what you're doing in advance but this is part of the model that we've tried to apply as Mac OS 10 as to how to provide as much power as a UNIX based environment can possibly have without having to be create you end users that have to be you know knowledgeable units administrators be careful that you don't repeat bad habits under Mac OS 10 there's a lot of different ways to approach things and you can probably solve virtually any problem you have with the mechanisms that are there but think about them before you do it don't just jump into the only way you can solve a problem is by using shared memory because you end up recreating the same set of problems of stability that have existed in the past there are other ways to be able to communicate between multiple applications than running on the system shared memory for example should be avoided and it's really important to look at the system you know to have a very system's view when you're writing your apps tools like what we showed with top and FSU's and stuff allows you to get a sense of how your application actually impacts the rest of the system there's a lot of other threads and processes and things that are running on the computer and in order to be able to make sure that the system behaves well with lots of different things occurring you need to take a few minutes and actually looking to see how your application affects the system always use the highest level of abstraction possible again they usually are very appropriate services to use either the carbon level or the Coco level or whatever and that you should only get involved in using bsd services or mark level api services if that's the only way to solve a given problem don't just jump right in and use vm allocate for example for doing memory management there usually are a lot of other things in the abstraction layers that exist that are helping to manage those resources and make sure that they you know are handled appropriately and remember that open source is an incredibly powerful tool this is something that we're doing which we think is pretty novel in terms of you know a adopting open source in a very big way as part of the major you know aspect of the product that we sell again this is not a little interesting side project you know Steve put it very clearly we're betting Apple very much on the success of Mac OS 10 and open-source is a very key part of the lower portion of that system get involved in it help participate you know there's a lot of email lists and ability to actually contribute code and other things help make the system better and also remember that there's an incredible amount of power that's exists here one of the big changes between Mac OS 9 and 10 is this introduction of a you know modern operating system well this is that modern operating system the ability to do threading and you know being able to do lots of things in power and parallel and things in the background important leverage UNIX applications and multiple application environments and so this is all incredibly powerful aspect of Mac OS 10 and something that you want to think about because I think it will enable you to take that next big step and lastly I want to remind people that there is a feedback form we'd love to hear your comments on Darwin and things the ways that you think we can make it better so encourage you to come to the if you're here at the last part of the session five o'clock on Friday we'll be here you