WWDC2000 Session 130

Transcript

Kind: captions Language: en ladies and gentlemen please welcome carbon techno manager David Wright thank you very much welcome to session 1 30 carbon the lower levels oh ok I'm going to see that's how to carve a low level in the book well I'm really excited to see how many of you all made it I know that a lots of questions that are going to be answered in this session we're asked during previous q and A's during the rest of the week and I told you guys to keep coming here and you did so I'm really glad and it's mixed with sadness because this is the last session of the conference so it's kind of weird I know like Christmas ending well for me anyway I don't know I don't know it's been a good conference for you guys I sure hope so yeah I'm so glad great we have so much information to get out and I'm so glad you were here to receive it we've certainly received a lot of good feedback have a lot to go back to apple with and work on to make bringing your applications on 210 a lot easier so anyway let's nail down the final details of what's involved for you as you bring your applications to 10 by carbonizing them and please welcome John orochi manager of the advanced tool box team at Apple hey John good afternoon wow glad everyone can make it on a five o'clock on a Friday afternoon so I'm going to take you through it says carbon lower levels it's a pretty generic term starting with a slide that I'm sure everyone here is seen and anyone here not seeing this slide yet here there's there's a alternative designs up on the web core services okay the it's a technology that I'm going to go through today are actually part of this box called core services it's a little confusing because when we initially had a lot of these technologies up and run they were part of the carbon runtime specifically they are part of the carbon stack if you will and what's happened over time is we've seen that we have some similar services that are available in all stacks when we decided to pack for these lower into the system and actually share them and that's why you'll see some familiar looking technologies down in the core services layer the other thing that you may not have noticed when you saw this kind of slide in other presentations is well the first thing you probably did notice it's an architectural diagram it's a layer diagram trying to describe the system but it's also real and that is the corresponds to the frameworks that we have on Mac OS 10 just to make sure we're all on the same page here framework is a term that's being bandied about a lot it really tends to be used in the same way in which you'd say shared library other folks call this ad I live a framework is really a collection of files and folders around grouped around a shared library it includes headers it might include the resources for this and i'll talk about frameworks quite a bit so if you are one of those folks who has actually installed deep before on your power books and you look in system library frameworks that's where we keep most of the frameworks on the system in particular you should note there there's there's one framework called core services core services is the lowest framework in the system above the system framework or the Darwin layer and it is intended for tools and and processes that would use the system in a way probably not use any graphic services not use any UI an example of things that use court services is the res tool that we have on the system you see running in in project builder okay so there's a lot of things that are in core services I'm not going to go over all of them I'm just going to go over a subset you should notice here that most of the things that are in core services are things that don't deal with you I or graphic in cases where it seems like that's not the case for example stata alias manager when it brings up a dialog or some UI associated with the NFL manager what we've actually done there factor that out of course services onto a framework above that an upper layer typically application services or even carbon or cocoa so what am I actually going to talk about in this session this subset of technologies and basically this list here is kind of roughly in chronological order as you would bring an application up from when when you double-click in the finder and that gets launched via the process manager I will be going through these technologies and kind of giving you an update on what's been happening with them but I'll also be looking at some of the things that are unique about them on 10 in particular I'm going to be highlighting some of the core services design goals performance is kind of a constant theme that we've seen throughout most of the core services technologies and almost all of us the slides to follow we'll have some some aspect of performance some impacted performance and then scalability and extensibility is essential for these services mostly because they are factored low in the system and that it really is essentially have these qualities at that level so that higher levels that build upon them can take advantage of that and then of course stability is one of the major points of Mac OS 10 and that's both from the point of view of looking at the api's that are in this layer but also from the point point of view of making sure the system itself that the actual underlying implementations of the technology are stable and then thread safety this layer this core services layer is the foundation for our plans moving forward with regards to thread safety the the there will be a slide later on exactly what services are thread safe and I'll speak about that a little bit when we get to those okay this slide should look a bit familiar from the carbon overview I just wanted to stress that there are lots of different application environments that are taking advantage of and using core services on including the classic environment which runs all in one address space carbon apps java apps java apps are a little bit different in that they may not actually use core services and carbon things until they until they actually use a UI so they're kind of a little bit more dynamic that way cocoa apps somewhat similar but there are some foundation technologies in core services that are used across the board okay so now I'm going to get into the technologies proper starting with code loading I don't think it's a secret you've pretty much heard the terms throughout the conference so far we do have to code loading mechanisms on Mac OS 10 from an architectural point of view this isn't something that we've set out to do that we designed this isn't something that's necessarily something we put on our list we need to code loading architectures the contrary it would simplify things to some degree to actually just have one especially in the areas of tools however what we were faced with when we started OS 10 or we're talking about carbon early early days we had a system was predominantly a mock o system and the terminology there is the dynamic loader or dy LD and it knows about these mako's shared libraries which we refer to as die lids and we thought about for a little while and actually went into of analysis on how much cost it would would incur we would incur to change over to a whole new set of tools and at that point in time we had other things that we were planning on for example if you'll remember two years back we were just then talking about changing the microkernel from the the earlier version of Mach and what we basically decided is that we did not need a whole sweeping change like this throughout the entire system and this was key for me I've been an apple for a very long time this was one of those decisions that really was instrumental in letting us continue to develop and continue to have a live system that's why you have DP one two three and now four because we made decisions like this at that time it's not ideal from a technical point of view but it is the right compromise and it's giving us both pretty much the best of both worlds we have a CFM world now that's very good and I'll get into that in a little bit later and the motto world that we kind of started from okay just again on the terminology because I know this can be confusing the way you talk about the kind of output file format of the shared libraries for CFM those are pests shared libraries and in general we refer to the Makos shared libraries as dyland okay so to go along with the code loading there is differences that you'll you'll find if you're developing ma co and CFM applications that are interesting to note for CFM as you are all already familiar with you end up using dummy libraries to link your application and this gives you basically with the the important information of where what libraries we need to get symbols from and what the name those symbols are and what after you've linked your application and you're actually running the code fragment manager makes sure i'm sure that at launch time right when you start your application up that all of those libraries that you need and all those symbols that you need are present right then and there it's kind of flip flopped with dild in la co linking an application of montgo application all of your libraries in all of the libraries that they depend on have to be available at link time and that's primarily because ma co supports what's what we refer to as lazy initialization at runtime so let me go through a scenario here you have an application you link amaka application you link it everything is good and then you're going along and running on a system and all of a sudden some symbol that you need is missing because you inadvertently remove that framework from the system that is something that can actually happen that is by design with the code loader on with addy yld and mock oh it's not by design that it just misses that symbol but it's by design that it loads things lazily as it needs it I will get to what we've done with CFM there that actually kind of compromises between the two when I get to that slide and then the other thing I want to make sure it's clear as of right now and be before we still do not have support for to level namespace in d yld we're working on it we expect to have it for the public beta but the basic problem there is that we have a big flat namespace and we have some tools that you may have seen here in performance talks and things like that that actually rely on that flat namespace so we're trying to figure out a way that we can actually preserve that investment in tools and move forward to a bi-level namespace that we know you need for your plugins and your shared libraries okay to go with the two code loading mechanisms we have to run times they are very similar kind of at a high level that the real difference is what gets passed around when you're looking at the assembly code particularly in terms of what a function pointer is a function pointer in the CFM runtime sometimes referred to as talk runtime a function pointer is really a pointer to a little bit of memory it has a fun function address you know the code address and another word that's used for data offsets in mock oh you actually have a function a function pointers a pointer to the code itself and the offsets are all fixed when you link the application or linked the library so that's one of the fundamental differences between them aside from that the two run times are pretty similar and pretty compatible and that brings us to mix mode the difference is that we have between the two run times are actually dealt with in a slight change to the mixed mode that you probably know and use regularly in the mac OS in general so basically what we did with mixed mode is we've changed from macros and the headers to actual function calls and this was because we had these two different runtimes to support and this is key to having a single CFM application binary that can run on both OS 8 9 and 10 the differences come in in when you do a new or a dispose upp and that's when we actually take into account the differences in between how you pass code addresses around at invoke time it's essentially the same and specifically the way that all gets invoked in your application is you link your CFM application with carbon lib right the carbon lib that lives on OS 10 has this little bit of glue that gets you from kind of a CFM world to mock Oh world that carbon live that's on 10 is different in that regard then the carbon lived it's on eight or nine okay now on to the memory model that we have an OS 10 this slide really is about what's not in the memory model you notice right at the top of the list I'm sure you've heard this before but there's no lo men the region corresponding to low mem is actually not even mapped so if you access it you'll take a fault crash or you'll break into your debugger that's by design that's when the key differences that helps with stability it helps with cases where you have code your unintentionally dereference email or looking at at low mmmm addresses and getting something that sometimes is valid and sometimes isn't because sometimes it's the last thing you wrote there or sometimes it's something that somebody else are out there this is one whole area of instability on the traditional Mac OS that were you just don't have on 10 and that actually brings me to a good point this is one of the real important reasons why you have to take your applications and try them out on 10 you have to qualify to your applications on tenth we're not saying that something that is perfect on 9 linking to carbon later is going to run unchanged on 10 as a matter of fact I would say the opposite I would say that if you run your app on Ken you're probably going to have a better app more stable app that it's very likely to run unmodified online okay so what else do you see in this memory map or what else don't you see well there's no system heat there's no globally out allocatable memory food function whose address you can rely on being around who exists you know beyond the lifespan of the application that again is a stability issue and I'll get to that a little bit more when we talk about shared memory no process manager heap it's really unnecessary the tent mem functions that you may be familiar with are just mapped to the same as the nan tenten m version so handle whether it's a temp temp memory handle or a nap you know just a new handle end up being the same thing and then well you don't have any other kind of heap you don't have to worry about zones and this is all mostly motivated by performance reasons and working better on a world where you actually have a very large address space like we do on 10 and then no direct device access is another thing that kind of comes with the map the memory map so what do you have in the memory map on 10 well first thing of note that that helps out a lot with performance particularly performance of the system it's code and read only data being shared these sections from from a vm point of view code and read only data their shared are the best kind of sharing that we have on the system because basically they're backed by wherever the original data and code came from in the case in the system most of the code and data you're talking about is coming from individual frameworks and it's basically only wanting one place on disk and is shared and mapped into every address space that needs to use it in addition other things that you'll find in your address space are going to be well it's light towards the beginning of the outer space you'll find code and data of your applications you'll find a stack for your main thread you'll find more than one thread running on Noah's 10 it's likely you'll find on the order of a couple in DP for running your application each one of these threads has their own stack surrounded by guard pages that deals with ish issues like flowing under or over the stack size stacks are very large I believe right now they're set at half a megabyte that's something we have to look at and tuned correctly and then in general we're talking about a very sparse address space lots of space lots of holes in the outer space it's discontinuous so the reason I'm going into that is that you shouldn't assume anything about about that address space you will find you know if you're sitting there probing in the debugger you will find a certain amount of determinism you'll see certain things loaded at the same address but that's not necessarily the case and because of the dynamic city of the dynamic loader and CFM you will find things you know on the first run of the app and you'll find certain things loaded code or data at a certain address and then do this very same thing and because between the two times something else got loaded you won't be loaded in the same address so don't make any assumptions about where you'll be loaded then the other thing I'd like to point out more on the practical note if you're looking at before there's two command line tools that really help you get a good grasp on this the first one is just sighs it and basically sighs when given a path to a shared library or an application will tell you that sections that your shared library application has things like how big your code is how big your data sections are what portion of those are read-only what portion are read right one of the things that we hit a lot as we brought code over from OS 8 is we had lots of data sections that were marked as writable which meant that they really didn't get shared and that was simply a matter of in the code saying that these were constant data sections at that point it became read only and we're fully shared that optimization is something I would look at with your own shared libraries in your own applications the other tool is a vm map which takes as an argument that the either the name or the PID of the process that is running and this gives you a snapshot if you will of a running process and there should be a one-to-one correspondence between the size and vm map I shouldn't say a one-to-one everything that you see inside you should see in the vm map you will also in vm map see the size of other things that are in your address space other frameworks that you're using you should be able to see read-only and rewrite sections and they're fairly clearly in talking to the tool folks there is an internal tool that we didn't get ready for DP for but it's basically a UI that lets you see this so you don't have to actually use command line tools but I thought I'd bring that up in case you're looking at these kinds of things in DP for ok so now we know about memory maps but what does this actually mean to you when you're doing memory allocation well the first thing is memory manager in general is fully supported on both pointer base handle based objects and I think where you have to keep in mind here is this is one of these areas where we have some api's in the memory manager that makes sense on eight and nine that don't make sense on 10 there's still part of carbon so you can write a single binary my the best example of this is the free mem call there isn't really a good answer for that on 10 we could tell you there's four gigabytes we could tell you there's half that it's not really something that that makes sense when you have a sparse address space and that's why we came up with a gift alt memory map sparse selector that's what you should check but when you're wondering whether you need to ask these questions whether you're wondering whether you really have to police your memory allocation some other differences that kind of fall into this boat is purging and for that matter just growing handles moving handles in general doesn't typically happen on OS 10 we won't do that the only way that handles generally move is if you've called set handle size on them so this is actually a counterpoint to what I said earlier about where you should qualify your app because this is an area that you could have something that runs fine on 10 and you write some new code and you don't lock your handle and you never sent handle size so that you can dereference that without a problem and you'll never find a bug on 10 but take that same code and move it back to eight or nine and you likely will find a problem and that's because eight or nine has a limited you know fixed size application heat the other thing I want to address on is kind of we spent quite a bit of time trying to understand the performance of memory allocation on 10 and one of the things we found is that the assumptions are almost reversed in a sense the memory manager on 89 made a very explicit design decision to favor handles over pointers and that was the right decision at the time because we did have a fixed size heap and basically the reverse is true on 10 in a vm based system like 10 what sparks address spaces you really be the most basic representation for memory is is a pointer and one of the things that kind of sounds counterintuitive but I really encourage you to do is to try to get your code set up so that you basically refrain from doing any kind of sub allocation that you may be doing very typical in your OS 8 9 code and the reason being here is that a lot of the tools on Noah's can depend on you using malloc directly or new pointer and new handle pandered handled directly the tools have been ripped to understand those two packages for memory allocation and the tools I'm talking about are going to help you find things like leaks they're going to help you find you know cases where you've overwritten a block memory underwritten this was all kind of gone over and one of the performance tool sessions but the point here is that at least in a debug version of your app you want to have your memory allocation factored so that it calls one of these routines directly so you can use the tools the other thing I would encourage you to do is do some measurements of your your application and see what they're like just with the system allocator either malik directly and you should probably should look at Mallik if you don't return it or don't rely on any of the features of the memory manager with regards to pointers and handles and recovering handles and things like that if you're going to be interacting with handle base api's in the toolbox of course you're required to use handles there but for your own internal allocation needs start off try malik there's a there's a big advantage to actually all of us using the same allocator and it actually is not something you typically see unless you're looking at the overall system performance the problem we initially had when we brought up brought over large subsystems early on in the carbon effort on OS 10 which we had a lot of sub allocation going on every big subsystem had their own memory allocation and the problem with that is that we have essentially different high-water marks in every subsystem that has allocations and that meant that we didn't really do a good job as a whole of getting rid of memory and freeing it up in a timely sense so take a look at that if you have issues as this specifically with Malik and its performance both size and speed let us know because we want Malik to be the system allocator we want it to be as fast as possible we want it to be as efficient in terms of speed I'm sorry in terms of size and that actually brings me to another point which is the problems we've been dealing with in terms of performance and allocation have largely been sized problems not speed problems and I think that's going to be the case with your applications in your shared libraries as well it's not just a matter of having the big sparse address space and being able to allocate a lot of memory but it's a matter of actually being able to account for your space and understand the relationship between things that get allocated underneath a system framework compared to what you really intended to do above it so again take a look at that if you have really good reasons or really good performance observations please share those with us because we really want to understand why you would think of doing something other than using malloc on the system and then lastly there's an issue that came up late in DP for we didn't get a fix in for it and basically one of the changes we were doing to the memory manager to enable d but additional debugging support ended us up with a memory manager doesn't allocate on a 16-byte boundary in particular this is important for altivec or velocity engine data which ignores some of the lower bits so if you have if you run in that situation there's a tech there's a release note on DP for with a workaround on how to do that essentially copying the memory out will fix this for the user beta okay shared memory this one has quite story to it too so you know early on in carbon we basically said new pointer sis and the system heat they have to go there's the stability problems around them they're being used way too much they're used between extensions and applications and those things don't essentially have the same counterparts on OS 10 we really looked at this closely and we listened to some of the developers that wanted that really knew what they wanted out of a shared memory system they wanted particular features and after a lot of consideration what we ended up with is saying you know the features that that most developers want this area are completely covered by the POSIX shared memory api's and it really makes no sense to try to come up with a system that works on both 9 and 10 regarding shared memory the two systems in this area are quite different the single address space versus multi address space nature of the system makes it really hard to kind of abstract that away this is something that you have to be aware of if you're dealing with shared memory and then of course to use shared memory safely you have to synchronize and that's what the politics semaphores are for those are fully documented on the DP for release the shared memory documentation didn't get there but I provided the URL here at the bottom of the screen then one additional note if you're a CFM application since currently there is no way to get to services that are in the system framework like the POSIX shared memory and semaphore api's you're going to need to get to them through CF bundle or a plug-in they're basically there's two different ways of doing this one you could you see a bundle to ask for the address of a routine in the system framework so you could inquire and then call that the other is you could factor your application to use bundles that are specific to a platform and we recommend doing this so that basically you have a mock o plug-in on 10 that links with the system uses these api s and then you would just invoke the plugin okay that gets us over shared memory and into the file manager the file manager is probably one of the largest chunks of work that we had to do over the last year the biggest disconnect or the biggest missing feature from the file manager on 10 with support for these volume formats that I have up here NFS ufs and other positive style volume formats um now why are these file formats important basically this gives us a certain level of interoperability that we've never really had with a mac before it's not just NFS and ufs it's anything that complies to a POSIX API it's anything that if you went to a file system talk it plugs into the BFS layer and this is a bunt this there's a bunch of opportunities here in up incoming file systems that we really wanted to be able to take advantage of the other point is that we just this wasn't really an afterthought for us we were coming from an environment that was heavily using ufs and NFS we've coming from the Mac os10 server environment and as a matter of fact internally we were using NFS and ufs very heavily I mean to this date I keep my source code in my home directory it's on an NFS server that's the way that I personally develop other folks choose whatever they want to I do that mostly because I tend to go from machine to machine and I like to have the same kind of environment keep my sources there locally they're backed up centrally that works beautifully on deep before but to do that we had to really look at how the file manager talked to these kind of volumes 41 the classic file manager AP is really kind of assumed a volume format they were very HFF centric so that posed a certain problem the biggest differences that we encountered in the volume formats had to do with two particular features of HFS one was forks resource forks the like just the general concept that you have more data associated with a file than just that one dat afford and then the other was the idea of persistent file IDs basically the backbone for aliases on HSS you have something similar in ufs called an inode but it doesn't work the same way and it isn't quite the same experience in terms of something that's persistent so we knew we had to support these volume formats and what we settled on was basically to two different solutions in a sense one we knew that the current api's were not very good for dealing with this and we actually had a luxury that we rarely seen in the industry we actually had some time to make sure that that the set of H a P is that we call now hfs+ api's worked well with this environment the the other thing that we did is standardized we knew we had to store the extra data that we're talking about the fork information and the the file ID essentially the catalog information from from HFS we had to score that somewhere persistently so for that we chose the apple double file format and by dealing with these volume formats by using apple double we basically what we get is the file manager API both the classic and the new hfs+ api's deal with volumes transparently even if their underlying they don't they may not have a catalog node or they may not have a resource for we store this in apple double format so that the HFS and hfs+ api's work transparently across all of these different volume formats of course we have support for the volume formats that we're already used to in necklace 89 and almost all of these are on d before the only exception is UDF and that's coming in addition to the different volume format we had to deal with something to essentially foreign concept one was the idea of mount points the idea that a volume can be mounted or contained within another volume and the other is symbolic links from unix we looked at this this work actually made its way into DP three and we've refined it since then but basically we came up with a mechanism whereby both mount points and since symbolic links are show up to you through the file manager API as aliases that is files and have alias bit set they're not really alias files this information is synthesized and the reason this is one of the rare cases in the file manager we actually synthesize something typically we're giving you exactly what's on the disk the other exception to that would be of course the apple double for Matt we're not actually giving you that information directly we're providing the extra information so the way that you need to support now points and symbolic links on OS 10 is just to respect the alias bit if you intend to resolve them to resolve them via the resolve alias file need to use that call as opposed to other variants because it isn't an alias file really now for those that are interested the reason we had to go this route is very a very pervasive assumption that most people have in dealing file system particularly in the iteration as you're going through a directory full of files you're not assuming that the v ref num which is sprinkled throughout most of our api's is going to change that is the case the mouth point and it's potentially the case with the symlink so the wit that's what got us to to a solution like this okay that's enough for there okay so the file manager is an area we put a lot of effort into and told you about kind of the differences in file system on on 10 the one thing that I really want you to come come away from this session with is there's really a lot of win in those ten to go and use the HFS plus API the HFS plus AP is our very well designed for this system there's one notion in particular in the HFS plus API that really pays off and that's the ability to be very specific about the information that you want to file so using the HMS plus AP is and asking for as little information or exactly the information that you need is a best solution on these volume formats that don't support some of the things that you may have been become used to in HS and then in particular you really have to try this out for yourself on these volume formats you can do that in DB for you can format a volume to use uff you can also if you have NFS servers in the general area you can mount them on NFS or over a CD the experience will be you'll be able to see it a lot easier just because of the general performance characteristics of those volume formats okay so then the other thing I wanted to bring up because people keep on hitting this over and over as they bring their apps as they carbonize and bring their apps to 10 are some assumptions that that are out there first is that V ref num start at minus 1 they typically don't start at minus 14 on OS 10 in the file manager freeleech need to treat those bereft gnomes as cookies we've run into a lot of code to basically you know quickly written code that basically works on 89 is something liking a looping between minus 1 and minus 10 assuming that they're going to find all volumes that way use the the volume calls to iterate through volumes that'll prevent that whole a full set of problems the other is the whole notion of file IDs file IDs like I said earlier are not available on some of these volume formats on some of these volume formats what we have to revert to is a path embedded in in the alias this is essentially like a minimal alias you can treat them exactly the same way and on these volume formats with current implementation the way it works on D before they'll have some limitations they won't behave like a normal alias you can break aliases more easily on these volume formats will be looking into things that we can do to improve that but particular changing an element of the path you know renaming a directory somewhere in the middle of the path is going to break that alias ok ok so I spoke about this one concept earlier that really helps out this is a tie in the performance and just working well on OSN and that's the the notion that you can be very specific about the information that you want when asking for information for a file using FS get catalog info the HFS a plus API for that if you just need the name of that file then use the bitmap that as an argument and just say you want the name that's a very cheap way of determining if the file is there same kind of approach works in some of these other cases when you're you're going through a directory what is it that you're really after are you trying to just enumerate my name or are you trying to find you know all files with a certain with an extension are you trying to find only the directories inside another directory these are all things that can be set in the bitmap and doing so will really help in particular on a network volume like NFS where each one of those file system operations ends up being some kind of typically some kind of pack it over the wire there's a certain amount of non-determinism to that on your typical network and volume iterations same thing goes for that set the bitmap and ask for what you want let's say that this is actually an area where the advice we gave you earlier on ended up biting us a little bit in terms of performance so long ago we basically said that the VC BQ is gone from carbon you can't just get this low ma'am and increment it and start walking through a volume control blocks that way the replacement technology for that was used the volume iteration api's works fine when the volumes that you're iterating over are for example NFS volumes it doesn't work so well again from a point of view of performance when you go through using volume iteration calls you're asking for a lot of data that often you don't want if you didn't just wanted the name or you wanted to get to the next ref num you probably don't want the size of the volume which can be very expensive to compute the size of blocks things like that some of the other information that comes back in some of these calls there's always been cheap on h of SN hfs+ not so in some of these volume formats again this is why you won't want to use FS get vol info in addition there's a just get ball params call which gives you a subset of the same information ok I'm going to go through a little bit of code here this should look familiar to most people this is the your typical you know go through a directory looking for files in the directory the dot-dot-dot is just this pseudocode you know that's a little bit of setup additional setup is not something you would compile and work correctly so you'll notice the the main call in a directory iteration this is the old pile manager api's this is PB get cat info sink sink in this case you notice that basically as you go through a directory you have to make this call for every item in the directory okay here's the replacement loop notice first same kind of structure code right I do while loop same kind of call you're asking for catalog info one big difference is it a bulk call so you can ask for more than one file information at a time the number of files is up to you it's actually a parameter for the FSK catalog info bolt so this is something that's tunable this is something that in DP for works down to a bulk call in the final manager but what we're going to do is actually move that even further posted before make a better performance enhancement and move that that call all the way down to the colonel that'll give us the best efficiency so now I want to actually can we get demo one up on stage I want to show you a quick demo application that we brought together just as a side here this was a you know in the tradition of WWDC done in i guess it was late last week now in a few hours but the surprise that we had is actually a lot of folks on on my team the low level the advanced Mac toolbox team hadn't really had a chance to to seriously use interface builder we don't typically do those kinds of things that have to do with interface and it's on the screen and this was it a basically was a surprise to actually see interface builder generating carbon app in almost no time flat without actually having to understand some of the new technologies that are there ok so what do I have here I have and go and hide this oh can I did ya see thank you ok on the top progress bar is the classic API so the PBG cat info sync call that we just saw it's basically that loop the bottom is the newer call that bulk call before actually say go on this I wanted to bring up one other thing I wanted to do this in a pretty fair way so I'll get to actually this the way I went about doing this a little bit later but basically I decided to do both of these tests on an MP threat so they're basically the only difference between the two executing is that the colonel time slicing them appropriately most of what these threads are doing shouldn't take that long in terms of cpu processing they should go in either block very quickly because that the OS de texas the data in question isn't there and act actually get that from the device or rip right through that and just fetch the next one if it's cached right the other thing I want to bring up is basically both of these the new and the old are both iterating through two different directories that have the same content so that you don't get any cash and effects from one disturbing the other and with that I'll just let them go it's a fairly small directory roughly double the time the results are 65 tix vs 31 pics this is extremely exaggerated in the case where you go on a CD a little bit less so on and a fesse and this is uff on DP for and you see the doubling is something that you can easily do and it one thing to note though is it really will depend on the content of what's in the directory and that gets back to you know the resource forks are there things that that aren't easily represented on the volume for Matt ok can we get slides back ok so we talked about the file manager again they're the biggest advice I have is to use hfs+ api's now we're on to the folder manager the biggest change the folder manager for OS 10 has been to support a concept that we call domain 0 h 10 supports a notion where you have a basically a system based what we ship to you that's read-only it's not modified by the user it's not typically modified after it's shipped that's kind of like the base domain that's what you start with above that is typically either a network or a local domain where you do install things in that are shared by all applications by all users and then finally you have the user domain which is roughly just for the users things that apply to the user domain or things that apply to all domains are examples are preferences where you'll clearly want to have a user having one set of preferences not shared with another user fonts may be another case where you have your own fonts you care about that you want to install and they're yours and you don't want anybody else to use them alternatively fonts could also be installed and one of the other domains it's actually domains has to do a lot with sharing and how certain types well known types of data get used the folder managers then expand their folder managers enhance to support these domains by using the first parameter to find folder instead of just supporting the ref numbs it also supports some well-known domain Constance it's important to note though that there are some types of things that the selectors that the folder manager supports that don't make sense and domains the two examples that I have here temporary and trash temporary items for example it's typically used when you want to do some kind of well that the best case I have is when you want to use PB exchange files you want to basically make sure that the file in question and you're going to save atomically is on the same volume right trash is another case if that ended up being in some different volume then you'd end up doing the copy for trash and it's not really what you want it either so to check for the support of this you just have checked this cos salt bit folder manager supports domains and act based on that the other thing that I wanted to bring up is you'll see a bullet item on a lot of these slides most of the lower level technologies have up incoming FS breath based ap is so if you had already started on hfs+ conversion you may have ran into the fact that you know yeah you're using hfs+ api's but then when you have to ask the holder manager for something it's in terms of you know an FS spec so we're slowly taking hfs+ api's and my guess is for the public beta you'll have variant api's in a bunch of these technologies that use them more directly okay resources there's been a lot of questions about resources in particular resource fork supported very quickly will go through the chain chain management api's these are things that have been there since carbon lived 10 so the they allow an insertion to remove removal of resources in the chain one thing to keep aware of no s10 is that we actually support file mapping of resource files that is we file map things that would be shared things that are roughly equate to the system file and what that means to you is that the data is really read only and this is very similar to a rom on older Macintoshes might get raw data in older macs you'll probably end up you can actually silently dereference a handle to ron data on OS 10 you won't be able to do that if you need that data if you want to modify that data you're going to have to use attached resource and then the resource manager is going to continue the trend in use have some FS ref based ap is available there been a lot of questions about resource forks and CF bundle I guess the biggest thing here is basically because we want to interoperate with other volume formats we've introduced a nap packaging specification that doesn't use resource fork it uses resources in data forks now why is that a good thing it's a good thing because that's kind of a lowest common denominator solution that transfers easily you can copy that off the web off of ftp you can copy it up to an NFS server copy it back without having to modify tools without having to understand about things like forks so there's some api's that enable this in the resource manager and this happens for you for free for your application resources if you're packaged you'll basically get to data fork resource files open for you assuming you've localized your app one if you have it and then just in case you haven't gotten to the any of the CF talks if you need to get to anything else within your application that's been packaged just to make sure you use the CF bundle api's okay code fragment manager there's been a big change we were actually gay able to get in we hadn't planned on making this available for DP for but we got far enough along that we did get it a preview of this technology in DP for its not on by default you're going to have to go and enable it in the release notes on DP for in the documentation folder there's a release note on the code fragment manager on how to actually enable it one of the key features of the the work we've done on this is really helps with launch time basically we were able to take calculation that happened at runtime previously make that a current link time so that CFM launches or a lot faster with this technology enabled the current default implementation the code fragment manager does not support data exports this new one does and in addition because of the way in which was the work was done at link time we support lazy initialization in the same way that dy LD does so do not pay the cost until you actually use the librarian question there's two particular issues on OS 10 resource-based fragments are not supported the workaround or the alternative to doing that is actually handling the whatever you want to load from a resource yourself by calling the resource manager and then using memory fragments instead and then general shared data sections are not supported and we already talked about the velocity engine issue okay the process manager process manager is fully functional on the current release it's compatible with that packages there's been some confusion about this basically if we preserve the illusion in the process manager that you kind of have in the finder if you ask the process information of a process that is app package the spec associated with it the FS spec is respect to the dot app the package not to the executable contained within it however we also support traditionally packaged single file applications and that works pretty much the same as it has thread manager okay it's a couple things I want to get through here on the thread manager one is that there's a lot of code out there that goes through great pains to make sure that they use threads in a memory savvy way they're trying to make sure they're not using too much memory with their threads managing the stack sizes and things like that most of that is unnecessary on 10 some of that you can't really do very well on tenth you can't really get the stack size of another thread particulars that thread happens to be running on another processor this isn't a case is just one of those kinds of things that wasn't anticipated when the original original thread manager came up came along these threads and MP threads are all layered on top of P threads in the thread manager case they're still completely cooperative so that is even though they're pthreads they all race for the same lock and only one of them can run at the time the other problem that has had that folks have had with these kinds of threads is that generally the way that you schedule them is off of null events which doesn't really work very well from a poor phone standpoint on 10 we've talked about England carbon event sessions and other sessions we really want you to be blocked waiting for events to come in if you're calling if you're blocked waiting for events to come in then you can't really be calling yield to any thread all the time that's part of the problem with thread manager threads and of course no concurrency however I do want to bring up that thread managers often the best solution for your you UI layer of your application why is that because well let me give this example if you are handling four different windows with four different pre-emptive threads and the user says you know command w the shortcut for closing the window to all four windows what's going to happen if those threads are pre-emptive is completely random there's not going to be any order you'll do it one time and all the windows will shut in one order you do for command WN and it'll be different this is not good for a decent predictable at you I you do want cause and effect in your UI the best way to kind of keep that is probably something like co-op your threads or even a single thread in for your UI and then have the backend of your application use a pre-emptive threat and that brings us to the MP API I already mentioned that they're layered on top of P threads these are the api's that we really want to push you towards these are the api's that worked frantically on carbon both on nine and on ten one difference is then some confusion on this is that MP constructs are not system global that is to say on a nine where everything works in an address space an MP q can be used to talk between prophecies that's not the case on 10 the empty package is a fairly thin layer on top of P threads it's a / process or / address space package then I just want to refer you to another route release note on how to use MP create task from CFM apps it's also on the DP for release and then I mentioned earlier different thread safe services these are the ones that we're talking about right now mostly I owe file i/o networking that kind of thing and one more thing about threading is that the models between nine and ten are particularly different when you're doing threaded threading and I oh something you want to look at is if you're using cooperative threads and asynchronous i/o that is not going to perform as well as synchronous i/o and pre-emptive threads on 10 it will still work that it's just not going to get your last little bit of performance okay Apple events apple event is basically the only solution that we have for cross process IPC inside of carbon there's been a little bit of there's been questions in the hallways about this and things like that so if you need to do cross process IPC this is the solution the other alternatives are not are only available on 10 our CF core foundation allows for some level of cross process communication and then Mauch messaging directly so the basic changes for Apple events is the desks are opaque and other than that they're a foundation technology that are being used throughout all the frameworks Coco carbon classic one of the things I didn't mention the process manager is all the applications within classic actually have an individual PSN so they're each uniquely targetable that way okay now I'd like to bring Steve Sellars up it's going to do a apple event related demo okay I know that we're short on time so I'm going to talk fast one of the things that's really exciting about Mac os10 is being able to mix all these different API so the demo i put together involves a version of apple works that just runs a little database here so this is a classic sorry i cook a carbon application that runs unmodified on mac OS 9 and 10 I just grabbed it off our server and double clicked on it so I've got a little database here and now i have a little tool in this window that you probably can't see very well and what it's going to do is start up a server that listens to connections only from the local hosts to execute applescript so that guy is going to start up I'm going to go over to the other machine and this machine out by the way is also running Apache so what we've got is a CGI script in the apache server going to talk to my command line tool that is going to send apple events to the to the database so we bring up a browser over here right so here's my window over here with a bunch of predefined searches and things and as I'm executing searches you can see that the database over here connects to the server performs the search and then displays the search now that's obviously not something you want your database to do all the time if this were a commercial database like one of you would develop you would return the data without updating the UI but you could do that in the background application without having to present to you I to the user at all so over here you can see the session transcript of what the perl script actually told the command line tool to send to the database over here and what gets returned is standard apple script results which are in turn interpreted by the perl script and turn into HTML all circuits at once because that's more interesting to watch over there so as you can see you can put together a real application a real workflow using carbon in your application okay Steve [Applause] okay we're at the more information section probably seeing the carbon documentation URL but that's up there again the to release notes that I referred to are on the DP for release in system developer documentation release notes carbon core HTML is going to go over some of the issues that I mentioned with the memory manager and code fragment manager and then I'm sorry memory manager then the code fragment manager has its own release note on how to enable the new vector libraries and now that summary so hfs+ api's that's the big one most of the applications that we've seen have just been ported no they're still not a lot of them have adopted at hfs+ api's now there's even a bigger reason to adopt these eight api's and in particular using these api's and using multiprocessing api's for a factored application is a very good proof it's a proven combination that's worked well on OS 10 so to get back to one of the carbon overview session scott forstall was actually up there saying that he had some challenges for you I have the same challenges and I really like to see that the development community take us up on this in particular with these core services we need feedback on things that may be impacting you in terms of performance in terms of features we need things earlier rather than later particularly these layers of the system we can't rev at the very last second just because of the dependencies so I encourage you to bring your app up on DB for to be beta with us when we ship the public beta and to release the same time as we release so is 10 ok now I'd like to bring up a few of the people on my team for some quick QA actually John we are out of time oh well you pledged Holden let you had a lot of information and you did so okay actually it looks like we have well I've ever naps we don't I can't even start it because of the time yeah okay well I can take some questions we can probably go yeah back over there so I thank you all so much for coming to this conference again the feedback that you've given us is so great i know the Apple engineering is Benway encouraged by the show and hopefully you have been too and we'll see you all next year [Music] you