WWDC2000 Session 130

Transcript

Kind: captions
Language: en
ladies and gentlemen please welcome
carbon techno
manager David Wright thank you very much
welcome to session 1 30 carbon the lower
levels oh ok I'm going to see that's how
to carve a low level in the book well
I'm really excited to see how many of
you all made it I know that a lots of
questions that are going to be answered
in this session we're asked during
previous q and A's during the rest of
the week and I told you guys to keep
coming here and you did so I'm really
glad and it's mixed with sadness because
this is the last session of the
conference so it's kind of weird I know
like Christmas ending well for me anyway
I don't know I don't know it's been a
good conference for you guys I sure hope
so yeah I'm so glad great we have so
much information to get out and I'm so
glad you were here to receive it we've
certainly received a lot of good
feedback have a lot to go back to apple
with and work on to make bringing your
applications on 210 a lot easier so
anyway let's nail down the final details
of what's involved for you as you bring
your applications to 10 by carbonizing
them and please welcome John orochi
manager of the advanced tool box team at
Apple hey John good afternoon wow glad
everyone can make it on a five o'clock
on a Friday afternoon so I'm going to
take you through it says carbon lower
levels it's a pretty generic term
starting with a slide that I'm sure
everyone here is seen and anyone here
not seeing this slide yet here there's
there's a alternative designs up on the
web core services okay the it's a
technology that I'm going to go through
today are actually part of this box
called core services it's a little
confusing because when we initially had
a lot of these technologies up and run
they were part of the carbon runtime
specifically they are part of the carbon
stack if you will and what's happened
over time is we've seen that we have
some similar services that are available
in all stacks when we decided to pack
for these lower into the system and
actually share them and that's why
you'll see some familiar looking
technologies down in the core services
layer the other thing that you may not
have noticed when you saw this kind of
slide in other presentations is well the
first thing you probably did notice it's
an architectural diagram it's a layer
diagram trying to describe the system
but it's also real and that is the
corresponds to the frameworks that we
have on Mac OS 10 just to make sure
we're all on the same page here
framework is a term that's being bandied
about a lot it really tends to be used
in the same way in which you'd say
shared library other folks call this ad
I live a framework is really a
collection of files and folders around
grouped around a shared library it
includes headers it might include the
resources for this and i'll talk about
frameworks quite a bit so if you are one
of those folks who has actually
installed deep before on your power
books and you look in system library
frameworks that's where we keep most of
the frameworks on the system in
particular you should note there there's
there's one framework called core
services core services is the lowest
framework in the system above the system
framework or the Darwin layer and it is
intended for tools and and processes
that would use the system in a way
probably not use any graphic services
not use any UI an example of things that
use court services is the res tool that
we have on the system you see running in
in project builder okay so there's a lot
of things that are in core services
I'm not going to go over all of them I'm
just going to go over a subset you
should notice here that most of the
things that are in core services are
things that don't deal with you I or
graphic in cases where it seems like
that's not the case for example stata
alias manager when it brings up a dialog
or some UI associated with the NFL
manager what we've actually done there
factor that out of course services onto
a framework above that an upper layer
typically application services or even
carbon or cocoa so what am I actually
going to talk about in this session this
subset of technologies and basically
this list here is kind of roughly in
chronological order as you would bring
an application up from when when you
double-click in the finder and that gets
launched via the process manager I will
be going through these technologies and
kind of giving you an update on what's
been happening with them but I'll also
be looking at some of the things that
are unique about them on 10 in
particular I'm going to be highlighting
some of the core services design goals
performance is kind of a constant theme
that we've seen throughout most of the
core services technologies and almost
all of us the slides to follow we'll
have some some aspect of performance
some impacted performance and then
scalability and extensibility is
essential for these services mostly
because they are factored low in the
system and that it really is essentially
have these qualities at that level so
that higher levels that build upon them
can take advantage of that and then of
course stability is one of the major
points of Mac OS 10 and that's both from
the point of view of looking at the
api's that are in this layer but also
from the point point of view of making
sure the system itself that the actual
underlying implementations of the
technology are stable and then thread
safety this layer this core services
layer is the foundation for
our plans moving forward with regards to
thread safety the the there will be a
slide later on exactly what services are
thread safe and I'll speak about that a
little bit when we get to those okay
this slide should look a bit familiar
from the carbon overview I just wanted
to stress that there are lots of
different application environments that
are taking advantage of and using core
services on including the classic
environment which runs all in one
address space carbon apps java apps java
apps are a little bit different in that
they may not actually use core services
and carbon things until they until they
actually use a UI so they're kind of a
little bit more dynamic that way cocoa
apps somewhat similar but there are some
foundation technologies in core services
that are used across the board okay so
now I'm going to get into the
technologies proper starting with code
loading I don't think it's a secret
you've pretty much heard the terms
throughout the conference so far we do
have to code loading mechanisms on Mac
OS 10 from an architectural point of
view this isn't something that we've set
out to do that we designed this isn't
something that's necessarily something
we put on our list we need to code
loading architectures the contrary it
would simplify things to some degree to
actually just have one especially in the
areas of tools however what we were
faced with when we started OS 10 or
we're talking about carbon early early
days we had a system was predominantly a
mock o system and the terminology there
is the dynamic loader or dy LD and it
knows about these mako's shared
libraries which we refer to as die lids
and we thought about for a little while
and actually went into
of analysis on how much cost it would
would incur we would incur to change
over to a whole new set of tools and at
that point in time we had other things
that we were planning on for example if
you'll remember two years back we were
just then talking about changing the
microkernel from the the earlier version
of Mach and what we basically decided is
that we did not need a whole sweeping
change like this throughout the entire
system and this was key for me I've been
an apple for a very long time this was
one of those decisions that really was
instrumental in letting us continue to
develop and continue to have a live
system that's why you have DP one two
three and now four because we made
decisions like this at that time it's
not ideal from a technical point of view
but it is the right compromise and it's
giving us both pretty much the best of
both worlds we have a CFM world now
that's very good and I'll get into that
in a little bit later and the motto
world that we kind of started from okay
just again on the terminology because I
know this can be confusing the way you
talk about the kind of output file
format of the shared libraries for CFM
those are pests shared libraries and in
general we refer to the Makos shared
libraries as dyland okay so to go along
with the code loading there is
differences that you'll you'll find if
you're developing ma co and CFM
applications that are interesting to
note for CFM as you are all already
familiar with you end up using dummy
libraries to link your application and
this gives you basically with the the
important information of where what
libraries we need to get symbols from
and what the name those symbols are and
what after you've linked your
application and you're actually running
the code fragment manager makes sure i'm
sure that at launch time right when you
start your application up that all of
those libraries that you need and all
those symbols that you need are
present right then and there it's kind
of flip flopped with dild in la co
linking an application of montgo
application all of your libraries in all
of the libraries that they depend on
have to be available at link time and
that's primarily because ma co supports
what's what we refer to as lazy
initialization at runtime so let me go
through a scenario here you have an
application you link amaka application
you link it everything is good and then
you're going along and running on a
system and all of a sudden some symbol
that you need is missing because you
inadvertently remove that framework from
the system that is something that can
actually happen that is by design with
the code loader on with addy yld and
mock oh it's not by design that it just
misses that symbol but it's by design
that it loads things lazily as it needs
it I will get to what we've done with
CFM there that actually kind of
compromises between the two when I get
to that slide and then the other thing I
want to make sure it's clear as of right
now and be before we still do not have
support for to level namespace in d yld
we're working on it we expect to have it
for the public beta but the basic
problem there is that we have a big flat
namespace and we have some tools that
you may have seen here in performance
talks and things like that that actually
rely on that flat namespace so we're
trying to figure out a way that we can
actually preserve that investment in
tools and move forward to a bi-level
namespace that we know you need for your
plugins and your shared libraries okay
to go with the two code loading
mechanisms we have to run times they are
very similar kind of at a high level
that the real difference is what gets
passed around when you're looking at the
assembly code particularly in terms of
what a function pointer is a function
pointer in the
CFM runtime sometimes referred to as
talk runtime a function pointer is
really a pointer to a little bit of
memory it has a fun function address you
know the code address and another word
that's used for data offsets in mock oh
you actually have a function a function
pointers a pointer to the code itself
and the offsets are all fixed when you
link the application or linked the
library so that's one of the fundamental
differences between them aside from that
the two run times are pretty similar and
pretty compatible and that brings us to
mix mode the difference is that we have
between the two run times are actually
dealt with in a slight change to the
mixed mode that you probably know and
use regularly in the mac OS in general
so basically what we did with mixed mode
is we've changed from macros and the
headers to actual function calls and
this was because we had these two
different runtimes to support and this
is key to having a single CFM
application binary that can run on both
OS 8 9 and 10 the differences come in in
when you do a new or a dispose upp and
that's when we actually take into
account the differences in between how
you pass code addresses around at invoke
time it's essentially the same and
specifically the way that all gets
invoked in your application is you link
your CFM application with carbon lib
right the carbon lib that lives on OS 10
has this little bit of glue that gets
you from kind of a CFM world to mock Oh
world that carbon live that's on 10 is
different in that regard then the carbon
lived it's on eight or nine
okay now on to the memory model that we
have an OS 10 this slide really is about
what's not in the memory model you
notice right at the top of the list I'm
sure you've heard this before but
there's no lo men the region
corresponding to low mem is actually not
even mapped so if you access it you'll
take a fault crash or you'll break into
your debugger that's by design that's
when the key differences that helps with
stability it helps with cases where you
have code your unintentionally
dereference email or looking at at low
mmmm addresses and getting something
that sometimes is valid and sometimes
isn't because sometimes it's the last
thing you wrote there or sometimes it's
something that somebody else are out
there this is one whole area of
instability on the traditional Mac OS
that were you just don't have on 10 and
that actually brings me to a good point
this is one of the real important
reasons why you have to take your
applications and try them out on 10 you
have to qualify to your applications on
tenth we're not saying that something
that is perfect on 9 linking to carbon
later is going to run unchanged on 10 as
a matter of fact I would say the
opposite I would say that if you run
your app on Ken you're probably going to
have a better app more stable app that
it's very likely to run unmodified
online okay so what else do you see in
this memory map or what else don't you
see well there's no system heat there's
no globally out allocatable memory food
function whose address you can rely on
being around who exists you know beyond
the lifespan of the application that
again is a stability issue and I'll get
to that a little bit more when we talk
about shared memory no process manager
heap it's really unnecessary the tent
mem functions that you may be familiar
with are just mapped to the same as the
nan tenten m
version so handle whether it's a temp
temp memory handle or a nap you know
just a new handle end up being the same
thing and then well you don't have any
other kind of heap you don't have to
worry about zones and this is all mostly
motivated by performance reasons and
working better on a world where you
actually have a very large address space
like we do on 10 and then no direct
device access is another thing that kind
of comes with the map the memory map so
what do you have in the memory map on 10
well first thing of note that that helps
out a lot with performance particularly
performance of the system it's code and
read only data being shared these
sections from from a vm point of view
code and read only data their shared are
the best kind of sharing that we have on
the system because basically they're
backed by wherever the original data and
code came from in the case in the system
most of the code and data you're talking
about is coming from individual
frameworks and it's basically only
wanting one place on disk and is shared
and mapped into every address space that
needs to use it in addition other things
that you'll find in your address space
are going to be well it's light towards
the beginning of the outer space you'll
find code and data of your applications
you'll find a stack for your main thread
you'll find more than one thread running
on Noah's 10 it's likely you'll find on
the order of a couple in DP for running
your application each one of these
threads has their own stack surrounded
by guard pages that deals with ish
issues like flowing under or over the
stack size stacks are very large I
believe right now they're set at half a
megabyte
that's something we have to look at and
tuned correctly and then in general
we're talking about a very sparse
address space lots of space lots of
holes in the outer space it's
discontinuous so the reason I'm going
into that is that you shouldn't assume
anything about about that address space
you will find you know if you're sitting
there probing in the debugger you will
find a certain amount of determinism
you'll see certain things loaded at the
same address but that's not necessarily
the case and because of the dynamic city
of the dynamic loader and CFM you will
find things you know on the first run of
the app and you'll find certain things
loaded code or data at a certain address
and then do this very same thing and
because between the two times something
else got loaded you won't be loaded in
the same address so don't make any
assumptions about where you'll be loaded
then the other thing I'd like to point
out more on the practical note if you're
looking at before there's two command
line tools that really help you get a
good grasp on this the first one is just
sighs it and basically sighs when given
a path to a shared library or an
application will tell you that sections
that your shared library application has
things like how big your code is how big
your data sections are what portion of
those are read-only what portion are
read right one of the things that we hit
a lot as we brought code over from OS 8
is we had lots of data sections that
were marked as writable which meant that
they really didn't get shared and that
was simply a matter of in the code
saying that these were constant data
sections at that point it became read
only and we're fully shared that
optimization is something I would look
at with your own shared libraries in
your own applications the other tool is
a vm map which takes as an argument that
the either the name or the PID of the
process that is running and this gives
you a snapshot if you will of a running
process
and there should be a one-to-one
correspondence between the size and vm
map I shouldn't say a one-to-one
everything that you see inside you
should see in the vm map you will also
in vm map see the size of other things
that are in your address space other
frameworks that you're using you should
be able to see read-only and rewrite
sections and they're fairly clearly in
talking to the tool folks there is an
internal tool that we didn't get ready
for DP for but it's basically a UI that
lets you see this so you don't have to
actually use command line tools but I
thought I'd bring that up in case you're
looking at these kinds of things in DP
for ok so now we know about memory maps
but what does this actually mean to you
when you're doing memory allocation well
the first thing is memory manager in
general is fully supported on both
pointer base handle based objects and I
think where you have to keep in mind
here is this is one of these areas where
we have some api's in the memory manager
that makes sense on eight and nine that
don't make sense on 10 there's still
part of carbon so you can write a single
binary my the best example of this is
the free mem call there isn't really a
good answer for that on 10 we could tell
you there's four gigabytes we could tell
you there's half that it's not really
something that that makes sense when you
have a sparse address space and that's
why we came up with a gift alt memory
map sparse selector that's what you
should check but when you're wondering
whether you need to ask these questions
whether you're wondering whether you
really have to police your memory
allocation some other differences that
kind of fall into this boat is purging
and for that matter just growing handles
moving handles in general doesn't
typically happen on OS 10 we won't do
that the only way that handles generally
move is if you've called set handle size
on them so this is actually a
counterpoint to what I said earlier
about where you should qualify your app
because this is an area that you could
have something that runs fine on 10
and you write some new code and you
don't lock your handle and you never
sent handle size so that you can
dereference that without a problem and
you'll never find a bug on 10 but take
that same code and move it back to eight
or nine and you likely will find a
problem and that's because eight or nine
has a limited you know fixed size
application heat the other thing I want
to address on is kind of we spent quite
a bit of time trying to understand the
performance of memory allocation on 10
and one of the things we found is that
the assumptions are almost reversed in a
sense the memory manager on 89 made a
very explicit design decision to favor
handles over pointers and that was the
right decision at the time because we
did have a fixed size heap and basically
the reverse is true on 10 in a vm based
system like 10 what sparks address
spaces you really be the most basic
representation for memory is is a
pointer and one of the things that kind
of sounds counterintuitive but I really
encourage you to do is to try to get
your code set up so that you basically
refrain from doing any kind of sub
allocation that you may be doing very
typical in your OS 8 9 code and the
reason being here is that a lot of the
tools on Noah's can depend on you using
malloc directly or new pointer and new
handle pandered handled directly the
tools have been ripped to understand
those two packages for memory allocation
and the tools I'm talking about are
going to help you find things like leaks
they're going to help you find you know
cases where you've overwritten a block
memory underwritten this was all kind of
gone over and one of the performance
tool sessions but
the point here is that at least in a
debug version of your app you want to
have your memory allocation factored so
that it calls one of these routines
directly so you can use the tools the
other thing I would encourage you to do
is do some measurements of your your
application and see what they're like
just with the system allocator either
malik directly and you should probably
should look at Mallik if you don't
return it or don't rely on any of the
features of the memory manager with
regards to pointers and handles and
recovering handles and things like that
if you're going to be interacting with
handle base api's in the toolbox of
course you're required to use handles
there but for your own internal
allocation needs start off try malik
there's a there's a big advantage to
actually all of us using the same
allocator and it actually is not
something you typically see unless
you're looking at the overall system
performance the problem we initially had
when we brought up brought over large
subsystems early on in the carbon effort
on OS 10 which we had a lot of sub
allocation going on every big subsystem
had their own memory allocation and the
problem with that is that we have
essentially different high-water marks
in every subsystem that has allocations
and that meant that we didn't really do
a good job as a whole of getting rid of
memory and freeing it up in a timely
sense so take a look at that if you have
issues as this specifically with Malik
and its performance both size and speed
let us know because we want Malik to be
the system allocator we want it to be as
fast as possible we want it to be as
efficient in terms of speed I'm sorry in
terms of size and that actually brings
me to another point which is the
problems we've been dealing with in
terms of performance and allocation have
largely been sized problems not speed
problems and I think that's going to be
the case with your applications in your
shared libraries as well
it's not just a matter of having the big
sparse address space and being able to
allocate a lot of memory but it's a
matter of actually being able to account
for your space and understand the
relationship between things that get
allocated underneath a system framework
compared to what you really intended to
do above it so again take a look at that
if you have really good reasons or
really good performance observations
please share those with us because we
really want to understand why you would
think of doing something other than
using malloc on the system and then
lastly there's an issue that came up
late in DP for we didn't get a fix in
for it and basically one of the changes
we were doing to the memory manager to
enable d but additional debugging
support ended us up with a memory
manager doesn't allocate on a 16-byte
boundary in particular this is important
for altivec or velocity engine data
which ignores some of the lower bits so
if you have if you run in that situation
there's a tech there's a release note on
DP for with a workaround on how to do
that essentially copying the memory out
will fix this for the user beta okay
shared memory this one has quite story
to it too so you know early on in carbon
we basically said new pointer sis and
the system heat they have to go there's
the stability problems around them
they're being used way too much they're
used between extensions and applications
and those things don't essentially have
the same counterparts on OS 10 we really
looked at this closely and we listened
to some of the developers that wanted
that really knew what they wanted out of
a shared memory system they wanted
particular features and after a lot of
consideration what we ended up with is
saying you know the features that that
most developers want
this area are completely covered by the
POSIX shared memory api's and it really
makes no sense to try to come up with a
system that works on both 9 and 10
regarding shared memory the two systems
in this area are quite different the
single address space versus multi
address space nature of the system makes
it really hard to kind of abstract that
away this is something that you have to
be aware of if you're dealing with
shared memory and then of course to use
shared memory safely you have to
synchronize and that's what the politics
semaphores are for those are fully
documented on the DP for release the
shared memory documentation didn't get
there but I provided the URL here at the
bottom of the screen then one additional
note if you're a CFM application since
currently there is no way to get to
services that are in the system
framework like the POSIX shared memory
and semaphore api's you're going to need
to get to them through CF bundle or a
plug-in they're basically there's two
different ways of doing this one you
could you see a bundle to ask for the
address of a routine in the system
framework so you could inquire and then
call that the other is you could factor
your application to use bundles that are
specific to a platform and we recommend
doing this so that basically you have a
mock o plug-in on 10 that links with the
system uses these api s and then you
would just invoke the plugin okay that
gets us over shared memory and into the
file manager the file manager is
probably one of the largest chunks of
work that we had to do over the last
year the biggest disconnect or the
biggest missing feature from the file
manager on 10 with support for these
volume formats that I have up here NFS
ufs and other positive style volume
formats
um
now why are these file formats important
basically this gives us a certain level
of interoperability that we've never
really had with a mac before it's not
just NFS and ufs it's anything that
complies to a POSIX API it's anything
that if you went to a file system talk
it plugs into the BFS layer and this is
a bunt this there's a bunch of
opportunities here in up incoming file
systems that we really wanted to be able
to take advantage of the other point is
that we just this wasn't really an
afterthought for us we were coming from
an environment that was heavily using
ufs and NFS we've coming from the Mac
os10 server environment and as a matter
of fact internally we were using NFS and
ufs very heavily I mean to this date I
keep my source code in my home directory
it's on an NFS server that's the way
that I personally develop other folks
choose whatever they want to I do that
mostly because I tend to go from machine
to machine and I like to have the same
kind of environment keep my sources
there locally they're backed up
centrally that works beautifully on deep
before but to do that we had to really
look at how the file manager talked to
these kind of volumes 41 the classic
file manager AP is really kind of
assumed a volume format they were very
HFF centric so that posed a certain
problem the biggest differences that we
encountered in the volume formats had to
do with two particular features of HFS
one was forks resource forks the like
just the general concept that you have
more data associated with a file than
just that one dat afford and then the
other was the idea of persistent file
IDs basically the backbone for aliases
on HSS
you have something similar in ufs called
an inode but it doesn't work the same
way and it isn't quite the same
experience in terms of something that's
persistent so we knew we had to support
these volume formats and what we settled
on was basically to two different
solutions in a sense one we knew that
the current api's were not very good for
dealing with this and we actually had a
luxury that we rarely seen in the
industry we actually had some time to
make sure that that the set of H a P is
that we call now hfs+ api's worked well
with this environment the the other
thing that we did is standardized we
knew we had to store the extra data that
we're talking about the fork information
and the the file ID essentially the
catalog information from from HFS we had
to score that somewhere persistently so
for that we chose the apple double file
format and by dealing with these volume
formats by using apple double we
basically what we get is the file
manager API both the classic and the new
hfs+ api's deal with volumes
transparently even if their underlying
they don't they may not have a catalog
node or they may not have a resource for
we store this in apple double format so
that the HFS and hfs+ api's work
transparently across all of these
different volume formats of course
we have support for the volume formats
that we're already used to in necklace
89 and almost all of these are on d
before the only exception is UDF and
that's coming in addition to the
different volume format we had to deal
with something to essentially foreign
concept one was the idea of mount points
the idea that a volume can be mounted or
contained within another volume and the
other is symbolic links from unix we
looked at this this work actually made
its way into DP three and we've refined
it since then but basically we came up
with a mechanism whereby both mount
points and since symbolic links are show
up to you through the file manager API
as aliases that is files and have alias
bit set they're not really alias files
this information is synthesized and the
reason this is one of the rare cases in
the file manager we actually synthesize
something typically we're giving you
exactly what's on the disk the other
exception to that would be of course the
apple double for Matt we're not actually
giving you that information directly
we're providing the extra information so
the way that you need to support now
points and symbolic links on OS 10 is
just to respect the alias bit if you
intend to resolve them to resolve them
via the resolve alias file need to use
that call as opposed to other variants
because it isn't an alias file really
now for those that are interested the
reason we had to go this route is very a
very pervasive assumption that most
people have in dealing file system
particularly in the iteration as you're
going through a directory full of files
you're not assuming that the v ref num
which is sprinkled throughout most of
our api's is going to change
that is the case the mouth point and
it's potentially the case with the
symlink so the wit that's what got us to
to a solution like this okay that's
enough for there okay so the file
manager is an area we put a lot of
effort into and told you about kind of
the differences in file system on on 10
the one thing that I really want you to
come come away from this session with is
there's really a lot of win in those ten
to go and use the HFS plus API the HFS
plus AP is our very well designed for
this system there's one notion in
particular in the HFS plus API that
really pays off and that's the ability
to be very specific about the
information that you want to file so
using the HMS plus AP is and asking for
as little information or exactly the
information that you need is a best
solution on these volume formats that
don't support some of the things that
you may have been become used to in HS
and then in particular you really have
to try this out for yourself on these
volume formats you can do that in DB for
you can format a volume to use uff you
can also if you have NFS servers in the
general area you can mount them on NFS
or over a CD the experience will be
you'll be able to see it a lot easier
just because of the general performance
characteristics of those volume formats
okay so then the other thing I wanted to
bring up because people keep on hitting
this over and over as they bring their
apps as they carbonize and bring their
apps to 10 are some assumptions that
that are out there first is that V ref
num start at minus 1 they typically
don't start at minus 14
on OS 10 in the file manager freeleech
need to treat those bereft gnomes as
cookies we've run into a lot of code to
basically you know quickly written code
that basically works on 89 is something
liking a looping between minus 1 and
minus 10 assuming that they're going to
find all volumes that way use the the
volume calls to iterate through volumes
that'll prevent that whole a full set of
problems the other is the whole notion
of file IDs file IDs like I said earlier
are not available on some of these
volume formats on some of these volume
formats what we have to revert to is a
path embedded in in the alias this is
essentially like a minimal alias you can
treat them exactly the same way and on
these volume formats with current
implementation the way it works on D
before they'll have some limitations
they won't behave like a normal alias
you can break aliases more easily on
these volume formats will be looking
into things that we can do to improve
that but particular changing an element
of the path you know renaming a
directory somewhere in the middle of the
path is going to break that alias ok ok
so I spoke about this one concept
earlier that really helps out this is a
tie in the performance and just working
well on OSN and that's the the notion
that you can be very specific about the
information that you want when asking
for information for a file using FS get
catalog info the HFS a plus API for that
if you just need the name of that file
then use the bitmap that as an argument
and just say you want the name that's a
very cheap way of determining if the
file is there same kind of approach
works in some of these other cases when
you're you're going through a directory
what is it that you're really after are
you trying to just enumerate my name or
are you trying to find you know all
files with a certain with an extension
are you trying to find only the
directories inside another directory
these are all things that can be set in
the bitmap and doing so will really help
in particular on a network volume like
NFS where each one of those file system
operations ends up being some kind of
typically some kind of pack it over the
wire there's a certain amount of
non-determinism to that on your typical
network and volume iterations same thing
goes for that set the bitmap and ask for
what you want let's say that this is
actually an area where the advice we
gave you earlier on ended up biting us a
little bit in terms of performance so
long ago we basically said that the VC
BQ is gone from carbon you can't just
get this low ma'am and increment it and
start walking through a volume control
blocks that way the replacement
technology for that was used the volume
iteration api's works fine when the
volumes that you're iterating over are
for example NFS volumes it doesn't work
so well again from a point of view of
performance when you go through using
volume iteration calls you're asking for
a lot of data that often you don't want
if you didn't just wanted the name or
you wanted to get to the next ref num
you probably don't want the size of the
volume which can be very expensive to
compute the size of blocks things like
that some of the other information that
comes back in some of these calls
there's always been cheap on h of SN
hfs+ not so in some of these volume
formats again this is why you won't want
to use FS get vol info in addition
there's a just get ball params call
which gives you a subset of the same
information ok I'm going to go through a
little bit of code here this should look
familiar to most people this is the your
typical you know go through a directory
looking for
files in the directory the dot-dot-dot
is just this pseudocode you know that's
a little bit of setup additional setup
is not something you would compile and
work correctly so you'll notice the the
main call in a directory iteration this
is the old pile manager api's this is PB
get cat info sink sink in this case you
notice that basically as you go through
a directory you have to make this call
for every item in the directory okay
here's the replacement loop notice first
same kind of structure code right I do
while loop same kind of call you're
asking for catalog info one big
difference is it a bulk call so you can
ask for more than one file information
at a time the number of files is up to
you it's actually a parameter for the
FSK catalog info bolt so this is
something that's tunable this is
something that in DP for works down to a
bulk call in the final manager but what
we're going to do is actually move that
even further posted before make a better
performance enhancement and move that
that call all the way down to the
colonel that'll give us the best
efficiency so now I want to actually can
we get demo one up on stage I want to
show you a quick demo application that
we brought together just as a side here
this was a you know in the tradition of
WWDC done in i guess it was late last
week now in a few hours but the surprise
that we had is actually a lot of folks
on on my team
the low level the advanced Mac toolbox
team hadn't really had a chance to to
seriously use interface builder we don't
typically do those kinds of things that
have to do with interface and it's on
the screen and this was it a basically
was a surprise to actually see interface
builder generating carbon app in almost
no time flat without actually having to
understand some of the new technologies
that are there ok so what do I have here
I have and go and hide this oh can I did
ya see thank you ok on the top progress
bar is the classic API so the PBG cat
info sync call that we just saw it's
basically that loop the bottom is the
newer call that bulk call before
actually say go on this I wanted to
bring up one other thing I wanted to do
this in a pretty fair way so I'll get to
actually this the way I went about doing
this a little bit later but basically I
decided to do both of these tests on an
MP threat so they're basically the only
difference between the two executing is
that the colonel time slicing them
appropriately most of what these threads
are doing shouldn't take that long in
terms of cpu processing they should go
in either block very quickly because
that the OS de texas the data in
question isn't there and act actually
get that from the device or rip right
through that and just fetch the next one
if it's cached right the other thing I
want to bring up is basically both of
these the new and the old are both
iterating through two different
directories that have the same content
so that you don't get any cash and
effects from one disturbing the other
and with that I'll just let them go
it's a fairly small directory roughly
double the time the results are 65 tix
vs 31 pics this is extremely exaggerated
in the case where you go on a CD a
little bit less so on and a fesse and
this is uff on DP for and you see the
doubling is something that you can
easily do and it one thing to note
though is it really will depend on the
content of what's in the directory and
that gets back to you know the resource
forks are there things that that aren't
easily represented on the volume for
Matt ok can we get slides back
ok so we talked about the file manager
again they're the biggest advice I have
is to use hfs+ api's now we're on to the
folder manager the biggest change the
folder manager for OS 10 has been to
support a concept that we call domain 0
h 10 supports a notion where you have a
basically a system based what we ship to
you that's read-only it's not modified
by the user it's not typically modified
after it's shipped that's kind of like
the base domain that's what you start
with above that is typically either a
network or a local domain where you do
install things in that are shared by all
applications by all users and then
finally you have the user domain which
is roughly just for the users things
that apply to the user domain or things
that apply to all domains are examples
are preferences where you'll clearly
want to have a user having one set of
preferences not shared with another user
fonts may be another case where you have
your own fonts you care about that you
want to install and they're yours and
you don't want anybody else to use them
alternatively fonts could also be
installed and one of the other domains
it's actually domains has to do a lot
with sharing and how certain types well
known types of data get used the folder
managers then expand their folder
managers enhance to support these
domains by using the first parameter to
find folder instead of just supporting
the ref numbs it also supports some
well-known domain Constance it's
important to note though that there are
some types of things that the selectors
that the folder manager supports that
don't make sense and domains the two
examples that I have here temporary and
trash temporary items for example it's
typically used when you want to do some
kind of
well that the best case I have is when
you want to use PB exchange files you
want to basically make sure that the
file in question and you're going to
save atomically is on the same volume
right trash is another case if that
ended up being in some different volume
then you'd end up doing the copy for
trash and it's not really what you want
it either so to check for the support of
this you just have checked this cos salt
bit folder manager supports domains and
act based on that the other thing that I
wanted to bring up is you'll see a
bullet item on a lot of these slides
most of the lower level technologies
have up incoming FS breath based ap is
so if you had already started on hfs+
conversion you may have ran into the
fact that you know yeah you're using
hfs+ api's but then when you have to ask
the holder manager for something it's in
terms of you know an FS spec so we're
slowly taking hfs+ api's and my guess is
for the public beta you'll have variant
api's in a bunch of these technologies
that use them more directly okay
resources there's been a lot of
questions about resources in particular
resource fork supported
very quickly will go through the chain
chain management api's these are things
that have been there since carbon lived
10 so the they allow an insertion to
remove removal of resources in the chain
one thing to keep aware of no s10 is
that we actually support file mapping of
resource files that is we file map
things that would be shared things that
are roughly equate to the system file
and what that means to you is that the
data is really read only and this is
very similar to a rom on older
Macintoshes might get raw data in older
macs you'll probably end up you can
actually silently dereference a handle
to ron data on OS 10 you won't be able
to do that if you need that data if you
want to modify that data you're going to
have to use attached resource and then
the resource manager is going to
continue the trend in use have some FS
ref based ap is available there been a
lot of questions about resource forks
and CF bundle I guess the biggest thing
here is basically because we want to
interoperate with other volume formats
we've introduced a nap packaging
specification that doesn't use resource
fork it uses resources in data forks now
why is that a good thing it's a good
thing because that's kind of a lowest
common denominator solution that
transfers easily you can copy that off
the web off of ftp you can copy it up to
an NFS server copy it back without
having to modify tools without having to
understand about things like forks so
there's some api's that enable this in
the resource manager and this happens
for you for free for your application
resources if you're packaged you'll
basically get to data fork resource
files open for you assuming you've
localized your app one if you have it
and then just in case you haven't gotten
to the any of the CF talks if you need
to get to anything else within your
application that's been packaged just to
make sure you use the CF bundle api's
okay code fragment manager there's been
a big change we were actually gay able
to get in we hadn't planned on making
this available for DP for but we got far
enough along that we did get it a
preview of this technology in DP for its
not on by default you're going to have
to go and enable it in the release notes
on DP for in the documentation folder
there's a release note on the code
fragment manager on how to actually
enable it one of the key features of the
the work we've done on this is really
helps with launch time basically we were
able to take calculation that happened
at runtime previously make that a
current link time so that CFM launches
or a lot faster with this technology
enabled the current default
implementation the code fragment manager
does not support data exports this new
one does and in addition because of the
way in which was the work was done at
link time we support lazy initialization
in the same way that dy LD does so do
not pay the cost until you actually use
the librarian question there's two
particular issues on OS 10
resource-based fragments are not
supported the workaround or the
alternative to doing that is actually
handling the whatever you want to load
from a resource yourself by calling the
resource manager and then using memory
fragments instead and then general
shared data sections are not supported
and we already talked about the velocity
engine issue okay the process manager
process manager is fully functional on
the current release it's compatible with
that packages there's been some
confusion about this basically if we
preserve the illusion in the process
manager that you kind of have in the
finder if you ask the process
information of a process that is app
package the spec associated with it the
FS spec
is respect to the dot app the package
not to the executable contained within
it however we also support traditionally
packaged single file applications and
that works pretty much the same as it
has thread manager okay it's a couple
things I want to get through here on the
thread manager one is that there's a lot
of code out there that goes through
great pains to make sure that they use
threads in a memory savvy way they're
trying to make sure they're not using
too much memory with their threads
managing the stack sizes and things like
that most of that is unnecessary on 10
some of that you can't really do very
well on tenth you can't really get the
stack size of another thread particulars
that thread happens to be running on
another processor this isn't a case is
just one of those kinds of things that
wasn't anticipated when the original
original thread manager came up came
along these threads and MP threads are
all layered on top of P threads in the
thread manager case they're still
completely cooperative so that is even
though they're pthreads they all race
for the same lock and only one of them
can run at the time the other problem
that has had that folks have had with
these kinds of threads is that generally
the way that you schedule them is off of
null events which doesn't really work
very well from a poor phone standpoint
on 10 we've talked about England carbon
event sessions and other sessions we
really want you to be blocked waiting
for events to come in if you're calling
if you're blocked waiting for events to
come in then you can't really be calling
yield to any thread all the time that's
part of the problem with thread manager
threads and of course no concurrency
however I do want to bring up that
thread managers often the best solution
for your you UI layer of your
application why is that because
well let me give this example if you are
handling four different windows with
four different pre-emptive threads and
the user says you know command w the
shortcut for closing the window to all
four windows what's going to happen if
those threads are pre-emptive is
completely random there's not going to
be any order you'll do it one time and
all the windows will shut in one order
you do for command WN and it'll be
different this is not good for a decent
predictable at you I you do want cause
and effect in your UI the best way to
kind of keep that is probably something
like co-op your threads or even a single
thread in for your UI and then have the
backend of your application use a
pre-emptive threat and that brings us to
the MP API I already mentioned that
they're layered on top of P threads
these are the api's that we really want
to push you towards these are the api's
that worked frantically on carbon both
on nine and on ten one difference is
then some confusion on this is that MP
constructs are not system global that is
to say on a nine where everything works
in an address space an MP q can be used
to talk between prophecies that's not
the case on 10 the empty package is a
fairly thin layer on top of P threads
it's a / process or / address space
package then I just want to refer you to
another route release note on how to use
MP create task from CFM apps it's also
on the DP for release and then I
mentioned earlier different thread safe
services these are the ones that we're
talking about right now mostly I owe
file i/o networking that kind of thing
and one more thing about threading is
that the models between nine and ten are
particularly different when you're doing
threaded threading and I oh
something you want to look at is if
you're using cooperative threads and
asynchronous i/o that is not going to
perform as well as synchronous i/o and
pre-emptive threads on 10 it will still
work that it's just not going to get
your last little bit of performance okay
Apple events apple event is basically
the only solution that we have for cross
process IPC inside of carbon there's
been a little bit of there's been
questions in the hallways about this and
things like that so if you need to do
cross process IPC this is the solution
the other alternatives are not are only
available on 10 our CF core foundation
allows for some level of cross process
communication and then Mauch messaging
directly so the basic changes for Apple
events is the desks are opaque and other
than that they're a foundation
technology that are being used
throughout all the frameworks Coco
carbon classic one of the things I
didn't mention the process manager is
all the applications within classic
actually have an individual PSN so
they're each uniquely targetable that
way okay now I'd like to bring Steve
Sellars up it's going to do a apple
event related demo okay I know that
we're short on time so I'm going to talk
fast one of the things that's really
exciting about Mac os10 is being able to
mix all these different API so the demo
i put together involves a version of
apple works that just runs a little
database here so this is a classic sorry
i cook a carbon application that runs
unmodified on mac OS 9 and 10 I just
grabbed it off our server and double
clicked on it so I've got a little
database here and now i have a little
tool in this window that you probably
can't see very well and what it's going
to do is start up a server that listens
to connections only from the local hosts
to execute
applescript so that guy is going to
start up I'm going to go over to the
other machine and this machine out by
the way is also running Apache so what
we've got is a CGI script in the apache
server going to talk to my command line
tool that is going to send apple events
to the to the database so we bring up a
browser over here right so here's my
window over here with a bunch of
predefined searches and things and as
I'm executing searches you can see that
the database over here connects to the
server performs the search and then
displays the search now that's obviously
not something you want your database to
do all the time if this were a
commercial database like one of you
would develop you would return the data
without updating the UI but you could do
that in the background application
without having to present to you I to
the user at all so over here you can see
the session transcript of what the perl
script actually told the command line
tool to send to the database over here
and what gets returned is standard apple
script results which are in turn
interpreted by the perl script and turn
into HTML all circuits at once because
that's more interesting to watch over
there so as you can see you can put
together a real application a real
workflow using carbon in your
application okay Steve
[Applause]
okay we're at the more information
section probably seeing the carbon
documentation URL but that's up there
again the to release notes that I
referred to are on the DP for release in
system developer documentation release
notes carbon core HTML is going to go
over some of the issues that I mentioned
with the memory manager and code
fragment manager and then I'm sorry
memory manager then the code fragment
manager has its own release note on how
to enable the new vector libraries and
now that summary so hfs+ api's that's
the big one most of the applications
that we've seen have just been ported no
they're still not a lot of them have
adopted at hfs+ api's now there's even a
bigger reason to adopt these eight api's
and in particular using these api's and
using multiprocessing api's for a
factored application is a very good
proof it's a proven combination that's
worked well on OS 10 so to get back to
one of the carbon overview session scott
forstall was actually up there saying
that he had some challenges for you I
have the same challenges and I really
like to see that the development
community take us up on this in
particular with these core services we
need feedback on things that may be
impacting you in terms of performance in
terms of features we need things earlier
rather than later particularly these
layers of the system we can't rev at the
very last second just because of the
dependencies so I encourage you to bring
your app up on DB for to be beta with us
when we ship the public beta and to
release the same time as we release so
is 10 ok now I'd like to bring up a few
of the people on my team for some quick
QA actually John
we are out of time oh well you pledged
Holden let you had a lot of information
and you did so okay actually it looks
like we have well I've ever naps we
don't I can't even start it because of
the time yeah okay well I can take some
questions we can probably go yeah back
over there so I thank you all so much
for coming to this conference again the
feedback that you've given us is so
great i know the Apple engineering is
Benway encouraged by the show and
hopefully you have been too and we'll
see you all next year
[Music]
you