WWDC2004 Session 106

Transcript

Kind: captions
Language: en
[Applause]
alright good morning this is the service
management for Mac os10 talk or as i
like to call it writing modern damon's
and mac OS 10 so what does that mean
let's jump into some terminology what is
a daemon a daemon is a long-running
background process people don't tend to
see it it's not in their immediate view
like the finder or Safari well we don't
just have damage on the system we have
super damon's to what is it super Damon
a super Damon is a daemon that proxies
work on behalf of other damage to make
their lives easier we also have a notion
of an agent which is more specifically a
daemon that runs in the context of a
user after you login something like the
ichat agent might run on the background
do a help i chat out finally something
that's very important for this talk is
the notion of a communication handled we
talking about this a lot communication
handles a file descriptor or a mock
board something used to talk to somebody
else be it near or far also some
assumptions I'd really like to assume
that most of you have prior experience
writing a mock or UNIX Damon also some
familiarity with the mock or UNIX system
calls to be very helpful let's jump
right in so what is this session about
it's all about background processes you
do work on behalf of a user either
directly let's say like someone telnet
then the telnet server is launched or
indirectly a thing like look up d which
is doing queries on behalf of
applications to look up users or
internet hosts or whatnot and finally
your code needs to get running at some
point you have work to do how do we know
when to run us at all now what's wrong
with the status quo I believe and we
believe the daimons deserve better
treatment and both the UNIX and mac OS
world Damon's were just prophecies which
disassociated themselves from the user
a might say core graphics nope don't
need you a call functions to remove
yourself from the users at context in
the mac OS 9 world we call these spaces
background applications again they were
just programs which said nope don't need
this don't need that oh yeah no don't
need this subsystem over there that's
not the way to go the solution we're
going to provide a new super Damon to
manage all the daimons on the system
it's designed to do work for you it's
designed to be flexible it's designed to
support messaging and control and
finally we're going to call it launched
deeb a good major component of this
since we want all damage to be use this
is that launched II will be open source
this is a critical objective of ours we
want UNIX damage to adopt this
technology things like Apache and bind
and you name it any unix Damon we'd like
to adopt this technology we want to have
it the open source we want to have it
run on freebsd and linux and solaris and
all those communiqués is it's in
everybody's best interest so we're going
to learn in this session issues a modern
Damon writer faces things that affect
you as a developer what launch T will
provide for you what it won't provide
for you and how to port an existing
Damon to launch deed also I'll talk
briefly about how to write a savvy Damon
so let's let's talk about some history
why do we need to write a new Damon well
in the UNIX world the classic super
Damon is ina d ina d is responsible for
things like running the telnet server or
the ftp server you name it well why was
it created it was created because there
was the observation was made that every
Damon bit did basically the same thing
it opened up the socket and sat there
waiting on accept and then forked a
child and then had the child deal with
the client well if they're all doing the
same thing we can have one Damon do that
for them and not have 20 Damon's all
running at the same time now in
hindsight I nandi was IP centric it also
assumed that
there's only one communication handle /
Damon this is definitely not true these
days with the invention of things like
SSL almost every day min these days is
listening on at least two ports one to
handle SSL traffic London to not handle
non SSL traffic also in the UNIX world
we had something called an it and it was
centric to a different kind of
technology it was centric to TT wise
things like the console login session or
a login session running on a serial port
off on the side that was you know I'll
find a dandy but it was only specific
towards the specific technology also in
to handle different kinds of daimons we
had this call-out starting from etsy RC
when the system booted we had a shell
script run it would then in turn run
other things and eventually a script of
yours would be run that would let your
daemon be launched well this is very
very static it only happens at boot
there's no notion of you know the world
might change we turn the system off I
mean we put it to sleep and woke up back
up somewhere else and also didn't d able
to shut down very well finally and they
also had cron at and batch there were
time centric that's great but maybe you
want to run for a different reason maybe
you want to be like send mail and run
once an hour to deal with spooled mail
but you also want to deal with incoming
connections on port 25 in the mock world
they had something similar to ina d
called maken it mocking it can launch
damon's on demand based on mach port IPC
that's great but it's only from awkwards
what about file descriptors maybe need
both so that yes that brings us to
today's problems there's missing
functionality first of all I nadee
doesn't handle UNIX domain sockets
despite them being file descriptors we
have file system Bates events to trigger
a daemon launch maybe your daemon is
this sits waiting for a male spool to
fill up and then try and drain it
periodically well we'd like to be able
to run your daemon when
and now that school directory gets files
in it also ina d and enid don't support
user-supplied jobs you know why can't a
normal user just submit a job to inet d
saying hey when somebody connects to
port five four three two one run my
little custom Damon you can't do that
today because the config file is in
flash etsy and only route can write to
that file that's a shame also multiple
event sources like we talked about with
I net D it's only allows you to have one
communication port and that's just not
practical these days we're still some
Damon's listen on both mocking UNIX file
descriptors something like the multicast
DNS responder is listening on a mach
port to handle requests from the GUI abs
but it's also listening on file
descriptor to handle the network event
there's no reason why we can't launch
that Damon on demand today except that
our super Damon's only support one
technology ultimately we believe that
time file system IPC events all these
various events sources need to be
supported in the sames it's super Damon
we need to be able to launch on demand
whenever any of your events happen also
a critical request by many people is
that their Damon automatically be
restarted if they inadvertently exit or
die for whatever reason so that's
something else we'd like to support that
not all the super dam and support so the
future we're going to have one Damon to
rule them all its support for
transferable based event sources at the
moment we support most file descriptors
and we support we will support mock
ports also support for user-supplied
jobs is we believe a very critical
feature we want users to be able to run
their own damage to support their own
needs and not need to run their own
super damage just this when we already
have one running on the system so what
does this mean first of all we want to
save you work in the UNIX world you need
to fork you need to call set said you
need to close stray file descriptors
reopen standard IO is Devon all it's
just a mess it
again it goes back to the we were a user
process now we need to disassociate
ourselves from the user input and these
are some of the items you need to do and
nobody likes trying to keep on top of
any new API and you need to call just to
disassociate themselves from the user
input what are we going to do when you
hit main your program will be pre
demonized you can check in and go we're
also going to support launching on
demand this helps you help us save
system resources we will have your
communication handles open and alive
even when you're not running and should
someone connect to them we will then
launch you it also improves the system
boot-up we don't need to run your entire
executable if we just need to open your
communication handle that's less work we
needed to do at boot and our users will
appreciate a faster boot-up it also
means we can do a parallel load of boot
we plan or actually we do implement the
registration of all the communication
handles of all the damage all at once
that means that if Damon a needs to talk
to Damon be we don't really have a
dependency problem because the
dependency is implicit by their
communication through a file descriptor
0 mach port we register them at the same
time we don't need to worry about it
which one gets launched first which
helps us in too if you're used to start
up by them as little external
dependencies we've been listing there
you don't need to track them anymore you
don't even need to list them also since
we want to support user-supplied jobs we
can support something called user agents
we're going to standardize the way
background processes run for user prot
users and allows them also to launch
those on demand we've never been able to
launch user agents on demand in any
standardized way this will help our
login performance which our users will
appreciate finally this is something I'm
really excited about we're going to
support messaging things like you know
hey Apple file server how many by
yourself tell me how many people are
connected tell me you know who they are
tell me about them we'd also like to
support controlling damage things like
hey AFP servers
you disconnect that user they're done
now or please shut down and the daemon
can come back to us and say okay I'm
ready to shut down now I've written out
my log files I've closed and cleaned all
my stayed thank you I'm done now this we
believe this is very important for
things like database servers which might
have a lot of work to do before they
shut down the old way in the UNIX world
was just to send a signal cross your
fingers and hope that they exit someday
and you know that's that's not great we
need to be able better than that but you
know the number of messages that we can
support or potentially endless and we
look forward to feedback from you on
what met kinds of messages you'd like to
see sent to you or messages you'd like
to send to us what would you like to
tell us about yourself now let's talk
about some case studies the real world
is what matters we have something called
cups beyond the system it's a common
unix printing system they have a daemon
today what they did to support
restarting is they use some mock API is
to check in with mock ahead and tell it
hey restart me if I die that was extra
code they needed to add it just to get
restarted in case they exit with launch
d there's no extra code we know you want
to keep running your a daemon you exist
to serve other people this is a
simplifies their port to Mac os10 and
makes life a lot easier for them
something I brought up earlier is the
multicast DNS responder it uses both
mock ports and UNIX file descriptors
we'd like to be able to launch that on
demand but today we have an ID on the
right hold just supporting file
descriptors I'm mocking it on the left
holding just mock ports they don't know
anything about each other one could
launch mdns responder based on
communication on one type of technology
but not the other leaving problems for
the mtns responder if they want to
launch on demand launch the handles both
nothing else does for launch on demand
and we're very excited about this
support for multiple technologies also
back to the user side let's bring
another case study example the ssh agent
is a classic example
of a command-line process that people
want to run to support their SSH
activities a lot of people usually end
up creating these complicated statements
at the beginning of their shell startup
just to make sure that they have only
one SSH agent running on the system so
they'll create lock files then they'll
try and see if they already have an
agent running and if not run it and then
unlock the whole you know state that
they built up well with launch Steve you
can just drop an XML plist into a
directory in your home directory and at
login time we will register the ssh
agent along launched on demand the next
time you talk to it and only one of them
for your login session which means all
that complicated locking and checking to
see if one's running nope don't need to
do that we're very excited about this
for user agents we hope to see more
cases where people use it also the
changes to ssh agent were very simple
this is something we're excited about
because it means less work for you guys
we know pretty much how demons operate
their state engines we know what
information they need in order to get
running we can get that to you right at
the beginning of your process and again
your pre damon eyes so you just take
your check-in information and you can
get running so okay I'm totally about
what launch T does but what doesn't it
do well there's a lot of other kinds of
events on the system we have config DS
database of key value pairs or config
these events we have the net info
database of key value pairs which can
come and go as the system runs we also
have the rendezvous service
advertisement some people want to launch
on demand just by virtue of another
entity showing up on the network we also
a very popular one is the io kid
namespace which is built on top of mock
ports and I okay at events again up on
top of infrastructure already which are
is built on top of mock boards now we
wait you know not i okayed events well
we have things like the Bluetooth Damon
to launch one bluetooth devices show up
we Apple have our own needs we know you
have your needs so this list that I just
gave a things we don't support is
subject to change we're probably most
likely add those subsystems in the
future as we figure out how to deal with
technologies that are built on top of
each other now porting let's start
jumping into the meat of things we have
a simple very simple IPC API and a very
simple rtti based object system to
support the message passing that we want
to use to communicate between launch d
and your daemon now what does the IPC
APA it looked like well it's kinda sorta
corefoundation well why not core
foundation portability just about what
we talked about earlier launch d is open
source this is a very important goal for
us to drive adoption and minimizing our
dependencies helps make adoption easier
also a stumbling block is the mach port
and file descriptor passing is not
supported by core foundation at the
moment it's very few people requested it
was just easier for us to implement a
simple object graph ourself and support
serializing and sending it across to the
other side finally really all we need is
our TTI dictionaries and arrays we're
just sending messages back and forth so
to give you an example of how simple it
is this is what the two basic CPAC api's
look like we have launched message which
supports sending an object graph and
getting an object graph and response and
if you're going to be doing any kind of
asynchronous programming we have a
simple API to get the file descriptor
that we're using on the backend to
communicate with launch key and add it
to your own run loop if you're using
case you select it's up to you we can
give you the file descriptor you can add
it and you can call launch message when
you get a when messages are available to
read now to jump more into the meat of
it a launch data t represents an object
graph it could be anything you know we
have the rtti that's what makes it fun
and you can introspect it as you get a
response back or you can send just about
any
you want also launch meet message while
it appears synchronous for the common
case is actually what launch method is
synchronous for the common case it
returns null and sets error no on
failure plain and simple for the simple
case now if you do send the message was
requesting that asynchronous messages we
sent in response you can then call
launch message with no message to send
by using null as a parameter and then
you'll get any asynchronous message back
and you can keep calling and keep
calling and keep calling until you get
all back and if errno is equal to zero
then there's no more asynchronous
messages and if it's obviously equal to
something not equal to 0 then there's a
problem now let's talk about those
messages we're sending back and forth
you know rtti and container classes are
fun we have dictionaries we have a raise
and we have some basic objects to match
the see basic types we have file
descriptors mock boards integers real
numbers boolean strings and finally
opaque data should we need it now again
it's just enough for IPC and no more we
have getting set operations for the
basic types and in the case of
dictionaries we allow insertion lookup
removal and iterate again all we need
for IPC for a raise gettin set index and
get count it's enough to introspect and
set up an object graph that's all we
need now to understand the object graphs
are going to be sending back and forth
let's talk about the XML plist that
you'll use to configure a daemon for us
to run into the later point in the
system you're going to have something
like a label to uniquely identify your
job and that really is the unique key
this is in case someone wants to control
your damn and the label is what they're
going to use to send it can turn it on
turn it off change some attribute of it
that's the important key you can have
things like the username you want the
damn and run as the group name program
name and pretty much any kind of
environmental setup you need to get the
daemon running you know I want the
daemon to run a networking directory I
want to have this argument and
environment variable any kind of setup
detail we plan on supporting now if you
notice at the bottom of the list back
there we have something called a vent
sources they're all about the details of
how to setup file descriptions and mock
boards you know who to connect to you
know where to listen particularly file
descriptors or can be very complicated
since they're different technologies we
can back them you've got ipv4 ipv6 UNIX
domain sockets it could be a new type of
technology that we don't know about the
requires different API is to set up the
file descriptor and we need all these
details put in one place so this is what
the event sources are now before we send
an XML plist over to launch key we
actually do some data distillation we're
going to turn those usernames into you
IDs the group names into G IDs and we're
going to take all those events sources
and actually translate them down to the
file descriptors and mock ports the back
some and then the resulting object graph
is much much simpler it's and requires
no lookups by launch d to find out the
actual details of anything and the great
thing is since we've translated things
to follow the Scriptures any kind of
security check the colonel would do we
don't need to replicate now let's look
at some example messages life's pretty
simple you might want to submit a job
and then you're going to have that you
know translated xml p list you might
want to remove a job well then you just
need the label you might want to get the
jobs you want to introspect launch do
you want to find out about what damages
are running on the system I don't get
you a big dictionary coming back you
might want to check in your daemon you
can call it send a check and message and
cut out the file descriptors and mach
port so you have allocated to you will
come back and you can proceed with them
as you wish and finally something that
supports for things like the ssh agent
is we need a notion of getting unsetting
and querying and environment that isn't
directly tied to your shell now
something that we're still working on is
becoming launch T savvy if you remember
I talked to you about sending
messages to control it tell me about the
users you have connected to you please i
want to disconnect this user over here
we're still figuring this stuff out we
look forward to your feedback and
definitely some updated documentation
for you in the tiger time frame so that
way you can understand what
introspection and control operations we
supply now let's hash things out again
so pre demonization we believe this is
very very important for you it means
again you're going to hit main and your
demons ready to run you don't need to
worry about disassociating yourself from
any kind of sub system on the system via
the graphics or maybe applescript i'm
actually don't program at that level but
at least in the UNIX world no forking no
call set said no closing straight file
descriptors left behind by who knows
what she'll life's good now you just
need to check in and go also again we
automatically start support restarting
so should you decide that your
configuration file change do you want to
rerun just exit will launch again should
next time someone connects to you also
something we hope you will take
advantage of is the multiple event
sources you might have a file descriptor
or 20 file descriptors we can listen on
all of them on your behalf and run you
whenever activity happens on any one of
them if you listen on a few mock ports
sure tell us about them well launch you
when activity happens on them too we
also the user agents are very excited
about should you have a foreground /
background application combination well
maybe you don't want maybe the user
doesn't want the foreground application
to necessarily run every time it log in
but they still want the background
activity to run a the example I like to
continuously point back to is the ichat
agent you have the menu lang it supports
showing you which users connected but
maybe you don't want I chat running a
plugin but you do want the menu you do
want to log in well with launch d we can
run the ichat agent at login without
running the foreground application
so to give you an example demo I would
like to show the ssh agent running under
launch d and i would like to show our
debug damon running under launch d and
acting as a pseudo web server to tell
you about launch d itself by doing the
introspection that i talked about so to
show you let's show you the web server I
have this is a representation the object
graph coming back from launch Deeb shows
you that here's our key we have the
combat apple to launch debug d we run it
on demand there's a boolean set to true
it's got a pit at 328 if I reload this
web page you'll see it you know we keep
launching on demand because this number
is changing we have some you know
various setup details in which you ID to
run it as if you notice it's running as
me the user finally there's two file
descriptors we're listening on they
happen to represent the ipv4 and ipv6
sockets or set up by the user and also
look at this we have an SSH agent it's
also running on demand it doesn't have a
pig set yet because no one's tried to
talk to them yet and this is the unix
domain socrates listening on so if we go
back to the terminal we can run ssh ad
and now if we go back here and run it
again we see that the ssh agent launched
on demand how did this all happen well
and goes back to the Simple pea list we
talked about dropping on the system if
you see until the library launch agents
we have a couple p list there's a launch
debug d that i talked about and the ssh
agent and they're pretty simple files we
got com openssh all we have is again
here's that label here's the program
arguments and the listeners which event
sources which get distilled down to an
actual file descriptor so in this case
we have a little secure socket with the
environmental key which if you're using
ssh agent if you're familiar with and
the socket type we need now that can
show you how simple it can be too oh I'm
i will show the
the apple so here's the debug dammit for
example it just listens on a port 1 2 3
4 5 happens to be a passive socket and
so it's like sock stream and what ends
up happening if you're familiar with the
get a tour info API is we take these
details we throw it a get a tour info
and take whatever at our info structures
come back and allocate a file descriptor
for each one of them and then send it
over to launch T and it just listens on
them so in this case ipv4 and ipv6
sockets came back and you know that
could change in the future and your
daemon wouldn't even need to know about
it a new type of communication
technology might come out and you don't
care as long as it's a file descriptor
and it comes back to you you can get
events off of it and handle them so that
we're very excited about this and I
believe it's time for Q&A now this is
yeah oh also for more information you
talk to Jason and yeah and this source
will be on the developer website at the
determined developer website