WWDC2001 Session 304

Transcript

Kind: captions
Language: en
good morning my name is Tom wire I'm the
network and communications technology
manager in developer relations I'm going
to introduce session 304 which is
extensible kernel networking services
hopefully many of you have made it over
from the previous session in room a one
so with that I'd like to introduce
Virant Dumont who is one of the core OS
networking engineers a good morning I'm
going to talk about the extensible
kernel networking services and in your
case in your case we'll see that really
mainly what we're talking about here are
kernel extensions and how to create a
kernel extension to add some networking
functionality to the kernel so as an
introduction Mac OS 10 networking
architecture is extensible so we get
mechanism where you can add firewalls or
VPNs or content filters or network
drivers so this is the goal here is that
a little bit like in Mac OS 9 where you
get a mechanism for extensions here you
can add your networking extension
without having to recompile the wall
kernel like you would do in a regular
freebsd type of environment so what
you'll learn this session will we'll
talk in some details about the network
kernel architecture so what we have for
Mac OS 10 and Darwin we kind of use both
terms because everything here is in
Darwin so it's all open source so you're
really welcome to look at all this you
know how to source yourself from from
the darwin kernel so and we'll see how
to to do and filter and intercept
packets from different points in the
kernel from the socket layer from the
lower levels and we'll see that and also
how to add network interfaces and
drivers
in the colonels and one of the
interesting things that we may you may
learn here is some interesting tips
about the specificities of Mac OS 10
kernel and some of the changes you know
coming from Mac OS 9 or even from some
FreeBSD or Darwin I mean that that are
different in Darwin from from FreeBSD so
there is different behaviors and some
tips here that you may you be interested
in so this is the pictures that you
probably have seen many times by now so
networking in Darwin is part of the bsd
kernel so I think it's it's always
interesting to say that there is really
three parts in in the kernel in Mac OS
10 you get the BSD subsystem which is a
lot of GAAP eyes that are sockets like
we'll talk about later and the file
system and we get the mark kernel which
is a base for VM and a bunch of all the
core services scheduling and so on and
also the i/o kit subsystem which is
using the driver so just because of this
networking as some different interaction
with the rest of the kernel you'll see
at the i/o kit layer or Mac kernel and
we get some some differences here so if
we go a little bit deeper in details
about the networking subsystem we see
that we can roughly decompensate it in
like four layers we get the socket part
which as you know and you probably were
probably in a Vincent session earlier
and you saw that socket is main EPI of
networking in 10 and so we get the
socket layer and we'll see that there is
some mechanism where you can extend the
socket layers you can plug yourself in
the socket layer to do filtering and
things like that the other parts the
second layer here from the top is a
network protocols so this is where
things like tcp/ip or appletalk or your
protocol lives we get another third
layer which is
something added in in the Darwin kernel
which you won't find in in FreeBSD which
is a data link interface layer we'll
talk about this in more details but
basically this is a an abstraction layer
for protocol and network interface to
meet for extensibility and also we get
the network interface layer which on one
side is going to communicate with IO kit
and you know absurdo drivers on the
bottom side and on the upper side is
going to talk to the GL il the data link
interface layer so we'll go in details
on those four layers so networking and
our way into to summarize this it's
currently based on FreeBSD 3-2 so we get
the tcp/ip stack socket and a bunch of
other services for networking from the
FreeBSD 3-2 implementation so if you
were a little bit aware of this you know
it's it's not the latest version of
FreeBSD but it's it's a very solid base
for what we're doing here so it's a very
robust and proven implementation FreeBSD
is used in many places and servers and
things like that
so what that gives us it's a it's a
stack which has a lot of life and it's
been you know a lot of problem have been
flushed already so we inherit all those
you know improvement and and and all
these things in the Darwin kernel
however we get some Apple enhancements
there for plenty of reasons that will
detail the FreeBSD stack by itself was
not completely meeting or needs so we're
trying to have a more dynamic approach
where people can load and unload without
you know rebooting and without
recompiling so we also have support for
mps the freebsd stack right now is not
really MP savvy so we get some mechanism
here for multi-threading and MP
efficiency in in the networking
subsystem intent so we also vincent
talked about a little bit before we also
tune some buffer allocation for both
client
and some you know high-performance for
gigabit and things like that so that's
also some of the modification we've done
to this stack and so we get this famous
datalink interface layer which is a
mechanism for extensibility at the
bottom of the stack for heading
protocols adding Network families one of
the biggest problem with freebsd is that
basically when you get a new driver or a
new class of driver and your family of
driver you more or less have to go
pretty deep into the kernel and we
compile the kernel and get that to get
that to be integrated in your in your
stack with Mac OS 10 + GL IL we we have
some mechanism to do this on the fly
basically and you can you can load and
unload your drivers add some type of
different family without having to
restart the machine so and as we said
it's extensible because of those plug-in
architectures and network kernel
extension we're going to just talk about
so the network kernel extensions what
are they they're they add extensibility
to the kernel networking they are
basically part of code that's going to
link against the kernel dynamically and
that will be part of the kernel so this
is a big responsibility
you as developer are going to make some
iron a filter a new protocol something
and using the network kernel extension
you'll link your code to the kernel code
as you know in Mac OS 10 and it's been
it's been said in all the session but
there is a pretty hard blender in
between user mode and kernel mode in
user mode basically whatever you do if
you crash your application you're not
going to bring down the machine when you
run in the kernel you're in protected
mode and you have access to all the
goodies and all the resources and it
means that if your your your code does
something wrong it's going to panic and
it's going to be a very bad experience
so kernel extension in that sense are
something that should be used only if
what you're trying to do can
done in user mode if what you are trying
to do as an application using networking
can be done in in user as we you know
without running in the kernel it's it's
always best to do so why because if your
application crash for some reason you'll
bring down your application user you
know may have to restart your
application but life will be fine
if something goes wrong in the kernel
it's a reboot and that's really
something we're trying to avoid that all
costs so this is a first word of caution
we're going to have several of them
during this talk kernel extensions are
absolute potential to crashing machines
you have to be really careful about what
is done there so what you can do in a
network extension in basically you can
do a filter in case so filter and case
are we'll see can modify and inject or
drop packets so I've added those
different layers we've we've seen before
you'll be able we're gonna go through
with a different type of filter in case
you can you can make but those filter
will intercept packets at some point and
you'll be able to do whatever you want
with a packet so you can decide to let
it go through you can decide to modify
it you can decide to swallow it
duplicate it whatever you trained your
you want to do you'll be able to do that
through filter and K so this is a very
powerful mechanism however as I said you
have the potential to you know if you
somebody if you sending back a packet
which is bad and it's gonna crash
somewhere up the stack you know it's
it's it's a bad thing so you get to be
really careful about what you're doing
Network kernel extensions are are using
the i/o kit functional it is so through
zio kit mechanism you can dynamic load
the dynamically load and unload the end
case and there is no need to reboot this
is just something that you can do by
yourself or you can have through a set
of dependency your nkb loaded
I will see in the next slide a thing so
again you're running in the kernel so
big liability be careful about your
pointers you know be careful about
you're doing when you're unloading you
have some special things to do to make
sure that you do live in a clean state
after your module unload so proceed with
caution so another important point is
that there is no real API the networking
subsystem is wide open
you're looking against all the symbol of
in the kernel so there is no guarantee
of binary compatibility in the future
just just a simple example here if we
change a structure or even worse if we
change a macro and you have linked with
an old version of this macro somehow
this is going to work you know I mean
you won't have any resolution problem
because this is a macro but in the lying
implementation of the macro is going to
be different from what is in the kernel
and your nke may work for a while but at
some point when it's going to get this
macro it's going to do the wrong stuff
and the consequences might be harmful
you know you can potentially panic or
even worse panic in two hours later
because you you freed the wrong stuff so
this is this is a problem here but you
get to be aware of this the binary
compatibility is not guaranteed in the
future because we may have to fix a
stack we may have to fix some of the
things so there is some mechanism in in
in I orchid that lets you declare
dependency and that will make sure that
you your n-k won't load if the kernel
version has changed and things like that
so so those are things that you need to
look into when you're doing an NK Darwin
is not BSD Darwin is based on BST so
some of the rules apply most of the
rules for if you're used to BSD apply
but not all of them and we'll go in some
of them we'll talk about things like
funnels or things like where you have to
be a little bit careful with MP so I
don't expect you code that you just got
from
from open source for the kernel part to
just compile and run in in Darwin it
probably will but you need to have to be
careful to look at some of the aspects
we'll talk about and you get a pretty
good eye on it and see what are your
areas that may give you problem into
Darwin so don't expect this to just
compile and Link it into Darwin it will
work you may have to do a little bit
more okay so the nke dynamic loadings so
as we said before this is provided by I
Oh kit so basically a kernel module is
it a filters that your protocol you know
whatever you do as a as an NK II will
have to required entry points a start
routine that would be called when your
module is loaded and stop routine which
is called when your module is unloaded
this is basically it all the rest is is
in term of of the of the NK i okay side
of things all the rest is going to be
the filter or whatever you're going to
use as a mechanism to plug yourself in
side the kernel so the N KS are not
automatically loaded you have you have
to load them or invoke their loading
through a startup script or through
dependency you can have a Shane of
dependency like that will require UNK to
be loaded this is pretty much what's
going on for some of the NK we get in in
the system library extensions as well
the Apple NK is live some of them are
loaded through scripts and some of them
are loaded through dependency so there
is three command-line tools that you
should be aware of for you know special
issue debugging and doing your own case
Kim K X load is going to load your n ke
came out stats will give you a list of
all the NK loaded and came out and load
will
G&K UNK you won't want load if there is
some symbol collision with what's in the
kernel this is I think in the next line
know one of the thing I wanted to add
here is that when you're gonna do and
then K you should use some prefixing for
for your your symbols there because
you're going to be in the same address
space in the same symbol space as a
kernel so if you declare a function
which is this is B connect right you're
going to have a problem because this
function is already inside the stack in
the kernel so you module won't load and
also we would like people to somehow
prefixes or symbols with their own you
know kind of prefix I don't know maybe
their vendor code Apple OS tied to to be
sure that they're not going to conflict
with anything else or any other other
vendor and K is that might load so
something to think about
so different we alluded to this a little
bit but different types of nke
so four main types socket filters
circuit filters are it's a socket layer
we'll go details on this so basically is
it lets you intercept socket calls
network protocols network protocols can
be implemented as nke also and there is
mechanism to add those Network protocol
to your to the Darwin stack dynamically
data link interface layer filters so
there's different types of filter here
for more of a lower-level type of
filtering and also network interfaces if
you add some different a new family a
new type of interface you might want you
might have to to have some nke to
describe that and let the protocol and
yell il know about you know family so
we'll go in those intruders so first of
all the socket layer so just for people
who are aware of the of the unique
the freebsd way of things you know the
second layer is right and there's a user
to kernel boundary it's got a pretty
interesting role here is that so socket
layer basically is doing all the copying
of buffers in and out of kernel space so
by this when your application under Mac
OS 10 when your application is trying to
do let's say ass and call it's gonna try
to send something on the network the
buffer live into your users read in your
application and they need to be put into
the kernel memory and this is done at
the socket layer the socket layer here
is going to take some memory I mean your
buffers and put them into the socket
buffer queues will go in details a
little bit more details about this and
the copy to the kernel space is done at
the socket layer one thing to note for
people were more aware freebsd is that
here your the execution of your of your
thread is going to continue in the
kernel so your users your user thread
will will do the same so we'll do that
from the from that's right so the socket
and the other side the socket layer is
where the protocol stack are going to
send the buffers and basically by a
mechanism a way get to get the
application awakens applications read
awaken we're going to come necessary
input thread and we'll by some mechanism
will get the application awaken and and
the buffer will be copied from the
socket layer back to user space if you
remember what Vinson was talking about
this morning about so was a xti sand
bath sighs well this is where they are
basically we get two sets of cues your
one for sending and one for receiving
and this is zebb some some you know some
parameter ball size so the socket
filters actually live in to the socket
layer so they're their way to intercept
packets both
coming from the application or going
from the application of going up to the
application so different sets of course
if you aware of the socket calls most of
them have some mechanism here for
filters so what we'll do is when you'll
create an nke socket filter will will
have a mechanism to know that your
socket your filter and into your code
need to be called and depending of which
call you added your filter to you will
be calling you'll receive the packets in
this side easier to modify drop you know
do nothing most of the time but you have
you have the flexibility to do something
with packets coming in and out of the
socket layer so this is one way of
putting a it's a Content filter I'm
talking about all this in this slide so
sockets again is the API for networking
this is the native API so everything is
going through the socket layer socket is
a glue in between application and
network protocols we talked about this
this is where you cross the boundaries
of the kernel to user binary it sits
above the network protocol so the
network protocols are not in the socket
layers the second layer is going to
decide which protocol need to be called
so I get this kind of a file structure
it's it's followings you know unique
type everything is a file system more or
less so it's it's a subset of that it's
got some specific calls but you know
it's it's it's following someone says
you can do a read or you can do a right
on your socket so we talked about this
socket has a pair of packet queues for
incoming and outgoing packets so your
packets are going to sit in in in those
queue specially on the way on the way up
until until basically your your
application is awakened and it's going
to grab those packets so as we said
though inside sockets have plugins each
of those call that SB receive for
example it's going to run
look through all the filters that are
associated to it before it actually does
the call so you have the opportunity to
just discard the packet if you want or
change it it's one-way
okay so socket filter NK yeah which we
just talked about this oh yeah the other
thing is you have you have two type of
socket filters here you can have which
probably is the simplest one a global
socket filter your your filter the
filters that you create is going to be
invoked for each sockets so every socket
is going to run through your filter even
if you don't care you'll say okay well I
don't care and you'll return and the
packet won't be touched and and in the
application I mean and the processing
will continue there is some a little bit
more complex way of doing things where
you can do programmatic circuit filters
that sim will only apply to a certain
type of sockets you know let's say you
just want something for for a web
traffic or something you have some way
to decide that your socket filter will
apply only to that you have to register
your n key and always Apple so those are
for people familiar with mac os9 those
are z/os types of n the types so that
said that let us identify the sockets
and there is no collision when you
insert your sockets your socket filter
can be run after you know some other
vendor x socket filter so you don't know
that
so we have we need to have some way to
identify all socket filters so you need
to use the NK ya know so see with DTS to
get your end alter and as we as we said
an example of this this is a content
filter where you're gonna decide based
on some you know your own criteria
what's what you want to let go through
in those packet set that you receive
each time you you get cold so yes sir
either you change them you you swallow
them you do whatever you want you
duplicate them that
which you know whatever you content
filter or use your circuit filter is
going to do so important points for the
for the socket in case there is no
built-in reference tracking what we mean
by this is that you have to in UNK you
need to keep track of where your socket
is your socket filter is inserted if you
are inserted in 15 you know different
sockets you'll receive you'll know that
and basically what it means like if
somebody asks you to unload you get to
keep track of those insertion and I
going to explain briefly what's going on
that when we run the socket filter code
basically there is a pointer to your
socket filter handler where you're going
to receive packets and if you don't
correctly remove all this when you are
node we're going to call into some codes
that doesn't exist and guess what it's
going to panic so it's all your
responsibility to take care of this and
refuse to unload it's it's pretty it's
pretty reasonable if your if your socket
filters in a state where it doesn't know
or for some reason cannot really unload
to the refuse to unload and people won't
be able to unload your your socket
filter but it's much better than
panicking five minutes later so that's
something you need to be aware and as a
warning you're in the kernel it's it's
pretty much wide open you know you have
wide open access to all the structure
all the thing but you're part of the
kernel you just a function if you're not
there you get to do the right stuff to
plug the hole basically and don't leave
you know null pointers and things like
that hanging around because that's
that's going to be a pretty bad user
experience so that's why we another word
of caution about using kernel extensions
you got to be really careful okay so now
that's that second layer when we looked
at what basically as a networking in
Darwin is the network protocols two
examples that come to mind here is
tcp/ip and appletalk so that's that's
more or less something which is pretty
close to what you'll find in freebsd as
a way to register protocol and add
protocols is pretty much the same we get
some some mechanism to let you do that
dynamically add and remove dynamically
your protocols and your domain so this
is a second layer we're going to talk
about so what's important to know here
is that a domain you find a protocol
family one of the big example here is PF
I net this is where all the TCP UDP and
IP live another one that you can think
about that we have in Darwin SPF I need
six which is four for the family of ipv6
so this is some things that that that is
pretty much covered in a bunch of bsd
books the protocol family and how the
protocol handler works so this is this
is what we have in darwin here from the
socket layer we're going to decide which
protocol handler depending of the of the
type of sockets you know the address
family you put in your socket and if you
have TCP you know you do a connect
we're gonna call TCP connect and this is
done and this is reduce structure where
you can add your own protocol if you
come with a next you know cure protocol
family that's gonna replace IP this is
where you're gonna add it so if you do
that you know please do it and put it in
that way take it so this is extensible
same thing can add your own protocol and
there is a mechanism to to declare this
from your NK again you can remove that
when you remove it be careful
clean up after yourself
and otherwise you know very bad thing
will happen in the kernel will panic
so another one the third layer here that
we we have in in the Dow in kernel is is
a new one
it's it's called Li L that telling
interface layer so what will go in
detail about this it's it's in between
the network protocols and the network
interfaces the goal of this layer is to
be basically a central point yeah
basically GL il that let us do
extensibility you can add protocols you
can add network interface network
interface type of families and also put
filters and GL il is really the central
point that's going to be the abstraction
layer for this
so this is where is a protocol attached
and say hey an IP and I'd like I'll be
interested in in receiving IP type
packets right
I'm appletalk I want to receive
appletalk packet an ipv6 I want ipv6 so
all those guys are gonna I'm gonna talk
to GL il and registers themselves for
requesting those types of packets on the
other side drivers or pseudo drivers
coming from the network interface so it
can be IO kid based drivers I mean I
okay base drivers we're not going to
talk in great details about those here
but they're basically drivers that are
moving stuff of some kind of a wire or
Wireless but you know they are moving
some physical bits if we can say that
well you get some sort of drivers like
tunneling devices and things like that
that just add stuff or remove things to
frames that have been generated
somewhere else or they can generate your
own frames but they don't they don't go
to a media somehow all those go to GL il
and register themselves and and give
some information about the type of
framing they do
the type of you know specificities they
have so GL al is here as a central point
to to end all those and add it to this
if I go back to the previous scheme here
in between those we got two types of
filter again we got in between the
network protocol we got the protocol
filters that register with the Li L and
say I want the IP packets for interface
you know zero you know your your
built-in Ethernet and you also have
interface filters that are registering
with GL al and there are more
lower-level those guys will see all
packets for these one interface say you
want to see everything coming from en 0
or C Airport GN 1 or whatever PPP and
you're gonna put an interface filter
here that will not be protocol dependent
you will get all your appletalk plus IP
plus you know whatever packets here well
as a protocol filter you will specify
that you're interested only in IP
packets or appletalk packets whatever
you want so it's so this mechanism here
it's different from BSD and this is
something you need to be aware of if you
take something like a driver or
something like that it's got to our
pseudo driver let's say channeling
driver it's got to follow some different
rules that it would in FreeBSD 3 or 4 it
will have to declare some GL il modules
there to register with GL al and and do
things differently there is some example
in the NK s and in the darwin code about
how to do that appletalk is an example i
p does that ipv6 that
so the interface layer the interface
layers is a four and I said ioki type
drivers or absolute drivers so they
attach to GL il and basically tell the
networking stack that they are available
so dynamically your airport is turned on
or something and the en one is going to
basically tell the LA hey I'm an eater
internet type of driver and you know
this this is where I am so it's a it's
more of a flexible mechanism for
attachment of detachment of interface on
the fly on the same way you register
your new you load your new protocol and
dynamically it's gonna tell GL il that
hey I'm protocol this type and they'll
take you know whatever snap ID for my
packets now I want all those packets you
know this interfaces that interface so
you'll see around the Geo tag which is
the cookie users you know the handle in
that you'll get four filters and you'll
get for protocol attachment or two
interface and all this is what identify
basically your unique connection to GL I
am so there is to go a little bit in
details about GL al as there is for
interface family per type of interface
to handle framing what I mean by this is
for enternet YP you need the address
resolution protocol and ARP is you know
is not really part of IP it's more in
between IP and Ethernet and there is a
way in DL il to give an interface family
and and specifies those kind of
specificity so here when you're gonna
our PR presentation is going to is going
to be register with GL al so if you come
if you're if you're with Ethernet and
you're using IP you don't have to do
anything if you come up with your know
your own media and needs your own
no address resolution protocol you may
have to look into this and and declare
your module with July yeah and okay so
this is the protocol module and so yeah
GL al also the modules will entail the
framing so you have two for your type of
interface if you go episode of device
pretty simple usually it's just moving
you know your your data pointer and
putting a few bytes anything but this is
what what is handled here
so there is a lot of details in this if
you're doing you know adding new type of
interface you really have to go into the
documentation and look for this the
filters are a little bit easier to do so
DL il filters so as we said before there
is local filters on top of the Li L and
and there's the protocols say the
difference what we do is that they see
all protocol diagram for an interface so
the little trick here is that you
register per interface your filter so if
you register for IP on en0 you'll see
valid IP frames for en 0 you won't see
any appletalk packets there but you
won't see IP packets for yen one for
your airport let's say I need to
register on both so they that kind of
low-level
however you get the interface filters
that give you even more flexibility
because basically at this point you'll
see the world frame so framing looking
at you know if it's an IP packet or an
apple TAC packet won't be done in the
interface filter so this is between the
interface and the family here you will
get access to the packets as they come
from the driver you'll get access to the
valid packets coming from the driver
so you'll see full frames and this is
where you can if you're trying to do
like a VPN solution or a firewall is
there one way where you could put your
nke is asking you know you know it could
but you could be doing that as a
protocol filter or
at the interface filter it really
depends what you're doing but those are
a good point for getting all the traffic
while things that the socket layer arm
or form you know getting things distinct
to an application you'll get all the
traffic for all all the sockets
basically or you know even if they don't
go to any socket if she just dropped in
the stack you'll see them at this point
it's we it's before any processing
bye-zee-bye the stacks so a good
solution for VPN and firewalls network
interface they're still freebsd STI fnet
structure you have one per interface
it's ioki based you have a lot of things
for internet internet as there's a lot
of sample and the family shows how to do
that for i okayed so there that's if
you're doing an internet driver or you
know some drivers that really talk to a
media you really need to look into i
okay because those driver don't live
into this four layers they live into i/o
kit however if you're doing episode of
device of some or something which is a
little bit in between like PDP you may
have to do some work in the i/o kit for
your driver you know dealing with media
side and also i'd see a network
interface layer here where you will have
some GL al work to get your stuff
registered with the lal and known by the
network stack network networking
subsystem is it's not part of i okay but
i okay that some way to basically give
the packets and you know call some GL al
functions to to make the interface well
not one case where if you're doing
absurd of driver like a tunneling device
you may you may do that only in the
networking subsystem you don't have to
go to i/o kit if all you do is take
packets you know add a header to it and
do some capsulation of some sort and
send it back to an internet or another
interface
you won't have to to go through our kids
that's that's the thing to know there is
an answer we just mentioned it sat here
because there's some confusion here but
those filter we're talking about our
different levels and people were coming
from UNIX may may know special in the
BSD side so BPF which is BBF is a is
really an i/o kit kind of tap the
difference it's it's well it's a
standard way and if you're not aware
coming from mine on FreeBSD 2 to get you
know things like sniffer type
application network traffic analyzer and
those kind of things those are tap from
the driver which will copy all the
frames back to BPF and and and get them
you know to your application which which
asking for BPF traffic the big
difference with the GL il filter is that
in GL il when you put like let's say an
interface filter you will get packets
but you'll decide to to let those
packets go through or make a copy and do
your own kind of tap functionality but
by default the packets there's one
instance of the packet here you get a
copy of the packet which is made and
also for internet that's pretty true for
Internet BPF opening the BPF device will
put the driver in promiscuous mode which
means you will get all the packets you
know basically seen by your interface
not just the one for your MAC address
and you know the various multicast or
broadcasts that you can get you will get
everything that is physically seen on
your segment so it's something you won't
see from Jil interface filter July
interface filter will get only the
packets that are valid for your for your
address it's not going to be in
promiscuous mode and again there are
some standard hooks in iokit network
interface for this so that's a good
model to follow if you're building your
own drivers it's it's it's a neat
utility to be able to use TCP dams
there's a bunch of
of you know services on top of that and
that's a pretty much low work to do to
get the BPF support in your new driver
so we just mentioned it here because
there's some confusion between those and
the JLA interface filters they are not
exactly the same level so no now your
kid interfaces need to support PPI folks
so that's four BPF okay
another important thing here that we
want to talk about about the networking
subsystem is the em buff so amber are
the memory buffers that we use all over
the system in the networking subsystem
to old network data if you're coming
from nine you're pretty much aware of
the M blocks which are used in in the OT
or the streams modules it's it's more or
less the same thing in in the bsd world
what we do with amber is that we're
gonna old either the packets coming from
the socket layer in m buffs or things
that are coming from from drivers so
iokit is gonna create some amber with
the packets received by let's say an
Internet driver and sends this back up
to the GL al of course and GLI L will
you know route those packets back to
tcp/ip or about or whatever but those
are all members that are manipulated so
amber are interesting because like M
blocks manipulate pointers to data so
once you're in the kernel there is no
more copy of data everything is done
through an buff so the drivers copies
are data from from you know ring buffer
to Ziemba actually and buff already and
they're passed up until they get to the
socket layer and then they'll be when
the application will be awakened and
we'll get data then they will be copied
back from kernel space to the user
application
memory and they'll be released at this
point so what's interesting in the in
Ziemba is that if we take the example of
a packet going down the example of ass
and you're sending some TCP traffic
what's going to happen and the socket
layer is that your packets from user
space will be copied in the kernel into
some m buff clusters into some man bath
and those n bath will be in the socket q
you know remembers socket / q we're
talking about earlier and what we're
going to do to send IP packet CP IP
packets is that we'll add an ember to a
part of the data that you sent and this
will logically point to the data in your
socket buffer and until and this is
going to go down to the driver and the
driver will send this and the driver
will will will say it's done but the
data in the socket buffer won't be
released until the data has been
acknowledged by the other side of the
TCP protocol so if we need to do a
retransmit of your off of this you know
packet that we took from your data we'll
do that by you know propelling a new
header and pointing to to your data but
your data will be the same in there so
there is no copy here it's only once we
know that all the data has been
acknowledged and we don't need it at the
circuit layers that it's going to be
released so that's something to know
about n buff for you as a you know
somebody who's gonna write in network
kernel extension you have a bunch of
function to access and bath to allocate
them both to many polite n buffs there
is pretty much everything you you can
think about to do with the n buff in n
buff Delta H is a good start into this
directory to look at however there is a
bunch of macro dealing with n buff
try to avoid using them as if possible
for the problem we talked about before
if we change implementation underneath
and the macro that makes let me let me
give some kernel panic if you're using
that so try to avoid the
who use if possible you know other thing
to know which is a little bit different
from from BSD is that we have a
different VM subsystem underneath in the
way we allocate memory is kind of
different
so in FreeBSD you're pretty much
guaranteed that you'll get memory when
you allocate an empath it's not the case
in Darwin so be aware of that that your
allocation of an buff can and will fail
there is two mode for an buff you can
ask for an empath with you know don't
wait which mean give me an F Buffy and
then buff if you have one two and all
your packets your data if you don't you
know just return this is something you
use on the Fast Pass let's say I'm you
transmit and receive pass you know in
your in case this be warned that you
you'll you'll may get you will get an
old back and you know probably the best
way in that case you drop your packet do
whatever is is it's good there you can
also ask for a wait mode but don't do
that on the fast pass because you're
gonna block the Strad what we're trying
to allocate memory so you're gonna block
you know potentially it's it's not very
good so do that for things that are a
low bandwidth kind of things you know
when you start your protocol and it need
some member you know to up front or
something like that but don't do that on
the fast path
so yeah the rule is do not block so
amber if you get to realize them there
is some rules it's not completely
depending you know socket buffers and
everything you'll you'll see that you
have to realize them or somebody's gonna
realize it for you there's no real
preset rules no I think that's it just
yeah just the warning is be careful with
em both be careful about the use of
macros and there is some come in like
netstat dash ends it's gonna tell you
how many Amba you're using a new system
and this
a really good way to see if you have a
leak in your NK if you see the number of
n buff you know going up in use it's
probably somewhere you forget to release
one and so you might you might want to
check that when you're doing debugging
of your encase from from the inside from
inside the kernel from gdb you can look
at MB stats it's gonna give you some
stats about the allocations number of
drops and things like that and again
it's gonna give you some information if
you forget to release a member in your
endgame another thing here that we have
in terms of kernel extensibility are the
kernel events what we have is basically
a new domain here the PF system which
from the socket protocol give you a way
by you know listening on a socket by to
see some kernel events so it's gonna
report events from kernel to user space
and those are you know pretty low
bandwidth events usually the kind of
events we have our things like your
interface you know you put the link so
link is up so link is down things like
that or the IP address change so we you
will receive an event on if you listen
this if you if you're connected socket
this is used mainly by the system
configuration you know configuration is
pretty pretty much the user of this you
can also add your own events to this
mechanism you can see if you have like
some specific driver or so specific you
know drive with families that you added
and you want to add your events and you
want to have an application listening to
this those events you can do that it's
it's not meant for high traffic it's
just a low bandwidth stuff but it's a
system PF system there is another thing
that we add in
in Darwin which is the network and grv
the PFN geography so that gives an
access to its essence I want to mention
for people from coming from nine it has
nothing to do with in the RV that you
you know the drivers native driver from
mac os9 it's it's really you know the
network driver here so while it gives
you is from the socket level give you
access to all the packets to the raw
packets and one you know you know you
can do as we said before try to avoid if
you can to do your your protocol or
things you are trying to do try to avoid
doing them in the kernel for all the
reason we stated before if you want to
do your own protocol in userland you
could use a PF and gr v to get some
packets there as an example in the oven
you can look at shared IP which is the
name of the kernel extension we're using
for doing the port sharing with classic
with classic networking is an example of
basically classic listen to PF and your
fe sockets to get its packet back and
you know in emulate gog you know driver
the LPI driver that way so the g LPI
drivers is really talking through a
socket to PF injury that's an example
for you if you're trying to do this kind
of things so right now works on internet
okay funnels funnels are mechanisms that
we we introduced in Darwin this is not
something you'll find in FreeBSD why do
we have this is basically as you know we
have an MP and we're an MP system so
we'll we have a multiprocessor and what
we want is to have a mode where we can
have you know performance in MP and the
networking stack in Darwin so from
FreeBSD is not completely MP safe let's
say and so funnels are a mechanism to
give the mutual exclusion to make sure
that
run into the networking code networking
from you know coming from i/o kit up to
the socket layer to the system calls
that nobody else is going to be running
into that code at the same time so we
got a mutex that we take from the socket
layer or from the packet level that's
gonna make sure that nobody else you
know can be running in let's say TCP
code and do something at the socket one
of problem to think about is you're
trying to send something on let's say
TCP doing some your you're sending some
data on a TCP socket at the same time
we're getting a disconnect if we didn't
have a system like phenols we're in a
multiprocessor environment you could be
doing your TCP send while the state of
your TCP transaction is being modified
by the packets the incoming packet and
we don't want to do that we are not
prepared for that so so way to do this
is to a funnel so basically in in the
Darwin kernel there is two fun answers
the network for know which is used by
the network stack and the kernel funnel
which is used by the rest of the biggest
D sub system if you remember the diagram
from before
inside the Mac os10 kernel we get BSD
subsystem which is more or less
networking plus file systems so it's
used by a file system so what it means
that in the file system or in the
networking you cannot have two
processors at the same time however by
this mechanism you can have one
processor running you know dealing with
packets in processing packets while the
other processor is doing filesystem sync
so that give us some good performance in
servers on MP environment where you can
have your Apache or you know Apple share
server do at the same time I have a one
processor and do some networking
activity while the others you know
flashing stuff on on the disk are doing
some net file access so problem with a
funnel is that
you need to be aware of them and and the
rule I stated that basically we have one
lock on top of the of the networking
stack at the socket layer and I mean
that's a system called layer and one at
the bottom is is is not completely true
so we're going to go into some of the
detail you need to do in your NK to deal
with funnels but that's the difference
from FreeBSD so that's something you
need to be aware of and the thing is
here that you need to deal with funnels
you cannot like say I don't care about
funnels your system is gonna be on
having problem if you don't deal with
that right okay so when to use them
so you need to set the network funnel
and specify that you want to work in the
network for all ins your module start
and stop why because you're called by
i/o kit basically when you're you're
your module starts and I orchid is not
running an under a funnel IO kit in the
mock part of the kernel don't need to
have an altar already NP completely safe
and they don't need that so you need
when you bring you're going to be called
you need to basically tell I'm gonna use
a network for now and their skulls to do
that same thing for the stop timeouts
are another one where you need to be you
know switching funnel or taking the
funnel taking the network final why
because you the timeouts are called
as a you know direct mapping from the
marker subsystem and you're not under
any funnel so if you run your code you
need to explicitly say hey I'm gonna run
in the networking code so I need to
grabs and network for now first and
there is also things when you're doing
things we're spreading if you create a
new thread and you need to explicitly
you know tell that this Redis to run to
the network for now yeah it's a
preemption point so each time you're
trying to grab the funnel or you you're
gonna leave the funnel you can be
preempted another thread in the kernel
can it can run so be aware of that your
state might change when you come back
from
you know from from asking for getting a
funnel or living a funnel just be aware
of that and more or less if you're aware
with FreeBSD and ESPN net SPL X and our
winds are more or less no ups I'm saying
more or less because all they do right
now is they don't they just make sure
that you're either under the network as
a colonel for now they don't they don't
have any active you know nesting or
anything like that you just do not have
the you know I mean and FreeBSD you
would protect yourself or a critical
section by raising you know SP on that
and say nobody else can can enter at
this point on tens are more less not but
if you're under the network from well
nobody's going to get in there so that's
four four funnels so that's something
you need to look at and look at or
kernel extensions in Darwin to see how
to use that appropriately NK control
there is a way p FN key to control the
Enki from a process from user mode so
you let's say you inserted the socket
filter or data link interface layer you
know filter and you want to have some
control to that you can have a special
mechanism like a conduit to control the
earth your your socket I mean your your
NK through this NK controls the PF NK
the NK manager is not loaded by default
so it's something you need to do and
we're we're kind of in in the changing
will change that so you know right now
it's it's work in progress
it's a character device and there is
some other way to that we encourage to
talk to your NK is you can go through
ioctl
we're going to intercept IOC else and
use that as you can the control
mechanism for your n ke are also socket
options and this is what we do if you
look in shared IP we use the adoption as
a you know control mechanism for ciently
so so VPN yeah we talked about a way to
implement a VPN so you could do that as
absolute device depending on what you're
doing the type of it
you're doing or you could do that as
Glif filter it really depends about how
your code is organized and which level
you think is appropriate for you to to
plug in be aware that the kami IPSec is
coming to Mac OS 10 so if your VPN
solution is using IPSec IPSec right now
can may IPSec is in Darwin you can take
a pic of that and this is gonna come and
be sometime in the future this will be
part of the Mac OS 10 kernel so you can
build your own darwin kernel with IPSec
right now and you know use that as a
base for your VPN solution if you're
using IPSec and talk to us if you're
interested we're really interested in
knowing how we can help you with that
summary I just want to go again because
the massagers
about the rule foreign keys you have to
be really careful about your dependency
you have some iokit mechanism to say hey
I'm linking against that kernel I don't
want to be loaded if you know it's
version of 15 of Mac OS 10 you get to
keep track of your resource and use age
you know nobody's gonna clean up after
you you have to do it do not block input
on the first pass you're gonna block the
world you know the world networking
stack here you're part of the networking
so you just have to behave and use those
rules you have to know your split
funnels be really careful about that
remember timeouts and anything that is
coming out of of of the of the kernel
funnel or the rest of the PS of the rest
of the kernel is probably not on there's
a funnel so just check with that and be
aware of binary compatibility in the
future as we said IPSec ipv6 are coming
the power of Darwin we get you know you
can you can look at that in the Darwin
kernel we're going to be based on that
PvP extensibility there will be a way to
get some plugins for PvP if you say you
want to have PvP over 80 nm or PPP over
some Azure new cool media you just
invented the N key control yeah socket
API is in flux it's gonna change a
little bit
and also in in the future we're planning
to get something to you know instead of
funnel some more finer grain locking of
sockets and things so we don't have to
use a funnel mechanism so that's pretty
much it
so I hope you get information there you
can have additional resources here
my quest and of course you've seen those
I'm going to go fast results darwin org
and freebsd are also good points for
information resources that just
disappear
the stevens books that we talked about
the implementation and design
implementation of bsd four four is also
pretty interesting however be aware of
those differences we talked about like
phenol yell IL and things like that that
make that gives you a pretty good idea
about what's going on but it's not
exactly what we have in darwin and you
know the network kernel extension PDF
file that has a very very complete
coverage about all the type of GL al
filter you can do in socket filters as
well this is where you're gonna dig into
all the details about how to do your UNK
and also kernel extension tutorial for
my orchids which is going to tell you
about how to build in case with project
builder and what are the rules for for
dependency and things like that and you
know tell us what you need from us you
know we're trying to make this
extensible we got some mechanisms that
we think cover some ground so tell us if
you need more what you think we should
change here and we're really interested
in getting your feedback on system and
the roadmap this morning was networking
overview session it's done already and
this afternoon pretty interesting just
history and network configuration
mobility we'll see how those guys are
using the events you know to get you
know some state information from the
stack and Thursday morning will all be
there for the network feedback forum
which is in room j1 just next
always fun so with that you know which
contact contact my boss
you