WWDC2003 Session 112

Transcript

Kind: captions
Language: en
hi I want to thank you all for coming
out here for this last session they were
only an hour and a half from the
margaritas i'm here today to talk to you
about CF network and this is roughly
what we're going to cover today we're
going to start out with an overview that
shows how CF Network fits into the
system as a whole and then we're going
to go through the basics of how you use
a CF network object all of the objects
are used in essentially the same manner
and we're just going to walk through
that basic paradigm after that we're
going to start looking at the actual
objects provided by CF network those can
be broken down into two categories first
we have some basic instructions in that
CF host and CF socket stream and then we
have some protocol implementations FTP
HTTP and rendezvous so with that let's
get started with the overview so those
are the questions we're going to try to
answer today what is CF network how does
it fit into the stack and when should
you use it I know with the introduction
of safar is that we've been talking
about a lot of different internet
knowledge ease here and staff networks
one of those so how do you use between
them how do you decide to ucf network
instead of one of the other libraries so
first and foremost CF network is a
library that abstract several basic
network protocols and I've listed those
protocols here dns host resolution
that's new and panther ftp also new in
Panther HTTP rendezvous and sockets
basic sockets the api's are designed in
the same style as core foundation in
fact all of our objects are exported as
CF types so you're going to retain and
release them the same way you would
retain and release any CF object and
further those api's all support
asynchrony this is the other main
purpose of CF network it's to take your
networking inputs and provide the
network events to you using the run loop
using the same the same asynchrony model
through which your your user events are
coming in
so what are the goals for CF network
first and foremost this is a power API
we are trying to expose the full power
of the underlying protocols to you we're
trying to also get strong integration
between the CF Network library and the
other libraries on the system
particularly like I said with the run
loop so that you can get your network
events in the same manner as all the
other events happening in your program
part of being a low part of being a
power API is providing high performance
so one of the other goals we have is to
make sure that our objects perform with
very nearly the same performance you
would get if you chose to write directly
to the socket layer but one of the
consequences of choosing to be this
power API is that we're not going to
protect you from any of the details this
is not a convenience API the way some of
the some of the other api's you've heard
about this week are so you need to know
how to use the run loop you need to know
the basics of the protocol that you're
taking advantage of so where does CF
Network fit into the overall
architecture of Mac OS 10 you've seen
this diagram probably a few billion
times by now Darwin the core OS at the
bottom applications all the way at the
top a couple levels of libraries in the
middle and if we look at the networking
concepts and the networking ap is this
is sort of how it breaks down the st
sockets is what comes from the low-level
core OS CF network is the first level of
abstraction above raw sockets and then
all of the other stronger higher-level
api's are built on top of CF network so
that includes nsurl and the WebKit for
instance and then the application sit
above that and I've listed here just
five of the clients that I know of the
take advantage of this networking stack
I kalin I think Safari of course mail
and I chat so when should you choose to
use CF network instead of one of those
higher libraries instead of going to the
nsurl api's or the WebKit 80
well UCF network if you particularly
need an API that that that is that low
level and high performing so you would
go there when you need extra control
like needing control of exactly when the
bites are going to come off the socket
or exactly when the bites are written to
the socket UCF network also if you need
detailed control of the protocol stack
the higher level API is often expose
protocol details so that you can tweak
and change things but that can become
cumbersome if you're ending up
constructing for instance an HTTP
request in its entirety in some
sometimes it's much easier to drop to CF
Network and just constructed all
yourself and also UCF network if you
want to avoid higher linkage you don't
want to bring in the app kit you don't
want to bring in in fact anything above
that core services layer well then CF
network can help you out but by all
means use the higher api's if that fits
your application the higher api's do
more things for you they're more
convenient they're easy to use they have
provide a lot of advanced features that
we do not at this layer so that's sort
of when you would go down to the level
of using CF network but suppose you're a
UNIX programmer and you've been working
with raw sockets how do you decide to
climb to the CF network layer well the
first and largest reason is because you
want that run loop integration you don't
want to deal with handling the select
loop yourself you'd rather be able to
integrate with your main event loop let
CF network do the work of finding out
when the socket has data available and
inform you in a way that fits seamlessly
into a regular application conf of
context another reason we see why people
come up to using CF network is because
all those sockets were perfectly fine
for them they discovered suddenly they
want to secure the socket and add SSL
encryption over the top well unless you
want to implement an SSL stack or get to
know an SSL library CF network might
help you out there it's a single it's a
a simple single option to turn on ssl on
any given sockets finally if you want an
object abstraction we can provide that
for you that's what we get from core
foundation that's just a simple
reference counting CF type abstraction
finally UCF what network if you know the
protocol you know HTTP you know how to
use it but you just don't want to write
the parse engine you don't want to write
the code that does all the details of
talking the protocol itself so when
should you not use CF network well use
nsurlconnection that's the new URL
loading api the kit that was introduced
with the safari SDK if you need one of
these other functions there one of these
other features first of all if you want
a more convenient fire-and-forget just
get me the data API nsurlconnection is
better suited to your needs you'll also
have to use nsurlconnection if you're
looking to handle any arbitrary URL the
nsurlconnection api is protocol agnostic
you don't have to know what kind of URL
you're working with CF network is not
you must know in advance that you're
doing HTTP or ftp if you need to add
your own protocol implementation if
you've got to have a way to put that to
implement gopher and plug it into the
existing stack that also happens at the
nsurlconnection layer and finally i've
just listened a couple large features
major features of the URL connection
layer caching and authentication we have
the raw hooks inside CF network to add
authentication we're going to look at
that today but it but nsurlconnection
goes way beyond that to being able to
manage your credential store and cleanly
inform you when authentication failures
occur and let you intervene in the whole
authentication process it also adds
sophisticated caching so that as urls
are accessed you can store those in disk
or in memory there's gives me on disk or
in memory automatically
so now I'm going to go on to talk a
little bit about the basic usage of CF
Network objects they all follow the same
basic model and not too surprisingly
they all start with creating the CF
network object you intend to use once
you've created the object you set
yourself up as a client all of these
api's are asynchronous that means
they're going to go out and monitor
something and when something interesting
happens they're going to turn around and
inform you well they inform you by
sending your client a call back and when
you set yourself up as the client you're
saying hey talk to me once that's done
the CF Network object needs to know what
thread it should it should talk to you
on we do that but you give the CF
network object that information by
scheduling the object on a particular
run loop remember with CF run loops one
thread one run loop it's a one-to-one
correspondence so when you schedule on a
run loop effectively what you're saying
is this is the thread I want my callback
on once that's done you start the CF
Network object you just tell it go and
do the thing that I created you to do
and then you just sit back and wait you
need to keep the run loop running if the
run loops not running then the CF
network object has no way to get time
from the CTU and it also has no way to
call you back most of the times the time
you don't have to do that yourself
because both app kid and hltv keep the
run loop running for you but if you're
on a secondary thread or if you're not
linking one of the GUI frameworks you're
going to need to call CF run Luke run or
one of its variants to get the run loop
going and eventually your callback will
be called at that point you find out
what happened and you field and you
filled the result finally when you're
done with the CF network object you need
to clean it up and dispose of it
all of the different CF Network objects
available follow this model in fact
several of the other objects inside core
foundation and system configuration also
follow this basic model so i think it'll
pay it will pay off to a become familiar
with it and understand how it works and
now i'm going to take you through an
example of that model we're going to
talk about CF host here since it's one
of the new objects and Panther CF host
allows you to do an asynchronous DNS
lookup and here what we're going to do
is look up the IP address for dub dub
dub apple com we start out by creating
the object CF host create with name and
we tell it what host name word we intend
to look up then we set up the client now
all of the CF Network clients are set up
in these contexts structures and it can
be a little confusing and daunting the
main thing to know is that your info
pointer is always the second argument
and if that info if you want us to treat
that info pointer is just a blind void
star all the all the other arguments can
be null if on the other hand that info
info argument is a CF type then the
other argument should be CF retain CF
relief CF copy description and you can
see examples of that in the
documentation and the example source so
you set up the context and then you call
CF host set client you're going to pass
the CF network object you're registering
with so here my host you're going to
pass the callback you want called and
then you're going to pass this context
rest which is how you get information
passed back to you about what you're
requesting once that's done you're ready
to schedule on the run loop here I'm
calling CF run loop get current to get
the current run loop so this essentially
says I want my call back on this very
thread the one that I'm executing on now
now that all of that context is set up
we're ready to start the CF Network up
object off and so here we call CF host
set info resolution
and tell us that what the info we want
resolved is the hosts IT addresses and
that we shouldn't wait and we do that by
running the run loop so then at some
point in the future hopefully you're
going to get a callback and call that's
going to look something like this the
first argument is always going to be the
CF Network object that's reporting the
result then they're going to be several
arguments that tell you what result you
have received so here's we have a type
info field to tell you what kind of
information is coming back an error
field in case an error has occurred and
then the last argument is always your
info pointer out from that context
structure and basically what you're
going to do in each of these callbacks
is first check to see if an error is
being reported if it is you have to
field the error if no error has been
reported you look and see what
information is being reported and you
and you do whatever you need to do there
so here if the type info is host
addresses we're being told that the
addresses are available we turn around
and call CF host get a dressing and this
returns an array of all of the IP
addresses for that we found and you can
do whatever processing you need on that
finally cleaning up with a callback
based API like this it's not as simple
as simply calling CF release when you're
done with the CF network object the
reason for that is that with a reference
counting API you can never know for
absolute sure that your reference was
the last one and so when you call
release that objects going to be
destroyed well since you can't know that
the object is going to be destroyed you
probably want a way to make sure that
your callback will never be called again
because the object may hang around but
you're not planning to so the way you do
that is by first calling set client null
that tells the object forget you ever
heard of me don't come back don't call
don't write once that's done you
unschedule from the run loop this is not
strictly necessary but it's a
performance boost because as long as
your schedules on the run loop that
object is going to keep using the run
loop to do work
after you've unscheduled if the object
is still doing work if you haven't told
the object okay stop you're done at this
point this is the time when you want to
want to call cancel or clothes or
whatever the call is that tells the
object you know stop doing anything I'm
no longer interested in the work finally
only after all of that is done should
you call CF release and forget your
reference to the object so that's the
basic the basic usage model for all of
the different CF Network types however
it's not the only usage model we support
we consider that model asynchronous
event-driven the basic idea is you tell
the object to go and at some point the
objects going to call you back and say
these interesting things have happened
however we do support other common
models polling and blocking we don't
recommend that you use them they are not
performed they will not perform as well
as the event driven model will but you
know if you're adapting old code that
uses one of these models or you're
working inside an architecture that
requires it you can use them and now I'm
going to move on to start talking about
the actual objects inside available from
CF networks and like I said we're going
to start out with the basic abstraction
there are two I want to talk about CF
host and CF socket stream CF host like
like I said is new and Panther and
allows you to asynchronously do a host
look up dns host lookup or a reverse DNS
host look up you can create a CF host
either from a hostname or from an IP
address and you see the two different CF
host create calls there and when you're
ready to start the asynchronous
resolution you'll call CFO start info
resolution you can on the arguments as
we saw on the code example determine
what times what it is that's going to be
resolved you have three choices you can
resolve IP addresses that's probably
going to be the most common
it's just a basic asynchronous DNS
lookup you can look up dns name so if
you started from an IP address you can
do a reverse lookup and finally you can
check reach ability to that host and a
word of warning there that reach ability
is to find the same way system
configurations reach ability is defined
in other words we're not actually going
out and pinging the remote host to see
if we can get a packet all the way there
and back we're just checking to make
sure there's a path configured off the
machine that is likely to get you to
where you want to go so with CF host
once your callback is triggered that
means that the information you need is
now available and you can now turn
around and retrieve that information out
of CF host by calling one of these three
functions CF host get addressing returns
the IP addresses CF host get names
returns the DNS names and CF hosts get
reach ability tells you whether whether
or not the host is reachable at this
time moving on to CF socket stream CF
socket stream is simply a stream wrapper
around a socket so sockets kit you can
read and write to them both so we return
a pair there's a CF read stream and a CF
right stream to go with the socket you
can create such a pair of streams to any
of these different things the CF hosts a
CF net service that you might have
gotten reported back via rendezvous a
hostname or poor end port pair an
arbitrary socket signature if you're
comfortable working with sock adders or
you can actually take a pre-existing raw
socket and wrap the streams around them
and the one catch to remember is that
there there is only one socket
underneath and any configuration you do
on one stream is going to affect the
other precisely because there is just
that one file descriptor underlying the
streams so the basic support for CF
socket stream actually resides inside
core foundation and you can find the
most basic creation routines in CF
stream th inside the core foundation
headers but CF Network adds several
interesting configuration options on top
of them
I already touched on one of them SSL or
TLS or TLS encryption and authentication
that's configurable from inside CF
network we also added socks proxy
support and you can find the details
about how you would use those inside CF
networks CF socket stream gauge on the
topic of proxies this is a common
question we have ok I've got this great
stream but I don't know how to set it up
so that it'll navigate the proxies that
configured on the computer well several
of the CF Network api's are set up to to
accept proxy configuration dictionaries
here I listed three of them CF socket 3m
HTTP stream and ftp stream because of
the layering of this API because this is
that Lola low-level API where we're
giving you full control over all the
configuration we will never apply a
proxy automatically however we tried to
make it as easy as possible for you to
apply the default proxies and the way to
do that is to call FC dynamic store copy
proxies that's the function that will
return the dictionary of priests of
proxies configured on the on the system
and then just pass that straight through
to us if you do that we will
automatically go through figure out what
the right proxy to use is and apply it
the reason why we do it this way so that
if you're in a situation where you know
you don't want to use the default proxy
there's no reason to go through the HTTP
firewall for instance if you know you're
hitting a local server you have a way to
do that so with that I'm going to move
on to talking about the network
protocols we have three to discuss we're
going to start with ftp because that's
new and Panther we're going to then talk
a little bit about HTTP and then finally
we're going to talk about the rendezvous
support inside CF networks and I've
listed here the types the CF types
associated with each of those different
different protocols CF ftp stream backs
ftp CF HTTP method in HTTP stream back
HTTP and rendezvous is a you access
rendezvous via CF net service and CF net
service browser so f PP stream is new
and Panther and you can use it to
download an ftp URL or to download a
directory listing for an ftp URL we
don't have upload and support in on the
CD that you got but we're promising it
to you by Panther and I'm going to just
walk you through a code example of how
you would download an ftp URL it's
pretty straightforward you create the
URL to the ftp location you're
interested in you call CF read stream
create with ftp URL you're getting a
read stream back and now you just use
the stream you way you would use any
strip you open it you read from it i do
not show the event driven model here but
normally you would schedule yourself and
wait for the stream to inform you that
there's bytes there are beds waiting for
your attention downloading a directory
listing is very similar you start out by
creating the URL but here the URL is a
directory URL instead of a file URL you
can tell that because it ends in the
slash and that's actually what week you
off of to figure out whether we're
looking at a directory or looking at a
raw file construct the read stream the
same way and read from the stree in the
same way however what you're getting
back the bites your reading or what the
same bytes that would have been
displayed if you had typed you know LS
from inside and ftp inside ftp in the
terminal so you need to do this extra
step of parsing out the resource listing
so i represented that by calling this
parse resource listing function and this
is what that that function is going to
look like so you got back this buffer
from the stream you have a length and a
number of bytes and what you're going to
do is call this other function CF ftp
create parsed resource listing giving
back the buffer and each time you do
that will scan the buffer and try to
find a directory listing if we find it
will tell you okay i just consumed the
first
bites and then we'll take those 80 bites
and parse out a full directory listing
and return that to you as a CF
dictionary so the dictionary has a bunch
of keys i'm not going to go into all of
them here they're all documented in the
headers but i show here she s dictionary
get value listing and then the resource
k CF ftp resource name and that will
return the actual name from the ftp
listing and then of course because we
told you we just consumed the first 80
bites you need to update your internal
buffers to say okay next time i call
this function i'm going to start 80
bytes further downstream and that
function the parse resource parse
resource listing function is going to
return 0 if there aren't enough bytes
available for us to get a full listing
or negative one if we can't understand
the bytes we got moving on to http CF
HTTP message is a CF type just a pure
data type doesn't have any asynchrony
built into it or anything like that it
simply represents an HTTP request or an
HTTP response once you have an HTTP
message you use CF HTTP stream to
actually perform the HTTP transaction to
take the request serialize it ship it
over a socket and get the response back
so your normal usage is going to be to
create an HTTP message for the quest
create an HTTP stream from the request
and then open and read from the stream
and that looks like this start out by
creating the HTTP URL construct a
request from that URL do any extra
configuration so here I'm setting the
user agent to my appt 10 to show you
what that would look like once the
message has been configured the way you
want it to be called CF read stream
create for HTTP request you're going to
get a rain screen back and again you're
going to use the read stream the same
way you would use any read stream set
yourself as a client open it wait to be
told bytes are available
now that's the basic usage for the HTTP
stack but there are a couple others that
are interesting to note sometimes you
don't want us to build the connection
the socket connection for you or
sometimes the bites aren't even going to
be coming from a socket they're coming
from a file sitting on disk or from some
piece of memory you got from somewhere
else you can still use CF HTTP message
to perform the HTTP parsing for you and
you do that by calling CF HTTP message
create empty and then calling a pen
bytes repeated repeatedly as the bites
are appended HTTP message will parse out
the interesting structure and then you
can ask it for I interesting values like
you can query the header field you can
find out what the status code is all of
that kind of thing now sometimes when
you're performing an HTTP transaction
there's a lot of payload data you're
performing a post of some large amount
of data or you're pushing a file back to
the server and you don't want to load
all of that into memory which is the
normal way of handling requests you
could accommodate that by calling CF
read stream create for streamed HTTP
request instead of the normal creation
function one of the arguments to that
function is a read stream and we will
automatically pull from that read stream
to send the body onwards and that's just
again to help you avoid having to hold
these large resources in memory finally
CF HTTP message can be used to add
authentication information there's a
function at authentication you pass it
the request you're about to send the
username and password that you want to
apply there a couple other arguments as
well and basically we will look at the
information you gave us and put together
the correct header fields and apply them
to the request so you don't have to
understand the details of digest
authentication for instance moving on to
rendezvous rendezvous was introduced in
Jaguar and the idea was to allow clients
to go out there on the on the network to
advertise services that they had
provided
then other other clients can go out and
look for those services and discover
them without having to without the user
needing to go through any configuration
work CF Network takes these services and
integrates it with the run loop so again
you can get receive information about
services available or advertised your
own service in a way that's well
integrated with the normal event
structure on the system there are two
types the CF net service and CF net
service browser CF net service
represents a network service it's a
single provider somewhere out there on
the network it is a printer it is a web
server a net service has a number of
attributes and I've listed them here the
domain is where is where that service
could be found so most often it's
thought local for the local network you
can also always pass the empty string
anywhere you're asked for a domain to
mean the local network the type is what
kind of services being provided you know
a printer or a web server the name is a
name you have chosen to to title your
service so becky's web server as opposed
to Apaches as opposed to apples and then
the address and the port is the IP
address and the port that should be used
to contact your service so you would use
a news excuse me you would use a net
service for two basic reasons either you
have a service yourself that you wish to
advertise so you want to announce your
presence on the network and so that
other clients can find you or because
you found a net service that someone
else has provided and you now want to go
and connect to it network service Prowse
ER is how you would go out to find those
services and each service browser
represents a search you can search
either for services for individual
services or for service domains to find
out where in a sense where are the
different places you could look for
services so to advertise your service
what you're going to do is create a
creative net service using CF net
service create
you're then going to register that
service by calling CF net service
register now this is a little this one
case is a little different from the
other CF Network cases because they're
the only event that could possibly be
reported to you is that an error or
naming conflict has occurred so usually
you're hoping that your callback isn't
going to be called if it is called it
means that there's a name conflict and
you should prompt the user for a new
name or you should choose a new name
yourself somehow handle that conflict
the other way you use in that service is
to resolve someone else's service
usually that means you got the net
service from a net service browser the
first thing to notice that if all you
want to do is connect to it you're done
just take the net service and pass it
into CF stream create pair and you'll
get back a read and write stream that's
directly connected to the service you're
interested in however you might also
want to know more information about the
service you might want to know what the
name is to display it to the user you
might want to pick and choose between
the IP addresses that that service is
advertised on to do that call CF net
service resolve that will start an
asynchronous resolution of all of the
information about the net service when
you get your call back you go back to
the net service and say ok tell me about
yourself and you can get all the
information then ok using a net service
browser create the browser using that
service browser create and then you
decide what you're going to search for
either domains where services are being
advertised or services within a given
domain and your callback will be called
as domains are found out on the network
your callback is also going to be called
as those services disappear so if you
leave a browser running for a long time
it'll call you back and say ok Becky's
printer is gone now oh these other
printers have become available so you
can track over a long time a long period
of time that way wow we're just sitting
through this we're going to move on to
the wrap up
all these sessions have already happened
however if you're looking looking back
on the conference later and going
through your DVDs here are some sessions
that might be of interest to you 403
Safari overview talked about all the
different technology pieces that go into
Suvari 105 rendezvous spent a lot a long
time talking about rendezvous itself 110
networking overview talked about all the
different networking ap is available on
Mac OS 10 there's a lot of time spent
there also talking about the basic
capabilities of the of the core OS
networking pieces advanced foundation
URL API is talked about those new URL
loading ap is that are available now
with safari with the Safari SDK savio
xavi a leg row so I'm going to invite up
to the stage now is is our evangelists
and you can contact him with any
questions or comments about the CF
Network ap is just to flash through the
documentation quickly there's a ton of
documentation about all kind of
different kinds of all different aspects
of the networking ap is on our system
just go to a DC home next documentation
networking and browse you'll be amazed
at how much you find but in particular I
wanted to highlight the CF Network
section and there's a very good
programming topic on using CF run loop
and finally rendezvous also has its own
section sample code and developer
examples networking I'm not going to go
through this in detail but there is a
good example of using HTTP there there's
also a good example of writing a server
and the client and getting them to
connect using rendezvous Mac network
Prague is a fabulous mailing list I'm on
that list most everybody I work with
that Apple is on that list it's for
discussing all aspects of network
programming and just go to list apple
com
and you can you can sign up there