WWDC2001 Session 302

Transcript

Kind: captions Language: en well I want to thank everyone for taking time out of the busy afternoon to attend the directory services talk my name is David O'Rourke and we'll be going into more detail about directory services which if some of you were in the Mac os10 server presentation you might have heard a little bit more about I'll be talking and later on we'll be inviting up a co-worker of mine Jason Townsend who will give you a little more detail on developing a directory plugin and a demo what we're going to cover today is what are directory services how are we using directory services in Apple software what are the api's look like what's plug-in development look like we'll give you a little demo and then hopefully cover some questions and answers at the end of the session the concept of directory services arose about three years ago when my management asked me to figure out what Apple's directory story was moving forward for Mac OS 10 server and one of the key pieces of feedback we had from our customers was they no longer wanted to use just an apple proprietary database like users and groups they wanted at least the option to integrate with an existing or perhaps a proprietary directory system so the concept is relatively simple allow Apple software to use customer data the implementation all of us are engineers in the room or have seen this engineers drop boxes is fairly straightforward we're going to build an abstraction API we're gonna put apple services on top of that abstraction and that abstraction will grant access to customer data the architecture a little more fleshed out we have Apple software based on directory services we have developers software that could be based on directory services and we provide plug-ins to to directory services that apple provides the primary directory service in Mac OS 10 is net info and that's our default plugin we also provide read-only access to LDAP v2 and part of the reason some of you are in the audience hopefully so that you can develop a directory access plugin for a directory service that you can make money on or possibly help your site the runtime of the API leverages the Mac os10 architecture a directory service client using directory service API calls into a Mac os10 framework that framework uses mock IPC to communicate with the daemon then talks to plugins to handle various requests the reply comes back up through mock IPC it's very secure and leverages the Michaelis 10 architecture status since last year I've given this talk now I think this is my third year up on stage talking about this so it'll be familiar if you were here the previous couple years hopefully this is new for some of you what have we done since last year the directory access api's are included in Mac OS 10 desktop this is not a server only technology you can use directory services in the desktop they are located in system library frameworks directory service framework the API manifests itself in five C header files you know going over header files is laborious and sometimes boring so they're there I'd encourage you to review them the directory plugin api SAR included in functional in the desktop and the server so this is something you can use in either location although the server leverages them more heavily than the desktop does there's also an SDK and the term SDKs a little over overused what we have is a collection of some sample code and some helper code that's available from Tom wire everything you need to actually develop and use directory services is already in Mac OS 10 with developer tools we are writing some sample code so the SDK includes the sample code and the documentation in one convenient place what's new since last year's we have the directory setup application which we only alluded to last year you'll be getting a demo of that later we have a search policy we received some feedback directory services embodies the concept of a search policy for authentication where what directories do we look into when the OS wants to authenticate a user we've also added a contact search policy which we think could be integrated into pins or email clients and mass search policies configured separately and could search public databases and it provides a lightweight way for you to provide a fine user functionality and your applications we will also be releasing by the SDK ad S wrappers which is some C++ sample code that was developed inside of the group that makes it a little easier to approach the directory service api's so all of that will be bundled up and made available contact home wire and I think we'll post it to the website when we talk ok what is directory setup well directory service has come out of the box and the first thing you need to know is if you're using net info you don't need to do it just works out of the box no one needs to visit directory set up no one needs to modify anything directory set up is for those clients that wish to use a directory other than net info and directory set up allows you to control what directory systems Mac os10 interacts with it's located in applications utilities you can enable and disable individual directory plugins right now there's only two the net info in the LDAP if third parties add more this is where using as a system administrator return those third party plugins on or off and it also provides the ability for you to put up your own human interface for configuring your plugin you'll see more about that later and I believe after this I have a sample screenshot yes if you look at the screen you this is a screenshot for Mac OS 10 the lock is in the lower left hand corner if this is a secure thing only the system administrator can change these settings and we provide the two standard plugins net info and LDAP if you wrote a third party plug-in or developed a plug-in it would be listed here the user could select your plug-in and click on the configure button there in the bottom center and that would launch a human interface or Coco or carbon application that you wrote which puts up your own user interface for configuring your plug-in the authentic the three tabs across the top of services authentication and contacts the services are for controlling the plugins authentication controls the authentication search policy and contacts allows the user to configure the contacts search policy what are the API is include they include full read/write capabilities you can do if your directory plug-in supports read/write you can do anything with a particular directory through these API smackle is ten servers administration software is based on these API for reading and writing that info the API offers the notion of simple complex and custom authentication methods proof of identity so if you have a custom authentication method that your particular directory system supports you can publish it and come up with a scheme that clients could use it the directory API is our standard C API and they're accessible from any runtime other than classic on Mac OS 10 so a mock Oh binary a cocoa app a carbon app we have a variety of applications calling directory services we have the Mac OS 10 server administration software which is which is a carbon application the directory set up calls directory API so that's a mock binary cocoa application and directory services calls itself which is running more at the POSIX layer so we're fairly confident you can call the directory service API from pretty much anywhere except classic on Mac OS 10 part of the directory services provides an abstraction for accessing it also provides an abstraction for the data and this is where we get some of the most questions so this year we've added this slide hopefully we'll do more next year but the abstraction that directory service presents as a notion of directory nodes at the top level and those are groups or collections of data if if you had multiple LDAP servers configured each one would manifest itself as a single directory node in the directory service abstraction API records come underneath nodes records are contained within nodes that consists of names and types attribute types with the data and attribute values come under attribute type so we're gonna go into that more detail in the next slide so directory nodes each directory node is published by a plug-in and a single plugin a plugin can register 0 or more nodes you as a plug-in developer can register as many directory nodes as you want that you want to grant clients access to hopefully you're matching that to some reality of some kind but when when someone opens the directory node using the directory service API it will dispatch to your plug-in and you will then be responsible for all calls regarding that directory node API calls to access directory nodes from a client level RDS opened ur node IDs get their node lists find their nodes and closed or notes they're relatively straightforward so use an API client if you wanted to work with a particular LDAP directory node you would call open their node and now you're talking to an LDAP server the important point is is it doesn't matter whether you're talking to an LDAP server an ActiveX server or a net info server open der node group gets you the session with that directory server regardless of what the protocol is an example path of a directory node would be slash LDAP v2 slash LDAP company.com that's how we configure our LDAP sessions so they are multi path and they are hierarchical so a directory plug-in can represent a hierarchical directory system into directory services records records reside in directory notes they consist of at least one name and exactly one type the record cannot simultaneously be a user and a group maybe it can in some directories but not in the directory service abstraction examples of a record name would be John Smith J Smith the important thing is the at least one name directory services supports the notion of multiple names per record and a sample record type is KD s standard record type users we have standard types to find for obvious record types users groups printers Athey servers web servers whatever you need to access can be listed inside of a directory new API calls to access records RDS open record just get record lists d/s create record and DEA's closed record there are more than that but that's some samples attribute types attribute types are contained within records and these are essentially the descriptors of what type of data you're fetching these are a UID home directory email address phone number any type of data is represented as an attribute type they're contained within records each record can contain 0 or each attribute type can contain 0 or more values an example attribute types are K DSN add or record name K DSN adder home directory and KD s1 add or password the naming convention in the API is consistent the KD s in attribute record name is that means that typically that's a multi value there are in instances of a record name the KD s one attribute password is that means in general you can expect that that attribute type would only have one value this is not enforced it's just it's it's it should help you as a plug-in developer and help you as a client know how many different pieces of data to expect for a particular attribute type API calls use to access attribute types or get attribute entry get at get record attribute info there are more than that attribute values these are the actual blobs of data once once you get down through the hierarchy you've opened your directory node you've opened your record you found the attribute type you're interested the phone number where do I get the list of values to each instance of a phone number those are attribute values attribute values are contained within the attribute type attribute values contain the raw data specified by the attribute type again the attribute type can lead you into expecting a certain type of data and format the API calls use to access attribute values or des kit attribute value DES get record attribute value by index and DES get record attribute value by ID these are all useful Nixon had to be used in all of our software come see me more if you want to know the differences between those types of attributes in particular directory services is data agnostic we are we treat all attribute values as binary blobs if your directory is capable of storing binary data we'll just pass it right on through to your plugin if your directory is incapable of storing binary data it would be up to the plugin to make some sort of modification to the data before it's stored it if it received binary data directory services is data agnostic although we provide the tools for you to discover and utilize the data apple adoption we're not just standing up here saying third party should adopt this API because it's a good idea we're using this in our server software and if any of you have helped server software I think you'll agree with me it's a very demanding development environment we have put our server software and qualified our server software on top of this architecture including admin GUI all the way down to high performance file servers so we believe the architecture works and we believe it will work for you administration uses it you built a full GUI on top of it and the apple server software products are based on third party enhancements or replacements of directory plugins can leverage apples directory by these API is that next I mean I'd like to spend a little bit of time on there's two ways for you to leverage apples investment in this API the first way is that if you have a server piece of server software that needs users and groups or attribute data you no longer have to write your own administration software for the user to set up and administer that because you can just sit on directory services and use Mac os10 server administration to create all of your data and you'll get access to it via directory services so you don't have to do your own administration software for simpler applications the other way you can leverage apples investment is say you want to use all of apples great server software but you have a directory you want it to use you can write one plug-in and pull the entire suite of direct of server software on to your directory so there's really a lot of leverage to be had in these api's both at the top level and at the low level API model usage the general model is that this allows why directory setup and this architecture allows apple software to be configured to use directory of customers choice we only ask for standard record types in their attributes so standard record types and attributes are documented in the directory services and server documentation this enables Apple and third-party software to access any directory system by now you should understand that customers configure directory access via the search policy and directory setup data accessed by OS and applications what type of data is typically requested by the server software and by the desktop software well obviously users and groups is a big winner but the Mac OS 10 desktop also requests mount information for specific file systems NFS an AFP home directory information and in some cases host information comes out of net info or could come out of LDAP by this architecture and that would be host name and IP address the previous Mac OS 10 server session if you weren't there I recommend you get the tape Greg Vaughn covered the issue that a lot of the configuration information in Mac OS 10 that normally stored in /se on UNIX has been migrated to net info well that's an accurate and an inaccurate statement olsommer really was been migrated to his directory services so now information or configure information stored in Mac OS 10 are required by Mac O's 10 can be accessed by a directory services Apple plugin usage and development apples develop three basic access plug-ins we've developed net info that's our our primary and default directory system LDAP be to read-only for Internet connectivity we get a lot of mileage out of the LDAP v2 we can talk to Active Directory we can talk to Novell talk to open LDAP and conduct I planet so if you can find an LDAP server to access your data you can get directory services to access the data as well we also implemented the search policy as the directory services plug-in that's another thing that gives us a lot of confidence in this architecture we use the owner the architecture to solve our own problems Apple software products being tested with these plugins so if you're developing a plug-in and you can make your plug-in behave like any one of these plugins there's a very good chance that it will work on modified apple desktop and server software uses the architecture to provide customer directory flexibility and we're really looking for developer plug-ins if you want to develop a plug-in for directory services contact tom contact me please get to us you're gonna need some help but we're gonna provide it for you and hopefully it'll be pretty simple at this time I'd like to invite Jason Townsend up on stage he's gonna go into some of the details of what it takes to develop a directory plugin and I'll be back later for a wrap-up in QA Thanks so I'm gonna talk about directory plugins and the main points we're gonna talk about are the structure how do you test your plugin once you've got it up and running and then a little more technical what what are the entry points and callbacks that you're interested in as a directory services plugin so plugins are dynamically loaded code modules they're actually CF plugins so we use core foundations CF load function to load them and you have to provide a set of entry points we'll go into detail on what each of those is later and then you can call back into directory services to do various things like primarily registering the nodes that you're going to provide service on so where does your plug-in live in the system all of the plugins that apple provides are stored inside the directory service framework with that big long nasty path you see at the top there but that is generally considered the place for for only apple plugins for your third party plugins you should put them in library directory services plugins and that's guaranteed to be a writable part of the file system for root and that's where we will load plugins from the other thing to keep in mind is you need to put the dot d s plug extension on your bundle so that directory service daemon will load it here's an example plug-in bundle you don't really need to worry too much about this if you use project builder it's just a matter of creating a CF plug-in and it will deal with all the packaging issues for you the main thing you need to be concerned about is the the plist file which has some of the metadata we use to to load your plugin and also to determine what app we should use to configure your plug-in when you're when you're showing up in directory setup and the user needs to configure how your plugin works so we have some sample code we've talked about before that is sort of a starting point for you with your plugin and it shows you stubs that you would use for all the API calls so basically all you need to do is fill in the functionality that you're providing and we handle setting up all of the the callbacks and show you how to use those as well so configuring your plug-in there's two keys in your plist file that are important for you to set up for your your plugin to be configure configurable in directory setup the first one is a CF bundle configure veil and the second is CF bundle config file so each of them is optional basically what will happen if for example you just want to provide a text file you can provide only the CF bundle config file key and we will just launch that text file with the default text editor probably text edit alternately you could have an application that you launch and just provide CF bundle config avail if you provide both then we will launch the application you specify with the file you specify and all this happens when your plugin is selected in directory setup and you click the configure button then we do this launching as appropriate so once you've got your plugin up and running how do you test it there is code on the system a lot of code on on Mac OS 10 server and even personal file showing on on 10 desktop uses directory services extensively but it's probably better to start out a little bit oh sorry wrong slide so first you got a copy you're plugging in to the the appropriate location in library directory services plugins the other point to keep in mind is that you need to kill and restart the directory service process this is a daemon process there's no GUI per se for it so the easiest way to do this is just use the terminal and kill and restart it and you need to be route in order to do that because directory service needs to run as route to access it's it's log files and such then you can take a look at the log file and library logs to see that your plugin loaded successfully and we keep other logging information in there so what I was getting into before I was getting a little bit ahead of myself when you're testing your plugin it's better instead of throwing a full client at it to begin with it's better to actually control the client side yourself as well and use the API so that you've provided so you can test a subset of the directory service API that you've created to that point for example you can leave out all of the the right portions and just test that you can find a user with get record list and you can you can do a deist dune or node or node off to authenticate that user rather than having to support all the functions that that a client would use because sometimes even for you know a fairly simple operation there might be api's in there that you haven't implemented yet so just because things don't work right off the bat doesn't mean that you haven't done things right you just need to add all of the appropriate calls in so we have a sample tool that has these these calls in there called es test tool that comes with our our sdk sample code and you can easily modify that or use what's already there it does simple things like look for a user create a user and and so on and you could also develop your own test tool especially if your plugin is going to provide specific functionality that that's outside of the standard like custom off methods then you might want to just develop your own tool to test those things out so what are the the entry points we have validate this is called once when your plugin is loaded the primary purpose of that is to provide a token to your plugin that you can use later and all of your callbacks so that allows us to recognize which plugin is calling back into directory services we call initialize on your plugin after all the plugins have load this is the point where you would probably load up your config file or do any startup initialization like that establish any any persistent network connections that you might need and so on set plug-in state is called when your plugin is enabled or disabled from directory set up and that'll tell you ok your plugin is now inactive you can go ahead and and close down any resources that you're using that you don't need anymore or bring them back online if you've been reactivated process requests this is where the bulk of the work is you can probably implement all these other functions you know in a couple hours really easily and we've got you know sample implementations in our in our sample code but process request is where all the API calls come through so essentially that's going to look like a big switch statement that looks at what the actual request was and then based on that you'll probably hand it off to another function - to deal with the particular request that's coming through and we're just stuffing all the parameters that are coming in into a big struct that you need to interpret and finally shutdown is called just before directory services shuts down so again another another time to release any resources that you're using before the directory service process goes away or save any any changes that you might need to save so the other half of it is the callbacks we have des register node which is how you publish your nodes and the guideline that we suggest you follow is to prefix the name of your node with your plugins name so the reason for this is that when a client makes an ADD es open door node call we don't know which plugin to use for that so if the name starts with the name of one of the plugins so for example slash net info / LDAP B - are the plugins that we have then we can directly go to that plug-in out of our table and dispatch directly to you if you give it some other name of course there's a possibility of collisions but the other thing is we have to scan through the list of plugins to find one that might handle and depending upon what what operations the plugins do to handle open-door node it can be less efficient es unregister node is used when you want to take a node out of the list that you're no longer providing service on so if that your config file changed and that that LDAP server is no longer there you would use that function and then we have D s debug log to log debugging information that's an ADA when you're when you're developing your plugin so as I mentioned before the directory service daemon runs as a root process and your plugin is considered a trusted entity so you really have to use your power wisely it's not it's not quite as bad as working with the kernel but you can really do a lot of damage if you don't know what you're doing is root so you need to make sure that you limit your access to the system to only what you need and also don't do anything on behalf of a client that that you're expecting the system to prevent you from doing so in other words you're running in a UID 0 process so anything you try to do will probably succeed so if you want to provide any kind of security then you need to be aware of that in your plugin and as I said before also you need to be logged in as root or at least have a root shell available to be able to develop a plugin so this means on Mac OS 10 desktop that you'll have to go into net info manager and enable root to get access to it or use sudo or other tools like that we also restrict the the plug-in directory library directory services plugins to only root access so this slide is a little bit more information probably more helpful to you as a directory services client but it's also applicable for plugins you've probably heard a lot at the conference about the performance tools and leaks and things like that and it's I can't stress enough that's it's very important on Mac OS 10 that you you clean up what you use because if you don't you're gonna make the Machine swap at some point and then your performance because it goes down the toilet so specific to directory services you need to use clothes and D Alec calls to clean things up when you're done with them so clothes is generally for references and references are representing memory that's allocated in the directory service daemon so even if you're using malloc debug you won't see these leaks so it's very important that you you look at take a look at the sample code you can see which functions correspond but there it's not just the open calls that you need to close on there are some other calls that allocate references particularly when you're parsing a buffer coming back from directory services that you need to close and if you don't close it means that the memory footprint will be increased until your your app closes down directory services completely then the d alec calls you free memory basically structures that that our framework is allocating for you so those would show up if you're leaking those those would show up in malloc debug for example and again a lot of those are allocated with alec calls but there are other ones that allocate structures like get record entry for example so you need to be aware of that too another one is release continued data this one is used when you're you're making a directory service call that returns a buffer to you but we can't fit all the data in one buffer so we'll fill the buffer and then give you a continue data for the next time around so you can get everything you need out of the buffer then make the call again with the continue data and go on and get a chain of responses so that's the normal the normal sequence of events if you don't want to do that then call release continue data after you've backed out of that that multi-step operation and that will allow us to free the memory on in the directory service daemon that's being used for that so again take a look at the sample code on this and that will help you see where the cleanup calls need to correspond and D s wrappers is a good a good example of what you need to do to clean up directory service structures so again if you if you're interested in plug-in development please talk to developer relations we can we can help you with this and and we want to help you with that and also take a look at the API and plugin documentation it's actually included with the ten developer tools so if you've installed ten developer tools it's on your hard disk already take a look at it it's very helpful in understanding the API and we've got all the all the constants and functions in there that you need also if you're going to develop a plugin you should look at our API and decide how are you going to map our abstraction onto your directory system so what is a node in your directory system what is a record and so on alright so now we're going to have a demo of a directory setup so can we switch over to demo one please no it's asleep we switched the demo one instead of two yes excellent thank you okay so I have here actually Mac OS 10 server on this machine but as we said before directory setup is is also part of of Mac OS 10 desktop so let's take a look at directory setup you saw the screenshot earlier we basically would come up showing you the list of plugins and also you notice that the interface came up locked so this means that I need to click the lock button to unlock it because what we're doing here in directory setup is actually changing who can login to the system so we really don't want anyone doing that other than an administrator so now that I've unlocked it I can make changes we have the ability to configure net info if you want to bind to a net info parent or just use the local right now we're just using local alright let's take a look at LDAP I showed you this was working for me before the before the session started so if you can figure LDAP you get a list of LDAP servers and we can go ahead and add a new server basically what we do here is we specify what mappings we want to use so and also what the IP address is and so on so say test LDAP server and you can put in an IP address or a domain name in the that field so I'll just put in a an IP address then we need to set up the mappings that we're going to use for our record types and our attribute types and this is kind of similar to the way that the tcp control panel works or the network setup works basically each line of this write text box represents one mapping but you can have multiple lines so you can map things to more than one search base for the record types we're using search bases as our mapping so you can change these to whatever you want for the purposes of just getting the file server working we really only need to worry about users mapping and we can leave the others there at their default or delete them it doesn't matter they're not actually necessary we also document in the ten server documentation for particular services what you would need to map in directory setup to use that service with LDAP so then on the data or attribute types again we have a list of our standard types and you can map those to the types and LDAP that you want to use and again it's multiple mapping so each line is one mapping so I could do for example UID and CN for record name and then for real name I could do UID so actually I really wanted to do CN so that's that's common name it's old absolute deviation of a common name then password if you're using using open LDAP for example or some LDAP server that has a krip pass where it's stored in it then we can use that in the password mapping and actually do a crypt off without sending a clear text passwords out over the network so if your LDAP server has a correct password stored for its users it's a good idea to map that if you don't then we can still do all that bind but the security is better if you're using crypt then the final tab here is access which allows you to configure whether you want to use anonymous access to talk to the LDAP server or specify a distinguished name and password if you need to bind to the server to do anything with it then this is important to setup so for example with Active Directory you would need to use this you can also specify the timeouts that you're going to use and the port number if you're using a non-standard port and then once you've set up that LDAP server it shows up in your list you can enable it when we see if I can bring up a server admin to show you some more of what what mackelson server is doing with directory services all right so the this is the server admin for Mac OS 10 server might look slightly familiar to you if you've worked with apple sure I P recent versions so the primary integration I wanted to show you is in users and groups so all of the users in groups information is is access through directory services and when we're populating the local net info database we use directory services to put that information in so if I create a user for example then that would be using directory service calls to set the password to set all the attributes that our services need to be in there and so on so and now once I've created a user there again it's available to all the services the other thing we have in in users and groups is the ability to use to search on the search path for example so now that I've added that user I can see that they are actually are on the search path okay so so basically this this just illustrates that the directory service API is provide everything you need even to do a full GUI like this and and both create the data and access it although you know for a directory services clients you probably would mostly be interested in just getting at the user names and doing authentication but you can also you can also populate the database too if you need to so I think I'll go ahead and stop there since directory setup doesn't seem to be happy today and I'll go ahead and turn it back over to David work to finish up my god I think we took a risk booting off the firewire drive and for those of you know it's not technically supported that ends the demo we're gonna do a wrap-up whoops I have this upside down directory services wrap-up so directory services and other after technologies this is a lower-level technology but it can be used to build some high-level solutions but it complements other technologies and it provides common framework for higher level software to access directory data another compliment to this is the NSL talk and network service location I'd recommend highly that you all tend that tomorrow if you're interested in directory services there's some exciting integration going on there the opportunities for developers are to directory enable your client software as you saw you could use the Mac OS 10 server administration to populate a local net info database and then using the directory API your application would inherit all the users set up by Mac OS 10 server administration if the user had configured the search policy to access LDAP your application would access LDAP so there's a lot of benefit in you enabling that in addition you could provide alternatives to Apple software with by either providing a competing web server or competing file server but still leverage the technology the setup technology and you can provide value by adding a directory plug-in directory plugin is clearly an opportunity we're actively looking for anybody willing to develop it to a directory plug-in we think there's opportunities here for people to either promote their own directory service or to access existing infrastructure and again the radbend client or the admin the Mac os10 server administration client uses directory services if your plugin is read/write there's there's integration there that i'm radmann our Mac OS 10 administration's offer could actually be used to set up users in your directory so the key messages is we're using this architecture for our own software the directory service api's brings directory access to all of apple software platform customers are leveraging this API we already have some customers using LDAP where there's some very large installations there's one customer in particular has over two million users in an LDAP server they took the MCOs Tim server package unpacked it configured their LDAP server end directory set up added it to the search policy and promptly logged on as any one of those users that they actually knew the password of and so opportunities exists for you to base your application on directory services or develop a plugin double the opportunity you can do your product on the high level you can do it at the low level or both resources directory service plugin and API documentation on the developer CD there Thomas wires the developer relations he has access to all the sample code that we've referred to and of course the Apple Developer connection you