WWDC2000 Session 417

Transcript

Kind: captions Language: en yeah I gotta say I'm just stunned but there's many of you here because I saw the original schedule maybe a month ago and I said oh look I'm after lunch on Friday and up against the aqua feedback forum it's just going to be me and a couple of close friends so this is I'm stunned by this so I'm going to be talking about large-scale web optics applications and because the content of this is a little more subjective than some of the earlier sessions I thought it's improve appropriate for me to let you know who I am so Steve Meyer consulting engagement manager from Apple eye services I've actually done web objects and next Apple related technologies here for just about two weeks short of 10 years first six years is that I was doing what you'd now call cocoa development two of those as a consultant another for doing shrink-wrap and in the last four years I've been with Apple doing the web objects development and just recently been working as the consultant consulting manager and what that means outside of the business aspect of writing proposals and things like that is I'm responsible for doing architectural reviews design reviews at the beginning of projects midway through projects sometimes when customers have done projects and they say okay can we have someone from Apple come in and just make sure we're on the right track before we go too far so I think I actually have a fairly good picture of what everybody has a background for what they want to do so introduction why is it important that you're here ok thing is that even though we work in a business where we tend to think of ok everybody's on the web now it's not even close to true matter of fact the big money isn't there yet I mean the government is just starting this state and local government is just starting this your conservative businesses you know caterpillar General Motors you look at the Dow and they're the ones who actually have a lot of money and they're used to spending a lot of money and I can tell you from the perspective of somebody who done the small startup thing that consulting to those kind of people is a great way to pay the mortgage while you're writing the next killer app the second thing here is that large-scale projects really are different you can't just say oh it's twice as hard place as many people the standard problem there it has to do with communication but there's another thing that Ernie kind of barely touched on which many people don't realize and this they stick around for a while and I like to use the analogy of a hill and if you could go with the picture that everybody kind of uses when they want to indicate steep learning curve so I want to use this in completely different context you've got this starts out gentle gets steeper and steeper and steeper and if you think of that in terms of the complexity of a problem what you find is that as everything in the tool space improves you're cutting little steps into that hill and what that means is year by year by year you're making hard problems simple you know that top step keep your icing but there's always a top step it means that at some point you reach the top and what you don't realize is that while we're making the harder and harder things easier we're making the first hard thing harder we're losing the middle ground so what you end up with is case where as soon as you get past the the easy stuff you're suddenly in the complex and you need all of the what people say baggage but it's not of large projects so what you're going to learn here mostly if I do my job is how does web objects fit in large project you know what do you want to do special with with objects when you're in a large project and of course what I think is most important what comes out of my experience of reviewing things is what not to do how to stay out of trouble so that is a large project if you have any of these things you've got a large project now that's kind of misleading because most of those things are related I mean no one is going to have a large project team working for a long time producing not much for a few people so it all does kind of pile together the one thing that's on that is not on this list and which kind of ties the whole thing together big money when I say a large-scale project it's not less than a million dollars okay I just use that as a number to my own number but so I want to start out with just give the general engineering practices if there's anybody who's run a large project is a project manager doesn't matter if you've ever used web objects at all the next ten minutes or so is going to be either boring or comforting to you and for all those people who have never been on part of a large project this will tell you what it is that is going to be expected of you so you always need these practices and for the most part web objects doesn't impose anything on it so first obviously project management and I'm going to try to go through these relatively quickly staffing we're not just talking about the development staffing here we're not just talking about database admins or network people we're talking about configuration management talking about techie mode technical documentation all that stuff what we as developers typically call overhead which that's wrong communication it's another white nice way of hiding a dirty word developers that means meetings and I think it means real meetings in the sense of actually talking to each other face-to-face video conferencing can work audio conferencing doesn't work email you just wishing if you think that's going to get it done scheduling not much to say there except that when you have a large project you look out and you see you don't really have to deliver for nine months and it's very easy to just kind of cruise through the first six of those so scheduling really should be you need to task things basically a week at a time if you if you're regularity is any courses in that you're not really scheduling and the last thing that you can't possibly track it okay source code control this is kind of a religion with anybody who's been on a large project you're not serious if you're not controlling your source and also it's not just source although this says that every single document associated with the project put it under the version control if nothing else you know you've got it all in a repository and somebody's responsible for backing up the repository you can never lose more than since the last time you checked in and in this case I actually have a recommendation I really like CVS it's the most common thing for our developers to use it's used internally by Apple and the biggest advantage for a large project is this the optimistic locking it means that you're not getting each other's way because let's see show of hands here who knows what optimistic locking is okay that's only 25% so I want to go through good explain that the idea is that when you have a source code control system a pessimistic system basically says that you can check out a document for viewing but you can't modify it until you say you know I want to lock this thing and assume you've done that no one else can lock it they've got it and this presentation is actually good an example what the problem with that is is that there's like 45 slides in this and I turned to the first draft about a week ago and then I flew back home to Virginia and I made a bunch of text changes to it the same time we had a graphics professional changing about for the graphics here we came back and since we didn't have any communication we both changed the thing so that's no control well let's suppose we put it under a pessimistic system what what happened is he would have noticed that I was away he would have checked it out I would have been somewhere else not being able to communicate with him I would have broken the lock anyway and I'd be off and running so in a large that's gone that's what's going to happen when you have a whole bunch of people who can't necessarily communicate all the time you just get end up breaking pessimistic locks all the time the optimistic locking scheme is that everybody's allowed to check it out it's one checking in that everybody has to sync up to make sure that they have the latest version now great example it turns out that between myself and a graphics guy we'd only both touch one slide so I could have put my stuff in with just one conflict and then we could have negotiate over that so that's why I'm a big fan bug tracking you know all the software is going to have bugs you can't track it you can't fix it simple enough I don't have a specific recommendation here we use a lot of different things including some homegrown things where it says web access here what I really mean is remote access because again if you got a large team sooner or later somebody is going to need to be able to access that thing when they're on vacation something like that the other elements here mostly just have to do with their what's good in supporting release management and some other things are we coming up real soon you need to have a build test team and of course whenever you hear that you think your team the team may actually be one guy for half time but there ask me somebody who's designated is you know this is the person who builds the code you need to have a fixed internal release schedule or start build schedule make a distinction between builds and releases because builds are really internal and what you need to have is somebody who can say you know noon every Friday it's my job to go put a tag on the whole CVS repository with the build name check the whole thing out and build it the nice thing that does for you is make sure you always do have something I can build at a moment's notice in case a customer suddenly decides that they'd like to see the latest features even though you haven't promised them or release and it also is good because it keeps the developers knowing that you know I should be getting stuff done every week there's things on the schedule every week and there are build names every week so if you say I'm going to fix this bug in build 34 you know exactly how many so they build 34 is so it's really good for that okay that's pretty much it for general stuff now let's talk about what it is that you need to support web objects itself first documentation now the project people are going to go well documentation is some things you need for everything anyway and the real reason I put it under the web objects thing is it web objects does have what I would call some special needs for documentation only some of these though like requirements requirements are pretty much the same everywhere and that they're you from your customers what you're likely to get is relatively vague high-level descriptions of functionality okay specifications is something else specifications what we want for specifications is we want to have actual prototype interfaces of all the screens and we want a navigation map and we want functional descriptions and what's supposed to occur between those now the customers aren't necessarily going to give you that and I'll get back to that in just a bit but that's what we call a spec it's not enough to just kind of say oh would we like to be an e-commerce system the design documentation actor is kind of interesting because as we usually do design documentation a lot of it looks like the specifications one nice thing you can do is you know for each of the the design mock-ups you've got go ahead and associate with that the you know a screenshot of will builder for what that component looks like and then you also you want to have documentation on a component by component basis that you know a one sheet on each component basically says here's what it is inputs outputs actions etc the deployment documentation is for the most part fairly simple it's if you had to install the software from scratch including really you know assume install the web server install the apps install the database etc and if the thing crashes how do you restart it that's pretty much what it comes down to documents like this it's really good to have pictures and I'll be showing you a picture a little bit but you know the kind of picture I'm talking about the other two things the coding in the user for the most part those are pretty standard documentation they're only included here for completeness so the hardware architecture you should expect to be working to case you start out this is a large project therefore eventually you're going to have multiple web servers multiple app servers multiple data sources that's not the same thing as necessarily saying multiple databases or a single database replicate or whatever that's more detail than I'm talking about right now but that's what we're looking at and I've simplified the picture a little bit you notice that only webserver one is connected to anything and only really app server one is connected further downstream otherwise lots of lines all over everywhere up in front there the the DNS that's that's referring to a DNS round-robin that basically masks the fact to the net that you've got multiple web servers so they both appear to be the same thing this is the kind of thing is that you can expect to see something like this in your system admin slash deployment documentation so that someone knows how to hook something up now I'm actually going to zip by an animation that wasn't supposed to be in here and move on to the software architecture that that hardware architecture locks you into already and that's that's actually partly a bad thing importantly a good thing first thing you're already dealing you know that because you have different Hardware separate Hardware multiple instances all that you start out designing for the problems you're going to have with data caching and synchronization and I'm guessing that many of you were here yesterday when Dan did is presentation on that so you should know the issues to look out for also given that once again if you've separated you know you've got multiple instances you know you've got data synchronization issues go ahead and break out the application into more than one application I mean don't dump all the functionality into one big thing and typically that's done by you pick the things that are read-only because you can deal with their synchronous synchronization one way you pick the things that are you know Publix can see that only maybe employees can see and then only people were just you administering the database can see and when you do that you also get the benefit that you can then separately optimize configurations for each of those one thing that is really quite helpful for that is if you've got any particular function usually this is an admin function where you're expecting it's going to take a long time it's going to put a big load on something you can schedule it so that that app only runs doing low-level times so you'll never be hitting your system at the same time your users are ok so you've got an architecture let's talk about some issues in application design I'm not going to teach you how to design an application so let's show you what parts of application designer likely to cause you problems in a big app ok first typical code ordination looks like this I mean and you have a data framework or more than one framework you've got multiple sources of different types it's good to get that off by itself that'll have your EO model and that'll have your custom classes you have this fixes framework which always accumulates and I'll talk about that a little bit but it happens and then you'll have your architectural frameworks your reusable components for the most part then you'll have one or more applications as a result your team looks like that you have an architecture team you've got a data modeling team you've got a couple app teams now it seems like every time I look at how someone has set up a team they've piled all the experience people onto the data modeling and architecture and there's a real problem with that well there's two real problems actually one has to do mmm one of them has to do with the fact that you're always going to be running the risk of having your data modeling and architecture being so sophisticated that your app teams can't use it the other problem with this is that and this is going to kind of come up severly you've got a maintainability problem there because at any particular point if your senior people who are most likely to leave so I I always like to have you make sure that even if we go with team here and again teams aren't necessarily very many people when I talked about a large project way it's beginning what is the large team a large team is maybe you know 15 or 20 people but the web objects part of that we're talking three four five I mean that's it if you have an app that takes 15 or 20 web objects developers I only know one of those in my personal experience and that was an effort that was going to replace a system the federal government currently spends three hundred million dollars a year to maintain so if you got one of those call me anyway the other thing about this is you have to make sure that you have planned communication and I'll get into this pretty shortly about what it is why the communication is necessary but the part that it's plan is what's important suppose you've got an architecture team they're building musical components if you have that once a month you have to make the architecture team stand up do a PowerPoint presentation on what they've done in the last month so the rest of the team's know what's available you can't count on them to just you know send out messages we've got this new cool reusable component come look at it doesn't work that way so plan the communication plan the meetings ok data modeling we're one of the data modeling team what the data modeling teams need to watch out for you got to make sure that you're modeling the process not just the static data where you can look at what the company does you need to be sure how are the applications going to use this data and the biggest thing that happens when you don't do that the next thing you over normalize there what that means is it you can never ever actually get to a meaningful collection of data to display to a user without having to follow a bunch of relationships or if your data fake database folks do a bunch of table joins I got to save like 80% of everything I see that I look at and I don't like the data modeling that's it and the nice thing about it is that there's some relatively simple fixes and you tend to get big performance wins when you do it and people are impressed the bad thing is that it shouldn't happen in the first place and this last one being cautious with the entity inheritance I don't know why it is that comes out this way but every time I run into a project where someone has used yet ID inheritance there's a problem on the project there's different problems but it just seems to be one of those things where you know if you see that someone's using entity inheritance ask my and make sure everyone knows what's happening abstraction of data interfaces is a great thing it's a it's a nolo concept from way back but as you can see I have a couple things that I don't like about it and really the biggest one here is the first one that if you try to abstract too much away about what your data sources are you end up losing any of the advantages that the people who designed the data sources built into them a great example is if you you can put an interface on top of us and if you forget to put prefetching of relationships into it you've blown any chance of optimizing your data access second thing is kind of similar if you're if you're trying to abstract things that just don't go together you could end up with something that doesn't really work for either that's actually a general case or the first one and the last is you kind of whenever you get into the of I'm going to take something and build a system that'll work with anything you always base it on what you know and when you come up with something that doesn't match what you abstracted to there's a really good chance instead of changing your abstraction you're just going to hacking some code to make it work with just that one source and I'm you defeated the entire purpose of doing extraction another thing it's great design for reuse although see that never really happens in the real world either what you have is designed for use redesign for reuse or design for use recognize you can reuse so you know you can tend to want to kind of make a second sweep through the whole app once you've got it done and figure out which bits you want to use next time but for a lot of people who have done even small apps before they have with them candy collection these are things that I had been useful before so they're actually or no it's not like you're always starting from scratch with reusable components the best thing for when it comes to reuse with web objects is the small reusable components small elements and if you want to know a good way to design them don't look at what low extension says as a framework the other thing that clearly from a you reuse standpoint is your data framework your enterprise objects if you model basically if you model the thing properly you can carry it around although you may need to modify it for the individual needs of applications and I'll talk about that too and lastly when you do this organize them in the things that don't have dependencies on one another it's a real shame when you find that there's a you know a really cool table implementation but it requires that you link in a framework that has somebody's own idea of what a set should be I found that a lot of times it's not only frustrating for the people who are trying to use it but there's almost you guarantee at some point someone's going to forget to drag along that utility is framework and it's just not going to work I got asked about this a lot from a design standpoint people want to go things be fast they hear who multi-threading let's do it and you know multi-threading is hard it's really hard there's a few cases where you really want to use it it's mostly in terms of somebody something that's listening and gets a request and spits requests off to somebody else to process and then starts listening again now you can think of you know the whoa adapter is doing that but you don't really want to because your application is doing that it's actually doing processing and I said it's really hard to write it's really hard to debug notice I didn't even put tests in the middle there because the assumption is you're going to have to debug it you're not going to get it right and the cool thing about this is that multi-threaded things fail under load and they come under load when you load test them and that's at the end of your development cycle so you're pretty much when you go down this road you're pretty much guaranteeing what you're going to get is a hard to track down bug just as you want to go to production even if it's a simple solution that's quite a lot of stress the benefit benefits of multi-threading tough to really be sure as you're designing you're going to get a chance to use them because unless you have the source to everything down the line that you're dealing with you can't be sure that somebody is in defeating the whole purpose you know whether it's occurring somewhere in the enterprise object framework in the adapter for a particular kind of database in the client libraries for that database in the connections to the server there's just too many places where you could spend a lot of time and find out that someone's just serializing it anyway however you have to do it web objects does support it I mean by default resources are requested and dealt with you know images that sort of thing in multiple threads and you can turn on the request handling and have a ball but I warned you so I'll really say there I go to I have analogy here that I like to use when someone says I'd like to go to multi-threading for performance and that is that thing at a grocery store you've got a couple to checkout lines bunch of people piling up behind it and what do they do when that happens they open up another checkout stand and people go move over to that and you're good to us that's starting up other instance now if they manage to actually fill up the whole store all the time they open another store mile down the road so that we're talking about bringing up and up another app server what you never ever had anybody do is go to the checkout clerks and say why don't you process some of the customers in parallel that's really kind of kind of the end of my feeling on multi-threading now when you're designing things you know that there are some warning signs and this is actually a warning is a strong word but I couldn't come up with one I really wanted this is when I go out and I review something I see one of these things happening I have three questions do you need to be doing this to get the functionality you want is there any other way to do this and right now do you have it documented to the point where I could have walked in not talked to you and found out the answers the questions I disaster and if the answer is yes to all these things I'm pretty happy but if the answer is no to any of these things then you know there's work to be done first of them kind of go together you subclassing a lot of Apple classes you're kind of intertwined with apples classes you know this is when you go to training this is what they show you this is the ideal you've got bottom four things are all ours top things yours and of course that top thing maybe multiple frameworks but it's all up there you're not really getting in the way of the web objects or the gyro control or any of that that's what I see a lot just room for me now I got to say that this is the kind of thing that the objective-c developers are much more likely to be able to get away with the thing is the objective-c gives you a much bigger gun to shoot yourself in the foot with but that's not saying I don't like at the language I like it but this kind of structure is something that's really common when you've found something this kind of accumulated bits of objective-c and another problem here is those little thin blue lines chances are they get dependencies between them and that does that violates one of those rules for usability I pour it out that you can't even if you decide you really like that one you know second blue line there it's got some functionality you need you may have to carry along the rest of it and that's bad more warning signs you got large application in session classes now this is partly a design issue and partly a coding issue every once in a while you'll want to sweep through the application and section classes and see what cross they've accumulated is people just put things in because they're it's a big global variable basically you can stuff anything you want in there but even from a design standpoint application and section classes aren't necessarily supposed to be all that heavy you're using objective-c in a Java project this is where I said I I like objective-c I like Java I'm not going to get into the religion nor wars over that but if you've chosen Java and I see Objective C anywhere in your project I know you've drawn with those blue lines similarly the next two actually are not all that bad I mean you're working with EEO access framework there's lots of benign reasons to be down at that level probably more than anything else here but still it's at that point you're down below the object level and all I really want to do is make sure you know justify your use the last one is new one I only really found out about this in the last two months or so and I think it may be the result of frustrated objective-c programmers this is an option they have to do some very strange things while sticking with Java and that's when you have custom key Val code that means that you're doing things with the bindings or you're doing things with the value for key you and your enterprise objects a it's a relatively benign instance of this is if you're changing your enterprise objects so that they can tell based on a key value whether or not you're asking for a let's say address from this particular page or that particular page you know if you just say put an asterisk on the end of the key and you've got something that can parse that out that that's relatively benign mostly it's a convenience it keeps you from having to put methods inside each of those components to go and get the value from the enterprise object and manipulate it as that page once it it's you know I'm against it but I'm not going to push it real hard the thing is this opens up it actually does and I kind of hate to say this because I'm telling you you know don't build the bomb this way is you can put anything in bindings and if you're overriding your they take value for binding you can do anything with it you can very easily just put a binding in it and says you know as long as the the binding starts with you know three exclamation points that means that the value that comes in is a block of Perl code and I want you to send that out to the Perl interpreter and take that value and that actually I've seen one not with pearls or something else and that's exactly what I mean by oh my god so that sort of thing is really hard to document too because wad files are not where you're looking for code and for as we as we move more and more into the abroad developer community more people are gonna be looking at this stuff in web objects builder and it's really hard to tell just as you're zipping down through the bindings you know this has an asterisk on the end that means if it's a number coming in we're going to divide it by two before we use it so no I don't like that a lot I think that there's that there's a there's a real possibility for abuse right now on that so let's assume you've managed to get through design without any major efforts you're on the prototyping and you got to divide the prototyping into things this is this is a customer issue the user interface prototyping now if you remember way back at the bidding I told you what a spec is and user interface prototyping is writing the spec and you have to have the customer know that make them understand until you've locked down like 90 percent of the user interface to a level detail that you've got screenshots they haven't yet specified the app and all the work that you're doing up to that point may not be final the technical thing you know that's where you're trying to decide whether or not you can actually hit that airline reservation system whether you can integrate the credit card processing with the Oracle database that sort of thing and you'll want to make the interface for that as ugly as you can you don't want to confuse the customer into thinking there's any possibility that the code rewriting now is anything but demonstrating that you can go do it I like to say when you're doing the demos you want to demo the user interface on a Mac demo the technical stuff on a PC and if you do that this throw it away thing works on its own wind a construction here I'm mostly talking to the developer assume the projects and then continue after you're gone this is a long project you know it's got to be at least six months and you got to be responsible and I have some some things to say about responsibility in a bit overall though the construction of a web objects application is no different than the rest of development it's not much I can say about web objects technology specific things as you're writing the code two things though they're really helpful and unfortunate I don't see enough of standardized society of standardizing keywords in your comments and I'll have an example on this it's really helpful to come up with a list of keywords that someone can search through the code for and that list of keywords ought to be put in your repository can hear that again document document document okay how about that anybody seen that kind of comment before interesting thing about that the first the first things that's actually two comments I mean separated by an indefinite amount of time and the first the first comment there I sort of made up although I've got I've seen something close to it the frightening thing the second comment I lifted verbatim from an app that's in production all I did was change the three initials there at the end of my own which may not have been all that necessary because when I asked around nobody could tell me who that was that's what I want to see and you don't have to add line every line like that but when you're doing something like you know clearly like that that first thing there it's a line of nothing but but key words it's great to have you know fix me as a really common one I think we all use that occasionally but if your standard I say yes indeed this is you know what we're going to use when we've got something that needs to be looked at if you can put keywords for the dependencies of every version of things that you need to work with that's really helpful you notice you've got enough text there that no one's likely to go around and put a hmm line after it and I put my full name on a date so at least you know people may not remember who I am even with a full name but there's a date there so they have some clue as to when it was an issue and the other nice thing actually about doing the whole the keywords for versions of stuff this is a three five problem in theory in practice I completely made this up what you need to do is every time you upgrade before somebody searches through for all the three five keywords verifies whether or not these comments are right and changes the three five to four which means you've never got any issues that are more than one version old of any of your products and you really like that trust me more construction things at this point the developers stopped wanting to go to meetings really the meetings have a purpose it's up to your management from a development standpoint it's up to whoever it is your interface to the rest of the team to make sure that you not go into the meetings you know

• go - and one thing is that I I hear it

once in a while and people try to convince me this is a bad thing 12 of us trooped into a room five minutes later we we walked out again what a waste of time like no that's perfect it took you an hour that's that but to walk into a room walk out in five minutes that means the system is working means everybody showed up like they were supposed to there's no opening questions from people who didn't show up and everything's on track if you were to take a look at it you'd find you take more time trying to coordinate among those 12 people whether or not they needed to have a meeting then they just have them all show up and this last item clever is bad you know I tried to I thought about making this less bold and I talked to other people in my team about this concept and you know excited I thought that you know maybe it's just me and after talking to them I decided it deserves its own slide okay clever is that and keep in mind this is clever in terms of coding this is not intelligent design this is clever coding and I have an example for you I think probably the cleverest I've ever been which is to say the grossest violation of this principles go back to 1992 I think it was I was doing what was what's now cocoa development and I really wanted to print panel to work differently so I went into the debugger I disassembled the appropriate part of the print panel code I examine that and figured out the memory locations of the private instance variables of it and a little pointer arithmetic and little guessing I managed to get it so when I wanted to I could stuff data into those places in memory that the print panel would go look to see what its state was and I could change it and it worked you know this was code that you know I in theory owned I was going to maintain nobody who's paying me to do this it was my this isn't something I was leaving behind for somebody at my expense although I'm standing here and I've got an Apple badge on when I have a badge means I'm not maintaining it I think I know who originally did have to maintain that and I liked him so even though there's maybe 20 lines of code and I had a page and a half a documentation on that whole thing when I look back on it I was much prouder of that achievement then that I am now and although it's hard for you to think forward and imagine how will I get a feel about that 10 years from now it's a pretty good guideline so how would you feel of a friend had to maintain this now the three items there I mean why is it bad outside of my story basically I had a an even bolder version of this slide and I decided to tone it down because probably without my words it would seem a little too harsh but first you can fool yourself and by that I mean a couple of things but you can have a problem with you may not actually understand everything you're doing you know I think the print panel worked it worked in all the cases I tested but I mean kind of wandering through the assembly code is not necessarily the best way to be sure if I had to go back to that now I mean admittedly that's eight years ago but if I had go back to maybe even six months later I may not know any idea what it was I was doing and that kind of follows on you can fold your team two different ways you can fool them in coming up with something it's so clever they have no idea how to use it or you can fool them because six months later you're gone you know you're the new CTO it show me the money Tom and they have to try to maintain it unless this is something that people don't think of you can fool us and by this this I mean the engineering team at Apple because everybody knows you know there's a gap there's no guarantee if you use private API and of course the print panel thing you know I knew there was no guarantee but what you don't really get is that there's not really a guarantee if you use public API in a way that nobody ever thought you could if you're depending upon your liked it order in which certain things happen to get called so you can know stuff things into a request here and pick up the fact that those values are in the response over there you run a risk and the the scariest thing that you can hear from a project management standpoint when someone's showing off something cool is ooh I bet Apple didn't know you could do that okay so you got the app built maybe testing you need to have a real separate environment to this you need to get your customers to duplicate the production as much as you possibly can testing Web Apps is a boring thing to do use automated tools for both the regression testing and for load testing although it's occasionally it's kind of neat to just get the whole accounting department to go at four o'clock come and get the app gotta give them a break in their day you use the humans for functional testing but the other thing you want to do is you want to have the occasional if you win stopping in doing your load testing because you're going to gather stats both from the inside of the app in terms of you know the low stats things and from the outside in terms of what the overall you know what a browser would see in terms of response time what you don't get is the human I thought this page would go quicker so while you're doing the load testing just have have a human sitting by doing some stuff and letting you know how they feel about it and we're back to meeting schedule the regular bug tracking meetings and make people will show up because once again the developers are very busy trying to do things other than fix bugs that someone mentioned the week ago so make sure that the right people attend each of those meetings not everybody but that you have them to really buy time you're down to the testing phase it's got to be daily performance now it works how fast can it go I'm cheating here there were like three different sessions you should have gone to if you wanted to know about performance these are the questions everybody asks though and the real point of this actually is not so much what theory is to say here is where I'm saying it if you haven't gone gotten to this far you can't answer those questions and too often I run in the case when I'm doing proposal type things that someone's going to say well I realize I haven't told you what the app does yet but how many boxes do I need so you can't do it I mean I have tried to come up with heuristics for this sort of thing to just say well if you happen to have you know a gigahertz Pentium machine running as your app server then assume that it you can support you know four transactions a second as long as no response is longer than X doesn't work you really do need that because you're either CPU bound or you're going to be memory bound on these things and you don't know which until you know how much memory it takes so never answer these questions till after you started testing that given assuming it's slow what do you want to do about it first bold item there I think if some people are going to say it well that's obvious but it isn't because it's not just what's most commonly slow its what's really slow that kills you from a user perspective let's say you have a whole bunch of pages that come back in two seconds and one page comes back in 10 now that thing shows up 10% of the time you can do the math you end up with a number that's you know three seconds or so for your response time within the app that's not what the user sees at all and for that I'm going to go back to the whole grocery store thing which is you know you've got your 15 items or less and you're standing behind somebody with 100 items and behind you is a bunch of people who also have to three items now from the checkout clerks perspective one person took a lot of time other people just zipped right through your back they're human so the approach that grocery stores take if they have 15 items or less and I actually kind of advocate the other side of this which is you should have lines for people with lots and lots of things and make them stand behind each other and it sounds like a joke but it actually it goes back to this as well I mean as app design it's a good thing sometimes to recognize when an operation is going to be expensive and shunt it off to the app served is handling the expensive operations and this is where the human factor comes in is people usually recognize that you know this is going to take a long time I want to find every hotel in Orlando that's great however for the people who think they're doing something that's really short and they end up behind somebody who's looking for every hotel in Orlando they're upset so 90% of the time when you're looking to improve the performance it's data access so you may look at modifying the data model or you may look at customizing versions of the data model for each of your apps because the readwrite app may have a different need then the read-only a lot of people don't think in terms of that because you've got your data framework well you don't have to have just one data framework you can have them optimized for the uses that they're going to have so think about that now lastly you're manually affection the data you need is something we almost always resort to in one case or another and when I say manually and it's not all that manual we're really saying here is rather than configure a well display group with a data source to go and get data go ahead and create effects pack yourself fetch objects with it and pass the array over to the display group and what that can do is you can then set things like the prefetching at a much finer granularity for what's being used at the time a little bit more on that data access this is something I see a little bit too much of don't have complicated bindings that just keep dotting along because in the middle there you're not really sure from a coding perspective how do you get from manager to spouse up to car that isn't even necessarily all in one data source to find out what the spouses car is made you may have to go up to some you know mainframe system it's much better in this case to go ahead figure out the relationships that you need fetch the data for that and just let you know if you put that thing inside a repetition you'd be here forever so anticipates the needs of your interface elements and try to provide them with their data ahead of time so that they don't have to do it themselves they're not smart about data access that way and the last thing here you'd think you'd never have this but you do is that customers will ask for I'd like the whole database please and sure they understand that the fetch takes forever and it's making the app go slow but they don't realize that they're then transmitted transferring like ten megabytes of data to their browser and they wonder why their network is slow why is my email not getting it every time someone tries to run the app so limit this stuff I mean at the very least go ahead and you put put it in the display groups turn on the batching and if nothing else that's going to show them when they see because they've got a zillion pages of batches what a silly idea is to try to fetch all this data at once okay you've gotten to the point you need a deployment team same thing not necessarily a whole big team to be a small team it could be a guy cool thing about this is if you go back to what I was saying about the builds about the releases about the testing you've got an environment that's just like it that you're going to go to production on and your deployment team gets to practice every single time you put out a test build so they really know what they're doing but that time there shouldn't be any last-minute do we know Solaris kind of question and gather as many stats as you can while you you actually got this thing running but you don't want to impact performance I don't want to see a bunch of things log it out to the screen to let you know what's going on I'm mostly here talking about you obviously get the wou statistics but also what you want to do is check the CPU loads on all the machines web server app server data servers etc check the memory on all these things I mean it doesn't cost that much just go ahead and check these the check point these things once an hour forever because if you ever need to know what happened you've got knowledge and knowledge is power last thing there the application is not finished when you deploy that's a function of the kind of customers we're talking about here and the time that kind of apps we're talking about you can expect a feature is going to not make it into version one it's the only way it's not going to be later than it is anyway therefore there's always a version 1.1 and for a management perspective that means you need to be thinking about who your maintenance team is ahead of time and you need to figure out not just your developers your maintenance team needs some of the same overhead as the rest some some of this stuff can be shared some of it can I kind of just this is a sad things that the management types like probably good things the development types I didn't put this on the slide here the biggest issue that I run into with deploying a big app nutrition this is the point whatever and leave and really when you're using something like you know web objects you've got it you know it's smart people and a high demand technology at any moment you have to be prepared to have somebody walk in and say you know I'm gone in two weeks that when you deploy you know there's a real good chance you to lose like 40 percent of your folks in the next month because there are a lot of people who want to see that make it and then that's when they'll move on and if you're not prepared for that it's devastating hey what went wrong this is this is actually not even my slide essentially this is this is a unknown thing within software development is if you never managed to get it done at all it's management's fault no doubt about it no matter how bad your engineering might be its management's fault they planned poorly they didn't follow their plan they picked the wrong people they use Windows whatever and and the second half of that really is if you're stuck at version one then your engineers did a lousy job unsaid here but I think we all understand is it when they're not within earshot it's the customers fault really but I'd like to switch that around because despite all the things that are in the way actually most of these apps actually do field then they're successful so let's look at what went right who gets the credit and I got to say I don't care how good your engineering is if you actually deploy a version one management gets the credit and when I say that I don't mean the deploying version one means you've got all the features in or even that you've got it all in on time it means that things went wrong and the customer didn't fire you and the team didn't quit and you got there the second thing there is no credit the engineering when the customer comes back and says I know I only wanted one language before now it needs four languages I know I was going to use this service to validate credit cards and it will have a response time of maybe five seconds with 100% reliability but I can get a lot cheaper thing from somebody will respond in two seconds with 75% reliability and that's actually an example from my own experience if you get that far no great at that point you want to congratulate the engineering folks you don't want to congratulate them too early though that's that's when they leave and to be honest to be fair would it be fair also not on this bullet item is if it works particularly if they happen to be an earshot credit the customer never hurts and it's actually true a lot of times when I talk to engineering teams about their experiences you know how's it going I hear my customer doesn't understand the problem my customer can't make a decision and the ever-popular my customer is stupid and see it's not true the customer is not stupid the customer has different priorities than you and doesn't necessarily have all the information to make a decision so if I can apologize to folks who maybe missed lunch and are thinking the candy is is really a good idea right now I'd like to go back to the grocery store just one more time as an analogy I mean if you've got an allergy you got a metaphor pounding in the ground follow your customer at some time to the grocery store and think in terms of you know they're heading down the cereal aisle and they stopped by the frosted flakes and if you're going to have a metropolitan area chances are they've got choices there they're going to have the Kellogg's Frosted Flakes you know the national brand we're going to have at least you know maybe a regional brand but certainly a store brand of it and you know they may pick up the boxes they'll look at the nutritional information something like that but you know they'll put them down and then what they've got a choice to face you know okay the national brand comes with Quicktime which is school the local brand cheaper and what happens sometime within like 10 seconds to pick up a box put it in the basket they move on nobody ever complains that the local brand doesn't have Quicktime nobody ever complains the frosted flakes are too expensive compared to the local brand so when you are doing those kinds of things with your customer when you're trying to tell them bad news schedule slippage whatever what you need to do is very simple you lay out the options for them give them the consequences of each of the options they have you make it very clear we can do this your way but there's a good chance if we lose this guy over here six months from now we'll have to pull that feature because we can't maintain it give the customer that kind of information the customer will make a decision once they've made the decision document the decision put it in your repository so in summary big projects are hard they are but that's all big projects that's not just web objects projects basically if you've got anybody who's ever done a hard project in any other technology they can manage a big project in web objects and web objects helps because you know we're always cutting steps so the one what is a hard project now and is a big project requires a big team this year may not next year and the last thing is I'm not it's not really everybody is suited for the big projects I mean some people just they want to be too creative they don't like the structure and that's fine there's lots of places for that but you know if you can stick it out in there it's there's really an immense sense of pride in having done something like that it's very much like building the tall building and what you'll find is that no matter how many little projects you might do it's always the big projects that you remember and incidentally it's always the big projects you put on your resume so because we are so close to the end of this there's like nothing else you can go to if you haven't learned by now all you've really got is the feedback for them to complain about the last 17 sessions most of these for more information you must have seen before and the web objects lab is closed so sorry about that you must have seen these names by now Tony on earth if you got any questions about all the stuff even you know something that's specifically for me go ahead and send it to Ernie Ernie will route it to me and now we're on to Q&A [Applause]