WWDC2000 Session 414

Transcript

Kind: captions
Language: en
we have with us Eric Bailey to discuss
performance metrics trying to squeeze
even more information out of your
application and we'll try and squeeze
some more information into your head as
well enjoy the session okay so I can get
this to go forward here there we go
I'm Eric Bailey and Internet services at
Apple where we deploy applications like
the Apple Store and AI tools I'll be
joined shortly by Stan German who works
in the web objects tools group it's
going to talk to you about some of the
new features in web objects 4 or 5 which
are going to help you measure the
performance of your well objects
application the sessions have probably
gone in the wrong order we wanted to
talk about performance first how do you
measure the performance of your
application then when you find the
problems go and optimize it which i
think was this morning and then deploy
it which was yesterday so today we're
going to talk about performance metrics
I'm going to talk about how to measure
the performance of your web objects
application and improve it we're going
to focus in on your application not all
the other parts of your environment and
we'll discover what factors will
influence the overall performance of
that environment and help you pinpoint
the code you can go and improve so
you've probably seen a picture like this
before which talks about the overall
architecture of a very typical web
objects application where here we see
that HTTP request lifecycle comes in
gets served by the web server into a web
objects application the green area a lot
of the times hitting database
information flowing back to what objects
back out through the web server into
your client browser the point of this
slide is this look is to show that there
are a lot of factors that influence the
overall performance of your application
it's not just inside the code that
you've written in the action
Asian business logic area of your well
objects application other factors do
come into play and it's important to
note what all those are and a lot of web
objects deployments have a lot of other
boxes in them and that you may be
accessing LDAP you may be relying on
files on the disk statting those reading
those in writing those out all those can
influence the overall performance of
your application so today we're going to
try to answer a few questions how do I
measure performance what does it mean
and am I getting good results we're
going to go through whoa snap switch is
a very useful tool for getting started
and then move on to into event logging
which is a new feature in web log fix
for five it will help you get into finer
grained detail what areas of your
application need your attention
there we go Thanks so the web server at
the top layer of this really acts as a
load balancer in the web objects
deployment environment it's basic job is
to send the forward the HTTP requests
onto an instance of your application and
in so doing it's basically an i/o
intensive task you get down into web
objects and then you may be making a
request upon the database all the
factors on the left are very important
to note especially database and other
sessions have talked about improving
your overall database performance
optimizing EOF making your queries more
efficient and although a lot of us take
it for granted there's a lot of i/o
network traffic going on there's latency
all these play into the overall
performance like I said and you want to
not take these for granted once you take
some measurements within web objects in
your database at HTTP server layer and
at the client end you'll get a better
feel for whether this is important or
not usually it's not a problem in a lot
of deployments but then the last Lefter
you've measured some traffic and felt
the load on your application you'll
figure out if you need to talk to your
sis ID mins about tuning your routers or
configuring something differently so the
first tool that's built into web objects
we're going to talk about as well stats
there's been in web objects for a couple
of releases and it's a good starting
point to figure out if your application
is is on the right foot it's not going
to get very fine-grained like event
logging will but early on in the
development cycle you should take a look
at what stats and figure out if your
pages are coming back your transaction
times are acceptable we'll talk about
what's acceptable in a minute and what
might not be acceptable
well stats like I said it's not very
fine-grain it measures transactions time
at a high level basically an HTTP
request comes into the webserver
forwards it on to your instance and as
soon as it gets there start the timer
and as you can see here the database is
downstream of that request so therefore
all of the time taken to do a sequel
query is going to be recorded in that
timer once we actually are finished
sending the response out of your
instance back to the adapter web objects
based adapter embedded in the server
that timer will stop so what steps is in
the well extensions framework and it's
always on as pretty low overhead well
it's not always on you can turn it off
there's low statistics store class and
we'll application which you can use to
modify well stats customize it to your
needs a little bit and you could turn it
off if you wanted to this is a
screenshot of the direct action called
low stats an example URL above of how
you would access it in the well info
center application as you'll see in the
screenshot we have 17 overall
transactions in the top left corner of
the table this is not many transactions
and I don't I don't expect it you should
read anything it about your web objects
applications performance with that few
transactions you want to get it warmed
up touch all the areas of your
application the first time you load a
page web objects has to read that
component off the disk initialize a few
classes internally and you really you
really need hundreds and even thousands
of transactions through your application
before you get a sense of the
performance but here we we hit well info
center a few times and 17 to be exact
and we had an average transaction time
of 1.0 3 6 seconds now that shouldn't
alarm us because like I said you want
more transactions but that's not great
I know that whoa InfoCenter though does
some indexing that startup and we would
expect this to a certain degree the next
number over this interesting is the
average idle time I had this application
running for a while before I took the
screenshot and here the idle time is
hours really but this is the amount of
time between requests so how long your
application sits idle this is a good
number to use to get a sense and
deployment it may have been mentioned
whether or not you have enough instances
or too many I know that on I tools we we
look at this number a fair amount to see
if we need to add more instances you
want to have it be probably I don't know
it's it's it's a hard number to figure
out exactly what you want it to be but
10 20 seconds is good higher than that
is is even better if you want to if you
expect bursty traffic you're going to
want to want to see the average idle
time in kind of low or normal traffic
modes be fairly high in the twenty to
thirty second range so that you have
enough instances that can handle a burst
of traffic moving down the page if you
scroll down you'll see a couple of
tables that detail their component
actions and the direct actions so the
component action statistics table on the
Left we see the name of the component
being returned this is important to note
that you know in the in the graph and
the diagram that I drew before web
objects application does take values
from request invokes an action and
returns a response the component that is
being used to return the response is
listed in the left column so although we
see in the menu component that the
average was very fast eighteen one
thousandth of a second that was
returning the menu page so it also
involves whatever was the what was going
on in your application when you took
values invoked an action and returned
that component so for instance the main
component is very slow one point three
oh six seconds now again we don't have
enough data to be
sure and the thing that I would point
out is noticed that the minimum is 16
1000th which would indicate that it did
fine on a few requests and the maximum
is seven point one to seven so like I
said I know well InfoCenter loads some
indexes and figures out some things when
it launches to better help you find
documentation so it has some warmup time
so we'd really want to hit this
application a thousand two thousand
times and we'll talk about how to do
that a little bit later so that was wool
stats and now I'm going to hand it over
to Stan German who's going to talk about
event logging which will help you get
into more finer grained detail about
where your application needs tuning all
right hello I'm going to tell you
something about will even blogging which
is a new feature it will set it in 4.5
Louis tez has been sitting around for a
while but this is a new feature and most
people don't even know it's there so I'm
going to tell you about it it's way more
fine-grained and low stats it doesn't do
just components you can use it for
logging any kinds of intervals it allows
you to identify the bottleneck of your
app well that's what performance tools
are all about and what's interesting is
building to any four point five four
point five complication that you are
running
yeah patience okay that works so what
you're going to learn here what is an
event what's the definition of an event
what are the building events with
objects comes with a lot of building
events so that you don't need to do
anything in your applications how do you
enable it how do you read the data that
it shows to you and if you're really
into it how do you define your own
custom events for tracking of your own
application specific things okay I'm
going to give up on this what are the
prerequisites to use a new event logging
well on any platform with objects
four-point-five will have this enabled
you can connect to it with any simple
web browser and you don't need to do
anything you don't need to link against
the special framework or library just
when the app looks slow you just connect
to it and find out what's going on right
now what is an event an event has
basically a starting point and an end
point so within the framework we do most
of those strategic places database
fetching component rendering we already
have the start point there then we do
our own stuff and we have this stop
point there and those events can be
nested let's say we have a page and the
page is being rendered so the page of
subcomponents so we have the start of
cage and then well start of component
one and then the component is rendering
itself and maybe needs to do a database
access so we have the database access in
there
boy I went logging is very very
efficient there is again no special
framework or library that you would need
to link against all event logging is
programmed in line so we don't do any
objective-c posing or anything like that
and all the analysis that you will see
in the demo is done at analysis time so
while we collecting the events it's
pretty efficient
and we have memory overflow protections
so that's when you for instance forget
it turned on its religious Ronald's out
of memory and kill your server that
actually is serving customers he had you
might be thinking well performance and
really efficient that what what word is
that on a 3 and Omega host g3 we can log
300,000 events per second sustained so
that's something
and again sustain that can go on for
hours and we won't die on that there's
no disk i/o involved I know that it's
pretty these days disk is very slow
compared to what it used to be relative
terms and also the way it's done it's
pure C methods we have some special
handling how to do it straight in see
instead of Objective C and Static Java
methods for maximum performance some
more numbers we have a custom memory
manager and garbage collector and so
let's say what's the worst thing that
can happen to you you turn on the event
logging and you just forget it turned on
and so the worst thing that can happen
to you is that you'll waste 4 megabytes
of memory per execution thread and our
own memory manager will recycle this
memory as it as time goes by so the most
the oldest events will be trashed and
the newest ones will be used and all of
those levels can be said so if you don't
want to waste 4 megabytes just on 1 or
whatever you can set that in per default
robustness as we all know nobody's
perfect and our applications do tend to
do bad things raise exceptions and we
handle exceptions gracefully for
instance you start an event and it's
nested nicely deeply and within the
database execution you hit exception
well what's going to happen the
exception handler is way way below and
all those nicely opened events they will
not be closed but the exception handling
mechanism will still catch it and if the
system finds out that for some strange
reason something's going wrong the
memories going
Way or something it would just turn
itself off and that's on by default you
would really have to tell it not to turn
itself off and continue logging the
system is very complete we have
strategic places where events are
built-in there's literally doesn't tell
them I didn't really want to list them
all but we have 4e adapter regardless
which adapter you have editing context
different component events again many
and for all pages and photo bindings
we'll see that in a demo later on the
user interface for this tool is accessed
through to direct action webpages
through any browsers that's the will
event setup page that is used to set up
the event logging and the even display
which is actually used for the data
analysis somebody cell phone is working
I had no signal here but events are
grouped into groups that kind of make
sense say the database context event is
a group of actually two events objects
with a specification and Save Changes it
doesn't really make too much sense to
log one without the other so they're
grouped together and this way we have
about a dozen of groups with a total of
you will see the setup page again like
three to five events per group the event
setup page again I'll go more in detail
into that in the demo is used to turn on
or off blogging you can also use it to
reset logging to clean the cache and it
will show a dynamic page of all events
that are relevant for application for
instance when you are using it Oracle
adapter it will not show the Sybase
event it will show only the Oracle event
and so on so if you have custom classes
and you have you even maybe your custom
events those will be listed and if you
don't they won't show up so it's all
dynamic the event display page is again
accessed writable
even displayed direct action and it
allows you to view the data to collect
the data in five different ways
depending on what particular thing
you're looking for in the demo I will
have talked about three of those most
useful ones so this is a more detailed
example of what we might encounter in a
real application execution this is the
same slide as before and say you have a
main page and it does something and then
within itself it builds a dynamic nested
component and that component
conditionally executes the database
fetch maybe it might not need it but
maybe it might compared to that we have
another page it's the same page but at
that point the conditional did not fire
and so there is no database access
needed so again for comparison we have
those two and all of the events logging
so too as to make it easier for you to
read the data on-screen is aggregated
when you have here this is the way it's
going to be displayed on the browser we
see that the main page could execute a
total of two times the method component
itself got also executed two times but
the database fetch was executed only
once and so we see that for instance
that total time spent in the main page
was 70 milliseconds over two instances
and the nested component it was 60
milliseconds in those two instances and
the database fetch lasted for 50
milliseconds at one time so what can we
read out of this we see that the main
page that for 10 milliseconds something
different than dealing with it's nested
component and that divided into event so
5 milliseconds each time and of course
the lion share of for whole page goes to
the database access which in one same
single axis we used up pretty much all
of the time
so this is one way to interpret the data
also you need to understand it the times
you're looking for what's not there
so like again between the main page and
the mystical
then there's ten milliseconds somewhere
and since you have the source code for
you can exactly see which part it is
there will be there will discuss three
different ways to display the data the
most the simplest one for when you're
just starting to performance enhance
your application will be view by page
and then by component you have a page
that has multiple embedded components
and so you'll see the most expensive
page on top then you will see that you
open up the link and there will be more
events in there sorted by component and
you can drill down into those components
into database access and whatnot there's
another view view by a it's very similar
to the one view by page and then
component except it will show all the
components flattened so in a really
sophisticated page you might find
yourself looking at a really long list
and it might be a little bit harder to
find out which component what database
access happened but it's a more complete
information and you can still drill down
into those supplements and finally the
unsorted view is also aggregated so the
same aggregation that I outlined
initially would take place but it shows
you everything flattened out throughout
your application so you see really the
worst thing right on top and the second
worst thing and I think this would be
the demo so yeah excellent what we have
here is a the think movie is very
popular with us on demos I'm sure you
see it before that's what we ship its in
as an example purely written in Java and
I wish if the source for that so you
know we have the home page with the
search page master index and what I'm
doing now I just you know click around
let's search for s so made clear a stand
by me and we have the simple I'm not
really going to talk about yet what they
want to see it's like let's say it's
slow and so we want to turn on event
logging so we go to the whoa you
and so age I hope I'm not the only one
who thinks that the keyboard is not the
best on the planet so here we have the
list of relevant events for this
application we have the database context
event editing context event application
event Association event that's the same
as like a wool binding it's probably
better name for that component and
component reference event so I'm just
going to say select all and well let's
go back to the application and I'm going
to play around
I just boys like to play and Star Wars
okay master index let's just click
around make the application do something
and let's go here scroll around a little
bit this will not be the most amazing
event lock again because we're not
showing we're not really exercising the
app I'll show you a page star Wahb with
lots of really cute geeky pictures this
is the web objects builder team as it
used to be back then one and here we
also have the EU F team look that's a
really special M on both teams here and
so let's not leave the walls came out
all right look at this this is a very
beautiful picture okay and now let's see
how the application did well Eve and
this play I proudly type with two
fingers well this one is a little bit
too big
does anybody know how to make the phone
smaller okay well I have a button on to
love it okay
it looks awkward but we will go first to
the show by page then component view
mode and here we see that all the pages
that our application has touched during
this brief exercise we log a total of
thirteen hundred seventy four events 91
events of that was an EF and 183 in web
objects so we see again I said we can
log 3 in 2000 events per second and we
see that there's not that many events
here so you really don't get any
performance hit and when you leave it
turned off you don't get any performance
hit whatsoever so let's see what what's
happening in the display video page we
see that it has received 643 calls and
it took a turn 33 milliseconds in total
while the display set page got about as
many calls but to go only 42
milliseconds so we know already worse
the problem is in this page alright and
now it's sorted by component this is a
pretty lame example because this page is
only one component itself so let's
assume there would be no more component
and here we see oh pretty much all of
the time 814 milliseconds out of the 833
were used in pulling the data for the
actor image so well it was kind of to be
expected because these are data blocks
but it shows you here and I can again
drill down and I see that actually of
all that time most of it was in your
binding most of that was actually in the
objects which write specification call
within EOF so you might want to look
maybe your page specification is not the
most optimal one etc but if the binding
still took some other time almost 100
milliseconds and you might want to look
into that the other display mode that I
mentioned is
the one group by page only and this will
be only very minor change you see there
is no component in their display video
page but if it were a real application
with more components it would actually
be nice to have them sorted by component
and otherwise it can still drill down
and you will see the axis which
objective sight spec and the last one
that I mentioned in the slides before
would be the one aggregated
hierarchically events which is just
without sorting without any sort of
hierarchy and we see immediately that
the data pool from the database is
listed topmost
didn't need to even drill down anywhere
however as you see the list is getting
longer and longer and if we had more
events it would be a little bit harder
to read and as I mentioned before with
the aggregation of events when we have a
similar call graph that we actually
coalesce that all that you see there's
also the raw mode that I usually don't
want to click on because it would list
this one five and twenty four times on
the webpage so this tool actually does a
lot of work for you just by looking for
the identical call graph and putting it
together so so much for the demo can we
switch back to slides
now what can you do what if you need to
customize it what we've shown here this
is the stock your event will event
logging and it does already quite some
stuff for you but if you need to
customize it the events are very very
general they're based to basic super
classes the will event and the EEO even
depending what you want to measure it
just came sub class that our events are
pretty much everywhere where it makes
sense as you can see and now let's say
that you want to measure something very
very specific let's say that it was no
database search that took so long but
for some really strange reason you're in
it code is taking so much time and in
your init code you're doing only three
four things and you want to find out
what's really so bad so what you can do
is this is a generic piece of code or
okay awake not in it you want to log
your own stuff so what you do there's a
global variable that's being set because
we set again very efficient so do just a
boolean task
if your event logging is enabled then
only in that case will incur that little
bit of overhead we'll create a new event
which is a static method new event of
class just a normal event class and you
can pass any string as a comment to it
as you saw in the table before in the
demo there's always like comments that
you can display and then you mark the
start of the event we also have atomic
events that just say like hello I was
here and they don't have any duration
but we will not really go into those
then you have your own code there that
you want to profile and then finally
when you're done if event is not equal
null or nail in my favorite language
then you mark the end of event and the
system will automatically logic it's
possible that within your profile code
here you would have some other start and
end event in there
and again if you raise an exception our
own exception garbage collector in a
sense will take care of that an
alternative that's a this was the cheap
solution now let's go a little bit into
the more sophisticated one you can
provide your own event classes say you
want to log some really specific things
about in which context or you call you
want not just like I was cold and I took
fifteen milliseconds but more I was
called from this component or I was
called from the other component and you
need to store some data in your instance
variables so you just create an empty
class and it's good enough and it will
be automatically added to the runtime
and the system now knows that it exists
and then you just call it oh there's a
description file that's a file my
component event dot description and
there's only one mandatory field that's
the e event group name which is my
whatever event and then do this do that
I will go back to this screen we see
here your event center mark start of
event event awake now that's the key
that you would be mad that will be the
do this or do that and that's just for
you as a programmer a really short
description and what's going to be shown
to the user in the whole table view of
Steve and display will be the long or
both description like operation number
one or another operation to relock so
this is the basic key value mapping only
the group name is mandatory when you
don't supply the other fields it will
just take whatever you pass in as a
string and you place it in the project
resource suitcase and it will
automatically be taken care of I think
that's about it for event logging so
I'll give back to Eric probably he knows
how to operate this the button that you
showed me did not work - right
so vent logging is really neat and I'm
looking forward to upgrading some of our
applications from 402 four or five just
get that feature and help us get into
tuning our applications at that high
level of detail and one of the things
that you that Stan and I both commented
on is that you need to do this under
stress sitting there and having one
person hit the application and logging
these events isn't enough it doesn't
give you enough detail doesn't tell you
what's happening when your application
is running so the next topic we're going
to talk about is a tool that's built
into web objects refer to it as a
playback and recording recording and
playback it allows you to generate load
on your web objects applications and
then the numbers that you've seen in
well stats and the event logging are
going to turn into something that you
can interpret and it'll be useful in
helping you forecast things like how
many instances you're going to need how
much hardware you're going to need and
where to go from there on your
deployment a couple of things I wanted
to go back and touch upon when you're
doing this kind of testing and you saw
that this measures using the real time
you want to make sure that what the
system that you're testing on doesn't
have a lot of other things running on it
if you have other servers running you
have other processes running they're
going to take time on the CPU and take
up resources that are going to skew
these results okay so playback on
recording as I mentioned is a load
testing feature built into web objects
I'm going to talk in a minute about what
it's what it's as I mentioned it's load
testing they'll talk in a minute about
what not to use it for because it's
intended for some very specific things
and I've heard and seen people misuse it
basically what it does is record
sessions you use a browser you click
through your application get the
coverage of the pages that you want and
it records that session
saves out the raw HTTP requests and
response
to disk and then you can use those as a
later time to play them back and
simulate traffic so I emphasized load
right that playback is intended for
prototyping load and it's once you do
this you get a good feel for how your
application is performing when it's in a
deployment environment one client in one
server doesn't really cut it so once you
start using the playback tool or some
other tool then you can access your
application with the browser and see how
it reacts and with one instance on your
development box then you can take this
on to you to your QA or staging
environment and see how it behaves there
on the class of hardware that you're
going to deploy on and get a feel for
how it's going to work there so I'm
saying it's free well yeah it's free if
you have the product there are other
tools I'll mention in a few minutes that
you could use that aren't free or that
aren't bundled with little objects that
do the same kind of thing and do other
things as well but playback manager is a
great start like whoa stats is for
getting the feel for our applications
performing the last point I want to make
is that we've used it very successfully
in Internet services and the Apple Store
is an example of a high volume internet
site that we use the playback tool over
and over fine tuner application continue
to reuse it and ended up with a very
stable application once we deployed it
so it's not a coverage tool it's not for
functional testing it's not scriptable
once you record that session that
session is going to get played back and
you can't change that you'd have to
record many sessions to get coverage in
this depending upon your application can
be very complicated
so that's why I emphasize load before
because if you're going to use it to try
to get to coverage you're going to want
to use some other tool or create a
custom script another limitation is that
it can't do very good validation on the
results of your pages so if you're
sending back dynamic data coming from
your database you're going to have some
challenges making it validate that
that's
okay all it really does is compare page
links and says you have this page is the
same as this one and actually continue
on to the next HTTP requests in the
session and it it can get tricky to
manage a high volume of virtual clients
with the tool if you want to simulate
tens of thousands of users it's not
going to scale up to your needs very
easily the playback manager can do this
it takes it'll take some of your time
and investment to get into that and set
that up it also take some hardware to do
this you're going to need more machines
to simulate that kind of traffic so the
first thing you have to do is make a
recording so here's an example you just
start think movies the application of
stand use earlier with this argument and
give it a path to tell it where to
record those sessions where it's going
to save those requests and responses
start your application launch your
browser click your application and when
you're done
stop your application that's all you
need to do in the session will be
several files will be stored in a
directory they'll be a directory created
inside slash temp like for example sync
movies dot RTC recording and there will
be all the HTTP requests and responses
in there as I mentioned there's a couple
of options for playback there's a
command-line interface which I'll show
you a screenshot of and there's playback
manager which allows you to launch some
java remote demons that can it can fan
out the load across to all those and
simulate more traffic in a lot of cases
the command line tool is going to do
just fine to do some of the things to
use event logging and well stats to
generate hundreds or thousands of
transactions just with one or two
clients just open a couple of terminal
windows get those going and you'll be
able to discover a lot of the things
that slow your application down so let's
take a look at the screen shot of that
command-line interface so here I took a
screen shot of running this
recording I did on hello world hello
world's a very simple application and
puts up a form and it you type in a text
and it sends the next request and
displays a page that says hello whatever
name you typed in now another important
thing about playback to mention when you
record this script it's recording the
rise to DP so if you entered in in my
case Eric for the hello world example
every time it did that it would send
back the same text so another limitation
there is it's not good for dynamic data
there's some other tools that can take
data and insert those into the forms in
your web objects application and
simulate more variety in your in
coverage in your form posting I wanted
to mention another thing on here you see
a number on the far left which is the
total number of requests that the tool
has sent there's a the time it took to
receive that response back and that's
the total amount of time measured from
the clients perspective now earlier we
saw low stats and we looked at event
logging and we got numbers that were
measured lower down in the stack right
so in this case we're looking at seven
one thousandth of a second for request
number one if we look at we did the same
measurement on well stats we should see
a smaller number because there is IO
going on the rise to DPS any being sent
back and forth from the browser that's
going to take at least some time so you
can use these numbers to make sure that
the numbers you're seeing on the server
side makes sense and then that overall
picture that I showed at the beginning
you're taking timings at the very top
inside web objects as soon as the
requests get in gets in and is being
sent back and then you use event logging
to go deeper in and analyze the smaller
code fragments that you've written in
the middle you'll see average which is a
moving average and this is a good number
to match up with one of those numbers
have begun top of low stats which tells
you what your overall transaction time
is and then it shows you bytes and I set
the different I set a parameter
you'll see on the top line to be very
large basically like I said it does
matching based on the page lengths and
if your page lengths that it gets back
don't match then it considers that they
failed transaction and you'll see that
all these passed and you can have it
ignore failures and keep going or you
can have it stopped when it when it gets
to a page that doesn't match if your
recording goes through your application
doesn't get into a lot of dynamic data
you don't have pages that are grow based
on how much data is in your database you
can set that percentage down a lot lower
and get more meaningful results and
actually use it for some validation but
that leads us into what other options
are there there are some third-party
tools we've used a few that will do more
for you than playback can I know of a
couple and another some others out there
there's mercury has one called lode
runner runs on NT RSW has one called the
test suite which runs on NT as well and
used a little bit of each of those to do
some testing and they're a lot better
for coverage and that you can script so
you can ask your you can write a script
with one of these tools let's say for
example an eye cards application on our
website you can have it go in and try to
send every one of those cards and we can
get better coverage to make sure all the
images load it's not doing what playback
and recording is doing where it's just
sending the HTML it actually fetches all
the images as well so we can make sure
that all of our images appear we don't
have broken links etc with a tool like
this and then you can write a script to
do the coverage so go touch every
category and I cards go try every card
make sure all the images load etc etc
this obviously will help you with
regression testing this your application
is not changing too much or you've made
a change that should not affect the user
interface you could use a tool like this
to make sure that you have no new bugs
and they're more flexible in the way
that they do validation so if you have a
string that appears on a page that's an
error server error error page not found
something like that
you can ask these tools hey if you ever
see this string report an error I want
to know about it and you can have it if
you have dynamic data and your page gets
much longer you may have some text at
the bottom of that page that signifies
that this was this was a valid retrieval
of data this page is fine you can say I
don't care whatever else is on the page
if you see this string it's a it's a
valid response and we should continue on
with the test the other thing that you
can do with these once you've set them
up is put a box just outside your
deployment environment outside your
firewall and have it just sit there and
execute a script every hour - or maybe
every couple minutes make sure that your
application is responding and doing the
things that you expect it to do which
you know turns it from a testing and
development tool into a deployment tool
where you can check for response times
make sure your application isn't down
and it's just you know unlimited with
the scripting capabilities you could add
to to do some checking of your site
so I want to wrap up with other factors
of influence performance and talk about
these a little bit we're talking back
them again I showed the architecture
diagram we were focused in on the green
box what you can do in web objects and
as I said before when you do a test like
this make sure that you're testing
something similar to what you're going
to be deploying with ultimately if you
can test in your deployment environment
that's the best thing you can do if you
have the same class of hardware that
you're going to use in your deployment
environment use that if you're going to
be if your database is going to have a
load on it from other sources not just
web objects seeming like that if you can
the more realistic you can make this
environment for testing the better
results you're going to get from event
logging and low statistics and the big
picture you know hardware the CPU the
CPU that you use the amount of RAM makes
a big difference and there's tools on
all the platforms we deploy on you can
use to monitor how busy is the CPU am i
utilizing this system very efficiently
is it maxed out as a CPU bound or is the
is a memory swapping out to disk check
the you know in Solaris or something as
simple as vmstat you can use to do that
and there's been some other sessions
that talked about database optimizations
advanced DOF you can do a lot of things
there to improve your overall database
performance obvious things turning
indexing on and if you can influence the
schema you know you could you can make
some adjustments there that are going to
affect the overall performance I'm not
going to even go down too far into the
single threaded versus multi-threaded
topic I'll just tell you a little bit of
what we found is that single threaded
apps do just find and the the
maintenance cost is very low you don't
you don't get into more complexity and a
lot of the apps in fact all the apps we
deploy our single threaded and they
perform just fine letting the the
operating system do the switching
raus multiple instances is sometimes
just as good is not worrying about all
the issues that go along with
multi-threaded and having said that you
can obviously run into situations where
you have long-running requests and
multi-threaded apps may be your answer
and as a stand pointed out with the
event logging features you can write
custom events my picture only showed a
database you could have other things
LDAP you may be relying on reading and
writing files a lot other factors write
custom events wrap those bits of code
with with custom events and figure out
you know how those impact your overall
system the more information you have
you'll be able to influence the other
parts of that picture that I drew
earlier maybe your problem is not in web
objects but it's in your database or or
somewhere else and the more data you
have you can go and optimize your code
and once that's done it's somebody
else's job so hopefully we've given you
a better idea for how to measure the
performance of your WebEx application
how to dig deeper get some numbers I'll
invite Stan back up here and we can take
your questions
[Applause]
you