If normal operation for your application is handling a lot of transactions at the same time (few dozen? many thousands?), at least some functional testing ought to get done while that is happening. Not all of it, and maybe even not most of it, it depends on how likely you think it might be that there'd be problems with stuff like resource contention. But at some point you probably need to evaluate the user experience in as realistic a setting as possible, and that includes the system workload.
For me at present, this is mostly covered by means of the "conference party". The product is a conference bridge with both audio and web support, and when we think the functionality in a release is pretty much working, we schedule some sessions where everyone available gets into a conference on a test system, and we do the sorts of things people do in a conference - talk, look at powerpoint presentations, do polls, mute lines, all that stuff. We're a very small company, so even if most employees are in on the conference, it is less than a span's worth of ports. I connect some Hammer spans and have a call load running during the conference party, and everyone is alert for voice quality and response time issues, as well as for anything that doesn't work as expected. It helps us know that everything really is all working together. |