I’m a member of a number of performance testing forums, mainly around LoadRunner. It must be on a monthly basis that I end up nearly screaming at the screen as I read yet another question being given the same answer. There are two things that cause me to shout:
Continue_on_error – the Holy Grail?
Some performance testers, seemingly mainly off shore but that may be my imagination, seem to think that they have found the “Holy Grail”. If the script won’t work then switch continue_on_error on. The script will then magically work! It won’t and it doesn’t!
In my opinion continue_on_error should NEVER be switched on in the run time settings. In fact “never” is probably too mild a word, but it’ll have to do. This option should only ever need using in short bursts, covering a few lines of code at a time. If you’ve switched it on because you’re capturing all the errors yourself then I have to ask the question “why are you capturing them all yourself?” LoadRunner has a perfectly capable facility for capturing these so why reinvent the wheel – you’re just adding complexity and reducing maintainability.
I never have continue_on_error switched on for more than a dozen lines of code. I switch it on when I have a response that I need to capture and I switch it back off immediately after the error that I don’t want LoadRunner to catch. I will add a plea to HP though, if I’ve switched off the LoadRunner error capturing why does LoadRunner insist on still failing the script when I said it was acceptable to get that return code? If I’ve requested that LoadRunner doesn’t check for errors then it shouldn’t say it has found an error.
As an example for HP
I’m currently working in an environment that uses single signon. If that fails then you’re presented with a login page. In our case the first request (single signon logon) fails with an http 401 – this is correct and the script then continues to the user/password logon screen. If I leave it in and catch the 401 then my script is still marked as failing by LoadRunner.
The same people also seem to think that if their page download times out then the answer is to increase the download timeout to 5 or 10 minutes from the default 2 minutes. This may indeed mean that the script now completes successfully but is a web page that takes 4 minutes to load acceptable? In most cases I would doubt it.
LoadRunner doesn’t help itself in this aspect though. If you have extended logging/Data returned by Server switched on then the timeout can be caused by LoadRunner writing its output log to the screen. If this happened then either don’t get the server responses or close the log window. The time is spent in writing the screen rather than recording the log. If you remove the output window then it’ll reopen when your script finishes but the script will run much faster if you’re downloading large pages or pdf’s.