StarBase Blog


Over the years I’ve developed a lot of LoadRunner scripts and have had to modify a whole lot more that may have once worked but now don’t. The scripts fall into two camps, and there seems to be a relationship between these camps and the scripters’ age and background.

In one camp the scripts are neat, well-structured and simple to maintain while in the other camp the scripts are left much closer to how LoadRunner originally generated them and are usually horrible to maintain.

Take time-to-understand-the-code

Take time to understand the code

The neat scripts tend to come from older testers who have a more varied background (and I have to admit this is my camp – the first computer language I learned was Algol 68; you’ll need to Google that one!).

I suspect that there’s a generational difference coming into play as the older guys probably fell into performance testing from a legacy programming background. The younger ones are more likely to have made a conscious decision that performance testing was what they wanted to do straight from university.

Let me, though, explain what I mean by neat, well structured, etc. and why it is really important…


Boot Camp for Unfit LoadRunner Scripts

The Importance of Neat, Well Structured Code

I don’t know why we call it scripting, its computer programming plain and simple! LoadRunner doesn’t generate very neat code (and make no mistake LoadRunner creates code) and this code has to be adapted and maintained.

LoadRunner has a nasty habit of creating very long code lines that scroll far off the right hand side of the screen, especially on web based scripts. This is the cause of most of the errors that I see and have to fix. The problem with all that code off the right hand side of the screen is that no one ever thinks to look for things over there. It’s so easy to lose your context when you scroll across and lose all the information on the left, you’re left with a single line of code with, seemingly, no beginning and no end so people try not to go there.

“Out of sight, out of mind” is often the fate of everything to the right of the screen size.

There’s a push by many vendors to market their tools as intuitive, simple to use, record and replay, “bringing performance testing to the masses” etc. (maybe I’m getting too cynical in my old age!). It seems to me that there’s a big tendency amongst the younger testing camp to buy into this and not to spend any time understanding the generated code which was so easy to record. They follow the rule – Just keep running it until it gets to the end and seems to have worked, then I’ve finished. There’s no real feeling that it’s necessary to understand the code generated by LoadRunner, if it reaches the end then it must be right, mustn’t it?

Well no! Not understanding all that code can hide problems with unfit scripts and comes back later to bite and the code stops working.

Examples of Code Hiding Problems with Scripts

A real example from a little while ago concerns an unnamed organisation where scripts had been written and testing completed. When the next test cycle came round none, and I mean none, of the scripts would work. The test team spent a couple of unsuccessful and frustrating weeks trying to get them to work.

That’s when StarBase were called in.

The first thing I did was to structure the code.

Why? Well, I just can’t work when I can only see half the code (and I’ve been bitten before by code I couldn’t see)!

I went through every line of code, breaking each URL down with a new line inserted before each parameter value, the parameters nested etc.. It’s boring as hell to do but usually pays dividends and doing it helps me understand exactly what is going on, how it all hangs together etc. and therefore why it isn’t working. It certainly did in this case!

The thing that broke the scripts? – 4 characters in the request, a year value! About the 12th parameter along was the year, I never did discover what purpose it served but the scripts had been written in one year and the current test cycle was in the next year so they didn’t work anymore as that year value hadn’t had parameters set. No one ever scrolled far enough right to even see that it was there, never mind correlate or work out the parameters.

Structuring the function calls and changing the presentation from fat and short into thin but long showed all the parameters and their relationship to one another.

Here’s an example from an application I’m testing at the moment. Hopefully it’ll show you what I mean, I think the second version is easier to follow (and you can see how much is missing off the right hand side of the first snip):

The original Fat and Short!

sid~003D{dyn_sapExtSid2}~E005ClientWidth~E0041643~E005ClientHeight~E004892~E005DocumentDomain~E004 {gen_config_url}~”E005ResponseData~E004delta~E003~E002~E003~E001LoadingPlaceHolder_Load~ E002Id~E004_loadingPlaceholder_~E003~E002ResponseData ~E004delta~E003~E002~E003″,

My restructured Thin but Long!

“ sap-ext-sid={dyn_sapExtSid2}”,
“ Custom_ClientInfos”

This restructuring effort also tends to show me the relationships between the various function calls as I have to read them as I go through and, for example, all the calls using the same referrer becomes much more obvious and lets me get an understanding of the application data flows as I work my way through it, (I tend to notice when I have to split the same data multiple times, common data tends to stand out). To complete the process I also restructure the body section if it’s an XML string or something similar, same reason, it helps me understand it when it’s in a structured manner. The example above shows some of the body and hopefully the second set is easier to read than the first.

When the problem was found it took about 5 minutes per script to fix and several test cycles later they haven’t needed changing again. A shame really, they had a great staff restaurant!

By now you might think that this is about structuring your code and that I’ve got diverted from the article’s title. Well, it’s not, it’s still about the need to spend the time to understand what’s been generated by the code generator as an integral part of your test development. The restructuring is just something that aids in this understanding, it’s a major part but still only one part.

Another Example

Here’s another true story to try and expand what I mean further.

Boot Camp for Unfit Scripts. Five or six years ago now StarBase were called in to take over a project where the tester was leaving the company.

I was assigned to lead the project.

The application was an online auction site and the scripts, as they were recorded, were really long – we’re talking thousands of lines of code. They had lots of identical calls in them that looked like heartbeat – “I’m still alive” calls and when you replayed the script caused errors in replay.

Removing them made the script run from end to end. Assuming (hoping?) they were insignificant and just made the code look complicated and as they all seemed to give the same – empty – response. The test developer had taken them out, leaving only the URL requests that built the pages.

When I tested these scripts everything seemed to work OK and the scripts ran to the end without failure – so far so good. This just didn’t feel right to me though, I just didn’t understand it!

Why had LoadRunner generated all this code if it could then just be cut out again? Why had the developer put it all in there if it wasn’t necessary? It just didn’t make sense to me. So I took some time to have a closer look and to question this as part of my process in understanding the generated code and what I was testing. Talking to the system architect it was mentioned that this site was designed to dynamically update every watching user when a bid was placed. But how was this done? There didn’t seem anything that would help do this in these finished scripts. The time being taken to understand the scripts was beginning to pay dividends. Could all this code that had been cut out be something to do with this? Yep, these calls weren’t “I’m still here” calls they were “has anybody placed a bid?” calls. (The functionality had been implemented via long poll calls that updated the web page if there was a bid or returned an empty response if there was not). Hence if no bids were placed, as was the case during script development as there was only one user on the system, then you just got a really long set of these calls, one every two minutes, until the tester had decided enough was enough and had stopped the recording

The real solution in the scripts was not to cut all these calls out but to build loops for these requests that reflected what was actually happening. (LoadRunner has caught up now and would handle it a lot better). The developer had just made the assumption that these were heartbeats without taking the time to understand what was going on. The code needed to be understood before it could be made to work properly. The tester’s lack of understanding of the code (and his perceived lack of need to understand it) with the decision to take all those calls out it all out had really removed the point of those scripts.Gold_Partner_Insignia_reverse

My conversations with the client also led me to the conclusion that a major emphasis in the testing was to test what happened when “x” users were looking at an auction and a bid was placed. (This was anything but obvious in the testing requirements document). It meant I also now had to look into how I could measure the time taken between placing a bid in one vuser and the bid being reflected in another vuser, (but that’s a different topic, and best left for another day, as is the need to write a good testing requirements document).

Test execution gave the answer, when it got to about 50 users it took too long to update the watchers and was outside the Service Level Agreement (SLA) documented in the End User License Agreement (EULA) for the site. This application functionality had to be rewritten before the site went live. Implementation had to be delayed for a month while this happened but the reputational implications of not fixing this before go-live would have been enormous. They were leaving themselves very open to complaints when someone’s session updated slowly and they missed the opportunity to increase their bid.


StarBase Star Logo

StarBase – specialists in Performance Testing for 20 years

In conclusion, it really is necessary to spend the time, and put up with the boredom, to understand the code generated by any code generator; including LoadRunner and most of the other first generation performance testing tools as part of the boot camp for Unfit scripts. The structuring exercise does it for me but if something else works for you then go that way instead. You’ll even find that you can build better tests, quicker, if you take the time to understand what’s going on (and the architecture of the application). Just take the time to understand things, it may feel like time wasted but it really isn’t, it’ll really pay dividends in your test development and make you a much better tester.

My next task, when I get the time, is to try and do a VuGen add-in that will restructure the code sections for me – but will that be counterproductive as if I don’t have to do it by hand I may not spend the time to understand the code properly? I guess time will tell!


Posted by:admin

Client Testimonials