Lessons learned? A number really
- The scripts are large but need very little correlation. Also parameterising anything other than some basic values will be very difficult. The Subject Matter Expert had built exactly the same reports where the only changes were to some of the dimensions. Each set of dimensions returned the same amount of data, rows and columns in the spread sheets. Parameterising to the extent of changing the number of data values being returned (rather than just the data values being returned) would have been almost impossible as the data definitions are also part of the message. It did mean that the effects of report sizes on the response times could not be measured but it was considered that this trade off was acceptable.
- The iteration execution time means thinking outside the bounds of regular performance testing as an iteration can take hours to execute. It’s quite common for someone, in our environment, to spend all day on what is really one Business process. In future cases it may be worth investigating the implication of dealing with each different report as a separate script and designing the testing around individual report execution frequencies rather than end to end processes.
- The need for production data volumes was really reinforced, this testing would have been much less useful without the full production data volumes.
- The data values need to be parameterised otherwise the server will not perform the updates, it can recognise when the values are the same as they were previously even if a message is being sent . We initially thought that leaving the values the same would still trigger an update as the server would just see a number of values, this wasn’t true. We found we really needed to change most of the values, rather than just one or two, to get the correct update profiles.
- BPC runs well at low levels of free server resources so you need server monitoring to determine the impact on server resources.
- As you would perhaps expect in this type of process, the correct indexes are very important. Perhaps not so obvious is that not implementing all available SAP Notes can have a major impact on response times.
Perhaps an expensive option ( I never saw the figures) but bringing in a SAP VTO team can give big benefits in respect of getting access to SAP internal resources and effective tuning recommendations.