The IT performance failure of the DVLA’s online road taxing site has hit the headlines, and thousands of customers have been unable to renew their car tax online, with some spending 13 hours online trying to get their car tax renewed.
The DVLA said the site had seen “an unprecedented volume of traffic”, and sent a series of tweets trying to explain the situation. One of these tweets caught my eye and I thought I would do a little digging into their problem…
One DVLA tweet says it all…
According to the tweet they managed to tax 250k vehicles ‘tonight’. The tweet was sent at 9:43 pm on 30 September, assuming ‘tonight’ means after sunset (approximately 6:40 pm in London), this means ‘tonight’ was a period of around 3 hours.
So this tweet tells us they managed to tax 250k vehicles in 3 hours, or 1389 a minute. The tweet also tells us they were experiencing a demand of 6000 a minute.
So how many requests should they have expected for IT Performance?
According to government figures there were 35.0 million vehicles licensed for use on the roads in Great Britain at the end of 2013, of which 29.1 million were cars
According to my Googling, it appears to be generally accepted that 1/3 of all new cars are registered at the plate change in March and September, that means that 19.4 million cars on the road were registered during the other 10 months – so we can assume 1.94 million cars on the road were registered in October. For IT performance calculation purposes let’s assume that all 1.94 million cars pay road tax in October, and that 25% (a guess) of vehicles paid 6 months duty in April so will also pay in October. That’s 2.425 million cars to pay tax in October.
We already know that the DVLA was only able to cope with a maximum of 250k vehicles during their peak 3 hour period – that’s ~10% of the anticipated tax requests. Is that reasonable for their IT performance?
- A lot of requests would still be done in the old fashioned way at the post office or by post, let’s say 50%
- Of the 50% done online, a lot would not have been left to the last minute, let’s say 70%
- Based on the above assumptions the peak period expectation would be 15% of the anticipated tax requests
So that leaves 363,750 vehicles to tax during the peak period, which is 45% more than 250,000 they could actually scale to. In fact, if each tax request was successful the load would be 2020 requests per minute not the 1389 a minute actually achieved.
So why was the demand 6000 requests per minute?
Of course, any of my assumptions could be wrong, or the actual number of people leaving it to the last minute to use the internet could be three times higher, but the most likely answer is that when presented with a time-out or error, the people wanting to tax their cars tried again and again – after all, if they didn’t get their car taxed they would be breaking the law!
It could have been so much worse…
At least in October there is a relatively small amount of car tax requests. If it had been March, which accounts for 1/5 of all car registrations, the story would have been much different. Using the assumptions outline above, 1.1 million cars would look to register in the peak period, that’s over 6000 a minute. And given the try again and again mentality of the public trying to tax their vehicles, that could equate to a massive 44,439 requests per minute. How would the DVLA IT performance system that can only successfully process 1389 requests per minute cope with 32 times that load?
Where did it all go wrong with their IT performance?
The tweet says it all – they believed their system was built to scale, but why didn’t it?
- Did they test the IT performance? Major system changes were introduced with the death of the tax disc – if these changes were not performance tested they may contain unidentified performance defects which reduced the scalability of the site
- Did they get their volumes wrong? They surely had all the information they needed. They must of known how many vehicles needed taxing each month – but had they anticipated the sudden surge in online use? Were they basing their figures on historical online use? Had they taken into account the media coverage of the death of the tax disc or thought about how this may change people’s habits?
- Was the performance testing inadequate? Even today I am amazed at how many performance tests simply don’t do what they are meant to. And what’s worse, the testers don’t even know! I’ve been called in to organisations to audit test capabilities and I’ve seen tests where 1000 users have all been getting an error page, but the testers have reported success, and where volumes have been misunderstood and tests have either significantly under tested volumes, leading to performance issues when live, or significantly over tested volumes leading to unnecessary hardware upgrades
Let’s keep our fingers crossed that the DVLA sort this problem out, and have also considered the peak in March otherwise this fiasco will be repeated month on month.