Is testing for girly-CIOs?
Interesting discussion caused by an article in the WSJ about the decision of ASU's chief technology officer to forgo full testing in an ERP implementation. How did he do that? Well, he just took a page book out of all the Web 2.0 startups and released the software in beta (That's a polite way of saying the product/service is not fully finished/tested.).
So who tests the application? Well, the poor users of course. And this is what has caused a firestorm. There have been cases of employees not getting paid, or in some cases overpaid. The CTO's justification for not fully testing? He's apparently managed to save over half the original $70m estimated price tag and reduced the time taken to bring the application to its users. Very impressive. But is this a strategy that other CIOs should follow?
I think not. While releasing a photo sharing program or email software such as Gmail in Beta can be justfied, doing this for software that going to matierally impact people lives can not.
- How would you feel if you tried to book a ticket from Phoenix to Chicago but ended up getting a ticket from Phoenix to San Diego? Right.
- What if you tried to order a shipment of 1000 pens for your company using your procurement system, but instead got a shipment of 1m pens? Oops.
- What if your mortgage company's loan origination system mishandled your closing date so that you couldn't close on your home? Hmm.
What's really shocking is that one would skimp on testing. Of all the items that one can offshore, testing and documentation of software are probably the items that are the least risky to offshore. Why for say $2m, the CIO could have got a team of say 65-70 testers testing the crap out of the system for say 6 months. Surely a wise investment given the $30m price tag.
What do you think? When is not testing software justified?
Comments
I do think it is a trend setter. May be it is too risky for financial systems, but for other systems, like knowledge management, its going to be one of the software development models.
Posted by: Vijayakumar | September 28, 2007 09:36 AM
>>>While releasing a photo sharing program or email software such as Gmail in Beta can be justfied
I disagree. Every software however small or big, however trivial or complex has users and stakeholders. I could be one of the users of a photosharing program and my life might be shattered by a wrong photo sent to me by a prospective lifepartner. So is true with a gmail program. If the program does not protect the privacy and other important security aspects around the mail commnunication - some one's life or business may be in trouble - who knows ....
The bottomline is stakeholders (who get affected by action/inaction of a product or service) decide what should be tested, how deep and so on. There cannot be general rules like ERP apps are important than other application that gmail when it comes to testing ...
BTW, you might be knowing about the popular notion in the ERP world -- they are packaged software well tested by ERP vendors. Minimum testing should be OK ...
Posted by: Shrini Kulkarni | February 28, 2008 01:42 PM
I dont think CIOs should follow the strategy of cutting the cost and reduce the application go live time by not testing the application.
"Save over half the original $70m estimated price tag and reduce the time taken to bring the application to its users" but they should realize the cost and time of fixing the production issues and the maintenance cost which increases after the application is live.
Other important thing, once users start finding issues (which could have identified by testing) they tend to stop using the application or sites etc, instead of reporting it. We not only loose business from users but also the investment made on the application is worth.
There are cases where employees lost their jobs because of using wrong reports (which are not tested) for decision making.
Testing is very important in any project and the need of independent test team is increasing day by day.
Posted by: Mahesh Gudipati | March 21, 2008 10:44 AM