Musings on Change Management Metrics - Part 1
The total number of changes has gone down - our Change Management process is a success ... Is it?
A while back, I was assisting an organization to implement ITIL based Change Management process. Apart from (the usual?) complexities of implementation, something that struck me was the trigger for this implementation. "There are far too many changes within our organization - we need to reduce the total number of changes".
So, why am I writing about this? Well, for one, I am still quite amazed by the number of organizations that use total number of Changes as a primary measure of how successful their Change Management program is.
Does it make sense? No? So, is "total number of changes" a wasted metric? Not really. Arguably, it is a relatively easy and very visible metric to measure. But is it sufficient? Does it give a sense of what impact these changes have had on services? Does it indicate whether testing happened rigorously? Or, for that matter, does it even indicate whether basic adherence to process happened or not - such as having a back-out plan, impact analysis carried out, etc. Obviously it's not a stand-alone metric.
So then, what's a better measure? Hmm ... we'll get to that in a bit.
V3 on Metrics
So, what does ITIL V3 have to say about metrics? And specifically what kind of metrics does it recommend for Change Management?
Moving beyond the traditional V2 approach which outlined a bunch of Critical Success Factors (CSFs) and Key Performance Indicators (KPIs), V3 offers a simple but rather interesting way of categorizing metrics. As an example, KPIs for Change Management, described in the Service Transition lifecycle, include:
- Output Measures - Metrics that measure the outcome of the process or answer the question - has it made a difference? Examples include reduction in the disruptions caused by unsuccessful changes.
- Workload Measures - Metrics that measure how many changes are being implemented in the environment or the volumes. And this is where our much-maligned but ever-popular metric of total number of changes fits in.
- Process Measures - Metrics that measure in-flight performance, including perceptions and satisfaction levels with the overall process including speed and ease of use.
So, what metrics should you be measuring? All of these?!
In my next blog, I would like to share with you some practices I have found useful in selecting which Change Management metrics to measure.
Oh and by the way, in case you are curious about what happened to the organization where we started out with reducing the total number of changes: We did manage to reduce both the number of changes as well as the business impact of those changes. But was there a direct co-relation between these two? Possible, but highly debatable!
Comments
Number of 'failed changes' could be one of the metrics to measure the effectiveness & efficiency of change management. Ideally there shouldn't be any Failed Changes in the system
Posted by: Roopa Nataraj | August 9, 2007 11:21 AM
Yes. And failed changes along with emergency changes is a quick and fairly good indicator of the health of an organization's IT Change Management process.
Posted by: Arvind Raman | October 26, 2007 10:55 AM
There should be a control mechanism to avoid abuse of "Emergency Change" classification.
Emergency changes as such bypass the CAB srutiny, hence there needs to be a control point somewhere. Maybe the change manager can be given necessary authority to accept/reject an Emergency Change.
It should be a constant drive to reduce the number of emergency changes.
Posted by: Roopa | November 2, 2007 12:43 PM
Interesting point you raise there Roopa. There has long been a debate around who "drives" an Emergency Change - is it the Change Owner (same role as the one who drives a normal change) or the Change Manager. Perhaps a topic for another blog in itself.
Pros and cons for both. But as you rightly point out, the organization's intent must be to reduce the number of Emergency Changes. Ensuring a Change Manager's role in driving such changes usually helps to progress towards that.
Posted by: Arvind Raman | November 4, 2007 10:38 AM
The challenge is in the real-time situation, when faced with a Severity 1 issue that needs a system reboot; do the technical teams run around the "administrative mill" (unfortunately a common perception about processes that the technical teams fancy!) or on the pretext of immediate business impact go ahead with a system reboot?!
An organisation that has an up-to-date Configuration Management DB with dependencies and risks identified for all CI's is better equipped in dealing with such uncertainties as above.
In the three-some triangle of People-Process-Technology, "People" need to play a pivotal role in leveraging the best out of Process and Technology.
Or is it that the best in class technology and processes be deployed to reduce errors due to Manual intervention?!
:-)
Posted by: Roopa | November 6, 2007 08:18 PM