Metrics in IT Projects
Contributor: Dmitriy Vinichenko, PM at Archer Software
Metrics are one of the most natural things in our lives. Just consider some of the questions we’re most often asked, or that we ask ourselves. How old are you? How many people are coming to your party? How many of those people do you actually want to see?
The same is true when it comes to the lifecycle of an IT project—almost everything can be calculated and measured. Once measurements are made, though, you will face the biggest challenge of them all – analyzing your data in such a way that all defined risks are predicted, and all issues reported and resolved. It’s not enough to just gather and share metrics. We need to analyze them as well.
First of all, let’s define what metrics are in the context of IT projects. Metrics measure processes, activities, resources, and deliverables within a broader quality control plan. . More specifically, a metric is a unit that is used to collect data in order to report on the state of a particular IT service. Metrics help us to develop predictability, improve our organization’s decision making ability, and lay out what is working and what is not within a given project. At their best, metrics help management make good decisions.
There are three basic types/levels of metrics:
- quantitative data (“raw” information about a specific variable)
- qualitative indicators (which give us derived and calculated dependencies)
- prognosis trends (which allow us to extrapolate data in order to make the right decisions moving forward).
One type of metric comes from another. At the beginning of gathering metrics, you will only be working with quantitative data. It comes from a team’s hourly reports, QA reports, daily meetings, client’s feedback, experience, etc. Remember – there is no useless information. There is only information which you are not yet able to apply
After collecting all of the information you need - you can start searching for dependences. Some dependencies are obvious, and appear after a cursory examination of the raw data. Other dependencies are harder to see, and exist between actions which we would have thought were totally unconnected. This brings us to a logical next question-how should we show these metrics once they’ve been collected?
Metrics reported to project stakeholders should be displayed in a short, clear way: values, dependencies, prognoses, and improvement actions should all be well-formatted and easy to understand.
There are six main criteria used as metrics in IT projects. They are as follows:
Time: how does the execution of a project stack up against the plan?
Cost: is the cost of developing the project within the allotted budget?
Resources: How much time is being spent on this project?
Scope: Is the project's scope in line with expectations?
Quality: Are quality problems being reviewed and fixed?
Actions: Are there any outstanding action items (things to be done) on the project?
It’s up to you how to share these metrics with all the participants in your project.
Of course there are many more characteristics in a project which can, and often should, be measured. That brings me to my next point: in order to be useful metrics must have the following three characteristics.
Functionality: Metrics must measure exactly what they are supposed measure, otherwise they will become effectively un-interpretable.
Cost Effectiveness: Metrics must deliver more value than the cost of gathering and analyzing them.
Consistency: Metrics should be the same across company operations. The continued analysis of these metrics provides insight into what is, or isn’t, working, thus allowing the project manager (PM) to take appropriate action.
These metrics also help in building a set of historical data to be used in similar projects moving forward. This makes planning a whole lot easier. The success of a given project can then be demonstrated by comparing sets of “before” and “after” data.
This ensures that the effort spent by a team in the collection and measurement of data for these metrics pays off in the form of continuous improvements, rather than just more busywork.