Project Management Book

...chapter 12 continued

Quality Measurements

Why might we want to measure quality? If we want to find out whether it's cheaper to find errors by simulation or by system testing we'll need to measure the cost effectiveness of each. And everyone already uses some quality measurements - we all count the number of bugs found in testing, don't we?

A lot of quality professionals go around muttering strange statistical terms like "standard deviation", "least squares fit" or "six sigma" which nobody else understands. If one is not careful the whole business of 'quality' becomes something far too complicated and esoteric for most people. (One organisation appointed a quality manager. Everyone expressed support - that was the right thing to be seen to do. He started to collect charts from projects and soon his office wall was full of them. Quality then became a matter of who could produce a chart so pretty that it would make it onto the wall and displace a rival's. They all had copies of the software which could produce lovely zigzag plots bounded by control lines, elegant bell shaped curves - great fun. Nobody had any idea what the charts meant and nobody used them for anything, but they seemed to make the quality manager happy.)

The purpose of quality measurement is not to produce pretty charts and statistically elegant metrics, it is to stimulate and direct quality improvement. We prefer simple raw numbers the average team member can understand that are used by the team to help them deliver better quality outputs.

One could devise hundreds of ways of measuring some aspect or other of quality - we will show just a couple to illustrate the principles.

Imagine your project team have just finished inspecting and simulating their business requirements document. The team report to you the number of errors they found in each of the document's five sections. What would you ask the team?

"Why were there so many errors in the 'change account type' section?" The chart shows there were 16 errors in 3 pages which is 5.3 per page (though they chose to show errors per 10 pages to avoid decimal points). That's four times more errors per page than any other section. You would want to know what that was. And then what would your follow on question be? "What are you going to do about it?" If you can ask those two questions - "why's it like that?", "what are you going to do about it?" - you can manage anything. We have found an error cluster, we can address the problem. The team say: "we did have a problem with that section, the user who wrote it didn't really have the right knowledge - it is currently being revised by a more experienced person and it should not be a running sore throughout the project." The measurements are beginning to enable us to manage quality.

A couple of months later the team show you the results of the user functions design document inspections. What would you be asking about this time?

The data conversion section of the UFD tops the poll so you'd certainly ask about that. Maybe the data transfer requirements, only 3 pages long, had been a bit vague and high level and only now at the design stage are the problems coming to light? Would you ask about anything else?

Too slow, missed your chance. A month or two later you get the results of the IT technical design inspections. What would really be worrying you now?

Change account type again. What are we being told? When the IT team inspected their technical design they found four times more errors in the change account type section than in any other part of the document. We'd want to know why.

If you're lucky the team say: "it was done by a trainee, he made a bit of a hash of it - not really his fault - but it has already been reworked and is now up to scratch." That would be good news. Bad news would be: "boss, the requirements for 'change account type' keep changing - it's chaotic. We need to go back and redo the requirements for 'change account type' and re-inspect them, redo the UFD and re-inspect it then completely redo the technical design for 'change account type' from scratch and re-inspect it."

Will that be painful for the project manager? It will, but a lot less painful than discovering the problem three months later in system test and then having to do all that rework plus re-write the code, delay testing and who knows what else. To be a good project manager it helps if you're a masochist: you are willing and able voluntarily to inflict pain upon yourself - with the aim of avoiding twice the pain being inflicted upon you involuntarily later. (In our example it would seem the team didn't spot the problem at the requirements stage - though perhaps they did but chose to ignore it or they failed to sort it out properly. One hopes they would at least have dealt with it at the technical design stage.)

The quality leader would show the project manager numbers like these each week or month. But he wouldn't just be showing the numbers - he'd be saying something like: "boss, we realised from the numbers there was a bit of a problem in this area and we have put John Spencer on it and he will have it revised and corrected by Friday." In other words the team, prompted by the quality leader, are using the measurements to help them identify and fix any quality weaknesses as the project progresses.

...next

Privacy Policy and Cookies