Define to Win!
If there is one defining indicator that shows whether a project manager really gets the concept of business focused ICT, it would have to be how they develop project acceptance criteria. Well, actually, sometimes this comes down to "whether they develop acceptance criteria", unfortunately.
The purpose of acceptance criteria for projects is widely misunderstood and you often see it overlooked or misconstrued. In particular, many project managers and engineers confuse it with user or technical requirements. This article explains how acceptance criteria should be used as top level indicators for the overall project (including the technology or solution), whereas user and technical requirements are typically confined to the solution. The article provides valuable pointers on how to build meaningful and measurable project acceptance criteria and how to use this to drive change management.
The misunderstanding of the purpose of acceptance criteria is ironic. After all, the whole intention of acceptance criteria is that we should kick off the project having a clear understanding between the people driving the need and the people driving the project!
The Entrepreneurial View
If you put yourself in the shoes of an executive or entrepreneur who wants a project done, it’s easier to see that the technology doing its job is just a subset of overall factors that will determine if a project is acceptable.
As the executive, you wouldn't want people complaining that they weren’t trained or that this system makes them stop everything else they were doing. You also want the project to be delivered in a certain timeframe or within a budget. You may want to know that people from other branches of your organization or even external parties have been involved – and let’s face it, sometimes their involvement is more important than what they have to say! These are all items that rarely appear within user or technical requirements.
Relationship to Benefits
An interesting, if slightly esoteric, observation is that the reasons an entrepreneur undertakes a project are often very well reflected in the benefits statements (if the business case has been correctly built).
However, this isn’t to say there’s a one-to-one relationship between acceptance criteria and benefits. There’s another aspect of acceptance you need to think about if you're taking that entrepreneurial view. An entrepreneur would only accept a project if nothing bad comes of it, or at least only if those risks were manageable! At the extremes, they must not end up in jail, or bankrupt because of a law suit. Now being less extreme, they are likely to care about whether their staff members are upset by the manner of development, whether anyone is put in danger and whether the business needs to come to a grinding halt to upload the new system.
Perhaps it's sad that it’s easier to explain what makes a good acceptance criterion by showing what makes a bad one? Here are some horrible examples, with an explanation of why they are so awful.
On time, on budget and to required quality.
Not exactly the thinking person’s version is it? In any case, this is circular as the quality criteria is meant to derive from the acceptance criteria!
(One criterion) Meets user requirements
Well again, it's circular but do the user requirements also cover, for example, training, transition downtime, design processes, legislative requirements and so on?
Too vague – processing of what? How much faster do we need to be?
A minimum million pixel interface on an LED platform
Here’s the solution – now what’s your problem!? Not likely to reflect the entrepreneurial view of the project is it?
The project must allow us to dominate world software markets.
Well besides probably being unrealistic it’s unmeasurable and it won’t be realized until well after the project is finished, so it can’t stop “acceptance."
So What Will Good Acceptance Criteria Look Like?
It’s hard to go past the expression SMART when talking about criteria of any sort. Specific, Measurable, Achievable, Relevant and Time-bound are essential. I would also add “Good Coverage” to the entire set of criteria. That is, they must cover not just the technical solution but the entire project. Here’s an example set that I quite liked (I’ve removed some identifying names):
This project as a whole, including the technical solution, will be deemed a success if:
- The project is delivered within 10% of budget and 1 month of schedule (to be approved at Gate 2).
- Human transaction processing times in the (call center) are reduced by 25% as specified in the business case.
- System stability is improved to Category 1 as specified in the business case.
- All user requirements developed by the group of users shown at point 3 are met.
- User requirements exercises are conducted early and include appropriate representatives from the (call center), Banking Products Branch, Audit and Control Division and ICT Support Branch.
- All systems developed are fully compatible with existing platforms including the proposed data warehouse technology in [project X].
- All users and system support personnel are trained directly (not via “train the trainer” or online courses).
- Transition does not involve major downtime or disruption to business as usual.
- The system meets stability and supportability requirements specified in ABC Technical Development Guidelines.
- The development does not otherwise breach any guidelines laid down by the organization from time to time or any State or Federal Laws (including copyright).
The project manager and project executive both had substantial parts of their salaries (around 30 per cent) attached to achievement of these criteria.
Getting the Executive Commitment
Of course we all know reality is that senior executives are often reluctant to put the effort into this. Sometimes they are even reluctant to commit to something that is this specific. I even had one executive tell me “I’m not signing up to something like that, I have one simple acceptance criterion … If I’m not happy then I’ll fire the project manager." Yep, dinosaurs are alive and walking amongst us but I’m pleased to report they are becoming rarer.
However, even enlightened executives can be busy and you just cannot make this stuff up for them. To get the commitment you usually need to guide them by the nose but be careful of doing their thinking for them. To break down reluctance to have a discussion, you might try asking in the negative – “So would you be happy if we shut down the call center for a week to implement this?" “No, then what would be an acceptable outcome?" (You need to frame the tone of these discussions to the personality you’re dealing with, of course!)
One thing I have noticed is that once an executive has committed once to good acceptance criteria, he or she will insist on them every time and their commitment becomes stronger and stronger with every successful project. The "converted" executive will want to check the compliance with these at every gate and will base a lot of their questioning around these criteria.
How can you possibly know if a project is a success unless there are some clear criteria laid down to measure that success?
References & Resources
All information based on author's personal experience.
Image Credits: sxc.hu/iprole