Test for Success
You've worked long and hard on your solution. Software and hardware are in place, and bug testing is complete. Next you have to face the users. User Acceptance Testing (UAT) is the means to validate the project with the end users. Real world use cases and user inputs and responses for specific test cases are examined. Business goals and pure functional requirements present one picture, but the day-to-day use of the solution may present a different scene. Obviously user feedback and approval is key. So, how do we test for success?
Tips for Acceptance
So, here are some tips to help gain user buy-in and satisfaction with the end results of the project:
1. Frame the Test on User's Goals
Projects are often defined by both business goals and by functional requirements. While these goals and requirements cannot be ignored and the project be successful, they are not the only factors. Users have goals and requirements that align and tie in with these. A solution that meets the business goals on paper but that the users hate and won't use, isn't much of a solution. Construct the tests to determine whether or not the user's goals are or are not met. Such a test may be identical with a business goal, or it may be an independent as well as supplemental requirement for success.
2. Is the Process Change an Improvement?
Did this process ever work? Did it exist before? Does it take less time now or more? Is it really better this way?
A straightforward set of questions such as: "Is this better/faster than the old method (If there was an old method)?" and "Does this make the job easier?" can go a long way. If not, use a separate section for feedback on what improvements are suggested. Don't go into testing for percentage improvements or partial success, though. Your test case should be clear.
3. Is It What They Asked for?
I've seen far too many times a user look at the results of development an say "Yes, this works, but it isn't what I need to do my job. I can just do 'x' instead." Or, "This isn't as secure." Or even "That's not what I expected." You may think that it is a bit late to be asking such questions, but with questions about improvements, asking point blank, "Is this what you expected?" can make sense. A solid Work Breakdown Structure (WBS) should have solved this potential problem.
4. Separate Test Case Validation from Feedback
Use Yes/No questions for verification of functionality. Provide a separate section for feedback & suggestions. Don't ask open ended questions. This bears repeating, and it may seem obvious. On one hand we want to find out more than whether it just works, as we've said. But to work effectively, we need objective answers quickly, and yes or no questions are the way to accomplish that. User Acceptance Testing needs to have yes or no, pass or fail (boolean) test cases defined.
As you can see, you need three sets of questions: first, does this feature or function work? Next, does it do what is intended? Then, is it what was asked for and expected? Note that these are all yes/no questions. A separate section for comments, questions, and feedback separate from the core of the test works best. With these three question types it's easy to see if you've met project goals and deliverables along with gaining the acceptance of the users. Bear in mind that you can't please all the people all the time. Some may never be satisfied or have different expectations. It's simple to define a percentage from the positive and negative responses to give a picture of opinions to stakeholders.