After you finish the usability test portion of your analysis of project implementation, you want to test the interface with people who will actually be using it.
How to conduct a user acceptance test
The purpose of the user acceptance test (UAT) is to show adherence to the project objectives, not to find bugs or software defects. It’s the final phase of testing, where users submit the software to real-world scenarios to verify it meets their needs.
Follow these steps to perform the user acceptance test:
Get an agreement with the customer/user on the test parameters.
Make sure you and the user are on the same page regarding the following items:
Set of acceptance test cases and pass criteria: As the business analyst, you typically assist by providing test cases and test scenarios to the people performing the UAT.
Test environment: You may want to test off-site or at a remote location if that environment is the real world in which the users operate. For example, remote salespeople are often in their cars or at Wi-Fi hotspots, so testing a solution for them may need to occur in one of those locations.
Test procedures: Confirm how users are going to test through the conditions.
Turn over any information about known defects in the system.
This step is beneficial for two reasons. One, your being upfront and honest is better than the users finding out on their own. Two, you can have a mitigation plan and explain to users when they’ll have a fix for the defects.
Let the users test the system according to the plan.
Have users sign off on the system, thus accepting it.
Getting users to sign off on the user acceptance test not only indicates acceptance; it’s also a major project milestone.
How to conduct a post-implementation user assessment
Although you as the business analyst should accept feedback about a system any time it’s given, you should also perform an official post-implementation assessment after the users have gotten training on and used the product for some time — usually about 3 to 6 months after implementation.
The real test of whether a system successfully meets its objective is whether the users are still using it several months after implementation. If so, the solution was successful and is meeting the need. If not, the process broke down somewhere, or the solution otherwise failed to meet that need.
Here’s how to perform a post-implementation user assessment:
After the initial 3-to-6-month usage period, schedule a session with the business users to conduct a post-implementation assessment.
During the session, elicit feedback either through observation or a facilitated workshop.
Get the users talking. Pay particular attention to their superlatives and get them to explain those statements. If they say the system is useless, drill down into why it’s useless. If they tell you it’s always crashing, ask them what actions they perform right before it crashes. After all, you want to understand their concerns because they can turn those into future requirements!
Note: Some of your observations may be the metrics (measurements of time, effort, or quality) surrounding a process. Make sure you’re familiar with the expectation and come prepared with a stopwatch!
Record your observations and users’ answers to your questions.
Provide the participants with your documentation and have them validate your information.
Make final changes based on users’ feedback.
If necessary, make recommendations for the next steps, which may be a new project, a maintenance fix, or a change in the process.
The post-implementation assessment isn’t a complaint session (“the system is horrible”) or a lessons-learned session (which concentrates on how you can improve the project processes). It’s designed to be a report-back of the product and whether/how well the product meets the business needs.