5-1 Evaluate Smart Device and Web App

Once integration testing is completed, your team needs to evaluate your smart device and web app by testing it with stakeholders to gather feedback on what is working well and what could be improved.

The evaluation will focus on testing the functionality, usability, and user experience of your solution. This evaluation can provide evidence of how well your solution meets your design requirements.

YOUR TASK

1. Identify one or more tasks that people could complete using your smart device and web app. Create a brief scenario describing a realistic context in which someone would need to complete the task(s).

   • For example, a scenario and task might be something like: "Imagine you are a student that needs to get something from your locker before your next class starts. Open up your smart locker, and then lock it again when you're done."

2. Recruit 3-5 participants that match one of your target stakeholder groups, as represented by your personas. Each participant will individually evaluate your team’s solution. Each evaluation will take about 5-10 minutes.

3. At the start of each evaluation, briefly introduce your solution to the participant using your team's value proposition. Be sure to mention that:

   • the product is being tested, not the participant

   • the product consists of a smart device and a web app that work together

   • the product is an early prototype, so you want constructive feedback on what’s working well and what could be improved

4. Describe the scenario to the participant, and identify the task that he or she should complete using the device and web app. (If you have multiple tasks to test, give the participant one task at a time.) Tell the participant to provide feedback by “Thinking Aloud” as he or she interacts with the device and web app.

   • Since the device is an early prototype (which may be very unfamiliar), it may be necessary at times to provide limited help if the participant doesn't understand what to do next.

   • However, as much as possible, let the participant figure out how to interact with the device.

5. As the participant completes the task(s), record notes on the feedback being provided.

6. At the end, have the participant complete this online evaluation survey.

Repeat Steps 3-6 with each participant.