3.1 Solution Evaluation

Your team will test your robot prototype and product website with people outside your team to evaluate whether the robot and website are useful, easy to use, and effective. Receiving constructive feedback will allow you to improve them before the public presentation.

You’ll need to recruit a total of 6-10 participants for this user testing:

• 3-5 participants should individually evaluate the robot

• A different set of 3-5 participants should individually evaluate the website

As a reminder, here are instructions for conducting user testing. For the robot prototype, the user testing will be slightly different because the participant will observe the robot completing tasks (rather than the participant completing tasks themselves).

You’ll also ask each participant to respond to a brief online survey after the user testing session. The surveys for the robot and product website will be slightly different. These two surveys have already been created as Google Forms:

1. Create a new folder called “Solution Evaluations” in your team’s Google Drive folder.

2. Save one copy of each survey in that new folder. (Links to both surveys are below.)

3. When you’re ready to administer a survey to a participant, open the specific survey, and click the preview icon in the upper right, which will allow the participant to respond.

Evaluation of Robot Prototype

The user testing of your robot prototype will require each participant to observe 1-3 task scenarios demonstrated by the robot. You'll ask the participant to "think aloud" as they observe the robot, in order to provide feedback.

Start each testing session by using your value proposition to summarize the robot's purpose. Briefly introduce the first task scenario, and then have the robot demonstrate the task. Be sure to record notes on the participant's feedback. Repeat for the remaining task scenarios.

• Evaluation Findings: Use this template to summarize your findings from the user testing.

• Post-Testing Survey: Have the participant respond to this survey after the user testing.

Evaluation of Product Website

The user testing of your product website will require each participant to find answers to specific content questions. Use these questions as the tasks for the website:

1. What does the product do?

2. Who is the product designed for?

3. What benefits does the product provide?

4. Evaluation Findings: Use this template to summarize your findings from the user testing.

5. Post-Testing Survey: Have the participant respond to this survey after the user testing.

Analyze Survey Data

After all the participant evaluations are completed, you’ll analyze the data from each survey.

Open one of the surveys in your team’s Google Drive folder. At the top of the survey form, click on the "Responses" tab, which should default to showing the summary data for all responses.

For each survey question, the summary data will display a histogram (similar to bar chart) of the response distribution:

• Responses categorized as 4 or 5 are positive because they indicate the participant either agreed or strongly agreed with the statement.

• Responses categorized as 3 are neutral because they indicate the participant neither agreed or disagreed with the statement.

• Responses categorized as 1 or 2 are negative because they indicate the participant either disagreed or strongly disagreed with the statement.

After each chart, you’ll see the written responses submitted by the participants to explain their ratings. You can compare these explanations against your team’s findings from the user testing to see if the survey responses revealed any new issues that might need to be addressed. If so, add these new issues to your evaluation findings document.

Create a summary document for each survey that shows the histogram charts (in order) of your team’s survey data. If you hover over the upper right corner of a chart, you can click a copy icon, and then paste that chart into the document.

In the next assignment, you’ll use the evaluation results to make refinements to your robot prototype and product website before the public presentation event.

❏ Deliverables

Submit all of the following:

• Evaluation findings for robot prototype

• Evaluation findings for product website

• Summary of survey data for robot prototype

• Summary of survey data for product website