How might we design learner-centered features aimed at improving student outcomes and engagement in phonics, personalized for various types of struggling readers?
Houghton Mifflin Harcourt is a leading publisher of educational content and technology. We are helping them to research and design new features for System 44, a reading intervention program for the most challenged readers in Grades 3 – 12+ that serves over 600,000 students in U.S.
We invite you to experience our solution. Try Endeavor!
Endeavor serves two purposes:
- engages students with a choose-your-own-adventure style narrative
- provides teachers with finer grained data of students’ performance on challenges embedded within the narrative
Through our user research and literature reviews, we found:
- some students were just going through the motions, they were highly extrinsically motivated, not at all intrinsically motivated
- there was untapped potential in providing data, of student actions on System 44, to their teacher
- UX RESEARCH
Go where the users are
As our team’s Research Lead, I guided our team in the exploratory research stage. We observed 7 class sessions, and interviewed 16 students and 5 teachers. Then we used affinity diagrams to draw out themes. Tip: when affinity diagramming, it is a good idea to make it a norm that people don’t talk, so that any one person’s influence is limited
2. DATA SYNTHESIS
Find a problem worth solving
We found that there was a sizeable percentage of students who found System 44 incredibly boring. One student we observed thought he had already done one part two times (we think he was so zoned out he couldn’t tell one part from another). Another student was on part 3 of 25, half-way through the year.
What these students had in common was an external motivation source. Something other than their own interest and goals was motivating them. Shifting their motivation from extrinsic to intrinsic seemed like a worthy goal. We hypothesized that solving this would take care of other issues such as a lack of progress, which was dire because they were very behind in their reading ability.
3. INTEGRATE LEARNING SCIENCE
Allow students to make their own choices in System 44 to shift their motivation.
Since we had already landed on shifting students’ motivation from extrinsically to intrinsically located as a goal, we conducted a quick literature review, and found a well-regarded theory that pointed us towards possible implementations. This theory, Self-Determination Theory was created by Edward Deci and Richard Ryan. Here’s a link to one of their papers, which has 28,401 citations.
There are three pillars to intrinsic motivation, they argue. Simply, shifting from extrinsic to intrinsic motivation requires strengthening people’s needs for competence, autonomy and relatedness.
We brainstormed potential solutions with all three pillars in mind, but in the end, one stuck out as both feasible and sorely lacking in System 44:
Autonomy, the need to feel as though we are making decisions that we want to make.
Collaborate with our client to decide on an idea
After generating 12 ideas we met with our clients, who flew to Pittsburgh.
Together, we created a 2×2 with Feasibility (our ability to deliver the solution) and Impact (on students) as the axes.
In the end, a high-impact, and feasible solution that we settled on was an interactive narrative.
Our mission statement became “We will develop an interactive storytelling prototype that motivates students to practice skills they struggle with the most, enabling them to become competent and confident in reading long passages.”
5. ITERATIVE DESIGN PROCESS
Use prototyping and user testing as a way to minimize risk
We created at least six prototypes of an interactive story over the following weeks, tested each with students and sought out feedback from teachers and our client partners.
With the first prototype, we needed to see if students actually liked having the autonomy of choosing their way through a narrative.
Here’s a screen from our first prototype.
Involve teachers: injecting game-based assessments into our prototype
Along with an interactive narrative, we created embedded game-based assessments to give teachers valuable data to personalize instruction with.
Here’s one of our initial game-based assessments, which looks a lot like those in System 44.
Our biggest challenge: ‘chocolate covered broccoli’
We got feedback from user tests that the game-based assessments (like the one above) felt “separate from the rest of the narrative.” This was an instance of the danger of chocolate covered broccoli, a common dilemma with educational games, which means taking something that’s dry and educational, and trying to make it fun by adding gamification. Here’s a link to what google has to say on chocolate covered broccoli.
So what’s the problem with chocolate covered broccoli anyway? Well, if students come to expect games with phonics, they won’t truly be motivated to read if there are no gamified rewards.
Remake our prototype to include well-integrated assessments
- Plan the story’s plot to include “scenes” in which we could add a game-based assessment as a logical climax of those scenes. This guided us in realizing how many assessments we needed to make.
- Brainstorm different ways that the students’ correct or incorrect answers would affect the story. For example, in our story, the students hide their gold from a pirate. If they fail to unlock a combination lock with the appropriate word, they lose gold. This should further engage them with the narrative, create opportunities for fine-grain data for teachers, and hopefully associate feelings of pleasure with reading in students.
- Create the assessments to provide useful data to teachers. We hypothesized that teachers would benefit from distinguishing students’ problems as comprehension based (not knowing what a word means after reading it) or decoding based (not being able to read, or decode, the word in the first place). Moreover, knowing which phonemes students mistakenly switch, would allow teachers to provide very personalized instruction.
What the students thought
After altering our prototype to have game-based assessments that were truly integrated into the narrative’s storyline, we tested it with our target audience. The result: all of them felt that the narrative functioned as a seamless whole!
6. FINAL PROTOTYPE
Present the key features
An inviting home screen. Students can pick a story matched to their interests.
‘Read it to me’ button and ‘gold count’ in the upper-right. Narrative text and choices (in orange) in the middle. A visual timeline on the bottom.
Game-based assessments. Listening to a word and attempting to comprehend its meaning. The robot is the students’ companion throughout the story.
Fine-grained data hooks. This is what students see if they fail the previous problem. Through multiple follow-up assessments, we can determine exactly where students are struggling.
Graphics revealing students performance. Students see the gold and items they collected and their performance on each of the four assessments, and have the opportunity to practice the words they got wrong.
The next System 44
With a new version of System 44 coming in 2019, we hope that our research and prototypes help Houghton Mifflin Harcourt in accomplishing the very reason they hired us: to make System 44 more engaging, effective, multi-sensory, and supportive of its most vulnerable users.
For me personally, this project showed me how learning science can be applied practically, and helped me improve my UX research & design, project management, cross-functional collaboration, and client relation skills.
Should I continue to work on this project, I would investigate the use of scaffolds to help learners who are especially unmotivated to access the text, and enjoy the story. To me, improved confidence, ability and motivation to read is achievable only with extensive (and hopefully, enjoyable) practice, with a combination of reading authentic texts and practicing targeted skills.
Time: 7 months
Methods: exploratory research (semi-structured interviews and observations), data synthesis (affinity diagramming), storyboarding, prototyping