Award-winner Ryan Baker looks to the future in online learning

Since he arrived at Penn GSE in 2016, Associate Professor Ryan Baker has been busy.

As the leader of Penn Center for Learning Analytics, Baker and his students are using big data to examine online and blended learning across educational settings. He’s delved into Massive Open Online Courses (MOOCs), asked questions about learner engagement and learner emotion, and studied complex reasoning in complex learning environments, among other areas.

While his research covers a broad range of learning systems, Baker says it is united by a search to make online learning work for students of all backgrounds and abilities, in the US and around the world. For his efforts, the Council of Scientific Society Presidents recently gave Baker its 2018 Educational Research Award.

With new research, we sat down with Baker to talk about what he’s learning in 2019.

Q: What are you and the center currently working on? 

A: We’ve built up our data infrastructure, but the exciting part is coming now. We’re able to investigate what works for different students and in courses of different designs. The results could give us new options for making online learning more equitable. 

Q: How so?

A: Adaptive learning only works if the models are actually good for the students using them.

When we build the models on a convenience population and we test them on a convenience population, they work for that convenience population.

The problem is — even with all data-based systems — it’s still easier to do research in suburban areas that are upper-middle class than urban and rural areas. If we don’t pay attention to these issues explicitly, we risk having a model that doesn’t work for many students. 

Q: And that can leave out students of color and students who grew up in poverty —

A: And subgroups we rarely focus on. We have research under review now that looks at military-connected students — students whose parents are in the military. Models built for non-military connected students don't tend to work as well on military connected students. They tend to just be less accurate. 

Q: How big of a problem is this for many of the students using programs that weren’t designed for them?

A: It’s not an obvious penalty, but the software just won’t work as well for them. The software makes predictions for kids, those predictions will be used to make decisions by the software or teachers, and those decisions will be kind of quietly wrong.

A kid who’s not at risk, the model will say they are at risk. Or the model will move them along too quickly. Or the adaptive learning system will not adapt the way they need it to. There are all these under-the-hood things that adaptive learning does now, and if the model is systematically off 7 percent more often for an African American kid, or a kid in a rural school, that’s a real equity issue over the long-term.

Q: How else are you using data to understand learning?

A: We’re getting better at detecting whether students are confused or frustrated, and how that differs across learning activities. We’re looking at problem-solving, and we’ve seen interesting things with erroneous examples, where a student is shown an incorrect conclusion and has to explain why it’s incorrect.

Erroneous examples can be more frustrating for students, but can also lead to more learning. It looks like confusion and frustration are actually associated with more learning.

Q: So is confusion a good thing?

A: It can be, if a course is built to support students through their confusion. But if the support isn’t there, confusion and frustration can become a bigger barrier than they are in a traditional classroom.

MOOCs are a great example. It can’t be overestimated how much an uptick in difficulty matters. When a video goes up that introduces a lot of new material, the data shows that’s the last video many students will look at. They just hit a brick wall.

Q: And that’s something it takes data to understand.

A: It all comes back to what we can do with newer, richer data to better understand students and drive forward their success.