Live from SXSWEdu: Who’s Khan Academy in bed with now?

The White House, the College Board, Bank of America, and Comcast.

What do they have in common? They’ve all announced partnerships with Khan Academy, the nonprofit provider of free educational videos and analytics tools, in the last 12 months.

These partnerships combine various aspects of cash, content, and distribution. But in each case, the stodgier established player is borrowing a little bit of Khan Academy’s halo. Maybe it’s because of their nonprofit status, maybe it’s Sal Khan’s geeky yet approachable personality, maybe it’s because the content is really pretty great and useful, but Khan Academy is the rare ed-tech company that has managed to reach millions of people while gaining a high media profile and transcending the MOOC backlash. They remain dedicated and associated in the public’s mind to providing free high quality educational resources to the world using technology. And so when a technology company wants to burnish its association with low-cost internet access; or the White House wants to update the image of college access, or the College Board wants to democratize test prep; or a big bank wants people to not hate them, they turn to one guy with a digital whiteboard and a camera on a tripod.

Sal Khan talks to David Coleman at the College Board

So it’s pretty clear what these organizations want out of KA. But what does KA want with them?Of course, the nonprofit needs cash to continue. But they’re also at risk of diluting their brand image, independence, and mission every time they ink a deal. The Comcast partnership, in particular, has raised eyebrows as it relates to net neutrality–an Internet provider appearing to favor a certain type of content.

I spoke to Monica Tran, Khan’s Product Strategy Lead, for insight on what KA is thinking as it enters into these partnerships.

“We are in a privileged position — we have a lot of interest from potential partners,” she said. “Given the fact that we’re pretty small and lean, we are highly thoughtful about what these partnerships can do for our learners.” She mentions that in the case of the College Board partnership, 70% of KA’s users are under 17, so these test prep resources are potentially quite appealing for that audience. And there’s a synergy with their college prep tools, which come out of the White House relationship.

I believe in the idea of giving learners what they want. But by aligning itself with SAT prep and college prep, Khan Academy is tethering itself to the educational establishment, diverging from its original, revolutionary vision: to give people a path to learn and demonstrate mastery without needing institutions as intermediaries.

Can online learning make teaching more human?

Screen Shot 2013-09-03 at 3.56.59 PM

Data-driven pedagogy. The phrase conjures a robotic, dull future that only intensifies the worst aspects of 20th-century, bureaucratic, industrial wasteland-style schooling, where learners are defined down to “users,” or even metonymized as disembodied “eyeballs,” and force-fed bits of disconnected information.

For a counternarrative, the question is simple. What can creative humans do with the power of data? One possible answer is that computer-powered analytics could expand humans’ ability to focus on the most human aspects of teaching and learning.

I reported earlier this year on a small experiment the video website Khan Academy ran to this end.

While browsing the web site, some Khan users saw a simple slogan added to the page next to, say, a math problem: “The more you learn today, the smarter you’ll be tomorrow.” The line linked to a further explanation of the concept of “mindset,” the famous body of research by Harvard psychologist Carol Dweck on growth, achievement and motivation.

Displaying that one line led to a 5% increase in problems attempted, proficiencies earned, and return visits to the site, compared to otherwise similar learners who did not see the line.

This week, Andrew Liu, Udacity’s data science intern, blogged about his own research with the data generated by that MOOC platform. Apparently the questions they are framing go along similar lines: toward psychological aspects of motivation and engagement.

Screen Shot 2013-08-27 at 4.55.42 PM

Model of student engagement over time.

Modeling student engagement over time.

“At Udacity, we now have the opportunity to take findings that originated from studies on tens of students in physical classrooms – such as Carol Dweck’s concept of growth mindset – and apply learnings to hundreds of thousands of students with improved teaching. But even more powerful is Udacity’s ability to conduct our own pedagogical research at scale on a rapidly growing worldwide classroom that was not even possible a year ago. Pedagogical areas we’re exploring include the importance of metacognition, expectation setting around formative assessment, and even new online challenges such as which characteristics of video keep students most engaged.” 

Mindset, metacognition (learning about learning), engagement–these are great research questions for educators to be looking at. They are not chiefly about automating the consumption and digestion of information, but about deepening the learner’s physical and emotional relationship with the process of learning.

It’s in part simply the growth of sample sizes that has some researchers so excited about what they might learn in the emerging field of data-driven pedagogy. I haven’t verified this independently, but I have often heard researchers repeat the notion that there are just very few large-scale randomized controlled trials out there comparing the efficacy of various classroom techniques and methodologies.  Sample sizes tend to be quite small and experimental effects hard to compare. (If there are counterexamples, I’d love to hear them).

A major example is the efficacy of online and blended learning itself. According to a comprehensive literature review published by Ithaka SR earlier this year, of over 1000 online and blended learning studies reviewed by the US Department of Education, only 45 met minimal criteria of having experimental research design and considering objective learning outcomes. Of those 45 studies, “most have sample sizes of a few dozen learners; only five include more than 400 learners.”

The kind of a/b trials that the Khan Academy and Udacity are doing, by contrast, can be easily run on hundreds of thousands of people.

Obviously there are relationships and aspects of the human dimension of learning that can’t be addressed with even the best data tracking and experimental design, or the largest sample sizes. There is an ever-present danger that the metrics chosen will tend to distort the nature of the undertaking itself. However, I can’t help but be a little optimistic that at least data scientists are starting with the right kinds of questions.