SXSWEdu: Gates Foundation vs. Microsoft Education: What’s the difference?

On my first day in Austin I had a terrific hourlong conversation with Cameron Evans, Chief Technology Officer for Microsoft Education. He had a lot of candid observations and great lines: “The vast amount of data in our education system can be used for good, and also for bad actors and bad reasons,” and on the need for professional development and parent education around new learning technologies: “You don’t want to be in a situation where you give people a library card but they can’t read.”

op-art by simosx on Flickr

op-art by simosx on Flickr

Toward the end I took the chance to ask him about a controversial point. It is an article of faith among many concerned about education that the extensive philanthropic influence of the Bill & Melinda Gates Foundation, headed by Microsoft founder Bill Gates, is deployed directly in service of the business interests of the Microsoft Corporation. When it comes to schools, where do the interests of one end and the other one begin? (Disclaimer: The Bill & Melinda Gates Foundation is among the many funders of The Hechinger Report, which publishes this blog.)

Evans greeted my question with a broad smile. It’s one that he fields fairly often.

“Bill and Melinda Gates have gone out of their way to make sure that the foundation’s work and the work of Microsoft have nothing to do with each other,” he starts, noting that the Gates Foundation often uses open-source technologies.

However, he allows, “There are times we do see eye to eye.” 

Not for nothing, Evans agrees that the Gates Foundation and Microsoft share a single mission. As the foundation puts it, “Bill & Melinda Gates Foundation works to help all people lead healthy, productive lives.” “We share that core mission: everybody should be able to participate fully in life without constraint or impediment.”

The true common ground between the company and the foundation, Evans tells me, has to do with education as a source of human capital and economic productivity. 

“We’re not producing enough human capital capacity in higher ed and K-12. We have an obligation as a corporation to help America be the dominant player in international competitiveness. We should be very clear about this. This is a national issue for a global company. It’s not something we have to apologize for or defend. If the Gates foundation shares that sentiment, so does Intel, so does Lumina, so do MacArthur and Carnegie.” He mentions Microsoft’s legislative advocacy on behalf of the H1B visa for foreign-born technology workers as a sign of their interest in global human capital.

I point out that he’s defaulted to talking about Microsoft’s interest in education from a corporate social responsibility standpoint. This is a party line in any ed-tech conversation, and I wish that corporations would drop it. We all know that Apple, Google, and Microsoft are not just in the education business for the love of adorable, ethnically & socioeconomically diverse children. If they’d be more transparent about where they see the opportunities, we could have an open and honest public discussion about the legitimate role of private actors in providing this public good.

Screen Shot 2014-03-04 at 11.01.23 PMgoogle-teachers


top to bottom: Cute students using Apple, Google, Microsoft

So Evans says this: Microsoft’s education business strategy is not primarily about hardware–a pragmatic attitude since they’re losing out on hardware.

“Some people think our sole goal is to get kids hooked on our products,” through deployments of items like Surface tablets in public schools, he said. “That they’ll be gateway products, students will be hooked for life. The consumer market today is far more dynamic than that. IT doesn’t decide what technology you use, whether in school or out of it. Consumers are choosing, not school districts, not anybody.”

What IS it about, then? Reading between the lines, by being a strong voice in workforce development and training, Microsoft helps ensure the the continued dominance of Microsoft software programs like Word, Excel, PowerPoint, and Access across the transition to mobile access in the global workplace. “As a technology supplier to governments, hospitals, individual corporations and entrepreneurs we have a responsibility to build capacity,”  Evans puts it. Microsoft has 1.5 billion users on the planet; only 55 million are in the US.

Secondly, on a much smaller scale than that global strategic interest, Microsoft is developing educational data and reporting systems that aim to give students, schools and parents embedded, formative, actionable, continuous feedback and insight about student growth and progress–and to give teachers and administrators performance feedback for professional development purposes. Evans was at SXSWEdu to promote this idea. “A teacher needs data on the student sitting in front of them right now. We need sophisticated, but simple tools to be able to communicate across the board, what is this data, and what decisions do I need to make that are different.”

The Gates Foundation, of course, cares a lot about data, reporting, and teacher quality as well. And here’s the rub: It can be difficult to trust any kind of separation between the interests of the foundation and the corporation because in many cases, not necessarily nefarious ones, they do in fact coincide.

Amplify unveils new digital curriculum

Amplify, the digital education brand owned by News Corp, unveiled its new digital curriculum today at the SXSWedu conference in Austin, TX. They offered me a preview on February 21st.

The home screen of Amplify's ELA curriculum.

The home screen of Amplify’s ELA curriculum.

At last year’s conference Amplify announced its own custom-built educational tablet with great fanfare. Since then it’s had a rocky rollout with a recall in Guilford County, NC due to broken screens and melted chargers. Amplify is reimbursing the county to the tune of $4.8 million in devices, services and cash.

This digital curriculum has been half of Amplify’s two-pronged strategy from the beginning, now it’s part of a do-over for its brand image. It is device-agnostic; it can run on iPad, Kindle or PC and in some cases smartphones. Amplify is launching with a Common Core-aligned English Language Arts curriculum for 6th, 7th and 8th graders, to be followed soon by social studies and science. It will be available this fall, starting at $45 per student.

Analytics for students.

Analytics for students.

The curriculum is a good example of what is coming, likely to most classrooms in the next decade, with the convergence of textbooks, materials and to some extent assessment in the digital format. As Joel Klein, former New York City schools superintendent and Amplify’s CEO, points out in our interview, the Common Core, as controversial as it is, creates both a great need for new materials and a unified marketplace for companies like his to create them.

Another analytics screen.

Another analytics screen.

“The NEA recently came out echoing what a lot of people have been saying –the Common Core is hard. Teachers are going to need the support.  You need immersive content that helps kids engage better in the learning experience. What we’re doing, and I say this in sincerity, I think it’s really unique, different, and it’s going to have a profound impact on the debate going forward.”

The Amplify ELA curriculum, explains president of Amplify Learning Larry Berger, was designed with three key metrics in mind: to get middle school students to do three times the typical amount of reading (which only gets you just above an hour a week), to write three times more pages (around 200, total, in 7th grade), and to get three times as much feedback both from the program directly and from teachers.

The lessons include a reading mode with highlighting, notes and vocabulary features, a writing mode with various word processing tools like a word counter, and a wide selection of digital multimedia content, including a library of 300 ebooks and games.

For example, students can watch actor Chadwick Boseman, star of last year’s Jackie Robinson bio-pic, dramatizing an excerpt of Frederick Douglass’s writings; or an animated version of Edgar Allen Poe’s The Raven, commissioned from an Oscar-winning animator with a voice performance by a Tony-winning actor and critical input from a Poe scholar at the University of Alabama. The resulting moody, black-and-white cartoon is part of one of the curriculum’s “quests,” a multi-day creative lesson plan that casts students as detectives seeking to solve Poe’s murder by reading his best-known works.

In addition to the official curriculum, Amplify is introducing a suite of 40 computer games designed by independent game designers. There is a World Of Warcraft-like universe called Lexica where students gain powers and points for their characters by reading books and playing grammar games, and a group of science games that bring to life processes like digestion. The games are designed to enrich and complement the curriculum, to compete for students’ free time, although some may also be assigned as homework.

All of this digital content, in my brief peek, appears high-quality and engaging. It’s designed with great attention to scope and sequencing. Berger points out that unlike a textbook publisher, Amplify tests and tweaks every lesson they develop on real students, first on a group of middle-schoolers who come to the headquarters after school (rewarded with pizza and gift cards) and then on students in classrooms.

The aspect of the digital convergence that may raise eyebrows is that products like these allow teachers, if they wish, to essentially put their classes on autopilot. Unlike a textbook, which may offer a few questions at the end of each chapter, Amplify, Berger told me, is designed to “orchestrate” or “choreograph” every five minutes of instruction, and often delivers it, along with feedback, directly to the student so the teacher doesn’t have to. This is a level of heavy lifting that some teachers will welcome taken off their shoulders; others, who prefer to curate classes on their own, may see it as intrusive or even a Trojan horse for increasing class sizes in a quest for efficiency. In either case, implementation and professional development will be everything.


Hechinger Report heads to #SXSWedu: Our top picks and tips!

art by open educator Alan Levine

art by open educator Alan Levine

South by Southwest Edu , this March 3-6 in Austin TX, has grown into a several-thousand-person big-tent conference about all things innovative in education: not just technology and entrepreneurship but new approaches to issues like inequality and social and emotional learning. “The same way seismic shifts in the music, film and technology industries fueled the growth of SXSW, the convergence of technology and innovation has set the stage for tremendous disruptions in the field of education,” Ron Reed, the director of the festival, told us recently. So it’s a natural place for Digital/Edu and the rest of the Hechinger Report to be.

After attending SXSW Interactive, Music, and/or Edu for five of the last six years, here are my top tips:

*The chance hallway encounters, meetups and party conversations tend to be far more valuable than *most* sessions (not ours of course). So leave room in your schedule for serendipity!

*Twitter is the best way to follow all the discussions and figure out where to go next. This year most sessions have their own #hashtags.

*Carry a water bottle, phone charger and energy bar, and wear comfortable shoes.

*Try to sit down for at least one great Austin meal. Here’s a bunch of recommendations.  Or just go to Stubb’s. (I’m a vegetarian so I like to go to Jo’s for brunch instead).

You can meet some members of the Hechinger Report team at the following sessions:

Tuesday, March 4, 12pm: Can the Liberal Arts Survive in an Age of Innovation? with Hechinger Editor in Chief Liz Willen, Scott Kinney, president of Capella University, and more.

Tuesday, March 4, 3pm: LAUNCHedu: Educator Insights (educator feedback on the LAUNCHedu startup competition) with me, Anya Kamenetz, Digital/Edu writer, top tech-teacher Jennie Magiera, and more.

Wednesday, March 5, 9am: How Media Shapes Our Perception of Technology with me, Betsy Corcoran from Edsurge and Paul Fain from Inside Higher Ed.

Wednesday, March 5, 12pm: Building a Better Teacher, with Jackie Mader, Hechinger Staff Writer, Kate Walsh of the National Council on Teacher Quality, and more.

Thursday, March 6, 9am: Hardwiring the Brain: Better, Faster, Stronger, with me and a bunch of brainiacs including Jan Plass, NYU professor of Digital Media & Learning Sciences and co-director of the Games for Learning Institute.

In addition to the above sessions, here’s some more we really want to get to. Leave your top picks in the comments!  

Monday, March 3, 3pm: Reclaiming the Promise Through Community Schools, a plenary discussion on the model of schools with wraparound services, moderated by AFT President Randi Weingarten.

Monday, March 3, 7pm at the Alamo Drafthouse: Ivory Tower, a screening of a new documentary on the future of higher education.

Tuesday, March 4, 10:30am: Data Privacy: What Parents DON’T Know Can Hurt You — an overview of a hot issue in the news, featuring Cameron Evans, Microsoft CTO.

Tuesday, March 4, 12pmInspiring Advocacy through News and Digital Media with folks from Do Something, Channel One, and NBC learn.

Wednesday, March 5, 12pm:  Wendy Kopp: There Are No Silver Bullets in Education –-Teach For All’s CEO, an increasingly controversial figure (aren’t they all?) argues there are no silver bullets to solving educational inequality (well, who told us there were?).


Bridging college-to-career

As of last June, 46% of recent college graduates were underemployed, meaning they had jobs that didn’t require a college degree, while a further 6% were unemployed. For employers, meanwhile, hiring young people means dealing with a lot of churn and an extensive training process; the average tenure in a first job is just 18 months.

With the cost of college and student loan debt rising, there’s increasing urgency to help graduates find fulfilling careers that fit their skills. A recent startup, Koru , is addressing itself to this problem.

Who We Are from Koru on Vimeo.

“A lot of young people feel like they’ve been sold a bill of goods,” says Josh Jarrett, the co-founder and Chief Learning Officer of Koru. “One young woman told us, ‘I never felt more abandoned than the day I graduated college. I was like Now What? I was on this 16 year long conveyor belt, I go to the end, and I was dropped off.”

For almost eight years, Jarrett supported other endeavors as a higher education program officer for the Gates Foundation. He made the decision to take the leap into entrepreneurship because of a strong belief in the power of both/and: both liberal arts education for its own sake combined with extremely practical hands-on experience.

The Koru model is for young people, from about the junior year of college through a few years after graduation, to work on a real challenge set by an employer: for example, do field research to determine the best way of reaching Millennial customers. To provide better context for their immersion in the world of work, they go through an MBA-lite curriculum covering concepts like design thinking and business analytics.  The model is blended learning, using the same kinds of collaboration software that young hires are likely to encounter in the office, like Basecamp and Asana.

Koru piloted its first, 10-day programs in Seattle earlier this year with companies including outdoor outfitter REI and online retailer zulily. They plan to launch a full set of four-week-long programs in Seattle over the summer, in San Francisco by the fall.  They just announced  a set of partner institutions that plan to offer Koru as a career services option. These are a well-known group of universities and liberal arts colleges: Bates, Brown University, Colorado College, Connecticut College, Denison University, Georgetown University, Mount Holyoke, Occidental, Pomona, University of Southern California, Vassar, Whitman, and Williams. “There will be a cost share between participants and employers,” Jarrett says–participants or their parents, and possibly their colleges, will pay part, and employers will contribute a placement fee if they end up hiring the participants.

Koru is part of a wave of what I call “educational popups” that seek to bridge the worlds of formal education and various industries. They are most numerous in the technology and design industries–Hacker School, General Assembly, and Hyper Island are all good examples.


#Schoolprivacyzone: emerging best practices for a contentious issue

Advocacy group Common Sense Media held a summit in Washington, DC on Monday as part of a national campaign on the highly contested topic of student data privacy.

The results of Common Sense Media's January survey show most parents concerned about student privacy.

The results of Common Sense Media’s January survey show most parents concerned about student privacy.

A recent study by Fordham University Law School found that as schools and districts adopt cloud computing services, they are transferring student information to third-party providers, often leaving it open to data mining and commercial purposes such as reselling and ad targeting. These services may be in violation of federal law. These agreements allow vendors to do whatever they want with student demographic records and other personal information. Of the districts studied, fewer than 25% of the agreements between districts and vendors specified the purpose for disclosures of student information, and fewer than 7% restricted the sale or marketing of student information by vendors in any way. And that is to say nothing of the risk of hacking or other security breaches.

There are a number of attempts in the works to establish better guidelines for the $8 billion educational software industry. The Software&Information Industry Association, a trade group, yesterday announced a list of best practices for agreements between software groups and schools:

That data should be used only for educational purposes, that its use should be fully disclosed and transparent and full consent obtained from families, that all reasonable security procedures should be followed and schools be notified in case of actual data breaches.

Even as the industry is taking baby steps to govern itself, lawmakers are converging on a solution with more teeth. California State Senator Darrell Steinberg just introduced a bill in that state enforcing some of these same principles: educational purposes only, encryption and deletion of data. Massachusetts Senator Ed Markey plans to do the same at the federal level.

At the Common Sense Media event, according to the lively discussion on Twitter, industry representatives like Cameron Evans of Microsoft and Joel Klein of Amplify argued that a rush to legislate might cause more problems than it solves. Best practices for data privacy and security continue to evolve as the technology does. The large-scale use of cloud computing and web-based data storage itself dates back only to the mid-2000s. It is difficult for the law to catch up. Also, while contracts may specify “educational purposes only,” the nature of the beast in ed-tech is that a large source of educational innovation is coming from for-profit startups whose involvement with the day-to-day experience of teachers and students is becoming increasingly intimate, if not intrusive. In practice the line between educational and commercial purposes may be somewhat blurry. As Katherine Varker, Associate General Counsel, McGraw-Hill Education, asked at the summit: ‘Where does targeted advertising end and personalized learning begin?’


Guest Post: A new approach to accountability

Note: Through March 27th I’ll be sharing this Digital/Edu space with some excellent professionals in the area of learning and innovation for one weekly guest post. In addition to bringing new voices and ideas to our readers, this will help me as I finish the first draft of my forthcoming book, The Test, on the past, present and future of testing in public schools, to be published by Public Affairs in 2015. Today’s guest poster, Joshua Starr, is superintendent of Montgomery County Public Schools , a national leader in the debate over 21st century skills, and @mcpssuper on Twitter.

JPS Headshot

School districts across the country are in the middle of the very difficult work of revising and reshaping their curriculum to align with the Common Core State Standards. (CCSS) And this work is worth doing. The CCSS provides our educators an opportunity to improve teaching and learning by organizing around a uniform set of high expectations. Moreover, consistency from state to state and district to district is essential if we truly want to improve public education in the United States.

In Montgomery County (Md.) Public Schools (MCPS), where I am superintendent, this work has been going on for several years. A CCSS-aligned curriculum is fully rolled out in elementary grades and is well underway in middle and high schools. Most people see the benefit, but there are growing pains. Teachers need more time for professional development; not everything works the way the folks in central office think it will; and many parents have questions about our new approach. And there remains tension about what the Common Core-aligned assessments will look like, what the role of AP, IB, SAT and ACT will be relative to PARCC, and how they will be used to support the improvement of instruction.

Good standards and assessments are extremely important to improving education for our students. But they are only a part of the answer. If we are truly committed to a new definition of “college and career ready,” we need to develop an accountability framework that supports that definition and it needs to look a lot different from No Child Left Behind (NCLB).

For more than a decade, nearly every school district in the United States has aligned its work to meet the “adequacy” of NCLB. Despite lofty mission statements about serving the “whole child” and developing “lifelong learners,” every school had a clear directive: Make AYP (Adequate Yearly Progress). This was true in MCPS, although the district did effectively hold itself to a higher bar.  MCPS encouraged more students to take Advanced Placement (AP) and International Baccalaureate (IB) Classes, paid for every 10th grader to take the PSAT, and encouraged students to be well prepared for the SAT, even offering prep classes in the schools. The results have been impressive.

About two-thirds of our graduates take at least one AP or IB test and more than half of our graduates earn a “college-ready” score on an AP or IB exam. MCPS has one of the highest SAT scores of any large district in the nation—an average of 1648 for the Class of 2013. And some of the biggest growth in these areas has been among African American, Hispanic and economically disadvantaged students.

However, all of this achievement has not necessarily translated into postsecondary success. About fifty percent of MCPS graduates earn a bachelor’s degree within six years of high school graduation. That is better than the state of Maryland or the country as a whole, but it is not reflective of our stellar K-12 results. And the achievement gap in college completion rates is stark: A white MCPS graduate is twice as likely as a black or Hispanic graduate to enroll in and successfully complete a bachelor’s degree.

This is not uniquely a problem in MCPS. Every school district—especially the larger ones—is dealing with these types of results. At the same time, industry leaders are telling us that our graduates don’t have skills like perseverance, teamwork and creative problem solving that are so important in the 21-century workplace.

MCPS is one of 15 school districts that are a part of the Large Countywide and Suburban District Consortium. The Consortium includes the leadership of successful districts that serve large, diverse student populations. In a recent meeting with a bipartisan group of U.S. Senators, we laid out the framework for a federal accountability system that will support meaningful outcomes for our students. I, and my colleagues in the Consortium, believe a reauthorization of the Elementary and Secondary Education Act (aka NCLB) must have these key elements:

  1. Establish 21st-century outcomes as the goal of all major education laws and policies
  2. Require academic content standards and student academic achievement standards aligned with college- and career-readiness outcomes
  3. Promote the development and use of high-quality assessment systems and other valid measures of college- and career-readiness
  4. Promote innovation and continuous improvement in schools and districts
  5. Ensure equitable access to effective educators
  6. Promote transparency, engagement, and shared accountability through appropriate reporting of data and information

I believe there is a need for a strong and clear federal accountability system. NCLB had major flaws, but it got us talking about data in a way we never have before and rightly held us accountable for the unacceptable performance of too many students that have been dis-served by American public education. As we consider the next generation of federal accountability, we need educators, families, industry leaders, higher education officials and policy leaders to come together to help define college and career readiness in a way that includes more than just test scores and graduation rates. We must take the opportunity to provide our students—and those that teach and advise them—with a coherent definition of success that will enable our children to thrive in the future that they want.

Measures of the heart: non-cognitive skills tests

Being smart isn’t all you need to succeed.

Non-cognitive or meta-cognitive skills such as self-awareness, self-control, empathy, communication and cooperation are as important or even more important to long-term success than cognitive factors like math and reading ability. The evidence is undeniable, but US school assessment and accountability systems are currently limited to measures of cognitive skills alone. If schools don’t measure non-cognitive skills, it is difficult to prioritize them.

There are several emerging attempts to measure non-cognitive skills, aka Mindsets & Essential Skills and Habits (MESH) in schools. Most use the simplest instrument available: self-reporting surveys.

  • The OECD gives the Program in International Assessment or PISA to about half a million students around the world. It’s widely publicized for its rankings on math and language achievement. But in addition to the two hours of word problems or essay questions, PISA students also complete a 30-minute questionnaire asking about things like self-efficacy, self-esteem, student-teacher relationships, and the school climate, making it one of the largest attempts to measure non-cognitive skills in the world. An infographic published on the website Buzzfeed based on PISA data sorted countries about evenly into four quadrants: happiest and least happy kids, and high and low test scores. Korea stands out for its miserable kids and high scores, Singapore has the happiest kids with the highest scores, Qatar and Argentina have sad, low-scoring kids and Peru and Indonesia are whistling away their weak scores. (The US falls into the unhappy, low-scoring bottom left quadrant.)


School leaders around the world are using PISA findings on the emotional health of their students. In Korea, in February 2013 the new president Park Geun-hye announced that the happiness of citizens would become a national priority, and the Ministry of Education has taken this up as a goal.

  •  In 2012 almost 500,000 students across the US in grades 5-12 completed the Gallup Student Poll, which asks about students’ hope, engagement, and well-being. Over the last 10 years, scores on the hope survey did a better job of predicting college persistence and GPA than high school GPA, SAT, and ACT scores. In the most recent survey, about a third of students were “struggling” and 43% were either “not engaged” or “actively disengaged” with school.
  • YouthTruth is a 20-minute online survey that’s been taken by about 250,000 students in grades 3 through 12 at 300 schools, covering school climate issues like bullying, classroom engagement, as well as optional topics like student motivation and grit. Schools are using YouthTruth to gather feedback not only on the school as a whole but also on individual teachers.
  • The State of California’s Department of Education, starting in 1998, has administered the largest statewide survey program in the nation looking at “resiliency, protective factors, and risk behaviors”, made up of the Healthy Kids Survey, the School Climate Survey, and a parent survey. In 2013-14 they updated the survey to track learning engagement, students’ connectedness to school, and a new optional social-emotional health section covering empathy, self-efficacy, self-awareness, persistence, emotion regulation, gratitude, ‘zest,’ and optimism.  The social-emotional health section for middle school students asks them to identify with statements like “each day I look forward to having a lot of fun,” and “I can do most things if I try.”

 Once schools have adopted one of these surveys, the next step is to incorporate them into decisionmaking processes. The Montgomery County Public Schools in Maryland has incorporated the Gallup hope survey into its strategic planning processes and goals.  A group called Transforming Education is advising the California Office to Reform Education, a group of eight big-city school districts in California that has obtained its own separate NCLB waiver. They are setting up one of the first large-scale accountability systems  that places non-cognitive measures as peers to test scores. Schools within these districts will be evaluated on 60% conventional test scores, 20% school culture/climate measures such as suspensions, expulsions, and bullying, and 20% social/emotional learning scores. 

The idea of attaching stakes to self-reported measures of happiness is fraught with peril. Surveys that are perfectly valid when administered as research tools may be distorted when they become the basis for school and teacher ratings. It’s far easier to game a self-reported survey than, say, a math test. To overcome these problems, researchers such as Patrick Kyllonen at ETS are trying to develop more sophisticated measures, such as asking students to provide a solution to a hypothetical interpersonal problem, demonstrating their emotional wisdom. Another approach is to combine multiple measures such as suspension rates and teacher reports with student survey data to get a more complete picture of the climate at a school. Regardless of the path chosen, we are headed towards a world where students’ emotional health and wellbeing gets full attention commensurate with its importance in determining their futures.

Guest Post: the latest MOOC Research

Note: Through March 27th I’ll be sharing this Digital/Edu space with some excellent professionals in the area of learning and innovation for one weekly guest post. In addition to bringing new voices and ideas to our readers, this will help me as I finish the first draft of my forthcoming book, The Test, on the past, present and future of testing in public schools, to be published by Public Affairs in 2015. Today’s guest poster, Justin Reich, is the Richard L. Menschel HarvardX Research Fellow and a Fellow at the Berkman Center for Internet & Society. He writes the EdTechResearcher blog for Education Week.

Framing MOOC Research: How Comparisons Shape Our Judgments

Last month, my colleagues and I on the HarvardX and MITx research teams jointly released a series of reports about the first 17 courses launched by HarvardX and MITx on the edX platform. We released a synthesis report with findings about all of the courses, and then 15 additional reports examining individual courses in more detail.

We tried to provide the public and our internal stakeholders with data that instructors can use to create better courses and that people can use to judge the state of the enterprise. We are keenly aware, however, that our data don’t have a single story to tell, and how people read our research depends upon how they approach the subject.

Here are two sets of facts, two possible frames for thinking about HarvardX courses:

Set 1: In September 2009, Boston’s WGBH published to YouTube a series of 12 videos from Michael Sandel’s Justice course at Harvard University. Each hourlong video is a combination of lecture from Sandel and facilitated Socratic dialogue among the hundreds of students who take his class every semester.

On YouTube, the first video has been played almost 5 million times. The second video over a million times. The next ten videos have been played about 300,000 to 400,000 times each. All told, the series has about 10 million views.

As a back of the envelope calculation (ignoring people who watch videos multiple times, who don’t finish videos, etc.), it seems unlikely that more than 6% of people who started the series finished the whole thing.

Set 2: Researchers at the Community College Research Center at Columbia University’s Teachers College have done some very interesting work examining online courses in community colleges.

Di Xu and Shanna Smith Jaggers have recently published studies that show that online course completion rates in two large systems in Virginia and Washington are lower than face-to-face course completion rates. In Virginia, completion rates in face-to-face courses were 81 percent, while online completion rates were 68 percent; in Washington, the rates of completion were 90 percent for face-to-face versus 82 percent for online. While online completion rates lag behind on-campus counterparts, the vast majority of students in both conditions earn a passing grade in the courses in which they enroll.

The first set of facts concerns online media available to anyone for learning and personal growth. The second set of facts concerns structured learning experiences offered by institutions of higher education. Both are are potential frames of reference for interpreting our research about HarvardX and MITx courses, each could lead to different comparisons and different judgments.

One of the most important findings from our research is that people use materials in edX courses in all kinds of ways. The 2012-2013 MITx and HarvardX courses had a little over 800,000 registrations. A little over 43,000 people earned a certification of completion in a course. Nearly 36,000 people opened up more than half of the units of a course, but did not earn a certificate. Over 450,000 people viewed less than half of the units of the course (without earning a certificate), and nearly 300,000 people who registered for a course never entered the courseware at all.

The figure below is a scatterplot of all 800,000 registrations. On the y-axis is the student’s grade in the course, and on the x-axis is the percentage of the units (or chapters) in a course that the student opening. As you can see, nearly the entire possibility space is full of points. There are people who are completing every snippet of course content, people who are auditing courses and ignore assessments, people dabbling in a fraction of the course, and people who are never showing up at all.


How should we judge these findings?

One thing we can do is compare these patterns of behavior to the patterns of behavior in community colleges. In community colleges, we are keenly concerned with completion rates. Courses are expensive, and students who fail and drop out not only miss the benefits of learning and certification, but they also lose the money they invested in enrollment. We might make the comparison that only about 6% of HarvardX and MITx registrants finish a course, but in Virginia 68% of online community college registrants finish a course.

We could also compare the online learning content hosted on edX to the PBS Justice videos hosted on YouTube. In online content, we expect to see a funnel of participation. We expect, that on any website, there will be some number of people who navigate to the site, a smaller number of people who register, a smaller number of people who participate in some way, and then a smaller yet number of people who engage the deepest possible ways. When we compare the HarvardX version of JusticeX with the PBS version of Justice, we find very similar patterns of participation.

paired figures

Patterns of persistence and completion in edX look pretty typical when compared with engagement funnels of online media, and they look pretty lousy when compared to community college retention rates. So which is the right comparison?

Faculty intent should play an important role in deciding the right frame of reference, the right yardstick, for judging open online courses.

Some MOOC faculty are primarily interested in sharing their ideas and relatively uninterested in certifying learners’ competency. Michael Sandel, through PBS Justice,, and JusticeX, is primarily interested in helping more people learn more about moral reasoning rather than certifying people at a particular level of skill or knowledge about moral reasoning. If he’s trying to maximize the number of learning experiences that people have, of both lighter touch and deeper engagement, then the frame of online learning media seems to be a fairer point of reference for judging the public impact of JusticeX.

By contrast, many of Sebastian Thrun’s courses at Udacity, especially his pilot programs with San Jose State University and Georgia Tech, are explicitly designed to replace or complement typical courses of study in higher education. Familiar higher education settings make a more sensible frame of reference in understanding these efforts.

As new forms of online learning proliferate, no doubt there will be even more sensible ways of contextualizing and framing new courses and new platforms for learning. As we debate how open online courses might reshape the future of higher education and lifelong learning, its worth paying close attention to how the points of comparison that we choose frame our interpretations and judgments.



The secrets of networked teens: Q&A with author danah boyd

danah boyd is a Principal Researcher at Microsoft Research, a Research Assistant Professor in Media, Culture, and Communication at New York University, and a Fellow at Harvard’s Berkman Center for Internet and Society. She’s one of the most knowledgeable resources in the field of young people and social media. Her highly anticipated book, It’s Complicated: the Social Lives of Networked Teens, based on ten years of field research with young people from a wide range of backgrounds, is officially released later this month. She agreed to share a preview with me and answer some questions for Digital/Edu’s readers.


Q. What do you think are the most common misconceptions held by media, educators, parents and the public about teens’ interactions online?

A. The overarching misconception that many adults have is that everything today is radically different than it was in the past because of social media.  At the end of the day, what teens want – and what they do – actually looks a lot like it did 20 years ago. Technology certainly inflects those practices in new ways, but the underlying desires to hang out and socialize, gossip and flirt still dominate.  Much to many adults’ surprise, the internet hasn’t made teens’ lives worse or more dangerous, although it certainly makes teens’ practices more visible. This visibility is often a source of anxiety, particularly when taken out of context.


Q. Throughout the book you take a nuanced, relatively optimistic stance toward technology. Can you explain how your research and views on, say, bullying or predators generally differ from the prevailing narratives in the media?

A. When it comes to online safety issues, there’s a tendency in popular discourse to extrapolate from extremely horrific cases and assume that this reflects what’s happening.  Numerous scholars have been tracking issues like sexual predation and bullying for a long time and what they’ve found contradicts what the public thinks. For example, study after study shows that bullying happens more frequently and with greater emotional duress at school than online.

Likewise, when sexual crimes involving the internet occur, they rarely take the form of a stranger identifying a minor and abducting her/him.  All too often, what happens involves abused or emotionally struggling youth seeking attention from adults knowing that sex is part of the picture.  This is not to say that it’s not problematic – it’s _deeply_ problematic – but that the kinds of interventions that we need are very different than what is needed to address abductions.

One of the goals of my book is to take such online safety issues and tease out where there is merit to the concern and where the way that the concern is presented is a distraction from the real risks that youth face.  I think that this is one of the things that I struggle with the most.  There are many youth out there who are seriously hurting, desperately seeking attention, and otherwise making visible the pain that they are in.  When we focus on the wrong things, we fail to help those youth who really need our help.

Q. Tell me about how Wikipedia is being used by students and the messages that educators are putting out about it.

A. Whenever I would ask educators about Wikipedia, they would consistently tell me that the site was inaccurate because anyone could edit it.  This refrain gets echoed by young people.  It doesn’t matter that studies have shown that Wikipedia is more accurate than the encyclopedia in many situations; the belief of misinformation is widespread.  What saddens me about this refrain is that it undermines a fantastic educational opportunity.

Unlike other content that youth encounter, the very production of Wikipedia’s material is visible. Anyone can see who edited the page, what edits they made, and how they debated what they saw to reach the final text.

The example that I give in the book is from the American Revolution. This entry had to be resolved by both British and American contributors who saw this historical event from very different perspective.  Even simple language like American references to “revolutionaries” are interpreted in British contexts as “terrorists.”  Watching the discussion page unfold over how to construct this article is itself tremendously educational.  And, yet, this opportunity is lost when we treat Wikipedia in a black-and-white way.

Q. Many of the people I talk to think that blended learning and one-to-one use of tablets or laptops will become the norm in American schools within a few years.How do you think this will affect the current patterns of interaction and behaviors that you observe among teens online?

A. It all depends on how devices are introduced and how they are contextualized pedagogically.  Technology won’t determine outcomes, good or bad.  But how teachers use that technology can.

The schools that I visited with 1:1 computer use had all introduced the machines very intentionally and with tremendous thought on the part of educators and parents.  Social dynamics that shape classrooms definitely spilled onto the online interactions too, but when teachers saw this as an extension of the classroom, they treated what they saw online as such.  What’s challenging to me is that I’m not at all sure how representative these early adopters are of what’s to come.

Q. Why are the social activities, both on and offline, that seem worthless or counterproductive to adults so compelling for teens?

A. Adults underestimate the value of hanging out. And yet, hanging out is where youth develop a sense of the world and their place in it. It’s where they learn to negotiate peer dynamics and social status. So much learning takes place when teens hang out but because it is not formal education, adults don’t recognize it as such.  Teens want to hang out because it’s fun but I think adults need to recognize that hanging out is educational too.

Guest Post: Khan Academy wants to help get kids ready for college

Note: Through March 27th I’ll be sharing this Digital/Edu space with some excellent professionals in the area of learning and innovation for one weekly guest post. In addition to bringing new voices and ideas to our readers, this will help me as I finish the first draft of my forthcoming book, The Test, on the past, present and future of testing in public schools, to be published by Public Affairs in 2015. This guest post is from the team at Khan Academy, the nonprofit that combines hundreds of free instructional videos with an adaptive learning platform and has millions of users per month.


Khan Academy’s landing page for new college resources.

Khan Academy’s landing page for new college resources.

At Khan Academy we’re making an increasingly concerted effort to create resources specifically designed to help students who want to go to college.  College hopefuls of all ages have always used us to refresh their math for school.  We recognize, however, that students also need help understanding what math they’re expected to know when they get to campus and the drawbacks of not having those foundations.

There is a big issue facing today’s college hopefuls. The gaps they have accumulated in their math knowledge over the course of K-12 often get in the way of a successful transition.  Two out of every three students are not prepared for college-level math courses. These students end up having to take extra math courses, which means extra student loans and a longer path to graduation.  For many this extra cost and delay becomes an insurmountable barrier.  Low graduation rates in the US reflect this, with as few as 46% of students who start college actually obtain a college degree.  The tough part in all of this is that students who aren’t able to graduate are locked out of a lot of today’s most exciting careers.


Two out of every three students are not prepared for college-level math courses. 

Given our mission to provide a free world-class education for anyone, anywhere, we’ve been hard at work creating new resources to make it easier for students to find and fix gaps in the math they need for college. So, when the White House called to invite Khan Academy to help students in Higher Ed reach their potential, we were really excited. We had already been creating new college prep materials, and saw the opportunity to make an even bigger impact by joining forces.

At the White House’s Expanding College Opportunity summit on January 16th, we had the opportunity to connect with many leaders in Higher Ed to share ideas and coordinate our efforts to help student success. Sal announced the launch of our college prep initiative, which you can check out here. We’re building up resources for this new section of our site to provide college study aids that will help students prepare for math placement tests and courses this Summer.

Educational technology is at the point today where personalized learning and individual instruction are actually feasible practically and financially. Gaps in knowledge are personal, and so is fixing them. One student might have moved school districts and missed geometry, while another was out sick the week they covered exponents.   In the past we’ve needed to move students through curriculum as a group because it’s difficult to know each student’s individual learning needs.  On Khan Academy, we’re capturing student progress in real time.  Faculty and students alike get immediate and granular feedback on where they are struggling and where they’re moving ahead.  Online resources like this are allowing educators to create courses where students move through materials at their own pace, they can skip the modules they know and work only on what they need to review.  It’s literally changing what’s possible.

Already, Khan Academy’s free resources are helping college hopefuls. Our team hears stories every day about students at every age using us to get ready to go to college.  We were particularly inspired by this young man, whose story was captured on the Humans of New York blog.


Moawia Eldeeb, student at Columbia University, as pictured on Humans of New York blog.

Moawia Eldeeb, student at Columbia University, as pictured on Humans of New York blog.

“I was born in Egypt…The first time I went to an actual school was middle school, but the whole school was in one classroom, and I was working as a delivery boy to help the family. It was illegal for me to be working that young, but I did. When I finally got into high school, my house burned down. We moved into a Red Cross Shelter, and the only way we could live there is if we all worked as volunteers. I got through high school by watching every single video on Khan Academy, and teaching myself everything that I had missed during the last nine years. Eventually I got into Queens College. I went there for two years and I just now transferred to Columbia on a scholarship.”

We would find our efforts well worth it just to impact one student’s life. But what’s incredibly motivating for our team is that we hear of stories like his every single day.

The new student resource center on Khan will be launching soon and building up over the spring.  But students can get started with our college prep resources today.

To check out more information on the White House’s Expanding College Opportunity initiative, click here.


« Newer PostsOlder Posts »