The White House, the College Board, Bank of America, and Comcast.
What do they have in common? They’ve all announced partnerships with Khan Academy, the nonprofit provider of free educational videos and analytics tools, in the last 12 months.
These partnerships combine various aspects of cash, content, and distribution. But in each case, the stodgier established player is borrowing a little bit of Khan Academy’s halo. Maybe it’s because of their nonprofit status, maybe it’s Sal Khan’s geeky yet approachable personality, maybe it’s because the content is really pretty great and useful, but Khan Academy is the rare ed-tech company that has managed to reach millions of people while gaining a high media profile and transcending the MOOC backlash. They remain dedicated and associated in the public’s mind to providing free high quality educational resources to the world using technology. And so when a technology company wants to burnish its association with low-cost internet access; or the White House wants to update the image of college access, or the College Board wants to democratize test prep; or a big bank wants people to not hate them, they turn to one guy with a digital whiteboard and a camera on a tripod.
Sal Khan talks to David Coleman at the College Board
So it’s pretty clear what these organizations want out of KA. But what does KA want with them?Of course, the nonprofit needs cash to continue. But they’re also at risk of diluting their brand image, independence, and mission every time they ink a deal. The Comcast partnership, in particular, has raised eyebrows as it relates to net neutrality–an Internet provider appearing to favor a certain type of content.
I spoke to Monica Tran, Khan’s Product Strategy Lead, for insight on what KA is thinking as it enters into these partnerships.
“We are in a privileged position — we have a lot of interest from potential partners,” she said. “Given the fact that we’re pretty small and lean, we are highly thoughtful about what these partnerships can do for our learners.” She mentions that in the case of the College Board partnership, 70% of KA’s users are under 17, so these test prep resources are potentially quite appealing for that audience. And there’s a synergy with their college prep tools, which come out of the White House relationship.
I believe in the idea of giving learners what they want. But by aligning itself with SAT prep and college prep, Khan Academy is tethering itself to the educational establishment, diverging from its original, revolutionary vision: to give people a path to learn and demonstrate mastery without needing institutions as intermediaries.
The big news coming out of SXSWEdu Wednesday was the New SAT. It even has a hashtag: #NewSAT.
The changes to the test taken by millions of students each year, which will start in 2016 with today’s 9th graders, are, at first glance, puzzling. The vocabulary section will be simplified, with fewer “gotcha” words. The math section will go into greater depth on fewer and more predictable topics. The reading section will involve more nonfiction and technical selections, including charts and graphs, and will be more predictable as well, with greatest-hits selections from US history like the Declaration of Independence and I Have a Dream.
Oh, and the essay section is now optional. After generations of testing, research has found that essay scores on the SATs are not predictive of anything important, and college admissions officers couldn’t agree on whether it was useful.
There are two big goals with the new SAT, as College Board director David Coleman had it at today’s announcement. At first blush, I don’t think it accomplishes either of them.
The first goal is to better align with what students are learning in high school and what they need to know in the workplace. Here’s a point-by-point comparison between the changes on the test and the Common Core curriculum, which College Board director David Coleman was very involved in designing–for example, less literature, more nonfiction.
Problem: the Common Core hasn’t been officially implemented yet in most states, yet it’s stirring up quite a bit of backlash. Nor was it tested or piloted. Who knows where it will be in five or 10 years? Will the test change again then?
Problem #2 : Seriously, you made the essay question optional? When verbal communication is the number one skill that employers look for? If you can’t devise a free response question that is useful or predictive, maybe you are not trying hard enough. Look at your own AP English exams or the UK GCEs if you need inspiration.
Employers aren’t clamoring for students who can pick the right answer from a given list, especially when the new SAT eliminates the penalty for blind guessing.
The second, and laudable goal is to make the exam more equitable by eliminating the influence of costly test preparation companies (which don’t, by the way, actually raise scores very much). The new test is designed to be both simpler to study for and harder to game. The College Board is also partnering with Khan Academy to provide free digital test prep–more on that soon so watch this space.
If you really want to make college admissions more equitable, here’s an idea: try getting rid of the SATs. Recently William Hiss of Bates College published a study looking at outcomes for 123,000 students and alumni at a wide range of institutions. These institutions belong to the growing group of 800 out of the nation’s 3000 colleges and universities that make the SATs optional for admission.
Most people who don’t submit their SAT scores, not surprisingly, have poor SAT scores. They are more likely to be first-generation college students, minorities, Pell Grant recipients, women and students with learning differences. Schools who go test-optional tend to get more diverse students as a result.
The bottom line: ”in a wide variety of settings, non-submitters are outperforming their standardized testing.” Hiss found few significant differences between students who submitted SATs and those who did not. ”Across the study, non-submitters … earned Cumulative GPAs that were only .05 lower than submitters, 2.83 versus 2.88. The difference in their graduation rates was .6%.”
Maybe the next step in the #NewSATs should be making the whole thing optional.
On my first day in Austin I had a terrific hourlong conversation with Cameron Evans, Chief Technology Officer for Microsoft Education. He had a lot of candid observations and great lines: “The vast amount of data in our education system can be used for good, and also for bad actors and bad reasons,” and on the need for professional development and parent education around new learning technologies: “You don’t want to be in a situation where you give people a library card but they can’t read.”
Toward the end I took the chance to ask him about a controversial point. It is an article of faith among many concerned about education that the extensive philanthropic influence of the Bill & Melinda Gates Foundation, headed by Microsoft founder Bill Gates, is deployed directly in service of the business interests of the Microsoft Corporation. When it comes to schools, where do the interests of one end and the other one begin? (Disclaimer: The Bill & Melinda Gates Foundation is among the many funders of The Hechinger Report, which publishes this blog.)
Evans greeted my question with a broad smile. It’s one that he fields fairly often.
“Bill and Melinda Gates have gone out of their way to make sure that the foundation’s work and the work of Microsoft have nothing to do with each other,” he starts, noting that the Gates Foundation often uses open-source technologies.
However, he allows, “There are times we do see eye to eye.”
Not for nothing, Evans agrees that the Gates Foundation and Microsoft share a single mission. As the foundation puts it, “Bill & Melinda Gates Foundation works to help all people lead healthy, productive lives.” ”We share that core mission: everybody should be able to participate fully in life without constraint or impediment.”
The true common ground between the company and the foundation, Evans tells me, has to do with education as a source of human capital and economic productivity.
“We’re not producing enough human capital capacity in higher ed and K-12. We have an obligation as a corporation to help America be the dominant player in international competitiveness. We should be very clear about this. This is a national issue for a global company. It’s not something we have to apologize for or defend. If the Gates foundation shares that sentiment, so does Intel, so does Lumina, so do MacArthur and Carnegie.” He mentions Microsoft’s legislative advocacy on behalf of the H1B visa for foreign-born technology workers as a sign of their interest in global human capital.
I point out that he’s defaulted to talking about Microsoft’s interest in education from a corporate social responsibility standpoint. This is a party line in any ed-tech conversation, and I wish that corporations would drop it. We all know that Apple, Google, and Microsoft are not just in the education business for the love of adorable, ethnically & socioeconomically diverse children. If they’d be more transparent about where they see the opportunities, we could have an open and honest public discussion about the legitimate role of private actors in providing this public good.
So Evans says this: Microsoft’s education business strategy is not primarily about hardware–a pragmatic attitude since they’re losing out on hardware.
“Some people think our sole goal is to get kids hooked on our products,” through deployments of items like Surface tablets in public schools, he said. “That they’ll be gateway products, students will be hooked for life. The consumer market today is far more dynamic than that. IT doesn’t decide what technology you use, whether in school or out of it. Consumers are choosing, not school districts, not anybody.”
What IS it about, then? Reading between the lines, by being a strong voice in workforce development and training, Microsoft helps ensure the the continued dominance of Microsoft software programs like Word, Excel, PowerPoint, and Access across the transition to mobile access in the global workplace. “As a technology supplier to governments, hospitals, individual corporations and entrepreneurs we have a responsibility to build capacity,” Evans puts it. Microsoft has 1.5 billion users on the planet; only 55 million are in the US.
Secondly, on a much smaller scale than that global strategic interest, Microsoft is developing educational data and reporting systems that aim to give students, schools and parents embedded, formative, actionable, continuous feedback and insight about student growth and progress–and to give teachers and administrators performance feedback for professional development purposes. Evans was at SXSWEdu to promote this idea. “A teacher needs data on the student sitting in front of them right now. We need sophisticated, but simple tools to be able to communicate across the board, what is this data, and what decisions do I need to make that are different.”
The Gates Foundation, of course, cares a lot about data, reporting, and teacher quality as well. And here’s the rub: It can be difficult to trust any kind of separation between the interests of the foundation and the corporation because in many cases, not necessarily nefarious ones, they do in fact coincide.
At last year’s conference Amplify announced its own custom-built educational tablet with great fanfare. Since then it’s had a rocky rollout with a recall in Guilford County, NC due to broken screens and melted chargers. Amplify is reimbursing the county to the tune of $4.8 million in devices, services and cash.
This digital curriculum has been half of Amplify’s two-pronged strategy from the beginning, now it’s part of a do-over for its brand image. It is device-agnostic; it can run on iPad, Kindle or PC and in some cases smartphones. Amplify is launching with a Common Core-aligned English Language Arts curriculum for 6th, 7th and 8th graders, to be followed soon by social studies and science. It will be available this fall, starting at $45 per student.
The curriculum is a good example of what is coming, likely to most classrooms in the next decade, with the convergence of textbooks, materials and to some extent assessment in the digital format. As Joel Klein, former New York City schools superintendent and Amplify’s CEO, points out in our interview, the Common Core, as controversial as it is, creates both a great need for new materials and a unified marketplace for companies like his to create them.
“The NEA recently came out echoing what a lot of people have been saying –the Common Core is hard. Teachers are going to need the support. You need immersive content that helps kids engage better in the learning experience. What we’re doing, and I say this in sincerity, I think it’s really unique, different, and it’s going to have a profound impact on the debate going forward.”
The Amplify ELA curriculum, explains president of Amplify Learning Larry Berger, was designed with three key metrics in mind: to get middle school students to do three times the typical amount of reading (which only gets you just above an hour a week), to write three times more pages (around 200, total, in 7th grade), and to get three times as much feedback both from the program directly and from teachers.
The lessons include a reading mode with highlighting, notes and vocabulary features, a writing mode with various word processing tools like a word counter, and a wide selection of digital multimedia content, including a library of 300 ebooks and games.
For example, students can watch actor Chadwick Boseman, star of last year’s Jackie Robinson bio-pic, dramatizing an excerpt of Frederick Douglass’s writings; or an animated version of Edgar Allen Poe’s The Raven, commissioned from an Oscar-winning animator with a voice performance by a Tony-winning actor and critical input from a Poe scholar at the University of Alabama. The resulting moody, black-and-white cartoon is part of one of the curriculum’s “quests,” a multi-day creative lesson plan that casts students as detectives seeking to solve Poe’s murder by reading his best-known works.
In addition to the official curriculum, Amplify is introducing a suite of 40 computer games designed by independent game designers. There is a World Of Warcraft-like universe called Lexica where students gain powers and points for their characters by reading books and playing grammar games, and a group of science games that bring to life processes like digestion. The games are designed to enrich and complement the curriculum, to compete for students’ free time, although some may also be assigned as homework.
All of this digital content, in my brief peek, appears high-quality and engaging. It’s designed with great attention to scope and sequencing. Berger points out that unlike a textbook publisher, Amplify tests and tweaks every lesson they develop on real students, first on a group of middle-schoolers who come to the headquarters after school (rewarded with pizza and gift cards) and then on students in classrooms.
The aspect of the digital convergence that may raise eyebrows is that products like these allow teachers, if they wish, to essentially put their classes on autopilot. Unlike a textbook, which may offer a few questions at the end of each chapter, Amplify, Berger told me, is designed to “orchestrate” or “choreograph” every five minutes of instruction, and often delivers it, along with feedback, directly to the student so the teacher doesn’t have to. This is a level of heavy lifting that some teachers will welcome taken off their shoulders; others, who prefer to curate classes on their own, may see it as intrusive or even a Trojan horse for increasing class sizes in a quest for efficiency. In either case, implementation and professional development will be everything.
South by Southwest Edu , this March 3-6 in Austin TX, has grown into a several-thousand-person big-tent conference about all things innovative in education: not just technology and entrepreneurship but new approaches to issues like inequality and social and emotional learning. “The same way seismic shifts in the music, film and technology industries fueled the growth of SXSW, the convergence of technology and innovation has set the stage for tremendous disruptions in the field of education,” Ron Reed, the director of the festival, told us recently. So it’s a natural place for Digital/Edu and the rest of the Hechinger Report to be.
After attending SXSW Interactive, Music, and/or Edu for five of the last six years, here are my top tips:
*The chance hallway encounters, meetups and party conversations tend to be far more valuable than *most* sessions (not ours of course). So leave room in your schedule for serendipity!
*Twitter is the best way to follow all the discussions and figure out where to go next. This year most sessions have their own #hashtags.
*Carry a water bottle, phone charger and energy bar, and wear comfortable shoes.
You can meet some members of the Hechinger Report team at the following sessions:
Tuesday, March 4, 12pm: Can the Liberal Arts Survive in an Age of Innovation? with Hechinger Editor in Chief Liz Willen, Scott Kinney, president of Capella University, and more.
Wednesday, March 5, 12pm: Building a Better Teacher, with Jackie Mader, Hechinger Staff Writer, Kate Walsh of the National Council on Teacher Quality, and more.
Thursday, March 6, 9am: Hardwiring the Brain: Better, Faster, Stronger, with me and a bunch of brainiacs including Jan Plass, NYU professor of Digital Media & Learning Sciences and co-director of the Games for Learning Institute.
In addition to the above sessions, here’s some more we really want to get to. Leave your top picks in the comments!
Monday, March 3, 3pm: Reclaiming the Promise Through Community Schools, a plenary discussion on the model of schools with wraparound services, moderated by AFT President Randi Weingarten.
Monday, March 3, 7pm at the Alamo Drafthouse: Ivory Tower, a screening of a new documentary on the future of higher education.
Tuesday, March 4, 10:30am: Data Privacy: What Parents DON’T Know Can Hurt You – an overview of a hot issue in the news, featuring Cameron Evans, Microsoft CTO.
Tuesday, March 4, 12pm: Inspiring Advocacy through News and Digital Media with folks from Do Something, Channel One, and NBC learn.
Wednesday, March 5, 12pm: Wendy Kopp: There Are No Silver Bullets in Education --Teach For All’s CEO, an increasingly controversial figure (aren’t they all?) argues there are no silver bullets to solving educational inequality (well, who told us there were?).
As of last June, 46% of recent college graduates were underemployed, meaning they had jobs that didn’t require a college degree, while a further 6% were unemployed. For employers, meanwhile, hiring young people means dealing with a lot of churn and an extensive training process; the average tenure in a first job is just 18 months.
With the cost of college and student loan debt rising, there’s increasing urgency to help graduates find fulfilling careers that fit their skills. A recent startup, Koru , is addressing itself to this problem.
“A lot of young people feel like they’ve been sold a bill of goods,” says Josh Jarrett, the founder and CEO of Koru. “One young woman told us, ‘I never felt more abandoned than the day I graduated college. I was like Now What? I was on this 16 year long conveyor belt, I go to the end, and I was dropped off.”
For almost eight years, Jarrett supported other endeavors as a higher education program officer for the Gates Foundation. He made the decision to take the leap into entrepreneurship because of a strong belief in the power of both/and: both liberal arts education for its own sake combined with extremely practical hands-on experience.
The Koru model is for young people, from about the junior year of college through a few years after graduation, to work on a real challenge set by an employer: for example, do field research to determine the best way of reaching Millennial customers. To provide better context for their immersion in the world of work, they go through an MBA-lite curriculum covering concepts like design thinking and business analytics. The model is blended learning, using the same kinds of collaboration software that young hires are likely to encounter in the office, like Basecamp and Asana.
Koru piloted its first, 10-day programs in Seattle earlier this year with companies including outdoor outfitter REI and online retailer zulily. They plan to launch a full set of four-week-long programs in Seattle over the summer, in San Francisco by the fall. They just announced a set of partner institutions that plan to offer Koru as a career services option. These are a well-known group of universities and liberal arts colleges: Bates, Brown University, Colorado College, Connecticut College, Denison University, Georgetown University, Mount Holyoke, Occidental, Pomona, University of Southern California, Vassar, Whitman, and Williams. “There will be a cost share between participants and employers,” Jarrett says–participants or their parents, and possibly their colleges, will pay part, and employers will contribute a placement fee if they end up hiring the participants.
Koru is part of a wave of what I call “educational popups” that seek to bridge the worlds of formal education and various industries. They are most numerous in the technology and design industries–Hacker School, General Assembly, and Hyper Island are all good examples.
Advocacy group Common Sense Media held a summit in Washington, DC on Monday as part of a national campaign on the highly contested topic of student data privacy.
A recent study by Fordham University Law School found that as schools and districts adopt cloud computing services, they are transferring student information to third-party providers, often leaving it open to data mining and commercial purposes such as reselling and ad targeting. These services may be in violation of federal law. These agreements allow vendors to do whatever they want with student demographic records and other personal information. Of the districts studied, fewer than 25% of the agreements between districts and vendors specified the purpose for disclosures of student information, and fewer than 7% restricted the sale or marketing of student information by vendors in any way. And that is to say nothing of the risk of hacking or other security breaches.
There are a number of attempts in the works to establish better guidelines for the $8 billion educational software industry. The Software&Information Industry Association, a trade group, yesterday announced a list of best practices for agreements between software groups and schools:
That data should be used only for educational purposes, that its use should be fully disclosed and transparent and full consent obtained from families, that all reasonable security procedures should be followed and schools be notified in case of actual data breaches.
Even as the industry is taking baby steps to govern itself, lawmakers are converging on a solution with more teeth. California State Senator Darrell Steinberg just introduced a bill in that state enforcing some of these same principles: educational purposes only, encryption and deletion of data. Massachusetts Senator Ed Markey plans to do the same at the federal level.
At the Common Sense Media event, according to the lively discussion on Twitter, industry representatives like Cameron Evans of Microsoft and Joel Klein of Amplify argued that a rush to legislate might cause more problems than it solves. Best practices for data privacy and security continue to evolve as the technology does. The large-scale use of cloud computing and web-based data storage itself dates back only to the mid-2000s. It is difficult for the law to catch up. Also, while contracts may specify “educational purposes only,” the nature of the beast in ed-tech is that a large source of educational innovation is coming from for-profit startups whose involvement with the day-to-day experience of teachers and students is becoming increasingly intimate, if not intrusive. In practice the line between educational and commercial purposes may be somewhat blurry. As Katherine Varker, Associate General Counsel, McGraw-Hill Education, asked at the summit: ’Where does targeted advertising end and personalized learning begin?’
Note: Through March 27th I’ll be sharing this Digital/Edu space with some excellent professionals in the area of learning and innovation for one weekly guest post. In addition to bringing new voices and ideas to our readers, this will help me as I finish the first draft of my forthcoming book, The Test, on the past, present and future of testing in public schools, to be published by Public Affairs in 2015. Today’s guest poster, Joshua Starr, is superintendent of Montgomery County Public Schools , a national leader in the debate over 21st century skills, and @mcpssuper on Twitter.
School districts across the country are in the middle of the very difficult work of revising and reshaping their curriculum to align with the Common Core State Standards. (CCSS) And this work is worth doing. The CCSS provides our educators an opportunity to improve teaching and learning by organizing around a uniform set of high expectations. Moreover, consistency from state to state and district to district is essential if we truly want to improve public education in the United States.
In Montgomery County (Md.) Public Schools (MCPS), where I am superintendent, this work has been going on for several years. A CCSS-aligned curriculum is fully rolled out in elementary grades and is well underway in middle and high schools. Most people see the benefit, but there are growing pains. Teachers need more time for professional development; not everything works the way the folks in central office think it will; and many parents have questions about our new approach. And there remains tension about what the Common Core-aligned assessments will look like, what the role of AP, IB, SAT and ACT will be relative to PARCC, and how they will be used to support the improvement of instruction.
Good standards and assessments are extremely important to improving education for our students. But they are only a part of the answer. If we are truly committed to a new definition of “college and career ready,” we need to develop an accountability framework that supports that definition and it needs to look a lot different from No Child Left Behind (NCLB).
For more than a decade, nearly every school district in the United States has aligned its work to meet the “adequacy” of NCLB. Despite lofty mission statements about serving the “whole child” and developing “lifelong learners,” every school had a clear directive: Make AYP (Adequate Yearly Progress). This was true in MCPS, although the district did effectively hold itself to a higher bar. MCPS encouraged more students to take Advanced Placement (AP) and International Baccalaureate (IB) Classes, paid for every 10th grader to take the PSAT, and encouraged students to be well prepared for the SAT, even offering prep classes in the schools. The results have been impressive.
About two-thirds of our graduates take at least one AP or IB test and more than half of our graduates earn a “college-ready” score on an AP or IB exam. MCPS has one of the highest SAT scores of any large district in the nation—an average of 1648 for the Class of 2013. And some of the biggest growth in these areas has been among African American, Hispanic and economically disadvantaged students.
However, all of this achievement has not necessarily translated into postsecondary success. About fifty percent of MCPS graduates earn a bachelor’s degree within six years of high school graduation. That is better than the state of Maryland or the country as a whole, but it is not reflective of our stellar K-12 results. And the achievement gap in college completion rates is stark: A white MCPS graduate is twice as likely as a black or Hispanic graduate to enroll in and successfully complete a bachelor’s degree.
This is not uniquely a problem in MCPS. Every school district—especially the larger ones—is dealing with these types of results. At the same time, industry leaders are telling us that our graduates don’t have skills like perseverance, teamwork and creative problem solving that are so important in the 21-century workplace.
MCPS is one of 15 school districts that are a part of the Large Countywide and Suburban District Consortium. The Consortium includes the leadership of successful districts that serve large, diverse student populations. In a recent meeting with a bipartisan group of U.S. Senators, we laid out the framework for a federal accountability system that will support meaningful outcomes for our students. I, and my colleagues in the Consortium, believe a reauthorization of the Elementary and Secondary Education Act (aka NCLB) must have these key elements:
- Establish 21st-century outcomes as the goal of all major education laws and policies
- Require academic content standards and student academic achievement standards aligned with college- and career-readiness outcomes
- Promote the development and use of high-quality assessment systems and other valid measures of college- and career-readiness
- Promote innovation and continuous improvement in schools and districts
- Ensure equitable access to effective educators
- Promote transparency, engagement, and shared accountability through appropriate reporting of data and information
I believe there is a need for a strong and clear federal accountability system. NCLB had major flaws, but it got us talking about data in a way we never have before. As we consider the next generation of federal accountability, we need educators, industry leaders, higher education officials and policy leaders to come together to help define college and career readiness in a way that includes more than just test scores and graduation rates. We must take the opportunity to provide our students—and those that teach and advise them—with a coherent definition of success that will enable our children to thrive in the future that they want.
Being smart isn’t all you need to succeed.
Non-cognitive or meta-cognitive skills such as self-awareness, self-control, empathy, communication and cooperation are as important or even more important to long-term success than cognitive factors like math and reading ability. The evidence is undeniable, but US school assessment and accountability systems are currently limited to measures of cognitive skills alone. If schools don’t measure non-cognitive skills, it is difficult to prioritize them.
There are several emerging attempts to measure non-cognitive skills, aka Mindsets & Essential Skills and Habits (MESH) in schools. Most use the simplest instrument available: self-reporting surveys.
- The OECD gives the Program in International Assessment or PISA to about half a million students around the world. It’s widely publicized for its rankings on math and language achievement. But in addition to the two hours of word problems or essay questions, PISA students also complete a 30-minute questionnaire asking about things like self-efficacy, self-esteem, student-teacher relationships, and the school climate, making it one of the largest attempts to measure non-cognitive skills in the world. An infographic published on the website Buzzfeed based on PISA data sorted countries about evenly into four quadrants: happiest and least happy kids, and high and low test scores. Korea stands out for its miserable kids and high scores, Singapore has the happiest kids with the highest scores, Qatar and Argentina have sad, low-scoring kids and Peru and Indonesia are whistling away their weak scores. (The US falls into the unhappy, low-scoring bottom left quadrant.)
School leaders around the world are using PISA findings on the emotional health of their students. In Korea, in February 2013 the new president Park Geun-hye announced that the happiness of citizens would become a national priority, and the Ministry of Education has taken this up as a goal.
- In 2012 almost 500,000 students across the US in grades 5-12 completed the Gallup Student Poll, which asks about students’ hope, engagement, and well-being. Over the last 10 years, scores on the hope survey did a better job of predicting college persistence and GPA than high school GPA, SAT, and ACT scores. In the most recent survey, about a third of students were “struggling” and 43% were either “not engaged” or “actively disengaged” with school.
- YouthTruth is a 20-minute online survey that’s been taken by about 250,000 students in grades 3 through 12 at 300 schools, covering school climate issues like bullying, classroom engagement, as well as optional topics like student motivation and grit. Schools are using YouthTruth to gather feedback not only on the school as a whole but also on individual teachers.
- The State of California’s Department of Education, starting in 1998, has administered the largest statewide survey program in the nation looking at “resiliency, protective factors, and risk behaviors”, made up of the Healthy Kids Survey, the School Climate Survey, and a parent survey. In 2013-14 they updated the survey to track learning engagement, students’ connectedness to school, and a new optional social-emotional health section covering empathy, self-efficacy, self-awareness, persistence, emotion regulation, gratitude, ‘zest,’ and optimism. The social-emotional health section for middle school students asks them to identify with statements like “each day I look forward to having a lot of fun,” and “I can do most things if I try.”
Once schools have adopted one of these surveys, the next step is to incorporate them into decisionmaking processes. The Montgomery County Public Schools in Maryland has incorporated the Gallup hope survey into its strategic planning processes and goals. A group called Transforming Education is advising the California Office to Reform Education, a group of eight big-city school districts in California that has obtained its own separate NCLB waiver. They are setting up one of the first large-scale accountability systems that places non-cognitive measures as peers to test scores. Schools within these districts will be evaluated on 60% conventional test scores, 20% school culture/climate measures such as suspensions, expulsions, and bullying, and 20% social/emotional learning scores.
The idea of attaching stakes to self-reported measures of happiness is fraught with peril. Surveys that are perfectly valid when administered as research tools may be distorted when they become the basis for school and teacher ratings. It’s far easier to game a self-reported survey than, say, a math test. To overcome these problems, researchers such as Patrick Kyllonen at ETS are trying to develop more sophisticated measures, such as asking students to provide a solution to a hypothetical interpersonal problem, demonstrating their emotional wisdom. Another approach is to combine multiple measures such as suspension rates and teacher reports with student survey data to get a more complete picture of the climate at a school. Regardless of the path chosen, we are headed towards a world where students’ emotional health and wellbeing gets full attention commensurate with its importance in determining their futures.
Note: Through March 27th I’ll be sharing this Digital/Edu space with some excellent professionals in the area of learning and innovation for one weekly guest post. In addition to bringing new voices and ideas to our readers, this will help me as I finish the first draft of my forthcoming book, The Test, on the past, present and future of testing in public schools, to be published by Public Affairs in 2015. Today’s guest poster, Justin Reich, is the Richard L. Menschel HarvardX Research Fellow and a Fellow at the Berkman Center for Internet & Society. He writes the EdTechResearcher blog for Education Week.
Framing MOOC Research: How Comparisons Shape Our Judgments
Last month, my colleagues and I on the HarvardX and MITx research teams jointly released a series of reports about the first 17 courses launched by HarvardX and MITx on the edX platform. We released a synthesis report with findings about all of the courses, and then 15 additional reports examining individual courses in more detail.
We tried to provide the public and our internal stakeholders with data that instructors can use to create better courses and that people can use to judge the state of the enterprise. We are keenly aware, however, that our data don’t have a single story to tell, and how people read our research depends upon how they approach the subject.
Here are two sets of facts, two possible frames for thinking about HarvardX courses:
Set 1: In September 2009, Boston’s WGBH published to YouTube a series of 12 videos from Michael Sandel’s Justice course at Harvard University. Each hourlong video is a combination of lecture from Sandel and facilitated Socratic dialogue among the hundreds of students who take his class every semester.
On YouTube, the first video has been played almost 5 million times. The second video over a million times. The next ten videos have been played about 300,000 to 400,000 times each. All told, the series has about 10 million views.
As a back of the envelope calculation (ignoring people who watch videos multiple times, who don’t finish videos, etc.), it seems unlikely that more than 6% of people who started the series finished the whole thing.
Set 2: Researchers at the Community College Research Center at Columbia University’s Teachers College have done some very interesting work examining online courses in community colleges.
Di Xu and Shanna Smith Jaggers have recently published studies that show that online course completion rates in two large systems in Virginia and Washington are lower than face-to-face course completion rates. In Virginia, completion rates in face-to-face courses were 81 percent, while online completion rates were 68 percent; in Washington, the rates of completion were 90 percent for face-to-face versus 82 percent for online. While online completion rates lag behind on-campus counterparts, the vast majority of students in both conditions earn a passing grade in the courses in which they enroll.
The first set of facts concerns online media available to anyone for learning and personal growth. The second set of facts concerns structured learning experiences offered by institutions of higher education. Both are are potential frames of reference for interpreting our research about HarvardX and MITx courses, each could lead to different comparisons and different judgments.
One of the most important findings from our research is that people use materials in edX courses in all kinds of ways. The 2012-2013 MITx and HarvardX courses had a little over 800,000 registrations. A little over 43,000 people earned a certification of completion in a course. Nearly 36,000 people opened up more than half of the units of a course, but did not earn a certificate. Over 450,000 people viewed less than half of the units of the course (without earning a certificate), and nearly 300,000 people who registered for a course never entered the courseware at all.
The figure below is a scatterplot of all 800,000 registrations. On the y-axis is the student’s grade in the course, and on the x-axis is the percentage of the units (or chapters) in a course that the student opening. As you can see, nearly the entire possibility space is full of points. There are people who are completing every snippet of course content, people who are auditing courses and ignore assessments, people dabbling in a fraction of the course, and people who are never showing up at all.
How should we judge these findings?
One thing we can do is compare these patterns of behavior to the patterns of behavior in community colleges. In community colleges, we are keenly concerned with completion rates. Courses are expensive, and students who fail and drop out not only miss the benefits of learning and certification, but they also lose the money they invested in enrollment. We might make the comparison that only about 6% of HarvardX and MITx registrants finish a course, but in Virginia 68% of online community college registrants finish a course.
We could also compare the online learning content hosted on edX to the PBS Justice videos hosted on YouTube. In online content, we expect to see a funnel of participation. We expect, that on any website, there will be some number of people who navigate to the site, a smaller number of people who register, a smaller number of people who participate in some way, and then a smaller yet number of people who engage the deepest possible ways. When we compare the HarvardX version of JusticeX with the PBS version of Justice, we find very similar patterns of participation.
Patterns of persistence and completion in edX look pretty typical when compared with engagement funnels of online media, and they look pretty lousy when compared to community college retention rates. So which is the right comparison?
Faculty intent should play an important role in deciding the right frame of reference, the right yardstick, for judging open online courses.
Some MOOC faculty are primarily interested in sharing their ideas and relatively uninterested in certifying learners’ competency. Michael Sandel, through PBS Justice, justiceharvard.org, and JusticeX, is primarily interested in helping more people learn more about moral reasoning rather than certifying people at a particular level of skill or knowledge about moral reasoning. If he’s trying to maximize the number of learning experiences that people have, of both lighter touch and deeper engagement, then the frame of online learning media seems to be a fairer point of reference for judging the public impact of JusticeX.
By contrast, many of Sebastian Thrun’s courses at Udacity, especially his pilot programs with San Jose State University and Georgia Tech, are explicitly designed to replace or complement typical courses of study in higher education. Familiar higher education settings make a more sensible frame of reference in understanding these efforts.
As new forms of online learning proliferate, no doubt there will be even more sensible ways of contextualizing and framing new courses and new platforms for learning. As we debate how open online courses might reshape the future of higher education and lifelong learning, its worth paying close attention to how the points of comparison that we choose frame our interpretations and judgments.