I’m currently working on a book about the past, present and future of assessment. For the “future” bit I get to talk to researchers like Ryan Baker at Columbia. He’s spent the last ten years working on systems that gather evidence about crucial parts of the learning process that would seem to be beyond the ken of a non-human teacher.
The basis for the observations comes from what’s called “semantic logs” within a computer learning platform, such as Khan Academy’s: Was it a hard or easy question? Did the student enter a right or wrong answer? How quickly did they answer it? How did it compare with their previous patterns of answers? The detectors gather evidence that students are gaming the system, drifting off-task, or making careless errors. They can extrapolate a range of emotional states, like confusion, flow, frustration, resistance, (which Baker calls memorably “WTF” behavior), engagement, motivation, excitement, delight, and yes, boredom.
Baker’s engagement detectors are embedded within systems currently being used by tens of thousands of students in classrooms from K-12 up to medical school. (Medical residents, he says, show the highest rate of “gaming the system,” aka trying to trick the software into letting them move on without learning anything, at rates up to 38% for a program that was supposed to teach them how to detect cancer.) His research, located at the forefront of the rapidly expanding field known as “educational data mining,” has a wide range of fascinating applications for anyone interested in blended learning.
Understanding how good these detectors currently are requires a bit of probability theory. To describe the accuracy of a diagnostic test, you need to compare the rate of true positives to the rate of false positives. The results for the “behavior detectors,” Baker says proudly, are about as good as first-line medical diagnostics. That is, if the question is whether someone is acting carelessly, off task, or gaming the system, his program will be right about as often as an HIV test was in the early 80s–0.7 or 0.8 (“fair” according to this rubric). For emotional states, which require a more sophisticated analysis, the results are closer to chance, but still have some usefulness. These accuracy scores are derived from systematic comparison with trained human observers in a classroom.
So why would someone want to build a computer program that can tell if you are bored?
- To improve computer tutoring programs. Let’s say a learning program provides several levels of hints before the right answer. You want to build something in that prevents a student from simple gaming techniques, such as pressing “hint, hint, hint, hint,” and then just entering the answer.
-To give students realtime feedback and personalization. ”I would like to see every kid get an educational experience tailored to their needs on multiple levels: cognitive, emotional, social,” says Baker. Let’s say the program knows you are easily frustrated, and gives you a few more “warmup” questions before moving on to a new task. Your friend is easily bored. She gets “challenge” questions at the start of every session to keep her on her toes.
- To improve classroom practice. Eventually as these systems become more common, “I would envision teachers having much more useful information about their kids,” says Baker. “Technology doesn’t get rid of the teacher, it allows them to focus on what people are best at: Dealing with students’ engagement, helping to support them, working on on one with kids who really need help.” In other words, though technology can provide the diagnostics for affective states that affect learning, it is often teachers that provide the best remedies.
-To reinvent educational research: This is a fascinating one to me.
“I’d like to see educational research have the same methodological scope and rigor that have transformed biology and physics,” Baker says. “Hopefully I would like to see research with, say, 75% of the richness of qualitative methods with ten times the scale of five years ago.”
Modeling qualitative factors related to learning opens up new possibilities for getting really rich answers to really interesting questions. ”Educational data mining often has some really nice subtle analyses. You can start to ask questions like: What’s the difference in impact between brief confusion and extended confusion?”
In case you’re wondering, I will clear up the confusion. Brief confusion is extremely helpful, even necessary, for optimal learning, but extended confusion is frustrating and kills motivation.
The very phrase “data mining” as applied to education ruffles feathers. It’s helpful to hear from an unabashedly enthusiastic research scientist, not an educational entrepreneur with a product to sell, about this topic. Privacy, he says, should be given due consideration. “The question is what the data is being used for,” he says. “We have a certain level of comfort with Amazon or Google knowing all this about us, so why not curriculum designers and developers? If we don’t allow education to benefit from the same technology as e-commerce, all we are saying is we don’t want our kids to have the best of what 21st c technology has to offer.”
If you’re interested in learning more, Baker has a free online Coursera course on “Big Data in Education” starting this Thursday. Over 30,000 people have signed up.
It’s a good season to learn about the use of technology in the classroom. Canvas, a learning management system-cum-MOOC platform, is running a MOOC on “digital tools for the K-12 classroom.” Simultaneously, Coursera is offering a MOOC for educators at all levels on blended learning starting this week. The course leaders are Michael Horn, education policy maven at the Christensen Institute; Brian Greenberg of the Silicon Schools Fund, a “venture philanthropy” outfit that gives seed money to start innovative new schools, and Rob Schwartz of the New Teacher Center , a nonprofit focused on “induction” and professional development for, yes, new teachers.
The blended learning course is more interesting to me because it focuses on the practice, not the tools. Participants are located all over the world:
And appear quite enthusiastic in the discussion forums, leaving hundreds of posts.
Like any good students, these participants are critically considering the source of the information.
“I’m skeptical,” writes one participant, Amanda Seal McAndrew, in a public post. “Why are you offering this MOOC? What’s in it for the sponsoring groups? So many of the people associated with your groups have business backgrounds. I’ve found a few that have education backgrounds, but it doesn’t look like they are in education anymore. Why not? I’m also curious why you don’t have a traditional educational institution as a partner. ”
“Skepticism is welcomed. If it helps, the three primary instructors in the course have over thirty years of classroom and school level experience between us,” responded one of the instructors, Brian Greenberg.
Horn, Greenberg and Schwartz clearly intend this course as a stake in the ground. They’re not simply out to explain what blended learning is from any neutral historical or sociological or otherwise academic perspective, but to argue for a particular definition, staking a rhetorical claim in a crowded, contested category. They are promoting something called “High-Quality Blended Learning” that includes, but goes beyond, the use of technology.
High-quality blended learning, they say in their first video chunks, takes place at least partly in and partly out of the classroom. It leverages technology for greater personalization and mastery-based or competency-based education.
Personalization means providing each students with the resources he or she needs, whether it’s solo practice time, group work, one on one tutoring, or multimedia presentation of information. Personalization resembles the pedagogical concept of differentiated instruction, but has a special connotation of the use of technology, rather than individual human attention, to customize an individual’s experience. From the Wikipedia entry for personalization:
“Personalization involves using technology to accommodate the differences between individuals. ..Social Network websites use personal data to provide relevant advertisements for their users. Websites like Google and Facebook are using account information to give better services. Personalization technology enables the dynamic insertion, customization or suggestion of content in any format that is relevant to the individual user, based on the user’s implicit behaviour and preferences, and explicitly given details.”
The scenario implied by the phrase “blended learning” is not that personalized learning is better than differentiated instruction by a skilled practitioner, but that personalized learning can aid and abet differentiated instruction, resulting in something more than the sum of its parts. Evidence for this view is still scant, though subjective; something I’ve dealt with in previous posts.
What of mastery-based or competency-based education? These are two more little phrases hiding a whole slew of arguments upending standard classroom practice.
Mastery-based education puts the focus on individual students mastering specific material. In a traditional class, the syllabus is tied to time. The entire class moves more or less at the same pace, completing assignments on specific due dates and sitting for exams at particular times. If you learn quickly and are bored, too bad. If you fall behind, too bad.
In the mastery-based model, students have more flexibility to stay with material until they fully demonstrate understanding. Again, this is enabled by personalization technology, but it also subtly shifts the focus from the teacher’s or class’s agenda to the individual student’s interest.
This very MOOC, like others, abides by these synchronous conventions–weekly syllabus, deadlines–for the purpose of promoting discussions among people studying the same material. Clearly there’s a tension between personalization and the productive aspects of learning in a group.
There’s a lot of food for thought in this first week’s session, with dozens of participants chiming in to give their own definitions of high-quality blended learning. I’ll be checking in with the MOOC as it goes on to see what else they come up with.
Maslow’s Hierarchy of Needs, Updated
For the past 15 years, Phil Shapiro has been taking donations of computers and setting them up for families that need them. He wrote to alert me of an issue he discovered recently when helping an Ethiopian immigrant family in Takoma Park, MD.
In its first 23 months, Internet Essentials has signed up 220,000 mostly urban households for Internet access that costs just $9.95 a month, making it the biggest digital divide program in the country. In order to qualify for the program, which also includes digital literacy training and the opportunity to buy low-cost refurbished computers, the household must have at least one child enrolled in the federal free or reduced lunch program at public school. Comcast is currently conducting a 23-state publicity tour to further tout the program. They’ve also recently increased the speeds available to Internet Essentials users, to 5 mbps downstream and 1 mbps upstream; by comparison, Comcast’s lowest cost cable and Internet package offers 20 mbps downloading speeds for $70 a month.
I’ve taken three donated computers to this family and I was expecting to get them all online with this cable modem service. Aha, but not so fast. Comcast’s telephone tech support tells me that Internet Essentials users cannot use Wi-Fi with their cable modems. Hmmm, but nowhere in Comcast’s printed literature about Internet Essentials is this limitation mentioned. And nowhere on the Comcast Internet Essentials web site is this limitation mentioned. Naturally, families who sign up for Internet Essentials get confused about this, but they are not well positioned to advocate for their needs…
The family that I was helping patiently waited for me while I talked on the phone. They could see that I spoke very politely with the tech support person. They also saw that I had reached the end of my patience.
The father regarded me with hopeful, tired eyes. He takes three buses to go to work each day. His children will not have Wi-Fi at his apartment though, despite the fact that he pays Comcast a monthly fee for broadband Internet service. They will not be able to use wireless laptops or surf the Web with tablets (unless the family pays for far more expensive 4G Internet service for tablets.)
Charlie Douglas, a Comcast spokesperson, confirms that Internet Essentials does not offer wi-fi. “A family that has a wifi modem could plug one in and that could be part of it. We don’t offer it as part of our $9.95 monthly service. We provide the cable modem which is a wired connection.” Wi-fi modems retail at about $75-$90.
Douglas says that to his knowledge, the families served by the program don’t miss the wi-fi access and don’t need it. “I haven’t heard anecdotally of this cohort of people asking for wireless options,” he says.
Phil Shapiro, who says he supports the Internet Essentials program, would disagree. He says that the Comcast customer support person told him, “Wireless Internet is not supported for Internet Essentials accounts. With Internet Essentials accounts there is insufficient bandwidth for Wi-Fi.” As an IT professional, Shapiro knew the bandwidth claim wasn’t true, but he says many poor families lacking technology expertise might believe it.
Comcast could be clearer with its customers about exactly what it’s offering. Their choice not to include, or even mention wi-fi access, which would lead to more connected devices and more usage, implies they are attempting to “throttle” or control the bandwidth being used by these lower-paying, low-income customers.
A bigger question is whether home wi-fi is really important for education. Many large school districts, such as Los Angeles, are adopting tablets for student use, which connect to the Internet only wirelessly. However, Douglas argued that wired devices are most useful for educational reasons.
“Many low-income Americans adopt wireless devices like smartphones, but those are poor tools for trying to do something like a homework assignment,” he says. “This particular program has been more focused on wired connectivity and helping to enable and facilitate a computer that would be connected with a wired connection.”
Thanks, everyone for your thoughtful responses and comments on part one of my review, linked here.
To summarize, I believe “privatization” and “corporate” are too simplistic a brand with which to lash the education-reform complex.
I believe a more subtle characterization with more explanatory power is that this is a technocratic and technophilic coalition, uniting conservatives, liberals, bureaucrats, politicians, entrepreneurs, executives, school leaders, and philanthropists–hey, even some teachers, parents and students!–in the basic conviction that schools must innovate, using technology and data.
This explanation is a little bit juicier, because there are far more people who accept the underlying technocratic/technophilic premise, even if they take it in very different directions and to different degrees, than those who swallow the case for privatization. So there’s abundant room for debate and disagreement.
Quoting myself earlier in the year:
“Is it possible to support real technological innovation in schools without supporting the privatization of schools? Can learning be personalized without destroying the public mission of schools? Do advocates of web-enabled learning necessarily stand in opposition to teachers’ unions?”
Ravitch’s answers to these questions would probably be no, no, and yes. While my answers are yes, yes, and no. Her view of technology in schools can be summarized in her book’s index:
.. technology’s replacing of, 19, 34, 250, 303″
Her book contains one reference to “blended learning” in quotes, as a Trojan horse for, you guessed it, technology replacing teachers. No references to (for example) open educational resources, connected educators, gamification, social media, or the Maker movement.
Is technology really being used to replace teachers? It’s hard to see the evidence, not in public schools. Pupil-teacher ratios in public schools have fallen steadily over the last 40 years, including from 15.9 to 1 to 15.2 to 1 between 2001 and 2011 (linking to the Google cache because NCES is down due to the government shutdown).
If “technology” is not (yet) replacing teachers, what in fact is it doing in schools? And what could it be doing for education? Here’s a couple of cases in point:
- Simply, acclimating students to the way work is done and life is lived in the developed world. This is the age of Google and ubiquitous smartphones. Why should they disappear at the school gate?
- Enabling students to share work to the open web.
- Connecting students with marine biologists or astrophysicists.
- Saving money on textbooks through Creative Commons-licensed resources.
- Customizing teaching for gifted students, students with disabilities, or other learning differences.
- Setting creative educators free to network with each other and reinvigorate their commitment to teaching and learning.
On the other hand, technology’s having deleterious effects too:
- Wasting money that could be used for wraparound services, basic supplies and, yes, maybe more teachers.
- Widening the gap between rich and poor classrooms (the digital divide).
- Exposing students’ data for the enrichment of private business.
How can this be? Well, technology is neither good, nor evil, nor even neutral. Nor is it beside the point or unworthy of serious consideration in any discussion of what can/should/must be done in education.
The technocratic/technophilic worldview does come with some characteristic and pernicious errors, which are worth investigating to root out. I believe it explains the current obsession with high-stakes standardized testing better than the “privatizer” lens. Technocrats must have data on which to base decisions, regardless of its quality, integrity, or relevance to the matter at hand.
It’s perhaps necessary to draw more specific battle lines among the terms “reformer,” “innovator,” and “privatizer.” The fact is, you can believe in locally, democratically controlled schools that are equitably and abundantly publicly funded and staffed with professional, well-paid teachers, and you can further hold that poverty eradication and other social progress around the family/maternal/child development matrix is equally important to any thing that happens under the umbrella called “school,” while also seeing a lot of merit in educational technology, innovation and data-driven decisionmaking. That’s what makes the current state of education reform so complicated and interesting.
Diane Ravitch’s Reign of Error: The Hoax of the Privatization Movement and the Danger to America’s Public Schools will undoubtedly be the best-selling book on school reform released in 2013. Ravitch’s 2011 book The Death and Life of the Great American School System was a bestseller, turning on the compelling story of her own apostasy as a former assistant secretary of education under George Bush the elder and a former supporter, now vital opponent, of testing and choice.
The new book contains no such personal story and few surprises, especially for those who have been reading Ravitch’s blog and following her on Twitter and in the media these past two years. Her argument can be summarized as follows:
1. Testing, accountability and choice together constitute a single “corporate reform movement” whose underlying aim is “privatization” of America’s public schools a la Milton Friedman; the dismantling of public schools and teachers unions and their replacement with charter schools and vouchers to the direct enrichment of a variety of private actors.
2. Contrary to the corporate reform narrative of failure, American schools are actually doing just fine.
3. “We must work both to improve schools and to reduce poverty, not to prioritize one over the other or say that schools come first, poverty later.” Racial segregation, like poverty, should be tackled directly.
4. Improving education means going back to basics. Chapters 21-32 are a long list of solutions, ranging from prenatal care for poor women, early childhood education, and wraparound social services to small class sizes, a broad and varied curriculum, extracurriculars, and better training and professional development for teachers. Restore local and democratic control of schools. Ban for-profits and charter chains. Replace high-stakes standardized tests with a “responsibility” model perhaps including national inspections.
It’s hard to argue with #3 and most of #4. #2 is curious. Ms. Ravitch spends a chapter attacking standardized tests yet turns around and points to one of the tests, the NAEP, as evidence of student progress. The NAEP is probably a better test to look at than most state tests, since it is the same everywhere and is given to random samples of students. Yet as Daniel Koretz, author of Measuring Up:What Educational Testing Really Tells Us, recently explained to me, the overall increase in student testing will alone usually cause scores to go up over time; when applied to IQ this is known as the “Flynn effect.” Similarly, Ravitch points to rising high school graduation rates as a sign of progress at a time when schools were being extensively scrutinized and judged on these same metrics. Yet in New York City, the nation’s largest public school district, even as graduation rates were rising, last year four out of five high school graduates required remediation when they got to the city’s community colleges.
Finally, if corporate school reform is such a disaster, how is it that schools are getting better?
Anyway, I agree with Ravitch that we have to look beyond metrics like test scores or even graduation rates when deciding whether schools are in fact doing just fine. There’s a reason that the reformers’ calls of “failure” and “broken” resonate so broadly with people. It’s because we’re living in a time of real anxiety about rapid change in the world.
Our economic system in the United States could do much more to abate or cushion that anxiety. I am for redistribution of wealth and basic economic protections enjoyed by the rest of the developed world like single-payer health care. May they come to pass in our time, Amen. But coming back to education, narrowly considered. Most of the jobs our children will be doing do not exist today. Nearly half of all current jobs may be replaced by technology and automation in some estimates. Our school system appears inadequate partly as a result of rising demand placed upon it. The real education reform debate of today is, how will our systems and methods of schooling evolve to meet these changes?
It is here, at #1, where I think Ravitch really misses the boat in a way that makes this book much less compelling than it could have been. When I look at the “corporate reform” coalition she identifies: “Major foundations, Wall Street hedge fund managers, entrepreneurs, and the U.S. Department of Education,” p.19, I don’t just see corporate capitalists on that list. (In fact, technically there are no corporations on that list.) I see technocrats and technophiles. These are people, Democrats and Republicans, executives and bureaucrats, who fundamentally believe that technical expertise rules supreme and that technological innovation is the engine of prosperity and all that makes America great.
Curiously, I think Diane Ravitch believes this too. At least, she pays lip service to this idea. “American technological innovations changed the way people live around the globe,”p.39…”the world’s leader in science and technology, the nation with the most powerful economy in the world.” p.72
I know there are hard-core right-wing Republican Tea Party/Grover Norquist/ALEC privatizers within the education-reform complex. I agree with Ravitch that the governor of my hometown state, Bobby Jindal, is probably one of them. I also agree that there are always plenty of people out to make a buck who need to be reined in.
But I think the big tent, the big umbrella, the unifying force here is a fascination with technology and innovation, not privatization per se. Bill Gates, to choose an example that Ravitch returns to often, was a ruthless CEO, but first he was a brilliant software engineer. Is it so hard to believe that in his third act, spending his personal wealth to try to tackle the world’s biggest problems, he’s influenced as much by the latter experience as by the former?
Technophilia explains why the ed-reform complex loves tests so much. It’s all that data, the number crunching that really gets them going. That is why they love charter schools: to pilot new ways of doing things. That is why they love to give tax money to private business owners; they believe that innovation thrives among private entrepreneurs and not in the public sector. That is why they love software and computers in classrooms and online teaching and learning.
It’s to the rightful role of innovation in education and computers in classrooms that I want to turn in part 2.
“Gravity” trailer via YouTube
Gravity, a new adventure movie about the perils of space exploration starring Sandra Bullock and George Clooney, is sharing some of its star power with a blended informal STEM-based learning program for kids. USC grad student Tara Chklovski founded Iridescent in 2006. It’s reached over 22,000 young people ages kindergarten and up with a combination of family science programs, delivered in New York and LA, and through partners in Chicago and the Bay Area. They have recently launched a website, the Curiosity Machine, where students can respond to challenges and get help online from dozens of professional mentors from the science and engineering worlds. ”The idea was to engage kids in STEM in a way that schools don’t do, by dealing with open-ended problems,” says Kevin Miklasz, director of digital learning at Iridescent. “Instead of lesson plans, we offer design challenges that are prompts for kids to solve complex problems with their own solutions.”
In the past Iridescent has partnered with the TV show Top Chef. For the Gravity-tie in design challenge, they asked participants to build a “space themed Rube Goldberg Machine” that performed some of the actions of a rocket launch, including 1) a launcher, 2) a transfer mechanism, and 3) an orbit well. The contest was judged by a panel of expert astrophysicists.
The first and second runner-ups were 14 and 16-year-old boys, but the winner was 13-year-old Eiley Hartzell-Jordan from Carrboro, North Carolina, who received mentoring online from Bonnie Lei, a Harvard biology student. She flew to New York City with her family to attend the movie premiere.
Iridescent is a near-perfect example of what Janet Coffey, at the Gordon and Betty Moore Foundation, calls the promise of informal STEM-based learning. ”A lot of the really rich experiences that get kids excited about pursuing science happen outside of school,” she says, naming settings like museums, libraries, summer camps, afterschool programs, clubs, Makerspaces and independent hobbies. “There’s an increasing awareness that factors like interest, motivation, and choice matter a lot for achievement and persistence [in academic subjects] and might even matter more than some of the cognitive things. Kids need opportunities to spark curiosity about science.” Her organization is investing resources in research that identifies the major emotional and social qualities that spur science learning, like self-confidence, a sense of belonging in a science-focused community, interest, motivation, and a fundamental belief in the importance of science.
Media, Miklasz says, may have its own role to play in spurring these kinds of experiences.
“Media can be an extremely subtle, nuanced, powerful way of reinforcing or dispelling stereotypes about scientists,” he says. “Portraying Sandra Bullock as a female scientist who is thinking hard and solving problems step by step–I’ve been really personally interested in how those impacts play out in kids’ perceptions.”
Scarcely a month ago, on August 27, the Los Angeles County Unified School District placed the first iPads in students’ hands at the outset of a $1 billion plan to give one to every single student in the nation’s second largest public school district ($500 million for devices, plus an additional $500 million for internet infrastructure upgrades, raised through construction bonds).
The project is now being resoundingly panned, as reports surfaced quickly of high school students going around the security software on the iPads to surf for non-approved content. The district has called a halt to students bringing iPads home amid disputes over who will be held responsible for loss or damage–parents or taxpayers.
On Friday I spoke to two LAUSD contractors who have first-hand knowledge of the rollout. They agreed to give an insiders’ view of the controversy on background. There’s an incredible litany of problems here that reads like a primer on what NOT to do with a major deployment of technology in a school district.
1. The Rush
Problem number one, from these contractors’ perspective, was the timeline. The iPad idea first surfaced in November as a proposal to spend $17 million in bond money coming to the district. There was a small pilot in the spring–not enough, says Contractor #1. “From an IT and security standpoint, it would be tough to pilot something in just a few months, let alone start phase I. I have a hard time believing that people in the district didn’t raise red flags to say, are you sure we’re doing this the best way possible?”
2. Training and Professional Development
The second big issue was a lack of training, professional development, and overall, a failure to recognize the human resource needs created by a big device rollout like this one. “Teachers were not trained in the system to manage the devices. Nobody at the school was trained. A couple people from the district that came out to sort of help and they had somebody at the school who was the de facto tech person, teaching teachers how to use it after it had been deployed,” says Contractor #1. Contractor #2 added: “The ELA (English) teachers got a 40 minute training, because they were responsible for giving them out. I don’t think any of the other teachers were trained on the mobile device management system.” Part of the reason that students found it so easy to turn off the security controls to surf the Web and access sites like Facebook, Youtube and Pandora might be that many teachers were unfamiliar with how the controls worked.
3. School to Home and Back
Taking school-issued devices home has pedagogical justifications, for homework, extra practice time, and making stronger connections between school and home. But there are some practical and theoretical objections to this idea.
During the pilot, Contractor #1 says, students weren’t allowed to take the iPads home. When they started going home, teachers quickly discovered that checking the devices out at the end of the day, and checking them back in in the morning, used up precious classroom time. Also, said contractor #2, “If kids didn’t want to do the work, they would come late purposely and not get an iPad. So in some classes, half the kids had them and half the kids didn’t, they were just sitting with their heads on the desk.” Parents, meanwhile, don’t want to be held liable for the loss, breakage or theft of the devices.
Contractor #1 had a different, more personal objection to the idea of students using a single device for work and home. “Being in IT, my professional device is separate from what I use at home. My daughter is five years old. She’s not old enough to understand that there’s a difference between your home life and school life and what’s acceptable in each place. Until she can segment that, I don’t want her being held responsible for any mistakes.”
4. Why iPads?
Los Angeles is paying a reported $678 apiece for these Apple iPads, higher than retail, although the price does include some educational software. That compares to as low as $250, retail, for a budget laptop. iPads don’t have a reputation as durable machines, and notably, they don’t have keyboards. “From the beginning I said, are they going to type at all? Is this not a skill? Are they going to require a keyboard?” said Contractor #1. Sure enough, just after Labor Day, the school district announced that they may be spending up to an additional $38 million on wireless keyboard accessories.
Barry Schwartz via TED.com
Barry Schwartz is a professor of psychology at Swarthmore College, the author of two best-selling books, by all accounts a terrific teacher and mentor, and in my personal experience, a mensch. He has a provocative piece on Slate this week that deserves attention. It is about paying attention.
See, in the age of information overload, no one has time, so everything has to be short. Tl;dr is an abbreviation used often online, in forums like Reddit, as a way of commenting on and dismissing someone else’s rant, diatribe, or impassioned outpouring. It stands for “too long; didn’t read.”
Articles are shortened to lists. Blogs are shortened to Tweets. And, Schwartz notes, with MOOCs the 45-minute college lecture–his own cherished medium–is being shortened to a series of five to eight- minute long video chunks interspersed with comprehension questions. This, he argues, is a sign of dumbing things down too far.
By catering to diminished attention, we are making a colossal and unconscionable mistake. The world is a complex and subtle place, and efforts to understand it and improve it must match its complexity and subtlety. We are treating as unalterable a characteristic that can be changed. Yes, there is no point in publishing a long article if no one will read it to the end. The question is, what does it take to get people to read things to the end?
Schwartz’s lament about the Information Age is, typically, couched as an injunction to educators. Teachers need to “make” students pay attention, instead of bowing to reduced attention spans with shorter, more attention-grabbing lessons.
Schwartz’s article, ironically, is fairly punchy and terse, but he still leaves room for nuance. One of his sharpest points is that he, himself, has become part of the problem.
Schwartz has delivered several popular TED talks which have been viewed online collectively millions of times. TED talks feature some of the world’s great minds taking on outrageously complex subjects, from quantum computing to molecular gastronomy, and they do so, famously, in just 18 entertaining, well-produced minutes. On the one hand, Schwartz says, this phenomenon is great, because it introduces so many people to great ideas (Ideas Worth Spreading is the slogan of TED). On the other hand, it’s terrible, because all the nuance is edited out.
I am all for nuance. Here’s a nuance I think is missing from his argument. The pithy, attention-grabbing intellectual style could actually be seen as a form of courtesy, a note to the idea that many people have something to say. There’s a power dynamic entirely different from that of the traditional classroom, where traditional professors, by virtue of their traditional power, claim the droit du seigneur to bore the bejeezus out of everyone by droning on with no editing whatsoever.
The Internet is a huge, buzzing conversation, not a lecture hall. No one is forced to listen to anyone else. Individual posts might be shorter, but rallies are long: You might write a thousand-word article, or post a ten-minute video, and it generates tens of thousands of words’ worth of comments and Tweets. That’s where the complexities truly emerge, in the back and forth. Overall, I don’t believe that our attention has diminished; it’s just that there’s so much more to pay attention to, and to contribute to as well. And isn’t this a better pedagogical model for encouraging people to grapple with complexity?
A few years ago, I interviewed Schwartz about his TED talks for an article and he had these (previously unpublished) comments about the experience. The first talk he gave was all pretty informal and off-the-cuff. The second time, he said, it was a much bigger deal.
“Having this incredible time constraint, I couldn’t count on myself to extemporize, because I didn’t have the time. I wanted to make sure that everything fit in. My shoot-from-the-hip-style was suppressed by the time limit. I took the preparation extremely seriously the second time…The truth is, it was worth it. It was incredibly well-received because I put so much effort into distilling a complicated set of ideas.”
This is the intellectual virtue and courtesy of the Internet age: recognizing that attention is scarce and exhaustible, you take just as much as you need.
In fact, you can see this in the evolution of the usage of tl;dr itself. These days, I often see it used not as a way to bite your thumb at others’ comments, but as a way to sum up your own–as a handy guide for the reader who may have other things to do. Cf:
tl;dr Brevity; soul of wit.
Sal Khan at The New York Times Schools for Tomorrow Conference, September 17, 2013.
As usual, the annual New York Times Schools for Tomorrow conference boasted a strong lineup of businesspeople, policymakers and academics on the pulse of educational innovation, performing for an audience of prematurely jaded journalists, ostentatiously Luddite faculty members, and businesspeople who thought their own companies should have been lauded from the stage.
MOOCs for the 90 Percent
Daphne Koller of Coursera, Sal Khan of Khan Academy, and Anant Agarwal of edX talked about the potential of their free course platforms, not as one-way broadcasters of video, but as analytics environments that collect vast amounts of data on the learning process and feed it back to curriculum developers for analysis. On the other hand, Dean Florez, of the 20 Million Minds foundation, was unsparing in pointing out that MOOCs as they exist today are not optimized to help the students he called “the 90 percent”–the nontraditional, working-two-jobs, first-in-their-families-to-go-to-college–the ones who truly need the convenience and affordability of online ed. Referencing the San Jose-Udacity pilot in which MOOCs offered to high school students in collaboration with San Jose State posted dismal pass rates, Florez said, “We need to dig in and work with the real students.”
The CEO Checkup
Alec Ross, formerly Hillary Clinton’s technology guru at the State Department who focuses on digital divide issues, made a point I hadn’t heard before in quite this way. He brought up the analogy of the “CEO Checkup,” a superpremium health care visit for an executive whose health is deemed to be of strategic interest to a company’s future, which costs tens of thousands of dollars and results in an overload of data. The fear is that wealthy parents will over-utilize the new computerized learning opportunities and diagnostics, resulting in highly individualized education programs and an even bigger edge for their kids, while poor kids struggle in schools that barely have Internet access.
The Power of Open–And Research
I was personally thrilled to see David Wiley, the pioneer of open educational content, now helping school districts and community colleges adopt free digital textbooks with Lumen Learning, and Candace Thille, who has taken her groundbreaking Open Learning Initiative from Carnegie Mellon to Stanford, on a panel about how the move to digital can, if done right, increase equity and access.
Thille in particular sounded a note that should resonate throughout the world of ed-tech. “Twenty hours in the lab can save you one hour in the library,” she quipped, pointing out that too many of the startups we see today are rushing forward with various models of instruction–game based, adaptive, what-have-you–without being grounded in an existing literature of decades of learning science. “That’s what I’ve spent the last 10 years of my life doing,” she said. “One thing we found is that learning is really complex…Once a colleague asked me, ‘why do you study learning? We all teach, it’s not rocket science.’ Well, actually it’s more complex than rocket science. Really understanding human learning at that episodic moment where you have change in thought is a complex process.”
I was bemused to hear the reaction of her fellow panelist, Karen Cator, lately of the Obama Administration, Apple, and now Digital Promise. Although she heads up a well-funded nonprofit dedicated to spreading best practices on technology and education, she seemed to dismiss out of hand the notion that published academic research could find its way into the hands of edupreneurs or decisionmakers. Karen, if you find Thille’s paper
“An Evaluation of Accelerated Learning in the CMU Open Learning Initiative Course “Logic & Proofs””, too hard going, I give a quick overview in pp.120-123 of DIY U.
This post draws on the ongoing research I’m doing for my next book, The Test, about the past, present and future of assessment in public schools.
This past week the New York Times Magazine published Jennifer Kahn’s big feature story on emotional intelligence. Like much recent coverage of education, it argued for the importance of a certain set of skills–noncognitive skills.
So-called noncognitive skills — attributes like self-restraint, persistence and self-awareness — might actually be better predictors of a person’s life trajectory than standard academic measures. A 2011 study using data collected on 17,000 British infants followed over 50 years found that a child’s level of mental well-being correlated strongly with future success. Similar studies have found that kids who develop these skills are not only more likely to do well at work but also to have longer marriages and to suffer less from depression and anxiety. Some evidence even shows that they will be physically healthier.
There’s evidence these skills can be learned, and they are starting to be taught. But there’s a big problem. Our current high-stakes testing regime, which is becoming the dominant standard by which schools and teachers are judged, funded, and allowed to keep doing what they’re doing, leaves very little room for anything other than math facts and reading facts. That’s it.
Anything that is not measured, is not managed, as the business axiom goes. So no matter how compelling the research and evidence for “non-cognitive skills” or “emotional intelligence,” or anything similar, our school system will never orient itself around these priorities unless we measure them.
But how? Multiple choice tests have been around for about 100 years. We are pretty confident in their ability to assess students’ knowledge of individual facts. And they are cheap to administer. But assessing a student’s emotional intelligence seems like something that can only be done qualitatively, one on one. That is far too messy and expensive to be adopted nationwide. Kahn alludes to the idea, which you typically hear, that emotional intelligence may boost performance on conventional standardized tests, but that’s not enough to ensure that emotional intelligence will be emphasized or taught. Particularly to the most disadvantaged students at the most underresourced schools who are more likely than others to be subject to intensive test prep.
Some researchers, however, are coming at this problem a different way. They are building computer-based tools like games and simulations to get at measuring the kinds of things we actually think are important for kids to know. Because they are computer based, once they are designed and built they are just as cheap to administer as a standard multiple choice test. One researcher that I spoke with recently is Dr. Dan Schwartz. He is director of the awesomely named AAALab, an acronym for Awesomely Adaptive Advanced Learning and Behavior, at Stanford University.
Schwartz spent a few decades in teaching and holds a PHD in Human Learning and Cognition from Columbia. I mention this because it’s actually unusual for people who design assessments to have either field experience in teaching, or knowledge of the ways humans learn and think. It may seem amazing, but I’ve learned that historically the field of psychometrics, which includes test design, is more or less completely separate from the field of learning or the profession of education. It’s almost as though medicine were divided into diagnosticians and treatment specialists who went to different schools and spoke different technical languages.
In any case, Schwartz is working on something called Choice-based Assessments. It’s a simple but revolutionary idea; don’t test kids’ knowledge, test how they approach the process of learning. “In our assessments we make little fun games, and to do well at the games you need to learn something. So they’re not just measures of what the student already knows, but attempts to measure how well they are prepared to continue learning when they’re no longer told exactly what to do.” That is, after formal schooling ends and lifelong learning continues. Or as they say on their website, these “interactive assessments can evaluate students in a context of choosing whether, what, how, and when to learn.”
One of the assessments currently under development is called Posterlets. It’s designed for students in middle school. Kids log on and their task is to make posters for a fun fair. They work in a little design program, drawing the graphics, putting in the text, etc. Then they choose to get feedback on their design from a group of animal characters. In each instance, they can choose whether they want to hear positive or negative feedback. (This evaluation is done automatically by the computer program, which is “smart” enough to tell whether the words are spelled right, whether the colors clash or the font sizes are too small). After that happens, the student has the option to change her design to incorporate the feedback. Then she gets to see how many funfair tickets were sold in response to a particular design.
A couple things strike me about Posterlets. First of all, it sounds more like a game than a test. It actually sounds fun. It also sounds far more like what you might actually do in a workplace or creative context than anything that usually happens on test days: coming up with a piece of creative work, seeking internal feedback, and reworking the work. It’s not so separate from the curriculum as a traditional test.
While it incorporates some material that you might find in a basic graphic design course, the content of Posterlets is secondary to what the simulation is really trying to get at: how does the student approach the learning process? In particular, says Schwartz, it looks at negative feedback. “The more negative feedback you chose, the better your poster gets.” Negative feedback, though it may be harder to hear, gives you more to go on. And a student who has the right level of resilience, motivation and persistence is more likely to be able to choose to hear the tough stuff on the way to getting great.
Posterlets is being tested as a means of evaluating design-based curricula. There are k-12 schools where students are taught to think like designers, a process by which they gather evidence, create novel solutions to problems, and test prototypes. “Good design based curricula emphasizes seeking feedback,” Schwartz explains. So a school that does a better job teaching that kind of skill should have students who do better at the Posterlets task.
To learn more about Schwartz’s research, I recommend this paper. In future posts I’ll talk about assessing students in informal learning contexts such as Makerspaces, and the concept of “stealth assessment.”