CogCubed is a video game you can reach out and touch

A recent paper published in Developmental Science reinforced how tactile experience is important to learning in the developing brain. The experiment looked at how toddlers in a high chair learned the names of novel (edible) substances. Those who were allowed to get their hands dirty, quite literally, exploring the texture of the different samples, picked up the names more quickly and remembered them longer.

How could this apply to the world of video games? Games are drawing wide interest as a tool for both learning and assessment, as I’ve written about here and here. But while they engage the senses of vision, hearing, and hand-eye coordination, they usually have left out the sense of touch. Until now.

CogCubed is a video game startup that uses Sifteo Cubes, an interactive game system developed at MIT that consists of a set of small cubes that look like tiny TV sets, each with a screen. When you move, stack or tap the cubes they communicate wirelessly and the image on the screen changes in response. This system is known as  a “tangible user interface,” a takeoff of “graphical user interface,” which is the term for the icon and window image navigation we are all familiar with from most computer operating systems.

The thought is that interacting with the cubes in three dimensions will be more engaging for both children and adults than using a keyboard, mouse, or other controller. According to CogCubed founder Kurt Roots, TUIs have been shown to be easier to learn than traditional GUIS, and they also tend to increase problem solving behaviors and improve spatial cognition.

CogCubed has created a game called Groundskeeper that looks a little bit like the old arcade game Whack-a-Mole. The company holds several patents relating to the capture and analysis of behavioral information while players are interacting with the game system.In a pilot study at the University of Minnesota, the game demonstrated surprising power to diagnose ADHD as people play. It could accurately detect the condition 75% to 78% of the time, an improvement over other existing methods. The success is not surprising given the level of detail: the system takes note of what the player is doing every one-tenth of a second for 30 minutes, for 30 different variables.

Clinical trials are continuing; CogCubed is pursuing FDA approval as a medical device to diagnose, and eventually treat, not only attention disorders, but other conditions affecting what is called “executive functioning” in the brain: anxiety, depression, autism spectrum disorders, traumatic brain injuries and Alzheimer’s. Clearly this is a growing area of research; how long do you think it will take before these devices and games are part of mainstream classroom practice?

 

 

Could a video game replace ADD medication?

atentiv

In America, 6.4 million children have been given diagnoses of attention disorders. That’s 11 percent of the school-age population.  Annual production of Ritalin-like drugs has quadrupled since the 1960s, and millions of children are taking these powerful stimulants every day.

Some argue that this “epidemic” is in fact an artifact of a test-driven, high-stakes, high-pressure school culture. Parents are eager to diagnose kids to get them extra resources, extra time to take tests, or simply an educational edge, and schools can exempt themselves from test targets if they have more kids classified as “disabled.”  A.D.H.D. diagnoses spiked 22 percent in the first four years after No Child Left Behind.

Regardless of the broader context, kids and families are clearly suffering. Sandra lives in Newton, Massachusetts and is the mother of a 9 year old boy who’s had trouble in school, both with behavior and grades, since kindergarten.

“It’s painful when your kid comes home and says, I feel like the dumbest kid in the class because all the other kids know the answers,” she says. “He tells me, Mom, when I’m in school I have a really hard time paying attention. I go into the clouds and I miss what the teacher said and I have to ask my friends to tell me what just happened.” Despite these problems, Sandra has been reluctant to medicate her son. She was happy to have the opportunity to enroll him in a clinical trial of a new technological solution, Atentiv.

Atentiv is, essentially, a video game with a brain-computer interface component. To play the game, children strap a headband around the forehead that uses an EEG to measure the brain’s pattern of electrical signals and transmits them wirelessly to the computer via Bluetooth. First, the player goes through a calibration process that measures the unique “signature” of the individual’s brainwaves in concentrating and distracted conditions while completing the Stroop Task, a common test of concentration.(Previous studies support the presence of unique EEG patterns for children diagnosed with ADD).

Once the system is calibrated the user plays a game that involves a character running through a landscape to complete tasks. When the player is distracted, the character slows down; the more she concentrates the faster the character moves. The result is something like a form of biofeedback–young children get an object demonstration of “growth mindset.” They grow in awareness of what distraction and concentration feels like, and they also grow in their sense of being able to control their mental state for better performance.

For Hayden, Sandra’s son, who took part in the clinical trial three times a week for 8 weeks, the results were dramatic. “I really saw a tremendous difference both at home and at school. I was thrilled.” Hayden’s handwriting got neater. His sleeping habits improved. His homework got better. His teacher was no longer calling with discipline problems. He got along better with his sister and with friends. He was easier to get out of the house in the morning.

Hayden is not alone. Trials of the Atentiv System have shown sustained improvements in 75-85% of children, as rated by parents, standardized tests, and direct measures of the brain waves. In this study, parents noticed significant improvement in inattentive and hyperactive-impulsive symptoms six months after the treatment began.

It’s hard to say how much of this is due to the futuristic wonders of neurofeedback technology, and how much of it is a placebo effect. The researchers in this independent study noted that the study did not have a control group that, say, played a regular video game, and the parents all knew that their kids were in the treatment condition, which may have caused them to exaggerate the positive effects.

Atentiv’s founders are not initially seeking FDA approval for their technology, so they can’t make outright claims about ADHD. Instead they are launching early next year with a consumer product aimed at parents who want a solution for their kids without drugs. CEO Eric Gordon is confident that the technology will eventually lead to clinical products for treating not just ADHD but memory and abstract reasoning, in children, adults and the elderly.

Sandra’s explanation for her son’s improvement is not about the technology itself, but about the broader message of the game.

“It’s all about self-esteem,” she says. “For Hayden, it helped him realize he was capable. And also, just the way the game and training materials talked to him about it: saying there’s nothing wrong with you, you’re not dumb. Look, your brain is like a Ferrari and we’re going to teach you to look at the green and red light.”

 

 

 

 

Redesigning for student-centered learning


Brendan Campbell teaches at Southeastern High School in Detroit, which is under the Education Achievement Authority of Michigan, a statewide recovery district for schools consistently in the bottom five percent according to test scores. In other words, it’s a failing school in a violent, poor, bankrupt city. But this fall, Campbell and his collaborators have used meager resources to construct a new approach to truly student-centered learning that is drawing interest and acclaim from educators and reformers all over the country: The Preparatory Academy at Southeastern, or PASE.

PASE students spend five hours of their school day in a big open space that’s been designed to feel like a college library, with quiet spaces for individual work, and places to meet with teachers or in groups. This time is theirs to prioritize and allocate over their core courses and up to two electives. The curriculum, based on the Common Core, Next Generation Science Standards and Michigan state standards, has been broken into manageable chunks–a set of learning targets in each subject, written in student-friendly “I Can” statements. Students move at their own pace towards mastery of each target through a sequence of  “learn, practice, apply, assess.”

On a given day, students could choose to attend a “scheduled learning opportunity,” such as a lecture or a science demonstration; watch a video of a previously recorded lecture by their teacher or a curated resource from elsewhere on the web; work on a group research project; or review peers’ work with reference to a rubric. At the “assess” stage of each target, they take a 3 to 10 question formative assessment. They must score at least 75% to get their “exit ticket” and move on to the next target; otherwise, they’ll conference with a teacher on what went wrong, and go back to pursue mastery.  Over the course of each unit, the students also work in groups to complete interdisciplinary performance assessments. In this first semester, students researched the science, ecological, health, and community benefits of planting a garden on campus and presented the case to the administration.

The planning process for teachers within PASE is novel. “When I used to make lesson plans it was focused on: I have 50 minutes to fill, if an activity takes 40 minutes, what happens for the last 10? That can mean a lot of busywork or wasted time,” says Campbell. “Now we’re only concerned about what is necessary in order to truly learn and master the content. Really, that’s a more productive use of student and teacher time.” Each student will be getting a different combination of direct instruction, reading, video, and more, so all the resources, and assessments, must be carefully curated to ensure that each student has access to what they need.

Each student has a laptop, and the program uses an online learning management system to help everyone keep track of students’ progress, but online is not the main focus. “We wanted it to be truly blended so there would be online and in-person components,” says Brendan Campbell. “Students always have a choice.” Students have the choice of up to two online electives during their PASE time, and have chosen from dozens from Psychology to Art History.

PASE isn’t designed for the strongest students in the school. Many must use their elective time to retake failed classes online–the average incoming reading level at the school is 4th or 5th grade. Students were asked to apply, and teachers were also asked to help identify those who they thought could benefit from the flexibility of the program and the chance to take ownership over their learning.

“One thing we’re struggling with is, do we hold all students to the same pace even thought they’re at different ability levels?” says Campbell. “Right now we’re at uniform pacing, and we want to create individual pacing for each student.”

I think what intrigues me most about this model is the sense of autonomy, respect and trust. Urban public schools like this one, majority minority and poor, have been criticized for forming a “school-to-prison pipeline.”  When someone’s experience of an institution is primarily about being forced to sit in a certain seat, to quietly listen to authorities speaking, to move from place to place at the sound of a bell, to be labeled as a discipline case if you don’t do it, and you’re never asked what you’re interested in or what you want to do, it’s hard not to see how that could interfere with motivation and learning at one’s best.

“We’re trying to build students’ understanding of how they learn, and what they need in order to be effective learners,” says Campbell. “We talk to them all the time about, ‘in college there’s not going to be a teacher telling you to be quiet.’ I’ve been blown away with how students have reacted to the level of trust and respect. They give it back to you almost all the time.” He says that students wear their PASE lanyards with pride, and are particularly proud of the many national and state visitors that the brand-new program has hosted.

On the other hand, it must be recognized that the PASE model is taking place within a grim economic and political context. They started this September with 70 students in grades 10 through 12, and ten weeks into the semester, because of staff cuts, they had to nearly double the size to 115 students. That’s with just four teachers presiding. If this model succeeds, it is possible that it might be used as an blueprint to lower student-teacher ratios in the name of hyper-efficient blended learning, which would certainly be an unintended consequence. In either case, the nation has a lot to learn from PASE’s first year.

MakerBot Academy: A 3-D Printer In Every Classroom

4095369665_d10ffc067e_o

 

 

A MakerBot is a tabletop-sized device that takes digital designs and builds them in the real world, layer by thin layer, out of a plastic-like roll of filament derived from corn. You can create or customize any design you can think of– a working prosthetic hand, or a scale model of the Eiffel Tower, or a set of chess pieces–or download someone else’s design from the Thingiverse, MakerBot’s free online library. Comparatively low-cost and easy to use, MakerBots are the popular edge of what technology observers and futurists–even President Obama–call a “new industrial revolution” of mass customization, where design is democratized and manufacturing is one day as decentralized as knowledge production has become. Plus, they’re just wondrously cool.

MakerBot CEO Bre Prettis used to teach public school in Seattle. “It’s always been a dream of mine to get this into more classrooms,” he says. This week he announced the MakerBot Academy  to do just that.

Teachers who want a MakerBot for their classroom are asked to register at the educational donation site DonorsChoose. There, their project can be funded through tax-deductible donations by individual donors. The architectural software company Autodesk has also agreed to sponsor a number of MakerBots, as has the director of MakerBot’s parent company, and Prettis personally is committed to funding them for high schools in his current home neighborhood of Brooklyn. For any teacher, MakerBot is making available a package of the printer itself, three rolls of filament, and a service and support plan at $2000, a $700 savings over the retail cost.

“A MakerBot is a manufacturing education in a box; it unlocks creativity and gets kids thinking about how things work,” Prettis says. He sees the machines functioning in classrooms in a range of ways.

On a practical level, MakerBots could be a relatively affordable way to furnish a steady stream of new materials and supplies for classrooms that might not otherwise be able to afford them, from a detailed model of a human heart to simple machines for use in physics. To start out, DonorsChoose told MakerBot that one of the most commonly requested items are “math manipulatives,” the blocks, wedges, counters and other toys that help kids learn geometry, arithmetic and more. MakerBot put out a design challenge to its Thingiverse community to submit ideas for new and creative manipulatives that can be downloaded, printed and used in classrooms. It’s exciting to think about students and teachers in different communities creating and sharing their own designs for classroom purposes.

 

Screen Shot 2013-11-13 at 4.14.34 PM

MakerBot is going to be sharing curricula created by teachers and companies like Autodesk to help teachers work the 3-D printers into lesson planning from kindergarten through high schools. Within the world of K-12 innovation, we’re oftentimes focused on the use of handheld devices with screens, but learning can be an intensely tactile process, and the MakerBot taps into that. A MakerBot is actually a great bridge between the virtual and physical world,” Prettis says. 

 

 

 

 

Coursera founder phones it in at open education conference

Coursera Infographic

#OpenEd2013 is the tenth annual installment of the premiere conference of the open education community, taking place right now in Utah. Open education is currently contested territory, with divisions highlighted yesterday by a flatfooted keynote from Andrew Ng, cofounder of Coursera, that played out to a baffled chorus of mockery on Twitter. Amid the jibes, there’s a serious issue at stake: will the future of education be dominated by a few closed platforms, and limited approaches to teaching, learning and knowledge, or will truly open innovation prevail?

Hope someone warned Ng that he can’t toe the standard Coursera line for #OpenEd13 talk. They have been doing this a lot longer than he has.— Amy Collier (@amcollier) November 6, 2013

Open education was first most closely identified with OER–digital educational resources such as MIT’s Open Courseware that carried an open license, such as the Creative Commons license, allowing them to be freely shared, reused and remixed. For self-identified open and connected educators, though, mostly from the higher ed world, openness wasn’t just a technical designation. They were concerned with democratizing education, making it accessible to all, peer-driven rather than hierarchical, emphasizing the fluid process of learning rather than the rigid gateways of accreditation–“an exploratory, community-created knowledge building process,” in the words of Athabasca University professor George Siemens. In this spirit, Siemens and Stephen Downes ran the first Massively Open Online Course, or MOOC, in 2008, with about 25 University of Manitoba students joined by 2500 students online. The topic–a bit meta– was “Connectivism and Connected Knowledge. ”

Today, of course, the term MOOC means something very, very different. From experiments pursued by a small group of learning and teaching enthusiasts, a handful of platforms — edX, Udacity, and Coursera— have emerged with tens of millions of dollars in backing from venture funders and foundations, hundreds of university partners, and millions of users. There is a dominant format for the MOOCs published by these platforms: they run from six to 14 weeks long, and consist of short video lecture “chunks” presented often by well-known professors, interspersed with multiple-choice comprehension questions, combined with readings, often homework assignments or an exam, and forums for discussion.

–I’m hoping Ng’s keynote is actually a bunch of short videos with intermittent quizzes. #opened13— Jonathan Becker (@jonbecker) November 6, 2013

Most of the MOOCs, while free to access currently, are not open-licensed–they are the intellectual property of the companies and institutions and thus can’t be downloaded, reused, or remixed freely.

–Why isn’t Coursera openly licensed? Ng says that its content creation costs too much money and that wouldn’t be sustainable #opened13 (sigh)— Audrey Watters (@audreywatters) November 6, 2013

Ng is the quieter of Coursera’s two cofounders. He’s also director of the Stanford Artificial Intelligence Lab, which means he has a deep intellectual interest in the growing field of “educational data mining,” learning research, training computers to grade essays and tracking student engagement. Coursera, like other large MOOC platforms, offers the opportunity to learn a great deal about the learning process, at least as it plays out online.

@jonbecker our analytics have determined your diminished interest and have now dispatched mentors to keep you engaged. #opened13— George Siemens (@gsiemens) November 6, 2013

His keynote, however, failed to address these research questions, and instead delivered a standard pitch about Coursera to people who are already quite aware of what it is. Also, unfortunately for a presentation on hybrid learning, there were technical problems.

–Oh the irony. Andrew Ng is Skyping in for his keynote at #opened13. So we’ve been asked to turn off devices to save bandwidth. MOOOOOOC!— Audrey Watters (@audreywatters) November 6, 2013

The irony is worth underlining: the OpenEd community, whose major criticism of MOOCs is that they enshrine the one-way, rigid lecture format, was asked not to respond via the open web while Ng was lecturing to them over a video link.

Within the open education world, as summarized by George Siemens’ keynote right after Ng’s, there are a range of feelings about MOOCs–both angst and hope. This is not just a group of hipsters who are upset that their favorite band suddenly got really popular, or merely professors angry that someone is turning their life’s work into a business.

–Sorry Dr Ng, but there are heaps of people who create engaging online learning inexpensively. Too bad you couldn’t meet them at #opened13— Brian Lamb (@brlamb) November 6, 2013

These are engaged, excited, experimental educators and learners, with values that they fear are getting lost as MOOCs get even more massive. They want their due as partners in the creation of a diverse and vital future of education.

@GardnerCampbell Ng’s talk had no sense of or much regard for its audience. Conversation would be great, but there’s a sense + #opened13 in which xMOOCers refuse to meaningfully engage thoughtful critiques that was symbolized by what went down #opened13 — Luke Waltzer (@lwaltzer) November 6, 2013

As the K-12 “connected educator” movement grows, this debate will be increasingly relevant across all levels of education. Do we want a future where mass market MOOCs and similar digital resources are primarily prepackaged and delivered to students via a vendor-like, consumption-based model? One that enshrines the several-week course and the talking-head lecturer as the central model of education? Or will more messy, diverse, participatory models of open education have the opportunity to spread and take root? Can the two approaches interact and maybe even reinforce each other?

This was clearly a missed opportunity to raise these questions and more.

–I’m dropping out of this keynote. #opened13— Jonathan Becker (@jonbecker) November 6, 2013

 

 

 

 

Apple has 94% of the classroom tablet market–Why that’s a scary stat

4488250788_83e9158cab_z

“Apple is making substantial contributions to society,” by “reinventing education,” CEO Tim Cook said during Monday’s quarterly earnings call. Celebrating their “highest September quarter revenue ever,” and their “best education quarter ever,” Cook and CFO Peter Oppenheimer highlighted global education more than any other growth market for the iPad specifically. Here are some of the examples they cited:

  • Over $1 billion in the education category specifically for the first time ever, out of $37 billion in total revenue.
  • Education sales were up 8% year over year.
  • In the U.K., the number of iPads sold to K-12 schools has more than doubled year-over-year.
  • In Latin America, over 70,000 iPads in use across 800 schools.
  • The Coachella Valley School District in California is currently distributing over 19,000 iPads to its student body.
  • Horry County School Districts in South Carolina will distribute 10,000 tablets to middle school students this year with a goal of providing all students in grades 3-12 with their own devices within three years.
  •  iBooks textbooks are now available across the U.S., U.K. and Australian national high school curricula.
  • Apple’s share of tablets in education is 94%. 

“It’s sort of unheard of,” crowed CEO Tim Cook. “I’ve never seen a market share that high before. So, we feel like we’re doing really well here and feel great to be making a contribution to education.”

The rise of the iPad in education has been impressive but not uncontroversial. As documented elsewhere on this blog, the devices have both passionate fans and passionate detractors among educators. Notably, Cook and Oppenheimer did not mention the ongoing $1 billion iPad rollout by the second largest school district in this country, LA Unified. Perhaps that’s because the rollout has been delayed due to security concerns and other implementation problems, a “fiasco” that led to resignation rumors for Superintendent John Deasy.

As I see it there are three major problems with one company having a 94% market share in a particular kind of classroom device. The first one is price and competition. In response to a question on the call about competition from the more modestly priced Google Chromebook, Tim Cook said “We do see Chromebooks in some places, but the vast majority of people are buying a PC/Mac or an iPad.”

Even with high-volume discounts, iPads can cost over $600 per student. Apple has consistently resisted making truly budget versions of its devices even when that meant losing market share to competitors, to say nothing of its purported social mission in an area like education. Not only are iPads expensive, they require monthly data plans for most applications, their components are near-impossible to repair or upgrade, and like most Apple products they are designed to be replaced every few years.

The second problem with massive iPad adoption is control of curriculum. The star iPad educators I’ve talked to are masters of the remix. They use dozens of different apps in their classrooms daily, most of which put students in an active role as content creators and communicators. It takes careful scaffolding and professional development to make teachers confident in applying this kind of technological creativity. In large scale iPad adoptions, especially those motivated by the transition to the Common Core, what we’re far more likely to see is Apple subcontracting out to providers like Pearson for an all-in-one, off-the-shelf curriculum solution that is little more than an automated textbook. As we’ve seen in LA, this can simultaneously disempower and confuse teachers, taking them out of the drivers’ seat, as well as lead to subpar experiences for students.

Finally, compared to bona fide computers, iPads are far less hackable. You can’t reprogram them without jailbreaking. Using them to write code is unwieldy. You can’t even open them up to change the battery. This puts an entire technological world off-limits in tablet-based classrooms.

Here’s a blog post by a friend of mine who is building a PC out of parts with his two school-aged daughters, using free videos found online as their coach. The girls will use the computer to play the wildly popular educational game Minecraft. The total cost is well under $300. Imagine if our public schools took this DIY approach to provisioning classrooms with computers.

Three student successes with iPads

 

The last time I wrote about iPads in the classroom, it was about a school district doing almost everything wrong. Today I talked to a teacher in the Chicago Public Schools who has a 180 degree view.

When the iPad first came out in 2010,  Jennie Magiera made fun of her friends for buying them: “Nice job–you got a giant iPhone that can’t make phone calls!!”  But when a grant bought iPads for her fourth and fifth grade class, the teacher quickly found a path to transforming her teaching and learning practice. While tests are only one measurement of success, she went from having just one student out of 15 “exceed” on state tests in fourth grade, to having 10 “exceed” the next year.

Just three years later she has gone from the classroom to helping other teachers implement one-to-one iPad programs, as the digital learning coordinator of the Academy of Urban School Leadership, a network of 29 public (non-charter) schools that are 90% free and reduced lunch. Her focus is on using technology to make good teachers better, and to let students be the best they can be.

“I could seriously sit here till we both passed out telling stories of powerful things that happen every day,” Magiera says. Here are three of those stories.

1) The second grade experts. 

The students in Magiera’s network are not “digital natives.” Most of them don’t have access to devices at home because of family income. Nevertheless, they are engaged by and excited about using computers, and because the teachers are learning to use them along with the students, there’s sometimes a role reversal in the learning process.

“We had three girls who came in during recess because it was cold and wanted to help us provision the tablets,” says Magiera, meaning setting them up to run certain kinds of apps. “Right away they started problem-solving: ‘She already hit that button…’ ‘Why don’t you try the green box in the upper left hand corner, since you already did the blue one in the lower right?'” They learned the term “microUSB” and created an organizing system to see which iPads were charged.  In this mundane technical activity, Magiera saw the students take on a real problem and work alongside adults in a way that seven-year-olds don’t always get a chance to do.

2) The shy one speaks.

Magiera had one very intelligent student who was afraid to speak up in class–she tried writing questions down on notecards, and warning him that she was going to call on him, but he would still freeze up. With the iPads she implemented a classroom “backchannel,” allowing students to participate in a text-based chat as part of the class discussion. In that forum, the boy blossomed. “I was not only able to see who was participating but the quantity and quality of their participation,” she says. “And what I found was this young man not only was the most vocal kid, he was the best in the conversation. He was hitting all the markers, responding to other students, coming up with novel ideas, supporting peers in a positive manner, and really thriving and flourishing in a community of thought when he didn’t want to speak up [before].”  

3) The troublemaker revealed.

Yet another student, she says, was constantly disruptive. His test scores and other math grades were poor. One day, she started using a technique called “screencasting,” a program where students can draw or write using a stylus and narrate at the same time, producing a video in a similar format to a Khan Academy video.

“The answer was 15 cents and he wrote $16. I would have thought he wasn’t paying attention or didn’t try. But when I go into his screencast video, it was 60 seconds of the best math I’ve ever seen as a math teacher.” The student had arrived at the wrong answer because of a tiny mistake, but he had devised his own original path through the problem, using his knowledge of fractions to create a system of proportions, a concept he wouldn’t be introduced to for another year or two. “He solved it completely on his own, narrated it beautifully, had the most amazing thought process.”  From watching this one minute of video, Magiera got insights into this student’s math skills that she hadn’t learned from having him in the classroom for over a year.

But the insights didn’t end there. Magiera then had the student rewatch his own video. She saw his reactions go from defiance (“lady, I already did it for you once, you want me to watch it now?”) to pride (“yeah! I got that!”) to dismay (“Oh my god, I messed that up! I can’t believe it! I was so close,”). And finally he asked her, “Can I do it again?” 

“I just about died,” Magiera says. “I was ready to burst into tears. This was a kid you could not get to do homework. Classwork was a struggle. Now he just heard his own thinking, which is really hard for a nine year old to do, and he wanted to improve authentically out of his own motivation. That was a feedback loop we did consistently from then on.”

 

An automatic boredom detector? Inside “educational data mining” research

8092131055_608198f390_z

I’m currently working on a book about the past, present and future of assessment. For the “future” bit I get to talk to researchers like Ryan Baker at Columbia. He’s spent the last ten years working on systems that gather evidence about crucial parts of the learning process that would seem to be beyond the ken of a non-human teacher.

The basis for the observations comes from what’s called “semantic logs” within a computer learning platform, such as Khan Academy’s: Was it a hard or easy question?  Did the student enter a right or wrong answer? How quickly did they answer it? How did it compare with their previous patterns of answers? The detectors gather evidence that students are gaming the system, drifting off-task, or making careless errors. They can extrapolate a range of emotional states, like confusion, flow, frustration, resistance, (which Baker calls memorably “WTF” behavior), engagement, motivation, excitement, delight, and yes, boredom.

Baker’s engagement detectors are embedded within systems currently being used by tens of thousands of students in classrooms from K-12 up to medical school. (Medical residents, he says, show the highest rate of “gaming the system,” aka trying to trick the software into letting them move on without learning anything, at rates up to 38% for a program that was supposed to teach them how to detect cancer.) His research, located at the forefront of the rapidly expanding field known as “educational data mining,” has a wide range of fascinating applications for anyone interested in blended learning.

Understanding how good these detectors currently are requires a bit of probability theory. To describe the accuracy of a diagnostic test, you need to compare the rate of true positives to the rate of false positives. The results for the “behavior detectors,” Baker says proudly, are about as good as first-line medical diagnostics. That is, if the question is whether someone is acting carelessly, off task, or gaming the system, his program will be right about as often as an HIV test was in the early 80s–0.7 or 0.8 (“fair” according to this rubric). For emotional states, which require a more sophisticated analysis, the results are closer to chance, but still have some usefulness. These accuracy scores are derived from systematic comparison with trained human observers in a classroom.

So why would someone want to build a computer program that can tell if you are bored?

To improve computer tutoring programs. Let’s say a learning program provides several levels of hints before the right answer. You want to build something in that prevents a student from simple gaming techniques, such as pressing “hint, hint, hint, hint,” and then just entering the answer.

To give students realtime feedback and personalization.  “I would like to see every kid get an educational experience tailored to their needs on multiple levels: cognitive, emotional, social,” says Baker. Let’s say the program knows you are easily frustrated, and gives you a few more “warmup” questions before moving on to a new task. Your friend is easily bored. She gets “challenge” questions at the start of every session to keep her on her toes.

To improve classroom practice. Eventually as these systems become more common, “I would envision teachers having much more useful information about their kids,” says Baker. “Technology doesn’t get rid of the teacher, it allows them to focus on what people are best at: Dealing with students’ engagement, helping to support them, working on on one with kids who really need help.” In other words, though technology can provide the diagnostics for affective states that affect learning, it is often teachers that provide the best remedies.

To reinvent educational research: This is a fascinating one to me. 

“I’d like to see educational research have the same methodological scope and rigor that have transformed biology and physics,” Baker says. “Hopefully I would like to see research with, say, 75% of the richness of qualitative methods with ten times the scale of five years ago.”

Modeling qualitative factors related to learning opens up new possibilities for getting really rich answers to really interesting questions. “Educational data mining often has some really nice subtle analyses. You can start to ask questions like: What’s the difference in impact between brief confusion and extended confusion?”

In case you’re wondering, I will clear up the confusion. Brief confusion is extremely helpful, even necessary, for optimal learning, but extended confusion is frustrating and kills motivation.

The very phrase “data mining” as applied to education ruffles feathers. It’s helpful to hear from an unabashedly enthusiastic research scientist, not an educational entrepreneur with a product to sell, about this topic. Privacy, he says, should be given due consideration. “The question is what the data is being used for,” he says. “We have a certain level of comfort with Amazon or Google knowing all this about us, so why not curriculum designers and developers? If we don’t allow education to benefit from the same technology as e-commerce, all we are saying is we don’t want our kids to have the best of what 21st c technology has to offer.”

If you’re interested in learning more, Baker has a free online Coursera course on “Big Data in Education” starting this Thursday. Over 30,000 people have signed up.

Can online learning make teaching more human?

Screen Shot 2013-09-03 at 3.56.59 PM

Data-driven pedagogy. The phrase conjures a robotic, dull future that only intensifies the worst aspects of 20th-century, bureaucratic, industrial wasteland-style schooling, where learners are defined down to “users,” or even metonymized as disembodied “eyeballs,” and force-fed bits of disconnected information.

For a counternarrative, the question is simple. What can creative humans do with the power of data? One possible answer is that computer-powered analytics could expand humans’ ability to focus on the most human aspects of teaching and learning.

I reported earlier this year on a small experiment the video website Khan Academy ran to this end.

While browsing the web site, some Khan users saw a simple slogan added to the page next to, say, a math problem: “The more you learn today, the smarter you’ll be tomorrow.” The line linked to a further explanation of the concept of “mindset,” the famous body of research by Harvard psychologist Carol Dweck on growth, achievement and motivation.

Displaying that one line led to a 5% increase in problems attempted, proficiencies earned, and return visits to the site, compared to otherwise similar learners who did not see the line.

This week, Andrew Liu, Udacity’s data science intern, blogged about his own research with the data generated by that MOOC platform. Apparently the questions they are framing go along similar lines: toward psychological aspects of motivation and engagement.

Screen Shot 2013-08-27 at 4.55.42 PM

Model of student engagement over time.

Modeling student engagement over time.

“At Udacity, we now have the opportunity to take findings that originated from studies on tens of students in physical classrooms – such as Carol Dweck’s concept of growth mindset – and apply learnings to hundreds of thousands of students with improved teaching. But even more powerful is Udacity’s ability to conduct our own pedagogical research at scale on a rapidly growing worldwide classroom that was not even possible a year ago. Pedagogical areas we’re exploring include the importance of metacognition, expectation setting around formative assessment, and even new online challenges such as which characteristics of video keep students most engaged.” 

Mindset, metacognition (learning about learning), engagement–these are great research questions for educators to be looking at. They are not chiefly about automating the consumption and digestion of information, but about deepening the learner’s physical and emotional relationship with the process of learning.

It’s in part simply the growth of sample sizes that has some researchers so excited about what they might learn in the emerging field of data-driven pedagogy. I haven’t verified this independently, but I have often heard researchers repeat the notion that there are just very few large-scale randomized controlled trials out there comparing the efficacy of various classroom techniques and methodologies.  Sample sizes tend to be quite small and experimental effects hard to compare. (If there are counterexamples, I’d love to hear them).

A major example is the efficacy of online and blended learning itself. According to a comprehensive literature review published by Ithaka SR earlier this year, of over 1000 online and blended learning studies reviewed by the US Department of Education, only 45 met minimal criteria of having experimental research design and considering objective learning outcomes. Of those 45 studies, “most have sample sizes of a few dozen learners; only five include more than 400 learners.”

The kind of a/b trials that the Khan Academy and Udacity are doing, by contrast, can be easily run on hundreds of thousands of people.

Obviously there are relationships and aspects of the human dimension of learning that can’t be addressed with even the best data tracking and experimental design, or the largest sample sizes. There is an ever-present danger that the metrics chosen will tend to distort the nature of the undertaking itself. However, I can’t help but be a little optimistic that at least data scientists are starting with the right kinds of questions.

The five most important ed-tech trends at SXSWedu

SAMSUNG

I’ve been on the ground in Austin for the South By Southwest Education Conference & Festival for 22 hours. In that time, I’ve interviewed six people, chatted with many more, and hit the Java Jive in the Hilton four times. Here’s what I see as the biggest trends coming out of the conference.

  1. Data and analytics. There seems to be a consensus, which Bill Gates will no doubt highlight in his keynote tomorrow, that the most important potential—as yet unrealized—contribution of technology to teaching and learning is the ability to extract meaningful insights from the myriad information that students generate as they travel through life on their learning journeys: diagnostics, individualized goals and plans, demographic information, performance evaluations, and on and on from cradle to mortarboard. Companies like InBloom and Engrade envision a teacher working like a doctor, synthesizing reams of test results and other information with the help of tech tools to arrive at the proper intervention for the proper moment.
  2. Games and adaptive learning. What makes video games fun is that they get harder as you get better at them, keeping you in the right “proximal zone” between bored and frustrated. “In the gaming world, when you don’t get the right outcome, you don’t feel like a failure, you say how do I adjust,” says Dreambox CEO Jessie Woolley-Wilson. This is what is meant, at its simplest, by adaptive learning. Game-like learning platforms range from Dreambox, a math program that “puts the learning in front,” in the words of Woolley-Wilson, to Kuato Studios, which later this month is debuting a fighting-robot coding game made by designers who worked on Call of Duty. Games and adaptive learning are intimately related to #1, data and analytics. In some sense, what defines a game is simply that the players are keeping score, so a key feature of online learning games is the constant generation of data that can, in theory, be used by teachers and parents in coaching mode to help direct students. Taken together, #1 and #2 form the megatrend/buzzword of “personalization”—the “mass customization” of learning.
  3. MOOCs. While many in the education space might be sick of hearing about Massively Open Online Courses, Coursera, edX, et al, they are still adding users and shaping the public imagination about what’s possible when classrooms open a window on the world.
  4. Makers and creativity. I was pleasantly surprised to see a Makerspace onsite at the convention center, where you could drop in and play with Legos, circuits and homemade play-doh. This hands-on, amateur, DIY stuff taps into a deep need for learners to accent what is most fully human, even as we are increasingly overwhelmed by virtual worlds. In addition, John Maeda, president of the Rhode Island School of Design, hosted an influential panel on STEM to STEAM—putting the arts into STEM education. He’s argued that the forward march of technology will lead to a higher premium being placed on the personal, well-designed and handmade.
  5. Going back to the classroom. “Where are the districts?” “Where are the teachers?” Aside from a few leaders of charter schools I’ve run into, most of whom were presenting, my impression is that there are few full-time educators here, let alone people who make IT purchasing decisions for school districts. Many sense a fundamental disconnect on both sides between the innovation conversation going on here and the real needs of teachers in classrooms. Hopefully that will change soon.