Teacher Feedback VS Machine Feedback

Do students prefer to learn from a machine?

A bit of a professional disclaimer to this week’s geek study. I do not believe that technology is ever an effective substitute for good teaching (and this is a belief rooted in quite a few empirical studies as well). Technology is a tool. My interest is how tools are leveraged by teachers to help and increase learning by students. I found the results of this study to be surprising. I’d also hesitate to draw huge conclusions from a study at a large education college and apply those results to K through 12 education.

That said, this is interesting.

Why Students Prefer to Learn from a Machine

While this article’s headline is hyperbolic, it does share a fascinating study from The International Journal of English Studies. Education students were placed in two groups. In one group, students received feedback on writing assignments from live instructors. A comparison group received feedback from a software program called Criterion. The study’s intent: to identify differences in how students responded to different forms of feedback. In short, Would students receive feedback better from one or the other and how would they act on that feedback?

The results?

[blockquote source=”Annie Murphy Paul”]The computer program appeared to transform the students’ approach to the process of receiving and acting on feedback.Comments and criticism from a human instructor actually had a negative effect on students’ attitudes about revision and on their willingness to write, the researchers note. By contrast, interactions with the computer produced overwhelmingly positive feelings, as well as an actual change in behavior—from “virtually never” revising, to revising and resubmitting at a rate of 100 percent. As a result of engaging in this process, the students’ writing improved; they repeated words less often, used shorter, simpler sentences, and corrected their grammar and spelling. These changes weren’t simply mechanical. Follow-up interviews with the study’s participants suggested that the computer feedback actually stimulated reflectiveness in the students—which, notably, feedback from instructors had not done.

Why would this be? First, the feedback from a computer program like Criterion is immediate and highly individualized—something not usually possible in big classes like those at Alexandria University, the site of the study by El Ebyary and Windeatt. Second, the researchers observed that for many students in the study, the process of improving their writing appeared to take on a game like quality, boosting their motivation to get better. Third, and most interesting, the students’ reactions to feedback seemed to be influenced by the impersonal, automated nature of the software.”[/blockquote]

There’s a lot to crunch through here. Part of me wonders if generational components affected the differences in responses? What social elements allow for students to respond better to a machine rather than an individual?

And does it really matter, as long as their writing improved?

The Role of Chance in Outcomes

Are the successes in my life a result of ability (the quality I produce) or chance?

NPR kicked off a story about the Mona Lisa. It’s a popular painting – the most popular painting. Each year thousands swarm the Louvre in Paris to get a glimpse of her enigmatic smile. The Mona Lisa is famous.

But is she famous because she’s a spectacular work of art? Or is she famous because she’s the result of chance encounters (i.e. because she was acquired by the King of France, displayed at the Louvre, a product of a masterful inventor who was acknowledged as such by Italian authorities)?

This question – chance vs ability – has historically been untestable. We live in one reality and in this reality the Mona Lisa is famous.

Until Princeton professor Matthew Salganik decided to come up with a rather genious experiment. He would create multiple identical worlds online filled with the same works of art, get thousands of people to pick which work of art they liked best, and if the same works of art rose to the top in each world, then we could reasonably conclude it was the actual art work that determined its greatness.

The experiment

From NPR:

[blockquote source=”Good Art Is Popular Because It’s Good. Right?”]

To test how much of success should be attributed to chance and how much to quality, Salganik created a website that randomly funneled the 30,000 teenagers he recruited online into nine identical worlds.

Each of these worlds exposed the teens to 48 songs from emerging artists — bands that hadn’t yet been signed so were totally unknown to the teens. The deal was that after listening to the songs, the teens could download the ones they liked best for free.

Now in one world — the control world — they couldn’t see which songs their peers were downloading so there was no social influence. But in the other eight, the teens could see which songs had been downloaded before, so they knew what other people thought was good.

“So we had the exact same 48 songs competing against each other, we had the exact same initial conditions, everything starts with zero downloads, and we have indistinguishable groups of participants, because they were randomly placed into the world,” Salganik says.


And what did he find?

[blockquote source=”Good Art Is Popular Because It’s Good. Right?”]

Different songs become popular in different histories — and not in small ways, either.

….there are many different things that have an impact on success and failure — money, race and a laundry list of other things — and after this work, which one person in the field described as a seminal paper, Salganik went on to do similar studies with parallel worlds that suggest that quality does have at least a limited role. It is hard to make things of very poor quality succeed — though after you meet a basic standard of quality, what becomes a huge hit and what doesn’t is essentially a matter of chance.


What does this mean?

I think studies like this give us moments to pause, look at the world around us, and consider the implications and how these implications affect our approach to the people around us.

Take my life as example. In terms of the biggest element of chance – who my parents are – I scored the jackpot. Middle class, a reasonably healthy pool of genes, educated, and willing to throw considerable resources at the development of their children. I try not to confuse the fact that I was born on third base with those who had to run, on their own, from the start of a pitch.

Or consider my current position as Director of Technology. One reason I landed this wonderful gig was that members of Hamilton CSD viewed a presentation by me at an informational session. The curious thing about that day was that I wasn’t supposed to present. My boss at the time had pneumonia and I filled the spot. Pneumonia as a matter of a chance.

If you think that much of a persons success (or lack of success) is up to chance, perhaps it allows you to empathize with an individual better. Or as Salganik points out, you treat them better.

I Like It’s Simplicity: OKRs

Slate has a short write up on Google’s internal grading system and I’m intrigued. They’re called Objectives and Key Results and their simplicity appeals to me, especially given the amorphous nature of this wonderful job.

Maybe I can get Keith to try them with me?

Questions for January

Heading into January, here are a few of the questions I’ll be wrestling with professionally and personally professionally. Hopefully I’ll post about some of the search for answers.

What is a fair cost for Chromebook support?

I’m in the process of writing an RFP for IT services and this is a pretty critical question as we continue to invest in Chromebooks. I’ve seen a wide variety of answers to this question. Mostly though, people don’t really know given that Chromebooks are pretty new to the scene (and I’m really curious to how much life we can get out of them).

Which professions will be replaced by robots?

I’m giving a presentation at OETC looking at this issue. Sort of scary (the question, not the presentation). And I find it curious that educators don’t ask this question, especially given that we’re in the business of preparing kids for the future.

What’s happening to MOOCs?

Related to the previous question I suppose. MOOCs seem to be imploding. Are MOOCs 2.0 around the corner?

Is it ok to distract my kids with games (as long as they’re educational) so I can get work done on snow days?

Because otherwise that OETC presentation won’t get finished.