Learning – James Rovira

Can We Think Clearly about MOOCs?

Yesterday I posted what seemed to me to be a short series of commonsense assertions that don’t even need stating. However, these obvious claims have now been supported by empirical research. The earth-shattering claims made by this research?

Earth shattering claim no. 1: Students who develop meaningful relationships with their professors are much more likely to do better in life.

Which leads me to take the next, most obvious step:

Earth shattering claim no. 2: MOOCs probably don’t provide an optimal learning experience so shouldn’t be offered for college credit.

What I received in response was a series of counterclaims in a small but friendly twitter war (see also Mark Snyder @radicaltotality and Ray Maxwell’s blog) involving a series of equally obvious conceptual errors, most of them involving how we deal with general claims. These were:

1. “Of course MOOCs allow for human interaction.” The first mistake made here is that of generalizing a specific claim. It’s probably the most common error into which educated people can fall, as the tendency is to think that developing a general principle out of specific data is the most important thing that you can do. However, shooting down an imagined general principle does not address the specific claim. That’s called “attacking a straw man.”

This response mistakes the general principle, for one thing. My claim wasn’t about “human interaction” in a broadly general sense, but about a specific type of mentoring relationship between students and faculty described in one specific study.

2. “This one MOOC has a great deal of interaction among students, the instructor, and assistants.” The first mistake being made here is the belief that exceptions somehow disprove the rule. Once I got this point across, respondents starting saying, “Yes, most MOOCs I’ve taken don’t involve any direct, individual interaction between the instructor and his or her students.” Right. That was my only point. By no means do I mean to criticize any MOOC that is in fact an exception. I think those deserve praise and recognition.

I will say I found it very encouraging to see a couple of students and the instructor rush to the defense of this one course, which was Al Filreis’s Modern and Contemporary American Poetry course on Coursera. I believe this particular course is exceptionally well run by a highly qualified and committed instructor. I’ve even signed up for it myself and plan to take it next Fall, if I have the time, because the course covers some relatively recent developments in poetry that I have yet to engage.

But I’m not done responding to this point yet, as it involves another mistake.

3.“This one MOOC has a great deal of interaction among students, the instructor, and assistants.” What I found out during this discussion was that 30,000 students at least initially signed up for this course. What we need to ask here is, “What do you mean by a ‘great deal’?”

I have been a college instructor for fourteen years, full time for ten. I have been teaching online graduate humanities courses since the Spring of 2008 as part of my full time teaching load. These online courses are all at the graduate level, and the ones that I have developed and regularly taught have included Creative Writing: Poetry, Literary Theory, Survey of British Literature, the Capstone Project, the Comprehensive Exam, and British History (I, II, III).

I teach a 4/4 load over Fall and Spring semesters. My seated and online classes are capped at 25 students each, and all but one of my classes require students to write at least 20 pages of finished academic writing per semester. So, yes, if all four of my classes are ever full, I will have 2000 pages of grading to do in a single semester, every semester. According to institutional policy, I am required to respond to all emails within 24 hours of receiving them, which I would do anyhow apart from this policy. In addition, I keep office hours, and any student at any time can walk in to see me during those times, not to mention my meeting them informally around campus at campus events.

Now my questions about “real” engagement are:

a. How many of these 30,000 students finished the course? (Typical completion rate for a MOOC is only 4%)
b. How many of those who finished had direct, individual attention from a tutor?
c. How many of those who finished had direct, individual attention from the instructor?
d. How often did that direct, individual attention go beyond a response to one or two emails?
e. How many of those 30,000 students feel that they are now in a mentoring relationship with the instructor or a tutor?

I’ve been doing this college teaching thing for too long to believe that the answer to any of those questions is anything other than “not many.” If the instructor replied to 2000 emails over the course of a ten-week term, which would be remarkable, that’s still only a 6.7% response rate to those who initially enrolled. That would be equivalent to me responding to only about 7 out of the 100 possible students that I teach every semester. No, that would not be me doing my job or running my classes very well at all. It might get me fired. But of course, if I had a 96% drop rate I’d only be teaching four students a semester anyhow. And I’d certainly get fired.

4. One last one: “We don’t have to fear MOOCs.” Neither should we have to fear clear reasoning about them. This response diverts attention from clear reasoning from specific detail.

Now I want to be very clear: I’m not knocking this MOOC or this instructor. I believe it is exceptional, and I have enrolled in it. But my point is that it is, in fact, exceptional, and that the rule for most MOOCs is in fact far worse than even what I’ve described here. And most importantly, we’re just talking about sending an email, not the individual attention given to a student by a teacher in a mentoring relationship. Even an exceptional MOOC doesn’t come close to the attention made possible by teachers in face to face settings.

So no, MOOCs should not be offered for college credit as a general rule. Am I saying that MOOCs are bad? No, just that they shouldn’t be offered for college credit as a general rule. As a means of distributing knowledge for free or cheap they’re wonderful. Am I saying that no one can learn or have meaningful learning experiences, even meaningful human interaction, in a MOOC? No, just that they shouldn’t be offered for college credit as a general rule. While these experiences are possible, they are rare, and hardly the norm.

It may be that some math and computer science courses for some student populations may be able to use MOOCs for college credit, but I would like to see research measuring learning outcomes and assessments in these courses first. Generally, they will only work well for higher performing students, I suspect, as remedial, and even many average students, need in-person guidance. If people could process a lot of text and just learn from that, all that they would need to do to learn is read a book. But, unfortunately, most kids coming out of high school aren’t there yet.

Now I would like to conclude by asking you to view the introductory video for Al Filreis’s Modern and Contemporary American Poetry course and then ask yourself one question: Would you rather be sitting in the room with these students or sitting in front of a computer screen watching them interact? I find it amusing that proponents of “traditional” education (which, you know, was responsible for every advancement in human knowledge from antiquity to the present) are called out for wanting more “seat time” when all that MOOC students do is sit in a seat in front of a computer instead of in a classroom. Until we can learn by jogging or by swimming, all learning comes from “seat time.” It’s just a matter of where the seats are located.

But that doesn’t matter. What matters is this question: which is the better learning experience? The videotaped course material itself, I think, demonstrates how MOOCs can’t really compare to a seated classroom. Small online courses that involve regular, direct interaction with the instructor are better, but face to face courses are best of all.

We do in fact have the money to pay for the education that our children and young adults need. About fifteen percent of the US military budget would double our education spending on the federal level. What’s going on is that a very few people who do not care for this country or its people are trying to gut education so that they don’t have to pay taxes to support it, so they’re pushing MOOCs for college credit. That’s really all that’s going on. The MOOC developers themselves may be idealistically wanting to widely and freely disseminate knowledge, but they are being used for other purposes as well.

If you don’t think face to face instruction is better than a MOOC, watch the video and decide for yourself. Again: would you rather be in the room with these students, or just watching them interact? How would you better learn?

Again, I think this course looks great, I think the instructor is committed, and I plan to take the course myself next fall. But let’s think clearly about what MOOCs can and cannot do.

Charismatic Teacher = High Evals, but not High Learning

Inside Higher Ed. has recently reported that a rather interesting study on student learning conducted by a group of psychologists supports what most of us who have been teaching for any length of time already knew all along: student perception of learning and student learning are two different things:

The researchers asked two groups of students to sit through the same lecture delivered in radically different styles. When asked afterward how much they felt they had learned, those who had experienced the more accomplished performance believed they had learned more than the second group. However, when tested, there was little difference found between them, with those attending the “better” lecture barely outperforming their poorly taught peers.

So students think they’ve learned more when they’re taught by more charismatic instructors, but when tested, they really haven’t. They learn about the same as those taught by instructors with poor self-presentation skills. I would always encourage faculty to strengthen their presentation skills, of course, just to make the learning experience more enjoyable, but we shouldn’t kid ourselves about how much this really means in terms of student learning.

The implications here are twofold:

1. Student course evaluations by themselves are no measure of teacher effectiveness, but we already knew that.

2. The hottest new thing — MOOCs — used by themselves reinforce ineffective educational practices:

Mazur said that despite modern technological advances, universities had work to do to redesign their lecture halls and rethink their teaching methods. “What is really worrying is that people are jumping on the massive open online course bandwagon, taking a failed model and putting it online. We need to rethink how people approach teaching,” he said.

I don’t see much difference between a MOOC and a 400 seat lecture class, but there’s a lot of difference between a MOOC and a smaller class involving direct student/faculty interaction that is heavy on instructor feedback, as the latter actually involves teaching and not just lecturing.

Now, let me share something even worse: high student course evaluations are negatively correlated with deep, long-term learning. In other words, when teachers engage in practices that help students retain what they’ve learned, they’re punished for it with low student evaluations according to this study:

Student evaluations are positively correlated with contemporaneous professor value-added and negatively correlated with follow-on student achievement. That is, students appear to reward higher grades in the introductory course but punish professors who increase deep learning (introductory course professor value-added in follow-on courses).

If you’re interested in the original report behind the IHE article, I’ve provided it below:

View this document on Scribd

Are Teachers Entertainers?

I’ve been following a discussion on LinkedIn in which one instructor has taken the position that teachers are entertainers because learning should be fun, and another is taking issue with him — he maybe agrees that learning should be fun, but he’s doubtful that teachers are entertainers. I think it wouldn’t hurt to consider how we use the words “fun” and “entertainment” when we think about classroom experiences and instructor’s roles.

We tend to say that we’re “entertained” by films, plays, concerts, stand-up comics, etc. Actors, comics, and musicians are entertainers. Being entertainers, they perform while we watch, and we enjoy what we’re watching. The important thing here is that when we’re being entertained, we’re passive. However, we have fun at the beach, the carnival, or when we’re playing games. When we’re having fun, we’re active. So by these definitions, whenever instructors are acting like entertainers, students are passive observers, but when students are having fun, they’re engaged — they’re doing something. So if instructors are entertainers, students aren’t having fun.

But to say that students should have fun in the classroom doesn’t seem quite right either: “fun” seems mindless (though it doesn’t have to be), and mindless isn’t what a college classroom should be. I think we should abandon the notion of fun altogether and adopt the idea of pleasure instead, so that we think about learning as an advanced form of pleasure. I’m drawing here from Book VII of Aristotle’s Nicomachean Ethics, in which he distinguishes carefully between the pleasures of the body and the pleasures of the soul:

Neither practical wisdom nor any state of being is impeded by the pleasure arising from it; it is foreign pleasures that impede, for the pleasures arising from thinking and learning will make us think and learn all the more.

There are pleasures that impede growth and pleasures that foster growth, and learning is one of the pleasures that foster growth. Aristotle calls these pleasures “natural.” They’re like eating. We eat so that we can live, and pleasure is a natural by-product of our eating, but not the purpose of eating. Learning is supposed to work the same way. We learn so that we can live and grow, and the natural by-product of learning is pleasure, but pleasure isn’t the purpose of learning. It’s just a by-product. When students truly learn, when a light comes on and they see something — maybe even the whole world — in ways that they haven’t before, those students experience a deep pleasure that makes them want to learn even more. As we learn, our ability to gain pleasure from learning grows with us.

Strategies for increasing the pleasure of learning:

“Big picture” teaching — teaching that relates the material to students’ own lived experiences, prior knowledge, and future lives.
Minimizing (though perhaps not eliminating) lectures and involving students in more activities: have them do something with the knowledge they acquire.
Problem solving activities are perhaps the best. Pose a problem for students to solve with the material at hand.

These strategies work, usually. Are they limited? Of course — by reasonable class sizes, by the amount of instructor support, and by students’ prior learning experiences, which usually involve having all of the imagination and pleasures of learning beaten out of them by test preparation instruction — and by hours of mindless fun in front of a television set or playing a video game. But I’ve also seen students resistant to learning have their heads turned by these strategies.