Charismatic Teacher = High Evals, but not High Learning

EvaluationsInside Higher Ed. has recently reported that a rather interesting study on student learning conducted by a group of psychologists supports what most of us who have been teaching for any length of time already knew all along: student perception of learning and student learning are two different things:

The researchers asked two groups of students to sit through the same lecture delivered in radically different styles. When asked afterward how much they felt they had learned, those who had experienced the more accomplished performance believed they had learned more than the second group. However, when tested, there was little difference found between them, with those attending the “better” lecture barely outperforming their poorly taught peers.

So students think they’ve learned more when they’re taught by more charismatic instructors, but when tested, they really haven’t. They learn about the same as those taught by instructors with poor self-presentation skills. I would always encourage faculty to strengthen their presentation skills, of course, just to make the learning experience more enjoyable, but we shouldn’t kid ourselves about how much this really means in terms of student learning.

The implications here are twofold:

1. Student course evaluations by themselves are no measure of teacher effectiveness, but we already knew that.

2. The hottest new thing — MOOCs — used by themselves reinforce ineffective educational practices:

Mazur said that despite modern technological advances, universities had work to do to redesign their lecture halls and rethink their teaching methods. “What is really worrying is that people are jumping on the massive open online course bandwagon, taking a failed model and putting it online. We need to rethink how people approach teaching,” he said.

I don’t see much difference between a MOOC and a 400 seat lecture class, but there’s a lot of difference between a MOOC and a smaller class involving direct student/faculty interaction that is  heavy on instructor feedback, as the latter actually involves teaching and not just lecturing.

Now, let me share something even worse: high student course evaluations are negatively correlated with deep, long-term learning. In other words, when teachers engage in practices that help students retain what they’ve learned, they’re punished for it with low student evaluations according to this study:

Student evaluations are positively correlated with contemporaneous professor value-added and negatively correlated with follow-on student achievement. That is, students appear to reward higher grades in the introductory course but punish professors who increase deep learning (introductory course professor value-added in follow-on courses).

If you’re interested in the original report behind the IHE article, I’ve provided it below:

Are Teachers Entertainers?

children-playing-250x249I’ve been following a discussion on LinkedIn in which one instructor has taken the position that teachers are entertainers because learning should be fun, and another is taking issue with him — he maybe agrees that learning should be fun, but he’s doubtful that teachers are entertainers. I think it wouldn’t hurt to consider how we use the words “fun” and “entertainment” when we think about classroom experiences and instructor’s roles.

We tend to say that we’re “entertained” by films, plays, concerts, stand-up comics, etc. Actors, comics, and musicians are entertainers. Being entertainers, they perform while we watch, and we enjoy what we’re watching. The important thing here is that when we’re being entertained, we’re passive. However, we have fun at the beach, the carnival, or when we’re playing games. When we’re having fun, we’re active. So by these definitions, whenever instructors are acting like entertainers, students are passive observers, but when students are having fun, they’re engaged — they’re doing something. So if instructors are entertainers, students aren’t having fun.

But to say that students should have fun in the classroom doesn’t seem quite right either: “fun” seems mindless (though it doesn’t have to be), and mindless isn’t what a college classroom should be. I think we should abandon the notion of fun altogether and adopt the idea of pleasure instead, so that we think about learning as an advanced form of pleasure. I’m drawing here from Book VII of Aristotle’s Nicomachean Ethics, in which he distinguishes carefully between the pleasures of the body and the pleasures of the soul:

Neither practical wisdom nor any state of being is impeded by the pleasure arising from it; it is foreign pleasures that impede, for the pleasures arising from thinking and learning will make us think and learn all the more.

There are pleasures that impede growth and pleasures that foster growth, and learning is one of the pleasures that foster growth. Aristotle calls these pleasures “natural.” They’re like eating. We eat so that we can live, and pleasure is a natural by-product of our eating, but not the purpose of eating. Learning is supposed to work the same way. We learn so that we can live and grow, and the natural by-product of learning is pleasure, but pleasure isn’t the purpose of learning. It’s just a by-product. When students truly learn, when a light comes on and they see something — maybe even the whole world — in ways that they haven’t before, those students experience a deep pleasure that makes them want to learn even more. As we learn, our ability to gain pleasure from learning grows with us.

Strategies for increasing the pleasure of learning:

  • “Big picture” teaching — teaching that relates the material to students’ own lived experiences, prior knowledge, and future lives.
  • Minimizing (though perhaps not eliminating) lectures and involving students in more activities: have them do something with the knowledge they acquire.
  • Problem solving activities are perhaps the best. Pose a problem for students to solve with the material at hand.

These strategies work, usually. Are they limited?  Of course — by reasonable class sizes, by the amount of instructor support, and by students’ prior learning experiences, which usually involve having all of the imagination and pleasures of learning beaten out of them by test preparation instruction — and by hours of mindless fun in front of a television set or playing a video game. But I’ve also seen students resistant to learning have their heads turned by these strategies.

 

On Being a Student (as a Human Being)

The chart below lists educational spending in twelve industrialized countries, comparing each country’s spending to student performance:
U.S. Education versus the World via Master of Arts in Teaching at USC
Via: MAT@USC | Master’s of Arts in Teaching

Educational spending for just these twelve countries combined is about $1.8 trillion, representing a massive investment of time, money, and resources to educate our children. And yes, this chart is only spending on school-aged children. Since the cutoff point is age 23, it doesn’t represent higher ed spending at the graduate level. And it doesn’t actually represent our total spending on education, which includes a host of para-educational  industries involving numerous vendors for everything from food to technology, the administration and scoring of certification tests, video production, spending on supplies, child care, and more. Educational institutions are facing an increasingly aggressive barrage of vendors hyping new technologies, and too many administrators are looking for technological magic bullets — because education doesn’t seem effective enough, salesmen selling new tech are more convincing than teachers saying they would like some support (but they need it).

How is the United States doing? We’re number one in spending (of course), number three in literacy, number five in number of years spent in school, number ten in math, and number nine in science — and okay, now I see why Obama is emphasizing math and science education.

Because we’re spending massive amounts of money already, and then trying to fix our educational shortcomings by spending even more massive amounts of money (just in new ways), I think we’re forgetting a few important things here.

First, education is a naturally occurring phenomenon. It’s not even limited to human beings — here’s a video you’ve probably already seen (insert shamelessly cute puppy video):

The most naturally occurring educational practice in nature is simply mimesis, or imitation. Mommy dog wants to teach puppy dog how to walk down a set of stairs, so it goes up and down the stairs until the puppy learns. We learn how to do something by watching others do it. Birds and cheetahs teach their young to hunt; herbivores teach their young to run away:

Human education has been going on — has been naturally occurring — for as long as there have been human beings. It’s become increasingly specialized, of course, with the advent of print, the development of new sciences and technologies, and the diversification of the workforce, but as of the late twentieth century the only technology actually needed for teaching is something to read, something to write with, and something to write on. So — get ready for the latest in cutting edge educational technology — I introduce you to the pad and pencil:

images

Are you impressed yet? We could add a calculator too, but well into the 1970s people were still using slide rules to do some of their advanced calculations. Anyone remember slide rules?

SlideRule

Because my father is an electrical engineer, slide rules are a childhood memory for me. But, I never learned how to use one. I grew up using calculators for advanced math.

So I’ll grant you a pad, a pencil, and a calculator, and you can make that calculator the most advanced graphing calculator that you want. That’s more than the minimal tech that we needed to educate our students throughout most of the twentieth century — which was the century that began space exploration, developed nuclear weapons, invented the computer, the internet, lasers, advanced study in genetics, magnetic imaging… the list goes on. For the most part, there was no such thing as online education until the last ten years of the twentieth century. There were no MOOCs, no educational research, and no brain research supporting it. For most of the twentieth century most of our educators didn’t have degrees in education. Most students in the US didn’t even work on computers until the last twenty years of the twentieth century. We made these advances using, horror of horrors, a host of “failed” practices, such as lectures delivered in lecture halls, but it does appear that students managed to learn.

Why? Because education is a naturally occurring phenomenon, and as it occurs in nature, human beings were teaching other human beings.

Of course, people even today are educating students with a lot less than what most people reading this blog have now.

What I would like us to do is forget for a moment about education as an industry, and education as an institution, and think about education just as human beings. You might remember sitting in your mother’s or father’s lap as they read to you. You might remember learning how to throw a ball. You might remember enjoying a good book, learning how to draw, finally understanding math. The point is that learning is fundamentally pleasurable. Learning is one of our great natural sources of pleasure.

It can be that way in school, too. I would like to encourage students reading this blog to try to extend the pleasures of learning to the classroom, and I’d like to encourage teachers reading this blog to consider how we can encourage the pleasures of learning within institutional settings. I think this change will require thinking very differently about education, though — in some cases, it might mean completely changing our thinking about education.

What I think kills the pleasures of learning is the fact that we’re forced to go to school for twelve years or more, and then once we go there, we’re made to do work, and that work is then graded — which feels like being in state of continual judgment. I would like to suggest adopting three attitudes that will help us recover the pleasures of learning in the classroom, and they mostly affect how we view the grades that we earn in school and the work that we do in school:

  • First, students do not work for teachers. Students work for themselves, and teachers work for the good of students. Students are not the teacher’s employer, however — teachers are employed by and accountable to a system, but their work within that system is for the benefit of students. When students think of their teacher as their employer, their time in school is nothing more than putting in time. What I tell my students, though, is that their minds are like muscles: when they work them, they get stronger, and when they leave them alone, they atrophy. In practical terms, every time teachers assign reading or writing or any other kind of homework, they are creating work for themselves. Assigned reading is reading the teacher needs to do and  assigned papers are papers that the teacher needs to grade. Teachers don’t receive any personal benefit from grading student work — trust me on this. Teachers who assign meaningful work and provide meaningful feedback are working for their students. Teachers who do not are working for themselves. The doing of the work and the grading of it is all for the student’s benefit.
  • Next, teachers grade student work, not students. I quit letting my students tell me that “I gave them an A on this paper” a long time ago. First, I didn’t give them anything — if they received an A, they earned an A, and I try to help them understand their grade by reviewing and explaining my rubrics and applying it to their papers consistently. But more importantly, I didn’t give them a grade of A. I assigned that grade to their paper. I’m not grading the student, I’m grading the student’s work.
  • Furthermore, teachers grade student performance, not student ability. Did you write a really bad paper? Did you write it the night before it was due? Do you really think that’s your best work? Of course not. But, sometimes it is, and sometimes our best work is bad, but even then, doing bad work is part of the learning process. Since teachers don’t grade students, but student work, grades are at most an indication of student’s progress on that specific assignment, not a global assessment of their future potential. An F grade does not necessarily mean an F student.

My suggestions here have to do with developing productive ways of thinking within the system, not with changing the system itself. I do think the system needs to change, and in a lot of ways. You can read my ideas for systemic change in other blog entries.

On Being a Student (in an Institution…)

Before I get around to talking about being a student in an institution (I write about being a student outside of an institution later), I would like you to consider three kinds of machines and how they differ: a hammer, a photocopier, and a computer.

These three kinds of machines represent three levels of complexity.

Three Kinds of Machines

English: A standard household claw hammer.
English: A standard household claw hammer. (Photo credit: Wikipedia)

A hammer is an example of the simplest type of machine. You can perform maybe three tasks with a hammer (beat things with it, pull nails, and use it as a paperweight), but it’s really just designed for two specific tasks — beating things and pulling nails. It’s very good at one or two tasks, but only those. A hammer actually doesn’t make a good paperweight, by the way. Because you can do other things with it, you’ll want to take it off your papers and use it, and then it quits being a paperweight. The best paperweights are useless for almost all functions but being paperweights. And of course if something really is only good for just sitting there, it may as well look good while it’s serving that purpose. Most importantly, the hammer is an inert object. It just lays there until you pick it up. It doesn’t do anything at all until you pick it up and do something with it.

Now, you might say that you can do many different things with a hammer, like chip wood, drive nails, or beat holes, but those are all just different ways of beating with it. You might say that you can throw a hammer, but unless you’re throwing the hammer at something, you’re throwing it for no reason at all (at least, no reason peculiar to a hammer — if you’re in a distance throwing contest, you could throw hammers, rocks, or frisbees), and throwing a hammer at something is just beating on it from a distance.

The next kind of machine, machines like photocopiers, are a little more complex. They too only perform one task, though. The photocopier in my office can scan and copy, but those are just two different ways of doing the same task, which is reproducing an image. It can staple and sort, but those tasks are only important because they are related to the copier’s primary task, which is reproducing an image.

When the spawn of Satan bothers to work at all, that is.

Photocopier of MaliceWe might think that what makes copiers more complex than hammers is the proliferation of moving parts. Since both copiers and hammers seem designed to perform a single task, it’s hard to compare them only on the basis of the work that they do. We might say that the single task that copiers perform is more complex than the one or two tasks that hammers perform, and that would be true, but I wouldn’t emphasize just number of moving parts as the most important order of complexity, or just the complexity of the task. I want to emphasize something else: copiers can be programmed to do their single task on their own, and once you’ve programmed them and started them running, they run until they’ve completed the task. So I can program my copier to make 500 copies double-sided and corner stapled, and once I’ve done so, it chugs away making copies and stapling them until the 500th copy is finished.

When the spawn of Satan bothers to work at all, that is.

This level of functionality is very different from a hammer, which just lays there until someone picks it up and uses it.

The most complex machine I’d like us to consider is a computer. Computers, like copiers, also need to be programmed, but their programming can get very sophisticated. Its normal functioning involves performing a highly diverse number of very complex tasks all at the same time, most of which the user is unaware — programming that runs beneath the user interface. Computers, as the most sophisticated type of machine under consideration here, can perform the most number of tasks and can work the most independently.

Now let’s consider the difference between any of these machines and a human being. Human beings can be programmed, but we can also program ourselves, and we can choose our programming. Furthermore, human beings are not limited in their behaviors to their programming — we can act beyond the parameters of our programming (in other words, creatively), and we can act in ways contradictory to our programming (in other words, annoyingly — but we can be deliberately annoying, unlike a computer, which is just passively annoying). Human beings are also capable of being self-directed. We can choose what to do and then go do it without any external stimulus. Even the most complex machines — computers — do not choose when they run independently. They’re just running established routines. When they act outside of those established routines, they crash.

Like the spawn of Satan that they can be.

Four Kinds of Students

I would like to suggest that our four options here represent four different types of students: hammers, photocopiers, computers, and human beings, and that student attitudes and institutional practices lead students to be one of these four types.

1. “Hammer” students do nothing until they’re forced to do it. School isn’t for learning but for earning grades. Curiosity and the potential for self-development play no role in this student’s motivation at all. This kind of student will do just what they are told to do and no more. They will do it when they are told and at no other time. Until they are told, they will lay there and do nothing. They are completely passive learners.

  • Institutional practices encourage students to behave like hammers when assessment drives education: students are not human beings developing their intellectual, social, creative, and emotional potentials, but are just test takers. The purpose of teaching in this model is to have students earn high grades with high test scores, and lessons offer no motivation for learning but test performance.
  • You might want to note that hammer students are still students. They participate in the educational task rather than resist it. They just do so as minimally as possible. Students who actively resist the educational task are prisoners. So are their teachers.

2. “Photocopier” students will work on their own to perform just the task(s) that instructors give them, but they won’t work beyond their given task. They repeat and repeat and repeat, but that’s it — they do not innovate, add, or create in relationship to the assigned tasks.They are better than hammers in that they have willingly accepted the educational task and will work on their own, but they are largely passive learners.

  • Institutional practices that encourage students to be photocopiers include teaching methods that emphasize only the acquisition of knowledge rather than the application of knowledge or the development of skills. Acquisition-based teaching requires students to take in information and then spit it back out in its original form. The more accurately the student can repeat acquired knowledge, the higher the student’s grade. The human mind is here being treated like a photocopier with no creative or critical potential at all.
  • Students taught to be photocopiers often want step by step instructions for all assignments. They feel anxious when they’re given a goal without being told exactly how to meet it.

3. “Computer” students are often A students. They can work on a variety of complex tasks independently, having developed a number of skills that they have learned to integrate into multiple kinds of tasks. They are capable of working on their own. Sounds great, doesn’t it? The problem here is that these students have not developed their potential for creative or critical thinking. They may write very competent, accurately documented, and grammatically correct papers but have problems with thesis development. They may have learned to write a thesis, but their best work is only a combination of what they’ve already been given. They never surprise their instructors, and they often tend to focus on figuring out what the instructor wants to hear and repeating that back to them in their papers. Depending upon the class, doing so can be a very complex task.

  • Institutional practices that encourage the development of this kind of student include limiting work to just one kind of methodology, or critical paradigms to just one or two — in short, students are not exposed to a meaningful diversity of ways of thinking about a topic or performing a task. Most of all, they are never encouraged to take risks in their thinking, to be creative. Instructors who write assignment instructions that tell students how to write their papers in significant detail (e.g., answer these questions about these possible literary works using this methodology) may be unintentionally derailing the development of student creativity and critical thinking skills.

4. Students who are fully developing themselves as human beings within an educational context see learning as play. Not frivolity, but serious play, the kind of play that creates new things with old materials, that relates existing material to the outside world and to one’s personal life, that changes and transforms and sees new possibilities for course material.

  • Institutional practices that encourage the development of this kind of student emphasize critical thinking, thesis development, creativity in thought, and problem solving (especially by posing impossible to solve or open-ended problems. Solving problems defined by the methodology is computer thinking — students may follow a complex routine, but they’re still following a set routine rather than thinking on their own). The best instruction explains the reasons for the class within the context of a student’s discipline and overall education, and it relates course material to big questions whenever possible.

Vending Machines

How Do We Respond?

First, I need to add a paragraph here that was not in my original draft. One reader reminded me that students can be photocopiers in one class, computers in another, and human beings in a third. That’s absolutely right. These are different ways that students postion themselves, or are positioned by institutional practices, in relationship to a specific course. Even hammer students may not necessarily be hammers in all of their classes. So while I’m describing four different kinds of students, I don’t mean to imply (though I probably have) that each individual student is one of these four types in some globally-defining sense of the word. It’s best to think of these four types of students as four types of student responses to material that can vary for an individual student from class to class.

If you are reading this blog as a student, I would like to encourage you to try to be a human being in all of your classes, however they are taught. Good students make the most out of even bad classes because they are driven to learn, not driven for grades or driven by their instructors. What you learn and do in class is of no benefit to your instructor. Your education, first and foremost, benefits yourself. Your instructor is not your employer — he or she is working for your benefit in class, as the work that your instructor assigns benefits you, not the instructor. For more details, read my next post.

If you are reading this blog as an instructor, I would like you to ask yourself about your classes — and what kind of student you’re creating with them. Are you treating your students like simple machines, complex machines, or human beings? I know that some classes, especially when taught to some student populations, need to be run on the lower end of the scale rather than the higher. Some students need to start out as hammers or copiers and then move up. I think courses designed like ladders — in which students attempt higher degrees of complexity as they move through the course — are best designed for beginning, introductory, or remedial students. We also need to consider this hierarchy in our curriculum design. By the time students are attempting 300 and definitely 400 level classes in their major, most assignments should be designed to serve the highest possible developmental ends.

I hope I’m being clear, though — I’m not laying the responsibility for student learning solely upon the instructor or solely upon the student. Students are responsible for being the type of student that they have chosen to be, and instructors are responsible for their educational practices and the type of student that they create with them. The success or failure of educational practices is dependent both upon the student and the instructor.

I would also like to suggest that this paradigm can help us define different management styles too. What kind of human being are we creating by our management practices? What kind of employees do we have, and what kind should we expect given institutional policies and our treatment of employees?

Understanding Four vs. Six-Year College Graduation Rates

If you’ve  been shopping for colleges you might have read about four and six year graduation rates. These rates are indicators of what percentage of entering freshmen finish college four years after starting and what percentage of entering freshmen finish six years after starting. Graduation rates beyond six years aren’t followed very closely, as most students finish within six years or not at all, and from the numbers I’ve seen, five and six year graduation rates tend to be very similar.

Time-to-Graduation too Often Overlooked” by Beth Akers and Matthew M. Chingos addresses the issue of time to graduation as a significant concern for students considering a college. They authors are concerned about the added costs involved in graduating six years after starting rather than four. Akers and Chingos provide quite a bit of useful data indicating that four and six year graduation rates can vary widely among institutions regardless of the institution’s quality on other measures, including the strength of incoming students’ ACT or SAT scores. Apparently, there’s no clear correlation among student scores, institutional quality, and differences between four and six year graduation rates:

One might assume that general metrics of college quality are good proxies for all sorts of outcomes, including time-to-degree. For example, perhaps institutions with good six-year graduation rates also have good four-year graduation rates so it doesn’t really matter which one students use.  But it turns out that this is not the case.  The average time-to-degree varies widely, even within institutions that seem to be of similar quality based on other measures.

What I would like to do here is help students think through why graduation rates might differ from school to school and how to evaluate those differences.

First, I agree with the basic point of both articles: students need to do all that they can to graduate in four years, because by doing so they’re saving money. These costs savings take the form of not paying for two years of additional tuition, room, board, and fees, and also in the form of two fewer years of lost income. The sooner you start work after college, the sooner your investment in college starts paying you back.

But it’s very important that we understand the range of possible reasons for these differences. Time to graduation is always just a matter of basic math: any four-year college that requires 120 credit hours to graduate requires 30 credits per year, or 15 credits (typically five classes) per semester. If a student passes five classes per semester every semester they will graduate in four years, period. So if 15-20% of a college’s students need an additional two years to graduate that’s not because of the degree structure or the college itself, unless significantly more than 120 credits are required for graduation.

Typical reasons that cause students to take more than four years to graduate include:

1. Failing or dropping classes — do this four or five times and you’ve added a semester to your completion time.
2. Sports injuries, if they cause you to drop out of school for a semester or more.
3. Serious family issues or illnesses, if they cause you to drop out of school for a semester or more.
4. Switching majors during or after your third year.
5. Adding minors, especially multiple minors.
6. Double majors.

At the end of this current semester of teaching, I have students who fit most if not all of the above categories.

Institutional reasons include:

7. Credit-intensive majors. The college usually can’t do anything about these requirements, as they are field-specific and are usually designed to meet state or federal requirements. For example, education majors at my current institution are all required to do coursework about equivalent to a double major. Those requirements are imposed by the state. Just know what you’re getting into if you select one of these majors.
8. The school doesn’t accept many or any transfer credits or students (i.e., most students start as freshmen), which raises the likelihood of more than four years to graduation. Transfer students will finish in two or three years, and if they graduate in four, they may have done it in four years at that institution but have taken much more than four years to get their degree.

Since my original draft, readers have suggested the following reasons for increased time to graduation.

9. One reader suggested that time to graduation may be increased if an institution has a significant population of non-traditional students, such as working adults. Working adults are often unable to attend full time so take longer to complete. My own alma mater, Rollins College, has an undergraduate evening program called the Hamilton Holt School that is largely populated by non-traditional students along with graduate programs with a similar student population. I suspect this school contributes to the 15% increase in six year graduation rates over four year graduation rates at Rollins College. I was one of these students. It took me nine years to complete my B.A. at three different institutions. I just happened to finish at Rollins College.
10. A college with a high population of remedial students will have longer time to graduation, as these courses often do not count for credit toward graduation. If a student has to take four remedial classes in two or three different subjects, that’s a semester added to their time to graduation. This problem may not be a negative if the college serves its remedial students well.
11. Are the college’s general education courses filled to overflowing? Do they offer too few sections for the students who need them? This problem is institutional and can extend time to graduation.
12. Does the college serve many veterans or those in the military? They often have to take a semester or more off for service. It’s a good thing that the college does so, but serving this population will affect a college’s stats on time to graduation.

So even if you see that there’s a big disconnect between four and six year graduation rates, you still don’t know what that means. If it’s because many students fail their classes, that means the school has meaningful academic standards but a weak student population, which is a mixed signal (bad in that students are weak, good in that at least the school has academic standards). If there’s a big difference between four- and six-year graduation rates because many students take credit-intensive majors or double majors, that’s a sign that the school has a high population of very motivated students, or very strong programs in credit-intensive majors, so that’s a good sign. Schools with a high four year graduation rates may have low academic standards, and faculty may be pressured to just pass everyone through, in which case a high four-year graduation rate can be a sign of a bad school. If the school has an evening program populated by working adults, then differences in four and six year graduation rates don’t tell you anything at all.

So finding out why these differences exist is what matters. The best thing to do is to consider this measure alongside many others, and compare your potential college’s graduation rates to national averages and other potential colleges. If they’re above the national average for their cohort, you’re on safe ground.

In general, four year private non-profit institutions have the best four to six year graduation rates, and for-profit institutions have the worst, running at about half the rate of public and private non-profit institutions.