Essay Robo-Graders Reward Form, Not Content

If you were appalled by the recently released study results that computers can score test essays as well as humans, take heart.  Machines are fast, but they’re superficial.

As NY Times reporter Michael Winrip notes in “Facing a Robo-Grader?  Just Keep Obfuscating Mellifluously,” ETS’s e-Rater can grade 16,000 essays in 20 seconds.  But as an MIT’s director of writing discovered, the computer falls for cheap tricks–big words, long sentences, empty transitions, and outright lies.

Instructors of writing still have to face the stack, but at least they have some job security.

…and April Is Especially Cruel for Those Who Assign Papers

Those of you who’ve taken the WI workshop series have heard our spiel about how to save time when dealing with papers.

Although it’s likely too late to redesign assignments, you might

  • Consider separating commenting from grading.  Research shows that students usually don’t carry writing feedback from the end of one class into the next semester.  Ever write comments on a paper due the last day of class–and find that many of your students don’t pick them up?  Sigh.

If you give students feedback on a paper in progress, however, they’re more likely to engage with it in the hope of improving their grade (or their learning–yes, we have those students, too).

  • Consider commenting early on a chunk.  If you have a paper due at the end of the semester, you might give some feedback on the thesis or introduction or give some feedback on a specific issue, such as organization.  You might collect a paragraph and note all the grammar errors that you expect to be eliminated throughout the final draft.
  • Consider making comments on the final paper only if students express interest.  Some professors ask for a self-addressed stamped envelope as a commitment.  Others schedule conferences (if you explain that there’s no penalty in just wanting a grade, a lot of students won’t sign up) and hand back the paper then or just comment orally.

You still have to grade, but at least you won’t feel as if you’re wasting your time casting pearls of wisdom that no one will bother to pick up.

Teaching Tip: Grading, Like it or Not

Grading is generally the least favorite part of teaching for me, and this blog entry on the 5 stages of grading suggests that I’m not alone (thanks to Brian Flanagan for posting). A quick Google search turned up “5 things I hate more than grading.” Some folks couldn’t come up with five things worse than grading; most mentioned life-threatening illnesses or surgery without anesthesia. And of course, there are the many Facebook entries that academics post this time of year e.g. “Grading is sucking out my soul.” With mid-term grades due this Friday, chances are good that your soul is a little battered about now too. Why do we find grading so awful? What can we do to make it better?

I think there are four main reasons why most of us hate grading:
1. We don’t like to judge people. As instructors, we are both coaches and judges, and most of us got into teaching for the coaching part, not the judging part. Assigning a grade doesn’t really seem to help anyone learn; it just rates the amount and quality of their learning. Plus, since learning is such a complex task, assigning a single number or letter that summarizes the total learning (or lack thereof) seems inadequate and not particularly helpful.

2. Grading policies are hard to formulate, and we’re never sure we have it right. We struggle to balance our grading criteria between fairness and rigor. We don’t want to be too easy (grade inflation alerts go off) or too difficult (student evaluations tank). Do we factor in improvement? Effort? Achievement alone? What about second language learners? The dilemmas just keep coming.

3. Grading can make us question ourselves as teachers. Exams or papers that don’t meet our expectations are discouraging, even depressing. Sometimes we blame the students e.g. Didn’t they study? How did they get admitted? Sometimes we blame ourselves. Perhaps all semester you thought you were doing a great job (or at least a competent one) and now you are wondering if you’re meant for this profession.

4. Grading can feel like a fight. Students complain, they challenge their grades directly, they attempt to cheat or plagiarize, they focus on the grades at the expense of the learning. It can feel like our job is to guard the tower of academe, defend ourselves and catch those miscreants.

How can we make it better?

Accept the role conflict inherent in grading, and make this explicit to yourself and your students. Separate out the coaching function from the evaluation function, and decide which one you are doing at a particular point in time. You might choose to only give comments without a grade or tell students “this is the grade you would get on this work right now” and then allow revision. You can give practice quiz or test questions with feedback before the real thing so that students (and you) get a sense of how they are doing and what their likely grade will be if they don’t do more.

Remember that grades are always subjective. You are an expert judge, but another expert judge in another context might have a different opinion. That’s OK. Establish your criteria and standards, communicate them to the students and use them to the best of your ability. Then move on.

Understand what grades mean to your students, and what an emotional issue they are for many of them. Talk and listen! Get student input on your standards and criteria, let them make suggestions and be very clear about what grades mean to you. Using rubrics or other grading schemes can help your students understand your grading decisions and lessen the complaining. You can even use rubrics for essays and share these with students. Obviously, you set the standards and criteria but you want your students to understand them fully and have some idea of WHY you are asking them to meet these criteria.

If your students are consistently failing to live up to your hopes for them, you will want to examine a couple of possibilities.

Are your expectations unrealistic? Do students have the background and preparation to achieve at the level you are expecting, or do they need additional support? You may need to change assignments, provide additional support or look at course pre-requisites. I am not advocating “dumbing down” a course here – but often we do not realize how difficult a task really is – especially when it comes to reading college level material and applying it.

Are your evaluation tools sharp? Look at your assignment or your test questions, and get some input from a colleague and from some students too. Is it clear what you want them to do? Vague questions or assignments frequently lead to confusion and difficulty grading the resulting products.

Are you requiring students to USE information before you assess them? If you lecture for several weeks and then give an exam, you have no idea if the students really understood anything you said until you start grading. And then it’s too late to do much about it. Ask students to demonstrate their understanding during class so you can check in on their learning. That way, you can catch errors and omissions while there is still a chance to make a difference. This doesn’t need to take a lot of time. Asking students to regularly write down questions they have at the end of class, using a brief exercise, or asking a few clicker questions lets you spot check understanding. Once you have taught a course once, you will be able to identify the concepts students struggle with the most – why not create an extra exercise to reinforce those problem areas? (And warn the students about those sticky spots, so they can spend extra time where it’s needed.)

I don’t think there’s any way to make grading easy, but I do think we can take a bit of the sting out. After all, nothing makes you feel prouder than reading a really good run of student exams or papers and knowing that they got it. If we can increase the percentage of good work that’s being done, we’ll be doing the students and ourselves a great service.

What do you hate most about grading? What do you do to make the experience better?

Fired for Tough Grading

Louisiana State has removed a demanding biology professor from an intro course midsemester and raised the grades in the class, reports Inside HigherEd.  Did administrators violate the prof’s academic freedom or rescue students from an untenable classroom situation?

For a data-driven review of grade inflation in higher ed, check out the splendid graphs at  Stuart Rojstaczer’s GradeInflation.com.

Teaching Tip #12 Situational despair: The end of semester paper

So, you gave a written assignment, with some directions and maybe even a rubric.  Now you’re sitting in your favorite grading spot and reading your students’ work.   The MIA thesis, the failing logic, the absent evidence, the atrocious editing – where did it all go so wrong?  Unfortunately it’s probably too late for this semester’s batch but how can you avoid repeating this experience?

First of all, realize that you’re not alone; I’ve experienced it and my conversations with faculty suggest that most of them have too.  I think that most of our despair stems from a common set of assumptions that many of us learned when we were students.  They go something like this:  Professor assigns paper with due date at the end of the semester and basic instructions.  No other communication is necessary.  Student hands in final paper; professor makes pithy and insightful comments.  Student internalizes said comments and applies them to papers at the end of the next semester, never making the same mistakes again.   Writing improves in a steady upward spiral. Grateful students send their donations and their children to the University.

So what’s wrong with this picture?  Quite a lot, as it turns out.  Research on transfer of learning from course to course shows that it does not happen easily or automatically.  Many students who are told to provide more supporting evidence in History 211 are unlikely to provide it in Marketing 301.  They may not even provide it in History 212 without a reminder.  Students need your help understanding the requirements of the discipline and the requirements of your particular assignment before they can meet them.  Passing EN 101-102 is not sufficient preparation for writing papers in every other course.

Students benefit most from feedback that is provided BEFORE the final paper is turned in.  This is a form of “just-in-time” teaching i.e. providing instruction at the moment it is most relevant and helpful.  Requiring drafts and revision allows you to catch errors before students get too far afield, and you also can direct students who need extra help to the LRC before it’s too late.  If you do this up front, the final papers will have fewer problems and you’ll probably be able to grade them faster.  In addition, plagiarism becomes much more difficult when drafts are required, which minimizes another end-of-semester nightmare.  If you’re allowing drafts but not requiring them, you are probably noticing that the students who need the most feedback don’t submit.  That’s why I strongly suggest requiring them.

When it comes to encouraging students to use what they learned in their earlier classes, start by asking/reminding them about what they already know.  Then encourage them to think about how they can apply what they know to THIS assignment, and what they will need to do differently.  Students do not know automatically that the passive voice they’ve been hunting down and killing in English class is required for science writing.  As an added benefit, when you model the transfer of knowledge and skills between courses, students may learn to do it themselves.  **Promotion alert**   If you’d like to improve your quality of (grading) life by helping your students write better, consider attending a Writing Intensive workshop.  You’ll learn lots of helpful strategies that can be used in any class, not just WI.  Contact Sylvia Whitman for details at Sylvia.whitman@marymount.edu.

Now, you may wondering —  what has happened to students today?  After all, you WERE able to transfer information and you DID improve your writing with relatively little feedback and without what some would characterize as “hand-holding”.    And we all have at least a couple of students in each class who DO get it; who think critically, write well and make us feel so much better about our teaching.  Why can’t they all be like that?

As many of you have heard me say, our students are not us.  I sometimes think back to my own undergraduate years (yes, it’s increasingly difficult).  I would have been horrified to turn in poorly written work, but I wonder how many of my classmates were that picky.  For the most part, we who stayed in academia WERE those few students who did the outstanding work.  What was everyone else doing?  I have a feeling that many of them were like the majority of our students now, stumbling along but in need of much more support than they got.  Those students did not have the benefit of what we have learned about teaching and learning over the last 30 years.  Our students can get the benefit of that knowledge, if we are brave enough to try something different, and willing to meet them where they are and not where we think they should be or wish they were.

As a follow up to last week’s post about  final exams,  I couldn’t possibly write a better entry than this one from the Center for Teaching and Learning at Brigham Young University:

http://ctl.byu.edu/the-final-exam-experience/

As always, looking forward to your thoughts.  Procrastinate on that grading and post a few!

Should we grade student participation?

I’m posting this link in lieu of this week’s teaching tip, since I’m down in Richmond learning to create online seminars. I’m really hoping you all will share your opinions and practices on student participation.
http://www.facultyfocus.com/articles/teaching-and-learning/reconsidering-grading-students-on-class-participation/

What’s Your Paper-Grading Practice?

Hofstra University fine arts professor Laurie Fendrich outlines how she tackles a stack of student essays in “4 days, 40 papers.” Team-teaching a large lecture course, Fendrich assigned one small essay, wrote all over it (too much IMHO), asked for a revision, and then awarded checks with plus or minus.  This assignment, however, she’s grading. It’s a 2-3 page essay.

In brief, she skims first, mostly intros and conclusions.  Then she reads a few papers carefully.

She reads 10 essays a day, three at a time before a break.  She allots 10 minutes per page for reading and commenting.  She puts them in piles: best, middle, worst.  She vets best and worst again, then divides the huge middle pile into its own best/middle/worst based most only her comments.

Finally, she assigns grades by pile.

How does her process compare to yours?