My revised quiz reflection assignment and the new homework reflection assigment

One of my very first posts on this blog discussed the quiz correction assignments that I use in my intro calc-based physics courses. I have since renamed them to quiz feedback assignments to better represent what I want the students to get out of these assignments. I would actually prefer to name them reflection assignments, but I feel like too many of the first-year science majors would frown on a word which sounds as touchy-feely as reflection so I settled on the likely to be more palatable “feedback”.

Today Kelly O’Shea posted about her experience with test corrections before and after implementing SBG in her classroom. I’m still a year away from my own SBG implementation, but her post got me thinking about an overdue change in my quiz feedback assignments and about introducing this type of assignment for homework as well.

Not all quiz mistakes lead to productive reflections

My quizzes are dominated by clicker-like conceptual multiple-choice questions and short answer questions which require a combination of translation of representation and short answer questions which one could consider to be analogous to an important step in a longer problem. The issue on the short answer questions is that I see a decent number of clerical errors (missing units, silly arithmetic errors, etc) where the student gains no further physics understanding by completing the full process that I ask for the quiz reflection assignments. So I added a clerical error category to the assignment that asks them to correct their clerical error and show how it leads to the correct answer instead of doing the full-blown diagnosis and generalization that I ask for the conceptual errors. The handout that I provide for the students is included in the post if you want more details.

Reflection assignments for homework

For my 3rd year (first four chapters of Grffiths) quantum mechanics course, I am going to offer the reflection assignments to my students for their homework as well as their quizzes. My plan is to mark each part of each homework question according to the following simple scheme:

  • Correct (full marks) – The solution and/or requested explanations are complete and correct. You will not be penalized for one or two small mechanical errors such as dropping a negative sign or a factor of 2pi.
  • Complete (half marks) – The solution and/or requested explanations are complete, but not entirely correct.
  • Incomplete (no marks) – The solution and/or requested explanations are not complete. Examples include a correct solution without the requested explanation, partially completed solutions, and solutions which jump over important steps.
And then the rest of the process looks the same as for the quiz feedback assignments except I will probably ask them to submit a solution which is complete and correct from the point of their initial mistake on to go along with the diagnosis and generalization.

Quiz Feedback Assignment Handout

Quiz Feedback Assignment (Version 2)
Last updated Sept 1, 2011 (Joss Ives)

Our quizzes are designed to be both a learning experience and an assessment of your current level of understanding of the material. For both these reasons, I offer you the opportunity to learn even more and to improve your quiz score by carefully reflecting on your performance to learn from it. Completing this assignment appropriately within two days of the quiz being returned will allow you to increase your quiz score by half of the points that you missed. This is an all-or-nothing assignment. It is intended only for those students who are interested in making a serious effort to improve their understanding. If it is incomplete or not done well, you will not receive any additional points. Late quiz feedback assignments can be submitted any time up until the date of the final exam, but will only earn back one quarter (instead of one half) of the points that you missed.

Please make sure to attach your quiz paper so I know what you are talking about. You can write or type your quiz corrections, but please put them on a separate sheet from the original quiz.

Types of Errors

For the purpose of this assignment I have divided the common types of errors into conceptual errors and clerical errors. These require slightly different correction processes, and these process are explained below. If your error seems to lie in some grey area between conceptual and clerical error, treat it as a conceptual error.

Conceptual Errors

These errors represent mistakes in your thinking, mistakes in setting up the problem, mistakes in translation of representation or incomplete understanding of concepts. Translation of representation is when you need to take information from one representation (such as word descriptions, graphs, motion diagrams, symbolic equations) and translate this information to another one of these representations. Incorrectly labeling a negative value as positive, grossly misreading a value off a graph or accidentally swapping your initial and final conditions are all considered conceptual errors in the translation of representation category.

To receive credit for your conceptual error feedback, you need to address the following two phases for each question or problem for which you did not receive full credit. See detailed description of each below:

1) Diagnosis Phase (DP) – Identify what went wrong.

In this phase you need to correctly identify your errors, and diagnose the nature of your difficulties as they relate to specific physics principles or concepts, a problem solving procedure, or beliefs about the nature of science and learning science.
Please note that an incorrect diagnosis or a merely descriptive work (such as simply noting the places where you made mistakes) is unacceptable. You need to analyze your thinking behind your mistakes, and explain the nature of these difficulties. Hence, in this phase you need to identify why you answered the way you did, where your understanding might have been weak, what you found difficult, what knowledge or skills you were missing that prevented you from correctly completing the solution, etc.

Poor Diagnosis – No description of thinking behind difficulty

  • “I was confused.”
  • “I thought it would be 5 N.”
  • “I picked the wrong equation.”
  • “I didn’t remember to use F=ma.”

Good Diagnosis – Focuses on reasons for actions

  • “I thought that the larger velocity would mean the larger force.”
  • “I knew it was angular momentum, but I didn’t apply it correctly – I neglected the angular momentum of the ball about the pivot point of the rod.”

2) Generalization Phase (GP) – Learn from your mistakes by generalizing beyond the specific problem.

In this phase you need to identify what deeper physics understanding you have gained from your diagnosis. By carefully thinking about the particular aspects that were problematic to you in approaching the question/problem, and correlating them with the correct solution, you should develop a better understanding of the basic physics principles. In your writing you should identify this new understanding and describe how it will prevent you from having similar problems in the future. Please note that merely stating the correct solution, by copying or paraphrasing another student’s solution for a question is unacceptable. You are expected to generalize beyond the specific problem to discuss the general principles of physics.

In your writing you are very welcome to identify not only your understanding of your mistakes, but also your appreciation for the aspects of your thinking that were already correct and successful in your original attempt. It is hoped that you will hold on to the good elements you already have and add new good ones by completing the feedback.

Poor Generalization – Focuses on generic activity

  • “I learned to read the question carefully”
  • “I learned to pick the right equations before solving a problem”

Poor Generalization – Focuses on the specific problem

  • “I learned that the amount of work from A to B is the same as the amount of work from B to C.”

Good Generalization – Generalizes beyond the specific problem

  • “I learned that the acceleration does not depend on the velocity. This is consistent with Newton’s Second Law which says that the acceleration depends only on the net force and the total mass.”

Clerical Errors

Clerical errors are those where you answered the question incorrectly in a way that was not due to a lack of physics understanding and where it is not reasonable to expect that you would be able to improve your physics understanding or mathematical fluency by learning from your mistakes. Examples of clerical errors include: your answer being incorrect due to a silly math error (accidental extra factor of 10 from a unit conversion, obvious arithmetic errors), forgetting to include units on your final answer, or making a mistake due to not reading the question carefully. These are errors where completing the Generalization Phase seems unproductive because the only thing sensible to write would be something along the lines of “I learned to read the question carefully” or “I need to be more careful of my arithmetic and always double-check my solutions.” Remember that errors in translation of representation are not considered to be clerical errors.

For clerical errors I ask you only to do one phase, the Correction Phase (CP). In this phase you identify your clerical error, how it led to your incorrect answer and how the corrected clerical error leads to a correct answer.

Acknowledgements: The quiz feedback assignment was originally developed by Charles Henderson (Western Michigan University) and Kathleen Harper (The Ohio State University) and much of the wording is theirs or is based on theirs.

Advertisements

12 Comments on “My revised quiz reflection assignment and the new homework reflection assigment”

  1. bretbenesh says:

    Hi Joss,

    This is good for (at least) two reasons:

    1) I like the all-or-nothing approach to the reflections.
    2) The examples of “Good Diagnosis”/”Poor Diagnosis” etc are extremely valuable.

    As usual, I might be stealing this.
    Bret

    • Joss Ives says:

      Of course, steal away.

      Anecdotally I find that they really reduce test anxiety, especially when combined with the group quizzes.

      I’m probably not going to do the all or nothing for the quantum homework feedback assignments, but I do really like using all or nothing for the quiz feedback assignments.

  2. Mylène says:

    Bret’s right — the examples really shine. I did something like this last year but didn’t know how to move my students past your “unacceptable” category except to write back to them individually. This reminds me that I need to model this process — they need to hear me think through it. One of my themes for this semester is “Make Mistakes Understandable” — when I demonstrate how I make my mistakes understandable, I think I will give examples of how to do it poorly as well as how to do it well.

    • Joss Ives says:

      Those examples come from Charles Henderson (I take no credit!). For the first couple rounds of correction assignments I am extra harsh on what is considered to be unacceptable and am quick to ask students to redo the assignment to earn the credit. This sets a high standard for their future work.

      I really like your plan Mylene. It is really important to model both high and low quality so that students can see the contrast and more easily try to figure out where they are along the continuum.

  3. Chris Goedde says:

    I may also steal this … thanks.

    Would you be willing to share a sample quiz, either here or via email? I’m especially curious about the various types of short answer questions you use.

  4. Chris Goedde says:

    One more question, if I may …

    Is there a level of student answer below which you don’t give credit for reflection? Suppose a question is left blank, or what’s there is just some minimal nonsense (random, wrong equations, etc.). How do you treat that for the purposes of the reflection?

    • Joss Ives says:

      Chris,

      In my experience so far with these (2 courses of 36 students each), students that are so unprepared as to leave questions blank or write only minimal nonsense are the same students that won’t bother to do the reflections. That being said, if one of those students leaves a blank or uses a completely wrong equation I have absolutely no issue if they want to go back and try to write a productive reflection to bring their mark from 0% up to 50% on that question. My hope is that they are learning some physics through that process.

      On a side note, I have some numbers on what fraction of students do the reflections based on the mark that they received on the overall quiz. I will post about that soon.

  5. I’m disturbed by one thing here: giving full marks for students who miss minus signs or constant factors. While these are small, easily corrected mistakes, they are still mistakes. I suppose it doesn’t matter if physicists are sloppy, but some of your students will be engineers, and if they are off on a calculation by a factor of -1 or 2pi it could be a disaster. Top marks should only be given for completely correct solutions (within the limitations of the model, of course). You should have a “clerical error” category for answers that are conceptually correct but have a tiny error, so that students don’t get the idea that “almost right” is as good as “right”.

    • Joss Ives says:

      In quantum I really try to emphasize that the process is much more important than the product. The questions tend to have these multi-page solutions and I don’t want to nickel and dime them to death for small mechanical errors.

      I was trying to go for the simplest possible marking system on this, but the clerical error category is something to consider. After a 4 page solution I have no real issue with (in principle) giving a student full marks if they dropped a 2pi somewhere, BUT in thinking about this I have realized the following: These students tend to all check their answers with each other before handing their homework in so they already know if there are clerical errors present before handing it in. I would like them to put some effort in to find those clerical error, and my proposed marking scheme gives them NO incentive to do so.

      I realize that I am now saying that it’s OK to have mechanical errors if you didn’t realize that they were there, but not OK if you did know they were there. I’m trying to sort this out.

  6. […] So I’ll introduce the components one by one: how to do a good-quality quiz correction (with inspiration from Joss), how to update the bar graph (their current grade), how to find an appropriate practice problem in […]

  7. […] was inspired by Joss Ives’ post on quiz reflection assignments to get specific about what I was looking for.  This all stems from a conversation I had with Kelly […]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s