Two-Stage Group Quizzes Part 1: What, How and Why

Note: This is the first in a series of posts, and is based on my fleshing out in more detail a poster that I presented at the fantastic FFPERPS conference last week. The basic points of group exam pros and cons, and the related references borrow very heavily from Ref. 5 (thanks Brett!). All quoted feedback is from my students.

Introduction

A two-stage group exam is form of assessment where students learn as part of the assessment. The idea is that the students write an exam individually, hand in their individual exams, and then re-write the same or similar exam in groups, where learning, volume and even fun are all had.

Instead of doing this for exams, I used the two-stage format for my weekly 20-30 minute quizzes. These quizzes would take place on the same day that the weekly homework was due (Mastering Physics so they had the instant feedback there as well), with the homework being do the moment class started. I used them for the first time Winter 2011 in an Introductory Calculus-based E&M course with an enrolment of 37 students.

The format of the weekly quiz (20-30 minutes total) was as follows:

  • The students wrote their 3-5 multiple-choice and short-answer question quiz individually and afterward handed these solo quizzes in.
  • The group quiz (typical group size = 3) consisted of most or all of individual questions as multiple-choice questions. Most questions had five choices.
  • The marks were weighted 75% for individual component and 25% for group component. The group component was not counted if it would drop an individual’s mark .

Immediate Feedback Assessment Technique (IF-AT)

IFAT card

The group quizzes were administered using IF-AT [1] cards, with correct answers being indicated by a star when covering is scratched off. This immediate feedback meant that the students always knew the correct answer by the end of the quiz, which was a time at which they were at their most curious, and thus the correct answer should be most meaningful to them. Typically, students that did not understand why a given answer was correct would call me over and ask for an explanation.

“I feel finding out the answers immediately after the quiz helps clarify where I went wrong instead of letting me think my answer was right over the weekend. It also works the same for right answers.”

My students found it to be a very fair marking system because they received partial marks based on their number of incorrect answers. My system was 2 points if you got it correct on the first try, and that went down by half for each wrong answer. So 1 wrong answer = 1 point credit, 2 wrong answers = 0.5 point credit, and so on.

The General Benefits of Group Exams

Peer interactions and feedback: Both students benefit when one explains an answer to the other.

“When I think I know what’s going on I can explain it and realize yes, I really do know what I’m talking about…and sometimes vice-versa.”

All students are given a good chance to participate: All students, including the shy or quiet, participate and are given a chance to explain their understanding and reasoning. I gave nine of these two-stage quizzes over the term and don’t remember seeing any students sitting on the sidelines letting the rest of their group figure it out. They were invested because their marks were on the line, and they genuinely (based on feedback) seemed to feel like it was a good opportunity for learning.

Development of collaboration skills [2]: I can’t really comment on this too much. These are not skills which I was able to notice a difference over the course of the term, but I would certainly believe that many students had some tangible level of development of these types of skills.

Students enjoy them and there is an increase in overall enjoyment of course [2]: The claim about the increase in overall enjoyment from Ref. 2 is not something I can comment on since I changed many things in this course from the first time I taught it so couldn’t pinpoint one thing which lead to higher student evaluations and a more positive classroom culture than the first time I taught this course (note this was only my second time teaching this course). But I can certainly tell you the feedback I got regarding the group quizzes was overwhelmingly positive.

“I (heart) group quizzes!!! I always learn, and it’s nice to argue out your thinking with the group. It really helps me understand. Plus, scratching the boxes is super fun.”

They promote higher-level thinking [3]: This claim from Ref. 3 is another one for which I cannot say I looked for, nor saw any evidence.

Increase in student retention of information (maybe) [4]: This is at the heart of a study which I plan to run this coming year. Ref. 4 saw a slight increase in student retention of information on questions for which students completed the group quizzes. But this study did not control for differences in time-on-task between the students that completed just the individual exams and those that complete individual and group exams. More discussion on my planned study in a future post.

Improved student learning [3,5]: I saw the same sort of evidence for this that is shown in Ref. 5. Groups of students, where none of them had the correct answer for a question on their solo quizzes, were able to get the correct answer on the group quiz nearly half the time. This shift from nobody having the correct answer to the group figuring out the correct answer is a very nice sign of learning.

Some Specific Benefits I Saw in My Implementation

The feedback comes when it is most useful to them: Immediate feedback via IF-AT provides them with correct answer when they are very receptive, after having spent time on their own and in group discussion pondering the question.

“It’s a good time to discuss and it’s the perfect time to learn, ‘cause right after the quiz, the ideas and thoughts stick to your mind. It’s gonna leave a great impression.”

Very high engagement from all students: I observed my students to be highly engaged with the questions at hand and were very excited (high-fives and other celebrations were delightfully common) when they got a challenging question correct.

Reduced student anxiety: due to (a) knowing that they could earn marks even if they were incorrect on individual portion, and (b) knowing that they would come away from the quiz knowing the correct answers. Point (a) is pure speculation on my part. Part (b) was a point made by multiple students when I asked them to provide feedback on the group quiz process.

Some Drawbacks to Group Exams

Longer exam/quiz time: This really wasn’t a big deal. It was typically less than an extra 10 minutes and the advantages were far too great to not give up that extra little bit of class time.

Some students feel very anxious about group work and interactions: This never came up in the feedback I received from the students, but I have friends and family who have discussed with me how much they dislike group work. Perhaps this specific implementation might have even been to their liking.

Social loafers and non-contributors earn same marks as the rest of the group: To my mind the potential student frustration from this was greatly moderated by all students writing the solo quizzes, as well as the group portion being worth only 25% of total quiz mark. And as I mentioned earlier, I do not remember noticing a non-contributor even once over the term.

Dominant group members can lead group astray when incorrect: This is another thing which, to my mind, the IF-AT sheets moderate greatly. Dominant group members can potentially minimize the contributions of other group members, but I do remember an “ease-your-mind” about dominant student issues point made by Jim Sibley when I first learned of IF-AT. Jim Sibley is at the University of British Columbia and is a proponent of Team-Based Learning. In this learning environment they use the IF-AT cards for reading quizzes at the start of a module. He told us that groups often go to the shy or quiet members as trusted answer sources when dominant group members are repeatedly incorrect.

References

[1] http://www.epsteineducation.com

[2] Stearns, S. (1996). Collaborative Exams as Learning Tools. College Teaching, 44, 111–112.

[3] Yuretich, R., Khan, S. & Leckie, R. (2001). Active-learning methods to improve student performance and scientific interest in a large introductory oceanography course. Journal of Geoscience Education, 49, 111–119.

[4] Cortright, R.N., Collins, H.L., Rodenbaugh D.W. & DiCarlo, S.T. (2003). Student retention of course content is improved by collaborative-group testing, Advan. Physiol. Edu. 27: 102-108.

[5]Gilley, B. & Harris, S. (2010). Group quizzes as a learning experience in an introductory lab, Poster presented at Geological Society of America 2010 Annual Meeting.

Updates

March 26, 2011 – Added: “These quizzes would take place on the same day that the weekly homework was due (Mastering Physics so they had the instant feedback there as well), with the homework being do the moment class started.”

Advertisements

22 Comments on “Two-Stage Group Quizzes Part 1: What, How and Why”

  1. Brian says:

    This is really cool. Where do you get the IF-AT scratch offs?

    It’s especially cool to see some shifts from no one correct individually to group being correct.

    • Joss Ives says:

      Hi Brian,

      I order them directly from Ref. 1. The ones I have are 10 questions per sheet with 5 choices each and cost $80/500 sheets. When you get 500, you get two different answer distributions. Some colleagues of mine ordered some near the end of the term and we made some trades so we each have four different answer distributions.

      While I’m on the topic I’m going to jump ahead to answer Andy’s question regarding if they started to see the patterns. Short answer is “not as far as I could tell.” With only two different answer distributions I had to be careful because it seems very reasonable that they would see the patterns. But they had to hand in the group quizzes and never saw them again. This worked fine because (1) they knew the correct answers and their grade when they handed the IF-AT sheets in, and (2) I posted the solutions to the solo quiz immediately after class and the group questions were always a subset of or the entire solo quiz. On more than one occasion I asked them to scratch a different row (say rows 5-8) instead of just the first four to keep them off the scent of the patterns. I told them that I was doing this because the correct answers on those rows better met my needs, which was sometimes partially true because I like to include legit “none of the above choices” and try to make sure those are always choice “E”.

  2. Karen Jones says:

    This is very interesting! I wonder how you chose the groups; did they mind who they worked with? Was there a rush to choose the ‘best’ students?
    I think there is alot of mileage in this idea, thanks for sharing

    • Joss Ives says:

      Hi Karen,

      Our 36-student lecture area is set up so students sit 3/bench and face the front. Students choose where to sit but typically end up staying where they sit on the first day. They work in these groups for clicker questions and whiteboarding so they are pretty used to the people they were working with and I ended up seeing a lot of friendships develop with their bench-mates. If I had to make a rough estimate, I would say that <5% of the students would prefer to work with other students than the ones at their bench if that could be made to happen in a way which was socially acceptable.

      In terms of choosing the "best" students, the overall average for the group portion was 92% so high enough that there wasn't any real worry about flocking to the "best" students. That being said, there were two very weak groups (perhaps I will post a histogram when I have a bit more time to prepare it) that really struggled with the group quizzes. It would have been nice if I could have had one of my stronger students sit with them on occasion. I might try to randomize groups for the group portion on occasion next term.

  3. Joss, this is fantastic! How hard is it to shuffle your answers to match the IF-AT answer keys? How many keys do you use (do students start to see any patterns?)?

    Now some tougher questions: the knock on multiple choice is you don’t capture the thinking process of the students. This IF-AT method is an awesome way of doing multiple choice (especially considering the instant feedback), but does it raise the overall assessment value of multiple choice? Ok, only one tougher question.

    Thanks so much for sharing this. I love how you took it to a conference and then shared both the poster info and some feedback you’ve learned.

    • Joss Ives says:

      Andy,

      In general it is super easy to shuffle answers to match the IF-AT key. The only two instances that it ends up being a bit of a hassle are when I don’t want to use all five choices, or when I want to use an “E” all of or none of the above type choice.

      The stuff related to assessment value of multiple choice questions is a big old can of worms (I smell future post). I will do my best to stick to what you bring up.

      Your main question is if IF-AT raises the value of multiple choice. I would say yes for two reasons.

      The first reason is that you can, on an IF-AT sheet, see which incorrect answers were chosen before the correct one. Approximately 50% of the time (proper stats in future post), a group will have only one incorrect answer on a question before answering correctly. So half the time that a group answers incorrectly you know exactly which incorrect choice they made before getting the correct answer. This is certainly more of a window into student thinking than you would have on regular multiple choice. I think that being able to see the trail of wrong answers that lead to a correct answer is a complimentary way of seeing student thinking to asking an “explain your reasoning” type of question. I was already planning on bringing in more “explain your reasoning” multiple-choice and short answer questions on the solo quizzes this coming year, but in light of what I have just written, I am extra planning on doing it. I will also try to figure out a way to ask groups to explain their reasoning on some questions without adding to much extra time to the quizzes.

      The second reason that I am arguing it raises the assessment value of multiple choice is from the student side and is related to the IF-AT selling points promoted by Epstein Education. I haven’t read through the papers on their site, but I am reasonably convinced that providing student with the correct answer when they are most curious AND allowing them to revise their incorrect thinking on the spot both have great learning value.

      Also related to the points you raise Andy, is the fact that I actually saw quite different performance on the short-answer vs. multiple-choice solo quiz questions, as well as how the groups did on these questions when converted to multiple-choice. This comparison is coming up in the next post in this series.

      I’m really starting to see conference poster sessions as a great venue to have conversations about the same sort of things that we all discuss through comments on each others posts. I didn’t do it at this last conference, but I want to start having a section on my posters along the lines of “Things I would love to discuss with you…”. There I could ask about people’s experiences with the thing I am presenting, thoughts on the research design used, if my statistical analysis is appropriate, etc. I have seen people leaving a marker and blank space or pages so that people can leave comments if the presenter is not standing at the poster at that moment. If the conference has a big enough back channel I could even try making a hashtag to discuss the poster on twitter. Fun!

  4. Joss,
    I really like this too. Did you think of trying to do this in something like webassign as well?

    • Joss Ives says:

      Hmmmmm. Group quizzes through webassign? I’m having trouble picturing a way to do it. Did you have a way in mind? (Please say yes)

      • John says:

        Well, I would think you could create assignments due in class, and allow 4-5 submissions. to get the same basic effect as the IF-AT tests. Webassign can even assign people to groups, if you like.

      • Joss Ives says:

        Got it! I was trying to come up with something outside of class, but your suggestion to use webassign (or any CMS) to do the work of the IF-AT sheets would certainly work.

  5. Joss Ives says:

    I didn’t explicitly mention this in the post, but the group portion of the quizzes was probably my favorite part of the week with these students. This is surprising (to me) since I wasn’t having any interactions with them, but their enthusiasm in completing these quizzes was amazing and it felt like a very non-stressful environment considering how they did on these questions did in fact affect their grades.

  6. bretbenesh says:

    Hi Joss,

    I like the idea, but I want to particularly compliment you on providing references in this posting. Everyone—including me—should do this more (both listing references AND using methods that have already been shown to be successful).
    Bret

    • Joss Ives says:

      Hi Bret,

      Thanks for the thumbs up!

      Part of why I started this blog was to force myself to do documentation of my instructional strategies that was thorough enough for others to decipher. This also involves doing some number crunching that I might otherwise put off indefinitely. The other reason was to get the seeds of talks and posters “on paper”, so that I could more easily put talks or posters together in the future based on ideas that I have already fleshed out to some degree. So I’m looking at my instructional strategies posts as the equivalent to the technical notes that I wrote in my experimental particle physics days and trying my best to hold myself to the expected level of academic thoroughness (references, background research, etc) that was expected for technical notes.

  7. […] questions, but they are related to the (inter)active class: group quizzes are something that I have previously posted about and I have also presented a poster on the topic. I give the students a weekly quiz that they write […]

  8. […] For the second term there was a group component to the quizzes and these group quizzes were administered using IF-AT (Immediate Feedback Assessment Technique) [4], a scratch and win multiple-choice sheet that lets students instantly know if they got the correct answer. The students would rewrite the quiz with their table-mates after handing in their solo quiz. The group quizzes and the IF-AT will be discussed in a future posts (Update Sept. 1, 2011 – My post discussing group quizzes and the IF-AT can be found here). […]

  9. […] have an older post that discusses these in detail, but I will summarize here. Every Friday we have a quiz. They write […]

  10. […] same data analysis as a poster that I presented the previous year (two-stage group quiz posts 0 and 1), but I added some comparisons to other similar […]

  11. […] we spend most of our class time doing clicker questions and whiteboard activities, and there is a weekly two-stage quiz (individual then group). I have added a single problem (well, closer to an exercise) to each weekly […]

  12. […] I read a post by Joss Ives about 2-stage quizzes (stage 1= solo, stage […]

  13. […] read a blog post about 2-stage quizzes and wanted to work them in in my standards-based grading […]

  14. […] read a blog post about 2-stage quizzes and wanted to work them in in my standards-based grading […]

  15. […] go into great detail about the benefits of a two-stage exam (as it has previously been covered here by @jossives), but this second stage gives students the immediate feedback that they tend to seek […]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s