Pre-class homework completion rates in my first-year courses

In my mind it is hard to get students to do pre-class homework (“pre-reading”) with much more than an 80% completion rate when averaged out over the term. It usually starts higher than this, but there is a slow trend toward less completion as the term wears on. After taking a more careful look at the five introductory courses in which I used pre-class assignment I have discovered that I was able to do much better than 80% in some of the later courses and want to share my data.

Descriptions of the five courses

The table below summarizes some of the key differences between each of the five introductory physics courses in which I used pre-class assignments. It may also be important to note that the majority of the students in Jan 2010 were the same students from Sep 2009, but not much more than half of the Jan 2013 students took my Sep 2012 course. For Jan 2013 only two of the students had previously taken a course with me.

Course Textbook Contribution to overall course grade Median completion rate (the numbers in brackets show the 1st and 3rd quartiles)
Sep 2009 (Mechanics) Young & Freedman – University Physics 11e Worth 8%, but drop 3 worst assignments. No opportunities for late submission or earning back lost marks. 0.73 (0.62,0.79)
Jan 2010 (E&M) Young & Freedman – University Physics 11e Worth 10%, but drop 2 worst assignments. No opportunities for late submission or earning back lost marks. 0.78 (0.74,0.89)
Sep 2011 (Mechanics) smartPhysics Worth 8%. Did not drop any assignments, but they could (re)submit at any point up until the final exam and earn half marks. 0.98 (0.96,0.98)
Jan 2012 (E&M) smartPhysics Worth 8%. Did not drop any assignments, but they could (re)submit at any point up until the final exam and earn half marks. 0.94 (0.93,0.98)
Jan 2013 (E&M) Halliday, Resnick & Walker – Fundamentals of Physics 9e & smartPhysics multimedia presentations Worth 10%. Did not drop any assignments, but they could (re)submit at any point up until the final exam and earn half marks. 0.93 (0.87,0.97)

Overall the style of question used was the same for each course, with the most common type of question being a fairly straight-forward clicker question (I discuss the resources and assignments a bit more in the next paragraph). I have not crunched the numbers, but scanning through results from the Jan 2013 course shows that students are answering the questions correctly somewhere in the 65-90% range and the questions used in that course were a mishmash of the Jan 2010 and Jan 2012 courses. Every question would have an “explain your answer” part. These assignments were graded on completion only, but their explanation had to show a reasonable level of effort to earn these completion marks. Depending on class size, I did not always read their explanations in detail, but always scanned every answer. For the first couple of assignments I always made sure to send some feedback to each student which would include an explanation of the correct answer if they answered incorrectly. Each question would also be discussed in class.

A rundown of how the resources and assignments varied by class:

  • For Sept 2009 and Jan 2010 I used a Blackboard assignment to give them the three questions each week and told them which sections of the textbook to read, and I didn’t do much to tell them to skip passages or examples that weren’t directly relevant.
  • For Sept 2010 and Jan 2012 I used smartPhysics (link to UIUC PER group page, where they were developed). These consist of multimedia presentations for each chapter/major topic, which have embedded conceptual questions (no student explanations required for these). After they are done the multimedia presentation, they then answer the actual pre-class questions, which are different from those embedded in the multimedia presentation. For the most part, the questions in their pre-class assignments were similar to the ones I was previously using except for the smartPhysics ones were often more difficult. Additionally, my one major criticism of smartPhysics is that I don’t feel they are pitched at the appropriate level for a student encountering the material for  the first time. For more on this, have a look at the second bullet in the “Random Notes” section of this post I did on pre-class assignments (link). One of the very nice things about smartphysics is that everything (the regular homework, the pre-class assignments and the multimedia presentations) all used the same web system.
  • For January 2013, I was back on assigning the pre-class assignments through Blackboard. The preamble for each of the pre-class assignments pointed them toward a smartPhysics multimedia presentation and the relevant sections of the textbook we were using. Students could use one, the other or both of these resources as they felt fit. I don’t think I ever surveyed them on their use of one over the other, but anecdotally I had the sense that way more were using the multimedia presentations.

The data

I present two graphs showing the same data from different perspectives. Figure 1 shows how the fraction of the class completing a given pre-class assignment varies over the course of the term. There is a noticeable downward trend in each course. Figure 2 shows the fraction of assignments completed by each student in each class.

This is the caption

Figure 1. For the first five graphs, the x-axis represents time going from the start of the course (left side) to the end of the course (right side). The box and whiskers plot compares the five courses according to the previously used colours, where the line in the boxes shows the median and the boxes show the 1st and 3rd quartiles. The whiskers are the matplotlib default; they extend to the most extreme data point within the 1.5*(75%-25%) range.

This is the caption

Figure 2. Histograms showing the fraction of assignments completed by each student. An ambiguous looking artifact appears in the 0 bin for Sept 2011, but all students in that course completed 70% or more of the pre-class assignments


There is clearly a large difference between the first two courses and the final three in terms of the rates at which students were completing these pre-class assignments. The fact that I saw 98% of these assignments completed one term is still shocking to me. I’m not sure how much each of the following factors contributed to the changes, but here are some of the potential factors…

  1. Multimedia presentations – students seem to find these easier to consume than reading the textbook. There is a study [Phys. Rev. ST Physics Ed. Research 8, 010103 (2012)] from
    Homeyra Sadaghiani at California State Polytechnic University where she did a controlled study comparing the multimedia presentations to readings in textbooks, and used the same pre-class assignments for both. In addition to finding that the multimedia presentation group did slightly better on the exams, she also found that the students had a favorable attitude toward the usefulness of the multimedia presentations, but that the textbook group had an unfavorable attitude toward the textbook reading assignments. But she also mentions that the multimedia group had a more favorable attitude toward clicker questions than the textbook section, and this alone could explain the difference in test performance as opposed to it having to do with the learning which takes place as part of the pre-class assignments. If the students in one section are buying into how the course is being run more than another, they are going to do a better job of engaging with all of the learning opportunities and as a result should be learning more. There are a variety of reasons why reading the textbook may be preferred to have them watching a video or multimedia presentation, but you can’t argue with the participation results.
  2. Generating buy-in – I have certainly found that, as time wears on, I have gotten better at generating buy-in for the way that my courses are run. I have gotten better at following up on the pre-class assignments in class and weaving the trends from their submissions into the class. However, for the Sep 2009 and Jan 2010 courses, that was the most personal feedback I have ever sent to students in an intro course on their pre-class assignments so I might have expected that getting better at generating buy-in might cancel out the decreased personal feedback.
  3. Changes in grading system – This may be a very large one and is tied to generating buy-in. For the first two courses I allowed them to drop their worst 3 or 2 pre-class assignments from their overall grade. In the later courses, I changed the system to being one where they could even submit the assignments late for half credit, but were not allowed to drop any. In the latter method I am clearly communicating to the students I think it is worth their time to complete all of the assignments.

In poking around through the UIUC papers and those from Sadaghiani, that 98% completion rate from my Sept 2011 course is really high, but is going to be an overestimate of how many people actually engaged with the pre-class material as opposed to trying to bluff their way through it. The smartPhysics system also gave them credit for completing the questions embedded in the multimedia presentations and I’m not presenting those numbers here, but when I scan the gradebooks, those that received credit for doing their pre-class assignments also always received credit for completing the embedded questions in the multimedia presentations. But, it is possible to skip slides to get to those so that doesn’t mean they actually fully consumed those presentations. Based on reviewing their explanations each week (with different degrees of thoroughness) and then docking grades accordingly, I would estimate that maybe 1 or 2 students managed to bluff their way through each week without actually consuming the presentation. That translates to 2-3% of these pre-class assignments.

Sadaghiani reported “78% of the MLM students completed 75% or more of the MLMs”, where MLM is what I have been calling the multimedia presentations. Colleagues of mine at UBC found (link to poster) that students self-reported to read their textbooks regularly in a course that used a quiz-based pre-class assignment (meaning that students were given marks for being correct as opposed to just participating, and in this case were not asked to explain their reasoning). 97% of the students actually took the weekly quizzes, but there is a discrepancy in numbers between those that took the quizzes and those that actually did the preparation.

With everything I have discussed here in mind, it seems that 80% or better is a good rule of thumb number for buy-in for pre-class activities, and that one can do even better than that with some additional effort.

Student opinions on what contributes to their learning in my intro E&M course

We are a couple of weeks away from our one and only term test in my intro calc-based electricity and magnetism course. This test comes in the second last week of the course and I pitch it to them as practicing for the final. This term test is worth 10-20% of their final grade and the final exam 30-40% of their final grade and these relative weights are meant to maximize the individual student’s grade.

Today I asked them how they feel the major course components are contributing to their learning:

How much do you feel that the following course component has contributed to your learning so far in this course?

This is a bit vague, but I told them to vote according to what contributes to their understanding of the physics in this course. It doesn’t necessarily mean what makes them feel the most prepared for the term test, but if that is how they wanted to interpret it, that would be just fine.

For each component that I discuss below, I will briefly discuss how it fits into the overall course. And you should have a sense of how the whole course works by the end.

The smartphysics pre-class assignments

The pre-class assignments are the engine that allow my course structure to work they way I want it to and I have been writing about them a lot lately (see my most recent post in a longer series). My specific implementation is detailed under ‘Reading assignments and other “learning before class” assignments’ in this post. The quick and dirty explanation is that, before coming to class, my students watch multimedia prelectures that have embedded conceptual multiple-choice questions. Afterward they answer 2-4 additional conceptual multiple-choice questions where they are asked to explain the reasoning behind each of their choices. They earn marks based on putting in an honest effort to explain their reasoning as opposed to choosing the correct answer. Then they show up to class ready to build on what they learned in the pre-class assignment.

Smartphysics pre-class assignments

A) A large contribution to my learning.
B) A small contribution to my learning, so I rarely complete them.
C) A small contribution to my learning, but they are worth marks so I complete them.
D) No contribution to my learning, so I rarely complete them.
E) No contribution to my learning, but they are worth marks so I complete them


The smartphysics online homework

The homework assignments are a combination of “Interactive Examples” and multi-part end-of-chapter-style problems.

The Interactive Examples tend to be fairly long and challenging problems where the online homework system takes the student through multiple steps of qualitative and quantitative analysis to arrive at the final answer. Some students seem to like these questions and others find them frustrating because they managed to figure out 90% of the problem on their own but are forced to step through all the intermediate guiding questions to get to the bit that is giving them trouble.

The multi-part end-of-chapter-style problems require, in theory, conceptual understanding to solve. In practice, I find that a lot of the students simply number mash until the correct answer comes out the other end, and then they don’t bother to step back and try to make sure that they understand why that particular number mashing combination gave them the correct answer. The default for the system (which is the way that I have left it) is that they can have as many tries as they like for each question and are never penalized as long as they find the correct answer. This seems to have really encouraged the mindless number mashing.

This is why their response regarding the learning value of the homework really surprised me. A sufficient number of them have admitted that they usually number mash, so I would have expected them not to place so much learning value on the homework.

The online smartphysics homework/b>

A) A large contribution to my learning.
B) A small contribution to my learning, so I rarely complete them.
C) A small contribution to my learning, but it is worth marks so I complete it.
D) No contribution to my learning, so I rarely complete it.
E) No contribution to my learning, but it is worth marks so I complete it


Studying for quizzes and other review outside of class time

Studying for quizzes and other review outside of class time

A) A large contribution to my learning.
B) A small contribution to my learning, but I do it anyway.
C) A small contribution to my learning so I don’t bother.
D) No contribution to my learning so I don’t bother.


Group quizzes

I have an older post that discusses these in detail, but I will summarize here. Every Friday we have a quiz. They write the quiz individually, hand it in, and then re-write the same quiz in groups. They receive instant feedback on their group quiz answers thanks to IF-AT multiple-choice scratch-and-win sheets and receive partial marks based on how many tries it took them to find the correct answer. Marks are awarded 75% for the individual portion and 25% for the group portion OR 100% for the individual portion if that would give them the better mark.

The questions are usually conceptual and often test the exact same conceptual step needed for them to get a correct answer on one of the homework questions (but not always with the same cover story). There are usually a lot of ranking tasks, which the students do not seem to like, but I do.

Group Quizzes

A) A large contribution to my learning.
B) A small contribution to my learning.
C) They don’t contribute to my learning.


Quiz Corrections

I have an older post that discusses these in detail, but I will again summarize here. For the quiz correction assignments they are asked, for each question, to diagnose what went wrong and then to generalize their new understanding of the physics involved. If they complete these assignments in the way I have asked, they earn back half of the marks they lost (e.g. a 60% quiz grade becomes 80%).

I am delighted to see that 42% of them find that these have a large contribution to their learning. The quizzes are worth 20% of their final grade, so I would have guessed that their perceived learning value would get lost in the quest for points.

Quiz Corrections

A) A large contribution to my learning.
B) A small contribution to my learning, so I rarely complete them.
C) A small contribution to my learning, but they are worth marks so I complete them.
D) No contribution to my learning, so I rarely complete them.
E) No contribution to my learning, but they are worth marks so I complete them.


In-class stuff

I am a full-on interactive engagement guy. I use clickers, in the question-driven instruction paradigm, as the driving force behind what happens during class time. Instead of working examples at the board, I either (A) use clicker questions to step the students through the example so that they are considering for themselves each of the important steps instead of me just showing them or (B) get them to work through examples in groups on whiteboards. Although I aspire to have the students report out there solutions in a future version of the course (“board meeting”), what I usually do when they work through the example on their whiteboards is wait until the majority of the groups are mostly done and then work through the example at the board with lots of their input, often generating clicker questions as we go.

The stuff we do in class

A) A large contribution to my learning.
B) A small contribution to my learning.
C) It doesn't contribute to my learning.


The take home messages

Groups quizzes rule! The students like them. I like them. The research tells us they are effective. Everybody wins. And they only take up approximately 10 minutes each week.

I need to step it up in terms of the perceived learning value of what we do in class. That 2/3rds number is somewhere between an accurate estimate and a small overestimate of the fraction of the students in class that at any moment are actively engaged with the task at hand. This class is 50% larger than my usual intro courses (54 students in this case) and I have been doing a much poorer job than usual of circulating and engaging individual students or groups during clicker questions and whiteboarding sessions. The other 1/3 of the students are a mix of students surfing/working on stuff for other classes (which I decided was something I was not going to fight in a course this size) and students that have adopted the “wait for him to tell us the answer” mentality. Peter Newbury talked about  these students in a recent post. I have lots of things in mind to improve both their perception and the actual learning value of what is happening in class. I will sit down and create a coherent plan of attack for the next round of courses.

I’m sure there are lots of other take home messages that I can pluck out of these data, but I will take make victory (group quizzes) and my needs improvement (working on the in class stuff) and look forward to continuing to work on course improvement.

Nonlinear narratives in the inverted classroom

I have temporarily taken over an introductory E&M course from one of my colleagues. I’m teaching the course using his format (and notes) which means that I am (A) lecturing and (B) not using pre-class assignments for the first time since 2006. In addition to his format, I am using the odd clicker question here and there.

The thing that has been the most interesting about lecturing in a non-inverted class has been the difference in narrative. In my regular courses, I assume that the students have had contact with all the major ideas from a given chapter or section before I even ask them the first clicker question. Because of this we are able to bring all the relevant ideas from a chapter to bear on each question if needed. This is what i am used to.

My colleague’s notes develop the ideas in a nice linear fashion and very easy to lecture from, but I just can’t stop myself from bringing in ideas that are multiple sections down the road. I am having a ton of trouble, even with a set of notes in front of me, letting the story develop according to a well laid-out narrative. It has simply been too long since I have presented material in this sort of a structured way. Note that when I give a talk at a conference it takes me a ton of practice to massage the talk I have prepared into something which I am able to deliver using a nice linear narrative. Even when it is nicely laid out, I will jump ahead to other ideas if I don’t spend some serious time practicing not doing that.

It has been really interesting being the one completely responsible for the narrative instead of sharing that responsibility with the resources that I provide for my pre-class assignments.

It has also been weird not having the inmates run the asylum.

Learning Before Class Strategies Part 2.5: Types of Video Lectures

A screen capture of a smartPhysics multimedia lesson

This is part 2.5 (the second half of part 2) of a series on pre-class learning strategies, which can be used as part of the flipped or inverted class. In this post I will discuss the different types of video lectures.

  • Part 1 focused on some common types of assignments/assessments that you can use.
  • Part 2 discussed the various types of resources you can provide your students for use in learning before class strategies.
  • Part 2.5 is this post
  • Part 3 will discuss some tips and some issues that I have come across trying to implement learning before class strategies.
  • I also had a quick update post that pointed to some recent articles/blog posts by others on the subject of pre-reading assignments.

Types of Video Lectures

In Part 2 of this series I used “video lecture” as a generic term to describe a lot of different audio+visual methods to get content into student hands. Note that the lines between some of these are pretty blurry (e.g., there’s no real practical difference between a whiteboard presentation  screencast and a video-recording of a chalk and talk).
  • Screencasts – These are recordings of your computer screen with your voice over top. Sometimes people narrate as they do something in a program (e.g., how to fit your data with a Gaussian in Gnuplot), narrate as they use the computer as a virtual whiteboard (e.g., Khan Academy) or even narrate a slide presentation that is being presented (sometimes with a talking head like Roger Freedman does).
  • Multimedia Presentations – You can think of these as a step beyond screencasts, with tools such as animation creating much tighter presentations than screencasts. The best example I have encountered is the collection of multimedia prelectures created by the folks at the University of Illinois Urbana-Champagne which are also included as part of their smartPhysics package. They use the PER literature  and the findings of research on multimedia learning to put these together in a way that is intended to maximize what students can learn from a direct instruction format.
  • Pencasts – These are recordings of what you write on paper using a special kind of pen (LiveScribe) which simultaneously records audio while you are writing. Watching a pencast is like sitting down beside somebody writing something down in a notebook while they talk you through it. Except you can click on anything they have written and rewatch it. It’s also possible to download a pdf of completed notes. I generated a whole bunch of these for my 3rd-year quantum course last term and the students gave me pretty positive feedback on the format. I also asked my students to each turn in one of their solutions as a pencast. This is a great idea that came to me from Andy Rundquist. What’s great about it is that they can’t just copy the solution from the web or a buddy. They have to make sure they understand what they did in the question before they sit down to present it as a pencast.
  • Video Recordings – You can record demonstrations, targeted lectures, etc. There are also many resources on the web (see some of the links on this the Active Class post).

Random Notes

  • I am not an advocate of traditional lecture and moving direct instruction onto a computer screen doesn’t change the fact that it is direct instruction. Most of the learning before class strategies I have been discussing rely on some sort of direct instruction to communicate initial content and I think video lectures can be effective in setting up what happens in the classroom later if they target ideas that are accessible to a novice.
  • Following up on the last bullet. I have been using smartPhysics this year for my calc-based intro physics course and I think their multimedia presentations are very well done, but ultimately they are communicating content at the same level as a regular textbook (see Part 2 of this series for my discussion on the suitability of textbooks as a first-contact resource). I don’t think it is a good use of student time or good-will for them to be confronted with challenging derivations two minutes after being introduced to a new topic. These pre-lectures are editable (as in I can take some of the material out) through the smartPhysics system so this term I have been aggressively trimming down the presentations so that the students get more of an introductory overview of the chapter as opposed to a 20-minute run through of the entire chapter. There’s a post slowly writing itself on my experiences with smartPhysics.
  • I have a tablet PC as my laptop, which should make it great for doing the virtual whiteboard thing, but some combination of the computer’s fan and other issues with the audio hardware introduces too much background noise (even when I have tried an external mic). I have tried post-recording noise reduction on the audio track and it worked pretty well, but that makes the whole thing fairly unsuitable for quick and dirty screencasts. I imagine that when I teach a comp phys class I will probably make more use of them for “this is how you do x” types of screencasts on a computer without noise issues.


 Part 3 some day.

Learning Before Class Strategies Part 2: Types of Resources to Provide Your Students

This is part 2 of a 3 part series on pre-class learning strategies, which can be used as part of the flipped or inverted class. In this post I will discuss some of the types of resources that you can provide your students to do their learning before class.

  • Part 1 focuses on some common types of assignments/assessments that you can use.
  • Part 2 is this post!
  • Part 2.5 discusses the types of video lectures in a bit more detail (this post was getting long)
  • Part 3 will discuss some tips and some issues that I have come across trying to implement learning before class strategies.
  • I also had a quick update post that pointed to some recent articles/blog posts by others on the subject of pre-reading assignments.

Photo by Kevin Dooley via Lifehacker

What I want out of these learning resources

To my mind, the purpose of most of the resources that I discuss below is to deliver some initial content to the students so that class time can be spent building on the ideas presented by that initial content. With this in mind, I don’t think that getting them to read the textbook is the most effective use of their time and can actually do some damage to any sort of buy-in that you have managed to generate toward the learning before class process. I say this because at their best textbooks attempt to be all of the following: (A) a resource to students on their first encounter with a topic, (B) a resource for students when working on their homework or studying for exams,  and (C) a reference for students after they have completed a course. I have yet to see a textbook that is structured in such a way that use (A) is clearly and consistently delineated from uses (B) and (C) so that it can be truly useful for use (A). I’m not saying it is not possible, but it would require a very targeted effort.
Like I said my use of learning before class strategies is so that my students show up in class ready to build on some initial understanding of the material.  It is not to help them develop their technical reading skills. And I don’t want them to show up feeling like they barely understood what was going on because of being overwhelmed by complicated derivations and examples that are more about symbolic manipulation than the underlying Physics. I will come back to this discussion in part 3, but since I am adding my $0.02 to the learning resources that I am presenting, I wanted to make my own personal take on the general purpose of most of these resources very clear.

Common types of learning resources to provide your students

These are types of learning before class resources that I have tried and I think is a fairly comprehensive list. Some of these are more well-suited to specific types of assignments (see Part 1) than others.

  1. Textbook – Despite everything I say above about textbooks (thus far) not being something that does a good job of being a resource for both first contact and reference, I do like Knight‘s intro physics textbook for being something that is quite readable for the student, even for first contact between the student and a new topic. Of course there are still many text elements (paragraphs, sections and examples) that I would ask the students to skip due to the amount of “mastering the basics” in class that is needed before this skipped content would be meaningful to any student other than the rare one that was determined to make sense of everything they they encountered in the textbook.
  2. Video lecture – These come in multiple flavours. Screencasting is becoming increasingly common, but there is also pencasting, highly-produced multimedia presentations like those found in smartPhysics, and video-recording a regular lecture. I will discuss each of these a little bit more in Part 2.5 of this series, but just comment on them as a group here. One of the strengths of the video lecture format is that it doesn’t take too much time to produce something coherent and of high enough quality that the students will find it useful. Anecdotally, I have found that students are much more willing to wrestle with slightly more challenging content in the video format than if I had just asked them to read very well presented notes on the same topic. Of course this is exactly why many (most?) students  are happy to attend traditional lectures, but would never think of reading the textbook covering the exact same content.
  3. Simulations – Inspired by Noah Podolefsky’s Global Physics Department talk on PhET simulations I have started basing some of my learning before class assignments on simulations. What I usually do is ask them to play with the simulation for 5-10 minutes and then send me 3 questions they had, things that they discovered or things that they found interesting in that time. After trying this type of simulation-based assignment out a few times I am finding that the students tend to generate questions that touch on most of the important points you would want to touch on, but instead frames the classroom discussions in terms of their curiosity instead of you telling them what’s important (even though it is the same actual content). The feedback I got from them indicated that they really seemed to like this type of pre-class assignment. If you don’t want to just let them play and tell you about it, you can try to focus their attention on specific things by asking them investigation types of questions like “what parameters affect X?”.
  4. Other targeted written resources – This is a grab bag category much like the video lecture one. This category includes
    • Your own written work meant to present the content at a level similar to a screencast or pencast that you would produce. A good example is the series of blog posts that Rhett Allain turned into Just Enough Physics;
    • Sections of textbooks that are targeted at a lower level than the given course. I often discovered when teaching Quantum that pointing my students to Knight’s discussions on the same topic would have been a goof “first encounter with a topic” resource or for intro physics I pointed my students toward physics for future presidents as an additional resource (before the book was published, all the chapter PDFs were available on that webpage).

    In many ways I prefer written resources to screencasts because (a) it is much easier to make small edits, and (b) I find it easier to piece together a few of them into a cohesive narrative. On the flip side, it is a lot less work to produce a screencast of acceptable quality than getting bogged down in writing something of acceptable quality. Or as Andy always say “I speak faster than I write.”

  5. Materials meant specifically to generate interest –  I have only tried it out once, but I was happy with how it worked out. You can give them a popular science or journal article, a chapter from a popular science book, a video, etc.  This one goes really well with the student generated question type of pre-class assignment (see Post 1 in this series) and then functions quite similar to the simulation-based pre-class assignments by letting the student questions frame the classroom discussion.
Overall I would say that I am the happiest when the combination of resource and assignment type allow the students to ask their own questions and then we can take those up in class. Like I said earlier, more often than not they ask the questions I was already planning on answering, but the difference feels huge between framing the discussion in terms of their questions and me telling them what I think is important. Yay!

Part 2.5 so very soon.

Learning Before Class Strategies – Quick Update

I am just about finished writing Part 2 of me Learning Before Class Strategies series of posts (Part 1 here), but I thought I would post links to a couple of recent posts on the subject of pre-reading assignments and an article from a recent UBC CWSEI Physics and Astronomy newsletter.

But first, here is a look at the relationship between exam averages and the marks my students earned for completing their smartPhysics Prelecture and Checkpoint assignments (marked for effort only) in my Fall 2011 introductory mechanics course. Not terribly compelling in terms of trying to sell my intro E&M students on the pre-class assignments. Fortunately Peter (see link below) gave me a bunch of good motivational ammo to pass on to them.

Sigh, the person that completed the least pre-class assignments had over an 80% average on exams.

Recent posts/articles of note on pre-reading assignments

Trying out a new type of simulation-based pre-class assignment

I always had trouble picturing the ground state of the deBroglie model of the Hydrogen atom. Now I don’t!

At this past week’s Global Physics Department virtual meeting Noah Podolefsky spoke with us about PhET simulations. Noah’s best practice suggestion was to let students play around with a simulation for 5-10 minutes before asking them to do anything specific. And when you ask them to do something specific, to use open / investigative questions (e.g., “explore all things that affect pH”, as opposed to cookbook directions such as “set the acid concentration to 0.010 M…”).

I asked Noah

I’m wondering how you would suggest using these in pre-class (JiTT-style) assignments. If I am ultimately going to give them some sort of a question (could be nice and open like you suggest), I feel like most students will jump to try to answer the question without first doing the “free play”. Any suggestions on getting them to do “free play” first?

Noah suggested getting them to play around with the sim and generate 2-3 questions or screenshots of “cool things” that they found, which Brian Frank echoed by suggesting I do the same thing I did when I got my Quantum Mechanics class to generate questions based on a reading. Andy Rundquist also suggested I could get them to screencast their interesting discoveries (instead of just screencapping).

My Quantum Mechanics class is in the middle of developing the Hydrogen wavefunctions (I showed them the shooting method results for the angular wavefunctions last class, thanks Andy!).  We’re skipping our regular pre-class assignment this week, so I sent them a bonus pre-class assignment before we look at the Hydrogen spectrum on Monday. Here’s a slightly paraphrased version of what I asked them to do with the “Models of the Hydrogen Atom” PhET simulation:

Spend 5-10 minutes playing around with the simulation. Generate 3 items of interest — these could be any combination of questions that you have, interesting observations that you made or descriptions of things that the simulation made really clear to you that you didn’t quite get before. You can take screen captures, generate screencasts or just send me regular old text.

I’m really interested to see what they come up with. I will make sure to report back.  Just for fun, I have embedded the simulation below

Models of the Hydrogen Atom

Click to Run

Learning Before Class Strategies Part 1: Types of Assignments

This is part 1 of a 3 part series on pre-class learning strategies, which can be used as part of the flipped or inverted class. I have discussed why I implement pre-class learning strategies in the past and want to focus instead more on the how in these posts.

  • Part 1 (this one) focuses on some common types of assignments/assessments that you can use.
  • Part 2 focuses on some different ways that you provide your students with the resources that they will be using to do some learning before coming to class.
  • Part 2.5 is a continuation of part 2 and discusses the types of video lectures in a bit more detail
  • Part 3 (eventually) will discuss some tips and some issues that I have come across trying to implement learning before class strategies.

Common pre-class assignments

These are types of pre-class assignments that I have tried out in my own classes or have heard of being in use out in the wild. Part 2 will go into more detail on the learning resources you can provide to your students, but most of these assignments types will work well with commonly used “flipped class” resources such as textbook readings, pencasts, screencasts and multimedia video.

    1. Reading Quiz – Reading quizzes are usually administered at the beginning of class and marked for correctness. I have used clickers, bubble sheets and index cards to administer them. The problem with reading quizzes is that they have to be reasonably easy  and target recall or very basic understanding so that students that put in an honest effort to do the reading will get most of the questions correct. As a result it is hard to come up with questions that aren’t reasonably easy to guess. The administration through index cards refers to the times that I have asked them to draw or explain something in a way that shows their understanding, but the overall performance of the class on these types of questions has usually ended up being low. I moved on from reading quizzes after just one term of using them.
    2. Guided Reading Quiz – Instead of asking the students to read and try to figure out what are the most important ideas from their pre-class resources, you supply them with a set of questions to guide their initial learning. And then a small handful of those questions are used for the start of class reading quiz. I have not tried this in my own courses, but I came across it in a effectiveness of peer instruction in computing paper by my friend and former CWSEI STLF Beth Simon. If I was teaching a giant section of a course, I think that I might use this method. If the students try to take a shortcut and just learn the answers to the questions, they still end up touching base with the most important points from the pre-class resources (assuming well-constructed questions) which means the assignment will still have accomplished its purpose. Haha, gotcha shortcut-takers.
    3. Pre-Class Online Participation Questions – This is my generic name for the type of pre-class assignments used in Just-in-Time-Teaching . Students are asked to answer questions online (through your CMS, online homework system or plain old email) at least a couple of hours before class, giving you time to review their answers and modify your lesson plan or seed your class notes with their words and questions. These questions are not marked for correctness, but are instead marked for completion, usually based on answers that show that the student put in some reasonable level of effort to learn the material. I get the most mileage out of asking them to answer relatively easy conceptual multiple-choice questions and asking them to explain their answer. Easy is a very relative term and I am usually happy if anywhere from 50-100% of the students are able to get the question correct after consuming the pre-class resources, but it is most important to me that I can see from their explanations that they had to mash the ideas around in their head a bit before being able to answer the question. For these questions I usually just pull a clicker question straight out of my notes. Other types of questions that work well for this type of assignment are estimation questions and short calculations (I will be discussing when these go poorly in Part 3). As I have discussed in a previous post, this is the style of pre-class assignment I currently use for all my courses (intro, upper-year and labs) other than my project-based upper-year lab course. One of my main open questions with this type of assignment (to be discussed more in Part 3) is how best to provide feedback to the students.
    4. Student-Generated Questions – (added an hour after the orginal post went live) I can’t believe I forgot to include this. As part of the pre-class online participation question assignments I usually provide an extra box on the web-form where they are encouraged to ask any questions that they have regarding the content in question. If there are some common themes to these questions I bring them up in class. For more isolated questions that won’t be addressed in class, I usually respond to that student’s question via email. This year in my 3rd-year quantum class I had the students generate some questions after reading an 8-page excerpt on the postulates of quantum mechanics and some compare/contrast points between classical and quantum mechanics. They generated fantastic questions and we spent a whole period going through these questions and tying the ideas from these questions to each other and to their previous courses. The best part was that if I had prepared a lecture to discuss those exact same ideas from their questions without having had them first generate the questions, they would have been nowhere near as invested in what I had to say. It really personalized the whole thing. I want to try this type of thing again in the future.
    5. Summaries – Get them to write a paragraph or three summarizing the main ideas or their understanding of the pre-class resources.
    6. Reflective Writing – The purpose of the student writing in this type of assignment is for them to focus on the ideas that they are having trouble understanding and to highlight or summarize those ideas through their writing. This type of assignment is marked for completion and evidence that they were writing for their own understanding, but is not marked for correctness. Calvin Kalman is a proponent of this type of writing to learn strategy.
If you have tried other types of assignments for learning before class, let me know about them. Part 2 coming soon.

Resources for selling and running an (inter)active intro physics class

This post is in response to Chad Orzel’s recent post about moving toward a more active classroom. He plans to get the students to read the textbook before coming to class, and then minimize lecture in class in favour of “in-class discussion/ problem solving/ questions/ etc.” At the end of the post he puts out a call for resources, which is where this post comes in.

There are three main things I want to discuss in this post, and (other than some links to specific clicker resources) they are all relevant to Chad or anybody else considering moving toward a more active classroom.

  1. Salesmanship is key. You need to generate buy-in from the students so that they truly believe that the reason you are doing all of this is so that they will learn more.
  2. When implementing any sort of “learn before class” strategy, you need to step back and decide what you realistically expect them to be able to learn  from reading the textbook or watching the multimedia pres
  3. The easiest first step toward a more (inter)active classroom is the appropriate use of clickers or some reasonable low-tech substitute.


KNA, a commenter on Chad’s post writes:
I also realized early on in my career that salesmanship is key. I need to explain why I want them to do the reading, and the 3 JiTT (ed. JiTT = Just-in-Time-Teaching) questions, and the homework problems sets, etc. My taking some time periodically to explain why it is all in their best interest (citing the PER studies, or showing them the correlation between homework done and exam grades), seems to help a lot with the end of term evals.

And I completely agree. I changed a lot of little things between my first and second year of teaching intro physics, but the thing that seemed to matter the most is that I managed to generate much more buy-in from the students the second year that I taught. Once they understood and believed that all the “crazy” stuff I was doing was for their benefit and was backed up by research, they followed me down all the different paths that I took them. My student evals, for basically the same course, went up significantly (0.75ish on a 5-point scale) between the first and second years.

A resource that I will point out for helping to generate student buy-in was put together for Peer Instruction (in Computer Science), but much of what is in there is applicable beyond Peer Instruction to the interactive classroom in general. Beth Simon (Lecturer at UCSD and former CWSEI STLF) made two screencasts to show/discuss how she generates student buy-in:

Reading assignments and other “learning before class” assignments

This seems to be a topic that I have posted about many times and for which I have had many conversations. I will briefly summarize my thoughts here, while pointing interested readers to some relevant posts and conversations.

When implementing “read the text before class” or any other type of “learn before class” assignments, you have to establish what exactly you want the students to get out of these assignments. My purpose for these types of assignments is to get them familiar with the terminology and lowest-level concepts, anything beyond that is what I want to work on in class. With that purpose in mind, not every single paragraph or section of a given chapter is relevant for my students to read before coming to class. I refer to this as “textbook overhead” and Mylene discussed this as part of a great post on student preparation for class.

I have tried reading quizzes at the beginning of class and found that it was too hard to pitch them at the exact right level that most of the students that did the reading would get them and that most of the students that didn’t do the reading wouldn’t get them.

Last year I used a modified version of the reading assignment portion of Jitt (this list was originally posted here):

  1. Assign reading
  2. Give them 3 questions. These questions are either directly from the JiTT book (I like their estimation questions) or are easy clicker questions pulled from my collection. For the clicker questions I ask them explain their reasoning in addition to simply answering the question.
  3. Get them to submit via web-form or email
  4. I respond to everybody’s submissions for each question to try to help clear up any mistakes in their thinking. I use a healthy dose of copy and paste after the first few and can make it through 30ish submissions in just over an hour.
  5. Give them some sort of credit for each question in which they made an effortful response whether they were correct or incorrect.

I was very happy with how this worked out. I think it really helped that I always responded to each and every one of their answers, even if it was nothing more than “great explanation” for a correct answer. I generated enough buy-in to have an average completion rate of 78% on these assignments over the term in my Mechanics course last time I taught it. I typically weight these assignments at 8-10% of their final grade so they have pretty strong (external) incentive for them to do them.

As I mentioned previously, my current thinking is that I want the initial presentation (reading or screencast) that the students encounter to be one that gets them familiar with terminology and low-level or core concepts. As Mylene says “It’s crazy to expect a single book to be both a reference for the pro and an introduction for the novice.” So that leaves me in a position where I need to generate my own “first-contact” reading materials or screencasts that best suit my needs and this is something that I am going to try out in my 3rd-year Quantum Mechanics course this fall.

It turns out that for intro physics there is an option which will save me this work. I am using smartPhysics this year (disclaimer: the publisher is providing the text and online access completely free to my students for the purposes of evaluation). To explain what smartPhysics is, I will pseudo-quote from something I previously wrote:

For those teaching intro physics that are more interested in screencasting/pre-class multimedia video presentations instead of pre-class reading assignments, you might wish to take a look at SmartPhysics. It’s a package developed by the PER group at UIUC that consists of online homework, online pre-class multimedia presentations and a shorter than usual textbook (read: cheaper than usual) because there are no end-of-chapter questions in the book, and the book’s presentation is geared more toward being a student reference since the multi-media presentations take care of the the “first time encountering a topic” level of exposition. My understanding is that they paid great attention to Mayer’s research on minimizing cognitive load during multimedia presentations. I will be using SmartPhysics for my first time this coming fall and will certainly write a post about my experience once I’m up and running.

Since writing that I have realized that the text from the textbook is more or less the transcript of the multimedia presentations so in a way this textbook actually is a reference for the pro and an introduction for the novice. They get into more challenging applications of concepts in their interactive examples which are part of the online homework assignments. For example, they don’t even mention objects landing at a different height than the launch height in the projectile motion portion of the textbook, but have an interactive example to look at this extension of projectile motion.

The thing with smartPhysics is that their checkpoint assignments are basically the same as the pre-class assignments I have been using so it should be a pretty seamless transition for me from that perspective. I still haven’t figured out how easy it is to give students direct feedback on their checkpoint assignment questions in smartPhysics, and remember that I consider that to be an important part of the student buy-in that I have managed to generate in the past.

(edit: the following discussion regarding reflective writing was added Aug 11) Another option for getting students to read the text before coming to class is reflective writing, which is promoted in Physics by Calvin Kalman (Concordia).  From “Enhancing Students’ Conceptual Understanding by Engaging Science Text with Reflective Writing as a Hermeneutical Circle“, CS Kalman, Science & Education, 2010:

For each section of the textbook that a student reads, they are supposed to first read the extract very carefully trying to zero in on what they don‘t understand, and all points that they would like to be clarified during the class using underlining, highlighting and/or summarizing the textual extract. They are then told to freewrite on the extract. “Write about what it means.” Try and find out exactly what you don‘t know, and try to understand through your writing the material you don‘t know.

This writing itself is not marked since the students are doing the writing for the purposes of their own understanding. But this writing can be marked for being complete.

Clicker questions and other (inter)active physics classroom resources

Chad doesn’t mention anywhere in his post that he is thinking of using clickers, but I highly recommend using them or a suitable low-tech substitute for promoting an (inter)active class.  I use a modified version of Mazur’s Peer Instruction and have blogged about my specific use of clickers in my class in the past. Many folks have implemented vanilla or modified peer instruction with cards and had great success.

Clicker question resources: My two favourite resources for intro physics clicker questions are:

I quite like the questions that Mazur includes in his book but find that they are too challenging for my students without appropriate scaffolding in the form of intermediate clicker questions which can be found in both the resources I list above.

Clicker-based examples: Chad expressed frustration that “when I do an example on the board, then ask them to do a similar problem themselves, they doodle aimlessly and say they don’t have any idea what to do.” To deal with this very issue, I have a continuum that I call clicker-based examples and will discuss the two most extreme cases that I use, but you can mash them together to produce anything in between:

  • The easier-for-students case is that, when doing an example or derivation, I do most of the work but get THEM to make the important mental jumps. For a typical example, I will identify 2-4 points in the example that would cause them some grief if they tried to do the example completely on their own. When I work this example at the board (or on my tablet) I will work through the example as usual, but when I get to one of the “grief” points I will pose a clicker question. These clicker questions might be things like “which free-body diagram is correct?”, “which of the following terms cancel?” or “which reasoning allowed me to go from step 3 to step 4?”
  • The other end of the spectrum is that I give them a harder question and still identify the “grief” points. But I instead get them to do all the work in small groups on whiteboards. I then help them through the question by posing the clicker questions at the appropriate times as they work through the problems. Sometimes I put all the clicker questions up at the beginning so they have an idea of the roadmap of working through the problem.

An excellent resource for questions to use in this way is Randy Knight’s 5 Easy Lessons, which is a supercharged instructor’s guide to his calculus-based intro book. The first time I used a lot of these questions I found that the students often threw their hands up in the air in confusion. So I would wander around the room (36 students) and note the points at which the students were stuck and generate on-the-fly clicker questions. The next year I was able to take advantage of those questions I had generated the previous year and then had all the “grief” points mapped out and the clicker questions prepared for my clicker-based examples.

Group Quizzes

Not related to clicker questions, but they are related to the (inter)active class: group quizzes are something that I have previously posted about and I have also presented a poster on the topic. I give the students a weekly quiz that they write individually first, and then after they have all been handed in they re-write the quiz in groups. Check out the post that I linked to if you want to learn more about exactly how I implement these as well as the pros and cons. Know that they are my single favourite thing that happens in my class due to it being the most animated I get to see the students being while discussing the application of physics concepts. It is loud and wonderful and I am trying to figure out how to show that there is a quantifiable learning benefit.

The Science Learnification (Almost) Weekly – June 19, 2011

This is a collection of things that tickled my science education fancy in the past couple of weeks or so.

Reflections on Standards-Based Grading

Lots of end-of-year reflections from SBG implementers

  • SBG with voice revisions – Andy Rundquist only accepts (re)assessments where he can hear the student’s voice. When they hand in a problem solution, it basically has to be a screencast or pencast (livescribe pen) submission. The post is his reflections on what worked, what didn’t and what to do next time.
  • Standards-Based Feedback and SBG Reflections – Bret Benesh has two SBG-posts one after the other. I was especially fond of the one on Standards-Based Feedback where he proposes that students would not receive standards-based grades throughout the term but would instead produce a portfolio of their work which best showed their mastery for each standard. This one got my mind racing and my fingers typing.
  • A Small Tweak and a Feedback Inequality – Dan Anderson posts about providing feedback-only on the first assessment in nerd form: Feedback > Feedback + Grade > Grade. This is his take on the same issue which lead Bret Benesh to thinking about Standards-Based Feedback, when there is a grade and feedback provided, the students focus all their attention on the grade. He also has a neat system of calculating the final score for an assessments.
  • Reflections on SBG – Roger Wistar (computer science teacher) discusses his SBG journey and the good and bad of his experience so far.


Flipped classrooms and screencasting

Peer Instruction

  • Why should I use peer instruction in my class? – Peter uses  a study on student (non)learning from video by the Kansas State Physics Education Research Group to help answer this question. The short answer is “Because they give the students and you to ability to assess the current level of understanding of the concepts. Current, right now, before it’s too late and the house of cards you’re so carefully building come crashing down.”

The tale of sciencegeekgirl’s career

Getting them to do stuff they are interested in

John Burk gets busy