In my mind it is hard to get students to do pre-class homework (“pre-reading”) with much more than an 80% completion rate when averaged out over the term. It usually starts higher than this, but there is a slow trend toward less completion as the term wears on. After taking a more careful look at the five introductory courses in which I used pre-class assignment I have discovered that I was able to do much better than 80% in some of the later courses and want to share my data.
Descriptions of the five courses
The table below summarizes some of the key differences between each of the five introductory physics courses in which I used pre-class assignments. It may also be important to note that the majority of the students in Jan 2010 were the same students from Sep 2009, but not much more than half of the Jan 2013 students took my Sep 2012 course. For Jan 2013 only two of the students had previously taken a course with me.
|Course||Textbook||Contribution to overall course grade||Median completion rate (the numbers in brackets show the 1st and 3rd quartiles)|
|Sep 2009 (Mechanics)||Young & Freedman – University Physics 11e||Worth 8%, but drop 3 worst assignments. No opportunities for late submission or earning back lost marks.||0.73 (0.62,0.79)|
|Jan 2010 (E&M)||Young & Freedman – University Physics 11e||Worth 10%, but drop 2 worst assignments. No opportunities for late submission or earning back lost marks.||0.78 (0.74,0.89)|
|Sep 2011 (Mechanics)||smartPhysics||Worth 8%. Did not drop any assignments, but they could (re)submit at any point up until the final exam and earn half marks.||0.98 (0.96,0.98)|
|Jan 2012 (E&M)||smartPhysics||Worth 8%. Did not drop any assignments, but they could (re)submit at any point up until the final exam and earn half marks.||0.94 (0.93,0.98)|
|Jan 2013 (E&M)||Halliday, Resnick & Walker – Fundamentals of Physics 9e & smartPhysics multimedia presentations||Worth 10%. Did not drop any assignments, but they could (re)submit at any point up until the final exam and earn half marks.||0.93 (0.87,0.97)|
Overall the style of question used was the same for each course, with the most common type of question being a fairly straight-forward clicker question (I discuss the resources and assignments a bit more in the next paragraph). I have not crunched the numbers, but scanning through results from the Jan 2013 course shows that students are answering the questions correctly somewhere in the 65-90% range and the questions used in that course were a mishmash of the Jan 2010 and Jan 2012 courses. Every question would have an “explain your answer” part. These assignments were graded on completion only, but their explanation had to show a reasonable level of effort to earn these completion marks. Depending on class size, I did not always read their explanations in detail, but always scanned every answer. For the first couple of assignments I always made sure to send some feedback to each student which would include an explanation of the correct answer if they answered incorrectly. Each question would also be discussed in class.
A rundown of how the resources and assignments varied by class:
- For Sept 2009 and Jan 2010 I used a Blackboard assignment to give them the three questions each week and told them which sections of the textbook to read, and I didn’t do much to tell them to skip passages or examples that weren’t directly relevant.
- For Sept 2010 and Jan 2012 I used smartPhysics (link to UIUC PER group page, where they were developed). These consist of multimedia presentations for each chapter/major topic, which have embedded conceptual questions (no student explanations required for these). After they are done the multimedia presentation, they then answer the actual pre-class questions, which are different from those embedded in the multimedia presentation. For the most part, the questions in their pre-class assignments were similar to the ones I was previously using except for the smartPhysics ones were often more difficult. Additionally, my one major criticism of smartPhysics is that I don’t feel they are pitched at the appropriate level for a student encountering the material for the first time. For more on this, have a look at the second bullet in the “Random Notes” section of this post I did on pre-class assignments (link). One of the very nice things about smartphysics is that everything (the regular homework, the pre-class assignments and the multimedia presentations) all used the same web system.
- For January 2013, I was back on assigning the pre-class assignments through Blackboard. The preamble for each of the pre-class assignments pointed them toward a smartPhysics multimedia presentation and the relevant sections of the textbook we were using. Students could use one, the other or both of these resources as they felt fit. I don’t think I ever surveyed them on their use of one over the other, but anecdotally I had the sense that way more were using the multimedia presentations.
I present two graphs showing the same data from different perspectives. Figure 1 shows how the fraction of the class completing a given pre-class assignment varies over the course of the term. There is a noticeable downward trend in each course. Figure 2 shows the fraction of assignments completed by each student in each class.
There is clearly a large difference between the first two courses and the final three in terms of the rates at which students were completing these pre-class assignments. The fact that I saw 98% of these assignments completed one term is still shocking to me. I’m not sure how much each of the following factors contributed to the changes, but here are some of the potential factors…
- Multimedia presentations – students seem to find these easier to consume than reading the textbook. There is a study [Phys. Rev. ST Physics Ed. Research 8, 010103 (2012)] from
Homeyra Sadaghiani at California State Polytechnic University where she did a controlled study comparing the multimedia presentations to readings in textbooks, and used the same pre-class assignments for both. In addition to finding that the multimedia presentation group did slightly better on the exams, she also found that the students had a favorable attitude toward the usefulness of the multimedia presentations, but that the textbook group had an unfavorable attitude toward the textbook reading assignments. But she also mentions that the multimedia group had a more favorable attitude toward clicker questions than the textbook section, and this alone could explain the difference in test performance as opposed to it having to do with the learning which takes place as part of the pre-class assignments. If the students in one section are buying into how the course is being run more than another, they are going to do a better job of engaging with all of the learning opportunities and as a result should be learning more. There are a variety of reasons why reading the textbook may be preferred to have them watching a video or multimedia presentation, but you can’t argue with the participation results.
- Generating buy-in – I have certainly found that, as time wears on, I have gotten better at generating buy-in for the way that my courses are run. I have gotten better at following up on the pre-class assignments in class and weaving the trends from their submissions into the class. However, for the Sep 2009 and Jan 2010 courses, that was the most personal feedback I have ever sent to students in an intro course on their pre-class assignments so I might have expected that getting better at generating buy-in might cancel out the decreased personal feedback.
- Changes in grading system – This may be a very large one and is tied to generating buy-in. For the first two courses I allowed them to drop their worst 3 or 2 pre-class assignments from their overall grade. In the later courses, I changed the system to being one where they could even submit the assignments late for half credit, but were not allowed to drop any. In the latter method I am clearly communicating to the students I think it is worth their time to complete all of the assignments.
In poking around through the UIUC papers and those from Sadaghiani, that 98% completion rate from my Sept 2011 course is really high, but is going to be an overestimate of how many people actually engaged with the pre-class material as opposed to trying to bluff their way through it. The smartPhysics system also gave them credit for completing the questions embedded in the multimedia presentations and I’m not presenting those numbers here, but when I scan the gradebooks, those that received credit for doing their pre-class assignments also always received credit for completing the embedded questions in the multimedia presentations. But, it is possible to skip slides to get to those so that doesn’t mean they actually fully consumed those presentations. Based on reviewing their explanations each week (with different degrees of thoroughness) and then docking grades accordingly, I would estimate that maybe 1 or 2 students managed to bluff their way through each week without actually consuming the presentation. That translates to 2-3% of these pre-class assignments.
Sadaghiani reported “78% of the MLM students completed 75% or more of the MLMs”, where MLM is what I have been calling the multimedia presentations. Colleagues of mine at UBC found (link to poster) that students self-reported to read their textbooks regularly in a course that used a quiz-based pre-class assignment (meaning that students were given marks for being correct as opposed to just participating, and in this case were not asked to explain their reasoning). 97% of the students actually took the weekly quizzes, but there is a discrepancy in numbers between those that took the quizzes and those that actually did the preparation.
With everything I have discussed here in mind, it seems that 80% or better is a good rule of thumb number for buy-in for pre-class activities, and that one can do even better than that with some additional effort.
We are a couple of weeks away from our one and only term test in my intro calc-based electricity and magnetism course. This test comes in the second last week of the course and I pitch it to them as practicing for the final. This term test is worth 10-20% of their final grade and the final exam 30-40% of their final grade and these relative weights are meant to maximize the individual student’s grade.
Today I asked them how they feel the major course components are contributing to their learning:
How much do you feel that the following course component has contributed to your learning so far in this course?
This is a bit vague, but I told them to vote according to what contributes to their understanding of the physics in this course. It doesn’t necessarily mean what makes them feel the most prepared for the term test, but if that is how they wanted to interpret it, that would be just fine.
For each component that I discuss below, I will briefly discuss how it fits into the overall course. And you should have a sense of how the whole course works by the end.
The smartphysics pre-class assignments
The pre-class assignments are the engine that allow my course structure to work they way I want it to and I have been writing about them a lot lately (see my most recent post in a longer series). My specific implementation is detailed under ‘Reading assignments and other “learning before class” assignments’ in this post. The quick and dirty explanation is that, before coming to class, my students watch multimedia prelectures that have embedded conceptual multiple-choice questions. Afterward they answer 2-4 additional conceptual multiple-choice questions where they are asked to explain the reasoning behind each of their choices. They earn marks based on putting in an honest effort to explain their reasoning as opposed to choosing the correct answer. Then they show up to class ready to build on what they learned in the pre-class assignment.
The smartphysics online homework
The homework assignments are a combination of “Interactive Examples” and multi-part end-of-chapter-style problems.
The Interactive Examples tend to be fairly long and challenging problems where the online homework system takes the student through multiple steps of qualitative and quantitative analysis to arrive at the final answer. Some students seem to like these questions and others find them frustrating because they managed to figure out 90% of the problem on their own but are forced to step through all the intermediate guiding questions to get to the bit that is giving them trouble.
The multi-part end-of-chapter-style problems require, in theory, conceptual understanding to solve. In practice, I find that a lot of the students simply number mash until the correct answer comes out the other end, and then they don’t bother to step back and try to make sure that they understand why that particular number mashing combination gave them the correct answer. The default for the system (which is the way that I have left it) is that they can have as many tries as they like for each question and are never penalized as long as they find the correct answer. This seems to have really encouraged the mindless number mashing.
This is why their response regarding the learning value of the homework really surprised me. A sufficient number of them have admitted that they usually number mash, so I would have expected them not to place so much learning value on the homework.
Studying for quizzes and other review outside of class time
I have an older post that discusses these in detail, but I will summarize here. Every Friday we have a quiz. They write the quiz individually, hand it in, and then re-write the same quiz in groups. They receive instant feedback on their group quiz answers thanks to IF-AT multiple-choice scratch-and-win sheets and receive partial marks based on how many tries it took them to find the correct answer. Marks are awarded 75% for the individual portion and 25% for the group portion OR 100% for the individual portion if that would give them the better mark.
The questions are usually conceptual and often test the exact same conceptual step needed for them to get a correct answer on one of the homework questions (but not always with the same cover story). There are usually a lot of ranking tasks, which the students do not seem to like, but I do.
I have an older post that discusses these in detail, but I will again summarize here. For the quiz correction assignments they are asked, for each question, to diagnose what went wrong and then to generalize their new understanding of the physics involved. If they complete these assignments in the way I have asked, they earn back half of the marks they lost (e.g. a 60% quiz grade becomes 80%).
I am delighted to see that 42% of them find that these have a large contribution to their learning. The quizzes are worth 20% of their final grade, so I would have guessed that their perceived learning value would get lost in the quest for points.
I am a full-on interactive engagement guy. I use clickers, in the question-driven instruction paradigm, as the driving force behind what happens during class time. Instead of working examples at the board, I either (A) use clicker questions to step the students through the example so that they are considering for themselves each of the important steps instead of me just showing them or (B) get them to work through examples in groups on whiteboards. Although I aspire to have the students report out there solutions in a future version of the course (“board meeting”), what I usually do when they work through the example on their whiteboards is wait until the majority of the groups are mostly done and then work through the example at the board with lots of their input, often generating clicker questions as we go.
The take home messages
Groups quizzes rule! The students like them. I like them. The research tells us they are effective. Everybody wins. And they only take up approximately 10 minutes each week.
I need to step it up in terms of the perceived learning value of what we do in class. That 2/3rds number is somewhere between an accurate estimate and a small overestimate of the fraction of the students in class that at any moment are actively engaged with the task at hand. This class is 50% larger than my usual intro courses (54 students in this case) and I have been doing a much poorer job than usual of circulating and engaging individual students or groups during clicker questions and whiteboarding sessions. The other 1/3 of the students are a mix of students surfing/working on stuff for other classes (which I decided was something I was not going to fight in a course this size) and students that have adopted the “wait for him to tell us the answer” mentality. Peter Newbury talked about these students in a recent post. I have lots of things in mind to improve both their perception and the actual learning value of what is happening in class. I will sit down and create a coherent plan of attack for the next round of courses.
I’m sure there are lots of other take home messages that I can pluck out of these data, but I will take make victory (group quizzes) and my needs improvement (working on the in class stuff) and look forward to continuing to work on course improvement.
I have temporarily taken over an introductory E&M course from one of my colleagues. I’m teaching the course using his format (and notes) which means that I am (A) lecturing and (B) not using pre-class assignments for the first time since 2006. In addition to his format, I am using the odd clicker question here and there.
The thing that has been the most interesting about lecturing in a non-inverted class has been the difference in narrative. In my regular courses, I assume that the students have had contact with all the major ideas from a given chapter or section before I even ask them the first clicker question. Because of this we are able to bring all the relevant ideas from a chapter to bear on each question if needed. This is what i am used to.
My colleague’s notes develop the ideas in a nice linear fashion and very easy to lecture from, but I just can’t stop myself from bringing in ideas that are multiple sections down the road. I am having a ton of trouble, even with a set of notes in front of me, letting the story develop according to a well laid-out narrative. It has simply been too long since I have presented material in this sort of a structured way. Note that when I give a talk at a conference it takes me a ton of practice to massage the talk I have prepared into something which I am able to deliver using a nice linear narrative. Even when it is nicely laid out, I will jump ahead to other ideas if I don’t spend some serious time practicing not doing that.
It has been really interesting being the one completely responsible for the narrative instead of sharing that responsibility with the resources that I provide for my pre-class assignments.
It has also been weird not having the inmates run the asylum.
This is part 2.5 (the second half of part 2) of a series on pre-class learning strategies, which can be used as part of the flipped or inverted class. In this post I will discuss the different types of video lectures.
- Part 1 focused on some common types of assignments/assessments that you can use.
- Part 2 discussed the various types of resources you can provide your students for use in learning before class strategies.
- Part 2.5 is this post
- Part 3 will discuss some tips and some issues that I have come across trying to implement learning before class strategies.
- I also had a quick update post that pointed to some recent articles/blog posts by others on the subject of pre-reading assignments.
Types of Video Lectures
- Screencasts – These are recordings of your computer screen with your voice over top. Sometimes people narrate as they do something in a program (e.g., how to fit your data with a Gaussian in Gnuplot), narrate as they use the computer as a virtual whiteboard (e.g., Khan Academy) or even narrate a slide presentation that is being presented (sometimes with a talking head like Roger Freedman does).
- Multimedia Presentations – You can think of these as a step beyond screencasts, with tools such as animation creating much tighter presentations than screencasts. The best example I have encountered is the collection of multimedia prelectures created by the folks at the University of Illinois Urbana-Champagne which are also included as part of their smartPhysics package. They use the PER literature and the findings of research on multimedia learning to put these together in a way that is intended to maximize what students can learn from a direct instruction format.
- Pencasts – These are recordings of what you write on paper using a special kind of pen (LiveScribe) which simultaneously records audio while you are writing. Watching a pencast is like sitting down beside somebody writing something down in a notebook while they talk you through it. Except you can click on anything they have written and rewatch it. It’s also possible to download a pdf of completed notes. I generated a whole bunch of these for my 3rd-year quantum course last term and the students gave me pretty positive feedback on the format. I also asked my students to each turn in one of their solutions as a pencast. This is a great idea that came to me from Andy Rundquist. What’s great about it is that they can’t just copy the solution from the web or a buddy. They have to make sure they understand what they did in the question before they sit down to present it as a pencast.
- Video Recordings – You can record demonstrations, targeted lectures, etc. There are also many resources on the web (see some of the links on this the Active Class post).
- I am not an advocate of traditional lecture and moving direct instruction onto a computer screen doesn’t change the fact that it is direct instruction. Most of the learning before class strategies I have been discussing rely on some sort of direct instruction to communicate initial content and I think video lectures can be effective in setting up what happens in the classroom later if they target ideas that are accessible to a novice.
- Following up on the last bullet. I have been using smartPhysics this year for my calc-based intro physics course and I think their multimedia presentations are very well done, but ultimately they are communicating content at the same level as a regular textbook (see Part 2 of this series for my discussion on the suitability of textbooks as a first-contact resource). I don’t think it is a good use of student time or good-will for them to be confronted with challenging derivations two minutes after being introduced to a new topic. These pre-lectures are editable (as in I can take some of the material out) through the smartPhysics system so this term I have been aggressively trimming down the presentations so that the students get more of an introductory overview of the chapter as opposed to a 20-minute run through of the entire chapter. There’s a post slowly writing itself on my experiences with smartPhysics.
- I have a tablet PC as my laptop, which should make it great for doing the virtual whiteboard thing, but some combination of the computer’s fan and other issues with the audio hardware introduces too much background noise (even when I have tried an external mic). I have tried post-recording noise reduction on the audio track and it worked pretty well, but that makes the whole thing fairly unsuitable for quick and dirty screencasts. I imagine that when I teach a comp phys class I will probably make more use of them for “this is how you do x” types of screencasts on a computer without noise issues.
This is part 2 of a 3 part series on pre-class learning strategies, which can be used as part of the flipped or inverted class. In this post I will discuss some of the types of resources that you can provide your students to do their learning before class.
- Part 1 focuses on some common types of assignments/assessments that you can use.
- Part 2 is this post!
- Part 2.5 discusses the types of video lectures in a bit more detail (this post was getting long)
- Part 3 will discuss some tips and some issues that I have come across trying to implement learning before class strategies.
- I also had a quick update post that pointed to some recent articles/blog posts by others on the subject of pre-reading assignments.
Photo by Kevin Dooley via Lifehacker
What I want out of these learning resources
Common types of learning resources to provide your students
These are types of learning before class resources that I have tried and I think is a fairly comprehensive list. Some of these are more well-suited to specific types of assignments (see Part 1) than others.
- Textbook – Despite everything I say above about textbooks (thus far) not being something that does a good job of being a resource for both first contact and reference, I do like Knight‘s intro physics textbook for being something that is quite readable for the student, even for first contact between the student and a new topic. Of course there are still many text elements (paragraphs, sections and examples) that I would ask the students to skip due to the amount of “mastering the basics” in class that is needed before this skipped content would be meaningful to any student other than the rare one that was determined to make sense of everything they they encountered in the textbook.
- Video lecture – These come in multiple flavours. Screencasting is becoming increasingly common, but there is also pencasting, highly-produced multimedia presentations like those found in smartPhysics, and video-recording a regular lecture. I will discuss each of these a little bit more in Part 2.5 of this series, but just comment on them as a group here. One of the strengths of the video lecture format is that it doesn’t take too much time to produce something coherent and of high enough quality that the students will find it useful. Anecdotally, I have found that students are much more willing to wrestle with slightly more challenging content in the video format than if I had just asked them to read very well presented notes on the same topic. Of course this is exactly why many (most?) students are happy to attend traditional lectures, but would never think of reading the textbook covering the exact same content.
- Simulations – Inspired by Noah Podolefsky’s Global Physics Department talk on PhET simulations I have started basing some of my learning before class assignments on simulations. What I usually do is ask them to play with the simulation for 5-10 minutes and then send me 3 questions they had, things that they discovered or things that they found interesting in that time. After trying this type of simulation-based assignment out a few times I am finding that the students tend to generate questions that touch on most of the important points you would want to touch on, but instead frames the classroom discussions in terms of their curiosity instead of you telling them what’s important (even though it is the same actual content). The feedback I got from them indicated that they really seemed to like this type of pre-class assignment. If you don’t want to just let them play and tell you about it, you can try to focus their attention on specific things by asking them investigation types of questions like “what parameters affect X?”.
- Other targeted written resources – This is a grab bag category much like the video lecture one. This category includes
- Your own written work meant to present the content at a level similar to a screencast or pencast that you would produce. A good example is the series of blog posts that Rhett Allain turned into Just Enough Physics;
- Sections of textbooks that are targeted at a lower level than the given course. I often discovered when teaching Quantum that pointing my students to Knight’s discussions on the same topic would have been a goof “first encounter with a topic” resource or for intro physics I pointed my students toward physics for future presidents as an additional resource (before the book was published, all the chapter PDFs were available on that webpage).
In many ways I prefer written resources to screencasts because (a) it is much easier to make small edits, and (b) I find it easier to piece together a few of them into a cohesive narrative. On the flip side, it is a lot less work to produce a screencast of acceptable quality than getting bogged down in writing something of acceptable quality. Or as Andy always say “I speak faster than I write.”
- Materials meant specifically to generate interest – I have only tried it out once, but I was happy with how it worked out. You can give them a popular science or journal article, a chapter from a popular science book, a video, etc. This one goes really well with the student generated question type of pre-class assignment (see Post 1 in this series) and then functions quite similar to the simulation-based pre-class assignments by letting the student questions frame the classroom discussion.
Part 2.5 so very soon.
I am just about finished writing Part 2 of me Learning Before Class Strategies series of posts (Part 1 here), but I thought I would post links to a couple of recent posts on the subject of pre-reading assignments and an article from a recent UBC CWSEI Physics and Astronomy newsletter.
But first, here is a look at the relationship between exam averages and the marks my students earned for completing their smartPhysics Prelecture and Checkpoint assignments (marked for effort only) in my Fall 2011 introductory mechanics course. Not terribly compelling in terms of trying to sell my intro E&M students on the pre-class assignments. Fortunately Peter (see link below) gave me a bunch of good motivational ammo to pass on to them.
Recent posts/articles of note on pre-reading assignments
- Stephanie Chasteen’s recent post “Do they do the reading? Helping students prepare for lecture”. As usual Stephanie manages to write something that contains tips, resources and research results, yet somehow manages to be concise.
- The article “Pre-Reading Assignments – Why they may be the most important homework for your students”, written by Louis Deslauriers, Cynthia Heiner, and Georg Rieger, in the Dec 2011 UBC CWSEI Physics and Astronomy newsletter. This has some best practices and some data regarding the sophistication level of student questions on weeks with pre-reading assignments vs. weeks without.
- Speaking of CWSEI, Peter Newbury recently had a nice post about motivating pre-reading assingments: “Motivation for pre-reading assignments”
At this past week’s Global Physics Department virtual meeting Noah Podolefsky spoke with us about PhET simulations. Noah’s best practice suggestion was to let students play around with a simulation for 5-10 minutes before asking them to do anything specific. And when you ask them to do something specific, to use open / investigative questions (e.g., “explore all things that affect pH”, as opposed to cookbook directions such as “set the acid concentration to 0.010 M…”).
I asked Noah
I’m wondering how you would suggest using these in pre-class (JiTT-style) assignments. If I am ultimately going to give them some sort of a question (could be nice and open like you suggest), I feel like most students will jump to try to answer the question without first doing the “free play”. Any suggestions on getting them to do “free play” first?
Noah suggested getting them to play around with the sim and generate 2-3 questions or screenshots of “cool things” that they found, which Brian Frank echoed by suggesting I do the same thing I did when I got my Quantum Mechanics class to generate questions based on a reading. Andy Rundquist also suggested I could get them to screencast their interesting discoveries (instead of just screencapping).
My Quantum Mechanics class is in the middle of developing the Hydrogen wavefunctions (I showed them the shooting method results for the angular wavefunctions last class, thanks Andy!). We’re skipping our regular pre-class assignment this week, so I sent them a bonus pre-class assignment before we look at the Hydrogen spectrum on Monday. Here’s a slightly paraphrased version of what I asked them to do with the “Models of the Hydrogen Atom” PhET simulation:
Spend 5-10 minutes playing around with the simulation. Generate 3 items of interest — these could be any combination of questions that you have, interesting observations that you made or descriptions of things that the simulation made really clear to you that you didn’t quite get before. You can take screen captures, generate screencasts or just send me regular old text.
I’m really interested to see what they come up with. I will make sure to report back. Just for fun, I have embedded the simulation below