Looking at the number of drafts submitted for project reports in the Advanced Lab

In this post I take a quick look at the number of drafts of their project papers that students submitted in my January 2012 Advanced Lab course. This course had a minimum bar for the paper grades and the students were allowed to revise and resubmit as many times as needed to get there, with an average of 3.22 drafts needed. I decided to look at these numbers for the purpose of communicating realistic expectations to students currently registered for my fall section of the course and thought I would share those numbers.

I am starting to prepare for my fall Advanced Lab course. Here is a quick overview of this course from a previous post:

This type of course, a standard course in most physics departments, is a standalone lab course without any associated lecture course. There is an amazing amount of variability from one Advanced Lab course to the next and they range in format from one experiment per week with everything already set up and cookbook procedures ready to be followed, to a single student-developed project over the entire term (or year!).

In my specific incarnation, we spend the first month doing some introductory activities to build up some foundational skills which are mostly related to data analysis and presentation. For the rest of the course pairs of students work on two month-long experimental physics projects. The students are guided to work on projects that can be viewed as being part of a larger research line, where they build on the work of previous students and future students will build on their work. Thus no two groups will ever perform identical experiments.

A major piece of the course is that they have to write a journal-style article to communicate the results of one of their projects. To help them practice revising their own writing and impress upon them that effective writing requires many revisions, I require that students earn a grade equivalent to a B on their paper according to this rubric, and are allowed to revise and resubmit as many times as needed to reach that threshold grade.

The overall grade for these papers was calculated as 25% from the first graded draft and 75% from the final draft. They were allowed to submit an initial draft, which was not graded, where I would spend a maximum of a half an hour reading over the paper and providing feedback. Students were encouraged to have a peer read through their paper and provide some feedback before submitting this initial draft. After reaching the threshold B-grade, they were allowed to resubmit one final draft. At some point in the revision process I also had a formal process where students provided each other with some peer feedback on their papers.

A quick summary of the numbers are in order. Of the twelve students, three of them gave up at some point before reaching threshold B-grade on the journal-style article. Those students were only given partial credit for the last grade that their paper received. Of the nine students whose papers reached the threshold B-grade, five of them submitted a final draft to improve their overall paper grade.

Of the 9 papers that were accepted (met the minimum grade threshold of a B), 5 of them were revised at least one additional time .

The number of drafts in this graph includes the initial ungraded draft, but does not include the final revision that 5 of 9 students submitted after their papers reached the B-grade threshold.

What is the take-home message here? Based on this system, students should expect to submit three or more drafts of a paper in order to meet the threshold grade.

This coming fall, I plan to adopt some new feedback strategies that  take the focus off grammatical correctness and similar issues in the hopes to focus more on the ideas in the papers. As part of this, I may move to a reviewer report style of feedback (for example, this is the one for AJP) and away from detailed rubrics, but I haven’t quite made up my mind on this yet. My grading philosophy in the course this fall will be that their course grade will represent the quality of the recommendation that I would give them in a reference letter based on their work in the course, and I want to do my best to make sure all of the individual components are assessed in ways that match up with this overall grading philosophy.

Student collision mini-projects from my fall comp-phys course

This past fall I had a revelation which I have yet to harness, but it is hiding out in my brain waiting to be incorporated into future courses. In two of my courses, I had the students work on mini-projects. This was the first time I had used mini-projects in a course and I was delighted with how independent the students were as compared to an overly prescribed task and I was also delighted with the quality of their work as compared to work from the regular prescribed tasks. Later in this post I have shared some videos of the comp-phys mini-projects, but I want to discuss a few things first.

In my digital electronics labs course they were asked to take the input from an analog sensor, apply some electronic decision-making to this input and provide some digital output related to the input. An example is to use a photoresistor to monitor room brightness and use 3 different colours of LED to provide feedback related to the room brightness: a red LED is lit if the room is dark, a yellow LED is lit if the room is of “standard” brightness and a green LED is lit if the room is extremely bright.

In my comp-phys course they were asked to make a collision simulation using Mathematica or Python where there has to be at least 3 different parameters which can be manipulated by the user (e.g., mass, velocity, coefficient of restitution, type of object) and at least one challenging piece of physics in the simulation (e.g., rolling friction, coefficient of restitution which varies between 0 and 1). Examples ideas that I provided included the ballistic pendulum or a 2D collision where you have to worry about the angle of attack.

In both cases, the task was designed to be something which should take approximately one week of the regular time that they are expected to put into the course. In both cases I had some small-in-scope expectations related to the documentation/presentation of the mini-project.  For the digital mini-project, I asked them to submit a complete circuit diagram and a brief video of them walking me through how the mini-project works. For the comp-phys mini-project, I asked for well-documented code and a brief document which highlighted the physics being  simulated and explained how it was implemented in the code.

Before I share the comp-phys mini-projects from the fall, I want to share an “in no particular order” list of things that I liked about the mini-projects above what I would see from a regular prescribed task or series of tasks:

  1. The students seemed much more willing to take on larger challenges with less support.
  2. The students were provided with the opportunity to bring some creativity into their work. There seems to be very few of these opportunities in most physics programs.
  3. The quality of student work was consistently higher than usual and competition played a small role in this. With the comp-phys mini-projects, students would show up to class and see what others had done and decide they had to step up their game by adding more bells and whistles than they had originally intended.
  4. The students had a lot more ownership of the learning task.

I suspect that Andy has seen a lot of these benefits since switching to SBG. A lot of the student submissions for standards that I have seen from his courses seem to involve some creativity and students taking on larger challenges that would normally be expected. The scope of those standards tends to be smaller than the mini-projects I am talking about here, but my experience with mini-projects certainly helps me appreciate even more how powerful SBG can be in terms of giving the students some choice in how they show their proficiency.

Mini-project playlist

Below is a playlist of no-audio videos of the 10 mini-projects from the comp-phys course. Each of them is in the neighborhood of 30 seconds long of me playing around with the various controls and then running the simulation one or two times. Some of them were done by groups. They’re pretty tiny in the embedded player so I would suggest going full-screen.