Student Interview Feedback for Advanced Lab, Spring 2012, Part 2

This is part 2 (part 1 here) of my post discussing feedback I got from a couple of my students after the conclusion of my Advanced Lab course. This went long again so it looks like there will have to be a part 3.

The 8-hour work-week and filling out time sheets

The combination of this having been only my second time teaching the course and my policy on student experiments always building on, but not repeating, the project of previous groups made it very hard for me to figure out projects of appropriate scope. So my solution was to ask that the students put in a minimum of 8 hours each week into the course and then I had to make sure that the projects consisted of sequences of achievable milestones. With that in place, I was happy to accept however far along each group made it with their projects as long as those 8 hours each week were actually spent working productively on the course.

So I got them to fill out and submit time sheets. I was worried that they would perceive these as being too strict or beneath them or terrible in some other way.

Interview feedback: No complaints. The time sheets were fine and did a good job of encouraging them to dedicate an appropriate amount of time to the course even when they felt like doing something else. Yay!

Future plan: It looks like I will continue to use these time sheets. The thoughtful-assessment part of my conscience doesn’t really like having to use these, but for the most part these students have never had to budget time for a longer project and they really need the scaffolding so that they don’t fall on their faces.

Oral and written dissemination

One of my major guiding principles in this course was (and continues to be) to try to make sure that the communication of their project was directed toward authentic audiences. For the weekly group meetings, they were bringing me up to speed on their project as well as informally presenting to their work to people with less project-specific expertise (the rest of their class-mates). Since projects are always meant to build on previous projects, their formal reports are going to be part of the literature used by the next group building on that same project. Their formal oral presentations were targeted at peers that lacked project-specific expertise (again the rest of the class).

The first time I taught the course, I had the students write journal-style articles (each partner wrote one). There were two problems. First, the partner that was not writing the article ended up contributing very little to the analysis and usually didn’t dig deep enough into how everything worked from either the theoretical or the experimental side of things (which is part of why I implemented the oral assessments into the course). Second, the background and theory sections often lacked an authentic audience for multiple reasons: (A) they were often vaguely repeating the work from a source journal article; (B) if they were building on a previous group’s work, writing their own background and theory sections would be mostly redundant; and (C) the topics were often deep enough that it was not reasonable to expect them to develop the project-specific expertise to do a very good job these sections.

So in this second incarnation of the course I decided to split the journal article up into two pieces, one for each partner: a LaTeX “technote” and a wiki entry for the background and theory. The idea was that future groups could add to the wiki entry, which would eliminate the redundancy of recreating essentially the same theory and background sections for future groups working on that research line. With the theory and background stripped out of the journal article, all I asked in the technote was that all equations and important terminology be clearly defined within the technote, and no other theory was needed. I thought this would have the added benefit of having both partners invested in a writing task for each project. But the whole thing did not work very well. The technotes worked fine, but the wiki entries ended up being so disconnected from the technotes that partners often didn’t even use the same notation between the wiki entry and technote.

It is worth noting that between a time crunch and the technote+wiki not working as well as I liked, I got the students to team up and write something a bit closer to a journal article for their second project.

In addition to their technote and wiki entry, each student gave a 12-15 minute formal oral presentation on one of their projects (each partner presented on the project for which they wrote the technote) instead of writing a final exam.

One of the things I wanted to discuss in the interview was what sort of improvements could we make to this dissemination process. I had some ideas in my head to discuss such as poster presentations and articles written for a lay-audience.

Interview feedback: for each project one person would write a journal article and the other would prepare and present a poster at a research symposium. The logistics of this still need to be worked out and we discussed a number of combinations of dissemination methods before coming to a consensus on this specific one. The interviewed students saw communicating science to a lay-audience (the research symposium attendees) as an important thing for them to practice.

Future plan: I really like how this combination makes each member of the group responsible for communicating all the important pieces of their project. Targeting them at different audiences means that they will be able to work together while still ultimately having to produce their own non-redundant (relative to each other) work.

Their are a lot of logistical issues to work out here. Our university has a student research day a couple of weeks before the end of our winter term and that would be a perfect place for them to present their poster. The problem is that, with a proper revision cycle for their poster, they will essentially have had to have completed both projects a month before the end of the term. I’m not certain I can make that work. We can always have our own research symposium, but it seems ideal to get involved with an existing one that already has an audience.

The other piece here is that I will probably ask them to keep the journal articles closer to the technotes than a real journal article (meaning bare-bones theory and background).

A vague notion of a plan dawned on me while proof-reading this post. I could probably get the timing with the student research symposium to work if I reduce the scope of each project by roughly a week and then in the final month of the course I could ask each group to revisit one of their experiments and push it a bit further forward. There are all sorts of problems with this plan, such as how they will disseminate this additional work and the experimental apparatus probably having been torn down, but it is still something to consider.

The timing of peer review for their lab reports

Each technote and journal article was allowed as many drafts as needed to get the paper up to “accepted for publication with minor revisions” standards (a B-grade) based on a very picky rubric. After that, they were allowed one final draft if they wished to try earn an A grade. A typical number of drafts was 3 or 4, but there were exceptions in both directions.

For the first report, I had each person do a peer review of another student’s submission. One of the questions I had on my mind for the feedback interview had to do with the timing of the peer review in the draft cycle. The first draft of the first paper is always an extremely rough thing to slog through, even those written by very strong students. Thus, asking them to do peer review on a first draft is asking them to do something very painful. But, having to critically apply a rubric and provide constructive feedback does wonders for getting students to pay much better attention to the specifics of the writing assignment and the sooner that happens in the course, the sooner that I see those improvements in their writing.

Interview feedback: not too sure if it is best to do peer review on a first or second draft. We discussed this for a bit, decided we could see both options as equally valid, and never came to any real conclusion.

Future plan: dunno yet. I could sign my course up for the Journal of the Advanced Undergraduate Physics Laboratory Investigation tool. They have peer review calibration tasks and the added benefit of anonymous peer reviewers from other institutions, but since JAUPLI is still small, the timing all has to work out magically well.

Student Interview Feedback for Advanced Lab, Spring 2012, Part 1

Once I started writing this it got pretty long so I will call this part 1 and work on part 2 another day.

A month ago I took a couple of my students to a local coffee shop, filled them full of treats, and poked their brains for a couple of hours about my Advanced Laboratory course that ended in April, 2012. I’m summarizing their feedback here to make sure I have a good record of their feedback and, of course, to share. For any given piece of feedback from the students I will try my best to explain the context and by the end you should have a pretty good idea of how the course looked and where it will be headed in the future.

Let’s start with my definition of an Advanced Laboratory course from an earlier post:

This type of course, a standard course in most physics departments, is a standalone lab course without any associated lecture course. There is an amazing amount of variability from one Advanced Lab course to the next and they range in format from one experiment per week with everything already set up and cookbook procedures ready to be followed, to a single student-developed project over the entire term (or year!).

In my specific incarnation, we spend the first month doing some introductory activities to build up some foundational skills which are mostly related to data analysis and presentation. For the rest of the course pairs of students work on two month-long experimental physics projects. The students are guided to work on projects that can be viewed as being part of a larger research line, where they build on the work of previous students and future students will build on their work. Thus no two groups will ever perform identical experiments.

Onto the feedback!

Weekly research group meetings

Each week we had a research group meeting where each group was asked to post a couple of figures or tables to the course wiki and quickly bring us up to speed on what they had done the previous week as well as what they planned on doing the next week. A very small part of their grade was based on how much they contributed to these research group meetings. My expectation was that, averaged over multiple meetings, each student would ask at least one meaningful question to the presenters per meeting or contribute in some other way to the conversations surrounding the projects of the other groups. I had twelve students enrolled so I split the class into two larger research groups so each research group consisted of myself and three pairs of students.

I was quite happy with how these meetings worked and felt it was really valuable for the presenters to have to frame things so that people other than me and the presenters actually understood what they were up to. Anecdotally it felt like students spent more of their initial time learning about the basics (experiment and theory) of their projects than in the past because they were going to have to explain things to somebody else.

Interview feedback: the meetings felt too long. Their initial suggestion was to make the meetings every other week or to time-limit them somehow. A research group meeting took somewhere between a half-hour and an hour each week. The actual presentations were reasonably concise, but there were always lots of questions from the other groups and feedback from me. This was also the place where we hashed out a lot of the gory details of what they should try to do in the coming week. But the thing was that the other groups often contributed to these discussions on what to tackle in the coming week so I felt like the whole process was extremely valuable for all parties. But it could probably be tightened up.

Future plan: Partners will alternate being the main presenter each week (previously they were both expected to present each week) and will be asked to present 1 or 2 tables of figures. The feedback was that it sometimes felt like a stretch to find that 2nd table or figure to present. The actual presentation will be limited to 5 minutes for a total of 10 minutes per project group between the presentation and the discussion afterwards. I won’t be strict on the time limits, but will be mindful of the clock to help prioritize which discussions to have at that moment and which ones can be saved for private discussions later. One of the students also suggested that having time limits on their presentation time would serve as good practice for their formal oral presentations later in the course and these did have strict time limits.

Parallel investigations

Twice during the intro sequence I tried to have a number of small groups working on different things and then had them report out to the class. The second time we did this I called them “parallel investigations” and sent them off to study goodness of fit, Monte-Carlo methods for fitting or Monte-Carlo methods for error analysis. In addition to orally reporting out their findings, I asked one partner to write their findings up in LaTeX and the other on the course wiki. These two write-ups were allowed to be identical and the reason that I used two formats was because one of the students was going to write up the background and theory for their first project on the wiki and the other was going to write up the analysis and results as a LaTeX “tech note”. Thus I wanted them to have some practice using these writing formats. Note that on the second project they were to switch who wrote on the wiki and who wrote the LaTeX “tech note”.

Interview feedback: this might not have been the best use of their time. Yup, I agree. In the end, for the investigations that they did not perform, each group was simply ont he receiving end of a mediocre lecture on the topic and never got a chance to actively engage with the ideas. This is the exact opposite of how I try to run my courses.

Future plan: I’m planning on completely restructuring how the first-month introductory stuff works and will talk about that a bit more later, but I think the parallel investigations idea as it existed is officially dead.

Introduction to LaTeX

This is a place where I have not offered my students a ton of support. I wrote a tutorial so that they can install miktex and texniccenter on their windows machines and gave them a couple of sample LaTeX documents that cover all the basics, but that’s it.

Interview feedback: they would like some coherent instruction and resources. For most of them this is their first time ever dealing with a markup language and the learning curve seems to be steeper than I have been admitting to myself.

Future plan: It looks like a crew of us on twitter are going to put together a LaTeX workshop for the Summer 2013 AAPT meeting and I am hoping that as part of this process we will have put together a straight forward introduction to LaTeX for physicists package that I can drop on my students like a big ol’ pile of awesomeness.

Notebook activity

From Student-Generated Scientific Inquiry (Leslie Atkins and Irene Salter) I used their lab notebook activity which uses pages taken from different famous scientists actual lab notebooks. The students are asked, in small groups, to take some notes on how these famous scientists took notes and organized their information. As a large group we then built a rubric for their lab notebooks based on their observations of the pages from the notebooks of the famous scientists. The students were highly engaged in this activity and seemed to be supportive of the rubric that we developed from this activity.

Interview feedback: they thought this activity was great. But they didn’t find it in the way that I expected. The two students I interviewed both had some previous experience with lab notebooks in research labs and had, in the past, put way too much emphasis on maintaining an immaculate lab notebook. This activity had let them know that it was OK to have rough notes in their lab notebook.

Future plan: I hate marking lab notebooks. It is the worst. And with so much of the work they do being digital these days it is really hard to find a solution that fits into their work flow and doesn’t involve pasting umpteen print-outs into their lab notebook. I’m actually planning on backing off of trying to get them to keep a really good lab notebook and emphasize getting them to report at the beginning and end of the day what they planned on doing and what they actually accomplished (science fair style!). I will be checking it every class period and it will be graded as complete or incomplete. Once I feel that I can get a group of students doing a consistently good job of this, I will consider the next step to take.