Category Archives: assessments

Project-Based Learning in New Software Engineering Course

A couple years ago, I wrote about a new course that would be offered in the 2017-2018 school year, Software Engineering. Last school year, the course was a great success, although there are many refinements I’m preparing for this upcoming school year. The second semester of the course is described as “small groups of students develop a software product as they iterate through software engineering development cycles to produce a software product”. Other teachers expressed the most interest in how to facilitate teams of students managing a semester-long project and how to assess them throughout the semester. Until last semester, I had never done this as a teacher. Fortunately, I remembered that I did this every day in my previous career as a software engineer.

One of my roles in my previous career was that of a methodologist with a specific focus on Agile Methodologies. After a review of the current state of Agile Methodologies, I decided that Scrum would be a good fit for my students. None of these students have any experience with Agile Software Development, and few have worked as part of a team on a substantial project. Scrum provided sufficient structure while maintaining the lightness appropriate for small teams. The activities in each sprint provide the opportunities for everything that is needed in project-based learning: planning, review, and reflection. I attempted to fulfill the role of Scrum Master. Each team had a product owner who was external to the class.

If you are interesting in a succinct overview of Scrum, I highly recommend the book, Scrum: A Breathtakingly Brief and Agile Introduction by Chris Sims and Hillary Louise Johnson. (If you want more, their book The Elements of Scrum is also excellent if the first book leaves you wanting more.)

I scheduled the teams into four, staggered, 4-week-long sprints. While all teams started and ended their project based on the start and end of the semester, the first and last sprint for each team were of varying length such that I could facilitate the activities with each team during class.

The key activity in Scrum is the Daily Scrum or stand-up meeting. Teams would hold their Daily Scrum twice a week. Having a four-week sprint seems long, and holding a Daily Scrum twice a week seems infrequent, but these teams aren’t developing software as a full-time job; they are in class for 50 minutes a day. At the beginning of the semester, I would facilitate every Daily Scrum meeting in my role as Scrum Master. Soon students became adapt and comfortable, and teams held their Daily Scrums while I observed from afar.

We focused on four activities in each sprint: Sprint Planning, Story Time, Sprint Review, and Retrospective. For each of these, the team would assign a student responsible for facilitating the activity, capturing the work, and sharing it with the team. With a team of four students and four sprints, each student would facilitate each of the four activities throughout the semester. The student facilitating the activity was assessed based on the student’s effectiveness in facilitating the activity, their demonstrated understanding of the activity, and the quality of the submitted document.

Repeated feedback is essential for both project-based learning and software development. Teams would receive feedback from their product owner during each sprint review (and more often as needed). Teams would receive feedback from me and each other during each retrospective.

In addition to these standard Scrum activities, I added an additional activity at the end of each sprint, a personal retrospective. Each student would evaluate themselves based on their role as a Scrum Team Member, reflect on the personal goal they set in the previous sprint, and set a new personal goal for the next sprint. Based on student feedback, for the upcoming school year, I will also have students evaluate their teammates.

At the end of the semester, we hold a Demo Event attended by all the students, their product owners, and other stakeholders. Each team presents their product in a modified version of the “3-in-5” presentation format. After the Demo Event, each student completes a final retrospective reflecting on the entire semester.

This structure worked well for my students and their projects last semester, and we will use the same structure next year with some refinements.

One final note – Scrum is used for all kinds of projects; not just software projects. After my experience last semester, I would encourage teachers to consider Scrum as a tool to facilitate any project-based learning of a significant duration.

Questions? Comments? Please comment here or each out to me on Twitter.

Electronic Lab Portfolios Aligned to AP Physics Science Practices

[Updated 15/7/2016, 10:54 PM: added links to two student lab portfolios.]

As I mentioned briefly in my reflection of the 2014-2015 school year, this past year, students created electronic lab portfolios for AP Physics 2. In summary:

  • many students demonstrated deeper metacognition than I have ever observed
  • several students struggled and their portfolios were incomplete
  • providing feedback and scoring consumed a huge amount of my time
  • structural changes made in the spring semester helped considerably

Structure

I was inspired to have students create electronic lab portfolios based on Chris Ludwig’s work and his presentation and our discussion at NSTA last year.

Before the start of the school year, using siteMaestro, I created a Google Site for each student based on a template that I created. I made both myself and the student owner of the site and kept the site otherwise private. The template consisted of two key portions of the site: a Lab Notebook, which provides a chronologically accounting of all labs; and a Lab Portfolio, which is the best representation of the student’s performance. I shared a document with the students that explained the purpose and distinction between the Lab Notebook and Lab Portfolio.

The lab portfolios were structured around the seven AP Physics Science Practices. I wanted students to evaluate and choose their best work that demonstrated their performance of each Science Practice. I also wanted the most critical and significant labs to be included; so, some labs were required to be in the lab portfolio. In the fall semester, I required that each student publishes at least two examples of their demonstration of each of the seven Science Practices.

I wanted students to think more deeply about the labs then they had in the past, and I didn’t want the lab portfolio to just be a collection of labs. So, in addition to the necessary lab report to demonstrate a given Science Practice, students also had to write a paragraph in which they reflected on why this lab was an excellent demonstration of their performance on the specific Science Practice.

The lab portfolio comprised 40% of the coursework grade for each semester. For the fall semester, the lab portfolio was scored at the end of the semester. I provide a few formal checkpoints throughout the fall semester where students would submit their portfolio (just a link to their site) and I would provide feedback on their labs and paragraphs.

Fall Semester

Many students wrote excellent paragraphs demonstrating a deeper understanding of Science Practices than anything I had previously read. Other students really struggled to distinguish between writing a lab report and writing a paragraph that provided evidence that they had performed a given Science Practice. I did create an example of both a lab report and lab portfolio reflection paragraph based on the shared experiment in first-year physics of the Constant Velocity Buggy Paradigm Lab. However, several students needed much more support to write these reflection paragraphs.

In general, those students who submitted their site for feedback had excellent portfolios by the end of the students; those who didn’t, underestimated the effort required and ended up with incomplete or poor-quality portfolios.

What I liked:

  • The metacognition and understanding of Science Practices demonstrated by many students.
  • Students deciding in which labs they most strongly performed each Science Practice.

What I Didn’t Like:

  • Several students struggled to distinguish a lab report from a paragraph providing evidence of performing a Science Practice.
  • Several students didn’t have enough support to complete a project of this magnitude and ended up with incomplete lab portfolios.
  • Providing feedback and scoring all of the lab portfolios over winter break consumed a huge amount of time.

Spring Semester

The spring semester has some different challenges and constraints:

  • We focus more on preparing for the AP exam and less on lab reports.
  • I don’t have the luxury of a two-week break to score lab portfolios at the end of the semester.

Based on these constraints and our experience during the fall semester, I made some changes for the spring semester. I selected seven required labs in the spring semester, one for each Science Practice. Each lab and reflection paragraph was due a few days after performing the lab, not at the end of the semester.

This had some advantages:

  • the portfolio was scored throughout the semester
  • students had more structure, which helped them stay current

and disadvantages:

  • no student choice in selection of labs to include in portfolio
  • no opportunity to revise a lab or reflection paragraph (the feedback could help them in labs later in the semester)

With these changes and students’ experience from the fall semester, the lab portfolios in the spring semester were largely successful. I think it is important to emphasize that both the changes and the students’ experience contributed to this success. I do not believe that the structure for the spring semester would lead to a more successful fall semester. The feedback I received from students at the end of the year was much more favorable concerning the structure in the spring semester than the structure in the fall semester.

Next Fall

I had the wonderful experience of being coached this year by Tony Borash. Tony provided guidance in many areas, one of which was making these adjustments for the spring semester and, more importantly, planning for next year. Together we were able to come up with a structure that will hopefully combine the strengths of the structure in the fall semester with the structure in the spring semester. My goals for these changes are to:

  • provide more structure for students
  • provide student choice
  • incorporate peer feedback

Here’s the plan for next fall:

  1. I choose the first lab. Students complete and submit the lab and the reflection paragraph. I provide feedback. Students make revisions and re-submit the lab and reflection paragraph. We review the best examples as a class.
  2. I choose the second lab. Students complete the lab and the reflection paragraph. Students provide peer feedback to each other. Students make revisions and submit the lab and reflection paragraph.
  3. Students choose the next lab to include in the portfolio. Students complete the lab and the reflection paragraph. Students provide peer feedback to each other. Students make revisions and submit the lab and reflection paragraph.
  4. Students choose some of the remaining labs, and I choose some of the remaining labs. Students complete the labs and reflection paragraphs. Students specify a subset of Science Practices on which they want formal feedback from me and on which they want feedback from their peers. Students make revisions and re-submit.

This past year, students included a link to their lab report in their lab portfolio and shared the lab report (as a Google Doc) with me. Next year, I will have students embed their lab report into the Google site. This will facilitate peer feedback and enable everyone to use comments within the Google site to provide feedback. I may still have students share the actual doc with me, as well as include a link, so I can provide more detailed suggestions directly within the document.

Student Examples

Conclusion

I’m pleased that my students and I are heading down this path and believe my students will gain a much deeper understanding of Science Practices as a result. While I shared this with my colleagues this past year, I also cautioned them that I didn’t have it figured out, and it wasn’t a smooth ride. I think electronic lab portfolios are an excellent way to assess student performance, and I hope that they will be used in other science courses in the future as they are a natural fit to the NGSS Science and Engineering Practices. I hope that after this next year, I will have something that will provide my colleagues with a stronger framework to adapt to their classes.

My AP Computer Science 2016 Solutions

I shared these with my students and thought that others may be interested as well. I typed up solutions to the 2016 AP Computer Science free response questions. The zip file includes a BlueJ project file and test code to verify solutions. As I tell my students, no guarantee that I wrote perfect solutions and there are multiple ways to answer these questions.

AP Physics B Assessments

As I’ve mentioned, I’m spending some time this summer preparing for the AP Physics B course that we will be teaching for the first time this fall. I recently finished creating the assessments for this course.

With one exception (fluids multiple choice), all questions are from previous AP Physics B exams. Thanks to the handy indexes available from Secure PGP, it was relatively easy to review relevant questions and problems and choose those I wanted.

While compiling the assessments, I refined the granularity of the units a bit.

Fall Semester

  • Special Relativity
  • Kinematics
  • Statics and Dynamics
  • Fluid Mechanics
  • Work, Energy, Power
  • Thermodynamics
  • Linear Momentum
  • Oscillations, Gravity, Waves
  • Capstone Project

Spring Semester

  • Electrostatics
  • Electric Circuits
  • Magnetic Fields and Electromagnetism
  • Geometric Optics
  • Physical Optics
  • Particle Physics
  • Atomic Physics and Quantum Effects
  • Nuclear Physics
  • Cosmology

For each unit, I compiled a quiz that contains representative free response problems to be used as a formative assessment. I then created an end-of-unit exam consisting of multiple choice and free response questions. The exam is intended to be completed in a 50-minute class period or less. To support the flavor of standards-based grading that I’m using in this class, I also created a reassessment consisting of multiple choice and free response questions. Scoring rubrics for all free response questions have also been compiled for each assessment.

I uploaded the assessments as an archive for each semester to Secure PGP. I included the original Pages documents as well as versions exported as PDFs and Word files. I hope that some of you find these helpful. Please let me know if you find any mistakes.

Critical Thinking Assessment

Those of us teaching physics have made a lot of changes this year. One major change is a focus on depth of understanding and critical thinking, which results in fewer topics covered. While I have qualitative evidence through formative assessments that students this year have developed stronger critical thinking and long-chains-of-reasoning skills, I’ve been bothered that I don’t have a summative assessment to measure this. Ideally, I would like an Force Concept Inventory pre-test/post-test equivalent for critical thinking. I’ve bookmarked the College and Work Readiness Assessment (CWRA), but that isn’t an assessment that I can administer to my own class. If you know of another, please let me know!

Due to our crazy calendar and snow days, seniors graduated two weeks ago and I’ve had relatively few students in my regular physics classes since then. We’ve been investigating color, polarization, mirrors, and lenses. Since these students had already completed their final with the seniors, I decided to use the scheduled final exam time this week to try a critical thinking assessment. I wanted them to read a passage that describes a physics phenomenon with which they were unfamiliar, make several observations of a somewhat related physics phenomenon that they had never seen, and propose and defend an explanation for this observed phenomenon based on the prior knowledge. They read about diffraction, observed various wavelengths of light passing through various double slits, and tried to formulate an explanation. We had previously learned about interference of waves (slinkies and beats), but not in the context of light. This is quite a series of inferences and connections for students to make during a final exam; so, I prepared a series of guiding questions to help them make the connections. When a student said they were stuck or were off-track, I gave them one of the five guiding questions. Some students needed all five; one, amazingly, didn’t need any.

Here is my reading passage, observation procedure, and guiding questions:

Download (PDF, 231KB)

The students did quite well connecting what they read about diffraction to what they observed to what they already knew about interference. Here are some of my favorite student comments.

The black parts are shadows, I think we see them because the light is being destructed?

In order to have constructive interference, one ray must travel one wavelength further than the first.

The blue filter causes the lines to be closer than that of the red filer because blue has a shorter wavelength and when it travels to the plate, it forms more concentric circles. Thus, there are more intersection of circles and more lines formed.

and my favorite (written without any guiding questions):

… It’s like light beats.

I also had some really creative explanations of the interference pattern. Students mentioned internal reflection in our eyes as well as lens effects due to the slits.

Students commented that this was unlike any final exam they had previously taken. In fact, several students in one class didn’t want to leave until they were satisfied they had a complete explanation. It certainly seemed more worthwhile than giving students a list of equations and a set of problems with numbers for them to plug in on their calculator. I think there is a kernel of a good idea here, but I need to develop it more. In my largest class, it was hard to manage since I had to interact with each student during the assessment, read their explanations, and give them the appropriate guiding questions. Sometimes this required me asking my own clarification questions and the ensuing discussion could be overheard by other students. If you have tried anything like this, please share your experience!