Virtual spring term wrap up

With all non-senior grades due yesterday morning (senior grades were due last week), spring term is finally and officially in the books!

Given that my research students started this week, and looking ahead to my schedule for the week (so! many! meetings!), I completed and submitted all of my grades at the senior grade deadline. I’ve now had a week to regroup and reflect on the term, and figure out what lessons I’ll take away from it.

In this post I’ll talk a bit about what went (surprisingly) well, what fell flat, and what I’d do differently next time, whether we’re in person, online, or some combination of the two.

What went well

Specificity. I tend to be very specific in my assignment prompts and in my assignment rubrics. Over time I’ve recognized that this is good and inclusive pedagogical practice, but honestly it was born out of necessity — as junior faculty in a male-dominated department, it was a defensive mechanism against students who questioned my pedagogy and right to be in the classroom, and/or felt I wasn’t qualified to “appropriately” assess their work. Specificity, it turns out, really helps students focus on what’s expected of them, particularly when they’re already feeling overwhelmed in an unfamiliar learning environment.

Organization. When most of your course delivery is asynchronous, Moodle gets unwieldy quickly. Even in non-virtual terms, Moodle gets unwieldy quickly. I’ve developed a template of sorts over the years for organizing my course weeks on Moodle with judicious use of labels and the assignment module, and I modified that to fit the flow of our virtual class. I also discovered that Google Calendar, when you embed it in Moodle, provides a much easier interface for students to figure out what’s due when than the built-in Moodle calendar, so I relied on that quite heavily. Finally, I’d never used Activity Completion before, but I leaned heavily on that to both control access to material (“complete this to unlock that”) and to give the students a way to keep track of what they’d completed and what they had left to do.

Consistent, stable teams. I decided early on to make the term-long project teams “collaboration teams” for the entire term. While a few teams struggled to connect with each other, the majority of teams connected effectively and formed mini-communities within the course, and these teams worked pretty much how I intended them to work. I suspect this “saved” a few students who might otherwise have gotten lost from completely falling off the wagon. (Unfortunately, it did not save all of these students.) For the most part, teams provided much-needed community within the larger course, and helped replicate some of the “table culture” that emerges in the face-to-face version of this course.

Team meetings. Boy, I wish I had done this earlier in the term! I met with each team over Zoom in the second-to-last week of the term, after peer and self evals came in and around the time the penultimate deliverable was due. I spend about half a class period doing this during in-person terms, and use it as a way to help teams figure out what to focus on for the final version of their project (are there features to jettison? are they focusing on user goals? what will get them closest to the vision they had at the start of the term?). This also provides a way for me to talk with students about team dynamics that emerged in the peer and self evaluations. I think the students were relieved to get some one-on-one focused time with me, something more personal than the weekly Zoom class meetings. And I made sure to do a quick non-course check-in with the students during this time, to focus on their well-being. This was a huge success.

Class discussions. It took me a while to figure out how to conduct these in a more authentic and engaged way. I used stable breakout rooms so that students were always paired up with the same students (their project teams), with some minor shuffling if only one student on a team showed up to class. I quickly realized that me jumping from room to room to check in did not work at all — it was clunky and immediately halted conversation. I started using Google Docs, placing all of the discussion prompts and spaces for notes there. (Sometimes I used a single document for the whole class; other times, each team had their own Google Doc.) Instead of room-hopping, I monitored the Google Doc(s), jumping in with comments in the Doc to redirect discussion or elevate a point, and popping into breakout rooms when it appeared a group was heading off-track or clearly lost. As an added bonus, the Google Docs provide a record of the discussion, so that students who could not participate in real time could still reference the notes and take-away points — and even students who participated could go back and review the take-away points. I definitely plan to use this strategy much more in the future, even in face-to-face courses! (And this might be the only effective way to conduct discussion in a socially-distanced, mask-wearing classroom.)

What fell flat

Building community. I had modest hopes at the start of the term that Slack could provide an acceptable way to build community asynchronously. I seriously underestimated the amount of work that building community online takes. Despite my best efforts (which were pretty lousy, I’ll admit!), interactions on Slack were pretty much one way between the students and me. My attempt to wrangle together a virtual project demonstration/feedback session (as several smaller combination showcase/office hours during Reading Days and Finals) failed to yield a single participant. I delve a little more deeply into the issues (and how I attempted to use badges to salvage the community) in this post on Carleton’s Learning and Teaching Center blog.

Office hours. Try as I might, I could not get students to utilize office hours on any sort of regular basis. I had both drop-in hours (no advance appointment necessary) and office hours by appointment, and both went over like lead balloons. I am not sure why students did not utilize these — fear? lack of a pre-existing relationship? (although even students I knew before this course failed to take advantage of office hours.) branding? I wonder if requiring students to make an office hours appointment with me in the first couple of weeks in the term would help fix this. At least it would remove the barrier to setting up the meeting and then showing up to the meeting.

Timing of deliverables. There was a mismatch in my head, over how I thought each week would flow, and how this matched up with when I had readings and reading activities due. Often this meant that our Wednesday synchronous class meetings hit on things that technically weren’t due until Friday. Luckily, this is easily fixable moving forward, particularly since now I have a slightly better sense of how to manage flexibility with keeping everyone roughly on the same track.

Flexibility in deadlines. Let me preface this by saying that flexibility was absolutely the right call, given all of the things students faced this spring and continue to face. For many, this flexibility allowed them to successfully complete and pass my course. Every major deliverable in the course had a de-facto 48 hour no-fault extension built in. Once students realized this, however, many of them treated that 48 hour buffer as the actual deadline. This made it more difficult for my grader and me to keep up with assessments, and to manage deliverables that built upon other deliverables. When most work revolves around team deliverables, there’s also the tricky balancing act of handling a student who needs some flexibility with the needs of the other students on the team — how does everyone navigate this minefield while meeting everyone’s individual learning goals and needs? I don’t have any great answers, but I did learn a lot this term and will use those lessons to craft better ways of handling this in the future.

Concrete takeaways and homework for the summer

I realized as I compiled my thoughts for this section that all of the takeaways revolve around one theme: providing students more autonomy and independence over their learning. Moving online forced us more into that mode, but continuing this mode offline is a good idea pedagogically anyway. So, what can I do to help students become more independent learners?

First, I want to explore the ideas of specifications and/or contract grading. I sort of did specifications grading-light this past term, with my rubrics following a “does not meet expectations”/”meets expectations”/”exceeds expectations” format, but I want to expand this idea out further. I also want to look into working with students on what they want to get out of a course and how to structure the pieces of the course to help them craft that for themselves.

Second, I want to continue integrating activity completion into my courses, to help students keep better track of expectations and due dates, and to control the release of information better (“do this before you can unlock that”). Ideally, this will also help students connect the dots of how all of the pieces relate and inter-relate.

Third, I suspect we’ll be in pandemic mode for quite some time, and that things may never quite get back to “normal”. Students may continue to engage in courses in multiple ways: in person, completely virtually, some fluid combination of these. How accessible are my courses, really? I learned a lot this term about the barriers to learning in my own materials — unclear labs, dense readings, opaque tool documentation, etc. I want students to grapple with the course concepts, not the how-tos.

Finally, I want to empower students to seek help and form connections with me and others in the course. How can I be more explicit in how I connect students to each other, and ensure that all students feel like they belong and are valued? How can I foster more back-and-forth interactions in both face-to-face and asynchronous conversations?


While this was not at all the term I was supposed to have, the struggles, the constant adjusting and readjusting on the fly, the dealing with students’ (and my own) trauma, have all made me a more reflective, more compassionate, and more effective educator. I am eager to apply these lessons moving forward, in making my courses more inclusive and accessible to all.

Week 8: The hard stuff

We’re heading swiftly toward the end of the term: next Wednesday is the last day of classes, and June 8 is the last day of finals. While at many times this term seemed like a colossal slog, now that we’re finishing up it seems to be moving at warp speed.

At this point in a normal term, I tend to ease up a bit. I know my students are stressed and tired (heck, I’m stressed and tired), so I refrain from assigning new, heavy things. The key focus now in Software Design is on finishing their term-long website projects, which keeps them plenty busy anyway. Integration is hard, so I want to give them the space and time to grapple with those tricky integration issues and annoying well-this-worked-before-why-is-it-crashing-now? bugs.

I’ve eased up this term, too, and perhaps it’s even more important now, as we’re all worn down by the fatigue of uncertainty, the struggles of learning in community online, and too much Zoom.

There are two course activities I typically do in the last couple of weeks in the term: a code review, and project presentations. Both, it turns out, are challenging to rework in a fully virtual environment. I still haven’t figured out how to pull off the project presentation piece, to be honest, although I really need to figure out something ASAP! But I did find a way to pull off the code review, and I’m eager to see how it goes.

Code review should be easy to pull off if you do it the way it’s normally done — as a way to review/test/try to break code for which you’ve submitted a pull request. This assumes access to a common repository and that you’re all working on the same codebase, which is not true here. My version of code review in Software Design resembles peer writing workshops. I divide students into “feedback groups” (usually two development teams per group), have them exchange code (usually a specific class and any helper classes necessary to understand that class), and have them review code more like you’d review writing. Groups project code onto one of the many monitors in the classroom and gather around tables to discuss it.

I considered several different ways to do this virtually and asynchronously, and weighed using various tools. In the end, I decided the costs of throwing yet another unfamiliar tool at my students outweighed any small benefits they’d derive from learning that tool. So students will use Google Docs, copying and pasting the relevant code into a Google Doc, and optionally applying syntax highlighting with one of the myriad tools out there. (I suggested Code Blocks, and I just found an online highlighter that actually connects to a website, unlike the others I tried.) Students will use the commenting feature to highlight and provide feedback on specific aspects of the code, and set up Slack channels in our course workspace for longer discussions about the code under review. This way, I can dip in and monitor the level, type, and content of feedback that teams exchange with each other. Which is actually a net benefit, because I’ll get much more information on how students review each others’ code than I do normally when I’m flitting around the room trying to listen in on each group’s feedback! (And I can provide a better post-mortem after the fact, using specific comments and examples.) In our synchronous class meeting today, before they start code review in earnest, I’ll have them work in teams to review a short snippet of code, so that they can practice the workflow and get feedback from me on how they’re reviewing code. I’m really eager to see how this goes.

The hard stuff, as I alluded to in the title of this post, leveled up this week, not just in my course, but in pretty much every aspect of my job:

  • I received unexpectedly bad news about my planned research project with students for the summer, and scrambled to work up a replacement project (after panicking, swearing, and throwing things, of course).
  • In a similar vein, I’m working with others on how to create a community of student researchers when they are all isolated from each other and remote. It’s not impossible, but it’s new to us and tricky to do well.
  • Students who struggled all term continue to struggle, meaning I’ve had some difficult conversations about what it will take to pass the course (even with our version of pass-fail grading, which is basically “pass with a C- or above”, “pass with a D”, and “fail”).
  • Discussions about next year continue, maybe not as fruitfully as I’d like and maybe with fewer answer and many more questions/unknowns than I’d like. It leads me to question whether the right conversations are not happening at all behind closed doors, or whether the right conversations are happening behind closed doors and the failure is in communicating this information to faculty and staff. Regardless, the end result is the same — more angst, more uncertainty, more anxiety about the future.
  • I have some big tasks on my plate — make sure research students get paid this summer (no small feat when it seems like everyone’s projects keep changing), assigning Comps (capstone) groups for next year, conducting oral exams for this year’s Comps students — that take plenty of time and mental/emotional energy, both of which are in short supply lately.

The best I can do is to do my best in managing my time and my energy levels, doing what I can when I can, and taking the time for self-care so that I can be present, mentally and emotionally, for all of the hard stuff on my plate. And to remember that now, especially, good enough is good enough. This will let me get to the end of the term in one piece, and leave me with enough emotional and mental space once the term concludes to reflect on the term and apply what I’ve learned to….whatever fall term and beyond look like.

Week 6: Engagement

We’re firmly in the second half of our abbreviated Spring Term now, and everyone is….looking ahead, anxiously. Looking ahead to the end of the term, yes, but also, increasingly, looking ahead to the summer and to next year.

I’ve spend much of my time this week getting a handle on the research funding situation for the student grants programs I oversee. Who’s still doing research? Who lost their opportunity? Who’s eligible to defer funding to next year? What might that process look like? And who, without that experience, finds themselves in a precarious financial situation, now that the summer income they banked on no longer exists?

In a similar vein, I’ve spent a lot of time thinking through how to train and mentor new undergraduate researchers in the art and science of research….remotely. So much of that first research experience (and beyond, too!) relies on ready access to your research mentor — popping down the hall to ask questions, working side-by-side in the lab, impromptu whiteboard sketching sessions. How do I replicate that when we’re all just boxes on a screen? How do I encourage students to “bug” me with their questions? How do I mimic the side-by-side work sessions in a way that feels natural? In short, how do I make sure students don’t fall through the cracks?

This last bit was one of the main points of discussion earlier this week in a meeting of the research cohort program I direct. I asked the students, “who feels comfortable going to a professor’s office hours to ask for help?” (they all were), then pointed out that seeking help from your research mentor is largely the same thing, except it’s like your research mentor has office hours all day long, just for you. Hopefully that sticks in their minds.

I’m thinking about engagement in my course, too — how to reach those who are not engaging with the material/their classmates, and how to incentivize engagement generally. I spent some time earlier this week creating Moodle badges, which seems like a small gesture but one that I’m willing to try. I have a badge each week that’s automatically awarded if they check off all of that week’s activities by Sunday night (of the following week). I have a badge for “attendance” (awarded to those who show up to EVERYTHING), badges for asking good questions and making astute observations, a teamwork badge, and a “helper” badge for anyone who helps out a classmate on Slack. These roughly correspond to the engagement I’d like to see in the class. We’ll see if that moves the needle at all.

Of course, the big question on everyone’s mind is “what happens in the fall?” (And beyond, because to think we’ll be back to any sort of “normal” anytime soon is…wishful thinking.) To that end, I’m already thinking ahead to what the elective I’m teaching, Computer Networks, should look like. How do I replicate the hands-on, exploration-heavy nature of that course if we’re partially or completely online? How do I best engage students in such a complex subject under still-unfamiliar-to-us learning conditions? I’ve already decided to forgo my usual tried-and-true textbook in favor of a freely-available, open-sourced, online textbook, which, because of the way it’s organized, forces me to radically redesign the course. I’ll have to think hard about what’s really fundamental content, and be comfortable with scrapping the rest. And I’m excited to try out a set of assignments that I first heard about at SIGCSE a few years back, which never quite fit into the way I taught the course — but is a much better fit with this other textbook.

As the term winds down, and as we head exhaustedly into summer, I’ll continue to look for ways to keep engagement — of those around me and of myself — alive and sustained. I’ll continue to encourage self-care to those around me — and to remind myself to do so, too. And I’ll do whatever I can to end this term on as much of a high note as possible.

Week 4: Exhaustion and energy levels

We’re limping to the end of Week 4 of Completely Virtual Spring Term, and everyone is exhausted.

Like most weeks, I’ve spent the better part of this week in Zoom meetings of various types, and in each one energy levels were noticeably low. My learning community, which meets every other week and is normally fairly engaged, was noticeably more subdued and resigned this week. I’m pretty sure one of my students in my synchronous class meeting yesterday, at one point, put their head down on whatever flat surface serves as a desk as I was talking. I’m running a meeting later today, and I’m already assuming that it will be similarly low energy.

I think there are several reasons for the exhaustion we’re all clearly feeling, besides whatever’s going on in our personal lives. The novelty of learning online is gone. There’s a ton of uncertainty about this summer (will we have summer research? when will a decision come down?) and next year (is it even possible to be back on campus in the fall?). We can’t plan with any certainty. The administration holds off on decision making so they can weigh the many factors and integrate new information as it comes in, which makes sense. At the same time, we all just want to know what’s going to happen so that we can prepare, which also makes sense. It’s like Waiting for Godot, except Godot is a deadly virus we don’t fully understand.

I suspected at the end of last week that my students were starting to drag, so I eased up a bit this week. No recorded lectures except for the Sunday one where I review the previous week and preview the current week. Time built in to work on their project proposals, so only 2 readings for the week. In yesterday’s synchronous class meeting, I straight-up lectured with a teeny bit of interactivity, which I rarely do — but there were a lot of questions related to the reading, and I think my students have a bit of teamwork fatigue, so it seemed like the right decision. They need to complete 2 labs, but they’re structured so that students can work on parts of them here and there, and have the option to work completely solo or alongside their teammates/classmates.

Normally, Midterm Break occurs the Monday of 6th week, but this term it’s this coming Monday (of 5th week), since the term is only 9 weeks instead of 10. We all clearly need this break at this point. I’ve been reflecting on changes I can make to the flow of the course for the second half. I’ll poll my students tomorrow to check in with how much time they’re spending on various activities, and what technical (and personal issues) continue to interfere with their learning. My kiddos’ school district declared a long weekend this weekend, so they have Friday and Monday off. We rarely all have off on my Midterm Break, so I’m thinking about what kind of safe, socially distanced adventures we could do as a family that day, just to do something different. (Note to self: check if state park pass is still valid!)

In the meantime, I’m experimenting with ways to increase engagement in my classes and the meetings I run. I plan to break the participants in today’s STEM Board meeting into smaller discussion groups in the hopes that people will feel more comfortable sharing ideas with fewer “face boxes” than on a screen of many “face boxes”. I held my first Q&A Friday class session last week, and ended up working through examples with students in a much smaller group. Even with the limitations of Zoom, that felt the closest to “real” teaching that I’ve done this term, and I ended the session rejuvenated instead of exhausted for a change. I tagged specific questions (and the students who posted them) in Slack that I promised to get to in Wednesday’s class, and I used chat more heavily than usual. Even though everyone could see the chat, I made sure to read/summarize contributions and attribute them to the students who made them, as a way of affirming their participation and their ideas. A few students have started privately messaging me questions/comments via chat during class, and I want to encourage that as an option for those too shy to participate in public. Engagement still falls short of what I’d like, but I’m taking baby steps to get it closer to that ideal.

Hopefully this weekend serves as a vital reset for all of us, and we come back ready to tackle whatever the second half of the term holds in store for us.

Week 3: Reality and routines

We’re now 3 weeks in to our virtual spring term.

This is the point where we should be in some sort of rhythm. Students know us better, we know our students better. The students know the expectations: what happens in a class period, how should I prepare for class, what’s due when. The faculty understand the class dynamics and manage class periods and expectations accordingly. Everything should be humming along.

But that’s in a normal term.

I do think we’ve reached an imperfect rhythm. Faculty at this point have pretty much figured out what technologies to use and what to let go — although many of us are still learning how to use them effectively.* Course structure is pretty much set. I’ve developed my own rhythm for preparing and posting materials for each week, and settled on a Moodle structure that seems to work ok.

I’m still trying to develop rhythms around grading and keeping track of student engagement. I thought I’d found a good structure for grading — and then my course staff student found a number of flaws and technical issues with that structure. Technical issues aside, I’m still trying to wrap my head around what should I be evaluating vs. what do I feel comfortable having my course staff student grade. (Also taking into account what they feel comfortable grading!)

Student engagement is trickier to keep on top of. Sure, Moodle lets me check student completion (and download this into a spreadsheet), and sure, I can probably check my Zoom logs to figure out who attended a class and who did not. But these feel unnatural. In a normal term, I pay attention to who’s attending and not attending class, which I can do by quickly scanning the room. I note who dominates small group discussions and who’s clearly tuning out or shutting out. I rely so heavily on visual, physical cues. Honestly, trading those instincts and skills for reviewing logs is exhausting, even though it seems like it should be much easier to measure engagement quantitatively than qualitatively.

Course rhythms aside, the constant uncertainty is clearly starting to wear on us collectively. Everyone seems a bit more fragile and frayed, quicker to jump to conclusions or to snap. It’s hard to focus when there’s so much up in the air.

This is the time of year where we should be finalizing plans for summer research, where planning for next fall ramps up. In our department, we’re still trying to hire a visitor for next year. I’m in charge of next year’s Comps (senior capstone projects), and we’re drafting project descriptions and reviewing each others’ drafts, in advance of the big topic reveal on May 7.

But we still don’t know whether we’ll have summer research this year, or if so what that looks like. And we have no idea what fall term looks like, either. So every plan we make needs at least one contingency plan, if not several. Will this Comps project work if we’re still virtual this fall? What does “mentoring new faculty” look like when everything is virtual? If we’re on campus but socially distancing (if that’s even possible, which I doubt it is), and we have to severely limit the number of students in a physical classroom, how badly does this torpedo our carefully-planned schedule? How do we ensure that our current majors (and the 50-some new majors that just declared) get into the classes they need to graduate if class sizes are severely limited?

I don’t want to end this post on a low note, and while things are challenging I also don’t want to leave you with the impression that everything is doom and gloom. So let me share a couple of positives from this week:

  1. My 7th grader and I, pre-pandemic, had a standing coffee and bagels date every Friday morning. We’d go to a local coffee shop near her school and I’d drop her off at school afterwards. We’ve found a way to continue that — every Friday morning we drive to an awesome local coffee shop in the next town over, go through the drive-through, and then sip our drinks as we drive home. It’s been a nice little grounding ritual for both of us.
  2. I’ve started including a bonus “hey, if you click through to this optional resource and watch/read it, respond to this set of bonus questions on Slack” in one lecture or activity per week. Not many students share out on Slack, but the ones that do post thoughtful and insightful reflections. This week’s option involved Microsoft Bob, and I appreciated the way they analyzed what Bob did well and what it got wrong. Reading these bonus reflections always makes me smile.

Wherever you’re reading this, and wherever you are in your term or semester or life, I hope you are staying safe and healthy, and that you’ve found a routine of sorts that works for you.


*Yesterday I learned — in real time — what Zoom co-hosts can and cannot do. Hilarity sometimes ensued.

Week 2: Managing expectations

We’re now in Week 2 of this bizarre and unasked-for experiment in online education. While I won’t say that I’ve settled into any comfortable pattern yet, I have settled into an uneasy acceptance. I am, as Steve Krug discusses in my course’s readings for the week, “muddling through” with the tools and the structure.

Last Friday, the college announced officially that we were not coming back to complete Spring Term on campus in May, that the entire term would be virtual, and canceling both Commencement and Reunion in June. Not a surprise, to be sure. I did not, however, expect the profound feelings of loss and sadness the announcement brought. I expect that it’s the finality imparted by the decision, the squashing of that last (and totally unrealistic) glimmer of hope that things would return to “normal” soon. This announcement is just another indicator that we’re not returning to any sort of “normal” anytime soon — not in a few weeks, and probably not in a few months. “Normal” changed overnight, and no one knows what “normal” is supposed to connote anymore.

This announcement framed Week 2 for me, and my Week 2 goals for my course:

  • Acknowledge the loss we’re all feeling (particularly the seniors)
  • Set and reinforce the weekly course rhythms
  • Develop workable and sustainable ways of “working together apart”

Acknowledge the loss we’re all feeling

Our amazing Instructional Continuity team (the director of our learning and teaching center, the academic technologists, and various IT staff) suggested that professors create a short video at the start of each week, summarizing the previous week and previewing/framing the content for the upcoming week. I often did this in my pre-pandemic Monday class meetings, so I eagerly adopted this suggestion. Doing this sort of review/preview is especially important in a course like Software Design, which can sometimes feel like “a loose collection of things that didn’t fit anywhere else in the curriculum”. I like to remind my students of how all of the threads weave together to form a whole.

I realized that it’s also helpful for me to acknowledge what’s happening outside of the course — reiterating that we can’t untangle ourselves from the context in which we’re learning, and that these are weird and hard times. This week, I started my video recognizing the sadness and loss we’re likely all feeling from the official announcement, and particularly the seniors. I ended my video acknowledging that some students in the class were struggling with much bigger issues, like the health of family members, increased family obligations, or other forms of uncertainty. I reaffirmed that while the course itself has particular learning objectives, my objective as a professor is to teach and lead with compassion above all else during this time.

Set and reinforce weekly course rhythms

Week 1 in Software Design is a bit odd, in that I cover 4 distinct topics in 3 days: framing the larger course questions; the qualities of effective teams and teamwork; a very quick overview of Agile development; and an introduction to git and GitHub. This term, the distinct topics actually worked in my favor. I could experiment with different tools and structures for each topic without worrying too much about how they all “flowed” together, because a normal Week 1 in this course doesn’t “flow” perfectly anyway.

Most of the remaining weeks in the course cover one, or two very tightly related, topics, and thus lend themselves better to a predictable structure. This week is one such week: we cover web usability and accessibility, with a first taste of ethics thrown in for good measure. This week also, the concepts are easily “chunkable” into mini-lecture/example and corresponding activity. And there’s a clear deliverable at the end of the week, the first big assignment of the term.

I currently struggle with how to time the asynchronous activity completion dates. I want to respect the need for my students to work when they can, which might not always be perfectly timed with our MWF schedule and may be disrupted at any time by technology issues or family expectations. This is particularly important because the asynchronous activities often have a team component to them, so I want to make sure teams have time to collaborate on the activity. But I also want to make sure completion dates are spaced out appropriately and that the students have certain things completed, if at all possible, by our synchronous class meeting.

To try and address this, I’ve used Activity Completion on Moodle more liberally this week. Not so much as a mandate, but more as a signal of “these are my expectations, and this is roughly the order in which I expect you to complete them”. I do “lock” some activities into a particular order — complete the reading quiz with a passing grade before you can access the lectures and activities related to that quiz and readings — but many of the activities just require the students to manually check off that they completed them.

Honestly, though, the biggest challenge for me is making sure I have the material ready to go for the start of the next week by the end of the previous week! I … did not succeed on this front this week. But hey, I’m learning.

Develop workable, sustainable ways of “working together apart”

I still struggle with the central question of how to create the collaborative, small group spirit I work so hard to foster in this course in a completely virtual environment. I placed the students into stable, 4-person teams and set up asynchronous activities so that they at least confer with each other when completing the activity. I use these teams during breakouts during our synchronous class meetings, too. (The technology failed me a bit on this front last week, but I think I resolved the issue.)

Beyond this, I want to re-create the between-teams sharing of ideas that often happens in a physical class meeting, or as teams work in our computer labs. I also want to make sure that students know that they can ask me questions just like they’d do in a physical classroom, or by popping into my office when my door’s open — basically, that they don’t and shouldn’t struggle alone.

I’m using Slack heavily for these last purposes. I set up a #coursework channel specifically for students to ask/answer questions or post comments as they work through the asynchronous activities. I am on Slack a lot anyway, but I told my students that I will definitely hang out on Slack during our scheduled meeting times, so if they post then they can expect to get an answer right away. I’ve also tried to be more responsive “outside hours” than I’d normally be, because I don’t want students to feel lost or discouraged.

That said, I am setting boundaries, too! But I’m still trying to strike that balance between being “available enough” and “too available”.

I also haven’t yet been successful at selling open office hours as “time available to work simultaneously with your peers with me available to answer questions in real time”, so I’m thinking of ways to modify the office hours model to make sense for me and for my students.


Beyond this, I continue to struggle with things that I suspect will be issues the entire term: the fatigue of being always-online and interacting in small Zoom windows for a good part of the day; the fact that everything related to virtual teaching takes 10 times longer than face-to-face teaching; the constant worry that students will fall through the cracks; the worry that we will lose members of the Carleton community to COVID-19. And, of course, managing my own expectations, for myself, continues to be a struggle — but I suppose that’s true of non-virtual terms as well.

Course planning in the time of COVID-19

Usually at this time of year, I start to regret my decision to work at a non-semester school. While most of my colleagues are starting to wrap up their spring semesters, I’m submitting final grades for Winter Term and frantically prepping for the start of Spring Term.

This year, of course, everything is different. Now, while most of my colleagues are struggling to figure out how to finish their spring semesters online, I’m struggling to figure out how to move an entire course online.

For the record, I think both situations are equally challenging. In the former case: how do you pivot the way you’ve been doing, well, everything for most of a semester, and finish out the course in a completely different environment from where it began? In the latter case: How do you take a 10 week in-person course, remove a week completely, and take it completely online, with just 2 weeks to prepare?

And, to throw another wrench in the works: what if only slightly above half of your students have “reliable, high-speed Internet access” at home or whatever place counts as home right now?

There’s a whole bunch of other things to consider, too.

  • What’s the minimum tech configuration I can assume my students’ computers have? What minimum tech configuration is fair to assume, for them and for me?
  • What do I do when a student’s technology can’t meet some minimum I’ve deemed necessary to be able to complete the coursework?
  • How do I take a course heavily centered around teamwork and team projects and move that online? Particularly when students are in different timezones and may not have the best Internet connectivity?
  • What will the mental state of my students, and my own mental state, be when the course starts and as the term progresses? How can I compassionately account for this while designing and delivering my course?

My thinking and planning continues to evolve, and there’s a lot I’m still trying desperately to figure out. But here’s where my thinking is right now.

Structure

  • Each week has a theme. This is similar to what I do now, anyway.
  • Each week is structured around 3 “days”, or topics. I’ll label these Day 1, Day 2, and Day 3, to signal to students my expectations that they’ll engage with the material 3 times a week, but that there’s flexibility as to when they engage.
  • 2 of the 3 days consist of completely asynchronous activities. This should help with the inconsistent Internet access issues, to some extent, as well as with some of the timezone issues.
  • These asynchronous activities will include short video lectures (powerpoint w/ audio, me sketching on a whiteboard, me demonstrating a technique or fleshing out a concept), followed by small group (and some individual) activities. I need to figure out how to collect something from these activities so that I can provide feedback and summarize these reports for the class.
  • The third day will have a short (30 minutes?) synchronous component, structured as a Q&A or a whole-class activity or something like that. This will always be during our class meeting time and always on the same day of the week, for consistency. And it will be recorded for those who can’t attend in person.
  • I will stick with targeted readings before each class, along w/ something they need to turn in to show me they engaged with the readings. This will largely be the same as I do now.

Team/small group engagement

I should explain here that I usually teach this course in an interactive classroom, fitted with small tables (each with its own computer and monitor) and lots of whiteboards. So there is a “table culture” in my course, where students end up interacting heavily with the people at their table in small group activities and discussions. I want to try and re-create that environment online, to the extent that I can. So far, I’m thinking:

  • Stable, small groups (3 students) that will also serve as project teams.
  • I will assign teams the first day of class, and teams will be based on timezones.
  • These teams stay stable for the entire term, unless something goes awry and I need to break up a toxic team.
  • Most of the asynchronous activities will be done in these groups. Sometimes I may combine 2 small groups if I think the activity would benefit by having more participants.
  • Having very small groups that share a timezone should in theory make it easier for groups to meet on their own synchronously. There also may be fewer connectivity glitches if they are trying to connect/coordinate with fewer students, rather than with an entire class.

Things I still have no clue about

  • I’m still working out how to manage the term-long team projects — to do this, I need to first understand what technology my students have. Can I assume, for instance, that they have some way to access a terminal? A text editor? A Python installation? Will something like repl.it work for what they’re doing?
  • Am I better served lopping a bit off of every topic, or of lopping off one or more topics entirely? One model has me cutting out the week+ of ethics material, but that seems to be exactly the wrong approach.
  • What is the correct number of smaller, lower-stakes assessments that will enhance student learning without overwhelming the students — or me?
  • How should I structure office hours? What does office hours look like when it’s completely online?
  • And, of course, since part of my research home is in computer networking, I remain deeply skeptical that any of our networks can actually support the load we’re about to throw at them….

My goal by the end of this week is to have the main structure of the course finalized and fleshed out, so that next week I can start constructing the activities, videos, demonstrations, etc. And hopefully, by the time April 6 rolls around, I’ll be ready to go….for some definition of “ready”.

Navigating the new normal

About an hour after I last posted, Carleton’s president emailed the campus with the announcement we’d all been sort of expecting anyway: Carleton’s moving all instruction online through at least Midterm Break in early May, and likely beyond.

The complete details, as one might imagine, are still very much in flux. Because how could they not be? We’re all in uncharted territory here. But what we do know: Finals end as planned today. Students have an extra day (until Wednesday) to vacate campus. Students can petition to stay if they really can’t leave, and we’ll have services to support them. Spring break’s extended by a week, until April 6, to help faculty and staff reconfigure courses and course support for online learning. Spring term will end on time, and be a week shorter than a normal spring term.

Everyone’s a little lost, and everything feels off right now. My extended Friday office hours were part instruction and part informal therapy session. I invited students to just show up even if they didn’t have questions if they felt unmoored and like they needed to be around others, and a few took me up on that, sitting and working or listening while I answered others’ questions. Some of my senior advisees are scrambling to graduate early, because they can, and I’ve been helping them navigate the ad-hoc accelerated process and think through their options.

We started immediately as a department thinking through some of the practical aspects of moving computer science instruction online, and have already agreed on some common tools to use and/or test drive before April 6. It’s certainly helped that many companies who provide online learning tools are moving to make those free to educators and students during this time. We arrived quickly at the minimum technical configuration our students need to have to participate remotely, which will help ITS plan to provide resources to students who need them. My chair has been an absolute rock star in all of this: attentive to the myriad details while still taking time to make sure we’re all comfortable in our teaching assignments and keeping us talking to each other. And our technical associate’s been working overtime to help us figure out the technical details of things like remote access to servers and other department resources. I feel as though we’re ahead of the curve as far as department preparedness goes.

This week, Grading All The Things and wrapping up winter term are my priorities, as well as making sure my kids, who are now on extended spring break through the end of the month, are not killing each other and/or spending 12 hours a day on screens. I have a ton of administrative tasks that fell off of last week’s to-do list when the announcement hit, that must get done this week. So any planning for spring at this point will be during those down moments when my mind wanders, or perhaps as a break from grading to quell my building anxiety. Next week, planning begins in earnest, and I hope to chronicle my thoughts and plans here as I, and we, navigate this new, strange normal.

How are you, and your institution, navigating your new normal?

Responding to midterm evaluations

I have a confession to make: I have fallen out of the habit of doing midterm course evaluations.

I don’t know why, exactly, I did. I’ve always found midterm evaluations to be useful as a way of taking the pulse of the course. Even if I think I have a pretty good sense of how things are going and what the major complaints are, I still learn useful information from my students. I’ve discovered problems with textbooks, problems with the timing of due dates, and problems with the sight lines in a classroom from midterm evaluations. I’ve learned surprising things about what things my students enjoy and what they value. (Hint: they are not always the same things.) I often use them as an opportunity to check the learning objectives I’ve set for the course, and how well students think we’re meeting those.

They are invaluable in helping me recalibrate my courses.

So I’d fallen out of the habit. But last term, I taught a course I haven’t taught since 2012. And it was a struggle to get back into the rhythm of the material and the pace of the class and the workload and such. So I thought that would be a golden opportunity to bring back the midterm evaluation into my life.

Boy, am I glad I did.

I’d had a sense that the class was going ok, but that something seemed off, and I couldn’t quite place why. The midterm evaluations illuminated the issues very quickly, and also explained why it was so hard for me to figure out the source of the issues. In a nutshell: The students couldn’t make sense of much of anything in the book. My version of flipped classroom pedagogy requires my students to do some targeted reading and complete targeted reading quizzes before class, so that I can see what they got out of the readings and where I need to spend some extra time before we dive into deeper practice with the material, and then deeper still on the problem sets. Since they were lost when reading the book, they were lost when attempting the quizzes. Sometimes they gave up, and sometimes they just kept guessing until they got the right answer. This means their foundation was shaky, and they were not really prepared to engage with the material in class. But the way I had the quizzes set up, it was hard for me to see the depth of the problems. And then, because they couldn’t engage with the material in class at the level I’d expect based on what I saw in the reading quizzes, they weren’t prepared to engage with the problem sets — which are challenging.

A perfect storm of chaos, basically.

I’ll admit that it was hard to read this set of evaluations. Mostly, I felt like I had let my class down, and berated myself for being a “bad professor”. Which is human but not really constructive. So I sat with the comments for a few days, planning how to address them. Clearly my course needed a restructuring, one that would work around the shortcomings of the textbook while still remaining true to my pedagogy. But I wasn’t sure how to do this.

As my class took an exam in the first half of the following Monday’s class, I finally figured out what I wanted to do, and sketched out a plan on slides. When the last student handed in the exam, I said we were going to talk about the evaluations. And I summarized their concerns. And outlined the “perfect storm” I described above, and how I thought this was affecting their learning. How finding a good textbook for this type of class at this level is difficult, and how I’ve successfully used this book for many years, but that’s irrelevant because what mattered most is that they struggled with it so that means it was not a good fit for this class, and how I will take this into account the next time I teach this class and have to pick a textbook.

And then I outlined a plan for the rest of the term, one that I hoped allowed them to attempt the readings more successfully and with a bit more focus (reminding them that I do give them hints on what to focus on, through a set of reading questions in addition to the quizzes). We would start each class explicitly with the key points from the readings listed on the board along with my impression of what concepts they found most challenging, giving them the opportunity to agree/disagree with my assessment. This might mean that I had to change my planned examples and exercises on the fly, which made me a bit nervous since I was still relearning the material along with them — but I promised to be honest with them when I felt like my free-wheeling might lead us into rough waters, and they seemed to be ok with that. That I’d dedicate more class time to giving them the opportunity to start/work on problem sets, so that they could ask questions right away and we could figure out together if they need some extra practice before attempting the problem sets.

I am grateful that I work at an institution where students feel safe enough to be honest with me about what is and is not working for their learning. I am grateful for the level of thoughtfulness and self-reflection my students bring to the learning process. Students here place high expectations on faculty and expect a lot from us (which they should!), and their feedback and observations make me a much better teacher. Responding to this feedback, and restructuring my course, proved an interesting challenge and a good opportunity for me to reflect on my pedagogy — not just my philosophy, but my practice.

I don’t think I completely hit the mark on everything I intended to do last term, but I do think things improved, for both me and the students. And, as I put together my course for spring term, I realized that I could and should incorporate some of these same practices (extra checking in with the students, being explicit about the key points from the readings) into that course as well. We’re only a week and a half into the new term, but I’m excited to see how things go. And, I’m already looking forward to what this batch of students will share on this term’s midterm evaluations.

Introducing CS 1 students to algorithmic bias via the Ethical Engine lab

There’s a lot of recent interest around the ethics of technology. From recent popular press books like Algorithms of OppressionAutomating Inequality, and Technically Wrong*, to news stories about algorithmic bias, it seems like everyone is grappling with the ethical impacts of technology. In the computer science education community, we’re having our own discussions (and have been for some time, although there seems to be an uptick in interest there) on where ethics “belongs” in the curriculum, and how we can incorporate ethics across the curriculum — including in introductory courses.

One initiative aimed at touching on ethical issues in CS 1 particularly caught my attention. In July 2017, Evan Peck, at Bucknell University, posted about a programming project he and Gabbi LaBorwit developed based on MIT’s Moral Machine, a reworking of the classic Trolley Problem for self-driving cars. This project, the Ethical Engine, had students design and implement an algorithm for the “brains” of a self-driving car, specifically how the car would react if it could only save its passengers or the pedestrians in the car’s way. After implementing and testing their own algorithms, students audited the algorithms other students in the class designed.

Justin Li at Occidental College built upon this lab, making some changes to the code and formalizing the reflection questions and analysis. He wrote about his experiences here. In particular, Justin’s edits focused more on student self-reflection, having them compare their algorithm’s decisions against their manual decisions and reflecting to what extent their algorithm’s decisions reflected or did not reflect their priorities.

I was intrigued by the idea of this lab, and Justin’s version seemed like it would fit well with Carleton students and with my learning goals for my intro course. I decided to integrate it into my fall term section of intro CS.

Like Evan and Justin, I’ve made my code and lab writeup freely available on GitHub. Here are links to all three code repositories:

Framework

Based on Justin’s and Evan’s writeups, I made several modifications to the code.

  • In the Person class, I added “nonbinary” as a third gender option. I went back and forth for a bit on how I wanted to phrase this option, and whether “nonbinary” captured enough of the nuance without getting us into the weeds, but ultimately decided this would be appropriate enough.
  • Also in the Person class, I removed “homeless” and “criminal” as occupations, since they didn’t really fit in that category, and made them boolean attributes, similar to “pregnant”. Any human could be homeless, but only adults could have the “criminal” attribute associated with them.
  • In the Scenario class, I removed the “crossing is illegal” and “pedestrians are in your lane” messages from the screen output, since in this version of the code these things are always true.

I also made it a bit clearer in the code where the students should make changes and add their implementation of the decision making algorithm they designed.

Execution

I scheduled the lab during Week 8 of our 10 week course, just after completing our unit on writing classes. We take a modified “objects-early” approach at Carleton in CS 1, meaning students use objects of predefined classes starting almost immediately, and learn to write their own classes later in the term. The lab mainly required students to utilize classes written by others, accessing the data and calling upon the methods in these classes, which conceivably they could have done earlier in the term. However, I found that slotting the lab in at this point in the term meant that students had a deeper understanding of the structure of the Person and Scenario classes, and could engage with the classes on a deeper level.

I spread the lab over two class periods, which seemed appropriate in terms of lab length. (In fact, one of the class periods was shortened because I gave a quiz that day, and the majority of the students had not finished the lab by the end of class, which leads me to believe that 2 whole class meeting periods at Carleton, or 140 minutes, would be appropriate for this lab.) As they do in all our class activities, students worked in assigned pairs using pair programming.

On the first day, students made their manual choices and designed their algorithm on paper. To ensure they did this without starting with the code, I required them to show their paper design to either my prefect (course TA) or myself. A few pairs were able to start implementing the code at the end of Day 1. On the second day, students implemented and tested their algorithms, and started working through the lab questions for their writeups. Most groups did not complete the lab in class and had to finish it on their own outside of class.

At the end of the first day, students submitted their manual log files. To complete the lab, students submitted their algorithm implementation, the manual and automatic logs, and a lab writeup.

Observations

Unexpectedly, students struggled the most with figuring out how to access the attributes of individual passengers and pedestrians. I quickly realized this is because I instruct students to access instance variables using accessor and mutator methods, but the code I gave them did not contain accessor/mutator methods. This is a change I plan to make in the code before I use this lab again. I also plan to look a bit more closely at the description of the Person and Scenario classes in the lab, since students sometimes got confused about which attributes belonged to Scenarios and which belonged to Persons.

Students exhibited a clear bias towards younger people, often coding this into their algorithms explicitly. One pair mentioned that while their algorithm explicitly favored younger people over the elderly, in their manual decisions they did “think of our grandmas”, which led to differences in their manual and automatic decisions in some places. A fair number of students in this class came from cultures where elders traditionally hold higher status than in the US, so the fact that this bias appeared so strongly surprised me somewhat. Pregnant women also got a boost in many students’ algorithms, which then had the effect of overfavoring women in the decisions — which many students noted in their writeups. While nearly all pairs explicitly favored humans over pets, a few pairs did give a small boost to dogs over cats, while no one gave any boost to cats. I’m not sure why this class was so biased against cats.

I was impressed by the thoughtfulness and nuance in many of the lab writeups. Most students were able to identify unexpected biases and reason appropriately about them. Many thoughtfully weighed in on differences in their algorithm’s choices versus the choices of their classmates’ algorithms, one pair even going so far as to reason about which type of self-driving car would be more marketable.

In the reflection question about the challenges of programming ethical self-driving cars, many students got hung up on the feasibility of a car “knowing” your gender, age, profession, etc, not to mention the same characteristics of random pedestrians, and being able to utilize these to make a split-second decision about whom to save. This is a fair point, and in the future I’ll do a better job framing this (although to be honest I’m not 100% sure what this will end up looking like).

One of the lab questions asked students to reflect on whether the use of attributes in the decision process is ethical, moral, or fair. Two separate pairs pointed out that the selection of attributes can make the decision fair, but not ethical; one pair pointed out the converse, that a decision could be ethical but not necessarily fair. I was impressed to see this recognition in student answers. Students who favored and used simpler decision making processes also provided some interesting thoughts about the limitations of both “simpler is better” and more nuanced decision-making processes, both of which may show unexpected bias in different ways.

Conclusions and takeaway points

Ten weeks is a very limited time for a course, so for any activity I add or contemplate in any course I teach, I weigh whether the learning outcomes are worth the time spent on the activity. In this case, they are. From a course concept perspective, the lab gave the students additional practice utilizing objects and developing and testing algorithms, using a real-world problem as context. This alone is worth the time spent. But the addition of the ethical analysis portion was also completely worth it. While I have yet to read my evaluations for the course, students informally commented during and after the exercise that they found the lab interesting and thought-provoking, and that it challenged their thinking in ways they did not expect going into an intro course. I worried a bit about students not taking the exercise seriously, and while I think that was true in a few cases, by and large the students engaged seriously with the lab and in discussions with their classmates.

I teach intro again in spring term, and I’m eager to try this lab again. The lab has already sparked some interest among my colleagues, and I’m hoping we can experiment with using this lab more broadly in our intro course sections, as a way to introduce ethics in computing early in our curriculum.

*all of which are excellent books, which you should definitely read if you haven’t done so already!