A Week of Blogging Fail

I went a whole week without blogging. BOO! I was commenting a bit more on LGM but really illness and class knocked me out. And then it snowballed.

I’ll try to post two a day for this week. But it might be slapdash.

My anxiety was brutal last week as I had to prepare a new class with material I inherited which was challenging to interpret much less prep.

Plus figure out how the changes affected the coursework. Yeek.

Per usual, it worked out. Per usual, it cost me an all nighter.

The lecture itself felt really really good. I’m working on whole class participation and I think it was working ok. I did a retrospective of the prior week first as outline and then again as slide snippets. I was able to emphasize how they should be pulling things to get in their heads. Then we applied it to the new material.

I came out tired and happy. And with a brutal sore throat. I also manage to spill not so hot tea over me and the desk. Yay.

Advertisements

Two Months of Nearly Daily Blogging to Go

I just might make it.

It’s been touch and go with classes and illness and feeling overwhelmed. I’ve not had the energy to write enough analytical stuff. My backlog there is growing.

I did fix a superfun bug in my exam authoring tool. It is a classic “dynamic typing plus truthiness” bug that gets static typing folks very excited.

I have a Python object that represents true-false questions. It inherits from an abstract question object. So far, fine. It is parsable from and printable to a simple format eg:

Q: This sentence is true!
+ F
Marks: 1
By: BJP

(Yes the sentence is true if you assume it’s true and false if you assume it’s false.)

This is fine until I used the “shuffle” option which reorganises the exam and question options. What I was getting out was:

Q: This sentence is true!
+ T
Marks: 1
By: BJP

This is bad! And has nothing to do with shuffling. The culprit looked like this:

options = '+ T' if self.key else '+ F'

This would have worked if the key was a Boolean. But it was a string: “True” or “False”. Any non empty string is truthy so the else never gets taken.

D’oh!

Either types on variables or no truthiness would have flushed this out. It was damned hard to spot esp when I wrote tests that constructed the object directly and set the key to a Boolean. (Unit testing didn’t work! It needed an integration test at least.)

Oy!

Two months to go…

Another 61511 Done

Four years ago, I made a push to change how we teach software engineering at the MSc level. I had ambitious plans about how to change the whole sequence, but I was going to start by taking over the first class in the sequence.

The first year was super tough as the person who has been teaching it took medical retirement (sadly). My early ideas just weren’t workable given me and the cohort.

I completely revamped it, esp the coursework and have edge closer and closer to something which has some good innovation. It needs another overhaul but this year went pretty smoothly (still some coursework marking to go).

I won’t said it’s best of breed because I don’t have a lot of comparisons. But it seems good and rather interesting. One class out of four isn’t enough to be transformative but it’s a start!

Plus I taught the whole day with a unicorn horn on my head. Good times!

UKIEPC 2018

It went ok! Yay!

We had 11 teams (out of 15 slots) with 32 people. 1 team was made up of PhD students (mine) so we’re ineligible to continue but had fun.

So much was last minute for me…but that didn’t seem to atypical from my observations on the Slack channel. It would have been all pleasure if not for the brutal-for-me timing.

Manchester had a team place in the top 10 (#6!). Cambridge dominated the top 4 slots. We had several teams solve 5 out of 13 and end up in the 30s. I think all teams solved at least 1 and perhaps 2 problems.

If you don’t train, that’s really an accomplishment.

Of course, now I have to organize a trip to the regionals. But not before I get some sleep.

Not An Easier Week

I thought it was going to be an easier week. Buuut…MSc project grading (done!), three exams to write (whoa, those are hugely not done), class prep (Not Done sob), and a programming contest (somewhat advanced)….it’s a bit much.

Sometimes, I can harness this pressure, but usually closer to the wire. Which I want to avoid!

Oh well. Next week is the last week of period 1. Now, I have a class in period 2 to panic over. Yay.

We Win the 2018 SWASA 10 Year Award

What is this award? Well:

The primary criteria will be the number of citations to the paper in ten years. We will use Google Scholar. While citation count will be the primary criteria, the panel will also consider other impact factors for the top-cited papers. If two or more papers are very close, more than one paper may be honored. The determination is done in the Spring of the calendar year for the conference.

That’s pretty cool! It also won best paper at the time. So yay! Matt’s thesis won the BCS Distinguished Dissertation award so Triple Yay!

When Uli and I were supervising Matt, we had plenary meetings. They were enjoyable but it’s difficult enough to get a word in edgewise with either Uli or myself on our own…when we’re bouncing off each other fuggetaboutit.

Grading Postmortem

I just finished the followup of grading a programming/software engineering assignment with a mostly automated toolkit. The goal was to have “next class” turnaround. It wasn’t quite same day but it was definitely within 20 hours, for the bulk of people. Some of the problem was that Blackboard is terrible. Just terrible. It refused to upload marks and feedback for people who had had multiple submissions and then sent me into a hellscape to try to enter them manually. So there were some upload errors (2 people didn’t have any feedback and a 1 had the wrong feedback due to cut and paste fail). Out of 49 submissions, I had 17 people report a problem or request a regrade. Of those 5 resulted in a change of mark for a total of 19 marks added (the total possible for each was 10 per assignment so 490 total ; 170 were originally given thus 10% of “rightful” marks went missing and needed a manual update; one of these was due to a rouge extra submission after the deadline that was the wrong one to grade for 3 points, so 8.5 missing marks were due to grader bugs).

Now the people with wrong marks generally got “0” often when it was obvious that they shouldn’t have. This was because their program would either crash in a new way or return a really unexpected result. In the later case, since we try to parse the program out put, we’d through an expected exception for that odd output. In both scenarios, this unexpected scenario would crash the grader before it wrote any feedback. Missing feedback was inferred to be an “upload” problem so the students got 0 and an unhelpful error message.

These were stupidly hard bugs to track down! But they point to a couple of holes in our robustness and test isolation approach (we’re generally pretty good on that). In general, I’d like to review the 0s before uploading to confirm but the tight time frame was just too much. It was a tradeoff between the real anxiety, pain, and confusions some students would feel at getting an erroneous 0, and delaying feedback. It’d have been great if I could have turned around the corrections more quickly, but I have only so much time and energy. All students who filed an issue got a resolution by the subsequent Monday evening at the latest. So, two full days with correct feedback before the next assignment. Obviously, quicker is always better, but this isn’t unreasonable.

At least two people were misled by the feedback which basically said “You are missing this file” when it should have said “You are missing at least one of this file or that directory.” Oops! That was mostly work for me than anything else.

In the same day lab, the students did an over the shoulder code review of each other’s first assignment. I wish I had gathered stats on problems found. I told everyone who wanted to file an issue to send me an email aftertheir code review discovered no problems and they had some simple test cases passing. In many of those cases, there were very obvious problems that a simple sanity test would have revealed and oddities in the code which lept out (to me).

I feel this justifies my decision not to return granular feedback or explicit tests. The program is very small and they have an oracle to test against (they are reverse engineering a small unix utility). The points awarded are few and  2 come from basically not messing up the submission format. 1 comes from following the spec requirement to use tabs as an output separator.

But the goal of these assignments is to get people thinking about software engineering, not programming per se. They need to reflect on their testing and release process and try to improve them. I had several students ask for detailed feedback so they would lose fewer marks on the next assignment and that’s precisely what I don’t want to do. The learning I’m trying to invoke isn’t “getting this program to pass my tests” but “becoming better at software engineering esp testing and problem solving and spec reading and…”.

It’s difficult, of course, for students to care about the real goals instead of the proxy rewards. That’s just being a person! All I can do is try to set up the proxy rewards and the rest of my teaching so as to promote the real goal as much as possible.

Giving students low marks drives a lot of anxiety and upset on my part. I hate it. I hate it because of their obvious suffering. I hate it because it can provoke angry reactions against me. I hate it because I love seeing people succeed.

But it seems necessary to achieve real learning. At least, I don’t see other ways that are as broadly effective.