Fail Fast Textbook Selection

I’m strongly considering changing my reading material for my software engineering class. I like Code Complete in a lot of ways, but it really does feel a bit old and a lot of bits are not super well organised or presented. And it’s big without being super nicely modularised. It’s not really a textbook. I’m planning a pretty significant reworking of the course (to consolidate some stuff) so this is the time to change.

I was looking at a text that has some good reviews and decent presence on Open Syllabus and a metric ton of supporting material. I’ve never used supporting material but one can see the attraction!

I’m skimming a copy starting with the intro. The intro of text tends to be a really weak bit especially if it’s didactic instead of tutorialesque, so I forgave the cutesy intro dialogue. There were some helpful fake graphs about characteristic error rates which seemed fun. Then I hit the following description in a list of “kinds of software”:

Artificial intelligence software—makes use of nonnumerical algorithms to solve complex problems that are not amenable to computation or straightforward analysis. Applications within this area include robotics, expert systems, pattern recognition (image and voice), artificial neural networks, theorem proving, and game playing.

Say what? “Nonnumerical algorithms”?!?!?! Right before talking about artificial neural networks??!?! Maaaaybe there’s a specialised enough variant of “numerical algorithm” (written primarily in Fortran?!?) where this is technically not wildly false, but it sure the hell is misleading here (given the standard distinction between symbolic and non-symbolic AI). Seriously bonkers.

But…ok. Does one extremely boneheaded bit of a sort of throwaway warrant tossing the whole thing? Maybe? I want to be fair. I can always guard against this in lecture…I guess. Then I hit:

  • Software has become deeply embedded in virtually every aspect of our lives, and as a consequence, the number of people who have an interest in the features and functions provided by a specific application has grown dramatically. When a new application or embedded system is to be built, many voices must be heard. And it sometimes seems that each of them has a slightly different idea of what software features and functions should be delivered. It follows that a concerted effort should be made to understand the problem before a software solution is developed.

Oy. I mean, the dude has chapters on iterative and agile processes, so there’s some course correction. But. Come. ON!

Ok, now I dump it from the list.

Advertisements

Error (and Other Condition) Handling

Two big challenges of programming are 1) resource management and 2) variant condition handling. Traditionally, we’ve focused on the specific variants of 1′) memory management and 2′) error handling because they are omnipresent cases. One nice thing in the recent renaissance on resource management is that the focus has generalised a lot to all sorts of resources (eg file descriptors). Less so in error handling though most exception systems actually handle all sorts of conditions and, given their pervasiveness in modern object oriented languages, often have interesting specialisations of their exceptions.

I estimate that there are three major eras of resource management: manual (only), garbage collection dominance, and the emerging RAII/scope based approaches. Rust (following C++) are the current champions though I doubt GC is going anywhere. (And I’m eliding virtual memory, memory mapped files, and so on.)

We currently are seeing a fair bit of language level experimentation with error handling. Consider that neither Go nor Rust have exceptions. One interesting place where the two managements bridge is Null vs option types.

While the dangers of null are well known (the worst mistake in computer science), I’m not yet sold on option types or error code checking in general.

The main grand challenge of memory management is how to make memory seem infinite and uniformly accessible while 1) being super efficient and 2) not pestering the programmer too much.

The main grand challenge of error handling is how to ensure that all errors (or variant conditions) are handled without 1) making the main line of the program obscure, 2) destroying the ability of the programmer to reason able the program, and 3) making the program robust. The big problem is that all solutions involve a ton of clutter. Exceptions try to reduce some of the clutter at the cost of making control flow extremely non local.

Part of the problem is that there’s so many types of error and error-situation combos and so many ways of handling them and handling them “appropriately” that it’s genuinely difficult to get a basic much less systematic handle on things.

I don’t have any firm ideas where to go with all this. I have an intuition that tooling could help a lot. One simple thing would be to see what errors are known to be possible at a given bit of code. Java’s checked exceptions, I think, are meant to help with that but it’s too much manual crap and too easy to escape. We want exception inference.

Similarly, we need ways to see all the sites and sources and handling of a condition. In general, lots of conditions are cross cutting and handing a hierarchy of throws is just super confusion. We want to cluster a throw by similarity of situation and appropriate handling.

I’m not sure any linear/lexical approach is going to work. It’s intimately related to testing and we need modular ways to manage t

Another 61511 Done

Four years ago, I made a push to change how we teach software engineering at the MSc level. I had ambitious plans about how to change the whole sequence, but I was going to start by taking over the first class in the sequence.

The first year was super tough as the person who has been teaching it took medical retirement (sadly). My early ideas just weren’t workable given me and the cohort.

I completely revamped it, esp the coursework and have edge closer and closer to something which has some good innovation. It needs another overhaul but this year went pretty smoothly (still some coursework marking to go).

I won’t said it’s best of breed because I don’t have a lot of comparisons. But it seems good and rather interesting. One class out of four isn’t enough to be transformative but it’s a start!

Plus I taught the whole day with a unicorn horn on my head. Good times!

Grading Postmortem

I just finished the followup of grading a programming/software engineering assignment with a mostly automated toolkit. The goal was to have “next class” turnaround. It wasn’t quite same day but it was definitely within 20 hours, for the bulk of people. Some of the problem was that Blackboard is terrible. Just terrible. It refused to upload marks and feedback for people who had had multiple submissions and then sent me into a hellscape to try to enter them manually. So there were some upload errors (2 people didn’t have any feedback and a 1 had the wrong feedback due to cut and paste fail). Out of 49 submissions, I had 17 people report a problem or request a regrade. Of those 5 resulted in a change of mark for a total of 19 marks added (the total possible for each was 10 per assignment so 490 total ; 170 were originally given thus 10% of “rightful” marks went missing and needed a manual update; one of these was due to a rouge extra submission after the deadline that was the wrong one to grade for 3 points, so 8.5 missing marks were due to grader bugs).

Now the people with wrong marks generally got “0” often when it was obvious that they shouldn’t have. This was because their program would either crash in a new way or return a really unexpected result. In the later case, since we try to parse the program out put, we’d through an expected exception for that odd output. In both scenarios, this unexpected scenario would crash the grader before it wrote any feedback. Missing feedback was inferred to be an “upload” problem so the students got 0 and an unhelpful error message.

These were stupidly hard bugs to track down! But they point to a couple of holes in our robustness and test isolation approach (we’re generally pretty good on that). In general, I’d like to review the 0s before uploading to confirm but the tight time frame was just too much. It was a tradeoff between the real anxiety, pain, and confusions some students would feel at getting an erroneous 0, and delaying feedback. It’d have been great if I could have turned around the corrections more quickly, but I have only so much time and energy. All students who filed an issue got a resolution by the subsequent Monday evening at the latest. So, two full days with correct feedback before the next assignment. Obviously, quicker is always better, but this isn’t unreasonable.

At least two people were misled by the feedback which basically said “You are missing this file” when it should have said “You are missing at least one of this file or that directory.” Oops! That was mostly work for me than anything else.

In the same day lab, the students did an over the shoulder code review of each other’s first assignment. I wish I had gathered stats on problems found. I told everyone who wanted to file an issue to send me an email aftertheir code review discovered no problems and they had some simple test cases passing. In many of those cases, there were very obvious problems that a simple sanity test would have revealed and oddities in the code which lept out (to me).

I feel this justifies my decision not to return granular feedback or explicit tests. The program is very small and they have an oracle to test against (they are reverse engineering a small unix utility). The points awarded are few and  2 come from basically not messing up the submission format. 1 comes from following the spec requirement to use tabs as an output separator.

But the goal of these assignments is to get people thinking about software engineering, not programming per se. They need to reflect on their testing and release process and try to improve them. I had several students ask for detailed feedback so they would lose fewer marks on the next assignment and that’s precisely what I don’t want to do. The learning I’m trying to invoke isn’t “getting this program to pass my tests” but “becoming better at software engineering esp testing and problem solving and spec reading and…”.

It’s difficult, of course, for students to care about the real goals instead of the proxy rewards. That’s just being a person! All I can do is try to set up the proxy rewards and the rest of my teaching so as to promote the real goal as much as possible.

Giving students low marks drives a lot of anxiety and upset on my part. I hate it. I hate it because of their obvious suffering. I hate it because it can provoke angry reactions against me. I hate it because I love seeing people succeed.

But it seems necessary to achieve real learning. At least, I don’t see other ways that are as broadly effective.

Some Good Reads

I really am trying to clean out my tabs and writing something in depth on each isn’t always cutting it. So here’s some quick hits (the link text generally isn’t the title:

  • Burying NoSQL for consistency failures. Essentially the argument is giving up consistency for availability (cf CAP theorem) is a bad move due to increased application complexity AND that many “NewSQL” systems aren’t consistent for a subtle implementation reason.
  • A beautiful performance study of grep tools by the author of ripgrep. Clear, fairly comprehensive, appropriately modest, it seems totally publishable to me. I learned a lot reading it and enjoyed doing so.
  • “Systems programming” != “low level programming” has a nice history of the term and concept. It’d be good to get an analysis of how the phrase “programming in the large” got in.
  • You should read all of Dan Luu, but you’d could do worse than starting with his “Hardware is Unforgiving”.
Four Down, hundreds to go.

Gitlab Has Kanban/Trello Style Boards

And they are linked to the issue tracker! Nice!

They aren’t as nice as Trello’s. The cards are very limited and don’t “flip over”. They don’t provide full access to the issue tracker, so adding comments, even adding full fledged issues, is hard to impossible from the board. However, I think for managing a workflow, it’s fine. A little clunky, but fine.

So now I can teach them in my software engineering class…which means I need to add them to my material…yay?

It’s panic time around here! Classes are….sooooo close!

CSS Misalignment

Well, here we are in day n of trying to add a simple logo to a showoff presentation. Showoff has a very neat feature set (esp for audience interaction) but is pretty garbage to dork with.  I mean, most HTML slideshow systems are, but showoff is screwing me pretty hard.

My current solution is to add the image to the content Markdown. That at least gets me somewhere even if I have to preprocess the Markdown. BUT, the image is aligned centre and I need it on the left. Now usually getting things to align centre is challenging for CSS with garbage like “margin-left:auto; margin-right:auto” being actual canonical moves (unless you are dealing with text or floating an image). Of course, I can’t use something like “align: left” but am off in some rathole of position, float, margin nonsense.

CSS has been around in some form since 1994. 1994. They’ve been deprecating HTML presentation stuff for quite some time now. But it’s just plain worse.

Now I’m sure if I spent enough time really learning all this shit, I would have some reasonable control. But…I don’t want to have to learn all this shit just to do some basic layout and I shouldn’t have too. LaTeX I sort of forgive just for it’s shear age, but 1994! With supposed active development since!

It makes me want to dork with transparent 1 pixel gifs.