Worse is Better and Back Again

Richard Gabriel, 1991

I and just about every designer of Common Lisp and CLOS has had extreme exposure to the MIT/Stanford style of design. The essence of this style can be captured by the phrase the right thing. To such a designer it is important to get all of the following characteristics right:

  • Simplicity — the design must be simple, both in implementation and interface. It is more important for the interface to be simple than the implementation.
  • Correctness — the design must be correct in all observable aspects. Incorrectness is simply not allowed.
  • Consistency — the design must not be inconsistent. A design is allowed to be slightly less simple and less complete to avoid inconsistency. Consistency is as important as correctness.
  • Completeness — the design must cover as many important situations as is practical. All reasonably expected cases must be covered. Simplicity is not allowed to overly reduce completeness.

I believe most people would agree that these are good characteristics. I will call the use of this philosophy of design the MIT approach Common Lisp (with CLOS) and Scheme represent the MIT approach to design and implementation.

The worse-is-better philosophy is only slightly different:

  • Simplicity — the design must be simple, both in implementation and interface. It is more important for the implementation to be simple than the interface. Simplicity is the most important consideration in a design.
  • Correctness — the design must be correct in all observable aspects. It is slightly better to be simple than correct.
  • Consistency — the design must not be overly inconsistent. Consistency can be sacrificed for simplicity in some cases, but it is better to drop those parts of the design that deal with less common circumstances than to introduce either implementational complexity or inconsistency.
  • Completeness — the design must cover as many important situations as is practical. All reasonably expected cases should be covered. Completeness can be sacrificed in favor of any other quality. In fact, completeness must be sacrificed whenever implementation simplicity is jeopardized. Consistency can be sacrificed to achieve completeness if simplicity is retained; especially worthless is consistency of interface.

Early Unix and C are examples of the use of this school of design, and I will call the use of this design strategy the New Jersey approach I have intentionally caricatured the worse-is-better philosophy to convince you that it is obviously a bad philosophy and that the New Jersey approach is a bad approach.

However, I believe that worse-is-better, even in its strawman form, has better survival characteristics than the-right-thing, and that the New Jersey approach when used for software is a better approach than the MIT approach.

Olin Shivers, 1998

* Preamble: 100% and 80% solutions
There’s a problem with tool design in the free software and academic
community. The tool designers are usually people who are building tools for
some larger goal. For example, let’s take the case of someone who wants to do
web hacking in Scheme. His Scheme system doesn’t have a sockets interface, so
he sits down and hacks one up for his particular Scheme implementation. Now,
socket API’s are not what this programmer is interested in; he wants to get on
with things and hack the exciting stuff — his real interest is Web services.
So he does a quick 80% job, which is adequate to get him up and running, and
then he’s on to his orignal goal.

Unfortunately, his quickly-built socket interface isn’t general. It just
covers the bits this particular hacker needed for his applications. So the
next guy that comes along and needs a socket interface can’t use this one.
Not only does it lack coverage, but the deep structure wasn’t thought out well
enough to allow for quality extension. So *he* does his *own* 80%
implementation. Five hackers later, five different, incompatible, ungeneral
implementations had been built. No one can use each others code.

The alternate way systems like this end up going over a cliff is that the
initial 80% system gets patched over and over again by subsequent hackers, and
what results is 80% bandaids and 20% structured code. When systems evolve
organically, it’s unsuprising and unavoidable that what one ends up with is a
horrible design — consider the DOS -> Win95 path.

As an alternative to five hackers doing five 80% solutions of the same
problem, we would be better off if each programmer picked a different task,
and really thought it through — a 100% solution. Then each time a programmer
solved a problem, no one else would have to redo the effort. Of course, it’s
true that 100% solutions are significantly harder to design and build than 80%
solutions. But they have one tremendous labor-savings advantage: you don’t
have to constantly reinvent the wheel. The up-front investment buys you
forward progress; you aren’t trapped endlessly reinventing the same awkward

But here’s what I’d really like: instead of tweaking regexps, you go do your
own 100% design or two. Because I’d like to use them. If everyone does just
one, then that’s all anyone has to do.

Kevlin Henney, 2017:

A common problem in component frameworks, class libraries, foundation services, and other infrastructure code is that many are designed to be general purpose without reference to concrete applications. This leads to a dizzying array of options and possibilities that are often unused or misused — or just not useful.

Generally, developers work on specific systems; specifically, the quest for unbounded generality rarely serves them well (if at all). The best route to generality is through understanding known, specific examples, focusing on their essence to find an essential common solution. Simplicity through experience rather than generality through guesswork.

Speculative generality accumulates baggage that becomes difficult or impossible to shift, thereby adding to the accidental complexity those in development must face in future.

Although many architects value generality, it should not be unconditional. People do not on the whole pay for — or need — generality: they tend to have a specific situation, and it is a solution to that specific situation that has value.

We can find generality and flexibility in trying to deliver specific solutions, but if we weigh anchor and forget the specifics too soon, we end up adrift in a sea of nebulous possibilities, a world of tricky configuration options, overloaded and overburdened parameter lists, long-winded interfaces, and not-quite-right abstractions. In pursuit of arbitrary flexibility, you can often lose valuable properties — whether intended or accidental — of alternative, simpler designs.

Ok, the last one is a bit more…specific…than the first two. But it’s fun to read it in juxtaposition with the first two. One way to try bridge the difference between Henney and Shivers is to not that Shivers is saying that we need more 100% designs and Henney is saying that we need a lot of specific experience to get to a good 100% design. But then the differences becomes stronger…Shivers doesn’t want people to hack up a bunch of 80% solutions while Henney, roughly, thinks we have to have them before we have a hope for a right 100% one.

My heart is with Shivers, but my head is with Henney.

I think I have some readings and an exam question for next year’s class.


Ah, Grading

Lost track of posting and lots of other things due to the whelm being over but not done. Some of the grading is going OK. Exams should be sortable. I hope to be back on posting track tomorrow.

Blackboard Learn 9.x Fail Encore

Last year, our installation of Blackboard could upload grades from a spreadsheet. So you could grade offline! Which is good, because we want to grade off line, esp. programs. But boo! You couldn’t upload feedback so even though the feedback was sitting in a column ready to go, we had to cut and paste it in. BOO!

But then, in spring, a service pack made it possible up load  (and download!) feedback. WOO! This is good! I can grade offline! I can use my tools! I can analyze stuff!

Except I now figure out that if I have multiple question tests, I can’t upload feedback OR MARKS for individuals questions in the test. Which, for something like, oh, I don’t know, a FINAL EXAM is a big deal.

It’s also going to suck for giving feedback. Lots of cutting and pasting in my future.

Software as a service folks of the world, there is a MINIMUM REQUIREMENT on you: Make sure your users can export and import your data. Easily. Very easily. Make it easy, ok? Use freaking XML if you have to. Just make it easy. From day 1. Until day always. For proper bonus points, make sure that simple things can be done simply. But if not that, just make sure we can do it.

Users of SAAS, demand this. DEMAND IT. If they can do it, you should worry.

A Cautionary Tale

It’s hard being a PhD student.

Having been one for quite a long time, I can speak quite passionately about it. Being a passionate person entails that I probably will at the drop of a hat.

Of course, lots of the difficulties with being a PhD student are simply a matter of life. I take a special interest because it was a defining condition of so much of much life and mentoring PhD students will is and will be such a condition for the rest of my life. So when I see a massive failure by a PhD student, I’m inclined to overreflect on it.

Kindred Winecoff posted quite a silly critique of Paul Krugman which was picked up by Henry Farrell. Now, Daniel Drezner has a similar, somewhat more nuanced view expressed with rather less vitrol and hyperbole. They share the same basic flaw: A hugely uncharitable misreading of Krugman as saying that the public bears absolutely no responsibility for since it had no influence on the massively disastrous Bush and Bush era policies. (I’m risking similar problems by not doing a very close exegesis of any of the articles. Furthermore, my generally pro-Krugman bent generates similar risks as Winecoff’s anti-Krugman bent.)

(The big error in this reading, AFAICT, is to miss the dialectic at several levels. The line Krugman is pushing back against is the one which justifies austerity measures with a massive negative effect on the poor and powerless along with irresponsible give aways to the rich and powerful. While there are piles of crap justifications, the key one here is that the public is irresponsible and the elites are relatively helpless in the face of massive public irresponsibility. (Think Santelli.) Whatever responsibility the public bears, I trust that it’s pretty obvious that this line is total nonsense and that’s Krugman’s core point. And, frankly, it’s the interesting point.)

Winecroff is now in a trap of their own making (yes, like Jane Austen, use the 3rd person plural as a neutral 3rd person singular). They gave a junky critique based on a junky reading and littered it with junky hyperbole, e.g.,

If Greenspan’s “with notably rare exceptions” deserves internet infamy, and it does, then surely Krugman’s less notable exceptions should too.

(Even if the junky reading were correct these are not remotely comparable. If the junky reading were correct, Krugman would be wrong (this is what Drezner tries, rather crappily afaict, to show). Greenspan is engaged in a kind of amazing and disgusting chuzpah in the service of some rather dangerous hackery.)

When appropriately (and gently!) chastized by Farrell, Winecroff fails to do the sensible thing that many commentators urged him to do: Take a moment, reflect, and back down. Instead, Winecroff doubles- and trebles-down on the silliness. The silliness is as every level including a classic “I’m leaving thread now” followed almost immediately by several more comments.

All this is relatively minor in the grand scheme of things: In the midst of an event like this, it’s really hard to turn oneself around. But given the systematic failures exhibited, I wonder if Winecroff is going to learn from it. If I were his supervisor (US: advisor), I would print all these out and go through them carefully. I’d probably focus more on the dialectic issues (e.g., problems with burden of proof, charity, self-awareness, tactics, and strategy, etc.). For example, it’s very unclear what Winecroff hopes to get out of the exchange. I’m afraid that bashing Krugman is core, which is really a worthless goal, esp. in this context. An easy win would have been to say, “Ok, let’s put my reading of Krugman aside (I’m not ready to give up on it, but maybe that’s because I really can’t stand him; I have to let that rest for awhile) and focus on the more interesting question of how to apportion responsibility for policy.”

This only wins if making the point is more important than making the bash. Which is why it’s a good move regardless of your goal if you are in hostile territory. It sidelines bashback for a while in favor of counterpoint. Given enough point and counterpoint, you might find your own goal moving from bashing to pointmaking. (This is not to say that bashing is worthless. Sometimes it’s very worthwhile indeed. But it needs to work, at the very least.)

As I said, Winecroff isn’t irrecoverable. I had a similar (more heated) exchange with a random PhD student on the web and they turned out just fine and we’re reasonble colleagues (I’m still a bit wary of them, though). Of course, I had a similar (even more heated) exchange which did not resolve favorably. If you find yourself in this circumstance, get as much reality checking as you can. Reflect. Talk to other (possibly critical) people. Don’t necessarily seek out supportive people, but people who will tell you when you’re off the rails. If you determine you have gone off the rails, apologize and retract and learn from the experience. In particular, learn something about your own strengths, weaknesses, and reactions.

Update: You don’t have to be a student to have major level fail as the Synthese scandal shows. The solution to such fails is the same.

However, the action Frances recommend (apologize first) works best in good faith circumstances. If there’s bad faith or bad blood admitting fault early can really, really screw you. Asking for time to think about it, or putting up similar disclaimers, can be useful. It really is the case that we fallible people sometimes can’t see the obvious. If you aren’t seeing it, then ask for some time to see it. “Hey folks, I’m seeing a lot of heat from people I generally respect but I’m not getting it. Can we hold things for a bit while I figure out for sure what’s going on?” is a reasonable move.

soul needs savin but i’m too damned shallow

(Thanks to Zoe for introducing me to Red (Elegy) and thus giving me this title.)

For my sins, I’m reading up on Virtual Learning Environments. I’m using Blackboard/Vista/WebCT (and they can’t get the name clear…doesn’t say much for the software) because 1) it’s sorta encouraged at the university level and 2) I’m eLearning Champion for the School of Computer Science. Yes, that’s an actual title. The University came up with it.

There’s a lot of that sort of thing in eLearning, I fear.

I also used Moodle rather shallowly and loathed it. I probably would loathe it less than Blackboard if I switched back, if only for the fact that it’s 1) open source, 2) free, and 3) downloadable. We pay tons of money for Blackboard and if I’m going to have a crappy experience I’d at least like the pleasure of being cheap.

Looking around, what I see is a lot of reinvention, and not just school specific stuff like Grades (look! discussion boards! mail! a blog! a calendar). Do we really need all this stuff built into a monolithic (even a modular monolithic) system?

The thing that doesn’t seem to be taken into consideration is portability and longevity. These are closely related. One of the things I set as a goal for the school is to support academic portfolios for our students. For example, if a student wants a letter of recommendation from me, it would be really handy to see what they’ve done. Similarly, I remember finding notes from long ago, and old textbooks, etc. None of these systems facilitate learning beyond the rather specific class they are set up for. That rather sucks. Who would turn to this stuff if they were interested in picking up, oh, Calculus later in life. The MIT OpenCourse stuff and iTunes U get the lectures out, so that’s nice, I suppose. Maybe there is stuff out there that transcends the individual instance of a course.

Maybe something built on Google Wave (for portability).

Dunno if I’ll do anything about it beyond writing this post. See the title!