Making Principled Unprincipled Choices

I like principled decision making. Indeed, few things inspired me as much as this quote from Leibniz:

if controversies were to arise, there would be be no more need of disputation between two philosophers than between two calculators. For it would suffice for them to take their pencils in their hands and to sit down at the abacus, and say to each other (and if they so wish also to a friend called to help): Let us calculate.

Alas, there’s no decision making situation where this vision holds, even in principle. But still, I like my decisions to conform to some articulable rationale, preferably in the form of some set of general rules.

But some of my rules are meta-rules which focus on resource use. Obviously, one goal of decision making rules in to maximise the chances of making the “right” choice. But for any metric of rightness (let’s say, an appliance with the best value for money) there’s a cost in the effort to assure the maximum (e.g., research, testing, comparing…lots of shopping). That cost can be quite large and interact with subsequent satisfaction in a variety of ways. I’m prone to this and, indeed, end up in decision paralysis.

In response to this, one of my meta-rules is “don’t over-sweat it”. So, for small stuff, this reduces to “don’t sweat the small stuff”. But, because of my anxiety structures, I tend to see certain classes of small stuff as big stuff. So, I dedicate some effort to seeing small stuff as small. Sometimes, this means making it invisible to me. Poor Zoe often has to make the actual purchase after I’ve done the research, or even make the decision after I’ve done the research. For various classes of minor, irrevocable sub-optimal decisions, I prefer not to know about them. I will obsess, and that doesn’t help anyone.

When the decision is essentially arbitrary (because all choices are incommensurable in toto, or their value is unknowable at the moment), I try to make myself flip a coin (metaphorically, at least). What I try to avoid is building a fake rationale (except when that enables the choosing or makes me happier with the arbitrary choice).

Technical (or teaching) decisions often are best treated as arbitrary, but we have tons of incentives to treat them as requiring a ton of analysis to make the “right” choice. At the moment, I’m evaluating what Python testing framework to use and teach in my software engineering class. I currently use doctest and unittest and have a pretty decent lesson plan around them. doctest is funky and unittest is bog standard. I’d consider dropping doctest because I need room and we don’t do enough xUnit style testing for them to really grasp it. They are also built into the standard library.

But then there’s pytest which seem fairly popular. It has some technical advantages, including a slew of plugins (including for regression testing and BDD style testing). It scales in complexity nicely…you can just write a test function and you’re done.

But, of course, it’s a third party thing and needs to be installed. Any plugins would have to be installed. Is it “better enough” to ignore the built in libraries? Or should I add it on with the builtin libraries? AND THERE MIGHT BE SOMETHING YET BETTER OUT THERE OH NOES!!!!

No. The key principle here is a meta-principle: Don’t invest too much more effort. Make a decision and stick with it. In the end, any of the choices will do and a big determiner will be “does it spark my interest now?” while the other will be “how much extra work is that?”

And that’s fine.

 

Advertisements

The Cyber Security Body Of Knowledge

This effort looks cool. Providing a codified overview of What We Should Know about cyber security could be very helpful, esp for teaching. I just read the Software Security Knowledge Area and it wasn’t a bad read. It felt a little “listy” without a good cognitive map. In particular, the cop out on the completeness of their taxonomy of faults. I don’t blame them on not claiming to be comprehensive, but I don’t know whether they cover the bulk or the important ones just from reading this. I should have a better sense of what I don’t know!

Then there was this thing that bugged me:

• A detection technique is sound for a given category of vulnerabilities if it can correctly conclude that a given program has no vulnerabilities of that category. An unsound detection technique on the other hand may have false negatives, i.e., actual vulnerabilities that the detection technique fails to find.

• A detection technique is complete for a given category of vulnerabilities, if any vulnerability it finds is an actual vulnerability. An incomplete detection technique on the other hand may have false positives, i.e. it may detect issues that do not turn out to be actual vulnerabilities.

Oy! This reverses the ordinary (i.e., mathematical logic) notions of soundness and completeness…sorta. They didn’t quite flip the meaning, but instead they focused on an entailment class that’s weird. Take soundness. The entailment they pick is, “Program P, for vulnerability class V, has no Vs.” It’s that “no” that messes it up. In order to conclude that there are no Vs it has to be the  case that ifthere was a V, it would find it. I.e., that it was complete with respect to “P has a V”. And I mean, the last sentence makes it clear that they are thinking at the “P has V” level. And, of course, their bogus complements focuses on the “P has a V” level, so they just screwed up. Sigh.

It would be much more straight forward to define a detection technique for V as a procedure which takes P as an input and returns a list of “P has a V” statements (with specific Vs). Then the technique is sound if it produces no false positives and complete if no false negatives. A sound and complete technique that returns the empty list allows us to conclude that the software is secure wrt V.

Then there’s this:

It is important to note, however, that some detection techniques are heuristic in nature, and hence the notions of soundness and completeness are not precisely defined for them. For instance, heuristic techniques that detect violations of secure coding practices as described in 2.3 are checking compliance with informally defined rules and recommendations, and it is not always possible to unambiguously define what false positives or false negatives are. Moreover, these approaches might highlight ’vulnerabilities’ that are maybe not exploitable at this point in time, but should be fixed nonetheless because they are ’near misses’, i.e., might become easily exploitable by future maintenance mistakes.

Sigh. Detection techniques that are “heuristic” are generally unsound or incomplete. What they seem to be talking about is problems (or maybe just infelicities) with the definition of some category of vulnerabilities.

Still! It’s in development and even as such, I’d point a student at it. These things aren’t supposed to substitute for textbooks, but they can be helpful as a quick orientation and sanity check.

Floating Point Explained Visually?

Tab cleanup time!

I’ve had this article on “explaining” floating point numbers in my phone’s browser for eons. Basically, my first glance made me go “huh” and I didn’t have the time or energy to figure it out at that moment and that’s what stuck with me.

But I need a post today and killing this tab suddenly was motivating.

One annoying thing about the article is that it doesn’t say what about floating point it’s explaining or why floating point works the way it does. It seems to be more about explaining how to convert from floating point notation in to standard decimal notation. Which…fair enough. I’m not sure the “scary formula” is such a problem or that the window metaphor is all that useful in the end. In the end, standard floating point is just scientific notation in base2 with a fixed width (thus a “moving” decimal point). If standard decimal scientific notation is confusing in the same way then maybe this visual/windowing explanation might help. But I’d start with scientific notation as most people will be a little familiar with it.

The big thing not quite explained is how (and why) various numbers are approximated and thus departures from various standard features of the rationals and reals. That seems to be the deep part of floating point, esp the bits that come from features of the base rather than features of the width.

Any rational number can be represented exactly as a pair (ie ratio!) of integers. The size of your integers constrain which rationals you can represent (ie max rational is maxint/1 and min (positive) rational (greater than zero) is 1/maxint). But in positional notations with a radix point (like standard decimal notation) we typically can’t capture all rationals because some rationals will have an infinite expansion. 1/3 in decimal, for example. (Though the decimal will be repeating.) Binary has a different set of numbers it can’t easily represent.

Ok the point of this isn’t to fully explain floating point but to get back to the idea that it has a lot of pieces so there’s a lot to understand. If you’re only going to explain a small piece of it say that! Say why this piece is useful to understand! Without this you’re problem typically making things worse.

“Actively Maintained” Claims Need Active Maintainence

Software is hard. Free (both as in speech and as in beer) software is hard and often not well compensated. I’m going to whine a bit about very free libraries which tends I bad form. I first want to sincerely thank the people who work on these libraries! You have my admiration and appreciation even as I criticize.

My learning experience on iOS (and specifically iPhone) based python development is extracting the results of the Twitter searches around “The President Sang Amazing Grace” and, I was hoping, to do some simple analytics. Maybe eventually do some sentiment analysis. In principle, you’d think the interesting bit, even moderately challenging part, is in the analyses. Per usual, just getting things up and going is a big PITA.

Out of the four libraries I found with a quick search, I settled on Twython, in spite of the name, because it had the nicest (still not complete) discussion of authentication. In particular, they clearly demonstrate the principle of least privilege by emphasizing the auth level sufficient for read only access. Since this is a large class of Twitter apps, I think it’s important to be emphasized. Yay! It also says that it’s actively maintained! Yay! Unlike those other loser libraries, I guess!

But searching Twitter is a pain. If you have a large number of Tweets in the results, you have to paginate. Furthermore, there’s a question as to how far back you can go (some search specific libs say 7-10 days is it, but the Twitter api seems to have some date range functionality).

In any case, Twython, like other high level libraries, provides a cursor abstraction. But the docs don’t say exactly what it abstracts over! Will it get the next page transparently? Do you need to be in page mode?

But worse, we have a bug of many years standing (with a stale pull request!) which causes a cursor to freaking loop on search results under some conditions. Well that seems to make cursors utterly useless (maybe they are useful out of box in some cases I don’t know, or maybe everyone writes loop detection code in their cursor loops).

Regardless, it makes the “actively maintained” claim…less than true. At the very least a note in the documentation is required but really this bug should be addressed if “actively maintained” is to have any meaning.

(It goes to show that one really REALLY has to work through any coursework one hopes to give. Reading the documentation is not sufficient!)

Demoing YAGNI with FizzBuzz

I use FizzBuzz as a warm up lab and as a key lecture. Tom Dalling has an excellent blog post where they iterate the implementation of FizzBuzz into a fully packaged, fully parameterized with sensible defaults, fully documented module. Which is horrific…completely larded with unnecessary complexity. And the punchline is “The Aristocrats”…er “You Ain’t Gonna Need It”.

I’m wondering if this would work as a lecture or even if it makes sense. Each step is supposed to be inevitable or at least sensible but there’s something made up about it. OTOH, it might go too far toward suspicion of abstraction.

It could but fun if the build was right. Perhaps it would be great if it also made clear when those moves made sense. Right now, it’s only negative. But a lot of those moves are good ones in some contexts.

Gantt Charts

Oh Gantt charts…the bane of many a student and grant proposal writer.

I spent a good chunk of today creating a starter kit for MSc projects on the School’s gitlab. I’m a bit believer in canned setup even when experience says they don’t work that great. At least, not for everyone. But hey, at the very least they might encourage people to use gitlab for their projects!

One thing that always causes problems is the god damned Gantt charts. Don’t get me wrong, a bit of speculative “big picture” planning is a good idea. But…I don’t think we train them in Gantt charts, nor do we supply nice software. Gantt charts are a nightmare! Most Gantt chart software is a nightmare!

I really wanted to use a textual syntax as the common denominator. The three candidates of note were:

  1. Mermaid (a Javascript library with reasonable Markdown integration)
  2. PlantUML which has a nice, if experimental, syntax with half assed rendering
  3. pgfgantt which is comprehensive, thoughtful, genius output and documentation that makes my gums bleed.

PlantUML has a sorta friendly syntax but seems to want to have every day as a slot, which makes a 9 month project unwieldy.

Mermaid refuses to put days on the top or have flexible slots (e.g., weeks). It also doesn’t have milestones (you have to fake it with a one day task). But it scales reasonably.

pgfgantt truly looks brilliant but I just couldn’t fight it today. It’s perfect for LaTeX docs but maybe not so much for Markdown. Plus, not really a friendly stand alone syntax. Definitely something to explore for another day.

With a lot of experimenting, I was able to coax a decent looking Gantt chart from Mermaid with reasonable source. Here’s the source:

gantt
    dateFormat  YYYY-MM-DD
    title Generic CS MSc Project Plan
       
    section Period 3 
        COMP66090            :done,  des1, 2018-01-29, 2018-03-07

    section Period 4
        Supervisory Meetings (term) :active, sup1, 2018-03-08, 2018-05-04
        POP Submission :sub1, 2018-05-10, 2018-05-11

    section Exams
        Exam Period :exams, 2018-05-06, 2018-06-06
        Semester End : 2018-06-08, 2018-06-09

    section Summer
        Supervisory Meetings : sup2, 2018-06-09, 2018-09-07
        Dissertation Submission :sub2, 2018-09-06, 2018-09-07

and here’s the output:

Not bad! Easy to edit. For students, they can just delete Period 3 and Period 4 and have a chart skeleton ready to go. Mermaid includes some dependency support (hence the names) but we really don’t need that for what we’re trying to do: Break a project into tasks, estimate times for a single person, and try to slot them in a calendar. This does the job.

Python Static Site Generators: External Dependencies

How “big” are they?

Yesterday, I looked at SLOC counts for 4 major Python static site generators. Here’s the summary of their counts wrt python files:

Package Python files Python SLOC
Nikola (plugins) 147 (75) 17,506 (8,510—48%)
Pelican 18 4,539
Ivy 33 1,004
Urubu 10 784

As you can see, there’s a pretty steep gradiant, even if we separate out Nikola’s plugins. But this only scratches the intellectual footprint of each. Given that “integrating third party components” is on the list of considerations, I thought I’d peek at their dependencies. This reminded me that Nikola might be a bit inflated as I installed it with “extras”. Since this option is easily available from pip it didn’t seem unreasonable, but there might be a “slimmer” version of Nikola. (A quick install of the non-extras version yields 500 more SLOC! No idea what’s up right now.)

Let’s look at the dependencies.

Nikola Pelican Ivy Urubu
Pygments
Pillow
python-dateutil
docutils
mako
Markdown
unidecode
lxml
Yapsy
PyRSS2Gen
logbook
blinker
setuptools
natsort
requests
piexif
doit
feedgenerator
jinja2
pygments
docutils
pytz
blinker
unidecode
six
python-dateutil
markdown
pygments
pyyaml
jinja2
libmonk
ibis
shortcodes
libjanus
syntex
jinja2
pygments
markdown
pyyaml
beautifulsoup4

(Note that the Pelican docs say you have to install Markdown yourself. So it really should be on this list.)

So, they all outsource:

  • Templating, typically jinja2 (thought that’s an extra for Nikola; mako is the “builtin”)
  • Syntax, typically markdown and/or Restructured Text
  • pygments, because nerds gotta have source code highlighting

Nikola has a fair bit of image stuff (Pillow, piexif). It supports image gallaries and the like out of the box.

Ivy has some homegrown utils for standard stuff (e.g.,libjanus for argument parsing).

Barring install issues, I’m not sure sheer number of packages should deter. It seems easy enough to focus down “on the core” and doing so in the context of a lot of other stuff seems valuable.

Nikola Extended

I wanted to look at the Nikola “extras” and “tests” dependency lists separately:

Extras Tests
Jinja2
husl
pyphen
micawber
pygal
typogrify
phpserialize
webassets
notebook
ipykernel
ghp-import2
ws4py
watchdog
PyYAML
toml
mock
coverage
pytest
pytest-cov
freezegun
codacy-coverage
colorama

That test stuff is really tempting both because it suggests that Nikola has great testing, but because they touch things I wanted to cover anyway.