Ontology Management on the Gartner Hype Cycle!

The Gartner hype cycle is an analytical construct (of sorts) which tries to capture the relation between a technology and the expectations we have for that technology. It’s based on the pretty reasonable observation that esp with new technology, there’s a tendency for expectations to outrun the current or even potential benefits. Everyone wants to use the new glittery magic, so vendors and specialising consultants do very well for a while. But it turns out that the new technology isn’t magic, so people find that they’ve spent a bunch of money and time and energy and they still have the problems the tech was supposed to magically solve. This leads to a crash in expectations and a backlash against the tech. But lots of new tech is actually useful, used appropriately, so some of the new tech, its shiny worn off, finds a place in our toolkit and tech landscape. The Gartner hype cycle is a pretty iconic graph with fun-ish labels:

(The y-axis gets different labels over time.)

And people try to operationalise it:

But I’m skeptical about a lot of this as being rigorously evaluate.

Of course, sometimes a tech takes off and doesn’t really stop. It goes pretty straight from trigger to productivity. The iPhone/iPhone style phones comes to mind. It Just Grew. It may level off as it hits saturation, but that’s a completely different phenomenon.

This is all pretty banal stuff, but Gartner takes it very seriously (they’ve branded it!).

ANYWAY, this year’s hype cycle, excitingly, includes ontology management for the first time! WE’RE ON THE MAP!

• 16 new technologies included in the Hype Cycle for the first time this year. These technologies include 4D Printing, Blockchain, General-Purpose Machine Intelligence, 802.11ax, Context Brokering, Neuromorphic Hardware, Data Broker PaaS (dbrPaaS), Personal Analytics, Smart Workspace, Smart Data Discovery, Commercial UAVs (Drones), Connected Home, Machine Learning, Nanotube Electronics, Software-Defined Anything (SDx), and Enterprise Taxonomy and Ontology Management,

Alas, if you look at the graph, we’re on the downslope into the Trough of Disllusionment:

And it has a “more than 10 years” to mainstream adoption label.

Ouch!

This is discouraging and perhaps hopeful. Remember that the hype cycle doesn’t tell you much about the quality, maturity, or utility of the technology, only the perception and influence of perception on the market. (To the degree you believe it at all.) 10 years to mainstream adoption is not 10 years from being a boon for your business or a viable business itself. It means you will often have a hard sell, because people are skeptical.

Update: Oh WordPress. Picture management please.

Music Monday: Zoe Mulford’s Small Brown Birds heading into endgame

Yay! Zoe had a couple of studio days last week and made a fair bit of progress on getting some more new album songs into the can. Most of the recording for these songs is done, so we’re in the editing stage. There are still some songs to come which will need recording.

(She had a lengthy period where not much was happening because of some vocal issues. Lots of nasal spray of various sorts later, things seem much better!)

You can hear a preview of one of the tracks:

I’m pretty sure that this is mostly done. It certainly hasn’t been mastered, but any further engineering tweaks will be minor.

You can compare this to an earlier production for a Holiday Sampler:

Should you prefer sensitive (noisy) or insensitive (lagging) poll aggregation?

There are quite a few poll aggregators and predictive models based on poll aggregation. This is a huge improvement on the status quo ante where our basic access to polling data was at the individual poll level.

Polls have error. Polls have biases (hidden and otherwise). Polls are a snapshot.

When you see a headline number of a poll, remember there are at least three factors: The poll’s data acquisition methodology (their sampling strategy, questions they ask, etc.), the actual data gathered, and the interpretation of that data. Each of these can have a very large effect on the headline numbers and any of them could easily reverse the rank order of the candidates. (See the wonderful Upshot article wherein they gave the same gathered data to 4 pollsters and got 4 different results which include a Trump and several Clinton leads. These pollsters were all doing a defensible job! No hackery there!)

Poll aggregation is, in effect, a poll of polls. So the same things feed in: their methodology (do you include 4-way race polls?), actual data, and interpretation (do you weight your averages?). As a result they can give you different results. For example, Talking Point Memo’s PollTracker:

http://core.talkingpointsmemo.com/pt/charts/contest/us-president-2016?f=%7B%22methodology%22%3A%5B%5D%2C%22new%22%3A1%2C%22r%22%3A%5B0%2C1%5D%2C%22population%22%3A%5B%5D%2C%22with_candidates%22%3A%5B%224e8b71050a30d83b5587ba54%22%2C%224e8b71050a30d83b5587c275%22%5D%2C%22pollsters%22%3A%5B%5D%7D

Is generally a bit more pessimistic about Clinton than the HuffPost Pollster

http://elections.huffingtonpost.com/pollster/2016-general-election-trump-vs-clinton/embed.js

And the RealClearPolitics one is more pessimistic about Clinton:

http://www.realclearpolitics.com/scripts/widget_embed.js?id=5491&width=450&height=338&key=general_election_trump_vs_clinton

(I’m going roughly by the number of times Trump’s trend line touches or crosses Clinton’s.)

When we get to forecasting models, we get even more variance. A forecasting model is a prediction of a candidates chances of winning, usually expressed as a probability. So if you see that Clinton has a 65% chance of winning, it’s not that she’s polling at 65%, but that she has a 65% chance of winning the election (which she might do by a razor thin margin!). For win probability, a very stable razor thin margin is better than a highly volatile large margin. Or it should be!

Some predictive models are more volatile than others. You can see this most easily on FiveThirtyEight’s prediction page because they have convenient radio buttons for selecting between three models with different levels of sensitivity to the polls (with the “nowcast” being is a “straightforward” poll aggregation). In contrast, Sam Wang’s model tends to move more slowly, by design.

So, which should you prefer?

In general, just as with polls, it’s good to look at multiple models. It gives you more information and reminds you that prediction is a tough tough game.

I think, in general, it’s worth being stable rather than highly reactive, so I tend to lean on less volatile models. There are several reasons:

1. We’re still pretty far out. Getting worked up about something that might be a statistical blip or a cyclic movement is pretty unwise. If some movement in the averages or forecasts is worth worrying about, then it will be durable and show up in all the models. Getting a “jump” on bad (or good) news isn’t really helpful, esp. as there’s little to do in response (for most of us). It’s similar to the stock market: Most of us aren’t equipped to do much short term trading efficiently, so it’s better off thinking long.
2. We really don’t know the underlying causal structure. One phenomenon that has been shown in the lab is “differential (non-)response”, that is, it is common that people respond (at all!) to polls depending on “(de)energising” events. Thus, consider convention bumps. Each candidate typically gets a boost in the polls that then fades after their convention. Why? Are people changing their mind? Are they really that fickle? Perhaps, but it also could be the case that there voting intentions (which is what we care about) don’t change, but whether and how they respond to polls changes. Thus, in addition to sampling error and other methodological and interpretive biases, we have the possibility that salient events might change polling results without there being a change in the phenomenon we’re trying to measure.
3. Given the strong negatives associated with a Trump victory, anything from a 10% on up is extremely worrisome. It’s worth being worried. If you can use that worry to prompt action, you should do it regardless of the current state of the polls.

So, prefer the more stable aggregators and forecasts. Also prefer the ones that are most inclusive of polls and minimise the “special sauce” in their models. If you want to know what a fundamentals model predicts, just use a separate prediction rather than trying to weave it into your polls based predictor. There’s enough interpretative variably that adding things which aren’t really made to work together is a bad idea. Better that each sort of evidential base has it’s own predictive model and you can compare them more or less directly.

Note: WordPress.com scrubbed all the embedding code for the aggregations. I’ll try to update with screenshots later. Sigh.

Update: PollyVote is a forecast model aggregator! So it saves you the work🙂 (It seems to have two levels of aggregation: It aggregates with a type of forecasting method, e.g., prediction market vs. econometric, and it aggregates over those types.) One interesting thing is that it provides a popular and EV vote total, as opposed to a win probability. Another is that it doesn’t incorporate error estimates (indeed, it’s hard to see how to do it). OTOH, it’s super simple and straightforward and covers the main sources of evidence. It will be interesting to see how it does in this weird weird year.

Blog Shout Out for Now Face North

Now Face North is a blog by LGM long time most-valued commenter JL. Any JL comment is worth reading. A JL comment about sexual or domestic abuse or rape victims (esp. about support) or activism is worth spending some serious time with. As we go into the election, her stuff will present a side of electioneering that you won’t typically see. Whether it’s about political trials and public defenders, the differential treatment of pro-Trump and anti-Trump by police at an RNC protest, or the ins and outs of being a Street Medic, there’s a lot there. It’s experientially grounded but clear, coherent, and thoughtful. And sometimes pretty funny:

Do you know where this march is going?

Okay, seriously, undercover/plainclothes cops, I don’t know why you all always seem to think that medics will know the answer to this question, but we usually don’t. Please stop asking me. Also, most of you are bad at pretending to be protesters. There are notable exceptions, but they are generally not the ones who meander up to street-medics fake-casually to ask where the march is going. If you’re not a cop and you’re asking me this question anyway, I still probably don’t know. Ask an organizer.

Add it to your rotation this election season. JL doesn’t post that frequently, but binge reading is a delight.

Grumpy about Textbooks

I definitely need to do more research but I don’t feel that there is a really solid textbook on software engineering. I use Steve McConnell’s Code Complete (second edition) and Making Software for readings.

These are both pretty good. Code Complete is a bible for many people (not for me!) but regardless it’s definitely on a “you should read this if you are a software engineer” list. It has a few problems though:

1. It’s not written with courses in mind, as far as I can tell. It introduces a lot of stuff and sometimes in a helpful order, but other times not. The “learning objects” are not clear at all.
2. It’s not super well written. You get a lot of interesting lists (e.g., of program qualities) but they are often not coherent, have some redundancies, are are perfunctorily designed. These often feel revelatory on a first read but if you try to work with them you get a bit grumpy. For example, we have 4 kinds of tests: unit, component, integration, and system. Unit and component test bits of the same size: a unit. The difference is whether the unit is maintained by one team (thus a unit test) or more than one team (a component test). This is bonkers. It’s esp. bonkers to compare with integration or system tests. It could be part of an interesting axis (who’s the owner vs. who’s writing the tests). But there are much better frameworks out there.
3. It’s a bit dated. The second edition came out in 2004 and is thus 12 years old. This doesn’t invalidate it per se, but given that the book itself has a prominent discussion of the need for life long learning because the fundamentals of software engineering keep changing, it’s a problem. I’d prefer something other than Basic as the “other” example language.
4. It pretends to focus on code construction, but has just enough architecture, etc. to be almost a reasonably complete text. But the scattershot approach is a bit disorienting.

If you read it cover to cover and absorbed it all with an appropriately skeptical eye and organised it appropriately, then you’d be in great shape.

My pal Mark suggested reoriented on The Pragmatic Programmer, which is another classic and definitely on the must read list. But a lot of my concerns apply to it too. (That there’s a basic divide between those pushing Code Complete and those pushing the Pragmatic Programmer is interesting. The lines I’ve seen is that Code Complete aspires to be encyclopaedic and the Pragmatic Programmer is more opinionated and thus effective. Roughly. They both feel scattered to me.)

I could try both (not this year!). I could go with Pragmatic Programmer because it’s smaller and thus they could possibly read the whole thing.

But neither feel satisfactory as a textbook. The systematicity and pedagogic logic just don’t seem to be there. So I’m left imposing some order on them.

Music Monday: Think Respect

I finished the Aretha Franklin biography and it was interesting. Her response in the press is exactly what you’d expect having read the book, so it has that going for it. It still didn’t really help me understand her musically. I’d really like to see “music biographies” emerge which trace the music and it’s connection (with audio obviously). However, here’s are two of her most famous and celebrated songs.

Think is awesome, of course:

esp. in the Blues Brothers:

Who doesn’t love Respect?

Interestingly, Respect was written and originally performed by Otis Redding:

It was Franklin who transformed it into a feminist anthem. Which is pretty cool.

While looking for the Redding version, I stumbled on his cover of (I can’t get no) Satisfaction:

Which Franklin also covered!

I guess I prefer Redding’s, because I think the arrangement is more interesting (the prominence of the driving beat during the verses is a pretty big and interesting departure from the Stones’).

Of course, the definitive cover is Devo’s deconstruction:

The performance on SNL is very interesting but it’s dropped off YouTube, at least, to my searching.

Music Monday: Eleanor Rigby

 

I’m listening to the audio version of “Respect: The Life of Aretha Franklin” by David Ritz. It’s one of this month’s freebies in my Scribd subscription. (The monthly selects have been pretty damn good on African American stuff!)  Aretha hates this biography (as the biography predicts), but it seems pretty positive overall. Maybe over the top. There’s a lot of quotes from collaborators and family and admirers about how awesome she was, but they tend not to give much insight in to what, technically speaking, made them so impressed by her. Now, don’t get me wrong, I remember loving Think in the Blues Brothers. And who doesn’t like Respect. If you don’t like Sisters are Doing it for Themselves, you are wrong:

But Franklin’s voice and performances didn’t essentially grab me. I like Lennox a lot more (in general and in Sisters). This may be my lack of understanding of Franklin’s idiom, i.e., the untutored ear problem.

In any case, I was delighted to find out that Franklin had covered several Beatles songs, including Eleanor Rigby. I love Eleanor Rigby (which was also probably, like for many people, my first exposure to a string quartet). It’s such a distinctive song on every level from lyrics to arrangement.

But…Franklin’s version has almost nothing to do with the Beatles version:

I don’t get why this arrangement exists. It seems like a completely different songs with most of the same lyrics rather than a reinterpretation. I know extreme reinterpretations exist and can be really interesting. I tend to be grumpy about them if I liked the original. For example, Beyonce’s Single Ladies:

doesn’t seem illuminated by Sara Bareilles…whatever…loungey?…version:

It seems competent. But it doesn’t do much for me after sorta laughing (because I think it’s a bit of humorous take).

Ray Charles’ version of Eleanor Rigby, in contrast, seems inspired:

I doubt it’ll be my canonical version, but it really is an amazing version. It captures the essence while reflecting it into a different idiom.

Contrast with this Alice Cooper version:

This is just the original arrangement, for the most part. If you prefer Cooper’s voice, I guess you’ll like it better.

Cássia Eller’s is sort of in between. I think it’s way farther from the baseline than Ray Charles’, but it’s somewhat closer than Franklin’s. It’s very weird! But hey, I keep finding these Brazillian covers:

Franklin still eludes me and maybe always will. But I’m still listening to the biography and will keep trying to train my ear.

Music Monday: Wannabe

I sorta missed the Spice Girls. I vaguely recall feminist analyses of Spice Girl flavoured “Girl Power”, but I didn’t hear “Wannabe” until a few years ago. It’s reasonably catchy and as I’m a sucker for long takes I rather enjoy the music video:

There are fundamental challenges with attempts to reclaim female sexuality under conditions of patriarchy. This point goes back at least to the sexual revolution wherein conceptions of sexual liberation often included various sorts of diminished consent for or sexual exploitation of women. Such challenges are, I now think, inevitable and we should take wins when we get them to the degree that they are wins. What threw “Wannabe” back on my radar is this reimagining of the video by the Global Goals into something unambiguously feminist:

This makes me just so happy! It’s wonderfully global and perfectly direct in the messages.

There are quite a few covers of “Wannabe” out there (including far too many thrash punk ones), but this Brazilian one seems the most interesting, esp. given that they did a translation:

And, of course, let’s not forget Adele talking about the Spice Girls (and singing along to “Wannabe”) on Carpool Karaoke:

(Sigh. WordPress doesn’t understand YouTube’s “link to a time point” feature. Follow the link if you want to jump to the Spice Girls bit.)

The Muddling of the Mental and the Physical

Nature also teaches me, through these sensations of pain, hunger, thirst and so on, that I (a thinking thing) am not merely in my body as a sailor is in a ship. Rather, I am closely joined to it—intermingled with it, so to speak—so that it and I form a unit. If this were not so, I wouldn’t feel pain when the body was hurt but would perceive the damage in an intellectual way, like a sailor seeing that his ship needs repairs. And when the body needed food or drink I would intellectually understand this fact instead of (as I do) having confused sensations of hunger and thirst. These sensations are confused mental events that arise from the union—the intermingling, as it were—of the mind with the body. Descartes, Meditation 6

Descartes is, of course, the arch-dualist. Mind and body are different substances with entirely different natures and can exist independently. Human beings, on the other hand, are not just their minds (even though the mind is the ego who’s existence we know first, and best). The things that teach us that we form a kind of unit — pain, hunger, thirst, etc. — are perceptions of the body which differ from how experience the rest of the world.

I was thinking about this because I’ve been feeling like crap for months now. Clearly there is a strong physical element, but equally so, there’s a strong mental component. They go back and forth in a complex dynamic but it’s not always clear which is which or even if they are fully separable. If I dry heave, it could be pure anxiety, a stomach virus, or a side effect of medication (perhaps for anxiety).

The most striking (for me) example in my personal history was the interaction between my inner ear issues and social anxiety. When I was a teen-ager, I developed an inner ear disorder that ranged from subtle to extremely overt (i.e., spinning for three days at a shot). But effect of the subtle variant was that in noisy environment with a fair bit of motion, my ability to distinguish my movement and other objects movement was diminished. (Think of being on a smooth and slow moving train when it just starts up and you’ve been distracted.) This can make you feel very uneasy and off balance and…anxious.

This inflected my experience of social gatherings…dances, parties, etc. When this got really going I would feel unsettled and uncomfortable and usually seek a quiet berth (kitchen, outside, or…not there). Part of this was undoubtably due to this inner ear phenomenon, but I had no idea that it even existed. So I interpreted this mostly physiological reaction as being a dislike of parties or part of my social anxiety. Which didn’t help the anxiety at all. On the contrary.

We know that many physical illness tend to have certain mental co-morbidities. Being sick sucks, so depression isn’t uncommon.

Our Cartesian unity…the fact that we are a big muddle of a complex system…makes life difficult. Our parts don’t swap easily.

Labour Leadership Election 2016: Pro Owen Smith

I voted for Owen Smith for Labour leader today and if you are eligible to vote, I encourage you to do so as well.

The current situation is pretty damn grim and likely to get grimmer both for Labour and for the UK. We need to at least staunch our wounds to survive. Corbyn is clearly not the person who can even try to do that.

Corbyn is successful in getting strong support from a large group of left leaning folks (from moderate to extreme left). (I count myself as pretty far left, but also as a devotee of harm minimisation over purity in political action.) This translates into strong support from general party members and many left leaning folks joining. Some of his policies are generally correct but often suffer from 1) lack of specificity (cf his anti-austerity measues), 2) poor prioritisation (cf. Trident), or 3) poor messaging (cf. Brexit, with the generous assumption he was actually pro-Remain) and sometimes all three. This makes him a terrible standard bearer for these policies.

Corbyn clearly cannot work effectively with the PLP or members of the political elites (including policy wonks). The mass resignation of the Shadow cabinet including former supporters is strong evidence thereof and there are increasing numbers of specific stories where incompetence or misprioritisation screw things up. As a fringe party member for decades, he just hasn’t been able to transcend that both in the perception of others and, as far as I can tell, his own thinking.

Corbyn faces a hugely hostile press. The coverage of him is decidedly unfair, but absent a plan to turn that around or mitigate it, it doesn’t matter that it’s unfair, merely that it exists. Plus, I would go so far to say that he exacerbates the problem. Stuff like not timing press releases correctly make things worse.

Corbyn is not seen as PM material by large swaths of the electorate. Putting aside the question of whether Labour can win the next general election (it’s going to be tough no matter what), voters often vote against MP candidates when they dislike the Leader, because they understand (correctly) that the way to vote for a PM is to vote for that candidate’s local MP. (This can be overridden, of course, by local excellence, but we’re talking about overall effects.) Corbyn’s multiple layers of general unpopularity will hurt all Labour’s races. The size of a loss matters.

Thus far, this has been all “vote against Corbyn” instead of “vote against Smith”. And I think there’s a very strong case to be made on that basis. I’m not quite “anyone but Corbyn” but it’s pretty close. Corbyn is close to being a Trump like figure for Labour: An outsider, outlier candidate who’s pretty toxic with the populace and terrible at conventional politics but caught the imagination of a fairly cohesive group with outlier views. This is a recipe for destruction.

Smith seems about as left a conventional politician we are likely to get with good relations throughout the political elite. He has been a party loyalist while trying to move the party left (which leads to some party line votes people take as evidence of his “blariteness”). Thus, I think he will play better with the press and the general populace and do better in getting the MPs together to fight in the next election (and to be a strong opposition now).

Defeating Corbyn is not, necessarily, a defeat for the left in Labour any more than Bernie Sanders loss was a defeat for the left in the Democratic party. Sanders helped push the Democrats much more to the left. If Corbyn doesn’t torch the party, that could happen as well. If the party falls apart, this will marginalise the left like almost nothing else could.

Contrariwise, recognising that polices are what matters and that we are, to say the least, living in a country which doesn’t easily tack left and being willing to strategically compromise is the best way to have a substantial and effective effect. We shouldn’t be invested in a person, but in a movement.

N.B. This article contains a compendium of links about issues with Corbyn. I find it mostly compelling though incomplete.