A bit of Russell on Super Intelligence

Stuart Russell (of Russell and Norvig, AI: A Modern Approach fame) has a book out on super intelligence. IEEE Spectrum has an excerpt:

AI research is making great strides toward its long-term goal of human-level or superhuman intelligent machines. If it succeeds in its current form, however, that could well be catastrophic for the human race. The reason is that the “standard model” of AI requires machines to pursue a fixed objective specified by humans. We are unable to specify the objective completely and correctly, nor can we anticipate or prevent the harms that machines pursuing an incorrect objective will create when operating on a global scale with superhuman capabilities. Already, we see examples such as social-media algorithms that learn to optimize click-through by manipulating human preferences, with disastrous consequences for democratic systems.

…

Surely, with so much at stake, the great minds of today are already doing this hard thinking—engaging in serious debate, weighing up the risks and benefits, seeking solutions, ferreting out loopholes in solutions, and so on. Not yet, as far as I am aware. Instead, a great deal of effort has gone into various forms of denial.

The rest of the except is a fun if depressing read though I take some of it with a grain of salt. I do agree with the general thrust that a priori arguments against the possibility of powerfully harmful AI (whether by impossibility arguments or by pollyanaish assertions of easy mitigations such as “turning it off”) are probably all very bad. I tend not to think that much about the problems of super intelligence (esp runaway malevolent super intelligence) because there are plenty of worrisome issues with existing pretty dumb and indifferent AI. I’m pretty skeptical about positive feedback loops to singularityish states with current technology (even 5-10 year technology) on lack of unfettered feedback loops and energy issues (i.e., we’re not yet able to have AIs design and build AIs in an improvement spiral and current compution to energy ratios aren’t hugely favourable to supporting superintelligences —our brains have amazingly low power requirements given what they can do; perhaps there are sneaky algorithms that would support super human general AI on current to near future hardware with reasonable energy consumption but I don’t see it).

There are some reasons to thinking that iterative runaway intelligence arguments aren’t all that strong…after all, just because a system is very smart doesn’t mean it has either the intelligence or the information to design a better system. I sincerely doubt, of example, that a super intelligence can just “figure out” what dark energy is without doing a lot of experimentation. How that experimentation turns out is unclear!

However, we don’t need magic super intelligence for AI systems to be dangerous in a variety of ways. While I don’t think it’s a bad idea, per se, to try to think about superintelligences and their possible emergence, the press for this has been ridiculous. There’s plenty of stuff to be concerned about with today’s systems and we should put a fair bit of focus on that.

Music Monday: Sacrilege

Great song but the original video is way too disturbing.This performance on Letterman is incredible.

One thing I find interesting about the song is that the lyrics seem better than they are because of the music and performance. I mean here’s all the distinct verses:

Fallen for a guy, fell down from the sky
Halo round his head
Feathers in a bed
In our bed, in our bed

…

Asked if I would try
To leave this all behind
Halo round his head
Feathers in a bed
In our bed, in our bed

And the two choral lines:

It’s sacrilege, sacrilege, sacrilege, you say

And I plead and I pray

So, the story is basically, narrator slept with a fallen angel. That’s it! No details.

But the song is so rich and evocative it doesn’t matter. The words are more than good enough to work with the music.

OTOH, compare with “Angel in the Storm”:

The music is terrific, but there’s so much poetry in the lyrics. Zoe seamlessly — effortlessly — weaves the mundane with the transcendent:

She’s had many lovers
And then vanished from their lives
She thinks that they’ll forgive her
If she carries them all with her
Like a necklace made of knives

I know that I’ll be one of those
Who waits for her return
But she never flies the same way twice
She says the more you sacrifice
The brighter you can burn

It would be interesting to hear a more intense arrangement of “Angel in the Storm”. The current one has a more blues grassy feel which is tremendous fun but doesn’t quite punch the way “Sacrilege” does.

Word2Vec Misleadings

Distributional semantics is all the rage and is very cool and omg it’s a bit of a fraud!

Just to be clear: there is nothing wrong with the algorithm itself! It is conceptually very interesting and works very well for a lot of cases. Done right, it can give a decent representation of word similarity or meaning. But the “King – Man + Woman = Queen” example by far overstates what the algorithm actually is capable of.

Here are some reasons why I think we should stop using that classical example to introduce Word2Vec:

1) It turns out that for the example to work in the first place, you have to include some ‘cheating’. The actual result would namely be King – Man + Woman = King. So, the resulting vector would be more similar to King than to Queen. The widely known example only works because the implementation of the algorithm will exclude the original vector from the possible results! That means the word vector for King – Man + Woman is closest to the word vector for King. Second comes Queen, which is what the routine will then pick. Quite disappointing, isn’t it?

Why yes, yes it is. (I wonder how close ‘Queen’ is to ‘King’ without the subtraction. Or under random subtractions on ‘King’.)

Gah! This drives me nuts. Please don’t mislead in this way!

Alaska is Destroying Its University System

Primary blame lies with the governor, but the legislature has failed:

The impasse over the Alaska system’s budget isn’t over yet. But on Friday the university officially lost 41 percent of its state funding in one fell swoop, as lawmakers failed to override Gov. Michael J. Dunleavy’s big cut to the system. Legislators in both the House and the Senate adjourned until next Wednesday, well past the Friday-night deadline for overriding Dunleavy’s line-item vetoes of the state budget.

Let me note that this is completely bonkers. Even if there were a case for shrinking the system in a variety of ways (but to half its size?!), there’s almost no more damaging ways than a fast, brutal slash:

Across the system, no one is hiring, no one is traveling, and no one is buying anything, unless it’s absolutely necessary. Officials are scrubbing any organizational memberships that cost money. About 2,500 staff members have been sent furlough notices, requiring them to take 10 unpaid days off during the year.

This is not forced on the state at all, it’s purely a political decision by the governor using a line item veto.

Which just shows how fucking stupid line item veto are esp with a challenging override mechanism.

I loved this bit of the budget memo (which really is bonkers):

While the University has consistently claimed that the Governor’s proposed budget would require it to close an entire campus, that is not at all clear. For instance, the University could consolidate its two engineering schools into one school in Fairbanks. It could consolidate its arts and sciences programs into one school in Anchorage. It could consolidate its business management programs into one school in Anchorage. And so on. Location disruption for students can be mitigated by live-streaming classes.

AND SO ON?!?!?! Well that’s a well thought out blueprint. Things aren’t clear but instead of working out a plan, let’s slash and pretend things will work out!

AND LIVE STREAMING WILL SAVE THE DAY!!!

One thing that really pissed me off is that I’m on board with complaining about top heavy administration…but it’s very unclear that this will fix that. It’s not like consolidation is needed to cut overpaid administrators! I mean, the redundancy isn’t the core problem!

Universities beware! High priced administrators will not save you and, indeed, will be used as a stick to beat you. Rebalance on your own terms.

Music Monday: If I had a Cello

I will not let my post count stay at 666, appealing as that might be! Time to start blogging again.

I’m picking two old Zoesongs, two of my absolute favourites and two which demonstrates just how fucking fantastic a songwriter she is. Both are “muted” overall, medium tempo, only a bit of range stretching and fairly simple arrangements. But the lyrics and music are devastating. Today, “If I had a cello” and next week “Songs of Love and Distance”.

This song is so heart achingly world weary yet…uplifting! It starts so strange:

Dont ask what I’m doing these days, I don’t know,

I lose track of it all

Is the interlocutor a daily acquaintance? I don’t think so. They seem like a far off person, at least temporally. Someone trying to catch up. Someone who we might have expected to lose track of the narrator’s all. But so has the narrator!

I’m alright I just lie here watching the light
as it slides down the wall

Depression? Or just…on the phone with an old friend or lover wondering wistfully. I always hear this as a call out to Paul Simon’s, “Obvious Child”:

I’ve been waking up at sunrise
I’ve been following the light across my room
I watch the night receive the room of my day
Some people say the sky is just the sky
But I say
Why deny the obvious child?
Why deny the obvious child?

(This is certainly one of the more elliptical verses of a song with a lot of elliptical at best bits.)

And some things have happened, but nothing’s really news
Maybe I’ll get a cello, and teach it to sing the blues

This not a blues song. It’s a meta blues song. The fantasy of getting a cello and mastering it to the point it sings the blues is so extravagant compared to the prior lines. Then we get the first round of the chorus:

If I had any sense I would say I was steady
If I had any say well I’ve said already
And you know, if I had me a heart I’d have something to lose
If I had me a cello, I’d teach it to sing the blues…

The first time I hear that it killed me. The way Zoe’s voice comes down on “blues” is so wishful, emotional, yet detached. I sometimes read it as about depression (esp with some dissociation and depersonalisation). Depression doesn’t always manifest itself as tearful sadness and more than grief does. Sometimes, you’re just lost. So far away from anything that feels real or good or true or happy that there’s just nothing. Numbness is a coping strategy but it doesn’t feel good. It’s distressing.

The second version deepens the depression theme, but the second chorus really hits the bluesiness:

And I don’t wanna cry and I don’t wanna holler
And there’s nothing to buy with another day’s dollar
And you know, I got nothin’ to say, nothin’ to lose
But if I had me a cello I’d teach it to sing the blues…

The transmutation of elements of the first chorus are spectacular. From  the sense of saying “I was steady” to not wanting to cry or holler feels like the surveying of the range of futile options (reinforced by the second line).

The start of the last verse makes me tear every time:

Don’t ask if I love you today, I don’t know
I’m a bit out of touch
I forgot what it feels like to care a whole lot
About anything much

I don’t feel like singing, there’s nothing in it now
If I had a cello, I might remember how!

 

This fixes the depression theory for me and I remember being the interlocutor (and the narrator). The loss of love and singing is only held off by the strange cello fantasy. The last chorus throws in some more blueness:

If I had me a plane I could be on the level
If I had me a soul, I could deal with the Devil
And you know, if I had me a heart I’d have something to lose
If I had me a cello, I would teach it to sing the blues

 

There’s an awesome coda:

And nobody dies of a second rate sorrow
I could get out of bed and feel fine tomorrow

This captures one of the most destructive aspects of depression, the idea that it’s no big deal and sorta a choice.

There’s a version of the recording where she double tracks the vocals on the chorus offset by about half a line. I loved it but her dad thought it made things too cluttered. I don’t know if I still have a copy of it but I will dig around.

As poetry alone, this is amazing. It’s definitely a “song” type of poetry, but the subtle imagery and progression are precise and evocative in multiple ways. The rhyme and rhythm are perfect and the tone varies from formaller to looser without any artificiality.

Then there’s the music. It’s sneaky then it soars. It fits the words so perfectly that it carries you along through their complexities. The way Zoe hits “touch/much” rhyme makes the whole verse into devastation for the interlocutor.

Travelling Moon has a lot of a lot of great songs on it. People should give it a listen!

Some Good Things

We had an 89.2% response rate for the PRES! We were number 1 across the university and smoked Maths (at 81.2%). I’m really happy about that! No one else really cares 🙂

I hope the data are useful. Obviously, the best out come is that everything’s great. The alternative is that our students are experiencing problems! But any which way it will be good to know.

I’m moving on to the next task which probably is revamping the employability support for PGR students.

The other good thing this week was that I got official notice of an award!

Dear Bijan

I’m delighted to inform you that you are the winner of the Manchester Doctoral College Supervisor of the Year Award 2019 (Faculty of Science and Engineering)

The Supervisor of the Year Award recognises an individual who has created the most supportive, stimulating and inspirational research environment for doctoral students.

I can’t tell you how much this delights me at a fundamental level. I really enjoy teaching of all kinds but PhD supervision has a special place for me. I’ve seen some really joyful transformations as students first approach, then understand, then enact research. I spent a long time as a PhD student and a lot of what we do as academics is closely related to what PhDs do.

I give it my all but that doesn’t always work. That my students would put me forth for this makes me hope that I’m doing something right.

I Hate Waiting

I clearly have a bit of obsessiveness. I hate waiting.

For the past 5+ weeks I’ve been working in my new role as Deputy PGR Director (i.e., Deputy Director of the PhD program) to get our students to respond to the Postgraduate Research Experience Survey (PRES), For those in the biz in the UK, it’s basically the NSS for PGR students, but much less abused. Our school uses this data to make real changes, we hope for the better. Good things have already come from it.

Last time (2017) our response rate was about 54% which put us amongst the lowest. Maths had ≈90%. The director tasked me with getting our rate up and I really went at it.

I emailed. I spammed the research symposium. I held contests. We contacted supervisors. I tracked our numbers obsessively (we get them each week). I knocked on doors.

Our last official rate (on Monday with 5 days to go) was 78% and my current estimate is 83%. The rest of the university seems to be lagging over 2017 (they were seriously lagging on Monday). We have a reasonable shot at the number one spot.

The survey closes in 45 minutes but I won’t know how we did until MONDAY!

Gonna be a long weekend.