I Hate Waiting

I clearly have a bit of obsessiveness. I hate waiting.

For the past 5+ weeks I’ve been working in my new role as Deputy PGR Director (i.e., Deputy Director of the PhD program) to get our students to respond to the Postgraduate Research Experience Survey (PRES), For those in the biz in the UK, it’s basically the NSS for PGR students, but much less abused. Our school uses this data to make real changes, we hope for the better. Good things have already come from it.

Last time (2017) our response rate was about 54% which put us amongst the lowest. Maths had ≈90%. The director tasked me with getting our rate up and I really went at it.

I emailed. I spammed the research symposium. I held contests. We contacted supervisors. I tracked our numbers obsessively (we get them each week). I knocked on doors.

Our last official rate (on Monday with 5 days to go) was 78% and my current estimate is 83%. The rest of the university seems to be lagging over 2017 (they were seriously lagging on Monday). We have a reasonable shot at the number one spot.

The survey closes in 45 minutes but I won’t know how we did until MONDAY!

Gonna be a long weekend.


Steven Salaita’s (Academic) Freedom

It seems that the current state of Salaita’s academic freedom is freedom from academia:

Even as I complicate honest work, I’m aware of how indebted I am to the notion.  It guided my exit from academe and my rejection of the pundit economy.  I’ve always overvalued recalcitrance, a compulsion, as I understand it, to vigorously avoid situations that require ass-kissing, usually resulting in significant reputational harm.  Since elementary school, I’ve searched for a space where I could conform to my surroundings without feeling unmoored from an inner sense of decency.  That space, it turns out, is equivalent to the volume of a school bus.

I pitched honest living to my parents when I told them about the new job.  Despite being aware of academe’s ruthless memory, they hoped that I’d one day be a professor again.  They probably still do.  In a better world, my redemption would happen in the United States.  I wanted to quell that expectation.  “Even if Harvard offered me a job I’d say no,” I proclaimed with earnest hyperbole.

They feigned support but didn’t believe me.  I understand why.  It’s hard to imagine coming of age in reverse.  Hollywood doesn’t make inspirational movies about struggling to overcome material comfort.  We don’t aspire to the working class.  Personal fulfillment occurs through economic uplift.  We go from the outdoors to the office, from the ghetto to the high-rise, from the bar rail to the capital.  That’s the dream, to become a celebrity or a tycoon or, in humbler fantasies, a bureaucrat.  But forward progress as material comfort is cultivated through the ubiquitous lie that upward mobility equals righteousness.  Honest living is a nice story we tell ourselves to rationalize privation, but in the real world money procures all the honesty we need.

For immigrants, these myths can be acute.  I could see my parents struggling between a filial instinct to nurture and an abrupt recognition of their failure.  My mom, a retired high school teacher, seemed interested in the logistics of transporting students, but my dad, the original professor, clenched his hands and stared across the table.  It’s the only time I’ve seen him avoid eye contact.

I’ve encounter, recently, students who are are studying computer science not because they like it, or chose it, but because their parents picked it for them. These students, even the good ones, are very unhappy.

I’ve been trying to communicate to PhD students that a PhD program is just one form of life and not one that everyone (or most people!) find worth while. It’s hard to accept that a PhD doesn’t make you better and that failing to get one doesn’t make you worse. It’s not clear that the best way to become a good researcher is to “finish your PhD”. For many fields, PhD projects are not very much like how you go on to do research.

Salaita remains an interesting case. His academic career was definitely and perniciously destroyed on false and illegitimate grounds. He was not a marginal figure but a minor “star” professor. His case was an egregious violation of both the spirit and letter of academic freedom. I should think that the standard advice is to make sure any new contract was fully enacted before resigning one’s old position, just in case.

So he was wronged but seems content with where he ended up. This demands respect. But it’s also a bit tricky to get right.

Of course, less dramatic versions of his story are playing out all the time. The adjunct crises is ongoing. Unemployable people with huge student debt are all over. We have to think hard about how higher education is going to work in the coming decade and to make sure we aren’t causing unnecessary harm.

Don’t Fuck Your Students

It’s past the time that academia adopt the bright line rule. It’s theoretically possible to design a regime governing romantic and sexual relationships between staff and students which allows them to exist which isn’t awful or at least as awful as what we have but:

  1. it would have to be pretty damn onerous, eg pre-registration of the relationship, careful monitoring of any effects on grade and soft benefits (most of them would need to be transferred to other people in most cases), etc.
  2. it benefits people who mutually want a relationship over those who are targeted for unwanted relationships; by being fairly onerous we can cut down the risk to the targeted but not as easily as the bright line regime.
<li>And, of course, enforcement is a perpetual problem. I'm less worried about over enforcement than under enforcement because historically the latter has been a much much much bigger problem. But the first exists as well. The bright line might make the first a bit more likely but I think it's easier to manage.</li>
<li><em>Every</em> policy has trade offs. There will be some innocent, innocuous couples which the bright line policy will either prevent or discourage. But the current policy is wrecking life after life.</li>
<li>Enough is enough.</li>

Universities Are Not Well Run

I read the news today, oh boy:

Students planning legal action against University of Manchester for lost teaching time during strikes

It’s understood that during talks between the student union and Manchester University, one suggestion has been that graduation fees – of £35 – could be waived in compensation for lost teaching time.

Remember that the people who were on strike are getting their pay docked. That is, the university feels that if we do not supply what they are paying us for then we don’t get paid on a pro rata and proportionate basis.y rough and low bally estimate is that students pay £50/day.

Now, to be fair, it’s complex. We didn’t shut down the whole university. But even then, it’s a bad look.

And nominal fee isn’t the whole story…it’s what’s been agreed upon for that fee.

Given that this whole thing was utterly unnecessary because the precipitating event was absurd, evil, and based on lies by the universities, they should step up.

But perhaps they are hard up?

Latest figures show increases in universities’ income, surpluses and reserves

UK universities’ income increased by £915m (2.7%) between 2015/16 and 2016/17, according to figures from the Higher Education Statistics Agency (HESA). During that time they made a surplus of £2.3bn and now have total reserves of £44.27bn.

The data also revealed an increase in capital expenditure, but that the proportion of money spent on staff had not improved. Compared to seven years ago, the percentage of expenditure spent on staff has fallen by 3.35%, but the percentage spent on capital expenditure has shot up by 34.9% over that period.

UCU said the figures made a mockery of universities’ claims that staff were a top priority. Staff in universities have seen their pay fall by around 20% since 2009,

So no. They just don’t care about spending money on staff.

I know that in some circles having shrinking staff salaries in a labor intensive industry would be thought to be good management. But those circles suck. They don’t care about the labour force.

And then there’s this lovely bit:

On Wednesday 25 April Senate endorsed the Faculty Leadership Team’s recommendation regarding a change to the School structure in the Faculty of Science and Engineering. Senate endorsed the proposal to reduce the number of Schools, from nine to two and name the new Schools the School of Engineering and the School of Natural Sciences.  In addition, the paper submitted to Senate included the following recommendations which did not require formal endorsement:

• the retention of the current School disciplines in ‘Departments’

• the formation of a Faculty Teaching College and a Faculty Research College.

Yes, a reshuffle that essentially is one part renaming and one part adding an extra level of management is a bold response to the many challenges we face! This makes me really confident that our senior leadership team is trying to acquire managerial cred for their next job…er…solve the tough problems…er…help US solve the tough problems of teaching and research in these troubled times.

Some Results On Student Evaluations

Erik posted about student evaluations last week and I took the opportunity to bring up the  famous Air Force Academy study. (the paper; an accessible blog post about it).

The setup is extraordinary:

Prior to the start of the freshman academic year, students take course placement exams in mathematics, chemistry, and select foreign languages. Scores on these exams are used to place students into the appropriate starting core courses (i.e., remedial math, Calculus I, Calculus II, etc.). Conditional on course placement, the USAFA registrar employs a stratified random assignment algorithm to place students into sections within each course/semester. The algorithm first assigns all female students evenly throughout all offered sections, then places male-recruited athletes, and then assigns all remaining students. Within each group (i.e., female, male athlete, and all remaining males), assignments are random with respect to academic ability and professor.12 Thus, students throughout their 4 years of study have no ability to choose their professors in required core courses. Faculty members teaching the same course use an identical syllabus and give the same exams during a common testing period. These institutional characteristics assure that there is no self-selection of students into (or out of) courses or toward certain professors.

They focused on math classes because the grades are highly normalised:

The integrity of our results depends on the percentage of points earned in core courses being a consistent measure of relative achievement across students. The manner in which student scores are determined at USAFAparticularly in the Math Department, allows us to rule out potential mechanisms for our results. Math professors grade only a small proportion of their own students’ exams, vastly reducing the ability of “easy” or “hard” grading professors to affect their students’ scores. All math exams are jointly graded by all professors teaching the course during that semester in “grading parties,” where Professor A grades question 1 and Professor B grades question 2 for all students taking the course. These aspects of grading allow us to rule out the possibility that professors have varying grading standards for equal student performance. Hence, our results are likely driven by the manner in which the course is taught by each professor.

In some core courses at USAFA, 5–10 percent of the overall course grade is earned by professor/section-specific quizzes and/or class participation. However, for the period of our study, the introductory calculus course at USAFA did not allow for any professor-specific assignments or quizzes. Thus, potential “bleeding heart” professors had no discretion to boost grades or to keep their students from failing their courses. For this reason, we present results in this study for the introductory calculus course and follow-on courses that require introductory calculus as a prerequisite.

This is really amazing! It’s pretty close to a highly controlled experiment!

They found a pretty strong effect from instructor quality:

The USAFA’s comprehensive core curriculum provides a unique opportunity to test how introductory course professors affect follow-on course achievement free from selection bias. The estimate of 2 Var (lj 1) is shown in row 2, column 2 of table 4 and indicates that introductory course professors significantly affect follow-on course achievement.19 The variance in follow-on course value-added is estimated to be 0.0025 (SD p 0.050). The magnitude of this effect is roughly equivalent to that estimated in the contemporaneous course and indicates that a one standard-deviation change in introductory professor quality results in a 0.05-standard-deviation change in follow-on course achievement.

The striking bit (for me) is their examination of student evaluation and grades:

Next, we examine the relationship between student evaluations of professors and student academic achievement as in Weinberg, Hashimoto, and Fleisher (2009). This analysis gives us a unique opportunity to compare the relationship between value-added models (currently used to measure primary and secondary teacher quality) and student evaluations (currently used to measure postsecondary teacher quality).

…In column 1, results for contemporaneous value-added are positive and statistically significant at the.05 level for scores on all six student evaluation questions. In contrast, results in column 2 for follow-on course value-added show that all six coefficients are negative, with three significant at the .05 level and three significant at the .10 level .Since proposals for teacher merit pay are often based on contemporaneous teacher value-added, we examine rank orders between our professor value-added estimates and student evaluation scores. We compute rank orders of career average student evaluation data for the question, “The instructor’s effectiveness in facilitating my learning in the course was,” by professor… As an illustration, the calculus professor in our sample 1 who ranks dead last in deep learning ranks sixth and seventh best in student evaluations and contemporaneous value-added, respectively.

Our findings show that introductory calculus professors significantly affect student achievement in both the contemporaneous course being taught and the follow-on related curriculum. However, these methodologies yield very different conclusions regarding which professors are measured as high quality, depending on the outcome of interest used. We find that less experienced and less qualified professors produce students who perform significantly better in the contemporaneous course being taught, whereas more experienced and highly qualified professors produce students who perform better in the follow-on related curriculum. Owing to the complexities of the education production function, where both students and faculty engage in optimizing behavior, we can only speculate as to the mechanism by which these effects may operate. Similar to elementary and secondary school teachers, who often have advance knowledge of assessment content in high-stakes testing systems, all professors teaching a given course at USAFA have an advance copy of the exam before it is given. Hence, educators in both settings must choose how much time to allocate to tasks that have great value for raising current scores but may have little value for lasting knowledge.

And the key bit:

Regardless of how these effects may operate, our results show that student evaluations reward professors who increase achievement in the contemporaneous course being taught, not those who increase deep learning. Using our various measures of teacher quality to rank-order teachers leads to profoundly different results. Since many U.S. colleges and universities use student evaluations as a measurement of teaching quality for academic promotion and tenure decisions, this finding draws into question the value and accuracy of this practice

Now, in the comment thread, people pointed out that this result, while strong, was narrow (restricted to fairly low level math and “adjacent” courses). These facts are related, of course. By focusing on the data with the fewest confounders, they narrowed the scope but increased the strength. However, there have been subsequent students. For example, “Evaluating students’ evaluations of professors“:

The empirical analysis is based on data for one enrollment cohort of undergraduate students at Bocconi University, an Italian private institution of tertiary education offering degree programs in economics, management, public policy and law. We select the cohort of the 1998/1999 freshmen because it is the only one available where students were randomly allocated to teaching classes for each of their compulsory courses.
The students entering Bocconi in the 1998/1999 academic year were offered 7 different degree programs but only three of them attracted enough students to require the splitting of lectures into more than one class: Management, Economics and Law&Management. Students in these programs were required to take a fixed sequence of compulsory courses that span over the first two years, a good part of their third year and, in a few cases, also their last year.

The exam questions were also the same for all students (within degree program), regardless of their classes. Specifically, one of the teachers in each course (normally a senior faculty member) acted as a coordinator, making sure that all classes progressed similarly during the term and addressing problems that might have arisen. The coordinator also prepared the exam paper, which was administered to all classes. Grading was delegated to the individualteachers, each ofthem marking the papers ofthe students in his/her own class. The coordinator would check that the distributions were similar across classes but grades were not curved, neither across nor within classes.

They also looked at evaluation/grade correlations:

In this section we investigate the relationship between our measures of teaching effectiveness and the evaluations teachers receive from their students. We concentrate on two core items from the evaluation questionnaires, namely overall teaching quality and the overall clarity of the lectures.

The key hit:

Our benchmark class effects are negatively associated with all the items that we consider, suggesting that teachers who are more effective in promoting future performance receive worse evaluations from their students. This relationship is statistically significant for all items (but logistics), and is of sizable magnitude. For example, a one-standard deviation increase in teacher effectiveness reduces the students’ evaluations of overall teaching quality by about 50% of a standard deviation. Such an effect could move a teacher who would otherwise receive a median evaluation down to the 31st percentile of the distribution. Effects of slightly smaller magnitude can be computed for lecturing clarity.

Finally, I did peek at a 2017 meta-analysis which overrides earlier meta-analyses. The abstract:

Student evaluation of teaching (SET) ratings are used to evaluate faculty’s teaching effectiveness based on a widespread belief that students learn more from highly rated professors. The key evidence cited in support of this belief are meta-analyses of multisection studies showing small-to-moderate correlations between SET ratings and student achievement (e.g., Cohen, 1980, 1981; Feldman, 1989). We re-analyzed previously published meta-analyses of the multisection studies and found that their findings were an artifact of small sample sized studies and publication bias. Whereas the small sample sized studies showed large and moderate correlation, the large sample sized studies showed no or only minimal correlation between SET ratings and learning. Our up-to-date meta-analysis of all multisection studies revealed no significant correlations between the SET ratings and learning. These findings suggest that institutions focused on student learning and career success may want to abandon SET ratings as a measure of faculty’s teaching effectiveness.

I mean, it’s brutal:

In combination, our new up-to-date meta-analyses based on nearly 100 multisection studies, as well as our re-analyses of the previous meta-analyses make it clear that the previous reports of “moderate” and “substantial” SET/learning correlations were artifacts of small size study effects. The best evidence − the meta-analyses of SET/learning correlations when prior learning/ability are taken into account − indicates that the SET/learning correlation is zero. Contrary to a multitude of reviews, reports, as well as self-help books aimed at new professors (a few of them quoted above), the simple scatterplots as well as more sophisticated meta-analyses methods indicate that students do not learn more from professors who receive higher SET ratings.

And student evaluatons are a substantial part of the Teaching Evaluation Framework.

So that’s really bad!

When we throw in gender (and other) biases, it seems clear we have a huge problem.

Sometimes There’s an Appearance of a Conflict of Interest Because There’s an Actual Conflict

So we’re about to go on strike. Again. To defend our pensions. Again. This is the big one…moving from a defined benefit to defined contribution. This shifts all the risk onto us, from the universities and will definitely involve big cuts, sometimes (if your retirement timing is bad) huge cuts to our pensions.

This sucks.

It seems that Oxford and Cambridge have played a large role in fucking the rest of us. Good show, you jerks. I think students should get compensation, after all, the unis dock our pay!

The union has been busy otherwise. They released (in conjunction with the Guardian) the results of a lot of Freedom of Information requests. Here’s the headline result:

The vice-chancellor was either on the committee that sets their pay or allowed to attend its meetings at 95% of UK universities. Three-quarters of institutions would not release full minutes of the pay committee’s meeting.

Oh, and surprise surprise, VC compensation has also risen…dramatically:

University vice-chancellors have enjoyed huge pay rises in recent years. The average pay (excluding pensions) for vice-chancellors in 2005/06 was £165,105. Over the next decade it increased by 56.2% to £257,904 in 2015/16. UCU said its report with the latest figures on senior pay and perks will be released shortly.

In case you were wondering, I’m not allowed to attend promotion or pay meetings about myself. Or sabbatical deciding meetings.

Now, would these massive increases happened anyway? Maybe! Why not! But it isn’t just an appearance of a conflict of interest…it really is a conflict of interest for the VC to attend (even) a meeting where their pay is discussed.

I checked out the findings to see how Manchester did and sadly:

Vice-chancellor/head of institution is a member of the remuneration committee? Although not a member, vice-chancellor/head of institution attends remuneration committee meetings? Remuneration committee minutes received 2016/17? Rationale for refusal to share minutes? Minutes redacted 2016/17?
The University of Manchester No response No response No response No response No response

That’s not the transparency I would hope for from my institution.

Needless to say, one would have thought, minimal proper governance dictates:

  1. VCs should not be involved in anyway with the committees setting their compensation.
  2. Their compensation shouldn’t be growing so much faster (or…faster) than the rest of us, barring very specific and exceptional circumstances.
  3. The facts about their membership and attendance and, indeed, the rationale for their compensation, should be public (with minimal redaction for properly private bits).
  4. This should all be watchdogged like hell.

This is a terrible look, both internally and externally, for the universities. That’s something the VCs are directly responsible for.


Yet Another Strike Day

A university is a community: staff and students, graduates and families, people in the town and people around the world. The university is its people. Buildings are more iconic; the physical environment conditions all our experiences and memories; but a university’s mission is the creation and dissemination of knowledge, and all knowledge exists in the people and communities that know it.

My union, the University and College Union (UCU), is in dispute with a large number of universities, and my local and my university (the University of Manchester) are part of that dispute. We are on strike today.

Strikers stand by the UCU banner with the slogans "Knowledge is Power" and "Unity is Strength".

Strikers by the UCU banner.

The Reason

It’s a pay dispute. The capsule UCU summary:

The dispute has arisen following a pay offer of just 1.1% from the universities’ employers, the Universities and Colleges Employers Association. UCU said universities could afford to pay more and the latest offer did little to address the real terms pay cut of 14.5% that its members have suffered since 2009. The squeeze on staff salaries comes despite vice-chancellors enjoying a 6.1% pay hike.

The union has also called for universities to commit to closing the gender pay gap and reducing the proportion of staff on casual and zero-hour contracts. On average, female academics across the sector are paid £6,103 per year less than male counterparts while 49% of university teachers are on insecure contracts.

Since 2010 the amount spent on staff by universities as a percentage of total income has dropped by 3%. However the total of cash in reserves has rocketed by 72% to stand at over £21bn.

(It started as a 1% offer. We pushed back. The Universities came back with a 1.1% offer. The faith is not good here.)

Note that we are in the midst of a £1 billion building spree  (after spending £750 million since 2004 on 10 buildings some of which are already falling apart and were hideous and misdesigned in the first place.

Now, I know not all money is the same, yadda yadda yadda. However, where is the billion pound effort to improve staff? Staff development, better pay, better recruitment…these are things that are not part of a giant, well funded master plan. Instead we get stagnant pay, increasing work loads, onerous busywork, metrics, targets, punitive measures, pensions slashed, etc. etc. etc. Less academic freedom and more centralisation of functions for the sake of centralisation.

The administrations should be actively figuring out how to “build up” staff which includes building up staff pay instead of having every drop wrung from them by disruptive action. Until they do, they are failing at their jobs.

The Tactics

This is part of a second round of targeted strikes after a national two day strike. Each local is picking a day designed to be maximally disruptive for their university. We are focused on exam boards (where we make final decisions on student grades). Oxford is targeting honorary degree and graduation ceremonies.

This is serious stuff. Disruption is disruptive. Student degrees may be delayed, complicating their job starts. Ceremonies that normally generate cherished memories are marred. It’s not something the union takes lightly. But there is really little choice. The administrations are not negotiating with us. Nor are they performing the basic function of nurturing the university. Neglecting staff issues is neglecting the foundations of the university.

It’s disruptive to staff. It’s not a “day off”. People are picketing. There’s work that’s delayed or created. Our pay is docked by the university. It costs us to go on strike (though often that is returned as a negotiation sweetener). It’s emotionally draining both to be faced by this situation and to engage in the action.

Simon Harper et al picketing one of the entrances to University Place.

Simon Harper et al picketing one of the entrances to University Place.

It is in the power of the adminstrations to prevent this disruption, to have prevented the risk of it. Coming in with 1% and “moving” to 1.1% isn’t a opening move in a sincere negotiation. Having no long range plan to address wage declines is not the background of this “discussion”.

As I often do, I look to Martin Luther King’s Letter from a Birmingham Jail

You may well ask: “Why direct action? Why sit ins, marches and so forth? Isn’t negotiation a better path?” You are quite right in calling for negotiation. Indeed, this is the very purpose of direct action. Nonviolent direct action seeks to create such a crisis and foster such a tension that a community which has constantly refused to negotiate is forced to confront the issue. It seeks so to dramatize the issue that it can no longer be ignored.

MLK was fighting a much, much more serious fight with dramatically greater stakes. but his description of direct action is applicable in a wide range of circumstances, including labour disputes.

(Read the whole thing! Read his books! It repays the effort many times in many circumstances. It is not merely of historical interest. King’s thinking is live, complex, theoretically interesting, and practically helpful.)

What you can do if you aren’t staff

Students and families can show solidarity. Write your university in support. Call for administrations to look toward staff development as the central part of their job, worthy of big efforts and active work. Stop and say hi to picketers!

Place the blame for the disruption where it belongs, with the administrations. Turn the situation into an opportunity to stand with others in your community. Instead of a marred experience, make it a proud moment.