Down in flames

August 25, 2018

I first encountered Larry Niven’s story/essay “Down in Flames” in the collection N-Space in high school. This was after I’d read Ringworld and most of Niven’s Known Space stories, so by the time I got to “Down in Flames” I had the context to get it. (You can read the whole thing for free here.)

Here’s the idea, from near the start:

On January 14, 1968, Norman Spinrad and I were at a party thrown by Tom & Terry Pinckard. We were filling coffee cups when Spinny started this whole thing.

“You ought to drop the known space series,” he said. “You’ll get stale.” (Quotes are not necessarily dead accurate.) I explained that I was writing stories outside the “known space” history, and that I would give up the series as soon as I ran out of things to say within its framework. Which would be soon.

“Then why don’t you write a novel that tears it to shreds? Don’t just abandon known space. Destroy it!”

“But how?” (I never asked why. Norman and I think alike in some ways.)

The rest of the piece is just working out the details.

“Down in Flames” brain-wormed me. Other than Ray Bradbury’s “A Sound of Thunder” I doubt if there is another short story I’ve read as many times. Mike once described the act of building something complex and beautiful and then destroying it as “magnificently profligate”, and that’s the exact quality of “Down in Flames” that appeals to me.

I also think it is a terrific* exercise for everyone who is a scientist, or who aspires to be one.

* In both the modern sense of “wonderful” and the archaic sense of “causing terror”.

Seriously, try it. Grab a piece of paper (or open a new doc, or whatever) and write down the ideas you’ve had that you hold most dear. And then imagine what it would take for all of them to be wrong. (When teams and organizations do this for their own futures, it’s called a pre-mortem, and there’s a whole managerially-oriented literature on it. I’d read “Down in Flames” instead.)

It feels like this! Borrowed from here.

Here are some questions to help you along:

  • Which of your chains of reasoning admit more than one end-point? If none of them might lead other places, then either you are the most amazing genius of all time (even Newton and Einstein made mistakes), or you are way behind the cutting edge, and your apparent flawlessness comes from working on things that are already settled.
  • If there is a line of evidence that could potentially falsify your pet hypothesis, have you checked it? Have you drawn any attention to it? Or have you gracefully elided it from your discussions in hopes that no-one will notice, at least until after you’re dead?
  • If there’s no line of evidence that could falsify your pet hypothesis, are you actually doing science?
  • Which of your own hypotheses do you have an emotional investment in?
  • Are there findings from a rival research team (real or imagined) that you would not be happy to see published, if they were accurate?
  • Which hypotheses do you not agree with, that you would be most dismayed to see proven correct?

[And yes, Karl, I know that according to some pedants hypotheses are never ‘proven’. It’s a theoretical exercise already, so just pretend they can be!]

I’ll close with one of my favorite quotes, originally published in a couple of tweets by Angus Johnson in May of 2017 (also archived here):

If skepticism means anything it means skepticism about the things you WANT to be true. It’s easy to be a skeptic about others’ views. Embracing a set of claims just because it happens to fit your priors doesn’t make you a skeptic. It makes you a rube, a mark, a schnook.

So, don’t be that rube. Burn down your house of ideas – or at least, mentally sift through the rubble and ashes and imagine how it might have burned down. And then be honest about that, minimally with yourself, and ideally with the world.

If you’re a true intellectual badass, blog the results. I will. It’s not fair to give you all homework – painful homework – and not take the medicine myself, so I’m going to do a “Down in Flames” on my whole oeuvre in the next a future post. Stay tuned!

Advertisements

I’ve been on Twitter since April 2011 — nearly six years. A few weeks ago, for the first time, something I tweeted broke the thousand-retweets barrier. And I am really unhappy about it. For two reasons.

First, it’s not my own content — it’s a screen-shot of Table 1 from Edwards and Roy (2017):

c49rdmlweaaa4if

And second, it’s so darned depressing.

The problem is a well-known one, and indeed one we have discussed here before: as soon as you try to measure how well people are doing, they will switch to optimising for whatever you’re measuring, rather than putting their best efforts into actually doing good work.

In fact, this phenomenon is so very well known and understood that it’s been given at least three different names by different people:

  • Goodhart’s Law is most succinct: “When a measure becomes a target, it ceases to be a good measure.”
  • Campbell’s Law is the most explicit: “The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”
  • The Cobra Effect refers to the way that measures taken to improve a situation can directly make it worse.

As I say, this is well known. There’s even a term for it in social theory: reflexivity. And yet we persist in doing idiot things that can only possibly have this result:

  • Assessing school-teachers on the improvement their kids show in tests between the start and end of the year (which obviously results in their doing all they can depress the start-of-year tests).
  • Assessing researchers by the number of their papers (which can only result in slicing into minimal publishable units).
  • Assessing them — heaven help us — on the impact factors of the journals their papers appear in (which feeds the brand-name fetish that is crippling scholarly communication).
  • Assessing researchers on whether their experiments are “successful”, i.e. whether they find statistically significant results (which inevitably results in p-hacking and HARKing).

What’s the solution, then?

I’ve been reading the excellent blog of economist Tim Harford, for a while. That arose from reading his even more excellent book The Undercover Economist (Harford 2007), which gave me a crash-course in the basics of how economies work, how markets help, how they can go wrong, and much more. I really can’t say enough good things about this book: it’s one of those that I feel everyone should read, because the issues are so important and pervasive, and Harford’s explanations are so clear.

In a recent post, Why central bankers shouldn’t have skin in the game, he makes this point:

The basic principle for any incentive scheme is this: can you measure everything that matters? If you can’t, then high-powered financial incentives will simply produce short-sightedness, narrow-mindedness or outright fraud. If a job is complex, multifaceted and involves subtle trade-offs, the best approach is to hire good people, pay them the going rate and tell them to do the job to the best of their ability.

I think that last part is pretty much how academia used to be run a few decades ago. Now I don’t want to get all misty-eyed and rose-tinted and nostalgic — especially since I wasn’t even involved in academia back then, and don’t know from experience what it was like. But could it be … could it possibly be … that the best way to get good research and publications out of scholars is to hire good people, pay them the going rate and tell them to do the job to the best of their ability?

[Read on to Why do we manage academia so badly?]

References

Bonus

Here is a nicely formatted full-page version of the Edwards and Roy table, for you to print out and stick on all the walls of your university. My thanks to David Roberts for preparing it.

Many thanks to everyone who played pin-the-skull-on-the-carnivore. The answers are down at the bottom of this post, so if you’ve just arrived here and want to take the challenge, go here before you scroll down.

To fill up some space, let me point out how crazy variable the skulls of black bears, Ursus americanus, are.

My bear skull - left lateral reversed

Here’s the one I helped dig up, missing the occipital region. Note the double inflection in the dorsal outline that separates the forehead from both the snout and top of the head, and the way the nasal bones stick out at a very different angle from the maxilla.

Page Museum black bear skull

Here’s the skull of a black bear from the La Brea tar pits, in the Page Museum in L.A. I don’t know if this one was female or juvenile or what, but the dorsal margin of the skull is one mostly-smooth curve from occiput almost to incisors, with the nasals scarcely deviating at all. Lest you think these differences were caused by evolutionary change rather than intraspecific variation, similar “roundhead” bear skulls from modern times are here and here and near the bottom of this page.

It’s this variability that first got me thinking about doing the Carnivore Skull Challenge. I saw a couple of photos of skulls of wolverines, and except for having carnassial cheek teeth instead of flatter premolars and molars, the wolverine skulls look like they could fit right into the span of black bear skull variability (in shape; obviously they’re not nearly as big). Then I saw a hyena skull and thought that it wasn’t that far off from a wolverine either. A little more searching for plausible distractors and I was all set.

Here are the answers, by the way:

Carnivore skull challenge - answersIt’s kind of ironic, then, that the first two people to venture identifications picked out the black bear right away. In the very first comment, Dean got it almost all right except for swapping the seal and the fossa. Dean was also the first to get all of the skulls correctly identified, albeit on his second pass. Markus Bühler (of Cthulhu-sculpting fame) was the first to get them all the first time. Tom Nutter, our own Darren Naish, and microecos Neil also aced the test, although in light of the Page Museum bear skull shown above, I was amused to see Darren’s “D: Bear. Because forehead.” I guess it’s one of those presence-of-forehead-means-bear, absence-of-forehead-does-not-rule-out-bear things that logicians are always going on about.

I was really happy to see people getting the wolverine and hyena mixed up, because they really do look strikingly similar to me. It’s almost like hyena + bear = wolverine.

Brian Engh asked on Facebook when I was going to do one for sauropods. Patience, good sir! It’s on my to-do list.

Finally, speaking of bear skulls, you can get a sweet tiny bronze one with a hinged jaw as part of this already successful Kickstarter, or from Fireandbone.com once the Kickstarter ends.

Something very different, and very unexpected, tomorrow.

Carnivore skull challenge

In this image I have assembled photos of skulls (or casts of skulls) of six extant carnivores. I exclusively used photos from the Skulls Unlimited website because they had all the taxa I wanted, lit about the same and photographed from similar angles. The omission of scale indicators is deliberate.

Your mission, should you choose to accept it, is to match these skulls with the animals they came from. Here are their currently-understood hierarchical relationships, scientific names, and common names (aside: I know this is ugly, is there a way to make nested tables in WordPress?).

Carnivora

– – Herpestoidea

– – – – Eupleridae

– – – – – – Fossa, Cryptoprocta ferox

– – – – Hyaenidae

– – – – – – Brown hyena, Hyaena brunnea

– – Arctoidea

– – – – Ursoidea

– – – – – – American black bear, Ursus americanus

– – – – Musteloidea

– – – – – – European badger, Meles meles

– – – – – – Wolverine, Gulo gulo

– – – – Pinnipedia

– – – – – – Mediterranean monk seal,  Monachus monachus

If you accept the challenge, leave your guesses as comments below, but only if you’ve played fair–no checking websites, references, or your own skull collection! Don’t worry about being wrong, I freely admit that I would have flunked this bigtime if anyone else had inflicted it on me. I decided to set up this challenge after I noticed the striking similarity between two of these critters in particular; I’ll tell you which two when I post the reveal in a day or two.