(Update: I’ve posted more on this subject here)
In my professional life, I deal with metrics on a regular basis. Computer and network security in particular is (in my opinion) struggling with the transition from a qualitative to quantitative field. And hey, at heart I’m a quant: I find it easier to improve aspects of my life that I can measure, be it miles per gallon, number of repetitions, pieces of sheet rock put up, dollars, calories, percent alcohol by volume, countries taken over by robot, etc. These are numbers that I can measure and track and use as a kind of mirror to study some aspect of my life and improve it.
One of my goals for this year is to improve my writing. I have plenty of ideas for that, but it occurs to me that I really know very few metrics on the subject. My first thought is that it’s simply a non-measurable activity. That may be the truth.
And yet… there are measurable things, things that some writers (*ahem*) already track or obsess over. And the intriguing thing is, there’s a kind of donut hole at work.
In the beginning, there is the word count. On a daily, monthly, and yearly basis, a writer can keep track of sheer words-on-page, bytes-wasted productivity. The derivative and the integral are both interesting: A small increase in words-per-day means producing fiction faster, and of course the total gives the volume of work produced. When editing, that can turn into words-removed, especially if you subscribe to Stephen King’s 10% rule (to wit, 2nd Draft = 1st Draft – 10%)
If you look at how writers can focus on and work with word count, you can see some of what I’m getting at. You can track it, graph it, whatever, and it gives you regular feedback about how you’re doing and how you need to do.
So far so good. Then, once the work is finished, there are metrics (sort of): how many stories I’ve finished, how many times a story or novel has been submitted, how long it’s been under consideration. Once published, there are all KINDS of metrics to obsess over: size of advance, number of downloads/purchases, Amazon ranking, etc. But I said “sort of” above because these mostly aren’t the sort of thing a writer can control. Tracking these may be gratifying or worrying, but isn’t necessarily useful.
But what happened to that middle ground? Is there really nothing I can usefully measure between word count and # of stories finished? I can sort of come up with some facetious ones: number of adverbs deleted, number of characters killed.
There may be useful metrics just above the word-count level in some of the things that come up in metrics for technical writing. Words per sentence, words/lines/sentences per paragraph, estimates of reading level (based, I believe, on scores for vocabulary choices?), these are all geared toward ease of comprehension. I don’t know how many of those are useful to track in general, but specific things might be helpful in response to feedback: If someone comes back and says that your story is very “dense” then it might be useful to try to bring an average-words-per-sentence metric down for that story. If that’s a common comment across a number of stories, maybe it’s useful to keep an eye on in general.
I draw a complete blank when it comes to plot-level or story-level analysis. I don’t even know where to start, or even what I want to know! Should I be concerned about number of characters? Number of subplots? Not all characters and subplots are created equal: is there a way to weight them? Maybe there are things I can track at the outline stage that might help.
If I can’t come up with actual things to measure, maybe I can still come up with a useful approach from the quantitative world, one that doesn’t come up with a number per se, but can boil down to a simple trackable answer: true/false, good/bad/neither. I’m thinking in particular of an approach I learned from Andrew Jaquith’s book “Security Metrics: Replacing Fear, Uncertainty, and Doubt” of breaking down the problem into falsifiable hypotheses and sub-hypotheses: You start off with “My computer is as secure as it can be”, then ask yourself how you could disprove that. You go on to, maybe, “My computer will not get viruses” and “My programs are all up to date” and maybe “My computer is not running any extraneous programs.” The point is, you break things down until you have a set of hypotheses which can, through measurement, be disproven. If you fail to disprove any of them, you turn around and treat the overarching hypothesis as true enough.
Maybe I can break down a plot analysis, not to a set of trackable numbers, but to a set of statements that I can affirm or deny. (I’ll leave open the question of whether there are useful hypotheses that can be disproven for the effectiveness of a work of fiction) But I don’t really know what they might look like. How do I break down “This plot is too complex” into a set of supporting statements in a useful way so that, at the bottom level, I can go back into the text and reliably change a “deny” into a “confirm”?
So I come to you in a state of confusion: Are there useful metrics to be discovered? Should I try to find and track them? Or is it (as I suspect) folly even to be thinking along these lines: Am I trying to quantify something inherently unmeasurable? Will I damage my fiction by thinking about it this way?
What do you think? I have dark brown eyes, which my grandmother says is because I’m so full of shit. But just how full of shit am I?
 This latter gets into the concept of attack surfaces, which IMO doesn’t really lead to useful (falsifiable) hypotheses, but I think that there’s a blog post there about how to use it to construct mystery plots. If you’re curious about it, I’ll go into it in comments.
[Edited To Add: A number of people come to this page looking for the source of the quotation in the title. Hi! Thanks for stopping by! The source of that particular formulation is unclear, but I have seen versions of it attributed to both Lord Kelvin and James Harrington. Only Harrington actually brings up “control”, as I recall.]