(Update: I’ve posted more on this subject here)
In my professional life, I deal with metrics on a regular basis. Computer and network security in particular is (in my opinion) struggling with the transition from a qualitative to quantitative field. And hey, at heart I’m a quant: I find it easier to improve aspects of my life that I can measure, be it miles per gallon, number of repetitions, pieces of sheet rock put up, dollars, calories, percent alcohol by volume, countries taken over by robot, etc. These are numbers that I can measure and track and use as a kind of mirror to study some aspect of my life and improve it.
One of my goals for this year is to improve my writing. I have plenty of ideas for that, but it occurs to me that I really know very few metrics on the subject. My first thought is that it’s simply a non-measurable activity. That may be the truth.
And yet… there are measurable things, things that some writers (*ahem*) already track or obsess over. And the intriguing thing is, there’s a kind of donut hole at work.
In the beginning, there is the word count. On a daily, monthly, and yearly basis, a writer can keep track of sheer words-on-page, bytes-wasted productivity. The derivative and the integral are both interesting: A small increase in words-per-day means producing fiction faster, and of course the total gives the volume of work produced. When editing, that can turn into words-removed, especially if you subscribe to Stephen King’s 10% rule (to wit, 2nd Draft = 1st Draft – 10%)
If you look at how writers can focus on and work with word count, you can see some of what I’m getting at. You can track it, graph it, whatever, and it gives you regular feedback about how you’re doing and how you need to do.
So far so good. Then, once the work is finished, there are metrics (sort of): how many stories I’ve finished, how many times a story or novel has been submitted, how long it’s been under consideration. Once published, there are all KINDS of metrics to obsess over: size of advance, number of downloads/purchases, Amazon ranking, etc. But I said “sort of” above because these mostly aren’t the sort of thing a writer can control. Tracking these may be gratifying or worrying, but isn’t necessarily useful.
But what happened to that middle ground? Is there really nothing I can usefully measure between word count and # of stories finished? I can sort of come up with some facetious ones: number of adverbs deleted, number of characters killed.
There may be useful metrics just above the word-count level in some of the things that come up in metrics for technical writing. Words per sentence, words/lines/sentences per paragraph, estimates of reading level (based, I believe, on scores for vocabulary choices?), these are all geared toward ease of comprehension. I don’t know how many of those are useful to track in general, but specific things might be helpful in response to feedback: If someone comes back and says that your story is very “dense” then it might be useful to try to bring an average-words-per-sentence metric down for that story. If that’s a common comment across a number of stories, maybe it’s useful to keep an eye on in general.
I draw a complete blank when it comes to plot-level or story-level analysis. I don’t even know where to start, or even what I want to know! Should I be concerned about number of characters? Number of subplots? Not all characters and subplots are created equal: is there a way to weight them? Maybe there are things I can track at the outline stage that might help.
If I can’t come up with actual things to measure, maybe I can still come up with a useful approach from the quantitative world, one that doesn’t come up with a number per se, but can boil down to a simple trackable answer: true/false, good/bad/neither. I’m thinking in particular of an approach I learned from Andrew Jaquith’s book “Security Metrics: Replacing Fear, Uncertainty, and Doubt” of breaking down the problem into falsifiable hypotheses and sub-hypotheses: You start off with “My computer is as secure as it can be”, then ask yourself how you could disprove that. You go on to, maybe, “My computer will not get viruses” and “My programs are all up to date” and maybe “My computer is not running any extraneous programs.”[1] The point is, you break things down until you have a set of hypotheses which can, through measurement, be disproven. If you fail to disprove any of them, you turn around and treat the overarching hypothesis as true enough.
Maybe I can break down a plot analysis, not to a set of trackable numbers, but to a set of statements that I can affirm or deny. (I’ll leave open the question of whether there are useful hypotheses that can be disproven for the effectiveness of a work of fiction) But I don’t really know what they might look like. How do I break down “This plot is too complex” into a set of supporting statements in a useful way so that, at the bottom level, I can go back into the text and reliably change a “deny” into a “confirm”?
So I come to you in a state of confusion: Are there useful metrics to be discovered? Should I try to find and track them? Or is it (as I suspect) folly even to be thinking along these lines: Am I trying to quantify something inherently unmeasurable? Will I damage my fiction by thinking about it this way?
What do you think? I have dark brown eyes, which my grandmother says is because I’m so full of shit. But just how full of shit am I?
[1] This latter gets into the concept of attack surfaces, which IMO doesn’t really lead to useful (falsifiable) hypotheses, but I think that there’s a blog post there about how to use it to construct mystery plots. If you’re curious about it, I’ll go into it in comments.
[Edited To Add: A number of people come to this page looking for the source of the quotation in the title. Hi! Thanks for stopping by! The source of that particular formulation is unclear, but I have seen versions of it attributed to both Lord Kelvin and James Harrington. Only Harrington actually brings up “control”, as I recall.]
John P. Murphy 
Hey John,
The 10% Solution (Ken Rand) appears to have some useful metrics tilted toward the lower levels. I picked it up based on a reference from Miranda Suri, but haven’t read it yet. Looks like it pretty closely follows King’s Rule.
Also Writing Fiction for Dummies (Randy Ingermanson) has a chapter on “Ten Steps to Analyzing Your Story”. I despise the name/premise of these books, but if you can get over that, it’s actually pretty good.
There will always be some level of subjectivity in analyzing things like plot and structure, but it doesn’t have to be completely wide open.
Oh, excellent. I’d heard of Rand’s book (probably when you did) but also do not have it yet. It’s cheap, I should get it.
Did not even know there was a … For Dummies book on writing fiction. Maybe I’ll flip through it next time I go to Borders. I’m also not a fan of the name. I’m always tempted to bring a bunch of “Is” stickers to a display of them…
As long as I’m the one assigning the numbers, then some of the subjectivity is irrelevant. The big issue, I think, is the meaning one assigns to them: there is no “right” set of scores for “good fiction”, but there might be “too high” or “too low” for a particular story, and that might help me when grappling with a piece that doesn’t work but I don’t know why.
Yea, I can appreciate this problem, as I deal with metrics at work on a regular basis too. I haven’t heard of these two books (thanks, Micah) but may look at them. My biggest problem is motivating myself, but maybe if I set more realistic, micro-goals, I will be more successful.
And I don’t know you as well as your grandmother does, so I’ll have to side with her on the brown eyes thing.
Micro-goals are useful, yeah. I’m terrible at setting reasonable ones, but on occasion I find one that works. The best ones, for me, are time-wise: I frequently tell myself that I just have to spend the next 15 minutes writing with the wi-fi off, then if I want to stop writing I can. (Usually this is for academic writing, not fiction)
And 😛
You make a good point about that middle ground between words written and stories published. It sort of seems like what you’re getting at is: okay, I’m writing more (or less) words, but are they *better*? Are they being put to better use? Am I improving? That does seem hard to measure. Maybe it’s looking at things like how often you get crits back that are largely positive versus largely negative? Or, how many drafts it takes to get a story into shape…or even how much more often your starting ideas are truly creative.
Bit abstract, I realize…but you got me thinking.
That’s it exactly: how do I measure progress between “I’m writing the right number of words” and “I’m shopping finished stories around”? The crits idea is interesting. In fact, it reminded me that I’m a member of an online workshop (that I don’t really use anymore) that includes ratings of different things like “Professionalism of Writing”, “Setting”, etc. Everyone uses different criteria (and not everyone uses the scores), so it’s hard to compare one rating to the next, but if you get enough scores for enough pieces, it would be good to see the numbers going up over time.
I encountered the above when looking for the source of the Control/Measure proverb.
My direct ideas for you are
a) sentence length: 20 average, broader distribution better, short ones at beginnings and ends of paragraphs. Can do same for paragraphs, except where there is a lot of dialogue. In the 50s there was a “fog index” promoted as useful for tech writers.
b) number of words over 3 syllables — none is too babyish, lots are too pedantic, pick a number that reflects expected audience. Counting them and words per sentence has added benefit of making you go over your work, unless you cheat by dividing total words by counted periods (find and replace will tell you how many you replaced).
c) amount of time getting feedback from others (precrits?), either via email or direct (skype counts as direct).
As for brown eyes, they can represent fertile earth as much as shit. Sounds like grandma has a problem; what color are hers? Lucky she can make fun of you to (about) your face. Some people are sneakier, tell you you’re OK or even great, and bite you when you’re not looking.
Measure/control is more important than just in writing or machinery — they are calls for balance, compromise (a balance you may not like) and seeing all sides even as they change (there aren’t always just two, and they aren’t always static). In short, they create uncertainty and require attention/analysis, thus opposing commitment to positions, often extreme, as dictated by intuition, gut feel and such drivers that cannot be crossexamined. A novel that dealt with this uncertainty might be well received, even as the committed readers squirm.
ALG
Hi Allan, thanks for stopping by!
Excellent suggestions, thank you. That fog index in particular sounds interesting. One of the big things to focus only on the things I can *actually* control when measuring. And you’re right — anything that encourages a writer to go word-by-word through the work will be helpful.
As for the source of “if you can’t measure it, you can’t control it”, I’ve seen various versions of that quote attributed to one James Harrington. Hope that helps!
I go for straight numbers because it’s something that I can easily count without spending too much time on. So for a draft, it’s a question of wordcount and for editing I count scenes polished. For the novel, I designated four passes (structure and sense, location, tension, and theme) and I tick off each scene as I go over it. Some scenes are easy and some take days, especially on the first pass. But I figure it evens out and to be honest, writing to word count works the same (1,000 words might take an hour, might take six).
Underlying this is the belief that if I continue, I will get better. I don’t consider myself a good judge of my own ability and I’m not fond of trying to apply “quality” metrics (the correct words per scene, beats, whatever). But I’m interested and engaged and reading on the subject so if I’m continuing to read and write voraciously and then edit that writing, it seems that my writing would logically improve. The probability of my ability stagnating is there but remote if I continue to put regular effort in, so doing it by the numbers becomes a valid metric for me.
I think this is true but I also believe in faeries so perhaps I’m not a reliable source.
There’s nothing wrong with believing in faeries 🙂 (Except that this damnable auto-correct won’t actually let me write the word “faerie” on the first try…)
There is absolutely no substitute for practice, I agree. Part of what makes me nervous about just sitting and practicing, though, is my worry of ingraining bad habits. This isn’t even coming from my experience writing, by the way, but from my execrable violin playing. I’ve had to repeatedly unlearn bad habits that I acquired sitting and sawing away. A lot of that is the lack of feedback (the cats just run away. Philistines.) and my own tin ear for music, though, and I’ve been fortunate in being able to get good critiques on my writing.