Yesterday’s excellent post by PJ Parrish about the first pages of this year’s nominees for the Edgar Allan Poe Mystery Award brought me back to the year when I served as a judge for best novel of the year. I looked at it an an honor–a rite of passage, of sorts, in the same vein as jury duty. I would set aside hundreds of hours of my life over the coming year as a means of paying tribute other writers, in service to this artform that I love so much.
Of necessity, much of the process is veiled in secrecy. As such, I have no idea if my judging experience bears any resemblance to that of any other review committee. And, to honor commitments I made to keep the process quiet, I won’t just keep titles to myself, but I won’t even mention the year in which I served. (Hint: It was a long time ago.)
We’re talking a lot of books. Every hardcover mystery, thriller, or genre-adjacent book published between January 1 and December 31 of the year under review. The number that I recall is 492, all delivered to the front door. The UPS driver got a very good Christmas bonus that year.
“Best book” is an absolute. This was the first speed bump for me. Best is best, hard stop. The standard is not really, really good, or “Wow, that’s an original take!” Best is “none better.” No silver medals here. It’s daunting.
The books don’t arrive all at once. In fact, it’s quite the opposite. During the first few months of the 12-month submission period, I received books at a rate of a manageable trickle. Easy-peasy. Well over 100 arrived during the month of December, a good number of those squeaking in just under the New Year’s Eve deadline. Bummer for those authors.
Judging is done by committee. As I recall, I was one of 7 judges on the best novel committee that year, each of us representing a different corner of the suspense writing universe–cozies, thrillers, hardboiled, etc. I don’t know if that’s typical, but I though it was a touch of brilliance. We were well wrangled by an under-appreciated and overworked committee chair. I forget the details of how it all worked exactly, but as tranches of books arrived, each of us would provide ranked lists of our favorites (top 10 at first, winnowed to top five toward the end), which often bore little resemblance to each other.
Confirmation bias is real. For me, it boiled down to best being best. Given the numbers involved, if what might have been the greatest book ever written didn’t become interesting before page 20, it surrendered its shot at being absolute best. (Ironically, if that book had been one of the initial submissions in the slow times, it might have had a shot. If it had been submitted in the December tranche, it would have been lucky to have a 10-page fuse.) Other judges refused to consider books written by certain famous names. Hey, judges are people, too.
Production values matter. This one is really an aside, but its an important one. There’s a look and feel to a well-produced book–factors that go above and beyond the quality of the writing itself–that have a big impact on the overall reading experience. Font size, binding, paper quality, and I’m sure a bunch of other qualities I don’t understand make a subliminal difference to a reader. Something for all of us to keep in mind.
It’s all friendly until the end. As I recall, our final submission deadline for nominees was sometime in early February. Seven judges, each reading roughly 500 books, represents 3,500 individual reading experiences. There’ll be disagreements. By this time, though, the obvious non-starters have been eliminated, and we were down to the last 20-30 books that not only were all very good (okay, I didn’t particularly like two of them), and we have to narrow it down to a total of one winner and four runners up. Exactly four, not five. In my year, the winner was the book that was common to each of the judges’ top-five lists, though not necessarily in the top slot. As for the runners up, that’s where the fighting occurred. My top two picks don’t appear anywhere on the final list. I’m confident that other judges can say the same thing.
So what do awards mean in the end?
Having won the Thriller Award for Best Paperback Original in 2016, I can tell you that it means a lot to be recognized by one’s peers. It’s humbling. And I deeply appreciate the honor.
But being involved in the process taught me that “best” does not, in fact, mean “best” because such a standard cannot exist in an arena as subjective as art. What “best” really means is engaging and entertaining enough to rate inclusion on lists that also include other writers whose works I admire and whose talent I envy.
The final takeaway is this: Cliche notwithstanding, the true honor lies in being nominated in the first place.
I have the feeling that some of the competitions I’ve submitted work to don’t actually read all the books – that they go by externals such as awards and sales.
It’s the only explanation I can understand for some of the winners (not that I’m the perfect judge, of course). I check the winners in competitions I’ve entered, hoping to learn.
NO ONE supervises the judging process, and when the judges are anonymous, they have no reputation to protect.
It sounds as if the committees you served on followed the correct process. But reading 500 books seems a lot, even if most judges don’t make it very far into most of them.
Kris’s post yesterday brought back memories for me, too. I judged the “Best Paperback Original” (which included e-books, although most publishers sent hard copies), and although the total number of entries was in the 300-350 range, our process was very much as you described it, John.
The big issue for me was the reasoning behind considering paperbacks (these were trade, not mm) as second class citizens to hard cover. It’s supposed to be about the words, not the binding.
Another peeve was publishers who didn’t follow the rules for submissions. When I voiced my protests, I was told it didn’t matter, and to judge them anyway. I’m sure I will never be asked to judge again.
Terry, as you might imagine since my Grave series is a paperback original (PBO) series, this stigma has been a sore subject for me for a long time–and the stigma has existed forever. But only for mass market, not so much for trade paper, which is becoming the dominant publishing format outside the USA.
Wow. How do you not get judges’ fatigue, having to read/evaluate 500 books?
Oh, fatigue doesn’t touch it. I’d never do it again.
Well said, John. To name a book as the best is purely subjective. I also agree, it’s humbling and heartwarming to be nominated for an award. Thanks for a great post today.
Thanks, John, for a look behind the curtain. Congrats on your award in 2016.
I agree with your take on awards. “that ‘best’ does not, in fact, mean ‘best’ because such a standard cannot exist in an arena as subjective as art.” Each reader has their own perspective, and they are legion.
I agree with you, John, that being nominated for an award is the true honor. If I remember correctly, Alfred Hitchcock never won an Academy Award as Best Director, but he was nominated multiple times and is considered one of the best ever.
Picking the winner from 500 entries is at best a … Well, maybe the judges should come up with a list of the top six and then roll a die to see who wins.
Ack, I think being nominated several times and never winning would be torture.
I normally read around 200 books per year. I’ve found myself in John’s December “mode” of not liking a book if the first ten or so pages didn’t grab me. As a newish author, I’m taking that into account. Quite a few of the ones P J highlighted didn’t make me want to continue enough to find the gold that may be hidden in them and that’s perfectly OK because there is a tribe for those sorts of books. For my tastes, a mix hits it best, with enough action to carry the atmospherics.
I think I would have made it the first 10 pages, separated out those that grabbed me, then read that pile more carefully. Fair?
I was a judge in easily fifty contest, mainly in genre categories, for various RWA chapters and EPIC’s Eppies for original digital ebooks. Most had questions about the book which we rated then totaled the scores. At that point, some of them asked to rate the books from best to worst. If the judges didn’t get all the books, the best of the ranked books were passed on to a new set of judges for the final sorting. The way the questions were written in some categories really made it impossible for some very good books to have a chance.
From my own sausage history.
In 2000, Microsoft announced the Frankfurt eBook Awards to honor those brave pioneers in a digital industry that was totally shunned by Big Publishing but fueled by small publishers, ebook publishers, and authors. Innovation in quality writing and storytelling was emphasized. The digital books had to have been published either digital first or digital/paper at the same time.
The people chosen to run the awards were from Big Publishing, and they told the judges not to read all those sad little ebooks from all those nobodies not anointed by NY. Some judges admitted this afterwards. The winner in the fiction category was a writer who had been writing the same book series for many years so no innovation there. His book also had never gone digital on the market. Instead, his publisher put up his book on a page within their website right before he was nominated and long after it was published in paper, and the book couldn’t be bought in a digital version.
Big Publishing then used much of the money for the award on a lavish party for themselves.
So, that’s how Big Publishing scammed Bill Gates for over $100,000, and why there was only one Frankfurt eBook Award.