Cracking The Big Mystery
Behind The Bestseller Lists

 

The bestseller list is the tip of the iceberg. — Michael Korda

By PJ Parrish

William Peter Blatty was hot off the blockbuster success of his book The Exorcist when he met the devil he couldn’t defeat — the New York Times best seller list.

Angered that his novel Legion, the sequel to The Exorcist, didn’t make the list, he sued the Times for $6 million, claiming the Times ignored actual sales figures from his publisher and that Legion was kept off the list because of “either negligence or intentional falsehood.”

It gets better. Or worse, depending on your point of view.

The Times, which had always claimed that the list was compiled from computer sales,  countered in court that its list “was not mathematically objective but was editorial content and thus protected under the Constitution as free speech.”  Blatty appealed to the Supreme Court, which declined to hear the case. Thus the ruling stood that the New York Times bestseller list was “editorial content, not objective factual content” and that they had the right to exclude whatever book they wanted.

And that, crime dogs, is pretty much where we still stand today.  How any book cracks the New York Times bestseller list remains, to paraphrase Churchill’s famous quote about Russia, “a riddle wrapped in a mystery wrapped in an enigma wrapped in fish wrap.”

I used to dream about being a bestseller.  Because this is what happens: Your publisher takes you to dinner at Le Bernardin. You get a new seven-figure contract with cover approval and world tour. Spielberg buys the rights. Your agent starts to return your calls. And you make so much dough your long-lost brother from Bullhead hits you up for a car loan.

Like I said…it’s just dream. We actually did make the Times list, with our third book Paint It Black. It was only what they call the “extended list” which means we didn’t crack the top 15 but hey, we hung on our toenails for a while.  We made the extended list two other times but have not repeated the feat recently. But that’s okay, because it’s sort of like making Eagle Scout. Once you get your badge, no one can take it away and you can wear that badge until your teeth fall out.

So I am not here to tell you that making a bestseller list is a fool’s goal. It doesn’t open doors so much as widen the crack, and it gives you credibility with readers, booksellers, critics and such. But I am here today to ask you not to think about it much. Because the bestseller list game is sort of rigged.

This is not news to many of you. But whenever I am asked about this subject by readers or some newbie writers, I am always shocked at their naïveté. What, you’re telling me it’s not based on real book sales? they gasp.

I don’t think much about bestseller lists anymore. I don’t even look at them when I read my New York Times book review section. But yesterday I did stop and read the paper’s “Inside the Times” article.  It was titled “We Don’t Have to Like ‘Best Sellers’.”  In it, once again, the Times felt compelled to explain to the world how it compiles its lists.

This controversy is not new. A book industry report in the 1940s found that best-seller lists were a poor indicator of sales, since they were based on “misleading data.” Fast-forward to a 2004 report that quoted a senior book marketing executive who said the rankings were “smoke and mirrors,” and a report in Book History found that many professionals in the book industry “scoffed at the notion that the lists are accurate.”

And writers have been trying to game the Times system since before the quill pen.  Jacquelyn Susann and Wayne Dwyer, among others, bulk-bought their own books to get on the list. And until recently, you could hire a company called ResultSource that would contract with you to manipulate lists through “bestseller campaigns.” (I tried to find their website but apparently ResultSource has since gone dark).

Last summer, an unknown book by an unknown author from an unknown publisher rocketed its way to first place on the Times’s young adult hardcover best-seller list. But the YA Twitter community discovered it wasn’t because a lot of people were reading Handbook for Mortals by Lani Sarem. The author and her publisher bought the book’s way onto the list by strategically ordering large numbers of the book from stores that report their sales to the New York Times. The Times quickly removed the book from its list.

So it’s no surprise the Times is still playing defense. Here’s a sample from their Q&A yesterday:

How do authors get on The New York Times best-seller lists? Do their books have to be sold at certain stores?

The New York Times best-seller lists are very competitive, which is what gives them the cachet they have within the book industry and with the public. Our lists reflect the reporting from our confidential panel of tens of thousands of retailers. We do not reveal those sources, in order to circumvent potential pressure on the booksellers and to prevent people from trying to game their way onto the lists.

Translation: The Times has a network of “reporting stores” which include selected independent bookstores and some but not all big-store outlets. The last figure I found was 4,000 stores and “undisclosed wholesalers.” The exact methodology is considered a trade secret. I have been told by store owners that the reporting figures are not even based on actual sales to customers but on the number of books ordered to stock.

How do authors get on The New York Times best-seller lists? Do their books have to be sold at certain stores?

The New York Times best-seller lists are very competitive, which is what gives them the cachet they have within the book industry and with the public. Our lists reflect the reporting from our confidential panel of tens of thousands of retailers. We do not reveal those sources, in order to circumvent potential pressure on the booksellers and to prevent people from trying to game their way onto the lists. A number of variables go into whether a book will rank on a given week. Weeks where there are blockbuster debuts in multiple categories will be different from quieter weeks. Rankings reflect unit sales reported on a confidential basis by vendors offering.

Do the books have to have been reviewed in The New York Times?

Books that get ranked may or may not get reviewed by the Book Review and vice versa. Our best-seller lists and the editorial decisions of The Times’s book editors and critics are entirely independent. This means our lists are not a judgment of literary merit made by the editors of the best-seller lists, who remain impartial to the results. These are best-seller lists, not best-reviewed lists.

Translation: But if you happen to work at the Times, some critics have charged, your book will not only get reviewed but it has a pretty good chance of being “considered” by the panel of folks who watch over the list. Which leads us to…

How do The Times’s ranking methods ensure objectivity?

The best-sellers desk is staffed by three full-time editors who work independently from our news, opinion and culture desks; from the Book Review and the books desk; and from our advertising department. Our nonfiction lists feature books from authors across ideological and political spectrums. In the last year, politicians and commentators who identify as conservative have performed as well as, if not better than, liberal ones on our lists. Trends depend on publishing schedules and what is happening in the cultural zeitgeist.

One question they don’t address, but one I am asked often is: How many books does it take to crack a list?  It depends…

On who else you’re competing against that week. On what time of year it is. On whether someone has a similar book already out there. And on what list you’re aiming for. The general figure these days (way down from the olden days when I started out) is you need to sell at least 5,000 in one week.  But that means from Monday to Sunday if you want to be a Publishers Weekly best-seller, and from Sunday to Saturday if you want to be a New York Times best-seller.

It’s a jungle out there, Martha. Even if you want to aim a little lower, say for USAToday, The Wall Street Journal, a regional list like the Chicago Tribune or maybe Indiebound, you have a whole different set of hoops to jump through for each. Every bestseller list out there is compiled differently. Here’s a breakdown I found at Vox.com:

Publishers Weekly: Compiles data from the Nielsen service BookScan, which is what most publishers use to track their competitors’ sales. BookScan claims it tracks 80-85  percent of the sales of printed books in the U.S. (Amazon, Barnes & Noble, Target, and Walmart, as well as a number of independent bookstores.) But it does not track books sold at independent bookstores that use older systems incompatible with BookScan’s tracking, or books sold outside of the general bookstore ecosystem, ie, at conferences or gift shops or toy stores, or even sales to libraries. It also doesn’t track the sales of e-books.

USA Today: Gets its data from both a handful of independent bookstores and many of the usual-suspect big sellers: Amazon, Barnes & Noble, Target, etc. It doesn’t make any claims about what share of books sales it tracks, so it’s a broad sampling of books sold every week from different types of stores. Again, like BookScan, it does not track books being sold outside the bookstore ecosystem.  It doesn’t divide its list into any specific categories, but instead reports the top 150 titles sold across all genres and in all formats except for audio. So your crime novel will compete against Michelle Obama’s Becoming. I know, I know…insane.

Indiebound: This is compiled by the American Booksellers Association. The ABA uses sales data drawn from about 550 independent bookstores to create its list, but it doesn’t rank titles by overall sales volume. Instead, it weights the books on its list according to the sales rank each one reaches at each individual store. I don’t understand that either but there it is…

Amazon: It has two different best-seller lists: Amazon Charts and Amazon Best Sellers. Charts comes out once a week, tracking the books that have sold the most copies in any format (on Amazon, and in its Kindle store, Audible store, and brick-and-mortar storefronts), and the most read or listened-to books on Kindle and Audible. It’s not broken down by category or format, and it only reflects what’s happening on Amazon and its subsidiaries. (Since Amazon has a 65 percent market share, that’s actually a pretty decent sampling.) Amazon Best Sellers, in contrast, is updated once an hour, and it is broken down by categories. This latter one is what we crime dogs fixate on.

Okay, you’re saying, what about us self-published guys? Do we have a chance at getting on any kind of list? Yes, you can crack the Amazon list.  We got to no. 1 briefly in the thriller category when we self-published our back-list title Dead of Winter. And it used to be alot easier before Amazon started messing with their algorhithms. There was a story every week about some self-pubbed phenom. But for reason behind my ken, that has tapered off. (Maybe some of you can explain in comment?) I did see a figure this week that was astonding — that you need to sell between 3,500 to 5,000 copies in a 24-hour period to hit no. 1 on Amazon. But then I also read recently that Lee Child sells a book every eight seconds…

By the way…those three books at the top of this blog today? You might recognize them. You might not know that they were all self-published before they were massive bestsellers.

But what about the Gray Lady? Well, according to their Q&A yesterday, here are the books they don’t track:  “perennial sellers, shopping guides, comics, reference and test preparation guides, required classroom reading, textbooks, journals, workbooks, calorie counters, puzzle books and self-published books.” If if makes you feel any better, this means the Bible doesn’t qualify. Neither does The New York Times Monday Through Friday Crossword Book, even though it is currently #3 on Amazon’s bestseller list.

So, does this matter? Is this something you should you worry about this?

Well, it’s a gold star on your homework, but it isn’t a true gauge of success. And here’s something weird I found:  Hitting the Times list works better for unknown  authors than the Lee Childs of the world.

According to an economics professor Alan Sorensen, who has studied the effect of bestseller lists on sales of hardcover fiction, relatively unknown writers get the biggest benefit, as much as a 57% increase in sales. But for perennial best-selling authors such as John Grisham, being on the list makes virtually no difference in sales. Most sales occur soon after a book hits the shelves and gradually peter out. “If anything, what appearing on the [bestseller] list does is not so much cause your sales to increase from one week to the next, but rather to decrease at a slower rate,” Sorensen said.

Why can’t the bestseller system be fixed?

With the sophistication of software now, you’d think there would be a better way to keep track of real book sales. The model, some say, is the music industry, with its bestseller list in Billboard. The magazine tracks every single album sold at every single music store in the United States. SoundScan, the company that began tracking CD and tape sales with a bar code system, was the force behind the creation of Bookscan.  But BookScan is too expensive for many bookstore owners.

And here’s the bigger rub: The publishing industry really doesn’t want a single list of what’s really selling. They want lots of different lists that they can manipulate to benefit their own bailiwicks.

So…write your book and kept your heads down, crime dogs. The rest will come.

Which brings us back to William Blatty.  Despite great reviews, The Exorcist laid such a giant sales egg at first that Harper and Row reported getting returns by “the carload.” But then sales took off and the book made the New York Times bestseller list for 57 straight weeks and at the No 1 spot for 17 of them.

And years later, not long after Blatty filed his lawsuit against the New York Times, Legion made it all the way to no. 15 the Times list for one week.

 

This Is (Almost) Halloween…

I know. Perhaps it is too early for me to be writing about Halloween. I’ve been seeing  merchandise for the unofficial holiday in stores since September 5th, however, so I’m actually behind the curve. Herewith please find my subjective list of Top Five frightening reads that will carry you through the next few weeks:

MISERY — I was given this newly published book as a present for Father’s Day 1987. I started reading it that afternoon and did not stop until I finished it that evening. Some dad, huh? Stephen King’s now-iconic tale of popular author Paul Sheldon’s extended visit with defrocked nurse Annie Wilkes — his Number One Fan — more than stands on its own merits. It makes/tops my list, however, because I had a relationship with someone very much like Annie, right down to her potentially dangerous mood changes and odd turns of phrase, the manifestation of which always preceded what I would come to call an “episode.” I read this book at least once a year, repenting at leisure and recalling the exhilarating sound of doom whistling by me at a near-miss.

THE SHINING — This tale about Jack Torrance, a struggling author with writer’s block the size of a Jersey Wall, and his family was already quite well known when it was adapted for a (lesser) film by Stanley Kubrick. I screamed twice while reading it. The first was during young Danny Torrance’s encounter with the girls in the hall.  To this day, when I am in a large hotel with a long, carpeted corridor, I think of Danny and the girls who wanted to play with him forever.The second was during the bathroom scene. I have, unbidden, remembered this scene at inopportune moments over the course of my adult life, with unhappiness ensuing. The book as a whole, however, is a terrific example of how to wring every bit of drama that can be wrung out of a single location.

THE EXORCIST by William Peter Blatty — This early 1970s novel was a potboiler for sure — and that is one of my highest compliments — but it is a cringe-inducing tale of demonic possession and the efforts of a heroic priest to save the life and soul of an innocent girl  which fed right into my Roman Catholic upbringing. My father, who spend serious and quality time in Seminary school, assisted in an exorcism and told me that Blatty’s account of possession was mild compared to what he witnessed. That might have been, but it is hard to believe that what (almost) Father Joe experienced was any more frightening than Blatty’s description.

‘SALEM’S LOT by, ummm, Stephen King — I have always enjoyed well-written vampire novels — there aren’t many of them — but there is a special place in my heart for this story of the Undead and love lost in a small town on its last legs. King’s second novel published under his own name is a textbook example of how to plant a slow, unnamable dread on the first page, nurture it, and grow it to full blossom stark terror. The television adaptation, with David Soul in the lead role, has its weaknesses but actually stands up quite well. A planned sequel was later incorporated into the Dark Tower series in THE WOLVES OF THE CALLA and SONG OF SUSANNAH but neither quite reach the atmospheric levels of fright found in this book.

THE BODY SNATCHERS by Jack Finney — I saw the 1956 movie Invasion of the Body Snatchers before I read the book upon which it is based. That august novel, although almost as old as I am, has held up much better than either myself or its film adaptation. Marketed as science fiction, THE BODY SNATCHERS is a paranoia-laden horror story about alien seed pods that land on earth and begin producing a duplicate replacement copy of each human being. You have almost certainly seen at least one of the three films based on the book but you can’t beat the source material on any level. Five-year-old mini-Me was also certain at one point that his parents had been pod-snatched. You might as well, but take a chance and pick up a copy of this classic if you’ve never read it.

You know what I’m going to ask now, I’m sure: what are your favorite horror/scary novels? And why? Thank you.