Machine Learning Tests Amazon Novels and Shows Book Covers Reflect Genre

We already know readers judge books by their covers. If you are still fighting this social truth, then close your eyes, take a deep breath and release your rebellion to the universe. Repeat after me, "Readers judge books by their covers".

If the above is true (and it is), can we say books in specific categories or genres have similar and distinct styles, fonts and images used in book cover design?

A quick scan of Goodreads can provide a visual for us to examine, but we could never go through as many books in as short a time, or with as much brain power as a computer could.

Brian Kenji Iwana and Seiichi Uchida conducted a study at the Kyushu University in Japan to determine if book genre could be guessed by cover design. What's interesting is they used a "Convolutional Neural Network (CNN) to predict the genre of a book based on the visual clues".

In case you care...

Convolutional Neural Networks (CNN), in particular, are multilayer neural networks that apply learned convolutional kernels, or filters, to transform input data as a method of feature extraction. The general idea is to use learned features rather than predesigned features as the feature representation for image recognition. Recent deep CNNs combine multiple convolutional layers along with fully-connected layers. By increasing the depth of the network, higher level features can be learned and discriminative parts of the images are exaggerated. These deep CNNs have had successes in many fields including digit-recognition,  and large-scale image recognition  (Judging...).

For the rest of us, I'll use the acronym CNN or "a computer" to avoid brain cramp.

Using "machine learning", Uchida and Iwana had their CNN analyze and genre-sort 137,788 book covers from Amazon.com. "Title, author and sub-categories were discarded" (Judging...).

Machine learning is a term used to describe a computer's ability to "learn" based on what it is taught. It isn't quite doing what it's told and it isn't quite true artificial intelligence. The computer is given some rules, which we call algorithms. Then, the computer runs these rules against the testing material or idea and can start to make predictions based on data patterns.

In this book cover study to determine if genre is recognizable in book cover design, the Japanese researchers trained their system on visual design using approx. 110,000 Amazon.com novels. After the system was trained, it tested it's genre learning against just under 14,000 novels to ensure pattern logic was working, then it ran the last 10% of the books in an effort to categorize the covers.

In some genres, such as cookbooks, technology and travel, the computer could recognize about 40% of the book categories.

Using a CNN model, we categorized book covers into genres and the results of the experiment had an accuracy of 21.9% for Top 1, 32.1% for Top 2, and 40.2% for Top 3 in 20-class classification (Judging...).

Other genres were more ambiguous and confusing and the system did not fair as well.

Another example of misleading cover images can be seen in the “Biographies & Memoirs” category. The difficulty of this category comes from a high rate of sharing qualities with other categories causing substantial ambiguity of the genre itself. A high number of misclassifications from the “Biographies & Memoirs” category went into “History". (Judging...)

The bottom line is this research showed a consistency between visual images within book genres.

In this paper, we presented the application of machine learning to predict the genre of a book based on its cover image. We showed that it is possible to draw a relationship between book cover images and genre using automatic recognition (Judging...).

So, what does this mean for authors? My personal conclusion is that readers, machines and designers will follow trending visual design for genre categories. If we, as authors, design our novel covers outside of those trends, we run the risk of our novels being purchased by the wrong readers, which can lead to unsatisfied customers and negative reviews, not to mention never building a tribe in the right genre readership. Book cover design outside of the visual cues for genre could also lead to algorithms misfiling/shelving a novel, or missed chances at having a novel connected to other books in its category.

If we weren't sure before, Iwana and Uchida's research has shown there are visual similarities between genre book cover design, and like they say... when in Amazon, do as the Amazonians do.





Sources:
"Deep Neural Network Learns to Judge Books by Their Covers" https://www.technologyreview.com/s/602807/deep-neural-network-learns-to-judge-books-by-their-covers/
"Judging a Book by its Cover" https://arxiv.org/abs/1610.09204