The two biggest issues with the Hemingway App you need to know are:
1. It Can’t Replace An Editor
First off, the Hemingway App can only look at sentence structure and word usage. It can’t evaluate the merits of a story, characters, narrative, etc. Nor is it going to point out errors in punctuation usage, wrong word usage, and more.
For example, here’s a passage I threw together with a bunch of intentional errors to see how the App graded it:
"Mark and Amy, ate some sandwiches by the see. “Gee, the weather really nice today” Mark said smiling. “Yes, it is,” Amy replied she leaned back and soaked in the sun."
Hemingway App graded this passage as a 2, which it considers “Good.” This passage contains nothing it dislikes, except for one adverb, which it considers acceptable. In spite of the fact that it’s riddled with errors.
So no, in no way is it a suitable substitute for a human editor.
2. Some Of The Greatest Works Of English Literature Fail The App
My favorite example of this is the absolutely astounding final paragraph of “The Dead,” which is the last story in James Joyce’s masterpiece Dubliners.
"It was falling, too, upon every part of the lonely churchyard on the hill where Michael Furey lay buried. It lay thickly drifted on the crooked crosses and headstones, on the spears of the little gate, on the barren thorns. His soul swooned slowly as he heard the snow falling faintly through the universe and faintly falling, like the descent of their last end, upon all the living and the dead."
As some will agree, this is one of the most beautiful passages ever written in the English language. So how does the Hemingway App grade it?
It says no phrases have simpler alternatives and that there are no uses of passive voice, which it likes. However, it found four adverbs, which the Hemingway App isn’t fond of – it tells Joyce to aim for one or fewer. And of the three sentences here, one is considered hard to read and the other very hard to read. And yet, these hard to read sentences are so lyrical and beautiful (particularly the last one) that it’s really hard not to see the literary merit of them.
Overall, Hemingway App grades this passage a 10, or an “OK.” (It wishes Joyce had been a 9.) It wishes James Joyce had been a better writer. I suspect other authors who have been hailed as classics, namely Charles Dickens, would fare even worse. Feel free to try out passages from some of your favorite books and see how they fare.
So remember that at the end of the day the App does help you take a second look at your writing and weed out some potential problems, but, you, as the author, have to decide when to follow the “rules” and when, for the sake of art, it’s best to break them.