Sherlock Holmes and the Mystery of Trash in node_modules

Tal Brenev April 16, 2019

So there I was, grepping through a project, when to my utmost surprise, this line of output leapt out at me:

node_modules/readline/test/fixtures/afile.txt:8670:came on, chromatic scales and diminished sevenths were heard in the

“Chromatic scales? Diminished sevenths?” I thought to myself. “Why would this terminology appear here, of all places?” So I put on my detective hat, and set out to find the source of this mysterious text.

Fortunately, solving this mystery did not require any deduction whatsoever, since the culprit was right there in the output of grep. So I promptly took off my detective hat, and proceeded to take a closer look at the offending file, hidden away in the node_modules directory.

And there it was, in all it’s glory: afile.txt. 39,772 lines. 343,011 words. 1.9 megabytes. At the very top of the file was this:

The Project Gutenberg EBook of The Adventures of Sherlock Holmes
by Sir Arthur Conan Doyle
(#15 in our series by Sir Arthur Conan Doyle)

Something about all this was odd, however. I’ve read The Adventures of Sherlock Holmes, and it definitely didn’t feel like it was 343,011 words long. I don’t recall chromatic scales ever being mentioned either. Upon further inspection, I realised that this behemoth did not consist exclusively of mystery novels. It also appears to contain a considerable chunk of Tolstoy’s War and Peace.

“But what in the world was this file doing in an npm module?” you might ask. Elementary, my dear reader! Somewhere inside the gargantuan dependency tree of the project was a module called readline. As the name suggests, this module reads in a file, line by line. Since you can’t test a module that reads files without a file, afile.txt was added as a test fixture. But for reasons that even Sherlock Holmes would not be able to deduce, these tests are distributed through npm alongside the actual package.

This module is guilty of crimes more heinous than anything that Sir Conan Doyle could ever come up with. Not only has it committed first-degree bloat, it has also wasted unfathomable amounts of electricity during its installation, which makes it liable for a charge of assaulting my battery.

Case closed. Take ‘em away, Lestrade!