Logo Ryan Petris

The Sorry State of Software Development

Posted on August 15, 2021 in Technology tagged Computer Software

I have been professionally developing software now for the last 12 years, and have been programming for at least the last 20. During that time, software development has changed significantly, and one of the more recent additions has caused both relief and frustration: package managers.

Package managers have been around for a long time in the open-source world, most linux distributions completely revolve around them. They make it easy to discover and install software; rather than having to go out on the internet or, gasp, to a store to get a piece of software; everything is just a command and download away.

While this has been the case for finished programs for a long time, this is only a recent development for software libraries, which have only become popular over the last 10 years or so. Is this a problem? No, not really, being able to run npm install underscore to install the popular underscore.js library, or pip install pyyaml to install the quasi-official PyYAML YAML parser for python makes it much easier to manage those dependencies and keep them up to date.

So, what's the problem them? Tiny, single-function libraries.

You see, with linux distribution package repositories, the packaging of the software to include in the repository is done by a team of maintainers. Those maintainers decide what would be beneficial to include in the repository, and what should be left out. You therefore effectively end up with a curated list of software to use with your distribution. With library repositories, however, typically anyone can submit a package without regard to the quality or usefulness of the package. On top of that, there's typically no review prior to the package being published, or really at any time unless someone reports that the package is malicious.

That's how you end up with the is-string library in NPM. This is a library that literally just checks if the given input is a string. In traditional software development, this would just be a function that you write quickly and throw into some utility class in your code; no external library needed! However, this gets even worse, as this simple library also has dependencies! It depends on has-tostringtag along with about a dozen additional development dependencies. And for icing on the cake, even has-tostringtag has dependencies!

This is how you end up with actual useful libraries, like Angular or React for example, pulling in literally hundreds of packages into your project. All of those libraries then need to be indexed by an IDE, scanned by whatever vulnerability scanner you're using, tree-shaken if you've turned that option on, and included in your final build.

This is dependency hell; it's why we can't have nice things anymore.