Just read the friggin tests
Posted on April 5, 2020 by sphaso

There’s a simple technique I’ve been using in my career that I now realize is not widely known. I’ll just spill it out: when I don’t know how to do something with a library, I read the unit tests. I’m crazy and I know it!

One trend I’ve noticed as I grow older is that I stopped relying on pre-digested information, especially Stack Overflow. In my first two years as a programmer I would rely on it quite often. Don’t remember how to use an API? google it, stack overflow it, copy-paste, see if it makes sense, done! At some point I grew a bit frustrated with the platform. Most answers were just bad if not wrong. If only there was a place where one could find reliable answers about a language or library… oh, you mean documentation? you’re cute.

I had heard the old adage of “read documentation!” but it never stuck. I had come across libraries whose documentation was just abysmal. Sometimes I wanted to know in which unit of measure a parameter was and all I could find was “parameter x represents x”, “x” doesn’t stand for anything, it’s the name of the parameter!
Then came Haskell. Haskell gets a lot of bad rap for a number of things and one is documentation. I used to beat this drum too but Haskell has two things that very few other languages have that doesn’t make it as bad: 1) a decent type system 2) Hoogle. The absence of tutorials, ready-made snippets etc. made me read documentation from Hoogle a lot. When I transitioned from .NET to Elixir documentation was again a godsend. Elixir documentation is very well made and library writers usually document their work equally well. In both cases there was little incentive to go to Stack Overflow and pray someone had a good (and updated) answer.
Looking around the office I would notice this trend: more junior developers had Stack Overflow open, older developers had documentation open. I talked a bit about it with other colleagues but few people seem to notice. For some it was a sign of maturity in one’s career, for others it was purely accidental. Take it as it is.

Sometimes however documentation is not enough. Sometimes what you read doesn’t match with what you’re seeing. Sometimes it looks like there is a bug or simply you need to know more about the internals. Your question is not answered by the documentation. Sometimes you’re simply curious about something. So what do you do? you go to the source. There’s something beautiful about being able to read the source code of a library, you’re almost like a journalist questioning their source. You read the code and hopefully you understand what’s going on. You don’t need to rely on someone’s else digestion. In an argument against yourself or others, what’s best to just resolve it by going to the source of the dilemma? the absolute truth of the matter is a few clicks away.

Another old adage no one listens to is “read code!”. Everyone agrees that they’re not reading enough quality code. I’m one of them. I usually just peer into the code to get my answer and dash out. If the library is small I might read the whole thing but it’s rare. It did happen in my career however that I had to read a lot of library code for the worst reason: I couldn’t do something. What I wanted to do was either not documented, wrongly documented, or the pieces were too hard to glue, or, you know, the library had a bug in a critical feature.

That’s a real head scratcher. What do you do when the tool doesn’t behave like it should? you read the code. That’s good, but the cognitive load sometimes might just be too big. I remember parsing through a persistence framework’s source just to understand why data wasn’t replicated, it’s not pretty. In case you think you’ve found a bug, how do you know it’s actually a bug and not just some settings or boilerplate you forgot? The code has all the answers but not all the code can fit in your brain. This is what this post is all about: read the tests.

How do you do X with this library? If the library is any good, there should be tests even before someone decided to write down a quick tutorial! Tests are code, tests save lives, and tests are documentation. When you break a test you might be breaking an API, which warrants you to go back and change documentation. Tests are the ultimate code snippet.