Captive Knowledge

What's the problem?

Algebra. You know it, you probably hate it. It's this stuff:

$$3x + 5 = x$$

(I nerd-sniped myself by writing this and had to solve it. To save you a little trouble, the solution is x = -5/2.)

Turns out, algebra is immensely useful in a wide variety of applications.

Now, imagine algebra wasn't public knowledge. Imagine some company, say, Google, had discovered algebra and held it as a proprietary technology that gave them a leg up over the competition. Imagine further that they called it something snazzy like Syntheticon, trademarked the name, and filed a patent for the IP.

Now no one else can use algebra. And heck, no one else even knows what it is. The patent filing is probably something irritatingly vague like: A rigorous method for reducing mathematical equations into simplified forms. If some bright individual working for another big corporation like Microsoft happened to re-discover it (a real Newton/Leibniz situation), they wouldn't even legally be allowed to use it without hearing from Google's lawyers.

No highschool algebra. No advanced mathematics. No calculus. No anything that relies on algebra, except at Google, where they discovered it and patented it. That Isaac Newton quote about standing on the shoulders of giants? Yeah, you can't do that anymore. There's just one giant, and it's wearing spiked pauldrons on its shoulders.

spiked_pauldrons.jpg

Now, obviously, this story isn't true. Google doesn't have a patent on algebra (Syntheticon ™*)*. But there is a great deal of information that exists in this imprisoned state.

Look at the platform on which I'm writing this article — HashNode! Quite recently, they released an editor called Neptune, which is pretty sweet, I'll tell ya what (I'm using it right now [or should I say "write now"?]). This is a very well-built, extremely useful web-based rich text editor that could benefit a lot of applications. I mean, consider the use cases for note-taking apps alone! I'm building my own note-taking app on the side and I have struggled to find a competent web-based text editing library that offers the features and performance my application needs. If the source code of Neptune were publicly available, I could just drop it into my app, or at the very least reference the code to see how they solved specific problems.

But I can't, and likely I never will be able to. This is a huge problem in the text-editing space in general — countless engineers have solved and re-solved the same damn rich text editing problems over and over and over, since Word 1.0 in 1983 (source) to Neptune in 2022. (Granted, Neptune offers many more features than Word 1.0 did, but I think you get my point.)

Why do we keep having to solve these problems over and over? Well, because all that knowledge is locked up in the organizations in which it was synthesized! This is captive knowledge.

It is knowledge that is being held "hostage" by an organization and has to be rediscovered over and over countless times by problem solvers across the world so they can use it in their organization. (Note that "organization" here may refer to an academic organization as well — more on that below.)

For more examples of this, look at the full-text search space! Google has refined its web search engine into a modern-day engineering marvel. It's the Eighth Wonder of the World, but, really, it belongs in a category of its own above that, because when's the last time the Pyramids of Giza helped you figure out if poinsettia plants are toxic to cats? (They are.) So many other companies are trying to compete with Google, like Bing, DuckDuckGo, and Ecosia. But have you tried these other search engines? I mean, c'mon. It's no comparison.

This is again largely because of captive knowledge. (In this case, it's also because Google has such a massive existing infrastructure that any new company that tries to compete with it is already at a huge disadvantage.) Google holds all that good good search knowledge close to its chest so that it can maintain its phenomenal competitive advantage over everyone else.

What happened to standing on the shoulders of giants? In a way, it's sort of becoming a relic of the past because more and more valuable information is imprisoned in this way.

And just to be clear, this isn't just a problem in industrial contexts. It's also a problem in academia! ACADEMIA, THE ONE PLACE WE TRUSTED! (Wait, who trusted it again?)

Don't believe me? Well, maybe the existence of Unpaywall can convince you. You can also read a bit more about paywalled research articles here and here. You can also just find an article and attempt to read it on a site like ScienceDirect.

Why does this happen?

I don't have all the answers, but I suspect these are the root causes of captive knowlege. I doubt any of them would surprise you:

• Money - Organizations want money. I can't blame them. I also want money, and I'm not even an organization! Knowledge is valuable; why would you share it with your competitors?

• Laws - Copyright law, trademarks, trade secrets, intellectual property, etc., all make it difficult or downright illegal to share knowledge.

• Unintentional neglect - How often do you even consider sharing something valuable that you learned? The thought might not even occur to you. I don't blame you! I've learned all kinds of stuff over the years that I've never shared with anyone. And oftentimes, I tell myself that it wouldn't be all that interesting to anyone else anyway.

What can we do?

I think I've described captive knowledge pretty well at this point, and have also complained a lot. But what can we do to make knowledge free? Well, none of the suggestions below should be surprising, but I'll suggest them anyway:

• Write about it publicly. Type up an article about your knowledge and post it somewhere others can read it for free! I'd love to read about your ant colony optimization for binary search, since I can't read the one on ResearchGate 😢.

• Make it open source. That code you wrote that solves a common but annoying problem? Post it on GitHub under a permissive license! Be the hero this city needs.

• Talk about it! Share your knowledge freely with the people around you! When you learn something cool, share it with your coworkers, or your family, or whoever you think would find it useful. The ol' "word-o-mouth" trick does wonders.

• Release your research paper to the public. This is a complex topic that covers copyright issues and researcher pay, but if you are a researcher, please consider posting your paper somewhere public if you legally can.

But it's not enough. Anyone who is working at a company knows they can't just go willy-nilly sharing company secrets or making internal projects open source. There needs to be a more systemic change in the way companies view information. So I guess I'll end this post with a plea:

An impassioned plea

Dear companies, researchers, and savants:

Please share your knowledge! By doing so, you advance all of humanity and you give future generations the ability to stand on your shoulders and discover new, greater things as a result.

Your contributions to the public library of human knowledge might just make history.