We’ve all read the Infinite Monkey Theorem by now. An infinite number of monkeys typing on a typewriter will eventually rewrite all the greatest works ever written. In theory, those monkeys would also write all the greatest works that ever will be written. While it is an interesting thought experiment, it has little practical application. We’ll never have infinite monkeys nor infinite time nor infinite resources to check their work and determine good output from bad output.
The Library of Babel is quite similar. An infinitely large library which contains books of all possible combinations of known language character sets and pictographs. The Library of Babel would contain a subset of books which contain all knowledge in the universe - as is representative by said languages. Just as no one can check all the monkey’s works, no one could read through all of the Library of Babel. If you owned the Library of Babel you would have all the answers to every possible question and, still, have nothing.
LLMs are sort of a precursor of the Library of Babel. Held within their vast multi-dimensional vector space of words and characters (tokens), is all knowledge that we’ve been able to publicly accumulate. Sans some niche edge cases, contemporary LLMs have vectors which point to most of humanity’s greatest written fiction, science, experiences, etc.
I can ask an LLM to continue this story:
BERNARDO. Who’s there?
Assuming I prompt it to just continue the story without trying to give me context, it will return:
FRANCISCO. Nay, answer me: stand, and unfold yourself.BERNARDO. Long live the king! …
FRANCISCO. Nay, answer me: stand, and unfold yourself.
BERNARDO. Long live the king!
There - within its embedding - is Shakespeare’s Hamlet. It stands to reason that a similar story to Hamlet is not far off. If I could take my finger and trace the vector space, I could find the words of another great work inspired by Shakespeare. Perhaps some poor, undiscovered, playwright in the 17th century wrote Hamlet II: Dead Guy Rises - an action thriller which would have blown their 17th century minds. LLMs are not infinite. They must be derivative based on existing works. So we’ll never truly (for now) see an LLM produce something new. The vector graph is simply too filtered. It only contains human written works. Sure, we try to mix and match pages, words, ideas, etc. But if we mix and match too much, we lose context, cogency, logic. We don’t have infinite resources to validate the good from the bad. The elusive recursive self-learning LLM might do that for us, but it might never understand the dichotomies of truth and fiction, entertaining and exhausting, or beauty and disgust.
Today, we can walk though the Library of LLM and pick out books. They are all derivative of something we’ve seen before by their very nature, but they will sometimes feel new or original. The Library of LLM doesn’t (usually) produce nonsensical works or works that are completely random. The Library of LLM is curated to try and only present books to you that have a vague sense of cogency.
When I sit before my keyboard - ready to write software - I must ask myself: is this within that vector space? Anyone who has studied art will know all work is derivative. The first cave painting was probably inspired by someone’s drawing in the sand. Sometimes the universe will give us originality and inspiration. The proverbial apple hitting Newton’s head comes to mind. But generally speaking, all human thought and creation is built on each other and our ancestors before us. I’m not trying to write software that is devoid of derivation, but I would like to write software that cannot be found in the Library of LLM.
Unfortunately, that has never been my forte. I am an average student, an average software engineer, and an average artist. Before LLMs I could write a piece of software which is close to another, but slightly different. Audio classifiers are common tech used by something like Spotify or Shazam to fingerprint songs. 10 years ago, I could have written an RF classifier for potentially helping people identify router models or classify a communication device. I didn’t have to be the first person to come up with the fingerprinting techniques. I just have to shift my audio spectrogram algorithms into RF. But now, I know it’s in the embedding. I know Library of LLM contains that book. Why bother writing it if it’s already written. Someone just needs to ask the librarian to go get it.
And that feeling has only grown worse over the years. I’m no longer inspired to contribute to the annals of software. What do I have to offer? I often find myself, in my limited free time, staring at an IDE wishing I had a reason to create some software, but I’m simply uninspired. LLMs can write entirely new languages, web applications, etc. in the time it takes for me to write a small subset. Not only that, but they are drawing on top experts in the field to do it. Their derivative work is based on the most well-known and consumed software. If I ask it to write an OS, it will draw inspiration from Linux. And would it truly be wrong to do so? Hell, I’d be pissed if it started generating a MS-DOS derivative. I don’t want it to write incoherent software, but I also don’t need it to rewrite Linux with some tiny, pointless, variance. The UNIX principles might be wrong. Who knows? Maybe someone will find monolithic software which does many things is better for some yet-to-be-understood reason. The Library of LLM will never have that book. But I don’t think I will be able to write that either.
So, what am I to do?