Something happened this year with the latest generation of LLMs, and I’m not talking about how they seem to have gotten a little worse at chat than the previous generation. No, what I’m talking about is how the latest generation of LLMs seem to be dramatically better at real work, like coding, parsing complex strings, making images and videos so realistic it is genuinely hard to tell them from manually recorded footage. It’s amazing, and for me it coincides with the kid gloves coming off their use at work. Where does that leave me? In desperate need to catch up with something I’ve been dabbling in for a long time.

So, I’m picking back up this blog to serve as my report to the ether about what I’m learning and working on. As usual my audience is actually myself, but hopefully someone finds something useful in what I share about what I’m learning along the way.

What you need to think about as AI starts to integrate in our world

What we are calling AI’s right now are actually just a very specific kind of AI, a “Narrow AI”, but that is changing quickly. In the numerous AI labs and company shops there are definitely some systems of AI’s, deep learning neural nets, and regular heuristic applications being glued together into what will surely qualify as Artificial General Intelligence’s. What you can buy as SaaS or build on your own hardware is pretty close, but there are a few key missing components that prevent current AI from being general intelligence:

The mostly missing pieces to general artificial intelligence

• Executive function: This is our overarching view of our own existence and sort of the guiding thought processes that combine our memories with our thoughts and intents. It is key to what humans consider intelligence, but LLMs, agentic systems, and such are only just getting functional memories, the ability to refine the memory and introspect the thought processes is just not there yet in consumer AI systems.

• Memory Refinement: We forget, and it’s not only a flaw, it’s one of the things that makes us able to use our memories appropriately. Digital systems today are designed not to forget, so that vector database with all the embedded memories of an AI’s inputs, thought processes and responses is perfect with everything readily available for instant recall. And a perfect memory in any general intelligence context is going to be a scalability nightmare in cognitive design. Our brains work well at the tasks they are evolved for because we only remember what we think is important to know(both consciously and subconsciously). In some ways this often feels like a flaw, but it is a feature that let’s us consider quickly and act decisively.

  Often, even without our conscious knowledge, we create false memories that help us keep a consistent world view. If we consider human intelligence as a the measure of artificial intelligence this is an important feature that must be integrated in artificial intelligence, and governed by the executive function so that it can be corrected when it strays too far from reality. We might be seeing something like this in the hallucinations of AI, but it is ungoverned and therefore purely a flaw. AI’s don’t know what they don’t know.

• A Consistent Internal World View: Our internal world view is vast, it extends from where we think we exist at any point in time to how we think we should behave in an ideal situation or the opposite. Our internal world view provides us with a way to integrate external stimuli with our goals and formulate tactics and plans to achieve our goals. It is a construct that our executive function can use and refine so we can better interact with the world around us in ever more effective ways as we adapt and change it. And even when we are battered by inconsistent external stimuli our internal world allows us to continue forward and find success.

What AI can do right now

Even without general intelligence, the AI's of today are capable of amazing feats once only thought capable by humans. Here I’m going to present a high level, non-exhaustive view of what I think are the most important features of LLMs and Deep Learning systems.

• Vast Amounts of Human Knowledge in Easily Accessible Stores: To a degree that has never existed before these models are the largest single conglomerations of human knowledge. The model training efforts are easily the largest collections ever assembled, with some even larger data sets yet to be trained into models. And on top of that they are all set in plain language. This is new, this is unique, this is a powerful tool.

• Natural to Us Interfaces: While natural language processing has been around a long time, it never really worked very well, now it does. LLM’s and their vast neural networks have finally cracked the problem of understanding plain human language in the wild. This is a tectonic shift in the way our tools can work. Since their inception computers have required rigidly perfect commands and data inputs to do any work for us, now we can quickly type out a request with misspellings, poor grammer, and slang words and most of the time it’s perfectly understood. This alone signals the end of an age of digital specialist who dedicated so much of their lives to learning domain specific languages.

• Human Like Problem Interpreting and Solving Capabilities: The ability to figure out what a problem is from incomplete information is a brand new feature for computers, this is practically an emergent feature of the large language model research. And on top of that the ability to solve some of these problems effectively is an astounding capability. Neither of these capabilities is perfect, but they are oh so human in their imperfection. And they will only get more accurate over time at both interpretation and solving as we refine and refactor and find new ways to make the models understand and solve our problems.

What you need to do now that AI is here

Well first off, you need to do things like what I am doing here. Hand typing my thoughts out into a regular interface, not relying on AI. While it would be easy to prompt through some of my local models and generate this entire blog post that is the last thing we as humans need to be doing. AI’s are a powerful tools that I’ll blog about how to use here, both setting them up locally and coding custom integrations and using API’s for some of the leading paid models available. But if one thing that should be clear isn’t clear, here it is:

Letting an AI do your work for you will not help you learn. If an AI can do everything for you, you have just proven that an AI can take your place. And if you lean on AI too much your own skills will atrophy. So pick up a pen and write, pick up a bush and paint, grab your guitar and strum, think through a skill and do it.

We can use AI’s as our force multiplier, to be the extra editor, be the extra researcher, be a whole team of programmers for us. But we fail if we get lazy and just let them take over.