7 Key Fundamentals AI Concepts Every Developer Needs to Know

If you only consume AI tools without understanding their core principles, you’re just repeating patterns—not creating new ones. AI is more than just throwing words at the latest version of ChatGPT or any other tool that emerges—and they will continue to emerge.

At this point, the advances in this field are so enormous that they cannot be stopped. This reminds me of an article I read in the Harvard Business Review that stated:

"The question isn’t whether AI will be good enough to take on more cognitive tasks but rather how we’ll adapt. Nobel Prize winner Daniel Kahneman is the world’s leading expert in human judgment and has spent a lifetime documenting its shortcomings. Yes, AI may have flaws, but human reasoning is deeply flawed, too. Therefore, “Clearly AI is going to win,” Kahneman remarked in 2021. "How people adjust is a fascinating problem.”

For me, the true creative potential of AI comes when you understand its core pieces and rearrange them into something unique—just like an artist mixing new colors.

This is not just about “learning AI”, but it’s about unlocking your creative power through the fundamentals—so you can build, adapt, invent and become a reference in your company.

So let's start!

Beyond the hype, what is AI?

Before we get into the nuts and bolts, it's essential to define AI beyond buzzwords. It is more than just machine learning models or LLMs predicting the next word in a sentence, but is about building intelligent systems that interact with the world, make decisions, adapt, and solve complex problems.

Agents: the intelligence behind the systems

If you open social media and look at discussions on AI, you’ll quickly find an overwhelming number of posts about autonomous agents—agents that control browsers, desktops, and perform tasks autonomously. Or maybe it's just my algorithm that's broken these days.

What bothers me, though, is how this is talked about like it’s something new—“Oh, the next wave of AI is autonomous agents!”—when in fact, agents have been a fundamental part of AI for decades. AI itself is about agents.

Everything from self-driving cars to game AI, even Alexa playing your favorite song on Spotify, is an agent. And if you don’t understand agents, you don’t understand AI.

The definition is simple: An agent is any entity that perceives its environment and acts upon it to achieve goals.

Artificial Intelligence: A Modern Approach

Think about the Pacman enviroment:

UC Berkeley

📖 Stuart Russell’s book sums it up well:

In a nutshell, AI has focused on the study and construction of agents that do the right thing. What counts as the right thing is defined by the objective that we provide to the agent.

For an agent to act intelligently, it must decide what actions to take.

So, how do agents actually make decisions? They search.

Search: Finding solutions

So, if the next action an agent should take is not obvious, them it need to plan and consider a sequence of actions that form a path to a goal state.

Many AI problems can be reduced to searching for the best solution in a vast space of possibilities. It is used in pathfinding, optimization, and problem-solving.

Think about the Pacman environment, in there, a search problem consists of:

- A state space:

UC Berkeley

- A successor function with actions and costs:

UC Berkeley

- A start state and a goal state

Search algorithms solves a lot of common problems, think about the NPCs (non-player characters) in video games as agents, they actually use A* (A-Star) for pathfinding around obstacles in real-time strategy.

And in talking about games, what if there are multiple agents competing? That’s where Game Theory comes in.

Game Theory

In few words, game theory studies how rational agents make decisions when outcomes depend not just on one’s own actions but also on the actions of others.

It might sound surprising, but many of AI’s biggest breakthroughs have come from teaching computers how to play games.

Why? Because games provide structured environments with:

✔ Clear rules

✔ Measurable goals

✔ Immediate feedback

When agents must make decisions quickly, they rely on heuristics—quick, rule-of-thumb strategies that guide search and decision-making in complex, uncertain environments.

Heuristics: Guiding the search

When I first started studying AI, I kept hearing the word heuristic—my professor would ask, “What’s the heuristic for this problem?” and I’d have a big question mark on my face.

Eventually, the concept clicked: heuristics are strategies or shortcuts that help simplify decision-making and reduce the heavy computational load of searching.

Imagine A*, the algorithm we mentioned for pathfinding. It uses a heuristic—like the Euclidean distance in a map—to guide its search. Instead of checking every possible path (which could be infinite), the heuristic gives tips to our agent about where to look first, making the process much faster.

UC Berkeley

Most real-world problems are way too complex to solve optimally all the time, so AI uses heuristics to approximate solutions at high speed. If you think of your AI as a detective, heuristics are the hunches that lead it toward the best clues without wasting time.

Still, even the best heuristic can’t control all the uncertainty out there—after all, real life isn’t a neat little puzzle like a Pacman board.

Probabilistic Reasoning

UC Berkeley

In the real world, everything from partial observability (your agent doesn’t see the whole picture) to random events to adversarial behavior can introduce loads of uncertainty. Your agent can’t always know exactly where it is, what it’s dealing with, or how the future will unfold.

This is where all those statistics classes we took finally pay off. Yes!

Probabilistic reasoning allows an agent to assess how likely different outcomes are and choose actions that maximize its chances of achieving a goal.

From Bayesian Networks to Monte Carlo simulations, probabilistic reasoning is essential for dealing with messy, real-world scenarios. In fact, it’s the backbone of decision theory, which states that an agent is rational if it picks the action that yields the highest expected utility, averaged over all possible outcomes. This is called the principle of maximum expected utility (MEU).

Translation... combine the power of probability theory with utility theory (how much you value an outcome), and you get decision theory—the blueprint for making the best possible choice in an uncertain world.

But how do AI systems plan ahead? That’s where Markov Decision Processes (MDP) come in.

Markov Decision Processes (MDP)

When our agent needs to plan ahead in an uncertain world, it can’t just rely on a single action sequence. That’s where Markov Decision Processes (MDPs) swoop in as the blueprint for systematic decision-making under uncertainty.

What does “Markov” mean? it means that given the current state, the future is independent of the past. So, each action outcome depends only on where we are right now—not on the entire history of how we got here.

Think about the gridworld below, if the environment were deterministic, a solution would be easy: [Up, Up, Right, Right, Right].

Unfortunately, the environment won’t always go along with this solution, because the actions are unreliable. What should we do?

UC Berkeley

- Transition Model: We use P(s' | s, a) to define the probability of landing in state s′ if we take action 'a' in state 's'.

- Reward Function: We assign a reward (or penalty) for being in certain states or taking certain actions (like −0.04 per step to encourage faster solutions).

- Policy: Instead of a single path, we define a policy π, which tells our agent the best action to take for any state it finds itself in.

Why does this matter? Because in a chaotic, real-world environment—whether it’s a self-driving car dealing with slippery roads or a robot in a warehouse with uncertain battery life—the agent needs a backup plan for every possible scenario. That’s exactly what a policy provides.

Keep the above in mind, now let's tie it all together with my favorite topic on this article.

When an agent learns from interacting with an environment defined by an MDP, it typically uses Reinforcement Learning (RL), let's understand it better!

Reinforcement Learning

That's the gold in the end of the rainbow, when it comes to unlocking sophisticated reasoning capabilities, reinforcement learning (RL) is the master key that ties together all the foundational concepts we've explored—from search and heuristics to probabilistic reasoning and MDPs.

UC Berkeley

As described in Stuart Russell’s work, reinforcement learning allows agents to interact with their environment, learning through rewards and penalties rather than relying solely on pre-labeled examples.

In contrast to supervised learning, where millions of examples may be needed, RL leverages simple reward signals (like win/loss or progress metrics) to guide learning, and in a context where training a LLM is very costly, it can be the key to reduce it.

The DeepSeek papers showcase this beautifully. DeepSeek-R1-Zero, trained with pure RL, naturally developed impressive reasoning behaviors—self-verification, extended chain-of-thought, and even spontaneous “aha moments.”

Experiment, Experiment, Experiment!

Just reading about AI won’t make you good at it. The best way to unlock creativity and AI expertise is to play with AI in experimental settings.

🔹 Gymnasium (OpenAI’s Gym fork): A sandbox for RL, train AI to play games, control robots, or optimize tasks.

🔹 Pacman AI (Berkeley Course): A hands-on project where you build AI agents for Pacman— perfect for learning Artificial Intelligence with a practical project

AI is not just about models—it’s about experimentation. The more you test, break, and rebuild AI, the more you’ll innovate and become a reference in your field.

Conclusion

If you want to go beyond being a user of AI and become a builder, start by:

✅ Understand the AI fundamentals: like search, decision-making, probabilistic reasoning, RL

✅ Study Reinforcement Learning & Decision Theory

✅ Experimenting with OpenAI Gymnasium, Pacman AI, and multi-agent AI simulations

Follow me on linkedin and keep an eye on my blog (creativedev.art) if you want more hands-on insights on building creative AI solutions.

See you on the next article! 🚀

Reference:

- Stuart Russell & Peter Norvig’s Artificial Intelligence: A Modern Approach

- UC Berkeley CS188 Intro to AI

- AlphaGo Documentary from Google DeepMind

- DeepSeek Research Papers