The Agentic Way

Bishwamohan JenaBishwamohan Jena
8 min read

Here is a blog stating 8 key engineering & design principles while working with LLM applications & agentic systems. This might be helpful for AI engineers working in LLM-based systems.

If you don't know about AI agents then you can refer to this blog by Anthropic.
Other resources are mentioned below the article.

1. Context > Prompt – Engineering the Real Input to LLMs

Prompt engineering got all the hype, but context engineering is where robust LLM systems are really built.

In early LLM exploration, clever prompts reigned supreme. But production-grade applications demand more than just “cleverness” — they need structured, dynamic, relevant context. That means feeding the model not just a task, but the entire situation: prior interactions, examples, retrieved data, memory, and even tool access if needed. It boils down to what does your model need to know right now so that it can take a decision sensible as per the scenario.

Why it's tricky:

  • Too little context = bad answers, hallucinations, and frustration.

  • Too much context = high costs, model confusion (the "needle in a haystack" problem), and slower responses.

  • Bad format = irrelevant outputs, misinterpretations.

Good context is:

  • Relevant (to the query)

  • Compact (fits in token limit)

  • Fresh (includes latest facts)

  • Structured (clearly organized)

💡 Tip: Build a context builder layer before every LLM call, not just a hardcoded prompt string.

Here is a post where Andrej Karpathy talks about that.

2. FAISS > Just RAG – When Search Deserves Optimization

When you need to find similar information fast, especially across massive datasets of text, FAISS (Facebook AI Similarity Search) is often a better choice.

While other methods exist, FAISS excels in its ability to perform highly efficient similarity searches on dense vectors.

Why is it "better"? It boils down to optimized indexing and search algorithms designed for scale. When you're dealing with millions or even billions of vector embeddings (the numerical representations of your text), a simple brute-force search becomes impractical.

FAISS offers various indexing structures that significantly speed up the search process, making it indispensable for real-time RAG systems or large-scale content recommendations.

⚠️ Note: A good index with basic models often outperforms a fancy model with no index.

Here’s more info about vector embeddings.

3. Not Everything Needs to Be an Agent

You don’t always need autonomous agents. Sometimes, a simple function is the best agent.

Agent frameworks like LangGraph and AutoGPT can feel powerful—but overusing them leads to fragility, slowness, and cost inflation.

When to Think Agents (and When Not To)

Here's a crucial consideration for any LLM application developer: Agents aren't always the answer. When building applications with LLMs, always aim for the simplest solution possible and only increase complexity when genuinely needed.

Agentic systems often trade increased latency and cost for better task performance. You need to consider if that tradeoff makes sense for your specific use case.

Use agents when:

  • Flexibility is key

  • Dynamic tool use is needed

  • Model-driven decision-making is required at scale

Otherwise, simple LLM + RAG + good in-context examples may be enough.

4. Parallel When Possible, Sequential When Critical

Building performant systems often feels like a race for speed. We want everything to happen instantly! However, true engineering brilliance lies in finding the right balance.

A crucial lesson in system design is that you can leverage parallelism for speed without sacrificing stability.

Example:

Implementing parallel PDF downloading using thread pools can drastically cut down the initial data acquisition time. But keep the summarization sequential. Why? Because summarization, especially with LLMs, can be resource-intensive and prone to instability.

Speed without stability is chaos; stability without speed is stagnation.

⚠️ Note: Sequential is sometimes faster — if it avoids retries, failures, or poor UX.

How to use multiprocessing in Python? video

5. LLM-Aware Tool Orchestration

It's tempting to think of LLMs as the ultimate "answer engines," capable of handling everything. But a more robust and reliable approach is to use LLMs as intelligent controllers and orchestrators of tools, rather than expecting them to generate the full response directly.

Instead of letting the LLM directly answer complex queries, we can designed it to coordinate various sub-tools that perform the action independently. This approach makes your system far more deterministic, interpretable, and less prone to hallucinations. It's a significant leap forward from naive "LLM-only" assistants, transforming the LLM from a sole performer into a masterful conductor of an entire orchestra of specialized tools.

Consider a real-world scenario:

Financial Analysis Tools: An LLM might be tasked with analyzing a company's financial health. It doesn't just "know" the answer. Instead, it uses tools to:

  • Retrieve the latest quarterly earnings report from a financial data API.

  • Call a spreadsheet tool to perform specific calculations (e.g., profit margins, debt-to-equity ratio).

  • Access a news sentiment analysis tool to gauge public perception.

  • Finally, the LLM compiles these tool outputs into a comprehensive financial overview.

In this case, the LLM's true power comes from its ability to intelligently decide when to use a tool, which tool to use, and how to interpret its results, leading to far more accurate, reliable, and grounded responses.

6. The Human Touch: Human-in-the-Loop for True Autonomy

When we talk about autonomous systems, it's easy to picture fully self-operating machines. But for complex, critical applications, true autonomy often involves a strategic human element. The principle of Human-in-the-Loop (HITL) control is vital for building robust LLM-powered systems..

This means designing your system to allow human intervention and feedback at critical junctures. Whether it's for validating a generated output, refining a search query, or correcting an interpretation, integrating human oversight provides a crucial safety net and a continuous feedback mechanism. It's not about stifling autonomy, but rather about ensuring accuracy, safety, and continuous improvement, especially as the system learns and evolves. An "autonomous" system that's also accountable and adaptable to human input is truly powerful.

A robust LLM system invites human intervention at critical junctures, such as:

  • Output validation

  • Search refinement

  • Result interpretation

7. When Small Teams Outsmart the "Big Brain" (Mixture-of-Experts Paradigm)

When we talk about building intelligent systems, the natural inclination is to create one massive, all-knowing "brain" model. A single, monolithic LLM that handles every conceivable task. But this seemingly powerful approach is often inefficient, costly, and can even limit overall performance.

A far more effective strategy, particularly for complex and scalable LLM applications, is to embrace modular agent design, drawing profound inspiration from the Mixture-of-Experts (MoE) paradigm that's been gaining significant traction in the research community (think Mixtral, for instance).

💡 What is Mixture-of-Experts (MoE)?

Imagine an AI system as a team of highly specialized consultants. Instead of one generalist trying to answer every question, you have experts in different domains: one for legal queries, another for coding, a third for creative writing, and so on.

A smart "gating network" acts as a dispatcher, intelligently routing each incoming query to the most relevant expert (or a small group of experts) for processing. Only the parameters of these selected experts are activated, not the entire model.

Monolithic LLMs:

Try to learn everything within one giant set of parameters.
This can lead to shallower knowledge across many domains, as the model struggles to balance competing expertise.

Modular Agents :

Each "expert" or specialized agent focuses on a distinct responsibility or data domain.
This allows for deep, highly refined expertise within each module.

🔻 The Problem with Monoliths:

As models grow, computational cost for both training and inference skyrockets. Every single inference requires activating all billions or trillions of parameters, even if only a tiny fraction is relevant to the current task.

🔺 The MoE Advantage:

By routing tasks to the most suitable sub-agent, you activate only a small, relevant subset of the model's total parameters. This leads to drastically reduced computational cost per inference, making systems faster and cheaper to run, even with an overall much larger number of total parameters.

You can scale the capacity of your system (by adding more experts) without proportionally increasing the computation for every single query.

8. The Power of Parallelization (Sectioning and Voting)

LLMs, despite their serial nature, can often work simultaneously on different parts of a task, with their outputs then aggregated programmatically. This workflow, parallelization, manifests in two key variations:

1. Sectioning:

Breaking a larger, complex task into independent subtasks that can be run in parallel by different LLM calls. Each LLM focuses on a specific section, and their outputs are then combined to form the complete result.

2. Voting:

Running the same task multiple times with slight variations or different models to get diverse outputs. This is particularly useful for tasks where accuracy is paramount, allowing you to aggregate results (e.g., take the majority vote) to achieve higher confidence.

Parallelization is highly effective when subtasks can be truly independent, leading to significant speed gains. It's also invaluable when multiple perspectives or attempts are needed to enhance confidence or explore different facets of a complex problem. For intricate tasks with many considerations, having LLMs focus on each aspect separately, perhaps in parallel, generally leads to better, more focused results.

💡 Use when tasks are large, high-stakes, or benefit from diverse outputs.

🔗 Further Reading & Resources

Thanks for reading! 👋
Have feedback or questions? Let’s connect in the comments.

Let’s Connect

If you found this useful or have thoughts to share, I’d love to hear from you:

1
Subscribe to my newsletter

Read articles from Bishwamohan Jena directly inside your inbox. Subscribe to the newsletter, and don't miss out.

agentic AIllmlanggraphlangchainfaissAI Engineering

Written by

Bishwamohan Jena
Bishwamohan Jena