What AI Transparency Is — and What It Isn’t: Layered Prompting vs. Interleaved Thinking

In my recent research, I came across a paper titled Explainable AI in Multi-Agent Systems: Advancing Transparency with Layered Prompting by Bashir Faisal and David Tunkel (Feb 2025). I wanted to highlight this paper because it introduces a term that I see coming up more and more often: prompt layering, sometimes referred to as layered prompting.

One of the things I appreciated about this paper is that it doesn’t just talk about transparency in theory—it offers a design approach that can actually be implemented. Too often, transparency is treated as an abstract goal. This paper grounds it in something practical: layered prompting. That alone makes it worth studying.

The reason I’m sharing this is because, as these ideas around AI transparency continue to grow, I’ve noticed this particular term tends to get blended together with other concepts like interleaved thinking or general reasoning behavior. While it’s understandable that these ideas might seem interchangeable, it’s important to pause and look at what they really mean—and how they’re different. Especially if we want to architect or design systems that are truly transparent and auditable.

Layered prompting is introduced as a method that helps agents break down their thinking into clear, easy-to-follow parts.

It’s a way to give shape to the agent’s reasoning so that we can see each step as it happens, in language that makes sense to people. Instead of just giving a final answer, the agent walks us through its thinking—what it noticed, what it assumed, how it worked things out, and what it decided.

“This enables modular thinking, justification surfacing, and integration of human feedback at multiple reasoning stages” (Faisal & Tunkel, 2025, p. 1).

That visibility is what sets layered prompting apart from other design patterns. It’s not just about how the agent thinks—it’s about how it shares its thinking back with us.

Think of it like a recipe.

I like to use baking a cake as an analogy. With layered prompting, you're not just asking the agent to give an answer—you’re giving it a recipe for how to show its thinking.

If you bake a cake with four layers—Observation, Assumption, Reasoning, and Decision—then that’s exactly what you’ll see when the agent responds. You’re not leaving it up to the agent to guess what structure to follow. You’re telling it, “This is how I want you to explain yourself.” And that recipe doesn’t just help the agent stay organized. It helps the rest of us—developers, researchers, stakeholders—see exactly what’s happening, layer by layer, all the way down.

Here’s how those four cake layers might play out with an agent that’s given the following mock task: “Summarize the key takeaways from the following article about rising sea levels and proposed coastal policy responses in under 100 words.”

• Observation:
  “The input text mentions a rising trend in sea levels along coastal regions.”

• Assumption:
  “I’m assuming the user wants a policy-related summary, not just scientific data.”

• Reasoning:
  “Given the policy angle, I’ll focus on legislation mentioned in the latter half of the text and summarize it first.”

• Decision:
  “I’ll respond with a 3-sentence summary highlighting coastal impact, recent bills, and projected timelines.”

Agent’s Response: “Rising sea levels are accelerating along U.S. coastlines, threatening infrastructure and ecosystems. The article highlights new legislative efforts aimed at shoreline protection, funding for climate resilience, and stricter building regulations in high-risk zones. Scientists emphasize the need for immediate policy action, while some regions push back due to economic concerns. The piece concludes by urging cross-sector collaboration to adapt to long-term environmental changes.”

Seeing the agent’s response is useful—but seeing how it got there is what makes it trustworthy. Each layer gives us insight into how the agent interpreted the task, what assumptions it made, and why it chose one path over another.

This is especially important in multi-agent systems, where complexity can scale quickly. Without a method like layered prompting, most of what agents are doing remains hidden. You might see the final output, but you have no idea why the agent made a certain choice, or whether it took a shortcut, misread something, or carried hidden bias. That’s why transparency matters—not just in theory, but in practice. When systems affect people’s health, safety, finances, education, or rights, we need to be able to ask: how was this decision made, and can we trust it?

That’s why transparency matters—not just in theory, but in practice. When systems affect people’s health, safety, finances, education, or rights, we need to be able to ask: how was this decision made, and can we trust it?

The study by Faisal and Tunkel (2025) offers several concrete examples of why this matters. When agents explained themselves step by step, users were much better at spotting reasoning errors and understanding how decisions were made. One section puts it this way:

“Layered prompting improved debugging efficiency by allowing developers to interrogate specific components of the decision-making process rather than relying on post-hoc interpretation methods that often lack clarity. A key finding from the study was the impact of layered prompting on user confidence in AI-driven recommendations. Participants reported higher trust in AI outputs when explanations followed a sequential, stepwise format that mirrored human reasoning.”
(Faisal & Tunkel, 2025, Results and Discussion section, p. 7)

These may seem like small technical choices, but they have a big impact. You can’t fix what you can’t see—and layered prompting gives us a way to see inside decisions we used to have to take on faith.

Now, let’s clear up a common confusion.

When people hear “prompt layering,” they often associate it with interleaved thinking. And I want to slow that down for a second because they are not the same thing.

Interleaved thinking is about how an agent reasons internally. It pauses between steps, revises its thoughts, anticipates what might happen next. It’s more reflective than linear reasoning, and it’s powerful in complex environments. But that thinking process, on its own, is still invisible. Unless you explicitly surface it, interleaved thinking is just that—thinking. Like someone sitting at the edge of their bed wondering whether to make a sandwich: Did I buy bread? Am I in the mood for toast? Is the knife clean? It’s all happening internally, quickly, silently. Unless you share it, no one knows.

And that’s why interleaved thinking, while helpful, is not transparency by itself. It only becomes useful when paired with something that can expose those thoughts to the outside. I explored this in more depth in my post Rethinking Citation Workflows with Multi-Agent Systems and Interleaved Thinking, where interleaved thinking helped agents pause and reflect, but without a method to surface those reflections, their reasoning remained hidden. That’s where mechanisms like layered prompting come in—they make that invisible thought process visible. They give structure to the agent’s reasoning and let others trace what it’s doing and why.

That visibility—surfacing the invisible—requires structure. And while layered prompting gives you control over that structure, it’s not the only method that helps reveal an agent’s thinking.

Enter another transparency method: Thought Blocks.

Thought blocks are a kind of default reasoning format used by Claude, Anthropic’s language model. When Claude responds, it sometimes labels parts of its output with cues like “Thought:”, “Action:”, or “Observation:”. These markers—what Anthropic calls thought blocks—offer a peek into the model’s decision-making flow. For example:

• Thought: I should check the user’s input for known error patterns.

• Action: Running the error-check tool.

• Observation: The tool flagged a formatting inconsistency.

That’s a form of transparency, too—but it works differently. Thought blocks aren’t something you, as a designer, define. They’re built into the model’s behavior. You can’t easily control which labels appear or how deep the model’s reasoning goes. It’s more like slicing into a cake someone else made: you can see the layers, but you didn’t choose how they were made.

With layered prompting, you bake the cake yourself. That’s the biggest difference. You define the structure, the sequence, and the level of detail. You might ask an agent to walk through its process in four layers—what it saw, what it assumed, how it reasoned, and what it decided. Or you might prompt it to surface ethical concerns, confidence levels, or alternatives. Whatever the format, the structure is yours to layer however you choose. They can be designed to match your values, risks, and system goals. That’s a key distinction.

Anthropic puts it this way:

“Transparency into the model’s mechanisms allows us to check whether it’s aligned with human values—and whether it’s worthy of our trust.”
(Anthropic – Tracing the thoughts of a large language model, 2024)

That’s the core idea. Thought blocks let us observe what the model is doing. Layered prompting lets us design how that observation happens—and do it consistently, across agents, teams, and use cases.

To bring it all into focus, I find it helpful to break transparency into two areas:

1. Agentic activity – This includes the prompts, tools, models, and reasoning the agent uses. It’s the decisions it makes, how it interacts with others, and the internal logic behind its behavior. Most of this stays hidden unless we intentionally surface it.

2. Transparency mechanisms – These are the methods or tools that make that hidden process visible. Without them, we’re left inferring behavior from outputs. With them, we can trace what’s happening under the hood—and make smarter, more accountable improvements.

To recap:
Interleaved thinking is how an agent reasons internally, but it’s not visible on its own.
Thought blocks expose some of that reasoning, but in a format you don’t fully control.
Layered prompting is a flexible design tool that lets you decide what structure to use, how visible it should be, and what needs to be traced.

That’s what makes it such a powerful step toward trustworthy AI. It’s not just about what the agent thinks—it’s about how we choose to see it. Transparency isn’t just a feature; it’s a responsibility. When systems affect real people, their decisions ought to be visible, traceable, and open to scrutiny. That’s how we build accountability. That’s how we build trust.

By no means are these methods perfect. There are still major strides—and real hurdles—we need to face as AI continues to weave itself into more parts of daily life. Transparency tools like layered prompting and thought blocks are just the beginning. In upcoming posts, I’ll be featuring research that explores these challenges more deeply, along with ideas for how we might build more trustworthy, accountable systems in the years ahead

Thinking about implementing AI or multi-agent systems? I’d love to help or answer any questions you have. I also offer workshops and strategy support—learn more on my website!