Copilot Bootcamp

Recently, I went to the Copilot Studio bootcamp at the head office of Microsoft in Denmark. It was a three day event where the Power CAT (customer advisory team) presented different learnings and led you through different labs focused on Copilot Studio. It was a great event to learn more about Copilot Studio and to see what everyone’s stance was on this technology. I wanted to take some time to write down what my key takeaways were and give my recommendations. These recommendations are the same ones I gave my team at Schiphol Airport Group where I am currently working.

Copilot Studio

Quickly a recap, although I think most readers of this article will know what Copilot Studio is. It is the equivalent of Chat GPT but then integrated within the ecosystem of Microsoft. Copilot Studio is, to be precise, a way to create “lightweight” Chat GPT’s which can also perform actions for you as a user. It is Microsoft’s push to keep up with the innovations we see on the market now with artificial intelligence.

Key Takeaways

Early-stage & fast-moving

What became clear and was stated early on is that everyone is trying to figure out how to use this new software. Even the Power CAT team also struggles with this learning curve. This team helps some of the biggest customers of Microsoft integrate Copilot Studio agents into their processes and they have short communication lines with the product team of Copilot Studio at Microsoft. This makes them able to tell the engineering team what is needed and what enterprise customers ask for. But, things are moving fast. New features are coming out on a weekly basis and this means everyone is learning on the fly how to use these features. At the same time they are trying to integrate new features in existing agents where it is possible that technical debt quickly piles up due to development work that becomes obsolete with new features. This makes it hard to anticipate what to develop.

Strong optimism

There was a broad agreement among the attendees that Copilot Studio would be a major disruptor at companies. This of course could be the confirmation bias bubble I was but nevertheless there was consensus among everyone: this was going to be the future. And I have to agree, currently I am using Chat GPT in my work on a daily basis. But, it is not connected to any of my applications which I use on a daily basis. Microsoft Teams chat, Azure DevOps Wiki and Jira are some of the applications I use on a daily basis and it would be great if I can search my DevOps wiki or create new Jira tickets on the fly in my teams chat while discussing new feature developments. And then I haven’t even mentioned other office employees who spend hours per week answering emails, looking for information in files or filling in Excel sheets. Copilot Studio could be the solution for these hours spent.

Few success stories

Nevertheless, very few production agents run stably with minimal defects; success stories are still rare. There weren’t many people at the bootcamp who already had stable agents running in production for clients or at their company. There is also the problem of the end user: they aren’t used to this new technology and give up too easily. While the product itself sometimes does not show the right behavior I also heard stories that users needed a lot of training and guidance into how to use this new product.

Licensing ambiguities

Furthermore there were a lot of questions about the message-count consumption and the cost drivers remain opaque. There are numerous ways of counting up the message consumption of your agent and while we have been used to a license based consumption in the past, Copilot Studio follows more of the Azure styled consumption where you are not entirely sure at the beginning of the month what your spending will be by the end. Another example of this are the MCP servers that are available through a click of a button in your tools section. It is not entirely clear yet how many messages these consume.

Feature decision gaps

Since new features are constantly coming out and the software is so new it is not entirely clear what to use when. For instance, guidance on when to use an Agent Flow versus discrete actions is still emerging. Or choosing between MCP server tools and standalone tools which come out of the box. The fact that we now use descriptions to guide an orchestrator does not make this any easier. At the same time we do not want to fall back into a chatbot structure where we draw out all the topics as we used to do in the Power Virtual Agents (the Copilot Studio predecessor) times.

Recommendations

Looking at my key takeaways and hearing about some things that were presented at the bootcamp I do have some recommendations for my team. We are going to continue with these recommendations for the foreseeable future. Also, since we currently still seeing the big push from Microsoft towards this agentic future.

Install the Copilot Studio Kit

At the end of the three days the Power CAT Team presented the Copilot Studio Kit. The Copilot Studio Kit is created by the Power CAT team and provides invaluable tools for creating and managing Copilot agents. It has a way of automating your tests for Copilot Studio agents, it has an adaptive card gallery, you can get a tenant-wide view of your agents and there are other components which are valuable and which you can use without inventing the wheel yourself. You can see this as the equivalent of the Centre Of Excellence toolkit. It is still supported and people are actively developing on it.

Start creating agents

Although I mentioned in my takeaways that there are few success stories this does not mean you stand still and do nothing. Start creating agents but be aware of bugs and limitations. My recommendation would be to not yet integrate critical processes into your agents. But, if you do not start investigating this technology and what it can do to increase the productivity of the people in your company you are going to fall behind. I discovered that there are many caveats and development decisions to take during the bootcamp which you can only learn when working on an agent of your own. So do not wait until it is too late, start now.

Monitor message consumption and Dataverse storage

Monitor messages and storage closely since it is very ambiguous and sometimes unclear how much is consumed and used. You can do this easily in the admin center of the Power Platform or on the agent configuration page itself. At the same time you can use the pay as you go feature to not be stuck with hundreds of unused messages at the end of the month. Also, your agents’ conversation transcripts are stored in Dataverse so if you really take off with agents you might end up using some extra Dataverse storage.

Find and document best practices

When best practices are found document them thoroughly so future builds can rely on them. This is something you should do in your team with any software you are using but for Copilot Studio it can be extra handy since there are not as many community members creating videos and blogs as for other software packages such as Power BI or Power Apps. Document the best practices and stick to them so you are going to have an easier time building that second and third agent!

Final thoughts

I love artificial intelligence and this whole Large Language Model wave we are riding now is exciting to me. I am able to create code faster and in languages I do not master. Copilot Studio is Microsoft’s answer to jump on the wave and I am here to ride it. Although, there are still some hurdles to take I see Microsoft is working hard to smooth these out. You need to stay in the loop before this takes over and you are left behind!