AI as Observer: Chronicling Tabletop RPGs

Last night at the gaming table…

They ascended cautiously, weapons ready. In a chamber on the upper floor, they found her - an emaciated figure kneeling within a circle of power. On either side stood skeletal guardians, each bearing six arms laden with ancient weapons. They turned toward the party with terrible purpose as the woman raised her hollow eyes.

"You've made it here," she rasped. "Perhaps for good or ill, you now share my fate. If we are to survive this place, it will take all of us."

I’m in a gaming group that plays a weekly in-person Pathfinder 2e game. Tabletop roleplaying games like Pathfinder 2e thrive on collaboration and creativity, but keeping track of every twist, turn, and NPC encounter can feel like a chore. In my group, no one is really interested in taking detailed notes, even though we all have laptops at the table.

I'm a tech nerd who likes to play with new tools, so I decided to see if AI could help.

For our current mini-campaign, Sands of Yore, I’ve adopted a workflow that uses audio recordings, transcription tools, and AI language models to generate session notes and weave an ongoing narrative. In this post, I’ll walk you through how I set this up, from the hardware to the software to the AI processing. It’s changed how we keep track of everything without disrupting the play at the table.

At the end of the post, you'll find links to the prompts and other tools I use, as well as the full campaign narrative as it exists today.

The Setup

Hardware: Recording the Session

I started out simple: just recording our sessions with the built-in mic on my MacBook Pro. It worked, but not well. Audio quality was sketchy, especially for players sitting farther from the laptop which led to sketchy transcripts. Over time, I upgraded to two flat condenser microphones designed for conference tables (Shure CVB-B/O). These are zip-tied to the ceiling in the dedicated game room we play in, keeping the table clear while capturing great audio. The mics connect to a TASCAM Portacapture X6 audio interface that plugs into the MacBook via USB-C. This audio hardware is overkill, but if you have seen my other posts, you’ll probably not be surprised.

Transcription: Turning Audio into Text

I’ve tried several transcription tools and am still iterating on this process:

• Whisper: Open-source and free, Whisper processes audio locally. It’s decent but doesn’t have speaker recognition and occasionally has weird issues, like looping errors in the transcript. It also requires several manual steps to process the audio after each session.

• Apple Notes: The new Apple Intelligence transcription let's you add an audio recording to a Note and it will generate a transcript from this audio when you stop recording. It's on par with Whisper. It’s local and free (as long as you’re in the Apple ecosystem), but it makes my laptop grind to a halt for several minutes when processing audio from a four-hour session and doesn’t identify speakers either.

• Otter AI: This cloud-based tool was the best by far for transcript quality. It’s accurate, identifies speakers, and even transcribes live during the session. The downside? It’s not free, and you’re uploading your audio to a third-party service, which raises privacy concerns.

Interestingly, even the lower-quality transcripts worked fine for our next step, but Otter made things easier by avoiding having to manually process audio files. I think the speaker identification (diarization) might help the LLM understand the transcript so I want to find a way to get that without relying on a cloud too. I’m looking at automating Whisper with a separate speaker diarization tool via n8n to get similar results without the privacy trade-off and with more automation. That’s still a work in progress.

The AI

Once I have a transcript, this is where the AI comes in. I’ve tried both ChatGPT and Claude for this process, and Claude is the current winner for our workflow, both because Sonnet 3.5 seems to be better at this kind of text processing but also because of the power of Claude "Projects". Here’s what we do:

1. Create the Transcript: Process the session audio into a text file and save it to our shared Google Drive.

2. Summarize the Session: Using a generic summary generation prompt, I paste the transcript into Claude and let it work its magic. If something important gets missed, I ask for tweaks to ensure nothing critical is left out. The prompt is generic, and campaign specific details come from reference documents that are part of the project (see below)

3. Generate the Narrative: A second generic prompt turns the same transcript into a more creative narrative, fleshing out character moments, world details, and story arcs. Again, I’ll ask Claude to refine it if needed.

4. Save and Share: Both the summary and narrative go into shared Google Docs. Everyone in the group can review them before the next session, and Claude uses them to keep continuity week-to-week.

The Claude project is tied to our Google Drive, so the LLM can reference background details to improve accuracy and depth:

• World-building notes from our GM that detail the setting, deities, and lore.

• A reference doc with PCs and key NPCs, including descriptions, motivations, and relationships.

• Previous session summaries and the ongoing narrative.

Storing everything in a shared Google Drive (including prompts) helps keep it organized and easy for everyone to access. And when you add a document to a Claude Project from Google Drive, Claude monitors it for changes over time and so the AI always have the latest content even as people make changes outside of Claude.

Conclusion

I'm very happy with this workflow.

• Imperfect is fine: Transcripts don’t need to be perfect. No one reads the raw transcripts, so their quality only needs to be good enough for the AI to work with. The AI can handle a lot of cleanup and still generate great results.

• Context limits: Claude’s project context capacity grows with each session. And 4-hour transcripts are big. If it becomes an issue, we might need to split the project into separate narratives and summaries or trim the background details.

We’re using this system for a short campaign of 5-8 sessions, but it’s been so effective that we’ll definitely bring it into our next long-term campaign. Having AI handle note-taking and storytelling not only saves time but also adds a richer narrative layer to the game. It’s like having an extra player at the table whose sole job is to record and enhance the story.

If you’re struggling with keeping notes or want to try something new with your Pathfinder or Dungeons & Dragons game, give it a shot. AI may be able to take the responsibility for tracking your sessions—and you’ll have some amazing stories to show for it.

References

• These are the two prompts I use: Summaries and Narratives

• Our full current campaign narrative: The Sands of Yore, an Adventure

• Our session summaries: Session Summaries

• Audio hardware used is two of these microphones: Shure CVB-B/O, and this audio interface: TASCAM Portacapture X6

• Audio transcription apps tried: Whisper Transcription, Apple Notes, Otter.ai

• AI platform: Claude