I Tried Using AI + RAG to Automate Compliance (APRA CPS 234 & ISO 27001) — Here’s What Happened

Lately, I’ve been tinkering with an idea that sits right at the intersection of AI, cybersecurity, and compliance frameworks. If that sounds like a strange mix, it kind of is — but hear me out.

Why I Started This

As a security engineer, I’ve lost count of the number of times I’ve been asked the classic compliance question:

“Can you show me evidence that satisfies both APRA CPS 234 and ISO 27001?”

If you’ve ever worked in risk, audit, or security assurance, you know this is where the pain begins. You end up hunting through policies, SIEM dashboards, log exports, and access control matrices to prove that yes, you really do have MFA enabled and yes, admin privileges are locked down.

It’s repetitive. It’s manual. And honestly, it’s the kind of work screaming to be automated.

That’s where my little experiment comes in: a Retrieval-Augmented Generation (RAG) system for compliance evidence.

The Big Idea

The concept is pretty simple:

1. Ingest your controls and evidence into a vector database (I used Qdrant with Semantic Kernel).

2. Map them against compliance frameworks like APRA CPS 234 (an Aussie banking standard) and ISO 27001 (the international information security standard).

3. Query in natural language:

   • “Show evidence that satisfies APRA CPS 234 and ISO access control.”

   • “What do I have for ITGC change management?”

And instead of scrolling through a SharePoint folder jungle, you get structured outputs like:

• Enforce MFA ✅

• Restrict admin privileges ✅

• Unique IDs ✅

• Role-based access ✅

• Central sign-in logs (immutable 12 months) ✅

Evidence attached: SIEM dashboard, sample log export, policy doc.

That’s the magic. It doesn’t just say “yes/no,” it actually points to where the proof lives.

Why APRA CPS 234 and ISO 27001?

I chose those two for a reason.

• APRA CPS 234 is a regulation in Australia that banks and financial services have to comply with. It’s all about protecting customer data and having tested incident response.

• ISO 27001 is broader and international — the gold standard for information security management systems.

If you can map your controls to both, you’re not only covering local regulatory ground but also aligning with global best practice. That’s powerful.

What I Learned (So Far)

• Framework language is tricky. “Access control” means slightly different things in APRA vs ISO, so you need to normalize how you tag controls.

• IDs matter. Qdrant yelled at me when I tried to push “AC-001” as an ID (apparently it only likes UUIDs or integers). Lesson learned.

• Evidence must be structured. A screenshot dumped in a folder won’t cut it — you need metadata: what control it satisfies, what framework it links to, and where it lives.

• The “aha moment.” Running my first query and seeing the model return a nice, clean mapping of MFA → APRA + ISO was honestly exciting. It felt like a glimpse of how compliance could work in the future.

Why This Experiment Matters

I’m not building a product here (yet). This is an experiment to explore how LLMs can reduce the grunt work in compliance. If you can turn framework crosswalks and evidence gathering into a conversational search, you:

• Save weeks of audit prep.

• Give auditors more confidence in your controls.

• Free up security engineers to focus on real risks instead of paperwork.

It’s not perfect. There are challenges around trust, evidence freshness, and context accuracy. But the direction feels right.

How I Built It (Step by Step)

This isn’t a polished product yet — but here’s what I did to get a working prototype.

1. Create a Conda environment

conda create -n compliance-rag python=3.12
conda activate compliance-rag

2. Install dependencies

   I used Semantic Kernel with Qdrant as the vector DB.

pip install semantic-kernel qdrant-client pyyaml

3. Define your controls

   I kept my compliance controls in a simple controls.yaml file. Example:

- id: "ac001"
  name: "Enforce MFA"
  description: "Multi-factor authentication is enforced for all admin accounts."
  frameworks:
    - ISO27001:A.5.15
    - APRA:CPS234:Controls/Testing/Incident
  evidence:
    - "SIEM dashboard screenshot"
    - "Access control policy"
    - "Sample log export"

⚠️ Gotcha: Qdrant doesn’t like string IDs like AC-001. It only accepts integers or UUIDs. I switched to lowercase UUID-style (ac001, ac002) to keep it simple.

4. Ingest into Qdrant

RESET_INDEX=1 conda run -n compliance-rag \
  python -m compliance_rag.cli ingest data/controls.yaml

5. Run a natural language query

RESET_INDEX=1 conda run -n compliance-rag \
  python -m compliance_rag.cli \
  "Show evidence that satisfies APRA CPS 234 and ISO access control" \
  --framework APRA:CPS234:Controls/Testing/Incident \
  --framework ISO27001:A.5.15

And the system responds with something like:

- Enforce MFA; restrict admin privileges; unique IDs; role-based access; 
  central sign-in logs (immutable 12 months) and alert on anomalous admin actions.  
Evidence: SIEM dashboard, policy, sample log export.  
[id: ac001 | frameworks: ISO27001:A.5.15, APRA:CPS234:Controls/Testing/Incident]  
Source: https://www.cdr.gov.au/for-providers/it-requirements-data-recipients

Sample Implementation

If you want to try this out yourself, I’ve pushed my code to GitHub here:
👉 github.com/viraj2252/compliance_rag

The repo includes:

• CLI tool for ingesting YAML controls into Qdrant

• Query runner for natural-language compliance checks

• Sample controls.yaml with APRA + ISO mappings

• Instructions to set up your own environment

It’s still a work-in-progress, but it should give you a head start if you’re experimenting with compliance + LLMs.

---

What’s Next

I want to:

• Expand beyond APRA and ISO to include CDR minimum controls and ITGC (SOX-style) checks.

• Experiment with attaching actual PDFs/logs/policies instead of just summaries.

• Add some kind of “confidence score” to see how strong the evidence match really is.

This is still early days — but it’s a fun playground for combining compliance frameworks, vector databases, and generative AI.

---

👉 Over to you:

• Would you trust AI to help prepare your audit evidence?

• What frameworks should I map next?