Managing Large IAM Policies: Overcoming Character Limits

Jakub DuchoňJakub Duchoň
4 min read

Table of contents

Introduction

While working with AWS Identity and Access Management (IAM), you rarely have to think about IAM character limits. AWS enforces a hard character limit of 6,144 characters on managed IAM policies, while inline policies for IAM users and roles are capped at 2,048 and 10,240 characters, respectively. But in most cases, you can split large policies across multiple managed policies and attach up to 10 (or 20 with a quota increase) of them to an IAM role.

But that’s not always an option. Permissions boundaries and inline policies don't give you that luxury. You're stuck with only one policy, so it must fit within the character limits.

I faced this issue during a recent project, and I want to share with you how I tackled it while maintaining the intended scope of permissions and keeping security in the first place.

Before we start

👋🏼 Hi! Found this helpful? Let's connect on LinkedIn! I'm always open to discussing AWS automation, governance, and security solutions with fellow AWS engineers.

The problem

During a recent project, I needed to implement an extensive permissions boundary that was exceeding the managed-policy character limit. You cannot attach multiple boundaries to a single IAM principal, nor increase this hard limit. I found myself in a tricky situation.

The obvious solution was to use wildcards to shorten the policy. But doing this manually would be tedious and risky. One misplaced wildcard could accidentally expand permissions, and that wasn’t acceptable.

The solution

I found it to be an interesting challenge, so to address it programmatically, I created a simple Python tool.

It takes an IAM policy as input and identifies opportunities for shortening the policy statements by introducing wildcards. It then outputs a more compact policy that behaves exactly like the original but with significantly fewer characters.

The tool is open-source and available on GitHub at github.com/imduchy/iam-minify.

How it works

At its core, IAM-minify uses a Trie data structure to identify patterns in policy statements that can be represented more concisely. This structure is particularly well-suited for IAM policy optimisation as it organises action statements hierarchically, grouping common substrings/prefixes.

Here's how the algorithm optimises policy statements while maintaining the intended permissions scope:

  1. Building the Action Tree: The tool first constructs a Trie from all IAM actions specified in the policy, where each node represents a character of an action string (e.g., "GetObject").

  2. Pattern Recognition: The algorithm traverses each IAM action and evaluates potential substitutions by performing various logic checks. It truncates a policy only if it can guarantee that no additional permissions would be granted beyond the original policy scope.

  3. Merging overlaps: After the initial shortening, the tool further optimises the list of IAM actions by merging overlaps. This step removes redundant actions and ensures the shortest possible safe representation of permissions.

  4. Output: As the last step, it outputs the optimised (shortened) policy.

Here's a simple example:

Before optimisation:

{
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:GetObjectVersion",
        "s3:GetObjectVersionTagging"
      ],
      "Resource": "*"
    }
  ]
}

After optimisation:

{
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject*"
      ],
      "Resource": "*"
    }
  ]
}

The critical security aspect here is that, unlike manual wildcard introduction (which can easily lead to over-permissive policies), IAM-minify only introduces wildcards to precisely match the original permission set.

Here’s a slightly more complex example where I passed the AWSQuicksightAthenaAccess policy through IAM-minify.

Before optimisation (2280 characters):

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "athena:BatchGetQueryExecution",
        "athena:CancelQueryExecution",
        "athena:GetCatalogs",
        "athena:GetExecutionEngine",
        "athena:GetExecutionEngines",
        "athena:GetNamespace",
        "athena:GetNamespaces",
        "athena:GetQueryExecution",
        "athena:GetQueryExecutions",
        "athena:GetQueryResults",
        "athena:GetQueryResultsStream",
        "athena:GetTable",
        "athena:GetTables",
        "athena:ListQueryExecutions",
        "athena:RunQuery",
        "athena:StartQueryExecution",
        "athena:StopQueryExecution",
        "athena:ListWorkGroups",
        "athena:ListEngineVersions",
        "athena:GetWorkGroup",
        "athena:GetDataCatalog",
        "athena:GetDatabase",
        "athena:GetTableMetadata",
        "athena:ListDataCatalogs",
        "athena:ListDatabases",
        "athena:ListTableMetadata"
      ],
      "Resource": ["*"]
    },
    {
      "Effect": "Allow",
      "Action": [
        "glue:CreateDatabase",
        "glue:DeleteDatabase",
        "glue:GetCatalog",
        "glue:GetCatalogs",
        "glue:GetDatabase",
        "glue:GetDatabases",
        "glue:UpdateDatabase",
        "glue:CreateTable",
        "glue:DeleteTable",
        "glue:BatchDeleteTable",
        "glue:UpdateTable",
        "glue:GetTable",
        "glue:GetTables",
        "glue:BatchCreatePartition",
        "glue:CreatePartition",
        "glue:DeletePartition",
        "glue:BatchDeletePartition",
        "glue:UpdatePartition",
        "glue:GetPartition",
        "glue:GetPartitions",
        "glue:BatchGetPartition"
      ],
      "Resource": ["*"]
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetBucketLocation",
        "s3:GetObject",
        "s3:ListBucket",
        "s3:ListBucketMultipartUploads",
        "s3:ListMultipartUploadParts",
        "s3:AbortMultipartUpload",
        "s3:CreateBucket",
        "s3:PutObject",
        "s3:PutBucketPublicAccessBlock"
      ],
      "Resource": ["arn:aws:s3:::aws-athena-query-results-*"]
    },
    {
      "Effect": "Allow",
      "Action": ["lakeformation:GetDataAccess"],
      "Resource": ["*"]
    }
  ]
}

After optimisation (1719 characters, 25% fewer characters than the original):

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "athena:BatchGetQ*",
        "athena:CancelQ*",
        "athena:GetCat*",
        "athena:GetD*",
        "athena:GetE*",
        "athena:GetNames*",
        "athena:GetQueryE*",
        "athena:GetQueryRe*",
        "athena:GetT*",
        "athena:GetW*",
        "athena:ListD*",
        "athena:ListEn*",
        "athena:ListQ*",
        "athena:ListTab*",
        "athena:ListW*",
        "athena:R*",
        "athena:StartQ*",
        "athena:StopQ*"
      ],
      "Resource": ["*"]
    },
    {
      "Effect": "Allow",
      "Action": [
        "glue:BatchC*",
        "glue:BatchDeleteP*",
        "glue:BatchDeleteTable",
        "glue:BatchGetP*",
        "glue:CreateDatab*",
        "glue:CreatePartition",
        "glue:CreateTable",
        "glue:DeleteDatab*",
        "glue:DeletePartition",
        "glue:DeleteTable",
        "glue:GetCatalog",
        "glue:GetCatalogs*",
        "glue:GetDatab*",
        "glue:GetPartition",
        "glue:GetPartitions*",
        "glue:GetTable",
        "glue:GetTables*",
        "glue:UpdateDatab*",
        "glue:UpdateP*",
        "glue:UpdateTable"
      ],
      "Resource": ["*"]
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:Ab*",
        "s3:CreateBucket",
        "s3:GetBucketLoc*",
        "s3:GetObject",
        "s3:ListBucket",
        "s3:ListBucketM*",
        "s3:ListMultip*",
        "s3:PutBucketPu*",
        "s3:PutObject"
      ],
      "Resource": ["arn:aws:s3:::aws-athena-query-results-*"]
    },
    {
      "Effect": "Allow",
      "Action": ["lakeformation:GetDataA*"],
      "Resource": ["*"]
    }
  ]
}

Conculsion

IAM-minify is an answer to a very specific problem. It transforms what would take me hours of manual policy review and optimisation into an automated process that takes seconds. Beyond time savings, it removes the risk of human error when working with complex policies.

It was a fun challenge, and it forced me to brush up on my algorithm skills, which I barely use in my day-to-day work.

If you've ever struggled with IAM policy character limits or are just curious, give IAM-minify a try. The tool is available at github.com/imduchy/iam-minify.

Before you go

👋🏼 Hi! Found this helpful? Let's connect on LinkedIn! I'm always open to discussing AWS IAM, governance, and security solutions with fellow AWS engineers.

0
Subscribe to my newsletter

Read articles from Jakub Duchoň directly inside your inbox. Subscribe to the newsletter, and don't miss out.

AWS IAMAWSPythonalgorithms

Written by

Jakub Duchoň
Jakub Duchoň

Hi, I'm Jakub 👋🏼 I'm an AWS consultant and full-stack developer, currently working as an AWS consultant. I've been involved in AWS since the beginning of my career, working primarily in "cloud enablement" teams, focusing on identity and access management, landing zones, governance, security, automation, and DevOps. Occasionally, I write about the interesting things I came across while helping clients on their AWS journey. This blog is where I share lessons learned, ideas, and tips, hoping some find it useful.