The Ultimate GitLab Import/Export Toolkit for Engineers


“I’m a DevOps engineer - if I’m doing it manually twice, I’m writing a script.”


A few weeks ago, I was handed a deceptively simple‑sounding task: spin up an entire playground environment, pipelines, history, everything for a complex micro‑services platform, from scratch.
The catch? All source code lived in GitLab, and there were dozens of projects spread across nested groups.
I scoured the web for a clean “export → import” recipe that would:

  • move commits, MRs, releases, tags, wikis - the whole lineage

  • preserve branch protections & default branches

  • run reliably for tens of repos without babysitting

Even GitLab’s docs stopped short of a full answer. Most blog posts boiled down to “click this button N times” or “write a bash loop and pray.”
Manual migration? Over my dead keyboard.
So I cracked open the GitLab API, dusted off my Python muscle memory, and the GitLab Migration & Branch‑Housekeeping Toolkit was born.


Why Another Solution?

Existing OptionsWhat This Toolkit Delivers
UI‑only exports (one project at a time)Bulk, unattended export/import of entire groups
3rd‑party “all‑in‑one” SaaS (pricey, opaque)100 % open‑source Python - reads every line
Incomplete migrations (no MRs, issues)Captures full project state, Inc. metadata
Risk of projects landing in root/AdministratorNamespace‑safe import with post‑audit cleanup
Branch sprawl post‑importAutomated branch hygiene—keeps demo → spawns demo2

A 50,000-ft View

flowchart TD
    subgraph Source GitLab
        A>All projects<br/>in `group-1`] --> B[export.py]
    end
    B -->|*.tar.gz| C((Secure<br/>storage))
    subgraph Destination GitLab
        D[import.py / selected...py] --> E((Group:<br/>group-1))
        E -->|Fix stray imports| F[cleanup.py]
        E -->|Delete stale branches| G[remove_obsolete_branches.py]
    end
  • export.py triggers asynchronous exports and downloads resulting archives

  • import.py / selected_import.py stream archives into the correct destination group

  • cleanup.py deletes projects accidentally imported under default user namespaces

  • Branch scripts enforce a single source of truth demo branch and spawn demo2


Under the Hood: Key Design Decisions

  1. Official python‑gitlab SDK: No scraping, no unofficial endpoints.

  2. Idempotency First: Every script can be re-run safely, as existing projects or branches are detected and skipped unless you opt in to overwrite.

  3. Streaming Downloads & Uploads: Exports are streamed in 1 MiB chunks; imports upload file-like objects, keeping memory steady.

  4. Namespace Double‑Lock: Imports supply both namespace (string) and namespace_id (int) to the API, all but eliminating the dreaded “imported to root” scenario.

  5. Branch Hygiene as First-Class Citizen: Post-import, the toolkit unprotects stale branches, deletes them, re-protects the guardians, and sets defaults because CI breaks are not an option.


Running the Toolkit in 6 Commands

git clone https://github.com/SubhanshuMG/gitlab-import-export.git
cd gitlab-import-export && python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

# 1 · bulk export
SRC_GITLAB=... SRC_TOKEN=... python scripts/export.py

# 2 · bulk import
DST_GITLAB=... DST_TOKEN=... python scripts/import.py  # or selected_import.py

# 3 · fix stray namespaces & branches
python scripts/cleanup.py
python scripts/remove_obsolete_branches.py
python scripts/selected_import.py  # edit SELECTED_PROJECTS = ["payment-service"]
python scripts/specific_project_remove_branches.py  # TARGET_PROJECT_NAME="payment-service"

Real‑World Payoff

  • 4× Speed‑up vs manual UI export/import (12 repos → 5 min)

  • Zero branch‑related pipeline failures post‑migration

  • Repeatable same scripts now run nightly to refresh our playground


Troubleshooting Cheat‑Sheet

SymptomDiagnosis & Fix
GitlabGetError: 404 on groupCheck PAT scope (read_api) and group path correctness
Import stuck at scheduledSidekiq busy; verify destination runners aren’t paused
Popen tmp/no space leftExports bigger than /tmp; set TMPDIR to a larger mount
Protected branch won’t deleteYou’re not a Maintainer; branch script auto‑unprotects but needs rights

Extending the Toolkit

  • Parallelize exports → Wrap the call loop in concurrent.futures.ThreadPoolExecutor

  • CI Variables & Releases → After import, iterate /projects/:id/variables & /releases

  • SaaS → Self‑Managed → Add mapping for group paths that changed between instances

  • GitLab => GitHub? → Swap endpoints; the logic stays 90 % identical

PRs are welcome, just fork and raise an issue!


The Repository

☑️ MIT‑licensed ☑️ Zero external dependencies (beyond python‑gitlab)
Check out the full source code and Python scripts w/ instructions:

GitHub → https://github.com/SubhanshuMG/gitlab-import-export.git


Final Words

If you’ve ever copy‑pasted repos one by one or worse, lost commit history during a migration; this toolkit is for you.
Fork it, bend it to your needs, and ship playground environments (or entire prod clones) with a single command.

Happy automating,

Signing off
Subhanshu Mohan Gupta (DevOps & Automation addict)

1
Subscribe to my newsletter

Read articles from Subhanshu Mohan Gupta directly inside your inbox. Subscribe to the newsletter, and don't miss out.

Written by

Subhanshu Mohan Gupta
Subhanshu Mohan Gupta

A passionate AI DevOps Engineer specialized in creating secure, scalable, and efficient systems that bridge development and operations. My expertise lies in automating complex processes, integrating AI-driven solutions, and ensuring seamless, secure delivery pipelines. With a deep understanding of cloud infrastructure, CI/CD, and cybersecurity, I thrive on solving challenges at the intersection of innovation and security, driving continuous improvement in both technology and team dynamics.