The Ultimate GitLab Import/Export Toolkit for Engineers


“I’m a DevOps engineer - if I’m doing it manually twice, I’m writing a script.”


A few weeks ago, I was handed a deceptively simple‑sounding task: spin up an entire playground environment, pipelines, history, everything for a complex micro‑services platform, from scratch.
The catch? All source code lived in GitLab, and there were dozens of projects spread across nested groups.
I scoured the web for a clean “export → import” recipe that would:

  • move commits, MRs, releases, tags, wikis - the whole lineage

  • preserve branch protections & default branches

  • run reliably for tens of repos without babysitting

Even GitLab’s docs stopped short of a full answer. Most blog posts boiled down to “click this button N times” or “write a bash loop and pray.”
Manual migration? Over my dead keyboard.
So I cracked open the GitLab API, dusted off my Python muscle memory, and the GitLab Migration & Branch‑Housekeeping Toolkit was born.


Why Another Solution?

Existing OptionsWhat This Toolkit Delivers
UI‑only exports (one project at a time)Bulk, unattended export/import of entire groups
3rd‑party “all‑in‑one” SaaS (pricey, opaque)100 % open‑source Python - reads every line
Incomplete migrations (no MRs, issues)Captures full project state, Inc. metadata
Risk of projects landing in root/AdministratorNamespace‑safe import with post‑audit cleanup
Branch sprawl post‑importAutomated branch hygiene—keeps demo → spawns demo2

A 50,000-ft View

flowchart TD
    subgraph Source GitLab
        A>All projects<br/>in `group-1`] --> B[export.py]
    end
    B -->|*.tar.gz| C((Secure<br/>storage))
    subgraph Destination GitLab
        D[import.py / selected...py] --> E((Group:<br/>group-1))
        E -->|Fix stray imports| F[cleanup.py]
        E -->|Delete stale branches| G[remove_obsolete_branches.py]
    end
  • export.py triggers asynchronous exports and downloads resulting archives

  • import.py / selected_import.py stream archives into the correct destination group

  • cleanup.py deletes projects accidentally imported under default user namespaces

  • Branch scripts enforce a single source of truth demo branch and spawn demo2


Under the Hood: Key Design Decisions

  1. Official python‑gitlab SDK: No scraping, no unofficial endpoints.

  2. Idempotency First: Every script can be re-run safely, as existing projects or branches are detected and skipped unless you opt in to overwrite.

  3. Streaming Downloads & Uploads: Exports are streamed in 1 MiB chunks; imports upload file-like objects, keeping memory steady.

  4. Namespace Double‑Lock: Imports supply both namespace (string) and namespace_id (int) to the API, all but eliminating the dreaded “imported to root” scenario.

  5. Branch Hygiene as First-Class Citizen: Post-import, the toolkit unprotects stale branches, deletes them, re-protects the guardians, and sets defaults because CI breaks are not an option.


Running the Toolkit in 6 Commands

git clone https://github.com/SubhanshuMG/gitlab-import-export.git
cd gitlab-import-export && python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

# 1 · bulk export
SRC_GITLAB=... SRC_TOKEN=... python scripts/export.py

# 2 · bulk import
DST_GITLAB=... DST_TOKEN=... python scripts/import.py  # or selected_import.py

# 3 · fix stray namespaces & branches
python scripts/cleanup.py
python scripts/remove_obsolete_branches.py
python scripts/selected_import.py  # edit SELECTED_PROJECTS = ["payment-service"]
python scripts/specific_project_remove_branches.py  # TARGET_PROJECT_NAME="payment-service"

Real‑World Payoff

  • 4× Speed‑up vs manual UI export/import (12 repos → 5 min)

  • Zero branch‑related pipeline failures post‑migration

  • Repeatable same scripts now run nightly to refresh our playground


Troubleshooting Cheat‑Sheet

SymptomDiagnosis & Fix
GitlabGetError: 404 on groupCheck PAT scope (read_api) and group path correctness
Import stuck at scheduledSidekiq busy; verify destination runners aren’t paused
Popen tmp/no space leftExports bigger than /tmp; set TMPDIR to a larger mount
Protected branch won’t deleteYou’re not a Maintainer; branch script auto‑unprotects but needs rights

Extending the Toolkit

  • Parallelize exports → Wrap the call loop in concurrent.futures.ThreadPoolExecutor

  • CI Variables & Releases → After import, iterate /projects/:id/variables & /releases

  • SaaS → Self‑Managed → Add mapping for group paths that changed between instances

  • GitLab => GitHub? → Swap endpoints; the logic stays 90 % identical

PRs are welcome, just fork and raise an issue!


The Repository

☑️ MIT‑licensed ☑️ Zero external dependencies (beyond python‑gitlab)
Check out the full source code and Python scripts w/ instructions:

GitHub → https://github.com/SubhanshuMG/gitlab-import-export.git


Final Words

If you’ve ever copy‑pasted repos one by one or worse, lost commit history during a migration; this toolkit is for you.
Fork it, bend it to your needs, and ship playground environments (or entire prod clones) with a single command.

Happy automating,

Signing off
Subhanshu Mohan Gupta (DevOps & Automation addict)

11
Subscribe to my newsletter

Read articles from Subhanshu Mohan Gupta directly inside your inbox. Subscribe to the newsletter, and don't miss out.

Written by

Subhanshu Mohan Gupta
Subhanshu Mohan Gupta

A passionate AI DevOps Engineer specialized in creating secure, scalable, and efficient systems that bridge development and operations. My expertise lies in automating complex processes, integrating AI-driven solutions, and ensuring seamless, secure delivery pipelines. With a deep understanding of cloud infrastructure, CI/CD, and cybersecurity, I thrive on solving challenges at the intersection of innovation and security, driving continuous improvement in both technology and team dynamics.