The Ultimate GitLab Import/Export Toolkit for Engineers


“I’m a DevOps engineer - if I’m doing it manually twice, I’m writing a script.”
A few weeks ago, I was handed a deceptively simple‑sounding task: spin up an entire playground environment, pipelines, history, everything for a complex micro‑services platform, from scratch.
The catch? All source code lived in GitLab, and there were dozens of projects spread across nested groups.
I scoured the web for a clean “export → import” recipe that would:
move commits, MRs, releases, tags, wikis - the whole lineage
preserve branch protections & default branches
run reliably for tens of repos without babysitting
Even GitLab’s docs stopped short of a full answer. Most blog posts boiled down to “click this button N times” or “write a bash loop and pray.”
Manual migration? Over my dead keyboard.
So I cracked open the GitLab API, dusted off my Python muscle memory, and the GitLab Migration & Branch‑Housekeeping Toolkit was born.
Why Another Solution?
Existing Options | What This Toolkit Delivers |
UI‑only exports (one project at a time) | Bulk, unattended export/import of entire groups |
3rd‑party “all‑in‑one” SaaS (pricey, opaque) | 100 % open‑source Python - reads every line |
Incomplete migrations (no MRs, issues) | Captures full project state, Inc. metadata |
Risk of projects landing in root/Administrator | Namespace‑safe import with post‑audit cleanup |
Branch sprawl post‑import | Automated branch hygiene—keeps demo → spawns demo2 |
A 50,000-ft View
flowchart TD
subgraph Source GitLab
A>All projects<br/>in `group-1`] --> B[export.py]
end
B -->|*.tar.gz| C((Secure<br/>storage))
subgraph Destination GitLab
D[import.py / selected...py] --> E((Group:<br/>group-1))
E -->|Fix stray imports| F[cleanup.py]
E -->|Delete stale branches| G[remove_obsolete_branches.py]
end
export.py triggers asynchronous exports and downloads resulting archives
import.py / selected_import.py stream archives into the correct destination group
cleanup.py deletes projects accidentally imported under default user namespaces
Branch scripts enforce a single source of truth
demo
branch and spawndemo2
Under the Hood: Key Design Decisions
Official
python‑gitlab
SDK: No scraping, no unofficial endpoints.Idempotency First: Every script can be re-run safely, as existing projects or branches are detected and skipped unless you opt in to overwrite.
Streaming Downloads & Uploads: Exports are streamed in 1 MiB chunks; imports upload file-like objects, keeping memory steady.
Namespace Double‑Lock: Imports supply both
namespace
(string) andnamespace_id
(int) to the API, all but eliminating the dreaded “imported to root” scenario.Branch Hygiene as First-Class Citizen: Post-import, the toolkit unprotects stale branches, deletes them, re-protects the guardians, and sets defaults because CI breaks are not an option.
Running the Toolkit in 6 Commands
git clone https://github.com/SubhanshuMG/gitlab-import-export.git
cd gitlab-import-export && python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
# 1 · bulk export
SRC_GITLAB=... SRC_TOKEN=... python scripts/export.py
# 2 · bulk import
DST_GITLAB=... DST_TOKEN=... python scripts/import.py # or selected_import.py
# 3 · fix stray namespaces & branches
python scripts/cleanup.py
python scripts/remove_obsolete_branches.py
python scripts/selected_import.py # edit SELECTED_PROJECTS = ["payment-service"]
python scripts/specific_project_remove_branches.py # TARGET_PROJECT_NAME="payment-service"
Real‑World Payoff
4× Speed‑up vs manual UI export/import (12 repos → 5 min)
Zero branch‑related pipeline failures post‑migration
Repeatable same scripts now run nightly to refresh our playground
Troubleshooting Cheat‑Sheet
Symptom | Diagnosis & Fix |
GitlabGetError: 404 on group | Check PAT scope (read_api ) and group path correctness |
Import stuck at scheduled | Sidekiq busy; verify destination runners aren’t paused |
Popen tmp/no space left | Exports bigger than /tmp ; set TMPDIR to a larger mount |
Protected branch won’t delete | You’re not a Maintainer; branch script auto‑unprotects but needs rights |
Extending the Toolkit
Parallelize exports → Wrap the call loop in
concurrent.futures.ThreadPoolExecutor
CI Variables & Releases → After import, iterate
/projects/:id/variables
&/releases
SaaS → Self‑Managed → Add mapping for group paths that changed between instances
GitLab => GitHub? → Swap endpoints; the logic stays 90 % identical
PRs are welcome, just fork and raise an issue!
The Repository
☑️ MIT‑licensed ☑️ Zero external dependencies (beyond python‑gitlab
)
Check out the full source code and Python scripts w/ instructions:
GitHub → https://github.com/SubhanshuMG/gitlab-import-export.git
Final Words
If you’ve ever copy‑pasted repos one by one or worse, lost commit history during a migration; this toolkit is for you.
Fork it, bend it to your needs, and ship playground environments (or entire prod clones) with a single command.
Happy automating,
Signing off
Subhanshu Mohan Gupta (DevOps & Automation addict)
Subscribe to my newsletter
Read articles from Subhanshu Mohan Gupta directly inside your inbox. Subscribe to the newsletter, and don't miss out.
Written by

Subhanshu Mohan Gupta
Subhanshu Mohan Gupta
A passionate AI DevOps Engineer specialized in creating secure, scalable, and efficient systems that bridge development and operations. My expertise lies in automating complex processes, integrating AI-driven solutions, and ensuring seamless, secure delivery pipelines. With a deep understanding of cloud infrastructure, CI/CD, and cybersecurity, I thrive on solving challenges at the intersection of innovation and security, driving continuous improvement in both technology and team dynamics.