Key Advantages of Harbor

Security: Strong authentication, RBAC, and security scanning features
Integration: Easy integration with Kubernetes and Docker
Performance: Fast image distribution and management
Scalability: High availability and replication support
Management: User-friendly web interface and project-based organization

By setting up our system on virtual machines instead of Kubernetes, we minimized the risk of downtime and simplified management.

Why Do We Use Separate Servers?

Uninterrupted Service: Operates independently from Kubernetes cluster maintenance
Resource Management: Dedicated resource allocation and optimization
Simple Management: Less complex infrastructure
Security: Isolated security layers

This setup ensures Harbor operates more stably and reliably while also making management easier.

Setup

Requirements:

Load Balancer - 2 Servers (HAProxy and Keepalived)
- 192.168.10.101 LoadBalancer-1
- 192.168.10.102 LoadBalancer-2
- 192.168.10.100 (Keepalived IP)
Main - 2 Harbor installations (same configurations)
- 192.168.10.111 Main-1
- 192.168.10.112 Main-2
NFS Cluster (Existing or new NFS, GlusterFS, CephFS Cluster)
- 192.168.10.120
PostgreSQL Cluster (An existing setup can be used)
- 192.168.10.121
Redis Cluster (An existing setup can be used)
- 192.168.10.122

An example server architecture configuration is shown above.

Load Balancer

As seen in the image, there are 2 Load Balancers in the system, and they operate in an active-passive structure.

Load Balancer-1

The HAProxy configuration is as follows. The only difference between Load Balancer-1 and Load Balancer-2 is the load balancer IP in the monitor-stats section.

Directs the Harbor main servers in an active-passive manner.
SSL control is done through /etc/haproxy/cert.pem. You can use it by assigning a domain to the IP 192.168.10.100.

cat /etc/haproxy/haproxy.cfg

global
    log /dev/log local0
    log /dev/log local1 notice
    chroot /var/lib/haproxy
    stats socket /run/haproxy/admin.sock mode 660 level admin
    stats timeout 30s
    user haproxy
    group haproxy
    daemon
    maxconn 10000
    tune.ssl.default-dh-param 2048

listen monitor-stats
    mode http
    bind 192.168.10.101:7000
    stats enable
    stats uri /

defaults
    log global
    option httplog
    option dontlognull
    timeout connect 5000ms
    timeout client  50000ms
    timeout server  50000ms
    timeout http-request 10s
    timeout http-keep-alive 10s
    errorfile 400 /etc/haproxy/errors/400.http
    errorfile 403 /etc/haproxy/errors/403.http
    errorfile 408 /etc/haproxy/errors/408.http
    errorfile 500 /etc/haproxy/errors/500.http
    errorfile 502 /etc/haproxy/errors/502.http
    errorfile 503 /etc/haproxy/errors/503.http
    errorfile 504 /etc/haproxy/errors/504.http

frontend harbor_frontend
    bind *:443 ssl crt /etc/haproxy/cert.pem
    mode http
    default_backend harbor_backend

backend harbor_backend
    mode http
    #option httpchk GET /
    server harbor1 192.168.10.111:443 ssl verify none check
    server harbor2 192.168.10.112:443 ssl verify none check backup

Keepalived

High availability between two load balancers:
- Primary server: 192.168.10.101
- Backup server: 192.168.10.102
The HAProxy service is checked every 2 seconds and automatically switches over in case of an issue
Provides uninterrupted access to the Harbor registry with virtual IP 192.168.10.100

cat /etc/keepalived/keepalived.conf

# Global Settings for notifications
global_defs {

}

# Define the script used to check if haproxy is still working
vrrp_script chk_haproxy {
    script "/usr/bin/killall -0 haproxy"
    interval 2
    weight 2
}

# Configuration for Virtual Interface
vrrp_instance LB_VIP {
    interface ens192
    state MASTER        # set to BACKUP on the peer machine
    priority 101        # set to  99 on the peer machine
    virtual_router_id 20

    smtp_alert          # Enable Notifications Via Email

    authentication {
        auth_type PASS
        auth_pass MYP@ssword    # Password for accessing vrrpd. Same on all devices
    }
    unicast_src_ip 192.168.10.101 # Private IP address of master
    unicast_peer {
        192.168.10.102
   }

    # The virtual ip address shared between the two loadbalancers
    virtual_ipaddress {
        192.168.10.100
    }

    # Use the Defined Script to Check whether to initiate a fail over
    track_script {
        chk_haproxy
    }
}

Loadbalancer-2

Loadbalancer-2 operates as a backup server. The HAProxy configuration is the same as Loadbalancer-1, with the only difference being the monitor-stats IP is 192.168.10.102.

In the Keepalived configuration, there are these important differences:

The state is set to BACKUP
The priority is lowered to 100 (lower than the main server)
The unicast_src_ip is its own IP, 192.168.10.102

global
    log /dev/log local0
    log /dev/log local1 notice
    chroot /var/lib/haproxy
    stats socket /run/haproxy/admin.sock mode 660 level admin
    stats timeout 30s
    user haproxy
    group haproxy
    daemon
    maxconn 10000
    tune.ssl.default-dh-param 2048

listen monitor-stats
    mode http
    bind 192.168.10.102:7000
    stats enable
    stats uri /

defaults
    log global
    option httplog
    option dontlognull
    timeout connect 5000ms
    timeout client  50000ms
    timeout server  50000ms
    timeout http-request 10s
    timeout http-keep-alive 10s
    errorfile 400 /etc/haproxy/errors/400.http
    errorfile 403 /etc/haproxy/errors/403.http
    errorfile 408 /etc/haproxy/errors/408.http
    errorfile 500 /etc/haproxy/errors/500.http
    errorfile 502 /etc/haproxy/errors/502.http
    errorfile 503 /etc/haproxy/errors/503.http
    errorfile 504 /etc/haproxy/errors/504.http

frontend harbor_frontend
    bind *:443 ssl crt /etc/haproxy/cert.pem
    mode http
    default_backend harbor_backend

backend harbor_backend
    mode http
    #option httpchk GET /
    server harbor1 192.168.10.111:443 ssl verify none check
    server harbor2 192.168.10.112:443 ssl verify none check backup

Keepalived configuration:

# Global Settings for notifications
global_defs {

}

# Define the script used to check if haproxy is still working
vrrp_script chk_haproxy {
    script "/usr/bin/killall -0 haproxy"
    interval 2
    weight 2
}

# Configuration for Virtual Interface
vrrp_instance LB_VIP {
    interface ens192
    state BACKUP        # set to BACKUP on the peer machine
    priority 100        # set to  99 on the peer machine
    virtual_router_id 20

    smtp_alert          # Enable Notifications Via Email

    authentication {
        auth_type PASS
        auth_pass MYP@ssword    # Password for accessing vrrpd. Same on all devices
    }
    unicast_src_ip 192.168.10.102 # Private IP address of master
    unicast_peer {
        192.168.10.101
   }

    # The virtual ip address shared between the two loadbalancers
    virtual_ipaddress {
        192.168.10.100
    }

    # Use the Defined Script to Check whether to initiate a fail over
    track_script {
        chk_haproxy
    }
}

Harbor Main Servers

For Harbor installation, you first need to install the requirements specified in the https://goharbor.io/docs/1.10/install-config/installation-prereqs/ document and perform system updates.

NFS Connection

Let's mount the /data directory to the NFS cluster on the Main-1 and Main-2 servers:

vi /etc/fstab
192.168.10.120:/mnt/harbor-data /data nfs4 rsize=1048576,wsize=1048576,noatime,nodiratime 0 0
mount -a

With this configuration, data will be stored on the NFS server. If there is an issue with the Main-1 server, the Main-2 (backup) server will automatically take over. Since the data is kept on the NFS cluster, it will be stored redundantly.

Harbor Installation (Same for Main-1 and Main-2)

wget <https://github.com/goharbor/harbor/releases/download/v2.11.1/harbor-online-installer-v2.11.1.tgz>
tar xfv harbor-online-installer-v2.11.1.tgz
cd harbor
mv harbor.yml.tmpl harbor.yml
./prepare
mv /serdarcanb.cert /data/cert/registry.serdarcanb.com.cert
mv /serdarcanb.key /data/cert/registry.serdarcanb.com.key
vi harbor.yml

./install.sh --with-trivy

Harbor Configuration

The content of the harbor.yml file for both servers:

http: 
  port: 80

hostname: registry.serdarcanb.com
https:
  port: 443
  certificate: /data/cert/registry.serdarcanb.com.cert
  private_key: /data/cert/registry.serdarcanb.com.key

harbor_admin_password: Serdarcanb!23
data_volume: /data

trivy:
  ignore_unfixed: true
  skip_update: false
  skip_java_db_update: false
  offline_scan: false
  security_check: vuln
  insecure: false
  timeout: 5m0s

jobservice:  
  max_job_workers: 10
  job_loggers:
    - STD_OUTPUT
    - FILE
  logger_sweeper_duration: 1 

notification:
  webhook_job_max_retry: 3
  webhook_job_http_client_timeout: 3 

log:
  level: info
  local:
    rotate_count: 50
    rotate_size: 200M
    location: /var/log/harbor
_version: 2.11.0

proxy:
  http_proxy:
  https_proxy:
  no_proxy:
  components:
    - core
    - jobservice
    - trivy

metric:
  enabled: true
  port: 9090
  path: /metrics

upload_purging:
  enabled: true
  age: 168h
  interval: 24h
  dryrun: false

cache:
  enabled: true
  expire_hours: 2

external_database:
  harbor:
    host: 192.168.10.121
    port: 5432
    db_name: harbor
    username: devops
    password: 1Serdarcan123
    ssl_mode: disable
    max_idle_conns: 10
    max_open_conns: 100
external_redis:
  host: 192.168.10.122
  port: 6379
  password: 1Serdarcan123
  registry_db_index: 1
  jobservice_db_index: 2
  chartmuseum_db_index: 3

This Harbor configuration file (harbor.yml) includes the following basic settings:

HTTP and HTTPS port settings (80 and 443)
SSL certificate configuration
Trivy security scanner settings
Job service configuration
Notification and logging settings
Metrics and cache configuration
External database connection (PostgreSQL - 192.168.10.121)
External Redis connection (192.168.10.122)

With this configuration, the Harbor container registry is set to use external database and Redis services for high availability.

For PostgreSQL and Redis redundancy, it can be set up in a cluster structure. For more detailed configuration options, you can check the https://goharbor.io/docs/2.12.0/install-config/configure-yml-file/ page.

Harbor Container Registry - HA Architecture Setup