You are probably already familiar with the work of Spatie and its contributors in the Laravel and Open Source ecosystem.

For many years, we have been using their Backup package (https://spatie.be/docs/laravel-backup) to back up our applications (databases and assets) on cloud provider's object storage like S3 or OpenStack Swift.

As our projects started utilizing more and more external disks to store assets and files generated by the application, such as PDF files, we encountered a limitation with the package — it only supports the backup of local files.

In this article, we will explore how to extend the package to back up external Laravel disks (S3 compatible, OpenStack Swift, ...).

Deep dive into the Package code

Let's take a closer look at the code of the package. Upon inspection, we can understand why it only handles local files:

• it uses the Symfony Finder to locate the files to back up

• it creates a temporary local folder to create a Zip file before sending it to the configured destination disks (after the backup process is complete, the temporary folder is deleted)

Making the Symfony Finder work with other filesystem types would require significant effort, and downloading the content of external disks to create a Zip file locally, and then sending it to other disks, is not an ideal solution...

However, there is good news. The package dispatches several events, such as when the Zip file is created, when the backup fails or succeeds, ...

The Approach

Based on these observations, we can devise a strategy to back up our external disks:

• Instead of creating a Zip file, we will perform a direct copy of the files to the destination disks

• We will listen to the following events to trigger our custom processes:

  • BackupWasSuccessful: to launch the copy operation

  • CleanupWasSuccessful: to clean up old backups

Implementation

Configuration

First, let's add a new entry in the package's configuration file to specify the disks we want to back up.

/config/backup.php

<?php

return [
    'backup' => [
        'source' => [
            // ...
            'disks' => [
                'assets',
                'customers',
            ],
            // ...
        ],
];

The logic

We will create a Subscriber to handle the events and implement our custom processes.

BackupWasSuccessful event

The BackupWasSuccessful event is the most suitable event to perform our custom copy operation. It is dispatched when the standard backup job has finished its work.

By accessing the backupDestination instance provided by the event, we can obtain the path of the Zip file on the destination disk.

Here's how we proceed:

• we compute the backup path. To simplify backup and cleanup,

  • the directory that will be created will have the same name as the Zip file

• the files / directories will be copied into a subdirectory that will have the same name as the external disk (here we back up two disks: assets and customers)

• we retrieve all the external disks we want to back up from the package's configuration file, and for each disk:

  • we retrieve all the files, including subdirectories.

  • we copy each file to the backup disk configured in the package.

CleanupWasSuccessful event

The CleanupWasSuccessful event is the most suitable event to perform our custom cleanup. It is dispatched when the standard cleanup job has finished its work.

Similar to the previous event, we can access the backupDestination instance provided by the event.

Here's how we proceed:

• we retrieve all the directories on the destination disk. These directories are expected to contain only the custom backups
  ❗All directories not related to backups in the backup directory will be deleted

• we retrieve all of the existing Zip backups from the backupDestination (we add fresh() method call because the Job removes the last backup from the collection, and as it's cached, we need to retrieve the updated list)

• we compute the directories to delete by identifying the directories that do not have a corresponding Zip file.

• we delete the directories that need to be cleaned up

Subscriber code

/app/Listeners/BackupEventSubscriber.php

<?php

namespace App\Listeners;

use Illuminate\Events\Dispatcher;
use Illuminate\Support\Facades\Storage;
use Illuminate\Support\Str;
use Spatie\Backup\Events\BackupWasSuccessful;
use Spatie\Backup\Events\CleanupWasSuccessful;
use Symfony\Component\Console\Output\ConsoleOutput;

class BackupEventSubscriber
{
    public function subscribe(Dispatcher $events): array
    {
        return [
            BackupWasSuccessful::class => 'handleBackup',
            CleanupWasSuccessful::class => 'handleCleanup',
        ];
    }

    public function handleBackup(BackupWasSuccessful $event): void
    {
        $backupDestination = $event->backupDestination;
        $backupPath = Str::replaceLast('.zip', '', $backupDestination->newestBackup()->path());

        $output = new ConsoleOutput();

        $sources = config('backup.backup.source.disks') ?? [];

        foreach ($sources as $disk) {
            $output->writeln('<info>Determining files to backup for disk ' . $disk . '...</info>');
            $destinationPath = $backupPath . '/' . $disk . '/';

            $files = Storage::disk($disk)->allFiles();
            $output->writeln('<info>Copying disk ' . $disk . ' to disk named backup: ' . count($files) . ' files</info>');

            foreach ($files as $file) {
                Storage::disk($backupDestination->diskName())->put(
                    $destinationPath . $file,
                    Storage::disk($disk)->get($file)
                );
            }
        }
    }

    public function handleCleanup(CleanupWasSuccessful $event): void
    {
        $backupDestination = $event->backupDestination;

        $output = new ConsoleOutput();
        $output->writeln('<info>Cleaning disks backups of ' . $backupDestination->backupName() . ' on disk ' . $backupDestination->diskName() . '...</info>');

        $directories = Storage::disk($backupDestination->diskName())->directories($backupDestination->backupName());

        if (count($directories) > 0) {
            $backups = $backupDestination
                ->fresh()
                ->backups()
                ->map(function ($backup) {
                    return Str::replaceLast('.zip', '', $backup->path());
                })
                ->toArray();

            $directoriesToDelete = array_filter($directories, function ($directory) use ($backups) {
                return !in_array($directory, $backups);
            });

            if (count($directoriesToDelete) > 0) {
                foreach ($directoriesToDelete as $directory) {
                    Storage::disk($backupDestination->diskName())->deleteDirectory($directory);
                }
            }
        }
    }
}

Register the Subscriber

The final step is to register the Subscriber in the Laravel EventServiceProvider

/app/Providers/EventServiceProvider.php

<?php

namespace App\Providers;

use App\Listeners\BackupEventSubscriber;
use Illuminate\Foundation\Support\Providers\EventServiceProvider as ServiceProvider;

class EventServiceProvider extends ServiceProvider
{
    /**
     * The subscriber classes to register.
     *
     * @var array
     */
    protected $subscribe = [
        BackupEventSubscriber::class,
    ];
}

---