Laravel Priority Queue Driver

Hi all,

One of my previous works for my OSS Organization - ShipSaaS - The Laravel Priority Queue Driver.

So I'd like to share it with you guys today 😉

Normal Prioritization in Laravel Queue

By default, Laravel Queue only has an alternative approach for Queue Prioritization, which create several dedicated queues and push your job there.

For example, I'm using DB driver for a quick example:

• default

• high

• low

Then I'll push the jobs like this:

TransferJob::dispatch($data)->onQueue('high');
UpdateDataJob::dispatch($data)->onQueue('low');
KybJob::dispatch($data); // default

Problems

Imagine you have 1000 TransferJob and 10 worker processes, a single job takes 30s

So Overall, it costs 5 mins to solve all Queue Msgs without any prioritization.

In reality, the number of jobs will continually increase, and certain constraints will arise, such as prioritizing transactions for VIP customers.

And create more dedicated queues will increase the complexity of the codebase. You have to manage them all in both code & infra.

Not so fun and scalable, right?

Meets Priority Queue Driver

Repository: https://github.com/shipsaas/laravel-priority-queue

PROS:

• You can compute the "weight" for each job, based on your constraints 🔥.

• Using the "DB" driver, with indexes, the job will be popped super fast 🏎️.

• High visibility, under DB, your DB ❇️.

• Needs to solve msgs faster? Increase the worker processes 🏃‍♂️.

• (Optional) You can create dedicated queues if you wanted to.

Installation

Check this out: https://github.com/shipsaas/laravel-priority-queue#installation

Usage

Pretty straightforward and the same as Laravel - Push Job:

MyJob::dispatch($data)->onConnection('database-priority');

// QueueFacade
Queue::connection('database-priority')
    ->push($myJob);

Define the $weight - build time:

class MyJob
{
    public int $weight = 950;
}

On runtime:

use ShipSaasPriorityQueue\Traits\UseJobPrioritization;

class MyJob implements ShouldQueue
{
    use UseJobPrioritization;

    public function __construct(public User $user)
    {
        // you can do this too
        $this->onConnection('database-priority');
    }

    public function getJobWeight() : int
    {
        return $this->user->isPro()
            ? 1000
            : 500;
    }
}

Testing

It is covered by:

• Unit Testing

• Integration Testing (queue:work)

Conclusions

With this driver, we hope to elevate your Laravel Queue Prioritization to a whole new level 🔥

If this package is helpful, please give it a ⭐️⭐️⭐️. Thank you!

Thanks for reading!