Laravel 4 Down Safe

Laravel is a fantastic PHP Framework that does most of the complex, and boring, application framework tasks for you, so you can focus on the application itself. One of the components it provides is a full Queue Worker system to make it very easy to add jobs into a queue and process them in the background.

Laravel has made the queuing system really simplistic, in order to make it easy to use, however it is here where it suffers from a big flaw: you can’t safely shut down a queue worker.

Let’s look at this command:

php artisan queue:listen

It will happily keep sitting there, processing jobs when there are any available. However, we have absolutely no idea if it’s actually processing a job or not – and therefore we don’t know when it’s a good time to kill the worker.

When I asked in the Laravel forums how to safely kill a worker, someone responded with this: “It should always be safe to kill a job, jobs should be coded like that, always assume it will happen.”. While this is a nice idea in theory, if you’re working with a really complex queue job (which queue jobs are designed for!), the effort involved in making sure your job can auto-recover from a crash may be immense. So I reject the notion that you just need to accept that jobs can be killed randomly.

Why do we need to kill a worker?

We need to kill a worker when we want to update the application code and/or database. By leaving the workers processing while we apply and update, we’ve opened the door for issues when a database field, or function call, no longer exist, or their behavior changes in an incompatible way.

Laravel provides a nice command to turn the application into maintenance mode: php artisan down. The queue:listen command will honor this after it finishes processing the current job – but you don’t know when that job is finished. Also, Laravel v4.2 recently added a new command, php artisan queue:work --daemon, which removes a lot of the overheads of the listen command – and completely ignores the maintenance mode setting.

So we either have to wait 30-60 seconds and hope the job is finished (for queue:listen), or we have to wait even longer and hope the queue is empty (for --daemon)… and we have no way of knowing either.

So, how can we deal with this?

I decided to find a simple solution to the problem. Our aim is to enable Maintenance Mode and be confident that the queue worker is not processing anything. How about we have the queue worker set Maintenance Mode, and keep the queue busy until the application is back online. This can be implemented as an Artisan command:

php artisan down:safe

With (summarised) code:

<?php
class DownSafe extends Command {

    protected $name = 'down:safe';

    public function fire()
    {
        // Push job onto queue
        Queue::push(function($job) {

            // Take Application down.
            touch(Config::get('app.manifest').'/down');

            // Add Log message
            Log::info("Application is down, pausing queue while maintenance happens.");

            // Loop, waiting for app to come back up
            while (App::isDownForMaintenance()) {
                echo ".";
                sleep(5);
            }

            // App is back online, kill worker to restart daemon.
            Log::info("Application is up, rebooting queue.");
            $job->delete();
            die();
        });

        // Wait until we're down for Maintenance
        while (!App::isDownForMaintenance()) {
            sleep(1);
        }

        $this->info("Application maintenance mode enabled.");
    }
}

When running php artisan down:safe, it will add a job into the queue and wait. When this job is processed, the application is taken down into maintenance mode and the queue worker remains looping that job – however command will finish so you can proceed with maintenance. Finally, when the application is brought back up with php artisan up, the queue worker is killed so it can reload the application environment with fresh code after the maintenance.

If running php artisan queue:listen, the listener will continue to operate throughout the process seamlessly. However, php artisan queue:work --daemon will die and need to be restarted.

This method only supports a single queue worker, however you could probably modify the idea with some flag files, add a job for each worker, and wait until they are all loaded.

Since I can see myself wanting to use this command in most of my projects, I’ve added it to GitHub and Packagist for anyone to use: valorin/l4-down-safe.

I hope my simple solution can be of use to you. I’d be very interested to hear from you if you do use it, and if you expand on the concept in any way.