Using Sidekiq with Rails to download CSV files in the background

Many-a-times, we come across processes that do not need user intervention, yet take up a lot of server time. This time could have been used, otherwise, to do multiple other tasks. Let’s take the example of downloading a large file as a csv. In such a case, when the data to be processed gets large enough, it’s always a good idea to run this process in the background and let the user continue using the application to do other tasks. Processes can be run in the background as background jobs.

Background jobs are jobs that can be run asynchronously in the background without any user intervention. Rails has Active Job that can be used to process background jobs and make them run on a variety of background adapters such as Sidekiq or Resque. To explain the process better, let’s take the example of a Bookstore and use Sidekiq as the background adapter.

Installing Sidekiq in Ubuntu 20.04

To use Sidekiq in Ubuntu, we will need to install redis as a prerequisite. Redis is used by Sidekiq as an in-memory data structure store.

After installing redis, We can go ahead and install Sidekiq. To do so, first open the Gemfile, add gem “sidekiq” and run bundle install in the terminal.

gem “sidekiq”bundle install

Now, open config/application.rb and inside class Application, enter the lines as shown:

class Application < Rails::Application
…
  config.active_job.queue_adapter = :sidekiq
…
end

Now, to get started with sidekiq, simply run

bundle exec sidekiq

You must be able to see some output on the terminal along with the sidekiq logo towards the end, to indicate a running sidekiq server.

Writing the background jobs

Installing Sidekiq in your project automatically creates app/jobs folder. All the background jobs concerning to this particular project reside in this folder. Moving back to our example, let’s say, we want to download details of all the books available in the bookshop.

Let’s create a job file_downloader_job.rb. This job will prepare the csv file with id, name and price of each book as the columns of the csv. Enter the following code in it:

class FileDownloaderJob < ApplicationJob
  queue_as :default  def perform(all)
    attributes = %w{book_id book_name price}
    CSV.generate(headers: true) do |csv|
      csv << attributes
      all.each do |book|
        csv << book.attributes.values_at(*attributes)
      end
    end
  end
end

Now, let’s call this job in the book.rb model. Here, you can generalize this process by writing the .to_csv function in application_record.rb model and writing variable attributes in the filedownloader job.

class Book < ActiveRecord::Base
  self.abstract_class = true  def self.to_csv
    FileDownloaderJob.new.perform(all)
  end
end

This job is called in the to_csv function to keep the controller clean. Now, write the following code in books_controller.rb.

class BooksController < ApplicationController
def index 
  books = Book.all respond_to do |format| 
    format.html {render status: :ok, json: books.organize()}
    format.csv { send_data books.to_csv.perform_async(*args), 
      filename: “books-#{Date.today}.csv” } 
  end 
end

And done! Your csv file that shows the contents of the Books table starts download in the background as soon as the index action of the books controller is called with .csv extension.

0
Subscribe to my newsletter

Read articles from NonStop io Technologies directly inside your inbox. Subscribe to the newsletter, and don't miss out.

Written by

NonStop io Technologies
NonStop io Technologies

Product Development as an Expertise Since 2015 Founded in August 2015, we are a USA-based Bespoke Engineering Studio providing Product Development as an Expertise. With 80+ satisfied clients worldwide, we serve startups and enterprises across San Francisco, Seattle, New York, London, Pune, Bangalore, Tokyo and other prominent technology hubs.