Scribd
Upload
7K views36 pages

Handling Long-Running Tasks in Rails

Uploaded by

Oleksiy Kovyrin
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
7K views36 pages

Handling Long-Running Tasks in Rails

Uploaded by

Oleksiy Kovyrin
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 36

Long-Running Tasks In Rails

Without Much Effort


Andy Stewart
April 2008
Not me
Still not me
Why?
User starts
something
lengthy
Delayed
task
Regular
task
Example
Synchronous link to Campaign Monitor
10
BackgroundJob (BJ) Spawn
Sparrow
Delayed Job (DJ)
AM4R
Conveyor
BackgrounDRb
Beanstalkd &
async-observer
Workling
BackgroundFu
Factors
Durability
Progress reporting
Scheduling
Serial vs. Parallel
Process management
Execution environment
Error handling
Learning curve
Installation burden
Constraints on task code
Durability
Durable Hopeful

BackgrounDRb
Background Job
Beanstalk
BackgroundFu
Spawn
Delayed Job
Sparrow
Workling
Workling
0% 50% 100%
Tool Method
Background Job Process’s stats
BackgroundFu Incremental
BackgrounDRb Ask workers
Workling DIY “return store”
Scheduling
BackgrounDRb

cron + {rake, script/runner,


whatever}
Delaying
(on purpose)

Delayed Job
Beanstalk
Process
Management
No Problem Hassle

Background Job
Everything else
Spawn
Examples
BackgrounDRb
# Lots of configuration
...

# Your worker code


class BillingWorker < BackgrounDRb::MetaWorker
set_worker_name :billing_worker

# Called when worker is loaded for the first time


def create(args = nil)
end

# The lengthy task.


def charge_customer(customer_id = nil)
# ... do stuff ...
end
end

# Your invocation code


class CustomersController < ApplicationController
def upgrade_account
# ...
MiddleMan.worker(:billing_worker).charge_customer(@customer.id)
end
end
Beanstalk
# Some configuration
...

# Comments controller
def create
@comment = Comment.new(params[:comment])
if @comment.save
BEANSTALK.yput({:type => "comment", :id => @comment.id}) rescue nil
# Then redirect and return
end
end

# Worker - Rake task


loop do
job = BEANSTALK.reserve
job_hash = job.ybody # ybody deserializes the job
case job_hash[:type]
when "comment"
if Comment.check_for_spam(job_hash[:id])
job.delete
else
job.bury
end
else
puts "Don't know what type of job this is: #{job_hash.inspect}"
end
end

Source: nubyonrails.com
Async Observer
# Some configuration
...

# Start some workers


$ ./vendor/plugins/async_observer/bin/worker

# Your code
class Person < ActiveRecord::Base
async_after_create do |person|
SiteStats.increment_members()
end

def befriend(other_person)
Friendship.async_send(:create, self, other_person)
end
end
Delayed Job
# Small config - db migration

# A job is a Ruby object with #perform


class NewsletterJob < Struct.new(:text, :emails)
def perform
emails.each { |e| NewsletterMailer.deliver_text_to_email(text, e) }
end
end

# Stick in queue
Delayed::Job.enqueue NewsletterJob.new('lorem ipsum...',
Customers.find(:all).collect(&:email))

# Delayed job
BatchImporter.new(Shop.find(1)).send_later(:import_massive_csv, massive_csv)

# Running tasks
$ rake jobs:work <CTRL-C to cancel>
Background Job
# Command line

$ Bj.submit "./jobs/background_job_to_run"

$ Bj.submit "./script/runner ./jobs/background_job_to_run"

# Within Rails

def upload_to_s3
Bj.submit "./script/runner ./jobs/s3_uploader.rb #{self.id}"
end

Source: slackworks.com
1
Background Job
Two-line installation
Zero configuration
No job-code constraints
Automatic* process management
Durable, prioritised, taggable jobs
Comprehensive job results
Clustering
* Manual if you want
Feed my spaniel
Buy my PeepCode PDF
Photo Credits:
1. BBC
2. Forever in Song (CD cover)
3. Me
4. Me

My Photos’ Licence:
Creative Commons Attribution-Noncommercial-No Derivative Works 2.0
UK: England & Wales Licence

Text’s Licence:
Creative Commons Attribution-Share Alike 2.0 UK: England & Wales Licence

You might also like