Quantcast
Channel: Active questions tagged redis+ruby-on-rails - Stack Overflow
Viewing all articles
Browse latest Browse all 873

Sidekiq, Puma and Redis config

$
0
0

I'm running a Rails 5.1 application on google AppEngine.

Below is my config

Procfile

worker: bundle exec sidekiq -C config/sidekiq.yaml
pubsub: bundle exec rake run_analytics_queue_processor

sidekiq.yaml

:concurrency: 2
:timeout: 30
:queues:
  - default

config/initializers/sidekiq.rb

# frozen_string_literal: true

require 'sidekiq/web'

url = CREDENTIALS[:redis_url]

Sidekiq::Web.use(Rack::Auth::Basic) do |user, password|
  [user, password] == [CREDENTIALS[:sidekiq_username], CREDENTIALS[:sidekiq_password]]
end

Sidekiq.configure_server do |config|
  config.redis = { url: url, id: nil }
end

Sidekiq.configure_client do |config|
  config.redis = { url: url, id: nil, size: 12 }
end

puma.rb

# frozen_string_literal: true

# Puma can serve each request in a thread from an internal thread pool.
# The `threads` method setting takes two numbers: a minimum and maximum.
# Any libraries that use thread pools should be configured to match
# the maximum value specified for Puma. Default is set to 5 threads for minimum
# and maximum; this matches the default thread size of Active Record.
#
workers 2
threads 6, 6

preload_app!

rackup      DefaultRackup

# Specifies the `port` that Puma will listen on to receive requests; default is 3000.
#
port        ENV.fetch('PORT') { 3000 }

# Specifies the `environment` that Puma will run in.
#
environment ENV.fetch('RAILS_ENV') { 'local' }

on_worker_boot do
  # Worker specific setup for Rails 4.1+
  # See: https://devcenter.heroku.com/articles/deploying-rails-applications-with-the-puma-web-server#on-worker-boot
  begin
    ActiveRecord::Base.connection.disconnect!
  rescue StandardError
    ActiveRecord::ConnectionNotEstablished
  end
  ActiveRecord::Base.establish_connection
end

# Specifies the number of `workers` to boot in clustered mode.
# Workers are forked webserver processes. If using threads and workers together
# the concurrency of the application would be max `threads` * `workers`.
# Workers do not work on JRuby or Windows (both of which do not support
# processes).
#
# workers ENV.fetch("WEB_CONCURRENCY") { 2 }

# Use the `preload_app!` method when specifying a `workers` number.
# This directive tells Puma to first boot the application and load code
# before forking the application. This takes advantage of Copy On Write
# process behavior so workers use less memory.
#
# preload_app!

# Allow puma to be restarted by `rails restart` command.
plugin :tmp_restart

In my procfile the bundle exec rake run_analytics_queue_processor is calling a rake job that endlessly runs and takes in messages from our PubSub service and queues them to a sidekiq/ActiveJob background job

I have been getting multiple Timeout Timeout::Error: Waited 1 sec issues when calling perform_later with sidekiq jobs. Looking up the issue it seems like the connection pooling for redis was incorrect. I believe it is threads * concurrency which should be 12 in my case, which is what I am setting the redis pool size to in my sidekiq.yaml

Does anything in my config look glaringly wrong?


Viewing all articles
Browse latest Browse all 873

Trending Articles