We are using redis(Elasticache) as our cache store.We are experiencing crashes in our app due to redis connection timeout.We have three app servers and use phusion passenger.At the time of crashes there are about 200 active connections.
/home/chillr/deploy/chillr-api/shared/vendor/bundle/ruby/2.2.0/gems/redis-3.2.1/lib/redis/connection/ruby.rb:55:in `rescue in _read_from_socket'/home/chillr/deploy/chillr-api/shared/vendor/bundle/ruby/2.2.0/gems/redis-3.2.1/lib/redis/connection/ruby.rb:48:in `_read_from_socket'/home/chillr/deploy/chillr-api/shared/vendor/bundle/ruby/2.2.0/gems/redis-3.2.1/lib/redis/connection/ruby.rb:41:in `gets'/home/chillr/deploy/chillr-api/shared/vendor/bundle/ruby/2.2.0/gems/redis-3.2.1/lib/redis/connection/ruby.rb:273:in `read'/home/chillr/deploy/chillr-api/shared/vendor/bundle/ruby/2.2.0/gems/redis-3.2.1/lib/redis/client.rb:248:in `block in read'/home/chillr/deploy/chillr-api/shared/vendor/bundle/ruby/2.2.0/gems/redis-3.2.1/lib/redis/client.rb:236:in `io'/home/chillr/deploy/chillr-api/shared/vendor/bundle/ruby/2.2.0/gems/redis-3.2.1/lib/redis/client.rb:247:in `read'/home/chillr/deploy/chillr-api/shared/vendor/bundle/ruby/2.2.0/gems/redis-3.2.1/lib/redis/client.rb:112:in `block in call'/home/chillr/deploy/chillr-api/shared/vendor/bundle/ruby/2.2.0/gems/redis-3.2.1/lib/redis/client.rb:217:in `block (2 levels) in process'/home/chillr/deploy/chillr-api/shared/vendor/bundle/ruby/2.2.0/gems/redis-3.2.1/lib/redis/client.rb:353:in `ensure_connected'/home/chillr/deploy/chillr-api/shared/vendor/bundle/ruby/2.2.0/gems/redis-3.2.1/lib/redis/client.rb:207:in `block in process'/home/chillr/deploy/chillr-api/shared/vendor/bundle/ruby/2.2.0/gems/redis-3.2.1/lib/redis/client.rb:292:in `logging'/home/chillr/deploy/chillr-api/shared/vendor/bundle/ruby/2.2.0/gems/redis-3.2.1/lib/redis/client.rb:206:in `process'/home/chillr/deploy/chillr-api/shared/vendor/bundle/ruby/2.2.0/gems/redis-3.2.1/lib/redis/client.rb:112:in `call'SlOWLOG commands
1) 1) (integer) 68673752) (integer) 14867015073) (integer) 620084) 1) "eval" 2) "for i, name in ipairs(redis.call('KEYS', 'cache:user_transaction_logs:*:9008245678')) do redis.call('DEL', name); end" 3) "0" 2) 1) (integer) 6867374 2) (integer) 1486701507 3) (integer) 61989 4) 1) "KEYS" 2) "cache:user_transaction_logs:*:9008245678" 3) 1) (integer) 6867373 2) (integer) 1486701507 3) (integer) 61026 4) 1) "eval" 2) "for i, name in ipairs(redis.call('KEYS', 'cache:user_transaction_logs:*:8888662136')) do redis.call('DEL', name); end" 3) "0" 4) 1) (integer) 6867372 2) (integer) 1486701507 3) (integer) 61006 4) 1) "KEYS" 2) "cache:user_transaction_logs:*:8888662136" 5) 1) (integer) 6867371 2) (integer) 1486701507 3) (integer) 63070 4) 1) "eval"We use
- Elasticache cache.
- t2.medium Phusion Passenger
No of connections at time of Crash 175..200
GEM redis (3.2.1) redis-rails (5.0.1)
What could be done