I have a rails app running which is using redis quite a lot - however - I'm seeing quite a few Redis::TimeoutError occurring here and there, from time to time. There is no pattern in the circumstances. It occurs both in the web app and in the background jobs (which is being processed using sidekiq) - not often but from time to time.
Now I have no idea how to track down the root cause of this and hence no idea how to fix it.
Here is a little background on my setup:
The redis instance is running on a separate physical server which is connected to both my web server and background server in a private local 1Gbit network. All servers are running ubuntu 12.04. The redis version is 2.6.10. I'm connecting from my rails app (which is 3.2) using an initializer like so:
require 'redis'require 'redis/objects'REDIS = Redis.new(:url => APP_CONFIG['REDIS_URL'])Redis.current = REDISThis is the output of redis-cli INFO:
# Serverredis_version:2.6.10redis_git_sha1:00000000redis_git_dirty:0redis_mode:standaloneos:Linux 3.2.0-38-generic x86_64arch_bits:64multiplexing_api:epollgcc_version:4.6.3process_id:28475run_id:d89bbb1b81d3169c4228cf23c0988ae437d496a1tcp_port:6379uptime_in_seconds:14913365uptime_in_days:172lru_clock:1507056# Clientsconnected_clients:233client_longest_output_list:0client_biggest_input_buf:0blocked_clients:19# Memoryused_memory:801637360used_memory_human:764.50Mused_memory_rss:594706432used_memory_peak:4295394784used_memory_peak_human:4.00Gused_memory_lua:31744mem_fragmentation_ratio:0.74mem_allocator:jemalloc-3.3.0# Persistenceloading:0rdb_changes_since_last_save:23166rdb_bgsave_in_progress:0rdb_last_save_time:1378219310rdb_last_bgsave_status:okrdb_last_bgsave_time_sec:4rdb_current_bgsave_time_sec:-1aof_enabled:0aof_rewrite_in_progress:0aof_rewrite_scheduled:0aof_last_rewrite_time_sec:-1aof_current_rewrite_time_sec:-1aof_last_bgrewrite_status:ok# Statstotal_connections_received:932395total_commands_processed:3088408103instantaneous_ops_per_sec:837rejected_connections:0expired_keys:31428evicted_keys:3007keyspace_hits:124093049keyspace_misses:53060192pubsub_channels:0pubsub_patterns:0latest_fork_usec:17651# Replicationrole:masterconnected_slaves:1slave0:192.168.0.2,6379,online# CPUused_cpu_sys:54000.21used_cpu_user:73692.52used_cpu_sys_children:36229.79used_cpu_user_children:420655.84# Keyspacedb0:keys=1498962,expires=1310In my redis config I have the following set:
\fidaemonize yespidfile /var/run/redis/redis-server.pidtimeout 0loglevel noticedatabases 1save 900 1save 300 10save 60 10000stop-writes-on-bgsave-error yesrdbcompression yesrdbchecksum yesdbfilename dump.rdbdir /var/lib/redisslave-serve-stale-data yesslave-read-only yesslave-priority 100maxclients 1000maxmemory 4GBmaxmemory-policy volatile-lruappendonly noappendfsync everysecno-appendfsync-on-rewrite noauto-aof-rewrite-percentage 100auto-aof-rewrite-min-size 64mblua-time-limit 5000slowlog-log-slower-than 10000slowlog-max-len 128hash-max-ziplist-entries 512hash-max-ziplist-value 64list-max-ziplist-entries 512list-max-ziplist-value 64set-max-intset-entries 512zset-max-ziplist-entries 128zset-max-ziplist-value 64activerehashing yesclient-output-buffer-limit normal 0 0 0client-output-buffer-limit slave 256mb 64mb 60client-output-buffer-limit pubsub 32mb 8mb 60