redis used_memory 比 used_memory_rss 大

Question

Redis Master大约有90个key，最长的key大约46字节，但是master占用了3GB的内存，这里是master info信息

# Server
redis_version:3.2.8
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:b45e9949f92f30de
redis_mode:standalone
os:Linux 3.10.0-327.36.2.el7.ppc64 ppc64
arch_bits:64
multiplexing_api:epoll
gcc_version:4.8.5
process_id:150358
run_id:acfc6247d94cf0c62a98694adf35e3ff9f1c0d9d
tcp_port:6379
uptime_in_seconds:3539
uptime_in_days:0
hz:10
lru_clock:14518804
executable:/home/redis/redis-3.2.8/config/redis-server
config_file:/home/redis/redis-3.2.8/config/server_6379.conf

# Clients
connected_clients:37
client_longest_output_list:0
client_biggest_input_buf:0
blocked_clients:0

# Memory
used_memory:3223126336
used_memory_human:3.00G
used_memory_rss:19988480
used_memory_rss_human:19.06M
used_memory_peak:3223657672
used_memory_peak_human:3.00G
total_system_memory:1071411167232
total_system_memory_human:997.83G
used_memory_lua:37888
used_memory_lua_human:37.00K
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
mem_fragmentation_ratio:0.01
mem_allocator:jemalloc-4.0.3

# Persistence
loading:0
rdb_changes_since_last_save:143046
rdb_bgsave_in_progress:0
rdb_last_save_time:1776122944
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:-1
rdb_current_bgsave_time_sec:-1
aof_enabled:1
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_last_write_status:ok
aof_current_size:9266469
aof_base_size:0
aof_pending_rewrite:0
aof_buffer_length:0
aof_rewrite_buffer_length:0
aof_pending_bio_fsync:0
aof_delayed_fsync:0

# Stats
total_connections_received:78
total_commands_processed:309390
instantaneous_ops_per_sec:126
total_net_input_bytes:21927610
total_net_output_bytes:62716490
instantaneous_input_kbps:8.79
instantaneous_output_kbps:12.20
rejected_connections:0
sync_full:2
sync_partial_ok:0
sync_partial_err:0
expired_keys:0
evicted_keys:0
keyspace_hits:47603
keyspace_misses:47731
pubsub_channels:1
pubsub_patterns:0
latest_fork_usec:206
migrate_cached_sockets:0

# Replication
role:master
connected_slaves:2
slave0:ip=10.124.152.8,port=6379,state=online,offset=9995541,lag=1
slave1:ip=10.124.152.7,port=6379,state=online,offset=9997441,lag=1
master_repl_offset:9998557
repl_backlog_active:1
repl_backlog_size:3221225472
repl_backlog_first_byte_offset:2
repl_backlog_histlen:9998556

# CPU
used_cpu_sys:7.61
used_cpu_user:3.37
used_cpu_sys_children:0.00
used_cpu_user_children:0.00

# Cluster
cluster_enabled:0

# Keyspace
db0:keys=90,expires=0,avg_ttl=0

以及slave info信息

# Memory
used_memory:761448
used_memory_human:743.60K
used_memory_rss:7536640
used_memory_rss_human:7.19M
used_memory_peak:823488
used_memory_peak_human:804.19K
total_system_memory:1071411167232
total_system_memory_human:997.83G
used_memory_lua:37888
used_memory_lua_human:37.00K
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
mem_fragmentation_ratio:9.90
mem_allocator:jemalloc-4.0.3
127.0.0.1:6379> info
# Server
redis_version:3.2.8
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:17a905ed68c0b83
redis_mode:standalone
os:Linux 3.10.0-327.36.2.el7.ppc64 ppc64
arch_bits:64
multiplexing_api:epoll
gcc_version:4.8.5
process_id:151704
run_id:2df76a29acc2910fff7e1ea77203caf0758b23dd
tcp_port:6379
uptime_in_seconds:3673
uptime_in_days:0
hz:10
lru_clock:14518975
executable:/home/redis/redis-3.2.8/config/redis-server
config_file:/home/redis/redis-3.2.8/config/server_6379.conf

# Clients
connected_clients:8
client_longest_output_list:0
client_biggest_input_buf:0
blocked_clients:0

# Memory
used_memory:803360
used_memory_human:784.53K
used_memory_rss:7536640
used_memory_rss_human:7.19M
used_memory_peak:844336
used_memory_peak_human:824.55K
total_system_memory:1071411167232
total_system_memory_human:997.83G
used_memory_lua:37888
used_memory_lua_human:37.00K
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
mem_fragmentation_ratio:9.38
mem_allocator:jemalloc-4.0.3

# Persistence
loading:0
rdb_changes_since_last_save:150387
rdb_bgsave_in_progress:0
rdb_last_save_time:1776122982
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:-1
rdb_current_bgsave_time_sec:-1
aof_enabled:1
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:0
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_last_write_status:ok
aof_current_size:9743688
aof_base_size:0
aof_pending_rewrite:0
aof_buffer_length:0
aof_rewrite_buffer_length:0
aof_pending_bio_fsync:0
aof_delayed_fsync:0

# Stats
total_connections_received:7
total_commands_processed:173861
instantaneous_ops_per_sec:58
total_net_input_bytes:11409472
total_net_output_bytes:13484792
instantaneous_input_kbps:3.73
instantaneous_output_kbps:6.38
rejected_connections:0
sync_full:0
sync_partial_ok:0
sync_partial_err:0
expired_keys:0
evicted_keys:0
keyspace_hits:0
keyspace_misses:0
pubsub_channels:1
pubsub_patterns:0
latest_fork_usec:203
migrate_cached_sockets:0

# Replication
role:slave
master_host:10.124.152.9
master_port:6379
master_link_status:up
master_last_io_seconds_ago:0
master_sync_in_progress:0
slave_repl_offset:10502941
slave_priority:100
slave_read_only:1
connected_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:3221225472
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0

# CPU
used_cpu_sys:4.19
used_cpu_user:1.72
used_cpu_sys_children:0.00
used_cpu_user_children:0.00

# Cluster
cluster_enabled:0

# Keyspace
db0:keys=90,expires=0,avg_ttl=0

造成这种情况的原因是什么？

Answer 1

来自 redis 文档:

Redis will not always free up (return) memory to the OS when keys are removed. This is not something special about Redis, but it is how most malloc() implementations work. For example if you fill an instance with 5GB worth of data, and then remove the equivalent of 2GB of data, the Resident Set Size (also known as the RSS, which is the number of memory pages consumed by the process) will probably still be around 5GB, even if Redis will claim that the user memory is around 3GB. This happens because the underlying allocator can't easily release the memory. For example often most of the removed keys were allocated in the same pages as the other keys that still exist.

1. used_memory 小于 used_memory_rss 表示内存碎片化。
2. used_memory 大于 used_memory_rss 意味着你的物理 RAM 已用完，部分 Redis 数据驻留在磁盘中交换空间。

但是，您的问题可能出在我们无法想象的地方。 90 个 46 字节的 key 绝对不会产生 used_memory_human:3.00G!更奇怪的是，Master 的info 输出显示itotal_system_memory_human:997.83G，这意味着您的服务器的总物理内存是997.83G!