大量更新应用程序中的 MySQL 性能调优

Question

我有一个应用程序，它非常重视对数据库的更新操作，而不是(更典型的)SELECT 检索。只是一个大概的猜测，但每个 SELECT 可能有 100 个更新。

使用的MySQL服务器是5.6.31-0ubuntu0.14.04.2-log。我对服务器的性能做了很多调整，包括设置如下选项:

innodb_change_buffer_max_size=50

(innodb_change_buffer_max_size 上的文档是 HERE。我将在本文末尾发布所有服务器变量。)

我遇到的问题是服务器性能中的周期性“打嗝”，导致小型 SQL 查询集群延迟几秒钟。例如，一系列查询，其中每个查询通常在几毫秒内完成，在“打嗝”期间每个查询将花费 3-110 秒。 (是的，有时会延迟近两分钟。)

虽然延迟集群中有许多不同的 SQL 查询，但集群中几乎所有查询都使用 MySQL 的“INSERT ... ON DUPLICATE KEY UPDATE”语句类型。例如:

INSERT INTO `PerfCounters` SET `PerfID`='scriptname',`PID`=27802
  ON DUPLICATE KEY UPDATE `ThisHour`=`ThisHour`+1;

这个特定示例的表结构是:

CREATE TABLE IF NOT EXISTS `PerfCounters` (
  `PerfID` varchar(30) NOT NULL,
  `PID` int(11) NOT NULL DEFAULT '0',
  `ThisHour` int(11) NOT NULL DEFAULT '1',
  PRIMARY KEY (`PerfID`,`PID`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

我越来越相信“打嗝”表明 MySQL 正忙于执行某种管理或维护功能，这些功能必须完成才能继续处理其他未决查询。但是，我不确定。

这是来自上述示例的慢查询日志的条目:

# User@Host: demouser[demouser] @ mysql.myhost.com [0.0.0.0]  Id: 453134
# Query_time: 2.822141  Lock_time: 0.000096 Rows_sent: 0  Rows_examined: 0
SET timestamp=1473074105;
INSERT INTO `PerfCounters` SET `PerfID`='scriptname',`PID`=27802 ON DUPLICATE KEY UPDATE `ThisHour`=`ThisHour`+1;

如您所见，延迟不是由于行锁或任何其他锁定争用问题造成的。事实上，因为“PID”(进程 ID)是复合键的一部分，并且没有两个并发任务可以具有相同的 PID，所以可以 100% 保证永远不会有行锁。

有人对这种特定情况的性能调优有什么建议吗？是否有已知的特定服务器变量设置可以提高更新频繁的应用程序的性能？

也许更重要的是:延迟“打嗝”真的表明 MySQL 正忙于执行某种高优先级的管理或维护操作吗？

服务器变量

auto_increment_increment=1
auto_increment_offset=1
autocommit=ON
automatic_sp_privileges=ON
avoid_temporal_upgrade=OFF
back_log=90
basedir=/usr
big_tables=OFF
bind_address=mysql.myhost.com
binlog_cache_size=32,768
binlog_checksum=CRC32
binlog_direct_non_transactional_updates=OFF
binlog_error_action=IGNORE_ERROR
binlog_format=STATEMENT
binlog_gtid_simple_recovery=OFF
binlog_max_flush_queue_time=0
binlog_order_commits=ON
binlog_row_image=FULL
binlog_rows_query_log_events=OFF
binlog_stmt_cache_size=32,768
binlogging_impossible_mode=IGNORE_ERROR
block_encryption_mode=aes-128-ecb
bulk_insert_buffer_size=8,388,608
character_set_client=utf8
character_set_connection=utf8
character_set_database=latin1
character_set_filesystem=binary
character_set_results=utf8
character_set_server=latin1
character_set_system=utf8
character_sets_dir=/usr/share/mysql/charsets/
collation_connection=utf8_general_ci
collation_database=latin1_swedish_ci
collation_server=latin1_swedish_ci
completion_type=NO_CHAIN
concurrent_insert=AUTO
connect_timeout=10
core_file=OFF
datadir=/var/lib/mysql/
date_format=%Y-%m-%d
datetime_format=%Y-%m-%d_%H:%i:%s
default_storage_engine=InnoDB
default_tmp_storage_engine=InnoDB
default_week_format=0
delay_key_write=ON
delayed_insert_limit=100
delayed_insert_timeout=300
delayed_queue_size=1,000
disconnect_on_expired_password=ON
div_precision_increment=4
end_markers_in_json=OFF
enforce_gtid_consistency=OFF
eq_range_index_dive_limit=10
error_count=0
event_scheduler=OFF
expire_logs_days=10
explicit_defaults_for_timestamp=OFF
external_user=
flush=OFF
flush_time=0
foreign_key_checks=ON
ft_boolean_syntax=+_-><()~*:""&|
ft_max_word_len=84
ft_min_word_len=4
ft_query_expansion_limit=20
ft_stopword_file=(built-in)
general_log=OFF
general_log_file=/var/log/mysql/mysql.log
group_concat_max_len=1,024
gtid_executed=
gtid_mode=OFF
gtid_next=AUTOMATIC
gtid_owned=
gtid_purged=
have_compress=YES
have_crypt=YES
have_dynamic_loading=YES
have_geometry=YES
have_openssl=DISABLED
have_profiling=YES
have_query_cache=YES
have_rtree_keys=YES
have_ssl=DISABLED
have_symlink=YES
host_cache_size=328
hostname=mysqlhost.com
identity=0
ignore_builtin_innodb=OFF
ignore_db_dirs=
init_connect=
init_file=
init_slave=
innodb_adaptive_flushing=ON
innodb_adaptive_flushing_lwm=10
innodb_adaptive_hash_index=ON
innodb_adaptive_max_sleep_delay=150,000
innodb_additional_mem_pool_size=8,388,608
innodb_api_bk_commit_interval=5
innodb_api_disable_rowlock=OFF
innodb_api_enable_binlog=OFF
innodb_api_enable_mdl=OFF
innodb_api_trx_level=0
innodb_autoextend_increment=64
innodb_autoinc_lock_mode=1
innodb_buffer_pool_dump_at_shutdown=OFF
innodb_buffer_pool_dump_now=OFF
innodb_buffer_pool_filename=ib_buffer_pool
innodb_buffer_pool_instances=8
innodb_buffer_pool_load_abort=OFF
innodb_buffer_pool_load_at_startup=OFF
innodb_buffer_pool_load_now=OFF
innodb_buffer_pool_size=2,147,483,648
innodb_change_buffer_max_size=50
innodb_change_buffering=all
innodb_checksum_algorithm=innodb
innodb_checksums=ON
innodb_cmp_per_index_enabled=OFF
innodb_commit_concurrency=0
innodb_compression_failure_threshold_pct=5
innodb_compression_level=6
innodb_compression_pad_pct_max=50
innodb_concurrency_tickets=5,000
innodb_data_file_path=ibdata1:12M:autoextend
innodb_data_home_dir=
innodb_disable_sort_file_cache=OFF
innodb_doublewrite=ON
innodb_fast_shutdown=1
innodb_file_format=Antelope
innodb_file_format_check=ON
innodb_file_format_max=Antelope
innodb_file_per_table=ON
innodb_flush_log_at_timeout=1
innodb_flush_log_at_trx_commit=1
innodb_flush_method=O_DIRECT
innodb_flush_neighbors=1
innodb_flushing_avg_loops=30
innodb_force_load_corrupted=OFF
innodb_force_recovery=0
innodb_ft_aux_table=
innodb_ft_cache_size=8,000,000
innodb_ft_enable_diag_print=OFF
innodb_ft_enable_stopword=ON
innodb_ft_max_token_size=84
innodb_ft_min_token_size=3
innodb_ft_num_word_optimize=2,000
innodb_ft_result_cache_limit=2,000,000,000
innodb_ft_server_stopword_table=
innodb_ft_sort_pll_degree=2
innodb_ft_total_cache_size=640,000,000
innodb_ft_user_stopword_table=
innodb_io_capacity=200
innodb_io_capacity_max=2,000
innodb_large_prefix=OFF
innodb_lock_wait_timeout=50
innodb_locks_unsafe_for_binlog=OFF
innodb_log_buffer_size=33,554,432
innodb_log_compressed_pages=ON
innodb_log_file_size=134,217,728
innodb_log_files_in_group=2
innodb_log_group_home_dir=./
innodb_lru_scan_depth=1,024
innodb_max_dirty_pages_pct=75
innodb_max_dirty_pages_pct_lwm=25
innodb_max_purge_lag=0
innodb_max_purge_lag_delay=0
innodb_mirrored_log_groups=1
innodb_monitor_disable=
innodb_monitor_enable=
innodb_monitor_reset=
innodb_monitor_reset_all=
innodb_old_blocks_pct=37
innodb_old_blocks_time=1,000
innodb_online_alter_log_max_size=134,217,728
innodb_open_files=2,000
innodb_optimize_fulltext_only=OFF
innodb_page_size=16,384
innodb_print_all_deadlocks=OFF
innodb_purge_batch_size=300
innodb_purge_threads=1
innodb_random_read_ahead=OFF
innodb_read_ahead_threshold=56
innodb_read_io_threads=4
innodb_read_only=OFF
innodb_replication_delay=0
innodb_rollback_on_timeout=OFF
innodb_rollback_segments=128
innodb_sort_buffer_size=1,048,576
innodb_spin_wait_delay=6
innodb_stats_auto_recalc=ON
innodb_stats_method=nulls_equal
innodb_stats_on_metadata=OFF
innodb_stats_persistent=ON
innodb_stats_persistent_sample_pages=20
innodb_stats_sample_pages=8
innodb_stats_transient_sample_pages=8
innodb_status_output=OFF
innodb_status_output_locks=OFF
innodb_strict_mode=OFF
innodb_support_xa=ON
innodb_sync_array_size=1
innodb_sync_spin_loops=30
innodb_table_locks=ON
innodb_thread_concurrency=0
innodb_thread_sleep_delay=10,000
innodb_tmpdir=
innodb_undo_directory=.
innodb_undo_logs=128
innodb_undo_tablespaces=0
innodb_use_native_aio=OFF
innodb_use_sys_malloc=ON
innodb_version=5.6.31
innodb_write_io_threads=4
insert_id=0
interactive_timeout=28,800
join_buffer_size=4,194,304
keep_files_on_create=OFF
key_buffer_size=301,989,888
key_cache_age_threshold=300
key_cache_block_size=1,024
key_cache_division_limit=100
large_files_support=ON
large_page_size=0
large_pages=OFF
last_insert_id=0
lc_messages=en_US
lc_messages_dir=/usr/share/mysql/
lc_time_names=en_US
license=GPL
local_infile=ON
lock_wait_timeout=31,536,000
locked_in_memory=OFF
log_bin=OFF
log_bin_basename=
log_bin_index=
log_bin_trust_function_creators=OFF
log_bin_use_v1_row_events=OFF
log_error=/var/log/mysql/error.log
log_output=FILE
log_queries_not_using_indexes=OFF
log_slave_updates=OFF
log_slow_admin_statements=OFF
log_slow_slave_statements=OFF
log_throttle_queries_not_using_indexes=0
log_warnings=2
long_query_time=10
low_priority_updates=OFF
lower_case_file_system=OFF
lower_case_table_names=0
master_info_repository=FILE
master_verify_checksum=OFF
max_allowed_packet=16,777,216
max_binlog_cache_size=18446744073709547520
max_binlog_size=1,073,741,824
max_binlog_stmt_cache_size=18446744073709547520
max_connect_errors=10,000
max_connections=200
max_delayed_threads=20
max_digest_length=1,024
max_error_count=64
max_heap_table_size=134,217,728
max_insert_delayed_threads=20
max_join_size=18446744073709551615
max_length_for_sort_data=1,024
max_prepared_stmt_count=16,382
max_relay_log_size=0
max_seeks_for_key=18446744073709551615
max_sort_length=1,024
max_sp_recursion_depth=0
max_tmp_tables=32
max_user_connections=0
max_write_lock_count=18446744073709551615
metadata_locks_cache_size=1,024
metadata_locks_hash_instances=8
min_examined_row_limit=0
multi_range_count=256
myisam_data_pointer_size=6
myisam_max_sort_file_size=9223372036853727232
myisam_mmap_size=18446744073709551615
myisam_recover_options=BACKUP,FORCE
myisam_repair_threads=1
myisam_sort_buffer_size=8,388,608
myisam_stats_method=nulls_unequal
myisam_use_mmap=OFF
net_buffer_length=16,384
net_read_timeout=30
net_retry_count=10
net_write_timeout=60
new=OFF
old=OFF
old_alter_table=OFF
old_passwords=0
open_files_limit=10,000
optimizer_prune_level=1
optimizer_search_depth=62
optimizer_switch=index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,engine_condition_pushdown=on,index_condition_pushdown=on,mrr=on,mrr_cost_based=on,block_nested_loop=on,batched_key_access=off,materialization=on,semijoin=on,loosescan=on,firstmatch=on,subquery_materialization_cost_based=on,use_index_extensions=on
optimizer_trace=enabled=off,one_line=off
optimizer_trace_features=greedy_search=on,range_optimizer=on,dynamic_range=on,repeated_subselect=on
optimizer_trace_limit=1
optimizer_trace_max_mem_size=16,384
optimizer_trace_offset=-1
performance_schema=OFF
performance_schema_accounts_size=-1
performance_schema_digests_size=-1
performance_schema_events_stages_history_long_size=-1
performance_schema_events_stages_history_size=-1
performance_schema_events_statements_history_long_size=-1
performance_schema_events_statements_history_size=-1
performance_schema_events_waits_history_long_size=-1
performance_schema_events_waits_history_size=-1
performance_schema_hosts_size=-1
performance_schema_max_cond_classes=80
performance_schema_max_cond_instances=-1
performance_schema_max_digest_length=1,024
performance_schema_max_file_classes=50
performance_schema_max_file_handles=32,768
performance_schema_max_file_instances=-1
performance_schema_max_mutex_classes=200
performance_schema_max_mutex_instances=-1
performance_schema_max_rwlock_classes=40
performance_schema_max_rwlock_instances=-1
performance_schema_max_socket_classes=10
performance_schema_max_socket_instances=-1
performance_schema_max_stage_classes=150
performance_schema_max_statement_classes=168
performance_schema_max_table_handles=-1
performance_schema_max_table_instances=-1
performance_schema_max_thread_classes=50
performance_schema_max_thread_instances=-1
performance_schema_session_connect_attrs_size=-1
performance_schema_setup_actors_size=100
performance_schema_setup_objects_size=100
performance_schema_users_size=-1
pid_file=/var/run/mysqld/mysqld.pid
plugin_dir=/usr/lib/mysql/plugin/
port=3,306
preload_buffer_size=32,768
profiling=OFF
profiling_history_size=15
protocol_version=10
proxy_user=
pseudo_slave_mode=OFF
pseudo_thread_id=517,506
query_alloc_block_size=8,192
query_cache_limit=5,242,880
query_cache_min_res_unit=4,096
query_cache_size=52,428,800
query_cache_type=OFF
query_cache_wlock_invalidate=OFF
query_prealloc_size=8,192
rand_seed1=0
rand_seed2=0
range_alloc_block_size=4,096
read_buffer_size=1,048,576
read_only=OFF
read_rnd_buffer_size=262,144
relay_log=
relay_log_basename=
relay_log_index=
relay_log_info_file=relay-log.info
relay_log_info_repository=FILE
relay_log_purge=ON
relay_log_recovery=OFF
relay_log_space_limit=0
report_host=
report_password=
report_port=3,306
report_user=
rpl_stop_slave_timeout=31,536,000
secure_auth=ON
secure_file_priv=
server_id=0
server_id_bits=32
server_uuid=993bf09d-45a2-11e6-98f0-fa163ecd49dc
show_old_temporals=OFF
simplified_binlog_gtid_recovery=OFF
skip_external_locking=ON
skip_name_resolve=OFF
skip_networking=OFF
skip_show_database=OFF
slave_allow_batching=OFF
slave_checkpoint_group=512
slave_checkpoint_period=300
slave_compressed_protocol=OFF
slave_exec_mode=STRICT
slave_load_tmpdir=/tmp
slave_max_allowed_packet=1,073,741,824
slave_net_timeout=3,600
slave_parallel_workers=0
slave_pending_jobs_size_max=16,777,216
slave_rows_search_algorithms=TABLE_SCAN,INDEX_SCAN
slave_skip_errors=OFF
slave_sql_verify_checksum=ON
slave_transaction_retries=10
slave_type_conversions=
slow_launch_time=2
slow_query_log=ON
slow_query_log_file=/var/log/mysql/mysql-slow.log
socket=/var/run/mysqld/mysqld.sock
sort_buffer_size=2,097,152
sql_auto_is_null=OFF
sql_big_selects=ON
sql_buffer_result=OFF
sql_log_bin=ON
sql_log_off=OFF
sql_mode=NO_ENGINE_SUBSTITUTION
sql_notes=ON
sql_quote_show_create=ON
sql_safe_updates=OFF
sql_select_limit=18446744073709551615
sql_slave_skip_counter=0
sql_warnings=OFF
ssl_ca=
ssl_capath=
ssl_cert=
ssl_cipher=
ssl_crl=
ssl_crlpath=
ssl_key=
storage_engine=InnoDB
stored_program_cache=256
sync_binlog=0
sync_frm=ON
sync_master_info=10,000
sync_relay_log=10,000
sync_relay_log_info=10,000
system_time_zone=PDT
table_definition_cache=1,400
table_open_cache=2,000
table_open_cache_instances=1
thread_cache_size=64
thread_concurrency=10
thread_handling=one-thread-per-connection
thread_stack=262,144
time_format=%H:%i:%s
time_zone=SYSTEM
timed_mutexes=OFF
timestamp=1473081753.003525
tmp_table_size=134,217,728
tmpdir=/tmp
transaction_alloc_block_size=8,192
transaction_allow_batching=OFF
transaction_prealloc_size=4,096
tx_isolation=REPEATABLE-READ
tx_read_only=OFF
unique_checks=ON
updatable_views_with_limit=YES
version=5.6.31-0ubuntu0.14.04.2-log
version_comment=(Ubuntu)
version_compile_machine=x86_64
version_compile_os=debian-linux-gnu
wait_timeout=28,800
warning_count=0

Answer 1

多少内存？如果你正在交换，那可能会导致严重的性能问题，甚至可能像你看到的那样打嗝。

binlog_error_action=IGNORE_ERROR  -- sweeping something under the rug?
innodb_buffer_pool_instances=8 -- but size only 2G; set instances=1
                          --  or set size to 70% of _available_ RAM
key_buffer_size=301,989,888 -- If not using MyISAM, set to 10M
long_query_time=10 -- change to 2 or less so you can catch slow queries.
max_heap_table_size=134,217,728 -- not more than 1% of RAM
tmp_table_size=134,217,728 -- not more than 1% of RAM
open_files_limit=10,000 -- that's rather high?

如果您想进一步分析，请提供SHOW GLOBAL STATUS(至少在正常运行一天后)。