mysql - 数据库大于 RAM 的 MySQL 上的 "innodb_buffer_pool_size"

Question

我有一台运行 MySQL 的 Linux 服务器，其中:

-12 Gb RAM 
-4 x Intel(R) Xeon(R) CPU E6510  @ 1.73GHz
-CentOS release 6.3
-MySQL 5.1.61

由于一些技术问题，我们不得不将服务器的 RAM 内存减少到 8 GB，目前我们还没有任何内存。现在，正因为如此，我们的服务器出现了很多性能问题。这是我们数据库的大小:

+--------+--------------------+---------+--------+--------+------------+---------+
| tables | table_schema       | rows    | data   | idx    | total_size | idxfrac |
+--------+--------------------+---------+--------+--------+------------+---------+
|     43 | XXXXXXXX           | 142.81M | 10.52G | 13.31G | 23.83G     |    1.27 |
|     44 | Test_XXXXXXXX      | 55.20M  | 3.57G  | 4.77G  | 8.33G      |    1.34 |
|     34 | XXXXXXXXXXXXXXXX   | 23.04M  | 1.39G  | 1.84G  | 3.24G      |    1.32 |
|     23 | mysql              | 0.00M   | 0.00G  | 0.00G  | 0.00G      |    0.16 |
|     28 | information_schema | NULL    | 0.00G  | 0.00G  | 0.00G      |    NULL |
+--------+--------------------+---------+--------+--------+------------+---------+

这是/etc/my.cnf 的内容:

[mysqld]

max_allowed_packet = 1024M
sort_buffer_size = 512M
max_connections=500

query_cache_size = 512M
query_cache_limit = 512M
query-cache-type = 2

table_cache = 800

thread_cache_size=8
key_buffer_size = 512M
read_buffer_size=64M
read_rnd_buffer_size=64M
myisam_sort_buffer_size=64M

innodb_flush_log_at_trx_commit=2
innodb_buffer_pool_size=7000M
innodb_additional_mem_pool_size=100M

...

我不知道我是否能够真正确定大小和 RAM 之间的关系。但情况是，当我有 12GB RAM 时，一切正常。 “innodb_buffer_pool_size”值为 10000M，性能非常好。但是现在，进行相同的操作需要 4 倍以上。

我们的应用基本上是一个数据库导出器，主要访问一个表，它有 72,314,541 个寄存器。

+--------+---------+--------+--------+------------+---------+
| tables | rows    | data   | idx    | total_size | idxfrac |
+--------+---------+--------+--------+------------+---------+
|      9 | 159.12M | 11.07G | 15.87G | 26.94G     |    1.43 |
+--------+---------+--------+--------+------------+---------+

目前我们正在测试更改“innodb_buffer_pool_size”的值，但看起来我们不会获得更多性能。现在的问题是，我们可以做些什么来提高 MySQL 的性能？

-Put more RAM (obviously)
-Change more variables on /etc/my.cnf?
-MySQL Partitioning for Performance
-...

欢迎所有的想法和贡献，

提前致谢

编辑:添加了表和查询的所有信息

DB的结构是一个从一些传感器接收信息并存储的系统。

测量表:我们从传感器接收到的测量值。

+--------------------+------------+------+-----+---------+----------------+
| Field              | Type       | Null | Key | Default | Extra          |
+--------------------+------------+------+-----+---------+----------------+
| id                 | bigint(20) | NO   | PRI | NULL    | auto_increment |
| version            | bigint(20) | NO   |     | NULL    |                |
| counter            | char(2)    | YES  |     | NULL    |                |
| datemeasurement_id | datetime   | NO   | MUL | NULL    |                |
| datereal_id        | datetime   | NO   | MUL | NULL    |                |
| delayed            | bit(1)     | NO   |     | NULL    |                |
| frequency          | tinyint(4) | YES  |     | NULL    |                |
| measuringentity_id | bigint(20) | NO   | MUL | NULL    |                |
| real               | bit(1)     | NO   |     | NULL    |                |
| tamper             | bit(1)     | NO   |     | NULL    |                |
| value              | float      | NO   |     | NULL    |                |
+--------------------+------------+------+-----+---------+----------------+

measuring_entity 表:一个传感器可以测量不止一件事(温度、湿度)。这些就是实体。

+--------------+------------+------+-----+---------+----------------+
| Field        | Type       | Null | Key | Default | Extra          |
+--------------+------------+------+-----+---------+----------------+
| id           | bigint(20) | NO   | PRI | NULL    | auto_increment |
| version      | bigint(20) | NO   |     | NULL    |                |
| household_id | varchar(4) | NO   | MUL | NULL    |                |
| operative    | bit(1)     | NO   |     | NULL    |                |
| type         | char(20)   | NO   |     | NULL    |                |
| unit         | char(3)    | NO   |     | NULL    |                |
| interval     | float      | YES  |     | NULL    |                |
+--------------+------------+------+-----+---------+----------------+

sensor_measuring_entity:一个传感器可以关联多个实体。

+--------------------+------------+------+-----+---------+-------+
| Field              | Type       | Null | Key | Default | Extra |
+--------------------+------------+------+-----+---------+-------+
| sensor_id          | bigint(20) | NO   | PRI | NULL    |       |
| measuringentity_id | bigint(20) | NO   | PRI | NULL    |       |
| version            | bigint(20) | NO   |     | NULL    |       |
+--------------------+------------+------+-----+---------+-------+

传感器表:传感器的信息，与上表中的测量实体相关。

+---------------------+-------------+------+-----+---------+----------------+
| Field               | Type        | Null | Key | Default | Extra          |
+---------------------+-------------+------+-----+---------+----------------+
| id                  | bigint(20)  | NO   | PRI | NULL    | auto_increment |
| version             | bigint(20)  | NO   |     | NULL    |                |
| battery             | bit(1)      | NO   |     | NULL    |                |
| identifier          | char(6)     | NO   |     | NULL    |                |
| installationdate_id | datetime    | NO   | MUL | NULL    |                |
| lastreceiveddate_id | datetime    | YES  | MUL | NULL    |                |
| location_id         | bigint(20)  | NO   | MUL | NULL    |                |
| operative           | bit(1)      | NO   |     | NULL    |                |
| tampererror         | smallint(6) | NO   |     | NULL    |                |
+---------------------+-------------+------+-----+---------+----------------+

位置表:传感器的放置位置。

+------------+------------+------+-----+---------+----------------+
| Field      | Type       | Null | Key | Default | Extra          |
+------------+------------+------+-----+---------+----------------+
| id         | bigint(20) | NO   | PRI | NULL    | auto_increment |
| version    | bigint(20) | NO   |     | NULL    |                |
| height     | tinyint(4) | YES  |     | NULL    |                |
| operative  | bit(1)     | NO   |     | NULL    |                |
| place      | char(15)   | NO   | MUL | NULL    |                |
| room       | char(15)   | NO   |     | NULL    |                |
| typesensor | char(15)   | NO   |     | NULL    |                |
| formaster  | bit(1)     | YES  |     | NULL    |                |
+------------+------------+------+-----+---------+----------------+

这是查询(针对所有房屋和所有传感器):

for (int z = 0; z < allHouses.length; z++) {
for (int j = 0; j < sensorlist.length; j++) {

sql.eachRow ("SELECT m.datemeasurement_id, s.identifier, me.type, m.value"
+ " FROM measurement as m"
+ " JOIN measuring_entity as me ON m.measuringentity_id = me.id"
+ " JOIN sensor_measuring_entity as sme ON sme.measuringentity_id = me.id"
+ " JOIN sensor as s ON sme.sensor_id = s.id"
+ " WHERE me.id = $actualmeid"
+ " AND me.household_id = '$mHouse'"
+ " AND m.datemeasurement_id >= '$cons_startDate'"
+ " AND m.datemeasurement_id <= '$cons_endDate'"
+ " AND m.datemeasurement_id > '$startDate'"
+ " AND m.datemeasurement_id < '$endDate'"
+ " ORDER BY datemeasurement_id")
{
}}

PD:一切都是一个 Grails 应用程序的一部分。

Answer 1

您分配了过多的 RAM 并开始进行交换。交换比缩小设置要糟糕得多。

对于您的 8GB，建议进行这些更改

innodb_buffer_pool_size = 5000Mquery_cache_size = 50M

如果你说你使用的是MyISAM还是InnoDB，我会给你更多提示。