MySQL优化存储过程中慢UNION查询-6ren

MySQL优化存储过程中慢UNION查询

转载作者：行者123 更新时间：2023-11-29 11:28:23

我的存储过程中有以下代码，它基本上将 SQL 文本连接到 while 循环中的一个大查询中，然后执行它。我有来自 7 个不同表的 7 个 UNION ALL 语句，其中包含一些输入参数(table_name1...7、start_date ，结束日期):

BEGIN

DECLARE v_max INT UNSIGNED DEFAULT 58; 
DECLARE v_counter INT UNSIGNED DEFAULT 1; 
SET @sql_text = '';
WHILE v_counter < v_max DO
-- Build SQL string
SET @sql_text = CONCAT(@sql_text, 
CONCAT(' SELECT 
(SELECT i.name FROM instrument i WHERE i.id =',v_counter,' LIMIT 1) AS Instrument,
(SELECT AVG(p.ask-p.bid) AS avg_spread
FROM ',table_name1,' p
WHERE p.instrument_id=',v_counter,' AND  FROM_UNIXTIME(p.system_datetime/1000000) BETWEEN \'',start_date,'\' AND \'',end_date,'\') AS One

,

(SELECT AVG(p.ask-p.bid) AS avg_spread
FROM ', table_name2,' p
WHERE p.instrument_id=',v_counter,' AND FROM_UNIXTIME(p.system_datetime/1000000) BETWEEN \'',start_date,'\' AND \'', end_date,'\') AS Two

,

(SELECT AVG(p.ask-p.bid) AS avg_spread
FROM ',table_name3,' p
WHERE p.instrument_id=',v_counter,' AND FROM_UNIXTIME(p.system_datetime/1000000) BETWEEN \'', start_date,'\' AND \'',end_date,'\') AS Three

/* ... more unions here ...*/
,

(SELECT AVG(p.ask-p.bid) AS avg_spread
FROM ',table_name7,' p
WHERE p.instrument_id=',v_counter,' AND FROM_UNIXTIME(p.system_datetime/1000000) BETWEEN \'', start_date,'\' AND \'',end_date,'\') AS Seven UNION ALL ')

);


SET v_counter=v_counter+1; 

END WHILE;

-- Remove extra UNION ALL from SQL string
SET @sql_text = LEFT(@sql_text,LENGTH(@sql_text) - LENGTH(' UNION ALL'));
-- SELECT LEFT(@sql_text,LENGTH(@sql_text) - LENGTH(' UNION ALL'));

-- Prepare and execute statement
PREPARE stmt3 FROM @sql_text;
EXECUTE stmt3;
DEALLOCATE PREPARE stmt3;
END

请问有什么办法可以优化这个查询(七个表平均每个表有350万条记录)，执行时间在6到25分钟之间？表是 MyISAM，相同，以下是它们的结构:

CREATE TABLE `prices_all_2016_06_13_5` (
    `id` INT(11) UNSIGNED NOT NULL AUTO_INCREMENT,
    `instrument_id` TINYINT(4) NOT NULL,
    `ask` DECIMAL(12,6) NOT NULL,
    `bid` DECIMAL(12,6) NOT NULL,
    `system_datetime` DECIMAL(20,0) NOT NULL,
    PRIMARY KEY (`id`) USING BTREE,
    INDEX `Index 3` (`instrument_id`) USING HASH,
    INDEX `Index 4` (`system_datetime`)
)
COLLATE='utf8_unicode_ci'
ENGINE=MyISAM
ROW_FORMAT=COMPACT
AUTO_INCREMENT=3882184
;

编辑:这是解释:

"id"    "select_type"   "table" "type"  "possible_keys" "key"   "key_len"   "ref"   "rows"  "Extra"
"1" "PRIMARY"   \N  \N  \N  \N  \N  \N  \N  "No tables used"
"9" "SUBQUERY"  \N  \N  \N  \N  \N  \N  \N  "Impossible WHERE noticed after reading const tables"
"8" "SUBQUERY"  "p" "ref"   "Index 3"   "Index 3"   "1" "const" "64566" "Using where"
"7" "SUBQUERY"  "p" "ref"   "Index 3"   "Index 3"   "1" "const" "71061" "Using where"
"6" "SUBQUERY"  "p" "ref"   "Index 3"   "Index 3"   "1" "const" "43979" "Using where"
"5" "SUBQUERY"  "p" "ref"   "Index 3"   "Index 3"   "1" "const" "69852" "Using where"
"4" "SUBQUERY"  "p" "ref"   "Index 3"   "Index 3"   "1" "const" "117355"    "Using where"
"3" "SUBQUERY"  "p" "ref"   "Index 3"   "Index 3"   "1" "const" "70672" "Using where"
"2" "SUBQUERY"  "i" "const" "PRIMARY"   "PRIMARY"   "4" "const" "1" \N
"10"    "UNION" \N  \N  \N  \N  \N  \N  \N  "No tables used"
"18"    "SUBQUERY"  \N  \N  \N  \N  \N  \N  \N  "Impossible WHERE noticed after reading const tables"
"17"    "SUBQUERY"  "p" "ref"   "Index 3"   "Index 3"   "1" "const" "52505" "Using where"
"16"    "SUBQUERY"  "p" "ref"   "Index 3"   "Index 3"   "1" "const" "68781" "Using where"
"15"    "SUBQUERY"  "p" "ref"   "Index 3"   "Index 3"   "1" "const" "40055" "Using where"
"14"    "SUBQUERY"  "p" "ref"   "Index 3"   "Index 3"   "1" "const" "32054" "Using where"
"13"    "SUBQUERY"  "p" "ref"   "Index 3"   "Index 3"   "1" "const" "57465" "Using where"
"12"    "SUBQUERY"  "p" "ref"   "Index 3"   "Index 3"   "1" "const" "33265" "Using where"
"11"    "SUBQUERY"  "i" "const" "PRIMARY"   "PRIMARY"   "4" "const" "1" \N
"19"    "UNION" \N  \N  \N  \N  \N  \N  \N  "No tables used"
"27"    "SUBQUERY"  \N  \N  \N  \N  \N  \N  \N  "Impossible WHERE noticed after reading const tables"
"26"    "SUBQUERY"  "p" "ref"   "Index 3"   "Index 3"   "1" "const" "75411" "Using where"
"25"    "SUBQUERY"  "p" "ref"   "Index 3"   "Index 3"   "1" "const" "91574" "Using where"
"24"    "SUBQUERY"  "p" "ref"   "Index 3"   "Index 3"   "1" "const" "47560" "Using where"
"23"    "SUBQUERY"  "p" "ref"   "Index 3"   "Index 3"   "1" "const" "80731" "Using where"
"22"    "SUBQUERY"  "p" "ref"   "Index 3"   "Index 3"   "1" "const" "139225"    "Using where"
"21"    "SUBQUERY"  "p" "ref"   "Index 3"   "Index 3"   "1" "const" "109131"    "Using where"
"20"    "SUBQUERY"  "i" "const" "PRIMARY"   "PRIMARY"   "4" "const" "1" \N
"28"    "UNION" \N  \N  \N  \N  \N  \N  \N  "No tables used"
"36"    "SUBQUERY"  \N  \N  \N  \N  \N  \N  \N  "Impossible WHERE noticed after reading const tables"
"35"    "SUBQUERY"  "p" "ref"   "Index 3"   "Index 3"   "1" "const" "67288" "Using where"
"34"    "SUBQUERY"  "p" "ref"   "Index 3"   "Index 3"   "1" "const" "83522" "Using where"
"33"    "SUBQUERY"  "p" "ref"   "Index 3"   "Index 3"   "1" "const" "46650" "Using where"
"32"    "SUBQUERY"  "p" "ref"   "Index 3"   "Index 3"   "1" "const" "50401" "Using where"
"31"    "SUBQUERY"  "p" "ref"   "Index 3"   "Index 3"   "1" "const" "73280" "Using where"
"30"    "SUBQUERY"  "p" "ref"   "Index 3"   "Index 3"   "1" "const" "50836" "Using where"
"29"    "SUBQUERY"  "i" "const" "PRIMARY"   "PRIMARY"   "4" "const" "1" \N
"37"    "UNION" \N  \N  \N  \N  \N  \N  \N  "No tables used"
"45"    "SUBQUERY"  \N  \N  \N  \N  \N  \N  \N  "Impossible WHERE noticed after reading const tables"
"44"    "SUBQUERY"  "p" "ref"   "Index 3"   "Index 3"   "1" "const" "59256" "Using where"
"43"    "SUBQUERY"  "p" "ref"   "Index 3"   "Index 3"   "1" "const" "54092" "Using where"
"42"    "SUBQUERY"  "p" "ref"   "Index 3"   "Index 3"   "1" "const" "41561" "Using where"
"41"    "SUBQUERY"  "p" "ref"   "Index 3"   "Index 3"   "1" "const" "51928" "Using where"
"40"    "SUBQUERY"  "p" "ref"   "Index 3"   "Index 3"   "1" "const" "64368" "Using where"
"39"    "SUBQUERY"  "p" "ref"   "Index 3"   "Index 3"   "1" "const" "60471" "Using where"
"38"    "SUBQUERY"  "i" "const" "PRIMARY"   "PRIMARY"   "4" "const" "1" \N
"46"    "UNION" \N  \N  \N  \N  \N  \N  \N  "No tables used"
"54"    "SUBQUERY"  \N  \N  \N  \N  \N  \N  \N  "Impossible WHERE noticed after reading const tables"
"53"    "SUBQUERY"  "p" "ref"   "Index 3"   "Index 3"   "1" "const" "59213" "Using where"
"52"    "SUBQUERY"  "p" "ref"   "Index 3"   "Index 3"   "1" "const" "56071" "Using where"
"51"    "SUBQUERY"  "p" "ref"   "Index 3"   "Index 3"   "1" "const" "47653" "Using where"
"50"    "SUBQUERY"  "p" "ref"   "Index 3"   "Index 3"   "1" "const" "46033" "Using where"
"49"    "SUBQUERY"  "p" "ref"   "Index 3"   "Index 3"   "1" "const" "56180" "Using where"
"48"    "SUBQUERY"  "p" "ref"   "Index 3"   "Index 3"   "1" "const" "47262" "Using where"
"47"    "SUBQUERY"  "i" "const" "PRIMARY"   "PRIMARY"   "4" "const" "1" \N
"55"    "UNION" \N  \N  \N  \N  \N  \N  \N  "No tables used"
"63"    "SUBQUERY"  \N  \N  \N  \N  \N  \N  \N  "Impossible WHERE noticed after reading const tables"
"62"    "SUBQUERY"  "p" "ref"   "Index 3"   "Index 3"   "1" "const" "69552" "Using where"
"61"    "SUBQUERY"  "p" "ref"   "Index 3"   "Index 3"   "1" "const" "100487"    "Using where"
"60"    "SUBQUERY"  "p" "ref"   "Index 3"   "Index 3"   "1" "const" "78156" "Using where"
"59"    "SUBQUERY"  "p" "ref"   "Index 3"   "Index 3"   "1" "const" "105721"    "Using where"
"58"    "SUBQUERY"  "p" "ref"   "Index 3"   "Index 3"   "1" "const" "128156"    "Using where"
"57"    "SUBQUERY"  "p" "ref"   "Index 3"   "Index 3"   "1" "const" "114533"    "Using where"
"56"    "SUBQUERY"  "i" "const" "PRIMARY"   "PRIMARY"   "4" "const" "1" ...

最佳答案

FROM_UNIXTIME(p.system_datetime/1000000)
      BETWEEN \'', start_date,'\' AND \'', end_date,'\'

不要隐藏函数中的列；不能使用索引。相反:

p.system_datetime
      BETWEEN 1000000*UNIX_TIMESTAMP(\'', start_date,'\')
          AND 1000000*UNIX_TIMESTAMP(\'', end_date,'\')

使用 DECIMAL(20,0) 而不是 DECIMAL(20,6) 从而避免 1000000 的原因是什么？另请注意，新的 MySQL 具有 DATETIME(6) 和 TIMESTAMP(6)，可为您提供所需的微秒分辨率。

重新编写查询后，p 将需要它来提高效率:

INDEX(instrument_id, system_datetime)

此外，如果这些表具有相同的结构，那么可能最好有一个表，而不是很多。它可能需要一个额外的列来区分每行属于哪个“表”。您似乎正在执行“通过表名称分区”(prices_all_2016_06_13_5 等)。

此外，切换到 InnoDB，并设计“正确的”主键(可能是“复合”)，将因“集群”而给您带来另一次性能提升。

分区可能值得也可能不值得。 (它通常买不到性能。)

我需要更好地了解您的疑问，以便提供进一步建议。

关于MySQL优化存储过程中慢UNION查询，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/37832391/

文章推荐： MySQL 指定天数的平均温度

文章推荐： sql - 使用 split_part 后替换空字段中的值

文章推荐： postgresql - Hibernate Postgresql select for update with outer join 问题

Mysql 查询 JOIN 查询
我有三张 table 。表 A 有选项名称(即颜色、尺寸)。表 B 有选项值名称(即蓝色、红色、黑色等)。表C通过将选项名称id和选项名称值id放在一起来建立关系。我的查询需要显示值和选项的名称，而
查询
在mysql中，如何计算一行中的非空单元格？我只想计算某些列之间的单元格，比如第 3-10 列之间的单元格。不是所有的列...同样，仅在该行中。最佳答案如果你想这样做，只能在 sql 中使用名称而
sql - 查询、 native 查询、命名查询和类型化查询之间的区别
关闭。这个问题需要多问focused 。目前不接受答案。想要改进此问题吗？更新问题，使其仅关注一个问题 editing this post . 已关闭 7 年前。 Improve this ques
elasticsearch - 在Elasticsearch查询中没有为[查询]注册的[查询]
我正在为版本7.6进行Elasticsearch查询我的查询是这样的: { "query": { "bool": { "should": [ {
sql - 查询、 native 查询、命名查询和类型化查询之间的区别
关闭。这个问题需要多问focused 。目前不接受答案。想要改进此问题吗？更新问题，使其仅关注一个问题 editing this post . 已关闭 7 年前。 Improve this ques
php - Mysql WHERE NOT EXISTS(查询)OR(查询)
是否可以编写一个查询来检查任一子查询(而不是一个子查询)是否正确？ SELECT * FROM employees e WHERE NOT EXISTS (
javascript - 查询。为表中的每一行发送 ajax 查询
我找到了很多关于我的问题的答案，但问题没有解决我有表格，有数据，例如: Data 1 Data 2 Data 3
salesforce - SOQL 查询 - 如何通过将字段设为小写并进行比较来编写 SOQL 查询？
以下查询返回错误: 查询: SELECT Id, FirstName, LastName, OwnerId, PersonEmail FROM Account WHERE lower(PersonEm
salesforce - SOQL 查询 - 如何通过将字段设为小写并进行比较来编写 SOQL 查询？
以下查询返回错误: 查询: SELECT Id, FirstName, LastName, OwnerId, PersonEmail FROM Account WHERE lower(PersonEm
Android SQLite 查询(我想解析一般的 SQL 查询)
我从 EditText 中获取了 String 值。以及提交查询的按钮。 String sql=editQuery.getText().toString();// SELECT * FROM empl
mysql 查询 - 为一个巨大的表优化现有的 MAX-MIN 查询
我有一个或多或少有效的查询(关于结果)，但处理大约需要 45 秒。这对于在 GUI 中呈现数据来说肯定太长了。所以我的需求是找到一个更快/更高效的查询(几毫秒左右会很好)我的数据表大约有 3000
SQL 查询 - 将 NULL 结果添加到 SELECT 查询
这是我第一次使用 Stack Overflow，所以我希望我以正确的方式提出这个问题。我有 2 个 SQL 查询，我正在尝试比较和识别缺失值，尽管我无法将 NULL 字段添加到第二个查询中以识别缺失
sql - 什么是动态 SQL 查询？何时需要使用动态 SQL 查询？
什么是动态 SQL 查询？何时需要使用动态 SQL 查询？我使用的是 SQL Server 2005。最佳答案这里有几篇文章: Introduction to Dynamic SQL Dynami
php - 在另一个 mysql 查询 while 循环中调用 mysql 查询
include "mysql.php"; $query= "SELECT ID,name,displayname,established,summary,searchlink,im
java - MySQL 查询 "select top 5"查询
我有一个查询要“转换”为 mysql。这是查询: select top 5 * from (select id, firstName, lastName, sum(fileSize) as To
c# - Entity Framework 查询 ToString 不会产生 SQL 查询
通过我的研究，我发现至少从 EF 4.1 开始，EF 查询上的 .ToString() 方法将返回要运行的 SQL。事实上，这对我来说非常有用，使用 Entity Framework 5 和 6。但
MySQL 查询(或 Doctrine 1.2 查询)- 从连接表和过滤器中获取最新项目
我在构造查询来执行以下操作时遇到问题: 按activity_type_id过滤联系人，仅显示最近事件具有所需activity_type_id或为NULL(无事件)的联系人表格结构如下: 一个联系人可
php - 如何在执行另一个 SQL 查询 x 分钟后执行一个 SQL 查询？
如何让我输入数据库的信息在输入数据 5 分钟后自行更新？假设我有一张 table : +--+--+-----+ |id|ip|count| +--+--+-----+ |
database - 如何在 N1QL 查询(Couchbase 查询)中使用 LENGTH() 字符串函数
我正在尝试搜索正好是 4 位数字的 ID，我知道我需要使用 LENGTH() 字符串函数，但找不到如何使用它的示例。我正在尝试以下(和其他变体)但它们不起作用。 SELECT max(car_id)
php - 将 SQL 查询 (+JOIN) 转换为 Symfony Propel 查询
我有一个在 mysql 上运行良好的 sql 查询(查询 + 连接): select sum(pa.price) from user u , purchase pu , pack pa where (

行者123

个人简介

我是一名优秀的程序员,十分优秀！

作者热门文章

滴滴打车优惠券免费领取

全站热门文章

首页

博学

6Ren·AI

商城

MySQL优化存储过程中慢UNION查询