sql-server - SQL Server 中增量批量删除和插入操作非常慢

Question

我有以下存储过程，用于在表中删除和插入行。运行速度很慢。

我已阅读各种提案并已实现:

• 批量删除行并使用派生表
• 禁用 FK
• where 子句中字段的索引。

所以应该采取的方式是:

1. 表包含大约 1000 万条记录。

2. 每天我需要刷新大约 15-30% 的内容。我就是用这个SP来做的。

来源:

CREATE PROCEDURE [dbo].[spIncrementalUpdate]
    -- Add the parameters for the stored procedure here
    @table_inc nvarchar(30),
    @table_target nvarchar(30),
    @table_date nvarchar(30),
    @field1 nvarchar(10),
    @field2 nvarchar(10)

AS
BEGIN
    -- SET NOCOUNT ON added to prevent extra result sets from
    -- interfering with SELECT statements.
    SET NOCOUNT ON;
    DECLARE @logmessage as nvarchar(2048)
    DECLARE @logdatacode as int
    DECLARE @cmd as nvarchar(max)
    DECLARE @fromjulian nvarchar(10)
    DECLARE @datadate datetime
    DECLARE @datadate_str nvarchar(10)
    DECLARE @rows int
    DECLARE @ParmDefinition nvarchar(500)

    select @fromjulian = '114000'
    print 'Using ' + @fromjulian
    --
    -- GET THE ROW COUNT OF THE INC TABLE.
    --
    IF @field2 = ''
        BEGIN
            SET @cmd = N'SELECT @retval = count(*) from ' + 
                @table_inc + ' where ' + @field1 +' > ' + @fromjulian
        END
    ELSE
        BEGIN
            SET @cmd = N'SELECT @retval = count(*) from ' + 
                @table_inc + ' where ' + @field1 +' > ' + @fromjulian + 
                ' OR ' + @field2 +' > ' + @fromjulian
        END
    SET @ParmDefinition = N'@retval int OUTPUT'
    EXEC sp_executesql @cmd, @ParmDefinition, @retval = @rows OUTPUT
    --
    if @rows <> 0
    BEGIN
        SET @logmessage = @table_inc + ' has ' + 
            cast(@rows as nvarchar(10)) + 
            ' rows after ' + @fromjulian + ', deleting'
        SET @logdatacode = 1000
        --
        -- Delete the records from original table based on @fromjulian
        --
        IF @field2 = ''
            BEGIN
                SET @cmd = N'delete ' + @table_target + 
                    ' from (select top(50000) * from ' + @table_target  +
                     ' where ' + @field1 + ' > ' + @fromjulian + 
                     ' order by ' + @field1 + ') ' + 
                     @table_target
                print @cmd
            END
        ELSE
            BEGIN
                SET @cmd = N'delete ' + @table_target + 
                    ' from (select top(50000) * from ' + 
                    @table_target  + 
                    ' where ' + @field1 + ' > ' + @fromjulian +  
                    ' OR ' + @field2 +' > ' + @fromjulian + 
                    ' order by ' + @field1 + ',' + @field2 + ') ' + 
                    @table_target
                print @cmd
            END 
        SELECT 1
        WHILE @@ROWCOUNT <> 0
        BEGIN
            EXEC sp_executesql @cmd
        END
        --
        -- Inserting the records to target from INC table
        --
        IF @field2 = ''
            BEGIN
                SET @cmd = N'insert into ' + @table_target + 
                ' select * from ' + @table_inc + 
                ' where ' + @field1 +' > ' + cast(@fromjulian as nvarchar(10))
                print @cmd
            END
        ELSE
            BEGIN
                SET @cmd = N'insert into ' + @table_target + 
                ' select * from ' + @table_inc + 
                ' where ' + @field1 +' > ' + cast(@fromjulian as nvarchar(10)) + 
                ' OR ' + @field2 +' > ' + cast(@fromjulian as nvarchar(10))
                print @cmd
            END

        print @cmd
        EXEC sp_executesql @cmd
    END
    ELSE
        BEGIN
            SET @logmessage = 'NO ROWS IN ' + @table_inc + ' AFTER ' + cast(@fromjulian as nvarchar(10))
            SET @logdatacode = 1001
        END
    --
    -- LOG
    --
    INSERT INTO YLA_GROUP.[dbo].[sysssislog]
           ([event]
           ,[computer]
           ,[operator]
           ,[source]
           ,[sourceid]
           ,[executionid]
           ,[starttime]
           ,[endtime]
           ,[datacode]
           ,[databytes]
           ,[message])
     VALUES
           ('spIncrementalUpdate','','','',NEWID(),NEWID(),getdate(),getdate(),@logdatacode,null,@logmessage)
    print cast(@logdatacode as nvarchar(10)) + ' - ' + @logmessage
END

Answer 1

由于您要批量删除行，因此应该从子查询中删除 ORDER BY 子句，因为这是没有必要的。

这是脚本的摘录:

IF @field2 = ''
    BEGIN
        SET @cmd = N'delete ' + @table_target + 
                   ' from (select top(50000) * from ' + @table_target  +
                   ' where ' + @field1 + ' > ' + @fromjulian + ') ' +  @table_target
        print @cmd
    END
ELSE
    BEGIN
        SET @cmd = N'delete ' + @table_target + 
                   ' from (select top(50000) * from ' + @table_target  + 
                   ' where ' + @field1 + ' > ' + @fromjulian +  
                   ' OR ' + @field2 +' > ' + @fromjulian + ') ' + @table_target
        print @cmd
    END 

请记住，排序可能是一项昂贵的操作，如果删除(在本例中是正确的)索引，成本甚至会更高。

除此之外，您无能为力。还要考虑使用动态 SQL，执行计划不会缓存。因此，每次在 @cmd 中执行命令时，查询引擎都需要计算最佳执行计划。不幸的是，据我所知，您需要动态 SQL。