C# ReadOnlySpan 与用于字符串剖析的子字符串

Question

我有一个相当简单的字符串扩展方法，该方法在我拥有的系统中被频繁调用，该方法正在执行大量字符串操作。我读了这篇文章 ( String.Substring() seems to bottleneck this code ) 并想我会尝试相同的方法，看看我是否可以通过改变我读取字符串的方式来获得一些性能。我的结果并不完全符合我的预期(我期待 ReadOnlySpan 提供显着的性能提升)，我想知道为什么会这样。在实际运行的生产代码中，我发现性能有非常轻微的损失。

我生成了一个包含约 115 万行字符串的文件，其中包含我关心的字符，对每个字符串调用该方法，并将结果转储到控制台。

我的结果(以毫秒为单位的运行时间)是:

ReadOnlySpan.IndexOf Framework 4.7.1: 68538
ReadOnlySpan.IndexOf Core 2.1: 64486

ReadOnlySpan.SequenceEqual Framework 4.7.1: 63650
ReadOnlySpan.SequenceEqual Core 2.1: 65071

substring Framework 4.7.1: 63508
substring Core 2.1: 64125

代码(从 Full Framework 到 Core 2.1 完全相同):

调用代码:

static void Main(string[] args)
{
    Stopwatch sw = new Stopwatch();
    sw.Start();

    var f = File.ReadAllLines("periods.CSV");

    foreach (string s in f)
    { Console.WriteLine(s.CountOccurrences(".")); }

    sw.Stop();
    Console.WriteLine("Done in " + sw.ElapsedMilliseconds + " ms");
    Console.ReadKey();
}

我的方法的原始子字符串形式:

public static int CountOccurrencesSub(this string val, string searchFor)
{
    if (string.IsNullOrEmpty(val) || string.IsNullOrEmpty(searchFor))
    { return 0; }

    int count = 0;

    for (int x = 0; x <= val.Length - searchFor.Length; x++)
    {
        if (val.Substring(x, searchFor.Length) == searchFor)
        { count++; }
    }

    return count;
}

ReadOnlySpan 版本(我已经使用 IndexOf 和 SequenceEqual 进行了相等性检查测试):

public static int CountOccurrences(this string val, string searchFor)
{
    if (string.IsNullOrEmpty(val) || string.IsNullOrEmpty(searchFor))
    { return 0; }

    int count = 0;

    ReadOnlySpan<char> vSpan = val.AsSpan();
    ReadOnlySpan<char> searchSpan = searchFor.AsSpan();

    for (int x = 0; x <= vSpan.Length - searchSpan.Length; x++)
    {
        if (vSpan.Slice(x, searchSpan.Length).SequenceEqual(searchSpan))
        { count++; }
    }

    return count;
}

相等比较是否在我调用的方法中进行了分配，因此没有提升？这对 ReadOnlySpan 来说不是一个好的应用程序吗？我只是老了，错过了什么吗？

Answer 1

虽然我来晚了一点，但我想我仍然可以为这个主题添加相关信息。

首先，说一下其他海报的尺寸。

OP 的结果显然不正确。正如评论中指出的那样，I/O 操作完全扭曲了统计数据。

已接受答案的海报在正确的轨道上。他的方法消除了缓慢的 I/O 操作，并明确关注基准测试的主题。但是，他没有提到使用的环境(尤其是 .NET 运行时)，而且他的“预热方法”也值得商榷。

绩效衡量是一项非常棘手的工作，很难做到正确。如果我想获得有效结果，我什至不会尝试自己编写代码。所以我决定使用广泛采用的 Benchmark.NET 检查这个问题图书馆。为了让这一切更有趣，我添加了第三个候选人。此实现使用 String.CompareOrdinal用于出现次数计数，我预计它会产生很好的结果。

基准

在测量开始之前(在全局设置阶段)，我生成了 1,000,000 行 lorem ipsum 文本。在整个测量过程中使用此数据。

每种方法都使用 1,000 行和 1,000,000 行以及较短(5 个字符长)和较长(39 个字符长)的搜索文本进行练习。

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;

namespace MyBenchmarks
{
#if NETCOREAPP2_1
    [CoreJob]
#else
    [ClrJob]
#endif
    [RankColumn, MarkdownExporterAttribute.StackOverflow]
    public class Benchmark
    {
        static readonly string[] words = new[]
        {
            "lorem", "ipsum", "dolor", "sit", "amet", "consectetuer",
            "adipiscing", "elit", "sed", "diam", "nonummy", "nibh", "euismod",
            "tincidunt", "ut", "laoreet", "dolore", "magna", "aliquam", "erat"
        };

        // borrowed from greg (https://stackoverflow.com/questions/4286487/is-there-any-lorem-ipsum-generator-in-c)
        static IEnumerable<string> LoremIpsum(Random random, int minWords, int maxWords, int minSentences, int maxSentences, int numLines)
        {
            var line = new StringBuilder();
            for (int l = 0; l < numLines; l++)
            {
                line.Clear();
                var numSentences = random.Next(maxSentences - minSentences) + minSentences + 1;
                for (int s = 0; s < numSentences; s++)
                {
                    var numWords = random.Next(maxWords - minWords) + minWords + 1;
                    line.Append(words[random.Next(words.Length)]);
                    for (int w = 1; w < numWords; w++)
                    {
                        line.Append(" ");
                        line.Append(words[random.Next(words.Length)]);
                    }
                    line.Append(". ");
                }
                yield return line.ToString();
            }
        }

        string[] lines;

        [Params(1000, 1_000_000)]
        public int N;

        [Params("lorem", "lorem ipsum dolor sit amet consectetuer")]
        public string SearchValue;

        [GlobalSetup]
        public void GlobalSetup()
        {
            lines = LoremIpsum(new Random(), 6, 8, 2, 3, 1_000_000).ToArray();
        }

        public static int CountOccurrencesSub(string val, string searchFor)
        {
            if (string.IsNullOrEmpty(val) || string.IsNullOrEmpty(searchFor))
            { return 0; }

            int count = 0;

            for (int x = 0; x <= val.Length - searchFor.Length; x++)
            {
                if (val.Substring(x, searchFor.Length) == searchFor)
                { count++; }
            }

            return count;
        }

        public static int CountOccurrences(string val, string searchFor)
        {
            if (string.IsNullOrEmpty(val) || string.IsNullOrEmpty(searchFor))
            { return 0; }

            int count = 0;

            ReadOnlySpan<char> vSpan = val.AsSpan();
            ReadOnlySpan<char> searchSpan = searchFor.AsSpan();

            for (int x = 0; x <= vSpan.Length - searchSpan.Length; x++)
            {
                if (vSpan.Slice(x, searchSpan.Length).SequenceEqual(searchSpan))
                { count++; }
            }

            return count;
        }

        public static int CountOccurrencesCmp(string val, string searchFor)
        {
            if (string.IsNullOrEmpty(val) || string.IsNullOrEmpty(searchFor))
            { return 0; }

            int count = 0;

            for (int x = 0; x <= val.Length - searchFor.Length; x++)
            {
                if (string.CompareOrdinal(val, x, searchFor, 0, searchFor.Length) == 0)
                { count++; }
            }

            return count;
        }


        [Benchmark(Baseline = true)]
        public int Substring()
        {
            int occurences = 0;
            for (var i = 0; i < N; i++)
                occurences += CountOccurrencesSub(lines[i], SearchValue);
            return occurences;
        }

        [Benchmark]
        public int Span()
        {
            int occurences = 0;
            for (var i = 0; i < N; i++)
                occurences += CountOccurrences(lines[i], SearchValue);
            return occurences;
        }

        [Benchmark]
        public int Compare()
        {
            int occurences = 0;
            for (var i = 0; i < N; i++)
                occurences += CountOccurrencesCmp(lines[i], SearchValue);
            return occurences;
        }
    }

    public class Program
    {
        public static void Main(string[] args)
        {
            BenchmarkRunner.Run<Benchmark>();
        }
    }
}

结果

NET 核心 2.1

BenchmarkDotNet=v0.11.0, OS=Windows 7 SP1 (6.1.7601.0)
Intel Core i3-4360 CPU 3.70GHz (Haswell), 1 CPU, 4 logical and 2 physical cores
Frequency=3604970 Hz, Resolution=277.3948 ns, Timer=TSC
.NET Core SDK=2.1.400
  [Host] : .NET Core 2.1.2 (CoreCLR 4.6.26628.05, CoreFX 4.6.26629.01), 64bit RyuJIT
  Core   : .NET Core 2.1.2 (CoreCLR 4.6.26628.05, CoreFX 4.6.26629.01), 64bit RyuJIT

Job=Core  Runtime=Core  

    Method |       N |          SearchValue |           Mean |           Error |          StdDev |         Median | Scaled | ScaledSD | Rank |
---------- |-------- |--------------------- |---------------:|----------------:|----------------:|---------------:|-------:|---------:|-----:|
 Substring |    1000 |                lorem |     2,149.4 us |       2.2763 us |       2.1293 us |     2,149.4 us |   1.00 |     0.00 |    3 |
      Span |    1000 |                lorem |       555.5 us |       0.2786 us |       0.2470 us |       555.5 us |   0.26 |     0.00 |    1 |
   Compare |    1000 |                lorem |     1,471.8 us |       0.2133 us |       0.1891 us |     1,471.8 us |   0.68 |     0.00 |    2 |
           |         |                      |                |                 |                 |                |        |          |      |
 Substring |    1000 | lorem(...)etuer [39] |     2,128.7 us |       1.0414 us |       0.9741 us |     2,128.6 us |   1.00 |     0.00 |    3 |
      Span |    1000 | lorem(...)etuer [39] |       388.9 us |       0.0440 us |       0.0412 us |       388.9 us |   0.18 |     0.00 |    1 |
   Compare |    1000 | lorem(...)etuer [39] |     1,215.6 us |       0.7016 us |       0.6220 us |     1,215.5 us |   0.57 |     0.00 |    2 |
           |         |                      |                |                 |                 |                |        |          |      |
 Substring | 1000000 |                lorem | 2,239,510.8 us | 241,887.0796 us | 214,426.5747 us | 2,176,083.7 us |   1.00 |     0.00 |    3 |
      Span | 1000000 |                lorem |   558,317.4 us |     447.3105 us |     418.4144 us |   558,338.9 us |   0.25 |     0.02 |    1 |
   Compare | 1000000 |                lorem | 1,471,941.2 us |     190.7533 us |     148.9276 us | 1,471,955.8 us |   0.66 |     0.05 |    2 |
           |         |                      |                |                 |                 |                |        |          |      |
 Substring | 1000000 | lorem(...)etuer [39] | 2,350,820.3 us |  46,974.4500 us | 115,229.1264 us | 2,327,187.2 us |   1.00 |     0.00 |    3 |
      Span | 1000000 | lorem(...)etuer [39] |   433,567.7 us |  14,445.7191 us |  42,593.5286 us |   417,333.4 us |   0.18 |     0.02 |    1 |
   Compare | 1000000 | lorem(...)etuer [39] | 1,299,065.2 us |  25,474.8504 us |  46,582.2045 us | 1,296,892.8 us |   0.55 |     0.03 |    2 |  

NET 框架 4.7.2

BenchmarkDotNet=v0.11.0, OS=Windows 7 SP1 (6.1.7601.0)
Intel Core i3-4360 CPU 3.70GHz (Haswell), 1 CPU, 4 logical and 2 physical cores
Frequency=3604960 Hz, Resolution=277.3956 ns, Timer=TSC
  [Host] : .NET Framework 4.7.2 (CLR 4.0.30319.42000), 64bit RyuJIT-v4.7.3062.0
  Clr    : .NET Framework 4.7.2 (CLR 4.0.30319.42000), 64bit RyuJIT-v4.7.3062.0

Job=Clr  Runtime=Clr  

    Method |       N |          SearchValue |           Mean |          Error |          StdDev |         Median | Scaled | ScaledSD | Rank |
---------- |-------- |--------------------- |---------------:|---------------:|----------------:|---------------:|-------:|---------:|-----:|
 Substring |    1000 |                lorem |     2,025.8 us |      2.4639 us |       1.9237 us |     2,025.4 us |   1.00 |     0.00 |    3 |
      Span |    1000 |                lorem |     1,216.6 us |      4.2994 us |       4.0217 us |     1,217.8 us |   0.60 |     0.00 |    1 |
   Compare |    1000 |                lorem |     1,295.5 us |      5.2427 us |       4.6475 us |     1,293.1 us |   0.64 |     0.00 |    2 |
           |         |                      |                |                |                 |                |        |          |      |
 Substring |    1000 | lorem(...)etuer [39] |     1,939.5 us |      0.4428 us |       0.4142 us |     1,939.3 us |   1.00 |     0.00 |    3 |
      Span |    1000 | lorem(...)etuer [39] |       944.9 us |      2.6648 us |       2.3622 us |       944.7 us |   0.49 |     0.00 |    1 |
   Compare |    1000 | lorem(...)etuer [39] |     1,002.0 us |      0.2475 us |       0.2067 us |     1,002.1 us |   0.52 |     0.00 |    2 |
           |         |                      |                |                |                 |                |        |          |      |
 Substring | 1000000 |                lorem | 2,065,805.7 us |  2,009.2139 us |   1,568.6619 us | 2,065,555.1 us |   1.00 |     0.00 |    3 |
      Span | 1000000 |                lorem | 1,209,976.4 us |  6,238.6091 us |   5,835.5982 us | 1,206,554.3 us |   0.59 |     0.00 |    1 |
   Compare | 1000000 |                lorem | 1,303,321.8 us |  1,257.7418 us |   1,114.9552 us | 1,303,330.1 us |   0.63 |     0.00 |    2 |
           |         |                      |                |                |                 |                |        |          |      |
 Substring | 1000000 | lorem(...)etuer [39] | 2,085,652.9 us | 62,651.7471 us | 168,309.8501 us | 1,973,522.2 us |   1.00 |     0.00 |    3 |
      Span | 1000000 | lorem(...)etuer [39] |   958,421.2 us |  3,703.5508 us |   3,464.3034 us |   958,324.9 us |   0.46 |     0.03 |    1 |
   Compare | 1000000 | lorem(...)etuer [39] | 1,007,936.8 us |    802.1730 us |     750.3531 us | 1,007,680.3 us |   0.49 |     0.04 |    2 |

结论

很明显，使用 Span 可以显着提高性能。有点令人惊讶的是，它在 .NET Core 上是 4-5 倍，而在 .NET Framework 上仅为 2 倍。这背后可能有什么原因？有人知道吗？

String.CompareOrdinal 的表现也相当不错。 我期待更好的结果，因为理论上它只是逐字节比较相同，但一点也不差。在 .NET Framework 上，它无论如何都是一个可行的选择。

搜索字符串的长度(当然除了四肢)似乎对结果没有太大影响。