java - Netty 4 替代 LineBasedFrameDecoder

Question

是否有更有效的方法来搜索 ByteBuf 上的行分隔符序列？特别是，我正在寻找一种方法来查找 \r\n 序列。

因为我想使用 bytebuf.forEachByte(ByteBufProcessor) 搜索 2 个字节 不起作用。

到目前为止我能找到的最简单的方法是使用LineBasedFrameDecoder。问题是我收到的一些消息可能很大(MB 范围)，就像 Norman Maurer 提到的 here处理这么大的消息时，循环遍历 ByteBuf 的效率非常低(可能需要几分钟才能找到分隔符)。

Answer 1

您仍然可以搜索两个字节:

public class CrLfProcessor implements ByteBufProcessor{
  private byte previousByte;

  @Override
  public boolean process(byte value) {
    if(previousByte == '\r'){
      if(value == '\n'){
        return false;
      }
    }

    previousByte = value;
    return true;
  }
}

<小时/>

这是测试各种优化的 JMH 基准测试:

Fork(1)
@State(Scope.Benchmark)
@OutputTimeUnit(TimeUnit.MILLISECONDS)
@Measurement(iterations = 10)
@Warmup(iterations = 10)
@BenchmarkMode(Mode.AverageTime)
public class ByteBufProcessorBenchmark {
  private static interface ByteProcessor {
    boolean process(byte value);
  }

  private static final int DATA_SIZE = 1024 * 1024;
  private byte[] data;

  @Setup(Level.Trial)
  public void setUp() {
    data = new byte[DATA_SIZE];
    Random random = new Random();
    random.nextBytes(data);
  }

  @Benchmark
  public void crFirst(Blackhole blackhole) {
    ByteProcessor byteProcessor = new ByteProcessor() {
      private byte previousByte;
      @Override
      public boolean process(byte value) {
        if(previousByte == '\r'){
          if(value == '\n'){
            return false;
          }
        }

        previousByte = value;
        return true;
      }
    };

    doProcess(byteProcessor, blackhole);
  }

  @Benchmark
  public void lfFirst(Blackhole blackhole) {
    ByteProcessor byteProcessor = new ByteProcessor() {
      private byte previousByte;
      @Override
      public boolean process(byte value) {
        if (value == '\n') {
          if(previousByte == '\r'){
            return false;
          }
        }

        previousByte = value;
        return true;
      }
    };

    doProcess(byteProcessor, blackhole);
  }

  @Benchmark
  public void crFirstUpdateCacheOnDemand(Blackhole blackhole) {
    ByteProcessor byteProcessor = new ByteProcessor() {
      private byte previousByte;
      @Override
      public boolean process(byte value) {
        if(previousByte == '\r'){
          if(value == '\n'){
            return false;
          }
          previousByte = 0;
        }else if(value == '\r'){
          previousByte = value;
        }
        return true;
      }
    };

    doProcess(byteProcessor, blackhole);
  }

  @Benchmark
  public void lfFirstUpdateCacheOnDemand(Blackhole blackhole) {
    ByteProcessor byteProcessor = new ByteProcessor() {
      private byte previousByte;
      @Override
      public boolean process(byte value) {
        if (value == '\n') {
          if(previousByte == '\r'){
            return false;
          }
          previousByte = 0;
        }else if(value == '\r'){
          previousByte = value;
        }
        return true;
      }
    };

    doProcess(byteProcessor, blackhole);
  }



@Benchmark
  public void consume(Blackhole blackhole){
    for(int i = 0; i < data.length; i++){
      blackhole.consume(data[i]);
    }
  }


  private void doProcess(ByteProcessor byteProcessor, Blackhole blackhole){
    for(int i = 0; i < data.length; i++){
      blackhole.consume(byteProcessor.process(data[i]));
    }
  }

}

结果如下:

# Run complete. Total time: 00:01:21

Benchmark                                             Mode  Cnt  Score   Error  Units
ByteBufProcessorBenchmark.crFirst                     avgt   10  4,211 ± 0,061  ms/op
ByteBufProcessorBenchmark.crFirstUpdateCacheOnDemand  avgt   10  4,285 ± 0,336  ms/op
ByteBufProcessorBenchmark.lfFirst                     avgt   10  4,375 ± 0,289  ms/op
ByteBufProcessorBenchmark.lfFirstUpdateCacheOnDemand  avgt   10  4,129 ± 0,075  ms/op
ByteBufProcessorBenchmark.consume                     avgt   10  3,126 ± 0,152  ms/op

正如您所看到的，最快的选项是 ByteBufProcessorBenchmark.lfFirstUpdateCacheOnDemand ，但与 ByteBufProcessorBenchmark.crFirst 的区别在于它并没有超过增加的复杂性。

还有你的性能要求是什么，因为每MB 4ms(包括你从结果中看到的黑洞需要3ms)恕我直言一点也不慢；最后你会得到每毫秒 1MB 的数据，这已经很不错了。