go - 优化网络代码

Question

我有一个 Go 服务器，它从大量流入数据的 TCP 客户端获取输入。该格式是自定义格式，结束分隔符可能出现在字节流中，因此它使用字节填充来解决此问题。

我正在我的代码中寻找热点，这引发了一个巨大的热点，我确信它可以变得更有效率，但我不太确定目前提供的 Go 函数如何。

代码如下，pprof 显示热点是 popPacketFromBuffer 命令。这会查看当前缓冲区，在接收到每个字节后自行查找 endDelimiter。如果连续有 2 个，则它在数据包本身内。

我确实看过使用 ReadBytes() 而不是 ReadByte() 但看起来我需要指定一个分隔符，我担心这会中断数据包中流？而且在任何情况下，这会比我现在所做的更有效率吗？

在 popPacketFromBuffer 函数中，for 循环是热点。

有什么想法吗？

// Read client data from channel
func (c *Client) listen() {

    reader := bufio.NewReader(c.conn)

    clientBuffer := new(bytes.Buffer)

    for {
        c.conn.SetDeadline(time.Now().Add(c.timeoutDuration))

        byte, err := reader.ReadByte()

        if err != nil {
            c.conn.Close()
            c.server.onClientConnectionClosed(c, err)
            return
        }

        wrErr := clientBuffer.WriteByte(byte)
        if wrErr != nil {
            log.Println("Write Error:", wrErr)
        }

        packet := popPacketFromBuffer(clientBuffer)
        if packet != nil {
            c.receiveMutex.Lock()
            packetSize := uint64(len(packet))
            c.bytesReceived += packetSize
            c.receiveMutex.Unlock()

            packetBuffer := bytes.NewBuffer(packet)

            b, err := uncompress(packetBuffer.Bytes())
            if err != nil {
                log.Println("Unzip Error:", err)
            } else {
                c.server.onNewMessage(c, b)
            }
        }

    }
}

func popPacketFromBuffer(buffer *bytes.Buffer) []byte {

    bufferLength := buffer.Len()

    if bufferLength >= 125000 { // 1MB in bytes is roughly this
        log.Println("Buffer is too large ", bufferLength)
        buffer.Reset()
        return nil
    }

    tempBuffer := buffer.Bytes()
    length := len(tempBuffer)

    // Return on zero length buffer submission
    if length == 0 {
        return nil
    }

    endOfPacket := -1

    // Determine the endOfPacket position by looking for an instance of our delimiter
    for i := 0; i < length-1; i++ {
        if tempBuffer[i] == endDelimiter {
            if tempBuffer[i+1] == endDelimiter {
                i++
            } else {
                // We found a single delimiter, so consider this the end of a packet
                endOfPacket = i - 2
                break
            }
        }
    }

    if endOfPacket != -1 {
        // Grab the contents of the provided packet
        extractedPacket := buffer.Bytes()

        // Extract the last byte as we were super greedy with the read operation to check for stuffing
        carryByte := extractedPacket[len(extractedPacket)-1]

        // Clear the main buffer now we have extracted a packet from it
        buffer.Reset()

        // Add the carryByte over to our new buffer
        buffer.WriteByte(carryByte)

        // Ensure packet begins with a valid startDelimiter
        if extractedPacket[0] != startDelimiter {
            log.Println("Popped a packet without a valid start delimiter")
            return nil
        }

        // Remove the start and end caps
        slice := extractedPacket[1 : len(extractedPacket)-2]

        return deStuffPacket(slice)
    }

    return nil
}

Answer 1

看起来每次从连接接收到每个字节时，您都会调用 popPacketFromBuffer()。然而，popPacketFromBuffer() 每次复制空洞缓冲区并检查每个字节的分隔符。也许这是压倒性的。对我来说你不需要循环

for i := 0; i < length-1; i++ {
        if tempBuffer[i] == endDelimiter {
            if tempBuffer[i+1] == endDelimiter {
                i++
            } else {
                // We found a single delimiter, so consider this the end of a packet
                endOfPacket = i - 2
                break
            }
        }
    }

在 popPacketFromBuffer() 中，也许只是测试最后两个字节而不是循环

if (buffer[len(buffer)-2] == endDelimiter) && (buffer[len(buffer)-1] != endDelimiter){
    //It's a packet
}

就足够了。