go - 如何实现S3宣传的5,500 HEAD rps？

Question

链接到s3通告的请求速率(每个前缀5500 rps):https://aws.amazon.com/about-aws/whats-new/2018/07/amazon-s3-announces-increased-request-rate-performance/

我的代码:

package main

import (
    "log"
    "math/rand"
    "net/http"
    "strconv"
    "time"

    "github.com/aws/aws-sdk-go/aws"
    "github.com/aws/aws-sdk-go/aws/session"
    "github.com/aws/aws-sdk-go/service/s3"
)

func main() {
    sess := session.Must(session.NewSession(&aws.Config{
        Region: aws.String("us-east-1"),
    }))
    client := s3.New(sess)
    for i := 0; i < 1000; i++ {
        go func() {
            r := rand.Intn(35000)
            start := time.Now()
            req, _ := client.HeadObjectRequest(&s3.HeadObjectInput{
                Key:    aws.String("one-prefix/" + strconv.Itoa(r)),
                Bucket: aws.String("MY-BUCKET"),
            })
            req.Sign()
            if _, err := http.DefaultClient.Do(req.HTTPRequest); err != nil {
                log.Println(err)
            }
            log.Println(r, "cost", time.Since(start))
        }()
    }
    <-time.After(10 * time.Second)
}

以m5.large(大约200 rps)完成这些goroutine的程序大约需要5 s，大约是广告5500 rps的一小部分。想知道我做错了什么以及如何使用s3产生更好的rp？

Answer 1

从您的时间安排中排除 session 设置，更重要的是，使用具有100Gb网络(或更多)的实例类型