.net - 为什么 byte[] 的最大大小为 2 GB - 57 B？

Question

在我的 64 位计算机上，此 C# 代码可以运行:

new byte[2L * 1024 * 1024 * 1024 - 57]

但是这个抛出了 OutOfMemoryException :

new byte[2L * 1024 * 1024 * 1024 - 56]

为什么？

据我了解，托管对象的最大大小为 2 GB，并且我正在创建的数组对象包含的字节数超出了我想要的字节数。即，有 4 个字节(或 8？)用于同步块(synchronized block)编号，8 个字节用于方法表引用，4 个字节用于数组大小。包括填充在内，这是 24 个字节，那么为什么我不能分配一个 2G - 24 个字节的数组呢？最大大小真的正好 2 GB 吗？如果是这样的话，剩下的 2 GB 用来做什么？

注意:我实际上不需要分配一个 200 万字节的数组。即使我这样做了，56 字节的开销也可以忽略不计。我可以使用自定义 BigArray<T> 轻松解决该限制.

Answer 1

您需要 56 字节的开销。最大大小实际上是 2,147,483,649-1 减 56。这就是为什么您的 minus 57 有效，而 minus 56 无效。

如Jon Skeet这里说:

However, in practical terms, I don't believe any implementation supports such huge arrays. The CLR has a per-object limit a bit short of 2GB, so even a byte array can't actually have 2147483648 elements. A bit of experimentation shows that on my box, the largest array you can create is new byte[2147483591]. (That's on the 64 bit .NET CLR; the version of Mono I've got installed chokes on that.)

另请参阅 InformIT article关于同一主题。它提供了以下代码来演示最大大小和开销:

class Program
{
  static void Main(string[] args)
  {
    AllocateMaxSize<byte>();
    AllocateMaxSize<short>();
    AllocateMaxSize<int>();
    AllocateMaxSize<long>();
    AllocateMaxSize<object>();
  }

  const long twogigLimit = ((long)2 * 1024 * 1024 * 1024) - 1;
  static void AllocateMaxSize<T>()
  {
    int twogig = (int)twogigLimit;
    int num;
    Type tt = typeof(T);
    if (tt.IsValueType)
    {
      num = twogig / Marshal.SizeOf(typeof(T));
    }
    else
    {
      num = twogig / IntPtr.Size;
    }

    T[] buff;
    bool success = false;
    do
    {
      try
      {
        buff = new T[num];
        success = true;
      }
      catch (OutOfMemoryException)
      {
        --num;
      }
    } while (!success);
    Console.WriteLine("Maximum size of {0}[] is {1:N0} items.", typeof(T).ToString(), num);
  }
}

最后，文章是这样说的:

If you do the math, you’ll see that the overhead for allocating an array is 56 bytes. There are some bytes left over at the end due to object sizes. For example, 268,435,448 64-bit numbers occupy 2,147,483,584 bytes. Adding the 56 byte array overhead gives you 2,147,483,640, leaving you 7 bytes short of 2 gigabytes.

编辑:

但是等等，还有更多!

环顾四周并与 Jon Skeet 交谈，他向我指出了他在 Of memory and strings 上写的一篇文章。在那篇文章中，他提供了一个尺寸表:

Type            x86 size            x64 size
object          12                  24
object[]        16 + length * 4     32 + length * 8
int[]           12 + length * 4     28 + length * 4
byte[]          12 + length         24 + length
string          14 + length * 2     26 + length * 2

先生。斯基特接着说道:

You might be forgiven for looking at the numbers above and thinking that the "overhead" of an object is 12 bytes in x86 and 24 in x64... but that's not quite right.

还有这个:

• There's a "base" overhead of 8 bytes per object in x86 and 16 per object in x64... given that we can store an Int32 of "real" data in x86 and still have an object size of 12, and likewise we can store two Int32s of real data in x64 and still have an object of x64.

• There's a "minimum" size of 12 bytes and 24 bytes respectively. In other words, you can't have a type which is just the overhead. Note how the "Empty" class takes up the same size as creating instances of Object... there's effectively some spare room, because the CLR doesn't like operating on an object with no data. (Note that a struct with no fields takes up space too, even for local variables.)

• The x86 objects are padded to 4 byte boundaries; on x64 it's 8 bytes (just as before)

最后 Jon Skeet 回复了 question I asked of him在另一个问题中，他指出(回应我向他展示的 InformIT article):

It looks like the article you're referring to is inferring the overhead just from the limit, which is silly IMO.

因此，为了回答您的问题，根据我的收集，实际开销为 24 字节，其中有 32 字节空闲空间。