c# - 使用 Alea GPU 加速嵌套循环和按位运算

Question

我正在尝试使用 Alea 来加速我正在处理的程序，但我需要一些帮助。

我需要做的是对存储在两个数组中的值进行大量的位计数和按位运算。

对于第一个数组的每个元素，我必须对第二个数组的每个元素执行按位 & 运算，然后计算 & 结果中设置为 1 的位。

如果结果大于/等于某个值，我需要退出内部 for 并转到我的第一个数组的下一个元素。

第一个数组通常很大，有数百万个元素，第二个数组通常少于 200.000 个元素。

尝试并行执行所有这些操作，这是我的代码:

[GpuManaged]
private long[] Check(long[] arr1, long[] arr2, int limit)
{
    Gpu.FreeAllImplicitMemory(true);
    var gpu = Gpu.Default;
    long[] result = new long[arr1.Length];
    gpu.For(0, arr1.Length, i =>
    {
        bool found = false;
        long b = arr1[i];
        for (int i2 = 0; i2 < arr2.Length; i2++)
        {
            if (LibDevice.__nv_popcll(b & arr2[i2]) >= limit)
            {
                found = true;
                break;
            }
        }
        if (!found)
        {
            result[i] = b;
        }
    });
    return result;
}

这按预期工作，但只比我在四核 CPU 上并行运行的版本快一点。

我肯定在这里遗漏了一些东西，这是我第一次尝试编写 GPU 代码。

顺便说一下，我的 NVIDIA 是 GeForce GT 740M。

编辑

以下代码比前一个代码快 2 倍，至少在我的 PC 上是这样。非常感谢 Michael Randall 为我指明了正确的方向。

private static int[] CheckWithKernel(Gpu gpu, int[] arr1, int[] arr2, int limit)
{
    var lp = new LaunchParam(16, 256);
    var result = new int[arr1.Length];
    try
    {
        using (var dArr1 = gpu.AllocateDevice(arr1))
        using (var dArr2 = gpu.AllocateDevice(arr2))
        using (var dResult = gpu.AllocateDevice<int>(arr1.Length))
        {
            gpu.Launch(Kernel, lp, arr1.Length, arr2.Length, dArr1.Ptr, dArr2.Ptr, dResult.Ptr, limit);
            Gpu.Copy(dResult, result);
            return result;
        }
    }
    finally
    {
        Gpu.Free(arr1);
        Gpu.Free(arr2);
        Gpu.Free(result);
    }
}

private static void Kernel(int a1, int a2, deviceptr<int> arr1, deviceptr<int> arr2, deviceptr<int> arr3, int limit)
{
    var iinit = blockIdx.x * blockDim.x + threadIdx.x;
    var istep = gridDim.x * blockDim.x;
    for (var i = iinit; i < a1; i += istep)
    {
        bool found = false;
        int b = arr1[i];
        for (var j = 0; j < a2; j++)
        {
            if (LibDevice.__nv_popcll(b & arr2[j]) >= limit)
            {
                found = true;
                break;
            }
        }
        if (!found)
        {
            arr3[i] = b;
        }
    }
}

Answer 1

更新

固定似乎不适用于 GCHandle.Alloc()

但是这个答案的重点是您将从直接内存访问中获得更大的性能增益。

http://www.aleagpu.com/release/3_0_3/doc/advanced_features_csharp.html

直接使用设备内存

Device memory provides even more flexibility as it also allows all kind of pointer arithmetics. Device memory is allocated with

Memory<T> Gpu.AllocateDevice<T>(int length)
Memory<T> Gpu.AllocateDevice<T>(T[] array)

The first overload creates a device memory object for the specified type T and length on the selected GPU. The second one allocates storage on the GPU and copies the .NET array into it. Both return a Memory<T> object, which implements IDisposable and can therefore support the using syntax which ensures proper disposal once the Memory<T> object goes out of scope. A Memory<T> object has properties to determine the length, the GPU or the device on which it lives. The Memory<T>.Ptr property returns a deviceptr<T>, which can be used in GPU code to access the actual data or to perform pointer arithmetics. The following example illustrates a simple use case of device pointers. The kernel only operates on part of the data, defined by an offset.

using (var dArg1 = gpu.AllocateDevice(arg1))
using (var dArg2 = gpu.AllocateDevice(arg2))
using (var dOutput = gpu.AllocateDevice<int>(Length/2))
{           
    // pointer arithmetics to access subset of data
    gpu.Launch(Kernel, lp, dOutput.Length, dOutput.Ptr, dArg1.Ptr + Length/2, dArg2.Ptr + Length / 2);

    var result = dOutput.ToArray();

    var expected = arg1.Skip(Length/2).Zip(arg2.Skip(Length/2), (x, y) => x + y);

    Assert.That(result, Is.EqualTo(expected));
}

原始答案

忽略正在进行的逻辑，或者这与 GPU 代码的相关性。但是，您可以通过使用 GCHandle.Alloc() 在内存中固定您的数组 来赞美您的并行 例程并可能加快速度。和 GCHandleType.Pinned 标记并使用直接指针 访问(如果您可以运行 unsafe 代码)

Notes

• You will cop a hit from pinning the memory, however for large arrays you can realize a lot of performance from direct access*

• You will have to mark your assembly unsafe in Build Properties*

• This is obviously untested and just an example*

• You could used fixed, however the Parallel Lambda makes it fiddlier

示例

private unsafe long[] Check(long[] arr1, long[] arr2, int limit)
{   
   Gpu.FreeAllImplicitMemory(true);
   var gpu = Gpu.Default;    
   var result = new long[arr1.Length];

   // Create some pinned memory
   var resultHandle = GCHandle.Alloc(result, GCHandleType.Pinned);
   var arr2Handle = GCHandle.Alloc(result, GCHandleType.Pinned);
   var arr1Handle = GCHandle.Alloc(result, GCHandleType.Pinned);

   // Get the addresses
   var resultPtr = (int*)resultHandle.AddrOfPinnedObject().ToPointer();
   var arr2Ptr = (int*)arr2Handle.AddrOfPinnedObject().ToPointer();
   var arr1Ptr = (int*)arr2Handle.AddrOfPinnedObject().ToPointer();

   // I hate nasty lambda statements. I always find local methods easier to read.    
   void Workload(int i)
   {
      var found = false;    
      var b = *(arr1Ptr + i);

      for (var j = 0; j < arr2.Length; j++)
      {
         if (LibDevice.__nv_popcll(b & *(arr2Ptr + j)) >= limit)
         {
            found = true;
            break;
         }
      }

      if (!found)
      {
         *(resultPtr + i) = b;
      }
   }

   try
   {
      gpu.For(0, arr1.Length, i => Workload(i));
   }
   finally 
   {
      // Make sure we free resources
      arr1Handle.Free();
      arr2Handle.Free();
      resultHandle.Free();
   } 
   return result;    
}

---

其他资源

GCHandle.Alloc Method (Object)

A new GCHandle that protects the object from garbage collection. This GCHandle must be released with Free when it is no longer needed.

GCHandleType Enumeration

Pinned : This handle type is similar to Normal, but allows the address of the pinned object to be taken. This prevents the garbage collector from moving the object and hence undermines the efficiency of the garbage collector. Use the Free method to free the allocated handle as soon as possible.

Unsafe Code and Pointers (C# Programming Guide)

In the common language runtime (CLR), unsafe code is referred to as unverifiable code. Unsafe code in C# is not necessarily dangerous; it is just code whose safety cannot be verified by the CLR. The CLR will therefore only execute unsafe code if it is in a fully trusted assembly. If you use unsafe code, it is your responsibility to ensure that your code does not introduce security risks or pointer errors.

• 请注意，此后有更新:

http://www.aleagpu.com/release/3_0_3/doc/advanced_features_csharp.html

现在是这样的:

http://www.aleagpu.com/release/3_0_4/doc/advanced_features_csharp.html

一些示例和信息在 3.0.4 版中已更改或移动。