
<p>There are many open source libraries that implement parallel versions of the algorithms in the C++ standard template libraries. Inevitably we get asked questions about how ArrayFire compares to the other libraries out in the open.</p>
<p>In this post we are going to compare the performance of ArrayFire to that of BoostCompute, HSA-Bolt, Intel TBB and Thrust. The benchmarks include the following commonly used vector algorithms across 3 different architectures.</p>
<ul>
<li><a href="#reductions">Reductions</a></li>
<li><a href="#scan">Scan</a></li>
<li><a href="#transform">Transform</a></li>
</ul>
<p>The following setup has been used for the benchmarking purposes. The code to reproduce the benchmarks is linked at the bottom of the post.</p>
<p><strong>Hardware:</strong></p>
<ul>
<li>NVIDIA Tesla K20</li>
<li>AMD FirePro S10000</li>
<li>Intel Xeon E5-2560v2</li>
</ul>
<h3 id="background">Background</h3>
<p><strong>ArrayFire</strong></p>
<p>ArrayFire provides high performance primitives across a <a href="http://www.sandbox.arrayfire.com/docs/group__func__categories.htm">wide spectrum of domains</a>. Objects of the base class, <a href="http://www.sandbox.arrayfire.com/docs/classaf_1_1array.htm">array</a>, can support up to 4 dimensions. It enables users to take advantage of parallelism at multiple levels by <a href="http://sandbox.arrayfire.com/how-to-write-vectorized-code/">writing vectorized code</a>.</p>
<p>ArrayFire supports NVIDIA GPUs using CUDA and OpenCL, AMD GPUs using OpenCL and Intel / AMD CPUs in serial and parallel (using OpenCL) modes. For this post, ArrayFire 3.0 has been used. You can download ArrayFire from the following locations.</p>
<ul>
<li><a href="http://sandbox.arrayfire.com/download/">Installers</a></li>
<li><a href="https://github.com/arrayfire/arrayfire">Build from source</a></li>
</ul>
<p><strong>BoostCompute</strong></p>
<p>Boost Compute is C++ template library for parallel computing based on OpenCL. Boost Compute provides accelerated versions of the algorithms present in the standard template library. It also provides an alternative C++ wrapper over common OpenCL C routines.</p>
<p>Boost Compute supports NVIDIA GPUs, AMD GPUs and Intel / AMD CPUs using OpenCL. You can download BoostCompute from the following locations.</p>
<ul>
<li><a href="https://github.com/boostorg/compute">Header only library</a></li>
</ul>
<p>ArrayFire uses BoostCompute for sorting, set operations and finding regions in image in our OpenCL backend.</p>
<p><strong>Hsa-Bolt</strong></p>
<p>Bolt is a C++ template library optimized for heterogenous computing. It supports routines built on top of OpenCL, C++ AMP, and TBB. It provides high performance implementations of common algorithms from the Standard Template Library.</p>
<p>You can download the source to Hsa-Bolt from the following locations.</p>
<ul>
<li><a href="https://github.com/HSA-Libraries/Bolt">Build from Source</a></li>
</ul>
<p><strong>Thrust</strong></p>
<p>Thrust is a parallel algorithms library providing high level C++ interafce on top of CUDA, OpenMP and TBB. It resembles C++ standard template library and provides accelerated versions of the commonly used algorithms.</p>
<p>Thrust supports NVIDIA GPUs using CUDA and Intel / AMD GPUs using OpenMP and TBB. You can download thrust from the following locations.</p>
<ul>
<li><a href="https://developer.nvidia.com/cuda-downloads">From CUDA 7.0</a></li>
<li><a href="https://github.com/thrust/thrust">Header only library</a></li>
</ul>
<p>ArrayFire uses BoostCompute for sorting, set operations and finding regions in image in our CUDA backend.</p>
<p><strong>Intel TBB</strong></p>
<p>Threading Building Blocks (TBB) is a C++ template library developed by intel for multi-core processors. It provides platform independent data structures and algorithms that decrease the complexity of working with multiple threads.</p>
<p>TBB supports multi-core processors (including the Xeon Phi). You can download TBB from the following locations.</p>
<ul>
<li><a href="https://www.threadingbuildingblocks.org/download">Source and binaries</a></li>
</ul>
<h3 id="reductions">Reductions</h3>
<p>The reduction algorithm reduces the vector into a single element by successively reducing two input elements to a single output element. The most commonly used operations are addition, minimum and maximum.</p>
<p>It is one of the most commonly used algorithms in scientific and engineering algorithms. It is most commonly used in statistics and machine learning.</p>
<iframe width="540" height="360" seamless frameborder="0" scrolling="no" src="https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1374058645&amp;format=interactive">
</iframe>
<ul>
<li>For sizes less than 10 million elements, ArrayFire is the fastest followed by HSA-Bolt and BoostCompute</li>
<li>For sizes approaching 100 million elements, HSA-Bolt is 10% faster ArrayFire, and 15% faster than BoostCompute</li>
</ul>
<iframe width="540" height="360" seamless frameborder="0" scrolling="no" src="https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1835396253&amp;format=interactive">
</iframe>
<ul>
<li>ArrayFire consistently performs better than other libraries for sizes less than 10 million elements</li>
<li>ArrayFire OpenCL performs better than ArrayFire CUDA for sizes greater than 1 million elements.</li>
<li>BoostCompute consistently performs better than Thrust</li>
<li>For sizes approaching 100 million elements, ArrayFire OpenCL is fastest followed by BoostCompute, Thrust and ArrayFire CUDA</li>
</ul>
<iframe width="540" height="360" seamless frameborder="0" scrolling="no" src="https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=492096912&amp;format=interactive">
</iframe>
<ul>
<li>Intel TBB is significantly faster than ArrayFire and BoostCompute for sizes less than 1 million elements</li>
<li>ArrayFire is about 20% to 30% slower for sizes greater than 1 million elements</li>
<li>BoostCompute is about 40% to 70% slower for sizes greater than 1 million elements</li>
</ul>
<h3 id="scan">Scan</h3>
<p>The scan algorithm is useful when performing condition select operations. It can also be used where cumulative distributions are needed.</p>
<iframe width="540" height="360" seamless frameborder="0" scrolling="no" src="https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1469250706&amp;format=interactive">
</iframe>
<ul>
<li>ArrayFire consistently performs better than other libraries</li>
<li>HSA-Bolt consistently performs better than BoostCompute</li>
</ul>
<iframe width="540" height="360" seamless frameborder="0" scrolling="no" src="https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1909166539&amp;format=interactive">
</iframe>
<ul>
<li>ArrayFire performs better than other libraries for sizes less than 1 million elements</li>
<li>ArrayFire OpenCL performs better than ArrayFire CUDA for sizes greater than 100000 elements</li>
<li>Thrust consistently performs better than BoostCompute</li>
<li>For sizes greater than 1 million elements, Thrust is 16% faster than ArrayFire OpenCL, 22% faster than ArrayFire CUDA</li>
</ul>
<iframe width="540" height="360" seamless frameborder="0" scrolling="no" src="https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=944922182&amp;format=interactive">
</iframe>
<ul>
<li>Intel TBB is significantly faster than ArrayFire and BoostCompute for all sizes</li>
</ul>
<h3 id="transform">Transform</h3>
<p>The transform operation performs the same operations on each element of the input array. For this benchmark a commonly used function called <code>saxpy</code> has been used.</p>
<iframe width="540" height="360" seamless frameborder="0" scrolling="no" src="https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=2057408484&amp;format=interactive">
</iframe>
<ul>
<li>ArrayFire consistently performs better than other libraries</li>
<li>BoostCompute consistently performs better than HSA-Bolt</li>
</ul>
<iframe width="540" height="360" seamless frameborder="0" scrolling="no" src="https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=180276342&amp;format=interactive">
</iframe>
<ul>
<li>For sizes less than 1 million elements, Thrust is the fastest, followed by ArrayFire CUDA, ArrayFire OpenCL and BoostCompute</li>
<li>For sizes greater than 1 million elements, BoostCompute is fastest followed by ArrayFire OpenCL, Thrust and ArrayFire CUDA</li>
</ul>
<iframe width="540" height="360" seamless frameborder="0" scrolling="no" src="https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=2014239669&amp;format=interactive">
</iframe>
<ul>
<li>For sizes less than 1 million elements, TBB is significantly faster than BoostCompute and ArrayFire</li>
<li>For sizes larger than 1 million elements, all three libraries perform within 5% of each other</li>
</ul>
<h3 id="conclusion">Conclusion</h3>
<ul>
<li>ArrayFire is the fastest library on NVIDIA and AMD GPUs for vectors on the order of 10 million elements</li>
<li><p>At even smaller sizes, ArrayFire's memory manager ammortizes the memory allocation and deallocation time, making it significantly faster than the alternatives</p></li>
<li>ArrayFire OpenCL is faster than ArrayFire CUDA for all the algorithms for NVIDIA GPUs at large sizes</li>
<li><p>Considering that we use similar optimizations and launch configurations, this is indeed perplexing</p></li>
<li><p>OpenCL on Xeon CPUs can provide comparable performance to TBB for certain algorithms</p></li>
</ul>
<p>Each library performed better than the others at some point in the benchmarks. Considering that all the libraries are open source, these results should provide a motivation for these libraries to learn from each other.</p>
<h3 id="links">Links:</h3>
<ul>
<li><a href="https://github.com/arrayfire/valhalla">Code</a></li>
<li><a href="https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/edit#gid=60225335">Raw data</a></li>
</ul>
