(8335,7,'2015-06-02 17:51:01','2015-06-02 17:51:01','[content_band inner_container=\"true\" no_margin=\"true\" border=\"none\" padding_top=\"10px\" padding_bottom=\"40px\"]\r\n[custom_headline type=\"left\" level=\"h2\"]\r\nArrayFire Installers\r\n[/custom_headline]\r\n[custom_headline type=\"left\" level=\"h5\"]\r\nEach download contains both CUDA and OpenCL files.\r\n[/custom_headline]\r\n[gap size=\"6%\"]\r\n\r\n[column type=\"one-fourth\"][button  block=\"true\" class=\"center-text\" type=\"flat\" shape=\"rounded\" href=\"http://go.sandbox.arrayfire.com/l/37882/2015-03-19/lvvyk\" size=\"regular\" ]Windows, v3.0\r\n(64-bit)[/button][/column] [column type=\"one-fourth\"][button block=\"true\" type=\"flat\"  shape=\"rounded\" href=\"http://go.sandbox.arrayfire.com/l/37882/2015-03-19/lw515\" size=\"regular\" ]Linux, v3.0\r\n(64-bit)[/button][/column][column type=\"one-fourth\"][button  block=\"true\" type=\"flat\"  shape=\"rounded\" href=\"http://go.sandbox.arrayfire.com/l/37882/2015-03-19/ltx4p\" size=\"regular\" ]Mac OSX, v3.0\r\n(64-bit)[/button][/column] [column type=\"one-fourth\" last=\"true\"][button block=\"true\"  type=\"flat\"  shape=\"rounded\" href=\"http://go.sandbox.arrayfire.com/l/37882/2015-03-19/lw51f\" size=\"regular\" ]Tegra K1 v3.0\r\n(32-bit)[/button][/column][gap size=\"9%\"]\r\n\r\nCheck out our GitHub repository to <a href=\"https://github.com/arrayfire/arrayfire\">build from the source</a>.\r\n\r\nLooking for an older installer? <a href=\"http://sandbox.arrayfire.com/installer_archive\">Try here</a>.[/content_band]\r\n<p style=\"text-align: left;\">[content_band inner_container=\"true\" no_margin=\"true\" border=\"none\" padding_top=\"10px\" padding_bottom=\"40px\" bg_color=\"#f1f1f1\"][column type=\"one-half\"][gap size=\"8%\"]<img class=\"aligncenter wp-image-5156\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/02/mobile_Exported1-e1395437261846.png\" alt=\"mobile_Exported\" width=\"233\" height=\"311\" />[/column][column type=\"one-half\" last=\"true\"]\r\n[custom_headline type=\"left\" level=\"h2\"]\r\nGreat. What now?\r\n[/custom_headline]</p>\r\n<p style=\"text-align: left;\">[gap size=\"5%\"]</p>\r\n<p style=\"text-align: left;\">Welcome to ArrayFire! You\'ve got an awesome library there—learn more about its features and functions on <a href=\"http://www.sandbox.arrayfire.com/docs/index.htm\">our documentation</a>.</p>\r\n<p style=\"text-align: left;\">In addition to our <a href=\"https://groups.google.com/forum/#!forum/arrayfire-users\">ArrayFire user group</a> and <a href=\"http://sandbox.arrayfire.com/blog/\">our blog</a>, we offer a plethora of <a href=\"http://sandbox.arrayfire.com/support/\">support options</a> to fit your needs.</p>\r\n<p style=\"text-align: left;\">Got a job that needs closer attention? Hire us! Our engineers are acceleration wizards. Check out our <a href=\"http://sandbox.arrayfire.com/consulting/\">consulting offerings</a>, or set up a free <a href=\"http://meetme.so/arrayfire\">technical consultation</a> with us today![/column]</p>\r\n<p style=\"text-align: left;\">[/content_band]</p>\r\n<p style=\"text-align: center;\"></p>','Download','','inherit','open','closed','','5931-revision-v1','','','2015-06-02 17:51:01','2015-06-02 17:51:01','',5931,'http://sandbox.arrayfire.com/5931-revision-v1/',0,'revision','',0),(8336,7,'2015-06-02 17:51:46','2015-06-02 17:51:46','[content_band inner_container=\"true\" no_margin=\"true\" border=\"none\" padding_top=\"10px\" padding_bottom=\"40px\"]\r\n[custom_headline type=\"left\" level=\"h2\"]\r\nArrayFire Installers\r\n[/custom_headline]\r\n[custom_headline type=\"left\" level=\"h5\"]\r\nEach download contains both CUDA and OpenCL files.\r\n[/custom_headline]\r\n[gap size=\"4%\"]\r\n\r\n[column type=\"one-fourth\"][button  block=\"true\" class=\"center-text\" type=\"flat\" shape=\"rounded\" href=\"http://go.sandbox.arrayfire.com/l/37882/2015-03-19/lvvyk\" size=\"regular\" ]Windows, v3.0\r\n(64-bit)[/button][/column] [column type=\"one-fourth\"][button block=\"true\" type=\"flat\"  shape=\"rounded\" href=\"http://go.sandbox.arrayfire.com/l/37882/2015-03-19/lw515\" size=\"regular\" ]Linux, v3.0\r\n(64-bit)[/button][/column][column type=\"one-fourth\"][button  block=\"true\" type=\"flat\"  shape=\"rounded\" href=\"http://go.sandbox.arrayfire.com/l/37882/2015-03-19/ltx4p\" size=\"regular\" ]Mac OSX, v3.0\r\n(64-bit)[/button][/column] [column type=\"one-fourth\" last=\"true\"][button block=\"true\"  type=\"flat\"  shape=\"rounded\" href=\"http://go.sandbox.arrayfire.com/l/37882/2015-03-19/lw51f\" size=\"regular\" ]Tegra K1 v3.0\r\n(32-bit)[/button][/column][gap size=\"9%\"]\r\n\r\nCheck out our GitHub repository to <a href=\"https://github.com/arrayfire/arrayfire\">build from the source</a>.\r\n\r\nLooking for an older installer? <a href=\"http://sandbox.arrayfire.com/installer_archive\">Try here</a>.[/content_band]\r\n<p style=\"text-align: left;\">[content_band inner_container=\"true\" no_margin=\"true\" border=\"none\" padding_top=\"10px\" padding_bottom=\"40px\" bg_color=\"#f1f1f1\"][column type=\"one-half\"][gap size=\"8%\"]<img class=\"aligncenter wp-image-5156\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/02/mobile_Exported1-e1395437261846.png\" alt=\"mobile_Exported\" width=\"233\" height=\"311\" />[/column][column type=\"one-half\" last=\"true\"]\r\n[custom_headline type=\"left\" level=\"h2\"]\r\nGreat. What now?\r\n[/custom_headline]</p>\r\n<p style=\"text-align: left;\">[gap size=\"5%\"]</p>\r\n<p style=\"text-align: left;\">Welcome to ArrayFire! You\'ve got an awesome library there—learn more about its features and functions on <a href=\"http://www.sandbox.arrayfire.com/docs/index.htm\">our documentation</a>.</p>\r\n<p style=\"text-align: left;\">In addition to our <a href=\"https://groups.google.com/forum/#!forum/arrayfire-users\">ArrayFire user group</a> and <a href=\"http://sandbox.arrayfire.com/blog/\">our blog</a>, we offer a plethora of <a href=\"http://sandbox.arrayfire.com/support/\">support options</a> to fit your needs.</p>\r\n<p style=\"text-align: left;\">Got a job that needs closer attention? Hire us! Our engineers are acceleration wizards. Check out our <a href=\"http://sandbox.arrayfire.com/consulting/\">consulting offerings</a>, or set up a free <a href=\"http://meetme.so/arrayfire\">technical consultation</a> with us today![/column]</p>\r\n<p style=\"text-align: left;\">[/content_band]</p>\r\n<p style=\"text-align: center;\"></p>','Download','','inherit','open','closed','','5931-revision-v1','','','2015-06-02 17:51:46','2015-06-02 17:51:46','',5931,'http://sandbox.arrayfire.com/5931-revision-v1/',0,'revision','',0),(8337,17728,'2015-06-10 22:44:34','2015-06-10 22:44:34','There are many open source libraries that implement parallel versions of the algorithms in the C++ standard template libraries. Inevitably we get asked questions about how ArrayFire compares to the other libraries out in the open.\r\n\r\nIn this post we are going to compare the performance of ArrayFire to that of BoostCompute, HSA-Bolt, Intel TBB and Thrust. The benchmarks include the following commonly used vector algorithms across 3 different architectures.\r\n<ul>\r\n	<li><a href=\"#reductions\">Reductions</a></li>\r\n	<li><a href=\"#scan\">Scan</a></li>\r\n	<li><a href=\"#transform\">Transform</a></li>\r\n</ul>\r\nThe following setup has been used for the benchmarking purposes. The code to reproduce the benchmarks is linked at the bottom of the post. The hardware used for the benchmarks is listed below:\r\n<ul>\r\n	<li>NVIDIA Tesla K20</li>\r\n	<li>AMD FirePro S10000</li>\r\n	<li>Intel Xeon E5-2560v2</li>\r\n</ul>\r\n<h3 id=\"background\">Background</h3>\r\n<strong>ArrayFire</strong>\r\n\r\nArrayFire provides high performance primitives across a <a href=\"http://www.sandbox.arrayfire.com/docs/group__func__categories.htm\">wide spectrum of domains</a>. Objects of the base class, <a href=\"http://www.sandbox.arrayfire.com/docs/classaf_1_1array.htm\">array</a>, can support up to 4 dimensions. It enables users to take advantage of parallelism at multiple levels by <a href=\"http://sandbox.arrayfire.com/how-to-write-vectorized-code/\">writing vectorized code</a>.\r\n\r\nArrayFire supports NVIDIA GPUs using CUDA and OpenCL, AMD GPUs using OpenCL and Intel / AMD CPUs in serial and parallel (using OpenCL) modes. For this post, ArrayFire 3.0 has been used. You can download ArrayFire from the following locations.\r\n<ul>\r\n	<li><a href=\"http://sandbox.arrayfire.com/download/\">Installers</a></li>\r\n	<li><a href=\"https://github.com/arrayfire/arrayfire\">Build from source</a></li>\r\n</ul>\r\n<strong>BoostCompute</strong>\r\n\r\nBoost Compute is C++ template library for parallel computing based on OpenCL. Boost Compute provides accelerated versions of the algorithms present in the standard template library. It also provides an alternative C++ wrapper over common OpenCL C routines.\r\n\r\nBoost Compute supports NVIDIA GPUs, AMD GPUs and Intel / AMD CPUs using OpenCL. You can download BoostCompute from the following locations.\r\n<ul>\r\n	<li><a href=\"https://github.com/boostorg/compute\">Header only library</a></li>\r\n</ul>\r\nArrayFire uses BoostCompute for sorting, set operations and finding regions in an image in our OpenCL backend.\r\n\r\n<strong>Hsa-Bolt</strong>\r\n\r\nBolt is a C++ template library optimized for heterogenous computing. It supports routines built on top of OpenCL, C++ AMP, and TBB. It provides high performance implementations of common algorithms from the Standard Template Library.\r\n\r\nYou can download the source to Hsa-Bolt from the following locations.\r\n<ul>\r\n	<li><a href=\"https://github.com/HSA-Libraries/Bolt\">Build from Source</a></li>\r\n</ul>\r\n<strong>Thrust</strong>\r\n\r\nThrust is a parallel algorithms library providing high level C++ interafce on top of CUDA, OpenMP and TBB. It resembles C++ standard template library and provides accelerated versions of the commonly used algorithms.\r\n\r\nThrust supports NVIDIA GPUs using CUDA and Intel / AMD GPUs using OpenMP and TBB. You can download thrust from the following locations.\r\n<ul>\r\n	<li><a href=\"https://developer.nvidia.com/cuda-downloads\">From CUDA 7.0</a></li>\r\n	<li><a href=\"https://github.com/thrust/thrust\">Header only library</a></li>\r\n</ul>\r\nArrayFire uses Thrust for sorting, set operations and finding regions in an image in our CUDA backend.\r\n\r\n<strong>Intel TBB</strong>\r\n\r\nThreading Building Blocks (TBB) is a C++ template library developed by intel for multi-core processors. It provides platform independent data structures and algorithms that decrease the complexity of working with multiple threads.\r\n\r\nTBB supports multi-core processors (including the Xeon Phi). You can download TBB from the following locations.\r\n<ul>\r\n	<li><a href=\"https://www.threadingbuildingblocks.org/download\">Source and binaries</a></li>\r\n</ul>\r\n<h3 id=\"reductions\">Reductions</h3>\r\nThe reduction algorithm reduces the vector into a single element by successively reducing two input elements to a single output element. The most commonly used operations are addition, minimum and maximum.\r\n\r\nIt is one of the most commonly used algorithms in scientific and engineering algorithms. Most notably in the fields of statistics and machine learning.\r\n\r\n<iframe width=\"540\" height=\"360\" seamless=\"\" frameborder=\"0\" scrolling=\"yes\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1374058645&amp;format=interactive\">\r\n</iframe>\r\n<iframe width=\"540\" height=\"360\" seamless=\"\" frameborder=\"0\" scrolling=\"yes\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1835396253&amp;format=interactive\">\r\n</iframe>\r\n<iframe width=\"540\" height=\"360\" seamless=\"\" frameborder=\"0\" scrolling=\"yes\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=492096912&amp;format=interactive\">\r\n</iframe>\r\n\r\n<strong>AMD FirePro S10000</strong>\r\n<ul>\r\n	<li>For sizes less than 10 million elements, ArrayFire is the fastest followed by HSA-Bolt and BoostComput</li>\r\n	<li>For sizes approaching 100 million elements, HSA-Bolt is 10% faster ArrayFire, and 15% faster than BoostCompute</li>\r\n</ul>\r\n<strong>NVIDIA Tesla K20</strong>\r\n<ul>\r\n	<li>ArrayFire consistently performs better than other libraries for sizes less than 10 million elements</li>\r\n	<li>ArrayFire OpenCL performs better than ArrayFire CUDA for sizes greater than 1 million elements.</li>\r\n	<li>BoostCompute consistently performs better than Thrust</li>\r\n	<li>For sizes approaching 100 million elements, ArrayFire OpenCL is fastest followed by BoostCompute, Thrust and ArrayFire CUDA</li>\r\n</ul>\r\n<strong>Intel Xeon E5-2560v2</strong>\r\n<ul>\r\n	<li>Intel TBB is significantly faster than ArrayFire and BoostCompute for sizes less than 1 million elements</li>\r\n	<li>ArrayFire is about 20% to 30% slower for sizes greater than 1 million elements</li>\r\n	<li>BoostCompute is about 40% to 70% slower for sizes greater than 1 million elements</li>\r\n</ul>\r\n<h3 id=\"scan\">Scan</h3>\r\nThe scan algorithm is useful when performing condition select operations. It can also be used where cumulative distributions are needed.\r\n\r\n<iframe width=\"540\" height=\"360\" seamless=\"\" frameborder=\"0\" scrolling=\"yes\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1469250706&amp;format=interactive\">\r\n</iframe>\r\n<iframe width=\"540\" height=\"360\" seamless=\"\" frameborder=\"0\" scrolling=\"yes\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1909166539&amp;format=interactive\">\r\n</iframe>\r\n<iframe width=\"540\" height=\"360\" seamless=\"\" frameborder=\"0\" scrolling=\"yes\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=944922182&amp;format=interactive\">\r\n</iframe>\r\n\r\n<strong>AMD FirePro S10000</strong>\r\n<ul>\r\n	<li>ArrayFire consistently performs better than other libraries</li>\r\n	<li>HSA-Bolt consistently performs better than BoostCompute</li>\r\n</ul>\r\n<strong>NVIDIA Tesla K20</strong>\r\n<ul>\r\n	<li>ArrayFire performs better than other libraries for sizes less than 1 million elements</li>\r\n	<li>ArrayFire OpenCL performs better than ArrayFire CUDA for sizes greater than 100000 elements</li>\r\n	<li>Thrust consistently performs better than BoostCompute</li>\r\n	<li>For sizes greater than 1 million elements, Thrust is 16% faster than ArrayFire OpenCL, 22% faster than ArrayFire CUDA</li>\r\n</ul>\r\n<strong>Intel Xeon E5-2560v2</strong>\r\n<ul>\r\n	<li>Intel TBB is significantly faster than ArrayFire and BoostCompute for all sizes</li>\r\n</ul>\r\n<h3 id=\"transform\">Transform</h3>\r\nThe transform operation performs the same operations on each element of the input array. For this benchmark a commonly used function called <code>saxpy</code> has been used.\r\n\r\n<iframe width=\"540\" height=\"360\" seamless=\"\" frameborder=\"0\" scrolling=\"yes\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=2057408484&amp;format=interactive\">\r\n</iframe>\r\n<iframe width=\"540\" height=\"360\" seamless=\"\" frameborder=\"0\" scrolling=\"yes\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=180276342&amp;format=interactive\">\r\n</iframe>\r\n<iframe width=\"540\" height=\"360\" seamless=\"\" frameborder=\"0\" scrolling=\"yes\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=2014239669&amp;format=interactive\">\r\n</iframe>\r\n\r\n<strong>AMD FirePro S10000</strong>\r\n<ul>\r\n	<li>ArrayFire consistently performs better than other libraries</li>\r\n	<li>BoostCompute consistently performs better than HSA-Bolt</li>\r\n</ul>\r\n<strong>NVIDIA Tesla K20</strong>\r\n<ul>\r\n	<li>For sizes less than 1 million elements, Thrust is the fastest, followed by ArrayFire CUDA, ArrayFire OpenCL and BoostCompute</li>\r\n	<li>For sizes greater than 1 million elements, BoostCompute is fastest followed by ArrayFire OpenCL, Thrust and ArrayFire CUDA</li>\r\n</ul>\r\n<strong>Intel Xeon E5-2560v2</strong>\r\n<ul>\r\n	<li>For sizes less than 1 million elements, TBB is significantly faster than BoostCompute and ArrayFire</li>\r\n	<li>For sizes larger than 1 million elements, all three libraries perform within 5% of each other</li>\r\n</ul>\r\n<h3 id=\"conclusion\">Conclusion</h3>\r\nOn AMD and NVIDIA GPUs, ArrayFire is the fastest library for vectors on the order of 10 million elements. At even smaller sizes, ArrayFire\'s memory manager ammortizes the memory allocation and deallocation time, making it significantly faster than the alternatives. While improvements can be made overall, one particular area that can be improved upon would be the scan algorithm at large sizes.\r\n\r\nOn Intel Xeons, TBB is significantly faster than OpenCL implementations. Improvements can and should be made in ArrayFire to extract the best performance out of multicore CPUs. Our roadmap for the next quarter includes a native parallel implementation for our CPU backend. This should bridge the performance gap seen in these benchmarks.\r\n\r\nEach library performed better than the others at some point in the benchmarks. Considering that all the libraries are open source, these results should provide a motivation for these libraries to learn from each other.\r\n<h3 id=\"links\">Links:</h3>\r\n<ul>\r\n	<li><a href=\"https://github.com/arrayfire/valhalla\">Code</a></li>\r\n	<li><a href=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/edit#gid=60225335\">Raw data</a></li>\r\n</ul>','Benchmarking parallel vector libraries','','publish','open','closed','','benchmarking-parallel-vector-libraries','','','2015-06-29 16:09:23','2015-06-29 16:09:23','',0,'http://sandbox.arrayfire.com/?p=8337',0,'post','',0),(8338,17728,'2015-06-09 11:54:38','2015-06-09 11:54:38','\r\n<p>We have had several people ask us how ArrayFire compares to other Open Source vector libraries. We always give them the same answer:</p>\r\n<blockquote>\r\n<p>We are as fast or faster than the other libraries out in the open.</p>\r\n</blockquote>\r\n<p>In this post, we will expand on that answer and compare the performance of ArrayFire to thrust, BoostCompute and HSA-Bolt.</p>\r\n<h3 id=\"background\">Background</h3>\r\n<h3 id=\"feature-comparison\">Feature Comparison</h3>\r\n<table>\r\n<thead>\r\n<tr class=\"header\">\r\n<th align=\"left\">Empty</th>\r\n<th align=\"left\">ArrayFire</th>\r\n<th align=\"left\">BoostCompute</th>\r\n<th align=\"left\">HSA-Bolt</th>\r\n<th align=\"left\">Thrust</th>\r\n</tr>\r\n</thead>\r\n<tbody>\r\n<tr class=\"odd\">\r\n<td align=\"left\">CUDA</td>\r\n<td align=\"left\"><strong>Yes</strong></td>\r\n<td align=\"left\">No</td>\r\n<td align=\"left\">No</td>\r\n<td align=\"left\"><strong>Yes</strong></td>\r\n</tr>\r\n<tr class=\"even\">\r\n<td align=\"left\">OpenCL</td>\r\n<td align=\"left\"><strong>Yes</strong></td>\r\n<td align=\"left\"><strong>Yes</strong></td>\r\n<td align=\"left\"><strong>Yes</strong></td>\r\n<td align=\"left\">No</td>\r\n</tr>\r\n<tr class=\"odd\">\r\n<td align=\"left\">CPU Fallback</td>\r\n<td align=\"left\"><strong>Yes</strong> <sup>1,2</sup></td>\r\n<td align=\"left\"><strong>Yes</strong> <sup>2</sup></td>\r\n<td align=\"left\"><strong>Yes</strong><sup>1,2</sup></td>\r\n<td align=\"left\"><strong>Yes</strong><sup>1,3,4</sup></td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n<p><sup>1: Serial CPU</sup> <sup>2: Parallel CPU support using OpenCL</sup> <sup>3: Parallel CPU support using OpenMP</sup> <sup>4: Parallel CPU support using TBB</sup></p>\r\n<h4 id=\"benchmarks\">Benchmarks</h4>','Benchmarking parallel vector libraries','','inherit','open','closed','','8337-revision-v1','','','2015-06-09 11:54:38','2015-06-09 11:54:38','',8337,'http://sandbox.arrayfire.com/8337-revision-v1/',0,'revision','',0),(8339,17728,'2015-06-09 12:50:07','2015-06-09 12:50:07','\r\n<p>We have had several people ask us how ArrayFire compares to other Open Source vector libraries. We always give them the same answer:</p>\r\n<blockquote>\r\n<p>We are as fast or faster than the other libraries out in the open.</p>\r\n</blockquote>\r\n<p>In this post, we will expand on that answer and compare the performance of ArrayFire to thrust, BoostCompute and HSA-Bolt.</p>\r\n<h3 id=\"background\">Background</h3>\r\n<h3 id=\"feature-comparison\">Feature Comparison</h3>\r\n<table>\r\n<thead>\r\n<tr class=\"header\">\r\n<th align=\"left\">Empty</th>\r\n<th align=\"left\">ArrayFire</th>\r\n<th align=\"left\">Boost.Compute</th>\r\n<th align=\"left\">HSA-Bolt</th>\r\n<th align=\"left\">Thrust</th>\r\n</tr>\r\n</thead>\r\n<tbody>\r\n<tr class=\"odd\">\r\n<td align=\"left\">CUDA</td>\r\n<td align=\"left\"><strong>Yes</strong></td>\r\n<td align=\"left\">No</td>\r\n<td align=\"left\">No</td>\r\n<td align=\"left\"><strong>Yes</strong></td>\r\n</tr>\r\n<tr class=\"even\">\r\n<td align=\"left\">OpenCL</td>\r\n<td align=\"left\"><strong>Yes</strong></td>\r\n<td align=\"left\"><strong>Yes</strong></td>\r\n<td align=\"left\"><strong>Yes</strong></td>\r\n<td align=\"left\">No</td>\r\n</tr>\r\n<tr class=\"odd\">\r\n<td align=\"left\">Parallel CPU</td>\r\n<td align=\"left\"><strong>Yes</strong> <sup>1</sup></td>\r\n<td align=\"left\"><strong>Yes</strong> <sup>1</sup></td>\r\n<td align=\"left\"><strong>Yes</strong><sup>1,3</sup></td>\r\n<td align=\"left\"><strong>Yes</strong><sup>1,2,3</sup></td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n<!-- | Empty     |  CUDA   | OpenCL  | CPU (Parallel) | -->\r\n<!-- |-----------|---------|---------|--------| -->\r\n<!-- | ArrayFire | **Yes** | **Yes** | **Yes** <sup>1</sup> | -->\r\n<!-- | Boost.Compute| **No** | **Yes** | **Yes** <sup>1</sup> | -->\r\n<!-- | HSA-Bolt  |  **No** | **Yes** | **Yes** <sup>1,3</sup> | -->\r\n<!-- | Thrust    | **Yes** | **No**  | **Yes** <sup>1,2,3</sup> | -->\r\n<ul>\r\n<li><sup>1: Using OpenCL</sup></li>\r\n<li><sup>2: Using OpenMP</sup></li>\r\n<li><sup>3: Using TBB</sup></li>\r\n</ul>\r\n<h4 id=\"benchmarks\">Benchmarks</h4>\r\n','Benchmarking parallel vector libraries','','inherit','open','closed','','8337-revision-v1','','','2015-06-09 12:50:07','2015-06-09 12:50:07','',8337,'http://sandbox.arrayfire.com/8337-revision-v1/',0,'revision','',0),(8340,17728,'2015-06-09 12:50:52','2015-06-09 12:50:52','\r\n<p>We have had several people ask us how ArrayFire compares to other Open Source vector libraries. We always give them the same answer:</p>\r\n<blockquote>\r\n<p>We are as fast or faster than the other libraries out in the open.</p>\r\n</blockquote>\r\n<p>In this post, we will expand on that answer and compare the performance of ArrayFire to thrust, BoostCompute and HSA-Bolt.</p>\r\n<h3 id=\"background\">Background</h3>\r\n<h3 id=\"feature-comparison\">Feature Comparison</h3>\r\n<table>\r\n<thead>\r\n<tr class=\"header\">\r\n<th align=\"left\">Empty</th>\r\n<th align=\"left\">ArrayFire</th>\r\n<th align=\"left\">Boost.Compute</th>\r\n<th align=\"left\">HSA-Bolt</th>\r\n<th align=\"left\">Thrust</th>\r\n</tr>\r\n</thead>\r\n<tbody>\r\n<tr class=\"odd\">\r\n<td align=\"left\">CUDA</td>\r\n<td align=\"left\"><strong>Yes</strong></td>\r\n<td align=\"left\">No</td>\r\n<td align=\"left\">No</td>\r\n<td align=\"left\"><strong>Yes</strong></td>\r\n</tr>\r\n<tr class=\"even\">\r\n<td align=\"left\">OpenCL</td>\r\n<td align=\"left\"><strong>Yes</strong></td>\r\n<td align=\"left\"><strong>Yes</strong></td>\r\n<td align=\"left\"><strong>Yes</strong></td>\r\n<td align=\"left\">No</td>\r\n</tr>\r\n<tr class=\"odd\">\r\n<td align=\"left\">Parallel CPU</td>\r\n<td align=\"left\"><strong>Yes</strong> <sup>1</sup></td>\r\n<td align=\"left\"><strong>Yes</strong> <sup>1</sup></td>\r\n<td align=\"left\"><strong>Yes</strong><sup>1,3</sup></td>\r\n<td align=\"left\"><strong>Yes</strong><sup>1,2,3</sup></td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n<ul>\r\n<li><sup>1: Using OpenCL</sup></li>\r\n<li><sup>2: Using OpenMP</sup></li>\r\n<li><sup>3: Using TBB</sup></li>\r\n</ul>\r\n<h4 id=\"benchmarks\">Benchmarks</h4>\r\n','Benchmarking parallel vector libraries','','inherit','open','closed','','8337-revision-v1','','','2015-06-09 12:50:52','2015-06-09 12:50:52','',8337,'http://sandbox.arrayfire.com/8337-revision-v1/',0,'revision','',0),(8344,17728,'2015-06-10 19:14:47','2015-06-10 19:14:47','\r\n<p>There are many open source libraries that implement parallel versions of the algorithms in the C++ standard template libraries. Inevitably we get asked questions about how ArrayFire compares to the other libraries out in the open.</p>\r\n<p>In this post we are going to compare the performance of ArrayFire to that of BoostCompute, HSA-Bolt, Intel TBB and Thrust. The benchmarks include the following commonly used vector algorithms across 3 different architectures.</p>\r\n<ul>\r\n<li><a href=\"#reductions\">Reductions</a></li>\r\n<li><a href=\"#scan\">Scan</a></li>\r\n<li><a href=\"#transform\">Transform</a></li>\r\n</ul>\r\n<p>The following setup has been used for the benchmarking purposes.</p>\r\n<p><strong>Hardware:</strong></p>\r\n<ul>\r\n<li>NVIDIA Tesla K20</li>\r\n<li>AMD FirePro S10000</li>\r\n<li>Intel Xeon E5-2560v2</li>\r\n</ul>\r\n<h3 id=\"background\">Background</h3>\r\n<p><strong>ArrayFire</strong></p>\r\n<p>ArrayFire provides high performance primitives across a <a href=\"http://www.sandbox.arrayfire.com/docs/group__func__categories.htm\">wide spectrum of domains</a>. Objects of the base class, <a href=\"http://www.sandbox.arrayfire.com/docs/classaf_1_1array.htm\"><code>array</code></a>, can support up to 4 dimensions. It enables users to take advantage of parallelism at multiple levels by <a href=\"http://sandbox.arrayfire.com/how-to-write-vectorized-code/\">writing vectorized code</a>.</p>\r\n<p>ArrayFire supports NVIDIA GPUs using CUDA and OpenCL, AMD GPUs using OpenCL and Intel / AMD CPUs in serial and parallel (using OpenCL) modes. For this post, ArrayFire 3.0 has been used. You can download ArrayFire from the following locations.</p>\r\n<ul>\r\n<li><a href=\"http://sandbox.arrayfire.com/download/\">Installers</a></li>\r\n<li><a href=\"https://github.com/arrayfire/arrayfire\">Build from source</a></li>\r\n</ul>\r\n<p><strong>BoostCompute</strong></p>\r\n<p>Boost Compute is C++ template library for parallel computing based on OpenCL. Boost Compute provides accelerated versions of the algorithms present in the standard template library. It also provides an alternative C++ wrapper over common OpenCL C routines.</p>\r\n<p>Boost Compute supports NVIDIA GPUs, AMD GPUs and Intel / AMD CPUs using OpenCL. You can download BoostCompute from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://github.com/boostorg/compute\">Header only library</a></li>\r\n</ul>\r\n<p><strong>HSA.Bolt</strong></p>\r\n<p>Bolt is a C++ template library optimized for heterogenous computing. It supports routines built on top of OpenCL, C++ AMP, and TBB. It provides high performance implementations of common algorithms from the Stnadard Template Library.</p>\r\n<p>You can download the source to HSA.Bolt from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://github.com/HSA-Libraries/Bolt\">Build from Source</a></li>\r\n</ul>\r\n<p><strong>Thrust</strong></p>\r\n<p>Thrust is a parallel algorithms library providing high level C++ interafce on top of CUDA, OpenMP and TBB. It resembles C++ standard template library and provides accelerated versions of the commonly used algorithms.</p>\r\n<p>Thrust supports NVIDIA GPUs using CUDA and Intel / AMD GPUs using OpenMP and TBB. You can download thrust from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://developer.nvidia.com/cuda-downloads\">From CUDA 7.0</a></li>\r\n<li><a href=\"https://github.com/thrust/thrust\">Header only library</a></li>\r\n</ul>\r\n<p><strong>Intel TBB</strong></p>\r\n<p>Threading Building Blocks (TBB) is a C++ template library developed by intel for multi-core processors. It provides platform independent data structures and algorithms that decrease the complexity of working with multiple threads.</p>\r\n<p>TBB supports multi-core processors (including the Xeon Phi). You can download TBB from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://www.threadingbuildingblocks.org/download\">Source and binaries</a></li>\r\n</ul>\r\n<h3 id=\"reductions\">Reductions</h3>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1374058645&amp;format=interactive\">\r\n</iframe>\r\n<p><br></p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1835396253&amp;format=interactive\">\r\n</iframe>\r\n<p><br></p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=492096912&amp;format=interactive\">\r\n</iframe>\r\n<h3 id=\"scan\">Scan</h3>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1469250706&amp;format=interactive\">\r\n</iframe>\r\n<p><br></p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1909166539&amp;format=interactive\">\r\n</iframe>\r\n<p><br></p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=944922182&amp;format=interactive\">\r\n</iframe>\r\n<h3 id=\"transform\">Transform</h3>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=2057408484&amp;format=interactive\">\r\n</iframe>\r\n<p><br></p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=180276342&amp;format=interactive\">\r\n</iframe>\r\n<p><br></p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=2014239669&amp;format=interactive\">\r\n</iframe>\r\n','Benchmarking parallel vector libraries','','inherit','open','closed','','8337-revision-v1','','','2015-06-10 19:14:47','2015-06-10 19:14:47','',8337,'http://sandbox.arrayfire.com/8337-revision-v1/',0,'revision','',0),(8345,17728,'2015-06-10 19:15:52','2015-06-10 19:15:52','\r\n<p>There are many open source libraries that implement parallel versions of the algorithms in the C++ standard template libraries. Inevitably we get asked questions about how ArrayFire compares to the other libraries out in the open.</p>\r\n<p>In this post we are going to compare the performance of ArrayFire to that of BoostCompute, HSA-Bolt, Intel TBB and Thrust. The benchmarks include the following commonly used vector algorithms across 3 different architectures.</p>\r\n<ul>\r\n<li><a href=\"#reductions\">Reductions</a></li>\r\n<li><a href=\"#scan\">Scan</a></li>\r\n<li><a href=\"#transform\">Transform</a></li>\r\n</ul>\r\n<p>The following setup has been used for the benchmarking purposes.</p>\r\n<p><strong>Hardware:</strong></p>\r\n<ul>\r\n<li>NVIDIA Tesla K20</li>\r\n<li>AMD FirePro S10000</li>\r\n<li>Intel Xeon E5-2560v2</li>\r\n</ul>\r\n<h3 id=\"background\">Background</h3>\r\n<p><strong>ArrayFire</strong></p>\r\n<p>ArrayFire provides high performance primitives across a <a href=\"http://www.sandbox.arrayfire.com/docs/group__func__categories.htm\">wide spectrum of domains</a>. Objects of the base class, <a href=\"http://www.sandbox.arrayfire.com/docs/classaf_1_1array.htm\">array</a>, can support up to 4 dimensions. It enables users to take advantage of parallelism at multiple levels by <a href=\"http://sandbox.arrayfire.com/how-to-write-vectorized-code/\">writing vectorized code</a>.</p>\r\n<p>ArrayFire supports NVIDIA GPUs using CUDA and OpenCL, AMD GPUs using OpenCL and Intel / AMD CPUs in serial and parallel (using OpenCL) modes. For this post, ArrayFire 3.0 has been used. You can download ArrayFire from the following locations.</p>\r\n<ul>\r\n<li><a href=\"http://sandbox.arrayfire.com/download/\">Installers</a></li>\r\n<li><a href=\"https://github.com/arrayfire/arrayfire\">Build from source</a></li>\r\n</ul>\r\n<p><strong>BoostCompute</strong></p>\r\n<p>Boost Compute is C++ template library for parallel computing based on OpenCL. Boost Compute provides accelerated versions of the algorithms present in the standard template library. It also provides an alternative C++ wrapper over common OpenCL C routines.</p>\r\n<p>Boost Compute supports NVIDIA GPUs, AMD GPUs and Intel / AMD CPUs using OpenCL. You can download BoostCompute from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://github.com/boostorg/compute\">Header only library</a></li>\r\n</ul>\r\n<p><strong>HSA.Bolt</strong></p>\r\n<p>Bolt is a C++ template library optimized for heterogenous computing. It supports routines built on top of OpenCL, C++ AMP, and TBB. It provides high performance implementations of common algorithms from the Stnadard Template Library.</p>\r\n<p>You can download the source to HSA.Bolt from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://github.com/HSA-Libraries/Bolt\">Build from Source</a></li>\r\n</ul>\r\n<p><strong>Thrust</strong></p>\r\n<p>Thrust is a parallel algorithms library providing high level C++ interafce on top of CUDA, OpenMP and TBB. It resembles C++ standard template library and provides accelerated versions of the commonly used algorithms.</p>\r\n<p>Thrust supports NVIDIA GPUs using CUDA and Intel / AMD GPUs using OpenMP and TBB. You can download thrust from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://developer.nvidia.com/cuda-downloads\">From CUDA 7.0</a></li>\r\n<li><a href=\"https://github.com/thrust/thrust\">Header only library</a></li>\r\n</ul>\r\n<p><strong>Intel TBB</strong></p>\r\n<p>Threading Building Blocks (TBB) is a C++ template library developed by intel for multi-core processors. It provides platform independent data structures and algorithms that decrease the complexity of working with multiple threads.</p>\r\n<p>TBB supports multi-core processors (including the Xeon Phi). You can download TBB from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://www.threadingbuildingblocks.org/download\">Source and binaries</a></li>\r\n</ul>\r\n<h3 id=\"reductions\">Reductions</h3>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1374058645&amp;format=interactive\">\r\n</iframe>\r\n<p><br></p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1835396253&amp;format=interactive\">\r\n</iframe>\r\n<p><br></p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=492096912&amp;format=interactive\">\r\n</iframe>\r\n<h3 id=\"scan\">Scan</h3>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1469250706&amp;format=interactive\">\r\n</iframe>\r\n<p><br></p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1909166539&amp;format=interactive\">\r\n</iframe>\r\n<p><br></p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=944922182&amp;format=interactive\">\r\n</iframe>\r\n<h3 id=\"transform\">Transform</h3>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=2057408484&amp;format=interactive\">\r\n</iframe>\r\n<p><br></p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=180276342&amp;format=interactive\">\r\n</iframe>\r\n<p><br></p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=2014239669&amp;format=interactive\">\r\n</iframe>','Benchmarking parallel vector libraries','','inherit','open','closed','','8337-revision-v1','','','2015-06-10 19:15:52','2015-06-10 19:15:52','',8337,'http://sandbox.arrayfire.com/8337-revision-v1/',0,'revision','',0),(8346,17728,'2015-06-10 21:14:33','2015-06-10 21:14:33','\r\n<p>There are many open source libraries that implement parallel versions of the algorithms in the C++ standard template libraries. Inevitably we get asked questions about how ArrayFire compares to the other libraries out in the open.</p>\r\n<p>In this post we are going to compare the performance of ArrayFire to that of BoostCompute, HSA-Bolt, Intel TBB and Thrust. The benchmarks include the following commonly used vector algorithms across 3 different architectures.</p>\r\n<ul>\r\n<li><a href=\"#reductions\">Reductions</a></li>\r\n<li><a href=\"#scan\">Scan</a></li>\r\n<li><a href=\"#transform\">Transform</a></li>\r\n</ul>\r\n<p>The following setup has been used for the benchmarking purposes.</p>\r\n<p><strong>Hardware:</strong></p>\r\n<ul>\r\n<li>NVIDIA Tesla K20</li>\r\n<li>AMD FirePro S10000</li>\r\n<li>Intel Xeon E5-2560v2</li>\r\n</ul>\r\n<h3 id=\"background\">Background</h3>\r\n<p><strong>ArrayFire</strong></p>\r\n<p>ArrayFire provides high performance primitives across a <a href=\"http://www.sandbox.arrayfire.com/docs/group__func__categories.htm\">wide spectrum of domains</a>. Objects of the base class, <a href=\"http://www.sandbox.arrayfire.com/docs/classaf_1_1array.htm\">array</a>, can support up to 4 dimensions. It enables users to take advantage of parallelism at multiple levels by <a href=\"http://sandbox.arrayfire.com/how-to-write-vectorized-code/\">writing vectorized code</a>.</p>\r\n<p>ArrayFire supports NVIDIA GPUs using CUDA and OpenCL, AMD GPUs using OpenCL and Intel / AMD CPUs in serial and parallel (using OpenCL) modes. For this post, ArrayFire 3.0 has been used. You can download ArrayFire from the following locations.</p>\r\n<ul>\r\n<li><a href=\"http://sandbox.arrayfire.com/download/\">Installers</a></li>\r\n<li><a href=\"https://github.com/arrayfire/arrayfire\">Build from source</a></li>\r\n</ul>\r\n<p><strong>BoostCompute</strong></p>\r\n<p>Boost Compute is C++ template library for parallel computing based on OpenCL. Boost Compute provides accelerated versions of the algorithms present in the standard template library. It also provides an alternative C++ wrapper over common OpenCL C routines.</p>\r\n<p>Boost Compute supports NVIDIA GPUs, AMD GPUs and Intel / AMD CPUs using OpenCL. You can download BoostCompute from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://github.com/boostorg/compute\">Header only library</a></li>\r\n</ul>\r\n<p>ArrayFire uses BoostCompute for sorting, set operations and finding regions in image in our OpenCL backend.</p>\r\n<p><strong>HSA.Bolt</strong></p>\r\n<p>Bolt is a C++ template library optimized for heterogenous computing. It supports routines built on top of OpenCL, C++ AMP, and TBB. It provides high performance implementations of common algorithms from the Stnadard Template Library.</p>\r\n<p>You can download the source to HSA.Bolt from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://github.com/HSA-Libraries/Bolt\">Build from Source</a></li>\r\n</ul>\r\n<p><strong>Thrust</strong></p>\r\n<p>Thrust is a parallel algorithms library providing high level C++ interafce on top of CUDA, OpenMP and TBB. It resembles C++ standard template library and provides accelerated versions of the commonly used algorithms.</p>\r\n<p>Thrust supports NVIDIA GPUs using CUDA and Intel / AMD GPUs using OpenMP and TBB. You can download thrust from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://developer.nvidia.com/cuda-downloads\">From CUDA 7.0</a></li>\r\n<li><a href=\"https://github.com/thrust/thrust\">Header only library</a></li>\r\n</ul>\r\n<p>ArrayFire uses BoostCompute for sorting, set operations and finding regions in image in our CUDA backend.</p>\r\n<p><strong>Intel TBB</strong></p>\r\n<p>Threading Building Blocks (TBB) is a C++ template library developed by intel for multi-core processors. It provides platform independent data structures and algorithms that decrease the complexity of working with multiple threads.</p>\r\n<p>TBB supports multi-core processors (including the Xeon Phi). You can download TBB from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://www.threadingbuildingblocks.org/download\">Source and binaries</a></li>\r\n</ul>\r\n<h3 id=\"reductions\">Reductions</h3>\r\n<p>The reduction algorithm is one of the most commonly used algorithms in scientific and engineering algorithms. It is most commonly used in statistics and machine learning.</p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1374058645&amp;format=interactive\">\r\n</iframe>\r\n<p><br></p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1835396253&amp;format=interactive\">\r\n</iframe>\r\n<p><br></p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=492096912&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>On NVIDIA K20</li>\r\n<li>ArrayFire consitently performs better than other libraries for sizes less than 10 million elements</li>\r\n<li>ArrayFire OpenCL consitently performs better than ArrayFire CUDA.</li>\r\n<li>BoostCompute consitently performs better than Thrust</li>\r\n<li><p>For sizes approaching 100 million elements, ArrayFire OpenCL is fastest followed by BoostCompute, Thrust and ArrayFire CUDA</p></li>\r\n<li>On AMD S10000</li>\r\n<li>ArrayFire consitently performs better than other libraries for sizes less than 10 million elements</li>\r\n<li>HSA-Bolt consistently performs better than BoostCompute</li>\r\n<li><p>For sizes approaching 100 million elements, HSA-Bolt performs the best followed by ArrayFire and BoostCompute</p></li>\r\n<li>On Intel E5-2560v2</li>\r\n<li>Intel TBB is significantly faster than ArrayFire and BoostCompute for sizes less than 1 million elements</li>\r\n<li>ArrayFire is about 20% to 30% slower for sizes greater than 1 million elements</li>\r\n<li><p>BoostCompute is about 40% to 70% slower for sizes greater than 1 million elements</p></li>\r\n</ul>\r\n<h3 id=\"scan\">Scan</h3>\r\n<p>The scan algorithm is useful when performing condition select operations. It can also be used where cumulative distributions are needed.</p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1469250706&amp;format=interactive\">\r\n</iframe>\r\n<p><br></p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1909166539&amp;format=interactive\">\r\n</iframe>\r\n<p><br></p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=944922182&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>On NVIDIA K20</li>\r\n<li>ArrayFire performs better than other libraries for sizes less than 1 million elements</li>\r\n<li>ArrayFire OpenCL consitently performs better than ArrayFire CUDA</li>\r\n<li>Thrust consistently performs better than BoostCompute</li>\r\n<li><p>For sizes greater than 1 million elements, Thrust is the fastest followed by ArrayFire OpenCL, ArrayFire CUDA and BoostCompute</p></li>\r\n<li>On AMD S10000</li>\r\n<li>ArrayFire consitently performs better than other libraries</li>\r\n<li><p>HSA-Bolt consistently performs better than BoostCompute</p></li>\r\n<li>On Intel E5-2560v2</li>\r\n<li><p>Intel TBB is significantly faster than ArrayFire and BoostCompute for all sizes</p></li>\r\n</ul>\r\n<h3 id=\"transform\">Transform</h3>\r\n<p>The transform operation performs the same operations on each element of the input array. For this benchmark a commonly used function called <code>saxpy</code> has been used. The formulation can be seen below.</p>\r\n<pre><code>Z = a * X + Y</code></pre>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=2057408484&amp;format=interactive\">\r\n</iframe>\r\n<p><br></p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=180276342&amp;format=interactive\">\r\n</iframe>\r\n<p><br></p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=2014239669&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>On NVIDIA K20</li>\r\n<li>For sizes less than 1 million elements, Thrust is the fastest, followed by ArrayFire and BoostCompute</li>\r\n<li><p>For sizes greater than 1 million elements, BoostCompute is fastest followed by ArrayFire OpenCL, Thrust and ArrayFire CUDA</p></li>\r\n<li>On AMD S10000</li>\r\n<li>ArrayFire consitently performs better than other libraries</li>\r\n<li><p>BoostCompute consistently performs better than HSA-Bolt</p></li>\r\n<li>On Intel E5-2560v2</li>\r\n<li>For sizes less than 1 million elements, TBB is significantly faster than BoostCompute and ArrayFire</li>\r\n<li><p>For sizes larger than 1 million elements, all three libraries perform within 5% of each other.</p></li>\r\n</ul>\r\n<h4 id=\"conclusion\">Conclusion</h4>\r\n<ul>\r\n<li>ArrayFire is the fastest library on NVIDIA and AMD GPUs for vectors on the order of 10 million elements</li>\r\n<li><p>At even smaller sizes, ArrayFire\'s memory manager ammortizes the memory allocation and deallocation time, making it significantly faster than the alternatives</p></li>\r\n<li>ArrayFire OpenCL is faster than ArrayFire CUDA for all the algorithms for NVIDIA GPUs</li>\r\n<li><p>Considering that we use similar optimizations and launch configurations, this is indeed perplexing</p></li>\r\n<li><p>OpenCL on Xeon CPUs can provide comparable performance to TBB for certain algorithms</p></li>\r\n</ul>\r\n<p>Each library performed better than the others at some point in the benchmarks. Considering that all the libraries are open source, these libraries should learn and borrow from each other.</p>\r\n','Benchmarking parallel vector libraries','','inherit','open','closed','','8337-revision-v1','','','2015-06-10 21:14:33','2015-06-10 21:14:33','',8337,'http://sandbox.arrayfire.com/8337-revision-v1/',0,'revision','',0),(8347,17728,'2015-06-10 21:23:32','2015-06-10 21:23:32','\r\n<p>There are many open source libraries that implement parallel versions of the algorithms in the C++ standard template libraries. Inevitably we get asked questions about how ArrayFire compares to the other libraries out in the open.</p>\r\n<p>In this post we are going to compare the performance of ArrayFire to that of BoostCompute, HSA-Bolt, Intel TBB and Thrust. The benchmarks include the following commonly used vector algorithms across 3 different architectures.</p>\r\n<ul>\r\n<li><a href=\"#reductions\">Reductions</a></li>\r\n<li><a href=\"#scan\">Scan</a></li>\r\n<li><a href=\"#transform\">Transform</a></li>\r\n</ul>\r\n<p>The following setup has been used for the benchmarking purposes.</p>\r\n<p><strong>Hardware:</strong></p>\r\n<ul>\r\n<li>NVIDIA Tesla K20</li>\r\n<li>AMD FirePro S10000</li>\r\n<li>Intel Xeon E5-2560v2</li>\r\n</ul>\r\n<h3 id=\"background\">Background</h3>\r\n<p><strong>ArrayFire</strong></p>\r\n<p>ArrayFire provides high performance primitives across a <a href=\"http://www.sandbox.arrayfire.com/docs/group__func__categories.htm\">wide spectrum of domains</a>. Objects of the base class, <a href=\"http://www.sandbox.arrayfire.com/docs/classaf_1_1array.htm\">array</a>, can support up to 4 dimensions. It enables users to take advantage of parallelism at multiple levels by <a href=\"http://sandbox.arrayfire.com/how-to-write-vectorized-code/\">writing vectorized code</a>.</p>\r\n<p>ArrayFire supports NVIDIA GPUs using CUDA and OpenCL, AMD GPUs using OpenCL and Intel / AMD CPUs in serial and parallel (using OpenCL) modes. For this post, ArrayFire 3.0 has been used. You can download ArrayFire from the following locations.</p>\r\n<ul>\r\n<li><a href=\"http://sandbox.arrayfire.com/download/\">Installers</a></li>\r\n<li><a href=\"https://github.com/arrayfire/arrayfire\">Build from source</a></li>\r\n</ul>\r\n<p><strong>BoostCompute</strong></p>\r\n<p>Boost Compute is C++ template library for parallel computing based on OpenCL. Boost Compute provides accelerated versions of the algorithms present in the standard template library. It also provides an alternative C++ wrapper over common OpenCL C routines.</p>\r\n<p>Boost Compute supports NVIDIA GPUs, AMD GPUs and Intel / AMD CPUs using OpenCL. You can download BoostCompute from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://github.com/boostorg/compute\">Header only library</a></li>\r\n</ul>\r\n<p>ArrayFire uses BoostCompute for sorting, set operations and finding regions in image in our OpenCL backend.</p>\r\n<p><strong>HSA.Bolt</strong></p>\r\n<p>Bolt is a C++ template library optimized for heterogenous computing. It supports routines built on top of OpenCL, C++ AMP, and TBB. It provides high performance implementations of common algorithms from the Stnadard Template Library.</p>\r\n<p>You can download the source to HSA.Bolt from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://github.com/HSA-Libraries/Bolt\">Build from Source</a></li>\r\n</ul>\r\n<p><strong>Thrust</strong></p>\r\n<p>Thrust is a parallel algorithms library providing high level C++ interafce on top of CUDA, OpenMP and TBB. It resembles C++ standard template library and provides accelerated versions of the commonly used algorithms.</p>\r\n<p>Thrust supports NVIDIA GPUs using CUDA and Intel / AMD GPUs using OpenMP and TBB. You can download thrust from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://developer.nvidia.com/cuda-downloads\">From CUDA 7.0</a></li>\r\n<li><a href=\"https://github.com/thrust/thrust\">Header only library</a></li>\r\n</ul>\r\n<p>ArrayFire uses BoostCompute for sorting, set operations and finding regions in image in our CUDA backend.</p>\r\n<p><strong>Intel TBB</strong></p>\r\n<p>Threading Building Blocks (TBB) is a C++ template library developed by intel for multi-core processors. It provides platform independent data structures and algorithms that decrease the complexity of working with multiple threads.</p>\r\n<p>TBB supports multi-core processors (including the Xeon Phi). You can download TBB from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://www.threadingbuildingblocks.org/download\">Source and binaries</a></li>\r\n</ul>\r\n<h3 id=\"reductions\">Reductions</h3>\r\n<p>The reduction algorithm is one of the most commonly used algorithms in scientific and engineering algorithms. It is most commonly used in statistics and machine learning.</p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1374058645&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>ArrayFire consitently performs better than other libraries for sizes less than 10 million elements</li>\r\n<li>HSA-Bolt consistently performs better than BoostCompute</li>\r\n<li>For sizes approaching 100 million elements, HSA-Bolt performs the best followed by ArrayFire and BoostCompute</li>\r\n</ul>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1835396253&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>ArrayFire consitently performs better than other libraries for sizes less than 10 million elements</li>\r\n<li>ArrayFire OpenCL consitently performs better than ArrayFire CUDA.</li>\r\n<li>BoostCompute consitently performs better than Thrust</li>\r\n<li>For sizes approaching 100 million elements, ArrayFire OpenCL is fastest followed by BoostCompute, Thrust and ArrayFire CUDA</li>\r\n</ul>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=492096912&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>Intel TBB is significantly faster than ArrayFire and BoostCompute for sizes less than 1 million elements</li>\r\n<li>ArrayFire is about 20% to 30% slower for sizes greater than 1 million elements</li>\r\n<li>BoostCompute is about 40% to 70% slower for sizes greater than 1 million elements</li>\r\n</ul>\r\n<h3 id=\"scan\">Scan</h3>\r\n<p>The scan algorithm is useful when performing condition select operations. It can also be used where cumulative distributions are needed.</p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1469250706&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>ArrayFire consitently performs better than other libraries</li>\r\n<li>HSA-Bolt consistently performs better than BoostCompute</li>\r\n</ul>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1909166539&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>ArrayFire performs better than other libraries for sizes less than 1 million elements</li>\r\n<li>ArrayFire OpenCL consitently performs better than ArrayFire CUDA</li>\r\n<li>Thrust consistently performs better than BoostCompute</li>\r\n<li>For sizes greater than 1 million elements, Thrust is the fastest followed by ArrayFire OpenCL, ArrayFire CUDA and BoostCompute</li>\r\n</ul>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=944922182&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>Intel TBB is significantly faster than ArrayFire and BoostCompute for all sizes</li>\r\n</ul>\r\n<h3 id=\"transform\">Transform</h3>\r\n<p>The transform operation performs the same operations on each element of the input array. For this benchmark a commonly used function called <code>saxpy</code> has been used.</p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=2057408484&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>ArrayFire consitently performs better than other libraries</li>\r\n<li>BoostCompute consistently performs better than HSA-Bolt</li>\r\n</ul>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=180276342&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>For sizes less than 1 million elements, Thrust is the fastest, followed by ArrayFire and BoostCompute</li>\r\n<li>For sizes greater than 1 million elements, BoostCompute is fastest followed by ArrayFire OpenCL, Thrust and ArrayFire CUDA</li>\r\n</ul>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=2014239669&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>For sizes less than 1 million elements, TBB is significantly faster than BoostCompute and ArrayFire</li>\r\n<li>For sizes larger than 1 million elements, all three libraries perform within 5% of each other.</li>\r\n</ul>\r\n<h4 id=\"conclusion\">Conclusion</h4>\r\n<ul>\r\n<li>ArrayFire is the fastest library on NVIDIA and AMD GPUs for vectors on the order of 10 million elements</li>\r\n<li><p>At even smaller sizes, ArrayFire\'s memory manager ammortizes the memory allocation and deallocation time, making it significantly faster than the alternatives</p></li>\r\n<li>ArrayFire OpenCL is faster than ArrayFire CUDA for all the algorithms for NVIDIA GPUs</li>\r\n<li><p>Considering that we use similar optimizations and launch configurations, this is indeed perplexing</p></li>\r\n<li><p>OpenCL on Xeon CPUs can provide comparable performance to TBB for certain algorithms</p></li>\r\n</ul>\r\n<p>Each library performed better than the others at some point in the benchmarks. Considering that all the libraries are open source, these results should provide a motivation for these libraries to learn from each other.</p>','Benchmarking parallel vector libraries','','inherit','open','closed','','8337-revision-v1','','','2015-06-10 21:23:32','2015-06-10 21:23:32','',8337,'http://sandbox.arrayfire.com/8337-revision-v1/',0,'revision','',0),(8351,17728,'2015-06-10 21:48:22','2015-06-10 21:48:22','\r\n<p>There are many open source libraries that implement parallel versions of the algorithms in the C++ standard template libraries. Inevitably we get asked questions about how ArrayFire compares to the other libraries out in the open.</p>\r\n<p>In this post we are going to compare the performance of ArrayFire to that of BoostCompute, HSA-Bolt, Intel TBB and Thrust. The benchmarks include the following commonly used vector algorithms across 3 different architectures.</p>\r\n<ul>\r\n<li><a href=\"#reductions\">Reductions</a></li>\r\n<li><a href=\"#scan\">Scan</a></li>\r\n<li><a href=\"#transform\">Transform</a></li>\r\n</ul>\r\n<p>The following setup has been used for the benchmarking purposes. The code to reproduce the benchmarks is linked at the bottom of the post.</p>\r\n<p><strong>Hardware:</strong></p>\r\n<ul>\r\n<li>NVIDIA Tesla K20</li>\r\n<li>AMD FirePro S10000</li>\r\n<li>Intel Xeon E5-2560v2</li>\r\n</ul>\r\n<h3 id=\"background\">Background</h3>\r\n<p><strong>ArrayFire</strong></p>\r\n<p>ArrayFire provides high performance primitives across a <a href=\"http://www.sandbox.arrayfire.com/docs/group__func__categories.htm\">wide spectrum of domains</a>. Objects of the base class, <a href=\"http://www.sandbox.arrayfire.com/docs/classaf_1_1array.htm\">array</a>, can support up to 4 dimensions. It enables users to take advantage of parallelism at multiple levels by <a href=\"http://sandbox.arrayfire.com/how-to-write-vectorized-code/\">writing vectorized code</a>.</p>\r\n<p>ArrayFire supports NVIDIA GPUs using CUDA and OpenCL, AMD GPUs using OpenCL and Intel / AMD CPUs in serial and parallel (using OpenCL) modes. For this post, ArrayFire 3.0 has been used. You can download ArrayFire from the following locations.</p>\r\n<ul>\r\n<li><a href=\"http://sandbox.arrayfire.com/download/\">Installers</a></li>\r\n<li><a href=\"https://github.com/arrayfire/arrayfire\">Build from source</a></li>\r\n</ul>\r\n<p><strong>BoostCompute</strong></p>\r\n<p>Boost Compute is C++ template library for parallel computing based on OpenCL. Boost Compute provides accelerated versions of the algorithms present in the standard template library. It also provides an alternative C++ wrapper over common OpenCL C routines.</p>\r\n<p>Boost Compute supports NVIDIA GPUs, AMD GPUs and Intel / AMD CPUs using OpenCL. You can download BoostCompute from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://github.com/boostorg/compute\">Header only library</a></li>\r\n</ul>\r\n<p>ArrayFire uses BoostCompute for sorting, set operations and finding regions in image in our OpenCL backend.</p>\r\n<p><strong>Hsa-Bolt</strong></p>\r\n<p>Bolt is a C++ template library optimized for heterogenous computing. It supports routines built on top of OpenCL, C++ AMP, and TBB. It provides high performance implementations of common algorithms from the Standard Template Library.</p>\r\n<p>You can download the source to Hsa-Bolt from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://github.com/HSA-Libraries/Bolt\">Build from Source</a></li>\r\n</ul>\r\n<p><strong>Thrust</strong></p>\r\n<p>Thrust is a parallel algorithms library providing high level C++ interafce on top of CUDA, OpenMP and TBB. It resembles C++ standard template library and provides accelerated versions of the commonly used algorithms.</p>\r\n<p>Thrust supports NVIDIA GPUs using CUDA and Intel / AMD GPUs using OpenMP and TBB. You can download thrust from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://developer.nvidia.com/cuda-downloads\">From CUDA 7.0</a></li>\r\n<li><a href=\"https://github.com/thrust/thrust\">Header only library</a></li>\r\n</ul>\r\n<p>ArrayFire uses BoostCompute for sorting, set operations and finding regions in image in our CUDA backend.</p>\r\n<p><strong>Intel TBB</strong></p>\r\n<p>Threading Building Blocks (TBB) is a C++ template library developed by intel for multi-core processors. It provides platform independent data structures and algorithms that decrease the complexity of working with multiple threads.</p>\r\n<p>TBB supports multi-core processors (including the Xeon Phi). You can download TBB from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://www.threadingbuildingblocks.org/download\">Source and binaries</a></li>\r\n</ul>\r\n<h3 id=\"reductions\">Reductions</h3>\r\n<p>The reduction algorithm reduces the vector into a single element by successively reducing two input elements to a single output element. The most commonly used operations are addition, minimum and maximum.</p>\r\n<p>It is one of the most commonly used algorithms in scientific and engineering algorithms. It is most commonly used in statistics and machine learning.</p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1374058645&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>For sizes less than 10 million elements, ArrayFire is the fastest followed by HSA-Bolt and BoostCompute</li>\r\n<li>For sizes approaching 100 million elements, HSA-Bolt is 10% faster ArrayFire, and 15% faster than BoostCompute</li>\r\n</ul>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1835396253&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>ArrayFire consistently performs better than other libraries for sizes less than 10 million elements</li>\r\n<li>ArrayFire OpenCL performs better than ArrayFire CUDA for sizes greater than 1 million elements.</li>\r\n<li>BoostCompute consistently performs better than Thrust</li>\r\n<li>For sizes approaching 100 million elements, ArrayFire OpenCL is fastest followed by BoostCompute, Thrust and ArrayFire CUDA</li>\r\n</ul>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=492096912&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>Intel TBB is significantly faster than ArrayFire and BoostCompute for sizes less than 1 million elements</li>\r\n<li>ArrayFire is about 20% to 30% slower for sizes greater than 1 million elements</li>\r\n<li>BoostCompute is about 40% to 70% slower for sizes greater than 1 million elements</li>\r\n</ul>\r\n<h3 id=\"scan\">Scan</h3>\r\n<p>The scan algorithm is useful when performing condition select operations. It can also be used where cumulative distributions are needed.</p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1469250706&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>ArrayFire consistently performs better than other libraries</li>\r\n<li>HSA-Bolt consistently performs better than BoostCompute</li>\r\n</ul>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1909166539&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>ArrayFire performs better than other libraries for sizes less than 1 million elements</li>\r\n<li>ArrayFire OpenCL performs better than ArrayFire CUDA for sizes greater than 100000 elements</li>\r\n<li>Thrust consistently performs better than BoostCompute</li>\r\n<li>For sizes greater than 1 million elements, Thrust is 16% faster than ArrayFire OpenCL, 22% faster than ArrayFire CUDA</li>\r\n</ul>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=944922182&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>Intel TBB is significantly faster than ArrayFire and BoostCompute for all sizes</li>\r\n</ul>\r\n<h3 id=\"transform\">Transform</h3>\r\n<p>The transform operation performs the same operations on each element of the input array. For this benchmark a commonly used function called <code>saxpy</code> has been used.</p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=2057408484&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>ArrayFire consistently performs better than other libraries</li>\r\n<li>BoostCompute consistently performs better than HSA-Bolt</li>\r\n</ul>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=180276342&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>For sizes less than 1 million elements, Thrust is the fastest, followed by ArrayFire CUDA, ArrayFire OpenCL and BoostCompute</li>\r\n<li>For sizes greater than 1 million elements, BoostCompute is fastest followed by ArrayFire OpenCL, Thrust and ArrayFire CUDA</li>\r\n</ul>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=2014239669&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>For sizes less than 1 million elements, TBB is significantly faster than BoostCompute and ArrayFire</li>\r\n<li>For sizes larger than 1 million elements, all three libraries perform within 5% of each other</li>\r\n</ul>\r\n<h4 id=\"conclusion\">Conclusion</h4>\r\n<ul>\r\n<li>ArrayFire is the fastest library on NVIDIA and AMD GPUs for vectors on the order of 10 million elements</li>\r\n<li><p>At even smaller sizes, ArrayFire\'s memory manager ammortizes the memory allocation and deallocation time, making it significantly faster than the alternatives</p></li>\r\n<li>ArrayFire OpenCL is faster than ArrayFire CUDA for all the algorithms for NVIDIA GPUs at large sizes</li>\r\n<li><p>Considering that we use similar optimizations and launch configurations, this is indeed perplexing</p></li>\r\n<li><p>OpenCL on Xeon CPUs can provide comparable performance to TBB for certain algorithms</p></li>\r\n</ul>\r\n<p>Each library performed better than the others at some point in the benchmarks. Considering that all the libraries are open source, these results should provide a motivation for these libraries to learn from each other.</p>\r\n<p>Links: - <a href=\"https://github.com/arrayfire/valhalla\">Code</a> - <a href=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/edit#gid=60225335\">Raw data</a></p>\r\n','Benchmarking parallel vector libraries','','inherit','open','closed','','8337-revision-v1','','','2015-06-10 21:48:22','2015-06-10 21:48:22','',8337,'http://sandbox.arrayfire.com/8337-revision-v1/',0,'revision','',0),(8352,17728,'2015-06-10 21:49:20','2015-06-10 21:49:20','\r\n<p>There are many open source libraries that implement parallel versions of the algorithms in the C++ standard template libraries. Inevitably we get asked questions about how ArrayFire compares to the other libraries out in the open.</p>\r\n<p>In this post we are going to compare the performance of ArrayFire to that of BoostCompute, HSA-Bolt, Intel TBB and Thrust. The benchmarks include the following commonly used vector algorithms across 3 different architectures.</p>\r\n<ul>\r\n<li><a href=\"#reductions\">Reductions</a></li>\r\n<li><a href=\"#scan\">Scan</a></li>\r\n<li><a href=\"#transform\">Transform</a></li>\r\n</ul>\r\n<p>The following setup has been used for the benchmarking purposes. The code to reproduce the benchmarks is linked at the bottom of the post.</p>\r\n<p><strong>Hardware:</strong></p>\r\n<ul>\r\n<li>NVIDIA Tesla K20</li>\r\n<li>AMD FirePro S10000</li>\r\n<li>Intel Xeon E5-2560v2</li>\r\n</ul>\r\n<h3 id=\"background\">Background</h3>\r\n<p><strong>ArrayFire</strong></p>\r\n<p>ArrayFire provides high performance primitives across a <a href=\"http://www.sandbox.arrayfire.com/docs/group__func__categories.htm\">wide spectrum of domains</a>. Objects of the base class, <a href=\"http://www.sandbox.arrayfire.com/docs/classaf_1_1array.htm\">array</a>, can support up to 4 dimensions. It enables users to take advantage of parallelism at multiple levels by <a href=\"http://sandbox.arrayfire.com/how-to-write-vectorized-code/\">writing vectorized code</a>.</p>\r\n<p>ArrayFire supports NVIDIA GPUs using CUDA and OpenCL, AMD GPUs using OpenCL and Intel / AMD CPUs in serial and parallel (using OpenCL) modes. For this post, ArrayFire 3.0 has been used. You can download ArrayFire from the following locations.</p>\r\n<ul>\r\n<li><a href=\"http://sandbox.arrayfire.com/download/\">Installers</a></li>\r\n<li><a href=\"https://github.com/arrayfire/arrayfire\">Build from source</a></li>\r\n</ul>\r\n<p><strong>BoostCompute</strong></p>\r\n<p>Boost Compute is C++ template library for parallel computing based on OpenCL. Boost Compute provides accelerated versions of the algorithms present in the standard template library. It also provides an alternative C++ wrapper over common OpenCL C routines.</p>\r\n<p>Boost Compute supports NVIDIA GPUs, AMD GPUs and Intel / AMD CPUs using OpenCL. You can download BoostCompute from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://github.com/boostorg/compute\">Header only library</a></li>\r\n</ul>\r\n<p>ArrayFire uses BoostCompute for sorting, set operations and finding regions in image in our OpenCL backend.</p>\r\n<p><strong>Hsa-Bolt</strong></p>\r\n<p>Bolt is a C++ template library optimized for heterogenous computing. It supports routines built on top of OpenCL, C++ AMP, and TBB. It provides high performance implementations of common algorithms from the Standard Template Library.</p>\r\n<p>You can download the source to Hsa-Bolt from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://github.com/HSA-Libraries/Bolt\">Build from Source</a></li>\r\n</ul>\r\n<p><strong>Thrust</strong></p>\r\n<p>Thrust is a parallel algorithms library providing high level C++ interafce on top of CUDA, OpenMP and TBB. It resembles C++ standard template library and provides accelerated versions of the commonly used algorithms.</p>\r\n<p>Thrust supports NVIDIA GPUs using CUDA and Intel / AMD GPUs using OpenMP and TBB. You can download thrust from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://developer.nvidia.com/cuda-downloads\">From CUDA 7.0</a></li>\r\n<li><a href=\"https://github.com/thrust/thrust\">Header only library</a></li>\r\n</ul>\r\n<p>ArrayFire uses BoostCompute for sorting, set operations and finding regions in image in our CUDA backend.</p>\r\n<p><strong>Intel TBB</strong></p>\r\n<p>Threading Building Blocks (TBB) is a C++ template library developed by intel for multi-core processors. It provides platform independent data structures and algorithms that decrease the complexity of working with multiple threads.</p>\r\n<p>TBB supports multi-core processors (including the Xeon Phi). You can download TBB from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://www.threadingbuildingblocks.org/download\">Source and binaries</a></li>\r\n</ul>\r\n<h3 id=\"reductions\">Reductions</h3>\r\n<p>The reduction algorithm reduces the vector into a single element by successively reducing two input elements to a single output element. The most commonly used operations are addition, minimum and maximum.</p>\r\n<p>It is one of the most commonly used algorithms in scientific and engineering algorithms. It is most commonly used in statistics and machine learning.</p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1374058645&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>For sizes less than 10 million elements, ArrayFire is the fastest followed by HSA-Bolt and BoostCompute</li>\r\n<li>For sizes approaching 100 million elements, HSA-Bolt is 10% faster ArrayFire, and 15% faster than BoostCompute</li>\r\n</ul>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1835396253&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>ArrayFire consistently performs better than other libraries for sizes less than 10 million elements</li>\r\n<li>ArrayFire OpenCL performs better than ArrayFire CUDA for sizes greater than 1 million elements.</li>\r\n<li>BoostCompute consistently performs better than Thrust</li>\r\n<li>For sizes approaching 100 million elements, ArrayFire OpenCL is fastest followed by BoostCompute, Thrust and ArrayFire CUDA</li>\r\n</ul>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=492096912&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>Intel TBB is significantly faster than ArrayFire and BoostCompute for sizes less than 1 million elements</li>\r\n<li>ArrayFire is about 20% to 30% slower for sizes greater than 1 million elements</li>\r\n<li>BoostCompute is about 40% to 70% slower for sizes greater than 1 million elements</li>\r\n</ul>\r\n<h3 id=\"scan\">Scan</h3>\r\n<p>The scan algorithm is useful when performing condition select operations. It can also be used where cumulative distributions are needed.</p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1469250706&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>ArrayFire consistently performs better than other libraries</li>\r\n<li>HSA-Bolt consistently performs better than BoostCompute</li>\r\n</ul>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1909166539&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>ArrayFire performs better than other libraries for sizes less than 1 million elements</li>\r\n<li>ArrayFire OpenCL performs better than ArrayFire CUDA for sizes greater than 100000 elements</li>\r\n<li>Thrust consistently performs better than BoostCompute</li>\r\n<li>For sizes greater than 1 million elements, Thrust is 16% faster than ArrayFire OpenCL, 22% faster than ArrayFire CUDA</li>\r\n</ul>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=944922182&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>Intel TBB is significantly faster than ArrayFire and BoostCompute for all sizes</li>\r\n</ul>\r\n<h3 id=\"transform\">Transform</h3>\r\n<p>The transform operation performs the same operations on each element of the input array. For this benchmark a commonly used function called <code>saxpy</code> has been used.</p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=2057408484&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>ArrayFire consistently performs better than other libraries</li>\r\n<li>BoostCompute consistently performs better than HSA-Bolt</li>\r\n</ul>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=180276342&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>For sizes less than 1 million elements, Thrust is the fastest, followed by ArrayFire CUDA, ArrayFire OpenCL and BoostCompute</li>\r\n<li>For sizes greater than 1 million elements, BoostCompute is fastest followed by ArrayFire OpenCL, Thrust and ArrayFire CUDA</li>\r\n</ul>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=2014239669&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>For sizes less than 1 million elements, TBB is significantly faster than BoostCompute and ArrayFire</li>\r\n<li>For sizes larger than 1 million elements, all three libraries perform within 5% of each other</li>\r\n</ul>\r\n<h4 id=\"conclusion\">Conclusion</h4>\r\n<ul>\r\n<li>ArrayFire is the fastest library on NVIDIA and AMD GPUs for vectors on the order of 10 million elements</li>\r\n<li><p>At even smaller sizes, ArrayFire\'s memory manager ammortizes the memory allocation and deallocation time, making it significantly faster than the alternatives</p></li>\r\n<li>ArrayFire OpenCL is faster than ArrayFire CUDA for all the algorithms for NVIDIA GPUs at large sizes</li>\r\n<li><p>Considering that we use similar optimizations and launch configurations, this is indeed perplexing</p></li>\r\n<li><p>OpenCL on Xeon CPUs can provide comparable performance to TBB for certain algorithms</p></li>\r\n</ul>\r\n<p>Each library performed better than the others at some point in the benchmarks. Considering that all the libraries are open source, these results should provide a motivation for these libraries to learn from each other.</p>\r\n<p>Links:</p>\r\n<ul>\r\n<li><a href=\"https://github.com/arrayfire/valhalla\">Code</a></li>\r\n<li><a href=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/edit#gid=60225335\">Raw data</a></li>\r\n</ul>\r\n','Benchmarking parallel vector libraries','','inherit','open','closed','','8337-revision-v1','','','2015-06-10 21:49:20','2015-06-10 21:49:20','',8337,'http://sandbox.arrayfire.com/8337-revision-v1/',0,'revision','',0),(8353,17728,'2015-06-10 21:50:21','2015-06-10 21:50:21','\r\n<p>There are many open source libraries that implement parallel versions of the algorithms in the C++ standard template libraries. Inevitably we get asked questions about how ArrayFire compares to the other libraries out in the open.</p>\r\n<p>In this post we are going to compare the performance of ArrayFire to that of BoostCompute, HSA-Bolt, Intel TBB and Thrust. The benchmarks include the following commonly used vector algorithms across 3 different architectures.</p>\r\n<ul>\r\n<li><a href=\"#reductions\">Reductions</a></li>\r\n<li><a href=\"#scan\">Scan</a></li>\r\n<li><a href=\"#transform\">Transform</a></li>\r\n</ul>\r\n<p>The following setup has been used for the benchmarking purposes. The code to reproduce the benchmarks is linked at the bottom of the post.</p>\r\n<p><strong>Hardware:</strong></p>\r\n<ul>\r\n<li>NVIDIA Tesla K20</li>\r\n<li>AMD FirePro S10000</li>\r\n<li>Intel Xeon E5-2560v2</li>\r\n</ul>\r\n<h3 id=\"background\">Background</h3>\r\n<p><strong>ArrayFire</strong></p>\r\n<p>ArrayFire provides high performance primitives across a <a href=\"http://www.sandbox.arrayfire.com/docs/group__func__categories.htm\">wide spectrum of domains</a>. Objects of the base class, <a href=\"http://www.sandbox.arrayfire.com/docs/classaf_1_1array.htm\">array</a>, can support up to 4 dimensions. It enables users to take advantage of parallelism at multiple levels by <a href=\"http://sandbox.arrayfire.com/how-to-write-vectorized-code/\">writing vectorized code</a>.</p>\r\n<p>ArrayFire supports NVIDIA GPUs using CUDA and OpenCL, AMD GPUs using OpenCL and Intel / AMD CPUs in serial and parallel (using OpenCL) modes. For this post, ArrayFire 3.0 has been used. You can download ArrayFire from the following locations.</p>\r\n<ul>\r\n<li><a href=\"http://sandbox.arrayfire.com/download/\">Installers</a></li>\r\n<li><a href=\"https://github.com/arrayfire/arrayfire\">Build from source</a></li>\r\n</ul>\r\n<p><strong>BoostCompute</strong></p>\r\n<p>Boost Compute is C++ template library for parallel computing based on OpenCL. Boost Compute provides accelerated versions of the algorithms present in the standard template library. It also provides an alternative C++ wrapper over common OpenCL C routines.</p>\r\n<p>Boost Compute supports NVIDIA GPUs, AMD GPUs and Intel / AMD CPUs using OpenCL. You can download BoostCompute from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://github.com/boostorg/compute\">Header only library</a></li>\r\n</ul>\r\n<p>ArrayFire uses BoostCompute for sorting, set operations and finding regions in image in our OpenCL backend.</p>\r\n<p><strong>Hsa-Bolt</strong></p>\r\n<p>Bolt is a C++ template library optimized for heterogenous computing. It supports routines built on top of OpenCL, C++ AMP, and TBB. It provides high performance implementations of common algorithms from the Standard Template Library.</p>\r\n<p>You can download the source to Hsa-Bolt from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://github.com/HSA-Libraries/Bolt\">Build from Source</a></li>\r\n</ul>\r\n<p><strong>Thrust</strong></p>\r\n<p>Thrust is a parallel algorithms library providing high level C++ interafce on top of CUDA, OpenMP and TBB. It resembles C++ standard template library and provides accelerated versions of the commonly used algorithms.</p>\r\n<p>Thrust supports NVIDIA GPUs using CUDA and Intel / AMD GPUs using OpenMP and TBB. You can download thrust from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://developer.nvidia.com/cuda-downloads\">From CUDA 7.0</a></li>\r\n<li><a href=\"https://github.com/thrust/thrust\">Header only library</a></li>\r\n</ul>\r\n<p>ArrayFire uses BoostCompute for sorting, set operations and finding regions in image in our CUDA backend.</p>\r\n<p><strong>Intel TBB</strong></p>\r\n<p>Threading Building Blocks (TBB) is a C++ template library developed by intel for multi-core processors. It provides platform independent data structures and algorithms that decrease the complexity of working with multiple threads.</p>\r\n<p>TBB supports multi-core processors (including the Xeon Phi). You can download TBB from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://www.threadingbuildingblocks.org/download\">Source and binaries</a></li>\r\n</ul>\r\n<h3 id=\"reductions\">Reductions</h3>\r\n<p>The reduction algorithm reduces the vector into a single element by successively reducing two input elements to a single output element. The most commonly used operations are addition, minimum and maximum.</p>\r\n<p>It is one of the most commonly used algorithms in scientific and engineering algorithms. It is most commonly used in statistics and machine learning.</p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1374058645&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>For sizes less than 10 million elements, ArrayFire is the fastest followed by HSA-Bolt and BoostCompute</li>\r\n<li>For sizes approaching 100 million elements, HSA-Bolt is 10% faster ArrayFire, and 15% faster than BoostCompute</li>\r\n</ul>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1835396253&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>ArrayFire consistently performs better than other libraries for sizes less than 10 million elements</li>\r\n<li>ArrayFire OpenCL performs better than ArrayFire CUDA for sizes greater than 1 million elements.</li>\r\n<li>BoostCompute consistently performs better than Thrust</li>\r\n<li>For sizes approaching 100 million elements, ArrayFire OpenCL is fastest followed by BoostCompute, Thrust and ArrayFire CUDA</li>\r\n</ul>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=492096912&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>Intel TBB is significantly faster than ArrayFire and BoostCompute for sizes less than 1 million elements</li>\r\n<li>ArrayFire is about 20% to 30% slower for sizes greater than 1 million elements</li>\r\n<li>BoostCompute is about 40% to 70% slower for sizes greater than 1 million elements</li>\r\n</ul>\r\n<h3 id=\"scan\">Scan</h3>\r\n<p>The scan algorithm is useful when performing condition select operations. It can also be used where cumulative distributions are needed.</p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1469250706&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>ArrayFire consistently performs better than other libraries</li>\r\n<li>HSA-Bolt consistently performs better than BoostCompute</li>\r\n</ul>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1909166539&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>ArrayFire performs better than other libraries for sizes less than 1 million elements</li>\r\n<li>ArrayFire OpenCL performs better than ArrayFire CUDA for sizes greater than 100000 elements</li>\r\n<li>Thrust consistently performs better than BoostCompute</li>\r\n<li>For sizes greater than 1 million elements, Thrust is 16% faster than ArrayFire OpenCL, 22% faster than ArrayFire CUDA</li>\r\n</ul>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=944922182&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>Intel TBB is significantly faster than ArrayFire and BoostCompute for all sizes</li>\r\n</ul>\r\n<h3 id=\"transform\">Transform</h3>\r\n<p>The transform operation performs the same operations on each element of the input array. For this benchmark a commonly used function called <code>saxpy</code> has been used.</p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=2057408484&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>ArrayFire consistently performs better than other libraries</li>\r\n<li>BoostCompute consistently performs better than HSA-Bolt</li>\r\n</ul>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=180276342&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>For sizes less than 1 million elements, Thrust is the fastest, followed by ArrayFire CUDA, ArrayFire OpenCL and BoostCompute</li>\r\n<li>For sizes greater than 1 million elements, BoostCompute is fastest followed by ArrayFire OpenCL, Thrust and ArrayFire CUDA</li>\r\n</ul>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=2014239669&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>For sizes less than 1 million elements, TBB is significantly faster than BoostCompute and ArrayFire</li>\r\n<li>For sizes larger than 1 million elements, all three libraries perform within 5% of each other</li>\r\n</ul>\r\n<h3 id=\"conclusion\">Conclusion</h3>\r\n<ul>\r\n<li>ArrayFire is the fastest library on NVIDIA and AMD GPUs for vectors on the order of 10 million elements</li>\r\n<li><p>At even smaller sizes, ArrayFire\'s memory manager ammortizes the memory allocation and deallocation time, making it significantly faster than the alternatives</p></li>\r\n<li>ArrayFire OpenCL is faster than ArrayFire CUDA for all the algorithms for NVIDIA GPUs at large sizes</li>\r\n<li><p>Considering that we use similar optimizations and launch configurations, this is indeed perplexing</p></li>\r\n<li><p>OpenCL on Xeon CPUs can provide comparable performance to TBB for certain algorithms</p></li>\r\n</ul>\r\n<p>Each library performed better than the others at some point in the benchmarks. Considering that all the libraries are open source, these results should provide a motivation for these libraries to learn from each other.</p>\r\n<h3 id=\"links\">Links:</h3>\r\n<ul>\r\n<li><a href=\"https://github.com/arrayfire/valhalla\">Code</a></li>\r\n<li><a href=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/edit#gid=60225335\">Raw data</a></li>\r\n</ul>\r\n','Benchmarking parallel vector libraries','','inherit','open','closed','','8337-revision-v1','','','2015-06-10 21:50:21','2015-06-10 21:50:21','',8337,'http://sandbox.arrayfire.com/8337-revision-v1/',0,'revision','',0),(8354,17728,'2015-06-10 21:51:36','2015-06-10 21:51:36','\r\n<p>There are many open source libraries that implement parallel versions of the algorithms in the C++ standard template libraries. Inevitably we get asked questions about how ArrayFire compares to the other libraries out in the open.</p>\r\n<p>In this post we are going to compare the performance of ArrayFire to that of BoostCompute, HSA-Bolt, Intel TBB and Thrust. The benchmarks include the following commonly used vector algorithms across 3 different architectures.</p>\r\n<ul>\r\n<li><a href=\"#reductions\">Reductions</a></li>\r\n<li><a href=\"#scan\">Scan</a></li>\r\n<li><a href=\"#transform\">Transform</a></li>\r\n</ul>\r\n<p>The following setup has been used for the benchmarking purposes. The code to reproduce the benchmarks is linked at the bottom of the post. The hardware used for the benchmarks is listed below:</p>\r\n<ul>\r\n<li>NVIDIA Tesla K20</li>\r\n<li>AMD FirePro S10000</li>\r\n<li>Intel Xeon E5-2560v2</li>\r\n</ul>\r\n<h3 id=\"background\">Background</h3>\r\n<p><strong>ArrayFire</strong></p>\r\n<p>ArrayFire provides high performance primitives across a <a href=\"http://www.sandbox.arrayfire.com/docs/group__func__categories.htm\">wide spectrum of domains</a>. Objects of the base class, <a href=\"http://www.sandbox.arrayfire.com/docs/classaf_1_1array.htm\">array</a>, can support up to 4 dimensions. It enables users to take advantage of parallelism at multiple levels by <a href=\"http://sandbox.arrayfire.com/how-to-write-vectorized-code/\">writing vectorized code</a>.</p>\r\n<p>ArrayFire supports NVIDIA GPUs using CUDA and OpenCL, AMD GPUs using OpenCL and Intel / AMD CPUs in serial and parallel (using OpenCL) modes. For this post, ArrayFire 3.0 has been used. You can download ArrayFire from the following locations.</p>\r\n<ul>\r\n<li><a href=\"http://sandbox.arrayfire.com/download/\">Installers</a></li>\r\n<li><a href=\"https://github.com/arrayfire/arrayfire\">Build from source</a></li>\r\n</ul>\r\n<p><strong>BoostCompute</strong></p>\r\n<p>Boost Compute is C++ template library for parallel computing based on OpenCL. Boost Compute provides accelerated versions of the algorithms present in the standard template library. It also provides an alternative C++ wrapper over common OpenCL C routines.</p>\r\n<p>Boost Compute supports NVIDIA GPUs, AMD GPUs and Intel / AMD CPUs using OpenCL. You can download BoostCompute from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://github.com/boostorg/compute\">Header only library</a></li>\r\n</ul>\r\n<p>ArrayFire uses BoostCompute for sorting, set operations and finding regions in image in our OpenCL backend.</p>\r\n<p><strong>Hsa-Bolt</strong></p>\r\n<p>Bolt is a C++ template library optimized for heterogenous computing. It supports routines built on top of OpenCL, C++ AMP, and TBB. It provides high performance implementations of common algorithms from the Standard Template Library.</p>\r\n<p>You can download the source to Hsa-Bolt from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://github.com/HSA-Libraries/Bolt\">Build from Source</a></li>\r\n</ul>\r\n<p><strong>Thrust</strong></p>\r\n<p>Thrust is a parallel algorithms library providing high level C++ interafce on top of CUDA, OpenMP and TBB. It resembles C++ standard template library and provides accelerated versions of the commonly used algorithms.</p>\r\n<p>Thrust supports NVIDIA GPUs using CUDA and Intel / AMD GPUs using OpenMP and TBB. You can download thrust from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://developer.nvidia.com/cuda-downloads\">From CUDA 7.0</a></li>\r\n<li><a href=\"https://github.com/thrust/thrust\">Header only library</a></li>\r\n</ul>\r\n<p>ArrayFire uses BoostCompute for sorting, set operations and finding regions in image in our CUDA backend.</p>\r\n<p><strong>Intel TBB</strong></p>\r\n<p>Threading Building Blocks (TBB) is a C++ template library developed by intel for multi-core processors. It provides platform independent data structures and algorithms that decrease the complexity of working with multiple threads.</p>\r\n<p>TBB supports multi-core processors (including the Xeon Phi). You can download TBB from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://www.threadingbuildingblocks.org/download\">Source and binaries</a></li>\r\n</ul>\r\n<h3 id=\"reductions\">Reductions</h3>\r\n<p>The reduction algorithm reduces the vector into a single element by successively reducing two input elements to a single output element. The most commonly used operations are addition, minimum and maximum.</p>\r\n<p>It is one of the most commonly used algorithms in scientific and engineering algorithms. It is most commonly used in statistics and machine learning.</p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1374058645&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>For sizes less than 10 million elements, ArrayFire is the fastest followed by HSA-Bolt and BoostCompute</li>\r\n<li>For sizes approaching 100 million elements, HSA-Bolt is 10% faster ArrayFire, and 15% faster than BoostCompute</li>\r\n</ul>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1835396253&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>ArrayFire consistently performs better than other libraries for sizes less than 10 million elements</li>\r\n<li>ArrayFire OpenCL performs better than ArrayFire CUDA for sizes greater than 1 million elements.</li>\r\n<li>BoostCompute consistently performs better than Thrust</li>\r\n<li>For sizes approaching 100 million elements, ArrayFire OpenCL is fastest followed by BoostCompute, Thrust and ArrayFire CUDA</li>\r\n</ul>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=492096912&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>Intel TBB is significantly faster than ArrayFire and BoostCompute for sizes less than 1 million elements</li>\r\n<li>ArrayFire is about 20% to 30% slower for sizes greater than 1 million elements</li>\r\n<li>BoostCompute is about 40% to 70% slower for sizes greater than 1 million elements</li>\r\n</ul>\r\n<h3 id=\"scan\">Scan</h3>\r\n<p>The scan algorithm is useful when performing condition select operations. It can also be used where cumulative distributions are needed.</p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1469250706&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>ArrayFire consistently performs better than other libraries</li>\r\n<li>HSA-Bolt consistently performs better than BoostCompute</li>\r\n</ul>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1909166539&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>ArrayFire performs better than other libraries for sizes less than 1 million elements</li>\r\n<li>ArrayFire OpenCL performs better than ArrayFire CUDA for sizes greater than 100000 elements</li>\r\n<li>Thrust consistently performs better than BoostCompute</li>\r\n<li>For sizes greater than 1 million elements, Thrust is 16% faster than ArrayFire OpenCL, 22% faster than ArrayFire CUDA</li>\r\n</ul>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=944922182&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>Intel TBB is significantly faster than ArrayFire and BoostCompute for all sizes</li>\r\n</ul>\r\n<h3 id=\"transform\">Transform</h3>\r\n<p>The transform operation performs the same operations on each element of the input array. For this benchmark a commonly used function called <code>saxpy</code> has been used.</p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=2057408484&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>ArrayFire consistently performs better than other libraries</li>\r\n<li>BoostCompute consistently performs better than HSA-Bolt</li>\r\n</ul>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=180276342&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>For sizes less than 1 million elements, Thrust is the fastest, followed by ArrayFire CUDA, ArrayFire OpenCL and BoostCompute</li>\r\n<li>For sizes greater than 1 million elements, BoostCompute is fastest followed by ArrayFire OpenCL, Thrust and ArrayFire CUDA</li>\r\n</ul>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=2014239669&amp;format=interactive\">\r\n</iframe>\r\n<ul>\r\n<li>For sizes less than 1 million elements, TBB is significantly faster than BoostCompute and ArrayFire</li>\r\n<li>For sizes larger than 1 million elements, all three libraries perform within 5% of each other</li>\r\n</ul>\r\n<h3 id=\"conclusion\">Conclusion</h3>\r\n<ul>\r\n<li>ArrayFire is the fastest library on NVIDIA and AMD GPUs for vectors on the order of 10 million elements</li>\r\n<li><p>At even smaller sizes, ArrayFire\'s memory manager ammortizes the memory allocation and deallocation time, making it significantly faster than the alternatives</p></li>\r\n<li>ArrayFire OpenCL is faster than ArrayFire CUDA for all the algorithms for NVIDIA GPUs at large sizes</li>\r\n<li><p>Considering that we use similar optimizations and launch configurations, this is indeed perplexing</p></li>\r\n<li><p>OpenCL on Xeon CPUs can provide comparable performance to TBB for certain algorithms</p></li>\r\n</ul>\r\n<p>Each library performed better than the others at some point in the benchmarks. Considering that all the libraries are open source, these results should provide a motivation for these libraries to learn from each other.</p>\r\n<h3 id=\"links\">Links:</h3>\r\n<ul>\r\n<li><a href=\"https://github.com/arrayfire/valhalla\">Code</a></li>\r\n<li><a href=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/edit#gid=60225335\">Raw data</a></li>\r\n</ul>','Benchmarking parallel vector libraries','','inherit','open','closed','','8337-revision-v1','','','2015-06-10 21:51:36','2015-06-10 21:51:36','',8337,'http://sandbox.arrayfire.com/8337-revision-v1/',0,'revision','',0),(8355,17728,'2015-06-10 22:21:36','2015-06-10 22:21:36','\r\n<p>There are many open source libraries that implement parallel versions of the algorithms in the C++ standard template libraries. Inevitably we get asked questions about how ArrayFire compares to the other libraries out in the open.</p>\r\n<p>In this post we are going to compare the performance of ArrayFire to that of BoostCompute, HSA-Bolt, Intel TBB and Thrust. The benchmarks include the following commonly used vector algorithms across 3 different architectures.</p>\r\n<ul>\r\n<li><a href=\"#reductions\">Reductions</a></li>\r\n<li><a href=\"#scan\">Scan</a></li>\r\n<li><a href=\"#transform\">Transform</a></li>\r\n</ul>\r\n<p>The following setup has been used for the benchmarking purposes. The code to reproduce the benchmarks is linked at the bottom of the post. The hardware used for the benchmarks is listed below:</p>\r\n<ul>\r\n<li>NVIDIA Tesla K20</li>\r\n<li>AMD FirePro S10000</li>\r\n<li>Intel Xeon E5-2560v2</li>\r\n</ul>\r\n<h3 id=\"background\">Background</h3>\r\n<p><strong>ArrayFire</strong></p>\r\n<p>ArrayFire provides high performance primitives across a <a href=\"http://www.sandbox.arrayfire.com/docs/group__func__categories.htm\">wide spectrum of domains</a>. Objects of the base class, <a href=\"http://www.sandbox.arrayfire.com/docs/classaf_1_1array.htm\">array</a>, can support up to 4 dimensions. It enables users to take advantage of parallelism at multiple levels by <a href=\"http://sandbox.arrayfire.com/how-to-write-vectorized-code/\">writing vectorized code</a>.</p>\r\n<p>ArrayFire supports NVIDIA GPUs using CUDA and OpenCL, AMD GPUs using OpenCL and Intel / AMD CPUs in serial and parallel (using OpenCL) modes. For this post, ArrayFire 3.0 has been used. You can download ArrayFire from the following locations.</p>\r\n<ul>\r\n<li><a href=\"http://sandbox.arrayfire.com/download/\">Installers</a></li>\r\n<li><a href=\"https://github.com/arrayfire/arrayfire\">Build from source</a></li>\r\n</ul>\r\n<p><strong>BoostCompute</strong></p>\r\n<p>Boost Compute is C++ template library for parallel computing based on OpenCL. Boost Compute provides accelerated versions of the algorithms present in the standard template library. It also provides an alternative C++ wrapper over common OpenCL C routines.</p>\r\n<p>Boost Compute supports NVIDIA GPUs, AMD GPUs and Intel / AMD CPUs using OpenCL. You can download BoostCompute from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://github.com/boostorg/compute\">Header only library</a></li>\r\n</ul>\r\n<p>ArrayFire uses BoostCompute for sorting, set operations and finding regions in image in our OpenCL backend.</p>\r\n<p><strong>Hsa-Bolt</strong></p>\r\n<p>Bolt is a C++ template library optimized for heterogenous computing. It supports routines built on top of OpenCL, C++ AMP, and TBB. It provides high performance implementations of common algorithms from the Standard Template Library.</p>\r\n<p>You can download the source to Hsa-Bolt from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://github.com/HSA-Libraries/Bolt\">Build from Source</a></li>\r\n</ul>\r\n<p><strong>Thrust</strong></p>\r\n<p>Thrust is a parallel algorithms library providing high level C++ interafce on top of CUDA, OpenMP and TBB. It resembles C++ standard template library and provides accelerated versions of the commonly used algorithms.</p>\r\n<p>Thrust supports NVIDIA GPUs using CUDA and Intel / AMD GPUs using OpenMP and TBB. You can download thrust from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://developer.nvidia.com/cuda-downloads\">From CUDA 7.0</a></li>\r\n<li><a href=\"https://github.com/thrust/thrust\">Header only library</a></li>\r\n</ul>\r\n<p>ArrayFire uses BoostCompute for sorting, set operations and finding regions in image in our CUDA backend.</p>\r\n<p><strong>Intel TBB</strong></p>\r\n<p>Threading Building Blocks (TBB) is a C++ template library developed by intel for multi-core processors. It provides platform independent data structures and algorithms that decrease the complexity of working with multiple threads.</p>\r\n<p>TBB supports multi-core processors (including the Xeon Phi). You can download TBB from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://www.threadingbuildingblocks.org/download\">Source and binaries</a></li>\r\n</ul>\r\n<h3 id=\"reductions\">Reductions</h3>\r\n<p>The reduction algorithm reduces the vector into a single element by successively reducing two input elements to a single output element. The most commonly used operations are addition, minimum and maximum.</p>\r\n<p>It is one of the most commonly used algorithms in scientific and engineering algorithms. Most notably in the fields of statistics and machine learning.</p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1374058645&amp;format=interactive\">\r\n</iframe>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1835396253&amp;format=interactive\">\r\n</iframe>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=492096912&amp;format=interactive\">\r\n</iframe>\r\n<p><strong>AMD FirePro S10000</strong></p>\r\n<ul>\r\n<li>For sizes less than 10 million elements, ArrayFire is the fastest followed by HSA-Bolt and BoostComput</li>\r\n<li>For sizes approaching 100 million elements, HSA-Bolt is 10% faster ArrayFire, and 15% faster than BoostCompute</li>\r\n</ul>\r\n<p><strong>NVIDIA Tesla K20</strong></p>\r\n<ul>\r\n<li>ArrayFire consistently performs better than other libraries for sizes less than 10 million elements</li>\r\n<li>ArrayFire OpenCL performs better than ArrayFire CUDA for sizes greater than 1 million elements.</li>\r\n<li>BoostCompute consistently performs better than Thrust</li>\r\n<li>For sizes approaching 100 million elements, ArrayFire OpenCL is fastest followed by BoostCompute, Thrust and ArrayFire CUDA</li>\r\n</ul>\r\n<p><strong>Intel Xeon E5-2560v2</strong></p>\r\n<ul>\r\n<li>Intel TBB is significantly faster than ArrayFire and BoostCompute for sizes less than 1 million elements</li>\r\n<li>ArrayFire is about 20% to 30% slower for sizes greater than 1 million elements</li>\r\n<li>BoostCompute is about 40% to 70% slower for sizes greater than 1 million elements</li>\r\n</ul>\r\n<h3 id=\"scan\">Scan</h3>\r\n<p>The scan algorithm is useful when performing condition select operations. It can also be used where cumulative distributions are needed.</p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1469250706&amp;format=interactive\">\r\n</iframe>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1909166539&amp;format=interactive\">\r\n</iframe>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=944922182&amp;format=interactive\">\r\n</iframe>\r\n<p><strong>AMD FirePro S10000</strong></p>\r\n<ul>\r\n<li>ArrayFire consistently performs better than other libraries</li>\r\n<li>HSA-Bolt consistently performs better than BoostCompute</li>\r\n</ul>\r\n<p>** NVIDIA Tesla K20**</p>\r\n<ul>\r\n<li>ArrayFire performs better than other libraries for sizes less than 1 million elements</li>\r\n<li>ArrayFire OpenCL performs better than ArrayFire CUDA for sizes greater than 100000 elements</li>\r\n<li>Thrust consistently performs better than BoostCompute</li>\r\n<li>For sizes greater than 1 million elements, Thrust is 16% faster than ArrayFire OpenCL, 22% faster than ArrayFire CUDA</li>\r\n</ul>\r\n<p><strong>Intel Xeon E5-2560v2</strong></p>\r\n<ul>\r\n<li>Intel TBB is significantly faster than ArrayFire and BoostCompute for all sizes</li>\r\n</ul>\r\n<h3 id=\"transform\">Transform</h3>\r\n<p>The transform operation performs the same operations on each element of the input array. For this benchmark a commonly used function called <code>saxpy</code> has been used.</p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=2057408484&amp;format=interactive\">\r\n</iframe>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=180276342&amp;format=interactive\">\r\n</iframe>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=2014239669&amp;format=interactive\">\r\n</iframe>\r\n<p><strong>AMD FirePro S10000</strong></p>\r\n<ul>\r\n<li>ArrayFire consistently performs better than other libraries</li>\r\n<li>BoostCompute consistently performs better than HSA-Bolt</li>\r\n</ul>\r\n<p><strong>NVIDIA Tesla K20</strong></p>\r\n<ul>\r\n<li>For sizes less than 1 million elements, Thrust is the fastest, followed by ArrayFire CUDA, ArrayFire OpenCL and BoostCompute</li>\r\n<li>For sizes greater than 1 million elements, BoostCompute is fastest followed by ArrayFire OpenCL, Thrust and ArrayFire CUDA</li>\r\n</ul>\r\n<p><strong>Intel Xeon E5-2560v2</strong></p>\r\n<ul>\r\n<li>For sizes less than 1 million elements, TBB is significantly faster than BoostCompute and ArrayFire</li>\r\n<li>For sizes larger than 1 million elements, all three libraries perform within 5% of each other</li>\r\n</ul>\r\n<h3 id=\"conclusion\">Conclusion</h3>\r\n<p>On AMD and NVIDIA GPUs, ArrayFire is the fastest library for vectors on the order of 10 million elements. At even smaller sizes, ArrayFire\'s memory manager ammortizes the memory allocation and deallocation time, making it significantly faster than the alternatives. While improvements can be made overall, one particular area that can be improved upon would be the scan algorithm at large sizes.</p>\r\n<p>On Intel Xeons, TBB is significantly faster than OpenCL implementations. Improvements can and should be made in ArrayFire to extract the best performance out of multicore CPUs. Our roadmap for the next quarter includes a native parallel implementation for our CPU backend. This should bridge the performance gap seen in these benchmarks.</p>\r\n<p>Each library performed better than the others at some point in the benchmarks. Considering that all the libraries are open source, these results should provide a motivation for these libraries to learn from each other.</p>\r\n<h3 id=\"links\">Links:</h3>\r\n<ul>\r\n<li><a href=\"https://github.com/arrayfire/valhalla\">Code</a></li>\r\n<li><a href=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/edit#gid=60225335\">Raw data</a></li>\r\n</ul>\r\n','Benchmarking parallel vector libraries','','inherit','open','closed','','8337-revision-v1','','','2015-06-10 22:21:36','2015-06-10 22:21:36','',8337,'http://sandbox.arrayfire.com/8337-revision-v1/',0,'revision','',0),(8356,17728,'2015-06-10 22:22:40','2015-06-10 22:22:40','\r\n<p>There are many open source libraries that implement parallel versions of the algorithms in the C++ standard template libraries. Inevitably we get asked questions about how ArrayFire compares to the other libraries out in the open.</p>\r\n<p>In this post we are going to compare the performance of ArrayFire to that of BoostCompute, HSA-Bolt, Intel TBB and Thrust. The benchmarks include the following commonly used vector algorithms across 3 different architectures.</p>\r\n<ul>\r\n<li><a href=\"#reductions\">Reductions</a></li>\r\n<li><a href=\"#scan\">Scan</a></li>\r\n<li><a href=\"#transform\">Transform</a></li>\r\n</ul>\r\n<p>The following setup has been used for the benchmarking purposes. The code to reproduce the benchmarks is linked at the bottom of the post. The hardware used for the benchmarks is listed below:</p>\r\n<ul>\r\n<li>NVIDIA Tesla K20</li>\r\n<li>AMD FirePro S10000</li>\r\n<li>Intel Xeon E5-2560v2</li>\r\n</ul>\r\n<h3 id=\"background\">Background</h3>\r\n<p><strong>ArrayFire</strong></p>\r\n<p>ArrayFire provides high performance primitives across a <a href=\"http://www.sandbox.arrayfire.com/docs/group__func__categories.htm\">wide spectrum of domains</a>. Objects of the base class, <a href=\"http://www.sandbox.arrayfire.com/docs/classaf_1_1array.htm\">array</a>, can support up to 4 dimensions. It enables users to take advantage of parallelism at multiple levels by <a href=\"http://sandbox.arrayfire.com/how-to-write-vectorized-code/\">writing vectorized code</a>.</p>\r\n<p>ArrayFire supports NVIDIA GPUs using CUDA and OpenCL, AMD GPUs using OpenCL and Intel / AMD CPUs in serial and parallel (using OpenCL) modes. For this post, ArrayFire 3.0 has been used. You can download ArrayFire from the following locations.</p>\r\n<ul>\r\n<li><a href=\"http://sandbox.arrayfire.com/download/\">Installers</a></li>\r\n<li><a href=\"https://github.com/arrayfire/arrayfire\">Build from source</a></li>\r\n</ul>\r\n<p><strong>BoostCompute</strong></p>\r\n<p>Boost Compute is C++ template library for parallel computing based on OpenCL. Boost Compute provides accelerated versions of the algorithms present in the standard template library. It also provides an alternative C++ wrapper over common OpenCL C routines.</p>\r\n<p>Boost Compute supports NVIDIA GPUs, AMD GPUs and Intel / AMD CPUs using OpenCL. You can download BoostCompute from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://github.com/boostorg/compute\">Header only library</a></li>\r\n</ul>\r\n<p>ArrayFire uses BoostCompute for sorting, set operations and finding regions in image in our OpenCL backend.</p>\r\n<p><strong>Hsa-Bolt</strong></p>\r\n<p>Bolt is a C++ template library optimized for heterogenous computing. It supports routines built on top of OpenCL, C++ AMP, and TBB. It provides high performance implementations of common algorithms from the Standard Template Library.</p>\r\n<p>You can download the source to Hsa-Bolt from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://github.com/HSA-Libraries/Bolt\">Build from Source</a></li>\r\n</ul>\r\n<p><strong>Thrust</strong></p>\r\n<p>Thrust is a parallel algorithms library providing high level C++ interafce on top of CUDA, OpenMP and TBB. It resembles C++ standard template library and provides accelerated versions of the commonly used algorithms.</p>\r\n<p>Thrust supports NVIDIA GPUs using CUDA and Intel / AMD GPUs using OpenMP and TBB. You can download thrust from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://developer.nvidia.com/cuda-downloads\">From CUDA 7.0</a></li>\r\n<li><a href=\"https://github.com/thrust/thrust\">Header only library</a></li>\r\n</ul>\r\n<p>ArrayFire uses BoostCompute for sorting, set operations and finding regions in image in our CUDA backend.</p>\r\n<p><strong>Intel TBB</strong></p>\r\n<p>Threading Building Blocks (TBB) is a C++ template library developed by intel for multi-core processors. It provides platform independent data structures and algorithms that decrease the complexity of working with multiple threads.</p>\r\n<p>TBB supports multi-core processors (including the Xeon Phi). You can download TBB from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://www.threadingbuildingblocks.org/download\">Source and binaries</a></li>\r\n</ul>\r\n<h3 id=\"reductions\">Reductions</h3>\r\n<p>The reduction algorithm reduces the vector into a single element by successively reducing two input elements to a single output element. The most commonly used operations are addition, minimum and maximum.</p>\r\n<p>It is one of the most commonly used algorithms in scientific and engineering algorithms. Most notably in the fields of statistics and machine learning.</p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1374058645&amp;format=interactive\">\r\n</iframe>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1835396253&amp;format=interactive\">\r\n</iframe>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=492096912&amp;format=interactive\">\r\n</iframe>\r\n<p><strong>AMD FirePro S10000</strong></p>\r\n<ul>\r\n<li>For sizes less than 10 million elements, ArrayFire is the fastest followed by HSA-Bolt and BoostComput</li>\r\n<li>For sizes approaching 100 million elements, HSA-Bolt is 10% faster ArrayFire, and 15% faster than BoostCompute</li>\r\n</ul>\r\n<p><strong>NVIDIA Tesla K20</strong></p>\r\n<ul>\r\n<li>ArrayFire consistently performs better than other libraries for sizes less than 10 million elements</li>\r\n<li>ArrayFire OpenCL performs better than ArrayFire CUDA for sizes greater than 1 million elements.</li>\r\n<li>BoostCompute consistently performs better than Thrust</li>\r\n<li>For sizes approaching 100 million elements, ArrayFire OpenCL is fastest followed by BoostCompute, Thrust and ArrayFire CUDA</li>\r\n</ul>\r\n<p><strong>Intel Xeon E5-2560v2</strong></p>\r\n<ul>\r\n<li>Intel TBB is significantly faster than ArrayFire and BoostCompute for sizes less than 1 million elements</li>\r\n<li>ArrayFire is about 20% to 30% slower for sizes greater than 1 million elements</li>\r\n<li>BoostCompute is about 40% to 70% slower for sizes greater than 1 million elements</li>\r\n</ul>\r\n<h3 id=\"scan\">Scan</h3>\r\n<p>The scan algorithm is useful when performing condition select operations. It can also be used where cumulative distributions are needed.</p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1469250706&amp;format=interactive\">\r\n</iframe>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1909166539&amp;format=interactive\">\r\n</iframe>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=944922182&amp;format=interactive\">\r\n</iframe>\r\n<p><strong>AMD FirePro S10000</strong></p>\r\n<ul>\r\n<li>ArrayFire consistently performs better than other libraries</li>\r\n<li>HSA-Bolt consistently performs better than BoostCompute</li>\r\n</ul>\r\n<p><strong>NVIDIA Tesla K20</strong></p>\r\n<ul>\r\n<li>ArrayFire performs better than other libraries for sizes less than 1 million elements</li>\r\n<li>ArrayFire OpenCL performs better than ArrayFire CUDA for sizes greater than 100000 elements</li>\r\n<li>Thrust consistently performs better than BoostCompute</li>\r\n<li>For sizes greater than 1 million elements, Thrust is 16% faster than ArrayFire OpenCL, 22% faster than ArrayFire CUDA</li>\r\n</ul>\r\n<p><strong>Intel Xeon E5-2560v2</strong></p>\r\n<ul>\r\n<li>Intel TBB is significantly faster than ArrayFire and BoostCompute for all sizes</li>\r\n</ul>\r\n<h3 id=\"transform\">Transform</h3>\r\n<p>The transform operation performs the same operations on each element of the input array. For this benchmark a commonly used function called <code>saxpy</code> has been used.</p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=2057408484&amp;format=interactive\">\r\n</iframe>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=180276342&amp;format=interactive\">\r\n</iframe>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=2014239669&amp;format=interactive\">\r\n</iframe>\r\n<p><strong>AMD FirePro S10000</strong></p>\r\n<ul>\r\n<li>ArrayFire consistently performs better than other libraries</li>\r\n<li>BoostCompute consistently performs better than HSA-Bolt</li>\r\n</ul>\r\n<p><strong>NVIDIA Tesla K20</strong></p>\r\n<ul>\r\n<li>For sizes less than 1 million elements, Thrust is the fastest, followed by ArrayFire CUDA, ArrayFire OpenCL and BoostCompute</li>\r\n<li>For sizes greater than 1 million elements, BoostCompute is fastest followed by ArrayFire OpenCL, Thrust and ArrayFire CUDA</li>\r\n</ul>\r\n<p><strong>Intel Xeon E5-2560v2</strong></p>\r\n<ul>\r\n<li>For sizes less than 1 million elements, TBB is significantly faster than BoostCompute and ArrayFire</li>\r\n<li>For sizes larger than 1 million elements, all three libraries perform within 5% of each other</li>\r\n</ul>\r\n<h3 id=\"conclusion\">Conclusion</h3>\r\n<p>On AMD and NVIDIA GPUs, ArrayFire is the fastest library for vectors on the order of 10 million elements. At even smaller sizes, ArrayFire\'s memory manager ammortizes the memory allocation and deallocation time, making it significantly faster than the alternatives. While improvements can be made overall, one particular area that can be improved upon would be the scan algorithm at large sizes.</p>\r\n<p>On Intel Xeons, TBB is significantly faster than OpenCL implementations. Improvements can and should be made in ArrayFire to extract the best performance out of multicore CPUs. Our roadmap for the next quarter includes a native parallel implementation for our CPU backend. This should bridge the performance gap seen in these benchmarks.</p>\r\n<p>Each library performed better than the others at some point in the benchmarks. Considering that all the libraries are open source, these results should provide a motivation for these libraries to learn from each other.</p>\r\n<h3 id=\"links\">Links:</h3>\r\n<ul>\r\n<li><a href=\"https://github.com/arrayfire/valhalla\">Code</a></li>\r\n<li><a href=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/edit#gid=60225335\">Raw data</a></li>\r\n</ul>','Benchmarking parallel vector libraries','','inherit','open','closed','','8337-revision-v1','','','2015-06-10 22:22:40','2015-06-10 22:22:40','',8337,'http://sandbox.arrayfire.com/8337-revision-v1/',0,'revision','',0),(8357,17728,'2015-06-11 14:39:40','2015-06-11 14:39:40','<p>There are many open source libraries that implement parallel versions of the algorithms in the C++ standard template libraries. Inevitably we get asked questions about how ArrayFire compares to the other libraries out in the open.</p>\r\n<p>In this post we are going to compare the performance of ArrayFire to that of BoostCompute, HSA-Bolt, Intel TBB and Thrust. The benchmarks include the following commonly used vector algorithms across 3 different architectures.</p>\r\n<ul>\r\n<li><a href=\"#reductions\">Reductions</a></li>\r\n<li><a href=\"#scan\">Scan</a></li>\r\n<li><a href=\"#transform\">Transform</a></li>\r\n</ul>\r\n<p>The following setup has been used for the benchmarking purposes. The code to reproduce the benchmarks is linked at the bottom of the post. The hardware used for the benchmarks is listed below:</p>\r\n<ul>\r\n<li>NVIDIA Tesla K20</li>\r\n<li>AMD FirePro S10000</li>\r\n<li>Intel Xeon E5-2560v2</li>\r\n</ul>\r\n<h3 id=\"background\">Background</h3>\r\n<p><strong>ArrayFire</strong></p>\r\n<p>ArrayFire provides high performance primitives across a <a href=\"http://www.sandbox.arrayfire.com/docs/group__func__categories.htm\">wide spectrum of domains</a>. Objects of the base class, <a href=\"http://www.sandbox.arrayfire.com/docs/classaf_1_1array.htm\">array</a>, can support up to 4 dimensions. It enables users to take advantage of parallelism at multiple levels by <a href=\"http://sandbox.arrayfire.com/how-to-write-vectorized-code/\">writing vectorized code</a>.</p>\r\n<p>ArrayFire supports NVIDIA GPUs using CUDA and OpenCL, AMD GPUs using OpenCL and Intel / AMD CPUs in serial and parallel (using OpenCL) modes. For this post, ArrayFire 3.0 has been used. You can download ArrayFire from the following locations.</p>\r\n<ul>\r\n<li><a href=\"http://sandbox.arrayfire.com/download/\">Installers</a></li>\r\n<li><a href=\"https://github.com/arrayfire/arrayfire\">Build from source</a></li>\r\n</ul>\r\n<p><strong>BoostCompute</strong></p>\r\n<p>Boost Compute is C++ template library for parallel computing based on OpenCL. Boost Compute provides accelerated versions of the algorithms present in the standard template library. It also provides an alternative C++ wrapper over common OpenCL C routines.</p>\r\n<p>Boost Compute supports NVIDIA GPUs, AMD GPUs and Intel / AMD CPUs using OpenCL. You can download BoostCompute from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://github.com/boostorg/compute\">Header only library</a></li>\r\n</ul>\r\n<p>ArrayFire uses BoostCompute for sorting, set operations and finding regions in an image in our OpenCL backend.</p>\r\n<p><strong>Hsa-Bolt</strong></p>\r\n<p>Bolt is a C++ template library optimized for heterogenous computing. It supports routines built on top of OpenCL, C++ AMP, and TBB. It provides high performance implementations of common algorithms from the Standard Template Library.</p>\r\n<p>You can download the source to Hsa-Bolt from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://github.com/HSA-Libraries/Bolt\">Build from Source</a></li>\r\n</ul>\r\n<p><strong>Thrust</strong></p>\r\n<p>Thrust is a parallel algorithms library providing high level C++ interafce on top of CUDA, OpenMP and TBB. It resembles C++ standard template library and provides accelerated versions of the commonly used algorithms.</p>\r\n<p>Thrust supports NVIDIA GPUs using CUDA and Intel / AMD GPUs using OpenMP and TBB. You can download thrust from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://developer.nvidia.com/cuda-downloads\">From CUDA 7.0</a></li>\r\n<li><a href=\"https://github.com/thrust/thrust\">Header only library</a></li>\r\n</ul>\r\n<p>ArrayFire uses BoostCompute for sorting, set operations and finding regions in an image in our CUDA backend.</p>\r\n<p><strong>Intel TBB</strong></p>\r\n<p>Threading Building Blocks (TBB) is a C++ template library developed by intel for multi-core processors. It provides platform independent data structures and algorithms that decrease the complexity of working with multiple threads.</p>\r\n<p>TBB supports multi-core processors (including the Xeon Phi). You can download TBB from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://www.threadingbuildingblocks.org/download\">Source and binaries</a></li>\r\n</ul>\r\n<h3 id=\"reductions\">Reductions</h3>\r\n<p>The reduction algorithm reduces the vector into a single element by successively reducing two input elements to a single output element. The most commonly used operations are addition, minimum and maximum.</p>\r\n<p>It is one of the most commonly used algorithms in scientific and engineering algorithms. Most notably in the fields of statistics and machine learning.</p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1374058645&amp;format=interactive\">\r\n</iframe>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1835396253&amp;format=interactive\">\r\n</iframe>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=492096912&amp;format=interactive\">\r\n</iframe>\r\n<p><strong>AMD FirePro S10000</strong></p>\r\n<ul>\r\n<li>For sizes less than 10 million elements, ArrayFire is the fastest followed by HSA-Bolt and BoostComput</li>\r\n<li>For sizes approaching 100 million elements, HSA-Bolt is 10% faster ArrayFire, and 15% faster than BoostCompute</li>\r\n</ul>\r\n<p><strong>NVIDIA Tesla K20</strong></p>\r\n<ul>\r\n<li>ArrayFire consistently performs better than other libraries for sizes less than 10 million elements</li>\r\n<li>ArrayFire OpenCL performs better than ArrayFire CUDA for sizes greater than 1 million elements.</li>\r\n<li>BoostCompute consistently performs better than Thrust</li>\r\n<li>For sizes approaching 100 million elements, ArrayFire OpenCL is fastest followed by BoostCompute, Thrust and ArrayFire CUDA</li>\r\n</ul>\r\n<p><strong>Intel Xeon E5-2560v2</strong></p>\r\n<ul>\r\n<li>Intel TBB is significantly faster than ArrayFire and BoostCompute for sizes less than 1 million elements</li>\r\n<li>ArrayFire is about 20% to 30% slower for sizes greater than 1 million elements</li>\r\n<li>BoostCompute is about 40% to 70% slower for sizes greater than 1 million elements</li>\r\n</ul>\r\n<h3 id=\"scan\">Scan</h3>\r\n<p>The scan algorithm is useful when performing condition select operations. It can also be used where cumulative distributions are needed.</p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1469250706&amp;format=interactive\">\r\n</iframe>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1909166539&amp;format=interactive\">\r\n</iframe>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=944922182&amp;format=interactive\">\r\n</iframe>\r\n<p><strong>AMD FirePro S10000</strong></p>\r\n<ul>\r\n<li>ArrayFire consistently performs better than other libraries</li>\r\n<li>HSA-Bolt consistently performs better than BoostCompute</li>\r\n</ul>\r\n<p><strong>NVIDIA Tesla K20</strong></p>\r\n<ul>\r\n<li>ArrayFire performs better than other libraries for sizes less than 1 million elements</li>\r\n<li>ArrayFire OpenCL performs better than ArrayFire CUDA for sizes greater than 100000 elements</li>\r\n<li>Thrust consistently performs better than BoostCompute</li>\r\n<li>For sizes greater than 1 million elements, Thrust is 16% faster than ArrayFire OpenCL, 22% faster than ArrayFire CUDA</li>\r\n</ul>\r\n<p><strong>Intel Xeon E5-2560v2</strong></p>\r\n<ul>\r\n<li>Intel TBB is significantly faster than ArrayFire and BoostCompute for all sizes</li>\r\n</ul>\r\n<h3 id=\"transform\">Transform</h3>\r\n<p>The transform operation performs the same operations on each element of the input array. For this benchmark a commonly used function called <code>saxpy</code> has been used.</p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=2057408484&amp;format=interactive\">\r\n</iframe>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=180276342&amp;format=interactive\">\r\n</iframe>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=2014239669&amp;format=interactive\">\r\n</iframe>\r\n<p><strong>AMD FirePro S10000</strong></p>\r\n<ul>\r\n<li>ArrayFire consistently performs better than other libraries</li>\r\n<li>BoostCompute consistently performs better than HSA-Bolt</li>\r\n</ul>\r\n<p><strong>NVIDIA Tesla K20</strong></p>\r\n<ul>\r\n<li>For sizes less than 1 million elements, Thrust is the fastest, followed by ArrayFire CUDA, ArrayFire OpenCL and BoostCompute</li>\r\n<li>For sizes greater than 1 million elements, BoostCompute is fastest followed by ArrayFire OpenCL, Thrust and ArrayFire CUDA</li>\r\n</ul>\r\n<p><strong>Intel Xeon E5-2560v2</strong></p>\r\n<ul>\r\n<li>For sizes less than 1 million elements, TBB is significantly faster than BoostCompute and ArrayFire</li>\r\n<li>For sizes larger than 1 million elements, all three libraries perform within 5% of each other</li>\r\n</ul>\r\n<h3 id=\"conclusion\">Conclusion</h3>\r\n<p>On AMD and NVIDIA GPUs, ArrayFire is the fastest library for vectors on the order of 10 million elements. At even smaller sizes, ArrayFire\'s memory manager ammortizes the memory allocation and deallocation time, making it significantly faster than the alternatives. While improvements can be made overall, one particular area that can be improved upon would be the scan algorithm at large sizes.</p>\r\n<p>On Intel Xeons, TBB is significantly faster than OpenCL implementations. Improvements can and should be made in ArrayFire to extract the best performance out of multicore CPUs. Our roadmap for the next quarter includes a native parallel implementation for our CPU backend. This should bridge the performance gap seen in these benchmarks.</p>\r\n<p>Each library performed better than the others at some point in the benchmarks. Considering that all the libraries are open source, these results should provide a motivation for these libraries to learn from each other.</p>\r\n<h3 id=\"links\">Links:</h3>\r\n<ul>\r\n<li><a href=\"https://github.com/arrayfire/valhalla\">Code</a></li>\r\n<li><a href=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/edit#gid=60225335\">Raw data</a></li>\r\n</ul>\r\n','Benchmarking parallel vector libraries','','inherit','open','closed','','8337-revision-v1','','','2015-06-11 14:39:40','2015-06-11 14:39:40','',8337,'http://sandbox.arrayfire.com/8337-revision-v1/',0,'revision','',0),(8359,7,'2015-06-15 15:24:36','2015-06-15 15:24:36','[content_band inner_container=\"true\" no_margin=\"true\" border=\"none\" padding_top=\"10px\" padding_bottom=\"40px\"]\r\n[custom_headline type=\"left\" level=\"h2\"]\r\nArrayFire Installers\r\n[/custom_headline]\r\n[custom_headline type=\"left\" level=\"h5\"]\r\nEach download contains both CUDA and OpenCL files.\r\n[/custom_headline]\r\n[gap size=\"4%\"]\r\n\r\n[column type=\"one-fourth\"][button  block=\"true\" class=\"center-text\" type=\"flat\" shape=\"rounded\" href=\"http://go.sandbox.arrayfire.com/l/37882/2015-03-19/lvvyk\" size=\"regular\" ]Windows, v3.0.1\r\n(64-bit)[/button][/column] [column type=\"one-fourth\"][button block=\"true\" type=\"flat\"  shape=\"rounded\" href=\"http://go.sandbox.arrayfire.com/l/37882/2015-03-19/lw515\" size=\"regular\" ]Linux, v3.0.1\r\n(64-bit)[/button][/column][column type=\"one-fourth\"][button  block=\"true\" type=\"flat\"  shape=\"rounded\" href=\"http://go.sandbox.arrayfire.com/l/37882/2015-03-19/ltx4p\" size=\"regular\" ]Mac OSX, v3.0.1\r\n(64-bit)[/button][/column] [column type=\"one-fourth\" last=\"true\"][button block=\"true\"  type=\"flat\"  shape=\"rounded\" href=\"http://go.sandbox.arrayfire.com/l/37882/2015-03-19/lw51f\" size=\"regular\" ]Tegra K1 v3.0.1\r\n(32-bit)[/button][/column][gap size=\"9%\"]\r\n\r\nCheck out our GitHub repository to <a href=\"https://github.com/arrayfire/arrayfire\">build from the source</a>.\r\n\r\nLooking for an older installer? <a href=\"http://sandbox.arrayfire.com/installer_archive\">Try here</a>.[/content_band]\r\n<p style=\"text-align: left;\">[content_band inner_container=\"true\" no_margin=\"true\" border=\"none\" padding_top=\"10px\" padding_bottom=\"40px\" bg_color=\"#f1f1f1\"][column type=\"one-half\"][gap size=\"8%\"]<img class=\"aligncenter wp-image-5156\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/02/mobile_Exported1-e1395437261846.png\" alt=\"mobile_Exported\" width=\"233\" height=\"311\" />[/column][column type=\"one-half\" last=\"true\"]\r\n[custom_headline type=\"left\" level=\"h2\"]\r\nGreat. What now?\r\n[/custom_headline]</p>\r\n<p style=\"text-align: left;\">[gap size=\"5%\"]</p>\r\n<p style=\"text-align: left;\">Welcome to ArrayFire! You\'ve got an awesome library there—learn more about its features and functions on <a href=\"http://www.sandbox.arrayfire.com/docs/index.htm\">our documentation</a>.</p>\r\n<p style=\"text-align: left;\">In addition to our <a href=\"https://groups.google.com/forum/#!forum/arrayfire-users\">ArrayFire user group</a> and <a href=\"http://sandbox.arrayfire.com/blog/\">our blog</a>, we offer a plethora of <a href=\"http://sandbox.arrayfire.com/support/\">support options</a> to fit your needs.</p>\r\n<p style=\"text-align: left;\">Got a job that needs closer attention? Hire us! Our engineers are acceleration wizards. Check out our <a href=\"http://sandbox.arrayfire.com/consulting/\">consulting offerings</a>, or set up a free <a href=\"http://meetme.so/arrayfire\">technical consultation</a> with us today![/column]</p>\r\n<p style=\"text-align: left;\">[/content_band]</p>\r\n<p style=\"text-align: center;\"></p>','Download','','inherit','open','closed','','5931-revision-v1','','','2015-06-15 15:24:36','2015-06-15 15:24:36','',5931,'http://sandbox.arrayfire.com/5931-revision-v1/',0,'revision','',0),(8360,17728,'2015-06-15 18:17:45','2015-06-15 18:17:45','\r\n<p>Coming off of our latest release cycle, we think it is a good time to talk about our release policy more openly. In this post we are going to talk about our versioning scheme and the expected release cycle.</p>\r\n<p>The versioning scheme of the library will follow the <a href=\"https://apr.apache.org/versioning.html\">Apache Portable Runtime</a> guidelines. Each release version number will be of the following format <code>arrayfire-x.y.z</code>. The releases are going to be categorized broadly into the following categories based on the frequency and scope.</p>\r\n<h4 id=\"bug-fix-releases\">Bug fix releases</h4>\r\n<p>A bugfix release will bump the 3rd digit of the release version. These releases will neither add new functions nor break API and ABI compatiblity from the previous Feature release.</p>\r\n<p>Bug fix releases will be infrequent, but will likely happen on the first day of the business week.</p>\r\n<p>For example, the next bugfix release is <a href=\"https://github.com/arrayfire/arrayfire/issues?utf8=%E2%9C%93&amp;q=milestone%3A3.0.2+\">3.0.2</a>. It is scheduled for June 29th.</p>\r\n<h4 id=\"feature-releases\">Feature releases</h4>\r\n<p>A feature release will bump the 2nd digit of the release number and reset the 3rd digit to 0.They will add new functionality to the library and try to improve performance across the board. These releases will be backwards compatible to the previous Major release. The new functions will come with header guards to enable users to target a particular (older) versions of the API.</p>\r\n<p>These releases will be made on a quarterly basis.</p>\r\n<p>For example, the next feature release is <a href=\"https://github.com/arrayfire/arrayfire/issues?utf8=%E2%9C%93&amp;q=milestone%3A3.1.0\">3.1.0</a>. It is scheduled for an early August release.</p>\r\n<h4 id=\"major-releases\">Major releases</h4>\r\n<p>A Major release will bump the 1st digit and reset 2nd and 3rd digits to 0. There is no fixed schedule for this. A major release will happen when a major feature is added to ArrayFire and / or there is a breaking change in our API. This will happen once every 15-24 months.</p>\r\n<p>For example, the next major release is 4.0.0. It will possibly come out late 2016 or early 2017.</p>\r\n<h4 id=\"tracking-our-progress\">Tracking our progress</h4>\r\n<p>You can track our progress by going to our <a href=\"https://github.com/arrayfire/arrayfire\">github page</a>. The <code>hotfixes-x.y.z</code> branch will be used for the Bug fix releases. The <code>devel</code> branch will be used for making progress to the next feature release. The <code>master</code> branch will contain the latest stable release.</p>\r\n<p>You can also look at <a href=\"https://github.com/arrayfire/arrayfire/milestones\">our milestones</a> to get a look at what is coming in future versions of ArrayFire. If you have any questions or any specific requests that you would like to see in future versions of ArrayFire, contact us at <script type=\"text/javascript\">\r\n<!--\r\nh=\'&#x61;&#114;&#114;&#x61;&#x79;&#102;&#x69;&#114;&#x65;&#46;&#x63;&#x6f;&#x6d;\';a=\'&#64;\';n=\'&#116;&#x65;&#x63;&#104;&#110;&#x69;&#x63;&#x61;&#108;\';e=n+a+h;\r\ndocument.write(\'<a h\'+\'ref\'+\'=\"ma\'+\'ilto\'+\':\'+e+\'\" clas\'+\'s=\"em\' + \'ail\">\'+e+\'<\\/\'+\'a\'+\'>\');\r\n// -->\r\n</script><noscript>&#116;&#x65;&#x63;&#104;&#110;&#x69;&#x63;&#x61;&#108;&#32;&#x61;&#116;&#32;&#x61;&#114;&#114;&#x61;&#x79;&#102;&#x69;&#114;&#x65;&#32;&#100;&#x6f;&#116;&#32;&#x63;&#x6f;&#x6d;</noscript>.</p>\r\n','ArrayFire release cycle','','publish','open','closed','','arrayfire-release-cycle','','','2015-06-15 18:17:45','2015-06-15 18:17:45','',0,'http://sandbox.arrayfire.com/?p=8360',0,'post','',0),(8361,11,'2015-06-16 22:09:05','0000-00-00 00:00:00','If I were asked to list 3 things to sell a user on ArrayFire, I would say:\n<li>Fast, Parallel, Interchangeable backends</li>\n<li>Open Source BSD License</li>\n<li>Cross Platform</li>\n\nThe last of these is what I shall be discussing today. The challenges of cross platform development and how we, at ArrayFire, deal with this problem.\nWith cross platform we are not just talking about supporting the 3 major operating systems (Linux, OSX, Windows), but also dealing with 32 and 64 bit, trying ArrayFire on new platforms such as ARM and the Tegra K1, and in future with Android.\n\nLets break this down into 3 separate categories\n1. Configuring and Building using CMake\n2. Upstream Libraries\n3. Continuous Integration using Jenkins\n4. Packaging\n\n## Configuring & Building With CMake\nWith ArrayFire 3.0, I, and most of the ArrayFire team, had our first extensive encounter with CMake. Initally ','ArrayFire - Open Source, Multiple Platforms','','draft','open','closed','','','','','2015-06-16 22:09:05','2015-06-16 22:09:05','',0,'http://sandbox.arrayfire.com/?p=8361',0,'post','',0),(8362,11,'2015-06-15 17:33:13','2015-06-15 17:33:13','If I were asked to list 3 things to sell a user on ArrayFire, I would say:\r\n<ul>\r\n	<li>Open Source BSD License</li>\r\n','ArrayFire - Open Source, Open Platform','','inherit','open','closed','','8361-revision-v1','','','2015-06-15 17:33:13','2015-06-15 17:33:13','',8361,'http://sandbox.arrayfire.com/8361-revision-v1/',0,'revision','',0),(8363,11,'2015-06-15 17:34:32','2015-06-15 17:34:32','If I were asked to list 3 things to sell a user on ArrayFire, I would say:\r\n<li>Fast, Parallel, Interchangeable backends</li>\r\n<li>Open Source BSD License</li>\r\n<li>Cross Platform</li>','ArrayFire - Open Source, Open Platform','','inherit','open','closed','','8361-revision-v1','','','2015-06-15 17:34:32','2015-06-15 17:34:32','',8361,'http://sandbox.arrayfire.com/8361-revision-v1/',0,'revision','',0),(8364,17728,'2015-06-15 17:54:07','2015-06-15 17:54:07','\r\n<p>Coming off of our latest release cycle, we think it is a good time to talk about our release policy more openly. In this post we are going to address our goals for the ArrayFire library\'s release cycle. The versioning scheme of the library will follow the <a href=\"https://apr.apache.org/versioning.html\">Apache Portable Runtime</a> guidelines.</p>\r\n<p>Each release version number will be of the following format <code>arrayfire-x.y.z</code>. The releases are going to be categorized broadly into the following categories based on the frequency and scope.</p>\r\n<h4 id=\"bug-fix-releases\">Bug fix releases:</h4>\r\n<p>A bugfix release will bump the 3rd digit of the release version. These releases are done every 2 weeks are so to provide bug fixes and critical performance improvements. These releases will neither add new functions nor break API and ABI compatiblity from the previous Feature release.</p>\r\n<p>For example, the next bugfix release is <a href=\"https://github.com/arrayfire/arrayfire/issues?utf8=%E2%9C%93&amp;q=milestone%3A3.0.2+\">3.0.2</a>. It is scheduled for June 29th.</p>\r\n<h4 id=\"feature-releases\">Feature releases:</h4>\r\n<p>A feature release will bump the 2nd digit of the release number and reset the 3rd digit to 0. These releases will be made on a quarterly basis. They will add new functionality to the library and try to improve performance across the board. These releases will be backwards compatible to the previous Major release.</p>\r\n<p>The new functions will come with header gaurds to enable users to target a particular (older) versions of the API.</p>\r\n<p>For example, the next feature release is <a href=\"https://github.com/arrayfire/arrayfire/issues?utf8=%E2%9C%93&amp;q=milestone%3A3.1\">3.1.0</a>. It is scheduled for an early August release.</p>\r\n<h4 id=\"major-releases\">Major releases:</h4>\r\n<p>A Major release will bump the 1st digit and reset 2nd and 3rd digits to 0. There is no fixed schedule for this. A major release will happen when a major feature is added to ArrayFire and / or there is a breaking change in our API. This will happen once every 15-24 months.</p>\r\n<p>For example, the next major release is 4.0.0. It will possibly come out late 2016 or early 2017.</p>\r\n<h4 id=\"tracking-our-progress\">Tracking our progress</h4>\r\n<p>You can track our progress by going to our <a href=\"https://github.com/arrayfire/arrayfire\">github page</a>. The <code>hotfixes-x.y.z</code> branch will be used for the Bug fix releases. The <code>devel</code> branch will be used for making progress to the next feature release. The <code>master</code> branch will contain the latest stable release.</p>\r\n<p>You can also look at <a href=\"https://github.com/arrayfire/arrayfire/milestones\">our milestones</a> to get a look at what is coming in future versions of ArrayFire. If you have any questions or any specific requests that you would like to see in future versions of ArrayFire, contact us at <code>technical@arrayfire.com</code>.</p>\r\n','ArrayFire release cycle','','inherit','open','closed','','8360-revision-v1','','','2015-06-15 17:54:07','2015-06-15 17:54:07','',8360,'http://sandbox.arrayfire.com/8360-revision-v1/',0,'revision','',0),(8365,17728,'2015-06-15 17:55:15','2015-06-15 17:55:15','\r\n<p>Coming off of our latest release cycle, we think it is a good time to talk about our release policy more openly. In this post we are going to address our goals for the ArrayFire library\'s release cycle. The versioning scheme of the library will follow the <a href=\"https://apr.apache.org/versioning.html\">Apache Portable Runtime</a> guidelines.</p>\r\n<p>Each release version number will be of the following format <code>arrayfire-x.y.z</code>. The releases are going to be categorized broadly into the following categories based on the frequency and scope.</p>\r\n<h4 id=\"bug-fix-releases\">Bug fix releases</h4>\r\n<p>A bugfix release will bump the 3rd digit of the release version. These releases are done every 2 weeks are so to provide bug fixes and critical performance improvements. These releases will neither add new functions nor break API and ABI compatiblity from the previous Feature release.</p>\r\n<p>For example, the next bugfix release is <a href=\"https://github.com/arrayfire/arrayfire/issues?utf8=%E2%9C%93&amp;q=milestone%3A3.0.2+\">3.0.2</a>. It is scheduled for June 29th.</p>\r\n<h4 id=\"feature-releases\">Feature releases</h4>\r\n<p>A feature release will bump the 2nd digit of the release number and reset the 3rd digit to 0. These releases will be made on a quarterly basis. They will add new functionality to the library and try to improve performance across the board. These releases will be backwards compatible to the previous Major release.</p>\r\n<p>The new functions will come with header gaurds to enable users to target a particular (older) versions of the API.</p>\r\n<p>For example, the next feature release is <a href=\"https://github.com/arrayfire/arrayfire/issues?utf8=%E2%9C%93&amp;q=milestone%3A3.1\">3.1.0</a>. It is scheduled for an early August release.</p>\r\n<h4 id=\"major-releases\">Major releases</h4>\r\n<p>A Major release will bump the 1st digit and reset 2nd and 3rd digits to 0. There is no fixed schedule for this. A major release will happen when a major feature is added to ArrayFire and / or there is a breaking change in our API. This will happen once every 15-24 months.</p>\r\n<p>For example, the next major release is 4.0.0. It will possibly come out late 2016 or early 2017.</p>\r\n<h4 id=\"tracking-our-progress\">Tracking our progress</h4>\r\n<p>You can track our progress by going to our <a href=\"https://github.com/arrayfire/arrayfire\">github page</a>. The <code>hotfixes-x.y.z</code> branch will be used for the Bug fix releases. The <code>devel</code> branch will be used for making progress to the next feature release. The <code>master</code> branch will contain the latest stable release.</p>\r\n<p>You can also look at <a href=\"https://github.com/arrayfire/arrayfire/milestones\">our milestones</a> to get a look at what is coming in future versions of ArrayFire. If you have any questions or any specific requests that you would like to see in future versions of ArrayFire, contact us at <code>technical@arrayfire.com</code>.</p>\r\n','ArrayFire release cycle','','inherit','open','closed','','8360-revision-v1','','','2015-06-15 17:55:15','2015-06-15 17:55:15','',8360,'http://sandbox.arrayfire.com/8360-revision-v1/',0,'revision','',0),(8366,17728,'2015-06-15 17:58:12','2015-06-15 17:58:12','\r\n<p>Coming off of our latest release cycle, we think it is a good time to talk about our release policy more openly. In this post we are going to address our goals for the ArrayFire library\'s release cycle. The versioning scheme of the library will follow the <a href=\"https://apr.apache.org/versioning.html\">Apache Portable Runtime</a> guidelines.</p>\r\n<p>Each release version number will be of the following format <code>arrayfire-x.y.z</code>. The releases are going to be categorized broadly into the following categories based on the frequency and scope.</p>\r\n<h4 id=\"bug-fix-releases\">Bug fix releases</h4>\r\n<p>A bugfix release will bump the 3rd digit of the release version. These releases are done every 2 weeks are so to provide bug fixes and critical performance improvements. These releases will neither add new functions nor break API and ABI compatiblity from the previous Feature release.</p>\r\n<p>For example, the next bugfix release is <a href=\"https://github.com/arrayfire/arrayfire/issues?utf8=%E2%9C%93&amp;q=milestone%3A3.0.2+\">3.0.2</a>. It is scheduled for June 29th.</p>\r\n<h4 id=\"feature-releases\">Feature releases</h4>\r\n<p>A feature release will bump the 2nd digit of the release number and reset the 3rd digit to 0. These releases will be made on a quarterly basis. They will add new functionality to the library and try to improve performance across the board. These releases will be backwards compatible to the previous Major release.</p>\r\n<p>The new functions will come with header gaurds to enable users to target a particular (older) versions of the API.</p>\r\n<p>For example, the next feature release is <a href=\"https://github.com/arrayfire/arrayfire/issues?utf8=%E2%9C%93&amp;q=milestone%3A3.1.0\">3.1.0</a>. It is scheduled for an early August release.</p>\r\n<h4 id=\"major-releases\">Major releases</h4>\r\n<p>A Major release will bump the 1st digit and reset 2nd and 3rd digits to 0. There is no fixed schedule for this. A major release will happen when a major feature is added to ArrayFire and / or there is a breaking change in our API. This will happen once every 15-24 months.</p>\r\n<p>For example, the next major release is 4.0.0. It will possibly come out late 2016 or early 2017.</p>\r\n<h4 id=\"tracking-our-progress\">Tracking our progress</h4>\r\n<p>You can track our progress by going to our <a href=\"https://github.com/arrayfire/arrayfire\">github page</a>. The <code>hotfixes-x.y.z</code> branch will be used for the Bug fix releases. The <code>devel</code> branch will be used for making progress to the next feature release. The <code>master</code> branch will contain the latest stable release.</p>\r\n<p>You can also look at <a href=\"https://github.com/arrayfire/arrayfire/milestones\">our milestones</a> to get a look at what is coming in future versions of ArrayFire. If you have any questions or any specific requests that you would like to see in future versions of ArrayFire, contact us at <code>technical@arrayfire.com</code>.</p>\r\n','ArrayFire release cycle','','inherit','open','closed','','8360-revision-v1','','','2015-06-15 17:58:12','2015-06-15 17:58:12','',8360,'http://sandbox.arrayfire.com/8360-revision-v1/',0,'revision','',0),(8367,17728,'2015-06-15 18:10:48','2015-06-15 18:10:48','\r\n<p>Coming off of our latest release cycle, we think it is a good time to talk about our release policy more openly. In this post we are going to address our goals for the ArrayFire library\'s release cycle. The versioning scheme of the library will follow the <a href=\"https://apr.apache.org/versioning.html\">Apache Portable Runtime</a> guidelines.</p>\r\n<p>Each release version number will be of the following format <code>arrayfire-x.y.z</code>. The releases are going to be categorized broadly into the following categories based on the frequency and scope.</p>\r\n<h4 id=\"bug-fix-releases\">Bug fix releases</h4>\r\n<p>A bugfix release will bump the 3rd digit of the release version. These releases will neither add new functions nor break API and ABI compatiblity from the previous Feature release.</p>\r\n<p>Bug fix releases will be infrequent, but will likely happen on the first day of the business week.</p>\r\n<p>For example, the next bugfix release is <a href=\"https://github.com/arrayfire/arrayfire/issues?utf8=%E2%9C%93&amp;q=milestone%3A3.0.2+\">3.0.2</a>. It is scheduled for June 29th.</p>\r\n<h4 id=\"feature-releases\">Feature releases</h4>\r\n<p>A feature release will bump the 2nd digit of the release number and reset the 3rd digit to 0. These releases will be made on a quarterly basis. They will add new functionality to the library and try to improve performance across the board. These releases will be backwards compatible to the previous Major release.</p>\r\n<p>The new functions will come with header gaurds to enable users to target a particular (older) versions of the API.</p>\r\n<p>For example, the next feature release is <a href=\"https://github.com/arrayfire/arrayfire/issues?utf8=%E2%9C%93&amp;q=milestone%3A3.1.0\">3.1.0</a>. It is scheduled for an early August release.</p>\r\n<h4 id=\"major-releases\">Major releases</h4>\r\n<p>A Major release will bump the 1st digit and reset 2nd and 3rd digits to 0. There is no fixed schedule for this. A major release will happen when a major feature is added to ArrayFire and / or there is a breaking change in our API. This will happen once every 15-24 months.</p>\r\n<p>For example, the next major release is 4.0.0. It will possibly come out late 2016 or early 2017.</p>\r\n<h4 id=\"tracking-our-progress\">Tracking our progress</h4>\r\n<p>You can track our progress by going to our <a href=\"https://github.com/arrayfire/arrayfire\">github page</a>. The <code>hotfixes-x.y.z</code> branch will be used for the Bug fix releases. The <code>devel</code> branch will be used for making progress to the next feature release. The <code>master</code> branch will contain the latest stable release.</p>\r\n<p>You can also look at <a href=\"https://github.com/arrayfire/arrayfire/milestones\">our milestones</a> to get a look at what is coming in future versions of ArrayFire. If you have any questions or any specific requests that you would like to see in future versions of ArrayFire, contact us at <code>technical@arrayfire.com</code>.</p>\r\n','ArrayFire release cycle','','inherit','open','closed','','8360-revision-v1','','','2015-06-15 18:10:48','2015-06-15 18:10:48','',8360,'http://sandbox.arrayfire.com/8360-revision-v1/',0,'revision','',0),(8368,17728,'2015-06-15 18:11:37','2015-06-15 18:11:37','\r\n<p>Coming off of our latest release cycle, we think it is a good time to talk about our release policy more openly. In this post we are going to address our goals for the ArrayFire library\'s release cycle. The versioning scheme of the library will follow the <a href=\"https://apr.apache.org/versioning.html\">Apache Portable Runtime</a> guidelines.</p>\r\n<p>Each release version number will be of the following format <code>arrayfire-x.y.z</code>. The releases are going to be categorized broadly into the following categories based on the frequency and scope.</p>\r\n<h4 id=\"bug-fix-releases\">Bug fix releases</h4>\r\n<p>A bugfix release will bump the 3rd digit of the release version. These releases will neither add new functions nor break API and ABI compatiblity from the previous Feature release.</p>\r\n<p>Bug fix releases will be infrequent, but will likely happen on the first day of the business week.</p>\r\n<p>For example, the next bugfix release is <a href=\"https://github.com/arrayfire/arrayfire/issues?utf8=%E2%9C%93&amp;q=milestone%3A3.0.2+\">3.0.2</a>. It is scheduled for June 29th.</p>\r\n<h4 id=\"feature-releases\">Feature releases</h4>\r\n<p>A feature release will bump the 2nd digit of the release number and reset the 3rd digit to 0.They will add new functionality to the library and try to improve performance across the board. These releases will be backwards compatible to the previous Major release. The new functions will come with header gaurds to enable users to target a particular (older) versions of the API.</p>\r\n<p>These releases will be made on a quarterly basis.</p>\r\n<p>For example, the next feature release is <a href=\"https://github.com/arrayfire/arrayfire/issues?utf8=%E2%9C%93&amp;q=milestone%3A3.1.0\">3.1.0</a>. It is scheduled for an early August release.</p>\r\n<h4 id=\"major-releases\">Major releases</h4>\r\n<p>A Major release will bump the 1st digit and reset 2nd and 3rd digits to 0. There is no fixed schedule for this. A major release will happen when a major feature is added to ArrayFire and / or there is a breaking change in our API. This will happen once every 15-24 months.</p>\r\n<p>For example, the next major release is 4.0.0. It will possibly come out late 2016 or early 2017.</p>\r\n<h4 id=\"tracking-our-progress\">Tracking our progress</h4>\r\n<p>You can track our progress by going to our <a href=\"https://github.com/arrayfire/arrayfire\">github page</a>. The <code>hotfixes-x.y.z</code> branch will be used for the Bug fix releases. The <code>devel</code> branch will be used for making progress to the next feature release. The <code>master</code> branch will contain the latest stable release.</p>\r\n<p>You can also look at <a href=\"https://github.com/arrayfire/arrayfire/milestones\">our milestones</a> to get a look at what is coming in future versions of ArrayFire. If you have any questions or any specific requests that you would like to see in future versions of ArrayFire, contact us at <code>technical@arrayfire.com</code>.</p>\r\n','ArrayFire release cycle','','inherit','open','closed','','8360-revision-v1','','','2015-06-15 18:11:37','2015-06-15 18:11:37','',8360,'http://sandbox.arrayfire.com/8360-revision-v1/',0,'revision','',0),(8369,17728,'2015-06-15 18:14:41','2015-06-15 18:14:41','\r\n<p>Coming off of our latest release cycle, we think it is a good time to talk about our release policy more openly. In this post we are going to talk about our versioning scheme and the expected release cycle.</p>\r\n<p>The versioning scheme of the library will follow the <a href=\"https://apr.apache.org/versioning.html\">Apache Portable Runtime</a> guidelines. Each release version number will be of the following format <code>arrayfire-x.y.z</code>. The releases are going to be categorized broadly into the following categories based on the frequency and scope.</p>\r\n<h4 id=\"bug-fix-releases\">Bug fix releases</h4>\r\n<p>A bugfix release will bump the 3rd digit of the release version. These releases will neither add new functions nor break API and ABI compatiblity from the previous Feature release.</p>\r\n<p>Bug fix releases will be infrequent, but will likely happen on the first day of the business week.</p>\r\n<p>For example, the next bugfix release is <a href=\"https://github.com/arrayfire/arrayfire/issues?utf8=%E2%9C%93&amp;q=milestone%3A3.0.2+\">3.0.2</a>. It is scheduled for June 29th.</p>\r\n<h4 id=\"feature-releases\">Feature releases</h4>\r\n<p>A feature release will bump the 2nd digit of the release number and reset the 3rd digit to 0.They will add new functionality to the library and try to improve performance across the board. These releases will be backwards compatible to the previous Major release. The new functions will come with header gaurds to enable users to target a particular (older) versions of the API.</p>\r\n<p>These releases will be made on a quarterly basis.</p>\r\n<p>For example, the next feature release is <a href=\"https://github.com/arrayfire/arrayfire/issues?utf8=%E2%9C%93&amp;q=milestone%3A3.1.0\">3.1.0</a>. It is scheduled for an early August release.</p>\r\n<h4 id=\"major-releases\">Major releases</h4>\r\n<p>A Major release will bump the 1st digit and reset 2nd and 3rd digits to 0. There is no fixed schedule for this. A major release will happen when a major feature is added to ArrayFire and / or there is a breaking change in our API. This will happen once every 15-24 months.</p>\r\n<p>For example, the next major release is 4.0.0. It will possibly come out late 2016 or early 2017.</p>\r\n<h4 id=\"tracking-our-progress\">Tracking our progress</h4>\r\n<p>You can track our progress by going to our <a href=\"https://github.com/arrayfire/arrayfire\">github page</a>. The <code>hotfixes-x.y.z</code> branch will be used for the Bug fix releases. The <code>devel</code> branch will be used for making progress to the next feature release. The <code>master</code> branch will contain the latest stable release.</p>\r\n<p>You can also look at <a href=\"https://github.com/arrayfire/arrayfire/milestones\">our milestones</a> to get a look at what is coming in future versions of ArrayFire. If you have any questions or any specific requests that you would like to see in future versions of ArrayFire, contact us at <code>technical@arrayfire.com</code>.</p>\r\n','ArrayFire release cycle','','inherit','open','closed','','8360-revision-v1','','','2015-06-15 18:14:41','2015-06-15 18:14:41','',8360,'http://sandbox.arrayfire.com/8360-revision-v1/',0,'revision','',0),(8370,17728,'2015-06-15 18:17:29','2015-06-15 18:17:29','\r\n<p>Coming off of our latest release cycle, we think it is a good time to talk about our release policy more openly. In this post we are going to talk about our versioning scheme and the expected release cycle.</p>\r\n<p>The versioning scheme of the library will follow the <a href=\"https://apr.apache.org/versioning.html\">Apache Portable Runtime</a> guidelines. Each release version number will be of the following format <code>arrayfire-x.y.z</code>. The releases are going to be categorized broadly into the following categories based on the frequency and scope.</p>\r\n<h4 id=\"bug-fix-releases\">Bug fix releases</h4>\r\n<p>A bugfix release will bump the 3rd digit of the release version. These releases will neither add new functions nor break API and ABI compatiblity from the previous Feature release.</p>\r\n<p>Bug fix releases will be infrequent, but will likely happen on the first day of the business week.</p>\r\n<p>For example, the next bugfix release is <a href=\"https://github.com/arrayfire/arrayfire/issues?utf8=%E2%9C%93&amp;q=milestone%3A3.0.2+\">3.0.2</a>. It is scheduled for June 29th.</p>\r\n<h4 id=\"feature-releases\">Feature releases</h4>\r\n<p>A feature release will bump the 2nd digit of the release number and reset the 3rd digit to 0.They will add new functionality to the library and try to improve performance across the board. These releases will be backwards compatible to the previous Major release. The new functions will come with header guards to enable users to target a particular (older) versions of the API.</p>\r\n<p>These releases will be made on a quarterly basis.</p>\r\n<p>For example, the next feature release is <a href=\"https://github.com/arrayfire/arrayfire/issues?utf8=%E2%9C%93&amp;q=milestone%3A3.1.0\">3.1.0</a>. It is scheduled for an early August release.</p>\r\n<h4 id=\"major-releases\">Major releases</h4>\r\n<p>A Major release will bump the 1st digit and reset 2nd and 3rd digits to 0. There is no fixed schedule for this. A major release will happen when a major feature is added to ArrayFire and / or there is a breaking change in our API. This will happen once every 15-24 months.</p>\r\n<p>For example, the next major release is 4.0.0. It will possibly come out late 2016 or early 2017.</p>\r\n<h4 id=\"tracking-our-progress\">Tracking our progress</h4>\r\n<p>You can track our progress by going to our <a href=\"https://github.com/arrayfire/arrayfire\">github page</a>. The <code>hotfixes-x.y.z</code> branch will be used for the Bug fix releases. The <code>devel</code> branch will be used for making progress to the next feature release. The <code>master</code> branch will contain the latest stable release.</p>\r\n<p>You can also look at <a href=\"https://github.com/arrayfire/arrayfire/milestones\">our milestones</a> to get a look at what is coming in future versions of ArrayFire. If you have any questions or any specific requests that you would like to see in future versions of ArrayFire, contact us at <script type=\"text/javascript\">\r\n<!--\r\nh=\'&#x61;&#114;&#114;&#x61;&#x79;&#102;&#x69;&#114;&#x65;&#46;&#x63;&#x6f;&#x6d;\';a=\'&#64;\';n=\'&#116;&#x65;&#x63;&#104;&#110;&#x69;&#x63;&#x61;&#108;\';e=n+a+h;\r\ndocument.write(\'<a h\'+\'ref\'+\'=\"ma\'+\'ilto\'+\':\'+e+\'\" clas\'+\'s=\"em\' + \'ail\">\'+e+\'<\\/\'+\'a\'+\'>\');\r\n// -->\r\n</script><noscript>&#116;&#x65;&#x63;&#104;&#110;&#x69;&#x63;&#x61;&#108;&#32;&#x61;&#116;&#32;&#x61;&#114;&#114;&#x61;&#x79;&#102;&#x69;&#114;&#x65;&#32;&#100;&#x6f;&#116;&#32;&#x63;&#x6f;&#x6d;</noscript>.</p>\r\n','ArrayFire release cycle','','inherit','open','closed','','8360-revision-v1','','','2015-06-15 18:17:29','2015-06-15 18:17:29','',8360,'http://sandbox.arrayfire.com/8360-revision-v1/',0,'revision','',0),(8371,11,'2015-06-16 21:59:39','2015-06-16 21:59:39','If I were asked to list 3 things to sell a user on ArrayFire, I would say:\r\n<li>Fast, Parallel, Interchangeable backends</li>\r\n<li>Open Source BSD License</li>\r\n<li>Cross Platform</li>\r\n\r\nThe last of these is what I shall be discussing today. The challenges of cross platform development and how we, at ArrayFire, deal with this problem.\r\nWith cross platform we are not just talking about supporting the 3 major operating systems (Linux, OSX, Windows), but also dealing with 32 and 64 bit, trying ArrayFire on new platforms such as ARM and the Tegra K1, and in future with Android.\r\n\r\nLets break this down into 3 separate categories\r\n1. Configuring and Building using CMake\r\n2. Upstream Libraries\r\n3. Continuous Integration using Jenkins\r\n4. Packaging\r\n\r\n## Configuring & Building With CMake\r\nWith ArrayFire 3.0, I, and most of the ArrayFire team, had our first extensive encounter with CMake. ','ArrayFire - Open Source, Multiple Platforms','','inherit','open','closed','','8361-revision-v1','','','2015-06-16 21:59:39','2015-06-16 21:59:39','',8361,'http://sandbox.arrayfire.com/8361-revision-v1/',0,'revision','',0),(8372,11,'2015-06-17 17:28:02','0000-00-00 00:00:00','We are starting a new series here about the tools we use at ArrayFire to make writing better code more productive.\r\n\r\nWe will be talking about hardware, linux distros, IDEs, editors, debuggers, miscellaneous software or anything that we believe helps us in increasing our productivity.\r\nYou will hear the eternal developer battles of Vim vs Emacs, Visual Studio vs Eclipse, dual vs single monitor, even lights on vs lights off (and more recently controllable smart lightning).\r\n\r\nSo sit tight for what is going to be an exciting series of posts.\r\n\r\nI\'ll be taking the first shot at this.\r\n\r\nI had to give it a long thought about what I really wanted to write about as the first post. I was in a dilemma between my Vim setup, Screen, gDebugger and my Visual Studio setup.\r\nAfter a long thought, I decided to write about <strong>my favorite Visual Studio Extensions</strong>.\r\n\r\nHere is a list of Visual Studio Extensions I use:\r\n* Productivity Power Tools\r\n* VsVim\r\n* CodeMaid\r\n* Brace Completer\r\n* Code Alignment\r\n* Indent Guides\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/dbcb8670-889e-4a54-a226-a48a15e4cace\" target=\"_blank\">Productivity Power Tools</a></strong>\r\nThis is by far my most used extension. It is the first thing I download whenever I install Visual Studio.\r\n\r\nI came across this extension when I saw someone using the \"Full Map Mode\" on sublime text and wanted the same thing for Visual Studio. And voila, Productivity Power Tools.\r\nWhen I installed it, I had no idea of how useful this was going to be. I started off using the Map modes for all sorts of navigation.\r\n\r\nOne of the most useful things from PPT is that it highlights all the places where there is a syntax errors in the full map mode using red dots. Simply howering over these red dots will show you the line of the error and clicking it will take you to that line. Similarly, warnings are indicated by yellow dots.\r\n\r\nAnother useful part is where the full map mode can be handy in showing a line elsewhere in the file that you may need to refer to for something like function arguments.\r\n\r\nThe \"Ctrl + Click\" Peek Definition is amazing. When working with a code base as big as ArrayFire, this comes in really handy. When you have a function that you want to peek into, ie. see its definition, simply Ctrl + Click this function. Not only functions, but any definition. If it is overloaded, then it will give you the option to click and select the definition you wish to see.\r\n\r\nI can write a whole blog post on Productivity Power Tools on its own. To simply summarize: if you do not have Productivity Power Tool, install it right now!\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/59ca71b3-a4a3-46ca-8fe1-0e90e3f79329/\" target=\"_blank\">VsVim</a></strong>\r\nFor a vim user, this is a must have. This extension provides all the best parts of Vim modes with the power of the Visual Studio environment. Simply copy your existing vimrc or configure VsVim manually. Either way, you are bound to make an extreme boost in productivity.\r\n\r\nOne of the things I have a love-hate relationship with, is the option to configure which Visual Studio shortcuts to use vs. which Vim shortcuts to use. For example, Ctrl+V in Visual Studio is for paste where as in Vim its for visual selection. Because the options are given to the user about which mode to choose for each shortcut, I am eternally confused about which shortcut I configured for which mode and and which computer (you have to manually configure each set of of shortcuts on each system and each version of Visual Studio).\r\n\r\nThis is just me ranting about one bad thing out of a thousand good ones.\r\n\r\nWhether you are a Vim user or not, VsVim is bound to make you write faster code by giving you the editing powers of Vim.\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/76293c4d-8c16-4f4a-aee6-21f83a571496\" target=\"_blank\">CodeMaid</a></strong>\r\nCodeMaid was a extension I discovered by experimentation. I went about installing all the popular extensions. When I found CodeMaid, it was love at first sight.\r\n\r\nThere isn\'t a single thing that I don\'t love about CodeMaid. I use practically every feature they provide. Code cleaning, navigation (\"digging\"), reorganizing just by drag and drop in the panel, formatting, wrapping code. I could do on and on about CodeMaid. But I can never do the justice that <a href=\"http://www.codemaid.net/documentation/\" target=\"_blank\">CodeMaid\'s Documentation</a> can do to its power. So check the documentation out and see how useful it can be.\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/0e33cb22-d4ac-4f5a-902f-aff5177cc94d\" target=\"_blank\">Brace Completer</a></strong>\r\nThis is a simple extension with one purpose. Always close the bracket.\r\n\r\nYou can configure this extension about which brace style you follow, inline, tabbed, new line always etc.\r\n\r\nConfigure it once, and reap the benefits forever.\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/7179e851-a263-44b7-a177-1d31e33c84fd\" target=\"_blank\">Code Alignment</a></strong>\r\nI would confess that I have an OCD when it comes to code symmetry and alignment. I was everything to be aligned. And this extension does just that.\r\n\r\nTell it what to align by - commas, equal sign, dot, arrow, and even custom text. It will do all the spacing for you.\r\n\r\nWhen writing kernels with SIMD, where you are writing multiple lines of the same operation, this is one of the most useful extensions to use.\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/e792686d-542b-474a-8c55-630980e72c30\" target=\"_blank\">Indent Guides</a></strong>\r\nA lot of people do not like indent guides. In their opinion, it makes the code look \"crummy\", with all those characters/lines showing spaces/tabs.\r\n\r\nBut given my association with alignment, I find it useful. I do share the opinion that its not the cleanest bit of code to look at, but then you are writing 4 levels of nested for loops, it has its perks.\r\n<hr>\r\nComment below to share your opinion on your favorite extensions. And keep a watch out for our next Tools at ArrayFire blog.','Tools at ArrayFire - Visual Studio Extensions','','draft','open','closed','','tools-at-arrayfire-vs-extensions','','','2015-06-17 17:28:02','2015-06-17 17:28:02','',0,'http://sandbox.arrayfire.com/?p=8372',0,'post','',0),(8373,11,'2015-06-17 15:07:44','2015-06-17 15:07:44','We are starting a new series here about the tools we use at ArrayFire to make writing better code more productive.\r\nWe will be talking about hardware, linux distros, IDEs, editors, debuggers, miscellaneous software or anything that we believe helps us in increasing our productivity.\r\nYou will hear the eternal developer battles of Vim vs Emacs, Visual Studio vs Eclipse, dual vs single monitor, even lights on vs lights off (and more recently controllable smart lightning).\r\nSo sit tight for what is going to be an exciting series of posts.\r\n\r\nI\'ll be taking the first shot at this.\r\nI had to give it a long thought about what I really wanted to write about as the first post. I was in a dilemma between my Vim setup, Screen, gDebugger and my Visual Studio setup.\r\nAfter a long thought, I decided to write about <strong>my favorite Visual Studio plugins</strong>.\r\n\r\n','Tools at ArrayFire','','inherit','open','closed','','8372-revision-v1','','','2015-06-17 15:07:44','2015-06-17 15:07:44','',8372,'http://sandbox.arrayfire.com/8372-revision-v1/',0,'revision','',0),(8374,11,'2015-06-17 15:55:16','2015-06-17 15:55:16','We are starting a new series here about the tools we use at ArrayFire to make writing better code more productive.\r\nWe will be talking about hardware, linux distros, IDEs, editors, debuggers, miscellaneous software or anything that we believe helps us in increasing our productivity.\r\nYou will hear the eternal developer battles of Vim vs Emacs, Visual Studio vs Eclipse, dual vs single monitor, even lights on vs lights off (and more recently controllable smart lightning).\r\nSo sit tight for what is going to be an exciting series of posts.\r\n\r\nI\'ll be taking the first shot at this.\r\n\r\nI had to give it a long thought about what I really wanted to write about as the first post. I was in a dilemma between my Vim setup, Screen, gDebugger and my Visual Studio setup.\r\nAfter a long thought, I decided to write about <strong>my favorite Visual Studio Extensions</strong>.\r\n\r\nHere is a list of Visual Studio Extensions I use:\r\nProductivity Power Tools\r\nVsVim\r\nCodeMaid\r\nBrace Completer\r\nCode Alignment\r\nIndent Guides\r\n\r\n<a href=\"https://visualstudiogallery.msdn.microsoft.com/dbcb8670-889e-4a54-a226-a48a15e4cace\" target=\"_blank\">Productivity Power Tools</a>\r\nThis is by far my most used extension. It is the first thing I download whenever I install Visual Studio.\r\nI can across this extension when I saw someone using the \"Full Map Mode\" on sublime text and wanted the same thing for Visual Studio. And voila, Productivity Power Tools.\r\nWhen I installed it, I had no idea of how useful this was going to be. I started off using the Map modes for all sorts of navigation.\r\nOne of the most useful things from PPT is that it highlights all the places where there is a syntax errors in the full map mode using red dots. Simply howering over these red dots will show you the line of the error and clicking it will take you to that line. Similarly, warnings are indicated by yellow dots.\r\nAnother useful part is where the full map mode can be handy in showing a line elsewhere in the file that you may need to refer to for something like function arguments.\r\n\r\nAnother favorite of mine is the \"Ctrl + Click\" Peek Definition. When working with a code base as big as ArrayFire, this comes in really handy. When you have a function that you want to peek into, ie. see its definition, simply Ctrl + Click this function. Not only functions, but any definition. If it is overloaded, then it will give you the option to click and select the definition you wish to see.\r\n\r\nI can write a whole blog post on Productivity Power Tools on its own. To simply summarize: if you do not have Productivity Power Tool, install it right now!\r\n\r\n<a href=\"https://visualstudiogallery.msdn.microsoft.com/59ca71b3-a4a3-46ca-8fe1-0e90e3f79329/\" target=\"_blank\">VsVim</a>\r\nFor a vim user, this is a must have. This extension provides all the best parts of Vim modes with the power of the Visual Studio environment. Simply copy your existing vimrc or configure VsVim manually. Either way, you are bound to make an extreme boost in productivity.\r\nOne of the things I have a love-hate relationship with, is the option to configure which Visual Studio shortcuts to use vs. which Vim shortcuts to use. For example, Ctrl+V in Visual Studio is for paste where as in Vim its for visual selection. Because the options are given to the user about which mode to choose for each shortcut, I am eternally confused about which shortcut I configured for which mode and and which computer (you have to manually configure each set of of shortcuts on each system and each version of Visual Studio).\r\nThis is just me ranting about one bad thing out of a thousand good ones.\r\nWhether you are a Vim user or not, VsVim is bound to make you write faster code by giving you the editing powers of Vim.\r\n\r\n<a href=\"https://visualstudiogallery.msdn.microsoft.com/76293c4d-8c16-4f4a-aee6-21f83a571496\" target=\"_blank\">CodeMaid</a>\r\nCodeMaid was a extension I discovered by experimentation. I went about installing all the popular extensions. When I found CodeMaid, it was love at first sight.\r\nThere isn\'t a single thing that I don\'t love about CodeMaid. I use practically every feature they provide. Code cleaning, navigation (\"digging\"), reorganizing just by drag and drop in the panel, formatting, wrapping code. I could do on and on about CodeMaid. But I can never do the justice that <a href=\"http://www.codemaid.net/documentation/\" target=\"_blank\">CodeMaid\'s Documentation</a> can do to its power. So check the documentation out and see how useful it can be.\r\n\r\n\r\n','Tools at ArrayFire','','inherit','open','closed','','8372-revision-v1','','','2015-06-17 15:55:16','2015-06-17 15:55:16','',8372,'http://sandbox.arrayfire.com/8372-revision-v1/',0,'revision','',0),(8375,11,'2015-06-17 15:55:33','2015-06-17 15:55:33','We are starting a new series here about the tools we use at ArrayFire to make writing better code more productive.\r\nWe will be talking about hardware, linux distros, IDEs, editors, debuggers, miscellaneous software or anything that we believe helps us in increasing our productivity.\r\nYou will hear the eternal developer battles of Vim vs Emacs, Visual Studio vs Eclipse, dual vs single monitor, even lights on vs lights off (and more recently controllable smart lightning).\r\nSo sit tight for what is going to be an exciting series of posts.\r\n\r\nI\'ll be taking the first shot at this.\r\n\r\nI had to give it a long thought about what I really wanted to write about as the first post. I was in a dilemma between my Vim setup, Screen, gDebugger and my Visual Studio setup.\r\nAfter a long thought, I decided to write about <strong>my favorite Visual Studio Extensions</strong>.\r\n\r\nHere is a list of Visual Studio Extensions I use:\r\nProductivity Power Tools\r\nVsVim\r\nCodeMaid\r\nBrace Completer\r\nCode Alignment\r\nIndent Guides\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/dbcb8670-889e-4a54-a226-a48a15e4cace\" target=\"_blank\">Productivity Power Tools</a>\r\n</strong>This is by far my most used extension. It is the first thing I download whenever I install Visual Studio.\r\nI can across this extension when I saw someone using the \"Full Map Mode\" on sublime text and wanted the same thing for Visual Studio. And voila, Productivity Power Tools.\r\nWhen I installed it, I had no idea of how useful this was going to be. I started off using the Map modes for all sorts of navigation.\r\nOne of the most useful things from PPT is that it highlights all the places where there is a syntax errors in the full map mode using red dots. Simply howering over these red dots will show you the line of the error and clicking it will take you to that line. Similarly, warnings are indicated by yellow dots.\r\nAnother useful part is where the full map mode can be handy in showing a line elsewhere in the file that you may need to refer to for something like function arguments.\r\n\r\nAnother favorite of mine is the \"Ctrl + Click\" Peek Definition. When working with a code base as big as ArrayFire, this comes in really handy. When you have a function that you want to peek into, ie. see its definition, simply Ctrl + Click this function. Not only functions, but any definition. If it is overloaded, then it will give you the option to click and select the definition you wish to see.\r\n\r\nI can write a whole blog post on Productivity Power Tools on its own. To simply summarize: if you do not have Productivity Power Tool, install it right now!\r\n\r\n<a href=\"https://visualstudiogallery.msdn.microsoft.com/59ca71b3-a4a3-46ca-8fe1-0e90e3f79329/\" target=\"_blank\">VsVim</a>\r\nFor a vim user, this is a must have. This extension provides all the best parts of Vim modes with the power of the Visual Studio environment. Simply copy your existing vimrc or configure VsVim manually. Either way, you are bound to make an extreme boost in productivity.\r\nOne of the things I have a love-hate relationship with, is the option to configure which Visual Studio shortcuts to use vs. which Vim shortcuts to use. For example, Ctrl+V in Visual Studio is for paste where as in Vim its for visual selection. Because the options are given to the user about which mode to choose for each shortcut, I am eternally confused about which shortcut I configured for which mode and and which computer (you have to manually configure each set of of shortcuts on each system and each version of Visual Studio).\r\nThis is just me ranting about one bad thing out of a thousand good ones.\r\nWhether you are a Vim user or not, VsVim is bound to make you write faster code by giving you the editing powers of Vim.\r\n\r\n<a href=\"https://visualstudiogallery.msdn.microsoft.com/76293c4d-8c16-4f4a-aee6-21f83a571496\" target=\"_blank\">CodeMaid</a>\r\nCodeMaid was a extension I discovered by experimentation. I went about installing all the popular extensions. When I found CodeMaid, it was love at first sight.\r\nThere isn\'t a single thing that I don\'t love about CodeMaid. I use practically every feature they provide. Code cleaning, navigation (\"digging\"), reorganizing just by drag and drop in the panel, formatting, wrapping code. I could do on and on about CodeMaid. But I can never do the justice that <a href=\"http://www.codemaid.net/documentation/\" target=\"_blank\">CodeMaid\'s Documentation</a> can do to its power. So check the documentation out and see how useful it can be.\r\n\r\n\r\n','Tools at ArrayFire','','inherit','open','closed','','8372-revision-v1','','','2015-06-17 15:55:33','2015-06-17 15:55:33','',8372,'http://sandbox.arrayfire.com/8372-revision-v1/',0,'revision','',0),(8376,11,'2015-06-17 16:08:50','2015-06-17 16:08:50','We are starting a new series here about the tools we use at ArrayFire to make writing better code more productive.\r\nWe will be talking about hardware, linux distros, IDEs, editors, debuggers, miscellaneous software or anything that we believe helps us in increasing our productivity.\r\nYou will hear the eternal developer battles of Vim vs Emacs, Visual Studio vs Eclipse, dual vs single monitor, even lights on vs lights off (and more recently controllable smart lightning).\r\nSo sit tight for what is going to be an exciting series of posts.\r\n\r\nI\'ll be taking the first shot at this.\r\n\r\nI had to give it a long thought about what I really wanted to write about as the first post. I was in a dilemma between my Vim setup, Screen, gDebugger and my Visual Studio setup.\r\nAfter a long thought, I decided to write about <strong>my favorite Visual Studio Extensions</strong>.\r\n\r\nHere is a list of Visual Studio Extensions I use:\r\nProductivity Power Tools\r\nVsVim\r\nCodeMaid\r\nBrace Completer\r\nCode Alignment\r\nIndent Guides\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/dbcb8670-889e-4a54-a226-a48a15e4cace\" target=\"_blank\">Productivity Power Tools</a>\r\n</strong>This is by far my most used extension. It is the first thing I download whenever I install Visual Studio.\r\nI can across this extension when I saw someone using the \"Full Map Mode\" on sublime text and wanted the same thing for Visual Studio. And voila, Productivity Power Tools.\r\nWhen I installed it, I had no idea of how useful this was going to be. I started off using the Map modes for all sorts of navigation.\r\nOne of the most useful things from PPT is that it highlights all the places where there is a syntax errors in the full map mode using red dots. Simply howering over these red dots will show you the line of the error and clicking it will take you to that line. Similarly, warnings are indicated by yellow dots.\r\nAnother useful part is where the full map mode can be handy in showing a line elsewhere in the file that you may need to refer to for something like function arguments.\r\n\r\nAnother favorite of mine is the \"Ctrl + Click\" Peek Definition. When working with a code base as big as ArrayFire, this comes in really handy. When you have a function that you want to peek into, ie. see its definition, simply Ctrl + Click this function. Not only functions, but any definition. If it is overloaded, then it will give you the option to click and select the definition you wish to see.\r\n\r\nI can write a whole blog post on Productivity Power Tools on its own. To simply summarize: if you do not have Productivity Power Tool, install it right now!\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/59ca71b3-a4a3-46ca-8fe1-0e90e3f79329/\" target=\"_blank\">VsVim</a></strong>\r\nFor a vim user, this is a must have. This extension provides all the best parts of Vim modes with the power of the Visual Studio environment. Simply copy your existing vimrc or configure VsVim manually. Either way, you are bound to make an extreme boost in productivity.\r\nOne of the things I have a love-hate relationship with, is the option to configure which Visual Studio shortcuts to use vs. which Vim shortcuts to use. For example, Ctrl+V in Visual Studio is for paste where as in Vim its for visual selection. Because the options are given to the user about which mode to choose for each shortcut, I am eternally confused about which shortcut I configured for which mode and and which computer (you have to manually configure each set of of shortcuts on each system and each version of Visual Studio).\r\nThis is just me ranting about one bad thing out of a thousand good ones.\r\nWhether you are a Vim user or not, VsVim is bound to make you write faster code by giving you the editing powers of Vim.\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/76293c4d-8c16-4f4a-aee6-21f83a571496\" target=\"_blank\">CodeMaid</a></strong>\r\nCodeMaid was a extension I discovered by experimentation. I went about installing all the popular extensions. When I found CodeMaid, it was love at first sight.\r\nThere isn\'t a single thing that I don\'t love about CodeMaid. I use practically every feature they provide. Code cleaning, navigation (\"digging\"), reorganizing just by drag and drop in the panel, formatting, wrapping code. I could do on and on about CodeMaid. But I can never do the justice that <a href=\"http://www.codemaid.net/documentation/\" target=\"_blank\">CodeMaid\'s Documentation</a> can do to its power. So check the documentation out and see how useful it can be.\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/0e33cb22-d4ac-4f5a-902f-aff5177cc94d\" target=\"_blank\">Brace Completer</a></strong>\r\nThis is a simple extension with one purpose. Always close the bracket.\r\nYou can configure this extension about which brace style you follow, inline, tabbed, new line always etc.\r\nConfigure it once, and reap the benefits forever.\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/0e33cb22-d4ac-4f5a-902f-aff5177cc94d\" target=\"_blank\">Brace Completer</a></strong>\r\nThis is a simple extension with one purpose. Always close the bracket.\r\nYou can configure this extension about which brace style you follow, inline, tabbed, new line always etc.\r\nConfigure it once, and reap the benefits forever.\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/7179e851-a263-44b7-a177-1d31e33c84fd\" target=\"_blank\">Code Alignment</a></strong>\r\nI would confess that I have an OCD when it comes to code symmetry and alignment. I was everything to be aligned. And this extension does just that.\r\nTell it what to align by - commas, equal sign, dot, arrow, and even custom text. It will do all the spacing for you.\r\nWhen writing kernels with SIMD, where you are writing multiple lines of the same operation, this is one of the most useful extensions to use.\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/e792686d-542b-474a-8c55-630980e72c30\" target=\"_blank\">Indent Guides</a></strong>\r\nA lot of people do not like indent guides. In their opinion, it makes the code look \"crummy\", with all those characters/lines showing spaces/tabs.\r\nBut given my association with alignment, I find it useful. I do share the opinion that its not the cleanest bit of code to look at, but then you are writing 4 levels of nested for loops, it has its perks.\r\n\r\nComment below to share your opinion on your favorite extensions. And keep a watch out for our next Tools at ArrayFire blog.','Tools at ArrayFire','','inherit','open','closed','','8372-revision-v1','','','2015-06-17 16:08:50','2015-06-17 16:08:50','',8372,'http://sandbox.arrayfire.com/8372-revision-v1/',0,'revision','',0),(8377,11,'2015-06-17 16:10:18','2015-06-17 16:10:18','We are starting a new series here about the tools we use at ArrayFire to make writing better code more productive.\r\nWe will be talking about hardware, linux distros, IDEs, editors, debuggers, miscellaneous software or anything that we believe helps us in increasing our productivity.\r\nYou will hear the eternal developer battles of Vim vs Emacs, Visual Studio vs Eclipse, dual vs single monitor, even lights on vs lights off (and more recently controllable smart lightning).\r\nSo sit tight for what is going to be an exciting series of posts.\r\n\r\nI\'ll be taking the first shot at this.\r\n\r\nI had to give it a long thought about what I really wanted to write about as the first post. I was in a dilemma between my Vim setup, Screen, gDebugger and my Visual Studio setup.\r\nAfter a long thought, I decided to write about <strong>my favorite Visual Studio Extensions</strong>.\r\n\r\nHere is a list of Visual Studio Extensions I use:\r\nProductivity Power Tools\r\nVsVim\r\nCodeMaid\r\nBrace Completer\r\nCode Alignment\r\nIndent Guides\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/dbcb8670-889e-4a54-a226-a48a15e4cace\" target=\"_blank\">Productivity Power Tools</a>\r\n</strong>This is by far my most used extension. It is the first thing I download whenever I install Visual Studio.\r\nI can across this extension when I saw someone using the \"Full Map Mode\" on sublime text and wanted the same thing for Visual Studio. And voila, Productivity Power Tools.\r\nWhen I installed it, I had no idea of how useful this was going to be. I started off using the Map modes for all sorts of navigation.\r\nOne of the most useful things from PPT is that it highlights all the places where there is a syntax errors in the full map mode using red dots. Simply howering over these red dots will show you the line of the error and clicking it will take you to that line. Similarly, warnings are indicated by yellow dots.\r\nAnother useful part is where the full map mode can be handy in showing a line elsewhere in the file that you may need to refer to for something like function arguments.\r\n\r\nAnother favorite of mine is the \"Ctrl + Click\" Peek Definition. When working with a code base as big as ArrayFire, this comes in really handy. When you have a function that you want to peek into, ie. see its definition, simply Ctrl + Click this function. Not only functions, but any definition. If it is overloaded, then it will give you the option to click and select the definition you wish to see.\r\n\r\nI can write a whole blog post on Productivity Power Tools on its own. To simply summarize: if you do not have Productivity Power Tool, install it right now!\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/59ca71b3-a4a3-46ca-8fe1-0e90e3f79329/\" target=\"_blank\">VsVim</a></strong>\r\nFor a vim user, this is a must have. This extension provides all the best parts of Vim modes with the power of the Visual Studio environment. Simply copy your existing vimrc or configure VsVim manually. Either way, you are bound to make an extreme boost in productivity.\r\nOne of the things I have a love-hate relationship with, is the option to configure which Visual Studio shortcuts to use vs. which Vim shortcuts to use. For example, Ctrl+V in Visual Studio is for paste where as in Vim its for visual selection. Because the options are given to the user about which mode to choose for each shortcut, I am eternally confused about which shortcut I configured for which mode and and which computer (you have to manually configure each set of of shortcuts on each system and each version of Visual Studio).\r\nThis is just me ranting about one bad thing out of a thousand good ones.\r\nWhether you are a Vim user or not, VsVim is bound to make you write faster code by giving you the editing powers of Vim.\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/76293c4d-8c16-4f4a-aee6-21f83a571496\" target=\"_blank\">CodeMaid</a></strong>\r\nCodeMaid was a extension I discovered by experimentation. I went about installing all the popular extensions. When I found CodeMaid, it was love at first sight.\r\nThere isn\'t a single thing that I don\'t love about CodeMaid. I use practically every feature they provide. Code cleaning, navigation (\"digging\"), reorganizing just by drag and drop in the panel, formatting, wrapping code. I could do on and on about CodeMaid. But I can never do the justice that <a href=\"http://www.codemaid.net/documentation/\" target=\"_blank\">CodeMaid\'s Documentation</a> can do to its power. So check the documentation out and see how useful it can be.\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/0e33cb22-d4ac-4f5a-902f-aff5177cc94d\" target=\"_blank\">Brace Completer</a></strong>\r\nThis is a simple extension with one purpose. Always close the bracket.\r\nYou can configure this extension about which brace style you follow, inline, tabbed, new line always etc.\r\nConfigure it once, and reap the benefits forever.\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/7179e851-a263-44b7-a177-1d31e33c84fd\" target=\"_blank\">Code Alignment</a></strong>\r\nI would confess that I have an OCD when it comes to code symmetry and alignment. I was everything to be aligned. And this extension does just that.\r\nTell it what to align by - commas, equal sign, dot, arrow, and even custom text. It will do all the spacing for you.\r\nWhen writing kernels with SIMD, where you are writing multiple lines of the same operation, this is one of the most useful extensions to use.\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/e792686d-542b-474a-8c55-630980e72c30\" target=\"_blank\">Indent Guides</a></strong>\r\nA lot of people do not like indent guides. In their opinion, it makes the code look \"crummy\", with all those characters/lines showing spaces/tabs.\r\nBut given my association with alignment, I find it useful. I do share the opinion that its not the cleanest bit of code to look at, but then you are writing 4 levels of nested for loops, it has its perks.\r\n\r\nComment below to share your opinion on your favorite extensions. And keep a watch out for our next Tools at ArrayFire blog.','Tools at ArrayFire','','inherit','open','closed','','8372-revision-v1','','','2015-06-17 16:10:18','2015-06-17 16:10:18','',8372,'http://sandbox.arrayfire.com/8372-revision-v1/',0,'revision','',0),(8378,11,'2015-06-17 16:10:58','2015-06-17 16:10:58','We are starting a new series here about the tools we use at ArrayFire to make writing better code more productive.\r\nWe will be talking about hardware, linux distros, IDEs, editors, debuggers, miscellaneous software or anything that we believe helps us in increasing our productivity.\r\nYou will hear the eternal developer battles of Vim vs Emacs, Visual Studio vs Eclipse, dual vs single monitor, even lights on vs lights off (and more recently controllable smart lightning).\r\nSo sit tight for what is going to be an exciting series of posts.\r\n\r\nI\'ll be taking the first shot at this.\r\n\r\nI had to give it a long thought about what I really wanted to write about as the first post. I was in a dilemma between my Vim setup, Screen, gDebugger and my Visual Studio setup.\r\nAfter a long thought, I decided to write about <strong>my favorite Visual Studio Extensions</strong>.\r\n\r\nHere is a list of Visual Studio Extensions I use:\r\nProductivity Power Tools\r\nVsVim\r\nCodeMaid\r\nBrace Completer\r\nCode Alignment\r\nIndent Guides\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/dbcb8670-889e-4a54-a226-a48a15e4cace\" target=\"_blank\">Productivity Power Tools</a>\r\n</strong>This is by far my most used extension. It is the first thing I download whenever I install Visual Studio.\r\nI can across this extension when I saw someone using the \"Full Map Mode\" on sublime text and wanted the same thing for Visual Studio. And voila, Productivity Power Tools.\r\nWhen I installed it, I had no idea of how useful this was going to be. I started off using the Map modes for all sorts of navigation.\r\nOne of the most useful things from PPT is that it highlights all the places where there is a syntax errors in the full map mode using red dots. Simply howering over these red dots will show you the line of the error and clicking it will take you to that line. Similarly, warnings are indicated by yellow dots.\r\nAnother useful part is where the full map mode can be handy in showing a line elsewhere in the file that you may need to refer to for something like function arguments.\r\n\r\nAnother favorite of mine is the \"Ctrl + Click\" Peek Definition. When working with a code base as big as ArrayFire, this comes in really handy. When you have a function that you want to peek into, ie. see its definition, simply Ctrl + Click this function. Not only functions, but any definition. If it is overloaded, then it will give you the option to click and select the definition you wish to see.\r\n\r\nI can write a whole blog post on Productivity Power Tools on its own. To simply summarize: if you do not have Productivity Power Tool, install it right now!\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/59ca71b3-a4a3-46ca-8fe1-0e90e3f79329/\" target=\"_blank\">VsVim</a></strong>\r\nFor a vim user, this is a must have. This extension provides all the best parts of Vim modes with the power of the Visual Studio environment. Simply copy your existing vimrc or configure VsVim manually. Either way, you are bound to make an extreme boost in productivity.\r\nOne of the things I have a love-hate relationship with, is the option to configure which Visual Studio shortcuts to use vs. which Vim shortcuts to use. For example, Ctrl+V in Visual Studio is for paste where as in Vim its for visual selection. Because the options are given to the user about which mode to choose for each shortcut, I am eternally confused about which shortcut I configured for which mode and and which computer (you have to manually configure each set of of shortcuts on each system and each version of Visual Studio).\r\nThis is just me ranting about one bad thing out of a thousand good ones.\r\nWhether you are a Vim user or not, VsVim is bound to make you write faster code by giving you the editing powers of Vim.\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/76293c4d-8c16-4f4a-aee6-21f83a571496\" target=\"_blank\">CodeMaid</a></strong>\r\nCodeMaid was a extension I discovered by experimentation. I went about installing all the popular extensions. When I found CodeMaid, it was love at first sight.\r\nThere isn\'t a single thing that I don\'t love about CodeMaid. I use practically every feature they provide. Code cleaning, navigation (\"digging\"), reorganizing just by drag and drop in the panel, formatting, wrapping code. I could do on and on about CodeMaid. But I can never do the justice that <a href=\"http://www.codemaid.net/documentation/\" target=\"_blank\">CodeMaid\'s Documentation</a> can do to its power. So check the documentation out and see how useful it can be.\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/0e33cb22-d4ac-4f5a-902f-aff5177cc94d\" target=\"_blank\">Brace Completer</a></strong>\r\nThis is a simple extension with one purpose. Always close the bracket.\r\nYou can configure this extension about which brace style you follow, inline, tabbed, new line always etc.\r\nConfigure it once, and reap the benefits forever.\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/7179e851-a263-44b7-a177-1d31e33c84fd\" target=\"_blank\">Code Alignment</a></strong>\r\nI would confess that I have an OCD when it comes to code symmetry and alignment. I was everything to be aligned. And this extension does just that.\r\nTell it what to align by - commas, equal sign, dot, arrow, and even custom text. It will do all the spacing for you.\r\nWhen writing kernels with SIMD, where you are writing multiple lines of the same operation, this is one of the most useful extensions to use.\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/e792686d-542b-474a-8c55-630980e72c30\" target=\"_blank\">Indent Guides</a></strong>\r\nA lot of people do not like indent guides. In their opinion, it makes the code look \"crummy\", with all those characters/lines showing spaces/tabs.\r\nBut given my association with alignment, I find it useful. I do share the opinion that its not the cleanest bit of code to look at, but then you are writing 4 levels of nested for loops, it has its perks.\r\n\r\n----------------------\r\nComment below to share your opinion on your favorite extensions. And keep a watch out for our next Tools at ArrayFire blog.','Tools at ArrayFire','','inherit','open','closed','','8372-revision-v1','','','2015-06-17 16:10:58','2015-06-17 16:10:58','',8372,'http://sandbox.arrayfire.com/8372-revision-v1/',0,'revision','',0),(8379,11,'2015-06-17 16:11:42','2015-06-17 16:11:42','We are starting a new series here about the tools we use at ArrayFire to make writing better code more productive.\r\nWe will be talking about hardware, linux distros, IDEs, editors, debuggers, miscellaneous software or anything that we believe helps us in increasing our productivity.\r\nYou will hear the eternal developer battles of Vim vs Emacs, Visual Studio vs Eclipse, dual vs single monitor, even lights on vs lights off (and more recently controllable smart lightning).\r\nSo sit tight for what is going to be an exciting series of posts.\r\n\r\nI\'ll be taking the first shot at this.\r\n\r\nI had to give it a long thought about what I really wanted to write about as the first post. I was in a dilemma between my Vim setup, Screen, gDebugger and my Visual Studio setup.\r\nAfter a long thought, I decided to write about <strong>my favorite Visual Studio Extensions</strong>.\r\n\r\nHere is a list of Visual Studio Extensions I use:\r\nProductivity Power Tools\r\nVsVim\r\nCodeMaid\r\nBrace Completer\r\nCode Alignment\r\nIndent Guides\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/dbcb8670-889e-4a54-a226-a48a15e4cace\" target=\"_blank\">Productivity Power Tools</a>\r\n</strong>This is by far my most used extension. It is the first thing I download whenever I install Visual Studio.\r\nI can across this extension when I saw someone using the \"Full Map Mode\" on sublime text and wanted the same thing for Visual Studio. And voila, Productivity Power Tools.\r\nWhen I installed it, I had no idea of how useful this was going to be. I started off using the Map modes for all sorts of navigation.\r\nOne of the most useful things from PPT is that it highlights all the places where there is a syntax errors in the full map mode using red dots. Simply howering over these red dots will show you the line of the error and clicking it will take you to that line. Similarly, warnings are indicated by yellow dots.\r\nAnother useful part is where the full map mode can be handy in showing a line elsewhere in the file that you may need to refer to for something like function arguments.\r\n\r\nAnother favorite of mine is the \"Ctrl + Click\" Peek Definition. When working with a code base as big as ArrayFire, this comes in really handy. When you have a function that you want to peek into, ie. see its definition, simply Ctrl + Click this function. Not only functions, but any definition. If it is overloaded, then it will give you the option to click and select the definition you wish to see.\r\n\r\nI can write a whole blog post on Productivity Power Tools on its own. To simply summarize: if you do not have Productivity Power Tool, install it right now!\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/59ca71b3-a4a3-46ca-8fe1-0e90e3f79329/\" target=\"_blank\">VsVim</a></strong>\r\nFor a vim user, this is a must have. This extension provides all the best parts of Vim modes with the power of the Visual Studio environment. Simply copy your existing vimrc or configure VsVim manually. Either way, you are bound to make an extreme boost in productivity.\r\nOne of the things I have a love-hate relationship with, is the option to configure which Visual Studio shortcuts to use vs. which Vim shortcuts to use. For example, Ctrl+V in Visual Studio is for paste where as in Vim its for visual selection. Because the options are given to the user about which mode to choose for each shortcut, I am eternally confused about which shortcut I configured for which mode and and which computer (you have to manually configure each set of of shortcuts on each system and each version of Visual Studio).\r\nThis is just me ranting about one bad thing out of a thousand good ones.\r\nWhether you are a Vim user or not, VsVim is bound to make you write faster code by giving you the editing powers of Vim.\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/76293c4d-8c16-4f4a-aee6-21f83a571496\" target=\"_blank\">CodeMaid</a></strong>\r\nCodeMaid was a extension I discovered by experimentation. I went about installing all the popular extensions. When I found CodeMaid, it was love at first sight.\r\nThere isn\'t a single thing that I don\'t love about CodeMaid. I use practically every feature they provide. Code cleaning, navigation (\"digging\"), reorganizing just by drag and drop in the panel, formatting, wrapping code. I could do on and on about CodeMaid. But I can never do the justice that <a href=\"http://www.codemaid.net/documentation/\" target=\"_blank\">CodeMaid\'s Documentation</a> can do to its power. So check the documentation out and see how useful it can be.\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/0e33cb22-d4ac-4f5a-902f-aff5177cc94d\" target=\"_blank\">Brace Completer</a></strong>\r\nThis is a simple extension with one purpose. Always close the bracket.\r\nYou can configure this extension about which brace style you follow, inline, tabbed, new line always etc.\r\nConfigure it once, and reap the benefits forever.\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/7179e851-a263-44b7-a177-1d31e33c84fd\" target=\"_blank\">Code Alignment</a></strong>\r\nI would confess that I have an OCD when it comes to code symmetry and alignment. I was everything to be aligned. And this extension does just that.\r\nTell it what to align by - commas, equal sign, dot, arrow, and even custom text. It will do all the spacing for you.\r\nWhen writing kernels with SIMD, where you are writing multiple lines of the same operation, this is one of the most useful extensions to use.\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/e792686d-542b-474a-8c55-630980e72c30\" target=\"_blank\">Indent Guides</a></strong>\r\nA lot of people do not like indent guides. In their opinion, it makes the code look \"crummy\", with all those characters/lines showing spaces/tabs.\r\nBut given my association with alignment, I find it useful. I do share the opinion that its not the cleanest bit of code to look at, but then you are writing 4 levels of nested for loops, it has its perks.\r\n<hr>\r\nComment below to share your opinion on your favorite extensions. And keep a watch out for our next Tools at ArrayFire blog.','Tools at ArrayFire','','inherit','open','closed','','8372-revision-v1','','','2015-06-17 16:11:42','2015-06-17 16:11:42','',8372,'http://sandbox.arrayfire.com/8372-revision-v1/',0,'revision','',0),(8380,11,'2015-06-17 16:12:09','2015-06-17 16:12:09','We are starting a new series here about the tools we use at ArrayFire to make writing better code more productive.\r\nWe will be talking about hardware, linux distros, IDEs, editors, debuggers, miscellaneous software or anything that we believe helps us in increasing our productivity.\r\nYou will hear the eternal developer battles of Vim vs Emacs, Visual Studio vs Eclipse, dual vs single monitor, even lights on vs lights off (and more recently controllable smart lightning).\r\nSo sit tight for what is going to be an exciting series of posts.\r\n\r\nI\'ll be taking the first shot at this.\r\n\r\nI had to give it a long thought about what I really wanted to write about as the first post. I was in a dilemma between my Vim setup, Screen, gDebugger and my Visual Studio setup.\r\nAfter a long thought, I decided to write about <strong>my favorite Visual Studio Extensions</strong>.\r\n\r\nHere is a list of Visual Studio Extensions I use:\r\n* Productivity Power Tools\r\n* VsVim\r\n* CodeMaid\r\n* Brace Completer\r\n* Code Alignment\r\n* Indent Guides\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/dbcb8670-889e-4a54-a226-a48a15e4cace\" target=\"_blank\">Productivity Power Tools</a>\r\n</strong>This is by far my most used extension. It is the first thing I download whenever I install Visual Studio.\r\nI can across this extension when I saw someone using the \"Full Map Mode\" on sublime text and wanted the same thing for Visual Studio. And voila, Productivity Power Tools.\r\nWhen I installed it, I had no idea of how useful this was going to be. I started off using the Map modes for all sorts of navigation.\r\nOne of the most useful things from PPT is that it highlights all the places where there is a syntax errors in the full map mode using red dots. Simply howering over these red dots will show you the line of the error and clicking it will take you to that line. Similarly, warnings are indicated by yellow dots.\r\nAnother useful part is where the full map mode can be handy in showing a line elsewhere in the file that you may need to refer to for something like function arguments.\r\n\r\nAnother favorite of mine is the \"Ctrl + Click\" Peek Definition. When working with a code base as big as ArrayFire, this comes in really handy. When you have a function that you want to peek into, ie. see its definition, simply Ctrl + Click this function. Not only functions, but any definition. If it is overloaded, then it will give you the option to click and select the definition you wish to see.\r\n\r\nI can write a whole blog post on Productivity Power Tools on its own. To simply summarize: if you do not have Productivity Power Tool, install it right now!\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/59ca71b3-a4a3-46ca-8fe1-0e90e3f79329/\" target=\"_blank\">VsVim</a></strong>\r\nFor a vim user, this is a must have. This extension provides all the best parts of Vim modes with the power of the Visual Studio environment. Simply copy your existing vimrc or configure VsVim manually. Either way, you are bound to make an extreme boost in productivity.\r\nOne of the things I have a love-hate relationship with, is the option to configure which Visual Studio shortcuts to use vs. which Vim shortcuts to use. For example, Ctrl+V in Visual Studio is for paste where as in Vim its for visual selection. Because the options are given to the user about which mode to choose for each shortcut, I am eternally confused about which shortcut I configured for which mode and and which computer (you have to manually configure each set of of shortcuts on each system and each version of Visual Studio).\r\nThis is just me ranting about one bad thing out of a thousand good ones.\r\nWhether you are a Vim user or not, VsVim is bound to make you write faster code by giving you the editing powers of Vim.\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/76293c4d-8c16-4f4a-aee6-21f83a571496\" target=\"_blank\">CodeMaid</a></strong>\r\nCodeMaid was a extension I discovered by experimentation. I went about installing all the popular extensions. When I found CodeMaid, it was love at first sight.\r\nThere isn\'t a single thing that I don\'t love about CodeMaid. I use practically every feature they provide. Code cleaning, navigation (\"digging\"), reorganizing just by drag and drop in the panel, formatting, wrapping code. I could do on and on about CodeMaid. But I can never do the justice that <a href=\"http://www.codemaid.net/documentation/\" target=\"_blank\">CodeMaid\'s Documentation</a> can do to its power. So check the documentation out and see how useful it can be.\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/0e33cb22-d4ac-4f5a-902f-aff5177cc94d\" target=\"_blank\">Brace Completer</a></strong>\r\nThis is a simple extension with one purpose. Always close the bracket.\r\nYou can configure this extension about which brace style you follow, inline, tabbed, new line always etc.\r\nConfigure it once, and reap the benefits forever.\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/7179e851-a263-44b7-a177-1d31e33c84fd\" target=\"_blank\">Code Alignment</a></strong>\r\nI would confess that I have an OCD when it comes to code symmetry and alignment. I was everything to be aligned. And this extension does just that.\r\nTell it what to align by - commas, equal sign, dot, arrow, and even custom text. It will do all the spacing for you.\r\nWhen writing kernels with SIMD, where you are writing multiple lines of the same operation, this is one of the most useful extensions to use.\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/e792686d-542b-474a-8c55-630980e72c30\" target=\"_blank\">Indent Guides</a></strong>\r\nA lot of people do not like indent guides. In their opinion, it makes the code look \"crummy\", with all those characters/lines showing spaces/tabs.\r\nBut given my association with alignment, I find it useful. I do share the opinion that its not the cleanest bit of code to look at, but then you are writing 4 levels of nested for loops, it has its perks.\r\n<hr>\r\nComment below to share your opinion on your favorite extensions. And keep a watch out for our next Tools at ArrayFire blog.','Tools at ArrayFire','','inherit','open','closed','','8372-revision-v1','','','2015-06-17 16:12:09','2015-06-17 16:12:09','',8372,'http://sandbox.arrayfire.com/8372-revision-v1/',0,'revision','',0),(8381,11,'2015-06-17 16:13:16','2015-06-17 16:13:16','We are starting a new series here about the tools we use at ArrayFire to make writing better code more productive.\r\nWe will be talking about hardware, linux distros, IDEs, editors, debuggers, miscellaneous software or anything that we believe helps us in increasing our productivity.\r\nYou will hear the eternal developer battles of Vim vs Emacs, Visual Studio vs Eclipse, dual vs single monitor, even lights on vs lights off (and more recently controllable smart lightning).\r\nSo sit tight for what is going to be an exciting series of posts.\r\n\r\nI\'ll be taking the first shot at this.\r\n\r\nI had to give it a long thought about what I really wanted to write about as the first post. I was in a dilemma between my Vim setup, Screen, gDebugger and my Visual Studio setup.\r\nAfter a long thought, I decided to write about <strong>my favorite Visual Studio Extensions</strong>.\r\n\r\nHere is a list of Visual Studio Extensions I use:\r\n* Productivity Power Tools\r\n* VsVim\r\n* CodeMaid\r\n* Brace Completer\r\n* Code Alignment\r\n* Indent Guides\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/dbcb8670-889e-4a54-a226-a48a15e4cace\" target=\"_blank\">Productivity Power Tools</a>\r\n</strong>This is by far my most used extension. It is the first thing I download whenever I install Visual Studio.\r\nI can across this extension when I saw someone using the \"Full Map Mode\" on sublime text and wanted the same thing for Visual Studio. And voila, Productivity Power Tools.\r\nWhen I installed it, I had no idea of how useful this was going to be. I started off using the Map modes for all sorts of navigation.\r\nOne of the most useful things from PPT is that it highlights all the places where there is a syntax errors in the full map mode using red dots. Simply howering over these red dots will show you the line of the error and clicking it will take you to that line. Similarly, warnings are indicated by yellow dots.\r\nAnother useful part is where the full map mode can be handy in showing a line elsewhere in the file that you may need to refer to for something like function arguments.\r\n\r\nAnother favorite of mine is the \"Ctrl + Click\" Peek Definition. When working with a code base as big as ArrayFire, this comes in really handy. When you have a function that you want to peek into, ie. see its definition, simply Ctrl + Click this function. Not only functions, but any definition. If it is overloaded, then it will give you the option to click and select the definition you wish to see.\r\n\r\nI can write a whole blog post on Productivity Power Tools on its own. To simply summarize: if you do not have Productivity Power Tool, install it right now!\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/59ca71b3-a4a3-46ca-8fe1-0e90e3f79329/\" target=\"_blank\">VsVim</a></strong>\r\nFor a vim user, this is a must have. This extension provides all the best parts of Vim modes with the power of the Visual Studio environment. Simply copy your existing vimrc or configure VsVim manually. Either way, you are bound to make an extreme boost in productivity.\r\nOne of the things I have a love-hate relationship with, is the option to configure which Visual Studio shortcuts to use vs. which Vim shortcuts to use. For example, Ctrl+V in Visual Studio is for paste where as in Vim its for visual selection. Because the options are given to the user about which mode to choose for each shortcut, I am eternally confused about which shortcut I configured for which mode and and which computer (you have to manually configure each set of of shortcuts on each system and each version of Visual Studio).\r\nThis is just me ranting about one bad thing out of a thousand good ones.\r\nWhether you are a Vim user or not, VsVim is bound to make you write faster code by giving you the editing powers of Vim.\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/76293c4d-8c16-4f4a-aee6-21f83a571496\" target=\"_blank\">CodeMaid</a></strong>\r\nCodeMaid was a extension I discovered by experimentation. I went about installing all the popular extensions. When I found CodeMaid, it was love at first sight.\r\nThere isn\'t a single thing that I don\'t love about CodeMaid. I use practically every feature they provide. Code cleaning, navigation (\"digging\"), reorganizing just by drag and drop in the panel, formatting, wrapping code. I could do on and on about CodeMaid. But I can never do the justice that <a href=\"http://www.codemaid.net/documentation/\" target=\"_blank\">CodeMaid\'s Documentation</a> can do to its power. So check the documentation out and see how useful it can be.\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/0e33cb22-d4ac-4f5a-902f-aff5177cc94d\" target=\"_blank\">Brace Completer</a></strong>\r\nThis is a simple extension with one purpose. Always close the bracket.\r\nYou can configure this extension about which brace style you follow, inline, tabbed, new line always etc.\r\nConfigure it once, and reap the benefits forever.\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/7179e851-a263-44b7-a177-1d31e33c84fd\" target=\"_blank\">Code Alignment</a></strong>\r\nI would confess that I have an OCD when it comes to code symmetry and alignment. I was everything to be aligned. And this extension does just that.\r\nTell it what to align by - commas, equal sign, dot, arrow, and even custom text. It will do all the spacing for you.\r\nWhen writing kernels with SIMD, where you are writing multiple lines of the same operation, this is one of the most useful extensions to use.\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/e792686d-542b-474a-8c55-630980e72c30\" target=\"_blank\">Indent Guides</a></strong>\r\nA lot of people do not like indent guides. In their opinion, it makes the code look \"crummy\", with all those characters/lines showing spaces/tabs.\r\nBut given my association with alignment, I find it useful. I do share the opinion that its not the cleanest bit of code to look at, but then you are writing 4 levels of nested for loops, it has its perks.\r\n<hr>\r\nComment below to share your opinion on your favorite extensions. And keep a watch out for our next Tools at ArrayFire blog.','Tools at ArrayFire - Visual Studio Extensions','','inherit','open','closed','','8372-revision-v1','','','2015-06-17 16:13:16','2015-06-17 16:13:16','',8372,'http://sandbox.arrayfire.com/8372-revision-v1/',0,'revision','',0),(8384,11,'2015-06-17 17:27:59','2015-06-17 17:27:59','We are starting a new series here about the tools we use at ArrayFire to make writing better code more productive.\r\n\r\nWe will be talking about hardware, linux distros, IDEs, editors, debuggers, miscellaneous software or anything that we believe helps us in increasing our productivity.\r\nYou will hear the eternal developer battles of Vim vs Emacs, Visual Studio vs Eclipse, dual vs single monitor, even lights on vs lights off (and more recently controllable smart lightning).\r\n\r\nSo sit tight for what is going to be an exciting series of posts.\r\n\r\nI\'ll be taking the first shot at this.\r\n\r\nI had to give it a long thought about what I really wanted to write about as the first post. I was in a dilemma between my Vim setup, Screen, gDebugger and my Visual Studio setup.\r\nAfter a long thought, I decided to write about <strong>my favorite Visual Studio Extensions</strong>.\r\n\r\nHere is a list of Visual Studio Extensions I use:\r\n* Productivity Power Tools\r\n* VsVim\r\n* CodeMaid\r\n* Brace Completer\r\n* Code Alignment\r\n* Indent Guides\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/dbcb8670-889e-4a54-a226-a48a15e4cace\" target=\"_blank\">Productivity Power Tools</a></strong>\r\nThis is by far my most used extension. It is the first thing I download whenever I install Visual Studio.\r\n\r\nI came across this extension when I saw someone using the \"Full Map Mode\" on sublime text and wanted the same thing for Visual Studio. And voila, Productivity Power Tools.\r\nWhen I installed it, I had no idea of how useful this was going to be. I started off using the Map modes for all sorts of navigation.\r\n\r\nOne of the most useful things from PPT is that it highlights all the places where there is a syntax errors in the full map mode using red dots. Simply howering over these red dots will show you the line of the error and clicking it will take you to that line. Similarly, warnings are indicated by yellow dots.\r\n\r\nAnother useful part is where the full map mode can be handy in showing a line elsewhere in the file that you may need to refer to for something like function arguments.\r\n\r\nThe \"Ctrl + Click\" Peek Definition is amazing. When working with a code base as big as ArrayFire, this comes in really handy. When you have a function that you want to peek into, ie. see its definition, simply Ctrl + Click this function. Not only functions, but any definition. If it is overloaded, then it will give you the option to click and select the definition you wish to see.\r\n\r\nI can write a whole blog post on Productivity Power Tools on its own. To simply summarize: if you do not have Productivity Power Tool, install it right now!\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/59ca71b3-a4a3-46ca-8fe1-0e90e3f79329/\" target=\"_blank\">VsVim</a></strong>\r\nFor a vim user, this is a must have. This extension provides all the best parts of Vim modes with the power of the Visual Studio environment. Simply copy your existing vimrc or configure VsVim manually. Either way, you are bound to make an extreme boost in productivity.\r\n\r\nOne of the things I have a love-hate relationship with, is the option to configure which Visual Studio shortcuts to use vs. which Vim shortcuts to use. For example, Ctrl+V in Visual Studio is for paste where as in Vim its for visual selection. Because the options are given to the user about which mode to choose for each shortcut, I am eternally confused about which shortcut I configured for which mode and and which computer (you have to manually configure each set of of shortcuts on each system and each version of Visual Studio).\r\n\r\nThis is just me ranting about one bad thing out of a thousand good ones.\r\n\r\nWhether you are a Vim user or not, VsVim is bound to make you write faster code by giving you the editing powers of Vim.\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/76293c4d-8c16-4f4a-aee6-21f83a571496\" target=\"_blank\">CodeMaid</a></strong>\r\nCodeMaid was a extension I discovered by experimentation. I went about installing all the popular extensions. When I found CodeMaid, it was love at first sight.\r\n\r\nThere isn\'t a single thing that I don\'t love about CodeMaid. I use practically every feature they provide. Code cleaning, navigation (\"digging\"), reorganizing just by drag and drop in the panel, formatting, wrapping code. I could do on and on about CodeMaid. But I can never do the justice that <a href=\"http://www.codemaid.net/documentation/\" target=\"_blank\">CodeMaid\'s Documentation</a> can do to its power. So check the documentation out and see how useful it can be.\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/0e33cb22-d4ac-4f5a-902f-aff5177cc94d\" target=\"_blank\">Brace Completer</a></strong>\r\nThis is a simple extension with one purpose. Always close the bracket.\r\n\r\nYou can configure this extension about which brace style you follow, inline, tabbed, new line always etc.\r\n\r\nConfigure it once, and reap the benefits forever.\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/7179e851-a263-44b7-a177-1d31e33c84fd\" target=\"_blank\">Code Alignment</a></strong>\r\nI would confess that I have an OCD when it comes to code symmetry and alignment. I was everything to be aligned. And this extension does just that.\r\n\r\nTell it what to align by - commas, equal sign, dot, arrow, and even custom text. It will do all the spacing for you.\r\n\r\nWhen writing kernels with SIMD, where you are writing multiple lines of the same operation, this is one of the most useful extensions to use.\r\n\r\n<strong><a href=\"https://visualstudiogallery.msdn.microsoft.com/e792686d-542b-474a-8c55-630980e72c30\" target=\"_blank\">Indent Guides</a></strong>\r\nA lot of people do not like indent guides. In their opinion, it makes the code look \"crummy\", with all those characters/lines showing spaces/tabs.\r\n\r\nBut given my association with alignment, I find it useful. I do share the opinion that its not the cleanest bit of code to look at, but then you are writing 4 levels of nested for loops, it has its perks.\r\n<hr>\r\nComment below to share your opinion on your favorite extensions. And keep a watch out for our next Tools at ArrayFire blog.','Tools at ArrayFire - Visual Studio Extensions','','inherit','open','closed','','8372-revision-v1','','','2015-06-17 17:27:59','2015-06-17 17:27:59','',8372,'http://sandbox.arrayfire.com/8372-revision-v1/',0,'revision','',0),(8385,17728,'2015-06-17 18:58:18','2015-06-17 18:58:18','<p>There are many open source libraries that implement parallel versions of the algorithms in the C++ standard template libraries. Inevitably we get asked questions about how ArrayFire compares to the other libraries out in the open.</p>\r\n<p>In this post we are going to compare the performance of ArrayFire to that of BoostCompute, HSA-Bolt, Intel TBB and Thrust. The benchmarks include the following commonly used vector algorithms across 3 different architectures.</p>\r\n<ul>\r\n<li><a href=\"#reductions\">Reductions</a></li>\r\n<li><a href=\"#scan\">Scan</a></li>\r\n<li><a href=\"#transform\">Transform</a></li>\r\n</ul>\r\n<p>The following setup has been used for the benchmarking purposes. The code to reproduce the benchmarks is linked at the bottom of the post. The hardware used for the benchmarks is listed below:</p>\r\n<ul>\r\n<li>NVIDIA Tesla K20</li>\r\n<li>AMD FirePro S10000</li>\r\n<li>Intel Xeon E5-2560v2</li>\r\n</ul>\r\n<h3 id=\"background\">Background</h3>\r\n<p><strong>ArrayFire</strong></p>\r\n<p>ArrayFire provides high performance primitives across a <a href=\"http://www.sandbox.arrayfire.com/docs/group__func__categories.htm\">wide spectrum of domains</a>. Objects of the base class, <a href=\"http://www.sandbox.arrayfire.com/docs/classaf_1_1array.htm\">array</a>, can support up to 4 dimensions. It enables users to take advantage of parallelism at multiple levels by <a href=\"http://sandbox.arrayfire.com/how-to-write-vectorized-code/\">writing vectorized code</a>.</p>\r\n<p>ArrayFire supports NVIDIA GPUs using CUDA and OpenCL, AMD GPUs using OpenCL and Intel / AMD CPUs in serial and parallel (using OpenCL) modes. For this post, ArrayFire 3.0 has been used. You can download ArrayFire from the following locations.</p>\r\n<ul>\r\n<li><a href=\"http://sandbox.arrayfire.com/download/\">Installers</a></li>\r\n<li><a href=\"https://github.com/arrayfire/arrayfire\">Build from source</a></li>\r\n</ul>\r\n<p><strong>BoostCompute</strong></p>\r\n<p>Boost Compute is C++ template library for parallel computing based on OpenCL. Boost Compute provides accelerated versions of the algorithms present in the standard template library. It also provides an alternative C++ wrapper over common OpenCL C routines.</p>\r\n<p>Boost Compute supports NVIDIA GPUs, AMD GPUs and Intel / AMD CPUs using OpenCL. You can download BoostCompute from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://github.com/boostorg/compute\">Header only library</a></li>\r\n</ul>\r\n<p>ArrayFire uses BoostCompute for sorting, set operations and finding regions in an image in our OpenCL backend.</p>\r\n<p><strong>Hsa-Bolt</strong></p>\r\n<p>Bolt is a C++ template library optimized for heterogenous computing. It supports routines built on top of OpenCL, C++ AMP, and TBB. It provides high performance implementations of common algorithms from the Standard Template Library.</p>\r\n<p>You can download the source to Hsa-Bolt from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://github.com/HSA-Libraries/Bolt\">Build from Source</a></li>\r\n</ul>\r\n<p><strong>Thrust</strong></p>\r\n<p>Thrust is a parallel algorithms library providing high level C++ interafce on top of CUDA, OpenMP and TBB. It resembles C++ standard template library and provides accelerated versions of the commonly used algorithms.</p>\r\n<p>Thrust supports NVIDIA GPUs using CUDA and Intel / AMD GPUs using OpenMP and TBB. You can download thrust from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://developer.nvidia.com/cuda-downloads\">From CUDA 7.0</a></li>\r\n<li><a href=\"https://github.com/thrust/thrust\">Header only library</a></li>\r\n</ul>\r\n<p>ArrayFire uses BoostCompute for sorting, set operations and finding regions in an image in our CUDA backend.</p>\r\n<p><strong>Intel TBB</strong></p>\r\n<p>Threading Building Blocks (TBB) is a C++ template library developed by intel for multi-core processors. It provides platform independent data structures and algorithms that decrease the complexity of working with multiple threads.</p>\r\n<p>TBB supports multi-core processors (including the Xeon Phi). You can download TBB from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://www.threadingbuildingblocks.org/download\">Source and binaries</a></li>\r\n</ul>\r\n<h3 id=\"reductions\">Reductions</h3>\r\n<p>The reduction algorithm reduces the vector into a single element by successively reducing two input elements to a single output element. The most commonly used operations are addition, minimum and maximum.</p>\r\n<p>It is one of the most commonly used algorithms in scientific and engineering algorithms. Most notably in the fields of statistics and machine learning.</p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"yes\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1374058645&amp;format=interactive\">\r\n</iframe>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"yes\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1835396253&amp;format=interactive\">\r\n</iframe>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"yes\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=492096912&amp;format=interactive\">\r\n</iframe>\r\n<p><strong>AMD FirePro S10000</strong></p>\r\n<ul>\r\n<li>For sizes less than 10 million elements, ArrayFire is the fastest followed by HSA-Bolt and BoostComput</li>\r\n<li>For sizes approaching 100 million elements, HSA-Bolt is 10% faster ArrayFire, and 15% faster than BoostCompute</li>\r\n</ul>\r\n<p><strong>NVIDIA Tesla K20</strong></p>\r\n<ul>\r\n<li>ArrayFire consistently performs better than other libraries for sizes less than 10 million elements</li>\r\n<li>ArrayFire OpenCL performs better than ArrayFire CUDA for sizes greater than 1 million elements.</li>\r\n<li>BoostCompute consistently performs better than Thrust</li>\r\n<li>For sizes approaching 100 million elements, ArrayFire OpenCL is fastest followed by BoostCompute, Thrust and ArrayFire CUDA</li>\r\n</ul>\r\n<p><strong>Intel Xeon E5-2560v2</strong></p>\r\n<ul>\r\n<li>Intel TBB is significantly faster than ArrayFire and BoostCompute for sizes less than 1 million elements</li>\r\n<li>ArrayFire is about 20% to 30% slower for sizes greater than 1 million elements</li>\r\n<li>BoostCompute is about 40% to 70% slower for sizes greater than 1 million elements</li>\r\n</ul>\r\n<h3 id=\"scan\">Scan</h3>\r\n<p>The scan algorithm is useful when performing condition select operations. It can also be used where cumulative distributions are needed.</p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1469250706&amp;format=interactive\">\r\n</iframe>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1909166539&amp;format=interactive\">\r\n</iframe>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=944922182&amp;format=interactive\">\r\n</iframe>\r\n<p><strong>AMD FirePro S10000</strong></p>\r\n<ul>\r\n<li>ArrayFire consistently performs better than other libraries</li>\r\n<li>HSA-Bolt consistently performs better than BoostCompute</li>\r\n</ul>\r\n<p><strong>NVIDIA Tesla K20</strong></p>\r\n<ul>\r\n<li>ArrayFire performs better than other libraries for sizes less than 1 million elements</li>\r\n<li>ArrayFire OpenCL performs better than ArrayFire CUDA for sizes greater than 100000 elements</li>\r\n<li>Thrust consistently performs better than BoostCompute</li>\r\n<li>For sizes greater than 1 million elements, Thrust is 16% faster than ArrayFire OpenCL, 22% faster than ArrayFire CUDA</li>\r\n</ul>\r\n<p><strong>Intel Xeon E5-2560v2</strong></p>\r\n<ul>\r\n<li>Intel TBB is significantly faster than ArrayFire and BoostCompute for all sizes</li>\r\n</ul>\r\n<h3 id=\"transform\">Transform</h3>\r\n<p>The transform operation performs the same operations on each element of the input array. For this benchmark a commonly used function called <code>saxpy</code> has been used.</p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=2057408484&amp;format=interactive\">\r\n</iframe>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=180276342&amp;format=interactive\">\r\n</iframe>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=2014239669&amp;format=interactive\">\r\n</iframe>\r\n<p><strong>AMD FirePro S10000</strong></p>\r\n<ul>\r\n<li>ArrayFire consistently performs better than other libraries</li>\r\n<li>BoostCompute consistently performs better than HSA-Bolt</li>\r\n</ul>\r\n<p><strong>NVIDIA Tesla K20</strong></p>\r\n<ul>\r\n<li>For sizes less than 1 million elements, Thrust is the fastest, followed by ArrayFire CUDA, ArrayFire OpenCL and BoostCompute</li>\r\n<li>For sizes greater than 1 million elements, BoostCompute is fastest followed by ArrayFire OpenCL, Thrust and ArrayFire CUDA</li>\r\n</ul>\r\n<p><strong>Intel Xeon E5-2560v2</strong></p>\r\n<ul>\r\n<li>For sizes less than 1 million elements, TBB is significantly faster than BoostCompute and ArrayFire</li>\r\n<li>For sizes larger than 1 million elements, all three libraries perform within 5% of each other</li>\r\n</ul>\r\n<h3 id=\"conclusion\">Conclusion</h3>\r\n<p>On AMD and NVIDIA GPUs, ArrayFire is the fastest library for vectors on the order of 10 million elements. At even smaller sizes, ArrayFire\'s memory manager ammortizes the memory allocation and deallocation time, making it significantly faster than the alternatives. While improvements can be made overall, one particular area that can be improved upon would be the scan algorithm at large sizes.</p>\r\n<p>On Intel Xeons, TBB is significantly faster than OpenCL implementations. Improvements can and should be made in ArrayFire to extract the best performance out of multicore CPUs. Our roadmap for the next quarter includes a native parallel implementation for our CPU backend. This should bridge the performance gap seen in these benchmarks.</p>\r\n<p>Each library performed better than the others at some point in the benchmarks. Considering that all the libraries are open source, these results should provide a motivation for these libraries to learn from each other.</p>\r\n<h3 id=\"links\">Links:</h3>\r\n<ul>\r\n<li><a href=\"https://github.com/arrayfire/valhalla\">Code</a></li>\r\n<li><a href=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/edit#gid=60225335\">Raw data</a></li>\r\n</ul>\r\n','Benchmarking parallel vector libraries','','inherit','open','closed','','8337-revision-v1','','','2015-06-17 18:58:18','2015-06-17 18:58:18','',8337,'http://sandbox.arrayfire.com/8337-revision-v1/',0,'revision','',0),(8386,17728,'2015-06-17 18:59:22','2015-06-17 18:59:22','<p>There are many open source libraries that implement parallel versions of the algorithms in the C++ standard template libraries. Inevitably we get asked questions about how ArrayFire compares to the other libraries out in the open.</p>\n<p>In this post we are going to compare the performance of ArrayFire to that of BoostCompute, HSA-Bolt, Intel TBB and Thrust. The benchmarks include the following commonly used vector algorithms across 3 different architectures.</p>\n<ul>\n<li><a href=\"#reductions\">Reductions</a></li>\n<li><a href=\"#scan\">Scan</a></li>\n<li><a href=\"#transform\">Transform</a></li>\n</ul>\n<p>The following setup has been used for the benchmarking purposes. The code to reproduce the benchmarks is linked at the bottom of the post. The hardware used for the benchmarks is listed below:</p>\n<ul>\n<li>NVIDIA Tesla K20</li>\n<li>AMD FirePro S10000</li>\n<li>Intel Xeon E5-2560v2</li>\n</ul>\n<h3 id=\"background\">Background</h3>\n<p><strong>ArrayFire</strong></p>\n<p>ArrayFire provides high performance primitives across a <a href=\"http://www.sandbox.arrayfire.com/docs/group__func__categories.htm\">wide spectrum of domains</a>. Objects of the base class, <a href=\"http://www.sandbox.arrayfire.com/docs/classaf_1_1array.htm\">array</a>, can support up to 4 dimensions. It enables users to take advantage of parallelism at multiple levels by <a href=\"http://sandbox.arrayfire.com/how-to-write-vectorized-code/\">writing vectorized code</a>.</p>\n<p>ArrayFire supports NVIDIA GPUs using CUDA and OpenCL, AMD GPUs using OpenCL and Intel / AMD CPUs in serial and parallel (using OpenCL) modes. For this post, ArrayFire 3.0 has been used. You can download ArrayFire from the following locations.</p>\n<ul>\n<li><a href=\"http://sandbox.arrayfire.com/download/\">Installers</a></li>\n<li><a href=\"https://github.com/arrayfire/arrayfire\">Build from source</a></li>\n</ul>\n<p><strong>BoostCompute</strong></p>\n<p>Boost Compute is C++ template library for parallel computing based on OpenCL. Boost Compute provides accelerated versions of the algorithms present in the standard template library. It also provides an alternative C++ wrapper over common OpenCL C routines.</p>\n<p>Boost Compute supports NVIDIA GPUs, AMD GPUs and Intel / AMD CPUs using OpenCL. You can download BoostCompute from the following locations.</p>\n<ul>\n<li><a href=\"https://github.com/boostorg/compute\">Header only library</a></li>\n</ul>\n<p>ArrayFire uses BoostCompute for sorting, set operations and finding regions in an image in our OpenCL backend.</p>\n<p><strong>Hsa-Bolt</strong></p>\n<p>Bolt is a C++ template library optimized for heterogenous computing. It supports routines built on top of OpenCL, C++ AMP, and TBB. It provides high performance implementations of common algorithms from the Standard Template Library.</p>\n<p>You can download the source to Hsa-Bolt from the following locations.</p>\n<ul>\n<li><a href=\"https://github.com/HSA-Libraries/Bolt\">Build from Source</a></li>\n</ul>\n<p><strong>Thrust</strong></p>\n<p>Thrust is a parallel algorithms library providing high level C++ interafce on top of CUDA, OpenMP and TBB. It resembles C++ standard template library and provides accelerated versions of the commonly used algorithms.</p>\n<p>Thrust supports NVIDIA GPUs using CUDA and Intel / AMD GPUs using OpenMP and TBB. You can download thrust from the following locations.</p>\n<ul>\n<li><a href=\"https://developer.nvidia.com/cuda-downloads\">From CUDA 7.0</a></li>\n<li><a href=\"https://github.com/thrust/thrust\">Header only library</a></li>\n</ul>\n<p>ArrayFire uses BoostCompute for sorting, set operations and finding regions in an image in our CUDA backend.</p>\n<p><strong>Intel TBB</strong></p>\n<p>Threading Building Blocks (TBB) is a C++ template library developed by intel for multi-core processors. It provides platform independent data structures and algorithms that decrease the complexity of working with multiple threads.</p>\n<p>TBB supports multi-core processors (including the Xeon Phi). You can download TBB from the following locations.</p>\n<ul>\n<li><a href=\"https://www.threadingbuildingblocks.org/download\">Source and binaries</a></li>\n</ul>\n<h3 id=\"reductions\">Reductions</h3>\n<p>The reduction algorithm reduces the vector into a single element by successively reducing two input elements to a single output element. The most commonly used operations are addition, minimum and maximum.</p>\n<p>It is one of the most commonly used algorithms in scientific and engineering algorithms. Most notably in the fields of statistics and machine learning.</p>\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"yes\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1374058645&amp;format=interactive\">\n</iframe>\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"yes\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1835396253&amp;format=interactive\">\n</iframe>\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"yes\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=492096912&amp;format=interactive\">\n</iframe>\n<p><strong>AMD FirePro S10000</strong></p>\n<ul>\n<li>For sizes less than 10 million elements, ArrayFire is the fastest followed by HSA-Bolt and BoostComput</li>\n<li>For sizes approaching 100 million elements, HSA-Bolt is 10% faster ArrayFire, and 15% faster than BoostCompute</li>\n</ul>\n<p><strong>NVIDIA Tesla K20</strong></p>\n<ul>\n<li>ArrayFire consistently performs better than other libraries for sizes less than 10 million elements</li>\n<li>ArrayFire OpenCL performs better than ArrayFire CUDA for sizes greater than 1 million elements.</li>\n<li>BoostCompute consistently performs better than Thrust</li>\n<li>For sizes approaching 100 million elements, ArrayFire OpenCL is fastest followed by BoostCompute, Thrust and ArrayFire CUDA</li>\n</ul>\n<p><strong>Intel Xeon E5-2560v2</strong></p>\n<ul>\n<li>Intel TBB is significantly faster than ArrayFire and BoostCompute for sizes less than 1 million elements</li>\n<li>ArrayFire is about 20% to 30% slower for sizes greater than 1 million elements</li>\n<li>BoostCompute is about 40% to 70% slower for sizes greater than 1 million elements</li>\n</ul>\n<h3 id=\"scan\">Scan</h3>\n<p>The scan algorithm is useful when performing condition select operations. It can also be used where cumulative distributions are needed.</p>\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"yes\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1469250706&amp;format=interactive\">\n</iframe>\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"yes\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1909166539&amp;format=interactive\">\n</iframe>\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"yes\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=944922182&amp;format=interactive\">\n</iframe>\n<p><strong>AMD FirePro S10000</strong></p>\n<ul>\n<li>ArrayFire consistently performs better than other libraries</li>\n<li>HSA-Bolt consistently performs better than BoostCompute</li>\n</ul>\n<p><strong>NVIDIA Tesla K20</strong></p>\n<ul>\n<li>ArrayFire performs better than other libraries for sizes less than 1 million elements</li>\n<li>ArrayFire OpenCL performs better than ArrayFire CUDA for sizes greater than 100000 elements</li>\n<li>Thrust consistently performs better than BoostCompute</li>\n<li>For sizes greater than 1 million elements, Thrust is 16% faster than ArrayFire OpenCL, 22% faster than ArrayFire CUDA</li>\n</ul>\n<p><strong>Intel Xeon E5-2560v2</strong></p>\n<ul>\n<li>Intel TBB is significantly faster than ArrayFire and BoostCompute for all sizes</li>\n</ul>\n<h3 id=\"transform\">Transform</h3>\n<p>The transform operation performs the same operations on each element of the input array. For this benchmark a commonly used function called <code>saxpy</code> has been used.</p>\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"yes\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=2057408484&amp;format=interactive\">\n</iframe>\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=180276342&amp;format=interactive\">\n</iframe>\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=2014239669&amp;format=interactive\">\n</iframe>\n<p><strong>AMD FirePro S10000</strong></p>\n<ul>\n<li>ArrayFire consistently performs better than other libraries</li>\n<li>BoostCompute consistently performs better than HSA-Bolt</li>\n</ul>\n<p><strong>NVIDIA Tesla K20</strong></p>\n<ul>\n<li>For sizes less than 1 million elements, Thrust is the fastest, followed by ArrayFire CUDA, ArrayFire OpenCL and BoostCompute</li>\n<li>For sizes greater than 1 million elements, BoostCompute is fastest followed by ArrayFire OpenCL, Thrust and ArrayFire CUDA</li>\n</ul>\n<p><strong>Intel Xeon E5-2560v2</strong></p>\n<ul>\n<li>For sizes less than 1 million elements, TBB is significantly faster than BoostCompute and ArrayFire</li>\n<li>For sizes larger than 1 million elements, all three libraries perform within 5% of each other</li>\n</ul>\n<h3 id=\"conclusion\">Conclusion</h3>\n<p>On AMD and NVIDIA GPUs, ArrayFire is the fastest library for vectors on the order of 10 million elements. At even smaller sizes, ArrayFire\'s memory manager ammortizes the memory allocation and deallocation time, making it significantly faster than the alternatives. While improvements can be made overall, one particular area that can be improved upon would be the scan algorithm at large sizes.</p>\n<p>On Intel Xeons, TBB is significantly faster than OpenCL implementations. Improvements can and should be made in ArrayFire to extract the best performance out of multicore CPUs. Our roadmap for the next quarter includes a native parallel implementation for our CPU backend. This should bridge the performance gap seen in these benchmarks.</p>\n<p>Each library performed better than the others at some point in the benchmarks. Considering that all the libraries are open source, these results should provide a motivation for these libraries to learn from each other.</p>\n<h3 id=\"links\">Links:</h3>\n<ul>\n<li><a href=\"https://github.com/arrayfire/valhalla\">Code</a></li>\n<li><a href=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/edit#gid=60225335\">Raw data</a></li>\n</ul>\n','Benchmarking parallel vector libraries','','inherit','open','closed','','8337-autosave-v1','','','2015-06-17 18:59:22','2015-06-17 18:59:22','',8337,'http://sandbox.arrayfire.com/8337-autosave-v1/',0,'revision','',0),(8387,17728,'2015-06-17 18:59:32','2015-06-17 18:59:32','<p>There are many open source libraries that implement parallel versions of the algorithms in the C++ standard template libraries. Inevitably we get asked questions about how ArrayFire compares to the other libraries out in the open.</p>\r\n<p>In this post we are going to compare the performance of ArrayFire to that of BoostCompute, HSA-Bolt, Intel TBB and Thrust. The benchmarks include the following commonly used vector algorithms across 3 different architectures.</p>\r\n<ul>\r\n<li><a href=\"#reductions\">Reductions</a></li>\r\n<li><a href=\"#scan\">Scan</a></li>\r\n<li><a href=\"#transform\">Transform</a></li>\r\n</ul>\r\n<p>The following setup has been used for the benchmarking purposes. The code to reproduce the benchmarks is linked at the bottom of the post. The hardware used for the benchmarks is listed below:</p>\r\n<ul>\r\n<li>NVIDIA Tesla K20</li>\r\n<li>AMD FirePro S10000</li>\r\n<li>Intel Xeon E5-2560v2</li>\r\n</ul>\r\n<h3 id=\"background\">Background</h3>\r\n<p><strong>ArrayFire</strong></p>\r\n<p>ArrayFire provides high performance primitives across a <a href=\"http://www.sandbox.arrayfire.com/docs/group__func__categories.htm\">wide spectrum of domains</a>. Objects of the base class, <a href=\"http://www.sandbox.arrayfire.com/docs/classaf_1_1array.htm\">array</a>, can support up to 4 dimensions. It enables users to take advantage of parallelism at multiple levels by <a href=\"http://sandbox.arrayfire.com/how-to-write-vectorized-code/\">writing vectorized code</a>.</p>\r\n<p>ArrayFire supports NVIDIA GPUs using CUDA and OpenCL, AMD GPUs using OpenCL and Intel / AMD CPUs in serial and parallel (using OpenCL) modes. For this post, ArrayFire 3.0 has been used. You can download ArrayFire from the following locations.</p>\r\n<ul>\r\n<li><a href=\"http://sandbox.arrayfire.com/download/\">Installers</a></li>\r\n<li><a href=\"https://github.com/arrayfire/arrayfire\">Build from source</a></li>\r\n</ul>\r\n<p><strong>BoostCompute</strong></p>\r\n<p>Boost Compute is C++ template library for parallel computing based on OpenCL. Boost Compute provides accelerated versions of the algorithms present in the standard template library. It also provides an alternative C++ wrapper over common OpenCL C routines.</p>\r\n<p>Boost Compute supports NVIDIA GPUs, AMD GPUs and Intel / AMD CPUs using OpenCL. You can download BoostCompute from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://github.com/boostorg/compute\">Header only library</a></li>\r\n</ul>\r\n<p>ArrayFire uses BoostCompute for sorting, set operations and finding regions in an image in our OpenCL backend.</p>\r\n<p><strong>Hsa-Bolt</strong></p>\r\n<p>Bolt is a C++ template library optimized for heterogenous computing. It supports routines built on top of OpenCL, C++ AMP, and TBB. It provides high performance implementations of common algorithms from the Standard Template Library.</p>\r\n<p>You can download the source to Hsa-Bolt from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://github.com/HSA-Libraries/Bolt\">Build from Source</a></li>\r\n</ul>\r\n<p><strong>Thrust</strong></p>\r\n<p>Thrust is a parallel algorithms library providing high level C++ interafce on top of CUDA, OpenMP and TBB. It resembles C++ standard template library and provides accelerated versions of the commonly used algorithms.</p>\r\n<p>Thrust supports NVIDIA GPUs using CUDA and Intel / AMD GPUs using OpenMP and TBB. You can download thrust from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://developer.nvidia.com/cuda-downloads\">From CUDA 7.0</a></li>\r\n<li><a href=\"https://github.com/thrust/thrust\">Header only library</a></li>\r\n</ul>\r\n<p>ArrayFire uses BoostCompute for sorting, set operations and finding regions in an image in our CUDA backend.</p>\r\n<p><strong>Intel TBB</strong></p>\r\n<p>Threading Building Blocks (TBB) is a C++ template library developed by intel for multi-core processors. It provides platform independent data structures and algorithms that decrease the complexity of working with multiple threads.</p>\r\n<p>TBB supports multi-core processors (including the Xeon Phi). You can download TBB from the following locations.</p>\r\n<ul>\r\n<li><a href=\"https://www.threadingbuildingblocks.org/download\">Source and binaries</a></li>\r\n</ul>\r\n<h3 id=\"reductions\">Reductions</h3>\r\n<p>The reduction algorithm reduces the vector into a single element by successively reducing two input elements to a single output element. The most commonly used operations are addition, minimum and maximum.</p>\r\n<p>It is one of the most commonly used algorithms in scientific and engineering algorithms. Most notably in the fields of statistics and machine learning.</p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"yes\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1374058645&amp;format=interactive\">\r\n</iframe>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"yes\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1835396253&amp;format=interactive\">\r\n</iframe>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"yes\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=492096912&amp;format=interactive\">\r\n</iframe>\r\n<p><strong>AMD FirePro S10000</strong></p>\r\n<ul>\r\n<li>For sizes less than 10 million elements, ArrayFire is the fastest followed by HSA-Bolt and BoostComput</li>\r\n<li>For sizes approaching 100 million elements, HSA-Bolt is 10% faster ArrayFire, and 15% faster than BoostCompute</li>\r\n</ul>\r\n<p><strong>NVIDIA Tesla K20</strong></p>\r\n<ul>\r\n<li>ArrayFire consistently performs better than other libraries for sizes less than 10 million elements</li>\r\n<li>ArrayFire OpenCL performs better than ArrayFire CUDA for sizes greater than 1 million elements.</li>\r\n<li>BoostCompute consistently performs better than Thrust</li>\r\n<li>For sizes approaching 100 million elements, ArrayFire OpenCL is fastest followed by BoostCompute, Thrust and ArrayFire CUDA</li>\r\n</ul>\r\n<p><strong>Intel Xeon E5-2560v2</strong></p>\r\n<ul>\r\n<li>Intel TBB is significantly faster than ArrayFire and BoostCompute for sizes less than 1 million elements</li>\r\n<li>ArrayFire is about 20% to 30% slower for sizes greater than 1 million elements</li>\r\n<li>BoostCompute is about 40% to 70% slower for sizes greater than 1 million elements</li>\r\n</ul>\r\n<h3 id=\"scan\">Scan</h3>\r\n<p>The scan algorithm is useful when performing condition select operations. It can also be used where cumulative distributions are needed.</p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"yes\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1469250706&amp;format=interactive\">\r\n</iframe>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"yes\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=1909166539&amp;format=interactive\">\r\n</iframe>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"yes\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=944922182&amp;format=interactive\">\r\n</iframe>\r\n<p><strong>AMD FirePro S10000</strong></p>\r\n<ul>\r\n<li>ArrayFire consistently performs better than other libraries</li>\r\n<li>HSA-Bolt consistently performs better than BoostCompute</li>\r\n</ul>\r\n<p><strong>NVIDIA Tesla K20</strong></p>\r\n<ul>\r\n<li>ArrayFire performs better than other libraries for sizes less than 1 million elements</li>\r\n<li>ArrayFire OpenCL performs better than ArrayFire CUDA for sizes greater than 100000 elements</li>\r\n<li>Thrust consistently performs better than BoostCompute</li>\r\n<li>For sizes greater than 1 million elements, Thrust is 16% faster than ArrayFire OpenCL, 22% faster than ArrayFire CUDA</li>\r\n</ul>\r\n<p><strong>Intel Xeon E5-2560v2</strong></p>\r\n<ul>\r\n<li>Intel TBB is significantly faster than ArrayFire and BoostCompute for all sizes</li>\r\n</ul>\r\n<h3 id=\"transform\">Transform</h3>\r\n<p>The transform operation performs the same operations on each element of the input array. For this benchmark a commonly used function called <code>saxpy</code> has been used.</p>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"yes\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=2057408484&amp;format=interactive\">\r\n</iframe>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"yes\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=180276342&amp;format=interactive\">\r\n</iframe>\r\n<iframe width=\"540\" height=\"360\" seamless frameborder=\"0\" scrolling=\"yes\" src=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/pubchart?oid=2014239669&amp;format=interactive\">\r\n</iframe>\r\n<p><strong>AMD FirePro S10000</strong></p>\r\n<ul>\r\n<li>ArrayFire consistently performs better than other libraries</li>\r\n<li>BoostCompute consistently performs better than HSA-Bolt</li>\r\n</ul>\r\n<p><strong>NVIDIA Tesla K20</strong></p>\r\n<ul>\r\n<li>For sizes less than 1 million elements, Thrust is the fastest, followed by ArrayFire CUDA, ArrayFire OpenCL and BoostCompute</li>\r\n<li>For sizes greater than 1 million elements, BoostCompute is fastest followed by ArrayFire OpenCL, Thrust and ArrayFire CUDA</li>\r\n</ul>\r\n<p><strong>Intel Xeon E5-2560v2</strong></p>\r\n<ul>\r\n<li>For sizes less than 1 million elements, TBB is significantly faster than BoostCompute and ArrayFire</li>\r\n<li>For sizes larger than 1 million elements, all three libraries perform within 5% of each other</li>\r\n</ul>\r\n<h3 id=\"conclusion\">Conclusion</h3>\r\n<p>On AMD and NVIDIA GPUs, ArrayFire is the fastest library for vectors on the order of 10 million elements. At even smaller sizes, ArrayFire\'s memory manager ammortizes the memory allocation and deallocation time, making it significantly faster than the alternatives. While improvements can be made overall, one particular area that can be improved upon would be the scan algorithm at large sizes.</p>\r\n<p>On Intel Xeons, TBB is significantly faster than OpenCL implementations. Improvements can and should be made in ArrayFire to extract the best performance out of multicore CPUs. Our roadmap for the next quarter includes a native parallel implementation for our CPU backend. This should bridge the performance gap seen in these benchmarks.</p>\r\n<p>Each library performed better than the others at some point in the benchmarks. Considering that all the libraries are open source, these results should provide a motivation for these libraries to learn from each other.</p>\r\n<h3 id=\"links\">Links:</h3>\r\n<ul>\r\n<li><a href=\"https://github.com/arrayfire/valhalla\">Code</a></li>\r\n<li><a href=\"https://docs.google.com/spreadsheets/d/1eWmehHURrormikwlS0ZtlW2cF6D54H4jRoBekDgfpfk/edit#gid=60225335\">Raw data</a></li>\r\n</ul>\r\n','Benchmarking parallel vector libraries','','inherit','open','closed','','8337-revision-v1','','','2015-06-17 18:59:32','2015-06-17 18:59:32','',8337,'http://sandbox.arrayfire.com/8337-revision-v1/',0,'revision','',0),(8388,1,'2015-06-17 21:38:07','2015-06-17 21:38:07','','Order &ndash; June 17, 2015 @ 09:38 PM','','wc-completed','open','closed','order_5581e8bb2cd1d','order-jun-17-2015-0938-pm','','','2015-06-17 21:38:09','2015-06-17 21:38:09','',0,'http://sandbox.arrayfire.com/?post_type=shop_order&#038;p=8388',0,'shop_order','',4),(8389,11,'2015-06-22 18:23:29','2015-06-22 18:23:29','<h3>Introduction</h3>\r\nGPUs are really good at doing math. The Achilles heel is when it comes to 64-bit double precision math. GPUs, at least consumer grade, are not built for high performance FP64. This is because they are targeted towards gamers and game developers, who do not really care about high precision compute. So vendors like NVIDIA and AMD do not cram FP64 compute cores in their GPUs.\r\n\r\nFor example, on a GTX 780 Ti, the FP64 performance is 1/24 FP32. Which means in an ideal case, running the same code by only changing float types to double types, would yield the single precision run time to be about 1/24th of the double precision time (time(FP32) = time(FP64)/24).\r\n\r\nKeep in mind, for compute-bound algorithms, such as GEMM and FFT, the theoretical best case for FP64 performance is 1:2 FP32, simply because it involves computing with double the number of bits as FP32. If the algorithms are memory bound, such as matrix transpose, then most GPUs will attain the 1:2 performance. The numbers we discuss below will all be compute-bound performance numbers.\r\n\r\nHow double precision performs really depends on the architecture of the GPU.\r\n\r\n<h3>Comparing NVIDIA GPUs</h3>\r\nLets take three almost identical cards: <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-780-ti\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-titan-black\" target=\"_blank\">GTX Titan Black</a> and the <a href=\"http://www.nvidia.com/object/tesla-servers.html\" target=\"_blank\">Tesla K40c</a>. All are Kepler <a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">GK110</a> based GPUs, with the same number of SMX and cores (15 SMX, 2880 cores) and the same bus width (384-bit). The 780 Ti and Titan Black even have nearly same base clock speeds (~880MHz; K40c is 745MHz) and identical memory clock speeds (7GHz; K40 is 6GHz).\r\n\r\n<strong>Single Precision Performance</strong>\r\nWith respect to single precision performance, all three are fairly in the same ball park. Which the 780 Ti and Titan Black sit just around 5.1 TFlops, owing to their similar clock speeds, the Tesla K40c drops in at 4.3 TFlops. You could give the Tesla a pass because it has a lower clock speed.\r\n\r\nAll three only vary significantly in three categories.\r\n\r\n<strong>1. Memory</strong>\r\nThis doesn\'t affect performance very much (at least when the sizes fit in all GPUs). The price difference based on market prices of other GPUs with similar memory size variations should not be that big either.\r\n\r\n<strong>2. Price</strong>\r\nAt launch, the GTX 780 Ti was priced at $699, $999 for the Titan Black and an estimated $5500 for the K40c. Well, that escalated quickly.\r\n(Price Sources: <a href=\"http://www.anandtech.com/show/7492/the-geforce-gtx-780-ti-review\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.anandtech.com/show/7765/nvidias-geforce-gtx-titan-black-no-compromises-for-gaming-compute\" target=\"_blank\">GTX Titan Black</a>, <a href=\"http://www.anandtech.com/show/7521/nvidia-launches-tesla-k40\" target=\"_blank\">K40</a>)\r\n\r\n<strong>3. Double Precision Performance</strong>\r\nSo if the 780 Ti and the Titan Black are practically the same in every respect, why is there a $300 difference in their price at launch (discounting memory size difference)?\r\n\r\nThe answer is in the double precision capabilities. The 780 Ti is physically locked at 1:24 FP32 where has the Titan Black has an ace up it\'s sleeve.\r\n\r\nFor the Titan Black, the magic happens in the driver. The Titan black\'s driver gives the user an option to choose the double precision performance between 1:3 and 1:24 FP32 (by switching the GPU to TCC mode). When the double precision performance is set to 1:24 FP32, which is the same as the 780 Ti, the the single precision performance of the Titan Black and 780 Ti are identical. But when the user sets the double precision performance to 1:3 FP32, the single precision performance is compromised to boost double precision performance and make it equal to the K40c. In other words, you can choose the performance of the Titan Black to match either the 780 Ti or the K40c based and your preference.\r\n\r\nThe K40c has a double precision performance of 1:3 without compromising the single precision performance. This is because the K40 is given a special double precision unit for every 3 single precision cores (<a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">white paper</a>). It combines the best of both worlds. NVIDIA also states that the Tesla GPUs go through a much more rigorous Q&A process which guarantees lesser failures and also has additional features such as ECC memory. Hence the large price difference.\r\n\r\n<h4>Summary of NVIDIA GPUs</h4>\r\n<strong>NVIDIA\'s GTX series</strong> are known for their great FP32 performance but are very poor in their FP64 performance. The performance generally ranges between <strong>1:24 (Kepler) and 1:32 (Maxwell)</strong>. The exceptions to this are the GTX Titan cards which blur the lines between the consumer GTX series and the professional Tesla/Quadro cards.\r\n\r\nThe <strong>Kepler architecture Quadro and Tesla series</strong> card provide full double precision performance with <strong>1:3 FP32</strong>. However, with the <strong>Quadro M6000</strong>, NVIDIA has decided to provide only minimal FP64 performance by giving it only <strong>1:32 of FP32 capability</strong> and touting the M6000 as the best graphics card rather than the best graphics+compute card like the Quadro K6000.\r\n\r\n<h3>AMD GPUs</h3>\r\n\r\nAMD GPUs perform fairly well for FP64 compared to FP32. Most AMD cards (including consumer/gaming series) will give between <strong>1:3 and 1:8 FP32 performance for FP64</strong>. The AMD Tahiti architectures tested in these benchmarks here do not suffer from the same problems FP64 problems as NVIDIA\'s GTX series and give a 1:4 performance. Newer Hawaii architecture consumer grade GPUs are expected to provide 1:8 performance.\r\n\r\nThe <strong>FirePro W9100, W8100 and S9150 will give you an incredible FP64 1:2 FP32 performance</strong>.\r\n\r\nOverall, AMD GPUs hold a reputation for good double precision performance ratios compared to their NVIDIA counterparts.\r\n\r\n<h3>Benchmarks</h3>\r\nHere are some benchmarks comparaing FP32 and FP64 GEMM.\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=381229711&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=722963661&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1083990145&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1438814030&amp;format=interactive\"></iframe>\r\n\r\n<h3>Conclusion</h3>\r\nThere is no perfect solution to the problem of choosing between FP32 and FP64 when it comes to GPUs. The best users can do is to decide whether they need the accuracy of double precision compute and how much compute they need, eventually going for a GPU that maximizes their productivity. Some applications like physics modelling and simulation, high accuracy financial computations etc which call for double precision accuracy at high performance require capable FP64 cards. Applications such as image and video processing, signal processing, statistics may not require such high precision and can get away with high FP32 performance only.\r\n\r\nDisclaimer: We receive GPUs from both NVIDIA and AMD. However, this blog does not promote or advocate any specific vendor.\r\n\r\nSources:\r\nWikipedia: <a href=\"https://en.wikipedia.org/wiki/GeForce_700_series\" target=\"_blank\">NVIDIA GeForce 700 Series</a>, <a href=\"https://en.wikipedia.org/wiki/Nvidia_Tesla\" target=\"_blank\">NVIDIA Tesla Series</a>, <a href=\"https://en.wikipedia.org/wiki/Nvidia_Quadro\" target=\"_blank\">NVIDIA Quadro Series</a>, <a href=\"https://en.wikipedia.org/wiki/List_of_AMD_graphics_processing_units#Comparison_table:_Workstation_GPUs\" target=\"_blank\">AMD FirePro Series</a>, <a href=\"https://en.wikipedia.org/wiki/List_of_AMD_graphics_processing_units\" target=\"_blank\">AMD GPUs</a>\r\nNVIDIA Website: <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-780-ti\" target=\"_blank\">NVIDIA GTX 780 Ti</a>, <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-titan-black\" target=\"_blank\">NVIDIA GTX Titan Black</a>, <a href=\"http://www.nvidia.com/object/tesla-servers.html\" target=\"_blank\">NVIDIA Tesla K40</a>\r\nAnandtech: <a href=\"http://www.anandtech.com/show/7492/the-geforce-gtx-780-ti-review\" target=\"_blank\">NVIDIA GTX 780 Ti</a>, <a href=\"http://www.anandtech.com/show/7765/nvidias-geforce-gtx-titan-black-no-compromises-for-gaming-compute\" target=\"_blank\">NVIDIA GTX Titan Black</a>, <a href=\"http://www.anandtech.com/show/7521/nvidia-launches-tesla-k40\" target=\"_blank\">NVIDIA Tesla K40</a>','Explaining FP64 performance on GPUs','','publish','open','closed','','explaining-fp64-performance-on-gpus','','','2015-06-24 19:01:22','2015-06-24 19:01:22','',0,'http://sandbox.arrayfire.com/?p=8389',0,'post','',12),(8390,11,'2015-06-22 12:55:23','2015-06-22 12:55:23','Let\'s start off with 2 charts.\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=381229711&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=722963661&amp;format=interactive\"></iframe>','Why FP64 performance on GPUs is so bad','','inherit','open','closed','','8389-revision-v1','','','2015-06-22 12:55:23','2015-06-22 12:55:23','',8389,'http://sandbox.arrayfire.com/8389-revision-v1/',0,'revision','',0),(8391,11,'2015-06-22 12:56:22','2015-06-22 12:56:22','Let\'s start off with 2 charts.\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=381229711&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=722963661&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1083990145&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1438814030&amp;format=interactive\"></iframe>','Why FP64 performance on GPUs is so bad','','inherit','open','closed','','8389-revision-v1','','','2015-06-22 12:56:22','2015-06-22 12:56:22','',8389,'http://sandbox.arrayfire.com/8389-revision-v1/',0,'revision','',0),(8392,11,'2015-06-22 13:34:15','2015-06-22 13:34:15','GPUs are really good at doing math. The Achilles heel is when it comes to 64-bit double precision math. GPUs, at least consumer grade, are not built for high performance FP64. This is because they are targetted towards gamers and game developers, who do not really care about high precision compute. So vendors like NVIDIA and AMD do not cram FP64 compute cores in their GPUs.\r\nFor example, on a GTX 780, the FP64 performance is 1/24 FP32. Which means in an ideal case, running the same code by only changing float types to double types, would yield the double precision run time to be about 1/24th of the single precision time.\r\n\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=381229711&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=722963661&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1083990145&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1438814030&amp;format=interactive\"></iframe>\r\n\r\n','Explainging FP64 performance on GPUs','','inherit','open','closed','','8389-revision-v1','','','2015-06-22 13:34:15','2015-06-22 13:34:15','',8389,'http://sandbox.arrayfire.com/8389-revision-v1/',0,'revision','',0),(8393,11,'2015-06-22 13:34:29','2015-06-22 13:34:29','GPUs are really good at doing math. The Achilles heel is when it comes to 64-bit double precision math. GPUs, at least consumer grade, are not built for high performance FP64. This is because they are targetted towards gamers and game developers, who do not really care about high precision compute. So vendors like NVIDIA and AMD do not cram FP64 compute cores in their GPUs.\r\nFor example, on a GTX 780, the FP64 performance is 1/24 FP32. Which means in an ideal case, running the same code by only changing float types to double types, would yield the double precision run time to be about 1/24th of the single precision time.\r\n\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=381229711&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=722963661&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1083990145&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1438814030&amp;format=interactive\"></iframe>\r\n\r\n','Explaining FP64 performance on GPUs','','inherit','open','closed','','8389-revision-v1','','','2015-06-22 13:34:29','2015-06-22 13:34:29','',8389,'http://sandbox.arrayfire.com/8389-revision-v1/',0,'revision','',0),(8394,11,'2015-06-22 14:13:00','2015-06-22 14:13:00','GPUs are really good at doing math. The Achilles heel is when it comes to 64-bit double precision math. GPUs, at least consumer grade, are not built for high performance FP64. This is because they are targetted towards gamers and game developers, who do not really care about high precision compute. So vendors like NVIDIA and AMD do not cram FP64 compute cores in their GPUs.\r\nFor example, on a GTX 780, the FP64 performance is 1/24 FP32. Which means in an ideal case, running the same code by only changing float types to double types, would yield the double precision run time to be about 1/24th of the single precision time.\r\n\r\nHow double precision performs really depends on the architecture of the GPU. Lets take 3 almost identical cards: GTX 780 Ti, GTX Titan Black and the Tesla K40c. All are <a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">GK110</a> based GPUs, with the same number of SMX and cores (15 SMX, 2880 cores) and the same bus width (384-bit). The 780 Ti and Titan Black even have nearly same base clock speeds (~880MHz; K40c is 745MHz) and identical memory clock speeds (7GHz; K40 is 6GHz).\r\nWith respect to single precision performance, all 3 are fairly in the same ball park. Which the 780 Ti and Titan Black sit just around 5.1 TFlops, owing to their similar clock speeds, the Tesla K40c drops in at 4.3 TFlops. You could give the Tesla a pass because it has a lower clock speed.\r\n\r\nAll 3 only vary significantly in 3 categories.\r\n\r\n1. Memory\r\nThis doesn\'t affect performance very much (at least when the sizes fit in all GPUs). The price difference based on market prices of other GPUs with similar memory size variations should not be that big either.\r\n\r\n2. Price\r\nAt launch, the GTX 780 Ti was priced at $699, $999 for the Titan Black and an estimated $5500 for the K40c. Well, that escalated quickly.\r\n(Price Sources: <a href=\"http://www.anandtech.com/show/7492/the-geforce-gtx-780-ti-review\" target=\"_blank\">GTX 780 Ti</a> <a href=\"http://www.anandtech.com/show/7765/nvidias-geforce-gtx-titan-black-no-compromises-for-gaming-compute\" target=\"_blank\">GTX Titan Black</a> <a href=\"http://www.anandtech.com/show/7521/nvidia-launches-tesla-k40\" target=\"_blank\">K40</a>)\r\n\r\n3. Double Precision Performance\r\nSo if the 780 Ti and the Titan Black are practically the same in every respect, why is there a $300 difference in their price at launch?\r\nThe 780 Ti is physically locked at 1:24 FP32.\r\nFor the Titan Black, the magic happens in the driver. The Titan black\'s driver gives the user an option to choose the double precision performance between 1:3 and 1:24 FP32. When the double precision performance is set to 1:24 FP32, which is the same as the 780 Ti, the the single precision performance between the Titan Black and 780 Ti are identical. But when the user sets the double precision performance to 1:3 FP32, the single precision performance is compromised to boost double precision performance and make it equal to the K40c.\r\nThe K40c has a double precision performance of 1:3 without compromising the single precision performance. This is because the K40 is given a special double precision unit for every 3 single precision cores. It combines the best of both worlds.\r\n\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=381229711&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=722963661&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1083990145&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1438814030&amp;format=interactive\"></iframe>','Explaining FP64 performance on GPUs','','inherit','open','closed','','8389-revision-v1','','','2015-06-22 14:13:00','2015-06-22 14:13:00','',8389,'http://sandbox.arrayfire.com/8389-revision-v1/',0,'revision','',0),(8395,11,'2015-06-22 14:18:57','2015-06-22 14:18:57','GPUs are really good at doing math. The Achilles heel is when it comes to 64-bit double precision math. GPUs, at least consumer grade, are not built for high performance FP64. This is because they are targeted towards gamers and game developers, who do not really care about high precision compute. So vendors like NVIDIA and AMD do not cram FP64 compute cores in their GPUs.\r\nFor example, on a GTX 780, the FP64 performance is 1/24 FP32. Which means in an ideal case, running the same code by only changing float types to double types, would yield the double precision run time to be about 1/24th of the single precision time.\r\n\r\nHow double precision performs really depends on the architecture of the GPU. Lets take 3 almost identical cards: GTX 780 Ti, GTX Titan Black and the Tesla K40c. All are <a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">GK110</a> based GPUs, with the same number of SMX and cores (15 SMX, 2880 cores) and the same bus width (384-bit). The 780 Ti and Titan Black even have nearly same base clock speeds (~880MHz; K40c is 745MHz) and identical memory clock speeds (7GHz; K40 is 6GHz).\r\nWith respect to single precision performance, all 3 are fairly in the same ball park. Which the 780 Ti and Titan Black sit just around 5.1 TFlops, owing to their similar clock speeds, the Tesla K40c drops in at 4.3 TFlops. You could give the Tesla a pass because it has a lower clock speed.\r\n\r\nAll 3 only vary significantly in 3 categories.\r\n\r\n1. Memory\r\nThis doesn\'t affect performance very much (at least when the sizes fit in all GPUs). The price difference based on market prices of other GPUs with similar memory size variations should not be that big either.\r\n\r\n2. Price\r\nAt launch, the GTX 780 Ti was priced at $699, $999 for the Titan Black and an estimated $5500 for the K40c. Well, that escalated quickly.\r\n(Price Sources: <a href=\"http://www.anandtech.com/show/7492/the-geforce-gtx-780-ti-review\" target=\"_blank\">GTX 780 Ti</a> <a href=\"http://www.anandtech.com/show/7765/nvidias-geforce-gtx-titan-black-no-compromises-for-gaming-compute\" target=\"_blank\">GTX Titan Black</a> <a href=\"http://www.anandtech.com/show/7521/nvidia-launches-tesla-k40\" target=\"_blank\">K40</a>)\r\n\r\n3. Double Precision Performance\r\nSo if the 780 Ti and the Titan Black are practically the same in every respect, why is there a $300 difference in their price at launch?\r\nThe 780 Ti is physically locked at 1:24 FP32.\r\n\r\nFor the Titan Black, the magic happens in the driver. The Titan black\'s driver gives the user an option to choose the double precision performance between 1:3 and 1:24 FP32. When the double precision performance is set to 1:24 FP32, which is the same as the 780 Ti, the the single precision performance between the Titan Black and 780 Ti are identical. But when the user sets the double precision performance to 1:3 FP32, the single precision performance is compromised to boost double precision performance and make it equal to the K40c.\r\n\r\nThe K40c has a double precision performance of 1:3 without compromising the single precision performance. This is because the K40 is given a special double precision unit for every 3 single precision cores. It combines the best of both worlds. NVIDIA also states that the Tesla GPUs go through a much more rigorous Q&A process which guarantees lesser failures and also has additional features such as ECC memory. Hence the large price difference.\r\n\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=381229711&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=722963661&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1083990145&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1438814030&amp;format=interactive\"></iframe>','Explaining FP64 performance on GPUs','','inherit','open','closed','','8389-revision-v1','','','2015-06-22 14:18:57','2015-06-22 14:18:57','',8389,'http://sandbox.arrayfire.com/8389-revision-v1/',0,'revision','',0),(8396,11,'2015-06-22 14:19:54','2015-06-22 14:19:54','GPUs are really good at doing math. The Achilles heel is when it comes to 64-bit double precision math. GPUs, at least consumer grade, are not built for high performance FP64. This is because they are targeted towards gamers and game developers, who do not really care about high precision compute. So vendors like NVIDIA and AMD do not cram FP64 compute cores in their GPUs.\r\n\r\nFor example, on a GTX 780 Ti, the FP64 performance is 1/24 FP32. Which means in an ideal case, running the same code by only changing float types to double types, would yield the double precision run time to be about 1/24th of the single precision time.\r\n\r\nHow double precision performs really depends on the architecture of the GPU. Lets take 3 almost identical cards: GTX 780 Ti, GTX Titan Black and the Tesla K40c. All are <a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">GK110</a> based GPUs, with the same number of SMX and cores (15 SMX, 2880 cores) and the same bus width (384-bit). The 780 Ti and Titan Black even have nearly same base clock speeds (~880MHz; K40c is 745MHz) and identical memory clock speeds (7GHz; K40 is 6GHz).\r\nWith respect to single precision performance, all 3 are fairly in the same ball park. Which the 780 Ti and Titan Black sit just around 5.1 TFlops, owing to their similar clock speeds, the Tesla K40c drops in at 4.3 TFlops. You could give the Tesla a pass because it has a lower clock speed.\r\n\r\nAll 3 only vary significantly in 3 categories.\r\n\r\n<strong>1. Memory</strong>\r\nThis doesn\'t affect performance very much (at least when the sizes fit in all GPUs). The price difference based on market prices of other GPUs with similar memory size variations should not be that big either.\r\n\r\n<strong>2. Price</strong>\r\nAt launch, the GTX 780 Ti was priced at $699, $999 for the Titan Black and an estimated $5500 for the K40c. Well, that escalated quickly.\r\n(Price Sources: <a href=\"http://www.anandtech.com/show/7492/the-geforce-gtx-780-ti-review\" target=\"_blank\">GTX 780 Ti</a> <a href=\"http://www.anandtech.com/show/7765/nvidias-geforce-gtx-titan-black-no-compromises-for-gaming-compute\" target=\"_blank\">GTX Titan Black</a> <a href=\"http://www.anandtech.com/show/7521/nvidia-launches-tesla-k40\" target=\"_blank\">K40</a>)\r\n\r\n<strong>3. Double Precision Performance</strong>\r\nSo if the 780 Ti and the Titan Black are practically the same in every respect, why is there a $300 difference in their price at launch?\r\nThe 780 Ti is physically locked at 1:24 FP32.\r\n\r\nFor the Titan Black, the magic happens in the driver. The Titan black\'s driver gives the user an option to choose the double precision performance between 1:3 and 1:24 FP32. When the double precision performance is set to 1:24 FP32, which is the same as the 780 Ti, the the single precision performance between the Titan Black and 780 Ti are identical. But when the user sets the double precision performance to 1:3 FP32, the single precision performance is compromised to boost double precision performance and make it equal to the K40c.\r\n\r\nThe K40c has a double precision performance of 1:3 without compromising the single precision performance. This is because the K40 is given a special double precision unit for every 3 single precision cores. It combines the best of both worlds. NVIDIA also states that the Tesla GPUs go through a much more rigorous Q&A process which guarantees lesser failures and also has additional features such as ECC memory. Hence the large price difference.\r\n\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=381229711&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=722963661&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1083990145&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1438814030&amp;format=interactive\"></iframe>','Explaining FP64 performance on GPUs','','inherit','open','closed','','8389-revision-v1','','','2015-06-22 14:19:54','2015-06-22 14:19:54','',8389,'http://sandbox.arrayfire.com/8389-revision-v1/',0,'revision','',0),(8397,11,'2015-06-22 14:20:15','2015-06-22 14:20:15','GPUs are really good at doing math. The Achilles heel is when it comes to 64-bit double precision math. GPUs, at least consumer grade, are not built for high performance FP64. This is because they are targeted towards gamers and game developers, who do not really care about high precision compute. So vendors like NVIDIA and AMD do not cram FP64 compute cores in their GPUs.\r\n\r\nFor example, on a GTX 780 Ti, the FP64 performance is 1/24 FP32. Which means in an ideal case, running the same code by only changing float types to double types, would yield the double precision run time to be about 1/24th of the single precision time.\r\n\r\nHow double precision performs really depends on the architecture of the GPU. Lets take 3 almost identical cards: GTX 780 Ti, GTX Titan Black and the Tesla K40c. All are <a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">GK110</a> based GPUs, with the same number of SMX and cores (15 SMX, 2880 cores) and the same bus width (384-bit). The 780 Ti and Titan Black even have nearly same base clock speeds (~880MHz; K40c is 745MHz) and identical memory clock speeds (7GHz; K40 is 6GHz).\r\nWith respect to single precision performance, all 3 are fairly in the same ball park. Which the 780 Ti and Titan Black sit just around 5.1 TFlops, owing to their similar clock speeds, the Tesla K40c drops in at 4.3 TFlops. You could give the Tesla a pass because it has a lower clock speed.\r\n\r\nAll 3 only vary significantly in 3 categories.\r\n\r\n<strong>1. Memory</strong>\r\nThis doesn\'t affect performance very much (at least when the sizes fit in all GPUs). The price difference based on market prices of other GPUs with similar memory size variations should not be that big either.\r\n\r\n<strong>2. Price</strong>\r\nAt launch, the GTX 780 Ti was priced at $699, $999 for the Titan Black and an estimated $5500 for the K40c. Well, that escalated quickly.\r\n(Price Sources: <a href=\"http://www.anandtech.com/show/7492/the-geforce-gtx-780-ti-review\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.anandtech.com/show/7765/nvidias-geforce-gtx-titan-black-no-compromises-for-gaming-compute\" target=\"_blank\">GTX Titan Black</a>, <a href=\"http://www.anandtech.com/show/7521/nvidia-launches-tesla-k40\" target=\"_blank\">K40</a>)\r\n\r\n<strong>3. Double Precision Performance</strong>\r\nSo if the 780 Ti and the Titan Black are practically the same in every respect, why is there a $300 difference in their price at launch?\r\nThe 780 Ti is physically locked at 1:24 FP32.\r\n\r\nFor the Titan Black, the magic happens in the driver. The Titan black\'s driver gives the user an option to choose the double precision performance between 1:3 and 1:24 FP32. When the double precision performance is set to 1:24 FP32, which is the same as the 780 Ti, the the single precision performance between the Titan Black and 780 Ti are identical. But when the user sets the double precision performance to 1:3 FP32, the single precision performance is compromised to boost double precision performance and make it equal to the K40c.\r\n\r\nThe K40c has a double precision performance of 1:3 without compromising the single precision performance. This is because the K40 is given a special double precision unit for every 3 single precision cores. It combines the best of both worlds. NVIDIA also states that the Tesla GPUs go through a much more rigorous Q&A process which guarantees lesser failures and also has additional features such as ECC memory. Hence the large price difference.\r\n\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=381229711&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=722963661&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1083990145&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1438814030&amp;format=interactive\"></iframe>','Explaining FP64 performance on GPUs','','inherit','open','closed','','8389-revision-v1','','','2015-06-22 14:20:15','2015-06-22 14:20:15','',8389,'http://sandbox.arrayfire.com/8389-revision-v1/',0,'revision','',0),(8398,11,'2015-06-22 14:22:06','2015-06-22 14:22:06','GPUs are really good at doing math. The Achilles heel is when it comes to 64-bit double precision math. GPUs, at least consumer grade, are not built for high performance FP64. This is because they are targeted towards gamers and game developers, who do not really care about high precision compute. So vendors like NVIDIA and AMD do not cram FP64 compute cores in their GPUs.\r\n\r\nFor example, on a GTX 780 Ti, the FP64 performance is 1/24 FP32. Which means in an ideal case, running the same code by only changing float types to double types, would yield the double precision run time to be about 1/24th of the single precision time.\r\n\r\nHow double precision performs really depends on the architecture of the GPU. Lets take 3 almost identical cards: <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-780-ti\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-titan-black\" target=\"_blank\">GTX Titan Black</a> and the <a href=\"http://www.nvidia.com/object/tesla-servers.html\" target=\"_blank\">Tesla K40c</a>. All are <a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">GK110</a> based GPUs, with the same number of SMX and cores (15 SMX, 2880 cores) and the same bus width (384-bit). The 780 Ti and Titan Black even have nearly same base clock speeds (~880MHz; K40c is 745MHz) and identical memory clock speeds (7GHz; K40 is 6GHz).\r\nWith respect to single precision performance, all 3 are fairly in the same ball park. Which the 780 Ti and Titan Black sit just around 5.1 TFlops, owing to their similar clock speeds, the Tesla K40c drops in at 4.3 TFlops. You could give the Tesla a pass because it has a lower clock speed.\r\n\r\nAll 3 only vary significantly in 3 categories.\r\n\r\n<strong>1. Memory</strong>\r\nThis doesn\'t affect performance very much (at least when the sizes fit in all GPUs). The price difference based on market prices of other GPUs with similar memory size variations should not be that big either.\r\n\r\n<strong>2. Price</strong>\r\nAt launch, the GTX 780 Ti was priced at $699, $999 for the Titan Black and an estimated $5500 for the K40c. Well, that escalated quickly.\r\n(Price Sources: <a href=\"http://www.anandtech.com/show/7492/the-geforce-gtx-780-ti-review\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.anandtech.com/show/7765/nvidias-geforce-gtx-titan-black-no-compromises-for-gaming-compute\" target=\"_blank\">GTX Titan Black</a>, <a href=\"http://www.anandtech.com/show/7521/nvidia-launches-tesla-k40\" target=\"_blank\">K40</a>)\r\n\r\n<strong>3. Double Precision Performance</strong>\r\nSo if the 780 Ti and the Titan Black are practically the same in every respect, why is there a $300 difference in their price at launch?\r\nThe 780 Ti is physically locked at 1:24 FP32.\r\n\r\nFor the Titan Black, the magic happens in the driver. The Titan black\'s driver gives the user an option to choose the double precision performance between 1:3 and 1:24 FP32. When the double precision performance is set to 1:24 FP32, which is the same as the 780 Ti, the the single precision performance between the Titan Black and 780 Ti are identical. But when the user sets the double precision performance to 1:3 FP32, the single precision performance is compromised to boost double precision performance and make it equal to the K40c.\r\n\r\nThe K40c has a double precision performance of 1:3 without compromising the single precision performance. This is because the K40 is given a special double precision unit for every 3 single precision cores. It combines the best of both worlds. NVIDIA also states that the Tesla GPUs go through a much more rigorous Q&A process which guarantees lesser failures and also has additional features such as ECC memory. Hence the large price difference.\r\n\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=381229711&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=722963661&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1083990145&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1438814030&amp;format=interactive\"></iframe>','Explaining FP64 performance on GPUs','','inherit','open','closed','','8389-revision-v1','','','2015-06-22 14:22:06','2015-06-22 14:22:06','',8389,'http://sandbox.arrayfire.com/8389-revision-v1/',0,'revision','',0),(8399,11,'2015-06-22 14:34:35','2015-06-22 14:34:35','GPUs are really good at doing math. The Achilles heel is when it comes to 64-bit double precision math. GPUs, at least consumer grade, are not built for high performance FP64. This is because they are targeted towards gamers and game developers, who do not really care about high precision compute. So vendors like NVIDIA and AMD do not cram FP64 compute cores in their GPUs.\r\n\r\nFor example, on a GTX 780 Ti, the FP64 performance is 1/24 FP32. Which means in an ideal case, running the same code by only changing float types to double types, would yield the double precision run time to be about 1/24th of the single precision time.\r\n\r\nKeep in mind, the theoretical best case for FP64 performance is 1:2 FP32, simply because it involves computing with double the number of bits as FP32.\r\n\r\nHow double precision performs really depends on the architecture of the GPU.\r\n\r\nLets take 3 almost identical cards: <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-780-ti\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-titan-black\" target=\"_blank\">GTX Titan Black</a> and the <a href=\"http://www.nvidia.com/object/tesla-servers.html\" target=\"_blank\">Tesla K40c</a>. All are <a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">GK110</a> based GPUs, with the same number of SMX and cores (15 SMX, 2880 cores) and the same bus width (384-bit). The 780 Ti and Titan Black even have nearly same base clock speeds (~880MHz; K40c is 745MHz) and identical memory clock speeds (7GHz; K40 is 6GHz).\r\n\r\n<strong>Single Precision Performance</strong>\r\nWith respect to single precision performance, all 3 are fairly in the same ball park. Which the 780 Ti and Titan Black sit just around 5.1 TFlops, owing to their similar clock speeds, the Tesla K40c drops in at 4.3 TFlops. You could give the Tesla a pass because it has a lower clock speed.\r\n\r\nAll 3 only vary significantly in 3 categories.\r\n\r\n<strong>1. Memory</strong>\r\nThis doesn\'t affect performance very much (at least when the sizes fit in all GPUs). The price difference based on market prices of other GPUs with similar memory size variations should not be that big either.\r\n\r\n<strong>2. Price</strong>\r\nAt launch, the GTX 780 Ti was priced at $699, $999 for the Titan Black and an estimated $5500 for the K40c. Well, that escalated quickly.\r\n(Price Sources: <a href=\"http://www.anandtech.com/show/7492/the-geforce-gtx-780-ti-review\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.anandtech.com/show/7765/nvidias-geforce-gtx-titan-black-no-compromises-for-gaming-compute\" target=\"_blank\">GTX Titan Black</a>, <a href=\"http://www.anandtech.com/show/7521/nvidia-launches-tesla-k40\" target=\"_blank\">K40</a>)\r\n\r\n<strong>3. Double Precision Performance</strong>\r\nSo if the 780 Ti and the Titan Black are practically the same in every respect, why is there a $300 difference in their price at launch (discounting memory size difference)?\r\nThe 780 Ti is physically locked at 1:24 FP32.\r\n\r\nFor the Titan Black, the magic happens in the driver. The Titan black\'s driver gives the user an option to choose the double precision performance between 1:3 and 1:24 FP32. When the double precision performance is set to 1:24 FP32, which is the same as the 780 Ti, the the single precision performance between the Titan Black and 780 Ti are identical. But when the user sets the double precision performance to 1:3 FP32, the single precision performance is compromised to boost double precision performance and make it equal to the K40c.\r\n\r\nThe K40c has a double precision performance of 1:3 without compromising the single precision performance. This is because the K40 is given a special double precision unit for every 3 single precision cores. It combines the best of both worlds. NVIDIA also states that the Tesla GPUs go through a much more rigorous Q&A process which guarantees lesser failures and also has additional features such as ECC memory. Hence the large price difference.\r\n\r\nHere are some benchmarks comparaing FP32 and FP64 GEMM.\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=381229711&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=722963661&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1083990145&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1438814030&amp;format=interactive\"></iframe>\r\n\r\nOne thing you would notice from the above charts is that the AMD GPUs performance fairly well and have a theoretical 1:3 FP32 performance for FP64. Most AMD cards will give between 1:3 and 1:8 FP32 performance for FP64. The architecture of AMD GPUs is such that do not suffer from the same problems as NVIDIA\'s GTX series. So if you want a cost effective FP64 card and don\'t need CUDA, you know what to look for.','Explaining FP64 performance on GPUs','','inherit','open','closed','','8389-revision-v1','','','2015-06-22 14:34:35','2015-06-22 14:34:35','',8389,'http://sandbox.arrayfire.com/8389-revision-v1/',0,'revision','',0),(8400,11,'2015-06-22 14:37:20','2015-06-22 14:37:20','GPUs are really good at doing math. The Achilles heel is when it comes to 64-bit double precision math. GPUs, at least consumer grade, are not built for high performance FP64. This is because they are targeted towards gamers and game developers, who do not really care about high precision compute. So vendors like NVIDIA and AMD do not cram FP64 compute cores in their GPUs.\r\n\r\nFor example, on a GTX 780 Ti, the FP64 performance is 1/24 FP32. Which means in an ideal case, running the same code by only changing float types to double types, would yield the double precision run time to be about 1/24th of the single precision time.\r\n\r\nKeep in mind, the theoretical best case for FP64 performance is 1:2 FP32, simply because it involves computing with double the number of bits as FP32.\r\n\r\nHow double precision performs really depends on the architecture of the GPU.\r\n\r\nLets take 3 almost identical cards: <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-780-ti\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-titan-black\" target=\"_blank\">GTX Titan Black</a> and the <a href=\"http://www.nvidia.com/object/tesla-servers.html\" target=\"_blank\">Tesla K40c</a>. All are Kepler <a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">GK110</a> based GPUs, with the same number of SMX and cores (15 SMX, 2880 cores) and the same bus width (384-bit). The 780 Ti and Titan Black even have nearly same base clock speeds (~880MHz; K40c is 745MHz) and identical memory clock speeds (7GHz; K40 is 6GHz).\r\n\r\n<strong>Single Precision Performance</strong>\r\nWith respect to single precision performance, all 3 are fairly in the same ball park. Which the 780 Ti and Titan Black sit just around 5.1 TFlops, owing to their similar clock speeds, the Tesla K40c drops in at 4.3 TFlops. You could give the Tesla a pass because it has a lower clock speed.\r\n\r\nAll 3 only vary significantly in 3 categories.\r\n\r\n<strong>1. Memory</strong>\r\nThis doesn\'t affect performance very much (at least when the sizes fit in all GPUs). The price difference based on market prices of other GPUs with similar memory size variations should not be that big either.\r\n\r\n<strong>2. Price</strong>\r\nAt launch, the GTX 780 Ti was priced at $699, $999 for the Titan Black and an estimated $5500 for the K40c. Well, that escalated quickly.\r\n(Price Sources: <a href=\"http://www.anandtech.com/show/7492/the-geforce-gtx-780-ti-review\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.anandtech.com/show/7765/nvidias-geforce-gtx-titan-black-no-compromises-for-gaming-compute\" target=\"_blank\">GTX Titan Black</a>, <a href=\"http://www.anandtech.com/show/7521/nvidia-launches-tesla-k40\" target=\"_blank\">K40</a>)\r\n\r\n<strong>3. Double Precision Performance</strong>\r\nSo if the 780 Ti and the Titan Black are practically the same in every respect, why is there a $300 difference in their price at launch (discounting memory size difference)?\r\nThe 780 Ti is physically locked at 1:24 FP32.\r\n\r\nFor the Titan Black, the magic happens in the driver. The Titan black\'s driver gives the user an option to choose the double precision performance between 1:3 and 1:24 FP32. When the double precision performance is set to 1:24 FP32, which is the same as the 780 Ti, the the single precision performance between the Titan Black and 780 Ti are identical. But when the user sets the double precision performance to 1:3 FP32, the single precision performance is compromised to boost double precision performance and make it equal to the K40c.\r\n\r\nThe K40c has a double precision performance of 1:3 without compromising the single precision performance. This is because the K40 is given a special double precision unit for every 3 single precision cores (<a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">white paper</a>). It combines the best of both worlds. NVIDIA also states that the Tesla GPUs go through a much more rigorous Q&A process which guarantees lesser failures and also has additional features such as ECC memory. Hence the large price difference.\r\n\r\nHere are some benchmarks comparaing FP32 and FP64 GEMM.\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=381229711&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=722963661&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1083990145&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1438814030&amp;format=interactive\"></iframe>\r\n\r\nOne thing you would notice from the above charts is that the AMD GPUs performance fairly well and have a theoretical 1:3 FP32 performance for FP64. Most AMD cards will give between 1:3 and 1:8 FP32 performance for FP64. The FirePro W9100 will give you an incredible FP64 1:2 FP32 performance. The architecture of AMD GPUs is such that do not suffer from the same problems as NVIDIA\'s GTX series. So if you want a cost effective FP64 card and don\'t need CUDA, you know what to look for.','Explaining FP64 performance on GPUs','','inherit','open','closed','','8389-revision-v1','','','2015-06-22 14:37:20','2015-06-22 14:37:20','',8389,'http://sandbox.arrayfire.com/8389-revision-v1/',0,'revision','',0),(8401,11,'2015-06-22 14:39:10','2015-06-22 14:39:10','GPUs are really good at doing math. The Achilles heel is when it comes to 64-bit double precision math. GPUs, at least consumer grade, are not built for high performance FP64. This is because they are targeted towards gamers and game developers, who do not really care about high precision compute. So vendors like NVIDIA and AMD do not cram FP64 compute cores in their GPUs.\r\n\r\nFor example, on a GTX 780 Ti, the FP64 performance is 1/24 FP32. Which means in an ideal case, running the same code by only changing float types to double types, would yield the double precision run time to be about 1/24th of the single precision time.\r\n\r\nKeep in mind, the theoretical best case for FP64 performance is 1:2 FP32, simply because it involves computing with double the number of bits as FP32.\r\n\r\nHow double precision performs really depends on the architecture of the GPU.\r\n\r\nLets take 3 almost identical cards: <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-780-ti\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-titan-black\" target=\"_blank\">GTX Titan Black</a> and the <a href=\"http://www.nvidia.com/object/tesla-servers.html\" target=\"_blank\">Tesla K40c</a>. All are Kepler <a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">GK110</a> based GPUs, with the same number of SMX and cores (15 SMX, 2880 cores) and the same bus width (384-bit). The 780 Ti and Titan Black even have nearly same base clock speeds (~880MHz; K40c is 745MHz) and identical memory clock speeds (7GHz; K40 is 6GHz).\r\n\r\n<strong>Single Precision Performance</strong>\r\nWith respect to single precision performance, all 3 are fairly in the same ball park. Which the 780 Ti and Titan Black sit just around 5.1 TFlops, owing to their similar clock speeds, the Tesla K40c drops in at 4.3 TFlops. You could give the Tesla a pass because it has a lower clock speed.\r\n\r\nAll 3 only vary significantly in 3 categories.\r\n\r\n<strong>1. Memory</strong>\r\nThis doesn\'t affect performance very much (at least when the sizes fit in all GPUs). The price difference based on market prices of other GPUs with similar memory size variations should not be that big either.\r\n\r\n<strong>2. Price</strong>\r\nAt launch, the GTX 780 Ti was priced at $699, $999 for the Titan Black and an estimated $5500 for the K40c. Well, that escalated quickly.\r\n(Price Sources: <a href=\"http://www.anandtech.com/show/7492/the-geforce-gtx-780-ti-review\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.anandtech.com/show/7765/nvidias-geforce-gtx-titan-black-no-compromises-for-gaming-compute\" target=\"_blank\">GTX Titan Black</a>, <a href=\"http://www.anandtech.com/show/7521/nvidia-launches-tesla-k40\" target=\"_blank\">K40</a>)\r\n\r\n<strong>3. Double Precision Performance</strong>\r\nSo if the 780 Ti and the Titan Black are practically the same in every respect, why is there a $300 difference in their price at launch (discounting memory size difference)?\r\nThe answer is in the double precision capabilities. The 780 Ti is physically locked at 1:24 FP32 where has the Titan Black has an ace up it\'s sleeve.\r\n\r\nFor the Titan Black, the magic happens in the driver. The Titan black\'s driver gives the user an option to choose the double precision performance between 1:3 and 1:24 FP32. When the double precision performance is set to 1:24 FP32, which is the same as the 780 Ti, the the single precision performance between the Titan Black and 780 Ti are identical. But when the user sets the double precision performance to 1:3 FP32, the single precision performance is compromised to boost double precision performance and make it equal to the K40c.\r\n\r\nThe K40c has a double precision performance of 1:3 without compromising the single precision performance. This is because the K40 is given a special double precision unit for every 3 single precision cores (<a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">white paper</a>). It combines the best of both worlds. NVIDIA also states that the Tesla GPUs go through a much more rigorous Q&A process which guarantees lesser failures and also has additional features such as ECC memory. Hence the large price difference.\r\n\r\nHere are some benchmarks comparaing FP32 and FP64 GEMM.\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=381229711&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=722963661&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1083990145&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1438814030&amp;format=interactive\"></iframe>\r\n\r\nOne thing you would notice from the above charts is that the AMD GPUs performance fairly well and have a theoretical 1:3 FP32 performance for FP64. Most AMD cards will give between 1:3 and 1:8 FP32 performance for FP64. The FirePro W9100 will give you an incredible FP64 1:2 FP32 performance. The architecture of AMD GPUs is such that do not suffer from the same problems as NVIDIA\'s GTX series. So if you want a cost effective FP64 card and don\'t need CUDA, you know what to look for.','Explaining FP64 performance on GPUs','','inherit','open','closed','','8389-revision-v1','','','2015-06-22 14:39:10','2015-06-22 14:39:10','',8389,'http://sandbox.arrayfire.com/8389-revision-v1/',0,'revision','',0),(8402,11,'2015-06-22 14:42:58','2015-06-22 14:42:58','GPUs are really good at doing math. The Achilles heel is when it comes to 64-bit double precision math. GPUs, at least consumer grade, are not built for high performance FP64. This is because they are targeted towards gamers and game developers, who do not really care about high precision compute. So vendors like NVIDIA and AMD do not cram FP64 compute cores in their GPUs.\r\n\r\nFor example, on a GTX 780 Ti, the FP64 performance is 1/24 FP32. Which means in an ideal case, running the same code by only changing float types to double types, would yield the double precision run time to be about 1/24th of the single precision time.\r\n\r\nKeep in mind, the theoretical best case for FP64 performance is 1:2 FP32, simply because it involves computing with double the number of bits as FP32.\r\n\r\nHow double precision performs really depends on the architecture of the GPU.\r\n\r\nLets take 3 almost identical cards: <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-780-ti\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-titan-black\" target=\"_blank\">GTX Titan Black</a> and the <a href=\"http://www.nvidia.com/object/tesla-servers.html\" target=\"_blank\">Tesla K40c</a>. All are Kepler <a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">GK110</a> based GPUs, with the same number of SMX and cores (15 SMX, 2880 cores) and the same bus width (384-bit). The 780 Ti and Titan Black even have nearly same base clock speeds (~880MHz; K40c is 745MHz) and identical memory clock speeds (7GHz; K40 is 6GHz).\r\n\r\n<strong>Single Precision Performance</strong>\r\nWith respect to single precision performance, all 3 are fairly in the same ball park. Which the 780 Ti and Titan Black sit just around 5.1 TFlops, owing to their similar clock speeds, the Tesla K40c drops in at 4.3 TFlops. You could give the Tesla a pass because it has a lower clock speed.\r\n\r\nAll 3 only vary significantly in 3 categories.\r\n\r\n<strong>1. Memory</strong>\r\nThis doesn\'t affect performance very much (at least when the sizes fit in all GPUs). The price difference based on market prices of other GPUs with similar memory size variations should not be that big either.\r\n\r\n<strong>2. Price</strong>\r\nAt launch, the GTX 780 Ti was priced at $699, $999 for the Titan Black and an estimated $5500 for the K40c. Well, that escalated quickly.\r\n(Price Sources: <a href=\"http://www.anandtech.com/show/7492/the-geforce-gtx-780-ti-review\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.anandtech.com/show/7765/nvidias-geforce-gtx-titan-black-no-compromises-for-gaming-compute\" target=\"_blank\">GTX Titan Black</a>, <a href=\"http://www.anandtech.com/show/7521/nvidia-launches-tesla-k40\" target=\"_blank\">K40</a>)\r\n\r\n<strong>3. Double Precision Performance</strong>\r\nSo if the 780 Ti and the Titan Black are practically the same in every respect, why is there a $300 difference in their price at launch (discounting memory size difference)?\r\n\r\nThe answer is in the double precision capabilities. The 780 Ti is physically locked at 1:24 FP32 where has the Titan Black has an ace up it\'s sleeve.\r\n\r\nFor the Titan Black, the magic happens in the driver. The Titan black\'s driver gives the user an option to choose the double precision performance between 1:3 and 1:24 FP32. When the double precision performance is set to 1:24 FP32, which is the same as the 780 Ti, the the single precision performance of the Titan Black and 780 Ti are identical. But when the user sets the double precision performance to 1:3 FP32, the single precision performance is compromised to boost double precision performance and make it equal to the K40c. In other words, you can choose the performance of the Titan Black to match either the 780 Ti or the K40c based and your preference.\r\n\r\nThe K40c has a double precision performance of 1:3 without compromising the single precision performance. This is because the K40 is given a special double precision unit for every 3 single precision cores (<a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">white paper</a>). It combines the best of both worlds. NVIDIA also states that the Tesla GPUs go through a much more rigorous Q&A process which guarantees lesser failures and also has additional features such as ECC memory. Hence the large price difference.\r\n\r\nHere are some benchmarks comparaing FP32 and FP64 GEMM.\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=381229711&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=722963661&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1083990145&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1438814030&amp;format=interactive\"></iframe>\r\n\r\nOne thing you would notice from the above charts is that the AMD GPUs performance fairly well and have a theoretical 1:3 FP32 performance for FP64. Most AMD cards will give between 1:3 and 1:8 FP32 performance for FP64. The FirePro W9100 will give you an incredible FP64 1:2 FP32 performance. The architecture of AMD GPUs is such that do not suffer from the same problems FP64 problems as NVIDIA\'s GTX series.\r\n\r\nDisclaimer: We receive GPUs from both NVIDIA and AMD. However, this blog does not promote or advocate any specific vendor.','Explaining FP64 performance on GPUs','','inherit','open','closed','','8389-revision-v1','','','2015-06-22 14:42:58','2015-06-22 14:42:58','',8389,'http://sandbox.arrayfire.com/8389-revision-v1/',0,'revision','',0);
INSERT INTO `wp_posts` VALUES (8403,11,'2015-06-22 14:43:39','2015-06-22 14:43:39','GPUs are really good at doing math. The Achilles heel is when it comes to 64-bit double precision math. GPUs, at least consumer grade, are not built for high performance FP64. This is because they are targeted towards gamers and game developers, who do not really care about high precision compute. So vendors like NVIDIA and AMD do not cram FP64 compute cores in their GPUs.\r\n\r\nFor example, on a GTX 780 Ti, the FP64 performance is 1/24 FP32. Which means in an ideal case, running the same code by only changing float types to double types, would yield the double precision run time to be about 1/24th of the single precision time.\r\n\r\nKeep in mind, the theoretical best case for FP64 performance is 1:2 FP32, simply because it involves computing with double the number of bits as FP32.\r\n\r\nHow double precision performs really depends on the architecture of the GPU.\r\n\r\nLets take 3 almost identical cards: <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-780-ti\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-titan-black\" target=\"_blank\">GTX Titan Black</a> and the <a href=\"http://www.nvidia.com/object/tesla-servers.html\" target=\"_blank\">Tesla K40c</a>. All are Kepler <a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">GK110</a> based GPUs, with the same number of SMX and cores (15 SMX, 2880 cores) and the same bus width (384-bit). The 780 Ti and Titan Black even have nearly same base clock speeds (~880MHz; K40c is 745MHz) and identical memory clock speeds (7GHz; K40 is 6GHz).\r\n\r\n<strong>Single Precision Performance</strong>\r\nWith respect to single precision performance, all 3 are fairly in the same ball park. Which the 780 Ti and Titan Black sit just around 5.1 TFlops, owing to their similar clock speeds, the Tesla K40c drops in at 4.3 TFlops. You could give the Tesla a pass because it has a lower clock speed.\r\n\r\nAll 3 only vary significantly in 3 categories.\r\n\r\n<strong>1. Memory</strong>\r\nThis doesn\'t affect performance very much (at least when the sizes fit in all GPUs). The price difference based on market prices of other GPUs with similar memory size variations should not be that big either.\r\n\r\n<strong>2. Price</strong>\r\nAt launch, the GTX 780 Ti was priced at $699, $999 for the Titan Black and an estimated $5500 for the K40c. Well, that escalated quickly.\r\n(Price Sources: <a href=\"http://www.anandtech.com/show/7492/the-geforce-gtx-780-ti-review\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.anandtech.com/show/7765/nvidias-geforce-gtx-titan-black-no-compromises-for-gaming-compute\" target=\"_blank\">GTX Titan Black</a>, <a href=\"http://www.anandtech.com/show/7521/nvidia-launches-tesla-k40\" target=\"_blank\">K40</a>)\r\n\r\n<strong>3. Double Precision Performance</strong>\r\nSo if the 780 Ti and the Titan Black are practically the same in every respect, why is there a $300 difference in their price at launch (discounting memory size difference)?\r\n\r\nThe answer is in the double precision capabilities. The 780 Ti is physically locked at 1:24 FP32 where has the Titan Black has an ace up it\'s sleeve.\r\n\r\nFor the Titan Black, the magic happens in the driver. The Titan black\'s driver gives the user an option to choose the double precision performance between 1:3 and 1:24 FP32. When the double precision performance is set to 1:24 FP32, which is the same as the 780 Ti, the the single precision performance of the Titan Black and 780 Ti are identical. But when the user sets the double precision performance to 1:3 FP32, the single precision performance is compromised to boost double precision performance and make it equal to the K40c. In other words, you can choose the performance of the Titan Black to match either the 780 Ti or the K40c based and your preference.\r\n\r\nThe K40c has a double precision performance of 1:3 without compromising the single precision performance. This is because the K40 is given a special double precision unit for every 3 single precision cores (<a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">white paper</a>). It combines the best of both worlds. NVIDIA also states that the Tesla GPUs go through a much more rigorous Q&A process which guarantees lesser failures and also has additional features such as ECC memory. Hence the large price difference.\r\n\r\n<strong>Benchmarks</strong>\r\nHere are some benchmarks comparaing FP32 and FP64 GEMM.\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=381229711&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=722963661&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1083990145&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1438814030&amp;format=interactive\"></iframe>\r\n\r\nOne thing you would notice from the above charts is that the AMD GPUs performance fairly well and have a theoretical 1:3 FP32 performance for FP64. Most AMD cards will give between 1:3 and 1:8 FP32 performance for FP64. The FirePro W9100 will give you an incredible FP64 1:2 FP32 performance. The architecture of AMD GPUs is such that do not suffer from the same problems FP64 problems as NVIDIA\'s GTX series.\r\n\r\nDisclaimer: We receive GPUs from both NVIDIA and AMD. However, this blog does not promote or advocate any specific vendor.','Explaining FP64 performance on GPUs','','inherit','open','closed','','8389-revision-v1','','','2015-06-22 14:43:39','2015-06-22 14:43:39','',8389,'http://sandbox.arrayfire.com/8389-revision-v1/',0,'revision','',0),(8404,11,'2015-06-22 14:50:49','2015-06-22 14:50:49','GPUs are really good at doing math. The Achilles heel is when it comes to 64-bit double precision math. GPUs, at least consumer grade, are not built for high performance FP64. This is because they are targeted towards gamers and game developers, who do not really care about high precision compute. So vendors like NVIDIA and AMD do not cram FP64 compute cores in their GPUs.\r\n\r\nFor example, on a GTX 780 Ti, the FP64 performance is 1/24 FP32. Which means in an ideal case, running the same code by only changing float types to double types, would yield the double precision run time to be about 1/24th of the single precision time.\r\n\r\nKeep in mind, the theoretical best case for FP64 performance is 1:2 FP32, simply because it involves computing with double the number of bits as FP32.\r\n\r\nHow double precision performs really depends on the architecture of the GPU.\r\n\r\nLets take 3 almost identical cards: <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-780-ti\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-titan-black\" target=\"_blank\">GTX Titan Black</a> and the <a href=\"http://www.nvidia.com/object/tesla-servers.html\" target=\"_blank\">Tesla K40c</a>. All are Kepler <a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">GK110</a> based GPUs, with the same number of SMX and cores (15 SMX, 2880 cores) and the same bus width (384-bit). The 780 Ti and Titan Black even have nearly same base clock speeds (~880MHz; K40c is 745MHz) and identical memory clock speeds (7GHz; K40 is 6GHz).\r\n\r\n<strong>Single Precision Performance</strong>\r\nWith respect to single precision performance, all 3 are fairly in the same ball park. Which the 780 Ti and Titan Black sit just around 5.1 TFlops, owing to their similar clock speeds, the Tesla K40c drops in at 4.3 TFlops. You could give the Tesla a pass because it has a lower clock speed.\r\n\r\nAll 3 only vary significantly in 3 categories.\r\n\r\n<strong>1. Memory</strong>\r\nThis doesn\'t affect performance very much (at least when the sizes fit in all GPUs). The price difference based on market prices of other GPUs with similar memory size variations should not be that big either.\r\n\r\n<strong>2. Price</strong>\r\nAt launch, the GTX 780 Ti was priced at $699, $999 for the Titan Black and an estimated $5500 for the K40c. Well, that escalated quickly.\r\n(Price Sources: <a href=\"http://www.anandtech.com/show/7492/the-geforce-gtx-780-ti-review\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.anandtech.com/show/7765/nvidias-geforce-gtx-titan-black-no-compromises-for-gaming-compute\" target=\"_blank\">GTX Titan Black</a>, <a href=\"http://www.anandtech.com/show/7521/nvidia-launches-tesla-k40\" target=\"_blank\">K40</a>)\r\n\r\n<strong>3. Double Precision Performance</strong>\r\nSo if the 780 Ti and the Titan Black are practically the same in every respect, why is there a $300 difference in their price at launch (discounting memory size difference)?\r\n\r\nThe answer is in the double precision capabilities. The 780 Ti is physically locked at 1:24 FP32 where has the Titan Black has an ace up it\'s sleeve.\r\n\r\nFor the Titan Black, the magic happens in the driver. The Titan black\'s driver gives the user an option to choose the double precision performance between 1:3 and 1:24 FP32. When the double precision performance is set to 1:24 FP32, which is the same as the 780 Ti, the the single precision performance of the Titan Black and 780 Ti are identical. But when the user sets the double precision performance to 1:3 FP32, the single precision performance is compromised to boost double precision performance and make it equal to the K40c. In other words, you can choose the performance of the Titan Black to match either the 780 Ti or the K40c based and your preference.\r\n\r\nThe K40c has a double precision performance of 1:3 without compromising the single precision performance. This is because the K40 is given a special double precision unit for every 3 single precision cores (<a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">white paper</a>). It combines the best of both worlds. NVIDIA also states that the Tesla GPUs go through a much more rigorous Q&A process which guarantees lesser failures and also has additional features such as ECC memory. Hence the large price difference.\r\n\r\n<strong>Benchmarks</strong>\r\nHere are some benchmarks comparaing FP32 and FP64 GEMM.\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=381229711&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=722963661&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1083990145&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1438814030&amp;format=interactive\"></iframe>\r\n\r\nOne thing you would notice from the above charts is that the AMD GPUs performance fairly well and have a theoretical 1:3 FP32 performance for FP64. Most AMD cards will give between 1:3 and 1:8 FP32 performance for FP64. The FirePro W9100 will give you an incredible FP64 1:2 FP32 performance. The architecture of AMD GPUs is such that do not suffer from the same problems FP64 problems as NVIDIA\'s GTX series.\r\n\r\nThere is no perfect solution to the problem of choosing between FP32 and FP64 when it comes to GPUs. The best users can do is to decide whether they need the accuracy of double precision compute and how much compute they need, eventually going for a GPU that maximizes their productivity.\r\n\r\nDisclaimer: We receive GPUs from both NVIDIA and AMD. However, this blog does not promote or advocate any specific vendor.','Explaining FP64 performance on GPUs','','inherit','open','closed','','8389-revision-v1','','','2015-06-22 14:50:49','2015-06-22 14:50:49','',8389,'http://sandbox.arrayfire.com/8389-revision-v1/',0,'revision','',0),(8405,11,'2015-06-22 15:04:58','2015-06-22 15:04:58','GPUs are really good at doing math. The Achilles heel is when it comes to 64-bit double precision math. GPUs, at least consumer grade, are not built for high performance FP64. This is because they are targeted towards gamers and game developers, who do not really care about high precision compute. So vendors like NVIDIA and AMD do not cram FP64 compute cores in their GPUs.\r\n\r\nFor example, on a GTX 780 Ti, the FP64 performance is 1/24 FP32. Which means in an ideal case, running the same code by only changing float types to double types, would yield the double precision run time to be about 1/24th of the single precision time.\r\n\r\nKeep in mind, the theoretical best case for FP64 performance is 1:2 FP32, simply because it involves computing with double the number of bits as FP32.\r\n\r\nHow double precision performs really depends on the architecture of the GPU.\r\n\r\nLets take 3 almost identical cards: <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-780-ti\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-titan-black\" target=\"_blank\">GTX Titan Black</a> and the <a href=\"http://www.nvidia.com/object/tesla-servers.html\" target=\"_blank\">Tesla K40c</a>. All are Kepler <a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">GK110</a> based GPUs, with the same number of SMX and cores (15 SMX, 2880 cores) and the same bus width (384-bit). The 780 Ti and Titan Black even have nearly same base clock speeds (~880MHz; K40c is 745MHz) and identical memory clock speeds (7GHz; K40 is 6GHz).\r\n\r\n<strong>Single Precision Performance</strong>\r\nWith respect to single precision performance, all 3 are fairly in the same ball park. Which the 780 Ti and Titan Black sit just around 5.1 TFlops, owing to their similar clock speeds, the Tesla K40c drops in at 4.3 TFlops. You could give the Tesla a pass because it has a lower clock speed.\r\n\r\nAll 3 only vary significantly in 3 categories.\r\n\r\n<strong>1. Memory</strong>\r\nThis doesn\'t affect performance very much (at least when the sizes fit in all GPUs). The price difference based on market prices of other GPUs with similar memory size variations should not be that big either.\r\n\r\n<strong>2. Price</strong>\r\nAt launch, the GTX 780 Ti was priced at $699, $999 for the Titan Black and an estimated $5500 for the K40c. Well, that escalated quickly.\r\n(Price Sources: <a href=\"http://www.anandtech.com/show/7492/the-geforce-gtx-780-ti-review\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.anandtech.com/show/7765/nvidias-geforce-gtx-titan-black-no-compromises-for-gaming-compute\" target=\"_blank\">GTX Titan Black</a>, <a href=\"http://www.anandtech.com/show/7521/nvidia-launches-tesla-k40\" target=\"_blank\">K40</a>)\r\n\r\n<strong>3. Double Precision Performance</strong>\r\nSo if the 780 Ti and the Titan Black are practically the same in every respect, why is there a $300 difference in their price at launch (discounting memory size difference)?\r\n\r\nThe answer is in the double precision capabilities. The 780 Ti is physically locked at 1:24 FP32 where has the Titan Black has an ace up it\'s sleeve.\r\n\r\nFor the Titan Black, the magic happens in the driver. The Titan black\'s driver gives the user an option to choose the double precision performance between 1:3 and 1:24 FP32. When the double precision performance is set to 1:24 FP32, which is the same as the 780 Ti, the the single precision performance of the Titan Black and 780 Ti are identical. But when the user sets the double precision performance to 1:3 FP32, the single precision performance is compromised to boost double precision performance and make it equal to the K40c. In other words, you can choose the performance of the Titan Black to match either the 780 Ti or the K40c based and your preference.\r\n\r\nThe K40c has a double precision performance of 1:3 without compromising the single precision performance. This is because the K40 is given a special double precision unit for every 3 single precision cores (<a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">white paper</a>). It combines the best of both worlds. NVIDIA also states that the Tesla GPUs go through a much more rigorous Q&A process which guarantees lesser failures and also has additional features such as ECC memory. Hence the large price difference.\r\n\r\n<strong>Benchmarks</strong>\r\nHere are some benchmarks comparaing FP32 and FP64 GEMM.\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=381229711&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=722963661&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1083990145&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1438814030&amp;format=interactive\"></iframe>\r\n\r\nOne thing you would notice from the above charts is that the AMD GPUs performance fairly well and have a theoretical 1:3 FP32 performance for FP64. Most AMD cards will give between 1:3 and 1:8 FP32 performance for FP64. The FirePro W9100 will give you an incredible FP64 1:2 FP32 performance. The AMD Tahiti architectures tested in these benchmarks here do not suffer from the same problems FP64 problems as NVIDIA\'s GTX series and give a 1:4 performance. Newer Hawaii architecture GPUs are expected to provide 1:8 performance. \r\n\r\nThere is no perfect solution to the problem of choosing between FP32 and FP64 when it comes to GPUs. The best users can do is to decide whether they need the accuracy of double precision compute and how much compute they need, eventually going for a GPU that maximizes their productivity. Some applications like physics modelling and simulation, high accuracy financial computations etc which call for double precision accuracy at high performance require capable FP64 cards. Applications such as image and video processing, signal processing, statistics may not require such high precision and can get away with high FP32 performance only.\r\n\r\nDisclaimer: We receive GPUs from both NVIDIA and AMD. However, this blog does not promote or advocate any specific vendor.','Explaining FP64 performance on GPUs','','inherit','open','closed','','8389-revision-v1','','','2015-06-22 15:04:58','2015-06-22 15:04:58','',8389,'http://sandbox.arrayfire.com/8389-revision-v1/',0,'revision','',0),(8406,11,'2015-06-22 15:38:34','2015-06-22 15:38:34','GPUs are really good at doing math. The Achilles heel is when it comes to 64-bit double precision math. GPUs, at least consumer grade, are not built for high performance FP64. This is because they are targeted towards gamers and game developers, who do not really care about high precision compute. So vendors like NVIDIA and AMD do not cram FP64 compute cores in their GPUs.\r\n\r\nFor example, on a GTX 780 Ti, the FP64 performance is 1/24 FP32. Which means in an ideal case, running the same code by only changing float types to double types, would yield the double precision run time to be about 1/24th of the single precision time.\r\n\r\nKeep in mind, the theoretical best case for FP64 performance is 1:2 FP32, simply because it involves computing with double the number of bits as FP32.\r\n\r\nHow double precision performs really depends on the architecture of the GPU.\r\n\r\nLets take 3 almost identical cards: <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-780-ti\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-titan-black\" target=\"_blank\">GTX Titan Black</a> and the <a href=\"http://www.nvidia.com/object/tesla-servers.html\" target=\"_blank\">Tesla K40c</a>. All are Kepler <a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">GK110</a> based GPUs, with the same number of SMX and cores (15 SMX, 2880 cores) and the same bus width (384-bit). The 780 Ti and Titan Black even have nearly same base clock speeds (~880MHz; K40c is 745MHz) and identical memory clock speeds (7GHz; K40 is 6GHz).\r\n\r\n<strong>Single Precision Performance</strong>\r\nWith respect to single precision performance, all 3 are fairly in the same ball park. Which the 780 Ti and Titan Black sit just around 5.1 TFlops, owing to their similar clock speeds, the Tesla K40c drops in at 4.3 TFlops. You could give the Tesla a pass because it has a lower clock speed.\r\n\r\nAll 3 only vary significantly in 3 categories.\r\n\r\n<strong>1. Memory</strong>\r\nThis doesn\'t affect performance very much (at least when the sizes fit in all GPUs). The price difference based on market prices of other GPUs with similar memory size variations should not be that big either.\r\n\r\n<strong>2. Price</strong>\r\nAt launch, the GTX 780 Ti was priced at $699, $999 for the Titan Black and an estimated $5500 for the K40c. Well, that escalated quickly.\r\n(Price Sources: <a href=\"http://www.anandtech.com/show/7492/the-geforce-gtx-780-ti-review\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.anandtech.com/show/7765/nvidias-geforce-gtx-titan-black-no-compromises-for-gaming-compute\" target=\"_blank\">GTX Titan Black</a>, <a href=\"http://www.anandtech.com/show/7521/nvidia-launches-tesla-k40\" target=\"_blank\">K40</a>)\r\n\r\n<strong>3. Double Precision Performance</strong>\r\nSo if the 780 Ti and the Titan Black are practically the same in every respect, why is there a $300 difference in their price at launch (discounting memory size difference)?\r\n\r\nThe answer is in the double precision capabilities. The 780 Ti is physically locked at 1:24 FP32 where has the Titan Black has an ace up it\'s sleeve.\r\n\r\nFor the Titan Black, the magic happens in the driver. The Titan black\'s driver gives the user an option to choose the double precision performance between 1:3 and 1:24 FP32 (by switch the GPU to TCC mode). When the double precision performance is set to 1:24 FP32, which is the same as the 780 Ti, the the single precision performance of the Titan Black and 780 Ti are identical. But when the user sets the double precision performance to 1:3 FP32, the single precision performance is compromised to boost double precision performance and make it equal to the K40c. In other words, you can choose the performance of the Titan Black to match either the 780 Ti or the K40c based and your preference.\r\n\r\nThe K40c has a double precision performance of 1:3 without compromising the single precision performance. This is because the K40 is given a special double precision unit for every 3 single precision cores (<a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">white paper</a>). It combines the best of both worlds. NVIDIA also states that the Tesla GPUs go through a much more rigorous Q&A process which guarantees lesser failures and also has additional features such as ECC memory. Hence the large price difference.\r\n\r\n<strong>Benchmarks</strong>\r\nHere are some benchmarks comparaing FP32 and FP64 GEMM.\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=381229711&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=722963661&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1083990145&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1438814030&amp;format=interactive\"></iframe>\r\n\r\nOne thing you would notice from the above charts is that the AMD GPUs performance fairly well and have a theoretical 1:3 FP32 performance for FP64. Most AMD cards will give between 1:3 and 1:8 FP32 performance for FP64. The FirePro W9100, W8100 and S9150 will give you an incredible FP64 1:2 FP32 performance. The AMD Tahiti architectures tested in these benchmarks here do not suffer from the same problems FP64 problems as NVIDIA\'s GTX series and give a 1:4 performance. Newer Hawaii architecture consumer grade GPUs are expected to provide 1:8 performance. \r\n\r\nThere is no perfect solution to the problem of choosing between FP32 and FP64 when it comes to GPUs. The best users can do is to decide whether they need the accuracy of double precision compute and how much compute they need, eventually going for a GPU that maximizes their productivity. Some applications like physics modelling and simulation, high accuracy financial computations etc which call for double precision accuracy at high performance require capable FP64 cards. Applications such as image and video processing, signal processing, statistics may not require such high precision and can get away with high FP32 performance only.\r\n\r\nDisclaimer: We receive GPUs from both NVIDIA and AMD. However, this blog does not promote or advocate any specific vendor.','Explaining FP64 performance on GPUs','','inherit','open','closed','','8389-revision-v1','','','2015-06-22 15:38:34','2015-06-22 15:38:34','',8389,'http://sandbox.arrayfire.com/8389-revision-v1/',0,'revision','',0),(8407,11,'2015-06-22 15:38:47','2015-06-22 15:38:47','GPUs are really good at doing math. The Achilles heel is when it comes to 64-bit double precision math. GPUs, at least consumer grade, are not built for high performance FP64. This is because they are targeted towards gamers and game developers, who do not really care about high precision compute. So vendors like NVIDIA and AMD do not cram FP64 compute cores in their GPUs.\r\n\r\nFor example, on a GTX 780 Ti, the FP64 performance is 1/24 FP32. Which means in an ideal case, running the same code by only changing float types to double types, would yield the double precision run time to be about 1/24th of the single precision time.\r\n\r\nKeep in mind, the theoretical best case for FP64 performance is 1:2 FP32, simply because it involves computing with double the number of bits as FP32.\r\n\r\nHow double precision performs really depends on the architecture of the GPU.\r\n\r\nLets take 3 almost identical cards: <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-780-ti\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-titan-black\" target=\"_blank\">GTX Titan Black</a> and the <a href=\"http://www.nvidia.com/object/tesla-servers.html\" target=\"_blank\">Tesla K40c</a>. All are Kepler <a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">GK110</a> based GPUs, with the same number of SMX and cores (15 SMX, 2880 cores) and the same bus width (384-bit). The 780 Ti and Titan Black even have nearly same base clock speeds (~880MHz; K40c is 745MHz) and identical memory clock speeds (7GHz; K40 is 6GHz).\r\n\r\n<strong>Single Precision Performance</strong>\r\nWith respect to single precision performance, all 3 are fairly in the same ball park. Which the 780 Ti and Titan Black sit just around 5.1 TFlops, owing to their similar clock speeds, the Tesla K40c drops in at 4.3 TFlops. You could give the Tesla a pass because it has a lower clock speed.\r\n\r\nAll 3 only vary significantly in 3 categories.\r\n\r\n<strong>1. Memory</strong>\r\nThis doesn\'t affect performance very much (at least when the sizes fit in all GPUs). The price difference based on market prices of other GPUs with similar memory size variations should not be that big either.\r\n\r\n<strong>2. Price</strong>\r\nAt launch, the GTX 780 Ti was priced at $699, $999 for the Titan Black and an estimated $5500 for the K40c. Well, that escalated quickly.\r\n(Price Sources: <a href=\"http://www.anandtech.com/show/7492/the-geforce-gtx-780-ti-review\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.anandtech.com/show/7765/nvidias-geforce-gtx-titan-black-no-compromises-for-gaming-compute\" target=\"_blank\">GTX Titan Black</a>, <a href=\"http://www.anandtech.com/show/7521/nvidia-launches-tesla-k40\" target=\"_blank\">K40</a>)\r\n\r\n<strong>3. Double Precision Performance</strong>\r\nSo if the 780 Ti and the Titan Black are practically the same in every respect, why is there a $300 difference in their price at launch (discounting memory size difference)?\r\n\r\nThe answer is in the double precision capabilities. The 780 Ti is physically locked at 1:24 FP32 where has the Titan Black has an ace up it\'s sleeve.\r\n\r\nFor the Titan Black, the magic happens in the driver. The Titan black\'s driver gives the user an option to choose the double precision performance between 1:3 and 1:24 FP32 (by switching the GPU to TCC mode). When the double precision performance is set to 1:24 FP32, which is the same as the 780 Ti, the the single precision performance of the Titan Black and 780 Ti are identical. But when the user sets the double precision performance to 1:3 FP32, the single precision performance is compromised to boost double precision performance and make it equal to the K40c. In other words, you can choose the performance of the Titan Black to match either the 780 Ti or the K40c based and your preference.\r\n\r\nThe K40c has a double precision performance of 1:3 without compromising the single precision performance. This is because the K40 is given a special double precision unit for every 3 single precision cores (<a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">white paper</a>). It combines the best of both worlds. NVIDIA also states that the Tesla GPUs go through a much more rigorous Q&A process which guarantees lesser failures and also has additional features such as ECC memory. Hence the large price difference.\r\n\r\n<strong>Benchmarks</strong>\r\nHere are some benchmarks comparaing FP32 and FP64 GEMM.\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=381229711&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=722963661&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1083990145&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1438814030&amp;format=interactive\"></iframe>\r\n\r\nOne thing you would notice from the above charts is that the AMD GPUs performance fairly well and have a theoretical 1:3 FP32 performance for FP64. Most AMD cards will give between 1:3 and 1:8 FP32 performance for FP64. The FirePro W9100, W8100 and S9150 will give you an incredible FP64 1:2 FP32 performance. The AMD Tahiti architectures tested in these benchmarks here do not suffer from the same problems FP64 problems as NVIDIA\'s GTX series and give a 1:4 performance. Newer Hawaii architecture consumer grade GPUs are expected to provide 1:8 performance. \r\n\r\nThere is no perfect solution to the problem of choosing between FP32 and FP64 when it comes to GPUs. The best users can do is to decide whether they need the accuracy of double precision compute and how much compute they need, eventually going for a GPU that maximizes their productivity. Some applications like physics modelling and simulation, high accuracy financial computations etc which call for double precision accuracy at high performance require capable FP64 cards. Applications such as image and video processing, signal processing, statistics may not require such high precision and can get away with high FP32 performance only.\r\n\r\nDisclaimer: We receive GPUs from both NVIDIA and AMD. However, this blog does not promote or advocate any specific vendor.','Explaining FP64 performance on GPUs','','inherit','open','closed','','8389-revision-v1','','','2015-06-22 15:38:47','2015-06-22 15:38:47','',8389,'http://sandbox.arrayfire.com/8389-revision-v1/',0,'revision','',0),(8408,11,'2015-06-22 15:40:19','2015-06-22 15:40:19','GPUs are really good at doing math. The Achilles heel is when it comes to 64-bit double precision math. GPUs, at least consumer grade, are not built for high performance FP64. This is because they are targeted towards gamers and game developers, who do not really care about high precision compute. So vendors like NVIDIA and AMD do not cram FP64 compute cores in their GPUs.\r\n\r\nFor example, on a GTX 780 Ti, the FP64 performance is 1/24 FP32. Which means in an ideal case, running the same code by only changing float types to double types, would yield the double precision run time to be about 1/24th of the single precision time.\r\n\r\nKeep in mind, the theoretical best case for FP64 performance is 1:2 FP32, simply because it involves computing with double the number of bits as FP32.\r\n\r\nHow double precision performs really depends on the architecture of the GPU.\r\n\r\nLets take 3 almost identical cards: <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-780-ti\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-titan-black\" target=\"_blank\">GTX Titan Black</a> and the <a href=\"http://www.nvidia.com/object/tesla-servers.html\" target=\"_blank\">Tesla K40c</a>. All are Kepler <a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">GK110</a> based GPUs, with the same number of SMX and cores (15 SMX, 2880 cores) and the same bus width (384-bit). The 780 Ti and Titan Black even have nearly same base clock speeds (~880MHz; K40c is 745MHz) and identical memory clock speeds (7GHz; K40 is 6GHz).\r\n\r\n<strong>Single Precision Performance</strong>\r\nWith respect to single precision performance, all 3 are fairly in the same ball park. Which the 780 Ti and Titan Black sit just around 5.1 TFlops, owing to their similar clock speeds, the Tesla K40c drops in at 4.3 TFlops. You could give the Tesla a pass because it has a lower clock speed.\r\n\r\nAll 3 only vary significantly in 3 categories.\r\n\r\n<strong>1. Memory</strong>\r\nThis doesn\'t affect performance very much (at least when the sizes fit in all GPUs). The price difference based on market prices of other GPUs with similar memory size variations should not be that big either.\r\n\r\n<strong>2. Price</strong>\r\nAt launch, the GTX 780 Ti was priced at $699, $999 for the Titan Black and an estimated $5500 for the K40c. Well, that escalated quickly.\r\n(Price Sources: <a href=\"http://www.anandtech.com/show/7492/the-geforce-gtx-780-ti-review\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.anandtech.com/show/7765/nvidias-geforce-gtx-titan-black-no-compromises-for-gaming-compute\" target=\"_blank\">GTX Titan Black</a>, <a href=\"http://www.anandtech.com/show/7521/nvidia-launches-tesla-k40\" target=\"_blank\">K40</a>)\r\n\r\n<strong>3. Double Precision Performance</strong>\r\nSo if the 780 Ti and the Titan Black are practically the same in every respect, why is there a $300 difference in their price at launch (discounting memory size difference)?\r\n\r\nThe answer is in the double precision capabilities. The 780 Ti is physically locked at 1:24 FP32 where has the Titan Black has an ace up it\'s sleeve.\r\n\r\nFor the Titan Black, the magic happens in the driver. The Titan black\'s driver gives the user an option to choose the double precision performance between 1:3 and 1:24 FP32 (by switching the GPU to TCC mode). When the double precision performance is set to 1:24 FP32, which is the same as the 780 Ti, the the single precision performance of the Titan Black and 780 Ti are identical. But when the user sets the double precision performance to 1:3 FP32, the single precision performance is compromised to boost double precision performance and make it equal to the K40c. In other words, you can choose the performance of the Titan Black to match either the 780 Ti or the K40c based and your preference.\r\n\r\nThe K40c has a double precision performance of 1:3 without compromising the single precision performance. This is because the K40 is given a special double precision unit for every 3 single precision cores (<a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">white paper</a>). It combines the best of both worlds. NVIDIA also states that the Tesla GPUs go through a much more rigorous Q&A process which guarantees lesser failures and also has additional features such as ECC memory. Hence the large price difference.\r\n\r\n<strong>Benchmarks</strong>\r\nHere are some benchmarks comparaing FP32 and FP64 GEMM.\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=381229711&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=722963661&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1083990145&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1438814030&amp;format=interactive\"></iframe>\r\n\r\nOne thing you would notice from the above charts is that the AMD GPUs perform fairly well for FP64 compared to FP32. Most AMD cards will give between 1:3 and 1:8 FP32 performance for FP64. The FirePro W9100, W8100 and S9150 will give you an incredible FP64 1:2 FP32 performance. The AMD Tahiti architectures tested in these benchmarks here do not suffer from the same problems FP64 problems as NVIDIA\'s GTX series and give a 1:4 performance. Newer Hawaii architecture consumer grade GPUs are expected to provide 1:8 performance.\r\n\r\nThere is no perfect solution to the problem of choosing between FP32 and FP64 when it comes to GPUs. The best users can do is to decide whether they need the accuracy of double precision compute and how much compute they need, eventually going for a GPU that maximizes their productivity. Some applications like physics modelling and simulation, high accuracy financial computations etc which call for double precision accuracy at high performance require capable FP64 cards. Applications such as image and video processing, signal processing, statistics may not require such high precision and can get away with high FP32 performance only.\r\n\r\nDisclaimer: We receive GPUs from both NVIDIA and AMD. However, this blog does not promote or advocate any specific vendor.','Explaining FP64 performance on GPUs','','inherit','open','closed','','8389-revision-v1','','','2015-06-22 15:40:19','2015-06-22 15:40:19','',8389,'http://sandbox.arrayfire.com/8389-revision-v1/',0,'revision','',0),(8409,11,'2015-06-22 15:44:54','2015-06-22 15:44:54','## Introduction\r\nGPUs are really good at doing math. The Achilles heel is when it comes to 64-bit double precision math. GPUs, at least consumer grade, are not built for high performance FP64. This is because they are targeted towards gamers and game developers, who do not really care about high precision compute. So vendors like NVIDIA and AMD do not cram FP64 compute cores in their GPUs.\r\n\r\nFor example, on a GTX 780 Ti, the FP64 performance is 1/24 FP32. Which means in an ideal case, running the same code by only changing float types to double types, would yield the double precision run time to be about 1/24th of the single precision time.\r\n\r\nKeep in mind, the theoretical best case for FP64 performance is 1:2 FP32, simply because it involves computing with double the number of bits as FP32.\r\n\r\nHow double precision performs really depends on the architecture of the GPU.\r\n\r\n## Comparing NVIDIA GPUs\r\nLets take 3 almost identical cards: <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-780-ti\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-titan-black\" target=\"_blank\">GTX Titan Black</a> and the <a href=\"http://www.nvidia.com/object/tesla-servers.html\" target=\"_blank\">Tesla K40c</a>. All are Kepler <a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">GK110</a> based GPUs, with the same number of SMX and cores (15 SMX, 2880 cores) and the same bus width (384-bit). The 780 Ti and Titan Black even have nearly same base clock speeds (~880MHz; K40c is 745MHz) and identical memory clock speeds (7GHz; K40 is 6GHz).\r\n\r\n<strong>Single Precision Performance</strong>\r\nWith respect to single precision performance, all 3 are fairly in the same ball park. Which the 780 Ti and Titan Black sit just around 5.1 TFlops, owing to their similar clock speeds, the Tesla K40c drops in at 4.3 TFlops. You could give the Tesla a pass because it has a lower clock speed.\r\n\r\nAll 3 only vary significantly in 3 categories.\r\n\r\n<strong>1. Memory</strong>\r\nThis doesn\'t affect performance very much (at least when the sizes fit in all GPUs). The price difference based on market prices of other GPUs with similar memory size variations should not be that big either.\r\n\r\n<strong>2. Price</strong>\r\nAt launch, the GTX 780 Ti was priced at $699, $999 for the Titan Black and an estimated $5500 for the K40c. Well, that escalated quickly.\r\n(Price Sources: <a href=\"http://www.anandtech.com/show/7492/the-geforce-gtx-780-ti-review\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.anandtech.com/show/7765/nvidias-geforce-gtx-titan-black-no-compromises-for-gaming-compute\" target=\"_blank\">GTX Titan Black</a>, <a href=\"http://www.anandtech.com/show/7521/nvidia-launches-tesla-k40\" target=\"_blank\">K40</a>)\r\n\r\n<strong>3. Double Precision Performance</strong>\r\nSo if the 780 Ti and the Titan Black are practically the same in every respect, why is there a $300 difference in their price at launch (discounting memory size difference)?\r\n\r\nThe answer is in the double precision capabilities. The 780 Ti is physically locked at 1:24 FP32 where has the Titan Black has an ace up it\'s sleeve.\r\n\r\nFor the Titan Black, the magic happens in the driver. The Titan black\'s driver gives the user an option to choose the double precision performance between 1:3 and 1:24 FP32 (by switching the GPU to TCC mode). When the double precision performance is set to 1:24 FP32, which is the same as the 780 Ti, the the single precision performance of the Titan Black and 780 Ti are identical. But when the user sets the double precision performance to 1:3 FP32, the single precision performance is compromised to boost double precision performance and make it equal to the K40c. In other words, you can choose the performance of the Titan Black to match either the 780 Ti or the K40c based and your preference.\r\n\r\nThe K40c has a double precision performance of 1:3 without compromising the single precision performance. This is because the K40 is given a special double precision unit for every 3 single precision cores (<a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">white paper</a>). It combines the best of both worlds. NVIDIA also states that the Tesla GPUs go through a much more rigorous Q&A process which guarantees lesser failures and also has additional features such as ECC memory. Hence the large price difference.\r\n\r\n## AMD GPUs\r\n\r\nAMD GPUs perform fairly well for FP64 compared to FP32. Most AMD cards (including consumer/gaming series) will give between 1:3 and 1:8 FP32 performance for FP64. The FirePro W9100, W8100 and S9150 will give you an incredible FP64 1:2 FP32 performance. The AMD Tahiti architectures tested in these benchmarks here do not suffer from the same problems FP64 problems as NVIDIA\'s GTX series and give a 1:4 performance. Newer Hawaii architecture consumer grade GPUs are expected to provide 1:8 performance. Overall, AMD GPUs hold a reputation for good double precision performance compared to their NVIDIA counterparts.\r\n\r\n## Benchmarks\r\nHere are some benchmarks comparaing FP32 and FP64 GEMM.\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=381229711&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=722963661&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1083990145&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1438814030&amp;format=interactive\"></iframe>\r\n\r\n## Conclusion\r\nThere is no perfect solution to the problem of choosing between FP32 and FP64 when it comes to GPUs. The best users can do is to decide whether they need the accuracy of double precision compute and how much compute they need, eventually going for a GPU that maximizes their productivity. Some applications like physics modelling and simulation, high accuracy financial computations etc which call for double precision accuracy at high performance require capable FP64 cards. Applications such as image and video processing, signal processing, statistics may not require such high precision and can get away with high FP32 performance only.\r\n\r\nDisclaimer: We receive GPUs from both NVIDIA and AMD. However, this blog does not promote or advocate any specific vendor.','Explaining FP64 performance on GPUs','','inherit','open','closed','','8389-revision-v1','','','2015-06-22 15:44:54','2015-06-22 15:44:54','',8389,'http://sandbox.arrayfire.com/8389-revision-v1/',0,'revision','',0),(8410,11,'2015-06-22 15:45:16','2015-06-22 15:45:16','##Introduction\r\nGPUs are really good at doing math. The Achilles heel is when it comes to 64-bit double precision math. GPUs, at least consumer grade, are not built for high performance FP64. This is because they are targeted towards gamers and game developers, who do not really care about high precision compute. So vendors like NVIDIA and AMD do not cram FP64 compute cores in their GPUs.\r\n\r\nFor example, on a GTX 780 Ti, the FP64 performance is 1/24 FP32. Which means in an ideal case, running the same code by only changing float types to double types, would yield the double precision run time to be about 1/24th of the single precision time.\r\n\r\nKeep in mind, the theoretical best case for FP64 performance is 1:2 FP32, simply because it involves computing with double the number of bits as FP32.\r\n\r\nHow double precision performs really depends on the architecture of the GPU.\r\n\r\n## Comparing NVIDIA GPUs\r\nLets take 3 almost identical cards: <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-780-ti\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-titan-black\" target=\"_blank\">GTX Titan Black</a> and the <a href=\"http://www.nvidia.com/object/tesla-servers.html\" target=\"_blank\">Tesla K40c</a>. All are Kepler <a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">GK110</a> based GPUs, with the same number of SMX and cores (15 SMX, 2880 cores) and the same bus width (384-bit). The 780 Ti and Titan Black even have nearly same base clock speeds (~880MHz; K40c is 745MHz) and identical memory clock speeds (7GHz; K40 is 6GHz).\r\n\r\n<strong>Single Precision Performance</strong>\r\nWith respect to single precision performance, all 3 are fairly in the same ball park. Which the 780 Ti and Titan Black sit just around 5.1 TFlops, owing to their similar clock speeds, the Tesla K40c drops in at 4.3 TFlops. You could give the Tesla a pass because it has a lower clock speed.\r\n\r\nAll 3 only vary significantly in 3 categories.\r\n\r\n<strong>1. Memory</strong>\r\nThis doesn\'t affect performance very much (at least when the sizes fit in all GPUs). The price difference based on market prices of other GPUs with similar memory size variations should not be that big either.\r\n\r\n<strong>2. Price</strong>\r\nAt launch, the GTX 780 Ti was priced at $699, $999 for the Titan Black and an estimated $5500 for the K40c. Well, that escalated quickly.\r\n(Price Sources: <a href=\"http://www.anandtech.com/show/7492/the-geforce-gtx-780-ti-review\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.anandtech.com/show/7765/nvidias-geforce-gtx-titan-black-no-compromises-for-gaming-compute\" target=\"_blank\">GTX Titan Black</a>, <a href=\"http://www.anandtech.com/show/7521/nvidia-launches-tesla-k40\" target=\"_blank\">K40</a>)\r\n\r\n<strong>3. Double Precision Performance</strong>\r\nSo if the 780 Ti and the Titan Black are practically the same in every respect, why is there a $300 difference in their price at launch (discounting memory size difference)?\r\n\r\nThe answer is in the double precision capabilities. The 780 Ti is physically locked at 1:24 FP32 where has the Titan Black has an ace up it\'s sleeve.\r\n\r\nFor the Titan Black, the magic happens in the driver. The Titan black\'s driver gives the user an option to choose the double precision performance between 1:3 and 1:24 FP32 (by switching the GPU to TCC mode). When the double precision performance is set to 1:24 FP32, which is the same as the 780 Ti, the the single precision performance of the Titan Black and 780 Ti are identical. But when the user sets the double precision performance to 1:3 FP32, the single precision performance is compromised to boost double precision performance and make it equal to the K40c. In other words, you can choose the performance of the Titan Black to match either the 780 Ti or the K40c based and your preference.\r\n\r\nThe K40c has a double precision performance of 1:3 without compromising the single precision performance. This is because the K40 is given a special double precision unit for every 3 single precision cores (<a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">white paper</a>). It combines the best of both worlds. NVIDIA also states that the Tesla GPUs go through a much more rigorous Q&A process which guarantees lesser failures and also has additional features such as ECC memory. Hence the large price difference.\r\n\r\n## AMD GPUs\r\n\r\nAMD GPUs perform fairly well for FP64 compared to FP32. Most AMD cards (including consumer/gaming series) will give between 1:3 and 1:8 FP32 performance for FP64. The FirePro W9100, W8100 and S9150 will give you an incredible FP64 1:2 FP32 performance. The AMD Tahiti architectures tested in these benchmarks here do not suffer from the same problems FP64 problems as NVIDIA\'s GTX series and give a 1:4 performance. Newer Hawaii architecture consumer grade GPUs are expected to provide 1:8 performance. Overall, AMD GPUs hold a reputation for good double precision performance compared to their NVIDIA counterparts.\r\n\r\n## Benchmarks\r\nHere are some benchmarks comparaing FP32 and FP64 GEMM.\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=381229711&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=722963661&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1083990145&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1438814030&amp;format=interactive\"></iframe>\r\n\r\n## Conclusion\r\nThere is no perfect solution to the problem of choosing between FP32 and FP64 when it comes to GPUs. The best users can do is to decide whether they need the accuracy of double precision compute and how much compute they need, eventually going for a GPU that maximizes their productivity. Some applications like physics modelling and simulation, high accuracy financial computations etc which call for double precision accuracy at high performance require capable FP64 cards. Applications such as image and video processing, signal processing, statistics may not require such high precision and can get away with high FP32 performance only.\r\n\r\nDisclaimer: We receive GPUs from both NVIDIA and AMD. However, this blog does not promote or advocate any specific vendor.','Explaining FP64 performance on GPUs','','inherit','open','closed','','8389-revision-v1','','','2015-06-22 15:45:16','2015-06-22 15:45:16','',8389,'http://sandbox.arrayfire.com/8389-revision-v1/',0,'revision','',0),(8411,11,'2015-06-22 15:46:21','2015-06-22 15:46:21','<h2>Introduction</h2>\r\nGPUs are really good at doing math. The Achilles heel is when it comes to 64-bit double precision math. GPUs, at least consumer grade, are not built for high performance FP64. This is because they are targeted towards gamers and game developers, who do not really care about high precision compute. So vendors like NVIDIA and AMD do not cram FP64 compute cores in their GPUs.\r\n\r\nFor example, on a GTX 780 Ti, the FP64 performance is 1/24 FP32. Which means in an ideal case, running the same code by only changing float types to double types, would yield the double precision run time to be about 1/24th of the single precision time.\r\n\r\nKeep in mind, the theoretical best case for FP64 performance is 1:2 FP32, simply because it involves computing with double the number of bits as FP32.\r\n\r\nHow double precision performs really depends on the architecture of the GPU.\r\n\r\n<h2>Comparing NVIDIA GPUs</h2>\r\nLets take 3 almost identical cards: <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-780-ti\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-titan-black\" target=\"_blank\">GTX Titan Black</a> and the <a href=\"http://www.nvidia.com/object/tesla-servers.html\" target=\"_blank\">Tesla K40c</a>. All are Kepler <a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">GK110</a> based GPUs, with the same number of SMX and cores (15 SMX, 2880 cores) and the same bus width (384-bit). The 780 Ti and Titan Black even have nearly same base clock speeds (~880MHz; K40c is 745MHz) and identical memory clock speeds (7GHz; K40 is 6GHz).\r\n\r\n<strong>Single Precision Performance</strong>\r\nWith respect to single precision performance, all 3 are fairly in the same ball park. Which the 780 Ti and Titan Black sit just around 5.1 TFlops, owing to their similar clock speeds, the Tesla K40c drops in at 4.3 TFlops. You could give the Tesla a pass because it has a lower clock speed.\r\n\r\nAll 3 only vary significantly in 3 categories.\r\n\r\n<strong>1. Memory</strong>\r\nThis doesn\'t affect performance very much (at least when the sizes fit in all GPUs). The price difference based on market prices of other GPUs with similar memory size variations should not be that big either.\r\n\r\n<strong>2. Price</strong>\r\nAt launch, the GTX 780 Ti was priced at $699, $999 for the Titan Black and an estimated $5500 for the K40c. Well, that escalated quickly.\r\n(Price Sources: <a href=\"http://www.anandtech.com/show/7492/the-geforce-gtx-780-ti-review\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.anandtech.com/show/7765/nvidias-geforce-gtx-titan-black-no-compromises-for-gaming-compute\" target=\"_blank\">GTX Titan Black</a>, <a href=\"http://www.anandtech.com/show/7521/nvidia-launches-tesla-k40\" target=\"_blank\">K40</a>)\r\n\r\n<strong>3. Double Precision Performance</strong>\r\nSo if the 780 Ti and the Titan Black are practically the same in every respect, why is there a $300 difference in their price at launch (discounting memory size difference)?\r\n\r\nThe answer is in the double precision capabilities. The 780 Ti is physically locked at 1:24 FP32 where has the Titan Black has an ace up it\'s sleeve.\r\n\r\nFor the Titan Black, the magic happens in the driver. The Titan black\'s driver gives the user an option to choose the double precision performance between 1:3 and 1:24 FP32 (by switching the GPU to TCC mode). When the double precision performance is set to 1:24 FP32, which is the same as the 780 Ti, the the single precision performance of the Titan Black and 780 Ti are identical. But when the user sets the double precision performance to 1:3 FP32, the single precision performance is compromised to boost double precision performance and make it equal to the K40c. In other words, you can choose the performance of the Titan Black to match either the 780 Ti or the K40c based and your preference.\r\n\r\nThe K40c has a double precision performance of 1:3 without compromising the single precision performance. This is because the K40 is given a special double precision unit for every 3 single precision cores (<a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">white paper</a>). It combines the best of both worlds. NVIDIA also states that the Tesla GPUs go through a much more rigorous Q&A process which guarantees lesser failures and also has additional features such as ECC memory. Hence the large price difference.\r\n\r\n<h2>AMD GPUs</h2>\r\n\r\nAMD GPUs perform fairly well for FP64 compared to FP32. Most AMD cards (including consumer/gaming series) will give between 1:3 and 1:8 FP32 performance for FP64. The FirePro W9100, W8100 and S9150 will give you an incredible FP64 1:2 FP32 performance. The AMD Tahiti architectures tested in these benchmarks here do not suffer from the same problems FP64 problems as NVIDIA\'s GTX series and give a 1:4 performance. Newer Hawaii architecture consumer grade GPUs are expected to provide 1:8 performance. Overall, AMD GPUs hold a reputation for good double precision performance compared to their NVIDIA counterparts.\r\n\r\n<h2>Benchmarks</h2>\r\nHere are some benchmarks comparaing FP32 and FP64 GEMM.\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=381229711&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=722963661&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1083990145&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1438814030&amp;format=interactive\"></iframe>\r\n\r\n<h2>Conclusion</h2>\r\nThere is no perfect solution to the problem of choosing between FP32 and FP64 when it comes to GPUs. The best users can do is to decide whether they need the accuracy of double precision compute and how much compute they need, eventually going for a GPU that maximizes their productivity. Some applications like physics modelling and simulation, high accuracy financial computations etc which call for double precision accuracy at high performance require capable FP64 cards. Applications such as image and video processing, signal processing, statistics may not require such high precision and can get away with high FP32 performance only.\r\n\r\nDisclaimer: We receive GPUs from both NVIDIA and AMD. However, this blog does not promote or advocate any specific vendor.','Explaining FP64 performance on GPUs','','inherit','open','closed','','8389-revision-v1','','','2015-06-22 15:46:21','2015-06-22 15:46:21','',8389,'http://sandbox.arrayfire.com/8389-revision-v1/',0,'revision','',0),(8412,11,'2015-06-22 15:46:59','2015-06-22 15:46:59','<h3>Introduction</h3>\r\nGPUs are really good at doing math. The Achilles heel is when it comes to 64-bit double precision math. GPUs, at least consumer grade, are not built for high performance FP64. This is because they are targeted towards gamers and game developers, who do not really care about high precision compute. So vendors like NVIDIA and AMD do not cram FP64 compute cores in their GPUs.\r\n\r\nFor example, on a GTX 780 Ti, the FP64 performance is 1/24 FP32. Which means in an ideal case, running the same code by only changing float types to double types, would yield the double precision run time to be about 1/24th of the single precision time.\r\n\r\nKeep in mind, the theoretical best case for FP64 performance is 1:2 FP32, simply because it involves computing with double the number of bits as FP32.\r\n\r\nHow double precision performs really depends on the architecture of the GPU.\r\n\r\n<h3>Comparing NVIDIA GPUs</h3>\r\nLets take 3 almost identical cards: <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-780-ti\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-titan-black\" target=\"_blank\">GTX Titan Black</a> and the <a href=\"http://www.nvidia.com/object/tesla-servers.html\" target=\"_blank\">Tesla K40c</a>. All are Kepler <a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">GK110</a> based GPUs, with the same number of SMX and cores (15 SMX, 2880 cores) and the same bus width (384-bit). The 780 Ti and Titan Black even have nearly same base clock speeds (~880MHz; K40c is 745MHz) and identical memory clock speeds (7GHz; K40 is 6GHz).\r\n\r\n<strong>Single Precision Performance</strong>\r\nWith respect to single precision performance, all 3 are fairly in the same ball park. Which the 780 Ti and Titan Black sit just around 5.1 TFlops, owing to their similar clock speeds, the Tesla K40c drops in at 4.3 TFlops. You could give the Tesla a pass because it has a lower clock speed.\r\n\r\nAll 3 only vary significantly in 3 categories.\r\n\r\n<strong>1. Memory</strong>\r\nThis doesn\'t affect performance very much (at least when the sizes fit in all GPUs). The price difference based on market prices of other GPUs with similar memory size variations should not be that big either.\r\n\r\n<strong>2. Price</strong>\r\nAt launch, the GTX 780 Ti was priced at $699, $999 for the Titan Black and an estimated $5500 for the K40c. Well, that escalated quickly.\r\n(Price Sources: <a href=\"http://www.anandtech.com/show/7492/the-geforce-gtx-780-ti-review\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.anandtech.com/show/7765/nvidias-geforce-gtx-titan-black-no-compromises-for-gaming-compute\" target=\"_blank\">GTX Titan Black</a>, <a href=\"http://www.anandtech.com/show/7521/nvidia-launches-tesla-k40\" target=\"_blank\">K40</a>)\r\n\r\n<strong>3. Double Precision Performance</strong>\r\nSo if the 780 Ti and the Titan Black are practically the same in every respect, why is there a $300 difference in their price at launch (discounting memory size difference)?\r\n\r\nThe answer is in the double precision capabilities. The 780 Ti is physically locked at 1:24 FP32 where has the Titan Black has an ace up it\'s sleeve.\r\n\r\nFor the Titan Black, the magic happens in the driver. The Titan black\'s driver gives the user an option to choose the double precision performance between 1:3 and 1:24 FP32 (by switching the GPU to TCC mode). When the double precision performance is set to 1:24 FP32, which is the same as the 780 Ti, the the single precision performance of the Titan Black and 780 Ti are identical. But when the user sets the double precision performance to 1:3 FP32, the single precision performance is compromised to boost double precision performance and make it equal to the K40c. In other words, you can choose the performance of the Titan Black to match either the 780 Ti or the K40c based and your preference.\r\n\r\nThe K40c has a double precision performance of 1:3 without compromising the single precision performance. This is because the K40 is given a special double precision unit for every 3 single precision cores (<a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">white paper</a>). It combines the best of both worlds. NVIDIA also states that the Tesla GPUs go through a much more rigorous Q&A process which guarantees lesser failures and also has additional features such as ECC memory. Hence the large price difference.\r\n\r\n<h3>AMD GPUs</h3>\r\n\r\nAMD GPUs perform fairly well for FP64 compared to FP32. Most AMD cards (including consumer/gaming series) will give between 1:3 and 1:8 FP32 performance for FP64. The FirePro W9100, W8100 and S9150 will give you an incredible FP64 1:2 FP32 performance. The AMD Tahiti architectures tested in these benchmarks here do not suffer from the same problems FP64 problems as NVIDIA\'s GTX series and give a 1:4 performance. Newer Hawaii architecture consumer grade GPUs are expected to provide 1:8 performance. Overall, AMD GPUs hold a reputation for good double precision performance compared to their NVIDIA counterparts.\r\n\r\n<h3>Benchmarks</h3>\r\nHere are some benchmarks comparaing FP32 and FP64 GEMM.\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=381229711&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=722963661&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1083990145&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1438814030&amp;format=interactive\"></iframe>\r\n\r\n<h3>Conclusion</h3>\r\nThere is no perfect solution to the problem of choosing between FP32 and FP64 when it comes to GPUs. The best users can do is to decide whether they need the accuracy of double precision compute and how much compute they need, eventually going for a GPU that maximizes their productivity. Some applications like physics modelling and simulation, high accuracy financial computations etc which call for double precision accuracy at high performance require capable FP64 cards. Applications such as image and video processing, signal processing, statistics may not require such high precision and can get away with high FP32 performance only.\r\n\r\nDisclaimer: We receive GPUs from both NVIDIA and AMD. However, this blog does not promote or advocate any specific vendor.','Explaining FP64 performance on GPUs','','inherit','open','closed','','8389-revision-v1','','','2015-06-22 15:46:59','2015-06-22 15:46:59','',8389,'http://sandbox.arrayfire.com/8389-revision-v1/',0,'revision','',0),(8413,11,'2015-06-22 15:57:23','2015-06-22 15:57:23','<h3>Introduction</h3>\r\nGPUs are really good at doing math. The Achilles heel is when it comes to 64-bit double precision math. GPUs, at least consumer grade, are not built for high performance FP64. This is because they are targeted towards gamers and game developers, who do not really care about high precision compute. So vendors like NVIDIA and AMD do not cram FP64 compute cores in their GPUs.\r\n\r\nFor example, on a GTX 780 Ti, the FP64 performance is 1/24 FP32. Which means in an ideal case, running the same code by only changing float types to double types, would yield the double precision run time to be about 1/24th of the single precision time.\r\n\r\nKeep in mind, the theoretical best case for FP64 performance is 1:2 FP32, simply because it involves computing with double the number of bits as FP32.\r\n\r\nHow double precision performs really depends on the architecture of the GPU.\r\n\r\n<h3>Comparing NVIDIA GPUs</h3>\r\nLets take 3 almost identical cards: <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-780-ti\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-titan-black\" target=\"_blank\">GTX Titan Black</a> and the <a href=\"http://www.nvidia.com/object/tesla-servers.html\" target=\"_blank\">Tesla K40c</a>. All are Kepler <a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">GK110</a> based GPUs, with the same number of SMX and cores (15 SMX, 2880 cores) and the same bus width (384-bit). The 780 Ti and Titan Black even have nearly same base clock speeds (~880MHz; K40c is 745MHz) and identical memory clock speeds (7GHz; K40 is 6GHz).\r\n\r\n<strong>Single Precision Performance</strong>\r\nWith respect to single precision performance, all 3 are fairly in the same ball park. Which the 780 Ti and Titan Black sit just around 5.1 TFlops, owing to their similar clock speeds, the Tesla K40c drops in at 4.3 TFlops. You could give the Tesla a pass because it has a lower clock speed.\r\n\r\nAll 3 only vary significantly in 3 categories.\r\n\r\n<strong>1. Memory</strong>\r\nThis doesn\'t affect performance very much (at least when the sizes fit in all GPUs). The price difference based on market prices of other GPUs with similar memory size variations should not be that big either.\r\n\r\n<strong>2. Price</strong>\r\nAt launch, the GTX 780 Ti was priced at $699, $999 for the Titan Black and an estimated $5500 for the K40c. Well, that escalated quickly.\r\n(Price Sources: <a href=\"http://www.anandtech.com/show/7492/the-geforce-gtx-780-ti-review\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.anandtech.com/show/7765/nvidias-geforce-gtx-titan-black-no-compromises-for-gaming-compute\" target=\"_blank\">GTX Titan Black</a>, <a href=\"http://www.anandtech.com/show/7521/nvidia-launches-tesla-k40\" target=\"_blank\">K40</a>)\r\n\r\n<strong>3. Double Precision Performance</strong>\r\nSo if the 780 Ti and the Titan Black are practically the same in every respect, why is there a $300 difference in their price at launch (discounting memory size difference)?\r\n\r\nThe answer is in the double precision capabilities. The 780 Ti is physically locked at 1:24 FP32 where has the Titan Black has an ace up it\'s sleeve.\r\n\r\nFor the Titan Black, the magic happens in the driver. The Titan black\'s driver gives the user an option to choose the double precision performance between 1:3 and 1:24 FP32 (by switching the GPU to TCC mode). When the double precision performance is set to 1:24 FP32, which is the same as the 780 Ti, the the single precision performance of the Titan Black and 780 Ti are identical. But when the user sets the double precision performance to 1:3 FP32, the single precision performance is compromised to boost double precision performance and make it equal to the K40c. In other words, you can choose the performance of the Titan Black to match either the 780 Ti or the K40c based and your preference.\r\n\r\nThe K40c has a double precision performance of 1:3 without compromising the single precision performance. This is because the K40 is given a special double precision unit for every 3 single precision cores (<a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">white paper</a>). It combines the best of both worlds. NVIDIA also states that the Tesla GPUs go through a much more rigorous Q&A process which guarantees lesser failures and also has additional features such as ECC memory. Hence the large price difference.\r\n\r\n<h4>Summary of NVIDIA GPUs<h4>\r\n<strong>NVIDIA\'s GTX series</strong> are known for their great FP32 performance but are very poor in their FP64 performance. The performance generally ranges between <strong>1:24 (Kepler) and 1:32 (Maxwell)</strong>. The exceptions to this are the GTX Titan cards which blur the lines between the consumer GTX series and the professional Tesla/Quadro cards.\r\n\r\nThe <strong>Kepler architecture Quadro and Tesla series</strong> card provide full double precision performance with <strong>1:3 FP32</strong>. However, with the <strong>Quadro M6000</strong>, NVIDIA has decided to provide only minimal FP64 performance by giving it only <strong>1:32 of FP32 capability</strong> and touting the M6000 as the best graphics card rather than the best graphics+compute card like the Quadro K6000.\r\n\r\n<h3>AMD GPUs</h3>\r\n\r\nAMD GPUs perform fairly well for FP64 compared to FP32. Most AMD cards (including consumer/gaming series) will give between <strong>1:3 and 1:8 FP32 performance for FP64</strong>. The AMD Tahiti architectures tested in these benchmarks here do not suffer from the same problems FP64 problems as NVIDIA\'s GTX series and give a 1:4 performance. Newer Hawaii architecture consumer grade GPUs are expected to provide 1:8 performance.\r\nThe <strong>FirePro W9100, W8100 and S9150 will give you an incredible FP64 1:2 FP32 performance</strong>. Overall, AMD GPUs hold a reputation for good double precision performance compared to their NVIDIA counterparts.\r\n\r\n<h3>Benchmarks</h3>\r\nHere are some benchmarks comparaing FP32 and FP64 GEMM.\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=381229711&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=722963661&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1083990145&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1438814030&amp;format=interactive\"></iframe>\r\n\r\n<h3>Conclusion</h3>\r\nThere is no perfect solution to the problem of choosing between FP32 and FP64 when it comes to GPUs. The best users can do is to decide whether they need the accuracy of double precision compute and how much compute they need, eventually going for a GPU that maximizes their productivity. Some applications like physics modelling and simulation, high accuracy financial computations etc which call for double precision accuracy at high performance require capable FP64 cards. Applications such as image and video processing, signal processing, statistics may not require such high precision and can get away with high FP32 performance only.\r\n\r\nDisclaimer: We receive GPUs from both NVIDIA and AMD. However, this blog does not promote or advocate any specific vendor.','Explaining FP64 performance on GPUs','','inherit','open','closed','','8389-revision-v1','','','2015-06-22 15:57:23','2015-06-22 15:57:23','',8389,'http://sandbox.arrayfire.com/8389-revision-v1/',0,'revision','',0),(8414,11,'2015-06-22 15:57:43','2015-06-22 15:57:43','<h3>Introduction</h3>\r\nGPUs are really good at doing math. The Achilles heel is when it comes to 64-bit double precision math. GPUs, at least consumer grade, are not built for high performance FP64. This is because they are targeted towards gamers and game developers, who do not really care about high precision compute. So vendors like NVIDIA and AMD do not cram FP64 compute cores in their GPUs.\r\n\r\nFor example, on a GTX 780 Ti, the FP64 performance is 1/24 FP32. Which means in an ideal case, running the same code by only changing float types to double types, would yield the double precision run time to be about 1/24th of the single precision time.\r\n\r\nKeep in mind, the theoretical best case for FP64 performance is 1:2 FP32, simply because it involves computing with double the number of bits as FP32.\r\n\r\nHow double precision performs really depends on the architecture of the GPU.\r\n\r\n<h3>Comparing NVIDIA GPUs</h3>\r\nLets take 3 almost identical cards: <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-780-ti\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-titan-black\" target=\"_blank\">GTX Titan Black</a> and the <a href=\"http://www.nvidia.com/object/tesla-servers.html\" target=\"_blank\">Tesla K40c</a>. All are Kepler <a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">GK110</a> based GPUs, with the same number of SMX and cores (15 SMX, 2880 cores) and the same bus width (384-bit). The 780 Ti and Titan Black even have nearly same base clock speeds (~880MHz; K40c is 745MHz) and identical memory clock speeds (7GHz; K40 is 6GHz).\r\n\r\n<strong>Single Precision Performance</strong>\r\nWith respect to single precision performance, all 3 are fairly in the same ball park. Which the 780 Ti and Titan Black sit just around 5.1 TFlops, owing to their similar clock speeds, the Tesla K40c drops in at 4.3 TFlops. You could give the Tesla a pass because it has a lower clock speed.\r\n\r\nAll 3 only vary significantly in 3 categories.\r\n\r\n<strong>1. Memory</strong>\r\nThis doesn\'t affect performance very much (at least when the sizes fit in all GPUs). The price difference based on market prices of other GPUs with similar memory size variations should not be that big either.\r\n\r\n<strong>2. Price</strong>\r\nAt launch, the GTX 780 Ti was priced at $699, $999 for the Titan Black and an estimated $5500 for the K40c. Well, that escalated quickly.\r\n(Price Sources: <a href=\"http://www.anandtech.com/show/7492/the-geforce-gtx-780-ti-review\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.anandtech.com/show/7765/nvidias-geforce-gtx-titan-black-no-compromises-for-gaming-compute\" target=\"_blank\">GTX Titan Black</a>, <a href=\"http://www.anandtech.com/show/7521/nvidia-launches-tesla-k40\" target=\"_blank\">K40</a>)\r\n\r\n<strong>3. Double Precision Performance</strong>\r\nSo if the 780 Ti and the Titan Black are practically the same in every respect, why is there a $300 difference in their price at launch (discounting memory size difference)?\r\n\r\nThe answer is in the double precision capabilities. The 780 Ti is physically locked at 1:24 FP32 where has the Titan Black has an ace up it\'s sleeve.\r\n\r\nFor the Titan Black, the magic happens in the driver. The Titan black\'s driver gives the user an option to choose the double precision performance between 1:3 and 1:24 FP32 (by switching the GPU to TCC mode). When the double precision performance is set to 1:24 FP32, which is the same as the 780 Ti, the the single precision performance of the Titan Black and 780 Ti are identical. But when the user sets the double precision performance to 1:3 FP32, the single precision performance is compromised to boost double precision performance and make it equal to the K40c. In other words, you can choose the performance of the Titan Black to match either the 780 Ti or the K40c based and your preference.\r\n\r\nThe K40c has a double precision performance of 1:3 without compromising the single precision performance. This is because the K40 is given a special double precision unit for every 3 single precision cores (<a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">white paper</a>). It combines the best of both worlds. NVIDIA also states that the Tesla GPUs go through a much more rigorous Q&A process which guarantees lesser failures and also has additional features such as ECC memory. Hence the large price difference.\r\n\r\n<h4>Summary of NVIDIA GPUs</h4>\r\n<strong>NVIDIA\'s GTX series</strong> are known for their great FP32 performance but are very poor in their FP64 performance. The performance generally ranges between <strong>1:24 (Kepler) and 1:32 (Maxwell)</strong>. The exceptions to this are the GTX Titan cards which blur the lines between the consumer GTX series and the professional Tesla/Quadro cards.\r\n\r\nThe <strong>Kepler architecture Quadro and Tesla series</strong> card provide full double precision performance with <strong>1:3 FP32</strong>. However, with the <strong>Quadro M6000</strong>, NVIDIA has decided to provide only minimal FP64 performance by giving it only <strong>1:32 of FP32 capability</strong> and touting the M6000 as the best graphics card rather than the best graphics+compute card like the Quadro K6000.\r\n\r\n<h3>AMD GPUs</h3>\r\n\r\nAMD GPUs perform fairly well for FP64 compared to FP32. Most AMD cards (including consumer/gaming series) will give between <strong>1:3 and 1:8 FP32 performance for FP64</strong>. The AMD Tahiti architectures tested in these benchmarks here do not suffer from the same problems FP64 problems as NVIDIA\'s GTX series and give a 1:4 performance. Newer Hawaii architecture consumer grade GPUs are expected to provide 1:8 performance.\r\nThe <strong>FirePro W9100, W8100 and S9150 will give you an incredible FP64 1:2 FP32 performance</strong>. Overall, AMD GPUs hold a reputation for good double precision performance compared to their NVIDIA counterparts.\r\n\r\n<h3>Benchmarks</h3>\r\nHere are some benchmarks comparaing FP32 and FP64 GEMM.\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=381229711&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=722963661&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1083990145&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1438814030&amp;format=interactive\"></iframe>\r\n\r\n<h3>Conclusion</h3>\r\nThere is no perfect solution to the problem of choosing between FP32 and FP64 when it comes to GPUs. The best users can do is to decide whether they need the accuracy of double precision compute and how much compute they need, eventually going for a GPU that maximizes their productivity. Some applications like physics modelling and simulation, high accuracy financial computations etc which call for double precision accuracy at high performance require capable FP64 cards. Applications such as image and video processing, signal processing, statistics may not require such high precision and can get away with high FP32 performance only.\r\n\r\nDisclaimer: We receive GPUs from both NVIDIA and AMD. However, this blog does not promote or advocate any specific vendor.','Explaining FP64 performance on GPUs','','inherit','open','closed','','8389-revision-v1','','','2015-06-22 15:57:43','2015-06-22 15:57:43','',8389,'http://sandbox.arrayfire.com/8389-revision-v1/',0,'revision','',0),(8415,11,'2015-06-22 15:58:22','2015-06-22 15:58:22','<h3>Introduction</h3>\r\nGPUs are really good at doing math. The Achilles heel is when it comes to 64-bit double precision math. GPUs, at least consumer grade, are not built for high performance FP64. This is because they are targeted towards gamers and game developers, who do not really care about high precision compute. So vendors like NVIDIA and AMD do not cram FP64 compute cores in their GPUs.\r\n\r\nFor example, on a GTX 780 Ti, the FP64 performance is 1/24 FP32. Which means in an ideal case, running the same code by only changing float types to double types, would yield the double precision run time to be about 1/24th of the single precision time.\r\n\r\nKeep in mind, the theoretical best case for FP64 performance is 1:2 FP32, simply because it involves computing with double the number of bits as FP32.\r\n\r\nHow double precision performs really depends on the architecture of the GPU.\r\n\r\n<h3>Comparing NVIDIA GPUs</h3>\r\nLets take 3 almost identical cards: <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-780-ti\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-titan-black\" target=\"_blank\">GTX Titan Black</a> and the <a href=\"http://www.nvidia.com/object/tesla-servers.html\" target=\"_blank\">Tesla K40c</a>. All are Kepler <a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">GK110</a> based GPUs, with the same number of SMX and cores (15 SMX, 2880 cores) and the same bus width (384-bit). The 780 Ti and Titan Black even have nearly same base clock speeds (~880MHz; K40c is 745MHz) and identical memory clock speeds (7GHz; K40 is 6GHz).\r\n\r\n<strong>Single Precision Performance</strong>\r\nWith respect to single precision performance, all 3 are fairly in the same ball park. Which the 780 Ti and Titan Black sit just around 5.1 TFlops, owing to their similar clock speeds, the Tesla K40c drops in at 4.3 TFlops. You could give the Tesla a pass because it has a lower clock speed.\r\n\r\nAll 3 only vary significantly in 3 categories.\r\n\r\n<strong>1. Memory</strong>\r\nThis doesn\'t affect performance very much (at least when the sizes fit in all GPUs). The price difference based on market prices of other GPUs with similar memory size variations should not be that big either.\r\n\r\n<strong>2. Price</strong>\r\nAt launch, the GTX 780 Ti was priced at $699, $999 for the Titan Black and an estimated $5500 for the K40c. Well, that escalated quickly.\r\n(Price Sources: <a href=\"http://www.anandtech.com/show/7492/the-geforce-gtx-780-ti-review\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.anandtech.com/show/7765/nvidias-geforce-gtx-titan-black-no-compromises-for-gaming-compute\" target=\"_blank\">GTX Titan Black</a>, <a href=\"http://www.anandtech.com/show/7521/nvidia-launches-tesla-k40\" target=\"_blank\">K40</a>)\r\n\r\n<strong>3. Double Precision Performance</strong>\r\nSo if the 780 Ti and the Titan Black are practically the same in every respect, why is there a $300 difference in their price at launch (discounting memory size difference)?\r\n\r\nThe answer is in the double precision capabilities. The 780 Ti is physically locked at 1:24 FP32 where has the Titan Black has an ace up it\'s sleeve.\r\n\r\nFor the Titan Black, the magic happens in the driver. The Titan black\'s driver gives the user an option to choose the double precision performance between 1:3 and 1:24 FP32 (by switching the GPU to TCC mode). When the double precision performance is set to 1:24 FP32, which is the same as the 780 Ti, the the single precision performance of the Titan Black and 780 Ti are identical. But when the user sets the double precision performance to 1:3 FP32, the single precision performance is compromised to boost double precision performance and make it equal to the K40c. In other words, you can choose the performance of the Titan Black to match either the 780 Ti or the K40c based and your preference.\r\n\r\nThe K40c has a double precision performance of 1:3 without compromising the single precision performance. This is because the K40 is given a special double precision unit for every 3 single precision cores (<a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">white paper</a>). It combines the best of both worlds. NVIDIA also states that the Tesla GPUs go through a much more rigorous Q&A process which guarantees lesser failures and also has additional features such as ECC memory. Hence the large price difference.\r\n\r\n<h4>Summary of NVIDIA GPUs</h4>\r\n<strong>NVIDIA\'s GTX series</strong> are known for their great FP32 performance but are very poor in their FP64 performance. The performance generally ranges between <strong>1:24 (Kepler) and 1:32 (Maxwell)</strong>. The exceptions to this are the GTX Titan cards which blur the lines between the consumer GTX series and the professional Tesla/Quadro cards.\r\n\r\nThe <strong>Kepler architecture Quadro and Tesla series</strong> card provide full double precision performance with <strong>1:3 FP32</strong>. However, with the <strong>Quadro M6000</strong>, NVIDIA has decided to provide only minimal FP64 performance by giving it only <strong>1:32 of FP32 capability</strong> and touting the M6000 as the best graphics card rather than the best graphics+compute card like the Quadro K6000.\r\n\r\n<h3>AMD GPUs</h3>\r\n\r\nAMD GPUs perform fairly well for FP64 compared to FP32. Most AMD cards (including consumer/gaming series) will give between <strong>1:3 and 1:8 FP32 performance for FP64</strong>. The AMD Tahiti architectures tested in these benchmarks here do not suffer from the same problems FP64 problems as NVIDIA\'s GTX series and give a 1:4 performance. Newer Hawaii architecture consumer grade GPUs are expected to provide 1:8 performance.\r\n\r\nThe <strong>FirePro W9100, W8100 and S9150 will give you an incredible FP64 1:2 FP32 performance</strong>.\r\n\r\nOverall, AMD GPUs hold a reputation for good double precision performance compared to their NVIDIA counterparts.\r\n\r\n<h3>Benchmarks</h3>\r\nHere are some benchmarks comparaing FP32 and FP64 GEMM.\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=381229711&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=722963661&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1083990145&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1438814030&amp;format=interactive\"></iframe>\r\n\r\n<h3>Conclusion</h3>\r\nThere is no perfect solution to the problem of choosing between FP32 and FP64 when it comes to GPUs. The best users can do is to decide whether they need the accuracy of double precision compute and how much compute they need, eventually going for a GPU that maximizes their productivity. Some applications like physics modelling and simulation, high accuracy financial computations etc which call for double precision accuracy at high performance require capable FP64 cards. Applications such as image and video processing, signal processing, statistics may not require such high precision and can get away with high FP32 performance only.\r\n\r\nDisclaimer: We receive GPUs from both NVIDIA and AMD. However, this blog does not promote or advocate any specific vendor.','Explaining FP64 performance on GPUs','','inherit','open','closed','','8389-revision-v1','','','2015-06-22 15:58:22','2015-06-22 15:58:22','',8389,'http://sandbox.arrayfire.com/8389-revision-v1/',0,'revision','',0),(8416,11,'2015-06-22 15:58:48','2015-06-22 15:58:48','<h3>Introduction</h3>\r\nGPUs are really good at doing math. The Achilles heel is when it comes to 64-bit double precision math. GPUs, at least consumer grade, are not built for high performance FP64. This is because they are targeted towards gamers and game developers, who do not really care about high precision compute. So vendors like NVIDIA and AMD do not cram FP64 compute cores in their GPUs.\r\n\r\nFor example, on a GTX 780 Ti, the FP64 performance is 1/24 FP32. Which means in an ideal case, running the same code by only changing float types to double types, would yield the double precision run time to be about 1/24th of the single precision time.\r\n\r\nKeep in mind, the theoretical best case for FP64 performance is 1:2 FP32, simply because it involves computing with double the number of bits as FP32.\r\n\r\nHow double precision performs really depends on the architecture of the GPU.\r\n\r\n<h3>Comparing NVIDIA GPUs</h3>\r\nLets take 3 almost identical cards: <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-780-ti\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-titan-black\" target=\"_blank\">GTX Titan Black</a> and the <a href=\"http://www.nvidia.com/object/tesla-servers.html\" target=\"_blank\">Tesla K40c</a>. All are Kepler <a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">GK110</a> based GPUs, with the same number of SMX and cores (15 SMX, 2880 cores) and the same bus width (384-bit). The 780 Ti and Titan Black even have nearly same base clock speeds (~880MHz; K40c is 745MHz) and identical memory clock speeds (7GHz; K40 is 6GHz).\r\n\r\n<strong>Single Precision Performance</strong>\r\nWith respect to single precision performance, all 3 are fairly in the same ball park. Which the 780 Ti and Titan Black sit just around 5.1 TFlops, owing to their similar clock speeds, the Tesla K40c drops in at 4.3 TFlops. You could give the Tesla a pass because it has a lower clock speed.\r\n\r\nAll 3 only vary significantly in 3 categories.\r\n\r\n<strong>1. Memory</strong>\r\nThis doesn\'t affect performance very much (at least when the sizes fit in all GPUs). The price difference based on market prices of other GPUs with similar memory size variations should not be that big either.\r\n\r\n<strong>2. Price</strong>\r\nAt launch, the GTX 780 Ti was priced at $699, $999 for the Titan Black and an estimated $5500 for the K40c. Well, that escalated quickly.\r\n(Price Sources: <a href=\"http://www.anandtech.com/show/7492/the-geforce-gtx-780-ti-review\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.anandtech.com/show/7765/nvidias-geforce-gtx-titan-black-no-compromises-for-gaming-compute\" target=\"_blank\">GTX Titan Black</a>, <a href=\"http://www.anandtech.com/show/7521/nvidia-launches-tesla-k40\" target=\"_blank\">K40</a>)\r\n\r\n<strong>3. Double Precision Performance</strong>\r\nSo if the 780 Ti and the Titan Black are practically the same in every respect, why is there a $300 difference in their price at launch (discounting memory size difference)?\r\n\r\nThe answer is in the double precision capabilities. The 780 Ti is physically locked at 1:24 FP32 where has the Titan Black has an ace up it\'s sleeve.\r\n\r\nFor the Titan Black, the magic happens in the driver. The Titan black\'s driver gives the user an option to choose the double precision performance between 1:3 and 1:24 FP32 (by switching the GPU to TCC mode). When the double precision performance is set to 1:24 FP32, which is the same as the 780 Ti, the the single precision performance of the Titan Black and 780 Ti are identical. But when the user sets the double precision performance to 1:3 FP32, the single precision performance is compromised to boost double precision performance and make it equal to the K40c. In other words, you can choose the performance of the Titan Black to match either the 780 Ti or the K40c based and your preference.\r\n\r\nThe K40c has a double precision performance of 1:3 without compromising the single precision performance. This is because the K40 is given a special double precision unit for every 3 single precision cores (<a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">white paper</a>). It combines the best of both worlds. NVIDIA also states that the Tesla GPUs go through a much more rigorous Q&A process which guarantees lesser failures and also has additional features such as ECC memory. Hence the large price difference.\r\n\r\n<h4>Summary of NVIDIA GPUs</h4>\r\n<strong>NVIDIA\'s GTX series</strong> are known for their great FP32 performance but are very poor in their FP64 performance. The performance generally ranges between <strong>1:24 (Kepler) and 1:32 (Maxwell)</strong>. The exceptions to this are the GTX Titan cards which blur the lines between the consumer GTX series and the professional Tesla/Quadro cards.\r\n\r\nThe <strong>Kepler architecture Quadro and Tesla series</strong> card provide full double precision performance with <strong>1:3 FP32</strong>. However, with the <strong>Quadro M6000</strong>, NVIDIA has decided to provide only minimal FP64 performance by giving it only <strong>1:32 of FP32 capability</strong> and touting the M6000 as the best graphics card rather than the best graphics+compute card like the Quadro K6000.\r\n\r\n<h3>AMD GPUs</h3>\r\n\r\nAMD GPUs perform fairly well for FP64 compared to FP32. Most AMD cards (including consumer/gaming series) will give between <strong>1:3 and 1:8 FP32 performance for FP64</strong>. The AMD Tahiti architectures tested in these benchmarks here do not suffer from the same problems FP64 problems as NVIDIA\'s GTX series and give a 1:4 performance. Newer Hawaii architecture consumer grade GPUs are expected to provide 1:8 performance.\r\n\r\nThe <strong>FirePro W9100, W8100 and S9150 will give you an incredible FP64 1:2 FP32 performance</strong>.\r\n\r\nOverall, AMD GPUs hold a reputation for good double precision performance ratios compared to their NVIDIA counterparts.\r\n\r\n<h3>Benchmarks</h3>\r\nHere are some benchmarks comparaing FP32 and FP64 GEMM.\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=381229711&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=722963661&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1083990145&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1438814030&amp;format=interactive\"></iframe>\r\n\r\n<h3>Conclusion</h3>\r\nThere is no perfect solution to the problem of choosing between FP32 and FP64 when it comes to GPUs. The best users can do is to decide whether they need the accuracy of double precision compute and how much compute they need, eventually going for a GPU that maximizes their productivity. Some applications like physics modelling and simulation, high accuracy financial computations etc which call for double precision accuracy at high performance require capable FP64 cards. Applications such as image and video processing, signal processing, statistics may not require such high precision and can get away with high FP32 performance only.\r\n\r\nDisclaimer: We receive GPUs from both NVIDIA and AMD. However, this blog does not promote or advocate any specific vendor.','Explaining FP64 performance on GPUs','','inherit','open','closed','','8389-revision-v1','','','2015-06-22 15:58:48','2015-06-22 15:58:48','',8389,'http://sandbox.arrayfire.com/8389-revision-v1/',0,'revision','',0),(8417,11,'2015-06-22 16:09:17','2015-06-22 16:09:17','<h3>Introduction</h3>\r\nGPUs are really good at doing math. The Achilles heel is when it comes to 64-bit double precision math. GPUs, at least consumer grade, are not built for high performance FP64. This is because they are targeted towards gamers and game developers, who do not really care about high precision compute. So vendors like NVIDIA and AMD do not cram FP64 compute cores in their GPUs.\r\n\r\nFor example, on a GTX 780 Ti, the FP64 performance is 1/24 FP32. Which means in an ideal case, running the same code by only changing float types to double types, would yield the double precision run time to be about 1/24th of the single precision time.\r\n\r\nKeep in mind, the theoretical best case for FP64 performance is 1:2 FP32, simply because it involves computing with double the number of bits as FP32.\r\n\r\nHow double precision performs really depends on the architecture of the GPU.\r\n\r\n<h3>Comparing NVIDIA GPUs</h3>\r\nLets take three almost identical cards: <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-780-ti\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-titan-black\" target=\"_blank\">GTX Titan Black</a> and the <a href=\"http://www.nvidia.com/object/tesla-servers.html\" target=\"_blank\">Tesla K40c</a>. All are Kepler <a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">GK110</a> based GPUs, with the same number of SMX and cores (15 SMX, 2880 cores) and the same bus width (384-bit). The 780 Ti and Titan Black even have nearly same base clock speeds (~880MHz; K40c is 745MHz) and identical memory clock speeds (7GHz; K40 is 6GHz).\r\n\r\n<strong>Single Precision Performance</strong>\r\nWith respect to single precision performance, all three are fairly in the same ball park. Which the 780 Ti and Titan Black sit just around 5.1 TFlops, owing to their similar clock speeds, the Tesla K40c drops in at 4.3 TFlops. You could give the Tesla a pass because it has a lower clock speed.\r\n\r\nAll three only vary significantly in three categories.\r\n\r\n<strong>1. Memory</strong>\r\nThis doesn\'t affect performance very much (at least when the sizes fit in all GPUs). The price difference based on market prices of other GPUs with similar memory size variations should not be that big either.\r\n\r\n<strong>2. Price</strong>\r\nAt launch, the GTX 780 Ti was priced at $699, $999 for the Titan Black and an estimated $5500 for the K40c. Well, that escalated quickly.\r\n(Price Sources: <a href=\"http://www.anandtech.com/show/7492/the-geforce-gtx-780-ti-review\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.anandtech.com/show/7765/nvidias-geforce-gtx-titan-black-no-compromises-for-gaming-compute\" target=\"_blank\">GTX Titan Black</a>, <a href=\"http://www.anandtech.com/show/7521/nvidia-launches-tesla-k40\" target=\"_blank\">K40</a>)\r\n\r\n<strong>3. Double Precision Performance</strong>\r\nSo if the 780 Ti and the Titan Black are practically the same in every respect, why is there a $300 difference in their price at launch (discounting memory size difference)?\r\n\r\nThe answer is in the double precision capabilities. The 780 Ti is physically locked at 1:24 FP32 where has the Titan Black has an ace up it\'s sleeve.\r\n\r\nFor the Titan Black, the magic happens in the driver. The Titan black\'s driver gives the user an option to choose the double precision performance between 1:3 and 1:24 FP32 (by switching the GPU to TCC mode). When the double precision performance is set to 1:24 FP32, which is the same as the 780 Ti, the the single precision performance of the Titan Black and 780 Ti are identical. But when the user sets the double precision performance to 1:3 FP32, the single precision performance is compromised to boost double precision performance and make it equal to the K40c. In other words, you can choose the performance of the Titan Black to match either the 780 Ti or the K40c based and your preference.\r\n\r\nThe K40c has a double precision performance of 1:3 without compromising the single precision performance. This is because the K40 is given a special double precision unit for every 3 single precision cores (<a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">white paper</a>). It combines the best of both worlds. NVIDIA also states that the Tesla GPUs go through a much more rigorous Q&A process which guarantees lesser failures and also has additional features such as ECC memory. Hence the large price difference.\r\n\r\n<h4>Summary of NVIDIA GPUs</h4>\r\n<strong>NVIDIA\'s GTX series</strong> are known for their great FP32 performance but are very poor in their FP64 performance. The performance generally ranges between <strong>1:24 (Kepler) and 1:32 (Maxwell)</strong>. The exceptions to this are the GTX Titan cards which blur the lines between the consumer GTX series and the professional Tesla/Quadro cards.\r\n\r\nThe <strong>Kepler architecture Quadro and Tesla series</strong> card provide full double precision performance with <strong>1:3 FP32</strong>. However, with the <strong>Quadro M6000</strong>, NVIDIA has decided to provide only minimal FP64 performance by giving it only <strong>1:32 of FP32 capability</strong> and touting the M6000 as the best graphics card rather than the best graphics+compute card like the Quadro K6000.\r\n\r\n<h3>AMD GPUs</h3>\r\n\r\nAMD GPUs perform fairly well for FP64 compared to FP32. Most AMD cards (including consumer/gaming series) will give between <strong>1:3 and 1:8 FP32 performance for FP64</strong>. The AMD Tahiti architectures tested in these benchmarks here do not suffer from the same problems FP64 problems as NVIDIA\'s GTX series and give a 1:4 performance. Newer Hawaii architecture consumer grade GPUs are expected to provide 1:8 performance.\r\n\r\nThe <strong>FirePro W9100, W8100 and S9150 will give you an incredible FP64 1:2 FP32 performance</strong>.\r\n\r\nOverall, AMD GPUs hold a reputation for good double precision performance ratios compared to their NVIDIA counterparts.\r\n\r\n<h3>Benchmarks</h3>\r\nHere are some benchmarks comparaing FP32 and FP64 GEMM.\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=381229711&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=722963661&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1083990145&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1438814030&amp;format=interactive\"></iframe>\r\n\r\n<h3>Conclusion</h3>\r\nThere is no perfect solution to the problem of choosing between FP32 and FP64 when it comes to GPUs. The best users can do is to decide whether they need the accuracy of double precision compute and how much compute they need, eventually going for a GPU that maximizes their productivity. Some applications like physics modelling and simulation, high accuracy financial computations etc which call for double precision accuracy at high performance require capable FP64 cards. Applications such as image and video processing, signal processing, statistics may not require such high precision and can get away with high FP32 performance only.\r\n\r\nDisclaimer: We receive GPUs from both NVIDIA and AMD. However, this blog does not promote or advocate any specific vendor.\r\n\r\nSources: Wikipedia, NVIDIA website.','Explaining FP64 performance on GPUs','','inherit','open','closed','','8389-revision-v1','','','2015-06-22 16:09:17','2015-06-22 16:09:17','',8389,'http://sandbox.arrayfire.com/8389-revision-v1/',0,'revision','',0),(8418,11,'2015-06-22 16:09:28','2015-06-22 16:09:28','<h3>Introduction</h3>\r\nGPUs are really good at doing math. The Achilles heel is when it comes to 64-bit double precision math. GPUs, at least consumer grade, are not built for high performance FP64. This is because they are targeted towards gamers and game developers, who do not really care about high precision compute. So vendors like NVIDIA and AMD do not cram FP64 compute cores in their GPUs.\r\n\r\nFor example, on a GTX 780 Ti, the FP64 performance is 1/24 FP32. Which means in an ideal case, running the same code by only changing float types to double types, would yield the double precision run time to be about 1/24th of the single precision time.\r\n\r\nKeep in mind, the theoretical best case for FP64 performance is 1:2 FP32, simply because it involves computing with double the number of bits as FP32.\r\n\r\nHow double precision performs really depends on the architecture of the GPU.\r\n\r\n<h3>Comparing NVIDIA GPUs</h3>\r\nLets take three almost identical cards: <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-780-ti\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-titan-black\" target=\"_blank\">GTX Titan Black</a> and the <a href=\"http://www.nvidia.com/object/tesla-servers.html\" target=\"_blank\">Tesla K40c</a>. All are Kepler <a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">GK110</a> based GPUs, with the same number of SMX and cores (15 SMX, 2880 cores) and the same bus width (384-bit). The 780 Ti and Titan Black even have nearly same base clock speeds (~880MHz; K40c is 745MHz) and identical memory clock speeds (7GHz; K40 is 6GHz).\r\n\r\n<strong>Single Precision Performance</strong>\r\nWith respect to single precision performance, all three are fairly in the same ball park. Which the 780 Ti and Titan Black sit just around 5.1 TFlops, owing to their similar clock speeds, the Tesla K40c drops in at 4.3 TFlops. You could give the Tesla a pass because it has a lower clock speed.\r\n\r\nAll three only vary significantly in three categories.\r\n\r\n<strong>1. Memory</strong>\r\nThis doesn\'t affect performance very much (at least when the sizes fit in all GPUs). The price difference based on market prices of other GPUs with similar memory size variations should not be that big either.\r\n\r\n<strong>2. Price</strong>\r\nAt launch, the GTX 780 Ti was priced at $699, $999 for the Titan Black and an estimated $5500 for the K40c. Well, that escalated quickly.\r\n(Price Sources: <a href=\"http://www.anandtech.com/show/7492/the-geforce-gtx-780-ti-review\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.anandtech.com/show/7765/nvidias-geforce-gtx-titan-black-no-compromises-for-gaming-compute\" target=\"_blank\">GTX Titan Black</a>, <a href=\"http://www.anandtech.com/show/7521/nvidia-launches-tesla-k40\" target=\"_blank\">K40</a>)\r\n\r\n<strong>3. Double Precision Performance</strong>\r\nSo if the 780 Ti and the Titan Black are practically the same in every respect, why is there a $300 difference in their price at launch (discounting memory size difference)?\r\n\r\nThe answer is in the double precision capabilities. The 780 Ti is physically locked at 1:24 FP32 where has the Titan Black has an ace up it\'s sleeve.\r\n\r\nFor the Titan Black, the magic happens in the driver. The Titan black\'s driver gives the user an option to choose the double precision performance between 1:3 and 1:24 FP32 (by switching the GPU to TCC mode). When the double precision performance is set to 1:24 FP32, which is the same as the 780 Ti, the the single precision performance of the Titan Black and 780 Ti are identical. But when the user sets the double precision performance to 1:3 FP32, the single precision performance is compromised to boost double precision performance and make it equal to the K40c. In other words, you can choose the performance of the Titan Black to match either the 780 Ti or the K40c based and your preference.\r\n\r\nThe K40c has a double precision performance of 1:3 without compromising the single precision performance. This is because the K40 is given a special double precision unit for every 3 single precision cores (<a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">white paper</a>). It combines the best of both worlds. NVIDIA also states that the Tesla GPUs go through a much more rigorous Q&A process which guarantees lesser failures and also has additional features such as ECC memory. Hence the large price difference.\r\n\r\n<h4>Summary of NVIDIA GPUs</h4>\r\n<strong>NVIDIA\'s GTX series</strong> are known for their great FP32 performance but are very poor in their FP64 performance. The performance generally ranges between <strong>1:24 (Kepler) and 1:32 (Maxwell)</strong>. The exceptions to this are the GTX Titan cards which blur the lines between the consumer GTX series and the professional Tesla/Quadro cards.\r\n\r\nThe <strong>Kepler architecture Quadro and Tesla series</strong> card provide full double precision performance with <strong>1:3 FP32</strong>. However, with the <strong>Quadro M6000</strong>, NVIDIA has decided to provide only minimal FP64 performance by giving it only <strong>1:32 of FP32 capability</strong> and touting the M6000 as the best graphics card rather than the best graphics+compute card like the Quadro K6000.\r\n\r\n<h3>AMD GPUs</h3>\r\n\r\nAMD GPUs perform fairly well for FP64 compared to FP32. Most AMD cards (including consumer/gaming series) will give between <strong>1:3 and 1:8 FP32 performance for FP64</strong>. The AMD Tahiti architectures tested in these benchmarks here do not suffer from the same problems FP64 problems as NVIDIA\'s GTX series and give a 1:4 performance. Newer Hawaii architecture consumer grade GPUs are expected to provide 1:8 performance.\r\n\r\nThe <strong>FirePro W9100, W8100 and S9150 will give you an incredible FP64 1:2 FP32 performance</strong>.\r\n\r\nOverall, AMD GPUs hold a reputation for good double precision performance ratios compared to their NVIDIA counterparts.\r\n\r\n<h3>Benchmarks</h3>\r\nHere are some benchmarks comparaing FP32 and FP64 GEMM.\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=381229711&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=722963661&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1083990145&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1438814030&amp;format=interactive\"></iframe>\r\n\r\n<h3>Conclusion</h3>\r\nThere is no perfect solution to the problem of choosing between FP32 and FP64 when it comes to GPUs. The best users can do is to decide whether they need the accuracy of double precision compute and how much compute they need, eventually going for a GPU that maximizes their productivity. Some applications like physics modelling and simulation, high accuracy financial computations etc which call for double precision accuracy at high performance require capable FP64 cards. Applications such as image and video processing, signal processing, statistics may not require such high precision and can get away with high FP32 performance only.\r\n\r\nDisclaimer: We receive GPUs from both NVIDIA and AMD. However, this blog does not promote or advocate any specific vendor.\r\n\r\nSources: Wikipedia, NVIDIA website, Anandtech.','Explaining FP64 performance on GPUs','','inherit','open','closed','','8389-revision-v1','','','2015-06-22 16:09:28','2015-06-22 16:09:28','',8389,'http://sandbox.arrayfire.com/8389-revision-v1/',0,'revision','',0),(8419,11,'2015-06-22 18:11:49','2015-06-22 18:11:49','<h3>Introduction</h3>\r\nGPUs are really good at doing math. The Achilles heel is when it comes to 64-bit double precision math. GPUs, at least consumer grade, are not built for high performance FP64. This is because they are targeted towards gamers and game developers, who do not really care about high precision compute. So vendors like NVIDIA and AMD do not cram FP64 compute cores in their GPUs.\r\n\r\nFor example, on a GTX 780 Ti, the FP64 performance is 1/24 FP32. Which means in an ideal case, running the same code by only changing float types to double types, would yield the double precision run time to be about 1/24th of the single precision time.\r\n\r\nKeep in mind, for compute-bound algorithms, such as GEMM and FFT, the theoretical best case for FP64 performance is 1:2 FP32, simply because it involves computing with double the number of bits as FP32. If the algorithms are memory bound, such as matrix transpose, then most GPUs will attain the 1:2 performance. The numbers we discuss below will all be compute-bound performance numbers.\r\n\r\nHow double precision performs really depends on the architecture of the GPU.\r\n\r\n<h3>Comparing NVIDIA GPUs</h3>\r\nLets take three almost identical cards: <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-780-ti\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-titan-black\" target=\"_blank\">GTX Titan Black</a> and the <a href=\"http://www.nvidia.com/object/tesla-servers.html\" target=\"_blank\">Tesla K40c</a>. All are Kepler <a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">GK110</a> based GPUs, with the same number of SMX and cores (15 SMX, 2880 cores) and the same bus width (384-bit). The 780 Ti and Titan Black even have nearly same base clock speeds (~880MHz; K40c is 745MHz) and identical memory clock speeds (7GHz; K40 is 6GHz).\r\n\r\n<strong>Single Precision Performance</strong>\r\nWith respect to single precision performance, all three are fairly in the same ball park. Which the 780 Ti and Titan Black sit just around 5.1 TFlops, owing to their similar clock speeds, the Tesla K40c drops in at 4.3 TFlops. You could give the Tesla a pass because it has a lower clock speed.\r\n\r\nAll three only vary significantly in three categories.\r\n\r\n<strong>1. Memory</strong>\r\nThis doesn\'t affect performance very much (at least when the sizes fit in all GPUs). The price difference based on market prices of other GPUs with similar memory size variations should not be that big either.\r\n\r\n<strong>2. Price</strong>\r\nAt launch, the GTX 780 Ti was priced at $699, $999 for the Titan Black and an estimated $5500 for the K40c. Well, that escalated quickly.\r\n(Price Sources: <a href=\"http://www.anandtech.com/show/7492/the-geforce-gtx-780-ti-review\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.anandtech.com/show/7765/nvidias-geforce-gtx-titan-black-no-compromises-for-gaming-compute\" target=\"_blank\">GTX Titan Black</a>, <a href=\"http://www.anandtech.com/show/7521/nvidia-launches-tesla-k40\" target=\"_blank\">K40</a>)\r\n\r\n<strong>3. Double Precision Performance</strong>\r\nSo if the 780 Ti and the Titan Black are practically the same in every respect, why is there a $300 difference in their price at launch (discounting memory size difference)?\r\n\r\nThe answer is in the double precision capabilities. The 780 Ti is physically locked at 1:24 FP32 where has the Titan Black has an ace up it\'s sleeve.\r\n\r\nFor the Titan Black, the magic happens in the driver. The Titan black\'s driver gives the user an option to choose the double precision performance between 1:3 and 1:24 FP32 (by switching the GPU to TCC mode). When the double precision performance is set to 1:24 FP32, which is the same as the 780 Ti, the the single precision performance of the Titan Black and 780 Ti are identical. But when the user sets the double precision performance to 1:3 FP32, the single precision performance is compromised to boost double precision performance and make it equal to the K40c. In other words, you can choose the performance of the Titan Black to match either the 780 Ti or the K40c based and your preference.\r\n\r\nThe K40c has a double precision performance of 1:3 without compromising the single precision performance. This is because the K40 is given a special double precision unit for every 3 single precision cores (<a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">white paper</a>). It combines the best of both worlds. NVIDIA also states that the Tesla GPUs go through a much more rigorous Q&A process which guarantees lesser failures and also has additional features such as ECC memory. Hence the large price difference.\r\n\r\n<h4>Summary of NVIDIA GPUs</h4>\r\n<strong>NVIDIA\'s GTX series</strong> are known for their great FP32 performance but are very poor in their FP64 performance. The performance generally ranges between <strong>1:24 (Kepler) and 1:32 (Maxwell)</strong>. The exceptions to this are the GTX Titan cards which blur the lines between the consumer GTX series and the professional Tesla/Quadro cards.\r\n\r\nThe <strong>Kepler architecture Quadro and Tesla series</strong> card provide full double precision performance with <strong>1:3 FP32</strong>. However, with the <strong>Quadro M6000</strong>, NVIDIA has decided to provide only minimal FP64 performance by giving it only <strong>1:32 of FP32 capability</strong> and touting the M6000 as the best graphics card rather than the best graphics+compute card like the Quadro K6000.\r\n\r\n<h3>AMD GPUs</h3>\r\n\r\nAMD GPUs perform fairly well for FP64 compared to FP32. Most AMD cards (including consumer/gaming series) will give between <strong>1:3 and 1:8 FP32 performance for FP64</strong>. The AMD Tahiti architectures tested in these benchmarks here do not suffer from the same problems FP64 problems as NVIDIA\'s GTX series and give a 1:4 performance. Newer Hawaii architecture consumer grade GPUs are expected to provide 1:8 performance.\r\n\r\nThe <strong>FirePro W9100, W8100 and S9150 will give you an incredible FP64 1:2 FP32 performance</strong>.\r\n\r\nOverall, AMD GPUs hold a reputation for good double precision performance ratios compared to their NVIDIA counterparts.\r\n\r\n<h3>Benchmarks</h3>\r\nHere are some benchmarks comparaing FP32 and FP64 GEMM.\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=381229711&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=722963661&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1083990145&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1438814030&amp;format=interactive\"></iframe>\r\n\r\n<h3>Conclusion</h3>\r\nThere is no perfect solution to the problem of choosing between FP32 and FP64 when it comes to GPUs. The best users can do is to decide whether they need the accuracy of double precision compute and how much compute they need, eventually going for a GPU that maximizes their productivity. Some applications like physics modelling and simulation, high accuracy financial computations etc which call for double precision accuracy at high performance require capable FP64 cards. Applications such as image and video processing, signal processing, statistics may not require such high precision and can get away with high FP32 performance only.\r\n\r\nDisclaimer: We receive GPUs from both NVIDIA and AMD. However, this blog does not promote or advocate any specific vendor.\r\n\r\nSources: Wikipedia, NVIDIA website, Anandtech.','Explaining FP64 performance on GPUs','','inherit','open','closed','','8389-revision-v1','','','2015-06-22 18:11:49','2015-06-22 18:11:49','',8389,'http://sandbox.arrayfire.com/8389-revision-v1/',0,'revision','',0),(8421,11,'2015-06-24 19:01:14','2015-06-24 19:01:14','<h3>Introduction</h3>\nGPUs are really good at doing math. The Achilles heel is when it comes to 64-bit double precision math. GPUs, at least consumer grade, are not built for high performance FP64. This is because they are targeted towards gamers and game developers, who do not really care about high precision compute. So vendors like NVIDIA and AMD do not cram FP64 compute cores in their GPUs.\n\nFor example, on a GTX 780 Ti, the FP64 performance is 1/24 FP32. Which means in an ideal case, running the same code by only changing float types to double types, would yield the single precision run time to be about 1/24th of the double precision time (time(F32) = time(FP64)/24.\n\nKeep in mind, for compute-bound algorithms, such as GEMM and FFT, the theoretical best case for FP64 performance is 1:2 FP32, simply because it involves computing with double the number of bits as FP32. If the algorithms are memory bound, such as matrix transpose, then most GPUs will attain the 1:2 performance. The numbers we discuss below will all be compute-bound performance numbers.\n\nHow double precision performs really depends on the architecture of the GPU.\n\n<h3>Comparing NVIDIA GPUs</h3>\nLets take three almost identical cards: <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-780-ti\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-titan-black\" target=\"_blank\">GTX Titan Black</a> and the <a href=\"http://www.nvidia.com/object/tesla-servers.html\" target=\"_blank\">Tesla K40c</a>. All are Kepler <a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">GK110</a> based GPUs, with the same number of SMX and cores (15 SMX, 2880 cores) and the same bus width (384-bit). The 780 Ti and Titan Black even have nearly same base clock speeds (~880MHz; K40c is 745MHz) and identical memory clock speeds (7GHz; K40 is 6GHz).\n\n<strong>Single Precision Performance</strong>\nWith respect to single precision performance, all three are fairly in the same ball park. Which the 780 Ti and Titan Black sit just around 5.1 TFlops, owing to their similar clock speeds, the Tesla K40c drops in at 4.3 TFlops. You could give the Tesla a pass because it has a lower clock speed.\n\nAll three only vary significantly in three categories.\n\n<strong>1. Memory</strong>\nThis doesn\'t affect performance very much (at least when the sizes fit in all GPUs). The price difference based on market prices of other GPUs with similar memory size variations should not be that big either.\n\n<strong>2. Price</strong>\nAt launch, the GTX 780 Ti was priced at $699, $999 for the Titan Black and an estimated $5500 for the K40c. Well, that escalated quickly.\n(Price Sources: <a href=\"http://www.anandtech.com/show/7492/the-geforce-gtx-780-ti-review\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.anandtech.com/show/7765/nvidias-geforce-gtx-titan-black-no-compromises-for-gaming-compute\" target=\"_blank\">GTX Titan Black</a>, <a href=\"http://www.anandtech.com/show/7521/nvidia-launches-tesla-k40\" target=\"_blank\">K40</a>)\n\n<strong>3. Double Precision Performance</strong>\nSo if the 780 Ti and the Titan Black are practically the same in every respect, why is there a $300 difference in their price at launch (discounting memory size difference)?\n\nThe answer is in the double precision capabilities. The 780 Ti is physically locked at 1:24 FP32 where has the Titan Black has an ace up it\'s sleeve.\n\nFor the Titan Black, the magic happens in the driver. The Titan black\'s driver gives the user an option to choose the double precision performance between 1:3 and 1:24 FP32 (by switching the GPU to TCC mode). When the double precision performance is set to 1:24 FP32, which is the same as the 780 Ti, the the single precision performance of the Titan Black and 780 Ti are identical. But when the user sets the double precision performance to 1:3 FP32, the single precision performance is compromised to boost double precision performance and make it equal to the K40c. In other words, you can choose the performance of the Titan Black to match either the 780 Ti or the K40c based and your preference.\n\nThe K40c has a double precision performance of 1:3 without compromising the single precision performance. This is because the K40 is given a special double precision unit for every 3 single precision cores (<a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">white paper</a>). It combines the best of both worlds. NVIDIA also states that the Tesla GPUs go through a much more rigorous Q&A process which guarantees lesser failures and also has additional features such as ECC memory. Hence the large price difference.\n\n<h4>Summary of NVIDIA GPUs</h4>\n<strong>NVIDIA\'s GTX series</strong> are known for their great FP32 performance but are very poor in their FP64 performance. The performance generally ranges between <strong>1:24 (Kepler) and 1:32 (Maxwell)</strong>. The exceptions to this are the GTX Titan cards which blur the lines between the consumer GTX series and the professional Tesla/Quadro cards.\n\nThe <strong>Kepler architecture Quadro and Tesla series</strong> card provide full double precision performance with <strong>1:3 FP32</strong>. However, with the <strong>Quadro M6000</strong>, NVIDIA has decided to provide only minimal FP64 performance by giving it only <strong>1:32 of FP32 capability</strong> and touting the M6000 as the best graphics card rather than the best graphics+compute card like the Quadro K6000.\n\n<h3>AMD GPUs</h3>\n\nAMD GPUs perform fairly well for FP64 compared to FP32. Most AMD cards (including consumer/gaming series) will give between <strong>1:3 and 1:8 FP32 performance for FP64</strong>. The AMD Tahiti architectures tested in these benchmarks here do not suffer from the same problems FP64 problems as NVIDIA\'s GTX series and give a 1:4 performance. Newer Hawaii architecture consumer grade GPUs are expected to provide 1:8 performance.\n\nThe <strong>FirePro W9100, W8100 and S9150 will give you an incredible FP64 1:2 FP32 performance</strong>.\n\nOverall, AMD GPUs hold a reputation for good double precision performance ratios compared to their NVIDIA counterparts.\n\n<h3>Benchmarks</h3>\nHere are some benchmarks comparaing FP32 and FP64 GEMM.\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=381229711&amp;format=interactive\"></iframe>\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=722963661&amp;format=interactive\"></iframe>\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1083990145&amp;format=interactive\"></iframe>\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1438814030&amp;format=interactive\"></iframe>\n\n<h3>Conclusion</h3>\nThere is no perfect solution to the problem of choosing between FP32 and FP64 when it comes to GPUs. The best users can do is to decide whether they need the accuracy of double precision compute and how much compute they need, eventually going for a GPU that maximizes their productivity. Some applications like physics modelling and simulation, high accuracy financial computations etc which call for double precision accuracy at high performance require capable FP64 cards. Applications such as image and video processing, signal processing, statistics may not require such high precision and can get away with high FP32 performance only.\n\nDisclaimer: We receive GPUs from both NVIDIA and AMD. However, this blog does not promote or advocate any specific vendor.\n\nSources:\nWikipedia: <a href=\"https://en.wikipedia.org/wiki/GeForce_700_series\" target=\"_blank\">NVIDIA GeForce 700 Series</a>, <a href=\"https://en.wikipedia.org/wiki/Nvidia_Tesla\" target=\"_blank\">NVIDIA Tesla Series</a>, <a href=\"https://en.wikipedia.org/wiki/Nvidia_Quadro\" target=\"_blank\">NVIDIA Quadro Series</a>, <a href=\"https://en.wikipedia.org/wiki/List_of_AMD_graphics_processing_units#Comparison_table:_Workstation_GPUs\" target=\"_blank\">AMD FirePro Series</a>, <a href=\"https://en.wikipedia.org/wiki/List_of_AMD_graphics_processing_units\" target=\"_blank\">AMD GPUs</a>\nNVIDIA Website: <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-780-ti\" target=\"_blank\">NVIDIA GTX 780 Ti</a>, <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-titan-black\" target=\"_blank\">NVIDIA GTX Titan Black</a>, <a href=\"http://www.nvidia.com/object/tesla-servers.html\" target=\"_blank\">NVIDIA Tesla K40</a>\nAnandtech: <a href=\"http://www.anandtech.com/show/7492/the-geforce-gtx-780-ti-review\" target=\"_blank\">NVIDIA GTX 780 Ti</a>, <a href=\"http://www.anandtech.com/show/7765/nvidias-geforce-gtx-titan-black-no-compromises-for-gaming-compute\" target=\"_blank\">NVIDIA GTX Titan Black</a>, <a href=\"http://www.anandtech.com/show/7521/nvidia-launches-tesla-k40\" target=\"_blank\">NVIDIA Tesla K40</a>','Explaining FP64 performance on GPUs','','inherit','open','closed','','8389-autosave-v1','','','2015-06-24 19:01:14','2015-06-24 19:01:14','',8389,'http://sandbox.arrayfire.com/8389-autosave-v1/',0,'revision','',0),(8422,11,'2015-06-22 19:09:28','2015-06-22 19:09:28','<h3>Introduction</h3>\r\nGPUs are really good at doing math. The Achilles heel is when it comes to 64-bit double precision math. GPUs, at least consumer grade, are not built for high performance FP64. This is because they are targeted towards gamers and game developers, who do not really care about high precision compute. So vendors like NVIDIA and AMD do not cram FP64 compute cores in their GPUs.\r\n\r\nFor example, on a GTX 780 Ti, the FP64 performance is 1/24 FP32. Which means in an ideal case, running the same code by only changing float types to double types, would yield the double precision run time to be about 1/24th of the single precision time.\r\n\r\nKeep in mind, for compute-bound algorithms, such as GEMM and FFT, the theoretical best case for FP64 performance is 1:2 FP32, simply because it involves computing with double the number of bits as FP32. If the algorithms are memory bound, such as matrix transpose, then most GPUs will attain the 1:2 performance. The numbers we discuss below will all be compute-bound performance numbers.\r\n\r\nHow double precision performs really depends on the architecture of the GPU.\r\n\r\n<h3>Comparing NVIDIA GPUs</h3>\r\nLets take three almost identical cards: <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-780-ti\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-titan-black\" target=\"_blank\">GTX Titan Black</a> and the <a href=\"http://www.nvidia.com/object/tesla-servers.html\" target=\"_blank\">Tesla K40c</a>. All are Kepler <a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">GK110</a> based GPUs, with the same number of SMX and cores (15 SMX, 2880 cores) and the same bus width (384-bit). The 780 Ti and Titan Black even have nearly same base clock speeds (~880MHz; K40c is 745MHz) and identical memory clock speeds (7GHz; K40 is 6GHz).\r\n\r\n<strong>Single Precision Performance</strong>\r\nWith respect to single precision performance, all three are fairly in the same ball park. Which the 780 Ti and Titan Black sit just around 5.1 TFlops, owing to their similar clock speeds, the Tesla K40c drops in at 4.3 TFlops. You could give the Tesla a pass because it has a lower clock speed.\r\n\r\nAll three only vary significantly in three categories.\r\n\r\n<strong>1. Memory</strong>\r\nThis doesn\'t affect performance very much (at least when the sizes fit in all GPUs). The price difference based on market prices of other GPUs with similar memory size variations should not be that big either.\r\n\r\n<strong>2. Price</strong>\r\nAt launch, the GTX 780 Ti was priced at $699, $999 for the Titan Black and an estimated $5500 for the K40c. Well, that escalated quickly.\r\n(Price Sources: <a href=\"http://www.anandtech.com/show/7492/the-geforce-gtx-780-ti-review\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.anandtech.com/show/7765/nvidias-geforce-gtx-titan-black-no-compromises-for-gaming-compute\" target=\"_blank\">GTX Titan Black</a>, <a href=\"http://www.anandtech.com/show/7521/nvidia-launches-tesla-k40\" target=\"_blank\">K40</a>)\r\n\r\n<strong>3. Double Precision Performance</strong>\r\nSo if the 780 Ti and the Titan Black are practically the same in every respect, why is there a $300 difference in their price at launch (discounting memory size difference)?\r\n\r\nThe answer is in the double precision capabilities. The 780 Ti is physically locked at 1:24 FP32 where has the Titan Black has an ace up it\'s sleeve.\r\n\r\nFor the Titan Black, the magic happens in the driver. The Titan black\'s driver gives the user an option to choose the double precision performance between 1:3 and 1:24 FP32 (by switching the GPU to TCC mode). When the double precision performance is set to 1:24 FP32, which is the same as the 780 Ti, the the single precision performance of the Titan Black and 780 Ti are identical. But when the user sets the double precision performance to 1:3 FP32, the single precision performance is compromised to boost double precision performance and make it equal to the K40c. In other words, you can choose the performance of the Titan Black to match either the 780 Ti or the K40c based and your preference.\r\n\r\nThe K40c has a double precision performance of 1:3 without compromising the single precision performance. This is because the K40 is given a special double precision unit for every 3 single precision cores (<a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">white paper</a>). It combines the best of both worlds. NVIDIA also states that the Tesla GPUs go through a much more rigorous Q&A process which guarantees lesser failures and also has additional features such as ECC memory. Hence the large price difference.\r\n\r\n<h4>Summary of NVIDIA GPUs</h4>\r\n<strong>NVIDIA\'s GTX series</strong> are known for their great FP32 performance but are very poor in their FP64 performance. The performance generally ranges between <strong>1:24 (Kepler) and 1:32 (Maxwell)</strong>. The exceptions to this are the GTX Titan cards which blur the lines between the consumer GTX series and the professional Tesla/Quadro cards.\r\n\r\nThe <strong>Kepler architecture Quadro and Tesla series</strong> card provide full double precision performance with <strong>1:3 FP32</strong>. However, with the <strong>Quadro M6000</strong>, NVIDIA has decided to provide only minimal FP64 performance by giving it only <strong>1:32 of FP32 capability</strong> and touting the M6000 as the best graphics card rather than the best graphics+compute card like the Quadro K6000.\r\n\r\n<h3>AMD GPUs</h3>\r\n\r\nAMD GPUs perform fairly well for FP64 compared to FP32. Most AMD cards (including consumer/gaming series) will give between <strong>1:3 and 1:8 FP32 performance for FP64</strong>. The AMD Tahiti architectures tested in these benchmarks here do not suffer from the same problems FP64 problems as NVIDIA\'s GTX series and give a 1:4 performance. Newer Hawaii architecture consumer grade GPUs are expected to provide 1:8 performance.\r\n\r\nThe <strong>FirePro W9100, W8100 and S9150 will give you an incredible FP64 1:2 FP32 performance</strong>.\r\n\r\nOverall, AMD GPUs hold a reputation for good double precision performance ratios compared to their NVIDIA counterparts.\r\n\r\n<h3>Benchmarks</h3>\r\nHere are some benchmarks comparaing FP32 and FP64 GEMM.\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=381229711&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=722963661&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1083990145&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1438814030&amp;format=interactive\"></iframe>\r\n\r\n<h3>Conclusion</h3>\r\nThere is no perfect solution to the problem of choosing between FP32 and FP64 when it comes to GPUs. The best users can do is to decide whether they need the accuracy of double precision compute and how much compute they need, eventually going for a GPU that maximizes their productivity. Some applications like physics modelling and simulation, high accuracy financial computations etc which call for double precision accuracy at high performance require capable FP64 cards. Applications such as image and video processing, signal processing, statistics may not require such high precision and can get away with high FP32 performance only.\r\n\r\nDisclaimer: We receive GPUs from both NVIDIA and AMD. However, this blog does not promote or advocate any specific vendor.\r\n\r\nSources:\r\nWikipedia: <a href=\"https://en.wikipedia.org/wiki/GeForce_700_series\" target=\"_blank\">NVIDIA GeForce 700 Series</a>, <a href=\"https://en.wikipedia.org/wiki/Nvidia_Tesla\" target=\"_blank\">NVIDIA Tesla Series</a>, <a href=\"https://en.wikipedia.org/wiki/Nvidia_Quadro\" target=\"_blank\">NVIDIA Quadro Series</a>, <a href=\"https://en.wikipedia.org/wiki/List_of_AMD_graphics_processing_units#Comparison_table:_Workstation_GPUs\" target=\"_blank\">AMD FirePro Series</a>, <a href=\"https://en.wikipedia.org/wiki/List_of_AMD_graphics_processing_units\" target=\"_blank\">AMD GPUs</a>\r\nNVIDIA Website: <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-780-ti\" target=\"_blank\">NVIDIA GTX 780 Ti</a>, <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-titan-black\" target=\"_blank\">NVIDIA GTX Titan Black</a>, <a href=\"http://www.nvidia.com/object/tesla-servers.html\" target=\"_blank\">NVIDIA Tesla K40</a>\r\nAnandtech: <a href=\"http://www.anandtech.com/show/7492/the-geforce-gtx-780-ti-review\" target=\"_blank\">NVIDIA GTX 780 Ti</a>, <a href=\"http://www.anandtech.com/show/7765/nvidias-geforce-gtx-titan-black-no-compromises-for-gaming-compute\" target=\"_blank\">NVIDIA GTX Titan Black</a>, <a href=\"http://www.anandtech.com/show/7521/nvidia-launches-tesla-k40\" target=\"_blank\">NVIDIA Tesla K40</a>','Explaining FP64 performance on GPUs','','inherit','open','closed','','8389-revision-v1','','','2015-06-22 19:09:28','2015-06-22 19:09:28','',8389,'http://sandbox.arrayfire.com/8389-revision-v1/',0,'revision','',0),(8425,11,'2015-06-24 19:01:22','2015-06-24 19:01:22','<h3>Introduction</h3>\r\nGPUs are really good at doing math. The Achilles heel is when it comes to 64-bit double precision math. GPUs, at least consumer grade, are not built for high performance FP64. This is because they are targeted towards gamers and game developers, who do not really care about high precision compute. So vendors like NVIDIA and AMD do not cram FP64 compute cores in their GPUs.\r\n\r\nFor example, on a GTX 780 Ti, the FP64 performance is 1/24 FP32. Which means in an ideal case, running the same code by only changing float types to double types, would yield the single precision run time to be about 1/24th of the double precision time (time(FP32) = time(FP64)/24).\r\n\r\nKeep in mind, for compute-bound algorithms, such as GEMM and FFT, the theoretical best case for FP64 performance is 1:2 FP32, simply because it involves computing with double the number of bits as FP32. If the algorithms are memory bound, such as matrix transpose, then most GPUs will attain the 1:2 performance. The numbers we discuss below will all be compute-bound performance numbers.\r\n\r\nHow double precision performs really depends on the architecture of the GPU.\r\n\r\n<h3>Comparing NVIDIA GPUs</h3>\r\nLets take three almost identical cards: <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-780-ti\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-titan-black\" target=\"_blank\">GTX Titan Black</a> and the <a href=\"http://www.nvidia.com/object/tesla-servers.html\" target=\"_blank\">Tesla K40c</a>. All are Kepler <a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">GK110</a> based GPUs, with the same number of SMX and cores (15 SMX, 2880 cores) and the same bus width (384-bit). The 780 Ti and Titan Black even have nearly same base clock speeds (~880MHz; K40c is 745MHz) and identical memory clock speeds (7GHz; K40 is 6GHz).\r\n\r\n<strong>Single Precision Performance</strong>\r\nWith respect to single precision performance, all three are fairly in the same ball park. Which the 780 Ti and Titan Black sit just around 5.1 TFlops, owing to their similar clock speeds, the Tesla K40c drops in at 4.3 TFlops. You could give the Tesla a pass because it has a lower clock speed.\r\n\r\nAll three only vary significantly in three categories.\r\n\r\n<strong>1. Memory</strong>\r\nThis doesn\'t affect performance very much (at least when the sizes fit in all GPUs). The price difference based on market prices of other GPUs with similar memory size variations should not be that big either.\r\n\r\n<strong>2. Price</strong>\r\nAt launch, the GTX 780 Ti was priced at $699, $999 for the Titan Black and an estimated $5500 for the K40c. Well, that escalated quickly.\r\n(Price Sources: <a href=\"http://www.anandtech.com/show/7492/the-geforce-gtx-780-ti-review\" target=\"_blank\">GTX 780 Ti</a>, <a href=\"http://www.anandtech.com/show/7765/nvidias-geforce-gtx-titan-black-no-compromises-for-gaming-compute\" target=\"_blank\">GTX Titan Black</a>, <a href=\"http://www.anandtech.com/show/7521/nvidia-launches-tesla-k40\" target=\"_blank\">K40</a>)\r\n\r\n<strong>3. Double Precision Performance</strong>\r\nSo if the 780 Ti and the Titan Black are practically the same in every respect, why is there a $300 difference in their price at launch (discounting memory size difference)?\r\n\r\nThe answer is in the double precision capabilities. The 780 Ti is physically locked at 1:24 FP32 where has the Titan Black has an ace up it\'s sleeve.\r\n\r\nFor the Titan Black, the magic happens in the driver. The Titan black\'s driver gives the user an option to choose the double precision performance between 1:3 and 1:24 FP32 (by switching the GPU to TCC mode). When the double precision performance is set to 1:24 FP32, which is the same as the 780 Ti, the the single precision performance of the Titan Black and 780 Ti are identical. But when the user sets the double precision performance to 1:3 FP32, the single precision performance is compromised to boost double precision performance and make it equal to the K40c. In other words, you can choose the performance of the Titan Black to match either the 780 Ti or the K40c based and your preference.\r\n\r\nThe K40c has a double precision performance of 1:3 without compromising the single precision performance. This is because the K40 is given a special double precision unit for every 3 single precision cores (<a href=\"http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf\" target=\"_blank\">white paper</a>). It combines the best of both worlds. NVIDIA also states that the Tesla GPUs go through a much more rigorous Q&A process which guarantees lesser failures and also has additional features such as ECC memory. Hence the large price difference.\r\n\r\n<h4>Summary of NVIDIA GPUs</h4>\r\n<strong>NVIDIA\'s GTX series</strong> are known for their great FP32 performance but are very poor in their FP64 performance. The performance generally ranges between <strong>1:24 (Kepler) and 1:32 (Maxwell)</strong>. The exceptions to this are the GTX Titan cards which blur the lines between the consumer GTX series and the professional Tesla/Quadro cards.\r\n\r\nThe <strong>Kepler architecture Quadro and Tesla series</strong> card provide full double precision performance with <strong>1:3 FP32</strong>. However, with the <strong>Quadro M6000</strong>, NVIDIA has decided to provide only minimal FP64 performance by giving it only <strong>1:32 of FP32 capability</strong> and touting the M6000 as the best graphics card rather than the best graphics+compute card like the Quadro K6000.\r\n\r\n<h3>AMD GPUs</h3>\r\n\r\nAMD GPUs perform fairly well for FP64 compared to FP32. Most AMD cards (including consumer/gaming series) will give between <strong>1:3 and 1:8 FP32 performance for FP64</strong>. The AMD Tahiti architectures tested in these benchmarks here do not suffer from the same problems FP64 problems as NVIDIA\'s GTX series and give a 1:4 performance. Newer Hawaii architecture consumer grade GPUs are expected to provide 1:8 performance.\r\n\r\nThe <strong>FirePro W9100, W8100 and S9150 will give you an incredible FP64 1:2 FP32 performance</strong>.\r\n\r\nOverall, AMD GPUs hold a reputation for good double precision performance ratios compared to their NVIDIA counterparts.\r\n\r\n<h3>Benchmarks</h3>\r\nHere are some benchmarks comparaing FP32 and FP64 GEMM.\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=381229711&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=722963661&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1083990145&amp;format=interactive\"></iframe>\r\n<iframe width=\"600\" height=\"371\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1438814030&amp;format=interactive\"></iframe>\r\n\r\n<h3>Conclusion</h3>\r\nThere is no perfect solution to the problem of choosing between FP32 and FP64 when it comes to GPUs. The best users can do is to decide whether they need the accuracy of double precision compute and how much compute they need, eventually going for a GPU that maximizes their productivity. Some applications like physics modelling and simulation, high accuracy financial computations etc which call for double precision accuracy at high performance require capable FP64 cards. Applications such as image and video processing, signal processing, statistics may not require such high precision and can get away with high FP32 performance only.\r\n\r\nDisclaimer: We receive GPUs from both NVIDIA and AMD. However, this blog does not promote or advocate any specific vendor.\r\n\r\nSources:\r\nWikipedia: <a href=\"https://en.wikipedia.org/wiki/GeForce_700_series\" target=\"_blank\">NVIDIA GeForce 700 Series</a>, <a href=\"https://en.wikipedia.org/wiki/Nvidia_Tesla\" target=\"_blank\">NVIDIA Tesla Series</a>, <a href=\"https://en.wikipedia.org/wiki/Nvidia_Quadro\" target=\"_blank\">NVIDIA Quadro Series</a>, <a href=\"https://en.wikipedia.org/wiki/List_of_AMD_graphics_processing_units#Comparison_table:_Workstation_GPUs\" target=\"_blank\">AMD FirePro Series</a>, <a href=\"https://en.wikipedia.org/wiki/List_of_AMD_graphics_processing_units\" target=\"_blank\">AMD GPUs</a>\r\nNVIDIA Website: <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-780-ti\" target=\"_blank\">NVIDIA GTX 780 Ti</a>, <a href=\"http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-titan-black\" target=\"_blank\">NVIDIA GTX Titan Black</a>, <a href=\"http://www.nvidia.com/object/tesla-servers.html\" target=\"_blank\">NVIDIA Tesla K40</a>\r\nAnandtech: <a href=\"http://www.anandtech.com/show/7492/the-geforce-gtx-780-ti-review\" target=\"_blank\">NVIDIA GTX 780 Ti</a>, <a href=\"http://www.anandtech.com/show/7765/nvidias-geforce-gtx-titan-black-no-compromises-for-gaming-compute\" target=\"_blank\">NVIDIA GTX Titan Black</a>, <a href=\"http://www.anandtech.com/show/7521/nvidia-launches-tesla-k40\" target=\"_blank\">NVIDIA Tesla K40</a>','Explaining FP64 performance on GPUs','','inherit','open','closed','','8389-revision-v1','','','2015-06-24 19:01:22','2015-06-24 19:01:22','',8389,'http://sandbox.arrayfire.com/8389-revision-v1/',0,'revision','',0),(8426,18528,'2015-06-29 16:04:09','2015-06-29 16:04:09','<p id=\"the-many-benefits-of-open-source-contributors\">Making the decision to open source your software is not an easy process. Indeed, here at ArrayFire our choice to <a href=\"https://github.com/arrayfire/arrayfire\">release ArrayFire</a> under the open source, commercially friendly, BSD 3-Clause License came only after many hours of consideration and philosophical discussion (e.g. see our <a href=\"http://notonlyluck.com/2014/07/31/the-decision-to-open-source-arrayfire/\">CEO\'s blog on these topics</a>). Thus far this decision has proven to be strictly beneficial to our company.</p>\r\n\r\n<h3 id=\"the-impact-of-third-party-contributions\">The impact of third-party contributions</h3>\r\nAlthough ArrayFire is primarily developed by our engineers, there are several <a href=\"https://github.com/arrayfire/arrayfire/graphs/contributors\">contributions from other</a> developers. Therefore we feel particularly compelled to elucidate how these contributions have improved the ArrayFire ecosystem.\r\n<h4>Packaging for Linux and OSX</h4>\r\nOne of the best parts of open source distribution is that your code can be packaged and distributed for a wide variety of operating systems. We <a href=\"http://sandbox.arrayfire.com/download\">distribute binary installers</a> for Windows, OSX, and Linux (x64), and the Jetson TK1. The Linux and Windows installers incorporate the Intel Math Kernel Library (MKL) and hence perform better than pure from-source builds. The OSX Installer uses the Accelerate Framework to give the best performance on OSX machines. Nevertheless, a few users have taken up the task of packaging ArrayFire for their favorite operating systems. <span style=\"line-height: 1.5;\">In particular, </span><a style=\"line-height: 1.5;\" href=\"https://github.com/ghisvail\">Ghislain Antony Vaillant</a><span style=\"line-height: 1.5;\"> created Debian packages for </span><a style=\"line-height: 1.5;\" href=\"https://github.com/clMathLibraries/clFFT\">clFFT</a><span style=\"line-height: 1.5;\">, </span><a style=\"line-height: 1.5;\" href=\"https://github.com/clMathLibraries/clBLAS\">clBLAS</a><span style=\"line-height: 1.5;\">, and </span><a style=\"line-height: 1.5;\" href=\"https://ftp-master.debian.org/new/arrayfire_3.0.1-1.html\">recently added</a><span style=\"line-height: 1.5;\"> ArrayFire\'s CPU backend. After clFFT and clBLAS are accepted his intention is to also package and submit our OpenCL backend to the Debian package repository. (It is unlikely that the CUDA backend will be packaged since CUDA has a license that does not agree with Debian standards.) </span><span style=\"line-height: 1.5;\">Around the same time </span><a style=\"line-height: 1.5;\" href=\"https://github.com/hmalphettes\">Hugues Malphettes</a><span style=\"line-height: 1.5;\"> created a Homebrew Formula for </span>installing ArrayFire on OSX from source<span style=\"line-height: 1.5;\">. Combined with Ghislain\'s recent work to package ArrayFire and its dependencies for Homebrew, it is easier than ever to deploy ArrayFire on OSX. [edit... since this post was originally written Ghislain\'s homebrew formula has been accepted to the official <a href=\"https://github.com/Homebrew/homebrew-science\">homebrew-science</a> repository.] </span><span style=\"line-height: 1.5;\">Not to be left out of the packaging fun, our own </span><a style=\"line-height: 1.5;\" href=\"https://github.com/pavanky\">Pavan Yalamanchili</a><span style=\"line-height: 1.5;\"> recently assumed the role as maintainer for the </span><a style=\"line-height: 1.5;\" href=\"https://aur.archlinux.org/packages/arrayfire-git/\">ArrayFire Arch Linux package</a>.\r\n<h4 id=\"arrayfire-js\">ArrayFire.js</h4>\r\nAmong the most fundamental third party contributions is the creation of Node.js bindings for ArrayFire by <a href=\"https://github.com/unbornchikken\">Gábor Mező</a>. Gábor has previously created bindings for (<a href=\"https://github.com/unbornchikken/NOOOCL\">NOOCL</a>) CMake (<a href=\"https://github.com/unbornchikken/cmake-js\">CMake.js</a>) and made several contributions to the Foreign Function Interface for Node, <a href=\"https://github.com/node-ffi/node-ffi\">node-ffi</a>. In less than a month Gábor created the bindings and contributed it to our GitHub organization. This new <a href=\"https://github.com/arrayfire/arrayfire_js\">ArrayFire.js</a> library lets you use the power of ArrayFire asyncronously from Javascript. We will write more about this project in the future.\r\n<h4 id=\"improving-the-code\">Improving the code</h4>\r\nVery early on we received input from several developers on how we might improve our CMake build system. <a href=\"https://github.com/glehmann\">Gaëtan Lehmann</a> added <a href=\"http://www.cmake.org/Wiki/CMake/Tutorials/How_to_create_a_ProjectConfig.cmake_file\">CMake project configuration files</a>. which automatically create the equivilant of CMake find scripts. This let us remove our (manually maintained) <code>FindArrayFire.cmake</code> script. Gaëtan also wins the award for squelching the most compiler warnings by cleaning up 1000+ unused function warnings issued by clang by <a href=\"https://github.com/arrayfire/arrayfire/commit/7084517cf72d9271076d17547e50fa7ae8b55370\">adding two lines of code</a>. Similarly, <a href=\"https://github.com/nathanjackson\">Nathan Jackson</a> discovered and fixed a race condition in a reduction kernel, and added several tests for complex values and <a href=\"https://github.com/FilipeMaia\">Filipe Maia</a> added support for <code>s64</code> and <code>u64 </code>datatypes by contributing six lines of code and fixed missing definitions for some proxy operators.\r\n<h3 id=\"how-can-you-help-\">How can you help?</h3>\r\nLike many open source projects, there are many ways to contribute to the ArrayFire library:\r\n<h4 id=\"reporting-issues\">Reporting issues</h4>\r\nThe most fundamental way you can contribute is to use our library and help us find problems and suggest methods by which the library may be improved. Using the <a href=\"https://github.com/arrayfire/arrayfire/issues\">ArrayFire GitHub issue tracker</a> you may\r\n<ul>\r\n	<li>make requests for new features</li>\r\n	<li>suggest performance improvements</li>\r\n	<li>report build issues</li>\r\n	<li>file bug reports</li>\r\n	<li>and report on new hardware/backend support status.</li>\r\n</ul>\r\nBe sure to read the <a href=\"https://github.com/arrayfire/arrayfire/blob/master/CONTRIBUTING.md\">contributing guidelines</a> prior to submitting a pull request.\r\n<h4 id=\"documentation-usage-examples-and-community-help\">Documentation, usage examples, and community help</h4>\r\nIf you are using ArrayFire and find someplace where you think our <a href=\"http://sandbox.arrayfire.com/docs/\">documentation</a> could be improved or want to create a <a href=\"http://sandbox.arrayfire.com/docs/examples.htm\">new example</a> we would be happy to hear from you. You may submit these improvements through either the <a href=\"https://github.com/arrayfire/arrayfire/issues\">ArrayFire GitHub issue tracker</a> or send us a message on the <a href=\"https://groups.google.com/forum/#!forum/arrayfire-users\">ArrayFire Google Group</a>.\r\n\r\nThe ArrayFire library also has many methods of providing community-driven help. We would be happy to have your assistance answering questions on the <a href=\"https://groups.google.com/forum/#!forum/arrayfire-users\">ArrayFire Google Group</a>, <a href=\"https://gitter.im/arrayfire/arrayfire?utm_source=badge&amp;utm_medium=badge&amp;utm_campaign=pr-badge&amp;utm_content=badge\">Gitter chat</a>, and <a href=\"https://github.com/arrayfire/arrayfire/issues\">GitHub issue tracker</a>. If the question can\'t be answered here, we also do email support.\r\n<h4 id=\"whitepapers\">Whitepapers</h4>\r\nIf you have found ArrayFire particularly useful in your field or want to exemplify how ArrayFire has improved your software, considering writing a whitepaper with us! We have written whitepapers on topics ranging low-level memory optimizations for HPC applications to the impact of ArrayFire\'s matrix multiplication abilities on high-level language wrappers. We have skilled writers on hand who will work with you to develop a great technical publication. Contact either <a href=\"mailto:sales@arrayfire.com\">sales@arrayfire.com</a> or <a href=\"mailto:technical@arrayfire.com\">technical@arrayfire.com</a> to get started.\r\n<h4 id=\"social-media\">Social media</h4>\r\nHelp spread the word about ArrayFire by following <a href=\"https://twitter.com/arrayfire\">@arrayfire</a> on Twitter, and <a href=\"https://plus.google.com/+Arrayfire\">ArrayFire</a> on Google Plus. You can also collect some <em>sweet, sweet karma</em> by posting our blogs and progress updates on the <a href=\"http://reddit.com/r/OpenCL\">/r/OpenCL</a>, <a href=\"http://reddit.com/r/CUDA\">/r/CUDA</a>, <a href=\"http://reddit.com/r/cpp\">/r/cpp</a> and <a href=\"http://reddit.com/r/programming\">/r/programming</a> subreddits and <a href=\"https://news.ycombinator.com/\">HackerNews</a>\r\n<h4 id=\"directly-fund-arrayfire-development\">Directly fund ArrayFire development</h4>\r\nLastly, if you are a researcher who uses ArrayFire in your work consider writing ArrayFire in as a subcontractor on your next grant. Similarly, if you love ArrayFire but need some additional functionality fast, consider hiring ArrayFire to write the code for you. We offer a full range of <a href=\"http://sandbox.arrayfire.com/consulting/\">consulting</a>, <a href=\"http://sandbox.arrayfire.com/support/\">support</a>, and OpenCL/CUDA/ArrayFire <a href=\"http://sandbox.arrayfire.com/training/\">training</a> options that fit almost any budget.\r\n<h3 id=\"conclusion\">Conclusion</h3>\r\nOne of the most overlooked benefits of making a project open source are contributions from third parties. Since ArrayFire was open sourced in November 2014 we have received a substantial number of contributions. We have third-parties packaging our software for OSX and various Linux distributions, creating wrappers so ArrayFire may be used in other languages, and contributing code changes that improve our build process. Even a person who does not feel comfortable writing code for the ArrayFire library core may contribute by reporting issues, improving our documentation, answering questions on our Google Group page, collaborating with ArrayFire LLC to write whitepapers, talking about us on social media, and even directly funding development of the library through ArrayFire\'s <a href=\"http://sandbox.arrayfire.com/consulting/\">consulting</a>, <a href=\"http://sandbox.arrayfire.com/support/\">support</a>, and OpenCL/CUDA/ArrayFire <a href=\"http://sandbox.arrayfire.com/training/\">training</a> services.','Contributors: A big benefit to open source software','','publish','open','closed','','open-source-benefits-contributors','','','2015-07-21 21:07:39','2015-07-21 21:07:39','',0,'http://sandbox.arrayfire.com/?p=8426',0,'post','',4),(8427,18528,'2015-06-25 18:34:34','2015-06-25 18:34:34','<p id=\"the-many-benefits-of-open-source-contributors\">Making the decision to open source your software is not an easy process. Indeed, here at ArrayFire our choice to <a href=\"https://github.com/arrayfire/arrayfire\">release ArrayFire</a> under the open source, commercially friendly, BSD 3-Clause License came only after many hours of consideration and philosophical discussion (e.g. see our <a href=\"http://notonlyluck.com/2014/07/31/the-decision-to-open-source-arrayfire/\">CEO\'s blog on these topics</a>). Thus far this decision has proven to be strictly beneficial to our company.</p>\r\n\r\n<h2 id=\"the-impact-of-third-party-contributions\">The impact of third-party contributions</h2>\r\nAlthough ArrayFire is primarily developed by our engineers, there are several<a href=\"https://github.com/arrayfire/arrayfire/graphs/contributors\">contributions from other</a> developers. Therefore we feel particularly compelled to elucidate how these contributions have improved the ArrayFire ecosystem.\r\n<h3 id=\"packaging-for-linux-and-osx\">Packaging for Linux and OSX</h3>\r\nOne of the best parts of open source distribution is that your code can be packaged and distributed for a wide variety of operating systems. We already<a href=\"http://sandbox.arrayfire.com/download\">distribute binary installers</a> for Windows, OSX, and Linux, but a few users have taken up the task of packaging ArrayFire for their favorite operating systems. In particular, <a href=\"https://github.com/ghisvail\">Ghislain Antony Vaillant</a> created Debian packages for <a href=\"https://github.com/clMathLibraries/clFFT\">clFFT</a>, <a href=\"https://github.com/clMathLibraries/clBLAS\">clBLAS</a>, and <a href=\"https://ftp-master.debian.org/new/arrayfire_3.0.1-1.html\">recently added</a> ArrayFire\'s CPU backend. After clFFT and clBLAS are accepted his intention is to also package and submit our OpenCL backend to the Debian package repository. Around the same time <a href=\"https://github.com/hmalphettes\">Hugues Malphettes</a> created a Homebrew Formula for <a href=\"https://github.com/sutoiku/homebrew-arrayfire\">installing ArrayFire on OSX from source</a>. Not to be left out of the packaging fun, our own <a href=\"https://github.com/pavanky\">Pavan Yalamanchili</a> recently assumed the role as maintainer for the <a href=\"https://aur.archlinux.org/packages/arrayfire-git/\">ArrayFire Arch Linux package</a>\r\n<h3 id=\"arrayfire-js\">ArrayFire.js</h3>\r\nAmong the most fundamental third party contributions is the creation of Node.js bindings for ArrayFire by <a href=\"https://github.com/unbornchikken\">Gábor Mező</a>. Gábor has previously created bindings for (<a href=\"https://github.com/unbornchikken/NOOOCL\">NOOCL</a>) CMake (<a href=\"https://github.com/unbornchikken/cmake-js\">CMake.js</a>) and made several contributions to the Foreign Function Interface for Node, <a href=\"https://github.com/node-ffi/node-ffi\">node-ffi</a>. In less than a month Gábor created the bindings and contributed it to our GitHub organization. This new <a href=\"https://github.com/arrayfire/arrayfire_js\">ArrayFire.js</a>library lets you use the power of ArrayFire asyncronously from Javascript. We will write more about this project in the future.\r\n<h3 id=\"improving-the-code\">Improving the code</h3>\r\nVery early on we received input from several developers on how we might improve our CMake build system. <a href=\"https://github.com/glehmann\">Gaëtan Lehmann</a> added <a href=\"http://www.cmake.org/Wiki/CMake/Tutorials/How_to_create_a_ProjectConfig.cmake_file\">CMake project configuration files</a>. which automatically create the equivilant of CMake find scripts. This let us remove our (manually maintained)<code>FindArrayFire.cmake</code> script. Gaëtan also wins the award for squelching the most compiler warnings by cleaning up 1000+ unused function warnings issued by clang by <a href=\"https://github.com/arrayfire/arrayfire/commit/7084517cf72d9271076d17547e50fa7ae8b55370\">adding two lines of code</a>. Similarly, <a href=\"https://github.com/nathanjackson\">Nathan Jackson</a>discovered and fixed a race condition in a reduction kernel, and added several tests for complex values and <a href=\"https://github.com/FilipeMaia\">Filipe Maia</a> added support for <code>s64</code> and <code>u64</code>datatypes by contributing six lines of code and fixed missing definitions for some proxy operators.\r\n<h2 id=\"how-can-you-help-\">How can you help?</h2>\r\nLike many open source projects, there are many ways to contribute to the ArrayFire library:\r\n<h3 id=\"reporting-issues\">Reporting issues</h3>\r\nThe most fundamental way you can contribute is to use our library and help us find problems and suggest methods by which the library may be improved. Using the <a href=\"https://github.com/arrayfire/arrayfire/issues\">ArrayFire GitHub issue tracker</a> you may\r\n<ul>\r\n	<li>make requests for new features</li>\r\n	<li>suggest performance improvements</li>\r\n	<li>report build issues</li>\r\n	<li>file bug reports</li>\r\n	<li>and report on new hardware/backend support status.</li>\r\n</ul>\r\nBe sure to read the <a href=\"https://github.com/arrayfire/arrayfire/blob/master/CONTRIBUTING.md\">contributing guidelines</a> prior to submitting a pull request.\r\n<h3 id=\"documentation-usage-examples-and-community-help\">Documentation, usage examples, and community help</h3>\r\nIf you are using ArrayFire and find someplace where you think our<a href=\"http://sandbox.arrayfire.com/docs/\">documentation</a> could be improved or want to create a <a href=\"http://sandbox.arrayfire.com/docs/examples.htm\">new example</a> we would be happy to hear from you. You may submit these improvements through either the<a href=\"https://github.com/arrayfire/arrayfire/issues\">ArrayFire GitHub issue tracker</a> or send us a message on the <a href=\"https://groups.google.com/forum/#!forum/arrayfire-users\">ArrayFire Google Group</a>\r\n<h3 id=\"whitepapers\">Whitepapers</h3>\r\nIf you have found ArrayFire particularly useful in your field or want to exemplify how ArrayFire has improved your software, considering writing a whitepaper with us! We have written whitepapers on topics ranging low-level memory optimizations for HPC applications to the impact of ArrayFire\'s matrix multiplication abilities on high-level language wrappers. We have skilled writers on hand who will work with you to develop a great technical publication. Contact either <a href=\"mailto:sales@arrayfire.com\">sales@arrayfire.com</a> or <a href=\"mailto:technical@arrayfire.com\">technical@arayfire.com</a> to get started.\r\n<h3 id=\"social-media\">Social media</h3>\r\nHelp spread the word about ArrayFire by following <a href=\"https://twitter.com/arrayfire\">@arrayfire</a> on Twitter, and<a href=\"https://plus.google.com/+Arrayfire\">ArrayFire</a> on Google Plus. You can also collect some sweet, sweet karma by posting our blogs and progress updates on the <a href=\"http://reddit.com/r/OpenCL\">/r/OpenCL</a> <a href=\"http://reddit.com/r/CUDA\">/r/CUDA</a> <a href=\"http://reddit.com/r/cpp\">/r/cpp</a> and<a href=\"http://reddit.com/r/programming\">/r/programming</a> subreddits and <a href=\"https://news.ycombinator.com/\">HackerNews</a>\r\n<h3 id=\"directly-fund-arrayfire-development\">Directly fund ArrayFire development</h3>\r\nLastly, if you are a researcher who uses ArrayFire in your work consider writing ArrayFire in as a subcontractor on your next grant. Similarly, if you love ArrayFire but need some additional functionality fast, consider hiring ArrayFire to write the code for you. We offer a full range of <a href=\"http://sandbox.arrayfire.com/consulting/\">consulting</a>, <a href=\"http://sandbox.arrayfire.com/support/\">support</a>, and OpenCL/CUDA/ArrayFire <a href=\"http://sandbox.arrayfire.com/training/\">training</a> options that fit almost any budget.\r\n<h2 id=\"conclusion\">Conclusion</h2>\r\nOne of the most overlooked benefits of making a project open source are contributions from third parties. Since ArrayFire was open sourced in November 2014 we have already received have been quite substantial. We have third-parties packaging our software for OSX and various Linux distributions, creating wrappers so ArrayFire maybe used in other languages, and many code changes that have improved our build process and code reliability. Even a person does not feel comfortable writing code for the ArrayFire library core, they may also contribute by reporting issues, improving our documentation, answering questions on our Google Group page, collaborating with ArrayFire LLC to write whitepapers, talking about us on social media, and even directly funding development of the library through ArrayFire\'s <a href=\"http://sandbox.arrayfire.com/consulting/\">consulting</a>, <a href=\"http://sandbox.arrayfire.com/support/\">support</a>, and OpenCL/CUDA/ArrayFire <a href=\"http://sandbox.arrayfire.com/training/\">training</a> services.','The many benefits of open source: contributors','','inherit','open','closed','','8426-revision-v1','','','2015-06-25 18:34:34','2015-06-25 18:34:34','',8426,'http://sandbox.arrayfire.com/8426-revision-v1/',0,'revision','',0),(8428,18528,'2015-06-25 18:36:22','2015-06-25 18:36:22','<p id=\"the-many-benefits-of-open-source-contributors\">Making the decision to open source your software is not an easy process. Indeed, here at ArrayFire our choice to <a href=\"https://github.com/arrayfire/arrayfire\">release ArrayFire</a> under the open source, commercially friendly, BSD 3-Clause License came only after many hours of consideration and philosophical discussion (e.g. see our <a href=\"http://notonlyluck.com/2014/07/31/the-decision-to-open-source-arrayfire/\">CEO\'s blog on these topics</a>). Thus far this decision has proven to be strictly beneficial to our company.</p>\r\n\r\n<h2 id=\"the-impact-of-third-party-contributions\">The impact of third-party contributions</h2>\r\nAlthough ArrayFire is primarily developed by our engineers, there are several<a href=\"https://github.com/arrayfire/arrayfire/graphs/contributors\">contributions from other</a> developers. Therefore we feel particularly compelled to elucidate how these contributions have improved the ArrayFire ecosystem.\r\n<h3 id=\"packaging-for-linux-and-osx\">Packaging for Linux and OSX</h3>\r\nOne of the best parts of open source distribution is that your code can be packaged and distributed for a wide variety of operating systems. We already<a href=\"http://sandbox.arrayfire.com/download\">distribute binary installers</a> for Windows, OSX, and Linux, but a few users have taken up the task of packaging ArrayFire for their favorite operating systems. In particular, <a href=\"https://github.com/ghisvail\">Ghislain Antony Vaillant</a> created Debian packages for <a href=\"https://github.com/clMathLibraries/clFFT\">clFFT</a>, <a href=\"https://github.com/clMathLibraries/clBLAS\">clBLAS</a>, and <a href=\"https://ftp-master.debian.org/new/arrayfire_3.0.1-1.html\">recently added</a> ArrayFire\'s CPU backend. After clFFT and clBLAS are accepted his intention is to also package and submit our OpenCL backend to the Debian package repository. Around the same time <a href=\"https://github.com/hmalphettes\">Hugues Malphettes</a> created a Homebrew Formula for <a href=\"https://github.com/sutoiku/homebrew-arrayfire\">installing ArrayFire on OSX from source</a>. Not to be left out of the packaging fun, our own <a href=\"https://github.com/pavanky\">Pavan Yalamanchili</a> recently assumed the role as maintainer for the <a href=\"https://aur.archlinux.org/packages/arrayfire-git/\">ArrayFire Arch Linux package</a>\r\n<h3 id=\"arrayfire-js\">ArrayFire.js</h3>\r\nAmong the most fundamental third party contributions is the creation of Node.js bindings for ArrayFire by <a href=\"https://github.com/unbornchikken\">Gábor Mező</a>. Gábor has previously created bindings for (<a href=\"https://github.com/unbornchikken/NOOOCL\">NOOCL</a>) CMake (<a href=\"https://github.com/unbornchikken/cmake-js\">CMake.js</a>) and made several contributions to the Foreign Function Interface for Node, <a href=\"https://github.com/node-ffi/node-ffi\">node-ffi</a>. In less than a month Gábor created the bindings and contributed it to our GitHub organization. This new <a href=\"https://github.com/arrayfire/arrayfire_js\">ArrayFire.js</a>library lets you use the power of ArrayFire asyncronously from Javascript. We will write more about this project in the future.\r\n<h3 id=\"improving-the-code\">Improving the code</h3>\r\nVery early on we received input from several developers on how we might improve our CMake build system. <a href=\"https://github.com/glehmann\">Gaëtan Lehmann</a> added <a href=\"http://www.cmake.org/Wiki/CMake/Tutorials/How_to_create_a_ProjectConfig.cmake_file\">CMake project configuration files</a>. which automatically create the equivilant of CMake find scripts. This let us remove our (manually maintained)<code>FindArrayFire.cmake</code> script. Gaëtan also wins the award for squelching the most compiler warnings by cleaning up 1000+ unused function warnings issued by clang by <a href=\"https://github.com/arrayfire/arrayfire/commit/7084517cf72d9271076d17547e50fa7ae8b55370\">adding two lines of code</a>. Similarly, <a href=\"https://github.com/nathanjackson\">Nathan Jackson</a>discovered and fixed a race condition in a reduction kernel, and added several tests for complex values and <a href=\"https://github.com/FilipeMaia\">Filipe Maia</a> added support for <code>s64</code> and <code>u64</code>datatypes by contributing six lines of code and fixed missing definitions for some proxy operators.\r\n<h2 id=\"how-can-you-help-\">How can you help?</h2>\r\nLike many open source projects, there are many ways to contribute to the ArrayFire library:\r\n<h3 id=\"reporting-issues\">Reporting issues</h3>\r\nThe most fundamental way you can contribute is to use our library and help us find problems and suggest methods by which the library may be improved. Using the <a href=\"https://github.com/arrayfire/arrayfire/issues\">ArrayFire GitHub issue tracker</a> you may\r\n<ul>\r\n	<li>make requests for new features</li>\r\n	<li>suggest performance improvements</li>\r\n	<li>report build issues</li>\r\n	<li>file bug reports</li>\r\n	<li>and report on new hardware/backend support status.</li>\r\n</ul>\r\nBe sure to read the <a href=\"https://github.com/arrayfire/arrayfire/blob/master/CONTRIBUTING.md\">contributing guidelines</a> prior to submitting a pull request.\r\n<h3 id=\"documentation-usage-examples-and-community-help\">Documentation, usage examples, and community help</h3>\r\nIf you are using ArrayFire and find someplace where you think our<a href=\"http://sandbox.arrayfire.com/docs/\">documentation</a> could be improved or want to create a <a href=\"http://sandbox.arrayfire.com/docs/examples.htm\">new example</a> we would be happy to hear from you. You may submit these improvements through either the<a href=\"https://github.com/arrayfire/arrayfire/issues\">ArrayFire GitHub issue tracker</a> or send us a message on the <a href=\"https://groups.google.com/forum/#!forum/arrayfire-users\">ArrayFire Google Group</a>\r\n<h3 id=\"whitepapers\">Whitepapers</h3>\r\nIf you have found ArrayFire particularly useful in your field or want to exemplify how ArrayFire has improved your software, considering writing a whitepaper with us! We have written whitepapers on topics ranging low-level memory optimizations for HPC applications to the impact of ArrayFire\'s matrix multiplication abilities on high-level language wrappers. We have skilled writers on hand who will work with you to develop a great technical publication. Contact either <a href=\"mailto:sales@arrayfire.com\">sales@arrayfire.com</a> or <a href=\"mailto:technical@arrayfire.com\">technical@arayfire.com</a> to get started.\r\n<h3 id=\"social-media\">Social media</h3>\r\nHelp spread the word about ArrayFire by following <a href=\"https://twitter.com/arrayfire\">@arrayfire</a> on Twitter, and<a href=\"https://plus.google.com/+Arrayfire\">ArrayFire</a> on Google Plus. You can also collect some sweet, sweet karma by posting our blogs and progress updates on the <a href=\"http://reddit.com/r/OpenCL\">/r/OpenCL</a> <a href=\"http://reddit.com/r/CUDA\">/r/CUDA</a> <a href=\"http://reddit.com/r/cpp\">/r/cpp</a> and<a href=\"http://reddit.com/r/programming\">/r/programming</a> subreddits and <a href=\"https://news.ycombinator.com/\">HackerNews</a>\r\n<h3 id=\"directly-fund-arrayfire-development\">Directly fund ArrayFire development</h3>\r\nLastly, if you are a researcher who uses ArrayFire in your work consider writing ArrayFire in as a subcontractor on your next grant. Similarly, if you love ArrayFire but need some additional functionality fast, consider hiring ArrayFire to write the code for you. We offer a full range of <a href=\"http://sandbox.arrayfire.com/consulting/\">consulting</a>, <a href=\"http://sandbox.arrayfire.com/support/\">support</a>, and OpenCL/CUDA/ArrayFire <a href=\"http://sandbox.arrayfire.com/training/\">training</a> options that fit almost any budget.\r\n<h2 id=\"conclusion\">Conclusion</h2>\r\nOne of the most overlooked benefits of making a project open source are contributions from third parties. Since ArrayFire was open sourced in November 2014 we have already received have been quite substantial. We have third-parties packaging our software for OSX and various Linux distributions, creating wrappers so ArrayFire maybe used in other languages, and many code changes that have improved our build process and code reliability. Even a person does not feel comfortable writing code for the ArrayFire library core, they may also contribute by reporting issues, improving our documentation, answering questions on our Google Group page, collaborating with ArrayFire LLC to write whitepapers, talking about us on social media, and even directly funding development of the library through ArrayFire\'s <a href=\"http://sandbox.arrayfire.com/consulting/\">consulting</a>, <a href=\"http://sandbox.arrayfire.com/support/\">support</a>, and OpenCL/CUDA/ArrayFire <a href=\"http://sandbox.arrayfire.com/training/\">training</a> services.','Contributors: A big benefit to open source software','','inherit','open','closed','','8426-revision-v1','','','2015-06-25 18:36:22','2015-06-25 18:36:22','',8426,'http://sandbox.arrayfire.com/8426-revision-v1/',0,'revision','',0),