Everyday developers ask us to predict how much speedup they can get with <a class="zem_slink" title="CUDA" href="http://en.wikipedia.org/wiki/CUDA" target="_blank" rel="wikipedia">CUDA</a> or <a class="zem_slink" title="OpenCL" href="http://en.wikipedia.org/wiki/OpenCL" target="_blank" rel="wikipedia">OpenCL</a>. Rather than gaze mysteriously into a crystal ball, we ask the developers questions to explore pertinent acceleration factors.

Note, we'll use the term <em>accelerator</em> to include GPUs, Xeon Phi coprocessor, APUs, FPGAs, and any other CUDA or OpenCL device. The principles we discuss below are equally applicable to all of these accelerators.

The following are some of the important factors that must be considered when estimating the potential for accelerated speedups:
<ul>
	<li><b>Hardware: </b> The more advanced the accelerator hardware, the more the speedup you get (e.g. the NVIDIA Kepler K20 outperforms the previous NVIDIA Fermi C2090 generation).</li>
</ul>
<ul>
	<li><b>Data Sizes:  </b>In general, accelerators will outperform CPUs to a larger degree as data sizes increase. Accelerators are only fast because they can exploit data parallelism. If there is not much data (e.g. only a few hundred elements in a vector), then there is not going to be much gain in performance. However, if there is a lot of data (e.g. more than 10,000 elements or a 100x100 matrix) then accelerators will be able to process those elements in parallel and exploit the data-parallelism. For an example of how speedups increase with data size, see the <a href="http://www.accelereyes.com/products/benchmarks_arrayfire">ArrayFire benchmarks</a>.</li>
</ul>
<ul>
	<li><b>Application:  </b>Speedup figures vary from application to application. Some operations may be computed faster than others on an accelerator. In general, the more parallelizable, the better the accelerator performance. And the fewer trips memory takes across the bus, the better. For examples of real application speedups, see the <a href="http://www.accelereyes.com/examples/case_studies">ArrayFire case studies</a>.</li>
</ul>
<ul>
	<li><b>CPU:  </b>In running speedup comparisons between the CPU and the GPU, the CPU speed matters. The CPU still plays an important role in heterogeneous computing throughput. For example, better CPUs help ArrayFire go faster, because ArrayFire does a lot of things (e.g. JIT compilation) on the CPU to keep the accelerator fully utilized.</li>
</ul>
<ul>
	<li><b>Code Quality:  </b>Better-written CPU code always leads to better CPU + accelerator code.</li>
</ul>
<ul>
	<li><strong>Choice of Software Tools:</strong>  Software tools can affect your potential for speedups. For instance, libraries outperform compilers in <a href="http://blog.accelereyes.com/blog/2012/04/11/no-free-lunch-for-gpu-compiler-directives/">providing productive performance</a> for accelerators.</li>
</ul>
What factors do you consider when predicting speedups?

&nbsp;
<h6 class="zemanta-related-title" style="font-size: 1em;">Related articles</h6>
<ul class="zemanta-article-ul zemanta-article-ul-image" style="margin: 0; padding: 0; overflow: hidden;">
	<li class="zemanta-article-ul-li-image zemanta-article-ul-li" style="padding: 0; background: none; list-style: none; display: block; float: left; vertical-align: top; text-align: left; width: 84px; font-size: 11px; margin: 2px 10px 10px 2px;"><a style="box-shadow: 0px 0px 4px #999; padding: 2px; display: block; border-radius: 2px; text-decoration: none;" href="http://withimagination.imgtec.com/index.php/powervr/mythbusters-imaginations-gpu-compute-special-edition-part-1" target="_blank"><img style="padding: 0; margin: 0; border: 0; display: block; width: 80px; max-width: 100%;" alt="" src="http://i.zemanta.com/139596145_80_80.jpg" /></a><a style="display: block; overflow: hidden; text-decoration: none; line-height: 12pt; height: 80px; padding: 5px 2px 0 2px;" href="http://withimagination.imgtec.com/index.php/powervr/mythbusters-imaginations-gpu-compute-special-edition-part-1" target="_blank">Mythbusters - Imagination's GPU Compute special edition (part 1)</a></li>
	<li class="zemanta-article-ul-li-image zemanta-article-ul-li" style="padding: 0; background: none; list-style: none; display: block; float: left; vertical-align: top; text-align: left; width: 84px; font-size: 11px; margin: 2px 10px 10px 2px;"><a style="box-shadow: 0px 0px 4px #999; padding: 2px; display: block; border-radius: 2px; text-decoration: none;" href="http://streamcomputing.eu/blog/2012-10-23/targetting-various-architectures-in-opencl-and-cuda/" target="_blank"><img style="padding: 0; margin: 0; border: 0; display: block; width: 80px; max-width: 100%;" alt="" src="http://i.zemanta.com/138441772_80_80.jpg" /></a><a style="display: block; overflow: hidden; text-decoration: none; line-height: 12pt; height: 80px; padding: 5px 2px 0 2px;" href="http://streamcomputing.eu/blog/2012-10-23/targetting-various-architectures-in-opencl-and-cuda/" target="_blank">Targetting various architectures in OpenCL and CUDA</a></li>
	<li class="zemanta-article-ul-li-image zemanta-article-ul-li" style="padding: 0; background: none; list-style: none; display: block; float: left; vertical-align: top; text-align: left; width: 84px; font-size: 11px; margin: 2px 10px 10px 2px;"><a style="box-shadow: 0px 0px 4px #999; padding: 2px; display: block; border-radius: 2px; text-decoration: none;" href="http://insidehpc.com/2012/11/25/gpu-accelerated-science-applications/" target="_blank"><img style="padding: 0; margin: 0; border: 0; display: block; width: 80px; max-width: 100%;" alt="" src="http://i.zemanta.com/noimg_12_80_80.jpg" /></a><a style="display: block; overflow: hidden; text-decoration: none; line-height: 12pt; height: 80px; padding: 5px 2px 0 2px;" href="http://insidehpc.com/2012/11/25/gpu-accelerated-science-applications/" target="_blank">GPU-accelerated Science Applications</a></li>
	<li class="zemanta-article-ul-li-image zemanta-article-ul-li" style="padding: 0; background: none; list-style: none; display: block; float: left; vertical-align: top; text-align: left; width: 84px; font-size: 11px; margin: 2px 10px 10px 2px;"><a style="box-shadow: 0px 0px 4px #999; padding: 2px; display: block; border-radius: 2px; text-decoration: none;" href="http://blog.accelereyes.com/blog/2012/08/13/option-pricing/" target="_blank"><img style="padding: 0; margin: 0; border: 0; display: block; width: 80px; max-width: 100%;" alt="" src="http://i.zemanta.com/131049345_80_80.jpg" /></a><a style="display: block; overflow: hidden; text-decoration: none; line-height: 12pt; height: 80px; padding: 5px 2px 0 2px;" href="http://blog.accelereyes.com/blog/2012/08/13/option-pricing/" target="_blank">Option Pricing</a></li>
	<li class="zemanta-article-ul-li-image zemanta-article-ul-li" style="padding: 0; background: none; list-style: none; display: block; float: left; vertical-align: top; text-align: left; width: 84px; font-size: 11px; margin: 2px 10px 10px 2px;"><a style="box-shadow: 0px 0px 4px #999; padding: 2px; display: block; border-radius: 2px; text-decoration: none;" href="http://openclnews.com/blog/amd_strengthens_opencl_ecosystem_with_new_tools_for_hpc_that_take_advantage" target="_blank"><img style="padding: 0; margin: 0; border: 0; display: block; width: 80px; max-width: 100%;" alt="" src="http://i.zemanta.com/noimg_9_80_80.jpg" /></a><a style="display: block; overflow: hidden; text-decoration: none; line-height: 12pt; height: 80px; padding: 5px 2px 0 2px;" href="http://openclnews.com/blog/amd_strengthens_opencl_ecosystem_with_new_tools_for_hpc_that_take_advantage" target="_blank">AMD strengthens OpenCL ecosystem with new tools for HPC that take advantage of GPU</a></li>
	<li class="zemanta-article-ul-li-image zemanta-article-ul-li" style="padding: 0; background: none; list-style: none; display: block; float: left; vertical-align: top; text-align: left; width: 84px; font-size: 11px; margin: 2px 10px 10px 2px;"><a style="box-shadow: 0px 0px 4px #999; padding: 2px; display: block; border-radius: 2px; text-decoration: none;" href="http://insidehpc.com/2012/12/14/gpu-testing-program-showing-kepler-speedups-of-2x-or-more-on-real-apps/" target="_blank"><img style="padding: 0; margin: 0; border: 0; display: block; width: 80px; max-width: 100%;" alt="" src="http://i.zemanta.com/noimg_0_80_80.jpg" /></a><a style="display: block; overflow: hidden; text-decoration: none; line-height: 12pt; height: 80px; padding: 5px 2px 0 2px;" href="http://insidehpc.com/2012/12/14/gpu-testing-program-showing-kepler-speedups-of-2x-or-more-on-real-apps/" target="_blank">GPU Testing Program Showing Kepler Speedups of 2X or More on Real Apps</a></li>
</ul>
<div class="zemanta-pixie" style="margin-top: 10px; height: 15px;"><img class="zemanta-pixie-img" style="border: none; float: right;" alt="" src="http://img.zemanta.com/pixy.gif?x-id=a382331e-bd95-4708-b119-c69d1af27ae8" /></div>