<strong>AccelerEyes</strong> is focused on not only providing the most easy to use GPU programming platform for CUDA capable GPUs by leveraging the MATLAB® language, our engineering organization is always looking for ways to improve the performance of all areas in the Jacket platform. A case in point is some recent work with <em>matrix multiplication</em>, specifically (Single General Matrix Multiply) SGEMM, or <a title="MTIMES" href="http://wiki.accelereyes.com/wiki/index.php/MTIMES">MTIMES</a>.  The Jacket 1.3 release was based on CUBLAS for matrix multiplication and given the importance of matrix multiplication to so many of our customers, we decided to find out if we could improve performance of the function.

<strong><em>Update: The new MTIMES routine in Jacket 1.4 has improved sigificantly since these benchmarks of the Release Candidate were taken. Have a look at the <a href="http://wiki.accelereyes.com/wiki/index.php/MTIMES_Benchmarks_(Jacket_1.4)" rel="nofollow">MTIMES Benchmarks</a> wiki for up-to-date performance results, including Fermi (Tesla C2050) benchmarks!</em></strong>

The following chart illustrates the improvements we were able to make with our <em>custom </em>GEMM implementation in the 1.4 Release Candidate. The performance timings for SGEMM at various sizes of square matrices were performed, comparing the <em>Jacket 1.3 Release</em> and the<em> Jacket 1.4 Release Candidate</em>.

<a href="http://blog.accelereyes.com/blog/wp-content/uploads/2010/06/13_14_mtimes_single_A_both_edit-e1277469888509.jpeg"><img class="alignnone size-full wp-image-514" title="1.3 1.4 mtimes single" alt="" src="http://blog.accelereyes.com/blog/wp-content/uploads/2010/06/13_14_mtimes_single_A_both_edit-e1277469888509.jpeg" width="683" height="580" /></a>

The graph above represents <strong>C=A*B</strong> timings for <em>every NxN matrix from 10x10 to 4000x4000</em>. Each data point is an average of 2 calls of MTIMES for each matrix size. Notice that Jacket 1.3 follows exactly 2 curves; the <em>lower</em> curve for certain multiples of 16 and 32, and the <em>upper</em> curve for everything else. The lower blue curve is attained by not using textures, as the data aligns and coalesces nicely in memory, and using other optimizations for certain data sizes. By switching to using only texture memory, <em>Jacket 1.4</em> sacrifices some performance slightly in certain cases, but along with other optimizations, it brings <strong>faster consistent run-times</strong>, gives <strong>enhanced <a title="GFOR" href="http://wiki.accelereyes.com/wiki/index.php/GFOR">GFOR</a></strong><strong> performance</strong> and enables <strong>native mixed matrix types</strong> support. In addition to texture memory, careful shared memory and register usage brings significant performance enhancements as well.
<div id="_mcePaste"><a href="http://blog.accelereyes.com/blog/wp-content/uploads/2010/06/CPU_14_mtimes_single_A2_all.png"><img class="alignnone size-full wp-image-512" title="1.3 CPU 1.4 mtimes single" alt="" src="http://blog.accelereyes.com/blog/wp-content/uploads/2010/06/CPU_14_mtimes_single_A2_all.png" width="687" height="571" /></a></div>
The above graph overlays a MATLAB R2009b 32-bit (CPU) benchmark of MTIMES. <a href="http://blog.accelereyes.com/blog/wp-content/uploads/2010/06/13_14_mtimes_single_AG_all-e1277416726380.png"><img class="alignnone size-full wp-image-499" title="1-3 vs 1-4 vs CPU GFLOPS" alt="" src="http://blog.accelereyes.com/blog/wp-content/uploads/2010/06/13_14_mtimes_single_AG_all-e1277416726380.png" width="714" height="620" /></a>

The graph above represents <strong>C=A*B</strong> GFLOPS for <em>every NxN matrix from 10x10 to 4000x4000</em>.

The<strong> GFLOPS formula</strong> used: <a href="http://blog.accelereyes.com/blog/wp-content/uploads/2010/06/CodeCogsEqn.gif"><img class="aligncenter size-full wp-image-484" title="MTIMES GFLOPS formula" alt="SGEMM GFLOPS formulat" src="http://blog.accelereyes.com/blog/wp-content/uploads/2010/06/CodeCogsEqn.gif" width="156" height="42" /></a>

Miscellaneous information regarding the Jacket installation and hardware used for testing:
<pre class="lang:default highlight:0 decode:true crayon-selected">&gt;&gt; ginfo
AccelerEyes Jacket v1.4.0rc2 (build 4971)
CUDA driver: 195.36.15, CUDA toolkit 3.0
Memory: 0 CPU-used, 23 GPU-used, 4034 GPU-free (in MB)
License Type: Designated Computer
License Features: jacket sdk mgl4 dla
Multi-GPU: Licensed for 4 GPUs
Detected CUDA-capable GPUs:
GPU0 Tesla C1060, 1265 MHz, 4095 MB VRAM, Compute 1.3 (single,double) (in use)
GPU1 GeForce 8400 GS, 896 MHz, 511 MB VRAM, Compute 1.1 (single)</pre>
<pre lang="bash">$ cat /proc/cpuinfo
vendor_id      : GenuineIntel
model name    : Pentium(R) Dual-Core  CPU  E5200  @ 2.50GHz
cpu MHz        : 2499.934
cache size     : 2048 KB
cpu cores      : 2
…</pre>
<strong>Update</strong>
Below is an example of the SGEMM performance on a Tesla C2050 (Fermi) card
-  Jacket 1.3 vs. Jacket 1.4 Final Release.
<a href="http://wiki.accelereyes.com/wiki/images/8/8a/Jacket14_sgemm.jpg"><img alt="" src="http://wiki.accelereyes.com/wiki/images/8/8a/Jacket14_sgemm.jpg" width="700" height="580" /></a>

<strong>References</strong>
* Volkov, V., and Demmel, J. W., <a title="Benchmarking GPUs to tune dense linear algebra" href="http://mc.stanford.edu/cgi-bin/images/6/65/SC08_Volkov_GPU.pdf">Benchmarking GPUs to tune dense linear algebra</a>, SC08.
* <a title="SGEMM code" href="http://forums.nvidia.com/index.php?showtopic=47689">SGEMM code</a> from Vasily Volkov.
* <a title="SGEMM code" href="http://forums.nvidia.com/index.php?showtopic=159033">SGEMM code</a> from Lung-Sheng Chien.
* <em><strong>See the </strong></em><a title="MTIMES benchmarks" href="http://wiki.accelereyes.com/wiki/index.php/MTIMES_Benchmarks_(Jacket_1.4)"><em><strong>MTIMES benchmarks</strong></em></a><em><strong> wiki page for more info!</strong></em>