(6742,7,'2014-09-18 21:36:56','2014-09-18 21:36:56','[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\" ]\r\n\r\n[column type=\"one-half\"][custom_headline type=\"left\" level=\"h3\" ]TRAINING[/custom_headline] Attendees will receive the latest industry knowledge and techniques for GPU computing in CUDA and OpenCL.\r\n\r\nArrayFire offers up to four days of specialized GPU training in CUDA and OpenCL programming. We provide customized on-site training courses for customers that have 3 or more attendees from their organization. We can travel to your location and provide a CUDA or OpenCL training course tailored to meet your application-specific needs. We also offer individual trainings at our Atlanta office. Please <a href=\"mailto:sales@arrayfire.com\">contact us</a> to learn more about our training offerings.\r\n\r\nWe recommend that attendees have a working knowledge of C/C++ in order to gain the most from the training courses. For more information about the content of our training lectures and hands-on practicums, please see the following training course syllabus.\r\n\r\n[/column]\r\n\r\n[column type=\"one-half\" last=\"true\" fade=\"true\" fade_animation=\"in-from-right\" class=\"center-text\"][gap size=\"20%\"][image src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/02/Professor-AF_Exported-e1395436350363.png\" ][/column]\r\n\r\n[column type=\"one-half\"]\r\n\r\n[custom_headline type=\"left\" level=\"h3\"]INCLUDED IN COURSE[/custom_headline]\r\n\r\n[icon_list] [icon_list_item type=\"angle-double-right\"]Instruction by an excellent and interesting expert, with hands-on exercises[/icon_list_item] [icon_list_item type=\"angle-double-right\"]Use of a laptop with CUDA and OpenCL capable GPUs and CPUs [/icon_list_item] [icon_list_item type=\"angle-double-right\"]Choice of Linux or Windows operating system[/icon_list_item][icon_list_item type=\"angle-double-right\"]Printed manual of lecture material[/icon_list_item][icon_list_item type=\"angle-double-right\"]Electronic copy of programming exercises[/icon_list_item][/icon_list]\r\n\r\n[/column][column type=\"one-half\" last=\"true\" class=\"center-text\"][gap size=\"15%\"][button type=\"flat\" shape=\"rounded\" block=\"true\" size=\"large\" href=\"mailto:sales@arrayfire.com?Subject=Training%20Inquiry\"] Register for Training[/button]\r\n\r\n[/column]\r\n\r\n&nbsp;\r\n\r\n[line]\r\n\r\n[custom_headline type=\"left\" level=\"h3\"]TRAINING COURSE SYLLABUS[/custom_headline] [column type=\"one-half\"] [custom_headline type=\"left\" level=\"h5\"  accent=\"false\"]Day 1, Introduction[/custom_headline] Lectures:\r\nGPU Computing Overview\r\nThe Programming Model\r\nBasic Dataset Mapping Techniques\r\nLibraries, ArrayFire\r\nProfiling Tools\r\n\r\nPractice:\r\nA Simple Kernel\r\nEquivalent ArrayFire Example\r\nUsing Libraries\r\nMonte Carlo Pi Estimation\r\nTiming and ArrayFire\r\nDebugging Code\r\n\r\n[custom_headline type=\"left\" level=\"h5\" accent=\"false\"]Day 3, Multi-GPU[/custom_headline] Lectures:\r\nMulti-GPU Use Cases\r\nMulti-GPUs: Contexts\r\nExisting Libraries\r\nScaling Across Multiple GPUs\r\n\r\nPractice:\r\nOut of Core Problems: Matrix Multiply\r\nTask Level Parallelism: Optimization\r\nArrayFire Multi-GPU[/column]\r\n\r\n[column type=\"one-half\" last=\"true\"] [custom_headline type=\"left\" level=\"h5\" accent=\"false\"]Day 2, Optimization[/custom_headline] Lectures:\r\nArchitecture: Grids, Blocks, and Threads\r\nMemory Model: Global, Shared, and Constant Memory\r\nAdvanced Mapping Techniques\r\nStreams: Asynchronos Launches and Concurrent Execution\r\nArrayFire: Lazy Evaluation and Code Vectorization\r\n\r\nPractice:\r\nMatrix Transpose\r\nOptimization Using Shared Memory\r\nMedian Filter\r\nOptimization Using Constant Memory\r\nStream Example\r\nArrayFire Example: Nearest Neighbor Algorithm\r\n\r\n[custom_headline type=\"left\" level=\"h5\" accent=\"false\"]Day 4, Algorithm Problems[/custom_headline]\r\n\r\nLectures and Practice:\r\nReductions\r\nScan Algorithms\r\nSort\r\nConvolution\r\nCustomer-Specific Problem [/column][/content_band]','Training, OPSC','','inherit','open','open','','6640-revision-v1','','','2014-09-18 21:36:56','2014-09-18 21:36:56','',6640,'http://sandbox.arrayfire.com/6640-revision-v1/',0,'revision','',0),(6743,7,'2014-09-18 21:37:49','2014-09-18 21:37:49','[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\" ]\r\n\r\n[column type=\"one-half\"][custom_headline type=\"left\" level=\"h3\" ]TRAINING[/custom_headline] Attendees will receive the latest industry knowledge and techniques for GPU computing in CUDA and OpenCL.\r\n\r\nArrayFire offers up to four days of specialized GPU training in CUDA and OpenCL programming. We provide customized on-site training courses for customers that have 3 or more attendees from their organization. We can travel to your location and provide a CUDA or OpenCL training course tailored to meet your application-specific needs. We also offer individual trainings at our Atlanta office. Please <a href=\"mailto:sales@arrayfire.com\">contact us</a> to learn more about our training offerings.\r\n\r\nWe recommend that attendees have a working knowledge of C/C++ in order to gain the most from the training courses. For more information about the content of our training lectures and hands-on practicums, please see the following training course syllabus.\r\n\r\n[/column]\r\n\r\n[column type=\"one-half\" last=\"true\" fade=\"true\" fade_animation=\"in-from-right\" class=\"center-text\"][gap size=\"15%\"][image src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/02/Professor-AF_Exported-e1395436350363.png\" ][/column]\r\n\r\n[column type=\"one-half\"]\r\n\r\n[custom_headline type=\"left\" level=\"h3\"]INCLUDED IN COURSE[/custom_headline]\r\n\r\n[icon_list] [icon_list_item type=\"angle-double-right\"]Instruction by an excellent and interesting expert, with hands-on exercises[/icon_list_item] [icon_list_item type=\"angle-double-right\"]Use of a laptop with CUDA and OpenCL capable GPUs and CPUs [/icon_list_item] [icon_list_item type=\"angle-double-right\"]Choice of Linux or Windows operating system[/icon_list_item][icon_list_item type=\"angle-double-right\"]Printed manual of lecture material[/icon_list_item][icon_list_item type=\"angle-double-right\"]Electronic copy of programming exercises[/icon_list_item][/icon_list]\r\n\r\n[/column][column type=\"one-half\" last=\"true\" class=\"center-text\"][gap size=\"20%\"][button type=\"flat\" shape=\"rounded\" block=\"true\" size=\"large\" href=\"mailto:sales@arrayfire.com?Subject=Training%20Inquiry\"] Register for Training[/button]\r\n\r\n[/column]\r\n\r\n&nbsp;\r\n\r\n[line]\r\n\r\n[custom_headline type=\"left\" level=\"h3\"]TRAINING COURSE SYLLABUS[/custom_headline] [column type=\"one-half\"] [custom_headline type=\"left\" level=\"h5\"  accent=\"false\"]Day 1, Introduction[/custom_headline] Lectures:\r\nGPU Computing Overview\r\nThe Programming Model\r\nBasic Dataset Mapping Techniques\r\nLibraries, ArrayFire\r\nProfiling Tools\r\n\r\nPractice:\r\nA Simple Kernel\r\nEquivalent ArrayFire Example\r\nUsing Libraries\r\nMonte Carlo Pi Estimation\r\nTiming and ArrayFire\r\nDebugging Code\r\n\r\n[custom_headline type=\"left\" level=\"h5\" accent=\"false\"]Day 3, Multi-GPU[/custom_headline] Lectures:\r\nMulti-GPU Use Cases\r\nMulti-GPUs: Contexts\r\nExisting Libraries\r\nScaling Across Multiple GPUs\r\n\r\nPractice:\r\nOut of Core Problems: Matrix Multiply\r\nTask Level Parallelism: Optimization\r\nArrayFire Multi-GPU[/column]\r\n\r\n[column type=\"one-half\" last=\"true\"] [custom_headline type=\"left\" level=\"h5\" accent=\"false\"]Day 2, Optimization[/custom_headline] Lectures:\r\nArchitecture: Grids, Blocks, and Threads\r\nMemory Model: Global, Shared, and Constant Memory\r\nAdvanced Mapping Techniques\r\nStreams: Asynchronos Launches and Concurrent Execution\r\nArrayFire: Lazy Evaluation and Code Vectorization\r\n\r\nPractice:\r\nMatrix Transpose\r\nOptimization Using Shared Memory\r\nMedian Filter\r\nOptimization Using Constant Memory\r\nStream Example\r\nArrayFire Example: Nearest Neighbor Algorithm\r\n\r\n[custom_headline type=\"left\" level=\"h5\" accent=\"false\"]Day 4, Algorithm Problems[/custom_headline]\r\n\r\nLectures and Practice:\r\nReductions\r\nScan Algorithms\r\nSort\r\nConvolution\r\nCustomer-Specific Problem [/column][/content_band]','Training, OPSC','','inherit','open','open','','6640-revision-v1','','','2014-09-18 21:37:49','2014-09-18 21:37:49','',6640,'http://sandbox.arrayfire.com/6640-revision-v1/',0,'revision','',0),(6744,1,'2014-09-19 05:34:59','2014-09-19 05:34:59','','Order &ndash; September 19, 2014 @ 05:34 AM','','wc-completed','open','closed','order_541bc07d5e325','order-sep-19-2014-0534-am','','','2014-09-19 05:35:02','2014-09-19 05:35:02','',0,'http://sandbox.arrayfire.com/?post_type=shop_order&#038;p=6744',0,'shop_order','',4),(6749,17795,'2014-09-24 00:29:28','2014-09-24 00:29:28','[content_band inner_container=\"true\" no_margin=\"true\" border=\"none\" padding_top=\"40px\" padding_bottom=\"40px\"] [custom_headline type=\"center\" level=\"h3\"]Thanks. This is the part when you download ArrayFire.[/custom_headline][gap size=\"3%\"]\r\n\r\n<hr />\r\n\r\n&nbsp;\r\n\r\n[custom_headline type=\"left\" level=\"h3\"]Desktop Version[/custom_headline]\r\n\r\n[column type=\"one-third\"] [gap size=\"15%\"][button  type=\"flat\" shape=\"rounded\" href=\"http://sandbox.arrayfire.com/installer_archive/2.1/ArrayFire-win-2.1.exe\" size=\"x-large\" block=\"true\"]Windows, v2.1 (64-bit)[/button] [gap size=\"10%\"] [/column][column type=\"one-third\"][gap size=\"15%\"][button  type=\"flat\" block=\"true\" shape=\"rounded\" href=\"http://sandbox.arrayfire.com/installer_archive/2.1/ArrayFire-linux-2.1.run\" size=\"x-large\" download ]Linux, v2.1 (32 &amp; 64-bit)[/button][/column] [column type=\"one-third\" last=\"true\"][gap size=\"15%\"][button  type=\"flat\" block=\"true\" shape=\"rounded\" href=\"http://sandbox.arrayfire.com/installer_archive/2.1/ArrayFire-maci-2.1.zip\" size=\"x-large\" ]Mac OSX, v2.1 (64-bit)[/button][/column]\r\n\r\n[custom_headline type=\"left\" level=\"h3\"]Mobile/Embedded Version[/custom_headline]\r\n\r\n[column type=\"one-third\"] [gap size=\"15%\"][button  type=\"flat\" shape=\"rounded\" href=\"http://sandbox.arrayfire.com/installer_archive/2.1/ArrayFire-TegraK1-2.1.zip\" size=\"x-large\" block=\"true\"]Linux, v2.1 (CUDA on Tegra)[/button] [gap size=\"10%\"] [/column][column type=\"one-third\"][gap size=\"15%\"][/column] [column type=\"one-third\" last=\"true\"][gap size=\"15%\"][/column]\r\n\r\nLooking for an old installer? <a href=\"http://sandbox.arrayfire.com/installer_archive\">Try here</a>.\r\n\r\nEach download contains both CUDA and OpenCL files. Your license file will determine your features, e.g. Free, Developer, Pro, or Enterprise.\r\n\r\n&nbsp;\r\n\r\n<hr />\r\n\r\n&nbsp;\r\n\r\n[custom_headline type=\"center\" level=\"h3\"]Awesome. Don\'t forget to check out our <span style=\"color: #ff6600;\"><a href=\"http://www.sandbox.arrayfire.com/docs/\"><span style=\"color: #ff6600;\">documentation</span></a></span> and <a href=\"http://forums.accelereyes.com/\"><span style=\"color: #ff6600;\">forums</span></a>![/custom_headline]\r\n<div id=\"arrayfire\">[gap size=\"3%\"]</div>\r\n[/content_band]','Download','','inherit','open','open','','5931-autosave-v1','','','2014-09-24 00:29:28','2014-09-24 00:29:28','',5931,'http://sandbox.arrayfire.com/5931-autosave-v1/',0,'revision','',0),(6750,17795,'2014-09-24 00:24:24','2014-09-24 00:24:24','[content_band inner_container=\"true\" no_margin=\"true\" border=\"none\" padding_top=\"40px\" padding_bottom=\"40px\"] [custom_headline type=\"center\" level=\"h3\"]Thanks. This is the part when you download ArrayFire.[/custom_headline][gap size=\"3%\"]\r\n\r\n<hr />\r\n\r\n&nbsp;\r\n\r\n[custom_headline type=\"left\" level=\"h3\"]Desktop Version[/custom_headline]\r\n\r\n[column type=\"one-third\"] [gap size=\"15%\"][button  type=\"flat\" shape=\"rounded\" href=\"http://sandbox.arrayfire.com/installer_archive/2.1/ArrayFire-win-2.1.exe\" size=\"x-large\" block=\"true\"]Windows, v2.1 (64-bit)[/button] [gap size=\"10%\"] [/column][column type=\"one-third\"][gap size=\"15%\"][button  type=\"flat\" block=\"true\" shape=\"rounded\" href=\"http://sandbox.arrayfire.com/installer_archive/2.1/ArrayFire-linux-2.1.run\" size=\"x-large\" ]Linux, v2.1 (32 &amp; 64-bit)[/button][/column] [column type=\"one-third\" last=\"true\"][gap size=\"15%\"][button  type=\"flat\" block=\"true\" shape=\"rounded\" href=\"http://sandbox.arrayfire.com/installer_archive/2.1/ArrayFire-maci-2.1.zip\" size=\"x-large\" ]Mac OSX, v2.1 (64-bit)[/button][/column]\r\n\r\n[custom_headline type=\"left\" level=\"h3\"]Mobile/Embedded Version[/custom_headline]\r\n\r\n[column type=\"one-third\"] [gap size=\"15%\"][button  type=\"flat\" shape=\"rounded\" href=\"http://sandbox.arrayfire.com/installer_archive/2.1/ArrayFire-TegraK1-2.1.zip\" size=\"x-large\" block=\"true\"]Linux, v2.1 (CUDA on Tegra)[/button] [gap size=\"10%\"] [/column][column type=\"one-third\"][gap size=\"15%\"][/column] [column type=\"one-third\" last=\"true\"][gap size=\"15%\"][/column]\r\n\r\nLooking for an old installer? <a href=\"http://sandbox.arrayfire.com/installer_archive\">Try here</a>.\r\n\r\nEach download contains both CUDA and OpenCL files. Your license file will determine your features, e.g. Free, Developer, Pro, or Enterprise.\r\n\r\n&nbsp;\r\n\r\n<hr />\r\n\r\n&nbsp;\r\n\r\n[custom_headline type=\"center\" level=\"h3\"]Awesome. Don\'t forget to check out our <span style=\"color: #ff6600;\"><a href=\"http://www.sandbox.arrayfire.com/docs/\"><span style=\"color: #ff6600;\">documentation</span></a></span> and <a href=\"http://forums.accelereyes.com/\"><span style=\"color: #ff6600;\">forums</span></a>![/custom_headline]\r\n<div id=\"arrayfire\">[gap size=\"3%\"]</div>\r\n[/content_band]','Download','','inherit','open','open','','5931-revision-v1','','','2014-09-24 00:24:24','2014-09-24 00:24:24','',5931,'http://sandbox.arrayfire.com/5931-revision-v1/',0,'revision','',0),(6757,11,'2014-10-03 15:28:10','2014-10-03 15:28:10','<h3>Introduction</h3>\r\nZero Copy access has been a part of the CUDA Toolkit for a long time (~2009). However, there were very few applications using the capability because, with time, GPU memory has become reasonably large. The only applications using zero copy access were mainly ones with extremely large memory requirements, such as database processing.\r\n\r\nZero Copy is a way to map host memory and access it directly over PCIe without doing an explicit memory transfer. It allows CUDA kernels to directly access host memory. Instead of reading data from global memory (limit ~200GB/s), data would be read over PCIe and be limited by PCIe bandwidth (upto 16GB/s). Hence there was no real performance advantage for most applications.\r\n\r\nHowever, with the release of the <a href=\"https://developer.nvidia.com/jetson-tk1\" title=\"Jetson TK1\" target=\"_blank\">Jetson TK1</a> board with the <a href=\"http://www.nvidia.com/object/tegra-k1-processor.html\" title=\"Tegra K1\" target=\"_blank\">Tegra K1</a> processor, this has become really useful. The Jetson board has 2GB of physical memory that is shared by the ARM CPU and the CUDA GPU. If a cudaMemcpy Host->Device is done on the Jetson, the memory is simply being copied to a new location on the same physical memory and retrieving a CUDA pointer from it. In such a scenario, Zero Copy access is perfect.\r\n\r\n<h3>Standard CUDA Pipeline</h3>\r\nLets take a look at a standard pipeline in CUDA.\r\n\r\n<pre>\r\n// Host Arrays\r\nfloat* h_in  = new float[sizeIn];\r\nfloat* h_out = new float[sizeOut];\r\n\r\n//Process h_in\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n\r\n// Allocate memory on the device\r\ncudaMalloc((void **) &d_in,  sizeIn ));\r\ncudaMalloc((void **) &d_out, sizeOut));\r\n\r\n// Copy array contents of input from the host (CPU) to the device (GPU)\r\ncudaMemcpy(d_in, h_in, sizeX * sizeY * sizeof(float), cudaMemcpyHostToDevice);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// Copy result back\r\ncudaMemcpy(h_out, d_out, sizeOut, cudaMemcpyDeviceToHost);\r\n\r\n// Continue processing on host using h_out\r\n\r\n</pre>\r\n\r\n<h3>Zero Copy Access CUDA Pipeline</h3>\r\nNow lets look at a pipeline using Zero Copy access.\r\n\r\n<pre>\r\n// Set flag to enable zero copy access\r\ncudaSetDeviceFlags(cudaDeviceMapHost);\r\n\r\n// Host Arrays\r\nfloat* h_in  = NULL;\r\nfloat* h_out = NULL;\r\n\r\n// Process h_in\r\n\r\n// Allocate host memory using CUDA allocation calls\r\ncudaHostAlloc((void **)&h_in,  sizeIn,  cudaHostAllocMapped);\r\ncudaHostAlloc((void **)&h_out, sizeOut, cudaHostAllocMapped);\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n// Get device pointer from host memory. No allocation or memcpy\r\ncudaHostGetDevicePointer((void **)&d_in,  (void *) h_in , 0);\r\ncudaHostGetDevicePointer((void **)&d_out, (void *) h_out, 0);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// No need to copy d_out back\r\n// Continue processing on host using h_out\r\n</pre>\r\n\r\nAs you can see, it is considerably simpler to use zero copy access on the Tegra from a coding perspective. The kernel code remains the same. You can also allocate host memory using the cudaHostAlloc call without using zero copy access. This allows the fast pinned memory transfer when doing a cudaMemcpy.\r\n\r\n<h3>Results</h3>\r\nSo how does it perform?\r\n\r\nWhen I ran a transpose kernel using zero copy vs using standard pipeline, I achieve a the following results for a 4096 x 4096 matrix. \r\n\r\n<table style=\"width:50%\">\r\n  <th>\r\n    <td>Pipeline</td>\r\n    <td>Bandwidth (GB/s)</td> \r\n    <td>Time (ms)</td>\r\n  </th>  \r\n  <tr>\r\n    <td></td>\r\n    <td>Standard Pipeline</td>\r\n    <td>3.0</td> \r\n    <td>45</td>\r\n  </tr>\r\n  <tr>\r\n    <td></td>\r\n    <td>Zero Copy Pipeline</td>\r\n    <td>5.8</td> \r\n    <td>23</td>\r\n  </tr>\r\n</table>\r\n\r\nThe bandwidth of a device-to-device copy kernel on the Tegra is ~6.6 GB/s.\r\n\r\nThe results are tremendous. Any result, that is as good as or better than the standard pipeline is worth doing using zero copy. This saves memory usage as it does not require duplication on host and device.\r\n\r\nThere a a few caveats though. If you are running multiple kernels without modifying the data on the host side, it may be wise to run the standard pipeline. There is no right answer. Since it is so simple to modify the code for memory transfer vs zero copy pointers, it would be best to run both techniques and benchmark them.\r\n\r\n<h3>Applications</h3>\r\nWe these results, we believe the Tegra K1 is a great option for streaming applications. Most streaming applications run image or signal processing algorithms, with a limitation of running at either 60 or 30 frames per second. On desktops, although the kernels themselves run well under the required time, the memory transfer times form a significant portion.\r\n\r\nThis is why the Tegra K1 can be great. We save 100% of the memory transfer times used by discrete GPUs by using Zero Copy access. This allow the Tegra K1 GPU to do the streaming operations within the performance constraints even though is is considerable under-powered compared to most desktop GPUs.\r\n\r\nOf course, streaming is not the only application. There are many more, and if do happen to try one or want to try your application on the K1, we would be extremely excited to hear about it.\r\n\r\n<h3>Code</h3>\r\nYou can find the entire code from my transpose exercise here: <a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/transpose.zip\">transpose</a>.\r\nNote: If you wish to run this on a x86 system, make sure you change the compute version in the makefile from 32 to 30 (for Kepler).','Zero Copy on Tegra K1','','publish','open','open','','zero-copy-on-tegra-k1','','','2014-10-07 15:00:46','2014-10-07 15:00:46','',0,'http://sandbox.arrayfire.com/?p=6757',0,'post','',8),(6759,11,'2014-10-02 22:07:12','2014-10-02 22:07:12','Zero Copy memory was introduced in CUDA quite a while back (~2009). However, there were very few takers for the capability because with time GPU memory has become reasonably large. The only takers were mainly folks with extremely large memory requirements, such as database processing.\r\n\r\nZero Copy memory is a way to map host memory and access it directly over PCIe without doing an explicit memory transfer. It is a way to allow CUDA kernels to directly access host memory. So instead of reading data from global memory (limit ~200GB/s), data would be read over PCIe and be limited by PCIe bandwidth (upto 16GB/s). Hence there was no real advantage.\r\n\r\nHowever, with the release of the Jetson TK1 which uses the Tegra K1 processor, this has become really useful. The Jetson board has 2GB of physical memory that is shared by host and the CUDA GPU. If a cudaMemcpy Host->Device is done on the Jetson, the memory is simply being copied to a new location on the same physical memory and retrieving a CUDA pointer from it.In such a scenario, Zero Copy memory is perfect.\r\n\r\nLets take a look at a standard pipeline in CUDA:\r\n\r\n<pre>\r\n// Host Arrays\r\nfloat* h_in  = new float[sizeIn];\r\nfloat* h_out = new float[sizeOut];\r\n\r\n//Process h_in\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n\r\n// Allocate memory on the device\r\ncudaMalloc((void **) &d_in,  sizeIn ));\r\ncudaMalloc((void **) &d_out, sizeOut));\r\n\r\n// Copy array contents of input from the host (CPU) to the device (GPU)\r\ncudaMemcpy(d_in, h_in, sizeX * sizeY * sizeof(float), cudaMemcpyHostToDevice);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// Copy result back\r\ncudaMemcpy(h_out, d_out, sizeOut, cudaMemcpyDeviceToHost);\r\n\r\n// Continue processing on host using h_out\r\n\r\n</pre>\r\n\r\nNow lets look at a pipeline using Zero Copy memory\r\n\r\n<pre>\r\n// Set flag to enable zero copy memory\r\ncudaSetDeviceFlags(cudaDeviceMapHost);\r\n\r\n// Host Arrays\r\nfloat* h_in  = NULL;\r\nfloat* h_out = NULL;\r\n\r\n// Process h_in\r\n\r\n// Allocate host memory using CUDA allocation calls\r\ncudaHostAlloc((void **)&h_in,  sizeIn,  cudaHostAllocMapped);\r\ncudaHostAlloc((void **)&h_out, sizeOut, cudaHostAllocMapped);\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n// Get device pointer from host memory. No allocation or memcpy\r\ncudaHostGetDevicePointer((void **)&d_a, (void *) a, 0);\r\ncudaHostGetDevicePointer((void **)&d_b, (void *) b, 0);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// No need to copy d_out back\r\n// Continue processing on host using h_out\r\n</pre>\r\n','Zero Copy on Tegra K1','','inherit','open','open','','6757-revision-v1','','','2014-10-02 22:07:12','2014-10-02 22:07:12','',6757,'http://sandbox.arrayfire.com/6757-revision-v1/',0,'revision','',0),(6761,11,'2014-10-02 22:34:16','2014-10-02 22:34:16','','transpose','','inherit','open','open','','transpose','','','2014-10-02 22:34:16','2014-10-02 22:34:16','',6757,'http://sandbox.arrayfire.com/wp-content/uploads/2014/10/transpose.zip',0,'attachment','application/zip',0),(6762,11,'2014-10-02 22:35:28','2014-10-02 22:35:28','Zero Copy memory was introduced in CUDA quite a while back (~2009). However, there were very few takers for the capability because with time GPU memory has become reasonably large. The only takers were mainly folks with extremely large memory requirements, such as database processing.\r\n\r\nZero Copy memory is a way to map host memory and access it directly over PCIe without doing an explicit memory transfer. It is a way to allow CUDA kernels to directly access host memory. So instead of reading data from global memory (limit ~200GB/s), data would be read over PCIe and be limited by PCIe bandwidth (upto 16GB/s). Hence there was no real advantage.\r\n\r\nHowever, with the release of the Jetson TK1 which uses the Tegra K1 processor, this has become really useful. The Jetson board has 2GB of physical memory that is shared by host and the CUDA GPU. If a cudaMemcpy Host->Device is done on the Jetson, the memory is simply being copied to a new location on the same physical memory and retrieving a CUDA pointer from it.In such a scenario, Zero Copy memory is perfect.\r\n\r\nLets take a look at a standard pipeline in CUDA:\r\n\r\n<pre>\r\n// Host Arrays\r\nfloat* h_in  = new float[sizeIn];\r\nfloat* h_out = new float[sizeOut];\r\n\r\n//Process h_in\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n\r\n// Allocate memory on the device\r\ncudaMalloc((void **) &d_in,  sizeIn ));\r\ncudaMalloc((void **) &d_out, sizeOut));\r\n\r\n// Copy array contents of input from the host (CPU) to the device (GPU)\r\ncudaMemcpy(d_in, h_in, sizeX * sizeY * sizeof(float), cudaMemcpyHostToDevice);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// Copy result back\r\ncudaMemcpy(h_out, d_out, sizeOut, cudaMemcpyDeviceToHost);\r\n\r\n// Continue processing on host using h_out\r\n\r\n</pre>\r\n\r\nNow lets look at a pipeline using Zero Copy memory\r\n\r\n<pre>\r\n// Set flag to enable zero copy memory\r\ncudaSetDeviceFlags(cudaDeviceMapHost);\r\n\r\n// Host Arrays\r\nfloat* h_in  = NULL;\r\nfloat* h_out = NULL;\r\n\r\n// Process h_in\r\n\r\n// Allocate host memory using CUDA allocation calls\r\ncudaHostAlloc((void **)&h_in,  sizeIn,  cudaHostAllocMapped);\r\ncudaHostAlloc((void **)&h_out, sizeOut, cudaHostAllocMapped);\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n// Get device pointer from host memory. No allocation or memcpy\r\ncudaHostGetDevicePointer((void **)&d_a, (void *) a, 0);\r\ncudaHostGetDevicePointer((void **)&d_b, (void *) b, 0);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// No need to copy d_out back\r\n// Continue processing on host using h_out\r\n</pre>\r\n\r\nAs you can see, it is considerably simpler to use zero copy memory on the Tegra from a coding perspective. The kernel code remains the same. You can also allocate host memory using the cudaHostAlloc call without using zero copy memory. This allows the fast pinned memory transfer when doing a cudaMemcpy.\r\n\r\nSo how does it perform?\r\n\r\nWhen I ran a transpose kernel using zero copy vs using standard pipeline, I achieve a bandwidth of 5.8 GB/s (23ms) vs 3.0 GB/s (45ms) for a 4096 x 4096 matrix. The bandwidth of a device-to-device copy kernel on the Tegra is ~6.6 GB/s.\r\n\r\nThe results are tremendous. There a a few caveats though. If you are running multiple kernels without modifying the data on the host side, it may be wise to run the standard pipeline. There is no right answer. Since it is so simple to modify the code for memory transfer vs zero copy pointers, it would be best to run both techniques and benchmark them.\r\n\r\nYou can find the entire code from my transpose exercise here<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/transpose.zip\">transpose</a>: \r\nNote: If you wish to run this on a x86 system, make sure you change the compute version in the makefile from 32 to 30 (for Kepler).','Zero Copy on Tegra K1','','inherit','open','open','','6757-revision-v1','','','2014-10-02 22:35:28','2014-10-02 22:35:28','',6757,'http://sandbox.arrayfire.com/6757-revision-v1/',0,'revision','',0),(6763,11,'2014-10-02 22:35:42','2014-10-02 22:35:42','Zero Copy memory was introduced in CUDA quite a while back (~2009). However, there were very few takers for the capability because with time GPU memory has become reasonably large. The only takers were mainly folks with extremely large memory requirements, such as database processing.\r\n\r\nZero Copy memory is a way to map host memory and access it directly over PCIe without doing an explicit memory transfer. It is a way to allow CUDA kernels to directly access host memory. So instead of reading data from global memory (limit ~200GB/s), data would be read over PCIe and be limited by PCIe bandwidth (upto 16GB/s). Hence there was no real advantage.\r\n\r\nHowever, with the release of the Jetson TK1 which uses the Tegra K1 processor, this has become really useful. The Jetson board has 2GB of physical memory that is shared by host and the CUDA GPU. If a cudaMemcpy Host->Device is done on the Jetson, the memory is simply being copied to a new location on the same physical memory and retrieving a CUDA pointer from it.In such a scenario, Zero Copy memory is perfect.\r\n\r\nLets take a look at a standard pipeline in CUDA:\r\n\r\n<pre>\r\n// Host Arrays\r\nfloat* h_in  = new float[sizeIn];\r\nfloat* h_out = new float[sizeOut];\r\n\r\n//Process h_in\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n\r\n// Allocate memory on the device\r\ncudaMalloc((void **) &d_in,  sizeIn ));\r\ncudaMalloc((void **) &d_out, sizeOut));\r\n\r\n// Copy array contents of input from the host (CPU) to the device (GPU)\r\ncudaMemcpy(d_in, h_in, sizeX * sizeY * sizeof(float), cudaMemcpyHostToDevice);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// Copy result back\r\ncudaMemcpy(h_out, d_out, sizeOut, cudaMemcpyDeviceToHost);\r\n\r\n// Continue processing on host using h_out\r\n\r\n</pre>\r\n\r\nNow lets look at a pipeline using Zero Copy memory\r\n\r\n<pre>\r\n// Set flag to enable zero copy memory\r\ncudaSetDeviceFlags(cudaDeviceMapHost);\r\n\r\n// Host Arrays\r\nfloat* h_in  = NULL;\r\nfloat* h_out = NULL;\r\n\r\n// Process h_in\r\n\r\n// Allocate host memory using CUDA allocation calls\r\ncudaHostAlloc((void **)&h_in,  sizeIn,  cudaHostAllocMapped);\r\ncudaHostAlloc((void **)&h_out, sizeOut, cudaHostAllocMapped);\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n// Get device pointer from host memory. No allocation or memcpy\r\ncudaHostGetDevicePointer((void **)&d_a, (void *) a, 0);\r\ncudaHostGetDevicePointer((void **)&d_b, (void *) b, 0);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// No need to copy d_out back\r\n// Continue processing on host using h_out\r\n</pre>\r\n\r\nAs you can see, it is considerably simpler to use zero copy memory on the Tegra from a coding perspective. The kernel code remains the same. You can also allocate host memory using the cudaHostAlloc call without using zero copy memory. This allows the fast pinned memory transfer when doing a cudaMemcpy.\r\n\r\nSo how does it perform?\r\n\r\nWhen I ran a transpose kernel using zero copy vs using standard pipeline, I achieve a bandwidth of 5.8 GB/s (23ms) vs 3.0 GB/s (45ms) for a 4096 x 4096 matrix. The bandwidth of a device-to-device copy kernel on the Tegra is ~6.6 GB/s.\r\n\r\nThe results are tremendous. There a a few caveats though. If you are running multiple kernels without modifying the data on the host side, it may be wise to run the standard pipeline. There is no right answer. Since it is so simple to modify the code for memory transfer vs zero copy pointers, it would be best to run both techniques and benchmark them.\r\n\r\nYou can find the entire code from my transpose exercise here: <a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/transpose.zip\">transpose</a>: \r\nNote: If you wish to run this on a x86 system, make sure you change the compute version in the makefile from 32 to 30 (for Kepler).','Zero Copy on Tegra K1','','inherit','open','open','','6757-revision-v1','','','2014-10-02 22:35:42','2014-10-02 22:35:42','',6757,'http://sandbox.arrayfire.com/6757-revision-v1/',0,'revision','',0),(6764,11,'2014-10-02 22:37:03','2014-10-02 22:37:03','<h3>Introduction</h3>\r\nZero Copy memory was introduced in CUDA quite a while back (~2009). However, there were very few takers for the capability because with time GPU memory has become reasonably large. The only takers were mainly folks with extremely large memory requirements, such as database processing.\r\n\r\nZero Copy memory is a way to map host memory and access it directly over PCIe without doing an explicit memory transfer. It is a way to allow CUDA kernels to directly access host memory. So instead of reading data from global memory (limit ~200GB/s), data would be read over PCIe and be limited by PCIe bandwidth (upto 16GB/s). Hence there was no real advantage.\r\n\r\nHowever, with the release of the Jetson TK1 which uses the Tegra K1 processor, this has become really useful. The Jetson board has 2GB of physical memory that is shared by host and the CUDA GPU. If a cudaMemcpy Host->Device is done on the Jetson, the memory is simply being copied to a new location on the same physical memory and retrieving a CUDA pointer from it.In such a scenario, Zero Copy memory is perfect.\r\n\r\n<h3>Standard CUDA Pipeline</h3>\r\nLets take a look at a standard pipeline in CUDA:\r\n\r\n<pre>\r\n// Host Arrays\r\nfloat* h_in  = new float[sizeIn];\r\nfloat* h_out = new float[sizeOut];\r\n\r\n//Process h_in\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n\r\n// Allocate memory on the device\r\ncudaMalloc((void **) &d_in,  sizeIn ));\r\ncudaMalloc((void **) &d_out, sizeOut));\r\n\r\n// Copy array contents of input from the host (CPU) to the device (GPU)\r\ncudaMemcpy(d_in, h_in, sizeX * sizeY * sizeof(float), cudaMemcpyHostToDevice);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// Copy result back\r\ncudaMemcpy(h_out, d_out, sizeOut, cudaMemcpyDeviceToHost);\r\n\r\n// Continue processing on host using h_out\r\n\r\n</pre>\r\n\r\n<h3>Zero Copy Memory CUDA Pipeline</h3>\r\nNow lets look at a pipeline using Zero Copy memory\r\n\r\n<pre>\r\n// Set flag to enable zero copy memory\r\ncudaSetDeviceFlags(cudaDeviceMapHost);\r\n\r\n// Host Arrays\r\nfloat* h_in  = NULL;\r\nfloat* h_out = NULL;\r\n\r\n// Process h_in\r\n\r\n// Allocate host memory using CUDA allocation calls\r\ncudaHostAlloc((void **)&h_in,  sizeIn,  cudaHostAllocMapped);\r\ncudaHostAlloc((void **)&h_out, sizeOut, cudaHostAllocMapped);\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n// Get device pointer from host memory. No allocation or memcpy\r\ncudaHostGetDevicePointer((void **)&d_a, (void *) a, 0);\r\ncudaHostGetDevicePointer((void **)&d_b, (void *) b, 0);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// No need to copy d_out back\r\n// Continue processing on host using h_out\r\n</pre>\r\n\r\nAs you can see, it is considerably simpler to use zero copy memory on the Tegra from a coding perspective. The kernel code remains the same. You can also allocate host memory using the cudaHostAlloc call without using zero copy memory. This allows the fast pinned memory transfer when doing a cudaMemcpy.\r\n\r\n<h3>Results</h3>\r\nSo how does it perform?\r\n\r\nWhen I ran a transpose kernel using zero copy vs using standard pipeline, I achieve a bandwidth of 5.8 GB/s (23ms) vs 3.0 GB/s (45ms) for a 4096 x 4096 matrix. The bandwidth of a device-to-device copy kernel on the Tegra is ~6.6 GB/s.\r\n\r\nThe results are tremendous. There a a few caveats though. If you are running multiple kernels without modifying the data on the host side, it may be wise to run the standard pipeline. There is no right answer. Since it is so simple to modify the code for memory transfer vs zero copy pointers, it would be best to run both techniques and benchmark them.\r\n\r\n<h3>Code</h3>\r\nYou can find the entire code from my transpose exercise here: <a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/transpose.zip\">transpose</a>: \r\nNote: If you wish to run this on a x86 system, make sure you change the compute version in the makefile from 32 to 30 (for Kepler).','Zero Copy on Tegra K1','','inherit','open','open','','6757-revision-v1','','','2014-10-02 22:37:03','2014-10-02 22:37:03','',6757,'http://sandbox.arrayfire.com/6757-revision-v1/',0,'revision','',0),(6765,11,'2014-10-02 22:40:59','2014-10-02 22:40:59','<h3>Introduction</h3>\r\nZero Copy memory was introduced in CUDA quite a while back (~2009). However, there were very few takers for the capability because with time GPU memory has become reasonably large. The only takers were mainly folks with extremely large memory requirements, such as database processing.\r\n\r\nZero Copy memory is a way to map host memory and access it directly over PCIe without doing an explicit memory transfer. It is a way to allow CUDA kernels to directly access host memory. So instead of reading data from global memory (limit ~200GB/s), data would be read over PCIe and be limited by PCIe bandwidth (upto 16GB/s). Hence there was no real advantage.\r\n\r\nHowever, with the release of the Jetson TK1 which uses the Tegra K1 processor, this has become really useful. The Jetson board has 2GB of physical memory that is shared by host and the CUDA GPU. If a cudaMemcpy Host->Device is done on the Jetson, the memory is simply being copied to a new location on the same physical memory and retrieving a CUDA pointer from it.In such a scenario, Zero Copy memory is perfect.\r\n\r\n<h3>Standard CUDA Pipeline</h3>\r\nLets take a look at a standard pipeline in CUDA:\r\n\r\n<pre>\r\n// Host Arrays\r\nfloat* h_in  = new float[sizeIn];\r\nfloat* h_out = new float[sizeOut];\r\n\r\n//Process h_in\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n\r\n// Allocate memory on the device\r\ncudaMalloc((void **) &d_in,  sizeIn ));\r\ncudaMalloc((void **) &d_out, sizeOut));\r\n\r\n// Copy array contents of input from the host (CPU) to the device (GPU)\r\ncudaMemcpy(d_in, h_in, sizeX * sizeY * sizeof(float), cudaMemcpyHostToDevice);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// Copy result back\r\ncudaMemcpy(h_out, d_out, sizeOut, cudaMemcpyDeviceToHost);\r\n\r\n// Continue processing on host using h_out\r\n\r\n</pre>\r\n\r\n<h3>Zero Copy Memory CUDA Pipeline</h3>\r\nNow lets look at a pipeline using Zero Copy memory\r\n\r\n<pre>\r\n// Set flag to enable zero copy memory\r\ncudaSetDeviceFlags(cudaDeviceMapHost);\r\n\r\n// Host Arrays\r\nfloat* h_in  = NULL;\r\nfloat* h_out = NULL;\r\n\r\n// Process h_in\r\n\r\n// Allocate host memory using CUDA allocation calls\r\ncudaHostAlloc((void **)&h_in,  sizeIn,  cudaHostAllocMapped);\r\ncudaHostAlloc((void **)&h_out, sizeOut, cudaHostAllocMapped);\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n// Get device pointer from host memory. No allocation or memcpy\r\ncudaHostGetDevicePointer((void **)&d_a, (void *) a, 0);\r\ncudaHostGetDevicePointer((void **)&d_b, (void *) b, 0);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// No need to copy d_out back\r\n// Continue processing on host using h_out\r\n</pre>\r\n\r\nAs you can see, it is considerably simpler to use zero copy memory on the Tegra from a coding perspective. The kernel code remains the same. You can also allocate host memory using the cudaHostAlloc call without using zero copy memory. This allows the fast pinned memory transfer when doing a cudaMemcpy.\r\n\r\n<h3>Results</h3>\r\nSo how does it perform?\r\n\r\nWhen I ran a transpose kernel using zero copy vs using standard pipeline, I achieve a bandwidth of 5.8 GB/s (23ms) for zero copy vs 3.0 GB/s (45ms) for a standard pipeline (4096 x 4096 matrix). The bandwidth of a device-to-device copy kernel on the Tegra is ~6.6 GB/s.\r\n\r\nThe results are tremendous. There a a few caveats though. If you are running multiple kernels without modifying the data on the host side, it may be wise to run the standard pipeline. There is no right answer. Since it is so simple to modify the code for memory transfer vs zero copy pointers, it would be best to run both techniques and benchmark them.\r\n\r\n<h3>Code</h3>\r\nYou can find the entire code from my transpose exercise here: <a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/transpose.zip\">transpose</a>: \r\nNote: If you wish to run this on a x86 system, make sure you change the compute version in the makefile from 32 to 30 (for Kepler).','Zero Copy on Tegra K1','','inherit','open','open','','6757-revision-v1','','','2014-10-02 22:40:59','2014-10-02 22:40:59','',6757,'http://sandbox.arrayfire.com/6757-revision-v1/',0,'revision','',0),(6766,11,'2014-10-02 22:42:24','2014-10-02 22:42:24','<h3>Introduction</h3>\r\nZero Copy memory was introduced in CUDA quite a while back (~2009). However, there were very few takers for the capability because with time GPU memory has become reasonably large. The only takers were mainly folks with extremely large memory requirements, such as database processing.\r\n\r\nZero Copy memory is a way to map host memory and access it directly over PCIe without doing an explicit memory transfer. It is a way to allow CUDA kernels to directly access host memory. So instead of reading data from global memory (limit ~200GB/s), data would be read over PCIe and be limited by PCIe bandwidth (upto 16GB/s). Hence there was no real advantage.\r\n\r\nHowever, with the release of the Jetson TK1 which uses the Tegra K1 processor, this has become really useful. The Jetson board has 2GB of physical memory that is shared by host and the CUDA GPU. If a cudaMemcpy Host->Device is done on the Jetson, the memory is simply being copied to a new location on the same physical memory and retrieving a CUDA pointer from it.In such a scenario, Zero Copy memory is perfect.\r\n\r\n<h3>Standard CUDA Pipeline</h3>\r\nLets take a look at a standard pipeline in CUDA:\r\n\r\n<pre>\r\n// Host Arrays\r\nfloat* h_in  = new float[sizeIn];\r\nfloat* h_out = new float[sizeOut];\r\n\r\n//Process h_in\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n\r\n// Allocate memory on the device\r\ncudaMalloc((void **) &d_in,  sizeIn ));\r\ncudaMalloc((void **) &d_out, sizeOut));\r\n\r\n// Copy array contents of input from the host (CPU) to the device (GPU)\r\ncudaMemcpy(d_in, h_in, sizeX * sizeY * sizeof(float), cudaMemcpyHostToDevice);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// Copy result back\r\ncudaMemcpy(h_out, d_out, sizeOut, cudaMemcpyDeviceToHost);\r\n\r\n// Continue processing on host using h_out\r\n\r\n</pre>\r\n\r\n<h3>Zero Copy Memory CUDA Pipeline</h3>\r\nNow lets look at a pipeline using Zero Copy memory\r\n\r\n<pre>\r\n// Set flag to enable zero copy memory\r\ncudaSetDeviceFlags(cudaDeviceMapHost);\r\n\r\n// Host Arrays\r\nfloat* h_in  = NULL;\r\nfloat* h_out = NULL;\r\n\r\n// Process h_in\r\n\r\n// Allocate host memory using CUDA allocation calls\r\ncudaHostAlloc((void **)&h_in,  sizeIn,  cudaHostAllocMapped);\r\ncudaHostAlloc((void **)&h_out, sizeOut, cudaHostAllocMapped);\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n// Get device pointer from host memory. No allocation or memcpy\r\ncudaHostGetDevicePointer((void **)&d_a, (void *) a, 0);\r\ncudaHostGetDevicePointer((void **)&d_b, (void *) b, 0);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// No need to copy d_out back\r\n// Continue processing on host using h_out\r\n</pre>\r\n\r\nAs you can see, it is considerably simpler to use zero copy memory on the Tegra from a coding perspective. The kernel code remains the same. You can also allocate host memory using the cudaHostAlloc call without using zero copy memory. This allows the fast pinned memory transfer when doing a cudaMemcpy.\r\n\r\n<h3>Results</h3>\r\nSo how does it perform?\r\n\r\nWhen I ran a transpose kernel using zero copy vs using standard pipeline, I achieve a <strong>bandwidth of 5.8 GB/s (23ms) for zero copy vs 3.0 GB/s (45ms) for a standard pipeline</strong> (4096 x 4096 matrix). The bandwidth of a device-to-device copy kernel on the Tegra is ~6.6 GB/s.\r\n\r\nThe results are tremendous. There a a few caveats though. If you are running multiple kernels without modifying the data on the host side, it may be wise to run the standard pipeline. There is no right answer. Since it is so simple to modify the code for memory transfer vs zero copy pointers, it would be best to run both techniques and benchmark them.\r\n\r\n<h3>Code</h3>\r\nYou can find the entire code from my transpose exercise here: <a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/transpose.zip\">transpose</a>: \r\nNote: If you wish to run this on a x86 system, make sure you change the compute version in the makefile from 32 to 30 (for Kepler).','Zero Copy on Tegra K1','','inherit','open','open','','6757-revision-v1','','','2014-10-02 22:42:24','2014-10-02 22:42:24','',6757,'http://sandbox.arrayfire.com/6757-revision-v1/',0,'revision','',0),(6767,11,'2014-10-02 22:42:49','2014-10-02 22:42:49','<h3>Introduction</h3>\r\nZero Copy memory was introduced in CUDA quite a while back (~2009). However, there were very few takers for the capability because with time GPU memory has become reasonably large. The only takers were mainly folks with extremely large memory requirements, such as database processing.\r\n\r\nZero Copy memory is a way to map host memory and access it directly over PCIe without doing an explicit memory transfer. It is a way to allow CUDA kernels to directly access host memory. So instead of reading data from global memory (limit ~200GB/s), data would be read over PCIe and be limited by PCIe bandwidth (upto 16GB/s). Hence there was no real advantage.\r\n\r\nHowever, with the release of the Jetson TK1 which uses the Tegra K1 processor, this has become really useful. The Jetson board has 2GB of physical memory that is shared by host and the CUDA GPU. If a cudaMemcpy Host->Device is done on the Jetson, the memory is simply being copied to a new location on the same physical memory and retrieving a CUDA pointer from it.In such a scenario, Zero Copy memory is perfect.\r\n\r\n<h3>Standard CUDA Pipeline</h3>\r\nLets take a look at a standard pipeline in CUDA:\r\n\r\n<pre>\r\n// Host Arrays\r\nfloat* h_in  = new float[sizeIn];\r\nfloat* h_out = new float[sizeOut];\r\n\r\n//Process h_in\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n\r\n// Allocate memory on the device\r\ncudaMalloc((void **) &d_in,  sizeIn ));\r\ncudaMalloc((void **) &d_out, sizeOut));\r\n\r\n// Copy array contents of input from the host (CPU) to the device (GPU)\r\ncudaMemcpy(d_in, h_in, sizeX * sizeY * sizeof(float), cudaMemcpyHostToDevice);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// Copy result back\r\ncudaMemcpy(h_out, d_out, sizeOut, cudaMemcpyDeviceToHost);\r\n\r\n// Continue processing on host using h_out\r\n\r\n</pre>\r\n\r\n<h3>Zero Copy Memory CUDA Pipeline</h3>\r\nNow lets look at a pipeline using Zero Copy memory\r\n\r\n<pre>\r\n// Set flag to enable zero copy memory\r\ncudaSetDeviceFlags(cudaDeviceMapHost);\r\n\r\n// Host Arrays\r\nfloat* h_in  = NULL;\r\nfloat* h_out = NULL;\r\n\r\n// Process h_in\r\n\r\n// Allocate host memory using CUDA allocation calls\r\ncudaHostAlloc((void **)&h_in,  sizeIn,  cudaHostAllocMapped);\r\ncudaHostAlloc((void **)&h_out, sizeOut, cudaHostAllocMapped);\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n// Get device pointer from host memory. No allocation or memcpy\r\ncudaHostGetDevicePointer((void **)&d_a, (void *) a, 0);\r\ncudaHostGetDevicePointer((void **)&d_b, (void *) b, 0);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// No need to copy d_out back\r\n// Continue processing on host using h_out\r\n</pre>\r\n\r\nAs you can see, it is considerably simpler to use zero copy memory on the Tegra from a coding perspective. The kernel code remains the same. You can also allocate host memory using the cudaHostAlloc call without using zero copy memory. This allows the fast pinned memory transfer when doing a cudaMemcpy.\r\n\r\n<h3>Results</h3>\r\nSo how does it perform?\r\n\r\nWhen I ran a transpose kernel using zero copy vs using standard pipeline, I achieve a <strong>bandwidth of 5.8 GB/s (23ms) for zero copy vs 3.0 GB/s (45ms) for a standard pipeline</strong> (4096 x 4096 matrix). The bandwidth of a device-to-device copy kernel on the Tegra is ~6.6 GB/s.\r\n\r\nThe results are tremendous. There a a few caveats though. If you are running multiple kernels without modifying the data on the host side, it may be wise to run the standard pipeline. There is no right answer. Since it is so simple to modify the code for memory transfer vs zero copy pointers, it would be best to run both techniques and benchmark them.\r\n\r\n<h3>Code</h3>\r\nYou can find the entire code from my transpose exercise here: <a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/transpose.zip\">transpose</a>.\r\nNote: If you wish to run this on a x86 system, make sure you change the compute version in the makefile from 32 to 30 (for Kepler).','Zero Copy on Tegra K1','','inherit','open','open','','6757-revision-v1','','','2014-10-02 22:42:49','2014-10-02 22:42:49','',6757,'http://sandbox.arrayfire.com/6757-revision-v1/',0,'revision','',0),(6769,11,'2014-10-02 22:53:41','2014-10-02 22:53:41','<h3>Introduction</h3>\r\nZero Copy memory was introduced in CUDA quite a while back (~2009). However, there were very few takers for the capability because with time GPU memory has become reasonably large. The only takers were mainly folks with extremely large memory requirements, such as database processing.\r\n\r\nZero Copy memory is a way to map host memory and access it directly over PCIe without doing an explicit memory transfer. It is a way to allow CUDA kernels to directly access host memory. So instead of reading data from global memory (limit ~200GB/s), data would be read over PCIe and be limited by PCIe bandwidth (upto 16GB/s). Hence there was no real advantage.\r\n\r\nHowever, with the release of the Jetson TK1 which uses the Tegra K1 processor, this has become really useful. The Jetson board has 2GB of physical memory that is shared by host and the CUDA GPU. If a cudaMemcpy Host->Device is done on the Jetson, the memory is simply being copied to a new location on the same physical memory and retrieving a CUDA pointer from it. In such a scenario, Zero Copy memory is perfect.\r\n\r\n<h3>Standard CUDA Pipeline</h3>\r\nLets take a look at a standard pipeline in CUDA:\r\n\r\n<pre>\r\n// Host Arrays\r\nfloat* h_in  = new float[sizeIn];\r\nfloat* h_out = new float[sizeOut];\r\n\r\n//Process h_in\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n\r\n// Allocate memory on the device\r\ncudaMalloc((void **) &d_in,  sizeIn ));\r\ncudaMalloc((void **) &d_out, sizeOut));\r\n\r\n// Copy array contents of input from the host (CPU) to the device (GPU)\r\ncudaMemcpy(d_in, h_in, sizeX * sizeY * sizeof(float), cudaMemcpyHostToDevice);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// Copy result back\r\ncudaMemcpy(h_out, d_out, sizeOut, cudaMemcpyDeviceToHost);\r\n\r\n// Continue processing on host using h_out\r\n\r\n</pre>\r\n\r\n<h3>Zero Copy Memory CUDA Pipeline</h3>\r\nNow lets look at a pipeline using Zero Copy memory\r\n\r\n<pre>\r\n// Set flag to enable zero copy memory\r\ncudaSetDeviceFlags(cudaDeviceMapHost);\r\n\r\n// Host Arrays\r\nfloat* h_in  = NULL;\r\nfloat* h_out = NULL;\r\n\r\n// Process h_in\r\n\r\n// Allocate host memory using CUDA allocation calls\r\ncudaHostAlloc((void **)&h_in,  sizeIn,  cudaHostAllocMapped);\r\ncudaHostAlloc((void **)&h_out, sizeOut, cudaHostAllocMapped);\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n// Get device pointer from host memory. No allocation or memcpy\r\ncudaHostGetDevicePointer((void **)&d_a, (void *) a, 0);\r\ncudaHostGetDevicePointer((void **)&d_b, (void *) b, 0);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// No need to copy d_out back\r\n// Continue processing on host using h_out\r\n</pre>\r\n\r\nAs you can see, it is considerably simpler to use zero copy memory on the Tegra from a coding perspective. The kernel code remains the same. You can also allocate host memory using the cudaHostAlloc call without using zero copy memory. This allows the fast pinned memory transfer when doing a cudaMemcpy.\r\n\r\n<h3>Results</h3>\r\nSo how does it perform?\r\n\r\nWhen I ran a transpose kernel using zero copy vs using standard pipeline, I achieve a <strong>bandwidth of 5.8 GB/s (23ms) for zero copy vs 3.0 GB/s (45ms) for a standard pipeline</strong> (4096 x 4096 matrix). The bandwidth of a device-to-device copy kernel on the Tegra is ~6.6 GB/s.\r\n\r\nThe results are tremendous. There a a few caveats though. If you are running multiple kernels without modifying the data on the host side, it may be wise to run the standard pipeline. There is no right answer. Since it is so simple to modify the code for memory transfer vs zero copy pointers, it would be best to run both techniques and benchmark them.\r\n\r\n<h3>Code</h3>\r\nYou can find the entire code from my transpose exercise here: <a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/transpose.zip\">transpose</a>.\r\nNote: If you wish to run this on a x86 system, make sure you change the compute version in the makefile from 32 to 30 (for Kepler).','Zero Copy on Tegra K1','','inherit','open','open','','6757-revision-v1','','','2014-10-02 22:53:41','2014-10-02 22:53:41','',6757,'http://sandbox.arrayfire.com/6757-revision-v1/',0,'revision','',0),(6771,11,'2014-10-03 14:47:37','2014-10-03 14:47:37','<h3>Introduction</h3>\r\nZero Copy memory was introduced in CUDA a quite a while back (~2009). However, there were very few takers for the capability because with time GPU memory has become reasonably large. The only takers were mainly folks with extremely large memory requirements, such as database processing.\r\n\r\nZero Copy is a way to map host memory and access it directly over PCIe without doing an explicit memory transfer. It allows CUDA kernels to directly access host memory. So instead of reading data from global memory (limit ~200GB/s), data would be read over PCIe and be limited by PCIe bandwidth (upto 16GB/s). Hence there was no real advantage.\r\n\r\nHowever, with the release of the Jetson TK1 which uses the Tegra K1 processor, this has become really useful. The Jetson board has 2GB of physical memory that is shared by host and the CUDA GPU. If a cudaMemcpy Host->Device is done on the Jetson, the memory is simply being copied to a new location on the same physical memory and retrieving a CUDA pointer from it. In such a scenario, Zero Copy memory is perfect.\r\n\r\n<h3>Standard CUDA Pipeline</h3>\r\nLets take a look at a standard pipeline in CUDA:\r\n\r\n<pre>\r\n// Host Arrays\r\nfloat* h_in  = new float[sizeIn];\r\nfloat* h_out = new float[sizeOut];\r\n\r\n//Process h_in\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n\r\n// Allocate memory on the device\r\ncudaMalloc((void **) &d_in,  sizeIn ));\r\ncudaMalloc((void **) &d_out, sizeOut));\r\n\r\n// Copy array contents of input from the host (CPU) to the device (GPU)\r\ncudaMemcpy(d_in, h_in, sizeX * sizeY * sizeof(float), cudaMemcpyHostToDevice);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// Copy result back\r\ncudaMemcpy(h_out, d_out, sizeOut, cudaMemcpyDeviceToHost);\r\n\r\n// Continue processing on host using h_out\r\n\r\n</pre>\r\n\r\n<h3>Zero Copy Memory CUDA Pipeline</h3>\r\nNow lets look at a pipeline using Zero Copy memory\r\n\r\n<pre>\r\n// Set flag to enable zero copy memory\r\ncudaSetDeviceFlags(cudaDeviceMapHost);\r\n\r\n// Host Arrays\r\nfloat* h_in  = NULL;\r\nfloat* h_out = NULL;\r\n\r\n// Process h_in\r\n\r\n// Allocate host memory using CUDA allocation calls\r\ncudaHostAlloc((void **)&h_in,  sizeIn,  cudaHostAllocMapped);\r\ncudaHostAlloc((void **)&h_out, sizeOut, cudaHostAllocMapped);\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n// Get device pointer from host memory. No allocation or memcpy\r\ncudaHostGetDevicePointer((void **)&d_a, (void *) a, 0);\r\ncudaHostGetDevicePointer((void **)&d_b, (void *) b, 0);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// No need to copy d_out back\r\n// Continue processing on host using h_out\r\n</pre>\r\n\r\nAs you can see, it is considerably simpler to use zero copy memory on the Tegra from a coding perspective. The kernel code remains the same. You can also allocate host memory using the cudaHostAlloc call without using zero copy memory. This allows the fast pinned memory transfer when doing a cudaMemcpy.\r\n\r\n<h3>Results and Applications</h3>\r\nSo how does it perform?\r\n\r\nWhen I ran a transpose kernel using zero copy vs using standard pipeline, I achieve a <strong>bandwidth of 5.8 GB/s (23ms) for zero copy vs 3.0 GB/s (45ms) for a standard pipeline</strong> (4096 x 4096 matrix). \r\n\r\n<table style=\"width:50%\">\r\n  <tr>\r\n    <td>Standard Pipeline</td>\r\n    <td>3.0</td> \r\n    <td>45</td>\r\n  </tr>\r\n  <tr>\r\n    <td>Zero Copy Pipeline</td>\r\n    <td>5.8</td> \r\n    <td>23</td>\r\n  </tr>\r\n</table>\r\n\r\nThe bandwidth of a device-to-device copy kernel on the Tegra is ~6.6 GB/s.\r\n\r\nThe results are tremendous. There a a few caveats though. If you are running multiple kernels without modifying the data on the host side, it may be wise to run the standard pipeline. There is no right answer. Since it is so simple to modify the code for memory transfer vs zero copy pointers, it would be best to run both techniques and benchmark them.\r\n\r\n<h3>Code</h3>\r\nYou can find the entire code from my transpose exercise here: <a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/transpose.zip\">transpose</a>.\r\nNote: If you wish to run this on a x86 system, make sure you change the compute version in the makefile from 32 to 30 (for Kepler).','Zero Copy on Tegra K1','','inherit','open','open','','6757-revision-v1','','','2014-10-03 14:47:37','2014-10-03 14:47:37','',6757,'http://sandbox.arrayfire.com/6757-revision-v1/',0,'revision','',0),(6772,11,'2014-10-03 14:49:17','2014-10-03 14:49:17','<h3>Introduction</h3>\r\nZero Copy memory was introduced in CUDA a quite a while back (~2009). However, there were very few takers for the capability because with time GPU memory has become reasonably large. The only takers were mainly folks with extremely large memory requirements, such as database processing.\r\n\r\nZero Copy is a way to map host memory and access it directly over PCIe without doing an explicit memory transfer. It allows CUDA kernels to directly access host memory. So instead of reading data from global memory (limit ~200GB/s), data would be read over PCIe and be limited by PCIe bandwidth (upto 16GB/s). Hence there was no real advantage.\r\n\r\nHowever, with the release of the Jetson TK1 which uses the Tegra K1 processor, this has become really useful. The Jetson board has 2GB of physical memory that is shared by host and the CUDA GPU. If a cudaMemcpy Host->Device is done on the Jetson, the memory is simply being copied to a new location on the same physical memory and retrieving a CUDA pointer from it. In such a scenario, Zero Copy memory is perfect.\r\n\r\n<h3>Standard CUDA Pipeline</h3>\r\nLets take a look at a standard pipeline in CUDA:\r\n\r\n<pre>\r\n// Host Arrays\r\nfloat* h_in  = new float[sizeIn];\r\nfloat* h_out = new float[sizeOut];\r\n\r\n//Process h_in\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n\r\n// Allocate memory on the device\r\ncudaMalloc((void **) &d_in,  sizeIn ));\r\ncudaMalloc((void **) &d_out, sizeOut));\r\n\r\n// Copy array contents of input from the host (CPU) to the device (GPU)\r\ncudaMemcpy(d_in, h_in, sizeX * sizeY * sizeof(float), cudaMemcpyHostToDevice);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// Copy result back\r\ncudaMemcpy(h_out, d_out, sizeOut, cudaMemcpyDeviceToHost);\r\n\r\n// Continue processing on host using h_out\r\n\r\n</pre>\r\n\r\n<h3>Zero Copy Memory CUDA Pipeline</h3>\r\nNow lets look at a pipeline using Zero Copy memory\r\n\r\n<pre>\r\n// Set flag to enable zero copy memory\r\ncudaSetDeviceFlags(cudaDeviceMapHost);\r\n\r\n// Host Arrays\r\nfloat* h_in  = NULL;\r\nfloat* h_out = NULL;\r\n\r\n// Process h_in\r\n\r\n// Allocate host memory using CUDA allocation calls\r\ncudaHostAlloc((void **)&h_in,  sizeIn,  cudaHostAllocMapped);\r\ncudaHostAlloc((void **)&h_out, sizeOut, cudaHostAllocMapped);\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n// Get device pointer from host memory. No allocation or memcpy\r\ncudaHostGetDevicePointer((void **)&d_a, (void *) a, 0);\r\ncudaHostGetDevicePointer((void **)&d_b, (void *) b, 0);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// No need to copy d_out back\r\n// Continue processing on host using h_out\r\n</pre>\r\n\r\nAs you can see, it is considerably simpler to use zero copy memory on the Tegra from a coding perspective. The kernel code remains the same. You can also allocate host memory using the cudaHostAlloc call without using zero copy memory. This allows the fast pinned memory transfer when doing a cudaMemcpy.\r\n\r\n<h3>Results and Applications</h3>\r\nSo how does it perform?\r\n\r\nWhen I ran a transpose kernel using zero copy vs using standard pipeline, I achieve a <strong>bandwidth of 5.8 GB/s (23ms) for zero copy vs 3.0 GB/s (45ms) for a standard pipeline</strong> (4096 x 4096 matrix). \r\n\r\n<table style=\"width:50%\">\r\n  <th>\r\n    <td>Pipeline</td>\r\n    <td>Bandwidth (GB/s)</td> \r\n    <td>Time (ms)</td>\r\n  </th>  \r\n  <tr>\r\n    <td>Standard Pipeline</td>\r\n    <td>3.0</td> \r\n    <td>45</td>\r\n  </tr>\r\n  <tr>\r\n    <td>Zero Copy Pipeline</td>\r\n    <td>5.8</td> \r\n    <td>23</td>\r\n  </tr>\r\n</table>\r\n\r\nThe bandwidth of a device-to-device copy kernel on the Tegra is ~6.6 GB/s.\r\n\r\nThe results are tremendous. There a a few caveats though. If you are running multiple kernels without modifying the data on the host side, it may be wise to run the standard pipeline. There is no right answer. Since it is so simple to modify the code for memory transfer vs zero copy pointers, it would be best to run both techniques and benchmark them.\r\n\r\n<h3>Code</h3>\r\nYou can find the entire code from my transpose exercise here: <a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/transpose.zip\">transpose</a>.\r\nNote: If you wish to run this on a x86 system, make sure you change the compute version in the makefile from 32 to 30 (for Kepler).','Zero Copy on Tegra K1','','inherit','open','open','','6757-revision-v1','','','2014-10-03 14:49:17','2014-10-03 14:49:17','',6757,'http://sandbox.arrayfire.com/6757-revision-v1/',0,'revision','',0),(6773,11,'2014-10-03 14:50:01','2014-10-03 14:50:01','<h3>Introduction</h3>\r\nZero Copy memory was introduced in CUDA a quite a while back (~2009). However, there were very few takers for the capability because with time GPU memory has become reasonably large. The only takers were mainly folks with extremely large memory requirements, such as database processing.\r\n\r\nZero Copy is a way to map host memory and access it directly over PCIe without doing an explicit memory transfer. It allows CUDA kernels to directly access host memory. So instead of reading data from global memory (limit ~200GB/s), data would be read over PCIe and be limited by PCIe bandwidth (upto 16GB/s). Hence there was no real advantage.\r\n\r\nHowever, with the release of the Jetson TK1 which uses the Tegra K1 processor, this has become really useful. The Jetson board has 2GB of physical memory that is shared by host and the CUDA GPU. If a cudaMemcpy Host->Device is done on the Jetson, the memory is simply being copied to a new location on the same physical memory and retrieving a CUDA pointer from it. In such a scenario, Zero Copy memory is perfect.\r\n\r\n<h3>Standard CUDA Pipeline</h3>\r\nLets take a look at a standard pipeline in CUDA:\r\n\r\n<pre>\r\n// Host Arrays\r\nfloat* h_in  = new float[sizeIn];\r\nfloat* h_out = new float[sizeOut];\r\n\r\n//Process h_in\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n\r\n// Allocate memory on the device\r\ncudaMalloc((void **) &d_in,  sizeIn ));\r\ncudaMalloc((void **) &d_out, sizeOut));\r\n\r\n// Copy array contents of input from the host (CPU) to the device (GPU)\r\ncudaMemcpy(d_in, h_in, sizeX * sizeY * sizeof(float), cudaMemcpyHostToDevice);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// Copy result back\r\ncudaMemcpy(h_out, d_out, sizeOut, cudaMemcpyDeviceToHost);\r\n\r\n// Continue processing on host using h_out\r\n\r\n</pre>\r\n\r\n<h3>Zero Copy Memory CUDA Pipeline</h3>\r\nNow lets look at a pipeline using Zero Copy memory\r\n\r\n<pre>\r\n// Set flag to enable zero copy memory\r\ncudaSetDeviceFlags(cudaDeviceMapHost);\r\n\r\n// Host Arrays\r\nfloat* h_in  = NULL;\r\nfloat* h_out = NULL;\r\n\r\n// Process h_in\r\n\r\n// Allocate host memory using CUDA allocation calls\r\ncudaHostAlloc((void **)&h_in,  sizeIn,  cudaHostAllocMapped);\r\ncudaHostAlloc((void **)&h_out, sizeOut, cudaHostAllocMapped);\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n// Get device pointer from host memory. No allocation or memcpy\r\ncudaHostGetDevicePointer((void **)&d_a, (void *) a, 0);\r\ncudaHostGetDevicePointer((void **)&d_b, (void *) b, 0);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// No need to copy d_out back\r\n// Continue processing on host using h_out\r\n</pre>\r\n\r\nAs you can see, it is considerably simpler to use zero copy memory on the Tegra from a coding perspective. The kernel code remains the same. You can also allocate host memory using the cudaHostAlloc call without using zero copy memory. This allows the fast pinned memory transfer when doing a cudaMemcpy.\r\n\r\n<h3>Results and Applications</h3>\r\nSo how does it perform?\r\n\r\nWhen I ran a transpose kernel using zero copy vs using standard pipeline, I achieve a <strong>bandwidth of 5.8 GB/s (23ms) for zero copy vs 3.0 GB/s (45ms) for a standard pipeline</strong> (4096 x 4096 matrix). \r\n\r\n<table style=\"width:50%\">\r\n  <th>\r\n    <td>Pipeline</td>\r\n    <td>Bandwidth (GB/s)</td> \r\n    <td>Time (ms)</td>\r\n  </th>  \r\n  <tr>\r\n    <td></td>\r\n    <td>Standard Pipeline</td>\r\n    <td>3.0</td> \r\n    <td>45</td>\r\n  </tr>\r\n  <tr>\r\n    <td></td>\r\n    <td>Zero Copy Pipeline</td>\r\n    <td>5.8</td> \r\n    <td>23</td>\r\n  </tr>\r\n</table>\r\n\r\nThe bandwidth of a device-to-device copy kernel on the Tegra is ~6.6 GB/s.\r\n\r\nThe results are tremendous. There a a few caveats though. If you are running multiple kernels without modifying the data on the host side, it may be wise to run the standard pipeline. There is no right answer. Since it is so simple to modify the code for memory transfer vs zero copy pointers, it would be best to run both techniques and benchmark them.\r\n\r\n<h3>Code</h3>\r\nYou can find the entire code from my transpose exercise here: <a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/transpose.zip\">transpose</a>.\r\nNote: If you wish to run this on a x86 system, make sure you change the compute version in the makefile from 32 to 30 (for Kepler).','Zero Copy on Tegra K1','','inherit','open','open','','6757-revision-v1','','','2014-10-03 14:50:01','2014-10-03 14:50:01','',6757,'http://sandbox.arrayfire.com/6757-revision-v1/',0,'revision','',0),(6774,11,'2014-10-03 15:13:48','2014-10-03 15:13:48','<h3>Introduction</h3>\r\nZero Copy memory was introduced in CUDA a quite a while back (~2009). However, there were very few takers for the capability because with time GPU memory has become reasonably large. The only takers were mainly folks with extremely large memory requirements, such as database processing.\r\n\r\nZero Copy is a way to map host memory and access it directly over PCIe without doing an explicit memory transfer. It allows CUDA kernels to directly access host memory. So instead of reading data from global memory (limit ~200GB/s), data would be read over PCIe and be limited by PCIe bandwidth (upto 16GB/s). Hence there was no real advantage.\r\n\r\nHowever, with the release of the Jetson TK1 which uses the Tegra K1 processor, this has become really useful. The Jetson board has 2GB of physical memory that is shared by host and the CUDA GPU. If a cudaMemcpy Host->Device is done on the Jetson, the memory is simply being copied to a new location on the same physical memory and retrieving a CUDA pointer from it. In such a scenario, Zero Copy memory is perfect.\r\n\r\n<h3>Standard CUDA Pipeline</h3>\r\nLets take a look at a standard pipeline in CUDA:\r\n\r\n<pre>\r\n// Host Arrays\r\nfloat* h_in  = new float[sizeIn];\r\nfloat* h_out = new float[sizeOut];\r\n\r\n//Process h_in\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n\r\n// Allocate memory on the device\r\ncudaMalloc((void **) &d_in,  sizeIn ));\r\ncudaMalloc((void **) &d_out, sizeOut));\r\n\r\n// Copy array contents of input from the host (CPU) to the device (GPU)\r\ncudaMemcpy(d_in, h_in, sizeX * sizeY * sizeof(float), cudaMemcpyHostToDevice);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// Copy result back\r\ncudaMemcpy(h_out, d_out, sizeOut, cudaMemcpyDeviceToHost);\r\n\r\n// Continue processing on host using h_out\r\n\r\n</pre>\r\n\r\n<h3>Zero Copy Memory CUDA Pipeline</h3>\r\nNow lets look at a pipeline using Zero Copy memory\r\n\r\n<pre>\r\n// Set flag to enable zero copy memory\r\ncudaSetDeviceFlags(cudaDeviceMapHost);\r\n\r\n// Host Arrays\r\nfloat* h_in  = NULL;\r\nfloat* h_out = NULL;\r\n\r\n// Process h_in\r\n\r\n// Allocate host memory using CUDA allocation calls\r\ncudaHostAlloc((void **)&h_in,  sizeIn,  cudaHostAllocMapped);\r\ncudaHostAlloc((void **)&h_out, sizeOut, cudaHostAllocMapped);\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n// Get device pointer from host memory. No allocation or memcpy\r\ncudaHostGetDevicePointer((void **)&d_a, (void *) a, 0);\r\ncudaHostGetDevicePointer((void **)&d_b, (void *) b, 0);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// No need to copy d_out back\r\n// Continue processing on host using h_out\r\n</pre>\r\n\r\nAs you can see, it is considerably simpler to use zero copy memory on the Tegra from a coding perspective. The kernel code remains the same. You can also allocate host memory using the cudaHostAlloc call without using zero copy memory. This allows the fast pinned memory transfer when doing a cudaMemcpy.\r\n\r\n<h3>Results</h3>\r\nSo how does it perform?\r\n\r\nWhen I ran a transpose kernel using zero copy vs using standard pipeline, I achieve a the following results for a 4096 x 4096 matrix. \r\n\r\n<table style=\"width:50%\">\r\n  <th>\r\n    <td>Pipeline</td>\r\n    <td>Bandwidth (GB/s)</td> \r\n    <td>Time (ms)</td>\r\n  </th>  \r\n  <tr>\r\n    <td></td>\r\n    <td>Standard Pipeline</td>\r\n    <td>3.0</td> \r\n    <td>45</td>\r\n  </tr>\r\n  <tr>\r\n    <td></td>\r\n    <td>Zero Copy Pipeline</td>\r\n    <td>5.8</td> \r\n    <td>23</td>\r\n  </tr>\r\n</table>\r\n\r\nThe bandwidth of a device-to-device copy kernel on the Tegra is ~6.6 GB/s.\r\n\r\nThe results are tremendous. Any result, that is as good as or better than the standard pipeline is worth doing using zero copy. This saves memory usage as it does not require duplication on host and device.\r\n\r\nThere a a few caveats though. If you are running multiple kernels without modifying the data on the host side, it may be wise to run the standard pipeline. There is no right answer. Since it is so simple to modify the code for memory transfer vs zero copy pointers, it would be best to run both techniques and benchmark them.\r\n\r\n<h3>Applications</h3>\r\nWe these results, we believe the Tegra K1 is a great option for streaming applications. Most streaming applications run image or signal processing algorithms, with a limitation of running at either 60 or 30 frames per second. On desktops, although the kernels themselves run well under the required time, the memory transfer times form a significant portion.\r\n\r\nThis is why the Tegra K1 can be great. We save 100% of the memory transfer times used by discrete GPUs by using Zero Copy memory. This allow the Tegra K1 GPU to do the streaming operations within the performance constraints even though is is considerable under-powered compared to most desktop GPUs.\r\n\r\nOf course, streaming is not the only application. There are many more, and if do happen to try one or want to try your application on the K1, we would be extremely excited to hear about it.\r\n\r\n<h3>Code</h3>\r\nYou can find the entire code from my transpose exercise here: <a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/transpose.zip\">transpose</a>.\r\nNote: If you wish to run this on a x86 system, make sure you change the compute version in the makefile from 32 to 30 (for Kepler).','Zero Copy on Tegra K1','','inherit','open','open','','6757-revision-v1','','','2014-10-03 15:13:48','2014-10-03 15:13:48','',6757,'http://sandbox.arrayfire.com/6757-revision-v1/',0,'revision','',0),(6775,11,'2014-10-03 15:16:57','2014-10-03 15:16:57','<h3>Introduction</h3>\r\nZero Copy memory was introduced in CUDA a quite a while back (~2009). However, there were very few takers for the capability because with time GPU memory has become reasonably large. The only takers were mainly folks with extremely large memory requirements, such as database processing.\r\n\r\nZero Copy is a way to map host memory and access it directly over PCIe without doing an explicit memory transfer. It allows CUDA kernels to directly access host memory. So instead of reading data from global memory (limit ~200GB/s), data would be read over PCIe and be limited by PCIe bandwidth (upto 16GB/s). Hence there was no real advantage.\r\n\r\nHowever, with the release of the Jetson TK1 which uses the Tegra K1 processor, this has become really useful. The Jetson board has 2GB of physical memory that is shared by host and the CUDA GPU. If a cudaMemcpy Host->Device is done on the Jetson, the memory is simply being copied to a new location on the same physical memory and retrieving a CUDA pointer from it. In such a scenario, Zero Copy memory is perfect.\r\n\r\n<h3>Standard CUDA Pipeline</h3>\r\nLets take a look at a standard pipeline in CUDA.\r\n\r\n<pre>\r\n// Host Arrays\r\nfloat* h_in  = new float[sizeIn];\r\nfloat* h_out = new float[sizeOut];\r\n\r\n//Process h_in\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n\r\n// Allocate memory on the device\r\ncudaMalloc((void **) &d_in,  sizeIn ));\r\ncudaMalloc((void **) &d_out, sizeOut));\r\n\r\n// Copy array contents of input from the host (CPU) to the device (GPU)\r\ncudaMemcpy(d_in, h_in, sizeX * sizeY * sizeof(float), cudaMemcpyHostToDevice);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// Copy result back\r\ncudaMemcpy(h_out, d_out, sizeOut, cudaMemcpyDeviceToHost);\r\n\r\n// Continue processing on host using h_out\r\n\r\n</pre>\r\n\r\n<h3>Zero Copy Memory CUDA Pipeline</h3>\r\nNow lets look at a pipeline using Zero Copy memory.\r\n\r\n<pre>\r\n// Set flag to enable zero copy memory\r\ncudaSetDeviceFlags(cudaDeviceMapHost);\r\n\r\n// Host Arrays\r\nfloat* h_in  = NULL;\r\nfloat* h_out = NULL;\r\n\r\n// Process h_in\r\n\r\n// Allocate host memory using CUDA allocation calls\r\ncudaHostAlloc((void **)&h_in,  sizeIn,  cudaHostAllocMapped);\r\ncudaHostAlloc((void **)&h_out, sizeOut, cudaHostAllocMapped);\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n// Get device pointer from host memory. No allocation or memcpy\r\ncudaHostGetDevicePointer((void **)&d_a, (void *) a, 0);\r\ncudaHostGetDevicePointer((void **)&d_b, (void *) b, 0);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// No need to copy d_out back\r\n// Continue processing on host using h_out\r\n</pre>\r\n\r\nAs you can see, it is considerably simpler to use zero copy memory on the Tegra from a coding perspective. The kernel code remains the same. You can also allocate host memory using the cudaHostAlloc call without using zero copy memory. This allows the fast pinned memory transfer when doing a cudaMemcpy.\r\n\r\n<h3>Results</h3>\r\nSo how does it perform?\r\n\r\nWhen I ran a transpose kernel using zero copy vs using standard pipeline, I achieve a the following results for a 4096 x 4096 matrix. \r\n\r\n<table style=\"width:50%\">\r\n  <th>\r\n    <td>Pipeline</td>\r\n    <td>Bandwidth (GB/s)</td> \r\n    <td>Time (ms)</td>\r\n  </th>  \r\n  <tr>\r\n    <td></td>\r\n    <td>Standard Pipeline</td>\r\n    <td>3.0</td> \r\n    <td>45</td>\r\n  </tr>\r\n  <tr>\r\n    <td></td>\r\n    <td>Zero Copy Pipeline</td>\r\n    <td>5.8</td> \r\n    <td>23</td>\r\n  </tr>\r\n</table>\r\n\r\nThe bandwidth of a device-to-device copy kernel on the Tegra is ~6.6 GB/s.\r\n\r\nThe results are tremendous. Any result, that is as good as or better than the standard pipeline is worth doing using zero copy. This saves memory usage as it does not require duplication on host and device.\r\n\r\nThere a a few caveats though. If you are running multiple kernels without modifying the data on the host side, it may be wise to run the standard pipeline. There is no right answer. Since it is so simple to modify the code for memory transfer vs zero copy pointers, it would be best to run both techniques and benchmark them.\r\n\r\n<h3>Applications</h3>\r\nWe these results, we believe the Tegra K1 is a great option for streaming applications. Most streaming applications run image or signal processing algorithms, with a limitation of running at either 60 or 30 frames per second. On desktops, although the kernels themselves run well under the required time, the memory transfer times form a significant portion.\r\n\r\nThis is why the Tegra K1 can be great. We save 100% of the memory transfer times used by discrete GPUs by using Zero Copy memory. This allow the Tegra K1 GPU to do the streaming operations within the performance constraints even though is is considerable under-powered compared to most desktop GPUs.\r\n\r\nOf course, streaming is not the only application. There are many more, and if do happen to try one or want to try your application on the K1, we would be extremely excited to hear about it.\r\n\r\n<h3>Code</h3>\r\nYou can find the entire code from my transpose exercise here: <a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/transpose.zip\">transpose</a>.\r\nNote: If you wish to run this on a x86 system, make sure you change the compute version in the makefile from 32 to 30 (for Kepler).','Zero Copy on Tegra K1','','inherit','open','open','','6757-revision-v1','','','2014-10-03 15:16:57','2014-10-03 15:16:57','',6757,'http://sandbox.arrayfire.com/6757-revision-v1/',0,'revision','',0),(6776,11,'2014-10-03 15:18:24','2014-10-03 15:18:24','<h3>Introduction</h3>\r\nZero Copy access was introduced in CUDA a quite a while back (~2009). However, there were very few takers for the capability because with time GPU memory has become reasonably large. The only takers were mainly folks with extremely large memory requirements, such as database processing.\r\n\r\nZero Copy is a way to map host memory and access it directly over PCIe without doing an explicit memory transfer. It allows CUDA kernels to directly access host memory. So instead of reading data from global memory (limit ~200GB/s), data would be read over PCIe and be limited by PCIe bandwidth (upto 16GB/s). Hence there was no real advantage.\r\n\r\nHowever, with the release of the Jetson TK1 which uses the Tegra K1 processor, this has become really useful. The Jetson board has 2GB of physical memory that is shared by host and the CUDA GPU. If a cudaMemcpy Host->Device is done on the Jetson, the memory is simply being copied to a new location on the same physical memory and retrieving a CUDA pointer from it. In such a scenario, Zero Copy memory is perfect.\r\n\r\n<h3>Standard CUDA Pipeline</h3>\r\nLets take a look at a standard pipeline in CUDA.\r\n\r\n<pre>\r\n// Host Arrays\r\nfloat* h_in  = new float[sizeIn];\r\nfloat* h_out = new float[sizeOut];\r\n\r\n//Process h_in\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n\r\n// Allocate memory on the device\r\ncudaMalloc((void **) &d_in,  sizeIn ));\r\ncudaMalloc((void **) &d_out, sizeOut));\r\n\r\n// Copy array contents of input from the host (CPU) to the device (GPU)\r\ncudaMemcpy(d_in, h_in, sizeX * sizeY * sizeof(float), cudaMemcpyHostToDevice);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// Copy result back\r\ncudaMemcpy(h_out, d_out, sizeOut, cudaMemcpyDeviceToHost);\r\n\r\n// Continue processing on host using h_out\r\n\r\n</pre>\r\n\r\n<h3>Zero Copy Memory CUDA Pipeline</h3>\r\nNow lets look at a pipeline using Zero Copy memory.\r\n\r\n<pre>\r\n// Set flag to enable zero copy memory\r\ncudaSetDeviceFlags(cudaDeviceMapHost);\r\n\r\n// Host Arrays\r\nfloat* h_in  = NULL;\r\nfloat* h_out = NULL;\r\n\r\n// Process h_in\r\n\r\n// Allocate host memory using CUDA allocation calls\r\ncudaHostAlloc((void **)&h_in,  sizeIn,  cudaHostAllocMapped);\r\ncudaHostAlloc((void **)&h_out, sizeOut, cudaHostAllocMapped);\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n// Get device pointer from host memory. No allocation or memcpy\r\ncudaHostGetDevicePointer((void **)&d_a, (void *) a, 0);\r\ncudaHostGetDevicePointer((void **)&d_b, (void *) b, 0);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// No need to copy d_out back\r\n// Continue processing on host using h_out\r\n</pre>\r\n\r\nAs you can see, it is considerably simpler to use zero copy memory on the Tegra from a coding perspective. The kernel code remains the same. You can also allocate host memory using the cudaHostAlloc call without using zero copy memory. This allows the fast pinned memory transfer when doing a cudaMemcpy.\r\n\r\n<h3>Results</h3>\r\nSo how does it perform?\r\n\r\nWhen I ran a transpose kernel using zero copy vs using standard pipeline, I achieve a the following results for a 4096 x 4096 matrix. \r\n\r\n<table style=\"width:50%\">\r\n  <th>\r\n    <td>Pipeline</td>\r\n    <td>Bandwidth (GB/s)</td> \r\n    <td>Time (ms)</td>\r\n  </th>  \r\n  <tr>\r\n    <td></td>\r\n    <td>Standard Pipeline</td>\r\n    <td>3.0</td> \r\n    <td>45</td>\r\n  </tr>\r\n  <tr>\r\n    <td></td>\r\n    <td>Zero Copy Pipeline</td>\r\n    <td>5.8</td> \r\n    <td>23</td>\r\n  </tr>\r\n</table>\r\n\r\nThe bandwidth of a device-to-device copy kernel on the Tegra is ~6.6 GB/s.\r\n\r\nThe results are tremendous. Any result, that is as good as or better than the standard pipeline is worth doing using zero copy. This saves memory usage as it does not require duplication on host and device.\r\n\r\nThere a a few caveats though. If you are running multiple kernels without modifying the data on the host side, it may be wise to run the standard pipeline. There is no right answer. Since it is so simple to modify the code for memory transfer vs zero copy pointers, it would be best to run both techniques and benchmark them.\r\n\r\n<h3>Applications</h3>\r\nWe these results, we believe the Tegra K1 is a great option for streaming applications. Most streaming applications run image or signal processing algorithms, with a limitation of running at either 60 or 30 frames per second. On desktops, although the kernels themselves run well under the required time, the memory transfer times form a significant portion.\r\n\r\nThis is why the Tegra K1 can be great. We save 100% of the memory transfer times used by discrete GPUs by using Zero Copy memory. This allow the Tegra K1 GPU to do the streaming operations within the performance constraints even though is is considerable under-powered compared to most desktop GPUs.\r\n\r\nOf course, streaming is not the only application. There are many more, and if do happen to try one or want to try your application on the K1, we would be extremely excited to hear about it.\r\n\r\n<h3>Code</h3>\r\nYou can find the entire code from my transpose exercise here: <a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/transpose.zip\">transpose</a>.\r\nNote: If you wish to run this on a x86 system, make sure you change the compute version in the makefile from 32 to 30 (for Kepler).','Zero Copy on Tegra K1','','inherit','open','open','','6757-revision-v1','','','2014-10-03 15:18:24','2014-10-03 15:18:24','',6757,'http://sandbox.arrayfire.com/6757-revision-v1/',0,'revision','',0),(6777,11,'2014-10-03 15:19:13','2014-10-03 15:19:13','<h3>Introduction</h3>\r\nZero Copy access was introduced in CUDA a quite a while back (~2009). However, there were very few takers for the capability because with time GPU memory has become reasonably large. The only takers were mainly folks with extremely large memory requirements, such as database processing.\r\n\r\nZero Copy is a way to map host memory and access it directly over PCIe without doing an explicit memory transfer. It allows CUDA kernels to directly access host memory. So instead of reading data from global memory (limit ~200GB/s), data would be read over PCIe and be limited by PCIe bandwidth (upto 16GB/s). Hence there was no real advantage.\r\n\r\nHowever, with the release of the Jetson TK1 which uses the Tegra K1 processor, this has become really useful. The Jetson board has 2GB of physical memory that is shared by host and the CUDA GPU. If a cudaMemcpy Host->Device is done on the Jetson, the memory is simply being copied to a new location on the same physical memory and retrieving a CUDA pointer from it. In such a scenario, Zero Copy access is perfect.\r\n\r\n<h3>Standard CUDA Pipeline</h3>\r\nLets take a look at a standard pipeline in CUDA.\r\n\r\n<pre>\r\n// Host Arrays\r\nfloat* h_in  = new float[sizeIn];\r\nfloat* h_out = new float[sizeOut];\r\n\r\n//Process h_in\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n\r\n// Allocate memory on the device\r\ncudaMalloc((void **) &d_in,  sizeIn ));\r\ncudaMalloc((void **) &d_out, sizeOut));\r\n\r\n// Copy array contents of input from the host (CPU) to the device (GPU)\r\ncudaMemcpy(d_in, h_in, sizeX * sizeY * sizeof(float), cudaMemcpyHostToDevice);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// Copy result back\r\ncudaMemcpy(h_out, d_out, sizeOut, cudaMemcpyDeviceToHost);\r\n\r\n// Continue processing on host using h_out\r\n\r\n</pre>\r\n\r\n<h3>Zero Copy Access CUDA Pipeline</h3>\r\nNow lets look at a pipeline using Zero Copy access.\r\n\r\n<pre>\r\n// Set flag to enable zero copy access\r\ncudaSetDeviceFlags(cudaDeviceMapHost);\r\n\r\n// Host Arrays\r\nfloat* h_in  = NULL;\r\nfloat* h_out = NULL;\r\n\r\n// Process h_in\r\n\r\n// Allocate host memory using CUDA allocation calls\r\ncudaHostAlloc((void **)&h_in,  sizeIn,  cudaHostAllocMapped);\r\ncudaHostAlloc((void **)&h_out, sizeOut, cudaHostAllocMapped);\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n// Get device pointer from host memory. No allocation or memcpy\r\ncudaHostGetDevicePointer((void **)&d_a, (void *) a, 0);\r\ncudaHostGetDevicePointer((void **)&d_b, (void *) b, 0);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// No need to copy d_out back\r\n// Continue processing on host using h_out\r\n</pre>\r\n\r\nAs you can see, it is considerably simpler to use zero copy access on the Tegra from a coding perspective. The kernel code remains the same. You can also allocate host memory using the cudaHostAlloc call without using zero copy access. This allows the fast pinned memory transfer when doing a cudaMemcpy.\r\n\r\n<h3>Results</h3>\r\nSo how does it perform?\r\n\r\nWhen I ran a transpose kernel using zero copy vs using standard pipeline, I achieve a the following results for a 4096 x 4096 matrix. \r\n\r\n<table style=\"width:50%\">\r\n  <th>\r\n    <td>Pipeline</td>\r\n    <td>Bandwidth (GB/s)</td> \r\n    <td>Time (ms)</td>\r\n  </th>  \r\n  <tr>\r\n    <td></td>\r\n    <td>Standard Pipeline</td>\r\n    <td>3.0</td> \r\n    <td>45</td>\r\n  </tr>\r\n  <tr>\r\n    <td></td>\r\n    <td>Zero Copy Pipeline</td>\r\n    <td>5.8</td> \r\n    <td>23</td>\r\n  </tr>\r\n</table>\r\n\r\nThe bandwidth of a device-to-device copy kernel on the Tegra is ~6.6 GB/s.\r\n\r\nThe results are tremendous. Any result, that is as good as or better than the standard pipeline is worth doing using zero copy. This saves memory usage as it does not require duplication on host and device.\r\n\r\nThere a a few caveats though. If you are running multiple kernels without modifying the data on the host side, it may be wise to run the standard pipeline. There is no right answer. Since it is so simple to modify the code for memory transfer vs zero copy pointers, it would be best to run both techniques and benchmark them.\r\n\r\n<h3>Applications</h3>\r\nWe these results, we believe the Tegra K1 is a great option for streaming applications. Most streaming applications run image or signal processing algorithms, with a limitation of running at either 60 or 30 frames per second. On desktops, although the kernels themselves run well under the required time, the memory transfer times form a significant portion.\r\n\r\nThis is why the Tegra K1 can be great. We save 100% of the memory transfer times used by discrete GPUs by using Zero Copy access. This allow the Tegra K1 GPU to do the streaming operations within the performance constraints even though is is considerable under-powered compared to most desktop GPUs.\r\n\r\nOf course, streaming is not the only application. There are many more, and if do happen to try one or want to try your application on the K1, we would be extremely excited to hear about it.\r\n\r\n<h3>Code</h3>\r\nYou can find the entire code from my transpose exercise here: <a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/transpose.zip\">transpose</a>.\r\nNote: If you wish to run this on a x86 system, make sure you change the compute version in the makefile from 32 to 30 (for Kepler).','Zero Copy on Tegra K1','','inherit','open','open','','6757-revision-v1','','','2014-10-03 15:19:13','2014-10-03 15:19:13','',6757,'http://sandbox.arrayfire.com/6757-revision-v1/',0,'revision','',0),(6778,11,'2014-10-03 15:20:11','2014-10-03 15:20:11','<h3>Introduction</h3>\r\nZero Copy access has been a part of the CUDA Toolkit for a long time (~2009). However, there were very few takers for the capability because with time GPU memory has become reasonably large. The only takers were mainly folks with extremely large memory requirements, such as database processing.\r\n\r\nZero Copy is a way to map host memory and access it directly over PCIe without doing an explicit memory transfer. It allows CUDA kernels to directly access host memory. So instead of reading data from global memory (limit ~200GB/s), data would be read over PCIe and be limited by PCIe bandwidth (upto 16GB/s). Hence there was no real advantage.\r\n\r\nHowever, with the release of the Jetson TK1 which uses the Tegra K1 processor, this has become really useful. The Jetson board has 2GB of physical memory that is shared by host and the CUDA GPU. If a cudaMemcpy Host->Device is done on the Jetson, the memory is simply being copied to a new location on the same physical memory and retrieving a CUDA pointer from it. In such a scenario, Zero Copy access is perfect.\r\n\r\n<h3>Standard CUDA Pipeline</h3>\r\nLets take a look at a standard pipeline in CUDA.\r\n\r\n<pre>\r\n// Host Arrays\r\nfloat* h_in  = new float[sizeIn];\r\nfloat* h_out = new float[sizeOut];\r\n\r\n//Process h_in\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n\r\n// Allocate memory on the device\r\ncudaMalloc((void **) &d_in,  sizeIn ));\r\ncudaMalloc((void **) &d_out, sizeOut));\r\n\r\n// Copy array contents of input from the host (CPU) to the device (GPU)\r\ncudaMemcpy(d_in, h_in, sizeX * sizeY * sizeof(float), cudaMemcpyHostToDevice);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// Copy result back\r\ncudaMemcpy(h_out, d_out, sizeOut, cudaMemcpyDeviceToHost);\r\n\r\n// Continue processing on host using h_out\r\n\r\n</pre>\r\n\r\n<h3>Zero Copy Access CUDA Pipeline</h3>\r\nNow lets look at a pipeline using Zero Copy access.\r\n\r\n<pre>\r\n// Set flag to enable zero copy access\r\ncudaSetDeviceFlags(cudaDeviceMapHost);\r\n\r\n// Host Arrays\r\nfloat* h_in  = NULL;\r\nfloat* h_out = NULL;\r\n\r\n// Process h_in\r\n\r\n// Allocate host memory using CUDA allocation calls\r\ncudaHostAlloc((void **)&h_in,  sizeIn,  cudaHostAllocMapped);\r\ncudaHostAlloc((void **)&h_out, sizeOut, cudaHostAllocMapped);\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n// Get device pointer from host memory. No allocation or memcpy\r\ncudaHostGetDevicePointer((void **)&d_a, (void *) a, 0);\r\ncudaHostGetDevicePointer((void **)&d_b, (void *) b, 0);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// No need to copy d_out back\r\n// Continue processing on host using h_out\r\n</pre>\r\n\r\nAs you can see, it is considerably simpler to use zero copy access on the Tegra from a coding perspective. The kernel code remains the same. You can also allocate host memory using the cudaHostAlloc call without using zero copy access. This allows the fast pinned memory transfer when doing a cudaMemcpy.\r\n\r\n<h3>Results</h3>\r\nSo how does it perform?\r\n\r\nWhen I ran a transpose kernel using zero copy vs using standard pipeline, I achieve a the following results for a 4096 x 4096 matrix. \r\n\r\n<table style=\"width:50%\">\r\n  <th>\r\n    <td>Pipeline</td>\r\n    <td>Bandwidth (GB/s)</td> \r\n    <td>Time (ms)</td>\r\n  </th>  \r\n  <tr>\r\n    <td></td>\r\n    <td>Standard Pipeline</td>\r\n    <td>3.0</td> \r\n    <td>45</td>\r\n  </tr>\r\n  <tr>\r\n    <td></td>\r\n    <td>Zero Copy Pipeline</td>\r\n    <td>5.8</td> \r\n    <td>23</td>\r\n  </tr>\r\n</table>\r\n\r\nThe bandwidth of a device-to-device copy kernel on the Tegra is ~6.6 GB/s.\r\n\r\nThe results are tremendous. Any result, that is as good as or better than the standard pipeline is worth doing using zero copy. This saves memory usage as it does not require duplication on host and device.\r\n\r\nThere a a few caveats though. If you are running multiple kernels without modifying the data on the host side, it may be wise to run the standard pipeline. There is no right answer. Since it is so simple to modify the code for memory transfer vs zero copy pointers, it would be best to run both techniques and benchmark them.\r\n\r\n<h3>Applications</h3>\r\nWe these results, we believe the Tegra K1 is a great option for streaming applications. Most streaming applications run image or signal processing algorithms, with a limitation of running at either 60 or 30 frames per second. On desktops, although the kernels themselves run well under the required time, the memory transfer times form a significant portion.\r\n\r\nThis is why the Tegra K1 can be great. We save 100% of the memory transfer times used by discrete GPUs by using Zero Copy access. This allow the Tegra K1 GPU to do the streaming operations within the performance constraints even though is is considerable under-powered compared to most desktop GPUs.\r\n\r\nOf course, streaming is not the only application. There are many more, and if do happen to try one or want to try your application on the K1, we would be extremely excited to hear about it.\r\n\r\n<h3>Code</h3>\r\nYou can find the entire code from my transpose exercise here: <a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/transpose.zip\">transpose</a>.\r\nNote: If you wish to run this on a x86 system, make sure you change the compute version in the makefile from 32 to 30 (for Kepler).','Zero Copy on Tegra K1','','inherit','open','open','','6757-revision-v1','','','2014-10-03 15:20:11','2014-10-03 15:20:11','',6757,'http://sandbox.arrayfire.com/6757-revision-v1/',0,'revision','',0),(6779,11,'2014-10-03 15:21:10','2014-10-03 15:21:10','<h3>Introduction</h3>\r\nZero Copy access has been a part of the CUDA Toolkit for a long time (~2009). However, there were very few applications using the capability because, with time, GPU memory has become reasonably large. The only applications using zero copy access were mainly ones with extremely large memory requirements, such as database processing.\r\n\r\nZero Copy is a way to map host memory and access it directly over PCIe without doing an explicit memory transfer. It allows CUDA kernels to directly access host memory. So instead of reading data from global memory (limit ~200GB/s), data would be read over PCIe and be limited by PCIe bandwidth (upto 16GB/s). Hence there was no real advantage.\r\n\r\nHowever, with the release of the Jetson TK1 which uses the Tegra K1 processor, this has become really useful. The Jetson board has 2GB of physical memory that is shared by host and the CUDA GPU. If a cudaMemcpy Host->Device is done on the Jetson, the memory is simply being copied to a new location on the same physical memory and retrieving a CUDA pointer from it. In such a scenario, Zero Copy access is perfect.\r\n\r\n<h3>Standard CUDA Pipeline</h3>\r\nLets take a look at a standard pipeline in CUDA.\r\n\r\n<pre>\r\n// Host Arrays\r\nfloat* h_in  = new float[sizeIn];\r\nfloat* h_out = new float[sizeOut];\r\n\r\n//Process h_in\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n\r\n// Allocate memory on the device\r\ncudaMalloc((void **) &d_in,  sizeIn ));\r\ncudaMalloc((void **) &d_out, sizeOut));\r\n\r\n// Copy array contents of input from the host (CPU) to the device (GPU)\r\ncudaMemcpy(d_in, h_in, sizeX * sizeY * sizeof(float), cudaMemcpyHostToDevice);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// Copy result back\r\ncudaMemcpy(h_out, d_out, sizeOut, cudaMemcpyDeviceToHost);\r\n\r\n// Continue processing on host using h_out\r\n\r\n</pre>\r\n\r\n<h3>Zero Copy Access CUDA Pipeline</h3>\r\nNow lets look at a pipeline using Zero Copy access.\r\n\r\n<pre>\r\n// Set flag to enable zero copy access\r\ncudaSetDeviceFlags(cudaDeviceMapHost);\r\n\r\n// Host Arrays\r\nfloat* h_in  = NULL;\r\nfloat* h_out = NULL;\r\n\r\n// Process h_in\r\n\r\n// Allocate host memory using CUDA allocation calls\r\ncudaHostAlloc((void **)&h_in,  sizeIn,  cudaHostAllocMapped);\r\ncudaHostAlloc((void **)&h_out, sizeOut, cudaHostAllocMapped);\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n// Get device pointer from host memory. No allocation or memcpy\r\ncudaHostGetDevicePointer((void **)&d_a, (void *) a, 0);\r\ncudaHostGetDevicePointer((void **)&d_b, (void *) b, 0);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// No need to copy d_out back\r\n// Continue processing on host using h_out\r\n</pre>\r\n\r\nAs you can see, it is considerably simpler to use zero copy access on the Tegra from a coding perspective. The kernel code remains the same. You can also allocate host memory using the cudaHostAlloc call without using zero copy access. This allows the fast pinned memory transfer when doing a cudaMemcpy.\r\n\r\n<h3>Results</h3>\r\nSo how does it perform?\r\n\r\nWhen I ran a transpose kernel using zero copy vs using standard pipeline, I achieve a the following results for a 4096 x 4096 matrix. \r\n\r\n<table style=\"width:50%\">\r\n  <th>\r\n    <td>Pipeline</td>\r\n    <td>Bandwidth (GB/s)</td> \r\n    <td>Time (ms)</td>\r\n  </th>  \r\n  <tr>\r\n    <td></td>\r\n    <td>Standard Pipeline</td>\r\n    <td>3.0</td> \r\n    <td>45</td>\r\n  </tr>\r\n  <tr>\r\n    <td></td>\r\n    <td>Zero Copy Pipeline</td>\r\n    <td>5.8</td> \r\n    <td>23</td>\r\n  </tr>\r\n</table>\r\n\r\nThe bandwidth of a device-to-device copy kernel on the Tegra is ~6.6 GB/s.\r\n\r\nThe results are tremendous. Any result, that is as good as or better than the standard pipeline is worth doing using zero copy. This saves memory usage as it does not require duplication on host and device.\r\n\r\nThere a a few caveats though. If you are running multiple kernels without modifying the data on the host side, it may be wise to run the standard pipeline. There is no right answer. Since it is so simple to modify the code for memory transfer vs zero copy pointers, it would be best to run both techniques and benchmark them.\r\n\r\n<h3>Applications</h3>\r\nWe these results, we believe the Tegra K1 is a great option for streaming applications. Most streaming applications run image or signal processing algorithms, with a limitation of running at either 60 or 30 frames per second. On desktops, although the kernels themselves run well under the required time, the memory transfer times form a significant portion.\r\n\r\nThis is why the Tegra K1 can be great. We save 100% of the memory transfer times used by discrete GPUs by using Zero Copy access. This allow the Tegra K1 GPU to do the streaming operations within the performance constraints even though is is considerable under-powered compared to most desktop GPUs.\r\n\r\nOf course, streaming is not the only application. There are many more, and if do happen to try one or want to try your application on the K1, we would be extremely excited to hear about it.\r\n\r\n<h3>Code</h3>\r\nYou can find the entire code from my transpose exercise here: <a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/transpose.zip\">transpose</a>.\r\nNote: If you wish to run this on a x86 system, make sure you change the compute version in the makefile from 32 to 30 (for Kepler).','Zero Copy on Tegra K1','','inherit','open','open','','6757-revision-v1','','','2014-10-03 15:21:10','2014-10-03 15:21:10','',6757,'http://sandbox.arrayfire.com/6757-revision-v1/',0,'revision','',0),(6780,11,'2014-10-03 15:21:59','2014-10-03 15:21:59','<h3>Introduction</h3>\r\nZero Copy access has been a part of the CUDA Toolkit for a long time (~2009). However, there were very few applications using the capability because, with time, GPU memory has become reasonably large. The only applications using zero copy access were mainly ones with extremely large memory requirements, such as database processing.\r\n\r\nZero Copy is a way to map host memory and access it directly over PCIe without doing an explicit memory transfer. It allows CUDA kernels to directly access host memory. Instead of reading data from global memory (limit ~200GB/s), data would be read over PCIe and be limited by PCIe bandwidth (upto 16GB/s). Hence there was no real performance advantage.\r\n\r\nHowever, with the release of the Jetson TK1 which uses the Tegra K1 processor, this has become really useful. The Jetson board has 2GB of physical memory that is shared by host and the CUDA GPU. If a cudaMemcpy Host->Device is done on the Jetson, the memory is simply being copied to a new location on the same physical memory and retrieving a CUDA pointer from it. In such a scenario, Zero Copy access is perfect.\r\n\r\n<h3>Standard CUDA Pipeline</h3>\r\nLets take a look at a standard pipeline in CUDA.\r\n\r\n<pre>\r\n// Host Arrays\r\nfloat* h_in  = new float[sizeIn];\r\nfloat* h_out = new float[sizeOut];\r\n\r\n//Process h_in\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n\r\n// Allocate memory on the device\r\ncudaMalloc((void **) &d_in,  sizeIn ));\r\ncudaMalloc((void **) &d_out, sizeOut));\r\n\r\n// Copy array contents of input from the host (CPU) to the device (GPU)\r\ncudaMemcpy(d_in, h_in, sizeX * sizeY * sizeof(float), cudaMemcpyHostToDevice);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// Copy result back\r\ncudaMemcpy(h_out, d_out, sizeOut, cudaMemcpyDeviceToHost);\r\n\r\n// Continue processing on host using h_out\r\n\r\n</pre>\r\n\r\n<h3>Zero Copy Access CUDA Pipeline</h3>\r\nNow lets look at a pipeline using Zero Copy access.\r\n\r\n<pre>\r\n// Set flag to enable zero copy access\r\ncudaSetDeviceFlags(cudaDeviceMapHost);\r\n\r\n// Host Arrays\r\nfloat* h_in  = NULL;\r\nfloat* h_out = NULL;\r\n\r\n// Process h_in\r\n\r\n// Allocate host memory using CUDA allocation calls\r\ncudaHostAlloc((void **)&h_in,  sizeIn,  cudaHostAllocMapped);\r\ncudaHostAlloc((void **)&h_out, sizeOut, cudaHostAllocMapped);\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n// Get device pointer from host memory. No allocation or memcpy\r\ncudaHostGetDevicePointer((void **)&d_a, (void *) a, 0);\r\ncudaHostGetDevicePointer((void **)&d_b, (void *) b, 0);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// No need to copy d_out back\r\n// Continue processing on host using h_out\r\n</pre>\r\n\r\nAs you can see, it is considerably simpler to use zero copy access on the Tegra from a coding perspective. The kernel code remains the same. You can also allocate host memory using the cudaHostAlloc call without using zero copy access. This allows the fast pinned memory transfer when doing a cudaMemcpy.\r\n\r\n<h3>Results</h3>\r\nSo how does it perform?\r\n\r\nWhen I ran a transpose kernel using zero copy vs using standard pipeline, I achieve a the following results for a 4096 x 4096 matrix. \r\n\r\n<table style=\"width:50%\">\r\n  <th>\r\n    <td>Pipeline</td>\r\n    <td>Bandwidth (GB/s)</td> \r\n    <td>Time (ms)</td>\r\n  </th>  \r\n  <tr>\r\n    <td></td>\r\n    <td>Standard Pipeline</td>\r\n    <td>3.0</td> \r\n    <td>45</td>\r\n  </tr>\r\n  <tr>\r\n    <td></td>\r\n    <td>Zero Copy Pipeline</td>\r\n    <td>5.8</td> \r\n    <td>23</td>\r\n  </tr>\r\n</table>\r\n\r\nThe bandwidth of a device-to-device copy kernel on the Tegra is ~6.6 GB/s.\r\n\r\nThe results are tremendous. Any result, that is as good as or better than the standard pipeline is worth doing using zero copy. This saves memory usage as it does not require duplication on host and device.\r\n\r\nThere a a few caveats though. If you are running multiple kernels without modifying the data on the host side, it may be wise to run the standard pipeline. There is no right answer. Since it is so simple to modify the code for memory transfer vs zero copy pointers, it would be best to run both techniques and benchmark them.\r\n\r\n<h3>Applications</h3>\r\nWe these results, we believe the Tegra K1 is a great option for streaming applications. Most streaming applications run image or signal processing algorithms, with a limitation of running at either 60 or 30 frames per second. On desktops, although the kernels themselves run well under the required time, the memory transfer times form a significant portion.\r\n\r\nThis is why the Tegra K1 can be great. We save 100% of the memory transfer times used by discrete GPUs by using Zero Copy access. This allow the Tegra K1 GPU to do the streaming operations within the performance constraints even though is is considerable under-powered compared to most desktop GPUs.\r\n\r\nOf course, streaming is not the only application. There are many more, and if do happen to try one or want to try your application on the K1, we would be extremely excited to hear about it.\r\n\r\n<h3>Code</h3>\r\nYou can find the entire code from my transpose exercise here: <a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/transpose.zip\">transpose</a>.\r\nNote: If you wish to run this on a x86 system, make sure you change the compute version in the makefile from 32 to 30 (for Kepler).','Zero Copy on Tegra K1','','inherit','open','open','','6757-revision-v1','','','2014-10-03 15:21:59','2014-10-03 15:21:59','',6757,'http://sandbox.arrayfire.com/6757-revision-v1/',0,'revision','',0),(6781,11,'2014-10-03 15:22:56','2014-10-03 15:22:56','<h3>Introduction</h3>\r\nZero Copy access has been a part of the CUDA Toolkit for a long time (~2009). However, there were very few applications using the capability because, with time, GPU memory has become reasonably large. The only applications using zero copy access were mainly ones with extremely large memory requirements, such as database processing.\r\n\r\nZero Copy is a way to map host memory and access it directly over PCIe without doing an explicit memory transfer. It allows CUDA kernels to directly access host memory. Instead of reading data from global memory (limit ~200GB/s), data would be read over PCIe and be limited by PCIe bandwidth (upto 16GB/s). Hence there was no real performance advantage for most applications.\r\n\r\nHowever, with the release of the Jetson TK1 which uses the Tegra K1 processor, this has become really useful. The Jetson board has 2GB of physical memory that is shared by host and the CUDA GPU. If a cudaMemcpy Host->Device is done on the Jetson, the memory is simply being copied to a new location on the same physical memory and retrieving a CUDA pointer from it. In such a scenario, Zero Copy access is perfect.\r\n\r\n<h3>Standard CUDA Pipeline</h3>\r\nLets take a look at a standard pipeline in CUDA.\r\n\r\n<pre>\r\n// Host Arrays\r\nfloat* h_in  = new float[sizeIn];\r\nfloat* h_out = new float[sizeOut];\r\n\r\n//Process h_in\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n\r\n// Allocate memory on the device\r\ncudaMalloc((void **) &d_in,  sizeIn ));\r\ncudaMalloc((void **) &d_out, sizeOut));\r\n\r\n// Copy array contents of input from the host (CPU) to the device (GPU)\r\ncudaMemcpy(d_in, h_in, sizeX * sizeY * sizeof(float), cudaMemcpyHostToDevice);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// Copy result back\r\ncudaMemcpy(h_out, d_out, sizeOut, cudaMemcpyDeviceToHost);\r\n\r\n// Continue processing on host using h_out\r\n\r\n</pre>\r\n\r\n<h3>Zero Copy Access CUDA Pipeline</h3>\r\nNow lets look at a pipeline using Zero Copy access.\r\n\r\n<pre>\r\n// Set flag to enable zero copy access\r\ncudaSetDeviceFlags(cudaDeviceMapHost);\r\n\r\n// Host Arrays\r\nfloat* h_in  = NULL;\r\nfloat* h_out = NULL;\r\n\r\n// Process h_in\r\n\r\n// Allocate host memory using CUDA allocation calls\r\ncudaHostAlloc((void **)&h_in,  sizeIn,  cudaHostAllocMapped);\r\ncudaHostAlloc((void **)&h_out, sizeOut, cudaHostAllocMapped);\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n// Get device pointer from host memory. No allocation or memcpy\r\ncudaHostGetDevicePointer((void **)&d_a, (void *) a, 0);\r\ncudaHostGetDevicePointer((void **)&d_b, (void *) b, 0);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// No need to copy d_out back\r\n// Continue processing on host using h_out\r\n</pre>\r\n\r\nAs you can see, it is considerably simpler to use zero copy access on the Tegra from a coding perspective. The kernel code remains the same. You can also allocate host memory using the cudaHostAlloc call without using zero copy access. This allows the fast pinned memory transfer when doing a cudaMemcpy.\r\n\r\n<h3>Results</h3>\r\nSo how does it perform?\r\n\r\nWhen I ran a transpose kernel using zero copy vs using standard pipeline, I achieve a the following results for a 4096 x 4096 matrix. \r\n\r\n<table style=\"width:50%\">\r\n  <th>\r\n    <td>Pipeline</td>\r\n    <td>Bandwidth (GB/s)</td> \r\n    <td>Time (ms)</td>\r\n  </th>  \r\n  <tr>\r\n    <td></td>\r\n    <td>Standard Pipeline</td>\r\n    <td>3.0</td> \r\n    <td>45</td>\r\n  </tr>\r\n  <tr>\r\n    <td></td>\r\n    <td>Zero Copy Pipeline</td>\r\n    <td>5.8</td> \r\n    <td>23</td>\r\n  </tr>\r\n</table>\r\n\r\nThe bandwidth of a device-to-device copy kernel on the Tegra is ~6.6 GB/s.\r\n\r\nThe results are tremendous. Any result, that is as good as or better than the standard pipeline is worth doing using zero copy. This saves memory usage as it does not require duplication on host and device.\r\n\r\nThere a a few caveats though. If you are running multiple kernels without modifying the data on the host side, it may be wise to run the standard pipeline. There is no right answer. Since it is so simple to modify the code for memory transfer vs zero copy pointers, it would be best to run both techniques and benchmark them.\r\n\r\n<h3>Applications</h3>\r\nWe these results, we believe the Tegra K1 is a great option for streaming applications. Most streaming applications run image or signal processing algorithms, with a limitation of running at either 60 or 30 frames per second. On desktops, although the kernels themselves run well under the required time, the memory transfer times form a significant portion.\r\n\r\nThis is why the Tegra K1 can be great. We save 100% of the memory transfer times used by discrete GPUs by using Zero Copy access. This allow the Tegra K1 GPU to do the streaming operations within the performance constraints even though is is considerable under-powered compared to most desktop GPUs.\r\n\r\nOf course, streaming is not the only application. There are many more, and if do happen to try one or want to try your application on the K1, we would be extremely excited to hear about it.\r\n\r\n<h3>Code</h3>\r\nYou can find the entire code from my transpose exercise here: <a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/transpose.zip\">transpose</a>.\r\nNote: If you wish to run this on a x86 system, make sure you change the compute version in the makefile from 32 to 30 (for Kepler).','Zero Copy on Tegra K1','','inherit','open','open','','6757-revision-v1','','','2014-10-03 15:22:56','2014-10-03 15:22:56','',6757,'http://sandbox.arrayfire.com/6757-revision-v1/',0,'revision','',0),(6782,11,'2014-10-03 15:25:36','2014-10-03 15:25:36','<h3>Introduction</h3>\r\nZero Copy access has been a part of the CUDA Toolkit for a long time (~2009). However, there were very few applications using the capability because, with time, GPU memory has become reasonably large. The only applications using zero copy access were mainly ones with extremely large memory requirements, such as database processing.\r\n\r\nZero Copy is a way to map host memory and access it directly over PCIe without doing an explicit memory transfer. It allows CUDA kernels to directly access host memory. Instead of reading data from global memory (limit ~200GB/s), data would be read over PCIe and be limited by PCIe bandwidth (upto 16GB/s). Hence there was no real performance advantage for most applications.\r\n\r\nHowever, with the release of the <a href=\"https://developer.nvidia.com/jetson-tk1\" title=\"Jetson TK1\" target=\"_blank\">Jetson TK1</a> board with the <a href=\"http://www.nvidia.com/object/tegra-k1-processor.html\" title=\"Tegra K1\" target=\"_blank\">Tegra K1</a> processor, this has become really useful. The Jetson board has 2GB of physical memory that is shared by host and the CUDA GPU. If a cudaMemcpy Host->Device is done on the Jetson, the memory is simply being copied to a new location on the same physical memory and retrieving a CUDA pointer from it. In such a scenario, Zero Copy access is perfect.\r\n\r\n<h3>Standard CUDA Pipeline</h3>\r\nLets take a look at a standard pipeline in CUDA.\r\n\r\n<pre>\r\n// Host Arrays\r\nfloat* h_in  = new float[sizeIn];\r\nfloat* h_out = new float[sizeOut];\r\n\r\n//Process h_in\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n\r\n// Allocate memory on the device\r\ncudaMalloc((void **) &d_in,  sizeIn ));\r\ncudaMalloc((void **) &d_out, sizeOut));\r\n\r\n// Copy array contents of input from the host (CPU) to the device (GPU)\r\ncudaMemcpy(d_in, h_in, sizeX * sizeY * sizeof(float), cudaMemcpyHostToDevice);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// Copy result back\r\ncudaMemcpy(h_out, d_out, sizeOut, cudaMemcpyDeviceToHost);\r\n\r\n// Continue processing on host using h_out\r\n\r\n</pre>\r\n\r\n<h3>Zero Copy Access CUDA Pipeline</h3>\r\nNow lets look at a pipeline using Zero Copy access.\r\n\r\n<pre>\r\n// Set flag to enable zero copy access\r\ncudaSetDeviceFlags(cudaDeviceMapHost);\r\n\r\n// Host Arrays\r\nfloat* h_in  = NULL;\r\nfloat* h_out = NULL;\r\n\r\n// Process h_in\r\n\r\n// Allocate host memory using CUDA allocation calls\r\ncudaHostAlloc((void **)&h_in,  sizeIn,  cudaHostAllocMapped);\r\ncudaHostAlloc((void **)&h_out, sizeOut, cudaHostAllocMapped);\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n// Get device pointer from host memory. No allocation or memcpy\r\ncudaHostGetDevicePointer((void **)&d_a, (void *) a, 0);\r\ncudaHostGetDevicePointer((void **)&d_b, (void *) b, 0);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// No need to copy d_out back\r\n// Continue processing on host using h_out\r\n</pre>\r\n\r\nAs you can see, it is considerably simpler to use zero copy access on the Tegra from a coding perspective. The kernel code remains the same. You can also allocate host memory using the cudaHostAlloc call without using zero copy access. This allows the fast pinned memory transfer when doing a cudaMemcpy.\r\n\r\n<h3>Results</h3>\r\nSo how does it perform?\r\n\r\nWhen I ran a transpose kernel using zero copy vs using standard pipeline, I achieve a the following results for a 4096 x 4096 matrix. \r\n\r\n<table style=\"width:50%\">\r\n  <th>\r\n    <td>Pipeline</td>\r\n    <td>Bandwidth (GB/s)</td> \r\n    <td>Time (ms)</td>\r\n  </th>  \r\n  <tr>\r\n    <td></td>\r\n    <td>Standard Pipeline</td>\r\n    <td>3.0</td> \r\n    <td>45</td>\r\n  </tr>\r\n  <tr>\r\n    <td></td>\r\n    <td>Zero Copy Pipeline</td>\r\n    <td>5.8</td> \r\n    <td>23</td>\r\n  </tr>\r\n</table>\r\n\r\nThe bandwidth of a device-to-device copy kernel on the Tegra is ~6.6 GB/s.\r\n\r\nThe results are tremendous. Any result, that is as good as or better than the standard pipeline is worth doing using zero copy. This saves memory usage as it does not require duplication on host and device.\r\n\r\nThere a a few caveats though. If you are running multiple kernels without modifying the data on the host side, it may be wise to run the standard pipeline. There is no right answer. Since it is so simple to modify the code for memory transfer vs zero copy pointers, it would be best to run both techniques and benchmark them.\r\n\r\n<h3>Applications</h3>\r\nWe these results, we believe the Tegra K1 is a great option for streaming applications. Most streaming applications run image or signal processing algorithms, with a limitation of running at either 60 or 30 frames per second. On desktops, although the kernels themselves run well under the required time, the memory transfer times form a significant portion.\r\n\r\nThis is why the Tegra K1 can be great. We save 100% of the memory transfer times used by discrete GPUs by using Zero Copy access. This allow the Tegra K1 GPU to do the streaming operations within the performance constraints even though is is considerable under-powered compared to most desktop GPUs.\r\n\r\nOf course, streaming is not the only application. There are many more, and if do happen to try one or want to try your application on the K1, we would be extremely excited to hear about it.\r\n\r\n<h3>Code</h3>\r\nYou can find the entire code from my transpose exercise here: <a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/transpose.zip\">transpose</a>.\r\nNote: If you wish to run this on a x86 system, make sure you change the compute version in the makefile from 32 to 30 (for Kepler).','Zero Copy on Tegra K1','','inherit','open','open','','6757-revision-v1','','','2014-10-03 15:25:36','2014-10-03 15:25:36','',6757,'http://sandbox.arrayfire.com/6757-revision-v1/',0,'revision','',0),(6783,11,'2014-10-03 15:26:15','2014-10-03 15:26:15','<h3>Introduction</h3>\r\nZero Copy access has been a part of the CUDA Toolkit for a long time (~2009). However, there were very few applications using the capability because, with time, GPU memory has become reasonably large. The only applications using zero copy access were mainly ones with extremely large memory requirements, such as database processing.\r\n\r\nZero Copy is a way to map host memory and access it directly over PCIe without doing an explicit memory transfer. It allows CUDA kernels to directly access host memory. Instead of reading data from global memory (limit ~200GB/s), data would be read over PCIe and be limited by PCIe bandwidth (upto 16GB/s). Hence there was no real performance advantage for most applications.\r\n\r\nHowever, with the release of the <a href=\"https://developer.nvidia.com/jetson-tk1\" title=\"Jetson TK1\" target=\"_blank\">Jetson TK1</a> board with the <a href=\"http://www.nvidia.com/object/tegra-k1-processor.html\" title=\"Tegra K1\" target=\"_blank\">Tegra K1</a> processor, this has become really useful. The Jetson board has 2GB of physical memory that is shared by the ARM CPU and the CUDA GPU. If a cudaMemcpy Host->Device is done on the Jetson, the memory is simply being copied to a new location on the same physical memory and retrieving a CUDA pointer from it. In such a scenario, Zero Copy access is perfect.\r\n\r\n<h3>Standard CUDA Pipeline</h3>\r\nLets take a look at a standard pipeline in CUDA.\r\n\r\n<pre>\r\n// Host Arrays\r\nfloat* h_in  = new float[sizeIn];\r\nfloat* h_out = new float[sizeOut];\r\n\r\n//Process h_in\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n\r\n// Allocate memory on the device\r\ncudaMalloc((void **) &d_in,  sizeIn ));\r\ncudaMalloc((void **) &d_out, sizeOut));\r\n\r\n// Copy array contents of input from the host (CPU) to the device (GPU)\r\ncudaMemcpy(d_in, h_in, sizeX * sizeY * sizeof(float), cudaMemcpyHostToDevice);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// Copy result back\r\ncudaMemcpy(h_out, d_out, sizeOut, cudaMemcpyDeviceToHost);\r\n\r\n// Continue processing on host using h_out\r\n\r\n</pre>\r\n\r\n<h3>Zero Copy Access CUDA Pipeline</h3>\r\nNow lets look at a pipeline using Zero Copy access.\r\n\r\n<pre>\r\n// Set flag to enable zero copy access\r\ncudaSetDeviceFlags(cudaDeviceMapHost);\r\n\r\n// Host Arrays\r\nfloat* h_in  = NULL;\r\nfloat* h_out = NULL;\r\n\r\n// Process h_in\r\n\r\n// Allocate host memory using CUDA allocation calls\r\ncudaHostAlloc((void **)&h_in,  sizeIn,  cudaHostAllocMapped);\r\ncudaHostAlloc((void **)&h_out, sizeOut, cudaHostAllocMapped);\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n// Get device pointer from host memory. No allocation or memcpy\r\ncudaHostGetDevicePointer((void **)&d_a, (void *) a, 0);\r\ncudaHostGetDevicePointer((void **)&d_b, (void *) b, 0);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// No need to copy d_out back\r\n// Continue processing on host using h_out\r\n</pre>\r\n\r\nAs you can see, it is considerably simpler to use zero copy access on the Tegra from a coding perspective. The kernel code remains the same. You can also allocate host memory using the cudaHostAlloc call without using zero copy access. This allows the fast pinned memory transfer when doing a cudaMemcpy.\r\n\r\n<h3>Results</h3>\r\nSo how does it perform?\r\n\r\nWhen I ran a transpose kernel using zero copy vs using standard pipeline, I achieve a the following results for a 4096 x 4096 matrix. \r\n\r\n<table style=\"width:50%\">\r\n  <th>\r\n    <td>Pipeline</td>\r\n    <td>Bandwidth (GB/s)</td> \r\n    <td>Time (ms)</td>\r\n  </th>  \r\n  <tr>\r\n    <td></td>\r\n    <td>Standard Pipeline</td>\r\n    <td>3.0</td> \r\n    <td>45</td>\r\n  </tr>\r\n  <tr>\r\n    <td></td>\r\n    <td>Zero Copy Pipeline</td>\r\n    <td>5.8</td> \r\n    <td>23</td>\r\n  </tr>\r\n</table>\r\n\r\nThe bandwidth of a device-to-device copy kernel on the Tegra is ~6.6 GB/s.\r\n\r\nThe results are tremendous. Any result, that is as good as or better than the standard pipeline is worth doing using zero copy. This saves memory usage as it does not require duplication on host and device.\r\n\r\nThere a a few caveats though. If you are running multiple kernels without modifying the data on the host side, it may be wise to run the standard pipeline. There is no right answer. Since it is so simple to modify the code for memory transfer vs zero copy pointers, it would be best to run both techniques and benchmark them.\r\n\r\n<h3>Applications</h3>\r\nWe these results, we believe the Tegra K1 is a great option for streaming applications. Most streaming applications run image or signal processing algorithms, with a limitation of running at either 60 or 30 frames per second. On desktops, although the kernels themselves run well under the required time, the memory transfer times form a significant portion.\r\n\r\nThis is why the Tegra K1 can be great. We save 100% of the memory transfer times used by discrete GPUs by using Zero Copy access. This allow the Tegra K1 GPU to do the streaming operations within the performance constraints even though is is considerable under-powered compared to most desktop GPUs.\r\n\r\nOf course, streaming is not the only application. There are many more, and if do happen to try one or want to try your application on the K1, we would be extremely excited to hear about it.\r\n\r\n<h3>Code</h3>\r\nYou can find the entire code from my transpose exercise here: <a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/transpose.zip\">transpose</a>.\r\nNote: If you wish to run this on a x86 system, make sure you change the compute version in the makefile from 32 to 30 (for Kepler).','Zero Copy on Tegra K1','','inherit','open','open','','6757-revision-v1','','','2014-10-03 15:26:15','2014-10-03 15:26:15','',6757,'http://sandbox.arrayfire.com/6757-revision-v1/',0,'revision','',0),(6786,17729,'2014-10-07 15:42:26','2014-10-07 15:42:26','Computer algorithms are extra friendly towards data sizes that are powers of two. GPU compute algorithms work particularly well with data sizes that are multiples of 32. In most real-world situations, however, data is rarely so conveniently sized. In today\'s post, we\'ll be looking at one such scenario related to GPU compute. Specifically, we\'ll provide you with some tips on how to optimize matrix transpose algorithm for a GPU.\r\n\r\nLet\'s start with the transpose kernel available from <a href=\"http://devblogs.nvidia.com/parallelforall/efficient-matrix-transpose-cuda-cc/\">NVIDIA\'s Parallel Forall blog</a>. It\'s been optimized to avoid bank conflicts as well, but only works on matrices with dimensions that are multiples of 32.\r\n<pre>\r\ntemplate<typename T>\r\n__global__\r\nvoid transpose32(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    __shared__ T shrdMem[TILE_DIM][TILE_DIM+1];\r\n\r\n    unsigned lx = threadIdx.x;\r\n    unsigned ly = threadIdx.y;\r\n\r\n    unsigned gx = lx + blockDim.x * blockIdx.x;\r\n    unsigned gy = ly + TILE_DIM   * blockIdx.y;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n\r\n    gx = lx + blockDim.x * blockIdx.y;\r\n    gy = ly + TILE_DIM   * blockIdx.x;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nA minor modification has been made to the kernel from the NVIDIA website by templating the matrix element data type such that we can use the same kernel with other integral data types. The variable <code>TILE_DIM</code> should have static constant qualifier and be initialized with a value of 32.\r\n\r\nNow we make the kernel applicable to matrices with any size by adding index checks. This requires making the following changes.\r\n<pre>\r\ntemplate<typename T>\r\n__global__\r\nvoid transposeNoSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (gx<dim0 && gy_<dim1)\r\n            shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (gx<dim1 && gy_<dim0)\r\n            out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nThe above kernel will check the read write indices for validity. Now let\'s add back the modifications to handle the special case where matrix dimensions are multiples of 32.\r\n<pre>\r\ntemplate<typename T, bool is32Multiple>\r\n__global__\r\nvoid transposeSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (is32Multiple || (gx<dim0 && gy_<dim1))\r\n            shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (is32Multiple || (gx<dim1 && gy_<dim0))\r\n            out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nNotice the additional boolean template parameter <code>is32Multiple</code> that is used to short circuit the <code>if</code> condition within the for loops inside the kernel. Wherever the transposeSC is called upon with boolean parameter value as true in the code, the resultant compiled code essentially eliminates the <code>if</code> condition.\r\n\r\nThe above kernel can be further optimized where any sort of index checks are completely avoided for all the blocks except the last one along a given dimension. Keep in mind that adding more short circuit parameters will increase the compilation time correspondingly.\r\n\r\nSimilar modifications can be done to OpenCL kernel using the kernel compile options to the function call <a href=\"https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clBuildProgram.html\"><code>clBuildProgram</code></a> and ensuring the kernels are compiled only once. We have recently done a post on how to ensure OpenCL kernels are compiled only once in applications. You can find the article <a href=\"http://sandbox.arrayfire.com/templating-and-caching-opencl-kernels/\">here</a>.\r\n\r\nHere are some bandwidth analysis results using the above three kernels on a 1024x1024 matrix. For the results, we used a launch configuration of 32x8 threads per block where each block processed a 32x32 sub-matrix from the input matrix.\r\n<table>\r\n<tbody>\r\n<tr>\r\n<th>Matrix Dimensions</th>\r\n<th>transpose32 Bandwidth</th>\r\n<th>transposeNoSC Bandwidth</th>\r\n<th>transposeSC Bandwidth</th>\r\n</tr>\r\n<tr>\r\n<td>1024x1024</td>\r\n<td>85.18 GB/s</td>\r\n<td>82.21 GB/s</td>\r\n<td>84.41 GB/s</td>\r\n</tr>\r\n</tbody>\r\n</table>\r\nKeep an eye out for future posts along this theme, in which we\'ll discuss tips to optimize CUDA/OpenCL kernels for different algorithms.','CUDA Optimization tips for Matrix Transpose in real world applications','','publish','open','open','','cuda-optimization-tips-for-matrix-transpose-in-real-world-applications','','http://sandbox.arrayfire.com/templating-and-caching-opencl-kernels/\nhttp://devblogs.nvidia.com/parallelforall/efficient-matrix-transpose-cuda-cc/','2014-10-07 20:53:39','2014-10-07 20:53:39','',0,'http://sandbox.arrayfire.com/?p=6786',0,'post','',1),(6787,17729,'2014-10-06 18:59:42','2014-10-06 18:59:42','In this post, we shall be looking at how to optimize transpose kernel for real world applications where the matrix dimensions are seldom multiples of 32.\r\n\r\nLet us start with the transpose kernel available from <a href=\"http://devblogs.nvidia.com/parallelforall/efficient-matrix-transpose-cuda-cc/\">here</a>. It has been optimized to avoid bank conflicts as well but only works with matrices with dimensions that are multiples of 32.\r\n<code>\r\ntemplate<typename T>\r\n__global__ \r\nvoid transpose32(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    __shared__ T shrdMem[TILE_DIM][TILE_DIM+1];\r\n\r\n    unsigned lx = threadIdx.x;\r\n    unsigned ly = threadIdx.y;\r\n\r\n    unsigned gx = lx + blockDim.x * blockIdx.x;\r\n    unsigned gy = ly + TILE_DIM   * blockIdx.y;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n\r\n    gx = lx + blockDim.x * blockIdx.y;\r\n    gy = ly + TILE_DIM   * blockIdx.x;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</code>\r\nI have done a minor modification to the kernel by templating the matrix element data type such that i can use the same kernel with other types such as double, int, unsigned etc.','','','inherit','open','open','','6786-revision-v1','','','2014-10-06 18:59:42','2014-10-06 18:59:42','',6786,'http://sandbox.arrayfire.com/6786-revision-v1/',0,'revision','',0),(6788,17729,'2014-10-06 19:00:42','2014-10-06 19:00:42','In this post, we shall be looking at how to optimize transpose kernel for real world applications where the matrix dimensions are seldom powers of 2.\r\n\r\nLet us start with the transpose kernel available from <a href=\"http://devblogs.nvidia.com/parallelforall/efficient-matrix-transpose-cuda-cc/\">here</a>. It has been optimized to avoid bank conflicts as well but only works with matrices with dimensions that are multiples of 32.\r\n<pre>\r\ntemplate<typename T>\r\n__global__ \r\nvoid transpose32(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    __shared__ T shrdMem[TILE_DIM][TILE_DIM+1];\r\n\r\n    unsigned lx = threadIdx.x;\r\n    unsigned ly = threadIdx.y;\r\n\r\n    unsigned gx = lx + blockDim.x * blockIdx.x;\r\n    unsigned gy = ly + TILE_DIM   * blockIdx.y;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n\r\n    gx = lx + blockDim.x * blockIdx.y;\r\n    gy = ly + TILE_DIM   * blockIdx.x;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nI have done a minor modification to the kernel by templating the matrix element data type such that i can use the same kernel with other types such as double, int, unsigned etc.','','','inherit','open','open','','6786-revision-v1','','','2014-10-06 19:00:42','2014-10-06 19:00:42','',6786,'http://sandbox.arrayfire.com/6786-revision-v1/',0,'revision','',0),(6789,17729,'2014-10-06 19:05:54','2014-10-06 19:05:54','In this post, we shall be looking at how to optimize transpose kernel for real world applications where the matrix dimensions are seldom powers of 2.\r\n\r\nLet us start with the transpose kernel available from <a href=\"http://devblogs.nvidia.com/parallelforall/efficient-matrix-transpose-cuda-cc/\">here</a>. It has been optimized to avoid bank conflicts as well but only works on matrices with dimensions that are multiples of 32.\r\n<pre>\r\ntemplate<typename T>\r\n__global__ \r\nvoid transpose32(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    __shared__ T shrdMem[TILE_DIM][TILE_DIM+1];\r\n\r\n    unsigned lx = threadIdx.x;\r\n    unsigned ly = threadIdx.y;\r\n\r\n    unsigned gx = lx + blockDim.x * blockIdx.x;\r\n    unsigned gy = ly + TILE_DIM   * blockIdx.y;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n\r\n    gx = lx + blockDim.x * blockIdx.y;\r\n    gy = ly + TILE_DIM   * blockIdx.x;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nA minor modification has been done to the kernel by templating the matrix element data type such that i can use the same kernel with other types such as double, int, unsigned etc.\r\n\r\nOne immediate thing we can do to make this kernel applicable to matrices that have dimensions which are not powers of 2 is to add index checking. Following are the changes required to do so.\r\n<pre>\r\ntemplate<typename T>\r\n__global__\r\nvoid transposeNoSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (gx<dim0 && gy_<dim1)\r\n            shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (gx<dim1 && gy_<dim0)\r\n            out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>','','','inherit','open','open','','6786-revision-v1','','','2014-10-06 19:05:54','2014-10-06 19:05:54','',6786,'http://sandbox.arrayfire.com/6786-revision-v1/',0,'revision','',0),(6790,17729,'2014-10-06 19:14:01','2014-10-06 19:14:01','In this post, we shall be looking at how to optimize transpose kernel for real world applications where the matrix dimensions are seldom powers of 2.\r\n\r\nLet us start with the transpose kernel available from <a href=\"http://devblogs.nvidia.com/parallelforall/efficient-matrix-transpose-cuda-cc/\">here</a>. It has been optimized to avoid bank conflicts as well but only works on matrices with dimensions that are multiples of 32.\r\n<pre>\r\ntemplate<typename T>\r\n__global__ \r\nvoid transpose32(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    __shared__ T shrdMem[TILE_DIM][TILE_DIM+1];\r\n\r\n    unsigned lx = threadIdx.x;\r\n    unsigned ly = threadIdx.y;\r\n\r\n    unsigned gx = lx + blockDim.x * blockIdx.x;\r\n    unsigned gy = ly + TILE_DIM   * blockIdx.y;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n\r\n    gx = lx + blockDim.x * blockIdx.y;\r\n    gy = ly + TILE_DIM   * blockIdx.x;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nA minor modification has been done to the kernel by templating the matrix element data type such that we can use the same kernel with other integral data types.\r\n\r\nNow, we make the kernel applicable to matrices with any size by adding index checks. Following are the changes required to do so.\r\n<pre>\r\ntemplate<typename T>\r\n__global__\r\nvoid transposeNoSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (gx<dim0 && gy_<dim1)\r\n            shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (gx<dim1 && gy_<dim0)\r\n            out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\n\r\nThere is a reason the above kernel is named transposeNoSC which shall become evident as we proceed to next modification. Lets add back the modifications to handle the special case where matrix dimensions are multiples of 32.\r\n<pre>\r\ntemplate<typename T, bool is32Multiple>\r\n__global__\r\nvoid transposeSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (is32Multiple || (gx<dim0 && gy_<dim1))\r\n            shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (is32Multiple || (gx<dim1 && gy_<dim0))\r\n            out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nNotice the additional boolean template parameter <code>is32Multiple</code> that used to short circuit the <code>if</code> condition within the for loops inside the kernel whenever the input matrix has dimensions that are multiple of 32. Where ever the transposeSC is called upon with boolean parameter value as true in the code, the resultant compiled code essentially eliminates the <code>if</code> condition.','','','inherit','open','open','','6786-revision-v1','','','2014-10-06 19:14:01','2014-10-06 19:14:01','',6786,'http://sandbox.arrayfire.com/6786-revision-v1/',0,'revision','',0),(6791,17729,'2014-10-06 19:21:04','2014-10-06 19:21:04','In this post, we shall be looking at how to optimize transpose kernel for real world applications where the matrix dimensions are seldom powers of 2.\r\n\r\nLet us start with the transpose kernel available from <a href=\"http://devblogs.nvidia.com/parallelforall/efficient-matrix-transpose-cuda-cc/\">here</a>. It has been optimized to avoid bank conflicts as well but only works on matrices with dimensions that are multiples of 32.\r\n<pre>\r\ntemplate<typename T>\r\n__global__ \r\nvoid transpose32(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    __shared__ T shrdMem[TILE_DIM][TILE_DIM+1];\r\n\r\n    unsigned lx = threadIdx.x;\r\n    unsigned ly = threadIdx.y;\r\n\r\n    unsigned gx = lx + blockDim.x * blockIdx.x;\r\n    unsigned gy = ly + TILE_DIM   * blockIdx.y;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n\r\n    gx = lx + blockDim.x * blockIdx.y;\r\n    gy = ly + TILE_DIM   * blockIdx.x;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nA minor modification has been done to the kernel by templating the matrix element data type such that we can use the same kernel with other integral data types.\r\n\r\nNow, we make the kernel applicable to matrices with any size by adding index checks. Following are the changes required to do so.\r\n<pre>\r\ntemplate<typename T>\r\n__global__\r\nvoid transposeNoSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (gx<dim0 && gy_<dim1)\r\n            shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (gx<dim1 && gy_<dim0)\r\n            out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\n\r\nThere is a reason the above kernel is named transposeNoSC which shall become evident as we proceed to next modification. Lets add back the modifications to handle the special case where matrix dimensions are multiples of 32.\r\n<pre>\r\ntemplate<typename T, bool is32Multiple>\r\n__global__\r\nvoid transposeSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (is32Multiple || (gx<dim0 && gy_<dim1))\r\n            shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (is32Multiple || (gx<dim1 && gy_<dim0))\r\n            out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nNotice the additional boolean template parameter <code>is32Multiple</code> that used to short circuit the <code>if</code> condition within the for loops inside the kernel. Where ever the transposeSC is called upon with boolean parameter value as true in the code, the resultant compiled code essentially eliminates the <code>if</code> condition. Hence the above kernel was named <code>transposeSC</code>(SC - Short Circuited) and the former kernel was named transposeNoSC(No Short Circuit compile time template parameter).\r\n\r\nSimilar modifications can be done to OpenCL kernel using the kernel compile options to the function call <code>clBuildProgram</code>.\r\n\r\nOne more optimization can be added to the above kernel where any sort of index checks are completely avoided for all the blocks except the last one. \r\n\r\nKeep an eye out for the upcoming parts of this series where we would be discussing tips and tricks to optimized CUDA/OpenCL kernels.','CUDA Optimization tricks for real world applications: Part 1','','inherit','open','open','','6786-revision-v1','','','2014-10-06 19:21:04','2014-10-06 19:21:04','',6786,'http://sandbox.arrayfire.com/6786-revision-v1/',0,'revision','',0),(6792,17729,'2014-10-06 19:21:28','2014-10-06 19:21:28','In this post, we shall be looking at how to optimize transpose kernel for real world applications where the matrix dimensions are seldom powers of 2.\r\n\r\nLet us start with the transpose kernel available from <a href=\"http://devblogs.nvidia.com/parallelforall/efficient-matrix-transpose-cuda-cc/\">here</a>. It has been optimized to avoid bank conflicts as well but only works on matrices with dimensions that are multiples of 32.\r\n<pre>\r\ntemplate<typename T>\r\n__global__ \r\nvoid transpose32(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    __shared__ T shrdMem[TILE_DIM][TILE_DIM+1];\r\n\r\n    unsigned lx = threadIdx.x;\r\n    unsigned ly = threadIdx.y;\r\n\r\n    unsigned gx = lx + blockDim.x * blockIdx.x;\r\n    unsigned gy = ly + TILE_DIM   * blockIdx.y;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n\r\n    gx = lx + blockDim.x * blockIdx.y;\r\n    gy = ly + TILE_DIM   * blockIdx.x;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nA minor modification has been done to the kernel by templating the matrix element data type such that we can use the same kernel with other integral data types.\r\n\r\nNow, we make the kernel applicable to matrices with any size by adding index checks. Following are the changes required to do so.\r\n<pre>\r\ntemplate<typename T>\r\n__global__\r\nvoid transposeNoSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (gx<dim0 && gy_<dim1)\r\n            shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (gx<dim1 && gy_<dim0)\r\n            out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\n\r\nThere is a reason the above kernel is named transposeNoSC which shall become evident as we proceed to next modification. Lets add back the modifications to handle the special case where matrix dimensions are multiples of 32.\r\n<pre>\r\ntemplate<typename T, bool is32Multiple>\r\n__global__\r\nvoid transposeSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (is32Multiple || (gx<dim0 && gy_<dim1))\r\n            shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (is32Multiple || (gx<dim1 && gy_<dim0))\r\n            out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nNotice the additional boolean template parameter <code>is32Multiple</code> that used to short circuit the <code>if</code> condition within the for loops inside the kernel. Where ever the transposeSC is called upon with boolean parameter value as true in the code, the resultant compiled code essentially eliminates the <code>if</code> condition. Hence the above kernel was named <code>transposeSC</code>(SC - Short Circuited) and the former kernel was named transposeNoSC(No Short Circuit compile time template parameter).\r\n\r\nSimilar modifications can be done to OpenCL kernel using the kernel compile options to the function call <code>clBuildProgram</code>.\r\n\r\nOne more optimization can be added to the above kernel where any sort of index checks are completely avoided for all the blocks except the last one. \r\n\r\nKeep an eye out for the upcoming parts of this series where we would be discussing tips and tricks to optimized CUDA/OpenCL kernels.','CUDA Optimization tips for real world applications: Part 1 - Matrix Transpose','','inherit','open','open','','6786-revision-v1','','','2014-10-06 19:21:28','2014-10-06 19:21:28','',6786,'http://sandbox.arrayfire.com/6786-revision-v1/',0,'revision','',0),(6793,17729,'2014-10-06 19:22:15','2014-10-06 19:22:15','In this post, we shall be looking at how to optimize transpose kernel for real world applications where the matrix dimensions are seldom powers of 2.\r\n\r\nLet us start with the transpose kernel available from <a href=\"http://devblogs.nvidia.com/parallelforall/efficient-matrix-transpose-cuda-cc/\">here</a>. It has been optimized to avoid bank conflicts as well but only works on matrices with dimensions that are multiples of 32.\r\n<pre>\r\ntemplate<typename T>\r\n__global__ \r\nvoid transpose32(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    __shared__ T shrdMem[TILE_DIM][TILE_DIM+1];\r\n\r\n    unsigned lx = threadIdx.x;\r\n    unsigned ly = threadIdx.y;\r\n\r\n    unsigned gx = lx + blockDim.x * blockIdx.x;\r\n    unsigned gy = ly + TILE_DIM   * blockIdx.y;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n\r\n    gx = lx + blockDim.x * blockIdx.y;\r\n    gy = ly + TILE_DIM   * blockIdx.x;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nA minor modification has been done to the kernel from NVIDIA website by templating the matrix element data type such that we can use the same kernel with other integral data types.\r\n\r\nNow, we make the kernel applicable to matrices with any size by adding index checks. Following are the changes required to do so.\r\n<pre>\r\ntemplate<typename T>\r\n__global__\r\nvoid transposeNoSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (gx<dim0 && gy_<dim1)\r\n            shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (gx<dim1 && gy_<dim0)\r\n            out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\n\r\nThere is a reason the above kernel is named transposeNoSC which shall become evident as we proceed to next modification. Lets add back the modifications to handle the special case where matrix dimensions are multiples of 32.\r\n<pre>\r\ntemplate<typename T, bool is32Multiple>\r\n__global__\r\nvoid transposeSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (is32Multiple || (gx<dim0 && gy_<dim1))\r\n            shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (is32Multiple || (gx<dim1 && gy_<dim0))\r\n            out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nNotice the additional boolean template parameter <code>is32Multiple</code> that used to short circuit the <code>if</code> condition within the for loops inside the kernel. Where ever the transposeSC is called upon with boolean parameter value as true in the code, the resultant compiled code essentially eliminates the <code>if</code> condition. Hence the above kernel was named <code>transposeSC</code>(SC - Short Circuited) and the former kernel was named transposeNoSC(No Short Circuit compile time template parameter).\r\n\r\nSimilar modifications can be done to OpenCL kernel using the kernel compile options to the function call <code>clBuildProgram</code>.\r\n\r\nOne more optimization can be added to the above kernel where any sort of index checks are completely avoided for all the blocks except the last one. \r\n\r\nKeep an eye out for the upcoming parts of this series where we would be discussing tips and tricks to optimized CUDA/OpenCL kernels.','CUDA Optimization tips for real world applications: Part 1 - Matrix Transpose','','inherit','open','open','','6786-revision-v1','','','2014-10-06 19:22:15','2014-10-06 19:22:15','',6786,'http://sandbox.arrayfire.com/6786-revision-v1/',0,'revision','',0),(6794,17729,'2014-10-06 19:28:38','2014-10-06 19:28:38','In this post, we shall be looking at how to optimize transpose kernel for real world applications where the matrix dimensions are seldom powers of 2.\r\n\r\nLet us start with the transpose kernel available from <a href=\"http://devblogs.nvidia.com/parallelforall/efficient-matrix-transpose-cuda-cc/\">here</a>. It has been optimized to avoid bank conflicts as well but only works on matrices with dimensions that are multiples of 32.\r\n<pre>\r\ntemplate<typename T>\r\n__global__ \r\nvoid transpose32(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    __shared__ T shrdMem[TILE_DIM][TILE_DIM+1];\r\n\r\n    unsigned lx = threadIdx.x;\r\n    unsigned ly = threadIdx.y;\r\n\r\n    unsigned gx = lx + blockDim.x * blockIdx.x;\r\n    unsigned gy = ly + TILE_DIM   * blockIdx.y;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n\r\n    gx = lx + blockDim.x * blockIdx.y;\r\n    gy = ly + TILE_DIM   * blockIdx.x;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nA minor modification has been done to the kernel from NVIDIA website by templating the matrix element data type such that we can use the same kernel with other integral data types.\r\n\r\nNow, we make the kernel applicable to matrices with any size by adding index checks. Following are the changes required to do so.\r\n<pre>\r\ntemplate<typename T>\r\n__global__\r\nvoid transposeNoSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (gx<dim0 && gy_<dim1)\r\n            shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (gx<dim1 && gy_<dim0)\r\n            out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\n\r\nThere is a reason the above kernel is named transposeNoSC which shall become evident as we proceed to next modification. Lets add back the modifications to handle the special case where matrix dimensions are multiples of 32.\r\n<pre>\r\ntemplate<typename T, bool is32Multiple>\r\n__global__\r\nvoid transposeSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (is32Multiple || (gx<dim0 && gy_<dim1))\r\n            shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (is32Multiple || (gx<dim1 && gy_<dim0))\r\n            out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nNotice the additional boolean template parameter <code>is32Multiple</code> that is used to short circuit the <code>if</code> condition within the for loops inside the kernel. Where ever the transposeSC is called upon with boolean parameter value as true in the code, the resultant compiled code essentially eliminates the <code>if</code> condition. Hence the above kernel was named <code>transposeSC</code>(SC - Short Circuited) and the former kernel was named transposeNoSC(No Short Circuit compile time template parameter).\r\n\r\nThe above kernel can be further optimized where any sort of index checks are completely avoided for all the blocks except the last one.\r\n\r\nSimilar modifications can be done to OpenCL kernel using the kernel compile options to the function call <a href=\"https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clBuildProgram.html\"><code>clBuildProgram</code></a> and ensuring the kernels are compiled only once. We have recently done a post on how to ensure OpenCL kernels are compiled only once in applications. You can find the article <a href=\"http://sandbox.arrayfire.com/templating-and-caching-opencl-kernels/\">here</a>.\r\n\r\nKeep an eye out for the future posts in this series where we would be discussing tips to optimize CUDA/OpenCL kernels for real world applications.','CUDA Optimization tips for real world applications: Part 1 - Matrix Transpose','','inherit','open','open','','6786-revision-v1','','','2014-10-06 19:28:38','2014-10-06 19:28:38','',6786,'http://sandbox.arrayfire.com/6786-revision-v1/',0,'revision','',0),(6795,17729,'2014-10-06 20:14:01','2014-10-06 20:14:01','In this post, we shall be looking at how to optimize transpose kernel for real world applications where the matrix dimensions are seldom powers of 2.\r\n\r\nLet us start with the transpose kernel available from <a href=\"http://devblogs.nvidia.com/parallelforall/efficient-matrix-transpose-cuda-cc/\">here</a>. It has been optimized to avoid bank conflicts as well but only works on matrices with dimensions that are multiples of 32.\r\n<pre>\r\ntemplate<typename T>\r\n__global__ \r\nvoid transpose32(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    __shared__ T shrdMem[TILE_DIM][TILE_DIM+1];\r\n\r\n    unsigned lx = threadIdx.x;\r\n    unsigned ly = threadIdx.y;\r\n\r\n    unsigned gx = lx + blockDim.x * blockIdx.x;\r\n    unsigned gy = ly + TILE_DIM   * blockIdx.y;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n\r\n    gx = lx + blockDim.x * blockIdx.y;\r\n    gy = ly + TILE_DIM   * blockIdx.x;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nA minor modification has been done to the kernel from NVIDIA website by templating the matrix element data type such that we can use the same kernel with other integral data types.\r\n\r\nNow, we make the kernel applicable to matrices with any size by adding index checks. Following are the changes required to do so.\r\n<pre>\r\ntemplate<typename T>\r\n__global__\r\nvoid transposeNoSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (gx<dim0 && gy_<dim1)\r\n            shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (gx<dim1 && gy_<dim0)\r\n            out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\n\r\nThere is a reason the above kernel is named transposeNoSC which shall become evident as we proceed to next modification. Lets add back the modifications to handle the special case where matrix dimensions are multiples of 32.\r\n<pre>\r\ntemplate<typename T, bool is32Multiple>\r\n__global__\r\nvoid transposeSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (is32Multiple || (gx<dim0 && gy_<dim1))\r\n            shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (is32Multiple || (gx<dim1 && gy_<dim0))\r\n            out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nNotice the additional boolean template parameter <code>is32Multiple</code> that is used to short circuit the <code>if</code> condition within the for loops inside the kernel. Where ever the transposeSC is called upon with boolean parameter value as true in the code, the resultant compiled code essentially eliminates the <code>if</code> condition. Hence the above kernel was named <code>transposeSC</code>(SC - Short Circuited) and the former kernel was named transposeNoSC(No Short Circuit compile time template parameter).\r\n\r\nThe above kernel can be further optimized where any sort of index checks are completely avoided for all the blocks except the last one along a given dimension.\r\n\r\nSimilar modifications can be done to OpenCL kernel using the kernel compile options to the function call <a href=\"https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clBuildProgram.html\"><code>clBuildProgram</code></a> and ensuring the kernels are compiled only once. We have recently done a post on how to ensure OpenCL kernels are compiled only once in applications. You can find the article <a href=\"http://sandbox.arrayfire.com/templating-and-caching-opencl-kernels/\">here</a>.\r\n\r\nKeep an eye out for the future posts in this series where we would be discussing tips to optimize CUDA/OpenCL kernels for real world applications.','CUDA Optimization tips for real world applications: Part 1 - Matrix Transpose','','inherit','open','open','','6786-revision-v1','','','2014-10-06 20:14:01','2014-10-06 20:14:01','',6786,'http://sandbox.arrayfire.com/6786-revision-v1/',0,'revision','',0),(6796,17729,'2014-10-06 20:54:29','2014-10-06 20:54:29','In this post, we shall be looking at how to optimize transpose kernel for real world applications where the matrix dimensions are seldom powers of 2.\r\n\r\nLet us start with the transpose kernel available from <a href=\"http://devblogs.nvidia.com/parallelforall/efficient-matrix-transpose-cuda-cc/\">here</a>. It has been optimized to avoid bank conflicts as well but only works on matrices with dimensions that are multiples of 32.\r\n<pre>\r\ntemplate<typename T>\r\n__global__ \r\nvoid transpose32(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    __shared__ T shrdMem[TILE_DIM][TILE_DIM+1];\r\n\r\n    unsigned lx = threadIdx.x;\r\n    unsigned ly = threadIdx.y;\r\n\r\n    unsigned gx = lx + blockDim.x * blockIdx.x;\r\n    unsigned gy = ly + TILE_DIM   * blockIdx.y;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n\r\n    gx = lx + blockDim.x * blockIdx.y;\r\n    gy = ly + TILE_DIM   * blockIdx.x;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nA minor modification has been done to the kernel from NVIDIA website by templating the matrix element data type such that we can use the same kernel with other integral data types.\r\n\r\nNow, we make the kernel applicable to matrices with any size by adding index checks. Following are the changes required to do so.\r\n<pre>\r\ntemplate<typename T>\r\n__global__\r\nvoid transposeNoSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (gx<dim0 && gy_<dim1)\r\n            shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (gx<dim1 && gy_<dim0)\r\n            out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\n\r\nThere is a reason the above kernel is named transposeNoSC which shall become evident as we proceed to next modification. Lets add back the modifications to handle the special case where matrix dimensions are multiples of 32.\r\n<pre>\r\ntemplate<typename T, bool is32Multiple>\r\n__global__\r\nvoid transposeSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (is32Multiple || (gx<dim0 && gy_<dim1))\r\n            shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (is32Multiple || (gx<dim1 && gy_<dim0))\r\n            out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nNotice the additional boolean template parameter <code>is32Multiple</code> that is used to short circuit the <code>if</code> condition within the for loops inside the kernel. Where ever the transposeSC is called upon with boolean parameter value as true in the code, the resultant compiled code essentially eliminates the <code>if</code> condition. Hence the above kernel was named <code>transposeSC</code>(SC - Short Circuited) and the former kernel was named transposeNoSC(No Short Circuit compile time template parameter).\r\n\r\nThe above kernel can be further optimized where any sort of index checks are completely avoided for all the blocks except the last one along a given dimension.\r\n\r\nSimilar modifications can be done to OpenCL kernel using the kernel compile options to the function call <a href=\"https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clBuildProgram.html\"><code>clBuildProgram</code></a> and ensuring the kernels are compiled only once. We have recently done a post on how to ensure OpenCL kernels are compiled only once in applications. You can find the article <a href=\"http://sandbox.arrayfire.com/templating-and-caching-opencl-kernels/\">here</a>.\r\n\r\nHere are some bandwidth analysis results using the above three kernels on different matrix dimensions.\r\n\r\n<iframe width=\"840\" height=\"616\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1tWhNejdDNFyYpGqs_x2Q7XVMlCYzPN1N6jDRIfKhYB8/pubchart?oid=682346979&amp;format=interactive\"></iframe>\r\n\r\nKeep an eye out for the future posts in this series where we would be discussing tips to optimize CUDA/OpenCL kernels for real world applications.','CUDA Optimization tips for real world applications: Part 1 - Matrix Transpose','','inherit','open','open','','6786-revision-v1','','','2014-10-06 20:54:29','2014-10-06 20:54:29','',6786,'http://sandbox.arrayfire.com/6786-revision-v1/',0,'revision','',0),(6797,17729,'2014-10-06 21:43:17','2014-10-06 21:43:17','Computer algorithms are some what extra friendly towards data sizes that are powers of two. Especially GPU compute algorithms are very favorable to data sizes that are multiples of 32(which is also a power of two, known to the GPUs by the name Warp/Wavefront). However, in most real world situations it is rather hard to find the data to be so conveniently sized. In today\'s post, we shall be looking at one of such scenarios related to GPU compute. To be specific, we shall provide you with some tips on how to optimize matrix transpose algorithm for a GPU.\r\n\r\nLet us start with the transpose kernel available from <a href=\"http://devblogs.nvidia.com/parallelforall/efficient-matrix-transpose-cuda-cc/\">NVIDIA\'s Parallel Forall blog</a>. It has been optimized to avoid bank conflicts as well but only works on matrices with dimensions that are multiples of 32.\r\n<pre>\r\ntemplate<typename T>\r\n__global__ \r\nvoid transpose32(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    __shared__ T shrdMem[TILE_DIM][TILE_DIM+1];\r\n\r\n    unsigned lx = threadIdx.x;\r\n    unsigned ly = threadIdx.y;\r\n\r\n    unsigned gx = lx + blockDim.x * blockIdx.x;\r\n    unsigned gy = ly + TILE_DIM   * blockIdx.y;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n\r\n    gx = lx + blockDim.x * blockIdx.y;\r\n    gy = ly + TILE_DIM   * blockIdx.x;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nA minor modification has been done to the kernel from NVIDIA website by templating the matrix element data type such that we can use the same kernel with other integral data types.\r\n\r\nNow, we make the kernel applicable to matrices with any size by adding index checks. Following are the changes required to do so.\r\n<pre>\r\ntemplate<typename T>\r\n__global__\r\nvoid transposeNoSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (gx<dim0 && gy_<dim1)\r\n            shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (gx<dim1 && gy_<dim0)\r\n            out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\n\r\nThere is a reason the above kernel is named transposeNoSC which shall become evident as we proceed to next modification. Lets add back the modifications to handle the special case where matrix dimensions are multiples of 32.\r\n<pre>\r\ntemplate<typename T, bool is32Multiple>\r\n__global__\r\nvoid transposeSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (is32Multiple || (gx<dim0 && gy_<dim1))\r\n            shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (is32Multiple || (gx<dim1 && gy_<dim0))\r\n            out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nNotice the additional boolean template parameter <code>is32Multiple</code> that is used to short circuit the <code>if</code> condition within the for loops inside the kernel. Where ever the transposeSC is called upon with boolean parameter value as true in the code, the resultant compiled code essentially eliminates the <code>if</code> condition. Hence the above kernel was named <code>transposeSC</code>(SC - Short Circuited) and the former kernel was named transposeNoSC(No Short Circuit compile time template parameter).\r\n\r\nThe above kernel can be further optimized where any sort of index checks are completely avoided for all the blocks except the last one along a given dimension.\r\n\r\nSimilar modifications can be done to OpenCL kernel using the kernel compile options to the function call <a href=\"https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clBuildProgram.html\"><code>clBuildProgram</code></a> and ensuring the kernels are compiled only once. We have recently done a post on how to ensure OpenCL kernels are compiled only once in applications. You can find the article <a href=\"http://sandbox.arrayfire.com/templating-and-caching-opencl-kernels/\">here</a>.\r\n\r\nHere are some bandwidth analysis results using the above three kernels on different matrix dimensions.\r\n\r\n<iframe width=\"840\" height=\"616\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1tWhNejdDNFyYpGqs_x2Q7XVMlCYzPN1N6jDRIfKhYB8/pubchart?oid=682346979&amp;format=interactive\"></iframe>\r\n\r\nKeep an eye out for the future posts in this series where we would be discussing tips to optimize CUDA/OpenCL kernels for real world applications.','CUDA Optimization tips for real world applications: Part 1 - Matrix Transpose','','inherit','open','open','','6786-revision-v1','','','2014-10-06 21:43:17','2014-10-06 21:43:17','',6786,'http://sandbox.arrayfire.com/6786-revision-v1/',0,'revision','',0),(6798,17729,'2014-10-06 21:50:04','2014-10-06 21:50:04','Computer algorithms are some what extra friendly towards data sizes that are powers of two. Especially GPU compute algorithms are very favorable to data sizes that are multiples of 32(which is also a power of two, known to the GPUs by the name Warp/Wavefront). However, in most real world situations it is rather hard to find the data to be so conveniently sized. In today\'s post, we shall be looking at one of such scenarios related to GPU compute. To be specific, we shall provide you with some tips on how to optimize matrix transpose algorithm for a GPU.\r\n\r\nLet us start with the transpose kernel available from <a href=\"http://devblogs.nvidia.com/parallelforall/efficient-matrix-transpose-cuda-cc/\">NVIDIA\'s Parallel Forall blog</a>. It has been optimized to avoid bank conflicts as well but only works on matrices with dimensions that are multiples of 32.\r\n<pre>\r\ntemplate<typename T>\r\n__global__ \r\nvoid transpose32(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    __shared__ T shrdMem[TILE_DIM][TILE_DIM+1];\r\n\r\n    unsigned lx = threadIdx.x;\r\n    unsigned ly = threadIdx.y;\r\n\r\n    unsigned gx = lx + blockDim.x * blockIdx.x;\r\n    unsigned gy = ly + TILE_DIM   * blockIdx.y;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n\r\n    gx = lx + blockDim.x * blockIdx.y;\r\n    gy = ly + TILE_DIM   * blockIdx.x;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nA minor modification has been done to the kernel from NVIDIA website by templating the matrix element data type such that we can use the same kernel with other integral data types.\r\n\r\nNow, we make the kernel applicable to matrices with any size by adding index checks. Following are the changes required to do so.\r\n<pre>\r\ntemplate<typename T>\r\n__global__\r\nvoid transposeNoSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (gx<dim0 && gy_<dim1)\r\n            shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (gx<dim1 && gy_<dim0)\r\n            out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\n\r\nThere is a reason the above kernel is named transposeNoSC which shall become evident as we proceed to next modification. Lets add back the modifications to handle the special case where matrix dimensions are multiples of 32.\r\n<pre>\r\ntemplate<typename T, bool is32Multiple>\r\n__global__\r\nvoid transposeSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (is32Multiple || (gx<dim0 && gy_<dim1))\r\n            shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (is32Multiple || (gx<dim1 && gy_<dim0))\r\n            out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nNotice the additional boolean template parameter <code>is32Multiple</code> that is used to short circuit the <code>if</code> condition within the for loops inside the kernel. Where ever the transposeSC is called upon with boolean parameter value as true in the code, the resultant compiled code essentially eliminates the <code>if</code> condition. Hence the above kernel was named <code>transposeSC</code>(SC - Short Circuited) and the former kernel was named transposeNoSC(No Short Circuit compile time template parameter).\r\n\r\nThe above kernel can be further optimized where any sort of index checks are completely avoided for all the blocks except the last one along a given dimension.\r\n\r\nSimilar modifications can be done to OpenCL kernel using the kernel compile options to the function call <a href=\"https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clBuildProgram.html\"><code>clBuildProgram</code></a> and ensuring the kernels are compiled only once. We have recently done a post on how to ensure OpenCL kernels are compiled only once in applications. You can find the article <a href=\"http://sandbox.arrayfire.com/templating-and-caching-opencl-kernels/\">here</a>.\r\n\r\nHere are some bandwidth analysis results using the above three kernels on different matrix dimensions.\r\n\r\n<iframe width=\"840\" height=\"616\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1tWhNejdDNFyYpGqs_x2Q7XVMlCYzPN1N6jDRIfKhYB8/pubchart?oid=682346979&amp;format=interactive\"></iframe>\r\n\r\nKeep an eye out for the future posts in this series where we would be discussing tips to optimize CUDA/OpenCL kernels for real world applications.','CUDA Optimization tips for Matrix Transpose in real world applications','','inherit','open','open','','6786-revision-v1','','','2014-10-06 21:50:04','2014-10-06 21:50:04','',6786,'http://sandbox.arrayfire.com/6786-revision-v1/',0,'revision','',0),(6799,17729,'2014-10-06 21:51:14','2014-10-06 21:51:14','Computer algorithms are some what extra friendly towards data sizes that are powers of two. Especially GPU compute algorithms are very favorable to data sizes that are multiples of 32(which is also a power of two, known to the GPUs by the name Warp/Wavefront). However, in most real world situations it is rather hard to find the data to be so conveniently sized. In today\'s post, we shall be looking at one of such scenarios related to GPU compute. To be specific, we shall provide you with some tips on how to optimize matrix transpose algorithm for a GPU.\r\n\r\nLet us start with the transpose kernel available from <a href=\"http://devblogs.nvidia.com/parallelforall/efficient-matrix-transpose-cuda-cc/\">NVIDIA\'s Parallel Forall blog</a>. It has been optimized to avoid bank conflicts as well but only works on matrices with dimensions that are multiples of 32.\r\n<pre>\r\ntemplate<typename T>\r\n__global__ \r\nvoid transpose32(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    __shared__ T shrdMem[TILE_DIM][TILE_DIM+1];\r\n\r\n    unsigned lx = threadIdx.x;\r\n    unsigned ly = threadIdx.y;\r\n\r\n    unsigned gx = lx + blockDim.x * blockIdx.x;\r\n    unsigned gy = ly + TILE_DIM   * blockIdx.y;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n\r\n    gx = lx + blockDim.x * blockIdx.y;\r\n    gy = ly + TILE_DIM   * blockIdx.x;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nA minor modification has been done to the kernel from NVIDIA website by templating the matrix element data type such that we can use the same kernel with other integral data types.\r\n\r\nNow, we make the kernel applicable to matrices with any size by adding index checks. Following are the changes required to do so.\r\n<pre>\r\ntemplate<typename T>\r\n__global__\r\nvoid transposeNoSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (gx<dim0 && gy_<dim1)\r\n            shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (gx<dim1 && gy_<dim0)\r\n            out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\n\r\nThere is a reason the above kernel is named transposeNoSC which shall become evident as we proceed to next modification. Lets add back the modifications to handle the special case where matrix dimensions are multiples of 32.\r\n<pre>\r\ntemplate<typename T, bool is32Multiple>\r\n__global__\r\nvoid transposeSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (is32Multiple || (gx<dim0 && gy_<dim1))\r\n            shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (is32Multiple || (gx<dim1 && gy_<dim0))\r\n            out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nNotice the additional boolean template parameter <code>is32Multiple</code> that is used to short circuit the <code>if</code> condition within the for loops inside the kernel. Where ever the transposeSC is called upon with boolean parameter value as true in the code, the resultant compiled code essentially eliminates the <code>if</code> condition. Hence the above kernel was named <code>transposeSC</code>(SC - Short Circuited) and the former kernel was named transposeNoSC(No Short Circuit compile time template parameter).\r\n\r\nThe above kernel can be further optimized where any sort of index checks are completely avoided for all the blocks except the last one along a given dimension.\r\n\r\nSimilar modifications can be done to OpenCL kernel using the kernel compile options to the function call <a href=\"https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clBuildProgram.html\"><code>clBuildProgram</code></a> and ensuring the kernels are compiled only once. We have recently done a post on how to ensure OpenCL kernels are compiled only once in applications. You can find the article <a href=\"http://sandbox.arrayfire.com/templating-and-caching-opencl-kernels/\">here</a>.\r\n\r\nHere are some bandwidth analysis results using the above three kernels on different matrix dimensions.\r\n\r\n<iframe width=\"840\" height=\"616\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1tWhNejdDNFyYpGqs_x2Q7XVMlCYzPN1N6jDRIfKhYB8/pubchart?oid=682346979&amp;format=interactive\"></iframe>\r\n\r\nKeep an eye out for the future posts along similar theme where we would be discussing tips to optimize CUDA/OpenCL kernels for different algorithms.','CUDA Optimization tips for Matrix Transpose in real world applications','','inherit','open','open','','6786-revision-v1','','','2014-10-06 21:51:14','2014-10-06 21:51:14','',6786,'http://sandbox.arrayfire.com/6786-revision-v1/',0,'revision','',0),(6800,17729,'2014-10-06 22:01:55','2014-10-06 22:01:55','Computer algorithms are some what extra friendly towards data sizes that are powers of two. Especially GPU compute algorithms are very favorable to data sizes that are multiples of 32. However, in most real world situations it is rather hard to find the data to be so conveniently sized. In today\'s post, we shall be looking at one of such scenarios related to GPU compute. To be specific, we shall provide you with some tips on how to optimize matrix transpose algorithm for a GPU.\r\n\r\nLet us start with the transpose kernel available from <a href=\"http://devblogs.nvidia.com/parallelforall/efficient-matrix-transpose-cuda-cc/\">NVIDIA\'s Parallel Forall blog</a>. It has been optimized to avoid bank conflicts as well but only works on matrices with dimensions that are multiples of 32.\r\n<pre>\r\ntemplate<typename T>\r\n__global__ \r\nvoid transpose32(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    __shared__ T shrdMem[TILE_DIM][TILE_DIM+1];\r\n\r\n    unsigned lx = threadIdx.x;\r\n    unsigned ly = threadIdx.y;\r\n\r\n    unsigned gx = lx + blockDim.x * blockIdx.x;\r\n    unsigned gy = ly + TILE_DIM   * blockIdx.y;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n\r\n    gx = lx + blockDim.x * blockIdx.y;\r\n    gy = ly + TILE_DIM   * blockIdx.x;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nA minor modification has been done to the kernel from NVIDIA website by templating the matrix element data type such that we can use the same kernel with other integral data types.\r\n\r\nNow, we make the kernel applicable to matrices with any size by adding index checks. Following are the changes required to do so.\r\n<pre>\r\ntemplate<typename T>\r\n__global__\r\nvoid transposeNoSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (gx<dim0 && gy_<dim1)\r\n            shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (gx<dim1 && gy_<dim0)\r\n            out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\n\r\nThe above kernel will check the read write indices for validity. Now, lets add back the modifications to handle the special case where matrix dimensions are multiples of 32.\r\n<pre>\r\ntemplate<typename T, bool is32Multiple>\r\n__global__\r\nvoid transposeSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (is32Multiple || (gx<dim0 && gy_<dim1))\r\n            shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (is32Multiple || (gx<dim1 && gy_<dim0))\r\n            out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nNotice the additional boolean template parameter <code>is32Multiple</code> that is used to short circuit the <code>if</code> condition within the for loops inside the kernel. Where ever the transposeSC is called upon with boolean parameter value as true in the code, the resultant compiled code essentially eliminates the <code>if</code> condition.\r\n\r\nThe above kernel can be further optimized where any sort of index checks are completely avoided for all the blocks except the last one along a given dimension.\r\n\r\nSimilar modifications can be done to OpenCL kernel using the kernel compile options to the function call <a href=\"https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clBuildProgram.html\"><code>clBuildProgram</code></a> and ensuring the kernels are compiled only once. We have recently done a post on how to ensure OpenCL kernels are compiled only once in applications. You can find the article <a href=\"http://sandbox.arrayfire.com/templating-and-caching-opencl-kernels/\">here</a>.\r\n\r\nHere are some bandwidth analysis results using the above three kernels on different matrix dimensions.\r\n\r\n<iframe width=\"840\" height=\"616\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1tWhNejdDNFyYpGqs_x2Q7XVMlCYzPN1N6jDRIfKhYB8/pubchart?oid=682346979&amp;format=interactive\"></iframe>\r\n\r\nKeep an eye out for the future posts along similar theme where we would be discussing tips to optimize CUDA/OpenCL kernels for different algorithms.','CUDA Optimization tips for Matrix Transpose in real world applications','','inherit','open','open','','6786-revision-v1','','','2014-10-06 22:01:55','2014-10-06 22:01:55','',6786,'http://sandbox.arrayfire.com/6786-revision-v1/',0,'revision','',0),(6801,17729,'2014-10-06 22:05:34','2014-10-06 22:05:34','Computer algorithms are some what extra friendly towards data sizes that are powers of two. Especially GPU compute algorithms are very favorable to data sizes that are multiples of 32. However, in most real world situations it is rather hard to find the data to be so conveniently sized. In today\'s post, we shall be looking at one of such scenarios related to GPU compute. To be specific, we shall provide you with some tips on how to optimize matrix transpose algorithm for a GPU.\r\n\r\nLet us start with the transpose kernel available from <a href=\"http://devblogs.nvidia.com/parallelforall/efficient-matrix-transpose-cuda-cc/\">NVIDIA\'s Parallel Forall blog</a>. It has been optimized to avoid bank conflicts as well but only works on matrices with dimensions that are multiples of 32.\r\n<pre>\r\ntemplate<typename T>\r\n__global__ \r\nvoid transpose32(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    __shared__ T shrdMem[TILE_DIM][TILE_DIM+1];\r\n\r\n    unsigned lx = threadIdx.x;\r\n    unsigned ly = threadIdx.y;\r\n\r\n    unsigned gx = lx + blockDim.x * blockIdx.x;\r\n    unsigned gy = ly + TILE_DIM   * blockIdx.y;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n\r\n    gx = lx + blockDim.x * blockIdx.y;\r\n    gy = ly + TILE_DIM   * blockIdx.x;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nA minor modification has been done to the kernel from NVIDIA website by templating the matrix element data type such that we can use the same kernel with other integral data types.\r\n\r\nNow, we make the kernel applicable to matrices with any size by adding index checks. Following are the changes required to do so.\r\n<pre>\r\ntemplate<typename T>\r\n__global__\r\nvoid transposeNoSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (gx<dim0 && gy_<dim1)\r\n            shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (gx<dim1 && gy_<dim0)\r\n            out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\n\r\nThe above kernel will check the read write indices for validity. Now, lets add back the modifications to handle the special case where matrix dimensions are multiples of 32.\r\n<pre>\r\ntemplate<typename T, bool is32Multiple>\r\n__global__\r\nvoid transposeSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (is32Multiple || (gx<dim0 && gy_<dim1))\r\n            shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (is32Multiple || (gx<dim1 && gy_<dim0))\r\n            out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nNotice the additional boolean template parameter <code>is32Multiple</code> that is used to short circuit the <code>if</code> condition within the for loops inside the kernel. Where ever the transposeSC is called upon with boolean parameter value as true in the code, the resultant compiled code essentially eliminates the <code>if</code> condition.\r\n\r\nThe above kernel can be further optimized where any sort of index checks are completely avoided for all the blocks except the last one along a given dimension. Keep in mind that, adding more short circuit parameters will increase the compilation time correspondingly.\r\n\r\nSimilar modifications can be done to OpenCL kernel using the kernel compile options to the function call <a href=\"https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clBuildProgram.html\"><code>clBuildProgram</code></a> and ensuring the kernels are compiled only once. We have recently done a post on how to ensure OpenCL kernels are compiled only once in applications. You can find the article <a href=\"http://sandbox.arrayfire.com/templating-and-caching-opencl-kernels/\">here</a>.\r\n\r\nHere are some bandwidth analysis results using the above three kernels on different matrix dimensions.\r\n\r\n<iframe width=\"840\" height=\"616\" seamless frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1tWhNejdDNFyYpGqs_x2Q7XVMlCYzPN1N6jDRIfKhYB8/pubchart?oid=682346979&amp;format=interactive\"></iframe>\r\n\r\nKeep an eye out for the future posts along similar theme where we would be discussing tips to optimize CUDA/OpenCL kernels for different algorithms.','CUDA Optimization tips for Matrix Transpose in real world applications','','inherit','open','open','','6786-revision-v1','','','2014-10-06 22:05:34','2014-10-06 22:05:34','',6786,'http://sandbox.arrayfire.com/6786-revision-v1/',0,'revision','',0),(6802,17729,'2014-10-06 22:10:05','2014-10-06 22:10:05','Computer algorithms are some what extra friendly towards data sizes that are powers of two. Especially GPU compute algorithms are very favorable to data sizes that are multiples of 32. However, in most real world situations it is rather hard to find the data to be so conveniently sized. In today\'s post, we shall be looking at one of such scenarios related to GPU compute. To be specific, we shall provide you with some tips on how to optimize matrix transpose algorithm for a GPU.\r\n\r\nLet us start with the transpose kernel available from <a href=\"http://devblogs.nvidia.com/parallelforall/efficient-matrix-transpose-cuda-cc/\">NVIDIA\'s Parallel Forall blog</a>. It has been optimized to avoid bank conflicts as well but only works on matrices with dimensions that are multiples of 32.\r\n<pre>\r\ntemplate<typename T>\r\n__global__ \r\nvoid transpose32(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    __shared__ T shrdMem[TILE_DIM][TILE_DIM+1];\r\n\r\n    unsigned lx = threadIdx.x;\r\n    unsigned ly = threadIdx.y;\r\n\r\n    unsigned gx = lx + blockDim.x * blockIdx.x;\r\n    unsigned gy = ly + TILE_DIM   * blockIdx.y;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n\r\n    gx = lx + blockDim.x * blockIdx.y;\r\n    gy = ly + TILE_DIM   * blockIdx.x;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nA minor modification has been done to the kernel from NVIDIA website by templating the matrix element data type such that we can use the same kernel with other integral data types.\r\n\r\nNow, we make the kernel applicable to matrices with any size by adding index checks. Following are the changes required to do so.\r\n<pre>\r\ntemplate<typename T>\r\n__global__\r\nvoid transposeNoSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (gx<dim0 && gy_<dim1)\r\n            shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (gx<dim1 && gy_<dim0)\r\n            out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\n\r\nThe above kernel will check the read write indices for validity. Now, lets add back the modifications to handle the special case where matrix dimensions are multiples of 32.\r\n<pre>\r\ntemplate<typename T, bool is32Multiple>\r\n__global__\r\nvoid transposeSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (is32Multiple || (gx<dim0 && gy_<dim1))\r\n            shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (is32Multiple || (gx<dim1 && gy_<dim0))\r\n            out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nNotice the additional boolean template parameter <code>is32Multiple</code> that is used to short circuit the <code>if</code> condition within the for loops inside the kernel. Where ever the transposeSC is called upon with boolean parameter value as true in the code, the resultant compiled code essentially eliminates the <code>if</code> condition.\r\n\r\nThe above kernel can be further optimized where any sort of index checks are completely avoided for all the blocks except the last one along a given dimension. Keep in mind that, adding more short circuit parameters will increase the compilation time correspondingly.\r\n\r\nSimilar modifications can be done to OpenCL kernel using the kernel compile options to the function call <a href=\"https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clBuildProgram.html\"><code>clBuildProgram</code></a> and ensuring the kernels are compiled only once. We have recently done a post on how to ensure OpenCL kernels are compiled only once in applications. You can find the article <a href=\"http://sandbox.arrayfire.com/templating-and-caching-opencl-kernels/\">here</a>.\r\n\r\nHere are some bandwidth analysis results using the above three kernels on different matrix dimensions.\r\n<table>\r\n<tr>\r\n<th>Matrix Dimensions</th> <th>32 Multiple Dimensions</th> <th>Generic</th><th>Generic Optimized</th>\r\n</tr>\r\n<tr>\r\n<td> 1024x1024 Matrix</td> <td> 85.1881</td> <td>82.2105</td><td>84.4135</td>\r\n</tr>\r\n</table>\r\n\r\nKeep an eye out for the future posts along similar theme where we would be discussing tips to optimize CUDA/OpenCL kernels for different algorithms.','CUDA Optimization tips for Matrix Transpose in real world applications','','inherit','open','open','','6786-revision-v1','','','2014-10-06 22:10:05','2014-10-06 22:10:05','',6786,'http://sandbox.arrayfire.com/6786-revision-v1/',0,'revision','',0),(6803,17729,'2014-10-06 22:11:09','2014-10-06 22:11:09','Computer algorithms are some what extra friendly towards data sizes that are powers of two. Especially GPU compute algorithms are very favorable to data sizes that are multiples of 32. However, in most real world situations it is rather hard to find the data to be so conveniently sized. In today\'s post, we shall be looking at one of such scenarios related to GPU compute. To be specific, we shall provide you with some tips on how to optimize matrix transpose algorithm for a GPU.\r\n\r\nLet us start with the transpose kernel available from <a href=\"http://devblogs.nvidia.com/parallelforall/efficient-matrix-transpose-cuda-cc/\">NVIDIA\'s Parallel Forall blog</a>. It has been optimized to avoid bank conflicts as well but only works on matrices with dimensions that are multiples of 32.\r\n<pre>\r\ntemplate<typename T>\r\n__global__ \r\nvoid transpose32(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    __shared__ T shrdMem[TILE_DIM][TILE_DIM+1];\r\n\r\n    unsigned lx = threadIdx.x;\r\n    unsigned ly = threadIdx.y;\r\n\r\n    unsigned gx = lx + blockDim.x * blockIdx.x;\r\n    unsigned gy = ly + TILE_DIM   * blockIdx.y;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n\r\n    gx = lx + blockDim.x * blockIdx.y;\r\n    gy = ly + TILE_DIM   * blockIdx.x;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nA minor modification has been done to the kernel from NVIDIA website by templating the matrix element data type such that we can use the same kernel with other integral data types.\r\n\r\nNow, we make the kernel applicable to matrices with any size by adding index checks. Following are the changes required to do so.\r\n<pre>\r\ntemplate<typename T>\r\n__global__\r\nvoid transposeNoSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (gx<dim0 && gy_<dim1)\r\n            shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (gx<dim1 && gy_<dim0)\r\n            out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\n\r\nThe above kernel will check the read write indices for validity. Now, lets add back the modifications to handle the special case where matrix dimensions are multiples of 32.\r\n<pre>\r\ntemplate<typename T, bool is32Multiple>\r\n__global__\r\nvoid transposeSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (is32Multiple || (gx<dim0 && gy_<dim1))\r\n            shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (is32Multiple || (gx<dim1 && gy_<dim0))\r\n            out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nNotice the additional boolean template parameter <code>is32Multiple</code> that is used to short circuit the <code>if</code> condition within the for loops inside the kernel. Where ever the transposeSC is called upon with boolean parameter value as true in the code, the resultant compiled code essentially eliminates the <code>if</code> condition.\r\n\r\nThe above kernel can be further optimized where any sort of index checks are completely avoided for all the blocks except the last one along a given dimension. Keep in mind that, adding more short circuit parameters will increase the compilation time correspondingly.\r\n\r\nSimilar modifications can be done to OpenCL kernel using the kernel compile options to the function call <a href=\"https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clBuildProgram.html\"><code>clBuildProgram</code></a> and ensuring the kernels are compiled only once. We have recently done a post on how to ensure OpenCL kernels are compiled only once in applications. You can find the article <a href=\"http://sandbox.arrayfire.com/templating-and-caching-opencl-kernels/\">here</a>.\r\n\r\nHere are some bandwidth analysis results using the above three kernels on different matrix dimensions.\r\n<table>\r\n<tr>\r\n<th>Matrix Dimensions</th> <th>transpose32 Bandwidth</th> <th>transposeNoSC Bandwidth</th> <th>transposeSC Bandwidth</th>\r\n</tr>\r\n<tr>\r\n<td> 1024x1024 </td> <td> 85.1881</td> <td>82.2105</td><td>84.4135</td>\r\n</tr>\r\n</table>\r\n\r\nKeep an eye out for the future posts along similar theme where we would be discussing tips to optimize CUDA/OpenCL kernels for different algorithms.','CUDA Optimization tips for Matrix Transpose in real world applications','','inherit','open','open','','6786-revision-v1','','','2014-10-06 22:11:09','2014-10-06 22:11:09','',6786,'http://sandbox.arrayfire.com/6786-revision-v1/',0,'revision','',0),(6804,17729,'2014-10-06 22:11:46','2014-10-06 22:11:46','Computer algorithms are some what extra friendly towards data sizes that are powers of two. Especially GPU compute algorithms are very favorable to data sizes that are multiples of 32. However, in most real world situations it is rather hard to find the data to be so conveniently sized. In today\'s post, we shall be looking at one of such scenarios related to GPU compute. To be specific, we shall provide you with some tips on how to optimize matrix transpose algorithm for a GPU.\r\n\r\nLet us start with the transpose kernel available from <a href=\"http://devblogs.nvidia.com/parallelforall/efficient-matrix-transpose-cuda-cc/\">NVIDIA\'s Parallel Forall blog</a>. It has been optimized to avoid bank conflicts as well but only works on matrices with dimensions that are multiples of 32.\r\n<pre>\r\ntemplate<typename T>\r\n__global__ \r\nvoid transpose32(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    __shared__ T shrdMem[TILE_DIM][TILE_DIM+1];\r\n\r\n    unsigned lx = threadIdx.x;\r\n    unsigned ly = threadIdx.y;\r\n\r\n    unsigned gx = lx + blockDim.x * blockIdx.x;\r\n    unsigned gy = ly + TILE_DIM   * blockIdx.y;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n\r\n    gx = lx + blockDim.x * blockIdx.y;\r\n    gy = ly + TILE_DIM   * blockIdx.x;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nA minor modification has been done to the kernel from NVIDIA website by templating the matrix element data type such that we can use the same kernel with other integral data types.\r\n\r\nNow, we make the kernel applicable to matrices with any size by adding index checks. Following are the changes required to do so.\r\n<pre>\r\ntemplate<typename T>\r\n__global__\r\nvoid transposeNoSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (gx<dim0 && gy_<dim1)\r\n            shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (gx<dim1 && gy_<dim0)\r\n            out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\n\r\nThe above kernel will check the read write indices for validity. Now, lets add back the modifications to handle the special case where matrix dimensions are multiples of 32.\r\n<pre>\r\ntemplate<typename T, bool is32Multiple>\r\n__global__\r\nvoid transposeSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (is32Multiple || (gx<dim0 && gy_<dim1))\r\n            shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (is32Multiple || (gx<dim1 && gy_<dim0))\r\n            out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nNotice the additional boolean template parameter <code>is32Multiple</code> that is used to short circuit the <code>if</code> condition within the for loops inside the kernel. Where ever the transposeSC is called upon with boolean parameter value as true in the code, the resultant compiled code essentially eliminates the <code>if</code> condition.\r\n\r\nThe above kernel can be further optimized where any sort of index checks are completely avoided for all the blocks except the last one along a given dimension. Keep in mind that, adding more short circuit parameters will increase the compilation time correspondingly.\r\n\r\nSimilar modifications can be done to OpenCL kernel using the kernel compile options to the function call <a href=\"https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clBuildProgram.html\"><code>clBuildProgram</code></a> and ensuring the kernels are compiled only once. We have recently done a post on how to ensure OpenCL kernels are compiled only once in applications. You can find the article <a href=\"http://sandbox.arrayfire.com/templating-and-caching-opencl-kernels/\">here</a>.\r\n\r\nHere are some bandwidth analysis results using the above three kernels on different matrix dimensions.\r\n<table>\r\n<tr>\r\n<th>Matrix Dimensions</th> <th>transpose32 Bandwidth</th> <th>transposeNoSC Bandwidth</th> <th>transposeSC Bandwidth</th>\r\n</tr>\r\n<tr>\r\n<td> 1024x1024 </td> <td> 85.1881 GB/s</td> <td>82.2105 GB/s</td><td>84.4135 GB/s</td>\r\n</tr>\r\n</table>\r\n\r\nKeep an eye out for the future posts along similar theme where we would be discussing tips to optimize CUDA/OpenCL kernels for different algorithms.','CUDA Optimization tips for Matrix Transpose in real world applications','','inherit','open','open','','6786-revision-v1','','','2014-10-06 22:11:46','2014-10-06 22:11:46','',6786,'http://sandbox.arrayfire.com/6786-revision-v1/',0,'revision','',0),(6805,17729,'2014-10-06 22:12:42','2014-10-06 22:12:42','Computer algorithms are some what extra friendly towards data sizes that are powers of two. Especially GPU compute algorithms are very favorable to data sizes that are multiples of 32. However, in most real world situations it is rather hard to find the data to be so conveniently sized. In today\'s post, we shall be looking at one of such scenarios related to GPU compute. To be specific, we shall provide you with some tips on how to optimize matrix transpose algorithm for a GPU.\r\n\r\nLet us start with the transpose kernel available from <a href=\"http://devblogs.nvidia.com/parallelforall/efficient-matrix-transpose-cuda-cc/\">NVIDIA\'s Parallel Forall blog</a>. It has been optimized to avoid bank conflicts as well but only works on matrices with dimensions that are multiples of 32.\r\n<pre>\r\ntemplate<typename T>\r\n__global__ \r\nvoid transpose32(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    __shared__ T shrdMem[TILE_DIM][TILE_DIM+1];\r\n\r\n    unsigned lx = threadIdx.x;\r\n    unsigned ly = threadIdx.y;\r\n\r\n    unsigned gx = lx + blockDim.x * blockIdx.x;\r\n    unsigned gy = ly + TILE_DIM   * blockIdx.y;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n\r\n    gx = lx + blockDim.x * blockIdx.y;\r\n    gy = ly + TILE_DIM   * blockIdx.x;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nA minor modification has been done to the kernel from NVIDIA website by templating the matrix element data type such that we can use the same kernel with other integral data types.\r\n\r\nNow, we make the kernel applicable to matrices with any size by adding index checks. Following are the changes required to do so.\r\n<pre>\r\ntemplate<typename T>\r\n__global__\r\nvoid transposeNoSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (gx<dim0 && gy_<dim1)\r\n            shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (gx<dim1 && gy_<dim0)\r\n            out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\n\r\nThe above kernel will check the read write indices for validity. Now, lets add back the modifications to handle the special case where matrix dimensions are multiples of 32.\r\n<pre>\r\ntemplate<typename T, bool is32Multiple>\r\n__global__\r\nvoid transposeSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (is32Multiple || (gx<dim0 && gy_<dim1))\r\n            shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (is32Multiple || (gx<dim1 && gy_<dim0))\r\n            out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nNotice the additional boolean template parameter <code>is32Multiple</code> that is used to short circuit the <code>if</code> condition within the for loops inside the kernel. Where ever the transposeSC is called upon with boolean parameter value as true in the code, the resultant compiled code essentially eliminates the <code>if</code> condition.\r\n\r\nThe above kernel can be further optimized where any sort of index checks are completely avoided for all the blocks except the last one along a given dimension. Keep in mind that, adding more short circuit parameters will increase the compilation time correspondingly.\r\n\r\nSimilar modifications can be done to OpenCL kernel using the kernel compile options to the function call <a href=\"https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clBuildProgram.html\"><code>clBuildProgram</code></a> and ensuring the kernels are compiled only once. We have recently done a post on how to ensure OpenCL kernels are compiled only once in applications. You can find the article <a href=\"http://sandbox.arrayfire.com/templating-and-caching-opencl-kernels/\">here</a>.\r\n\r\nHere are some bandwidth analysis results using the above three kernels on different matrix dimensions.\r\n<table>\r\n<tr>\r\n<th>Matrix Dimensions</th> <th>transpose32 Bandwidth</th> <th>transposeNoSC Bandwidth</th> <th>transposeSC Bandwidth</th>\r\n</tr>\r\n<tr>\r\n<td> 1024x1024 </td> <td> 85.18 GB/s</td> <td>82.21 GB/s</td><td>84.41 GB/s</td>\r\n</tr>\r\n</table>\r\n\r\nKeep an eye out for the future posts along similar theme where we would be discussing tips to optimize CUDA/OpenCL kernels for different algorithms.','CUDA Optimization tips for Matrix Transpose in real world applications','','inherit','open','open','','6786-revision-v1','','','2014-10-06 22:12:42','2014-10-06 22:12:42','',6786,'http://sandbox.arrayfire.com/6786-revision-v1/',0,'revision','',0),(6808,7,'2014-10-07 13:16:07','2014-10-07 13:16:07','Computer algorithms are extra friendly towards data sizes that are powers of two. GPU compute algorithms work particularly well with data sizes that are multiples of 32. In most real-world situations, however, data is rarely so conveniently sized. In today\'s post, we\'ll be looking at one such scenario related to GPU compute. Specifically, we\'ll provide you with some tips on how to optimize matrix transpose algorithm for a GPU.\r\n\r\nLet\'s start with the transpose kernel available from <a href=\"http://devblogs.nvidia.com/parallelforall/efficient-matrix-transpose-cuda-cc/\">NVIDIA\'s Parallel Forall blog</a>. It\'s been optimized to avoid bank conflicts as well, but only works on matrices with dimensions that are multiples of 32.\r\n<pre>template\r\n__global__ \r\nvoid transpose32(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    __shared__ T shrdMem[TILE_DIM][TILE_DIM+1];\r\n\r\n    unsigned lx = threadIdx.x;\r\n    unsigned ly = threadIdx.y;\r\n\r\n    unsigned gx = lx + blockDim.x * blockIdx.x;\r\n    unsigned gy = ly + TILE_DIM   * blockIdx.y;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat &lt; TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n\r\n    gx = lx + blockDim.x * blockIdx.y;\r\n    gy = ly + TILE_DIM   * blockIdx.x;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat &lt; TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nA minor modification has been made to the kernel from the NVIDIA website by templating the matrix element data type such that we can use the same kernel with other integral data types.\r\n\r\nNow we make the kernel applicable to matrices with any size by adding index checks. This requires making the following changes.\r\n<pre>template\r\n__global__\r\nvoid transposeNoSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat &lt; TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (gx</pre>\r\nThe above kernel will check the read write indices for validity. Now let\'s add back the modifications to handle the special case where matrix dimensions are multiples of 32.\r\n<pre>template\r\n__global__\r\nvoid transposeSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat &lt; TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (is32Multiple || (gx</pre>\r\nNotice the additional boolean template parameter <code>is32Multiple</code> that is used to short circuit the <code>if</code> condition within the for loops inside the kernel. Wherever the transposeSC is called upon with boolean parameter value as true in the code, the resultant compiled code essentially eliminates the <code>if</code> condition.\r\n\r\nThe above kernel can be further optimized where any sort of index checks are completely avoided for all the blocks except the last one along a given dimension. Keep in mind that adding more short circuit parameters will increase the compilation time correspondingly.\r\n\r\nSimilar modifications can be done to OpenCL kernel using the kernel compile options to the function call <a href=\"https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clBuildProgram.html\"><code>clBuildProgram</code></a> and ensuring the kernels are compiled only once. We have recently done a post on how to ensure OpenCL kernels are compiled only once in applications. You can find the article <a href=\"http://sandbox.arrayfire.com/templating-and-caching-opencl-kernels/\">here</a>.\r\n\r\nHere are some bandwidth analysis results using the above three kernels on different matrix dimensions.\r\n<table>\r\n<tbody>\r\n<tr>\r\n<th>Matrix Dimensions</th>\r\n<th>transpose32 Bandwidth</th>\r\n<th>transposeNoSC Bandwidth</th>\r\n<th>transposeSC Bandwidth</th>\r\n</tr>\r\n<tr>\r\n<td>1024x1024</td>\r\n<td>85.18 GB/s</td>\r\n<td>82.21 GB/s</td>\r\n<td>84.41 GB/s</td>\r\n</tr>\r\n</tbody>\r\n</table>\r\nKeep an eye out for future posts along this theme, in which we\'ll discuss tips to optimize CUDA/OpenCL kernels for different algorithms.','CUDA Optimization tips for Matrix Transpose in real world applications','','inherit','open','open','','6786-revision-v1','','','2014-10-07 13:16:07','2014-10-07 13:16:07','',6786,'http://sandbox.arrayfire.com/6786-revision-v1/',0,'revision','',0),(6809,17729,'2014-10-07 14:33:51','2014-10-07 14:33:51','Computer algorithms are extra friendly towards data sizes that are powers of two. GPU compute algorithms work particularly well with data sizes that are multiples of 32. In most real-world situations, however, data is rarely so conveniently sized. In today\'s post, we\'ll be looking at one such scenario related to GPU compute. Specifically, we\'ll provide you with some tips on how to optimize matrix transpose algorithm for a GPU.\r\n\r\nLet\'s start with the transpose kernel available from <a href=\"http://devblogs.nvidia.com/parallelforall/efficient-matrix-transpose-cuda-cc/\">NVIDIA\'s Parallel Forall blog</a>. It\'s been optimized to avoid bank conflicts as well, but only works on matrices with dimensions that are multiples of 32.\r\n<pre>template\r\n__global__ \r\nvoid transpose32(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    __shared__ T shrdMem[TILE_DIM][TILE_DIM+1];\r\n\r\n    unsigned lx = threadIdx.x;\r\n    unsigned ly = threadIdx.y;\r\n\r\n    unsigned gx = lx + blockDim.x * blockIdx.x;\r\n    unsigned gy = ly + TILE_DIM   * blockIdx.y;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat &lt; TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n\r\n    gx = lx + blockDim.x * blockIdx.y;\r\n    gy = ly + TILE_DIM   * blockIdx.x;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat &lt; TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nA minor modification has been made to the kernel from the NVIDIA website by templating the matrix element data type such that we can use the same kernel with other integral data types.\r\n\r\nNow we make the kernel applicable to matrices with any size by adding index checks. This requires making the following changes.\r\n<pre>template\r\n__global__\r\nvoid transposeNoSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat &lt; TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (gx</pre>\r\nThe above kernel will check the read write indices for validity. Now let\'s add back the modifications to handle the special case where matrix dimensions are multiples of 32.\r\n<pre>template\r\n__global__\r\nvoid transposeSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat &lt; TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (is32Multiple || (gx</pre>\r\nNotice the additional boolean template parameter <code>is32Multiple</code> that is used to short circuit the <code>if</code> condition within the for loops inside the kernel. Wherever the transposeSC is called upon with boolean parameter value as true in the code, the resultant compiled code essentially eliminates the <code>if</code> condition.\r\n\r\nThe above kernel can be further optimized where any sort of index checks are completely avoided for all the blocks except the last one along a given dimension. Keep in mind that adding more short circuit parameters will increase the compilation time correspondingly.\r\n\r\nSimilar modifications can be done to OpenCL kernel using the kernel compile options to the function call <a href=\"https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clBuildProgram.html\"><code>clBuildProgram</code></a> and ensuring the kernels are compiled only once. We have recently done a post on how to ensure OpenCL kernels are compiled only once in applications. You can find the article <a href=\"http://sandbox.arrayfire.com/templating-and-caching-opencl-kernels/\">here</a>.\r\n\r\nHere are some bandwidth analysis results using the above three kernels on a 1024x1024 matrix.\r\n<table>\r\n<tbody>\r\n<tr>\r\n<th>Matrix Dimensions</th>\r\n<th>transpose32 Bandwidth</th>\r\n<th>transposeNoSC Bandwidth</th>\r\n<th>transposeSC Bandwidth</th>\r\n</tr>\r\n<tr>\r\n<td>1024x1024</td>\r\n<td>85.18 GB/s</td>\r\n<td>82.21 GB/s</td>\r\n<td>84.41 GB/s</td>\r\n</tr>\r\n</tbody>\r\n</table>\r\nKeep an eye out for future posts along this theme, in which we\'ll discuss tips to optimize CUDA/OpenCL kernels for different algorithms.','CUDA Optimization tips for Matrix Transpose in real world applications','','inherit','open','open','','6786-revision-v1','','','2014-10-07 14:33:51','2014-10-07 14:33:51','',6786,'http://sandbox.arrayfire.com/6786-revision-v1/',0,'revision','',0),(6810,17729,'2014-10-07 14:38:14','2014-10-07 14:38:14','Computer algorithms are extra friendly towards data sizes that are powers of two. GPU compute algorithms work particularly well with data sizes that are multiples of 32. In most real-world situations, however, data is rarely so conveniently sized. In today\'s post, we\'ll be looking at one such scenario related to GPU compute. Specifically, we\'ll provide you with some tips on how to optimize matrix transpose algorithm for a GPU.\r\n\r\nLet\'s start with the transpose kernel available from <a href=\"http://devblogs.nvidia.com/parallelforall/efficient-matrix-transpose-cuda-cc/\">NVIDIA\'s Parallel Forall blog</a>. It\'s been optimized to avoid bank conflicts as well, but only works on matrices with dimensions that are multiples of 32.\r\n<pre>\r\ntemplate<typename T>\r\n__global__\r\nvoid transpose32(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    __shared__ T shrdMem[TILE_DIM][TILE_DIM+1];\r\n\r\n    unsigned lx = threadIdx.x;\r\n    unsigned ly = threadIdx.y;\r\n\r\n    unsigned gx = lx + blockDim.x * blockIdx.x;\r\n    unsigned gy = ly + TILE_DIM   * blockIdx.y;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n\r\n    gx = lx + blockDim.x * blockIdx.y;\r\n    gy = ly + TILE_DIM   * blockIdx.x;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nA minor modification has been made to the kernel from the NVIDIA website by templating the matrix element data type such that we can use the same kernel with other integral data types.\r\n\r\nNow we make the kernel applicable to matrices with any size by adding index checks. This requires making the following changes.\r\n<pre>\r\ntemplate<typename T>\r\n__global__\r\nvoid transposeNoSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (gx<dim0 && gy_<dim1)\r\n            shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (gx<dim1 && gy_<dim0)\r\n            out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nThe above kernel will check the read write indices for validity. Now let\'s add back the modifications to handle the special case where matrix dimensions are multiples of 32.\r\n<pre>\r\ntemplate<typename T, bool is32Multiple>\r\n__global__\r\nvoid transposeSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (is32Multiple || (gx<dim0 && gy_<dim1))\r\n            shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (is32Multiple || (gx<dim1 && gy_<dim0))\r\n            out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nNotice the additional boolean template parameter <code>is32Multiple</code> that is used to short circuit the <code>if</code> condition within the for loops inside the kernel. Wherever the transposeSC is called upon with boolean parameter value as true in the code, the resultant compiled code essentially eliminates the <code>if</code> condition.\r\n\r\nThe above kernel can be further optimized where any sort of index checks are completely avoided for all the blocks except the last one along a given dimension. Keep in mind that adding more short circuit parameters will increase the compilation time correspondingly.\r\n\r\nSimilar modifications can be done to OpenCL kernel using the kernel compile options to the function call <a href=\"https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clBuildProgram.html\"><code>clBuildProgram</code></a> and ensuring the kernels are compiled only once. We have recently done a post on how to ensure OpenCL kernels are compiled only once in applications. You can find the article <a href=\"http://sandbox.arrayfire.com/templating-and-caching-opencl-kernels/\">here</a>.\r\n\r\nHere are some bandwidth analysis results using the above three kernels on a 1024x1024 matrix.\r\n<table>\r\n<tbody>\r\n<tr>\r\n<th>Matrix Dimensions</th>\r\n<th>transpose32 Bandwidth</th>\r\n<th>transposeNoSC Bandwidth</th>\r\n<th>transposeSC Bandwidth</th>\r\n</tr>\r\n<tr>\r\n<td>1024x1024</td>\r\n<td>85.18 GB/s</td>\r\n<td>82.21 GB/s</td>\r\n<td>84.41 GB/s</td>\r\n</tr>\r\n</tbody>\r\n</table>\r\nKeep an eye out for future posts along this theme, in which we\'ll discuss tips to optimize CUDA/OpenCL kernels for different algorithms.','CUDA Optimization tips for Matrix Transpose in real world applications','','inherit','open','open','','6786-revision-v1','','','2014-10-07 14:38:14','2014-10-07 14:38:14','',6786,'http://sandbox.arrayfire.com/6786-revision-v1/',0,'revision','',0),(6812,11,'2014-10-07 15:00:46','2014-10-07 15:00:46','<h3>Introduction</h3>\r\nZero Copy access has been a part of the CUDA Toolkit for a long time (~2009). However, there were very few applications using the capability because, with time, GPU memory has become reasonably large. The only applications using zero copy access were mainly ones with extremely large memory requirements, such as database processing.\r\n\r\nZero Copy is a way to map host memory and access it directly over PCIe without doing an explicit memory transfer. It allows CUDA kernels to directly access host memory. Instead of reading data from global memory (limit ~200GB/s), data would be read over PCIe and be limited by PCIe bandwidth (upto 16GB/s). Hence there was no real performance advantage for most applications.\r\n\r\nHowever, with the release of the <a href=\"https://developer.nvidia.com/jetson-tk1\" title=\"Jetson TK1\" target=\"_blank\">Jetson TK1</a> board with the <a href=\"http://www.nvidia.com/object/tegra-k1-processor.html\" title=\"Tegra K1\" target=\"_blank\">Tegra K1</a> processor, this has become really useful. The Jetson board has 2GB of physical memory that is shared by the ARM CPU and the CUDA GPU. If a cudaMemcpy Host->Device is done on the Jetson, the memory is simply being copied to a new location on the same physical memory and retrieving a CUDA pointer from it. In such a scenario, Zero Copy access is perfect.\r\n\r\n<h3>Standard CUDA Pipeline</h3>\r\nLets take a look at a standard pipeline in CUDA.\r\n\r\n<pre>\r\n// Host Arrays\r\nfloat* h_in  = new float[sizeIn];\r\nfloat* h_out = new float[sizeOut];\r\n\r\n//Process h_in\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n\r\n// Allocate memory on the device\r\ncudaMalloc((void **) &d_in,  sizeIn ));\r\ncudaMalloc((void **) &d_out, sizeOut));\r\n\r\n// Copy array contents of input from the host (CPU) to the device (GPU)\r\ncudaMemcpy(d_in, h_in, sizeX * sizeY * sizeof(float), cudaMemcpyHostToDevice);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// Copy result back\r\ncudaMemcpy(h_out, d_out, sizeOut, cudaMemcpyDeviceToHost);\r\n\r\n// Continue processing on host using h_out\r\n\r\n</pre>\r\n\r\n<h3>Zero Copy Access CUDA Pipeline</h3>\r\nNow lets look at a pipeline using Zero Copy access.\r\n\r\n<pre>\r\n// Set flag to enable zero copy access\r\ncudaSetDeviceFlags(cudaDeviceMapHost);\r\n\r\n// Host Arrays\r\nfloat* h_in  = NULL;\r\nfloat* h_out = NULL;\r\n\r\n// Process h_in\r\n\r\n// Allocate host memory using CUDA allocation calls\r\ncudaHostAlloc((void **)&h_in,  sizeIn,  cudaHostAllocMapped);\r\ncudaHostAlloc((void **)&h_out, sizeOut, cudaHostAllocMapped);\r\n\r\n// Device arrays\r\nfloat *d_out, *d_in;\r\n// Get device pointer from host memory. No allocation or memcpy\r\ncudaHostGetDevicePointer((void **)&d_in,  (void *) h_in , 0);\r\ncudaHostGetDevicePointer((void **)&d_out, (void *) h_out, 0);\r\n\r\n// Launch the GPU kernel\r\nkernel<<<blocks, threads>>>(d_out, d_in);\r\n\r\n// No need to copy d_out back\r\n// Continue processing on host using h_out\r\n</pre>\r\n\r\nAs you can see, it is considerably simpler to use zero copy access on the Tegra from a coding perspective. The kernel code remains the same. You can also allocate host memory using the cudaHostAlloc call without using zero copy access. This allows the fast pinned memory transfer when doing a cudaMemcpy.\r\n\r\n<h3>Results</h3>\r\nSo how does it perform?\r\n\r\nWhen I ran a transpose kernel using zero copy vs using standard pipeline, I achieve a the following results for a 4096 x 4096 matrix. \r\n\r\n<table style=\"width:50%\">\r\n  <th>\r\n    <td>Pipeline</td>\r\n    <td>Bandwidth (GB/s)</td> \r\n    <td>Time (ms)</td>\r\n  </th>  \r\n  <tr>\r\n    <td></td>\r\n    <td>Standard Pipeline</td>\r\n    <td>3.0</td> \r\n    <td>45</td>\r\n  </tr>\r\n  <tr>\r\n    <td></td>\r\n    <td>Zero Copy Pipeline</td>\r\n    <td>5.8</td> \r\n    <td>23</td>\r\n  </tr>\r\n</table>\r\n\r\nThe bandwidth of a device-to-device copy kernel on the Tegra is ~6.6 GB/s.\r\n\r\nThe results are tremendous. Any result, that is as good as or better than the standard pipeline is worth doing using zero copy. This saves memory usage as it does not require duplication on host and device.\r\n\r\nThere a a few caveats though. If you are running multiple kernels without modifying the data on the host side, it may be wise to run the standard pipeline. There is no right answer. Since it is so simple to modify the code for memory transfer vs zero copy pointers, it would be best to run both techniques and benchmark them.\r\n\r\n<h3>Applications</h3>\r\nWe these results, we believe the Tegra K1 is a great option for streaming applications. Most streaming applications run image or signal processing algorithms, with a limitation of running at either 60 or 30 frames per second. On desktops, although the kernels themselves run well under the required time, the memory transfer times form a significant portion.\r\n\r\nThis is why the Tegra K1 can be great. We save 100% of the memory transfer times used by discrete GPUs by using Zero Copy access. This allow the Tegra K1 GPU to do the streaming operations within the performance constraints even though is is considerable under-powered compared to most desktop GPUs.\r\n\r\nOf course, streaming is not the only application. There are many more, and if do happen to try one or want to try your application on the K1, we would be extremely excited to hear about it.\r\n\r\n<h3>Code</h3>\r\nYou can find the entire code from my transpose exercise here: <a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/transpose.zip\">transpose</a>.\r\nNote: If you wish to run this on a x86 system, make sure you change the compute version in the makefile from 32 to 30 (for Kepler).','Zero Copy on Tegra K1','','inherit','open','open','','6757-revision-v1','','','2014-10-07 15:00:46','2014-10-07 15:00:46','',6757,'http://sandbox.arrayfire.com/6757-revision-v1/',0,'revision','',0),(6813,17729,'2014-10-07 20:53:33','2014-10-07 20:53:33','Computer algorithms are extra friendly towards data sizes that are powers of two. GPU compute algorithms work particularly well with data sizes that are multiples of 32. In most real-world situations, however, data is rarely so conveniently sized. In today\'s post, we\'ll be looking at one such scenario related to GPU compute. Specifically, we\'ll provide you with some tips on how to optimize matrix transpose algorithm for a GPU.\n\nLet\'s start with the transpose kernel available from <a href=\"http://devblogs.nvidia.com/parallelforall/efficient-matrix-transpose-cuda-cc/\">NVIDIA\'s Parallel Forall blog</a>. It\'s been optimized to avoid bank conflicts as well, but only works on matrices with dimensions that are multiples of 32.\n<pre>\ntemplate<typename T>\n__global__\nvoid transpose32(T * out, const T * in, unsigned dim0, unsigned dim1)\n{\n    __shared__ T shrdMem[TILE_DIM][TILE_DIM+1];\n\n    unsigned lx = threadIdx.x;\n    unsigned ly = threadIdx.y;\n\n    unsigned gx = lx + blockDim.x * blockIdx.x;\n    unsigned gy = ly + TILE_DIM   * blockIdx.y;\n\n#pragma unroll\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\n        unsigned gy_ = gy+repeat;\n        shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\n    }\n    __syncthreads();\n\n    gx = lx + blockDim.x * blockIdx.y;\n    gy = ly + TILE_DIM   * blockIdx.x;\n\n#pragma unroll\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\n        unsigned gy_ = gy+repeat;\n        out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\n    }\n}\n</pre>\nA minor modification has been made to the kernel from the NVIDIA website by templating the matrix element data type such that we can use the same kernel with other integral data types. The variable <code>TILE_DIM</code> should have static constant qualifier and be initialized with a value of 32.\n\nNow we make the kernel applicable to matrices with any size by adding index checks. This requires making the following changes.\n<pre>\ntemplate<typename T>\n__global__\nvoid transposeNoSC(T * out, const T * in, unsigned dim0, unsigned dim1)\n{\n    /*\n     * same code as transpose32 kernel\n     */\n#pragma unroll\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\n        unsigned gy_ = gy+repeat;\n        if (gx<dim0 && gy_<dim1)\n            shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\n    }\n    __syncthreads();\n    /*\n     * same code as transpose32 kernel\n     */\n#pragma unroll\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\n        unsigned gy_ = gy+repeat;\n        if (gx<dim1 && gy_<dim0)\n            out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\n    }\n}\n</pre>\nThe above kernel will check the read write indices for validity. Now let\'s add back the modifications to handle the special case where matrix dimensions are multiples of 32.\n<pre>\ntemplate<typename T, bool is32Multiple>\n__global__\nvoid transposeSC(T * out, const T * in, unsigned dim0, unsigned dim1)\n{\n    /*\n     * same code as transpose32 kernel\n     */\n#pragma unroll\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\n        unsigned gy_ = gy+repeat;\n        if (is32Multiple || (gx<dim0 && gy_<dim1))\n            shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\n    }\n    __syncthreads();\n    /*\n     * same code as transpose32 kernel\n     */\n#pragma unroll\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\n        unsigned gy_ = gy+repeat;\n        if (is32Multiple || (gx<dim1 && gy_<dim0))\n            out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\n    }\n}\n</pre>\nNotice the additional boolean template parameter <code>is32Multiple</code> that is used to short circuit the <code>if</code> condition within the for loops inside the kernel. Wherever the transposeSC is called upon with boolean parameter value as true in the code, the resultant compiled code essentially eliminates the <code>if</code> condition.\n\nThe above kernel can be further optimized where any sort of index checks are completely avoided for all the blocks except the last one along a given dimension. Keep in mind that adding more short circuit parameters will increase the compilation time correspondingly.\n\nSimilar modifications can be done to OpenCL kernel using the kernel compile options to the function call <a href=\"https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clBuildProgram.html\"><code>clBuildProgram</code></a> and ensuring the kernels are compiled only once. We have recently done a post on how to ensure OpenCL kernels are compiled only once in applications. You can find the article <a href=\"http://sandbox.arrayfire.com/templating-and-caching-opencl-kernels/\">here</a>.\n\nHere are some bandwidth analysis results using the above three kernels on a 1024x1024 matrix. For the results, we used a launch configuration of 32x8 threads per block where each block processed a 32x32 sub-matrix from the input matrix.\n<table>\n<tbody>\n<tr>\n<th>Matrix Dimensions</th>\n<th>transpose32 Bandwidth</th>\n<th>transposeNoSC Bandwidth</th>\n<th>transposeSC Bandwidth</th>\n</tr>\n<tr>\n<td>1024x1024</td>\n<td>85.18 GB/s</td>\n<td>82.21 GB/s</td>\n<td>84.41 GB/s</td>\n</tr>\n</tbody>\n</table>\nKeep an eye out for future posts along this theme, in which we\'ll discuss tips to optimize CUDA/OpenCL kernels for different algorithms.','CUDA Optimization tips for Matrix Transpose in real world applications','','inherit','open','open','','6786-autosave-v1','','','2014-10-07 20:53:33','2014-10-07 20:53:33','',6786,'http://sandbox.arrayfire.com/6786-autosave-v1/',0,'revision','',0),(6814,17729,'2014-10-07 20:53:39','2014-10-07 20:53:39','Computer algorithms are extra friendly towards data sizes that are powers of two. GPU compute algorithms work particularly well with data sizes that are multiples of 32. In most real-world situations, however, data is rarely so conveniently sized. In today\'s post, we\'ll be looking at one such scenario related to GPU compute. Specifically, we\'ll provide you with some tips on how to optimize matrix transpose algorithm for a GPU.\r\n\r\nLet\'s start with the transpose kernel available from <a href=\"http://devblogs.nvidia.com/parallelforall/efficient-matrix-transpose-cuda-cc/\">NVIDIA\'s Parallel Forall blog</a>. It\'s been optimized to avoid bank conflicts as well, but only works on matrices with dimensions that are multiples of 32.\r\n<pre>\r\ntemplate<typename T>\r\n__global__\r\nvoid transpose32(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    __shared__ T shrdMem[TILE_DIM][TILE_DIM+1];\r\n\r\n    unsigned lx = threadIdx.x;\r\n    unsigned ly = threadIdx.y;\r\n\r\n    unsigned gx = lx + blockDim.x * blockIdx.x;\r\n    unsigned gy = ly + TILE_DIM   * blockIdx.y;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n\r\n    gx = lx + blockDim.x * blockIdx.y;\r\n    gy = ly + TILE_DIM   * blockIdx.x;\r\n\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nA minor modification has been made to the kernel from the NVIDIA website by templating the matrix element data type such that we can use the same kernel with other integral data types. The variable <code>TILE_DIM</code> should have static constant qualifier and be initialized with a value of 32.\r\n\r\nNow we make the kernel applicable to matrices with any size by adding index checks. This requires making the following changes.\r\n<pre>\r\ntemplate<typename T>\r\n__global__\r\nvoid transposeNoSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (gx<dim0 && gy_<dim1)\r\n            shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (gx<dim1 && gy_<dim0)\r\n            out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nThe above kernel will check the read write indices for validity. Now let\'s add back the modifications to handle the special case where matrix dimensions are multiples of 32.\r\n<pre>\r\ntemplate<typename T, bool is32Multiple>\r\n__global__\r\nvoid transposeSC(T * out, const T * in, unsigned dim0, unsigned dim1)\r\n{\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (is32Multiple || (gx<dim0 && gy_<dim1))\r\n            shrdMem[ly + repeat][lx] = in[gy_ * dim0 + gx];\r\n    }\r\n    __syncthreads();\r\n    /*\r\n     * same code as transpose32 kernel\r\n     */\r\n#pragma unroll\r\n    for (unsigned repeat = 0; repeat < TILE_DIM; repeat += blockDim.y) {\r\n        unsigned gy_ = gy+repeat;\r\n        if (is32Multiple || (gx<dim1 && gy_<dim0))\r\n            out[gy_ * dim0 + gx] = shrdMem[lx][ly + repeat];\r\n    }\r\n}\r\n</pre>\r\nNotice the additional boolean template parameter <code>is32Multiple</code> that is used to short circuit the <code>if</code> condition within the for loops inside the kernel. Wherever the transposeSC is called upon with boolean parameter value as true in the code, the resultant compiled code essentially eliminates the <code>if</code> condition.\r\n\r\nThe above kernel can be further optimized where any sort of index checks are completely avoided for all the blocks except the last one along a given dimension. Keep in mind that adding more short circuit parameters will increase the compilation time correspondingly.\r\n\r\nSimilar modifications can be done to OpenCL kernel using the kernel compile options to the function call <a href=\"https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clBuildProgram.html\"><code>clBuildProgram</code></a> and ensuring the kernels are compiled only once. We have recently done a post on how to ensure OpenCL kernels are compiled only once in applications. You can find the article <a href=\"http://sandbox.arrayfire.com/templating-and-caching-opencl-kernels/\">here</a>.\r\n\r\nHere are some bandwidth analysis results using the above three kernels on a 1024x1024 matrix. For the results, we used a launch configuration of 32x8 threads per block where each block processed a 32x32 sub-matrix from the input matrix.\r\n<table>\r\n<tbody>\r\n<tr>\r\n<th>Matrix Dimensions</th>\r\n<th>transpose32 Bandwidth</th>\r\n<th>transposeNoSC Bandwidth</th>\r\n<th>transposeSC Bandwidth</th>\r\n</tr>\r\n<tr>\r\n<td>1024x1024</td>\r\n<td>85.18 GB/s</td>\r\n<td>82.21 GB/s</td>\r\n<td>84.41 GB/s</td>\r\n</tr>\r\n</tbody>\r\n</table>\r\nKeep an eye out for future posts along this theme, in which we\'ll discuss tips to optimize CUDA/OpenCL kernels for different algorithms.','CUDA Optimization tips for Matrix Transpose in real world applications','','inherit','open','open','','6786-revision-v1','','','2014-10-07 20:53:39','2014-10-07 20:53:39','',6786,'http://sandbox.arrayfire.com/6786-revision-v1/',0,'revision','',0),(6820,7,'2014-10-10 18:13:40','2014-10-10 18:13:40','','Training','','publish','open','closed','','6820','','','2015-05-01 21:26:34','2015-05-01 21:26:34','',0,'http://sandbox.arrayfire.com/?p=6820',5,'nav_menu_item','',0),(6827,7,'2014-10-10 18:13:40','2014-10-10 18:13:40','','Support','','publish','open','closed','','6827','','','2015-05-01 21:26:34','2015-05-01 21:26:34','',0,'http://sandbox.arrayfire.com/?p=6827',4,'nav_menu_item','',0),(6828,7,'2014-10-10 18:13:40','2014-10-10 18:13:40','','Mobile','','publish','open','closed','','6828','','','2015-05-01 21:26:34','2015-05-01 21:26:34','',0,'http://sandbox.arrayfire.com/?p=6828',6,'nav_menu_item','',0),(6829,7,'2014-10-10 18:13:40','2014-10-10 18:13:40','','Documentation','','publish','open','closed','','documentation-3','','','2015-05-01 21:26:34','2015-05-01 21:26:34','',0,'http://sandbox.arrayfire.com/?p=6829',8,'nav_menu_item','',0),(6830,7,'2014-10-10 18:13:40','2014-10-10 18:13:40','','Forums','','publish','open','closed','','forums','','','2015-05-01 21:26:34','2015-05-01 21:26:34','',0,'http://sandbox.arrayfire.com/?p=6830',9,'nav_menu_item','',0),(6834,7,'2014-10-10 18:28:15','2014-10-10 18:28:15','[content_band inner_container=\"true\" no_margin=\"true\" border=\"none\" padding_top=\"40px\" padding_bottom=\"40px\"][column type=\"two-fifths\"]\r\n\r\n[custom_headline type=\"left\" level=\"h3\"]If you want faster code, you\'ve come to the right place.[/custom_headline][gap size=\"15%\"][button  type=\"flat\" shape=\"rounded\" href=\"http://sandbox.arrayfire.com/download\" size=\"x-large\" block=\"true\"]Download ArrayFire[/button]\r\n\r\n[gap size=\"10%\"]\r\n\r\n[button  type=\"flat\" block=\"true\" shape=\"rounded\" href=\"http://www.meetme.so/arrayfire\" size=\"x-large\" ]Schedule a consultation[/button]\r\n\r\n[/column][column type=\"three-fifths\" last=\"true\"][gap size=\"7%\"]\r\n<div id=\"wistia_rj2bar3xxp\" class=\"wistia_embed\" style=\"width: 640px; height: 388px;\"></div>\r\n<script src=\"//fast.wistia.com/assets/external/E-v1.js\" charset=\"ISO-8859-1\"></script><script>// <![CDATA[\r\nwistiaEmbed = Wistia.embed(\"rj2bar3xxp\");\r\n// ]]></script>\r\n[/column]\r\n<div id=\"arrayfire\"></div>\r\n[/content_band]\r\n\r\n[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"]\r\n\r\n[column type=\"one-third\" ][custom_headline type=\"left\" level=\"h3\" class=\"center-text\"]FASTER CODE[/custom_headline][line]ArrayFire is a blazing fast software library for GPU computing. Its easy-to-use API and array-based function set make GPU programming simple. A few lines of code in ArrayFire can replace dozens of lines of raw GPU code, saving you valuable time and lowering development costs.[/column]\r\n\r\n[column type=\"one-third\" ][custom_headline type=\"left\" level=\"h3\" class=\"center-text\"]HARDWARE NEUTRAL[/custom_headline][line]ArrayFire supports both CUDA-capable NVIDIA GPUs and most OpenCL devices, including AMD GPUs/APUs and Intel Xeon Phi co-processors. It also supports mobile OpenCL devices from ARM, Qualcomm, and others. We want your code to run as fast as possible, regardless of the hardware.[/column]\r\n\r\n[column type=\"one-third\" last=\"true\" ][custom_headline type=\"left\" level=\"h3\" class=\"center-text\"]OPEN SOURCE[/custom_headline][line]ArrayFire is open source—it\'s all there and it\'s all free. Be sure to check out all of the ArrayFire functions (and a lot more!) in our well-composed documentation. And don\'t forget to visit our forums to connect with the ArrayFire community and pick up acceleration tips and innovative best practices.[/column][gap size=\"1%\"]\r\n\r\n[column type=\"one-third\" class=\"center-text\"][button  type=\"flat\" shape=\"rounded\" href=\"http://www.meetme.so/arrayfire\" size=\"regular\" ]Learn more[/button][/column]\r\n\r\n[column type=\"one-third\" class=\"center-text\"][button  type=\"flat\" shape=\"rounded\" href=\"http://www.meetme.so/arrayfire\" size=\"regular\" ]Learn more[/button][/column]\r\n\r\n[column type=\"one-third\" class=\"center-text\" last=\"true\"][button  type=\"flat\" shape=\"rounded\" href=\"http://www.meetme.so/arrayfire\" size=\"regular\" ]Learn more[/button][/column]\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n<div id=\"hardware\">[/content_band]</div>\r\n<div>[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"]</div>\r\n<div></div>\r\n<div></div>\r\n<div>[/content_band]</div>','Home OPSC','','inherit','open','open','','6613-revision-v1','','','2014-10-10 18:28:15','2014-10-10 18:28:15','',6613,'http://sandbox.arrayfire.com/6613-revision-v1/',0,'revision','',0),(6835,7,'2014-10-10 19:27:40','2014-10-10 19:27:40','','mini-motorcycle-01','','inherit','open','open','','mini-motorcycle-01','','','2014-10-10 19:27:40','2014-10-10 19:27:40','',6613,'http://sandbox.arrayfire.com/wp-content/uploads/2014/09/mini-motorcycle-01.png',0,'attachment','image/png',0),(6836,7,'2014-10-10 19:29:43','2014-10-10 19:29:43','','mini-plane-01','','inherit','open','open','','mini-plane-01','','','2014-10-10 19:29:43','2014-10-10 19:29:43','',6613,'http://sandbox.arrayfire.com/wp-content/uploads/2014/09/mini-plane-01.png',0,'attachment','image/png',0),(6837,7,'2014-10-10 19:50:05','2014-10-10 19:50:05','','LM-logo-700','','inherit','open','open','','lm-logo-700','','','2014-10-10 19:50:05','2014-10-10 19:50:05','',6613,'http://sandbox.arrayfire.com/wp-content/uploads/2014/09/LM-logo-700.jpg',0,'attachment','image/jpeg',0),(6838,7,'2014-10-10 19:52:36','2014-10-10 19:52:36','','Cisco_logo_2006','','inherit','open','open','','cisco_logo_2006','','','2014-10-10 19:52:36','2014-10-10 19:52:36','',6613,'http://sandbox.arrayfire.com/wp-content/uploads/2014/09/Cisco_logo_2006.png',0,'attachment','image/png',0),(6839,7,'2014-10-10 19:52:59','2014-10-10 19:52:59','','baker hughes logo','','inherit','open','open','','baker-hughes-logo','','','2014-10-10 19:52:59','2014-10-10 19:52:59','',6613,'http://sandbox.arrayfire.com/wp-content/uploads/2014/09/baker-hughes-logo.png',0,'attachment','image/png',0),(6840,7,'2014-10-10 20:03:38','2014-10-10 20:03:38','','Logo_2013_Google','','inherit','open','open','','logo_2013_google','','','2014-10-10 20:03:38','2014-10-10 20:03:38','',6613,'http://sandbox.arrayfire.com/wp-content/uploads/2014/09/Logo_2013_Google.png',0,'attachment','image/png',0),(6841,7,'2014-10-10 20:19:58','2014-10-10 20:19:58','[content_band inner_container=\"true\" no_margin=\"true\" border=\"none\" padding_top=\"40px\" padding_bottom=\"40px\"][column type=\"two-fifths\"]\r\n\r\n[custom_headline type=\"left\" level=\"h3\"]If you want faster code, you\'ve come to the right place.[/custom_headline][gap size=\"15%\"][button  type=\"flat\" shape=\"rounded\" href=\"http://sandbox.arrayfire.com/download\" size=\"x-large\" block=\"true\"]Download ArrayFire[/button]\r\n\r\n[gap size=\"10%\"]\r\n\r\n[button  type=\"flat\" block=\"true\" shape=\"rounded\" href=\"http://www.meetme.so/arrayfire\" size=\"x-large\" ]Schedule a consultation[/button]\r\n\r\n[/column][column type=\"three-fifths\" last=\"true\"][gap size=\"7%\"]\r\n<div id=\"wistia_rj2bar3xxp\" class=\"wistia_embed\" style=\"width: 640px; height: 388px;\"></div>\r\n<script src=\"//fast.wistia.com/assets/external/E-v1.js\" charset=\"ISO-8859-1\"></script><script>// <![CDATA[\r\nwistiaEmbed = Wistia.embed(\"rj2bar3xxp\");\r\n// ]]></script>\r\n[/column]\r\n<div id=\"arrayfire\"></div>\r\n[/content_band]\r\n\r\n[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"]\r\n\r\n[column type=\"one-third\" ][custom_headline type=\"left\" level=\"h3\" class=\"center-text\"][image src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/01/car-e1412967813807.png\" class=\"mbn\"] FASTER CODE[/custom_headline][line]ArrayFire is a blazing fast software library for GPU computing. Its easy-to-use API and array-based function set make GPU programming simple. A few lines of code in ArrayFire can replace dozens of lines of raw GPU code, saving you valuable time and lowering development costs.[/column]\r\n\r\n[column type=\"one-third\" ][custom_headline type=\"left\" level=\"h3\" class=\"center-text\"][image src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/09/mini-motorcycle-01-e1412969290876.png\" class=\"mbn\"] HARDWARE NEUTRAL[/custom_headline][line]ArrayFire supports both CUDA-capable NVIDIA GPUs and most OpenCL devices, including AMD GPUs/APUs and Intel Xeon Phi co-processors. It also supports mobile OpenCL devices from ARM, Qualcomm, and others. We want your code to run as fast as possible, regardless of the hardware.[/column]\r\n\r\n[column type=\"one-third\" last=\"true\" ][custom_headline type=\"left\" level=\"h3\" class=\"center-text\"][image src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/09/mini-plane-01-e1412969527633.png\" class=\"mbn\"] OPEN SOURCE[/custom_headline][line]ArrayFire is open source—it\'s all there and it\'s all free. Be sure to check out all of the ArrayFire functions (and a lot more!) in our well-composed documentation. And don\'t forget to visit our forums to connect with the ArrayFire community and pick up acceleration tips and innovative best practices.[/column][gap size=\"1%\"]\r\n\r\n[column type=\"one-third\" class=\"center-text\"][button  type=\"flat\" shape=\"rounded\" href=\"http://www.meetme.so/arrayfire\" size=\"regular\" ]Learn more[/button][/column]\r\n\r\n[column type=\"one-third\" class=\"center-text\"][button  type=\"flat\" shape=\"rounded\" href=\"http://www.meetme.so/arrayfire\" size=\"regular\" ]Learn more[/button][/column]\r\n\r\n[column type=\"one-third\" class=\"center-text\" last=\"true\"][button  type=\"flat\" shape=\"rounded\" href=\"http://www.meetme.so/arrayfire\" size=\"regular\" ]Learn more[/button][/column][gap size=5%][/content_band]\r\n<div>\r\n\r\n[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"][gap size=5%]\r\n\r\n[column type=\"one-fourth\" class=\"center-text\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/04/1800contactlogo.png\"><img class=\"aligncenter size-full wp-image-5395\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/04/1800contactlogo.png\" alt=\"1800contactlogo\" width=\"236\" height=\"40\" /></a>[/column] [column type=\"one-fourth\" class=\"center-text\"]\r\n\r\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/04/Koch-Industries-e1409845503117.jpg\"><img class=\"aligncenter size-large wp-image-5390\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/04/Koch-Industries-1024x217.jpg\" alt=\"Koch-Industries\" width=\"742\" height=\"157\" /></a>[/column] [column type=\"one-fourth\" class=\"center-text\"][image src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/09/Logo_2013_Google.png\"]\r\n\r\n&nbsp;\r\n\r\n[/column][column type=\"one-fourth\" class=\"center-text\" last=\"true\"]\r\n\r\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/04/nasa-e1396387230473.png\"><img class=\"aligncenter size-full wp-image-5384\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/04/nasa-e1396387230473.png\" alt=\"nasa\" width=\"253\" height=\"75\" /></a>[/column][column type=\"one-fourth\" class=\"center-text\"]<img class=\"aligncenter size-full wp-image-6837\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/09/LM-logo-700.jpg\" alt=\"LM-logo-700\" width=\"704\" height=\"200\" />\r\n\r\n[/column][column type=\"one-fourth\" class=\"center-text\"]<img class=\"aligncenter size-full wp-image-6838\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/09/Cisco_logo_2006.png\" alt=\"Cisco_logo_2006\" width=\"307\" height=\"180\" />\r\n\r\n[/column][column type=\"one-fourth\" class=\"center-text\"]<img class=\"aligncenter size-full wp-image-5396\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/04/LFG-B-HORZ_202_K-e1396448935778.jpg\" alt=\"LFG-B-HORZ_202_K\" width=\"239\" height=\"75\" />[/column][column type=\"one-fourth\" class=\"center-text\" last=\"true\"]<img class=\"aligncenter size-full wp-image-6839\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/09/baker-hughes-logo.png\" alt=\"baker hughes logo\" width=\"449\" height=\"248\" />[/column]\r\n\r\n[/content_band]\r\n\r\n</div>','Home OPSC','','inherit','open','open','','6613-revision-v1','','','2014-10-10 20:19:58','2014-10-10 20:19:58','',6613,'http://sandbox.arrayfire.com/6613-revision-v1/',0,'revision','',0),(6851,7,'2014-10-10 20:30:52','2014-10-10 20:30:52','[content_band inner_container=\"true\" no_margin=\"true\" border=\"none\" padding_top=\"40px\" padding_bottom=\"40px\"][column type=\"two-fifths\"]\r\n\r\n[custom_headline type=\"left\" level=\"h3\"]If you want faster code, you\'ve come to the right place.[/custom_headline][gap size=\"15%\"][button  type=\"flat\" shape=\"rounded\" href=\"http://sandbox.arrayfire.com/download\" size=\"x-large\" block=\"true\"]Download ArrayFire[/button]\r\n\r\n[gap size=\"10%\"]\r\n\r\n[button  type=\"flat\" block=\"true\" shape=\"rounded\" href=\"http://www.meetme.so/arrayfire\" size=\"x-large\" ]Schedule a consultation[/button]\r\n\r\n[/column][column type=\"three-fifths\" last=\"true\"][gap size=\"7%\"]\r\n<div id=\"wistia_rj2bar3xxp\" class=\"wistia_embed\" style=\"width: 640px; height: 388px;\"></div>\r\n<script src=\"//fast.wistia.com/assets/external/E-v1.js\" charset=\"ISO-8859-1\"></script><script>// <![CDATA[\r\nwistiaEmbed = Wistia.embed(\"rj2bar3xxp\");\r\n// ]]></script>\r\n[/column]\r\n<div id=\"arrayfire\"></div>\r\n[/content_band]\r\n\r\n[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"]\r\n\r\n[column type=\"one-third\" ][custom_headline type=\"left\" level=\"h3\" class=\"center-text\"][image src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/01/car-e1412967813807.png\" class=\"mbn\"] FASTER CODE[/custom_headline][line]ArrayFire is a blazing fast software library for GPU computing. Its easy-to-use API and array-based function set make GPU programming simple. A few lines of code in ArrayFire can replace dozens of lines of raw GPU code, saving you valuable time and lowering development costs.[/column]\r\n\r\n[column type=\"one-third\" ][custom_headline type=\"left\" level=\"h3\" class=\"center-text\"][image src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/09/mini-motorcycle-01-e1412969290876.png\" class=\"mbn\"] HARDWARE NEUTRAL[/custom_headline][line]ArrayFire supports both CUDA-capable NVIDIA GPUs and most OpenCL devices, including AMD GPUs/APUs and Intel Xeon Phi co-processors. It also supports mobile OpenCL devices from ARM, Qualcomm, and others. We want your code to run as fast as possible, regardless of the hardware.[/column]\r\n\r\n[column type=\"one-third\" last=\"true\" ][custom_headline type=\"left\" level=\"h3\" class=\"center-text\"][image src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/09/mini-plane-01-e1412969527633.png\" class=\"mbn\"] OPEN SOURCE[/custom_headline][line]ArrayFire is open source—it\'s all there and it\'s all free. Be sure to check out all of the ArrayFire functions (and a lot more!) in our well-composed documentation. And don\'t forget to visit our forums to connect with the ArrayFire community and pick up acceleration tips and innovative best practices.[/column][gap size=\"1%\"]\r\n\r\n[column type=\"one-third\" class=\"center-text\"][button  type=\"flat\" shape=\"rounded\" href=\"http://sandbox.arrayfire.com/faster-code-opsc\" size=\"regular\" ]Learn more[/button][/column]\r\n\r\n[column type=\"one-third\" class=\"center-text\"][button  type=\"flat\" shape=\"rounded\" href=\"http://http://sandbox.arrayfire.com/hardware-neutral-opsc\" size=\"regular\" ]Learn more[/button][/column]\r\n\r\n[column type=\"one-third\" class=\"center-text\" last=\"true\"][button  type=\"flat\" shape=\"rounded\" href=\"http://sandbox.arrayfire.com/open-source-opsc\" size=\"regular\" ]Learn more[/button][/column][gap size=5%][/content_band]\r\n<div>\r\n\r\n[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"][gap size=5%]\r\n\r\n[column type=\"one-fourth\" class=\"center-text\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/04/1800contactlogo.png\"><img class=\"aligncenter size-full wp-image-5395\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/04/1800contactlogo.png\" alt=\"1800contactlogo\" width=\"236\" height=\"40\" /></a>[/column] [column type=\"one-fourth\" class=\"center-text\"]\r\n\r\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/04/Koch-Industries-e1409845503117.jpg\"><img class=\"aligncenter size-large wp-image-5390\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/04/Koch-Industries-1024x217.jpg\" alt=\"Koch-Industries\" width=\"742\" height=\"157\" /></a>[/column] [column type=\"one-fourth\" class=\"center-text\"][image src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/09/Logo_2013_Google.png\"]\r\n\r\n&nbsp;\r\n\r\n[/column][column type=\"one-fourth\" class=\"center-text\" last=\"true\"]\r\n\r\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/04/nasa-e1396387230473.png\"><img class=\"aligncenter size-full wp-image-5384\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/04/nasa-e1396387230473.png\" alt=\"nasa\" width=\"253\" height=\"75\" /></a>[/column][column type=\"one-fourth\" class=\"center-text\"]<img class=\"aligncenter size-full wp-image-6837\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/09/LM-logo-700.jpg\" alt=\"LM-logo-700\" width=\"704\" height=\"200\" />\r\n\r\n[/column][column type=\"one-fourth\" class=\"center-text\"]<img class=\"aligncenter size-full wp-image-6838\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/09/Cisco_logo_2006.png\" alt=\"Cisco_logo_2006\" width=\"307\" height=\"180\" />\r\n\r\n[/column][column type=\"one-fourth\" class=\"center-text\"]<img class=\"aligncenter size-full wp-image-5396\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/04/LFG-B-HORZ_202_K-e1396448935778.jpg\" alt=\"LFG-B-HORZ_202_K\" width=\"239\" height=\"75\" />[/column][column type=\"one-fourth\" class=\"center-text\" last=\"true\"]<img class=\"aligncenter size-full wp-image-6839\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/09/baker-hughes-logo.png\" alt=\"baker hughes logo\" width=\"449\" height=\"248\" />[/column]\r\n\r\n[/content_band]\r\n\r\n</div>','Home OPSC','','inherit','open','open','','6613-revision-v1','','','2014-10-10 20:30:52','2014-10-10 20:30:52','',6613,'http://sandbox.arrayfire.com/6613-revision-v1/',0,'revision','',0),(6853,7,'2014-10-10 20:38:42','2014-10-10 20:38:42','[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"][column type=\"one-half\"][custom_headline type=\"left\" level=\"h3\"]HIRE THE ARRAYFIRE TEAM[/custom_headline]\r\n\r\nHire us to accelerate your code, train your pros, or even write new code for your organization!\r\n\r\nWe’ve already helped thousands of organizations speed up their code. We’ve accelerated algorithms,  sped up data sets, and solved challenging problems of all types in the past, and we’re confident that we can accelerate your code, too.\r\n\r\nStart by setting up a free consultation to discuss your project with an ArrayFire expert today!\r\n\r\n[gap size=\"10%\"][button  type=\"flat\" shape=\"rounded\" href=\"http://meetme.so/arrayfire\" size=\"large\" block=\"true\"]Schedule a consultation[/button][/column][column type=\"one-half\" last=\"true\" class=\"center-text\"][gap size=\"15%\"][image src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/01/Bat-signal-e1391790016951.png\"][/column] [gap size=\"7%\"] [line]\r\n\r\n[custom_headline type=\"left\" level=\"h3\"]WHERE WE\'VE BEEN LATELY[/custom_headline]\r\n\r\n&nbsp;\r\n\r\n[column type=\"one-third\" class=\"center-text\"][image src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/04/LFG-B-HORZ_202_K-e1396448935778.jpg\"][/column][column type=\"one-third\" class=\"center-text\"][image src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/02/reveal_logo-e1396449044651.jpg\"][/column][column type=\"one-third\" last=\"true\" class=\"center-text\"][gap size=\"5%\"] [image src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/04/1800contactlogo.png\"][/column]\r\n\r\n[column type=\"one-third\" ][line]Lincoln Financial Group ported their CPU code (C/C++) to CUDA, and this enabled them to speed up the necessary financial calculations. Their primary objective was to reduce the time taken to run this code. Before porting to CUDA, they required 10-12 hours to run the entire code. After porting to CUDA, they were able to do the same work in about 30 minutes.[/column][column type=\"one-third\" ][line]Reveal Imaging, an SAIC company, uses ArrayFire in developing threat-detection products for transportation systems—such as cargo and baggage screening in airports—and in securing facilities and public events.[/column][column type=\"one-third\" last=\"true\" ][line]The Glasses.com app relies on computer vision to recognize the features of a user’s face. The application then generates an accurate, high-fidelity, three-dimensional model of that face, and virtually places selected frames on the image.[/column]\r\n<div></div>\r\n[/content_band]','Consulting, OPSC','','inherit','open','open','','6634-revision-v1','','','2014-10-10 20:38:42','2014-10-10 20:38:42','',6634,'http://sandbox.arrayfire.com/6634-revision-v1/',0,'revision','',0),(6854,7,'2014-10-10 20:39:22','2014-10-10 20:39:22','[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"][column type=\"one-half\"][custom_headline type=\"left\" level=\"h3\"]HIRE THE ARRAYFIRE TEAM[/custom_headline]\r\n\r\nHire us to accelerate your code, train your pros, or even write new code for your organization!\r\n\r\nWe’ve already helped thousands of organizations speed up their code. We’ve accelerated algorithms,  sped up data sets, and solved challenging problems of all types in the past, and we’re confident that we can accelerate your code, too.\r\n\r\nStart by setting up a free consultation to discuss your project with an ArrayFire expert today!\r\n\r\n[gap size=\"10%\"][button  type=\"flat\" shape=\"rounded\" href=\"http://meetme.so/arrayfire\" size=\"large\" block=\"true\"]Schedule a consultation[/button][/column][column type=\"one-half\" last=\"true\" class=\"center-text\"][gap size=\"15%\"][image src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/01/Bat-signal-e1391790016951.png\"][/column] [gap size=\"7%\"] [line]\r\n\r\n[custom_headline type=\"left\" level=\"h3\"]ARRAYFIRE USERS[/custom_headline]\r\n\r\n&nbsp;\r\n\r\n[column type=\"one-third\" class=\"center-text\"][image src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/04/LFG-B-HORZ_202_K-e1396448935778.jpg\"][/column][column type=\"one-third\" class=\"center-text\"][image src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/02/reveal_logo-e1396449044651.jpg\"][/column][column type=\"one-third\" last=\"true\" class=\"center-text\"][gap size=\"5%\"] [image src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/04/1800contactlogo.png\"][/column]\r\n\r\n[column type=\"one-third\" ][line]Lincoln Financial Group ported their CPU code (C/C++) to CUDA, and this enabled them to speed up the necessary financial calculations. Their primary objective was to reduce the time taken to run this code. Before porting to CUDA, they required 10-12 hours to run the entire code. After porting to CUDA, they were able to do the same work in about 30 minutes.[/column][column type=\"one-third\" ][line]Reveal Imaging, an SAIC company, uses ArrayFire in developing threat-detection products for transportation systems—such as cargo and baggage screening in airports—and in securing facilities and public events.[/column][column type=\"one-third\" last=\"true\" ][line]The Glasses.com app relies on computer vision to recognize the features of a user’s face. The application then generates an accurate, high-fidelity, three-dimensional model of that face, and virtually places selected frames on the image.[/column]\r\n<div></div>\r\n[/content_band]','Consulting, OPSC','','inherit','open','open','','6634-revision-v1','','','2014-10-10 20:39:22','2014-10-10 20:39:22','',6634,'http://sandbox.arrayfire.com/6634-revision-v1/',0,'revision','',0),(6855,7,'2014-10-10 20:39:46','2014-10-10 20:39:46','[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"][column type=\"one-half\"][custom_headline type=\"left\" level=\"h3\"]HIRE THE ARRAYFIRE TEAM[/custom_headline]\r\n\r\nHire us to accelerate your code, train your pros, or even write new code for your organization!\r\n\r\nWe’ve already helped thousands of organizations speed up their code. We’ve accelerated algorithms,  sped up data sets, and solved challenging problems of all types in the past, and we’re confident that we can accelerate your code, too.\r\n\r\nStart by setting up a free consultation to discuss your project with an ArrayFire expert today!\r\n\r\n[gap size=\"10%\"][button  type=\"flat\" shape=\"rounded\" href=\"http://meetme.so/arrayfire\" size=\"large\" block=\"true\"]Schedule a consultation[/button][/column][column type=\"one-half\" last=\"true\" class=\"center-text\"][gap size=\"15%\"][image src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/01/Bat-signal-e1391790016951.png\"][/column] [gap size=\"7%\"] [line]\r\n\r\n&nbsp;\r\n\r\n[column type=\"one-third\" class=\"center-text\"][image src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/04/LFG-B-HORZ_202_K-e1396448935778.jpg\"][/column][column type=\"one-third\" class=\"center-text\"][image src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/02/reveal_logo-e1396449044651.jpg\"][/column][column type=\"one-third\" last=\"true\" class=\"center-text\"][gap size=\"5%\"] [image src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/04/1800contactlogo.png\"][/column]\r\n\r\n[column type=\"one-third\" ][line]Lincoln Financial Group ported their CPU code (C/C++) to CUDA, and this enabled them to speed up the necessary financial calculations. Their primary objective was to reduce the time taken to run this code. Before porting to CUDA, they required 10-12 hours to run the entire code. After porting to CUDA, they were able to do the same work in about 30 minutes.[/column][column type=\"one-third\" ][line]Reveal Imaging, an SAIC company, uses ArrayFire in developing threat-detection products for transportation systems—such as cargo and baggage screening in airports—and in securing facilities and public events.[/column][column type=\"one-third\" last=\"true\" ][line]The Glasses.com app relies on computer vision to recognize the features of a user’s face. The application then generates an accurate, high-fidelity, three-dimensional model of that face, and virtually places selected frames on the image.[/column]\r\n<div></div>\r\n[/content_band]','Consulting, OPSC','','inherit','open','open','','6634-revision-v1','','','2014-10-10 20:39:46','2014-10-10 20:39:46','',6634,'http://sandbox.arrayfire.com/6634-revision-v1/',0,'revision','',0),(6866,17796,'2014-10-23 14:49:13','2014-10-23 14:49:13','A while back I wrote a blog on triangle counting in networks using CUDA (see <a href=\"http://sandbox.arrayfire.com/triangle-counting-in-graphs-on-the-gpu-part-1/\" title=\"Part 1\" target=\"_blank\"> Part 1</a> for the post). In this post, I will cover in more detail the internals of the algorithm and the CUDA implementation. Before I take a deep dive into the details of the algorithm, I want to remind the reader that there are multiple ways for finding triangles in a graph. Our approach is based off the intersection of two adjacency lists and finding the common elements in both those lists. Two additional approaches would simply be to compare all the possible node-triplets, either in the graph or via matrix multiplication of the incidence array. The latter of these two approaches can be computationally demanding--especially for sparse networks. This makes the adjacency list intersection approach preferable in most cases. For the intersection to be efficient, the lists are typically presorted. This by no means is an exaggerated requirement.\r\n\r\nAn extended discussion on the pros and cons of the different approaches can be found in both the papers by <a href=\"http://www.emis.ams.org/journals/JGAA/accepted/2005/SchankWagner2005.9.2.pdf\" title=\"Finding, counting and listing all triangles in large graphs, an experimental study\" target=\"_blank\">Shanck & Wagner</a> and in <a href=\"http://www.cc.gatech.edu/grads/o/ogreen3/_docs/2014LoadBalancecClusteringCoefficients.pdf\" title=\"Load Balanced Clustering Coefficients\" target=\"_blank\">Green et al.</a> \r\n\r\nThe significance of the adjacency lists being sorted is in fact important. The main reason is that the intersecting two sort lists is very similar conceptually to merging two sorted lists. Thus, parallel merge algorithms can be adopted for the intersection.\r\n\r\nThe key difference between a merge and the intersection is that most parallel merge algorithms offer exclusive writing (in terms of location) in the output array--meaning each thread is responsible for merging independent sections of the output and, in fact, each thread knows the exact position in the output array that it will write to. For intersection this is more complicated, as the exact position to which a thread will need to write to in a sorted fashion is dependent on the sorted input and how many prior intersections were found. Fortunately, in the case of triangle counting, the actual set of overlapping elements is not necessary--rather, only the number of common elements is required. This allows us to use the recently developed <a href=\"http://arxiv.org/pdf/1406.2628\" title=\"Merge Path\" target=\"_blank\">Merge Path</a> and <a href=\"http://www.cc.gatech.edu/~bader/papers/GPUMergePath-ICS2012.pdf\" title=\"GPU Merge Path\" target=\"_blank\">GPU Merge Path</a> algorithms for our purposes. Merge Path is a visual approach for merging two sorted arrays into one sorted array. \r\n\r\nMerge Path restates the problem of merging two sorted arrays into the traversal of a 2-d grid from the top-left corner to the bottom-right corner with the only legal moves being downwards and to the right. This can be seen in the figure below. The traversal of the grid is dependent on the values of the input arrays (one has been placed at the top of the grid and the other has been placed on the left of the grid). The decision on either moving down or to the right is dependent on the location of the indices of the path. As the path starts at index (0,0), the value A[0] is compared with the value B[0]. Based on the comparison, the movement direction is decided. The movement direction essentially increases the pointer-index of one of these arrays. Thus, only elements on the path are computed (compared).\r\n\r\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/merge.png\"><img src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/merge-300x300.png\" alt=\"merge\" width=\"300\" height=\"300\" class=\"alignnone size-medium wp-image-6952\" /></a> <a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/intersect1.png\"><img src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/intersect1-300x300.png\" alt=\"intersect\" width=\"300\" height=\"300\" class=\"alignnone size-medium wp-image-6955\" /></a>\r\n\r\nUsing the Merge Path concept, it is possible to partition the work into equal length paths such that each core is responsible for merging an equal number of elements. With a little augmentation to the movement pattern, we can change Merge Path to support intersection. This involves adding an additional move option: down-right. This move occurs when the elements pointed by the indices in both lists are equal. So why is all this important? Well, the simple truth is that this allows us to split the adjacency list intersection between multiple GPU threads. This also permits executing several different adjacency list executions concurrently in a single CUDA block.\r\n\r\nIn my next blog I will cover performance and give some additional details on benefits of doing multiple list intersections concurrently. A full tech-report on this will be published in the near future, so stay tuned.','Triangle Counting in Graphs on the GPU (Part 2)','','publish','open','open','','triangle-counting-in-graphs-on-the-gpu-part-2','','\nhttp://sandbox.arrayfire.com/triangle-counting-in-graphs-on-the-gpu-part-1/','2014-10-23 14:49:13','2014-10-23 14:49:13','',0,'http://sandbox.arrayfire.com/?p=6866',0,'post','',2),(6867,17796,'2014-10-14 20:37:20','2014-10-14 20:37:20','A while back I wrote a blog on triangle counting in networks, see<a href=\"http://sandbox.arrayfire.com/triangle-counting-in-graphs-on-the-gpu-part-1/\" title=\"Part 1\" target=\"_blank\"> Part 1</a> ','Triangle Counting in Graphs on the GPU (Part 2)','','inherit','open','open','','6866-revision-v1','','','2014-10-14 20:37:20','2014-10-14 20:37:20','',6866,'http://sandbox.arrayfire.com/6866-revision-v1/',0,'revision','',0),(6868,17796,'2014-10-14 20:55:11','2014-10-14 20:55:11','A while back I wrote a blog on triangle counting in networks using CUDA (see <a href=\"http://sandbox.arrayfire.com/triangle-counting-in-graphs-on-the-gpu-part-1/\" title=\"Part 1\" target=\"_blank\"> Part 1</a> for the post). In this post, I will cover in more detail the internals of the algorithm and the CUDA implementation. Before I take a deep dive into the details of the algorithm, I want to remind the reader that there are multiple ways for finding triangles in a graph. Our approach is based off the intersection of two adjacency lists and finding the common elements in both those lists. Two additional approaches would simply be to compare all the possible node-triplets in the graph or using via matrix multiplication of the incidence array. The last of these two approaches can be computationally  demanding, especially for sparse networks. This makes the adjacency list intersection approach preferable in most cases. For the intersection to be efficient, typically the lists are presorted. This by no means is an exaggerated requirement. \r\nAn extended discussion on the pros and cons of the different approaches can be found in both the papers by <a href=\"http://www.emis.ams.org/journals/JGAA/accepted/2005/SchankWagner2005.9.2.pdf\" title=\"Finding, counting and listing all triangles in large graphs, an experimental study\" target=\"_blank\">Shanck & Wagner</a> and in <a href=\"http://www.cc.gatech.edu/grads/o/ogreen3/_docs/2014LoadBalancecClusteringCoefficients.pdf\" title=\"Load Balanced Clustering Coefficients\" target=\"_blank\">Green et al.</a> ','Triangle Counting in Graphs on the GPU (Part 2)','','inherit','open','open','','6866-revision-v1','','','2014-10-14 20:55:11','2014-10-14 20:55:11','',6866,'http://sandbox.arrayfire.com/6866-revision-v1/',0,'revision','',0),(6869,17796,'2014-10-14 21:32:00','2014-10-14 21:32:00','A while back I wrote a blog on triangle counting in networks using CUDA (see <a href=\"http://sandbox.arrayfire.com/triangle-counting-in-graphs-on-the-gpu-part-1/\" title=\"Part 1\" target=\"_blank\"> Part 1</a> for the post). In this post, I will cover in more detail the internals of the algorithm and the CUDA implementation. Before I take a deep dive into the details of the algorithm, I want to remind the reader that there are multiple ways for finding triangles in a graph. Our approach is based off the intersection of two adjacency lists and finding the common elements in both those lists. Two additional approaches would simply be to compare all the possible node-triplets in the graph or using via matrix multiplication of the incidence array. The last of these two approaches can be computationally  demanding, especially for sparse networks. This makes the adjacency list intersection approach preferable in most cases. For the intersection to be efficient, typically the lists are presorted. This by no means is an exaggerated requirement. \r\nAn extended discussion on the pros and cons of the different approaches can be found in both the papers by <a href=\"http://www.emis.ams.org/journals/JGAA/accepted/2005/SchankWagner2005.9.2.pdf\" title=\"Finding, counting and listing all triangles in large graphs, an experimental study\" target=\"_blank\">Shanck & Wagner</a> and in <a href=\"http://www.cc.gatech.edu/grads/o/ogreen3/_docs/2014LoadBalancecClusteringCoefficients.pdf\" title=\"Load Balanced Clustering Coefficients\" target=\"_blank\">Green et al.</a> \r\n\r\nThe significance of the adjacency lists being, while not exaggerated, is in fact important. The main reason is that the intersecting two sort lists is very similar conceptually to merging two sorted lists. Thus, parallel merge algorithms can adopted for the intersection.\r\nThe key difference between a merge and the intersection, is that most parallel merge algorithms offer exclusive writing in the output array - meaning each thread is responsible for merging independent sections of the output and in fact each thread knows the exact position in the output array that it will write to. For intersection, this is more complicated as the exact position to which a thread will need to write to in a sorted fashion is dependent on the sorted input. Fortunately, in the case of triangle counting, the actual set of overlapping elements is not necessary - rather only the number of common elements is required. This in fact allows us to use the recently developed <a href=\"http://arxiv.org/pdf/1406.2628\" title=\"Merge Path\" target=\"_blank\">MergePath</a> and <a href=\"http://www.cc.gatech.edu/~bader/papers/GPUMergePath-ICS2012.pdf\" title=\"GPU Merge Path\" target=\"_blank\">GPU-MergePath</a> algorithms.\r\n\r\n\r\n\r\n','Triangle Counting in Graphs on the GPU (Part 2)','','inherit','open','open','','6866-revision-v1','','','2014-10-14 21:32:00','2014-10-14 21:32:00','',6866,'http://sandbox.arrayfire.com/6866-revision-v1/',0,'revision','',0),(6871,17796,'2014-10-14 21:35:03','2014-10-14 21:35:03','A while back I wrote a blog on triangle counting in networks using CUDA (see <a href=\"http://sandbox.arrayfire.com/triangle-counting-in-graphs-on-the-gpu-part-1/\" title=\"Part 1\" target=\"_blank\"> Part 1</a> for the post). In this post, I will cover in more detail the internals of the algorithm and the CUDA implementation. Before I take a deep dive into the details of the algorithm, I want to remind the reader that there are multiple ways for finding triangles in a graph. Our approach is based off the intersection of two adjacency lists and finding the common elements in both those lists. Two additional approaches would simply be to compare all the possible node-triplets in the graph or using via matrix multiplication of the incidence array. The last of these two approaches can be computationally  demanding, especially for sparse networks. This makes the adjacency list intersection approach preferable in most cases. For the intersection to be efficient, typically the lists are presorted. This by no means is an exaggerated requirement. \r\nAn extended discussion on the pros and cons of the different approaches can be found in both the papers by <a href=\"http://www.emis.ams.org/journals/JGAA/accepted/2005/SchankWagner2005.9.2.pdf\" title=\"Finding, counting and listing all triangles in large graphs, an experimental study\" target=\"_blank\">Shanck & Wagner</a> and in <a href=\"http://www.cc.gatech.edu/grads/o/ogreen3/_docs/2014LoadBalancecClusteringCoefficients.pdf\" title=\"Load Balanced Clustering Coefficients\" target=\"_blank\">Green et al.</a> \r\n\r\nThe significance of the adjacency lists being, while not exaggerated, is in fact important. The main reason is that the intersecting two sort lists is very similar conceptually to merging two sorted lists. Thus, parallel merge algorithms can adopted for the intersection.\r\nThe key difference between a merge and the intersection, is that most parallel merge algorithms offer exclusive writing in the output array - meaning each thread is responsible for merging independent sections of the output and in fact each thread knows the exact position in the output array that it will write to. For intersection, this is more complicated as the exact position to which a thread will need to write to in a sorted fashion is dependent on the sorted input. Fortunately, in the case of triangle counting, the actual set of overlapping elements is not necessary - rather only the number of common elements is required. This in fact allows us to use the recently developed <a href=\"http://arxiv.org/pdf/1406.2628\" title=\"Merge Path\" target=\"_blank\">MergePath</a> and <a href=\"http://www.cc.gatech.edu/~bader/papers/GPUMergePath-ICS2012.pdf\" title=\"GPU Merge Path\" target=\"_blank\">GPU-MergePath</a> algorithms for our purposes.\r\n\r\n\r\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/MPexample.pdf\">MPexample</a>\r\n\r\n','Triangle Counting in Graphs on the GPU (Part 2)','','inherit','open','open','','6866-revision-v1','','','2014-10-14 21:35:03','2014-10-14 21:35:03','',6866,'http://sandbox.arrayfire.com/6866-revision-v1/',0,'revision','',0),(6872,17796,'2014-10-14 21:47:12','2014-10-14 21:47:12','A while back I wrote a blog on triangle counting in networks using CUDA (see <a href=\"http://sandbox.arrayfire.com/triangle-counting-in-graphs-on-the-gpu-part-1/\" title=\"Part 1\" target=\"_blank\"> Part 1</a> for the post). In this post, I will cover in more detail the internals of the algorithm and the CUDA implementation. Before I take a deep dive into the details of the algorithm, I want to remind the reader that there are multiple ways for finding triangles in a graph. Our approach is based off the intersection of two adjacency lists and finding the common elements in both those lists. Two additional approaches would simply be to compare all the possible node-triplets in the graph or using via matrix multiplication of the incidence array. The last of these two approaches can be computationally  demanding, especially for sparse networks. This makes the adjacency list intersection approach preferable in most cases. For the intersection to be efficient, typically the lists are presorted. This by no means is an exaggerated requirement. \r\nAn extended discussion on the pros and cons of the different approaches can be found in both the papers by <a href=\"http://www.emis.ams.org/journals/JGAA/accepted/2005/SchankWagner2005.9.2.pdf\" title=\"Finding, counting and listing all triangles in large graphs, an experimental study\" target=\"_blank\">Shanck & Wagner</a> and in <a href=\"http://www.cc.gatech.edu/grads/o/ogreen3/_docs/2014LoadBalancecClusteringCoefficients.pdf\" title=\"Load Balanced Clustering Coefficients\" target=\"_blank\">Green et al.</a> \r\n\r\nThe significance of the adjacency lists being, while not exaggerated, is in fact important. The main reason is that the intersecting two sort lists is very similar conceptually to merging two sorted lists. Thus, parallel merge algorithms can adopted for the intersection.\r\nThe key difference between a merge and the intersection, is that most parallel merge algorithms offer exclusive writing in the output array - meaning each thread is responsible for merging independent sections of the output and in fact each thread knows the exact position in the output array that it will write to. For intersection, this is more complicated as the exact position to which a thread will need to write to in a sorted fashion is dependent on the sorted input. Fortunately, in the case of triangle counting, the actual set of overlapping elements is not necessary - rather only the number of common elements is required. This in fact allows us to use the recently developed <a href=\"http://arxiv.org/pdf/1406.2628\" title=\"Merge Path\" target=\"_blank\">MergePath</a> and <a href=\"http://www.cc.gatech.edu/~bader/papers/GPUMergePath-ICS2012.pdf\" title=\"GPU Merge Path\" target=\"_blank\">GPU-MergePath</a> algorithms for our purposes. Merge Path is a visual approach for merging two sorted arrays into one sorted array. \r\n\r\nMergePath restates the problem of merging two sorted arrays into the traversal of a 2-d grid from the top-left corner to the bottom-right corner with the only legal moves being downwards and to the right. This can be seen in the figure below. The traversal of the grid is in fact dependent on the values of the input arrays (one has been placed at the top of the grid and the other has been placed on the left of the grid). The decision on either moving down or two the right is dependent on the location of the indices of the path. As the path starts at index (0,0), the values A[0] is compared with the value B[0]. Based on the comparison, the movement direction is decided on. The movement direction essentially increases the pointer-index of one of these arrays. Thus, only elements on the path are computed (compared).\r\n\r\n\r\n','Triangle Counting in Graphs on the GPU (Part 2)','','inherit','open','open','','6866-revision-v1','','','2014-10-14 21:47:12','2014-10-14 21:47:12','',6866,'http://sandbox.arrayfire.com/6866-revision-v1/',0,'revision','',0),(6875,17796,'2014-10-15 14:57:06','2014-10-15 14:57:06','A while back I wrote a blog on triangle counting in networks using CUDA (see <a href=\"http://sandbox.arrayfire.com/triangle-counting-in-graphs-on-the-gpu-part-1/\" title=\"Part 1\" target=\"_blank\"> Part 1</a> for the post). In this post, I will cover in more detail the internals of the algorithm and the CUDA implementation. Before I take a deep dive into the details of the algorithm, I want to remind the reader that there are multiple ways for finding triangles in a graph. Our approach is based off the intersection of two adjacency lists and finding the common elements in both those lists. Two additional approaches would simply be to compare all the possible node-triplets in the graph or using via matrix multiplication of the incidence array. The last of these two approaches can be computationally  demanding, especially for sparse networks. This makes the adjacency list intersection approach preferable in most cases. For the intersection to be efficient, typically the lists are presorted. This by no means is an exaggerated requirement. \r\nAn extended discussion on the pros and cons of the different approaches can be found in both the papers by <a href=\"http://www.emis.ams.org/journals/JGAA/accepted/2005/SchankWagner2005.9.2.pdf\" title=\"Finding, counting and listing all triangles in large graphs, an experimental study\" target=\"_blank\">Shanck & Wagner</a> and in <a href=\"http://www.cc.gatech.edu/grads/o/ogreen3/_docs/2014LoadBalancecClusteringCoefficients.pdf\" title=\"Load Balanced Clustering Coefficients\" target=\"_blank\">Green et al.</a> \r\n\r\nThe significance of the adjacency lists being, while not exaggerated, is in fact important. The main reason is that the intersecting two sort lists is very similar conceptually to merging two sorted lists. Thus, parallel merge algorithms can adopted for the intersection.\r\nThe key difference between a merge and the intersection, is that most parallel merge algorithms offer exclusive writing in the output array - meaning each thread is responsible for merging independent sections of the output and in fact each thread knows the exact position in the output array that it will write to. For intersection, this is more complicated as the exact position to which a thread will need to write to in a sorted fashion is dependent on the sorted input. Fortunately, in the case of triangle counting, the actual set of overlapping elements is not necessary - rather only the number of common elements is required. This in fact allows us to use the recently developed <a href=\"http://arxiv.org/pdf/1406.2628\" title=\"Merge Path\" target=\"_blank\">MergePath</a> and <a href=\"http://www.cc.gatech.edu/~bader/papers/GPUMergePath-ICS2012.pdf\" title=\"GPU Merge Path\" target=\"_blank\">GPU-MergePath</a> algorithms for our purposes. Merge Path is a visual approach for merging two sorted arrays into one sorted array. \r\n\r\nMergePath restates the problem of merging two sorted arrays into the traversal of a 2-d grid from the top-left corner to the bottom-right corner with the only legal moves being downwards and to the right. This can be seen in the figure below. The traversal of the grid is in fact dependent on the values of the input arrays (one has been placed at the top of the grid and the other has been placed on the left of the grid). The decision on either moving down or two the right is dependent on the location of the indices of the path. As the path starts at index (0,0), the values A[0] is compared with the value B[0]. Based on the comparison, the movement direction is decided on. The movement direction essentially increases the pointer-index of one of these arrays. Thus, only elements on the path are computed (compared).\r\n\r\nUsing the Merge-Path concept, it is possible to partition the work into equal length paths such that each core is responsible for merging an equal number of elements. With a little augmentation to the movement pattern, we can in fact change Merge-Path to support intersection. This involves adding an additional move option: down-right. This move occurs when the elements pointed by the indices in both lists are equal.\r\n\r\n\r\n','Triangle Counting in Graphs on the GPU (Part 2)','','inherit','open','open','','6866-revision-v1','','','2014-10-15 14:57:06','2014-10-15 14:57:06','',6866,'http://sandbox.arrayfire.com/6866-revision-v1/',0,'revision','',0),(6889,7,'2014-10-15 18:46:16','2014-10-15 18:46:16','[content_band inner_container=\"true\" no_margin=\"true\" border=\"none\" padding_top=\"40px\" padding_bottom=\"40px\"][column type=\"two-fifths\"]\r\n\r\n[custom_headline type=\"left\" level=\"h3\"]If you want faster code, you\'ve come to the right place.[/custom_headline][gap size=\"15%\"][button  type=\"flat\" shape=\"rounded\" href=\"http://sandbox.arrayfire.com/download\" size=\"x-large\" block=\"true\"]Download ArrayFire[/button]\r\n\r\n[gap size=\"10%\"]\r\n\r\n[button  type=\"flat\" block=\"true\" shape=\"rounded\" href=\"http://www.meetme.so/arrayfire\" size=\"x-large\" ]Schedule a consultation[/button]\r\n\r\n[/column][column type=\"three-fifths\" last=\"true\"][gap size=\"7%\"]\r\n<div id=\"wistia_rj2bar3xxp\" class=\"wistia_embed\" style=\"width: 640px; height: 388px;\"></div>\r\n<script src=\"//fast.wistia.com/assets/external/E-v1.js\" charset=\"ISO-8859-1\"></script><script>// <![CDATA[\r\nwistiaEmbed = Wistia.embed(\"rj2bar3xxp\");\r\n// ]]></script>[/column]\r\n<div id=\"arrayfire\"></div>\r\n[/content_band]\r\n\r\n[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"][column type=\"one-half\"][custom_headline type=\"left\" level=\"h3\"]Faster code[/custom_headline]\r\n\r\nArrayFire is a fast, hardware-neutral software library for GPU computing with an easy-to-use API. Its array-based function set makes GPU programming simple. A few lines of code in ArrayFire can replace dozens of lines of raw GPU code, saving you valuable time and lowering development costs.\r\n\r\n[gap size=\"10%\"][button  type=\"flat\" shape=\"rounded\" href=\"http://sandbox.arrayfire.com/download\" size=\"x-large\" ]Download ArrayFire[/button][/column][column type=\"one-half\" last=\"true\" class=\"center-text\"][gap size=\"5%\"][image src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/01/Triumvirate_Exported-e1409844599610.png\"][/column] [gap size=\"5%\"]\r\n\r\n[column type=\"one-third\" class=\"center-text\"][image src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/09/Logo_2013_Google.png\"][line]Video content analysis is the basis for categorizing videos and enabling search by content. Google used ArrayFire to extract motion features in video, improving performance by substantially accelerating the solution of the L1-regularized least-squares optimization problem.[/column][column type=\"one-third\" class=\"center-text\"][image src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/04/Koch-Industries-e1396387359637.jpg\"][line]Andrew Shin, Market Risk Manager of Koch Supply &amp; Trading, achieves significant performance increases on option pricing algorithms using ArrayFire software, \"My buddy and I are, at best, novice programmers and we couldn\'t imagine having to figure out how to code all this in CUDA.\"[/column][column type=\"one-third\" last=\"true\" class=\"center-text\"][image src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/04/nasa-e1396387230473.png\"][line]The main thrust of this research was improving Mars rover image compression via GPUs and genetic algorithms. With ArrayFire software and GPUs, the researchers were able to achieve 5X speedups on the larger data sizes.[/column]\r\n<div id=\"hardware\"></div>\r\n[/content_band]\r\n[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"]\r\n\r\n[column type=\"one-half\"][gap size=\"15%\"]\r\n\r\n<img class=\"aligncenter\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/01/CUDA-logo.jpg\" alt=\"CUDA logo\" width=\"231\" height=\"140\" />\r\n\r\n<img class=\"aligncenter\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/01/OpenCLLogo_678x452.png\" alt=\"OpenCL\" width=\"326\" height=\"217\" />\r\n\r\n[/column][column type=\"one-half\" last=\"true\"][gap size=\"10%\"]\r\n\r\n[custom_headline type=\"left\" level=\"h3\"] ArrayFire for CUDA and OpenCL[/custom_headline]\r\n\r\nArrayFire supports CUDA-capable NVIDIA GPUs and most OpenCL devices, including AMD GPUs/APUs and Intel Xeon Phi co-processors. It also supports mobile OpenCL devices from ARM, Qualcomm, and others.\r\n\r\nArrayFire is a high-performance software library designed for maximum productivity and speed without the hassle of writing difficult low-level device code. Each of ArrayFire\'s functions has been hand-tuned by our CUDA and OpenCL experts.\r\n\r\n[/column]\r\n<div id=\"mobile\"></div>\r\n[/content_band]\r\n\r\n[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"]\r\n[column type=\"one-half\"][custom_headline type=\"left\" level=\"h3\"  ]Mobile GPU Computing[/custom_headline]\r\n\r\nMobile GPUs offer accelerated performance. If you would like to use ArrayFire in a mobile application or if you would like to hire our mobile acceleration services, email us at <a href=\"mailto:sales@arrayfire.com\" target=\"_blank\">sales@arrayfire.com</a> and include a description of your application.\r\n\r\nOur clients have achieved between 5X to 30X speed boosts for convolutions and other compute functions on mobile devices. It enables:[icon_list][icon_list_item type=\"angle-double-right\"]Real-time video processing[/icon_list_item][icon_list_item type=\"angle-double-right\"]Better augmented reality [/icon_list_item][icon_list_item type=\"angle-double-right\"]Better computational photograph[/icon_list_item][icon_list_item type=\"angle-double-right\"]Faster data processing[/icon_list_item][/icon_list]\r\n\r\nAll mobile devices have a GPU which can deliver greater compute throughput than the CPU. Why not use the GPU to make your app faster?[/column][column type=\"one-half\" fade=\"true\" last=\"true\" fade_animation=\"in-from-right\" class=\"center-text\"][gap size=\"10%\"][image src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/02/mobile_Exported1-e1395437261846.png\"][/column]\r\n<div id=\"documentation\"></div>\r\n[/content_band]\r\n\r\n[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"][column type=\"one-half\"][custom_headline type=\"left\" level=\"h3\"]Documentation[/custom_headline]\r\n<h5>ArrayFire for C, C++, and Fortran</h5>\r\nOpen the hood—take a look at the beautiful nitty-gritty.\r\n\r\n[icon type=\"angle-double-right\"] <a href=\"http://www.sandbox.arrayfire.com/docs/installation.htm\" target=\"_blank\">CUDA &amp; OpenCL C/C++</a>\r\n<h5>Functions</h5>\r\nHundreds of functions, hand-tuned and optimized by our engineers.\r\n\r\n[icon type=\"angle-double-right\"] <a title=\"ArrayFire Functions\" href=\"http://www.sandbox.arrayfire.com/docs/modules.htm\">ArrayFire Functions</a>\r\n<h5>User Forums</h5>\r\nEvery post gets a response from ArrayFire engineers.\r\n\r\n[icon type=\"angle-double-right\"] <a href=\"http://forums.accelereyes.com/\" target=\"_blank\">User Forums</a>\r\n\r\n[/column][column class=\"center-text\" type=\"one-half\" last=\"true\" fade=\"true\" fade_animation=\"in-from-right\" ][gap size=\"5%\"][image src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/01/gears1-e1395437543311.png\"][/column]\r\n<div id=\"benchmarks\"></div>\r\n[/content_band]\r\n\r\n[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"]\r\n[column type=\"one-half\" fade=\"true\" fade_animation=\"in-from-left\" ][gap size=\"5%\"][image src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/02/Benchmark-guy-torso-opposite-e1392155948298.png\"][/column]\r\n\r\n[column type=\"one-half\" last=\"true\"][custom_headline type=\"left\" level=\"h3\" ]Benchmarks[/custom_headline]\r\n\r\nAll benchmarks were performed on a NVIDIA Tesla C2050 with an Intel i7-950 processor. The benchmarks compare ArrayFire with several popular CPU-based acceleration libraries, including <a href=\"http://software.intel.com/en-us/articles/intel-mkl/\" target=\"_new\">Intel Math Kernel Library</a>, <a href=\"http://software.intel.com/en-us/articles/intel-ipp/\" target=\"_new\">Intel Integrated Performance Primitives</a>, <a href=\"http://eigen.tuxfamily.org/index.php?title=Main_Page\" target=\"_new\">Eigen</a>, and <a href=\"http://arma.sourceforge.net/\" target=\"_new\">Armadillo</a>.[/column]\r\n\r\n[column type=\"one-fifth\"].[/column][column type=\"three-fifths\"][tab_nav type=\"four-up\"][tab_nav_item title=\"Base\" active=\"true\"][tab_nav_item title=\"Linear Algebra\" active=\"\"][tab_nav_item title=\"Convolution\" active=\"\"][tab_nav_item title=\"Utility\" active=\"\"][/tab_nav][tabs][tab active=\"true\"]<img src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/01/matrix_multiply-1024x730.png\" alt=\"Matrix Multiply\" width=\"491\" height=\"350\" />[line]<img src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/01/matrix_transpose-1024x730.png\" alt=\"Matrix Transpose\" width=\"491\" height=\"350\" />[line]<img src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/01/fft-1024x730.png\" alt=\"FFT\" width=\"491\" height=\"350\" />[/tab][tab]<img src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/01/lu-1024x730.png\" alt=\"LU\" width=\"491\" height=\"350\" />[line]<img src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/01/svd-1024x730.png\" alt=\"SVD\" width=\"491\" height=\"350\" />[line]<img src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/01/solve-1024x730.png\" alt=\"Solve\" width=\"491\" height=\"350\" />[/tab][tab]<img src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/01/convs-1024x730.png\" alt=\"Convs\" width=\"491\" height=\"350\" />[line]<img src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/01/conv-1024x730.png\" alt=\"Conv\" width=\"491\" height=\"350\" />[/tab][tab]<img src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/01/sum-1024x730.png\" alt=\"sum\" width=\"491\" height=\"350\" />[line]<img src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/01/sort-1024x730.png\" alt=\"sort\" width=\"491\" height=\"350\" />[line]<img src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/01/scan-1024x730.png\" alt=\"scan\" width=\"491\" height=\"350\" />[/tab][/tabs][/column][column type=\"one-fifth\" last=\"true\"].[/column]\r\n<div id=\"free\"></div>\r\n[/content_band]\r\n\r\n[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\" ]\r\n\r\n[column type=\"one-half\"].[/column][column type=\"one-fourth\"].[custom_headline type=\"center\"  level=\"h3\" class=\"mtn\"]ArrayFire Free[/custom_headline][/column][column type=\"one-fourth\" last=\"true\"].[/column][column type=\"one-fourth\"  fade=\"true\" fade_animation=\"in-from-left\" ][image src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/01/right-leaning-e1391788779101.png\"][/column][column type=\"one-fourth\"]\r\n<table>\r\n<tbody>\r\n<tr>\r\n<td>[custom_headline type=\"center\" level=\"h5\"]Functions[/custom_headline]</td>\r\n</tr>\r\n<tr>\r\n<td>[icon type=\"check-circle\" style=\"color: #8cc540;\"] Basic Operations</td>\r\n</tr>\r\n<tr>\r\n<td>[icon type=\"check-circle\" style=\"color: #8cc540;\"] Arithmetic</td>\r\n</tr>\r\n<tr>\r\n<td>[icon type=\"check-circle\" style=\"color: #8cc540;\"] Signal Processing</td>\r\n</tr>\r\n<tr>\r\n<td>[icon type=\"check-circle\" style=\"color: #8cc540;\"] Image Processing</td>\r\n</tr>\r\n<tr>\r\n<td>[icon type=\"check-circle\" style=\"color: #8cc540;\"] Statistics</td>\r\n</tr>\r\n<tr>\r\n<td>[icon type=\"check-circle\" style=\"color: #8cc540;\"] Visualization</td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n[gap size=\"17%\"][button type=\"flat\" block=\"true\" shape=\"rounded\" size=\"large\"  href=\"http://sandbox.arrayfire.com/download\"]Download ArrayFire[/button] [/column][column type=\"one-fourth\"]\r\n<table>\r\n<tbody>\r\n<tr>\r\n<td>[custom_headline type=\"center\" level=\"h5\"]Hardware[/custom_headline]</td>\r\n</tr>\r\n<tr>\r\n<td>[icon type=\"check-circle\" style=\"color: #8cc540;\"] CUDA</td>\r\n</tr>\r\n<tr>\r\n<td>[icon type=\"check-circle\" style=\"color: #8cc540;\"] OpenCL</td>\r\n</tr>\r\n<tr>\r\n<td>[custom_headline type=\"center\" level=\"h5\"]Languages[/custom_headline]</td>\r\n</tr>\r\n<tr>\r\n<td>[icon type=\"check-circle\" style=\"color: #8cc540;\"] C/C++</td>\r\n</tr>\r\n<tr>\r\n<td>[icon type=\"check-circle\" style=\"color: #8cc540;\"] Fortran</td>\r\n</tr>\r\n<tr>\r\n<td>[custom_headline type=\"center\" level=\"h5\"]Advanced[/custom_headline]</td>\r\n</tr>\r\n<tr>\r\n<td> [icon type=\"check-circle\" style=\"color: #8cc540;\"] GFOR Loop</td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n[/column][column type=\"one-fourth\" last=\"true\"]\r\n<table>\r\n<tbody>\r\n<tr>\r\n<td>[custom_headline type=\"center\" level=\"h5\"]Platforms[/custom_headline]</td>\r\n</tr>\r\n<tr>\r\n<td>[icon type=\"check-circle\" style=\"color: #8cc540;\"] Windows</td>\r\n</tr>\r\n<tr>\r\n<td>[icon type=\"check-circle\" style=\"color: #8cc540;\"] Linux</td>\r\n</tr>\r\n<tr>\r\n<td>[icon type=\"check-circle\" style=\"color: #8cc540;\"] Mac</td>\r\n</tr>\r\n<tr>\r\n<td>[custom_headline type=\"center\" level=\"h5\"]Data Types[/custom_headline]</td>\r\n</tr>\r\n<tr>\r\n<td>[icon type=\"check-circle\" style=\"color: #8cc540;\"] All types</td>\r\n</tr>\r\n<tr>\r\n<td>[custom_headline type=\"center\" level=\"h5\"]Support[/custom_headline]</td>\r\n</tr>\r\n<tr>\r\n<td> [icon type=\"check-circle\" style=\"color: #8cc540;\"] Forums</td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n<div id=\"licensing\"></div>\r\n[/column][gap size=\"8%\"]\r\n\r\n[pricing_table id=\"pricing\" columns=\"3\"][pricing_table_column  featured=\"false\" title=\"ArrayFire Developer\" currency=\"$\" price=\"499 / year\" interval=\"$149 / year (Academic)\"] [icon_list] [icon_list_item type=\"star\" ]Dedicated license [/icon_list_item] [icon_list_item  type=\"times-circle\"] Starts at 2 GPUs or devices [/icon_list_item] [icon_list_item type=\"times-circle\"] Linear algebra [/icon_list_item] [icon_list_item type=\"times-circle\"] Sparse matrices [/icon_list_item][icon_list_item type=\"times-circle\"] Phone support [/icon_list_item][icon_list_item type=\"times-circle\"] Premium support &amp; consulting [/icon_list_item][/icon_list] [button href=\" http://sandbox.arrayfire.com/product-category/arrayfire/\"] Purchase [/button] [/pricing_table_column] [pricing_table_column featured=\"true\" featured_sub=\"Most Popular\" title=\"ArrayFire Pro\" currency=\"$\" price=\"1,499 / year\" interval=\"$525 / year (Academic)\"] [icon_list] [icon_list_item type=\"star\"] Dedicated license [/icon_list_item] [icon_list_item type=\"star\"] Starts at 2 GPUs or devices [/icon_list_item] [icon_list_item type=\"star\"] Linear algebra [/icon_list_item][icon_list_item type=\"star\"] Sparse matrices [/icon_list_item][icon_list_item type=\"times-circle\"] Phone support [/icon_list_item][icon_list_item type=\"times-circle\"] Premium support &amp; consulting [/icon_list_item] [/icon_list] [button href=\" http://sandbox.arrayfire.com/product-category/arrayfire/\"]Purchase [/button] [/pricing_table_column] [pricing_table_column featured=\"false\" title=\"ArrayFire Enterprise\" currency=\"$\" price=\"2,999 / year\" interval=\"$1,049 / year (Academic)\"] [icon_list] [icon_list_item type=\"star\"] Dedicated license [/icon_list_item] [icon_list_item type=\"star\"] Starts at 2 GPUs or devices [/icon_list_item] [icon_list_item type=\"star\"] Linear algebra [/icon_list_item][icon_list_item type=\"star\"]Sparse matrices [/icon_list_item][icon_list_item type=\"star\"] Phone support[/icon_list_item][icon_list_item type=\"star\"]Premium support &amp; consulting [/icon_list_item][/icon_list] [button href=\" http://sandbox.arrayfire.com/product-category/arrayfire/\"] Purchase [/button] [/pricing_table_column] [/pricing_table]\r\n<div id=\"whitepapers\"></div>\r\n[/content_band]\r\n\r\n[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\" ]\r\n\r\n[column type=\"one-half\"][gap size=\"20%\"][image type=\"thumbnail\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/02/wp2.jpg\"][button href=\"http://go.sandbox.arrayfire.com/l/37882/2014-05-29/dw83/37882/18534/arrayfire_matrix_computations_chrzeszczyk.pdf\" shape=\"rounded\" size=\"regular\" block=\"true\"]Download <em>Matrix Computations on the GPU with ArrayFire for C/C++</em>[/button][/column][column type=\"one-half\" last=\"true\"]\r\n\r\n[custom_headline type=\"left\" level=\"h3\" ]Whitepapers[/custom_headline]\r\n\r\n<img src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/02/Whitepaper-1.png\" alt=\"Whitepaper 1\" width=\"466\" height=\"374\" />\r\n\r\n&nbsp;\r\n\r\n[/column][gap size=\"10%\"]\r\n\r\n[column type=\"one-half\"]<img src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/02/Whitepaper-2.png\" alt=\"Whitepaper 2\" width=\"442\" height=\"304\" />[/column][column type=\"one-half\" last=\"true\"][image type=\"thumbnail\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/02/wp1.jpg\"][button href=\"http://go.sandbox.arrayfire.com/l/37882/2014-05-29/dw81/37882/18532/arrayfire_whitepaper.pdf\" shape=\"rounded\" size=\"regular\"  block=\"true\" float=\"right\"]Download <em>High-level GPU Computing in C, C++, and Fortran</em>[/button][/column]\r\n<div id=\"casestudies\"></div>\r\n[/content_band]\r\n\r\n[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\" ]\r\n\r\n[column type=\"one-half\"][gap size=\"12%\"][custom_headline type=\"left\" level=\"h3\" ]Case Studies[/custom_headline]\r\n\r\nLearn how ArrayFire has worked in real code, including applications in academia, finance, government, life sciences, manufacturing, media, and oil &amp; gas.[/column][column type=\"one-half\" last=\"true\" fade=\"true\" fade_animation=\"in-from right\"][image src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/02/model-A-e1409844543721.png\"][/column][custom_headline type=\"left\" level=\"h5\"]Academia [/custom_headline][accordion][accordion_item title=\"Accelerating LTE Simulation | Tsinghua University\"]\r\n\r\nAuthors: Yuan Gao, Yin Sun, Chun Hui Zhou, Xin Su, Xi Bin Xu, Shi Dong Zhou, Tsinghua University\r\nSpeedup: 3X\r\n\r\nFast simulations are a driving force in several research projects. However, the accompanying long simulation times can tend to be a drag in many of these projects. In this article, we shall bring up the example of the work on 3GPP LTE System Simulation by Yuan Gao et al (from Tsinghua University, Beijing) and demonstrate how the use of ArrayFire software can significantly improve the simulator performance and lead to faster validation times in simulation projects.[/accordion_item]\r\n\r\n[accordion_item title=\"High Performance Compressive Sensing | Rice University\"]\r\n\r\nAuthors: Nabor Reyna and Wotao Yin from Rice University\r\nSpeedup: 5X\r\n\r\nThis work deals with reconstruction of signals using partial Fourier matrices (RecPF). The major computational components of the algorithm involve shrinkage and FFTs. ArrayFire software is employed to accelerate this compute-heavy code.[/accordion_item]\r\n\r\n[accordion_item title=\"Power System Simulations | Indian Institute of Technology, Roorkee, India\"]\r\nAuthors: Indian Institute of Technology, Roorkee\r\nSpeedup: 35X\r\n\r\nPower flow studies are one of the most important aspects of power system planning and operation. The power flow reveals the sinusoidal steady state characteristics of the entire system - voltages, real and reactive power generated, and absorbed and line losses- elucidating the voltage magnitudes and angles at each bus, the generation of each generating unit, and real and reactive power losses in the system. All this is necessary to ensure the security, economy, and control of electrical energy distribution. Learn how ArrayFire software can deliver magnitudes of performance improvement over CPU-based solutions.[/accordion_item]\r\n\r\n[accordion_item title=\"Antenna Array Simulations | University of Naples Federico II\"]\r\n\r\nAuthors: A. Capozzoli, C. Cucio, A. Liseno at University of Naples Federico II\r\nSpeedup: 24X\r\n\r\nAntenna array design involves repeated simulation to tune the many parameters involved, and waiting around for simulations to finish is no fun. Offloading the optimization problem onto the GPU cuts that time down significantly. In their recent paper, Capozzoli, Curcio, and Liseno of University of Naples Federico II demonstrated how a simple modification to their echo generator array simulation took advantage of the GPU to bring immediate speedups.[/accordion_item]\r\n\r\n[accordion_item title=\"Laplace Transform Inversion | Acunum Algorithms and Simulations\"]\r\n\r\nAuthors: Patrick Kano and Moysey Brio at Acunum Algorithms and Simulations\r\nSpeedup: 3.8X\r\n\r\nThe numerical inversion of the Laplace transform is a long standing problem due its implicit ill-posedness. Patrick Kano and Moysey Brio of Acunum Algorithms and Simulations, with their experience in computational methods and algorithm development, found a solution that not only works, but is very fast.[/accordion_item]\r\n\r\n[accordion_item title=\"Compressed Sensing for Image Reconstruction | College of Engineering, Roorkee, India\"]\r\n\r\nAuthors: Kuldeep Yadav, Ankush Mittal, M. A. Ansar, and Avi Srivastava, College of Engineering, Roorkee, India\r\nSpeedup: 8X\r\n\r\nCompressed sensing is very critical in the areas of medical image reconstruction, image acquisition or sensor networks. An algorithm for compressed sensing developed using a Basis Pursuit Algorithm shows over 8X speedup when run on an NVIDIA GPU.[/accordion_item]\r\n\r\n[accordion_item title=\"Fat/Water Reconstruction for Medical Images | Case Western Reserve University\"]\r\n\r\nAuthors: D. H. Johnson, S. Narayan, C. A. Flask, and D. L. Wilson, Case Western Reserve University\r\nSpeedup: 11.6X\r\n\r\nCase Western Reserve University researchers turned to GPUs running ArrayFire software to develop a fast and robust version of the \"Iterative Decomposition of water and fat with an Echo Asymmetry and Least-squares\" (IDEAL) reconstruction algorithm. This algorithm uses a lot of Image Processing algorithms for reconstruction, and was shown to achieve very high speedups.\r\n[/accordion_item]\r\n\r\n[/accordion]\r\n<h5>Finance</h5>\r\n[accordion][accordion_item title=\"Option Pricing | Koch Supply and Trading\"]\r\n\r\nAuthors: Koch Supply &amp; Trading\r\nSpeedup: 51.8X\r\n\r\nAndrew Shin, Market Risk Manager of Koch Supply &amp; Trading, achieves significant performance increases on option pricing algorithms using ArrayFire software to accelerate his code with GPUs. Andrew says, \"My buddy and I are, at best, novice programmers and we couldn\'t imagine having to figure out how to code all this in CUDA.\" But he found ArrayFire software to be straight-forward. With these results, he says he can see ArrayFire software and GPUs populating Koch\'s mark-to-futures cube, which contains its assets, simulations, and simulated asset prices.[/accordion_item]\r\n\r\n[accordion_item title=\"GPU Computing in Automated Trader | Automated Trader\"]\r\n\r\nAuthors: Automated Trader\r\nSpeedup: 37.5X\r\n\r\nThe Q1 2012 issue of Automated Trader contains an excellent Mashup piece reviewing software for algorithmic trading. The article provides a wonderful glimpse into the 1-2 month adventure of Andy Webb, Automated Trader.s Founder, and Wrecking Crew building a fast trading platform from several technologies. The full trading platform they built was quite extensive. The part that caught our eye was the core computational component of the pipeline. That component involved permuting 1,000 potential pairs with cointegration tests for 350 time windows on each potential pair.\r\n[/accordion_item]\r\n[accordion_item title=\"GPUs in Quantitative Analytics and Finance | Private Bank\"]\r\n\r\nThe world of Quantitative finance is all about getting accurate results really, really fast. ArrayFire is working with one of the largest banks in Spain to maximize their output using GPUs. Click the link below for an overview of the uses of GPU computing in finance.[/accordion_item][/accordion]\r\n<h5>Government</h5>\r\n[accordion][accordion_item title=\"Powering Mars Research | NASA and UAA in Anchorage\" ]\r\n\r\nAuthors: NASA and UAA in Anchorage\r\nSpeedup: 5X\r\n\r\nThe main thrust of this research is improving mars rover image compression via GPUs and genetic algorithms. With ArrayFire software and GPUs, the researchers were able to achieve 5X speedups on the larger data sizes. The algorithm works by pairing neighboring pixels with a random one and then adjusting the random pixel based on whether it incrementally improves the original image. Babb described the algorithm as an embarrassingly parallel process, ideally suited to GPU acceleration. He estimates he has been able to achieve a 20 to 30 percent error reduction in subjects like fingerprints and satellite imagery.[/accordion_item]\r\n\r\n[accordion_item title=\"Radar Image Formation | System Planning Corporation\"]\r\n\r\nAuthors: Gary Rubin and Earl Sager - System Planning Corporation\r\nSpeedup: ~45X\r\n\r\nRadar imaging is computationally intensive. As a result, many imaging algorithms apply FFT-based approximations. While efficient, these algorithms sacrifice data fidelity for speed. Other algorithms better preserve information, but are often too slow for many applications. At System Planning Corporation (SPC) , we have implemented a SAR/ISAR imaging routine based on the Backprojection algorithm. Using ArrayFire software, we have demonstrated speedups of roughly 45x for large datasets.[/accordion_item]\r\n\r\n[accordion_item title=\"Radar Clutter Reduction | System Planning Corporation\"]\r\n\r\nAuthors: David Berger and Gary Rubin - System Planning Corporation\r\nSpeedup: 5-10X\r\n\r\nSystem Planning Corporation (SPC) uses ArrayFire software to accelerate radar processing algorithms. The system processes raw data from marine navigation radars using a variety of thresholding techniques to extract real targets from clutter. This involves highly data-parallel processing in which each radar pulse is subjected to the same computations; very few operations occur across multiple pulses. Using ArrayFire software, SPC has achieved 10x speed improvements relative to a Core i7-920 CPU and 5x improvements relative to a realtime DSP implementation.[/accordion_item]\r\n\r\n[accordion_item title=\"Novel Algorithms for Linear Algebra | SAIC\" ]\r\n\r\nAuthors: Nolan Davis and Daniel Redig, SAIC\r\nSpeedup: 3.5X\r\n\r\nNolan Davis and Daniel Redig at SAIC recently presented work on Hybrid GPU/Multicore Solutions for Large Linear Algebra Problems where they developed a novel algorithm for LU decomposition, one of the most important routines in linear algebra. They presented a Hybrid CPU/GPU computing approach, where problems too large to fit in GPU memory can also be solved faster than using only the CPU.\r\n[/accordion_item]\r\n[accordion_item title=\"Geolocation | BAE Systems\"]\r\n\r\nAuthors: BAE Systems\r\nSpeedup: 17X\r\n\r\nGeolocation is the identification of the real-world geographic location of a target of interest. In this application, the system receives the signal with an array of several antennas and computes the direction of arrival of the radio energy by measuring the time difference of arrival (or the phase difference) at the different antennas.\r\n[/accordion_item]\r\n[accordion_item title=\"Tsunami Modeling | University of Minnesota, Boise State, Saint Scholastica, and NCAR\"]\r\n\r\nAuthors: University of Minnesota, Boise State, Saint Scholastica, and NCAR\r\nSpeedup: 3-5X\r\n\r\nNatural catastrophic disasters like tsunamis commonly strike with little warning. For most people, tsunamis are underrated as major hazards. People sometimes wrongly believe that they occur infrequently and only along distant coasts. Tsunamis are usually caused by earthquakes. Seismic signals can give some margin of warning since the speed of tsunami waves travels at 1/30 the speed of seismic waves. Still there is little time between the creation of the tsunami and its impact making fast processing critical to producing effective warning systems. ArrayFire software was used to run an RBF simulation on the GPU with a time to solution not available by other alternatives.\r\n[/accordion_item][/accordion]\r\n<h5>Life Sciences</h5>\r\n[accordion][accordion_item title=\"Parallelized Gene Predictors | University of Quebec\" ]\r\n\r\nAuthors: University of Quebec\r\nSpeedup: 43X\r\n\r\nComputerized approaches to studying the human genome are challenged by the exploding amount of data, which doubles roughly every 6 months. In order to deal with this burgeoning datasets, demands for faster processing power continue to arise. This work focuses on predicting genes using frequency analysis with FFTs and with an equivalent technique known as Goertzel\'s algorithm. In these applications, the emphasis of this paper is to propose tools to geneticists and molecular biologists for the prediction or identification of new genes using existing complementary strategies. The criteria for these tools are speed, reliability, accuracy and ease of use, thus requiring little training.\r\n[/accordion_item]\r\n[accordion_item title=\"Pathology Advances with GPUs | Northeastern University\" ]\r\n\r\nAuthors: Laboratory for Spectral Diagnosis at Northeastern University\r\nSpeedup: 100X+\r\n\r\nOne element of the hyperspectral image analysis workflow that requires more than a traditional desktop workstation or personal computer is Hierarchical Cluster analysis (HCA). HCA requires a large amount of data space and substantial computation time (~11 hours) for typical datasets using a single processor personal computer. Rather than following the traditional approach of moving to a lower level programming language like C or C++ and complex parallel programming paradigms such as OpenMP or the Massage Passing Interface (MPI), the lab utilized graphics processing units, or GPUs, and the ArrayFire software platform. The solution allowed the lab to dramatically increase the performance of the analysis while substantially decreasing the amount of calendar time to reach the desired results.\r\n[/accordion_item]\r\n[accordion_item title=\"Hepatitis C Virus - Mutation Modeling | Centers for Disease Control and Prevention\" ]\r\n\r\nAuthors: CDC Research and Development Team\r\nSpeedup: ~20X\r\n\r\nThis case study provides a look at biological research regarding coordinated mutations of the Hepatitis C Virus (HCV). ArrayFire provided collaborative R&amp;D resources and greatly improved the speed of this HCV research with the use of parallelization, reducing the computing time from 40 days to less than 1 day. Most importantly, the conclusion of the case study illustrates the the relative price-performance of personal supercomputers that leverage GPUs and ArrayFire software provides a compelling solution versus other architectures and approaches.\r\n[/accordion_item]\r\n[accordion_item title=\"Accelerating the SPM Package for Neuroimaging | Georgia Institute of Technology\" ]\r\n\r\nAuthors: Georgia Institute of Technology\r\nSpeedup: 3.5X\r\n\r\nThe Georgia Tech team explores the value of ArrayFire software and GPUs for fMRI workflows within the popular SPM - Statistical Parametric Mapping software widely used in neuroscience research.\r\n[/accordion_item]\r\n[accordion_item title=\"Medical Image Compression | Indian Institute of Technology, Roorkee, India\" ]\r\n\r\nAuthors: Jaideep Singh, Ipseeta Aruni, R. Balasubramanian - IIT - Roorkee, India\r\nSpeedup: 38X\r\n\r\nThis study presents the acceleration of Haar wavelet-based image compression algorithm for medical imaging on the Graphics Processing Unit (GPU) using ArrayFire software. Due to bandwidth and image size constraints of medical imaging systems, image compression plays a vital role in reducing the bit rate of transmission or storage. Wavelet-based image compression provides the most promising approach for high quality image compression.\r\n[/accordion_item]\r\n[accordion_item title=\"Brain Displacement | Spencer Technologies\"]\r\n\r\nAuthors: Spencer Technologies\r\nSpeedup: 12X\r\n\r\nSpencer describes how ArrayFire software facilitates the development of fast algorithms enabling observation of brain displacement across depth with sampling density that far surpasses previous benchmarks.\r\n[/accordion_item]\r\n[accordion_item title=\"Multidimensional Scaling for Genomics | Leibniz Institute of Plant Genetics and Crop Plant Research\" ]\r\n\r\nAuthors: Leibniz Institute of Plant Genetics and Crop Plant Research\r\nSpeedup: 20-35X\r\n\r\nMultidimensional scaling (MDS) is a general computing technique to turn a distance matrix into a set of reconstructed points with pair-wise relationships approximating the original distances by points located in a usually low-dimensional space. ArrayFire software is used to enhance execution of the HiT-MDS procedure and delivers considerable performance improvement.\r\n[/accordion_item]\r\n[accordion_item title=\"Drug Delivery Model | Georgia Institute of Technology\"]\r\n\r\nAuthors: Georgia Institute of Technology\r\nSpeedup: 70X\r\n\r\nIn this work, the researchers simulate the delivery of a novel nanoparticle chemotherapy drug to cancerous tissue. Simulation allows scientists to predict experimental outcomes and thus reduce the cost of development and time to clinical relevance. The simulation model includes blood vessels, tumor cells, and healthy cells and an engine to calculate the spatial distributions of both drug and oxygen. ArrayFire software is used to speed up the diffusion calculations for the drug and oxygen within the tissue.\r\n[/accordion_item]\r\n[accordion_item title=\"Biomedical Infrared Spectroscopy | University of Manchester and Nofima Mat, Norway\" ]\r\n\r\nAuthors: University of Manchester and Nofima Mat, Norway\r\nSpeedup: Hours of runtime reduction\r\n\r\nThe authors present an iterative algorithm that applies full Mie scattering theory and avoids noise accumulation in their iterative algorithm by integrating a curve-fitting step. ArrayFire software along with NVIDIA GPUs are leveraged to reduce the time added by the curve-fitting step.\r\n[/accordion_item][/accordion]\r\n<h5>Manufacturing</h5>\r\n[accordion][accordion_item title=\"Feature Learning on Images | Stanford\" ]\r\n\r\nAuthors: Andrew Ng, Stanford University\r\nSpeedup: Ability to process many images in parallel\r\n\r\nStanford researchers in Andrew Ng\'s group used GPUs and ArrayFire software to speed up their work on Feature Learning Architectures. They decided to use ArrayFire software for this study because of the need to quickly evaluate many architectures on thousands of images. ArrayFire software taps into the immense computing power of GPUs and speeds up research utilizing many images.\r\n[/accordion_item]\r\n[accordion_item title=\"Tomography of Vegetation - Filtered Back-Projection and Non-Uniform FFTs | Universita di Napoli Federico II\" ]\r\n\r\nAuthors: Drs. Capozzoli, Curcio, di Vico, and Liseno, Universita di Napoli Federico II\r\nSpeedup: 10X\r\n\r\nIn order to investigate changes of forest biomass, scientists use microwave tomography to image the vegetation. At the smallest scale, individual plants can be imaged to investigate branching and growth, but even synthetic aperture radar can reveal large-scale changes in regional ecology. To the right, you can see the experimental setup to image an individual plant.\r\n[/accordion_item]\r\n[accordion_item title=\"Action Recognition with Independent Subspace Analysis | Stanford University\" ]\r\n\r\nAuthors: Quoc Le, Will Zou, Serena Yeung, Andrew Ng, Stanford University\r\nSpeedup: 4.4X\r\n\r\nIn a paper at this year\'s CVPR 2011, entitled \"Learning hierarchical invariant spatio-temporal features for action recognition with independent subspace analysis\", the authors explain how their unsupervised feature learning algorithm competes with other algorithms that are hand crafted or use learned features. For their training purposes, they used a multi-layered stacked convolutional ISA (Independent subspace analysis) network. An ISA is used for learning features from image patches without supervision.\r\n[/accordion_item][/accordion]\r\n<h5>Media and Computer Vision</h5>\r\n[accordion][accordion_item title=\"Fast Computer Vision with OpenCV and ArrayFire | OpenCV Blogger\" ]\r\n\r\nAuthors: OpenCV Blogger\r\nSpeedup: ~10X\r\n\r\nThe OpenCV library is the defacto standard for doing computer vision and image processing research projects. OpenCV includes several hundreds of computer vision algorithms, aimed for use in realtime vision applications. This case study shows how to use both libraries together. There is a simple example application that demonstrates using OpenCV for webcam access and ArrayFire for some basic processing routines and displaying results.\r\n[/accordion_item]\r\n[accordion_item title=\"Video Processing | Google\" ]\r\n\r\nAuthors: Google and Stanford\r\nSpeedup: 10-20X\r\n\r\nVideo content analysis is the basis for categorizing videos and enabling search by content. Growing interest in using sparse-coding methods to extract motion features in video in support of video content analysis led to the application of ArrayFire software to improve performance by substantially accelerating the solution of the L1-regularized least-squares optimization problem.\r\n[/accordion_item]\r\n[accordion_item title=\"Action Recognition with Independent Subspace Analysis | Stanford University\"]\r\n\r\nAuthors: Quoc Le, Will Zou, Serena Yeung, Andrew Ng, Stanford University\r\nSpeedup: 4.4X\r\n\r\nIn a paper at this year\'s CVPR 2011, entitled \"Learning hierarchical invariant spatio-temporal features for action recognition with independent subspace analysis\", the authors explain how their unsupervised feature learning algorithm competes with other algorithms that are hand crafted or use learned features. For their training purposes, they used a multi-layered stacked convolutional ISA (Independent subspace analysis) network. An ISA is used for learning features from image patches without supervision.\r\n[/accordion_item]\r\n[accordion_item title=\"Music Beat Analysis | Georgia Tech\"]\r\n\r\nAuthors: Vidhur Vohra - Georgia Tech\r\nSpeedup: 15X\r\n\r\nDid you ever wonder how the music visualizer in your media player works? Watching it pulsate in synchrony with the beats of the song is almost as entertaining as listening to the song itself! Researchers have been attempting to detect beats in audio signals for many years, and there are many techniques available, from the simplest (and least accurate) to more complicated algorithms that are highly accurate. All algorithms, though, perform some form of signal processing and frequency analysis, applications highly suited to GPU Computing.\r\n[/accordion_item]\r\n[accordion_item title=\"Optimization Methods for Deep Learning | Stanford Artificial Intelligence Laboratory\" ]\r\n\r\nAuthors: Stanford Artificial Intelligence Laboratory\r\nSpeedup: Improved accuracy\r\n\r\nResearchers at SAIL (Stanford Artificial Intelligence Laboratory), have done it again. They have successfully used ArrayFire software to speed up the training part of Deep Learning algorithms. In their paper titled .On Optimization Methods for Deep Learning., they experiment with some of the well known training algorithms and demostrate their scalability across parallel architectures (GPUs as well as multi-machine networks). The algorithms include SGDs (Stochastic Gradient Descent) L-BFGS (Limited BFGS used for solving non-linear problems), CG (Conjugate Gradient).\r\n[/accordion_item]\r\n[accordion_item title=\"Feature Learning on Images | Stanford University\" ]\r\n\r\nAuthors: Andrew Ng, Stanford University\r\nSpeedup: Ability to process many images in parallel\r\n\r\nStanford researchers in Andrew Ng’s group used ArrayFire software to speed up their work on Feature Learning Architectures. They decided to use ArrayFire software for this study because of the need to quickly evaluate many architectures on thousands of images. ArrayFire software taps into the immense computing power of GPUs and speeds up research utilizing many images.\r\n[/accordion_item]\r\n[accordion_item title=\"Digital Holography for Imaging | National University of Ireland, Maynooth\" ]\r\n\r\nAuthors: Nitesh Pandey, Damien Kelly, Bryan Hennelly and Thomas Naughton from the National University of Ireland, Maynooth\r\nSpeedup: 17X\r\n\r\nDigital holography is a powerful imaging technique with many new applications like true 3D display. It allows the capture of both amplitude and phase information of the light reflected off the surface of 3D objects. Researchers at the National University of Ireland, Maynooth are developing techniques based on digital holography for 3D display applications.\r\nReconstruction of large digital holograms can be computationally intensive to generate on CPUs, but GPUs running ArrayFire software offer amazing possibilities.\r\n[/accordion_item][/accordion]\r\n<h5>Oil and Gas</h5>\r\n[accordion][accordion_item title=\"3D Mantle Convection - Geodynamics | Boise State, University of Colorado, University of Minnesota\" ]\r\n\r\nAuthors: Boise State, University of Colorado, University of Minnesota\r\nSpeedup: 2.5-4.5X\r\n\r\nThe authors introduce a GPU implementation of a three-dimensional mantle convection modeling at a high Rayleigh number to the solid earth geophysics community. They outline code development time, compare performance of CPUs versus GPUs, and deliver powerful visualizations.\r\n[/accordion_item]\r\n[accordion_item title=\"Ground Water Simulations | Louisiana State University\" ]\r\n\r\nAuthors: Kevin R. Tubbs and Frank T-C. Tsai at Louisiana State University\r\nSpeedup: &gt;20X\r\n\r\nA lattice Boltzmann method for solving the shallow water equations and the advection-dispersion equation is developed and implemented on graphics processing unit (GPU)-based architectures. The proposed LBM is implemented to an NVIDIA Computing Processor in a single GPU workstation. GPU computing is performed using ArrayFire software. Mass transport with velocity-dependent dispersion in shallow water flow is simulated by combining the MRT-LBM model and the TRT-LBM model. The GPU parallel performance increases as the grid size increases. The results indicate the promise of the GPU-accelerated LBM for modeling mass transport phenomena in shallow water flows.\r\n[/accordion_item]\r\n[accordion_item title=\"Shallow Water Fluid Flow | Louisiana State University\" ]\r\n\r\nAuthors: Louisiana State University\r\nSpeedup: &gt;10X\r\n\r\nA lattice Boltzmann method (LBM) on high performance computing (HPC) environments for three-dimensional shallow water flow fields coupled to mass transport is developed. LBM is an attractive method for solving the multilayered shallow water equations because the extension to multilayer is straight forward with all of the simplicities and advantages of the LBM in mass transport in shallow water flows and the LBM performance on central processing unit (CPU)-based and graphics processing unit (GPU)-based HPC environments.\r\n[/accordion_item][/accordion]\r\n<div id=\"consulting\"></div>\r\n[/content_band]\r\n<div id=\"services\"></div>\r\n[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"][column type=\"one-half\"][custom_headline type=\"left\" level=\"h3\"]Hire our acceleration gang[/custom_headline]\r\n\r\nHire us to accelerate your code. Or to train your pros. Or even to write new code for your organization!\r\n\r\nWe’ve already helped thousands of organizations speedup their code. We’ve accelerated algorithms, data sets, and challenging problems of all types in the past, and we’re confident that we can accelerate your code too.\r\n\r\n[gap size=\"10%\"][button  type=\"flat\" shape=\"rounded\" href=\"http://meetme.so/arrayfire\" size=\"x-large\" ]Schedule a consultation[/button][/column][column type=\"one-half\" last=\"true\" class=\"center-text\"][gap size=\"5%\"][image src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/01/Bat-signal-e1391790016951.png\"][/column] [gap size=\"7%\"]\r\n\r\n[column type=\"one-third\" class=\"center-text\"][image src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/04/LFG-B-HORZ_202_K-e1396448935778.jpg\"][line]Lincoln Financial Group ported their CPU code (C/C++) to CUDA, and this enabled them to speed up the necessary financial calculations. Their primary objective was to reduce the time taken to run this code. Before porting to CUDA, they required 10-12 hours to run the entire code. After porting to CUDA, they were able to do the same work in about 30 minutes.[/column][column type=\"one-third\" class=\"center-text\"][image src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/02/reveal_logo-e1396449044651.jpg\"][line]Reveal Imaging, an SAIC company, uses ArrayFire in developing threat-detection products for transportation systems—such as cargo and baggage screening in airports—and in securing facilities and public events.[/column][column type=\"one-third\" last=\"true\" class=\"center-text\"][image src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/04/1800contactlogo.png\"][line]The Glasses.com app relies on computer vision to recognize the features of a user’s face. The application then generates an accurate, high-fidelity, three-dimensional model of that face, and virtually places selected frames on the image.[/column]\r\n<div id=\"starterpackage\"></div>\r\n[/content_band]\r\n\r\n[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"]\r\n\r\n[column type=\"one-half\" last=\"false\" fade=\"true\" fade_animation=\"in-from-left\" ][gap size=\"10%\"][image src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/01/comingatyou-e1409844224608.png\"][/column][column type=\"one-half\" last=\"true\"] [custom_headline type=\"left\" level=\"h3\"]Starter Package ($500)* [/custom_headline] Hire us for 1-day. We set you up with one engineer dedicated for a full day, as well as support from senior and test engineers throughout the day. Here\'s how it works:\r\n\r\n[icon_list] [icon_list_item type=\"angle-double-right\"]Initial phone call[/icon_list_item] [icon_list_item type=\"angle-double-right\"]You send us your code or algorithm [/icon_list_item] [icon_list_item type=\"angle-double-right\"]We analyze your code for potential GPU benefit[/icon_list_item][icon_list_item type=\"angle-double-right\"]We accelerate as much of the code as is possible within the 1-day effort, applying our extensive GPU software libraries[/icon_list_item][icon_list_item type=\"angle-double-right\"]We produce a final report, along with code[/icon_list_item] [/icon_list] The final report outlines the best approach to follow for full GPU-acceleration of your code.\r\n\r\n&nbsp;\r\n\r\n[button type=\"flat\" shape=\"rounded\" size=\"regular\" href=\"http://sandbox.arrayfire.com/product/consulting-starter-package/\" title=\"Starter Package\"]Buy the Starter Package[/button]\r\n\r\n&nbsp;\r\n\r\n*Limit one (1) per customer.\r\n\r\n[/column]\r\n<div id=\"customcode\"></div>\r\n[/content_band] [content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\" ]\r\n\r\n[column type=\"one-half\"][custom_headline type=\"left\" level=\"h3\" ]Custom Coding Work[/custom_headline]\r\n\r\n[custom_headline type=\"left\" level=\"h6\" ]Why Choose Us?[/custom_headline] [icon_list] [icon_list_item type=\"angle-double-right\"]Deep experience, already servicing thousands of customers[/icon_list_item] [icon_list_item type=\"angle-double-right\"]Quality work, with leading GPU software [/icon_list_item] [icon_list_item type=\"angle-double-right\"]Efficient delivery, with ArrayFire\'s code base we can deliver results fast[/icon_list_item][icon_list_item type=\"angle-double-right\"]Proven, we make customers happy through faster code[/icon_list_item][/icon_list] [custom_headline type=\"left\" level=\"h6\" ]Our Strengths[/custom_headline] [icon_list] [icon_list_item type=\"angle-double-right\"]Excellence in GPU code[/icon_list_item] [icon_list_item type=\"angle-double-right\"]Well-written, efficient C, C++, Fortran, Java, or .NET [/icon_list_item] [icon_list_item type=\"angle-double-right\"]Application porting and integration[/icon_list_item][icon_list_item type=\"angle-double-right\"]Tuning software to peak hardware performance[/icon_list_item][icon_list_item type=\"angle-double-right\"]Scalability of algorithms to handle enterprise-level programs[/icon_list_item] [/icon_list][button type=\"flat\" shape=\"rounded\" size=\"regular\" href=\"http://meetme.so/arrayfire\" title=\"Tech Consultation\"]Schedule a consultation[/button] [/column][column type=\"one-half\"  fade=\"true\" fade_animation=\"in-from-right\" last=\"true\" ][gap size=\"25%\"][image src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/02/Holoham-e1409844116266.png\"][/column]\r\n<div id=\"training\"></div>\r\n[/content_band] [content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\" ]\r\n\r\n[column type=\"one-half\"][custom_headline type=\"left\" level=\"h3\" ]Training[/custom_headline] Attendees will receive the latest industry knowledge and techniques for GPU computing in CUDA and OpenCL.\r\n\r\nArrayFire offers up to four days of specialized GPU training in CUDA and OpenCL programming. We provide customized on-site training courses for customers that have 3 or more attendees from their organization. We can travel to your location and provide a CUDA or OpenCL training course tailored to meet your application-specific needs. We also offer individual trainings at our Atlanta office. Please <a href=\"mailto:sales@arrayfire.com\">contact us</a> to learn more about our training offerings.\r\n\r\nWe recommend that attendees have a working knowledge of C/C++ in order to gain the most from the training courses. For more information about the content of our training lectures and hands-on practicums, please see the following training course syllabus.\r\n\r\n[/column]\r\n\r\n[column type=\"one-half\" last=\"true\" fade=\"true\" fade_animation=\"in-from-right\" class=\"center-text\"][gap size=\"15%\"][image src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/02/Professor-AF_Exported-e1395436350363.png\" ][/column]\r\n\r\n[column type=\"one-half\" class=\"center-text\"] [gap size=\"17%\"]\r\n<div>[button type=\"flat\" shape=\"rounded\" size=\"jumbo\" href=\"mailto:sales@arrayfire.com?Subject=Training%20Inquiry\"] Register for Training[/button]</div>\r\n[/column][column type=\"one-half\" last=\"true\"][custom_headline type=\"left\" level=\"h3\"]Included in Course[/custom_headline]\r\n\r\n[icon_list] [icon_list_item type=\"angle-double-right\"]Instruction by an excellent and interesting expert, with hands-on exercises[/icon_list_item] [icon_list_item type=\"angle-double-right\"]Use of a laptop with CUDA and OpenCL capable GPUs and CPUs [/icon_list_item] [icon_list_item type=\"angle-double-right\"]Choice of Linux or Windows operating system[/icon_list_item][icon_list_item type=\"angle-double-right\"]Printed manual of lecture material[/icon_list_item][icon_list_item type=\"angle-double-right\"]Electronic copy of programming exercises[/icon_list_item][/icon_list]\r\n\r\n[/column]\r\n\r\n<span style=\"line-height: 1.5em;\">[gap size=\"5%\"]</span>\r\n\r\n[custom_headline type=\"left\" level=\"h3\"]Training Course Syllabus[/custom_headline] [column type=\"one-half\"] [custom_headline type=\"left\" level=\"h5\"  accent=\"false\"]Day 1, Introduction[/custom_headline] Lectures:\r\nGPU Computing Overview\r\nThe Programming Model\r\nBasic Dataset Mapping Techniques\r\nLibraries, ArrayFire\r\nProfiling Tools\r\n\r\nPractice:\r\nA Simple Kernel\r\nEquivalent ArrayFire Example\r\nUsing Libraries\r\nMonte Carlo Pi Estimation\r\nTiming and ArrayFire\r\nDebugging Code\r\n\r\n[custom_headline type=\"left\" level=\"h5\" accent=\"false\"]Day 3, Multi-GPU[/custom_headline] Lectures:\r\nMulti-GPU Use Cases\r\nMulti-GPUs: Contexts\r\nExisting Libraries\r\nScaling Across Multiple GPUs\r\n\r\nPractice:\r\nOut of Core Problems: Matrix Multiply\r\nTask Level Parallelism: Optimization\r\nArrayFire Multi-GPU[/column]\r\n\r\n[column type=\"one-half\" last=\"true\"] [custom_headline type=\"left\" level=\"h5\" accent=\"false\"]Day 2, Optimization[/custom_headline] Lectures:\r\nArchitecture: Grids, Blocks, and Threads\r\nMemory Model: Global, Shared, and Constant Memory\r\nAdvanced Mapping Techniques\r\nStreams: Asynchronos Launches and Concurrent Execution\r\nArrayFire: Lazy Evaluation and Code Vectorization\r\n\r\nPractice:\r\nMatrix Transpose\r\nOptimization Using Shared Memory\r\nMedian Filter\r\nOptimization Using Constant Memory\r\nStream Example\r\nArrayFire Example: Nearest Neighbor Algorithm\r\n\r\n[custom_headline type=\"left\" level=\"h5\" accent=\"false\"]Day 4, Algorithm Problems[/custom_headline]\r\n\r\nLectures and Practice:\r\nReductions\r\nScan Algorithms\r\nSort\r\nConvolution\r\nCustomer-Specific Problem [/column][/content_band]','Home','','inherit','open','open','','245-revision-v1','','','2014-10-15 18:46:16','2014-10-15 18:46:16','',245,'http://sandbox.arrayfire.com/245-revision-v1/',0,'revision','',0),(6894,7,'2014-10-15 21:07:56','2014-10-15 21:07:56','','glasses logo, promo-01','','inherit','open','open','','glasses-logo-promo-01','','','2014-10-15 21:07:56','2014-10-15 21:07:56','',0,'http://sandbox.arrayfire.com/wp-content/uploads/2014/10/glasses-logo-promo-01.png',0,'attachment','image/png',0),(6895,7,'2014-10-15 21:10:52','2014-10-15 21:10:52','','lincoln logo, promo-01','','inherit','open','open','','lincoln-logo-promo-01','','','2014-10-15 21:10:52','2014-10-15 21:10:52','',0,'http://sandbox.arrayfire.com/wp-content/uploads/2014/10/lincoln-logo-promo-01.png',0,'attachment','image/png',0),(6896,7,'2014-10-15 21:16:13','2014-10-15 21:16:13','','reveal logo, promo-01','','inherit','open','open','','reveal-logo-promo-01','','','2014-10-15 21:16:13','2014-10-15 21:16:13','',0,'http://sandbox.arrayfire.com/wp-content/uploads/2014/10/reveal-logo-promo-01.png',0,'attachment','image/png',0),(6899,7,'2015-02-05 15:49:13','2015-02-05 15:49:13','[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"][column type=\"one-fifth\"].[/column][column type=\"three-fifths\"][custom_headline type=\"left\" level=\"h3\"]Glasses.com App with 3Dfit Technology[/custom_headline][/column][column type=\"one-fifth\" last=\"true\"].[/column][gap size=\"3%\"]\r\n\r\n[column type=\"one-fifth\"].[/column][column type=\"three-fifths\"]<img class=\"alignright wp-image-6894 size-full\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/glasses-logo-promo-01.png\" alt=\"glasses logo, promo-01\" width=\"360\" height=\"120\" />Many find the experience of purchasing new glasses stressful. Even simply trying on frames can be uncomfortable when done in a public setting. Glasses.com set out to improve the experience by providing customers a way of virtually trying on glasses frames in the comfort of their homes via smart phone or tablet.\r\n\r\nThe Glasses.com app takes a picture of the user’s face, creates a 3D model of facial features, and virtually places selected frames on the user’s 3D image. The user is then free to view the 3D image from a variety of angles.\r\n\r\nThe application relies on computer vision to recognize the features of a user’s face and then generate an accurate, high-fidelity, 3D model of that face. Virtual frames can then be positioned on the 3D model.\r\n<blockquote>This app is amazing! It takes a 3D video of your face in just a few steps, and the rendering is spot on! RayBan Aviators are always a little big on me, and the app showed them on my face exactly as they fit in the store! —davidandrew</blockquote>\r\n<h4>Mission</h4>\r\nGlasses.com brought our engineers an excellent, fully-functioning version of the application; they simply wanted it accelerated. The app took full advantage of the CPU, which—left unchanged—would necessitate Glasses.com’s purchasing and managing a large amount of hardware. Further, the application was running at a slow thirty seconds per session, resulting in a sub-optimal user experience. Our engineers were tasked with parallelizing the process and reducing the session runtime.\r\n\r\nGlasses.com has several machines they are using in production. Their main setup consists of a Haswell/Ivy Bridge CPU paired with a GTX 690. The machine is able to run 4 processes concurrently. In the past, Glasses.com used a 32 core Intel process with a couple of K20X. They are still using these machines, but they are not the main configuration. They are also making use of the Amazon EC2 instances in case they exceed internal capacity.\r\n<h4>Action</h4>\r\nWe began by analyzing the application and targeting the process which would have the greatest impact on performance.\r\n\r\nThe most pressing concern was the rendering portion of code, which was running on the CPU. We implemented rendering in ArrayFire, which is specially optimized for CUDA GPUs. Further, we noticed that while the application was rendering-intensive, much of the rendering did not need to be precise—we could make estimations without compromising quality. This estimated rendering speedup operation was critical to the success of the project.\r\n\r\nWe discovered further opportunities for speedup within the architecture itself. The original app processed each frame in a separate thread; we accelerated this by combining each operation into a single kernel and running that kernel once. By moving the parallelization to the GPU we effectively took advantage of the strength of that architecture.\r\n<h4>Results</h4>\r\nUltimately, parallelizing the application and incorporating the ArrayFire software library accelerated the application to about ten seconds per session—roughly three times as fast as the original code, and twice as fast as the target speed. We further exceeded expectations by improving general stability when handling errors. <img class=\"aligncenter size-full wp-image-6903\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/glassesdotcom-benchmarks.jpg\" alt=\"glassesdotcom benchmarks\" width=\"700\" height=\"442\" />\r\n\r\nThe improvements in speed, stability, and user experience have resonated with the application’s target market, resulting in a 3.5/5 star rating in the Apple Store.\r\n<h6>About glasses.com</h6>\r\nGlasses.com launched in 2011 as an online retailer of prescription and non-prescription glasses and sunglasses, carrying a wide-range of styles and name brands such as Oakley, Ray-Ban, Kate Spade, Burberry, Maui Jim, and more.[/column][column type=\"one-fifth\" last=\"true\"].[/column][/content_band]','Case Study: glasses.com App with 3Dfit Technology','','publish','open','open','','case-study-glasses-com-app-with-3dfit-technology','','','2015-02-05 15:49:13','2015-02-05 15:49:13','',0,'http://sandbox.arrayfire.com/?page_id=6899',0,'page','',0),(6900,7,'2014-10-15 21:29:02','2014-10-15 21:29:02','Adopting a new look is tough! Haircut, clothing style, glasses—whatever the change may be, finding the right fit is stressful. Glasses.com set out to improve the experience by providing customers a way of virtually trying on glasses in the comfort of their homes via smart phone or tablet.\r\nThe Glasses.com app takes a picture of the user’s face, creates a 3D model of facial features, and virtually places selected frames on the user’s 3D image. The user is then free to view the 3D image from a wide variety of angles.\r\n\r\nThe application relies on computer vision to recognize the features of a user’s face and to then generate an accurate, high-fidelity, 3D model of that face, such that the eyewear fits properly.\r\n\r\nGlasses.com brought our engineers an excellent, fully-functioning version of the application, seeking acceleration. The app took full advantage of the CPU, which would necessitate Glasses.com’s purchasing and managing a large amount of hardware. Further, the application was running at a slow thirty seconds per session, resulting in a sub-optimal user experience. Our engineers were tasked with parallelizing the process and reducing the session runtime.\r\n\r\nGlasses.com has several machines they are using in production. Their main setup consists of a Haswell/Ivy Bridge CPU paired with a GTX 690. The machine is able to run 4 processes concurrently. In the past, Glasses.com used a 32 core Intel process with a couple of K20X. They are still using these machines, but they are not the main configuration. They are also making use of the Amazon EC2 instances in case they exceed internal capacity.\r\n\r\nWe began by analyzing the application and targeting the process which would have the greatest impact on performance.\r\n<blockquote>This app is amazing! It takes a 3D video of your face in just a few steps, and the rendering is spot on! RayBan Aviators are always a little big on me, and the app showed them on my face exactly as they fit in the store! -davidandrew</blockquote>\r\nThe most pressing concern was the rendering portion of code, which was running on the CPU. We implemented rendering in ArrayFire, which is specially optimized for CUDA GPUs. Further, we noticed that while the application was rendering-intensive, much of the rendering did not need to be precise—we could make estimations without compromising quality. This estimated rendering speedup operation was critical to the success of the project.\r\n\r\nWe discovered further opportunities for speedup within the architecture itself. The original app processed each frame in a separate thread; we accelerated this by combining each operation into a single kernel and running that kernel once. By moving the parallelization to the GPU we effectively took advantage of the strength of that architecture.\r\n\r\nUltimately, parallelizing the application and incorporating the ArrayFire software library accelerated the application to about ten seconds per session—roughly three times as fast as the original code, and twice as fast as the target speed. We further exceeded expectations by improving general stability when handling errors.\r\n\r\nThe improvements in speed, stability, and user experience have resonated with the application’s target market, resulting in a 3.5/5 star rating in the Apple Store.','Case Study: glasses.com App with 3Dfit Technology','','inherit','open','open','','6899-revision-v1','','','2014-10-15 21:29:02','2014-10-15 21:29:02','',6899,'http://sandbox.arrayfire.com/6899-revision-v1/',0,'revision','',0),(6901,7,'2014-10-15 21:30:45','2014-10-15 21:30:45','[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"]\r\n\r\nAdopting a new look is tough! Haircut, clothing style, glasses—whatever the change may be, finding the right fit is stressful. Glasses.com set out to improve the experience by providing customers a way of virtually trying on glasses in the comfort of their homes via smart phone or tablet.\r\nThe Glasses.com app takes a picture of the user’s face, creates a 3D model of facial features, and virtually places selected frames on the user’s 3D image. The user is then free to view the 3D image from a wide variety of angles.\r\n\r\nThe application relies on computer vision to recognize the features of a user’s face and to then generate an accurate, high-fidelity, 3D model of that face, such that the eyewear fits properly.\r\n\r\nGlasses.com brought our engineers an excellent, fully-functioning version of the application, seeking acceleration. The app took full advantage of the CPU, which would necessitate Glasses.com’s purchasing and managing a large amount of hardware. Further, the application was running at a slow thirty seconds per session, resulting in a sub-optimal user experience. Our engineers were tasked with parallelizing the process and reducing the session runtime.\r\n\r\nGlasses.com has several machines they are using in production. Their main setup consists of a Haswell/Ivy Bridge CPU paired with a GTX 690. The machine is able to run 4 processes concurrently. In the past, Glasses.com used a 32 core Intel process with a couple of K20X. They are still using these machines, but they are not the main configuration. They are also making use of the Amazon EC2 instances in case they exceed internal capacity.\r\n\r\nWe began by analyzing the application and targeting the process which would have the greatest impact on performance.\r\n<blockquote>This app is amazing! It takes a 3D video of your face in just a few steps, and the rendering is spot on! RayBan Aviators are always a little big on me, and the app showed them on my face exactly as they fit in the store! -davidandrew</blockquote>\r\nThe most pressing concern was the rendering portion of code, which was running on the CPU. We implemented rendering in ArrayFire, which is specially optimized for CUDA GPUs. Further, we noticed that while the application was rendering-intensive, much of the rendering did not need to be precise—we could make estimations without compromising quality. This estimated rendering speedup operation was critical to the success of the project.\r\n\r\nWe discovered further opportunities for speedup within the architecture itself. The original app processed each frame in a separate thread; we accelerated this by combining each operation into a single kernel and running that kernel once. By moving the parallelization to the GPU we effectively took advantage of the strength of that architecture.\r\n\r\nUltimately, parallelizing the application and incorporating the ArrayFire software library accelerated the application to about ten seconds per session—roughly three times as fast as the original code, and twice as fast as the target speed. We further exceeded expectations by improving general stability when handling errors.\r\n\r\nThe improvements in speed, stability, and user experience have resonated with the application’s target market, resulting in a 3.5/5 star rating in the Apple Store.\r\n\r\n[/content_band]','Case Study: glasses.com App with 3Dfit Technology','','inherit','open','open','','6899-revision-v1','','','2014-10-15 21:30:45','2014-10-15 21:30:45','',6899,'http://sandbox.arrayfire.com/6899-revision-v1/',0,'revision','',0),(6902,7,'2014-10-15 21:32:13','2014-10-15 21:32:13','[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"][custom_headline type=\"left\" level=\"h3\"]glasses.com App with 3Dfit Technology[/custom_headline]\r\n\r\nAdopting a new look is tough! Haircut, clothing style, glasses—whatever the change may be, finding the right fit is stressful. Glasses.com set out to improve the experience by providing customers a way of virtually trying on glasses in the comfort of their homes via smart phone or tablet.\r\nThe Glasses.com app takes a picture of the user’s face, creates a 3D model of facial features, and virtually places selected frames on the user’s 3D image. The user is then free to view the 3D image from a wide variety of angles.\r\n\r\nThe application relies on computer vision to recognize the features of a user’s face and to then generate an accurate, high-fidelity, 3D model of that face, such that the eyewear fits properly.\r\n\r\nGlasses.com brought our engineers an excellent, fully-functioning version of the application, seeking acceleration. The app took full advantage of the CPU, which would necessitate Glasses.com’s purchasing and managing a large amount of hardware. Further, the application was running at a slow thirty seconds per session, resulting in a sub-optimal user experience. Our engineers were tasked with parallelizing the process and reducing the session runtime.\r\n\r\nGlasses.com has several machines they are using in production. Their main setup consists of a Haswell/Ivy Bridge CPU paired with a GTX 690. The machine is able to run 4 processes concurrently. In the past, Glasses.com used a 32 core Intel process with a couple of K20X. They are still using these machines, but they are not the main configuration. They are also making use of the Amazon EC2 instances in case they exceed internal capacity.\r\n\r\nWe began by analyzing the application and targeting the process which would have the greatest impact on performance.\r\n<blockquote>This app is amazing! It takes a 3D video of your face in just a few steps, and the rendering is spot on! RayBan Aviators are always a little big on me, and the app showed them on my face exactly as they fit in the store! -davidandrew</blockquote>\r\nThe most pressing concern was the rendering portion of code, which was running on the CPU. We implemented rendering in ArrayFire, which is specially optimized for CUDA GPUs. Further, we noticed that while the application was rendering-intensive, much of the rendering did not need to be precise—we could make estimations without compromising quality. This estimated rendering speedup operation was critical to the success of the project.\r\n\r\nWe discovered further opportunities for speedup within the architecture itself. The original app processed each frame in a separate thread; we accelerated this by combining each operation into a single kernel and running that kernel once. By moving the parallelization to the GPU we effectively took advantage of the strength of that architecture.\r\n\r\nUltimately, parallelizing the application and incorporating the ArrayFire software library accelerated the application to about ten seconds per session—roughly three times as fast as the original code, and twice as fast as the target speed. We further exceeded expectations by improving general stability when handling errors.\r\n\r\nThe improvements in speed, stability, and user experience have resonated with the application’s target market, resulting in a 3.5/5 star rating in the Apple Store.\r\n\r\n[/content_band]','Case Study: glasses.com App with 3Dfit Technology','','inherit','open','open','','6899-revision-v1','','','2014-10-15 21:32:13','2014-10-15 21:32:13','',6899,'http://sandbox.arrayfire.com/6899-revision-v1/',0,'revision','',0),(6903,7,'2014-10-15 21:33:28','2014-10-15 21:33:28','','glassesdotcom benchmarks','','inherit','open','open','','glassesdotcom-benchmarks-2','','','2014-10-15 21:33:28','2014-10-15 21:33:28','',6899,'http://sandbox.arrayfire.com/wp-content/uploads/2014/10/glassesdotcom-benchmarks.jpg',0,'attachment','image/jpeg',0),(6904,7,'2014-10-15 21:34:32','2014-10-15 21:34:32','[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"][custom_headline type=\"left\" level=\"h3\"]glasses.com App with 3Dfit Technology[/custom_headline]\r\n\r\nAdopting a new look is tough! Haircut, clothing style, glasses—whatever the change may be, finding the right fit is stressful. Glasses.com set out to improve the experience by providing customers a way of virtually trying on glasses in the comfort of their homes via smart phone or tablet.\r\nThe Glasses.com app takes a picture of the user’s face, creates a 3D model of facial features, and virtually places selected frames on the user’s 3D image. The user is then free to view the 3D image from a wide variety of angles.\r\n\r\nThe application relies on computer vision to recognize the features of a user’s face and to then generate an accurate, high-fidelity, 3D model of that face, such that the eyewear fits properly.\r\n\r\nGlasses.com brought our engineers an excellent, fully-functioning version of the application, seeking acceleration. The app took full advantage of the CPU, which would necessitate Glasses.com’s purchasing and managing a large amount of hardware. Further, the application was running at a slow thirty seconds per session, resulting in a sub-optimal user experience. Our engineers were tasked with parallelizing the process and reducing the session runtime.\r\n\r\nGlasses.com has several machines they are using in production. Their main setup consists of a Haswell/Ivy Bridge CPU paired with a GTX 690. The machine is able to run 4 processes concurrently. In the past, Glasses.com used a 32 core Intel process with a couple of K20X. They are still using these machines, but they are not the main configuration. They are also making use of the Amazon EC2 instances in case they exceed internal capacity.\r\n\r\nWe began by analyzing the application and targeting the process which would have the greatest impact on performance.\r\n<blockquote>This app is amazing! It takes a 3D video of your face in just a few steps, and the rendering is spot on! RayBan Aviators are always a little big on me, and the app showed them on my face exactly as they fit in the store! -davidandrew</blockquote>\r\nThe most pressing concern was the rendering portion of code, which was running on the CPU. We implemented rendering in ArrayFire, which is specially optimized for CUDA GPUs. Further, we noticed that while the application was rendering-intensive, much of the rendering did not need to be precise—we could make estimations without compromising quality. This estimated rendering speedup operation was critical to the success of the project.\r\n\r\nWe discovered further opportunities for speedup within the architecture itself. The original app processed each frame in a separate thread; we accelerated this by combining each operation into a single kernel and running that kernel once. By moving the parallelization to the GPU we effectively took advantage of the strength of that architecture.\r\n\r\nUltimately, parallelizing the application and incorporating the ArrayFire software library accelerated the application to about ten seconds per session—roughly three times as fast as the original code, and twice as fast as the target speed. We further exceeded expectations by improving general stability when handling errors. <img class=\"aligncenter size-full wp-image-6903\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/glassesdotcom-benchmarks.jpg\" alt=\"glassesdotcom benchmarks\" width=\"700\" height=\"442\" />\r\n\r\nThe improvements in speed, stability, and user experience have resonated with the application’s target market, resulting in a 3.5/5 star rating in the Apple Store.\r\n\r\n[/content_band]','Case Study: glasses.com App with 3Dfit Technology','','inherit','open','open','','6899-revision-v1','','','2014-10-15 21:34:32','2014-10-15 21:34:32','',6899,'http://sandbox.arrayfire.com/6899-revision-v1/',0,'revision','',0),(6905,7,'2014-10-15 21:37:33','2014-10-15 21:37:33','[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"][custom_headline type=\"left\" level=\"h3\"]glasses.com App with 3Dfit Technology[/custom_headline]\r\n\r\n[column=\"one-fifth\"][/column][column=\"three-fifths\"]Adopting a new look is tough! Haircut, clothing style, glasses—whatever the change may be, finding the right fit is stressful. Glasses.com set out to improve the experience by providing customers a way of virtually trying on glasses in the comfort of their homes via smart phone or tablet.\r\nThe Glasses.com app takes a picture of the user’s face, creates a 3D model of facial features, and virtually places selected frames on the user’s 3D image. The user is then free to view the 3D image from a wide variety of angles.\r\n\r\nThe application relies on computer vision to recognize the features of a user’s face and to then generate an accurate, high-fidelity, 3D model of that face, such that the eyewear fits properly.\r\n\r\nGlasses.com brought our engineers an excellent, fully-functioning version of the application, seeking acceleration. The app took full advantage of the CPU, which would necessitate Glasses.com’s purchasing and managing a large amount of hardware. Further, the application was running at a slow thirty seconds per session, resulting in a sub-optimal user experience. Our engineers were tasked with parallelizing the process and reducing the session runtime.\r\n\r\nGlasses.com has several machines they are using in production. Their main setup consists of a Haswell/Ivy Bridge CPU paired with a GTX 690. The machine is able to run 4 processes concurrently. In the past, Glasses.com used a 32 core Intel process with a couple of K20X. They are still using these machines, but they are not the main configuration. They are also making use of the Amazon EC2 instances in case they exceed internal capacity.\r\n\r\nWe began by analyzing the application and targeting the process which would have the greatest impact on performance.\r\n<blockquote>This app is amazing! It takes a 3D video of your face in just a few steps, and the rendering is spot on! RayBan Aviators are always a little big on me, and the app showed them on my face exactly as they fit in the store! -davidandrew</blockquote>\r\nThe most pressing concern was the rendering portion of code, which was running on the CPU. We implemented rendering in ArrayFire, which is specially optimized for CUDA GPUs. Further, we noticed that while the application was rendering-intensive, much of the rendering did not need to be precise—we could make estimations without compromising quality. This estimated rendering speedup operation was critical to the success of the project.\r\n\r\nWe discovered further opportunities for speedup within the architecture itself. The original app processed each frame in a separate thread; we accelerated this by combining each operation into a single kernel and running that kernel once. By moving the parallelization to the GPU we effectively took advantage of the strength of that architecture.\r\n\r\nUltimately, parallelizing the application and incorporating the ArrayFire software library accelerated the application to about ten seconds per session—roughly three times as fast as the original code, and twice as fast as the target speed. We further exceeded expectations by improving general stability when handling errors. <img class=\"aligncenter size-full wp-image-6903\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/glassesdotcom-benchmarks.jpg\" alt=\"glassesdotcom benchmarks\" width=\"700\" height=\"442\" />\r\n\r\nThe improvements in speed, stability, and user experience have resonated with the application’s target market, resulting in a 3.5/5 star rating in the Apple Store.[/column][column=\"one-fifth\" last=\"true\"][/column][/content_band]','Case Study: glasses.com App with 3Dfit Technology','','inherit','open','open','','6899-revision-v1','','','2014-10-15 21:37:33','2014-10-15 21:37:33','',6899,'http://sandbox.arrayfire.com/6899-revision-v1/',0,'revision','',0),(6906,7,'2014-10-15 21:37:59','2014-10-15 21:37:59','[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"][custom_headline type=\"left\" level=\"h3\"]glasses.com App with 3Dfit Technology[/custom_headline]\r\n\r\n[column=\"one-fifth\"].[/column][column=\"three-fifths\"]Adopting a new look is tough! Haircut, clothing style, glasses—whatever the change may be, finding the right fit is stressful. Glasses.com set out to improve the experience by providing customers a way of virtually trying on glasses in the comfort of their homes via smart phone or tablet.\r\nThe Glasses.com app takes a picture of the user’s face, creates a 3D model of facial features, and virtually places selected frames on the user’s 3D image. The user is then free to view the 3D image from a wide variety of angles.\r\n\r\nThe application relies on computer vision to recognize the features of a user’s face and to then generate an accurate, high-fidelity, 3D model of that face, such that the eyewear fits properly.\r\n\r\nGlasses.com brought our engineers an excellent, fully-functioning version of the application, seeking acceleration. The app took full advantage of the CPU, which would necessitate Glasses.com’s purchasing and managing a large amount of hardware. Further, the application was running at a slow thirty seconds per session, resulting in a sub-optimal user experience. Our engineers were tasked with parallelizing the process and reducing the session runtime.\r\n\r\nGlasses.com has several machines they are using in production. Their main setup consists of a Haswell/Ivy Bridge CPU paired with a GTX 690. The machine is able to run 4 processes concurrently. In the past, Glasses.com used a 32 core Intel process with a couple of K20X. They are still using these machines, but they are not the main configuration. They are also making use of the Amazon EC2 instances in case they exceed internal capacity.\r\n\r\nWe began by analyzing the application and targeting the process which would have the greatest impact on performance.\r\n<blockquote>This app is amazing! It takes a 3D video of your face in just a few steps, and the rendering is spot on! RayBan Aviators are always a little big on me, and the app showed them on my face exactly as they fit in the store! -davidandrew</blockquote>\r\nThe most pressing concern was the rendering portion of code, which was running on the CPU. We implemented rendering in ArrayFire, which is specially optimized for CUDA GPUs. Further, we noticed that while the application was rendering-intensive, much of the rendering did not need to be precise—we could make estimations without compromising quality. This estimated rendering speedup operation was critical to the success of the project.\r\n\r\nWe discovered further opportunities for speedup within the architecture itself. The original app processed each frame in a separate thread; we accelerated this by combining each operation into a single kernel and running that kernel once. By moving the parallelization to the GPU we effectively took advantage of the strength of that architecture.\r\n\r\nUltimately, parallelizing the application and incorporating the ArrayFire software library accelerated the application to about ten seconds per session—roughly three times as fast as the original code, and twice as fast as the target speed. We further exceeded expectations by improving general stability when handling errors. <img class=\"aligncenter size-full wp-image-6903\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/glassesdotcom-benchmarks.jpg\" alt=\"glassesdotcom benchmarks\" width=\"700\" height=\"442\" />\r\n\r\nThe improvements in speed, stability, and user experience have resonated with the application’s target market, resulting in a 3.5/5 star rating in the Apple Store.[/column][column=\"one-fifth\" last=\"true\"].[/column][/content_band]','Case Study: glasses.com App with 3Dfit Technology','','inherit','open','open','','6899-revision-v1','','','2014-10-15 21:37:59','2014-10-15 21:37:59','',6899,'http://sandbox.arrayfire.com/6899-revision-v1/',0,'revision','',0),(6907,7,'2014-10-15 21:39:47','2014-10-15 21:39:47','[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"][custom_headline type=\"left\" level=\"h3\"]glasses.com App with 3Dfit Technology[/custom_headline]\r\n\r\n[column type=\"one-fifth\"].[/column][column type=\"three-fifths\"]Adopting a new look is tough! Haircut, clothing style, glasses—whatever the change may be, finding the right fit is stressful. Glasses.com set out to improve the experience by providing customers a way of virtually trying on glasses in the comfort of their homes via smart phone or tablet.\r\nThe Glasses.com app takes a picture of the user’s face, creates a 3D model of facial features, and virtually places selected frames on the user’s 3D image. The user is then free to view the 3D image from a wide variety of angles.\r\n\r\nThe application relies on computer vision to recognize the features of a user’s face and to then generate an accurate, high-fidelity, 3D model of that face, such that the eyewear fits properly.\r\n\r\nGlasses.com brought our engineers an excellent, fully-functioning version of the application, seeking acceleration. The app took full advantage of the CPU, which would necessitate Glasses.com’s purchasing and managing a large amount of hardware. Further, the application was running at a slow thirty seconds per session, resulting in a sub-optimal user experience. Our engineers were tasked with parallelizing the process and reducing the session runtime.\r\n\r\nGlasses.com has several machines they are using in production. Their main setup consists of a Haswell/Ivy Bridge CPU paired with a GTX 690. The machine is able to run 4 processes concurrently. In the past, Glasses.com used a 32 core Intel process with a couple of K20X. They are still using these machines, but they are not the main configuration. They are also making use of the Amazon EC2 instances in case they exceed internal capacity.\r\n\r\nWe began by analyzing the application and targeting the process which would have the greatest impact on performance.\r\n<blockquote>This app is amazing! It takes a 3D video of your face in just a few steps, and the rendering is spot on! RayBan Aviators are always a little big on me, and the app showed them on my face exactly as they fit in the store! -davidandrew</blockquote>\r\nThe most pressing concern was the rendering portion of code, which was running on the CPU. We implemented rendering in ArrayFire, which is specially optimized for CUDA GPUs. Further, we noticed that while the application was rendering-intensive, much of the rendering did not need to be precise—we could make estimations without compromising quality. This estimated rendering speedup operation was critical to the success of the project.\r\n\r\nWe discovered further opportunities for speedup within the architecture itself. The original app processed each frame in a separate thread; we accelerated this by combining each operation into a single kernel and running that kernel once. By moving the parallelization to the GPU we effectively took advantage of the strength of that architecture.\r\n\r\nUltimately, parallelizing the application and incorporating the ArrayFire software library accelerated the application to about ten seconds per session—roughly three times as fast as the original code, and twice as fast as the target speed. We further exceeded expectations by improving general stability when handling errors. <img class=\"aligncenter size-full wp-image-6903\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/glassesdotcom-benchmarks.jpg\" alt=\"glassesdotcom benchmarks\" width=\"700\" height=\"442\" />\r\n\r\nThe improvements in speed, stability, and user experience have resonated with the application’s target market, resulting in a 3.5/5 star rating in the Apple Store.[/column][column type=\"one-fifth\" last=\"true\"].[/column][/content_band]','Case Study: glasses.com App with 3Dfit Technology','','inherit','open','open','','6899-revision-v1','','','2014-10-15 21:39:47','2014-10-15 21:39:47','',6899,'http://sandbox.arrayfire.com/6899-revision-v1/',0,'revision','',0),(6908,7,'2014-10-15 21:40:22','2014-10-15 21:40:22','[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"][custom_headline type=\"left\" level=\"h3\"]glasses.com App with 3Dfit Technology[/custom_headline][gap size=\"5%\"]\r\n\r\n[column type=\"one-fifth\"].[/column][column type=\"three-fifths\"]Adopting a new look is tough! Haircut, clothing style, glasses—whatever the change may be, finding the right fit is stressful. Glasses.com set out to improve the experience by providing customers a way of virtually trying on glasses in the comfort of their homes via smart phone or tablet.\r\nThe Glasses.com app takes a picture of the user’s face, creates a 3D model of facial features, and virtually places selected frames on the user’s 3D image. The user is then free to view the 3D image from a wide variety of angles.\r\n\r\nThe application relies on computer vision to recognize the features of a user’s face and to then generate an accurate, high-fidelity, 3D model of that face, such that the eyewear fits properly.\r\n\r\nGlasses.com brought our engineers an excellent, fully-functioning version of the application, seeking acceleration. The app took full advantage of the CPU, which would necessitate Glasses.com’s purchasing and managing a large amount of hardware. Further, the application was running at a slow thirty seconds per session, resulting in a sub-optimal user experience. Our engineers were tasked with parallelizing the process and reducing the session runtime.\r\n\r\nGlasses.com has several machines they are using in production. Their main setup consists of a Haswell/Ivy Bridge CPU paired with a GTX 690. The machine is able to run 4 processes concurrently. In the past, Glasses.com used a 32 core Intel process with a couple of K20X. They are still using these machines, but they are not the main configuration. They are also making use of the Amazon EC2 instances in case they exceed internal capacity.\r\n\r\nWe began by analyzing the application and targeting the process which would have the greatest impact on performance.\r\n<blockquote>This app is amazing! It takes a 3D video of your face in just a few steps, and the rendering is spot on! RayBan Aviators are always a little big on me, and the app showed them on my face exactly as they fit in the store! -davidandrew</blockquote>\r\nThe most pressing concern was the rendering portion of code, which was running on the CPU. We implemented rendering in ArrayFire, which is specially optimized for CUDA GPUs. Further, we noticed that while the application was rendering-intensive, much of the rendering did not need to be precise—we could make estimations without compromising quality. This estimated rendering speedup operation was critical to the success of the project.\r\n\r\nWe discovered further opportunities for speedup within the architecture itself. The original app processed each frame in a separate thread; we accelerated this by combining each operation into a single kernel and running that kernel once. By moving the parallelization to the GPU we effectively took advantage of the strength of that architecture.\r\n\r\nUltimately, parallelizing the application and incorporating the ArrayFire software library accelerated the application to about ten seconds per session—roughly three times as fast as the original code, and twice as fast as the target speed. We further exceeded expectations by improving general stability when handling errors. <img class=\"aligncenter size-full wp-image-6903\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/glassesdotcom-benchmarks.jpg\" alt=\"glassesdotcom benchmarks\" width=\"700\" height=\"442\" />\r\n\r\nThe improvements in speed, stability, and user experience have resonated with the application’s target market, resulting in a 3.5/5 star rating in the Apple Store.[/column][column type=\"one-fifth\" last=\"true\"].[/column][/content_band]','Case Study: glasses.com App with 3Dfit Technology','','inherit','open','open','','6899-revision-v1','','','2014-10-15 21:40:22','2014-10-15 21:40:22','',6899,'http://sandbox.arrayfire.com/6899-revision-v1/',0,'revision','',0),(6909,7,'2014-10-15 21:42:51','2014-10-15 21:42:51','[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"][custom_headline type=\"left\" level=\"h3\"]glasses.com App with 3Dfit Technology[/custom_headline][gap size=\"5%\"]\r\n\r\n[column type=\"one-fifth\"].[/column][column type=\"three-fifths\"]Adopting a new look is tough! Haircut, clothing style, glasses—whatever the change may be, finding the right fit is stressful. Glasses.com set out to improve the experience by providing customers a way of virtually trying on glasses in the comfort of their homes via smart phone or tablet.\r\nThe Glasses.com app takes a picture of the user’s face, creates a 3D model of facial features, and virtually places selected frames on the user’s 3D image. The user is then free to view the 3D image from a wide variety of angles.\r\n\r\nThe application relies on computer vision to recognize the features of a user’s face and to then generate an accurate, high-fidelity, 3D model of that face, such that the eyewear fits properly.\r\n\r\nGlasses.com brought our engineers an excellent, fully-functioning version of the application, seeking acceleration. The app took full advantage of the CPU, which would necessitate Glasses.com’s purchasing and managing a large amount of hardware. Further, the application was running at a slow thirty seconds per session, resulting in a sub-optimal user experience. Our engineers were tasked with parallelizing the process and reducing the session runtime.\r\n\r\nGlasses.com has several machines they are using in production. Their main setup consists of a Haswell/Ivy Bridge CPU paired with a GTX 690. The machine is able to run 4 processes concurrently. In the past, Glasses.com used a 32 core Intel process with a couple of K20X. They are still using these machines, but they are not the main configuration. They are also making use of the Amazon EC2 instances in case they exceed internal capacity.\r\n\r\nWe began by analyzing the application and targeting the process which would have the greatest impact on performance.\r\n<blockquote>This app is amazing! It takes a 3D video of your face in just a few steps, and the rendering is spot on! RayBan Aviators are always a little big on me, and the app showed them on my face exactly as they fit in the store! —davidandrew</blockquote>\r\nThe most pressing concern was the rendering portion of code, which was running on the CPU. We implemented rendering in ArrayFire, which is specially optimized for CUDA GPUs. Further, we noticed that while the application was rendering-intensive, much of the rendering did not need to be precise—we could make estimations without compromising quality. This estimated rendering speedup operation was critical to the success of the project.\r\n\r\nWe discovered further opportunities for speedup within the architecture itself. The original app processed each frame in a separate thread; we accelerated this by combining each operation into a single kernel and running that kernel once. By moving the parallelization to the GPU we effectively took advantage of the strength of that architecture.\r\n\r\nUltimately, parallelizing the application and incorporating the ArrayFire software library accelerated the application to about ten seconds per session—roughly three times as fast as the original code, and twice as fast as the target speed. We further exceeded expectations by improving general stability when handling errors. <img class=\"aligncenter size-full wp-image-6903\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/glassesdotcom-benchmarks.jpg\" alt=\"glassesdotcom benchmarks\" width=\"700\" height=\"442\" />\r\n\r\nThe improvements in speed, stability, and user experience have resonated with the application’s target market, resulting in a 3.5/5 star rating in the Apple Store.[/column][column type=\"one-fifth\" last=\"true\"].[/column][/content_band]','Case Study: glasses.com App with 3Dfit Technology','','inherit','open','open','','6899-revision-v1','','','2014-10-15 21:42:51','2014-10-15 21:42:51','',6899,'http://sandbox.arrayfire.com/6899-revision-v1/',0,'revision','',0),(6910,7,'2014-10-15 21:43:45','2014-10-15 21:43:45','[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"][custom_headline type=\"left\" level=\"h3\"]glasses.com App with 3Dfit Technology[/custom_headline][gap size=\"5%\"]\r\n\r\n[column type=\"one-fifth\"].[/column][column type=\"three-fifths\"]<img class=\"alignleft size-full wp-image-6894\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/glasses-logo-promo-01.png\" alt=\"glasses logo, promo-01\" width=\"360\" height=\"120\" />Adopting a new look is tough! Haircut, clothing style, glasses—whatever the change may be, finding the right fit is stressful. Glasses.com set out to improve the experience by providing customers a way of virtually trying on glasses in the comfort of their homes via smart phone or tablet.\r\nThe Glasses.com app takes a picture of the user’s face, creates a 3D model of facial features, and virtually places selected frames on the user’s 3D image. The user is then free to view the 3D image from a wide variety of angles.\r\n\r\nThe application relies on computer vision to recognize the features of a user’s face and to then generate an accurate, high-fidelity, 3D model of that face, such that the eyewear fits properly.\r\n\r\nGlasses.com brought our engineers an excellent, fully-functioning version of the application, seeking acceleration. The app took full advantage of the CPU, which would necessitate Glasses.com’s purchasing and managing a large amount of hardware. Further, the application was running at a slow thirty seconds per session, resulting in a sub-optimal user experience. Our engineers were tasked with parallelizing the process and reducing the session runtime.\r\n\r\nGlasses.com has several machines they are using in production. Their main setup consists of a Haswell/Ivy Bridge CPU paired with a GTX 690. The machine is able to run 4 processes concurrently. In the past, Glasses.com used a 32 core Intel process with a couple of K20X. They are still using these machines, but they are not the main configuration. They are also making use of the Amazon EC2 instances in case they exceed internal capacity.\r\n\r\nWe began by analyzing the application and targeting the process which would have the greatest impact on performance.\r\n<blockquote>This app is amazing! It takes a 3D video of your face in just a few steps, and the rendering is spot on! RayBan Aviators are always a little big on me, and the app showed them on my face exactly as they fit in the store! —davidandrew</blockquote>\r\nThe most pressing concern was the rendering portion of code, which was running on the CPU. We implemented rendering in ArrayFire, which is specially optimized for CUDA GPUs. Further, we noticed that while the application was rendering-intensive, much of the rendering did not need to be precise—we could make estimations without compromising quality. This estimated rendering speedup operation was critical to the success of the project.\r\n\r\nWe discovered further opportunities for speedup within the architecture itself. The original app processed each frame in a separate thread; we accelerated this by combining each operation into a single kernel and running that kernel once. By moving the parallelization to the GPU we effectively took advantage of the strength of that architecture.\r\n\r\nUltimately, parallelizing the application and incorporating the ArrayFire software library accelerated the application to about ten seconds per session—roughly three times as fast as the original code, and twice as fast as the target speed. We further exceeded expectations by improving general stability when handling errors. <img class=\"aligncenter size-full wp-image-6903\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/glassesdotcom-benchmarks.jpg\" alt=\"glassesdotcom benchmarks\" width=\"700\" height=\"442\" />\r\n\r\nThe improvements in speed, stability, and user experience have resonated with the application’s target market, resulting in a 3.5/5 star rating in the Apple Store.[/column][column type=\"one-fifth\" last=\"true\"].[/column][/content_band]','Case Study: glasses.com App with 3Dfit Technology','','inherit','open','open','','6899-revision-v1','','','2014-10-15 21:43:45','2014-10-15 21:43:45','',6899,'http://sandbox.arrayfire.com/6899-revision-v1/',0,'revision','',0),(6911,7,'2014-10-15 21:44:49','2014-10-15 21:44:49','[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"][custom_headline type=\"left\" level=\"h3\"]glasses.com App with 3Dfit Technology[/custom_headline][gap size=\"5%\"]\r\n\r\n[column type=\"one-fifth\"].[/column][column type=\"three-fifths\"]<img class=\"alignleft size-full wp-image-6894\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/glasses-logo-promo-01.png\" alt=\"glasses logo, promo-01\" width=\"360\" height=\"120\" />Adopting a new look is tough! Haircut, clothing style, glasses—whatever the change may be, finding the right fit is stressful. Glasses.com set out to improve the experience by providing customers a way of virtually trying on glasses in the comfort of their homes via smart phone or tablet.\r\nThe Glasses.com app takes a picture of the user’s face, creates a 3D model of facial features, and virtually places selected frames on the user’s 3D image. The user is then free to view the 3D image from a wide variety of angles.\r\n\r\nThe application relies on computer vision to recognize the features of a user’s face and to then generate an accurate, high-fidelity, 3D model of that face, such that the eyewear fits properly.\r\n<blockquote>This app is amazing! It takes a 3D video of your face in just a few steps, and the rendering is spot on! RayBan Aviators are always a little big on me, and the app showed them on my face exactly as they fit in the store! —davidandrew</blockquote>\r\nGlasses.com brought our engineers an excellent, fully-functioning version of the application, seeking acceleration. The app took full advantage of the CPU, which would necessitate Glasses.com’s purchasing and managing a large amount of hardware. Further, the application was running at a slow thirty seconds per session, resulting in a sub-optimal user experience. Our engineers were tasked with parallelizing the process and reducing the session runtime.\r\n\r\nGlasses.com has several machines they are using in production. Their main setup consists of a Haswell/Ivy Bridge CPU paired with a GTX 690. The machine is able to run 4 processes concurrently. In the past, Glasses.com used a 32 core Intel process with a couple of K20X. They are still using these machines, but they are not the main configuration. They are also making use of the Amazon EC2 instances in case they exceed internal capacity.\r\n\r\nWe began by analyzing the application and targeting the process which would have the greatest impact on performance.\r\n\r\nThe most pressing concern was the rendering portion of code, which was running on the CPU. We implemented rendering in ArrayFire, which is specially optimized for CUDA GPUs. Further, we noticed that while the application was rendering-intensive, much of the rendering did not need to be precise—we could make estimations without compromising quality. This estimated rendering speedup operation was critical to the success of the project.\r\n\r\nWe discovered further opportunities for speedup within the architecture itself. The original app processed each frame in a separate thread; we accelerated this by combining each operation into a single kernel and running that kernel once. By moving the parallelization to the GPU we effectively took advantage of the strength of that architecture.\r\n\r\nUltimately, parallelizing the application and incorporating the ArrayFire software library accelerated the application to about ten seconds per session—roughly three times as fast as the original code, and twice as fast as the target speed. We further exceeded expectations by improving general stability when handling errors. <img class=\"aligncenter size-full wp-image-6903\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/glassesdotcom-benchmarks.jpg\" alt=\"glassesdotcom benchmarks\" width=\"700\" height=\"442\" />\r\n\r\nThe improvements in speed, stability, and user experience have resonated with the application’s target market, resulting in a 3.5/5 star rating in the Apple Store.[/column][column type=\"one-fifth\" last=\"true\"].[/column][/content_band]','Case Study: glasses.com App with 3Dfit Technology','','inherit','open','open','','6899-revision-v1','','','2014-10-15 21:44:49','2014-10-15 21:44:49','',6899,'http://sandbox.arrayfire.com/6899-revision-v1/',0,'revision','',0),(6912,7,'2014-10-16 20:05:29','2014-10-16 20:05:29','[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"][column type=\"one-fifth\"].[/column][column type=\"three-fifths\"][custom_headline type=\"left\" level=\"h3\"]glasses.com App with 3Dfit Technology[/custom_headline][/column][column type=\"one-fifth\"].[/column][gap size=\"5%\"]\r\n\r\n[column type=\"one-fifth\"].[/column][column type=\"three-fifths\"]<img class=\"alignleft size-full wp-image-6894\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/glasses-logo-promo-01.png\" alt=\"glasses logo, promo-01\" width=\"360\" height=\"120\" />Adopting a new look is tough! Haircut, clothing style, glasses—whatever the change may be, finding the right fit is stressful. Glasses.com set out to improve the experience by providing customers a way of virtually trying on glasses in the comfort of their homes via smart phone or tablet.\r\nThe Glasses.com app takes a picture of the user’s face, creates a 3D model of facial features, and virtually places selected frames on the user’s 3D image. The user is then free to view the 3D image from a wide variety of angles.\r\n\r\nThe application relies on computer vision to recognize the features of a user’s face and to then generate an accurate, high-fidelity, 3D model of that face, such that the eyewear fits properly.\r\n<blockquote>This app is amazing! It takes a 3D video of your face in just a few steps, and the rendering is spot on! RayBan Aviators are always a little big on me, and the app showed them on my face exactly as they fit in the store! —davidandrew</blockquote>\r\nGlasses.com brought our engineers an excellent, fully-functioning version of the application, seeking acceleration. The app took full advantage of the CPU, which would necessitate Glasses.com’s purchasing and managing a large amount of hardware. Further, the application was running at a slow thirty seconds per session, resulting in a sub-optimal user experience. Our engineers were tasked with parallelizing the process and reducing the session runtime.\r\n\r\nGlasses.com has several machines they are using in production. Their main setup consists of a Haswell/Ivy Bridge CPU paired with a GTX 690. The machine is able to run 4 processes concurrently. In the past, Glasses.com used a 32 core Intel process with a couple of K20X. They are still using these machines, but they are not the main configuration. They are also making use of the Amazon EC2 instances in case they exceed internal capacity.\r\n\r\nWe began by analyzing the application and targeting the process which would have the greatest impact on performance.\r\n\r\nThe most pressing concern was the rendering portion of code, which was running on the CPU. We implemented rendering in ArrayFire, which is specially optimized for CUDA GPUs. Further, we noticed that while the application was rendering-intensive, much of the rendering did not need to be precise—we could make estimations without compromising quality. This estimated rendering speedup operation was critical to the success of the project.\r\n\r\nWe discovered further opportunities for speedup within the architecture itself. The original app processed each frame in a separate thread; we accelerated this by combining each operation into a single kernel and running that kernel once. By moving the parallelization to the GPU we effectively took advantage of the strength of that architecture.\r\n\r\nUltimately, parallelizing the application and incorporating the ArrayFire software library accelerated the application to about ten seconds per session—roughly three times as fast as the original code, and twice as fast as the target speed. We further exceeded expectations by improving general stability when handling errors. <img class=\"aligncenter size-full wp-image-6903\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/glassesdotcom-benchmarks.jpg\" alt=\"glassesdotcom benchmarks\" width=\"700\" height=\"442\" />\r\n\r\nThe improvements in speed, stability, and user experience have resonated with the application’s target market, resulting in a 3.5/5 star rating in the Apple Store.[/column][column type=\"one-fifth\" last=\"true\"].[/column][/content_band]','Case Study: glasses.com App with 3Dfit Technology','','inherit','open','open','','6899-revision-v1','','','2014-10-16 20:05:29','2014-10-16 20:05:29','',6899,'http://sandbox.arrayfire.com/6899-revision-v1/',0,'revision','',0),(6913,7,'2014-10-16 20:05:48','2014-10-16 20:05:48','[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"][column type=\"one-fifth\"].[/column][column type=\"three-fifths\"][custom_headline type=\"left\" level=\"h3\"]glasses.com App with 3Dfit Technology[/custom_headline][/column][column type=\"one-fifth\" last=\"true\"].[/column][gap size=\"5%\"]\r\n\r\n[column type=\"one-fifth\"].[/column][column type=\"three-fifths\"]<img class=\"alignleft size-full wp-image-6894\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/glasses-logo-promo-01.png\" alt=\"glasses logo, promo-01\" width=\"360\" height=\"120\" />Adopting a new look is tough! Haircut, clothing style, glasses—whatever the change may be, finding the right fit is stressful. Glasses.com set out to improve the experience by providing customers a way of virtually trying on glasses in the comfort of their homes via smart phone or tablet.\r\nThe Glasses.com app takes a picture of the user’s face, creates a 3D model of facial features, and virtually places selected frames on the user’s 3D image. The user is then free to view the 3D image from a wide variety of angles.\r\n\r\nThe application relies on computer vision to recognize the features of a user’s face and to then generate an accurate, high-fidelity, 3D model of that face, such that the eyewear fits properly.\r\n<blockquote>This app is amazing! It takes a 3D video of your face in just a few steps, and the rendering is spot on! RayBan Aviators are always a little big on me, and the app showed them on my face exactly as they fit in the store! —davidandrew</blockquote>\r\nGlasses.com brought our engineers an excellent, fully-functioning version of the application, seeking acceleration. The app took full advantage of the CPU, which would necessitate Glasses.com’s purchasing and managing a large amount of hardware. Further, the application was running at a slow thirty seconds per session, resulting in a sub-optimal user experience. Our engineers were tasked with parallelizing the process and reducing the session runtime.\r\n\r\nGlasses.com has several machines they are using in production. Their main setup consists of a Haswell/Ivy Bridge CPU paired with a GTX 690. The machine is able to run 4 processes concurrently. In the past, Glasses.com used a 32 core Intel process with a couple of K20X. They are still using these machines, but they are not the main configuration. They are also making use of the Amazon EC2 instances in case they exceed internal capacity.\r\n\r\nWe began by analyzing the application and targeting the process which would have the greatest impact on performance.\r\n\r\nThe most pressing concern was the rendering portion of code, which was running on the CPU. We implemented rendering in ArrayFire, which is specially optimized for CUDA GPUs. Further, we noticed that while the application was rendering-intensive, much of the rendering did not need to be precise—we could make estimations without compromising quality. This estimated rendering speedup operation was critical to the success of the project.\r\n\r\nWe discovered further opportunities for speedup within the architecture itself. The original app processed each frame in a separate thread; we accelerated this by combining each operation into a single kernel and running that kernel once. By moving the parallelization to the GPU we effectively took advantage of the strength of that architecture.\r\n\r\nUltimately, parallelizing the application and incorporating the ArrayFire software library accelerated the application to about ten seconds per session—roughly three times as fast as the original code, and twice as fast as the target speed. We further exceeded expectations by improving general stability when handling errors. <img class=\"aligncenter size-full wp-image-6903\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/glassesdotcom-benchmarks.jpg\" alt=\"glassesdotcom benchmarks\" width=\"700\" height=\"442\" />\r\n\r\nThe improvements in speed, stability, and user experience have resonated with the application’s target market, resulting in a 3.5/5 star rating in the Apple Store.[/column][column type=\"one-fifth\" last=\"true\"].[/column][/content_band]','Case Study: glasses.com App with 3Dfit Technology','','inherit','open','open','','6899-revision-v1','','','2014-10-16 20:05:48','2014-10-16 20:05:48','',6899,'http://sandbox.arrayfire.com/6899-revision-v1/',0,'revision','',0),(6914,7,'2014-10-16 20:06:31','2014-10-16 20:06:31','[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"][column type=\"one-fifth\"].[/column][column type=\"three-fifths\"][custom_headline type=\"left\" level=\"h3\"]glasses.com App with 3Dfit Technology[/custom_headline][/column][column type=\"one-fifth\" last=\"true\"].[/column][gap size=\"5%\"]\r\n\r\n[column type=\"one-fifth\"].[/column][column type=\"three-fifths\"]<img class=\"alignright wp-image-6894 size-full\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/glasses-logo-promo-01.png\" alt=\"glasses logo, promo-01\" width=\"360\" height=\"120\" />Adopting a new look is tough! Haircut, clothing style, glasses—whatever the change may be, finding the right fit is stressful. Glasses.com set out to improve the experience by providing customers a way of virtually trying on glasses in the comfort of their homes via smart phone or tablet.\r\nThe Glasses.com app takes a picture of the user’s face, creates a 3D model of facial features, and virtually places selected frames on the user’s 3D image. The user is then free to view the 3D image from a wide variety of angles.\r\n\r\nThe application relies on computer vision to recognize the features of a user’s face and to then generate an accurate, high-fidelity, 3D model of that face, such that the eyewear fits properly.\r\n<blockquote>This app is amazing! It takes a 3D video of your face in just a few steps, and the rendering is spot on! RayBan Aviators are always a little big on me, and the app showed them on my face exactly as they fit in the store! —davidandrew</blockquote>\r\nGlasses.com brought our engineers an excellent, fully-functioning version of the application, seeking acceleration. The app took full advantage of the CPU, which would necessitate Glasses.com’s purchasing and managing a large amount of hardware. Further, the application was running at a slow thirty seconds per session, resulting in a sub-optimal user experience. Our engineers were tasked with parallelizing the process and reducing the session runtime.\r\n\r\nGlasses.com has several machines they are using in production. Their main setup consists of a Haswell/Ivy Bridge CPU paired with a GTX 690. The machine is able to run 4 processes concurrently. In the past, Glasses.com used a 32 core Intel process with a couple of K20X. They are still using these machines, but they are not the main configuration. They are also making use of the Amazon EC2 instances in case they exceed internal capacity.\r\n\r\nWe began by analyzing the application and targeting the process which would have the greatest impact on performance.\r\n\r\nThe most pressing concern was the rendering portion of code, which was running on the CPU. We implemented rendering in ArrayFire, which is specially optimized for CUDA GPUs. Further, we noticed that while the application was rendering-intensive, much of the rendering did not need to be precise—we could make estimations without compromising quality. This estimated rendering speedup operation was critical to the success of the project.\r\n\r\nWe discovered further opportunities for speedup within the architecture itself. The original app processed each frame in a separate thread; we accelerated this by combining each operation into a single kernel and running that kernel once. By moving the parallelization to the GPU we effectively took advantage of the strength of that architecture.\r\n\r\nUltimately, parallelizing the application and incorporating the ArrayFire software library accelerated the application to about ten seconds per session—roughly three times as fast as the original code, and twice as fast as the target speed. We further exceeded expectations by improving general stability when handling errors. <img class=\"aligncenter size-full wp-image-6903\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/glassesdotcom-benchmarks.jpg\" alt=\"glassesdotcom benchmarks\" width=\"700\" height=\"442\" />\r\n\r\nThe improvements in speed, stability, and user experience have resonated with the application’s target market, resulting in a 3.5/5 star rating in the Apple Store.[/column][column type=\"one-fifth\" last=\"true\"].[/column][/content_band]','Case Study: glasses.com App with 3Dfit Technology','','inherit','open','open','','6899-revision-v1','','','2014-10-16 20:06:31','2014-10-16 20:06:31','',6899,'http://sandbox.arrayfire.com/6899-revision-v1/',0,'revision','',0),(6915,7,'2014-10-16 20:09:07','2014-10-16 20:09:07','[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"][column type=\"one-fifth\"].[/column][column type=\"three-fifths\"][custom_headline type=\"left\" level=\"h3\"]glasses.com App with 3Dfit Technology[/custom_headline][/column][column type=\"one-fifth\" last=\"true\"].[/column][gap size=\"5%\"]\r\n\r\n[column type=\"one-fifth\"].[/column][column type=\"three-fifths\"]<img class=\"alignright wp-image-6894 size-full\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/glasses-logo-promo-01.png\" alt=\"glasses logo, promo-01\" width=\"360\" height=\"120\" />Adopting a new look is tough! Haircut, clothing style, glasses—whatever the change may be, finding the right fit is stressful. Glasses.com set out to improve the experience by providing customers a way of virtually trying on glasses in the comfort of their homes via smart phone or tablet.\r\n\r\nThe Glasses.com app takes a picture of the user’s face, creates a 3D model of facial features, and virtually places selected frames on the user’s 3D image. The user is then free to view the 3D image from a wide variety of angles.\r\n\r\nThe application relies on computer vision to recognize the features of a user’s face and to then generate an accurate, high-fidelity, 3D model of that face, such that the eyewear fits properly.\r\n<blockquote>This app is amazing! It takes a 3D video of your face in just a few steps, and the rendering is spot on! RayBan Aviators are always a little big on me, and the app showed them on my face exactly as they fit in the store! —davidandrew</blockquote>\r\n<h3>Mission</h3>\r\nGlasses.com brought our engineers an excellent, fully-functioning version of the application, seeking acceleration. The app took full advantage of the CPU, which would necessitate Glasses.com’s purchasing and managing a large amount of hardware. Further, the application was running at a slow thirty seconds per session, resulting in a sub-optimal user experience. Our engineers were tasked with parallelizing the process and reducing the session runtime.\r\n\r\nGlasses.com has several machines they are using in production. Their main setup consists of a Haswell/Ivy Bridge CPU paired with a GTX 690. The machine is able to run 4 processes concurrently. In the past, Glasses.com used a 32 core Intel process with a couple of K20X. They are still using these machines, but they are not the main configuration. They are also making use of the Amazon EC2 instances in case they exceed internal capacity.\r\n<h3>Action</h3>\r\nWe began by analyzing the application and targeting the process which would have the greatest impact on performance.\r\n\r\nThe most pressing concern was the rendering portion of code, which was running on the CPU. We implemented rendering in ArrayFire, which is specially optimized for CUDA GPUs. Further, we noticed that while the application was rendering-intensive, much of the rendering did not need to be precise—we could make estimations without compromising quality. This estimated rendering speedup operation was critical to the success of the project.\r\n\r\nWe discovered further opportunities for speedup within the architecture itself. The original app processed each frame in a separate thread; we accelerated this by combining each operation into a single kernel and running that kernel once. By moving the parallelization to the GPU we effectively took advantage of the strength of that architecture.\r\n<h3>Results</h3>\r\nUltimately, parallelizing the application and incorporating the ArrayFire software library accelerated the application to about ten seconds per session—roughly three times as fast as the original code, and twice as fast as the target speed. We further exceeded expectations by improving general stability when handling errors. <img class=\"aligncenter size-full wp-image-6903\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/glassesdotcom-benchmarks.jpg\" alt=\"glassesdotcom benchmarks\" width=\"700\" height=\"442\" />\r\n\r\nThe improvements in speed, stability, and user experience have resonated with the application’s target market, resulting in a 3.5/5 star rating in the Apple Store.[/column][column type=\"one-fifth\" last=\"true\"].[/column][/content_band]','Case Study: glasses.com App with 3Dfit Technology','','inherit','open','open','','6899-revision-v1','','','2014-10-16 20:09:07','2014-10-16 20:09:07','',6899,'http://sandbox.arrayfire.com/6899-revision-v1/',0,'revision','',0),(6916,7,'2014-10-16 20:09:35','2014-10-16 20:09:35','[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"][column type=\"one-fifth\"].[/column][column type=\"three-fifths\"][custom_headline type=\"left\" level=\"h3\"]Glasses.com App with 3Dfit Technology[/custom_headline][/column][column type=\"one-fifth\" last=\"true\"].[/column][gap size=\"5%\"]\r\n\r\n[column type=\"one-fifth\"].[/column][column type=\"three-fifths\"]<img class=\"alignright wp-image-6894 size-full\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/glasses-logo-promo-01.png\" alt=\"glasses logo, promo-01\" width=\"360\" height=\"120\" />Adopting a new look is tough! Haircut, clothing style, glasses—whatever the change may be, finding the right fit is stressful. Glasses.com set out to improve the experience by providing customers a way of virtually trying on glasses in the comfort of their homes via smart phone or tablet.\r\n\r\nThe Glasses.com app takes a picture of the user’s face, creates a 3D model of facial features, and virtually places selected frames on the user’s 3D image. The user is then free to view the 3D image from a wide variety of angles.\r\n\r\nThe application relies on computer vision to recognize the features of a user’s face and to then generate an accurate, high-fidelity, 3D model of that face, such that the eyewear fits properly.\r\n<blockquote>This app is amazing! It takes a 3D video of your face in just a few steps, and the rendering is spot on! RayBan Aviators are always a little big on me, and the app showed them on my face exactly as they fit in the store! —davidandrew</blockquote>\r\n<h3>Mission</h3>\r\nGlasses.com brought our engineers an excellent, fully-functioning version of the application, seeking acceleration. The app took full advantage of the CPU, which would necessitate Glasses.com’s purchasing and managing a large amount of hardware. Further, the application was running at a slow thirty seconds per session, resulting in a sub-optimal user experience. Our engineers were tasked with parallelizing the process and reducing the session runtime.\r\n\r\nGlasses.com has several machines they are using in production. Their main setup consists of a Haswell/Ivy Bridge CPU paired with a GTX 690. The machine is able to run 4 processes concurrently. In the past, Glasses.com used a 32 core Intel process with a couple of K20X. They are still using these machines, but they are not the main configuration. They are also making use of the Amazon EC2 instances in case they exceed internal capacity.\r\n<h3>Action</h3>\r\nWe began by analyzing the application and targeting the process which would have the greatest impact on performance.\r\n\r\nThe most pressing concern was the rendering portion of code, which was running on the CPU. We implemented rendering in ArrayFire, which is specially optimized for CUDA GPUs. Further, we noticed that while the application was rendering-intensive, much of the rendering did not need to be precise—we could make estimations without compromising quality. This estimated rendering speedup operation was critical to the success of the project.\r\n\r\nWe discovered further opportunities for speedup within the architecture itself. The original app processed each frame in a separate thread; we accelerated this by combining each operation into a single kernel and running that kernel once. By moving the parallelization to the GPU we effectively took advantage of the strength of that architecture.\r\n<h3>Results</h3>\r\nUltimately, parallelizing the application and incorporating the ArrayFire software library accelerated the application to about ten seconds per session—roughly three times as fast as the original code, and twice as fast as the target speed. We further exceeded expectations by improving general stability when handling errors. <img class=\"aligncenter size-full wp-image-6903\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/glassesdotcom-benchmarks.jpg\" alt=\"glassesdotcom benchmarks\" width=\"700\" height=\"442\" />\r\n\r\nThe improvements in speed, stability, and user experience have resonated with the application’s target market, resulting in a 3.5/5 star rating in the Apple Store.[/column][column type=\"one-fifth\" last=\"true\"].[/column][/content_band]','Case Study: glasses.com App with 3Dfit Technology','','inherit','open','open','','6899-revision-v1','','','2014-10-16 20:09:35','2014-10-16 20:09:35','',6899,'http://sandbox.arrayfire.com/6899-revision-v1/',0,'revision','',0),(6917,7,'2014-10-16 20:10:17','2014-10-16 20:10:17','[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"][column type=\"one-fifth\"].[/column][column type=\"three-fifths\"][custom_headline type=\"left\" level=\"h3\"]Glasses.com App with 3Dfit Technology[/custom_headline][/column][column type=\"one-fifth\" last=\"true\"].[/column][gap size=\"5%\"]\r\n\r\n[column type=\"one-fifth\"].[/column][column type=\"three-fifths\"]<img class=\"alignright wp-image-6894 size-full\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/glasses-logo-promo-01.png\" alt=\"glasses logo, promo-01\" width=\"360\" height=\"120\" />Adopting a new look is tough! Haircut, clothing style, glasses—whatever the change may be, finding the right fit is stressful. Glasses.com set out to improve the experience by providing customers a way of virtually trying on glasses in the comfort of their homes via smart phone or tablet.\r\n\r\nThe Glasses.com app takes a picture of the user’s face, creates a 3D model of facial features, and virtually places selected frames on the user’s 3D image. The user is then free to view the 3D image from a wide variety of angles.\r\n\r\nThe application relies on computer vision to recognize the features of a user’s face and to then generate an accurate, high-fidelity, 3D model of that face, such that the eyewear fits properly.\r\n<blockquote>This app is amazing! It takes a 3D video of your face in just a few steps, and the rendering is spot on! RayBan Aviators are always a little big on me, and the app showed them on my face exactly as they fit in the store! —davidandrew</blockquote>\r\n<h4>Mission</h4>\r\nGlasses.com brought our engineers an excellent, fully-functioning version of the application, seeking acceleration. The app took full advantage of the CPU, which would necessitate Glasses.com’s purchasing and managing a large amount of hardware. Further, the application was running at a slow thirty seconds per session, resulting in a sub-optimal user experience. Our engineers were tasked with parallelizing the process and reducing the session runtime.\r\n\r\nGlasses.com has several machines they are using in production. Their main setup consists of a Haswell/Ivy Bridge CPU paired with a GTX 690. The machine is able to run 4 processes concurrently. In the past, Glasses.com used a 32 core Intel process with a couple of K20X. They are still using these machines, but they are not the main configuration. They are also making use of the Amazon EC2 instances in case they exceed internal capacity.\r\n<h4>Action</h4>\r\nWe began by analyzing the application and targeting the process which would have the greatest impact on performance.\r\n\r\nThe most pressing concern was the rendering portion of code, which was running on the CPU. We implemented rendering in ArrayFire, which is specially optimized for CUDA GPUs. Further, we noticed that while the application was rendering-intensive, much of the rendering did not need to be precise—we could make estimations without compromising quality. This estimated rendering speedup operation was critical to the success of the project.\r\n\r\nWe discovered further opportunities for speedup within the architecture itself. The original app processed each frame in a separate thread; we accelerated this by combining each operation into a single kernel and running that kernel once. By moving the parallelization to the GPU we effectively took advantage of the strength of that architecture.\r\n<h4>Results</h4>\r\nUltimately, parallelizing the application and incorporating the ArrayFire software library accelerated the application to about ten seconds per session—roughly three times as fast as the original code, and twice as fast as the target speed. We further exceeded expectations by improving general stability when handling errors. <img class=\"aligncenter size-full wp-image-6903\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/glassesdotcom-benchmarks.jpg\" alt=\"glassesdotcom benchmarks\" width=\"700\" height=\"442\" />\r\n\r\nThe improvements in speed, stability, and user experience have resonated with the application’s target market, resulting in a 3.5/5 star rating in the Apple Store.[/column][column type=\"one-fifth\" last=\"true\"].[/column][/content_band]','Case Study: glasses.com App with 3Dfit Technology','','inherit','open','open','','6899-revision-v1','','','2014-10-16 20:10:17','2014-10-16 20:10:17','',6899,'http://sandbox.arrayfire.com/6899-revision-v1/',0,'revision','',0),(6918,7,'2014-10-16 20:15:59','2014-10-16 20:15:59','[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"][column type=\"one-fifth\"].[/column][column type=\"three-fifths\"][custom_headline type=\"left\" level=\"h3\"]Glasses.com App with 3Dfit Technology[/custom_headline][/column][column type=\"one-fifth\" last=\"true\"].[/column][gap size=\"5%\"]\r\n\r\n[column type=\"one-fifth\"].[/column][column type=\"three-fifths\"]<img class=\"alignright wp-image-6894 size-full\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/glasses-logo-promo-01.png\" alt=\"glasses logo, promo-01\" width=\"360\" height=\"120\" />Adopting a new look is tough! Haircut, clothing style, glasses—whatever the change may be, finding the right fit is stressful. Glasses.com set out to improve the experience by providing customers a way of virtually trying on glasses in the comfort of their homes via smart phone or tablet.\r\n\r\nThe Glasses.com app takes a picture of the user’s face, creates a 3D model of facial features, and virtually places selected frames on the user’s 3D image. The user is then free to view the 3D image from a wide variety of angles.\r\n\r\nThe application relies on computer vision to recognize the features of a user’s face and to then generate an accurate, high-fidelity, 3D model of that face, such that the eyewear fits properly.\r\n<blockquote>This app is amazing! It takes a 3D video of your face in just a few steps, and the rendering is spot on! RayBan Aviators are always a little big on me, and the app showed them on my face exactly as they fit in the store! —davidandrew</blockquote>\r\n<h4>Mission</h4>\r\nGlasses.com brought our engineers an excellent, fully-functioning version of the application, seeking acceleration. The app took full advantage of the CPU, which would necessitate Glasses.com’s purchasing and managing a large amount of hardware. Further, the application was running at a slow thirty seconds per session, resulting in a sub-optimal user experience. Our engineers were tasked with parallelizing the process and reducing the session runtime.\r\n\r\nGlasses.com has several machines they are using in production. Their main setup consists of a Haswell/Ivy Bridge CPU paired with a GTX 690. The machine is able to run 4 processes concurrently. In the past, Glasses.com used a 32 core Intel process with a couple of K20X. They are still using these machines, but they are not the main configuration. They are also making use of the Amazon EC2 instances in case they exceed internal capacity.\r\n<h4>Action</h4>\r\nWe began by analyzing the application and targeting the process which would have the greatest impact on performance.\r\n\r\nThe most pressing concern was the rendering portion of code, which was running on the CPU. We implemented rendering in ArrayFire, which is specially optimized for CUDA GPUs. Further, we noticed that while the application was rendering-intensive, much of the rendering did not need to be precise—we could make estimations without compromising quality. This estimated rendering speedup operation was critical to the success of the project.\r\n\r\nWe discovered further opportunities for speedup within the architecture itself. The original app processed each frame in a separate thread; we accelerated this by combining each operation into a single kernel and running that kernel once. By moving the parallelization to the GPU we effectively took advantage of the strength of that architecture.\r\n<h4>Results</h4>\r\nUltimately, parallelizing the application and incorporating the ArrayFire software library accelerated the application to about ten seconds per session—roughly three times as fast as the original code, and twice as fast as the target speed. We further exceeded expectations by improving general stability when handling errors. <img class=\"aligncenter size-full wp-image-6903\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/glassesdotcom-benchmarks.jpg\" alt=\"glassesdotcom benchmarks\" width=\"700\" height=\"442\" />\r\n\r\nThe improvements in speed, stability, and user experience have resonated with the application’s target market, resulting in a 3.5/5 star rating in the Apple Store.\r\n<h4>About glasses.com</h4>\r\nGlasses.com launched in 2011 as an online retailer of prescription and non-prescription glasses and sunglasses, carrying a wide-range of styles and name brands such as Oakley, Ray-Ban, Kate Spade, Burberry, Maui Jim, and more.[/column][column type=\"one-fifth\" last=\"true\"].[/column][/content_band]','Case Study: glasses.com App with 3Dfit Technology','','inherit','open','open','','6899-revision-v1','','','2014-10-16 20:15:59','2014-10-16 20:15:59','',6899,'http://sandbox.arrayfire.com/6899-revision-v1/',0,'revision','',0),(6919,7,'2014-10-16 20:16:31','2014-10-16 20:16:31','[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"][column type=\"one-fifth\"].[/column][column type=\"three-fifths\"][custom_headline type=\"left\" level=\"h3\"]Glasses.com App with 3Dfit Technology[/custom_headline][/column][column type=\"one-fifth\" last=\"true\"].[/column][gap size=\"3%\"]\r\n\r\n[column type=\"one-fifth\"].[/column][column type=\"three-fifths\"]<img class=\"alignright wp-image-6894 size-full\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/glasses-logo-promo-01.png\" alt=\"glasses logo, promo-01\" width=\"360\" height=\"120\" />Adopting a new look is tough! Haircut, clothing style, glasses—whatever the change may be, finding the right fit is stressful. Glasses.com set out to improve the experience by providing customers a way of virtually trying on glasses in the comfort of their homes via smart phone or tablet.\r\n\r\nThe Glasses.com app takes a picture of the user’s face, creates a 3D model of facial features, and virtually places selected frames on the user’s 3D image. The user is then free to view the 3D image from a wide variety of angles.\r\n\r\nThe application relies on computer vision to recognize the features of a user’s face and to then generate an accurate, high-fidelity, 3D model of that face, such that the eyewear fits properly.\r\n<blockquote>This app is amazing! It takes a 3D video of your face in just a few steps, and the rendering is spot on! RayBan Aviators are always a little big on me, and the app showed them on my face exactly as they fit in the store! —davidandrew</blockquote>\r\n<h4>Mission</h4>\r\nGlasses.com brought our engineers an excellent, fully-functioning version of the application, seeking acceleration. The app took full advantage of the CPU, which would necessitate Glasses.com’s purchasing and managing a large amount of hardware. Further, the application was running at a slow thirty seconds per session, resulting in a sub-optimal user experience. Our engineers were tasked with parallelizing the process and reducing the session runtime.\r\n\r\nGlasses.com has several machines they are using in production. Their main setup consists of a Haswell/Ivy Bridge CPU paired with a GTX 690. The machine is able to run 4 processes concurrently. In the past, Glasses.com used a 32 core Intel process with a couple of K20X. They are still using these machines, but they are not the main configuration. They are also making use of the Amazon EC2 instances in case they exceed internal capacity.\r\n<h4>Action</h4>\r\nWe began by analyzing the application and targeting the process which would have the greatest impact on performance.\r\n\r\nThe most pressing concern was the rendering portion of code, which was running on the CPU. We implemented rendering in ArrayFire, which is specially optimized for CUDA GPUs. Further, we noticed that while the application was rendering-intensive, much of the rendering did not need to be precise—we could make estimations without compromising quality. This estimated rendering speedup operation was critical to the success of the project.\r\n\r\nWe discovered further opportunities for speedup within the architecture itself. The original app processed each frame in a separate thread; we accelerated this by combining each operation into a single kernel and running that kernel once. By moving the parallelization to the GPU we effectively took advantage of the strength of that architecture.\r\n<h4>Results</h4>\r\nUltimately, parallelizing the application and incorporating the ArrayFire software library accelerated the application to about ten seconds per session—roughly three times as fast as the original code, and twice as fast as the target speed. We further exceeded expectations by improving general stability when handling errors. <img class=\"aligncenter size-full wp-image-6903\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/glassesdotcom-benchmarks.jpg\" alt=\"glassesdotcom benchmarks\" width=\"700\" height=\"442\" />\r\n\r\nThe improvements in speed, stability, and user experience have resonated with the application’s target market, resulting in a 3.5/5 star rating in the Apple Store.\r\n<h4>About glasses.com</h4>\r\nGlasses.com launched in 2011 as an online retailer of prescription and non-prescription glasses and sunglasses, carrying a wide-range of styles and name brands such as Oakley, Ray-Ban, Kate Spade, Burberry, Maui Jim, and more.[/column][column type=\"one-fifth\" last=\"true\"].[/column][/content_band]','Case Study: glasses.com App with 3Dfit Technology','','inherit','open','open','','6899-revision-v1','','','2014-10-16 20:16:31','2014-10-16 20:16:31','',6899,'http://sandbox.arrayfire.com/6899-revision-v1/',0,'revision','',0),(6920,7,'2014-10-16 20:23:57','2014-10-16 20:23:57','[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"][column type=\"one-fifth\"].[/column][column type=\"three-fifths\"][custom_headline type=\"left\" level=\"h3\"]Glasses.com App with 3Dfit Technology[/custom_headline][/column][column type=\"one-fifth\" last=\"true\"].[/column][gap size=\"3%\"]\r\n\r\n[column type=\"one-fifth\"].[/column][column type=\"three-fifths\"]<img class=\"alignright wp-image-6894 size-full\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/glasses-logo-promo-01.png\" alt=\"glasses logo, promo-01\" width=\"360\" height=\"120\" />Many find the experience of purchasing new glasses stressful. Even simply trying on frames can be uncomfortable when done in public. Glasses.com set out to improve the experience by providing customers a way of virtually trying on glasses in the comfort of their homes via smart phone or tablet.\r\n\r\nThe Glasses.com app takes a picture of the user’s face, creates a 3D model of facial features, and virtually places selected frames on the user’s 3D image. The user is then free to view the 3D image from a wide variety of angles.\r\n\r\nThe application relies on computer vision to recognize the features of a user’s face and to then generate an accurate, high-fidelity, 3D model of that face, such that the eyewear fits properly.\r\n<blockquote>This app is amazing! It takes a 3D video of your face in just a few steps, and the rendering is spot on! RayBan Aviators are always a little big on me, and the app showed them on my face exactly as they fit in the store! —davidandrew</blockquote>\r\n<h4>Mission</h4>\r\nGlasses.com brought our engineers an excellent, fully-functioning version of the application, seeking acceleration. The app took full advantage of the CPU, which would necessitate Glasses.com’s purchasing and managing a large amount of hardware. Further, the application was running at a slow thirty seconds per session, resulting in a sub-optimal user experience. Our engineers were tasked with parallelizing the process and reducing the session runtime.\r\n\r\nGlasses.com has several machines they are using in production. Their main setup consists of a Haswell/Ivy Bridge CPU paired with a GTX 690. The machine is able to run 4 processes concurrently. In the past, Glasses.com used a 32 core Intel process with a couple of K20X. They are still using these machines, but they are not the main configuration. They are also making use of the Amazon EC2 instances in case they exceed internal capacity.\r\n<h4>Action</h4>\r\nWe began by analyzing the application and targeting the process which would have the greatest impact on performance.\r\n\r\nThe most pressing concern was the rendering portion of code, which was running on the CPU. We implemented rendering in ArrayFire, which is specially optimized for CUDA GPUs. Further, we noticed that while the application was rendering-intensive, much of the rendering did not need to be precise—we could make estimations without compromising quality. This estimated rendering speedup operation was critical to the success of the project.\r\n\r\nWe discovered further opportunities for speedup within the architecture itself. The original app processed each frame in a separate thread; we accelerated this by combining each operation into a single kernel and running that kernel once. By moving the parallelization to the GPU we effectively took advantage of the strength of that architecture.\r\n<h4>Results</h4>\r\nUltimately, parallelizing the application and incorporating the ArrayFire software library accelerated the application to about ten seconds per session—roughly three times as fast as the original code, and twice as fast as the target speed. We further exceeded expectations by improving general stability when handling errors. <img class=\"aligncenter size-full wp-image-6903\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/glassesdotcom-benchmarks.jpg\" alt=\"glassesdotcom benchmarks\" width=\"700\" height=\"442\" />\r\n\r\nThe improvements in speed, stability, and user experience have resonated with the application’s target market, resulting in a 3.5/5 star rating in the Apple Store.\r\n<h4>About glasses.com</h4>\r\nGlasses.com launched in 2011 as an online retailer of prescription and non-prescription glasses and sunglasses, carrying a wide-range of styles and name brands such as Oakley, Ray-Ban, Kate Spade, Burberry, Maui Jim, and more.[/column][column type=\"one-fifth\" last=\"true\"].[/column][/content_band]','Case Study: glasses.com App with 3Dfit Technology','','inherit','open','open','','6899-revision-v1','','','2014-10-16 20:23:57','2014-10-16 20:23:57','',6899,'http://sandbox.arrayfire.com/6899-revision-v1/',0,'revision','',0),(6921,7,'2014-10-16 20:28:10','2014-10-16 20:28:10','[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"][column type=\"one-fifth\"].[/column][column type=\"three-fifths\"][custom_headline type=\"left\" level=\"h3\"]Glasses.com App with 3Dfit Technology[/custom_headline][/column][column type=\"one-fifth\" last=\"true\"].[/column][gap size=\"3%\"]\r\n\r\n[column type=\"one-fifth\"].[/column][column type=\"three-fifths\"]<img class=\"alignright wp-image-6894 size-full\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/glasses-logo-promo-01.png\" alt=\"glasses logo, promo-01\" width=\"360\" height=\"120\" />Many find the experience of purchasing new glasses stressful. Even simply trying on frames can be uncomfortable when done in a public setting. Glasses.com set out to improve the experience by providing customers a way of virtually trying on glasses frames in the comfort of their homes via smart phone or tablet.\r\n\r\nThe Glasses.com app takes a picture of the user’s face, creates a 3D model of facial features, and virtually places selected frames on the user’s 3D image. The user is then free to view the 3D image from a variety of angles.\r\n\r\nThe application relies on computer vision to recognize the features of a user’s face and then generate an accurate, high-fidelity, 3D model of that face. Virtual frames can then be positioned on the 3D model.\r\n<blockquote>This app is amazing! It takes a 3D video of your face in just a few steps, and the rendering is spot on! RayBan Aviators are always a little big on me, and the app showed them on my face exactly as they fit in the store! —davidandrew</blockquote>\r\n<h4>Mission</h4>\r\nGlasses.com brought our engineers an excellent, fully-functioning version of the application, seeking acceleration. The app took full advantage of the CPU, which would necessitate Glasses.com’s purchasing and managing a large amount of hardware. Further, the application was running at a slow thirty seconds per session, resulting in a sub-optimal user experience. Our engineers were tasked with parallelizing the process and reducing the session runtime.\r\n\r\nGlasses.com has several machines they are using in production. Their main setup consists of a Haswell/Ivy Bridge CPU paired with a GTX 690. The machine is able to run 4 processes concurrently. In the past, Glasses.com used a 32 core Intel process with a couple of K20X. They are still using these machines, but they are not the main configuration. They are also making use of the Amazon EC2 instances in case they exceed internal capacity.\r\n<h4>Action</h4>\r\nWe began by analyzing the application and targeting the process which would have the greatest impact on performance.\r\n\r\nThe most pressing concern was the rendering portion of code, which was running on the CPU. We implemented rendering in ArrayFire, which is specially optimized for CUDA GPUs. Further, we noticed that while the application was rendering-intensive, much of the rendering did not need to be precise—we could make estimations without compromising quality. This estimated rendering speedup operation was critical to the success of the project.\r\n\r\nWe discovered further opportunities for speedup within the architecture itself. The original app processed each frame in a separate thread; we accelerated this by combining each operation into a single kernel and running that kernel once. By moving the parallelization to the GPU we effectively took advantage of the strength of that architecture.\r\n<h4>Results</h4>\r\nUltimately, parallelizing the application and incorporating the ArrayFire software library accelerated the application to about ten seconds per session—roughly three times as fast as the original code, and twice as fast as the target speed. We further exceeded expectations by improving general stability when handling errors. <img class=\"aligncenter size-full wp-image-6903\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/glassesdotcom-benchmarks.jpg\" alt=\"glassesdotcom benchmarks\" width=\"700\" height=\"442\" />\r\n\r\nThe improvements in speed, stability, and user experience have resonated with the application’s target market, resulting in a 3.5/5 star rating in the Apple Store.\r\n<h4>About glasses.com</h4>\r\nGlasses.com launched in 2011 as an online retailer of prescription and non-prescription glasses and sunglasses, carrying a wide-range of styles and name brands such as Oakley, Ray-Ban, Kate Spade, Burberry, Maui Jim, and more.[/column][column type=\"one-fifth\" last=\"true\"].[/column][/content_band]','Case Study: glasses.com App with 3Dfit Technology','','inherit','open','open','','6899-revision-v1','','','2014-10-16 20:28:10','2014-10-16 20:28:10','',6899,'http://sandbox.arrayfire.com/6899-revision-v1/',0,'revision','',0),(6922,7,'2014-10-16 20:32:12','2014-10-16 20:32:12','[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"][column type=\"one-fifth\"].[/column][column type=\"three-fifths\"][custom_headline type=\"left\" level=\"h3\"]Glasses.com App with 3Dfit Technology[/custom_headline][/column][column type=\"one-fifth\" last=\"true\"].[/column][gap size=\"3%\"]\r\n\r\n[column type=\"one-fifth\"].[/column][column type=\"three-fifths\"]<img class=\"alignright wp-image-6894 size-full\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/glasses-logo-promo-01.png\" alt=\"glasses logo, promo-01\" width=\"360\" height=\"120\" />Many find the experience of purchasing new glasses stressful. Even simply trying on frames can be uncomfortable when done in a public setting. Glasses.com set out to improve the experience by providing customers a way of virtually trying on glasses frames in the comfort of their homes via smart phone or tablet.\r\n\r\nThe Glasses.com app takes a picture of the user’s face, creates a 3D model of facial features, and virtually places selected frames on the user’s 3D image. The user is then free to view the 3D image from a variety of angles.\r\n\r\nThe application relies on computer vision to recognize the features of a user’s face and then generate an accurate, high-fidelity, 3D model of that face. Virtual frames can then be positioned on the 3D model.\r\n<blockquote>This app is amazing! It takes a 3D video of your face in just a few steps, and the rendering is spot on! RayBan Aviators are always a little big on me, and the app showed them on my face exactly as they fit in the store! —davidandrew</blockquote>\r\n<h4>Mission</h4>\r\nGlasses.com brought our engineers an excellent, fully-functioning version of the application; they simply wanted it accelerated. The app took full advantage of the CPU, which—left unchanged—would necessitate Glasses.com’s purchasing and managing a large amount of hardware. Further, the application was running at a slow thirty seconds per session, resulting in a sub-optimal user experience. Our engineers were tasked with parallelizing the process and reducing the session runtime.\r\n\r\nGlasses.com has several machines they are using in production. Their main setup consists of a Haswell/Ivy Bridge CPU paired with a GTX 690. The machine is able to run 4 processes concurrently. In the past, Glasses.com used a 32 core Intel process with a couple of K20X. They are still using these machines, but they are not the main configuration. They are also making use of the Amazon EC2 instances in case they exceed internal capacity.\r\n<h4>Action</h4>\r\nWe began by analyzing the application and targeting the process which would have the greatest impact on performance.\r\n\r\nThe most pressing concern was the rendering portion of code, which was running on the CPU. We implemented rendering in ArrayFire, which is specially optimized for CUDA GPUs. Further, we noticed that while the application was rendering-intensive, much of the rendering did not need to be precise—we could make estimations without compromising quality. This estimated rendering speedup operation was critical to the success of the project.\r\n\r\nWe discovered further opportunities for speedup within the architecture itself. The original app processed each frame in a separate thread; we accelerated this by combining each operation into a single kernel and running that kernel once. By moving the parallelization to the GPU we effectively took advantage of the strength of that architecture.\r\n<h4>Results</h4>\r\nUltimately, parallelizing the application and incorporating the ArrayFire software library accelerated the application to about ten seconds per session—roughly three times as fast as the original code, and twice as fast as the target speed. We further exceeded expectations by improving general stability when handling errors. <img class=\"aligncenter size-full wp-image-6903\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/glassesdotcom-benchmarks.jpg\" alt=\"glassesdotcom benchmarks\" width=\"700\" height=\"442\" />\r\n\r\nThe improvements in speed, stability, and user experience have resonated with the application’s target market, resulting in a 3.5/5 star rating in the Apple Store.\r\n<h4>About glasses.com</h4>\r\nGlasses.com launched in 2011 as an online retailer of prescription and non-prescription glasses and sunglasses, carrying a wide-range of styles and name brands such as Oakley, Ray-Ban, Kate Spade, Burberry, Maui Jim, and more.[/column][column type=\"one-fifth\" last=\"true\"].[/column][/content_band]','Case Study: glasses.com App with 3Dfit Technology','','inherit','open','open','','6899-revision-v1','','','2014-10-16 20:32:12','2014-10-16 20:32:12','',6899,'http://sandbox.arrayfire.com/6899-revision-v1/',0,'revision','',0),(6923,7,'2015-04-08 20:28:30','2015-04-08 20:28:30','[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"][column type=\"one-fifth\"].[/column][column type=\"three-fifths\"][custom_headline type=\"left\" level=\"h3\"]Reveal Imaging, an SAIC Company[/custom_headline][/column][column type=\"one-fifth\" last=\"true\"].[/column][gap size=\"3%\"]\r\n\r\n[column type=\"one-fifth\"].[/column][column type=\"three-fifths\"]<img class=\"alignright size-full wp-image-6896\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/reveal-logo-promo-01.png\" alt=\"reveal logo, promo-01\" width=\"360\" height=\"120\" />Iterative reconstruction from projections is typically a function reserved for the medical industry. This approach allows medical professionals to reconstruct computed tomography (CT) images from acquired data; the reconstructed images display virtual “slices” of a scanned object, such as a heart or brain.\r\n\r\nIn reconstructing an image from the data, algorithms are used to approximate the solution. The closer the solution, the better the reconstruction. Multiple iterations produce more accurate solutions, and, thus, higher quality images. This technique, however, consumes computation time; speeding the process through the GPU is one solution to this problem.\r\n<h4>Mission</h4>\r\nReveal Imaging, an SAIC company, uses this CT technology in developing threat-detection products for transportation systems—such as cargo and baggage screening in airports—and in securing facilities and public events. Speed is critical in their technology, and they approached us seeking acceleration.\r\n\r\nReveal Imaging was running its cone-beam reconstructor via CUDA code previously implemented by a different contractor, but the performance was sub-par. They tasked ArrayFire with improving that performance on their GTX690s.\r\n<h4>Action</h4>\r\nOur engineers approached this by following standard CUDA practices. The project consisted mostly of cleaning up the existing code to allow it to run faster.\r\n\r\nFirst we removed the dependence of the permitted slice thickness on the number of GPU cards in the system. Then we allowed for one of the GPUs in the system to be excluded from those used by the reconstruction code and dedicated to driving a display. Next, we assured that the code met specifications with the latest CUDA compilers. Finally, we implemented performance enhancements, bottleneck reductions, and bug fixes previously identified.\r\n<h4>Results</h4>\r\nThe conclusion of the project was followed by a three week testing period, during which time Reveal Imaging determined reliability and overall satisfaction of the code’s performance. The code’s original speed was measured at around 200 slices per second—through our optimization efforts we were able to increase that number to 320 slices per second. Upon completion of the testing period, Reveal Imaging was very pleased with the results and end performance of their code.\r\n<blockquote>The changes that you made sped up the system more than expected! How often does anybody get to say that things work better than expected?  —Reveal Imaging</blockquote>\r\nIn the end, Reveal Imaging was able to focus on other development aspects of their project while we focused on what we do best—working relentlessly to achieve the best possible results in parallel computing.\r\n<h6>About SAIC</h6>\r\nScience Applications International Corporation (SAIC) is a leading technology integrator that provides full lifecycle services and solutions in the technical, engineering, and enterprise IT markets. SAIC has approximately 13,000 employees worldwide.[/column][column type=\"one-fifth\" last=\"true\"].[/column][/content_band]','Case Study: Reveal Imaging','','publish','open','open','','case-study-reveal-imaging','','','2015-04-08 20:28:30','2015-04-08 20:28:30','',0,'http://sandbox.arrayfire.com/?page_id=6923',0,'page','',0),(6924,7,'2014-10-16 20:44:36','2014-10-16 20:44:36','[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"][column type=\"one-fifth\"].[/column][column type=\"three-fifths\"][custom_headline type=\"left\" level=\"h3\"]Glasses.com App with 3Dfit Technology[/custom_headline][/column][column type=\"one-fifth\" last=\"true\"].[/column][gap size=\"3%\"]\r\n\r\n[column type=\"one-fifth\"].[/column][column type=\"three-fifths\"]<img class=\"alignright size-full wp-image-6896\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/reveal-logo-promo-01.png\" alt=\"reveal logo, promo-01\" width=\"360\" height=\"120\" />Many find the experience of purchasing new glasses stressful. Even simply trying on frames can be uncomfortable when done in a public setting. Glasses.com set out to improve the experience by providing customers a way of virtually trying on glasses frames in the comfort of their homes via smart phone or tablet.\r\n\r\nThe Glasses.com app takes a picture of the user’s face, creates a 3D model of facial features, and virtually places selected frames on the user’s 3D image. The user is then free to view the 3D image from a variety of angles.\r\n\r\nThe application relies on computer vision to recognize the features of a user’s face and then generate an accurate, high-fidelity, 3D model of that face. Virtual frames can then be positioned on the 3D model.\r\n<blockquote>This app is amazing! It takes a 3D video of your face in just a few steps, and the rendering is spot on! RayBan Aviators are always a little big on me, and the app showed them on my face exactly as they fit in the store! —davidandrew</blockquote>\r\n<h4>Mission</h4>\r\nGlasses.com brought our engineers an excellent, fully-functioning version of the application; they simply wanted it accelerated. The app took full advantage of the CPU, which—left unchanged—would necessitate Glasses.com’s purchasing and managing a large amount of hardware. Further, the application was running at a slow thirty seconds per session, resulting in a sub-optimal user experience. Our engineers were tasked with parallelizing the process and reducing the session runtime.\r\n\r\nGlasses.com has several machines they are using in production. Their main setup consists of a Haswell/Ivy Bridge CPU paired with a GTX 690. The machine is able to run 4 processes concurrently. In the past, Glasses.com used a 32 core Intel process with a couple of K20X. They are still using these machines, but they are not the main configuration. They are also making use of the Amazon EC2 instances in case they exceed internal capacity.\r\n<h4>Action</h4>\r\nWe began by analyzing the application and targeting the process which would have the greatest impact on performance.\r\n\r\nThe most pressing concern was the rendering portion of code, which was running on the CPU. We implemented rendering in ArrayFire, which is specially optimized for CUDA GPUs. Further, we noticed that while the application was rendering-intensive, much of the rendering did not need to be precise—we could make estimations without compromising quality. This estimated rendering speedup operation was critical to the success of the project.\r\n\r\nWe discovered further opportunities for speedup within the architecture itself. The original app processed each frame in a separate thread; we accelerated this by combining each operation into a single kernel and running that kernel once. By moving the parallelization to the GPU we effectively took advantage of the strength of that architecture.\r\n<h4>Results</h4>\r\nUltimately, parallelizing the application and incorporating the ArrayFire software library accelerated the application to about ten seconds per session—roughly three times as fast as the original code, and twice as fast as the target speed. We further exceeded expectations by improving general stability when handling errors. <img class=\"aligncenter size-full wp-image-6903\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/glassesdotcom-benchmarks.jpg\" alt=\"glassesdotcom benchmarks\" width=\"700\" height=\"442\" />\r\n\r\nThe improvements in speed, stability, and user experience have resonated with the application’s target market, resulting in a 3.5/5 star rating in the Apple Store.\r\n<h4>About glasses.com</h4>\r\nGlasses.com launched in 2011 as an online retailer of prescription and non-prescription glasses and sunglasses, carrying a wide-range of styles and name brands such as Oakley, Ray-Ban, Kate Spade, Burberry, Maui Jim, and more.[/column][column type=\"one-fifth\" last=\"true\"].[/column][/content_band]','Case Study: Reveal Imaging','','inherit','open','open','','6923-revision-v1','','','2014-10-16 20:44:36','2014-10-16 20:44:36','',6923,'http://sandbox.arrayfire.com/6923-revision-v1/',0,'revision','',0),(6925,7,'2014-10-16 20:51:32','2014-10-16 20:51:32','[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"][column type=\"one-fifth\"].[/column][column type=\"three-fifths\"][custom_headline type=\"left\" level=\"h3\"]Reveal Imaging, an SAIC Company[/custom_headline][/column][column type=\"one-fifth\" last=\"true\"].[/column][gap size=\"3%\"]\r\n\r\n[column type=\"one-fifth\"].[/column][column type=\"three-fifths\"]<img class=\"alignright size-full wp-image-6896\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/reveal-logo-promo-01.png\" alt=\"reveal logo, promo-01\" width=\"360\" height=\"120\" />Iterative reconstruction from projections is typically a function reserved for the medical industry. This approach allows medical professionals to reconstruct computed tomography (CT) images from acquired data; the reconstructed images display virtual “slices” of a scanned object, such as a heart or brain.\r\nIn reconstructing an image from the data, algorithms are used to approximate the solution. The closer the solution, the better the reconstruction. Multiple iterations produce more accurate solutions, and, thus, higher quality images. This technique, however, consumes computation time; speeding the process through the GPU is one solution to this problem.\r\n<blockquote>The changes that you made sped up the system more than expected! How often does anybody get to say that things work better than expected?  —Reveal Imaging</blockquote>\r\n<h4>Mission</h4>\r\nReveal Imaging, an SAIC company, uses this same technology in developing threat-detection products for transportation systems—such as cargo and baggage screening in airports—and in securing facilities and public events.\r\n\r\nReveal Imaging was running its cone-beam reconstructor via CUDA code previously implemented by a different contractor, but the performance was sub-par. They tasked ArrayFire with improving that performance on their GTX690s.\r\n<h4>Action</h4>\r\nOur engineers approached this by following standard CUDA practices. The project consisted mostly of cleaning up the existing code to allow it to run faster.\r\n\r\nFirst we removed the dependence of the permitted slice thickness on the number of GPU cards in the system. Then we allowed for one of the GPUs in the system to be excluded from those used by the reconstruction code and dedicated to driving a display. Next, we assured that the code met specifications with the latest CUDA compilers through version 5.0. Finally, we implemented performance enhancements, bottleneck reductions, and bug fixes previously identified.\r\n<h4>Results</h4>\r\nThe conclusion of the project was followed by a three week testing period, during which time Reveal Imaging determined reliability and overall satisfaction of the code’s performance. The code’s original speed was measured at around 200 slices per second—through our optimization efforts we were able to increase that number to 320 slices per second. Upon completion of the testing period Reveal Imaging was very pleased with the results and end performance of their code.\r\n\r\nIn the end, Reveal Imaging was able to focus on other development aspects of their project while we focused on what we do best—working relentlessly to achieve the best possible results in parallel computing.\r\n<h4>About SAIC</h4>\r\nScience Applications International Corporation (SAIC) is a leading technology integrator that provides full lifecycle services and solutions in the technical, engineering, and enterprise IT markets. SAIC has approximately 13,000 employees worldwide.[/column][column type=\"one-fifth\" last=\"true\"].[/column][/content_band]','Case Study: Reveal Imaging','','inherit','open','open','','6923-revision-v1','','','2014-10-16 20:51:32','2014-10-16 20:51:32','',6923,'http://sandbox.arrayfire.com/6923-revision-v1/',0,'revision','',0),(6926,7,'2014-10-16 20:54:24','2014-10-16 20:54:24','[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"][column type=\"one-fifth\"].[/column][column type=\"three-fifths\"][custom_headline type=\"left\" level=\"h3\"]Reveal Imaging, an SAIC Company[/custom_headline][/column][column type=\"one-fifth\" last=\"true\"].[/column][gap size=\"3%\"]\r\n\r\n[column type=\"one-fifth\"].[/column][column type=\"three-fifths\"]<img class=\"alignright size-full wp-image-6896\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/reveal-logo-promo-01.png\" alt=\"reveal logo, promo-01\" width=\"360\" height=\"120\" />Iterative reconstruction from projections is typically a function reserved for the medical industry. This approach allows medical professionals to reconstruct computed tomography (CT) images from acquired data; the reconstructed images display virtual “slices” of a scanned object, such as a heart or brain.\r\n\r\nIn reconstructing an image from the data, algorithms are used to approximate the solution. The closer the solution, the better the reconstruction. Multiple iterations produce more accurate solutions, and, thus, higher quality images. This technique, however, consumes computation time; speeding the process through the GPU is one solution to this problem.\r\n<blockquote>The changes that you made sped up the system more than expected! How often does anybody get to say that things work better than expected?  —Reveal Imaging</blockquote>\r\n<h4>Mission</h4>\r\nReveal Imaging, an SAIC company, uses this CT technology in developing threat-detection products for transportation systems—such as cargo and baggage screening in airports—and in securing facilities and public events. Speed is critical in their technology, and they approached us seeking acceleration.\r\n\r\nReveal Imaging was running its cone-beam reconstructor via CUDA code previously implemented by a different contractor, but the performance was sub-par. They tasked ArrayFire with improving that performance on their GTX690s.\r\n<h4>Action</h4>\r\nOur engineers approached this by following standard CUDA practices. The project consisted mostly of cleaning up the existing code to allow it to run faster.\r\n\r\nFirst we removed the dependence of the permitted slice thickness on the number of GPU cards in the system. Then we allowed for one of the GPUs in the system to be excluded from those used by the reconstruction code and dedicated to driving a display. Next, we assured that the code met specifications with the latest CUDA compilers through version 5.0. Finally, we implemented performance enhancements, bottleneck reductions, and bug fixes previously identified.\r\n<h4>Results</h4>\r\nThe conclusion of the project was followed by a three week testing period, during which time Reveal Imaging determined reliability and overall satisfaction of the code’s performance. The code’s original speed was measured at around 200 slices per second—through our optimization efforts we were able to increase that number to 320 slices per second. Upon completion of the testing period Reveal Imaging was very pleased with the results and end performance of their code.\r\n\r\nIn the end, Reveal Imaging was able to focus on other development aspects of their project while we focused on what we do best—working relentlessly to achieve the best possible results in parallel computing.\r\n<h4>About SAIC</h4>\r\nScience Applications International Corporation (SAIC) is a leading technology integrator that provides full lifecycle services and solutions in the technical, engineering, and enterprise IT markets. SAIC has approximately 13,000 employees worldwide.[/column][column type=\"one-fifth\" last=\"true\"].[/column][/content_band]','Case Study: Reveal Imaging','','inherit','open','open','','6923-revision-v1','','','2014-10-16 20:54:24','2014-10-16 20:54:24','',6923,'http://sandbox.arrayfire.com/6923-revision-v1/',0,'revision','',0),(6927,7,'2014-10-16 20:55:17','2014-10-16 20:55:17','[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"][column type=\"one-fifth\"].[/column][column type=\"three-fifths\"][custom_headline type=\"left\" level=\"h3\"]Reveal Imaging, an SAIC Company[/custom_headline][/column][column type=\"one-fifth\" last=\"true\"].[/column][gap size=\"3%\"]\r\n\r\n[column type=\"one-fifth\"].[/column][column type=\"three-fifths\"]<img class=\"alignright size-full wp-image-6896\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/reveal-logo-promo-01.png\" alt=\"reveal logo, promo-01\" width=\"360\" height=\"120\" />Iterative reconstruction from projections is typically a function reserved for the medical industry. This approach allows medical professionals to reconstruct computed tomography (CT) images from acquired data; the reconstructed images display virtual “slices” of a scanned object, such as a heart or brain.\r\n\r\nIn reconstructing an image from the data, algorithms are used to approximate the solution. The closer the solution, the better the reconstruction. Multiple iterations produce more accurate solutions, and, thus, higher quality images. This technique, however, consumes computation time; speeding the process through the GPU is one solution to this problem.\r\n<h4>Mission</h4>\r\nReveal Imaging, an SAIC company, uses this CT technology in developing threat-detection products for transportation systems—such as cargo and baggage screening in airports—and in securing facilities and public events. Speed is critical in their technology, and they approached us seeking acceleration.\r\n\r\nReveal Imaging was running its cone-beam reconstructor via CUDA code previously implemented by a different contractor, but the performance was sub-par. They tasked ArrayFire with improving that performance on their GTX690s.\r\n<h4>Action</h4>\r\nOur engineers approached this by following standard CUDA practices. The project consisted mostly of cleaning up the existing code to allow it to run faster.\r\n\r\nFirst we removed the dependence of the permitted slice thickness on the number of GPU cards in the system. Then we allowed for one of the GPUs in the system to be excluded from those used by the reconstruction code and dedicated to driving a display. Next, we assured that the code met specifications with the latest CUDA compilers through version 5.0. Finally, we implemented performance enhancements, bottleneck reductions, and bug fixes previously identified.\r\n<h4>Results</h4>\r\nThe conclusion of the project was followed by a three week testing period, during which time Reveal Imaging determined reliability and overall satisfaction of the code’s performance. The code’s original speed was measured at around 200 slices per second—through our optimization efforts we were able to increase that number to 320 slices per second. Upon completion of the testing period Reveal Imaging was very pleased with the results and end performance of their code.\r\n<blockquote>The changes that you made sped up the system more than expected! How often does anybody get to say that things work better than expected?  —Reveal Imaging</blockquote>\r\nIn the end, Reveal Imaging was able to focus on other development aspects of their project while we focused on what we do best—working relentlessly to achieve the best possible results in parallel computing.\r\n<h4>About SAIC</h4>\r\nScience Applications International Corporation (SAIC) is a leading technology integrator that provides full lifecycle services and solutions in the technical, engineering, and enterprise IT markets. SAIC has approximately 13,000 employees worldwide.[/column][column type=\"one-fifth\" last=\"true\"].[/column][/content_band]','Case Study: Reveal Imaging','','inherit','open','open','','6923-revision-v1','','','2014-10-16 20:55:17','2014-10-16 20:55:17','',6923,'http://sandbox.arrayfire.com/6923-revision-v1/',0,'revision','',0),(6928,7,'2015-04-08 21:03:09','2015-04-08 21:03:09','[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"][column type=\"one-fifth\"].[/column][column type=\"three-fifths\"][custom_headline type=\"left\" level=\"h3\"]Fortune 300 Financial Company[/custom_headline][/column][column type=\"one-fifth\" last=\"true\"].[/column][gap size=\"3%\"]\r\n\r\n[column type=\"one-fifth\"].[/column][column type=\"three-fifths\"]\r\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2015/04/philadelphia-skyline.png\"><img class=\"aligncenter size-full wp-image-9081\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2015/04/philadelphia-skyline.png\" alt=\"philadelphia-skyline\" width=\"720\" height=\"160\" /></a>\r\n\r\nA Fortune 300 Financial Company located in the North East USA ported their CPU code (C/C++) to CUDA and this enabled them to speed up the necessary financial calculations. Their primary objective was to reduce the time taken to run this code. Before porting to CUDA, they required ten to twelve hours to run the entire code. After porting to CUDA, they were able to do the same work in about 30 minutes.\r\n\r\nThe Client has several C/C++ programmers in their team and they trained many of them to use CUDA. Their code used CUDA to speed up a calculation using Monte Carlo methods to analyze various scenarios in their hedge-fund projection system (HPS). The major portion of their calculations were carried out on independent policies and thus the entire process was ideal for parallel programming (no communication bottlenecks).\r\n<h4>Mission</h4>\r\nThe CPU code was slow and this was holding them back. Because the code took the entire night to run, they were not able to run several different scenarios and take corrective action based on the results of the code. The primary reason to move to GPUs was to speed up the calculations.\r\n\r\nThe Client invested heavily into building CPU+GPU clusters relying on AMAX. They had the code running on four compute nodes, with four GPUs on each node. Each GPU was a NVIDIA K20X. The code ran on 16 GPUs. They were in the process of installing an even bigger system, with 64 total GPUs.\r\n<h4>Action</h4>\r\nOur initial plan was adjusted as the project progressed, and we identified key areas that needed improvement. The first couple of weeks were somewhat tricky because we were not able to get a working version of the codebase that was representative of the code they were actually running. This was because the code was in a very active state of development and integration on their side.\r\n\r\nWe made several suggestions to improve the efficiency of the codebase, but most of these were minor improvements. There were two levels of parallelism in the code—at the kernel level (GPU CUDA code) and the cluster level (MPI). One of the really useful suggestions we made was relevant at the MPI level, where we suggested they organize their policies in a random manner to distribute the computing load in a more uniform manner. This led to a major efficiency increase. This suggestion was immediately implemented by their engineers. Another useful contribution we made was to stress-test their code to get an estimate of how many more policies they could handle. This was really simple to do, but was a big hit within the company.\r\n\r\nIn the begging of the project, we were unable to access a working version of their code. Additionally, we were not able to access their machines from our laptops. Both of these challenges were overcome in the following week and no significant adjustments to the original plan were required.\r\n<h4>Results</h4>\r\nThe final deliverable consisted of a detailed technical report covering six modules of our assessment of Client\'s HPS, including the following system reviews: code base, performance and optimization, fault tolerance, scalability, compatibility with emerging CUDA trends, and disaster recovery. We also delivered an executive summary of our findings suitable for higher management.\r\n\r\nThe Client had already developed the code base for their HPS. They engaged ArrayFire to obtain an expert assessment of their work, and to get a \"stamp of approval\" that the code will work as intended and is of sufficiently good quality. We successfully delivered on all objectives. Our assessment and findings were fully documented in nearly 100 pages of detailed technical reports and slide modules. These contained a wealth of significant benchmarks and visuals related to the code base and our assessment of their HPS.\r\n\r\nThe Client was extremely satisfied with our input and suggestions. We met or exceeded their expectations in all project objectives and deliverables.\r\n[/column][column type=\"one-fifth\" last=\"true\"].[/column][/content_band]\r\n\r\nGraphics credit <a href=\"http://cliparts.co/clipart/2385767\">cliparts.co</a>','Case Study: Fortune 300 Financial Company','','publish','open','closed','','case-study-fortune-300-financial-company','','','2016-08-23 16:12:21','2016-08-23 20:12:21','',0,'http://sandbox.arrayfire.com/?page_id=6928',0,'page','',0),(6929,7,'2014-10-16 21:04:35','2014-10-16 21:04:35','[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"][column type=\"one-fifth\"].[/column][column type=\"three-fifths\"][custom_headline type=\"left\" level=\"h3\"]Reveal Imaging, an SAIC Company[/custom_headline][/column][column type=\"one-fifth\" last=\"true\"].[/column][gap size=\"3%\"]\r\n\r\n[column type=\"one-fifth\"].[/column][column type=\"three-fifths\"]<img class=\"alignright size-full wp-image-6896\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/reveal-logo-promo-01.png\" alt=\"reveal logo, promo-01\" width=\"360\" height=\"120\" />Iterative reconstruction from projections is typically a function reserved for the medical industry. This approach allows medical professionals to reconstruct computed tomography (CT) images from acquired data; the reconstructed images display virtual “slices” of a scanned object, such as a heart or brain.\r\n\r\nIn reconstructing an image from the data, algorithms are used to approximate the solution. The closer the solution, the better the reconstruction. Multiple iterations produce more accurate solutions, and, thus, higher quality images. This technique, however, consumes computation time; speeding the process through the GPU is one solution to this problem.\r\n<h4>Mission</h4>\r\nReveal Imaging, an SAIC company, uses this CT technology in developing threat-detection products for transportation systems—such as cargo and baggage screening in airports—and in securing facilities and public events. Speed is critical in their technology, and they approached us seeking acceleration.\r\n\r\nReveal Imaging was running its cone-beam reconstructor via CUDA code previously implemented by a different contractor, but the performance was sub-par. They tasked ArrayFire with improving that performance on their GTX690s.\r\n<h4>Action</h4>\r\nOur engineers approached this by following standard CUDA practices. The project consisted mostly of cleaning up the existing code to allow it to run faster.\r\n\r\nFirst we removed the dependence of the permitted slice thickness on the number of GPU cards in the system. Then we allowed for one of the GPUs in the system to be excluded from those used by the reconstruction code and dedicated to driving a display. Next, we assured that the code met specifications with the latest CUDA compilers through version 5.0. Finally, we implemented performance enhancements, bottleneck reductions, and bug fixes previously identified.\r\n<h4>Results</h4>\r\nThe conclusion of the project was followed by a three week testing period, during which time Reveal Imaging determined reliability and overall satisfaction of the code’s performance. The code’s original speed was measured at around 200 slices per second—through our optimization efforts we were able to increase that number to 320 slices per second. Upon completion of the testing period Reveal Imaging was very pleased with the results and end performance of their code.\r\n<blockquote>The changes that you made sped up the system more than expected! How often does anybody get to say that things work better than expected?  —Reveal Imaging</blockquote>\r\nIn the end, Reveal Imaging was able to focus on other development aspects of their project while we focused on what we do best—working relentlessly to achieve the best possible results in parallel computing.\r\n<h4>About SAIC</h4>\r\nScience Applications International Corporation (SAIC) is a leading technology integrator that provides full lifecycle services and solutions in the technical, engineering, and enterprise IT markets. SAIC has approximately 13,000 employees worldwide.[/column][column type=\"one-fifth\" last=\"true\"].[/column][/content_band]','Case Study: Lincoln Financial Group','','inherit','open','open','','6928-revision-v1','','','2014-10-16 21:04:35','2014-10-16 21:04:35','',6928,'http://sandbox.arrayfire.com/6928-revision-v1/',0,'revision','',0),(6930,7,'2014-10-16 21:04:50','2014-10-16 21:04:50','[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"][column type=\"one-fifth\"].[/column][column type=\"three-fifths\"][custom_headline type=\"left\" level=\"h3\"]Lincoln Financial Group[/custom_headline][/column][column type=\"one-fifth\" last=\"true\"].[/column][gap size=\"3%\"]\r\n\r\n[column type=\"one-fifth\"].[/column][column type=\"three-fifths\"]<img class=\"alignright size-full wp-image-6896\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/reveal-logo-promo-01.png\" alt=\"reveal logo, promo-01\" width=\"360\" height=\"120\" />Iterative reconstruction from projections is typically a function reserved for the medical industry. This approach allows medical professionals to reconstruct computed tomography (CT) images from acquired data; the reconstructed images display virtual “slices” of a scanned object, such as a heart or brain.\r\n\r\nIn reconstructing an image from the data, algorithms are used to approximate the solution. The closer the solution, the better the reconstruction. Multiple iterations produce more accurate solutions, and, thus, higher quality images. This technique, however, consumes computation time; speeding the process through the GPU is one solution to this problem.\r\n<h4>Mission</h4>\r\nReveal Imaging, an SAIC company, uses this CT technology in developing threat-detection products for transportation systems—such as cargo and baggage screening in airports—and in securing facilities and public events. Speed is critical in their technology, and they approached us seeking acceleration.\r\n\r\nReveal Imaging was running its cone-beam reconstructor via CUDA code previously implemented by a different contractor, but the performance was sub-par. They tasked ArrayFire with improving that performance on their GTX690s.\r\n<h4>Action</h4>\r\nOur engineers approached this by following standard CUDA practices. The project consisted mostly of cleaning up the existing code to allow it to run faster.\r\n\r\nFirst we removed the dependence of the permitted slice thickness on the number of GPU cards in the system. Then we allowed for one of the GPUs in the system to be excluded from those used by the reconstruction code and dedicated to driving a display. Next, we assured that the code met specifications with the latest CUDA compilers through version 5.0. Finally, we implemented performance enhancements, bottleneck reductions, and bug fixes previously identified.\r\n<h4>Results</h4>\r\nThe conclusion of the project was followed by a three week testing period, during which time Reveal Imaging determined reliability and overall satisfaction of the code’s performance. The code’s original speed was measured at around 200 slices per second—through our optimization efforts we were able to increase that number to 320 slices per second. Upon completion of the testing period Reveal Imaging was very pleased with the results and end performance of their code.\r\n<blockquote>The changes that you made sped up the system more than expected! How often does anybody get to say that things work better than expected?  —Reveal Imaging</blockquote>\r\nIn the end, Reveal Imaging was able to focus on other development aspects of their project while we focused on what we do best—working relentlessly to achieve the best possible results in parallel computing.\r\n<h4>About SAIC</h4>\r\nScience Applications International Corporation (SAIC) is a leading technology integrator that provides full lifecycle services and solutions in the technical, engineering, and enterprise IT markets. SAIC has approximately 13,000 employees worldwide.[/column][column type=\"one-fifth\" last=\"true\"].[/column][/content_band]','Case Study: Lincoln Financial Group','','inherit','open','open','','6928-revision-v1','','','2014-10-16 21:04:50','2014-10-16 21:04:50','',6928,'http://sandbox.arrayfire.com/6928-revision-v1/',0,'revision','',0),(6931,7,'2014-10-16 21:05:10','2014-10-16 21:05:10','[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"][column type=\"one-fifth\"].[/column][column type=\"three-fifths\"][custom_headline type=\"left\" level=\"h3\"]Lincoln Financial Group[/custom_headline][/column][column type=\"one-fifth\" last=\"true\"].[/column][gap size=\"3%\"]\r\n\r\n[column type=\"one-fifth\"].[/column][column type=\"three-fifths\"]<img class=\"alignright size-full wp-image-6895\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/lincoln-logo-promo-01.png\" alt=\"lincoln logo, promo-01\" width=\"360\" height=\"120\" />Iterative reconstruction from projections is typically a function reserved for the medical industry. This approach allows medical professionals to reconstruct computed tomography (CT) images from acquired data; the reconstructed images display virtual “slices” of a scanned object, such as a heart or brain.\r\n\r\nIn reconstructing an image from the data, algorithms are used to approximate the solution. The closer the solution, the better the reconstruction. Multiple iterations produce more accurate solutions, and, thus, higher quality images. This technique, however, consumes computation time; speeding the process through the GPU is one solution to this problem.\r\n<h4>Mission</h4>\r\nReveal Imaging, an SAIC company, uses this CT technology in developing threat-detection products for transportation systems—such as cargo and baggage screening in airports—and in securing facilities and public events. Speed is critical in their technology, and they approached us seeking acceleration.\r\n\r\nReveal Imaging was running its cone-beam reconstructor via CUDA code previously implemented by a different contractor, but the performance was sub-par. They tasked ArrayFire with improving that performance on their GTX690s.\r\n<h4>Action</h4>\r\nOur engineers approached this by following standard CUDA practices. The project consisted mostly of cleaning up the existing code to allow it to run faster.\r\n\r\nFirst we removed the dependence of the permitted slice thickness on the number of GPU cards in the system. Then we allowed for one of the GPUs in the system to be excluded from those used by the reconstruction code and dedicated to driving a display. Next, we assured that the code met specifications with the latest CUDA compilers through version 5.0. Finally, we implemented performance enhancements, bottleneck reductions, and bug fixes previously identified.\r\n<h4>Results</h4>\r\nThe conclusion of the project was followed by a three week testing period, during which time Reveal Imaging determined reliability and overall satisfaction of the code’s performance. The code’s original speed was measured at around 200 slices per second—through our optimization efforts we were able to increase that number to 320 slices per second. Upon completion of the testing period Reveal Imaging was very pleased with the results and end performance of their code.\r\n<blockquote>The changes that you made sped up the system more than expected! How often does anybody get to say that things work better than expected?  —Reveal Imaging</blockquote>\r\nIn the end, Reveal Imaging was able to focus on other development aspects of their project while we focused on what we do best—working relentlessly to achieve the best possible results in parallel computing.\r\n<h4>About SAIC</h4>\r\nScience Applications International Corporation (SAIC) is a leading technology integrator that provides full lifecycle services and solutions in the technical, engineering, and enterprise IT markets. SAIC has approximately 13,000 employees worldwide.[/column][column type=\"one-fifth\" last=\"true\"].[/column][/content_band]','Case Study: Lincoln Financial Group','','inherit','open','open','','6928-revision-v1','','','2014-10-16 21:05:10','2014-10-16 21:05:10','',6928,'http://sandbox.arrayfire.com/6928-revision-v1/',0,'revision','',0),(6932,7,'2014-10-17 14:57:38','2014-10-17 14:57:38','[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"][column type=\"one-fifth\"].[/column][column type=\"three-fifths\"][custom_headline type=\"left\" level=\"h3\"]Glasses.com App with 3Dfit Technology[/custom_headline][/column][column type=\"one-fifth\" last=\"true\"].[/column][gap size=\"3%\"]\r\n\r\n[column type=\"one-fifth\"].[/column][column type=\"three-fifths\"]<img class=\"alignright wp-image-6894 size-full\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/glasses-logo-promo-01.png\" alt=\"glasses logo, promo-01\" width=\"360\" height=\"120\" />Many find the experience of purchasing new glasses stressful. Even simply trying on frames can be uncomfortable when done in a public setting. Glasses.com set out to improve the experience by providing customers a way of virtually trying on glasses frames in the comfort of their homes via smart phone or tablet.\r\n\r\nThe Glasses.com app takes a picture of the user’s face, creates a 3D model of facial features, and virtually places selected frames on the user’s 3D image. The user is then free to view the 3D image from a variety of angles.\r\n\r\nThe application relies on computer vision to recognize the features of a user’s face and then generate an accurate, high-fidelity, 3D model of that face. Virtual frames can then be positioned on the 3D model.\r\n<blockquote>This app is amazing! It takes a 3D video of your face in just a few steps, and the rendering is spot on! RayBan Aviators are always a little big on me, and the app showed them on my face exactly as they fit in the store! —davidandrew</blockquote>\r\n<h4>Mission</h4>\r\nGlasses.com brought our engineers an excellent, fully-functioning version of the application; they simply wanted it accelerated. The app took full advantage of the CPU, which—left unchanged—would necessitate Glasses.com’s purchasing and managing a large amount of hardware. Further, the application was running at a slow thirty seconds per session, resulting in a sub-optimal user experience. Our engineers were tasked with parallelizing the process and reducing the session runtime.\r\n\r\nGlasses.com has several machines they are using in production. Their main setup consists of a Haswell/Ivy Bridge CPU paired with a GTX 690. The machine is able to run 4 processes concurrently. In the past, Glasses.com used a 32 core Intel process with a couple of K20X. They are still using these machines, but they are not the main configuration. They are also making use of the Amazon EC2 instances in case they exceed internal capacity.\r\n<h4>Action</h4>\r\nWe began by analyzing the application and targeting the process which would have the greatest impact on performance.\r\n\r\nThe most pressing concern was the rendering portion of code, which was running on the CPU. We implemented rendering in ArrayFire, which is specially optimized for CUDA GPUs. Further, we noticed that while the application was rendering-intensive, much of the rendering did not need to be precise—we could make estimations without compromising quality. This estimated rendering speedup operation was critical to the success of the project.\r\n\r\nWe discovered further opportunities for speedup within the architecture itself. The original app processed each frame in a separate thread; we accelerated this by combining each operation into a single kernel and running that kernel once. By moving the parallelization to the GPU we effectively took advantage of the strength of that architecture.\r\n<h4>Results</h4>\r\nUltimately, parallelizing the application and incorporating the ArrayFire software library accelerated the application to about ten seconds per session—roughly three times as fast as the original code, and twice as fast as the target speed. We further exceeded expectations by improving general stability when handling errors. <img class=\"aligncenter size-full wp-image-6903\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/glassesdotcom-benchmarks.jpg\" alt=\"glassesdotcom benchmarks\" width=\"700\" height=\"442\" />\r\n\r\nThe improvements in speed, stability, and user experience have resonated with the application’s target market, resulting in a 3.5/5 star rating in the Apple Store.\r\n<h5>About glasses.com</h5>\r\nGlasses.com launched in 2011 as an online retailer of prescription and non-prescription glasses and sunglasses, carrying a wide-range of styles and name brands such as Oakley, Ray-Ban, Kate Spade, Burberry, Maui Jim, and more.[/column][column type=\"one-fifth\" last=\"true\"].[/column][/content_band]','Case Study: glasses.com App with 3Dfit Technology','','inherit','open','open','','6899-revision-v1','','','2014-10-17 14:57:38','2014-10-17 14:57:38','',6899,'http://sandbox.arrayfire.com/6899-revision-v1/',0,'revision','',0),(6933,7,'2014-10-17 14:58:11','2014-10-17 14:58:11','[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"][column type=\"one-fifth\"].[/column][column type=\"three-fifths\"][custom_headline type=\"left\" level=\"h3\"]Glasses.com App with 3Dfit Technology[/custom_headline][/column][column type=\"one-fifth\" last=\"true\"].[/column][gap size=\"3%\"]\r\n\r\n[column type=\"one-fifth\"].[/column][column type=\"three-fifths\"]<img class=\"alignright wp-image-6894 size-full\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/glasses-logo-promo-01.png\" alt=\"glasses logo, promo-01\" width=\"360\" height=\"120\" />Many find the experience of purchasing new glasses stressful. Even simply trying on frames can be uncomfortable when done in a public setting. Glasses.com set out to improve the experience by providing customers a way of virtually trying on glasses frames in the comfort of their homes via smart phone or tablet.\r\n\r\nThe Glasses.com app takes a picture of the user’s face, creates a 3D model of facial features, and virtually places selected frames on the user’s 3D image. The user is then free to view the 3D image from a variety of angles.\r\n\r\nThe application relies on computer vision to recognize the features of a user’s face and then generate an accurate, high-fidelity, 3D model of that face. Virtual frames can then be positioned on the 3D model.\r\n<blockquote>This app is amazing! It takes a 3D video of your face in just a few steps, and the rendering is spot on! RayBan Aviators are always a little big on me, and the app showed them on my face exactly as they fit in the store! —davidandrew</blockquote>\r\n<h4>Mission</h4>\r\nGlasses.com brought our engineers an excellent, fully-functioning version of the application; they simply wanted it accelerated. The app took full advantage of the CPU, which—left unchanged—would necessitate Glasses.com’s purchasing and managing a large amount of hardware. Further, the application was running at a slow thirty seconds per session, resulting in a sub-optimal user experience. Our engineers were tasked with parallelizing the process and reducing the session runtime.\r\n\r\nGlasses.com has several machines they are using in production. Their main setup consists of a Haswell/Ivy Bridge CPU paired with a GTX 690. The machine is able to run 4 processes concurrently. In the past, Glasses.com used a 32 core Intel process with a couple of K20X. They are still using these machines, but they are not the main configuration. They are also making use of the Amazon EC2 instances in case they exceed internal capacity.\r\n<h4>Action</h4>\r\nWe began by analyzing the application and targeting the process which would have the greatest impact on performance.\r\n\r\nThe most pressing concern was the rendering portion of code, which was running on the CPU. We implemented rendering in ArrayFire, which is specially optimized for CUDA GPUs. Further, we noticed that while the application was rendering-intensive, much of the rendering did not need to be precise—we could make estimations without compromising quality. This estimated rendering speedup operation was critical to the success of the project.\r\n\r\nWe discovered further opportunities for speedup within the architecture itself. The original app processed each frame in a separate thread; we accelerated this by combining each operation into a single kernel and running that kernel once. By moving the parallelization to the GPU we effectively took advantage of the strength of that architecture.\r\n<h4>Results</h4>\r\nUltimately, parallelizing the application and incorporating the ArrayFire software library accelerated the application to about ten seconds per session—roughly three times as fast as the original code, and twice as fast as the target speed. We further exceeded expectations by improving general stability when handling errors. <img class=\"aligncenter size-full wp-image-6903\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/glassesdotcom-benchmarks.jpg\" alt=\"glassesdotcom benchmarks\" width=\"700\" height=\"442\" />\r\n\r\nThe improvements in speed, stability, and user experience have resonated with the application’s target market, resulting in a 3.5/5 star rating in the Apple Store.\r\n<h6>About glasses.com</h6>\r\nGlasses.com launched in 2011 as an online retailer of prescription and non-prescription glasses and sunglasses, carrying a wide-range of styles and name brands such as Oakley, Ray-Ban, Kate Spade, Burberry, Maui Jim, and more.[/column][column type=\"one-fifth\" last=\"true\"].[/column][/content_band]','Case Study: glasses.com App with 3Dfit Technology','','inherit','open','open','','6899-revision-v1','','','2014-10-17 14:58:11','2014-10-17 14:58:11','',6899,'http://sandbox.arrayfire.com/6899-revision-v1/',0,'revision','',0),(6936,7,'2014-10-17 15:55:29','2014-10-17 15:55:29','[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"][column type=\"one-fifth\"].[/column][column type=\"three-fifths\"][custom_headline type=\"left\" level=\"h3\"]Lincoln Financial Group[/custom_headline][/column][column type=\"one-fifth\" last=\"true\"].[/column][gap size=\"3%\"]\r\n\r\n[column type=\"one-fifth\"].[/column][column type=\"three-fifths\"]<img class=\"alignright size-full wp-image-6895\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/lincoln-logo-promo-01.png\" alt=\"lincoln logo, promo-01\" width=\"360\" height=\"120\" /><span class=\"il\">Lincoln</span> Financial Group (LFG) had ported their CPU code (C/C++) to CUDA and this enabled them to speed up the necessary financial calculations. Their primary objective was to reduce the time taken to run this code. Before porting to CUDA, they required 10-12 hours to run the entire code. After porting to CUDA, they were able to do the same work in about 30 min.\r\n\r\nLFG has several C/C++ programmers in their team and they trained many of them to use CUDA. Their code used CUDA to speed up a calculation using Monte Carlo methods to analyze various scenarios in their so-called hedge-fund projection system (HPS). The major portion of their calculations were carried out on independent policies and thus the entire process was ideal for parallel programming (no communication bottlenecks).\r\n<h4>Mission</h4>\r\nThe CPU code was slow and this was holding them back. Because the code took the entire night to run, they were not able to run several different scenarios and take corrective action based on the results of the code. The primary reason to move to GPUs was to speed up the calculations.\r\n\r\nLFG invested heavily into building CPU+GPU clusters and I think the vendor they used to build this hardware was AMAX. Based on what I remember, they had the code running on 4 compute nodes, with 4 GPUs on each node. Each GPU was a NVIDIA K20X. The code ran on 16 GPUs. They were in the process of installing an even bigger system, with 64 total GPUs.\r\n<h4>Action</h4>\r\nWe did have a plan of action, outlined in our SOW. This plan was adjusted as the project progressed and we identified key areas that needed improvement. The first couple of weeks were somewhat tricky because we were not able to get a working version of the codebase that was representative of the code they were actually running. This was because the code was in a very active state of development and integration on their side.\r\n\r\nWe made several suggestions to improve the efficiency of the codebase, but most of these were minor improvements. There were two levels of parallelism in the code: at the kernel level (GPU CUDA code) and the cluster level (MPI). One of the really useful suggestions we made was relevant at the MPI level, where we suggested that that organize their policies in a random manner to distribute the computing load in a more uniform manner. This led to a major efficiency increase. This suggestion was immediately implemented by their engineers. Another useful contribution we made was to stress-test their code to get an estimate of how many more policies they could handle. This was really simple to do, but it seems this was a big hit with their management.\r\n\r\nThe most frustrating part was at the very beginning of the assignment, when we were simply unable to get a working version of their code. Additionally, we were not able to access their machines from our laptops (both Jimi and me had macs). Both problems were solved in the following week. No significant adjustments to the original plan were required.\r\n<h4>Results</h4>\r\nThe final deliverable was a series of slide decks and detailed technical reports that summarized our findings and analyses about the performance of their codebase. Also required was an executive summary of our findings suitable for higher management.\r\n<blockquote>The changes that you made sped up the system more than expected! How often does anybody get to say that things work better than expected?  —Reveal Imaging</blockquote>\r\nMy understanding is that they had already developed the codebase using GPUs and their main interest in giving us this project was to get an external assessment of their work and get a \"stamp of approval\" that the code will work as intended and is of sufficiently good quality. In this sense, yes, we were able to provide them with what they wanted.\r\n\r\nIn general, LFG was extremely happy with our input and suggestions and they got what they were looking for when they hired us. I would say that we met their expectations in all areas and exceeded their expectations in a select few.\r\n\r\nWe documented all our findings in the form of detailed technical reports and slide-decks. These contain a wealth of significant benchmarks and visuals related to the codebase and our experiments with it. These documents are all available in Google docs in team-&gt;consulting-&gt;LFG.\r\n<h6>About SAIC</h6>\r\nHeadquartered in the Philadelphia region, Lincoln Financial Group is a Fortune 500 company offering a diverse range of financial services and solutions. With a strong focus on four core business areas — life insurance, annuities, retirement plan services, and group protection — our business is built around supporting, preserving, and enhancing our customer\'s lifestyles and providing better retirement outcomes. Led by over 9,000 employees, Lincoln Financial provides the tools and advice to help individuals take charge of their futures.[/column][column type=\"one-fifth\" last=\"true\"].[/column][/content_band]','Case Study: Lincoln Financial Group','','inherit','open','open','','6928-revision-v1','','','2014-10-17 15:55:29','2014-10-17 15:55:29','',6928,'http://sandbox.arrayfire.com/6928-revision-v1/',0,'revision','',0),(6937,7,'2014-10-17 15:56:17','2014-10-17 15:56:17','[content_band inner_container=\"true\" no_margin=\"true\" border=\"top\" padding_top=\"40px\" padding_bottom=\"40px\"][column type=\"one-fifth\"].[/column][column type=\"three-fifths\"][custom_headline type=\"left\" level=\"h3\"]Lincoln Financial Group[/custom_headline][/column][column type=\"one-fifth\" last=\"true\"].[/column][gap size=\"3%\"]\r\n\r\n[column type=\"one-fifth\"].[/column][column type=\"three-fifths\"]<img class=\"alignright size-full wp-image-6895\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/lincoln-logo-promo-01.png\" alt=\"lincoln logo, promo-01\" width=\"360\" height=\"120\" /><span class=\"il\">Lincoln</span> Financial Group (LFG) had ported their CPU code (C/C++) to CUDA and this enabled them to speed up the necessary financial calculations. Their primary objective was to reduce the time taken to run this code. Before porting to CUDA, they required 10-12 hours to run the entire code. After porting to CUDA, they were able to do the same work in about 30 min.\r\n\r\nLFG has several C/C++ programmers in their team and they trained many of them to use CUDA. Their code used CUDA to speed up a calculation using Monte Carlo methods to analyze various scenarios in their so-called hedge-fund projection system (HPS). The major portion of their calculations were carried out on independent policies and thus the entire process was ideal for parallel programming (no communication bottlenecks).\r\n<h4>Mission</h4>\r\nThe CPU code was slow and this was holding them back. Because the code took the entire night to run, they were not able to run several different scenarios and take corrective action based on the results of the code. The primary reason to move to GPUs was to speed up the calculations.\r\n\r\nLFG invested heavily into building CPU+GPU clusters and I think the vendor they used to build this hardware was AMAX. Based on what I remember, they had the code running on 4 compute nodes, with 4 GPUs on each node. Each GPU was a NVIDIA K20X. The code ran on 16 GPUs. They were in the process of installing an even bigger system, with 64 total GPUs.\r\n<h4>Action</h4>\r\nWe did have a plan of action, outlined in our SOW. This plan was adjusted as the project progressed and we identified key areas that needed improvement. The first couple of weeks were somewhat tricky because we were not able to get a working version of the codebase that was representative of the code they were actually running. This was because the code was in a very active state of development and integration on their side.\r\n\r\nWe made several suggestions to improve the efficiency of the codebase, but most of these were minor improvements. There were two levels of parallelism in the code: at the kernel level (GPU CUDA code) and the cluster level (MPI). One of the really useful suggestions we made was relevant at the MPI level, where we suggested that that organize their policies in a random manner to distribute the computing load in a more uniform manner. This led to a major efficiency increase. This suggestion was immediately implemented by their engineers. Another useful contribution we made was to stress-test their code to get an estimate of how many more policies they could handle. This was really simple to do, but it seems this was a big hit with their management.\r\n\r\nThe most frustrating part was at the very beginning of the assignment, when we were simply unable to get a working version of their code. Additionally, we were not able to access their machines from our laptops (both Jimi and me had macs). Both problems were solved in the following week. No significant adjustments to the original plan were required.\r\n<h4>Results</h4>\r\nThe final deliverable was a series of slide decks and detailed technical reports that summarized our findings and analyses about the performance of their codebase. Also required was an executive summary of our findings suitable for higher management.\r\n<blockquote>The changes that you made sped up the system more than expected! How often does anybody get to say that things work better than expected?  —Reveal Imaging</blockquote>\r\nMy understanding is that they had already developed the codebase using GPUs and their main interest in giving us this project was to get an external assessment of their work and get a \"stamp of approval\" that the code will work as intended and is of sufficiently good quality. In this sense, yes, we were able to provide them with what they wanted.\r\n\r\nIn general, LFG was extremely happy with our input and suggestions and they got what they were looking for when they hired us. I would say that we met their expectations in all areas and exceeded their expectations in a select few.\r\n\r\nWe documented all our findings in the form of detailed technical reports and slide-decks. These contain a wealth of significant benchmarks and visuals related to the codebase and our experiments with it. These documents are all available in Google docs in team-&gt;consulting-&gt;LFG.\r\n<h6>About Lincoln Financial Group</h6>\r\nHeadquartered in the Philadelphia region, Lincoln Financial Group is a Fortune 500 company offering a diverse range of financial services and solutions. With a strong focus on four core business areas — life insurance, annuities, retirement plan services, and group protection — our business is built around supporting, preserving, and enhancing our customer\'s lifestyles and providing better retirement outcomes. Led by over 9,000 employees, Lincoln Financial provides the tools and advice to help individuals take charge of their futures.[/column][column type=\"one-fifth\" last=\"true\"].[/column][/content_band]','Case Study: Lincoln Financial Group','','inherit','open','open','','6928-revision-v1','','','2014-10-17 15:56:17','2014-10-17 15:56:17','',6928,'http://sandbox.arrayfire.com/6928-revision-v1/',0,'revision','',0),