(9324,11,'2016-09-20 18:09:33','2016-09-20 22:09:33','With the release of <strong>ArrayFire v3.4</strong>, we have added a number of features that will help our vast range of users. Our users have been asking us for a way to reduce the memory footprint of their data on the GPU. And now we have answered our users\' requests.\r\nIntroducing Sparse Matrices in ArrayFire!\r\n\r\n<h4>Sparse Data Structure</h4>\r\nA sparse data structure is one where all the non-zero elements are not stored. Sparse matrices are useful when the number of zero-values elements are &gt;&gt; number of non-zero elements (sparsity of the matrix is high).\r\nThe sparse data structure in general is stored as 3 arrays.\r\n<ul>\r\n<li>A data or values array containing all the non-zero elements</li>\r\n<li>A vector for row indices (based on storage format).</li>\r\n<li>A vector for column indices (based on storage format).</li>\r\n</ul>\r\n\r\nThere are many ways to store sparse matrices, the most prominent of which are\r\n<ul>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_row_.28CSR.2C_CRS_or_Yale_format.29\">Compressed Sparse Row</a></li>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_column_.28CSC_or_CCS.29\">Compressed Storage Column</a></li>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Coordinate_list_.28COO.29\">Coordinate Format</a>.</li>\r\n</ul>\r\n\r\nArrayFire supports CSR and COO format, with CSC support coming in future releases.\r\n\r\nA more detailed description of Sparse Matrix can be found on <a href=\"https://en.wikipedia.org/wiki/Sparse_matrix\">Wikipedia</a>.\r\n\r\n<h4>Sparse in ArrayFire</h4>\r\nIn ArrayFire, Sparse arrays use the same user facing object - af::array. This makes working with sparse arrays all the more seamless.\r\nWe have added a function <a href=\"http://arrayfire.org/docs/group__method__mat.htm#ga5945f95549bae25f75694a69d98ca584\">af::array::issparse()</a> that allows users to check if an array is sparse or not.\r\nAlthough externally, there is no difference between a sparse and dense array, internally, the sparse array is stored as 3 independent arrays along with other meta data.\r\nThese arrays, as well as other properties can be fetched using the <a href=\"http://arrayfire.org/docs/group__sparse__func.htm\">sparse functions</a>.\r\n\r\n<h4>Sparse Functionality</h4>\r\nAs of v3.4, the following features are supported for sparse arrays:\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm\">Creation for sparse arrays.</a>\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae4621e58bfaf6d80668309824fde86c0\">From 3 (data, row, column) sparse host or device memory vectors.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae9a843c7e93b05e6b090ca32b271ebe9\">From 3 (data, row, column) sparse af::arrays.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">From dense af::array.</a></li>\r\n<li>To create a sparse array from host or device dense array, first create a dense af::array, and then pass it to create sparse af::array from dense af::array.</li>\r\n</ul>\r\n</li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__convert__to.htm\">Storage format conversions.</a>\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">Dense to CSR/COO.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__dense.htm\">CSR/COO to dense.</a></li>\r\n</ul>\r\n</li>\r\n<li><a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">Sparse-Dense Matrix Multiplication (SPMM/SPMV)</a>\r\n<ul>\r\n<li>Sparse Matrix - Dense Matrix</li>\r\n<li>Sparse Matrix - Dense Vector</li>\r\n<li>Options for Sparse Matrix to be transposed.</li>\r\n</ul>\r\n</li>\r\n</ul>\r\n\r\nThe Sparse-Dense Matrix Multiplcation is a implemented as a part of <a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">af::matmul()</a>. This is part of making the usage of sparse arrays seamless with regular arrays.\r\n\r\n<h4>Example</h4>\r\n<pre>\r\n/*******************************************************\r\n * Copyright (c) 2014, ArrayFire\r\n * All rights reserved.\r\n *\r\n * This file is distributed under 3-clause BSD license.\r\n * The complete license agreement can be obtained at:\r\n * http://sandbox.arrayfire.com/licenses/BSD-3-Clause\r\n ********************************************************/\r\n#include <arrayfire.h>\r\n#include <iostream>\r\n\r\nusing namespace af;\r\n\r\nstatic size_t dimension = 16 * 1024;\r\nstatic const int maxIter = 10;\r\nstatic const int sparsityFactor = 7;\r\n\r\nstatic array A;\r\nstatic array As;  // Sparse A\r\nstatic array x0;\r\nstatic array b;\r\n\r\nvoid setupInputs()\r\n{\r\n    // Generate a random input: A\r\n    array T = randu(dimension, dimension, f32);\r\n    // Create 0s in input.\r\n    // Anything that is no divisible by sparsityFactor will become 0.\r\n    A = floor(T * 1000);\r\n    A = A * ((A % sparsityFactor) == 0) / 1000;\r\n    // Make it positive definite\r\n    A = transpose(A) + A + A.dims(0)*identity(A.dims(0), A.dims(0), f32);\r\n\r\n    // Make A sparse as As\r\n    As = sparse(A);\r\n\r\n    // Generate x0: Random guess\r\n    x0 = randu(A.dims(0), f32);\r\n\r\n    //Generate b\r\n    b = matmul(A, x0);\r\n\r\n    // Run evals on inputs\r\n    A .eval();\r\n    sparseGetValues(As).eval();\r\n    sparseGetRowIdx(As).eval();\r\n    sparseGetColIdx(As).eval();\r\n    x0.eval();\r\n    b .eval();\r\n    af::sync();\r\n}\r\n\r\nvoid sparseConjugateGradient(void)\r\n{\r\n    array x = constant(0, b.dims(), f32);\r\n    array r = b - matmul(As, x);\r\n    array p = r;\r\n\r\n    for (int i = 0; i < maxIter; ++i) {\r\n        array Ap = matmul(As, p);\r\n        array alpha_num = dot(r, r);\r\n        array alpha_den = dot(p, Ap);\r\n        array alpha = alpha_num/alpha_den;\r\n        r = r - tile(alpha, Ap.dims())*Ap;\r\n        x = x + tile(alpha, Ap.dims())*p;\r\n        array beta_num = dot(r, r);\r\n        array beta = beta_num/alpha_num;\r\n        p = r + tile(beta, p.dims())*p;\r\n    }\r\n}\r\n\r\nvoid denseConjugateGradient(void)\r\n{\r\n    array x = constant(0, b.dims(), f32);\r\n    array r = b - matmul(A, x);\r\n    array p = r;\r\n\r\n    for (int i = 0; i < maxIter; ++i) {\r\n        array Ap = matmul(A, p);\r\n        array alpha_num = dot(r, r);\r\n        array alpha_den = dot(p, Ap);\r\n        array alpha = alpha_num/alpha_den;\r\n        r = r - tile(alpha, Ap.dims())*Ap;\r\n        x = x + tile(alpha, Ap.dims())*p;\r\n        array beta_num = dot(r, r);\r\n        array beta = beta_num/alpha_num;\r\n        p = r + tile(beta, p.dims())*p;\r\n    }\r\n}\r\n\r\nvoid checkConjugateGradient(const af::array in)\r\n{\r\n    array x = constant(0, b.dims(), f32);\r\n    array r = b - matmul(in, x);\r\n    array p = r;\r\n\r\n    for (int i = 0; i < maxIter; ++i) {\r\n        array Ap = matmul(in, p);\r\n        array alpha_num = dot(r, r);\r\n        array alpha_den = dot(p, Ap);\r\n        array alpha = alpha_num/alpha_den;\r\n        r = r - tile(alpha, Ap.dims())*Ap;\r\n        x = x + tile(alpha, Ap.dims())*p;\r\n        array beta_num = dot(r, r);\r\n        array beta = beta_num/alpha_num;\r\n        p = r + tile(beta, p.dims())*p;\r\n    }\r\n    array res = x0 - x;\r\n\r\n    std::cout<<\"Final difference in solutions:\\n\";\r\n    af_print(dot(res, res));\r\n}\r\n\r\nint main(int argc, char *argv[])\r\n{\r\n    af::info();\r\n    setupInputs();\r\n\r\n    std::cout << \"Verifying Dense Conjugate Gradient:\" << std::endl;\r\n    checkConjugateGradient(A);\r\n\r\n    std::cout << \"Verifying Sparse Conjugate Gradient:\" << std::endl;\r\n    checkConjugateGradient(As);\r\n\r\n    af::sync();\r\n\r\n    std::cout << \"Dense Conjugate Gradient Time: \"\r\n              << timeit(denseConjugateGradient) * 1000 << \"ms\" << std::endl;\r\n    af::sync();\r\n\r\n    std::cout << \"Sparse Conjugate Gradient Time: \"\r\n              << timeit(sparseConjugateGradient) * 1000 << \"ms\" << std::endl;\r\n    af::sync();\r\n\r\n    return 0;\r\n}\r\n</pre>\r\n<h4>Performance and Memory Footprint</h4>\r\n\r\nAs with all ArrayFire features, Sparse support is available across all backends and all architectures.','Sparse Matrices in ArrayFire','','inherit','closed','closed','','9190-revision-v1','','','2016-09-20 18:09:33','2016-09-20 22:09:33','',9190,'http://sandbox.arrayfire.com/9190-revision-v1/',0,'revision','',0),(9325,11,'2016-09-20 18:23:15','2016-09-20 22:23:15','With the release of <strong>ArrayFire v3.4</strong>, we have added a number of features that will help our vast range of users. Our users have been asking us for a way to reduce the memory footprint of their data on the GPU. And now we have answered our users\' requests.\r\nIntroducing Sparse Matrices in ArrayFire!\r\n\r\n<h4>Sparse Data Structure</h4>\r\nA sparse data structure is one where all the non-zero elements are not stored. Sparse matrices are useful when the number of zero-values elements are &gt;&gt; number of non-zero elements (sparsity of the matrix is high).\r\nThe sparse data structure in general is stored as 3 arrays.\r\n<ul>\r\n<li>A data or values array containing all the non-zero elements</li>\r\n<li>A vector for row indices (based on storage format).</li>\r\n<li>A vector for column indices (based on storage format).</li>\r\n</ul>\r\n\r\nThere are many ways to store sparse matrices, the most prominent of which are\r\n<ul>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_row_.28CSR.2C_CRS_or_Yale_format.29\">Compressed Sparse Row</a></li>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_column_.28CSC_or_CCS.29\">Compressed Storage Column</a></li>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Coordinate_list_.28COO.29\">Coordinate Format</a>.</li>\r\n</ul>\r\n\r\nArrayFire supports CSR and COO format, with CSC support coming in future releases.\r\n\r\nA more detailed description of Sparse Matrix can be found on <a href=\"https://en.wikipedia.org/wiki/Sparse_matrix\">Wikipedia</a>.\r\n\r\n<h4>Sparse in ArrayFire</h4>\r\nIn ArrayFire, Sparse arrays use the same user facing object - af::array. This makes working with sparse arrays all the more seamless.\r\nWe have added a function <a href=\"http://arrayfire.org/docs/group__method__mat.htm#ga5945f95549bae25f75694a69d98ca584\">af::array::issparse()</a> that allows users to check if an array is sparse or not.\r\nAlthough externally, there is no difference between a sparse and dense array, internally, the sparse array is stored as 3 independent arrays along with other meta data.\r\nThese arrays, as well as other properties can be fetched using the <a href=\"http://arrayfire.org/docs/group__sparse__func.htm\">sparse functions</a>.\r\n\r\n<h4>Sparse Functionality</h4>\r\nAs of v3.4, the following features are supported for sparse arrays:\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm\">Creation for sparse arrays.</a>\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae4621e58bfaf6d80668309824fde86c0\">From 3 (data, row, column) sparse host or device memory vectors.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae9a843c7e93b05e6b090ca32b271ebe9\">From 3 (data, row, column) sparse af::arrays.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">From dense af::array.</a></li>\r\n<li>To create a sparse array from host or device dense array, first create a dense af::array, and then pass it to create sparse af::array from dense af::array.</li>\r\n</ul>\r\n</li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__convert__to.htm\">Storage format conversions.</a>\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">Dense to CSR/COO.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__dense.htm\">CSR/COO to dense.</a></li>\r\n</ul>\r\n</li>\r\n<li><a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">Sparse-Dense Matrix Multiplication (SPMM/SPMV)</a>\r\n<ul>\r\n<li>Sparse Matrix - Dense Matrix</li>\r\n<li>Sparse Matrix - Dense Vector</li>\r\n<li>Options for Sparse Matrix to be transposed.</li>\r\n</ul>\r\n</li>\r\n</ul>\r\n\r\nThe Sparse-Dense Matrix Multiplcation is a implemented as a part of <a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">af::matmul()</a>. This is part of making the usage of sparse arrays seamless with regular arrays.\r\n\r\n<h4>Example</h4>\r\n<pre>\r\n/*******************************************************\r\n * Copyright (c) 2014, ArrayFire\r\n * All rights reserved.\r\n *\r\n * This file is distributed under 3-clause BSD license.\r\n * The complete license agreement can be obtained at:\r\n * http://sandbox.arrayfire.com/licenses/BSD-3-Clause\r\n ********************************************************/\r\n#include <arrayfire.h>\r\n#include <iostream>\r\n\r\nusing namespace af;\r\n\r\nstatic size_t dimension = 16 * 1024;\r\nstatic const int maxIter = 10;\r\nstatic const int sparsityFactor = 7;\r\n\r\nstatic array A;\r\nstatic array spA;  // Sparse A\r\nstatic array x0;\r\nstatic array b;\r\n\r\nvoid setupInputs()\r\n{\r\n    // Generate a random input: A\r\n    array T = randu(dimension, dimension, f32);\r\n    // Create 0s in input.\r\n    // Anything that is no divisible by sparsityFactor will become 0.\r\n    A = floor(T * 1000);\r\n    A = A * ((A % sparsityFactor) == 0) / 1000;\r\n    // Make it positive definite\r\n    A = transpose(A) + A + A.dims(0)*identity(A.dims(0), A.dims(0), f32);\r\n\r\n    // Make A sparse as spA\r\n    spA = sparse(A);\r\n\r\n    // Generate x0: Random guess\r\n    x0 = randu(A.dims(0), f32);\r\n\r\n    //Generate b\r\n    b = matmul(A, x0);\r\n\r\n    // Run evals on inputs\r\n    A .eval();\r\n    sparseGetValues(spA).eval();\r\n    sparseGetRowIdx(spA).eval();\r\n    sparseGetColIdx(spA).eval();\r\n    x0.eval();\r\n    b .eval();\r\n    af::sync();\r\n}\r\n\r\nvoid conjugateGradient(const af::array in)\r\n{\r\n    array x = constant(0, b.dims(), f32);\r\n    array r = b - matmul(in, x);\r\n    array p = r;\r\n\r\n    for (int i = 0; i < maxIter; ++i) {\r\n        array Ap = matmul(in, p);\r\n        array alpha_num = dot(r, r);\r\n        array alpha_den = dot(p, Ap);\r\n        array alpha = alpha_num/alpha_den;\r\n        r = r - tile(alpha, Ap.dims())*Ap;\r\n        x = x + tile(alpha, Ap.dims())*p;\r\n        array beta_num = dot(r, r);\r\n        array beta = beta_num/alpha_num;\r\n        p = r + tile(beta, p.dims())*p;\r\n    }\r\n    array res = x0 - x;\r\n\r\n    std::cout<<\"Final difference in solutions:\\n\";\r\n    af_print(dot(res, res));\r\n}\r\n\r\nint main(int argc, char *argv[])\r\n{\r\n    af::info();\r\n    setupInputs();\r\n\r\n    std::cout << \"Verifying Dense Conjugate Gradient:\" << std::endl;\r\n    conjugateGradient(A);\r\n\r\n    std::cout << \"Verifying Sparse Conjugate Gradient:\" << std::endl;\r\n    conjugateGradient(spA);\r\n\r\n    return 0;\r\n}\r\n</pre>\r\n\r\n<h4>Performance and Memory Footprint</h4>\r\nWith a <code>sparsityFactor</code> of 7, the sparsity of A is about 26.38% (for CUDA backend). This means a tremendous amount of space and performance can be achieved by using sparse arrays.\r\nFor the CUDA backend benchmarking on a Tesla K40c, we achieved a time of 64.94ms for running the <code>conjugateGradient</code> using dense matrix and 39.31ms for sparse Matrix. That is almost a 2x speedup! Add in the fact that not all computation in the conjugate gradient function is sparse, the speedup for sparse vs dense matrix multiplication is extremely good.\r\n\r\nAs with all ArrayFire features, Sparse support is available across all backends and all architectures.','Sparse Matrices in ArrayFire','','inherit','closed','closed','','9190-revision-v1','','','2016-09-20 18:23:15','2016-09-20 22:23:15','',9190,'http://sandbox.arrayfire.com/9190-revision-v1/',0,'revision','',0),(9326,11,'2016-09-20 18:24:30','2016-09-20 22:24:30','With the release of <strong>ArrayFire v3.4</strong>, we have added a number of features that will help our vast range of users. Our users have been asking us for a way to reduce the memory footprint of their data on the GPU. And now we have answered our users\' requests.\r\nIntroducing Sparse Matrices in ArrayFire!\r\n\r\n<h4>Sparse Data Structure</h4>\r\nA sparse data structure is one where all the non-zero elements are not stored. Sparse matrices are useful when the number of zero-values elements are &gt;&gt; number of non-zero elements (sparsity of the matrix is high).\r\nThe sparse data structure in general is stored as 3 arrays.\r\n<ul>\r\n<li>A data or values array containing all the non-zero elements</li>\r\n<li>A vector for row indices (based on storage format).</li>\r\n<li>A vector for column indices (based on storage format).</li>\r\n</ul>\r\n\r\nThere are many ways to store sparse matrices, the most prominent of which are\r\n<ul>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_row_.28CSR.2C_CRS_or_Yale_format.29\">Compressed Sparse Row</a></li>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_column_.28CSC_or_CCS.29\">Compressed Storage Column</a></li>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Coordinate_list_.28COO.29\">Coordinate Format</a>.</li>\r\n</ul>\r\n\r\nArrayFire supports CSR and COO format, with CSC support coming in future releases.\r\n\r\nA more detailed description of Sparse Matrix can be found on <a href=\"https://en.wikipedia.org/wiki/Sparse_matrix\">Wikipedia</a>.\r\n\r\n<h4>Sparse in ArrayFire</h4>\r\nIn ArrayFire, Sparse arrays use the same user facing object - af::array. This makes working with sparse arrays all the more seamless.\r\nWe have added a function <a href=\"http://arrayfire.org/docs/group__method__mat.htm#ga5945f95549bae25f75694a69d98ca584\">af::array::issparse()</a> that allows users to check if an array is sparse or not.\r\nAlthough externally, there is no difference between a sparse and dense array, internally, the sparse array is stored as 3 independent arrays along with other meta data.\r\nThese arrays, as well as other properties can be fetched using the <a href=\"http://arrayfire.org/docs/group__sparse__func.htm\">sparse functions</a>.\r\n\r\n<h4>Sparse Functionality</h4>\r\nAs of v3.4, the following features are supported for sparse arrays:\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm\">Creation for sparse arrays.</a>\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae4621e58bfaf6d80668309824fde86c0\">From 3 (data, row, column) sparse host or device memory vectors.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae9a843c7e93b05e6b090ca32b271ebe9\">From 3 (data, row, column) sparse af::arrays.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">From dense af::array.</a></li>\r\n<li>To create a sparse array from host or device dense array, first create a dense af::array, and then pass it to create sparse af::array from dense af::array.</li>\r\n</ul>\r\n</li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__convert__to.htm\">Storage format conversions.</a>\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">Dense to CSR/COO.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__dense.htm\">CSR/COO to dense.</a></li>\r\n</ul>\r\n</li>\r\n<li><a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">Sparse-Dense Matrix Multiplication (SPMM/SPMV)</a>\r\n<ul>\r\n<li>Sparse Matrix - Dense Matrix</li>\r\n<li>Sparse Matrix - Dense Vector</li>\r\n<li>Options for Sparse Matrix to be transposed.</li>\r\n</ul>\r\n</li>\r\n</ul>\r\n\r\nThe Sparse-Dense Matrix Multiplcation is a implemented as a part of <a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">af::matmul()</a>. This is part of making the usage of sparse arrays seamless with regular arrays.\r\n\r\n<h4>Example</h4>\r\n<pre>\r\n/*******************************************************\r\n * Copyright (c) 2014, ArrayFire\r\n * All rights reserved.\r\n *\r\n * This file is distributed under 3-clause BSD license.\r\n * The complete license agreement can be obtained at:\r\n * http://sandbox.arrayfire.com/licenses/BSD-3-Clause\r\n ********************************************************/\r\n#include <arrayfire.h>\r\n#include <iostream>\r\n\r\nusing namespace af;\r\n\r\nstatic size_t dimension = 16 * 1024;\r\nstatic const int maxIter = 10;\r\nstatic const int sparsityFactor = 7;\r\n\r\nstatic array A;\r\nstatic array spA;  // Sparse A\r\nstatic array x0;\r\nstatic array b;\r\n\r\nvoid setupInputs()\r\n{\r\n    // Generate a random input: A\r\n    array T = randu(dimension, dimension, f32);\r\n    // Create 0s in input.\r\n    // Anything that is no divisible by sparsityFactor will become 0.\r\n    A = floor(T * 1000);\r\n    A = A * ((A % sparsityFactor) == 0) / 1000;\r\n    // Make it positive definite\r\n    A = transpose(A) + A + A.dims(0)*identity(A.dims(0), A.dims(0), f32);\r\n\r\n    // Make A sparse as spA\r\n    spA = sparse(A);\r\n\r\n    // Generate x0: Random guess\r\n    x0 = randu(A.dims(0), f32);\r\n\r\n    //Generate b\r\n    b = matmul(A, x0);\r\n\r\n    // Run evals on inputs\r\n    A .eval();\r\n    sparseGetValues(spA).eval();\r\n    sparseGetRowIdx(spA).eval();\r\n    sparseGetColIdx(spA).eval();\r\n    x0.eval();\r\n    b .eval();\r\n    af::sync();\r\n\r\n    std::cout << \"Sparsity of A = \" <<  100.f * (float)sparseGetNNZ(spA) / (float)spA.elements() << std::endl;\r\n}\r\n\r\nvoid conjugateGradient(const af::array in)\r\n{\r\n    array x = constant(0, b.dims(), f32);\r\n    array r = b - matmul(in, x);\r\n    array p = r;\r\n\r\n    for (int i = 0; i < maxIter; ++i) {\r\n        array Ap = matmul(in, p);\r\n        array alpha_num = dot(r, r);\r\n        array alpha_den = dot(p, Ap);\r\n        array alpha = alpha_num/alpha_den;\r\n        r = r - tile(alpha, Ap.dims())*Ap;\r\n        x = x + tile(alpha, Ap.dims())*p;\r\n        array beta_num = dot(r, r);\r\n        array beta = beta_num/alpha_num;\r\n        p = r + tile(beta, p.dims())*p;\r\n    }\r\n    array res = x0 - x;\r\n\r\n    std::cout<<\"Final difference in solutions:\\n\";\r\n    af_print(dot(res, res));\r\n}\r\n\r\nint main(int argc, char *argv[])\r\n{\r\n    af::info();\r\n    setupInputs();\r\n\r\n    std::cout << \"Verifying Dense Conjugate Gradient:\" << std::endl;\r\n    conjugateGradient(A);\r\n\r\n    std::cout << \"Verifying Sparse Conjugate Gradient:\" << std::endl;\r\n    conjugateGradient(spA);\r\n\r\n    return 0;\r\n}\r\n</pre>\r\n\r\n<h4>Performance and Memory Footprint</h4>\r\nWith a <code>sparsityFactor</code> of 7, the sparsity of A is about 26.38% (for CUDA backend). This means a tremendous amount of space and performance can be achieved by using sparse arrays.\r\nFor the CUDA backend benchmarking on a Tesla K40c, we achieved a time of 64.94ms for running the <code>conjugateGradient</code> using dense matrix and 39.31ms for sparse Matrix. That is almost a 2x speedup! Add in the fact that not all computation in the conjugate gradient function is sparse, the speedup for sparse vs dense matrix multiplication is extremely good.\r\n\r\nAs with all ArrayFire features, Sparse support is available across all backends and all architectures.','Sparse Matrices in ArrayFire','','inherit','closed','closed','','9190-revision-v1','','','2016-09-20 18:24:30','2016-09-20 22:24:30','',9190,'http://sandbox.arrayfire.com/9190-revision-v1/',0,'revision','',0),(9327,11,'2016-09-20 18:31:51','2016-09-20 22:31:51','With the release of <strong>ArrayFire v3.4</strong>, we have added a number of features that will help our vast range of users. Our users have been asking us for a way to reduce the memory footprint of their data on the GPU. And now we have answered our users\' requests.\r\nIntroducing Sparse Matrices in ArrayFire!\r\n\r\n<h4>Sparse Data Structure</h4>\r\nA sparse data structure is one where all the non-zero elements are not stored. Sparse matrices are useful when the number of zero-values elements are &gt;&gt; number of non-zero elements (sparsity of the matrix is high).\r\nThe sparse data structure in general is stored as 3 arrays.\r\n<ul>\r\n<li>A data or values array containing all the non-zero elements</li>\r\n<li>A vector for row indices (based on storage format).</li>\r\n<li>A vector for column indices (based on storage format).</li>\r\n</ul>\r\n\r\nThere are many ways to store sparse matrices, the most prominent of which are\r\n<ul>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_row_.28CSR.2C_CRS_or_Yale_format.29\">Compressed Sparse Row</a></li>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_column_.28CSC_or_CCS.29\">Compressed Storage Column</a></li>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Coordinate_list_.28COO.29\">Coordinate Format</a>.</li>\r\n</ul>\r\n\r\nArrayFire supports CSR and COO format, with CSC support coming in future releases.\r\n\r\nA more detailed description of Sparse Matrix can be found on <a href=\"https://en.wikipedia.org/wiki/Sparse_matrix\">Wikipedia</a>.\r\n\r\n<h4>Sparse in ArrayFire</h4>\r\nIn ArrayFire, Sparse arrays use the same user facing object - af::array. This makes working with sparse arrays all the more seamless.\r\nWe have added a function <a href=\"http://arrayfire.org/docs/group__method__mat.htm#ga5945f95549bae25f75694a69d98ca584\">af::array::issparse()</a> that allows users to check if an array is sparse or not.\r\nAlthough externally, there is no difference between a sparse and dense array, internally, the sparse array is stored as 3 independent arrays along with other meta data.\r\nThese arrays, as well as other properties can be fetched using the <a href=\"http://arrayfire.org/docs/group__sparse__func.htm\">sparse functions</a>.\r\n\r\n<h4>Sparse Functionality</h4>\r\nAs of v3.4, the following features are supported for sparse arrays:\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm\">Creation for sparse arrays.</a>\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae4621e58bfaf6d80668309824fde86c0\">From 3 (data, row, column) sparse host or device memory vectors.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae9a843c7e93b05e6b090ca32b271ebe9\">From 3 (data, row, column) sparse af::arrays.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">From dense af::array.</a></li>\r\n<li>To create a sparse array from host or device dense array, first create a dense af::array, and then pass it to create sparse af::array from dense af::array.</li>\r\n</ul>\r\n</li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__convert__to.htm\">Storage format conversions.</a>\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">Dense to CSR/COO.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__dense.htm\">CSR/COO to dense.</a></li>\r\n</ul>\r\n</li>\r\n<li><a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">Sparse-Dense Matrix Multiplication (SPMM/SPMV)</a>\r\n<ul>\r\n<li>Sparse Matrix - Dense Matrix</li>\r\n<li>Sparse Matrix - Dense Vector</li>\r\n<li>Options for Sparse Matrix to be transposed.</li>\r\n</ul>\r\n</li>\r\n</ul>\r\n\r\nThe Sparse-Dense Matrix Multiplcation is a implemented as a part of <a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">af::matmul()</a>. This is part of making the usage of sparse arrays seamless with regular arrays.\r\n\r\n<h4>Example</h4>\r\nThe example we shall use is a conjugate gradient solver. The inputs will be the same except that for the sparse conjugate gradient we will convert the A matrix to sparse.\r\n<pre>\r\n/*******************************************************\r\n * Copyright (c) 2014, ArrayFire\r\n * All rights reserved.\r\n *\r\n * This file is distributed under 3-clause BSD license.\r\n * The complete license agreement can be obtained at:\r\n * http://sandbox.arrayfire.com/licenses/BSD-3-Clause\r\n ********************************************************/\r\n#include <arrayfire.h>\r\n#include <iostream>\r\n\r\nusing namespace af;\r\n\r\nstatic size_t dimension = 16 * 1024;\r\nstatic const int maxIter = 10;\r\nstatic const int sparsityFactor = 7;\r\n\r\nstatic array A;\r\nstatic array spA;  // Sparse A\r\nstatic array x0;\r\nstatic array b;\r\n\r\nvoid setupInputs()\r\n{\r\n    // Generate a random input: A\r\n    array T = randu(dimension, dimension, f32);\r\n    // Create 0s in input.\r\n    // Anything that is no divisible by sparsityFactor will become 0.\r\n    A = floor(T * 1000);\r\n    A = A * ((A % sparsityFactor) == 0) / 1000;\r\n    // Make it positive definite\r\n    A = transpose(A) + A + A.dims(0)*identity(A.dims(0), A.dims(0), f32);\r\n\r\n    // Make A sparse as spA\r\n    spA = sparse(A);\r\n\r\n    // Generate x0: Random guess\r\n    x0 = randu(A.dims(0), f32);\r\n\r\n    //Generate b\r\n    b = matmul(A, x0);\r\n\r\n    // Run evals on inputs\r\n    A .eval();\r\n    sparseGetValues(spA).eval();\r\n    sparseGetRowIdx(spA).eval();\r\n    sparseGetColIdx(spA).eval();\r\n    x0.eval();\r\n    b .eval();\r\n    af::sync();\r\n\r\n    std::cout << \"Sparsity of A = \" <<  100.f * (float)sparseGetNNZ(spA) / (float)spA.elements() << std::endl;\r\n}\r\n\r\nvoid conjugateGradient(const af::array in)\r\n{\r\n    array x = constant(0, b.dims(), f32);\r\n    array r = b - matmul(in, x);\r\n    array p = r;\r\n\r\n    for (int i = 0; i < maxIter; ++i) {\r\n        array Ap = matmul(in, p);\r\n        array alpha_num = dot(r, r);\r\n        array alpha_den = dot(p, Ap);\r\n        array alpha = alpha_num/alpha_den;\r\n        r = r - tile(alpha, Ap.dims())*Ap;\r\n        x = x + tile(alpha, Ap.dims())*p;\r\n        array beta_num = dot(r, r);\r\n        array beta = beta_num/alpha_num;\r\n        p = r + tile(beta, p.dims())*p;\r\n    }\r\n    array res = x0 - x;\r\n\r\n    std::cout<<\"Final difference in solutions:\\n\";\r\n    af_print(dot(res, res));\r\n}\r\n\r\nint main(int argc, char *argv[])\r\n{\r\n    af::info();\r\n    setupInputs();\r\n\r\n    std::cout << \"Verifying Dense Conjugate Gradient:\" << std::endl;\r\n    conjugateGradient(A);\r\n\r\n    std::cout << \"Verifying Sparse Conjugate Gradient:\" << std::endl;\r\n    conjugateGradient(spA);\r\n\r\n    return 0;\r\n}\r\n</pre>\r\n\r\n<h4>Performance and Memory Footprint</h4>\r\nWith a <code>sparsityFactor</code> of 7, the sparsity of A is about 26.38% (for CUDA backend). This means a tremendous amount of space and performance can be achieved by using sparse arrays. For a matrix of size 16M x 16M of float type, this meant that the dense matrix use 1 GB of memory while the sparse array, stored in CSR format, only used 527 MB - halving the memory consumption!\r\nFor the CUDA backend benchmarking on a Tesla K40c, we achieved a time of 64.94ms for running the <code>conjugateGradient</code> using dense matrix and 39.31ms for sparse Matrix. That is almost a 2x speedup! Add in the fact that not all computation in the conjugate gradient function is sparse, the speedup for sparse vs dense matrix multiplication is extremely good.\r\n\r\nAs with all ArrayFire features, Sparse support is available across all backends and all architectures.','Sparse Matrices in ArrayFire','','inherit','closed','closed','','9190-revision-v1','','','2016-09-20 18:31:51','2016-09-20 22:31:51','',9190,'http://sandbox.arrayfire.com/9190-revision-v1/',0,'revision','',0),(9328,11,'2016-09-20 18:33:35','2016-09-20 22:33:35','With the release of <strong>ArrayFire v3.4</strong>, we have added a number of features that will help our vast range of users. Our users have been asking us for a way to reduce the memory footprint of their data on the GPU. And now we have answered our users\' requests.\r\nIntroducing Sparse Matrices in ArrayFire!\r\n\r\n<h4>Sparse Data Structure</h4>\r\nA sparse data structure is one where all the non-zero elements are not stored. Sparse matrices are useful when the number of zero-values elements are &gt;&gt; number of non-zero elements (sparsity of the matrix is high).\r\nThe sparse data structure in general is stored as 3 arrays.\r\n<ul>\r\n<li>A data or values array containing all the non-zero elements</li>\r\n<li>A vector for row indices (based on storage format).</li>\r\n<li>A vector for column indices (based on storage format).</li>\r\n</ul>\r\n\r\nThere are many ways to store sparse matrices, the most prominent of which are\r\n<ul>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_row_.28CSR.2C_CRS_or_Yale_format.29\">Compressed Sparse Row</a></li>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_column_.28CSC_or_CCS.29\">Compressed Storage Column</a></li>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Coordinate_list_.28COO.29\">Coordinate Format</a>.</li>\r\n</ul>\r\n\r\nArrayFire supports CSR and COO format, with CSC support coming in future releases.\r\n\r\nA more detailed description of Sparse Matrix can be found on <a href=\"https://en.wikipedia.org/wiki/Sparse_matrix\">Wikipedia</a>.\r\n\r\n<h4>Sparse in ArrayFire</h4>\r\nIn ArrayFire, Sparse arrays use the same user facing object - af::array. This makes working with sparse arrays all the more seamless.\r\nWe have added a function <a href=\"http://arrayfire.org/docs/group__method__mat.htm#ga5945f95549bae25f75694a69d98ca584\">af::array::issparse()</a> that allows users to check if an array is sparse or not.\r\nAlthough externally, there is no difference between a sparse and dense array, internally, the sparse array is stored as 3 independent arrays along with other meta data.\r\nThese arrays, as well as other properties can be fetched using the <a href=\"http://arrayfire.org/docs/group__sparse__func.htm\">sparse functions</a>.\r\n\r\n<h4>Sparse Functionality</h4>\r\nAs of v3.4, the following features are supported for sparse arrays:\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm\">Creation for sparse arrays.</a>\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae4621e58bfaf6d80668309824fde86c0\">From 3 (data, row, column) sparse host or device memory vectors.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae9a843c7e93b05e6b090ca32b271ebe9\">From 3 (data, row, column) sparse af::arrays.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">From dense af::array.</a></li>\r\n<li>To create a sparse array from host or device dense array, first create a dense af::array, and then pass it to create sparse af::array from dense af::array.</li>\r\n</ul>\r\n</li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__convert__to.htm\">Storage format conversions.</a>\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">Dense to CSR/COO.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__dense.htm\">CSR/COO to dense.</a></li>\r\n</ul>\r\n</li>\r\n<li><a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">Sparse-Dense Matrix Multiplication (SPMM/SPMV)</a>\r\n<ul>\r\n<li>Sparse Matrix - Dense Matrix</li>\r\n<li>Sparse Matrix - Dense Vector</li>\r\n<li>Options for Sparse Matrix to be transposed.</li>\r\n</ul>\r\n</li>\r\n</ul>\r\n\r\nThe Sparse-Dense Matrix Multiplcation is a implemented as a part of <a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">af::matmul()</a>. This is part of making the usage of sparse arrays seamless with regular arrays.\r\n\r\nAs with all ArrayFire features, Sparse support is available across all backends and all architectures.\r\n\r\n<h4>Example</h4>\r\nThe example we shall use is a conjugate gradient solver. The inputs will be the same except that for the sparse conjugate gradient we will convert the A matrix to sparse.\r\n<pre>\r\n/*******************************************************\r\n * Copyright (c) 2014, ArrayFire\r\n * All rights reserved.\r\n *\r\n * This file is distributed under 3-clause BSD license.\r\n * The complete license agreement can be obtained at:\r\n * http://sandbox.arrayfire.com/licenses/BSD-3-Clause\r\n ********************************************************/\r\n#include <arrayfire.h>\r\n#include <iostream>\r\n\r\nusing namespace af;\r\n\r\nstatic size_t dimension = 16 * 1024;\r\nstatic const int maxIter = 10;\r\nstatic const int sparsityFactor = 7;\r\n\r\nstatic array A;\r\nstatic array spA;  // Sparse A\r\nstatic array x0;\r\nstatic array b;\r\n\r\nvoid setupInputs()\r\n{\r\n    // Generate a random input: A\r\n    array T = randu(dimension, dimension, f32);\r\n    // Create 0s in input.\r\n    // Anything that is no divisible by sparsityFactor will become 0.\r\n    A = floor(T * 1000);\r\n    A = A * ((A % sparsityFactor) == 0) / 1000;\r\n    // Make it positive definite\r\n    A = transpose(A) + A + A.dims(0)*identity(A.dims(0), A.dims(0), f32);\r\n\r\n    // Make A sparse as spA\r\n    spA = sparse(A);\r\n\r\n    // Generate x0: Random guess\r\n    x0 = randu(A.dims(0), f32);\r\n\r\n    //Generate b\r\n    b = matmul(A, x0);\r\n\r\n    // Run evals on inputs\r\n    A .eval();\r\n    sparseGetValues(spA).eval();\r\n    sparseGetRowIdx(spA).eval();\r\n    sparseGetColIdx(spA).eval();\r\n    x0.eval();\r\n    b .eval();\r\n    af::sync();\r\n\r\n    std::cout << \"Sparsity of A = \" <<  100.f * (float)sparseGetNNZ(spA) / (float)spA.elements() << std::endl;\r\n}\r\n\r\nvoid conjugateGradient(const af::array in)\r\n{\r\n    array x = constant(0, b.dims(), f32);\r\n    array r = b - matmul(in, x);\r\n    array p = r;\r\n\r\n    for (int i = 0; i < maxIter; ++i) {\r\n        array Ap = matmul(in, p);\r\n        array alpha_num = dot(r, r);\r\n        array alpha_den = dot(p, Ap);\r\n        array alpha = alpha_num/alpha_den;\r\n        r = r - tile(alpha, Ap.dims())*Ap;\r\n        x = x + tile(alpha, Ap.dims())*p;\r\n        array beta_num = dot(r, r);\r\n        array beta = beta_num/alpha_num;\r\n        p = r + tile(beta, p.dims())*p;\r\n    }\r\n    array res = x0 - x;\r\n\r\n    std::cout<<\"Final difference in solutions:\\n\";\r\n    af_print(dot(res, res));\r\n}\r\n\r\nint main(int argc, char *argv[])\r\n{\r\n    af::info();\r\n    setupInputs();\r\n\r\n    std::cout << \"Verifying Dense Conjugate Gradient:\" << std::endl;\r\n    conjugateGradient(A);\r\n\r\n    std::cout << \"Verifying Sparse Conjugate Gradient:\" << std::endl;\r\n    conjugateGradient(spA);\r\n\r\n    return 0;\r\n}\r\n</pre>\r\n\r\n<h4>Performance and Memory Footprint</h4>\r\nFor the CUDA backend benchmarking on a Tesla K40c, we achieved a time of 64.94ms for running the <code>conjugateGradient</code> using dense matrix and 39.31ms for sparse Matrix. That is almost a 2x speedup! Add in the fact that not all computation in the conjugate gradient function is sparse, the speedup for sparse vs dense matrix multiplication is extremely good.\r\n\r\nWith a <code>sparsityFactor</code> of 7, the sparsity of A is about 26.38% (for CUDA backend). This means a tremendous amount of space and performance can be achieved by using sparse arrays. For a matrix of size 16M x 16M of float type, this meant that the dense matrix use 1 GB of memory while the sparse array, stored in CSR format, only used 527 MB - halving the memory consumption!','Sparse Matrices in ArrayFire','','inherit','closed','closed','','9190-revision-v1','','','2016-09-20 18:33:35','2016-09-20 22:33:35','',9190,'http://sandbox.arrayfire.com/9190-revision-v1/',0,'revision','',0),(9329,18673,'2016-09-20 18:35:46','2016-09-20 22:35:46','','rot_2','','inherit','open','closed','','rot_2','','','2016-09-20 18:35:46','2016-09-20 22:35:46','',9166,'http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_2.png',0,'attachment','image/png',0),(9330,18673,'2016-09-20 18:35:47','2016-09-20 22:35:47','','rot_3','','inherit','open','closed','','rot_3','','','2016-09-20 18:35:47','2016-09-20 22:35:47','',9166,'http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_3.png',0,'attachment','image/png',0),(9331,18673,'2016-09-20 18:35:48','2016-09-20 22:35:48','','s_0','3x image scaling using nearest-neighbor sampling','inherit','open','closed','','s_0','','','2016-09-20 18:52:11','2016-09-20 22:52:11','',9166,'http://sandbox.arrayfire.com/wp-content/uploads/2016/09/s_0.png',0,'attachment','image/png',0),(9332,18673,'2016-09-20 18:35:50','2016-09-20 22:35:50','','s_1','3x image scaling using bilinear-cosine sampling','inherit','open','closed','','s_1','','','2016-09-20 18:52:07','2016-09-20 22:52:07','',9166,'http://sandbox.arrayfire.com/wp-content/uploads/2016/09/s_1.png',0,'attachment','image/png',0),(9333,11,'2016-09-20 18:35:52','2016-09-20 22:35:52','With the release of <strong>ArrayFire v3.4</strong>, we have added a number of features that will help our vast range of users. Our users have been asking us for a way to reduce the memory footprint of their data on the GPU. And now we have answered our users\' requests.\r\nIntroducing Sparse Matrices in ArrayFire!\r\n\r\n<h4>Sparse Data Structure</h4>\r\nA sparse data structure is one where all the non-zero elements are not stored. Sparse matrices are useful when the number of zero-values elements are &gt;&gt; number of non-zero elements (sparsity of the matrix is high).\r\nThe sparse data structure in general is stored as 3 arrays.\r\n<ul>\r\n<li>A data or values array containing all the non-zero elements</li>\r\n<li>A vector for row indices (based on storage format).</li>\r\n<li>A vector for column indices (based on storage format).</li>\r\n</ul>\r\n\r\nThere are many ways to store sparse matrices, the most prominent of which are\r\n<ul>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_row_.28CSR.2C_CRS_or_Yale_format.29\">Compressed Sparse Row</a></li>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_column_.28CSC_or_CCS.29\">Compressed Storage Column</a></li>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Coordinate_list_.28COO.29\">Coordinate Format</a>.</li>\r\n</ul>\r\n\r\nArrayFire supports CSR and COO format, with CSC support coming in future releases.\r\n\r\nA more detailed description of Sparse Matrix can be found on <a href=\"https://en.wikipedia.org/wiki/Sparse_matrix\">Wikipedia</a>.\r\n\r\n<h4>Sparse in ArrayFire</h4>\r\nIn ArrayFire, Sparse arrays use the same user facing object - af::array. This makes working with sparse arrays all the more seamless.\r\nWe have added a function <a href=\"http://arrayfire.org/docs/group__method__mat.htm#ga5945f95549bae25f75694a69d98ca584\">af::array::issparse()</a> that allows users to check if an array is sparse or not.\r\nAlthough externally, there is no difference between a sparse and dense array, internally, the sparse array is stored as 3 independent arrays along with other meta data.\r\nThese arrays, as well as other properties can be fetched using the <a href=\"http://arrayfire.org/docs/group__sparse__func.htm\">sparse functions</a>.\r\n\r\n<h4>Sparse Functionality</h4>\r\nAs of v3.4, the following features are supported for sparse arrays:\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm\">Creation for sparse arrays.</a>\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae4621e58bfaf6d80668309824fde86c0\">From 3 (data, row, column) sparse host or device memory vectors.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae9a843c7e93b05e6b090ca32b271ebe9\">From 3 (data, row, column) sparse af::arrays.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">From dense af::array.</a></li>\r\n<li>To create a sparse array from host or device dense array, first create a dense af::array, and then pass it to create sparse af::array from dense af::array.</li>\r\n</ul>\r\n</li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__convert__to.htm\">Storage format conversions.</a>\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">Dense to CSR/COO.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__dense.htm\">CSR/COO to dense.</a></li>\r\n</ul>\r\n</li>\r\n<li><a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">Sparse-Dense Matrix Multiplication (SPMM/SPMV)</a>\r\n<ul>\r\n<li>Sparse Matrix - Dense Matrix</li>\r\n<li>Sparse Matrix - Dense Vector</li>\r\n<li>Options for Sparse Matrix to be transposed.</li>\r\n</ul>\r\n</li>\r\n</ul>\r\n\r\nThe Sparse-Dense Matrix Multiplcation is a implemented as a part of <a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">af::matmul()</a>. This is part of making the usage of sparse arrays seamless with regular arrays.\r\n\r\nAs with all ArrayFire features, Sparse support is available across all backends and all architectures.\r\n\r\n<h4>Example</h4>\r\nThe example we shall use is a conjugate gradient solver. The inputs will be the same except that for the sparse conjugate gradient we will convert the A matrix to sparse.\r\n<pre>\r\n/*******************************************************\r\n * Copyright (c) 2014, ArrayFire\r\n * All rights reserved.\r\n *\r\n * This file is distributed under 3-clause BSD license.\r\n * The complete license agreement can be obtained at:\r\n * http://sandbox.arrayfire.com/licenses/BSD-3-Clause\r\n ********************************************************/\r\n#include <arrayfire.h>\r\n#include <iostream>\r\n\r\nusing namespace af;\r\n\r\nstatic size_t dimension = 16 * 1024;\r\nstatic const int maxIter = 10;\r\nstatic const int sparsityFactor = 7;\r\n\r\nstatic array A;\r\nstatic array spA;  // Sparse A\r\nstatic array x0;\r\nstatic array b;\r\n\r\nvoid setupInputs()\r\n{\r\n    // Generate a random input: A\r\n    array T = randu(dimension, dimension, f32);\r\n    // Create 0s in input.\r\n    // Anything that is no divisible by sparsityFactor will become 0.\r\n    A = floor(T * 1000);\r\n    A = A * ((A % sparsityFactor) == 0) / 1000;\r\n    // Make it positive definite\r\n    A = transpose(A) + A + A.dims(0)*identity(A.dims(0), A.dims(0), f32);\r\n\r\n    // Make A sparse as spA\r\n    spA = sparse(A);\r\n\r\n    // Generate x0: Random guess\r\n    x0 = randu(A.dims(0), f32);\r\n\r\n    //Generate b\r\n    b = matmul(A, x0);\r\n\r\n    // Run evals on inputs\r\n    A .eval();\r\n    sparseGetValues(spA).eval();\r\n    sparseGetRowIdx(spA).eval();\r\n    sparseGetColIdx(spA).eval();\r\n    x0.eval();\r\n    b .eval();\r\n    af::sync();\r\n\r\n    std::cout << \"Sparsity of A = \" <<  100.f * (float)sparseGetNNZ(spA) / (float)spA.elements() << std::endl;\r\n}\r\n\r\nvoid conjugateGradient(const af::array in)\r\n{\r\n    array x = constant(0, b.dims(), f32);\r\n    array r = b - matmul(in, x);\r\n    array p = r;\r\n\r\n    for (int i = 0; i < maxIter; ++i) {\r\n        array Ap = matmul(in, p);\r\n        array alpha_num = dot(r, r);\r\n        array alpha_den = dot(p, Ap);\r\n        array alpha = alpha_num/alpha_den;\r\n        r = r - tile(alpha, Ap.dims())*Ap;\r\n        x = x + tile(alpha, Ap.dims())*p;\r\n        array beta_num = dot(r, r);\r\n        array beta = beta_num/alpha_num;\r\n        p = r + tile(beta, p.dims())*p;\r\n    }\r\n    array res = x0 - x;\r\n\r\n    std::cout<<\"Final difference in solutions:\\n\";\r\n    af_print(dot(res, res));\r\n}\r\n\r\nint main(int argc, char *argv[])\r\n{\r\n    af::info();\r\n    setupInputs();\r\n\r\n    std::cout << \"Verifying Dense Conjugate Gradient:\" << std::endl;\r\n    conjugateGradient(A);\r\n\r\n    std::cout << \"Verifying Sparse Conjugate Gradient:\" << std::endl;\r\n    conjugateGradient(spA);\r\n\r\n    return 0;\r\n}\r\n</pre>\r\n\r\n<h4>Performance and Memory Footprint</h4>\r\nUsing the CUDA backend on a Tesla K40c, the conjugate gradient function took a time of 64.94ms for dense matrix and 39.31ms for sparse Matrix. That is almost a 2x speedup! Add in the fact that not all computation in the conjugate gradient function is sparse, the speedup for sparse vs dense matrix multiplication is extremely good.\r\n\r\nWith a <code>sparsityFactor</code> of 7, the sparsity of A is about 26.38% (for CUDA backend). This means a tremendous amount of space and performance can be achieved by using sparse arrays. For a matrix of size 16M x 16M of float type, this meant that the dense matrix use 1 GB of memory while the sparse array, stored in CSR format, only used 527 MB - halving the memory consumption!','Sparse Matrices in ArrayFire','','inherit','closed','closed','','9190-revision-v1','','','2016-09-20 18:35:52','2016-09-20 22:35:52','',9190,'http://sandbox.arrayfire.com/9190-revision-v1/',0,'revision','',0),(9334,18673,'2016-09-20 18:35:53','2016-09-20 22:35:53','','s_2','3x image scaling using bicubic sampling','inherit','open','closed','','s_2','','','2016-09-20 18:52:05','2016-09-20 22:52:05','',9166,'http://sandbox.arrayfire.com/wp-content/uploads/2016/09/s_2.png',0,'attachment','image/png',0),(9335,18673,'2016-09-20 18:35:54','2016-09-20 22:35:54','','rot_0','','inherit','open','closed','','rot_0','','','2016-09-20 18:35:54','2016-09-20 22:35:54','',9166,'http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_0.png',0,'attachment','image/png',0),(9336,18673,'2016-09-20 18:35:56','2016-09-20 22:35:56','','rot_1','','inherit','open','closed','','rot_1','','','2016-09-20 18:35:56','2016-09-20 22:35:56','',9166,'http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_1.png',0,'attachment','image/png',0),(9337,18673,'2016-09-20 18:36:18','2016-09-20 22:36:18','','ap1_3','Cubic interpolation','inherit','open','closed','','ap1_3','','','2016-09-20 18:36:42','2016-09-20 22:36:42','',9166,'http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_3.png',0,'attachment','image/png',0),(9338,18673,'2016-09-20 18:36:19','2016-09-20 22:36:19','','ap1_0','Original data points','inherit','open','closed','','ap1_0','','','2016-09-20 18:36:33','2016-09-20 22:36:33','',9166,'http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_0.png',0,'attachment','image/png',0),(9339,18673,'2016-09-20 18:36:19','2016-09-20 22:36:19','','ap1_1','Linear interpolation','inherit','open','closed','','ap1_1','','','2016-09-20 18:36:51','2016-09-20 22:36:51','',9166,'http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_1.png',0,'attachment','image/png',0),(9340,18673,'2016-09-20 18:36:20','2016-09-20 22:36:20','','ap1_2','Linear cosine interpolation','inherit','open','closed','','ap1_2','','','2016-09-20 18:37:11','2016-09-20 22:37:11','',9166,'http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_2.png',0,'attachment','image/png',0),(9341,11,'2016-09-20 18:36:22','2016-09-20 22:36:22','With the release of <strong>ArrayFire v3.4</strong>, we have added a number of features that will help our vast range of users. Our users have been asking us for a way to reduce the memory footprint of their data on the GPU. And now we have answered our users\' requests.\r\nIntroducing Sparse Matrices in ArrayFire!\r\n\r\n<h4>Sparse Data Structure</h4>\r\nA sparse data structure is one where all the non-zero elements are not stored. Sparse matrices are useful when the number of zero-values elements are &gt;&gt; number of non-zero elements (sparsity of the matrix is high).\r\nThe sparse data structure in general is stored as 3 arrays.\r\n<ul>\r\n<li>A data or values array containing all the non-zero elements</li>\r\n<li>A vector for row indices (based on storage format).</li>\r\n<li>A vector for column indices (based on storage format).</li>\r\n</ul>\r\n\r\nThere are many ways to store sparse matrices, the most prominent of which are\r\n<ul>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_row_.28CSR.2C_CRS_or_Yale_format.29\">Compressed Sparse Row</a></li>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_column_.28CSC_or_CCS.29\">Compressed Storage Column</a></li>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Coordinate_list_.28COO.29\">Coordinate Format</a>.</li>\r\n</ul>\r\n\r\nArrayFire supports CSR and COO format, with CSC support coming in future releases.\r\n\r\nA more detailed description of Sparse Matrix can be found on <a href=\"https://en.wikipedia.org/wiki/Sparse_matrix\">Wikipedia</a>.\r\n\r\n<h4>Sparse in ArrayFire</h4>\r\nIn ArrayFire, Sparse arrays use the same user facing object - af::array. This makes working with sparse arrays all the more seamless.\r\nWe have added a function <a href=\"http://arrayfire.org/docs/group__method__mat.htm#ga5945f95549bae25f75694a69d98ca584\">af::array::issparse()</a> that allows users to check if an array is sparse or not.\r\nAlthough externally, there is no difference between a sparse and dense array, internally, the sparse array is stored as 3 independent arrays along with other meta data.\r\nThese arrays, as well as other properties can be fetched using the <a href=\"http://arrayfire.org/docs/group__sparse__func.htm\">sparse functions</a>.\r\n\r\n<h4>Sparse Functionality</h4>\r\nAs of v3.4, the following features are supported for sparse arrays:\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm\">Creation for sparse arrays.</a>\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae4621e58bfaf6d80668309824fde86c0\">From 3 (data, row, column) sparse host or device memory vectors.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae9a843c7e93b05e6b090ca32b271ebe9\">From 3 (data, row, column) sparse af::arrays.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">From dense af::array.</a></li>\r\n<li>To create a sparse array from host or device dense array, first create a dense af::array, and then pass it to create sparse af::array from dense af::array.</li>\r\n</ul>\r\n</li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__convert__to.htm\">Storage format conversions.</a>\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">Dense to CSR/COO.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__dense.htm\">CSR/COO to dense.</a></li>\r\n</ul>\r\n</li>\r\n<li><a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">Sparse-Dense Matrix Multiplication (SPMM/SPMV)</a>\r\n<ul>\r\n<li>Sparse Matrix - Dense Matrix</li>\r\n<li>Sparse Matrix - Dense Vector</li>\r\n<li>Options for Sparse Matrix to be transposed.</li>\r\n</ul>\r\n</li>\r\n</ul>\r\n\r\nThe Sparse-Dense Matrix Multiplcation is a implemented as a part of <a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">af::matmul()</a>. This is part of making the usage of sparse arrays seamless with regular arrays.\r\n\r\nAs with all ArrayFire features, Sparse support is available across all backends and all architectures.\r\n\r\n<h4>Example</h4>\r\nThe example we shall use is a conjugate gradient solver. The inputs will be the same except that for the sparse conjugate gradient we will convert the A matrix to sparse.\r\n<pre>\r\n/*******************************************************\r\n * Copyright (c) 2014, ArrayFire\r\n * All rights reserved.\r\n *\r\n * This file is distributed under 3-clause BSD license.\r\n * The complete license agreement can be obtained at:\r\n * http://sandbox.arrayfire.com/licenses/BSD-3-Clause\r\n ********************************************************/\r\n#include <arrayfire.h>\r\n#include <iostream>\r\n\r\nusing namespace af;\r\n\r\nstatic size_t dimension = 16 * 1024;\r\nstatic const int maxIter = 10;\r\nstatic const int sparsityFactor = 7;\r\n\r\nstatic array A;\r\nstatic array spA;  // Sparse A\r\nstatic array x0;\r\nstatic array b;\r\n\r\nvoid setupInputs()\r\n{\r\n    // Generate a random input: A\r\n    array T = randu(dimension, dimension, f32);\r\n    // Create 0s in input.\r\n    // Anything that is no divisible by sparsityFactor will become 0.\r\n    A = floor(T * 1000);\r\n    A = A * ((A % sparsityFactor) == 0) / 1000;\r\n    // Make it positive definite\r\n    A = transpose(A) + A + A.dims(0)*identity(A.dims(0), A.dims(0), f32);\r\n\r\n    // Make A sparse as spA\r\n    spA = sparse(A);\r\n\r\n    // Generate x0: Random guess\r\n    x0 = randu(A.dims(0), f32);\r\n\r\n    //Generate b\r\n    b = matmul(A, x0);\r\n\r\n    // Run evals on inputs\r\n    A .eval();\r\n    sparseGetValues(spA).eval();\r\n    sparseGetRowIdx(spA).eval();\r\n    sparseGetColIdx(spA).eval();\r\n    x0.eval();\r\n    b .eval();\r\n    af::sync();\r\n\r\n    std::cout << \"Sparsity of A = \" <<  100.f * (float)sparseGetNNZ(spA) / (float)spA.elements() << std::endl;\r\n}\r\n\r\nvoid conjugateGradient(const af::array in)\r\n{\r\n    array x = constant(0, b.dims(), f32);\r\n    array r = b - matmul(in, x);\r\n    array p = r;\r\n\r\n    for (int i = 0; i < maxIter; ++i) {\r\n        array Ap = matmul(in, p);\r\n        array alpha_num = dot(r, r);\r\n        array alpha_den = dot(p, Ap);\r\n        array alpha = alpha_num/alpha_den;\r\n        r = r - tile(alpha, Ap.dims())*Ap;\r\n        x = x + tile(alpha, Ap.dims())*p;\r\n        array beta_num = dot(r, r);\r\n        array beta = beta_num/alpha_num;\r\n        p = r + tile(beta, p.dims())*p;\r\n    }\r\n    array res = x0 - x;\r\n\r\n    std::cout<<\"Final difference in solutions:\\n\";\r\n    af_print(dot(res, res));\r\n}\r\n\r\nint main(int argc, char *argv[])\r\n{\r\n    af::info();\r\n    setupInputs();\r\n\r\n    std::cout << \"Verifying Dense Conjugate Gradient:\" << std::endl;\r\n    conjugateGradient(A);\r\n\r\n    std::cout << \"Verifying Sparse Conjugate Gradient:\" << std::endl;\r\n    conjugateGradient(spA);\r\n\r\n    return 0;\r\n}\r\n</pre>\r\n\r\n<h4>Performance and Memory Footprint</h4>\r\nUsing the CUDA backend on a Tesla K40c, the conjugate gradient function took a time of 64.94ms for dense matrix and 39.31ms for sparse Matrix. That is almost a 2x speedup! Add in the fact that not all computation in the conjugate gradient function is sparse, the speedup for sparse vs dense matrix multiplication is extremely good.\r\n\r\nWith a <code>sparsityFactor</code> of 7, the sparsity of A is about 26.38%. This means a tremendous amount of space and performance can be achieved by using sparse arrays. For a matrix of size 16M x 16M of float type, this meant that the dense matrix use 1 GB of memory while the sparse array, stored in CSR format, only used 527 MB - halving the memory consumption!','Sparse Matrices in ArrayFire','','inherit','closed','closed','','9190-revision-v1','','','2016-09-20 18:36:22','2016-09-20 22:36:22','',9190,'http://sandbox.arrayfire.com/9190-revision-v1/',0,'revision','',0),(9342,11,'2016-09-20 18:36:52','2016-09-20 22:36:52','With the release of <strong>ArrayFire v3.4</strong>, we have added a number of features that will help our vast range of users. Our users have been asking us for a way to reduce the memory footprint of their data on the GPU. And now we have answered our users\' requests.\r\nIntroducing Sparse Matrices in ArrayFire!\r\n\r\n<h4>Sparse Data Structure</h4>\r\nA sparse data structure is one where all the non-zero elements are not stored. Sparse matrices are useful when the number of zero-values elements are &gt;&gt; number of non-zero elements (sparsity of the matrix is high).\r\nThe sparse data structure in general is stored as 3 arrays.\r\n<ul>\r\n<li>A data or values array containing all the non-zero elements</li>\r\n<li>A vector for row indices (based on storage format).</li>\r\n<li>A vector for column indices (based on storage format).</li>\r\n</ul>\r\n\r\nThere are many ways to store sparse matrices, the most prominent of which are\r\n<ul>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_row_.28CSR.2C_CRS_or_Yale_format.29\">Compressed Sparse Row</a></li>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_column_.28CSC_or_CCS.29\">Compressed Storage Column</a></li>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Coordinate_list_.28COO.29\">Coordinate Format</a>.</li>\r\n</ul>\r\n\r\nArrayFire supports CSR and COO format, with CSC support coming in future releases.\r\n\r\nA more detailed description of Sparse Matrix can be found on <a href=\"https://en.wikipedia.org/wiki/Sparse_matrix\">Wikipedia</a>.\r\n\r\n<h4>Sparse in ArrayFire</h4>\r\nIn ArrayFire, Sparse arrays use the same user facing object - af::array. This makes working with sparse arrays all the more seamless.\r\nWe have added a function <a href=\"http://arrayfire.org/docs/group__method__mat.htm#ga5945f95549bae25f75694a69d98ca584\">af::array::issparse()</a> that allows users to check if an array is sparse or not.\r\nAlthough externally, there is no difference between a sparse and dense array, internally, the sparse array is stored as 3 independent arrays along with other meta data.\r\nThese arrays, as well as other properties can be fetched using the <a href=\"http://arrayfire.org/docs/group__sparse__func.htm\">sparse functions</a>.\r\n\r\n<h4>Sparse Functionality</h4>\r\nAs of v3.4, the following features are supported for sparse arrays:\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm\">Creation for sparse arrays.</a>\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae4621e58bfaf6d80668309824fde86c0\">From 3 (data, row, column) sparse host or device memory vectors.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae9a843c7e93b05e6b090ca32b271ebe9\">From 3 (data, row, column) sparse af::arrays.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">From dense af::array.</a></li>\r\n<li>To create a sparse array from host or device dense array, first create a dense af::array, and then pass it to create sparse af::array from dense af::array.</li>\r\n</ul>\r\n</li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__convert__to.htm\">Storage format conversions.</a>\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">Dense to CSR/COO.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__dense.htm\">CSR/COO to dense.</a></li>\r\n</ul>\r\n</li>\r\n<li><a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">Sparse-Dense Matrix Multiplication (SPMM/SPMV)</a>\r\n<ul>\r\n<li>Sparse Matrix - Dense Matrix</li>\r\n<li>Sparse Matrix - Dense Vector</li>\r\n<li>Options for Sparse Matrix to be transposed.</li>\r\n</ul>\r\n</li>\r\n</ul>\r\n\r\nThe Sparse-Dense Matrix Multiplcation is a implemented as a part of <a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">af::matmul()</a>. This is part of making the usage of sparse arrays seamless with regular arrays.\r\n\r\nAs with all ArrayFire features, Sparse support is available across all backends and all architectures.\r\n\r\n<h4>Example</h4>\r\nThe example we shall use is a conjugate gradient solver. The inputs will be the same except that for the sparse conjugate gradient we will convert the A matrix to sparse.\r\n<pre>\r\n/*******************************************************\r\n * Copyright (c) 2014, ArrayFire\r\n * All rights reserved.\r\n *\r\n * This file is distributed under 3-clause BSD license.\r\n * The complete license agreement can be obtained at:\r\n * http://sandbox.arrayfire.com/licenses/BSD-3-Clause\r\n ********************************************************/\r\n#include <arrayfire.h>\r\n#include <iostream>\r\n\r\nusing namespace af;\r\n\r\nstatic size_t dimension = 16 * 1024;\r\nstatic const int maxIter = 10;\r\nstatic const int sparsityFactor = 7;\r\n\r\nstatic array A;\r\nstatic array spA;  // Sparse A\r\nstatic array x0;\r\nstatic array b;\r\n\r\nvoid setupInputs()\r\n{\r\n    // Generate a random input: A\r\n    array T = randu(dimension, dimension, f32);\r\n    // Create 0s in input.\r\n    // Anything that is no divisible by sparsityFactor will become 0.\r\n    A = floor(T * 1000);\r\n    A = A * ((A % sparsityFactor) == 0) / 1000;\r\n    // Make it positive definite\r\n    A = transpose(A) + A + A.dims(0)*identity(A.dims(0), A.dims(0), f32);\r\n\r\n    // Make A sparse as spA\r\n    spA = sparse(A);\r\n\r\n    // Generate x0: Random guess\r\n    x0 = randu(A.dims(0), f32);\r\n\r\n    //Generate b\r\n    b = matmul(A, x0);\r\n\r\n    // Run evals on inputs\r\n    A .eval();\r\n    sparseGetValues(spA).eval();\r\n    sparseGetRowIdx(spA).eval();\r\n    sparseGetColIdx(spA).eval();\r\n    x0.eval();\r\n    b .eval();\r\n    af::sync();\r\n\r\n    std::cout << \"Sparsity of A = \" <<  100.f * (float)sparseGetNNZ(spA) / (float)spA.elements() << std::endl;\r\n}\r\n\r\nvoid conjugateGradient(const af::array in)\r\n{\r\n    array x = constant(0, b.dims(), f32);\r\n    array r = b - matmul(in, x);\r\n    array p = r;\r\n\r\n    for (int i = 0; i < maxIter; ++i) {\r\n        array Ap = matmul(in, p);\r\n        array alpha_num = dot(r, r);\r\n        array alpha_den = dot(p, Ap);\r\n        array alpha = alpha_num/alpha_den;\r\n        r = r - tile(alpha, Ap.dims())*Ap;\r\n        x = x + tile(alpha, Ap.dims())*p;\r\n        array beta_num = dot(r, r);\r\n        array beta = beta_num/alpha_num;\r\n        p = r + tile(beta, p.dims())*p;\r\n    }\r\n    array res = x0 - x;\r\n\r\n    std::cout<<\"Final difference in solutions:\\n\";\r\n    af_print(dot(res, res));\r\n}\r\n\r\nint main(int argc, char *argv[])\r\n{\r\n    af::info();\r\n    setupInputs();\r\n\r\n    std::cout << \"Verifying Dense Conjugate Gradient:\" << std::endl;\r\n    conjugateGradient(A);\r\n\r\n    std::cout << \"Verifying Sparse Conjugate Gradient:\" << std::endl;\r\n    conjugateGradient(spA);\r\n\r\n    return 0;\r\n}\r\n</pre>\r\n\r\n<h4>Performance and Memory Footprint</h4>\r\nUsing the CUDA backend on a Tesla K40c, the conjugate gradient function took a time of 64.94ms for dense matrix and 39.31ms for sparse Matrix. That is almost a 2x speedup! Add in the fact that not all computation in the conjugate gradient function is sparse, the speedup for sparse vs dense matrix multiplication is extremely good.\r\n\r\nWith a <code>sparsityFactor</code> of 7, the sparsity of A is about 26.38%. This means a tremendous amount of space and performance can be achieved by using sparse arrays. For a floating point matrix of size 16M x 16M, a dense matrix uses 1 GB of memory while the sparse array, stored in CSR format, only used 527 MB - halving the memory consumption!','Sparse Matrices in ArrayFire','','inherit','closed','closed','','9190-revision-v1','','','2016-09-20 18:36:52','2016-09-20 22:36:52','',9190,'http://sandbox.arrayfire.com/9190-revision-v1/',0,'revision','',0),(9343,11,'2016-09-20 18:41:10','2016-09-20 22:41:10','With the release of <strong>ArrayFire v3.4</strong>, we have added a number of features that will help our vast range of users. Our users have been asking us for a way to reduce the memory footprint of their data on the GPU. And now we have answered our users\' requests.\r\nIntroducing Sparse Matrices in ArrayFire!\r\n\r\n<h4>Sparse Data Structure</h4>\r\nA sparse data structure is one where all the non-zero elements are not stored. Sparse matrices are useful when the number of zero-values elements are &gt;&gt; number of non-zero elements (sparsity of the matrix is high).\r\nThe sparse data structure in general is stored as 3 arrays.\r\n<ul>\r\n<li>A data or values array containing all the non-zero elements</li>\r\n<li>A vector for row indices (based on storage format).</li>\r\n<li>A vector for column indices (based on storage format).</li>\r\n</ul>\r\n\r\nThere are many ways to store sparse matrices, the most prominent of which are\r\n<ul>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_row_.28CSR.2C_CRS_or_Yale_format.29\">Compressed Sparse Row</a></li>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_column_.28CSC_or_CCS.29\">Compressed Storage Column</a></li>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Coordinate_list_.28COO.29\">Coordinate Format</a>.</li>\r\n</ul>\r\n\r\nArrayFire supports CSR and COO format, with CSC support coming in future releases.\r\n\r\nA more detailed description of Sparse Matrix can be found on <a href=\"https://en.wikipedia.org/wiki/Sparse_matrix\">Wikipedia</a>.\r\n\r\n<h4>Sparse in ArrayFire</h4>\r\nIn ArrayFire, Sparse arrays use the same user facing object - af::array. This makes working with sparse arrays all the more seamless.\r\nWe have added a function <a href=\"http://arrayfire.org/docs/group__method__mat.htm#ga5945f95549bae25f75694a69d98ca584\">af::array::issparse()</a> that allows users to check if an array is sparse or not.\r\nAlthough externally, there is no difference between a sparse and dense array, internally, the sparse array is stored as 3 independent arrays along with other meta data.\r\nThese arrays, as well as other properties can be fetched using the <a href=\"http://arrayfire.org/docs/group__sparse__func.htm\">sparse functions</a>.\r\n\r\n<h4>Sparse Functionality</h4>\r\nAs of v3.4, the following features are supported for sparse arrays:\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm\">Creation for sparse arrays.</a>\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae4621e58bfaf6d80668309824fde86c0\">From 3 (data, row, column) sparse host or device memory vectors.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae9a843c7e93b05e6b090ca32b271ebe9\">From 3 (data, row, column) sparse af::arrays.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">From dense af::array.</a></li>\r\n<li>To create a sparse array from host or device dense array, first create a dense af::array, and then pass it to create sparse af::array from dense af::array.</li>\r\n</ul>\r\n</li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__convert__to.htm\">Storage format conversions.</a>\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">Dense to CSR/COO.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__dense.htm\">CSR/COO to dense.</a></li>\r\n</ul>\r\n</li>\r\n<li><a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">Sparse-Dense Matrix Multiplication (SPMM/SPMV)</a>\r\n<ul>\r\n<li>Sparse Matrix - Dense Matrix</li>\r\n<li>Sparse Matrix - Dense Vector</li>\r\n<li>Options for Sparse Matrix to be transposed.</li>\r\n</ul>\r\n</li>\r\n</ul>\r\n\r\nThe Sparse-Dense Matrix Multiplcation is a implemented as a part of <a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">af::matmul()</a>. This is part of making the usage of sparse arrays seamless with regular arrays.\r\n\r\nAs with all ArrayFire features, Sparse support is available across all backends and all architectures.\r\n\r\n<h4>Example</h4>\r\nThe example we shall use is a conjugate gradient solver. The inputs will be the same except that for the sparse conjugate gradient we will convert the A matrix to sparse.\r\n<pre>\r\n/*******************************************************\r\n * Copyright (c) 2014, ArrayFire\r\n * All rights reserved.\r\n *\r\n * This file is distributed under 3-clause BSD license.\r\n * The complete license agreement can be obtained at:\r\n * http://sandbox.arrayfire.com/licenses/BSD-3-Clause\r\n ********************************************************/\r\n#include <arrayfire.h>\r\n#include <iostream>\r\n\r\nusing namespace af;\r\n\r\nstatic size_t dimension = 16 * 1024;\r\nstatic const int maxIter = 10;\r\nstatic const int sparsityFactor = 7;\r\n\r\nstatic array A;\r\nstatic array spA;  // Sparse A\r\nstatic array x0;\r\nstatic array b;\r\n\r\nvoid setupInputs()\r\n{\r\n    // Generate a random input: A\r\n    array T = randu(dimension, dimension, f32);\r\n    // Create 0s in input.\r\n    // Anything that is no divisible by sparsityFactor will become 0.\r\n    A = floor(T * 1000);\r\n    A = A * ((A % sparsityFactor) == 0) / 1000;\r\n    // Make it positive definite\r\n    A = transpose(A) + A + A.dims(0)*identity(A.dims(0), A.dims(0), f32);\r\n\r\n    // Make A sparse as spA\r\n    spA = sparse(A);\r\n\r\n    // Generate x0: Random guess\r\n    x0 = randu(A.dims(0), f32);\r\n\r\n    //Generate b\r\n    b = matmul(A, x0);\r\n\r\n    std::cout << \"Sparsity of A = \" <<  100.f * (float)sparseGetNNZ(spA) / (float)spA.elements() << std::endl;\r\n    std::cout << \"Memory Usage of A = \" <<  A.bytes() << \" bytes\" << std::endl;\r\n    std::cout << \"Memory Usage of spA = \"\r\n              << sparseGetValues(spA).bytes()\r\n               + sparseGetRowIdx(spA).bytes()\r\n               + sparseGetColIdx(spA).bytes()\r\n              << \" bytes\" << std::endl;\r\n}\r\n\r\nvoid conjugateGradient(const af::array in)\r\n{\r\n    array x = constant(0, b.dims(), f32);\r\n    array r = b - matmul(in, x);\r\n    array p = r;\r\n\r\n    for (int i = 0; i < maxIter; ++i) {\r\n        array Ap = matmul(in, p);\r\n        array alpha_num = dot(r, r);\r\n        array alpha_den = dot(p, Ap);\r\n        array alpha = alpha_num/alpha_den;\r\n        r = r - tile(alpha, Ap.dims())*Ap;\r\n        x = x + tile(alpha, Ap.dims())*p;\r\n        array beta_num = dot(r, r);\r\n        array beta = beta_num/alpha_num;\r\n        p = r + tile(beta, p.dims())*p;\r\n    }\r\n    array res = x0 - x;\r\n\r\n    std::cout<<\"Final difference in solutions:\\n\";\r\n    af_print(dot(res, res));\r\n}\r\n\r\nint main(int argc, char *argv[])\r\n{\r\n    af::info();\r\n    setupInputs();\r\n\r\n    std::cout << \"Verifying Dense Conjugate Gradient:\" << std::endl;\r\n    conjugateGradient(A);\r\n\r\n    std::cout << \"Verifying Sparse Conjugate Gradient:\" << std::endl;\r\n    conjugateGradient(spA);\r\n\r\n    return 0;\r\n}\r\n</pre>\r\n\r\n<h4>Performance and Memory Footprint</h4>\r\nUsing the CUDA backend on a Tesla K40c, the conjugate gradient function took a time of 64.94ms for dense matrix and 39.31ms for sparse Matrix. That is almost a 2x speedup! Add in the fact that not all computation in the conjugate gradient function is sparse, the speedup for sparse vs dense matrix multiplication is extremely good.\r\n\r\nWith a <code>sparsityFactor</code> of 7, the sparsity of A is about 26.38%. This means a tremendous amount of space and performance can be achieved by using sparse arrays. For a floating point matrix of size 16M x 16M, a dense matrix uses 1 GB of memory while the sparse array, stored in CSR format, only used 527 MB - halving the memory consumption!\r\n\r\n<h4>Conclusion></h4>\r\nNothing excites us more than delivering on user requests for features and seeing them solve complex problems in diverse fields. Sparse arrays will now let users store more data on the GPU while allowing faster computation, thus allowing to solve for bigger, more complex problem.\r\n\r\nThe ArrayFire GitHub page is the place for all things ArrayFire. We look forward to your comments, feature requests and anything else that you may want to talk about.','Sparse Matrices in ArrayFire','','inherit','closed','closed','','9190-revision-v1','','','2016-09-20 18:41:10','2016-09-20 22:41:10','',9190,'http://sandbox.arrayfire.com/9190-revision-v1/',0,'revision','',0),(9344,11,'2016-09-20 18:41:47','2016-09-20 22:41:47','With the release of <strong>ArrayFire v3.4</strong>, we have added a number of features that will help our vast range of users. Our users have been asking us for a way to reduce the memory footprint of their data on the GPU. And now we have answered our users\' requests.\r\nIntroducing Sparse Matrices in ArrayFire!\r\n\r\n<h4>Sparse Data Structure</h4>\r\nA sparse data structure is one where all the non-zero elements are not stored. Sparse matrices are useful when the number of zero-values elements are &gt;&gt; number of non-zero elements (sparsity of the matrix is high).\r\nThe sparse data structure in general is stored as 3 arrays.\r\n<ul>\r\n<li>A data or values array containing all the non-zero elements</li>\r\n<li>A vector for row indices (based on storage format).</li>\r\n<li>A vector for column indices (based on storage format).</li>\r\n</ul>\r\n\r\nThere are many ways to store sparse matrices, the most prominent of which are\r\n<ul>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_row_.28CSR.2C_CRS_or_Yale_format.29\">Compressed Sparse Row</a></li>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_column_.28CSC_or_CCS.29\">Compressed Storage Column</a></li>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Coordinate_list_.28COO.29\">Coordinate Format</a>.</li>\r\n</ul>\r\n\r\nArrayFire supports CSR and COO format, with CSC support coming in future releases.\r\n\r\nA more detailed description of Sparse Matrix can be found on <a href=\"https://en.wikipedia.org/wiki/Sparse_matrix\">Wikipedia</a>.\r\n\r\n<h4>Sparse in ArrayFire</h4>\r\nIn ArrayFire, Sparse arrays use the same user facing object - af::array. This makes working with sparse arrays all the more seamless.\r\nWe have added a function <a href=\"http://arrayfire.org/docs/group__method__mat.htm#ga5945f95549bae25f75694a69d98ca584\">af::array::issparse()</a> that allows users to check if an array is sparse or not.\r\nAlthough externally, there is no difference between a sparse and dense array, internally, the sparse array is stored as 3 independent arrays along with other meta data.\r\nThese arrays, as well as other properties can be fetched using the <a href=\"http://arrayfire.org/docs/group__sparse__func.htm\">sparse functions</a>.\r\n\r\n<h4>Sparse Functionality</h4>\r\nAs of v3.4, the following features are supported for sparse arrays:\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm\">Creation for sparse arrays.</a>\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae4621e58bfaf6d80668309824fde86c0\">From 3 (data, row, column) sparse host or device memory vectors.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae9a843c7e93b05e6b090ca32b271ebe9\">From 3 (data, row, column) sparse af::arrays.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">From dense af::array.</a></li>\r\n<li>To create a sparse array from host or device dense array, first create a dense af::array, and then pass it to create sparse af::array from dense af::array.</li>\r\n</ul>\r\n</li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__convert__to.htm\">Storage format conversions.</a>\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">Dense to CSR/COO.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__dense.htm\">CSR/COO to dense.</a></li>\r\n</ul>\r\n</li>\r\n<li><a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">Sparse-Dense Matrix Multiplication (SPMM/SPMV)</a>\r\n<ul>\r\n<li>Sparse Matrix - Dense Matrix</li>\r\n<li>Sparse Matrix - Dense Vector</li>\r\n<li>Options for Sparse Matrix to be transposed.</li>\r\n</ul>\r\n</li>\r\n</ul>\r\n\r\nThe Sparse-Dense Matrix Multiplcation is a implemented as a part of <a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">af::matmul()</a>. This is part of making the usage of sparse arrays seamless with regular arrays.\r\n\r\nAs with all ArrayFire features, Sparse support is available across all backends and all architectures.\r\n\r\n<h4>Example</h4>\r\nThe example we shall use is a conjugate gradient solver. The inputs will be the same except that for the sparse conjugate gradient we will convert the A matrix to sparse.\r\n<pre>\r\n/*******************************************************\r\n * Copyright (c) 2014, ArrayFire\r\n * All rights reserved.\r\n *\r\n * This file is distributed under 3-clause BSD license.\r\n * The complete license agreement can be obtained at:\r\n * http://sandbox.arrayfire.com/licenses/BSD-3-Clause\r\n ********************************************************/\r\n#include <arrayfire.h>\r\n#include <iostream>\r\n\r\nusing namespace af;\r\n\r\nstatic size_t dimension = 16 * 1024;\r\nstatic const int maxIter = 10;\r\nstatic const int sparsityFactor = 7;\r\n\r\nstatic array A;\r\nstatic array spA;  // Sparse A\r\nstatic array x0;\r\nstatic array b;\r\n\r\nvoid setupInputs()\r\n{\r\n    // Generate a random input: A\r\n    array T = randu(dimension, dimension, f32);\r\n    // Create 0s in input.\r\n    // Anything that is no divisible by sparsityFactor will become 0.\r\n    A = floor(T * 1000);\r\n    A = A * ((A % sparsityFactor) == 0) / 1000;\r\n    // Make it positive definite\r\n    A = transpose(A) + A + A.dims(0)*identity(A.dims(0), A.dims(0), f32);\r\n\r\n    // Make A sparse as spA\r\n    spA = sparse(A);\r\n\r\n    // Generate x0: Random guess\r\n    x0 = randu(A.dims(0), f32);\r\n\r\n    //Generate b\r\n    b = matmul(A, x0);\r\n\r\n    std::cout << \"Sparsity of A = \" <<  100.f * (float)sparseGetNNZ(spA) / (float)spA.elements() << std::endl;\r\n    std::cout << \"Memory Usage of A = \" <<  A.bytes() << \" bytes\" << std::endl;\r\n    std::cout << \"Memory Usage of spA = \"\r\n              << sparseGetValues(spA).bytes()\r\n               + sparseGetRowIdx(spA).bytes()\r\n               + sparseGetColIdx(spA).bytes()\r\n              << \" bytes\" << std::endl;\r\n}\r\n\r\nvoid conjugateGradient(const af::array in)\r\n{\r\n    array x = constant(0, b.dims(), f32);\r\n    array r = b - matmul(in, x);\r\n    array p = r;\r\n\r\n    for (int i = 0; i < maxIter; ++i) {\r\n        array Ap = matmul(in, p);\r\n        array alpha_num = dot(r, r);\r\n        array alpha_den = dot(p, Ap);\r\n        array alpha = alpha_num/alpha_den;\r\n        r = r - tile(alpha, Ap.dims())*Ap;\r\n        x = x + tile(alpha, Ap.dims())*p;\r\n        array beta_num = dot(r, r);\r\n        array beta = beta_num/alpha_num;\r\n        p = r + tile(beta, p.dims())*p;\r\n    }\r\n    array res = x0 - x;\r\n\r\n    std::cout<<\"Final difference in solutions:\\n\";\r\n    af_print(dot(res, res));\r\n}\r\n\r\nint main(int argc, char *argv[])\r\n{\r\n    af::info();\r\n    setupInputs();\r\n\r\n    std::cout << \"Verifying Dense Conjugate Gradient:\" << std::endl;\r\n    conjugateGradient(A);\r\n\r\n    std::cout << \"Verifying Sparse Conjugate Gradient:\" << std::endl;\r\n    conjugateGradient(spA);\r\n\r\n    return 0;\r\n}\r\n</pre>\r\n\r\n<h4>Performance and Memory Footprint</h4>\r\nUsing the CUDA backend on a Tesla K40c, the conjugate gradient function took a time of 64.94ms for dense matrix and 39.31ms for sparse Matrix. That is almost a 2x speedup! Add in the fact that not all computation in the conjugate gradient function is sparse, the speedup for sparse vs dense matrix multiplication is extremely good.\r\n\r\nWith a <code>sparsityFactor</code> of 7, the sparsity of A is about 26.38%. This means a tremendous amount of space and performance can be achieved by using sparse arrays. For a floating point matrix of size 16M x 16M, a dense matrix uses 1 GB of memory while the sparse array, stored in CSR format, only used 527 MB - halving the memory consumption!\r\n\r\n<h4>Conclusion></h4>\r\nNothing excites us more than delivering on user requests for features and seeing them solve complex problems in diverse fields. Sparse arrays will now let users store more data on the GPU while allowing faster computation, thus allowing to solve for bigger, more complex problem.\r\n\r\nThe <a href=\"https://github.com/arrayfire/arrayfire\">ArrayFire GitHub</a> page is the place for all things ArrayFire. We look forward to your comments, feature requests and anything else that you may want to talk about.','Sparse Matrices in ArrayFire','','inherit','closed','closed','','9190-revision-v1','','','2016-09-20 18:41:47','2016-09-20 22:41:47','',9190,'http://sandbox.arrayfire.com/9190-revision-v1/',0,'revision','',0),(9345,11,'2016-09-20 18:42:19','2016-09-20 22:42:19','With the release of <strong>ArrayFire v3.4</strong>, we have added a number of features that will help our vast range of users. Our users have been asking us for a way to reduce the memory footprint of their data on the GPU. And now we have answered our users\' requests.\r\nIntroducing Sparse Matrices in ArrayFire!\r\n\r\n<h4>Sparse Data Structure</h4>\r\nA sparse data structure is one where all the non-zero elements are not stored. Sparse matrices are useful when the number of zero-values elements are &gt;&gt; number of non-zero elements (sparsity of the matrix is high).\r\nThe sparse data structure in general is stored as 3 arrays.\r\n<ul>\r\n<li>A data or values array containing all the non-zero elements</li>\r\n<li>A vector for row indices (based on storage format).</li>\r\n<li>A vector for column indices (based on storage format).</li>\r\n</ul>\r\n\r\nThere are many ways to store sparse matrices, the most prominent of which are\r\n<ul>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_row_.28CSR.2C_CRS_or_Yale_format.29\">Compressed Sparse Row</a></li>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_column_.28CSC_or_CCS.29\">Compressed Storage Column</a></li>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Coordinate_list_.28COO.29\">Coordinate Format</a>.</li>\r\n</ul>\r\n\r\nArrayFire supports CSR and COO format, with CSC support coming in future releases.\r\n\r\nA more detailed description of Sparse Matrix can be found on <a href=\"https://en.wikipedia.org/wiki/Sparse_matrix\">Wikipedia</a>.\r\n\r\n<h4>Sparse in ArrayFire</h4>\r\nIn ArrayFire, Sparse arrays use the same user facing object - af::array. This makes working with sparse arrays all the more seamless.\r\nWe have added a function <a href=\"http://arrayfire.org/docs/group__method__mat.htm#ga5945f95549bae25f75694a69d98ca584\">af::array::issparse()</a> that allows users to check if an array is sparse or not.\r\nAlthough externally, there is no difference between a sparse and dense array, internally, the sparse array is stored as 3 independent arrays along with other meta data.\r\nThese arrays, as well as other properties can be fetched using the <a href=\"http://arrayfire.org/docs/group__sparse__func.htm\">sparse functions</a>.\r\n\r\n<h4>Sparse Functionality</h4>\r\nAs of v3.4, the following features are supported for sparse arrays:\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm\">Creation for sparse arrays.</a>\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae4621e58bfaf6d80668309824fde86c0\">From 3 (data, row, column) sparse host or device memory vectors.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae9a843c7e93b05e6b090ca32b271ebe9\">From 3 (data, row, column) sparse af::arrays.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">From dense af::array.</a></li>\r\n<li>To create a sparse array from host or device dense array, first create a dense af::array, and then pass it to create sparse af::array from dense af::array.</li>\r\n</ul>\r\n</li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__convert__to.htm\">Storage format conversions.</a>\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">Dense to CSR/COO.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__dense.htm\">CSR/COO to dense.</a></li>\r\n</ul>\r\n</li>\r\n<li><a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">Sparse-Dense Matrix Multiplication (SPMM/SPMV)</a>\r\n<ul>\r\n<li>Sparse Matrix - Dense Matrix</li>\r\n<li>Sparse Matrix - Dense Vector</li>\r\n<li>Options for Sparse Matrix to be transposed.</li>\r\n</ul>\r\n</li>\r\n</ul>\r\n\r\nThe Sparse-Dense Matrix Multiplcation is a implemented as a part of <a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">af::matmul()</a>. This is part of making the usage of sparse arrays seamless with regular arrays.\r\n\r\nAs with all ArrayFire features, Sparse support is available across all backends and all architectures.\r\n\r\n<h4>Example</h4>\r\nThe example we shall use is a conjugate gradient solver. The inputs will be the same except that for the sparse conjugate gradient we will convert the A matrix to sparse.\r\n<pre>\r\n/*******************************************************\r\n * Copyright (c) 2014, ArrayFire\r\n * All rights reserved.\r\n *\r\n * This file is distributed under 3-clause BSD license.\r\n * The complete license agreement can be obtained at:\r\n * http://sandbox.arrayfire.com/licenses/BSD-3-Clause\r\n ********************************************************/\r\n#include <arrayfire.h>\r\n#include <iostream>\r\n\r\nusing namespace af;\r\n\r\nstatic size_t dimension = 16 * 1024;\r\nstatic const int maxIter = 10;\r\nstatic const int sparsityFactor = 7;\r\n\r\nstatic array A;\r\nstatic array spA;  // Sparse A\r\nstatic array x0;\r\nstatic array b;\r\n\r\nvoid setupInputs()\r\n{\r\n    // Generate a random input: A\r\n    array T = randu(dimension, dimension, f32);\r\n    // Create 0s in input.\r\n    // Anything that is no divisible by sparsityFactor will become 0.\r\n    A = floor(T * 1000);\r\n    A = A * ((A % sparsityFactor) == 0) / 1000;\r\n    // Make it positive definite\r\n    A = transpose(A) + A + A.dims(0)*identity(A.dims(0), A.dims(0), f32);\r\n\r\n    // Make A sparse as spA\r\n    spA = sparse(A);\r\n\r\n    // Generate x0: Random guess\r\n    x0 = randu(A.dims(0), f32);\r\n\r\n    //Generate b\r\n    b = matmul(A, x0);\r\n\r\n    std::cout << \"Sparsity of A = \"\r\n              <<  100.f * (float)sparseGetNNZ(spA) / (float)spA.elements()\r\n              << std::endl;\r\n    std::cout << \"Memory Usage of A = \" <<  A.bytes() << \" bytes\" << std::endl;\r\n    std::cout << \"Memory Usage of spA = \"\r\n              << sparseGetValues(spA).bytes()\r\n               + sparseGetRowIdx(spA).bytes()\r\n               + sparseGetColIdx(spA).bytes()\r\n              << \" bytes\" << std::endl;\r\n}\r\n\r\nvoid conjugateGradient(const af::array in)\r\n{\r\n    array x = constant(0, b.dims(), f32);\r\n    array r = b - matmul(in, x);\r\n    array p = r;\r\n\r\n    for (int i = 0; i < maxIter; ++i) {\r\n        array Ap = matmul(in, p);\r\n        array alpha_num = dot(r, r);\r\n        array alpha_den = dot(p, Ap);\r\n        array alpha = alpha_num/alpha_den;\r\n        r = r - tile(alpha, Ap.dims())*Ap;\r\n        x = x + tile(alpha, Ap.dims())*p;\r\n        array beta_num = dot(r, r);\r\n        array beta = beta_num/alpha_num;\r\n        p = r + tile(beta, p.dims())*p;\r\n    }\r\n    array res = x0 - x;\r\n\r\n    std::cout<<\"Final difference in solutions:\\n\";\r\n    af_print(dot(res, res));\r\n}\r\n\r\nint main(int argc, char *argv[])\r\n{\r\n    af::info();\r\n    setupInputs();\r\n\r\n    std::cout << \"Verifying Dense Conjugate Gradient:\" << std::endl;\r\n    conjugateGradient(A);\r\n\r\n    std::cout << \"Verifying Sparse Conjugate Gradient:\" << std::endl;\r\n    conjugateGradient(spA);\r\n\r\n    return 0;\r\n}\r\n</pre>\r\n\r\n<h4>Performance and Memory Footprint</h4>\r\nUsing the CUDA backend on a Tesla K40c, the conjugate gradient function took a time of 64.94ms for dense matrix and 39.31ms for sparse Matrix. That is almost a 2x speedup! Add in the fact that not all computation in the conjugate gradient function is sparse, the speedup for sparse vs dense matrix multiplication is extremely good.\r\n\r\nWith a <code>sparsityFactor</code> of 7, the sparsity of A is about 26.38%. This means a tremendous amount of space and performance can be achieved by using sparse arrays. For a floating point matrix of size 16M x 16M, a dense matrix uses 1 GB of memory while the sparse array, stored in CSR format, only used 527 MB - halving the memory consumption!\r\n\r\n<h4>Conclusion></h4>\r\nNothing excites us more than delivering on user requests for features and seeing them solve complex problems in diverse fields. Sparse arrays will now let users store more data on the GPU while allowing faster computation, thus allowing to solve for bigger, more complex problem.\r\n\r\nThe <a href=\"https://github.com/arrayfire/arrayfire\">ArrayFire GitHub</a> page is the place for all things ArrayFire. We look forward to your comments, feature requests and anything else that you may want to talk about.','Sparse Matrices in ArrayFire','','inherit','closed','closed','','9190-revision-v1','','','2016-09-20 18:42:19','2016-09-20 22:42:19','',9190,'http://sandbox.arrayfire.com/9190-revision-v1/',0,'revision','',0),(9346,18673,'2016-09-20 18:42:30','2016-09-20 22:42:30','There are a number of  additions and updates to image based features in the new v3.4 release of ArrayFire. Among the updates are:\r\n<ul>\r\n 	<li>New interpolation methods for several existing functions\r\n<ul>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__signal__func__approx1.htm\">approx1</a>, <a href=\"http://arrayfire.org/docs/group__signal__func__approx2.htm\">approx2</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__transform.htm\">transform</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__resize.htm\">resize</a></li>\r\n</ul>\r\n</li>\r\n 	<li>Functions for image <a href=\"http://arrayfire.org/docs/group__image__func__moments.htm\">moments</a></li>\r\n</ul>\r\nThis blog post will display some typical use cases for these new features.\r\n\r\n<span style=\"line-height: 1.5\">[custom_headline type=\"left\" level=\"h2\" looks_like=\"h2\"]Interpolation methods[/custom_headline]</span>\r\n\r\nArrayFire v3.4 implements several new <a href=\"http://arrayfire.org/docs/defines_8h.htm#ac0b7c1cfd1fb146314d96f17bbfffa68\"> interpolation methods </a>for 1-d and 2-d domains. The new interpolation methods for 1-d functions are:\r\n<ul>\r\n 	<li><strong>AF_INTERP_LINEAR_COSINE</strong></li>\r\n 	<li><strong>AF_INTERP_CUBIC</strong></li>\r\n</ul>\r\nand for 2-d functions are:\r\n<ul>\r\n 	<li><strong>AF_INTERP_BILINEAR_COSINE</strong></li>\r\n 	<li><strong>AF_INTERP_BICUBIC</strong></li>\r\n</ul>\r\nThe behavior of the <a href=\"http://arrayfire.org/docs/defines_8h.htm#ac0b7c1cfd1fb146314d96f17bbfffa68\">interpolation methods</a> can be seen in the following pictures.\r\n\r\n&nbsp;\r\n\r\n[caption id=\"attachment_9338\" align=\"alignleft\" width=\"300\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_0.png\"><img class=\"size-medium wp-image-9338\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_0-300x271.png\" alt=\"Original data points\" width=\"300\" height=\"271\" /></a> Original data points[/caption]\r\n\r\n[caption id=\"attachment_9339\" align=\"alignleft\" width=\"300\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_1.png\"><img class=\"size-medium wp-image-9339\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_1-300x267.png\" alt=\"Linear interpolation\" width=\"300\" height=\"267\" /></a> Linear interpolation[/caption]\r\n\r\n[caption id=\"attachment_9340\" align=\"alignleft\" width=\"300\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_2.png\"><img class=\"size-medium wp-image-9340\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_2-300x252.png\" alt=\"Linear cosine interpolation\" width=\"300\" height=\"252\" /></a> Linear cosine interpolation[/caption]\r\n\r\n[caption id=\"attachment_9337\" align=\"alignleft\" width=\"300\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_3.png\"><img class=\"size-medium wp-image-9337\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_3-300x255.png\" alt=\"Cubic interpolation\" width=\"300\" height=\"255\" /></a> Cubic interpolation[/caption]\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\nA common use for interpolation is image filtering. Given a coarse image, we can resample it to be smoother.\r\n<pre class=\"lang:c++ decode:true\" title=\"Interpolating images with approx2()\">af::array img = af::randu(7,7); //create a random image\r\n\r\n//define sample points for interpolation\r\naf::array Xs = af::seq(0, 6, 0.1f);\r\naf::array Ys = af::seq(0, 6, 0.1f);\r\nXs = af::tile(Xs, 1, Ys.dims(0));\r\nYs = af::tile(Ys.T(), Xs.dims(0));\r\n\r\n//interpolate based on specific method\r\naf::array img_bilinear = af::approx2(img, Xs, Ys, AF_INTERP_BILINEAR);\r\naf::array img_bilinearcos = af::approx2(img, Xs, Ys, AF_INTERP_BILINEAR_COSINE);\r\naf::array img_bicubic = af::approx2(img, Xs, Ys, AF_INTERP_BICUBIC_SPLINE);</pre>\r\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/Capture.png\"><img class=\"aligncenter size-full wp-image-9245\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/Capture.png\" alt=\"Capture\" width=\"1025\" height=\"243\" /></a>\r\n\r\nThe new interpolation methods further apply to several similar image transformation functions:\r\n<ul>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__resize.htm\">Resize</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__rotate.htm\">Rotate</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__scale.htm\">Scale</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__skew.htm\">Skew</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__transform.htm\">Transform</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__translate.htm\">Translate</a></li>\r\n</ul>\r\nDifferent interpolation methods greatly affect the final result of image processing operations. In general, bilinear and bilinear-cosine produce a good result when upscaling an image, however they often result in a blurry look. The bicubic method tends to sharpen the image resulting in slightly stronger borders and a better overall look. Below are some examples of this behavior using image rotation and image scaling.\r\n\r\n&nbsp;\r\n\r\n[caption id=\"attachment_9260\" align=\"aligncenter\" width=\"300\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_quality.png\"><img class=\"wp-image-9260 size-medium\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_quality-300x119.png\" alt=\"Image rotation with different interpolation methods.\" width=\"300\" height=\"119\" /></a> Image rotation with different interpolation methods. Left to right: nearest neighbor, bilinear, bilinear cosine, cubic[/caption]\r\n\r\n[caption id=\"attachment_9261\" align=\"aligncenter\" width=\"300\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/scale.png\"><img class=\"size-medium wp-image-9261\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/scale-300x276.png\" alt=\"3x image scaling with different interpolation methods.\" width=\"300\" height=\"276\" /></a> 3x image scaling with different sampling methods. Top to bottom: nearest neighbor, bilinear cosine, bicubic[/caption]\r\n\r\n<span style=\"line-height: 1.5\">[custom_headline type=\"left\" level=\"h2\" looks_like=\"h2\"]Image moments[/custom_headline]</span>\r\n\r\nImage <a href=\"http://arrayfire.org/docs/group__image__func__moments.htm\">moments </a>are another new addition to ArrayFire v3.4. Image moments are weighted averages of pixels in an image which provide useful properties of an image. A common use of image moments is to find the center of mass or area (or gray level sum) of an image.\r\n\r\nCurrently, ArrayFire calculates all<a href=\"http://arrayfire.org/docs/defines_8h.htm#ad3ee8e32709131cf4126dfa26210aec0\"> first order moments</a>. Each moment can be returned individually or all first-order moments can be calculated at once. All moments can be found as follows:\r\n<pre class=\"lang:c++ decode:true\">af::array moments = af::moments(input_image, AF_MOMENT_FIRST_ORDER);</pre>\r\n<div class=\"fragment\">\r\n<div class=\"line\">Here is an example of how the shorthand versions might be used to find the area and center of mass of an image:</div>\r\n</div>\r\n<pre class=\"lang:default decode:true\" title=\"Calculating center of mass of grayscale image\">double m00, m01, m10;\r\naf::moments(&amp;m00, aimg, AF_MOMENT_M00);\r\naf::moments(&amp;m01, aimg, AF_MOMENT_M01);\r\naf::moments(&amp;m10, aimg, AF_MOMENT_M10);\r\n\r\ndouble area = m00;\r\ndouble x_center = m10 / m00;\r\ndouble y_center = m01 / m00;\r\n</pre>\r\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/com.png\"><img class=\"aligncenter wp-image-9246 size-medium\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/com-300x300.png\" alt=\"com\" width=\"300\" height=\"300\" /></a>\r\n\r\n[custom_headline type=\"left\" level=\"h2\" looks_like=\"h2\"]Download[/custom_headline]\r\n\r\nArrayFire v3.4 can be downloaded from these locations:\r\n<ul>\r\n 	<li><a href=\"http://sandbox.arrayfire.com/download/\">Binaries</a></li>\r\n 	<li><a href=\"https://github.com/arrayfire\">Source code</a></li>\r\n</ul>\r\n[custom_headline type=\"left\" level=\"h2\" looks_like=\"h2\"]Community[/custom_headline]\r\n\r\nArrayFire is continually improving through the addition of new image processing and computer vision functions. We look forward to hearing your feedback on our user group and encourage you to share your ArrayFire stories there to inspire and contribute back to our open source community.','New image-based features in ArrayFire v3.4','','inherit','closed','closed','','9166-revision-v1','','','2016-09-20 18:42:30','2016-09-20 22:42:30','',9166,'http://sandbox.arrayfire.com/9166-revision-v1/',0,'revision','',0),(9347,11,'2016-09-20 18:42:35','2016-09-20 22:42:35','With the release of <strong>ArrayFire v3.4</strong>, we have added a number of features that will help our vast range of users. Our users have been asking us for a way to reduce the memory footprint of their data on the GPU. And now we have answered our users\' requests.\r\nIntroducing Sparse Matrices in ArrayFire!\r\n\r\n<h4>Sparse Data Structure</h4>\r\nA sparse data structure is one where all the non-zero elements are not stored. Sparse matrices are useful when the number of zero-values elements are &gt;&gt; number of non-zero elements (sparsity of the matrix is high).\r\nThe sparse data structure in general is stored as 3 arrays.\r\n<ul>\r\n<li>A data or values array containing all the non-zero elements</li>\r\n<li>A vector for row indices (based on storage format).</li>\r\n<li>A vector for column indices (based on storage format).</li>\r\n</ul>\r\n\r\nThere are many ways to store sparse matrices, the most prominent of which are\r\n<ul>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_row_.28CSR.2C_CRS_or_Yale_format.29\">Compressed Sparse Row</a></li>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_column_.28CSC_or_CCS.29\">Compressed Storage Column</a></li>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Coordinate_list_.28COO.29\">Coordinate Format</a>.</li>\r\n</ul>\r\n\r\nArrayFire supports CSR and COO format, with CSC support coming in future releases.\r\n\r\nA more detailed description of Sparse Matrix can be found on <a href=\"https://en.wikipedia.org/wiki/Sparse_matrix\">Wikipedia</a>.\r\n\r\n<h4>Sparse in ArrayFire</h4>\r\nIn ArrayFire, Sparse arrays use the same user facing object - af::array. This makes working with sparse arrays all the more seamless.\r\nWe have added a function <a href=\"http://arrayfire.org/docs/group__method__mat.htm#ga5945f95549bae25f75694a69d98ca584\">af::array::issparse()</a> that allows users to check if an array is sparse or not.\r\nAlthough externally, there is no difference between a sparse and dense array, internally, the sparse array is stored as 3 independent arrays along with other meta data.\r\nThese arrays, as well as other properties can be fetched using the <a href=\"http://arrayfire.org/docs/group__sparse__func.htm\">sparse functions</a>.\r\n\r\n<h4>Sparse Functionality</h4>\r\nAs of v3.4, the following features are supported for sparse arrays:\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm\">Creation for sparse arrays.</a>\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae4621e58bfaf6d80668309824fde86c0\">From 3 (data, row, column) sparse host or device memory vectors.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae9a843c7e93b05e6b090ca32b271ebe9\">From 3 (data, row, column) sparse af::arrays.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">From dense af::array.</a></li>\r\n<li>To create a sparse array from host or device dense array, first create a dense af::array, and then pass it to create sparse af::array from dense af::array.</li>\r\n</ul>\r\n</li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__convert__to.htm\">Storage format conversions.</a>\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">Dense to CSR/COO.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__dense.htm\">CSR/COO to dense.</a></li>\r\n</ul>\r\n</li>\r\n<li><a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">Sparse-Dense Matrix Multiplication (SPMM/SPMV)</a>\r\n<ul>\r\n<li>Sparse Matrix - Dense Matrix</li>\r\n<li>Sparse Matrix - Dense Vector</li>\r\n<li>Options for Sparse Matrix to be transposed.</li>\r\n</ul>\r\n</li>\r\n</ul>\r\n\r\nThe Sparse-Dense Matrix Multiplcation is a implemented as a part of <a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">af::matmul()</a>. This is part of making the usage of sparse arrays seamless with regular arrays.\r\n\r\nAs with all ArrayFire features, Sparse support is available across all backends and all architectures.\r\n\r\n<h4>Example</h4>\r\nThe example we shall use is a conjugate gradient solver. The inputs will be the same except that for the sparse conjugate gradient we will convert the A matrix to sparse.\r\n<pre>\r\n#include <arrayfire.h>\r\n#include <iostream>\r\n\r\nusing namespace af;\r\n\r\nstatic size_t dimension = 16 * 1024;\r\nstatic const int maxIter = 10;\r\nstatic const int sparsityFactor = 7;\r\n\r\nstatic array A;\r\nstatic array spA;  // Sparse A\r\nstatic array x0;\r\nstatic array b;\r\n\r\nvoid setupInputs()\r\n{\r\n    // Generate a random input: A\r\n    array T = randu(dimension, dimension, f32);\r\n    // Create 0s in input.\r\n    // Anything that is no divisible by sparsityFactor will become 0.\r\n    A = floor(T * 1000);\r\n    A = A * ((A % sparsityFactor) == 0) / 1000;\r\n    // Make it positive definite\r\n    A = transpose(A) + A + A.dims(0)*identity(A.dims(0), A.dims(0), f32);\r\n\r\n    // Make A sparse as spA\r\n    spA = sparse(A);\r\n\r\n    // Generate x0: Random guess\r\n    x0 = randu(A.dims(0), f32);\r\n\r\n    //Generate b\r\n    b = matmul(A, x0);\r\n\r\n    std::cout << \"Sparsity of A = \"\r\n              <<  100.f * (float)sparseGetNNZ(spA) / (float)spA.elements()\r\n              << std::endl;\r\n    std::cout << \"Memory Usage of A = \" <<  A.bytes() << \" bytes\" << std::endl;\r\n    std::cout << \"Memory Usage of spA = \"\r\n              << sparseGetValues(spA).bytes()\r\n               + sparseGetRowIdx(spA).bytes()\r\n               + sparseGetColIdx(spA).bytes()\r\n              << \" bytes\" << std::endl;\r\n}\r\n\r\nvoid conjugateGradient(const af::array in)\r\n{\r\n    array x = constant(0, b.dims(), f32);\r\n    array r = b - matmul(in, x);\r\n    array p = r;\r\n\r\n    for (int i = 0; i < maxIter; ++i) {\r\n        array Ap = matmul(in, p);\r\n        array alpha_num = dot(r, r);\r\n        array alpha_den = dot(p, Ap);\r\n        array alpha = alpha_num/alpha_den;\r\n        r = r - tile(alpha, Ap.dims())*Ap;\r\n        x = x + tile(alpha, Ap.dims())*p;\r\n        array beta_num = dot(r, r);\r\n        array beta = beta_num/alpha_num;\r\n        p = r + tile(beta, p.dims())*p;\r\n    }\r\n    array res = x0 - x;\r\n\r\n    std::cout<<\"Final difference in solutions:\\n\";\r\n    af_print(dot(res, res));\r\n}\r\n\r\nint main(int argc, char *argv[])\r\n{\r\n    af::info();\r\n    setupInputs();\r\n\r\n    std::cout << \"Verifying Dense Conjugate Gradient:\" << std::endl;\r\n    conjugateGradient(A);\r\n\r\n    std::cout << \"Verifying Sparse Conjugate Gradient:\" << std::endl;\r\n    conjugateGradient(spA);\r\n\r\n    return 0;\r\n}\r\n</pre>\r\n\r\n<h4>Performance and Memory Footprint</h4>\r\nUsing the CUDA backend on a Tesla K40c, the conjugate gradient function took a time of 64.94ms for dense matrix and 39.31ms for sparse Matrix. That is almost a 2x speedup! Add in the fact that not all computation in the conjugate gradient function is sparse, the speedup for sparse vs dense matrix multiplication is extremely good.\r\n\r\nWith a <code>sparsityFactor</code> of 7, the sparsity of A is about 26.38%. This means a tremendous amount of space and performance can be achieved by using sparse arrays. For a floating point matrix of size 16M x 16M, a dense matrix uses 1 GB of memory while the sparse array, stored in CSR format, only used 527 MB - halving the memory consumption!\r\n\r\n<h4>Conclusion></h4>\r\nNothing excites us more than delivering on user requests for features and seeing them solve complex problems in diverse fields. Sparse arrays will now let users store more data on the GPU while allowing faster computation, thus allowing to solve for bigger, more complex problem.\r\n\r\nThe <a href=\"https://github.com/arrayfire/arrayfire\">ArrayFire GitHub</a> page is the place for all things ArrayFire. We look forward to your comments, feature requests and anything else that you may want to talk about.','Sparse Matrices in ArrayFire','','inherit','closed','closed','','9190-revision-v1','','','2016-09-20 18:42:35','2016-09-20 22:42:35','',9190,'http://sandbox.arrayfire.com/9190-revision-v1/',0,'revision','',0),(9348,11,'2016-09-20 18:43:11','2016-09-20 22:43:11','With the release of <strong>ArrayFire v3.4</strong>, we have added a number of features that will help our vast range of users. Our users have been asking us for a way to reduce the memory footprint of their data on the GPU. And now we have answered our users\' requests.\r\nIntroducing Sparse Matrices in ArrayFire!\r\n\r\n<h4>Sparse Data Structure</h4>\r\nA sparse data structure is one where all the non-zero elements are not stored. Sparse matrices are useful when the number of zero-values elements are &gt;&gt; number of non-zero elements (sparsity of the matrix is high).\r\nThe sparse data structure in general is stored as 3 arrays.\r\n<ul>\r\n<li>A data or values array containing all the non-zero elements</li>\r\n<li>A vector for row indices (based on storage format).</li>\r\n<li>A vector for column indices (based on storage format).</li>\r\n</ul>\r\n\r\nThere are many ways to store sparse matrices, the most prominent of which are\r\n<ul>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_row_.28CSR.2C_CRS_or_Yale_format.29\">Compressed Sparse Row</a></li>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_column_.28CSC_or_CCS.29\">Compressed Storage Column</a></li>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Coordinate_list_.28COO.29\">Coordinate Format</a>.</li>\r\n</ul>\r\n\r\nArrayFire supports CSR and COO format, with CSC support coming in future releases.\r\n\r\nA more detailed description of Sparse Matrix can be found on <a href=\"https://en.wikipedia.org/wiki/Sparse_matrix\">Wikipedia</a>.\r\n\r\n<h4>Sparse in ArrayFire</h4>\r\nIn ArrayFire, Sparse arrays use the same user facing object - af::array. This makes working with sparse arrays all the more seamless.\r\nWe have added a function <a href=\"http://arrayfire.org/docs/group__method__mat.htm#ga5945f95549bae25f75694a69d98ca584\">af::array::issparse()</a> that allows users to check if an array is sparse or not.\r\nAlthough externally, there is no difference between a sparse and dense array, internally, the sparse array is stored as 3 independent arrays along with other meta data.\r\nThese arrays, as well as other properties can be fetched using the <a href=\"http://arrayfire.org/docs/group__sparse__func.htm\">sparse functions</a>.\r\n\r\n<h4>Sparse Functionality</h4>\r\nAs of v3.4, the following features are supported for sparse arrays:\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm\">Creation for sparse arrays.</a>\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae4621e58bfaf6d80668309824fde86c0\">From 3 (data, row, column) sparse host or device memory vectors.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae9a843c7e93b05e6b090ca32b271ebe9\">From 3 (data, row, column) sparse af::arrays.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">From dense af::array.</a></li>\r\n<li>To create a sparse array from host or device dense array, first create a dense af::array, and then pass it to create sparse af::array from dense af::array.</li>\r\n</ul>\r\n</li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__convert__to.htm\">Storage format conversions.</a>\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">Dense to CSR/COO.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__dense.htm\">CSR/COO to dense.</a></li>\r\n</ul>\r\n</li>\r\n<li><a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">Sparse-Dense Matrix Multiplication (SPMM/SPMV)</a>\r\n<ul>\r\n<li>Sparse Matrix - Dense Matrix</li>\r\n<li>Sparse Matrix - Dense Vector</li>\r\n<li>Options for Sparse Matrix to be transposed.</li>\r\n</ul>\r\n</li>\r\n</ul>\r\n\r\nThe Sparse-Dense Matrix Multiplcation is a implemented as a part of <a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">af::matmul()</a>. This is part of making the usage of sparse arrays seamless with regular arrays.\r\n\r\nAs with all ArrayFire features, Sparse support is available across all backends and all architectures.\r\n\r\n<h4>Example</h4>\r\nThe example we shall use is a conjugate gradient solver. The inputs will be the same except that for the sparse conjugate gradient we will convert the A matrix to sparse.\r\n<pre>\r\n#include <arrayfire.h>\r\n#include <iostream>\r\n\r\nusing namespace af;\r\n\r\nstatic size_t dimension = 16 * 1024;\r\nstatic const int maxIter = 10;\r\nstatic const int sparsityFactor = 7;\r\n\r\nstatic array A;\r\nstatic array spA;  // Sparse A\r\nstatic array x0;\r\nstatic array b;\r\n\r\nvoid setupInputs()\r\n{\r\n    // Generate a random input: A\r\n    array T = randu(dimension, dimension, f32);\r\n    // Create 0s in input.\r\n    // Anything that is no divisible by sparsityFactor will become 0.\r\n    A = floor(T * 1000);\r\n    A = A * ((A % sparsityFactor) == 0) / 1000;\r\n    // Make it positive definite\r\n    A = transpose(A) + A + A.dims(0)*identity(A.dims(0), A.dims(0), f32);\r\n\r\n    // Make A sparse as spA\r\n    spA = sparse(A);\r\n\r\n    // Generate x0: Random guess\r\n    x0 = randu(A.dims(0), f32);\r\n\r\n    //Generate b\r\n    b = matmul(A, x0);\r\n\r\n    std::cout << \"Sparsity of A = \"\r\n              <<  100.f * (float)sparseGetNNZ(spA) / (float)spA.elements()\r\n              << std::endl;\r\n    std::cout << \"Memory Usage of A = \" <<  A.bytes() << \" bytes\" << std::endl;\r\n    std::cout << \"Memory Usage of spA = \"\r\n              << sparseGetValues(spA).bytes()\r\n               + sparseGetRowIdx(spA).bytes()\r\n               + sparseGetColIdx(spA).bytes()\r\n              << \" bytes\" << std::endl;\r\n}\r\n\r\nvoid conjugateGradient(const af::array in)\r\n{\r\n    array x = constant(0, b.dims(), f32);\r\n    array r = b - matmul(in, x);\r\n    array p = r;\r\n\r\n    for (int i = 0; i < maxIter; ++i) {\r\n        array Ap = matmul(in, p);\r\n        array alpha_num = dot(r, r);\r\n        array alpha_den = dot(p, Ap);\r\n        array alpha = alpha_num/alpha_den;\r\n        r = r - tile(alpha, Ap.dims())*Ap;\r\n        x = x + tile(alpha, Ap.dims())*p;\r\n        array beta_num = dot(r, r);\r\n        array beta = beta_num/alpha_num;\r\n        p = r + tile(beta, p.dims())*p;\r\n    }\r\n    array res = x0 - x;\r\n\r\n    std::cout<<\"Final difference in solutions:\\n\";\r\n    af_print(dot(res, res));\r\n}\r\n\r\nint main(int argc, char *argv[])\r\n{\r\n    af::info();\r\n    setupInputs();\r\n\r\n    std::cout << \"Verifying Dense Conjugate Gradient:\" << std::endl;\r\n    conjugateGradient(A);\r\n\r\n    std::cout << \"Verifying Sparse Conjugate Gradient:\" << std::endl;\r\n    conjugateGradient(spA);\r\n\r\n    return 0;\r\n}\r\n</pre>\r\n\r\n<h4>Performance and Memory Footprint</h4>\r\nUsing the CUDA backend on a Tesla K40c, the conjugate gradient function took a time of 64.94ms for dense matrix and 39.31ms for sparse Matrix. That is almost a 2x speedup! Add in the fact that not all computation in the conjugate gradient function is sparse, the speedup for sparse vs dense matrix multiplication is extremely good.\r\n\r\nWith a <code>sparsityFactor</code> of 7, the sparsity of A is about 26.38%. This means a tremendous amount of space and performance can be achieved by using sparse arrays. For a floating point matrix of size 16M x 16M, a dense matrix uses 1 GB of memory while the sparse array, stored in CSR format, only used 527 MB - halving the memory consumption!\r\n\r\n<h4>Conclusion</h4>\r\nNothing excites us more than delivering on user requests for features and seeing them solve complex problems in diverse fields. Sparse arrays will now let users store more data on the GPU while allowing faster computation, thus allowing to solve for bigger, more complex problem.\r\n\r\nThe <a href=\"https://github.com/arrayfire/arrayfire\">ArrayFire GitHub</a> page is the place for all things ArrayFire. We look forward to your comments, feature requests and anything else that you may want to talk about.','Sparse Matrices in ArrayFire','','inherit','closed','closed','','9190-revision-v1','','','2016-09-20 18:43:11','2016-09-20 22:43:11','',9190,'http://sandbox.arrayfire.com/9190-revision-v1/',0,'revision','',0),(9349,18673,'2016-09-20 18:52:45','2016-09-20 22:52:45','There are a number of  additions and updates to image based features in the new v3.4 release of ArrayFire. Among the updates are:\r\n<ul>\r\n 	<li>New interpolation methods for several existing functions\r\n<ul>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__signal__func__approx1.htm\">approx1</a>, <a href=\"http://arrayfire.org/docs/group__signal__func__approx2.htm\">approx2</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__transform.htm\">transform</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__resize.htm\">resize</a></li>\r\n</ul>\r\n</li>\r\n 	<li>Functions for image <a href=\"http://arrayfire.org/docs/group__image__func__moments.htm\">moments</a></li>\r\n</ul>\r\nThis blog post will display some typical use cases for these new features.\r\n\r\n<span style=\"line-height: 1.5\">[custom_headline type=\"left\" level=\"h2\" looks_like=\"h2\"]Interpolation methods[/custom_headline]</span>\r\n\r\nArrayFire v3.4 implements several new <a href=\"http://arrayfire.org/docs/defines_8h.htm#ac0b7c1cfd1fb146314d96f17bbfffa68\"> interpolation methods </a>for 1-d and 2-d domains. The new interpolation methods for 1-d functions are:\r\n<ul>\r\n 	<li><strong>AF_INTERP_LINEAR_COSINE</strong></li>\r\n 	<li><strong>AF_INTERP_CUBIC</strong></li>\r\n</ul>\r\nand for 2-d functions are:\r\n<ul>\r\n 	<li><strong>AF_INTERP_BILINEAR_COSINE</strong></li>\r\n 	<li><strong>AF_INTERP_BICUBIC</strong></li>\r\n</ul>\r\nThe behavior of the <a href=\"http://arrayfire.org/docs/defines_8h.htm#ac0b7c1cfd1fb146314d96f17bbfffa68\">interpolation methods</a> can be seen in the following pictures.\r\n\r\n&nbsp;\r\n\r\n[caption id=\"attachment_9338\" align=\"aligncenter\" width=\"300\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_0.png\"><img class=\"size-medium wp-image-9338\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_0-300x271.png\" alt=\"Original data points\" width=\"300\" height=\"271\" /></a> Original data points[/caption]\r\n\r\n[caption id=\"attachment_9339\" align=\"aligncenter\" width=\"300\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_1.png\"><img class=\"size-medium wp-image-9339\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_1-300x267.png\" alt=\"Linear interpolation\" width=\"300\" height=\"267\" /></a> Linear interpolation[/caption]\r\n\r\n[caption id=\"attachment_9340\" align=\"aligncenter\" width=\"300\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_2.png\"><img class=\"size-medium wp-image-9340\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_2-300x252.png\" alt=\"Linear cosine interpolation\" width=\"300\" height=\"252\" /></a> Linear cosine interpolation[/caption]\r\n\r\n[caption id=\"attachment_9337\" align=\"aligncenter\" width=\"300\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_3.png\"><img class=\"size-medium wp-image-9337\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_3-300x255.png\" alt=\"Cubic interpolation\" width=\"300\" height=\"255\" /></a> Cubic interpolation[/caption]\r\n\r\nA common use for interpolation is image filtering. Given a coarse image, we can resample it to be smoother.\r\n<pre class=\"lang:c++ decode:true\" title=\"Interpolating images with approx2()\">af::array img = af::randu(7,7); //create a random image\r\n\r\n//define sample points for interpolation\r\naf::array Xs = af::seq(0, 6, 0.1f);\r\naf::array Ys = af::seq(0, 6, 0.1f);\r\nXs = af::tile(Xs, 1, Ys.dims(0));\r\nYs = af::tile(Ys.T(), Xs.dims(0));\r\n\r\n//interpolate based on specific method\r\naf::array img_bilinear = af::approx2(img, Xs, Ys, AF_INTERP_BILINEAR);\r\naf::array img_bilinearcos = af::approx2(img, Xs, Ys, AF_INTERP_BILINEAR_COSINE);\r\naf::array img_bicubic = af::approx2(img, Xs, Ys, AF_INTERP_BICUBIC_SPLINE);</pre>\r\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/Capture.png\"><img class=\"aligncenter size-full wp-image-9245\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/Capture.png\" alt=\"Capture\" width=\"1025\" height=\"243\" /></a>\r\n\r\nThe new interpolation methods further apply to several similar image transformation functions:\r\n<ul>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__resize.htm\">Resize</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__rotate.htm\">Rotate</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__scale.htm\">Scale</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__skew.htm\">Skew</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__transform.htm\">Transform</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__translate.htm\">Translate</a></li>\r\n</ul>\r\nDifferent interpolation methods greatly affect the final result of image processing operations. In general, bilinear and bilinear-cosine produce a good result when upscaling an image, however they often result in a blurry look. The bicubic method tends to sharpen the image resulting in slightly stronger borders and a better overall look. Below are some examples of this behavior using image rotation and image scaling.\r\n\r\n[caption id=\"attachment_9335\" align=\"alignleft\" width=\"182\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_0.png\"><img class=\"wp-image-9335 size-medium\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_0-182x300.png\" alt=\"rot_0\" width=\"182\" height=\"300\" /></a> Image rotation with nearest neighbor sampling.[/caption]\r\n\r\n[caption id=\"attachment_9336\" align=\"aligncenter\" width=\"190\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_1.png\"><img class=\"wp-image-9336 size-medium\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_1-190x300.png\" alt=\"rot_1\" width=\"190\" height=\"300\" /></a> Image rotation with bilinear sampling.[/caption]\r\n\r\n[caption id=\"attachment_9329\" align=\"alignleft\" width=\"184\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_2.png\"><img class=\"wp-image-9329 size-medium\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_2-184x300.png\" alt=\"rot_2\" width=\"184\" height=\"300\" /></a> Image rotation with bilinear-cosine sampling.[/caption]\r\n\r\n[caption id=\"attachment_9330\" align=\"aligncenter\" width=\"184\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_3.png\"><img class=\"wp-image-9330 size-medium\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_3-184x300.png\" alt=\"rot_3\" width=\"184\" height=\"300\" /></a> Image rotation with bicubic sampling.[/caption]\r\n\r\n[caption id=\"attachment_9331\" align=\"aligncenter\" width=\"300\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/s_0.png\"><img class=\"size-medium wp-image-9331\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/s_0-300x84.png\" alt=\"3x image scaling using nearest-neighbor sampling\" width=\"300\" height=\"84\" /></a> 3x image scaling using nearest-neighbor sampling[/caption]\r\n\r\n[caption id=\"attachment_9332\" align=\"aligncenter\" width=\"300\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/s_1.png\"><img class=\"size-medium wp-image-9332\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/s_1-300x85.png\" alt=\"3x image scaling using bilinear-cosine sampling\" width=\"300\" height=\"85\" /></a> 3x image scaling using bilinear-cosine sampling[/caption]\r\n\r\n[caption id=\"attachment_9334\" align=\"aligncenter\" width=\"300\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/s_2.png\"><img class=\"size-medium wp-image-9334\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/s_2-300x85.png\" alt=\"3x image scaling using bicubic sampling\" width=\"300\" height=\"85\" /></a> 3x image scaling using bicubic sampling[/caption]\r\n\r\n<span style=\"line-height: 1.5\">[custom_headline type=\"left\" level=\"h2\" looks_like=\"h2\"]Image moments[/custom_headline]</span>\r\n\r\nImage <a href=\"http://arrayfire.org/docs/group__image__func__moments.htm\">moments </a>are another new addition to ArrayFire v3.4. Image moments are weighted averages of pixels in an image which provide useful properties of an image. A common use of image moments is to find the center of mass or area (or gray level sum) of an image.\r\n\r\nCurrently, ArrayFire calculates all<a href=\"http://arrayfire.org/docs/defines_8h.htm#ad3ee8e32709131cf4126dfa26210aec0\"> first order moments</a>. Each moment can be returned individually or all first-order moments can be calculated at once. All moments can be found as follows:\r\n<pre class=\"lang:c++ decode:true\">af::array moments = af::moments(input_image, AF_MOMENT_FIRST_ORDER);</pre>\r\n<div class=\"fragment\">\r\n<div class=\"line\">Here is an example of how the shorthand versions might be used to find the area and center of mass of an image:</div>\r\n</div>\r\n<pre class=\"lang:default decode:true\" title=\"Calculating center of mass of grayscale image\">double m00, m01, m10;\r\naf::moments(&amp;m00, aimg, AF_MOMENT_M00);\r\naf::moments(&amp;m01, aimg, AF_MOMENT_M01);\r\naf::moments(&amp;m10, aimg, AF_MOMENT_M10);\r\n\r\ndouble area = m00;\r\ndouble x_center = m10 / m00;\r\ndouble y_center = m01 / m00;\r\n</pre>\r\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/com.png\"><img class=\"aligncenter wp-image-9246 size-medium\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/com-300x300.png\" alt=\"com\" width=\"300\" height=\"300\" /></a>\r\n\r\n[custom_headline type=\"left\" level=\"h2\" looks_like=\"h2\"]Download[/custom_headline]\r\n\r\nArrayFire v3.4 can be downloaded from these locations:\r\n<ul>\r\n 	<li><a href=\"http://sandbox.arrayfire.com/download/\">Binaries</a></li>\r\n 	<li><a href=\"https://github.com/arrayfire\">Source code</a></li>\r\n</ul>\r\n[custom_headline type=\"left\" level=\"h2\" looks_like=\"h2\"]Community[/custom_headline]\r\n\r\nArrayFire is continually improving through the addition of new image processing and computer vision functions. We look forward to hearing your feedback on our user group and encourage you to share your ArrayFire stories there to inspire and contribute back to our open source community.','New image-based features in ArrayFire v3.4','','inherit','closed','closed','','9166-revision-v1','','','2016-09-20 18:52:45','2016-09-20 22:52:45','',9166,'http://sandbox.arrayfire.com/9166-revision-v1/',0,'revision','',0),(9350,11,'2016-09-20 18:53:36','2016-09-20 22:53:36','With the release of <strong>ArrayFire v3.4</strong>, we have added a number of features that will help our vast range of users. Our users have been asking us for a way to reduce the memory footprint of their data on the GPU. And now we have answered our users\' requests.\r\nIntroducing Sparse Matrices in ArrayFire!\r\n\r\n<h4>Sparse Data Structure</h4>\r\nA sparse data structure is one where all the non-zero elements are not stored. Sparse matrices are useful when the number of zero-values elements are &gt;&gt; number of non-zero elements (sparsity of the matrix is high).\r\nThe sparse data structure in general is stored as 3 arrays.\r\n<ul>\r\n<li>A data or values array containing all the non-zero elements</li>\r\n<li>A vector for row indices (based on storage format).</li>\r\n<li>A vector for column indices (based on storage format).</li>\r\n</ul>\r\n\r\nThere are many ways to store sparse matrices, the most prominent of which are\r\n<ul>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_row_.28CSR.2C_CRS_or_Yale_format.29\">Compressed Sparse Row</a></li>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_column_.28CSC_or_CCS.29\">Compressed Storage Column</a></li>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Coordinate_list_.28COO.29\">Coordinate Format</a>.</li>\r\n</ul>\r\n\r\nArrayFire supports CSR and COO format, with CSC support coming in future releases.\r\n\r\nA more detailed description of Sparse Matrix can be found on <a href=\"https://en.wikipedia.org/wiki/Sparse_matrix\">Wikipedia</a>.\r\n\r\n<h4>Sparse in ArrayFire</h4>\r\nIn ArrayFire, Sparse arrays use the same user facing object - af::array. This makes working with sparse arrays all the more seamless.\r\nWe have added a function <a href=\"http://arrayfire.org/docs/group__method__mat.htm#ga5945f95549bae25f75694a69d98ca584\">af::array::issparse()</a> that allows users to check if an array is sparse or not.\r\nAlthough externally, there is no difference between a sparse and dense array, internally, the sparse array is stored as 3 independent arrays along with other meta data.\r\nThese arrays, as well as other properties can be fetched using the <a href=\"http://arrayfire.org/docs/group__sparse__func.htm\">sparse functions</a>.\r\n\r\n<h4>Sparse Functionality</h4>\r\nAs of v3.4, the following features are supported for sparse arrays:\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm\">Creation for sparse arrays.</a>\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae4621e58bfaf6d80668309824fde86c0\">From 3 (data, row, column) sparse host or device memory vectors.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae9a843c7e93b05e6b090ca32b271ebe9\">From 3 (data, row, column) sparse af::arrays.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">From dense af::array.</a></li>\r\n<li>To create a sparse array from host or device dense array, first create a dense af::array, and then pass it to create sparse af::array from dense af::array.</li>\r\n</ul>\r\n</li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__convert__to.htm\">Storage format conversions.</a>\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">Dense to CSR/COO.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__dense.htm\">CSR/COO to dense.</a></li>\r\n</ul>\r\n</li>\r\n<li><a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">Sparse-Dense Matrix Multiplication (SPMM/SPMV)</a>\r\n<ul>\r\n<li>Sparse Matrix - Dense Matrix</li>\r\n<li>Sparse Matrix - Dense Vector</li>\r\n<li>Options for Sparse Matrix to be transposed.</li>\r\n</ul>\r\n</li>\r\n</ul>\r\n\r\nThe Sparse-Dense Matrix Multiplcation is a implemented as a part of <a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">af::matmul()</a>. This is part of making the usage of sparse arrays seamless with regular arrays.\r\n\r\nAs with all ArrayFire features, Sparse support is available across all backends and all architectures.\r\n\r\n<h4>Example</h4>\r\nThe example we shall use is a conjugate gradient solver. The inputs will be the same except that for the sparse conjugate gradient we will convert the A matrix to sparse.\r\n<pre>\r\n#include <arrayfire.h>\r\n#include <iostream>\r\n\r\nusing namespace af;\r\n\r\nstatic size_t dimension = 16 * 1024;\r\nstatic const int maxIter = 10;\r\nstatic const int sparsityFactor = 7;\r\n\r\nstatic array A;\r\nstatic array spA;  // Sparse A\r\nstatic array x0;\r\nstatic array b;\r\n\r\nvoid setupInputs()\r\n{\r\n    // Generate a random input: A\r\n    array T = randu(dimension, dimension, f32);\r\n    // Create 0s in input.\r\n    // Anything that is no divisible by sparsityFactor will become 0.\r\n    A = floor(T * 1000);\r\n    A = A * ((A % sparsityFactor) == 0) / 1000;\r\n    // Make it positive definite\r\n    A = transpose(A) + A + A.dims(0)*identity(A.dims(0), A.dims(0), f32);\r\n\r\n    // Make A sparse as spA\r\n    spA = sparse(A);\r\n\r\n    // Generate x0: Random guess\r\n    x0 = randu(A.dims(0), f32);\r\n\r\n    //Generate b\r\n    b = matmul(A, x0);\r\n\r\n    std::cout << \"Sparsity of A = \"\r\n              <<  100.f * (float)sparseGetNNZ(spA) / (float)spA.elements()\r\n              << std::endl;\r\n    std::cout << \"Memory Usage of A = \" <<  A.bytes() << \" bytes\" << std::endl;\r\n    std::cout << \"Memory Usage of spA = \"\r\n              << sparseGetValues(spA).bytes()\r\n               + sparseGetRowIdx(spA).bytes()\r\n               + sparseGetColIdx(spA).bytes()\r\n              << \" bytes\" << std::endl;\r\n}\r\n\r\nvoid conjugateGradient(const af::array in)\r\n{\r\n    array x = constant(0, b.dims(), f32);\r\n    array r = b - matmul(in, x);\r\n    array p = r;\r\n\r\n    for (int i = 0; i < maxIter; ++i) {\r\n        array Ap = matmul(in, p);\r\n        array alpha_num = dot(r, r);\r\n        array alpha_den = dot(p, Ap);\r\n        array alpha = alpha_num/alpha_den;\r\n        r = r - tile(alpha, Ap.dims())*Ap;\r\n        x = x + tile(alpha, Ap.dims())*p;\r\n        array beta_num = dot(r, r);\r\n        array beta = beta_num/alpha_num;\r\n        p = r + tile(beta, p.dims())*p;\r\n    }\r\n    array res = x0 - x;\r\n\r\n    std::cout<<\"Final difference in solutions:\\n\";\r\n    af_print(dot(res, res));\r\n}\r\n\r\nint main(int argc, char *argv[])\r\n{\r\n    af::info();\r\n    setupInputs();\r\n\r\n    std::cout << \"Verifying Dense Conjugate Gradient:\" << std::endl;\r\n    conjugateGradient(A);\r\n\r\n    std::cout << \"Verifying Sparse Conjugate Gradient:\" << std::endl;\r\n    conjugateGradient(spA);\r\n\r\n    return 0;\r\n}\r\n</pre>\r\n\r\n<h4>Performance and Memory Footprint</h4>\r\nUsing the CUDA backend on a Tesla K40c, the conjugate gradient function took a time of 64.94ms for dense matrix and 39.31ms for sparse Matrix. That is almost a 2x speedup! Add in the fact that not all computation in the conjugate gradient function is sparse, the speedup for sparse vs dense matrix multiplication is extremely good.\r\n\r\nWith a <code>sparsityFactor</code> of 7, the sparsity of A is about 26.38%. This means a tremendous amount of space and performance can be achieved by using sparse arrays. For a floating point matrix of size 16M x 16M, a dense matrix uses 1 GB of memory while the sparse array, stored in CSR format, only used 527 MB - halving the memory consumption!\r\n\r\n<table>\r\n  <tr>\r\n    <th>Size</th>\r\n    <th>Dense Matrix Size</th>\r\n    <th>Sparse Matrix Size</th>\r\n    <th>CG Time with Dense</th>\r\n    <th>CG Time with Sparse</th>\r\n  </tr>\r\n  <tr>\r\n    <th>16M x 16M</th>\r\n    <th>1024 MB</th>\r\n    <th>527 MB</th>\r\n    <th>64.94ms</th>\r\n    <th>39.31ms</th>\r\n  </tr>\r\n  <tr>\r\n    <th>4M x 4M</th>\r\n    <th>64 MB</th>\r\n    <th>33 MB</th>\r\n    <th>4.752ms</th>\r\n    <th>3.935ms</th>\r\n  </tr>\r\n</table>\r\n\r\n<h4>Conclusion</h4>\r\nNothing excites us more than delivering on user requests for features and seeing them solve complex problems in diverse fields. Sparse arrays will now let users store more data on the GPU while allowing faster computation, thus allowing to solve for bigger, more complex problem.\r\n\r\nThe <a href=\"https://github.com/arrayfire/arrayfire\">ArrayFire GitHub</a> page is the place for all things ArrayFire. We look forward to your comments, feature requests and anything else that you may want to talk about.','Sparse Matrices in ArrayFire','','inherit','closed','closed','','9190-revision-v1','','','2016-09-20 18:53:36','2016-09-20 22:53:36','',9190,'http://sandbox.arrayfire.com/9190-revision-v1/',0,'revision','',0),(9351,11,'2016-09-20 18:54:38','2016-09-20 22:54:38','With the release of <strong>ArrayFire v3.4</strong>, we have added a number of features that will help our vast range of users. Our users have been asking us for a way to reduce the memory footprint of their data on the GPU. And now we have answered our users\' requests.\r\nIntroducing Sparse Matrices in ArrayFire!\r\n\r\n<h4>Sparse Data Structure</h4>\r\nA sparse data structure is one where all the non-zero elements are not stored. Sparse matrices are useful when the number of zero-values elements are &gt;&gt; number of non-zero elements (sparsity of the matrix is high).\r\nThe sparse data structure in general is stored as 3 arrays.\r\n<ul>\r\n<li>A data or values array containing all the non-zero elements</li>\r\n<li>A vector for row indices (based on storage format).</li>\r\n<li>A vector for column indices (based on storage format).</li>\r\n</ul>\r\n\r\nThere are many ways to store sparse matrices, the most prominent of which are\r\n<ul>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_row_.28CSR.2C_CRS_or_Yale_format.29\">Compressed Sparse Row</a></li>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_column_.28CSC_or_CCS.29\">Compressed Storage Column</a></li>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Coordinate_list_.28COO.29\">Coordinate Format</a>.</li>\r\n</ul>\r\n\r\nArrayFire supports CSR and COO format, with CSC support coming in future releases.\r\n\r\nA more detailed description of Sparse Matrix can be found on <a href=\"https://en.wikipedia.org/wiki/Sparse_matrix\">Wikipedia</a>.\r\n\r\n<h4>Sparse in ArrayFire</h4>\r\nIn ArrayFire, Sparse arrays use the same user facing object - af::array. This makes working with sparse arrays all the more seamless.\r\nWe have added a function <a href=\"http://arrayfire.org/docs/group__method__mat.htm#ga5945f95549bae25f75694a69d98ca584\">af::array::issparse()</a> that allows users to check if an array is sparse or not.\r\nAlthough externally, there is no difference between a sparse and dense array, internally, the sparse array is stored as 3 independent arrays along with other meta data.\r\nThese arrays, as well as other properties can be fetched using the <a href=\"http://arrayfire.org/docs/group__sparse__func.htm\">sparse functions</a>.\r\n\r\n<h4>Sparse Functionality</h4>\r\nAs of v3.4, the following features are supported for sparse arrays:\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm\">Creation for sparse arrays.</a>\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae4621e58bfaf6d80668309824fde86c0\">From 3 (data, row, column) sparse host or device memory vectors.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae9a843c7e93b05e6b090ca32b271ebe9\">From 3 (data, row, column) sparse af::arrays.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">From dense af::array.</a></li>\r\n<li>To create a sparse array from host or device dense array, first create a dense af::array, and then pass it to create sparse af::array from dense af::array.</li>\r\n</ul>\r\n</li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__convert__to.htm\">Storage format conversions.</a>\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">Dense to CSR/COO.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__dense.htm\">CSR/COO to dense.</a></li>\r\n</ul>\r\n</li>\r\n<li><a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">Sparse-Dense Matrix Multiplication (SPMM/SPMV)</a>\r\n<ul>\r\n<li>Sparse Matrix - Dense Matrix</li>\r\n<li>Sparse Matrix - Dense Vector</li>\r\n<li>Options for Sparse Matrix to be transposed.</li>\r\n</ul>\r\n</li>\r\n</ul>\r\n\r\nThe Sparse-Dense Matrix Multiplcation is a implemented as a part of <a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">af::matmul()</a>. This is part of making the usage of sparse arrays seamless with regular arrays.\r\n\r\nAs with all ArrayFire features, Sparse support is available across all backends and all architectures.\r\n\r\n<h4>Example</h4>\r\nThe example we shall use is a conjugate gradient solver. The inputs will be the same except that for the sparse conjugate gradient we will convert the A matrix to sparse.\r\n<pre>\r\n#include <arrayfire.h>\r\n#include <iostream>\r\n\r\nusing namespace af;\r\n\r\nstatic size_t dimension = 16 * 1024;\r\nstatic const int maxIter = 10;\r\nstatic const int sparsityFactor = 7;\r\n\r\nstatic array A;\r\nstatic array spA;  // Sparse A\r\nstatic array x0;\r\nstatic array b;\r\n\r\nvoid setupInputs()\r\n{\r\n    // Generate a random input: A\r\n    array T = randu(dimension, dimension, f32);\r\n    // Create 0s in input.\r\n    // Anything that is no divisible by sparsityFactor will become 0.\r\n    A = floor(T * 1000);\r\n    A = A * ((A % sparsityFactor) == 0) / 1000;\r\n    // Make it positive definite\r\n    A = transpose(A) + A + A.dims(0)*identity(A.dims(0), A.dims(0), f32);\r\n\r\n    // Make A sparse as spA\r\n    spA = sparse(A);\r\n\r\n    // Generate x0: Random guess\r\n    x0 = randu(A.dims(0), f32);\r\n\r\n    //Generate b\r\n    b = matmul(A, x0);\r\n\r\n    std::cout << \"Sparsity of A = \"\r\n              <<  100.f * (float)sparseGetNNZ(spA) / (float)spA.elements()\r\n              << std::endl;\r\n    std::cout << \"Memory Usage of A = \" <<  A.bytes() << \" bytes\" << std::endl;\r\n    std::cout << \"Memory Usage of spA = \"\r\n              << sparseGetValues(spA).bytes()\r\n               + sparseGetRowIdx(spA).bytes()\r\n               + sparseGetColIdx(spA).bytes()\r\n              << \" bytes\" << std::endl;\r\n}\r\n\r\nvoid conjugateGradient(const af::array in)\r\n{\r\n    array x = constant(0, b.dims(), f32);\r\n    array r = b - matmul(in, x);\r\n    array p = r;\r\n\r\n    for (int i = 0; i < maxIter; ++i) {\r\n        array Ap = matmul(in, p);\r\n        array alpha_num = dot(r, r);\r\n        array alpha_den = dot(p, Ap);\r\n        array alpha = alpha_num/alpha_den;\r\n        r = r - tile(alpha, Ap.dims())*Ap;\r\n        x = x + tile(alpha, Ap.dims())*p;\r\n        array beta_num = dot(r, r);\r\n        array beta = beta_num/alpha_num;\r\n        p = r + tile(beta, p.dims())*p;\r\n    }\r\n    array res = x0 - x;\r\n\r\n    std::cout<<\"Final difference in solutions:\\n\";\r\n    af_print(dot(res, res));\r\n}\r\n\r\nint main(int argc, char *argv[])\r\n{\r\n    af::info();\r\n    setupInputs();\r\n\r\n    std::cout << \"Verifying Dense Conjugate Gradient:\" << std::endl;\r\n    conjugateGradient(A);\r\n\r\n    std::cout << \"Verifying Sparse Conjugate Gradient:\" << std::endl;\r\n    conjugateGradient(spA);\r\n\r\n    return 0;\r\n}\r\n</pre>\r\n\r\n<h4>Performance and Memory Footprint</h4>\r\nUsing the CUDA backend on a Tesla K40c, the conjugate gradient function took a time of 64.94ms for dense matrix and 39.31ms for sparse Matrix. That is almost a 2x speedup! Add in the fact that not all computation in the conjugate gradient function is sparse, the speedup for sparse vs dense matrix multiplication is extremely good.\r\n\r\nWith a <code>sparsityFactor</code> of 7, the sparsity of A is about 26.38%. This means a tremendous amount of space and performance can be achieved by using sparse arrays. For a floating point matrix of size 16M x 16M, a dense matrix uses 1 GB of memory while the sparse array, stored in CSR format, only used 527 MB - halving the memory consumption!\r\n\r\n<table>\r\n  <tr>\r\n    <th>Size</th>\r\n    <th>Matrix Size (Dense)</th>\r\n    <th>Matrix Size (Sparse)</th>\r\n    <th>CG Time (Dense)</th>\r\n    <th>CG Time (Sparse)</th>\r\n  </tr>\r\n  <tr>\r\n    <th>16M x 16M</th>\r\n    <th>1024 MB</th>\r\n    <th>527 MB</th>\r\n    <th>64.94ms</th>\r\n    <th>39.31ms</th>\r\n  </tr>\r\n  <tr>\r\n    <th>4M x 4M</th>\r\n    <th>64 MB</th>\r\n    <th>33 MB</th>\r\n    <th>4.752ms</th>\r\n    <th>3.935ms</th>\r\n  </tr>\r\n</table>\r\n\r\n<h4>Conclusion</h4>\r\nNothing excites us more than delivering on user requests for features and seeing them solve complex problems in diverse fields. Sparse arrays will now let users store more data on the GPU while allowing faster computation, thus allowing to solve for bigger, more complex problem.\r\n\r\nThe <a href=\"https://github.com/arrayfire/arrayfire\">ArrayFire GitHub</a> page is the place for all things ArrayFire. We look forward to your comments, feature requests and anything else that you may want to talk about.','Sparse Matrices in ArrayFire','','inherit','closed','closed','','9190-revision-v1','','','2016-09-20 18:54:38','2016-09-20 22:54:38','',9190,'http://sandbox.arrayfire.com/9190-revision-v1/',0,'revision','',0),(9352,11,'2016-09-20 18:55:04','2016-09-20 22:55:04','With the release of <strong>ArrayFire v3.4</strong>, we have added a number of features that will help our vast range of users. Our users have been asking us for a way to reduce the memory footprint of their data on the GPU. And now we have answered our users\' requests.\r\nIntroducing Sparse Matrices in ArrayFire!\r\n\r\n<h4>Sparse Data Structure</h4>\r\nA sparse data structure is one where all the non-zero elements are not stored. Sparse matrices are useful when the number of zero-values elements are &gt;&gt; number of non-zero elements (sparsity of the matrix is high).\r\nThe sparse data structure in general is stored as 3 arrays.\r\n<ul>\r\n<li>A data or values array containing all the non-zero elements</li>\r\n<li>A vector for row indices (based on storage format).</li>\r\n<li>A vector for column indices (based on storage format).</li>\r\n</ul>\r\n\r\nThere are many ways to store sparse matrices, the most prominent of which are\r\n<ul>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_row_.28CSR.2C_CRS_or_Yale_format.29\">Compressed Sparse Row</a></li>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_column_.28CSC_or_CCS.29\">Compressed Storage Column</a></li>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Coordinate_list_.28COO.29\">Coordinate Format</a>.</li>\r\n</ul>\r\n\r\nArrayFire supports CSR and COO format, with CSC support coming in future releases.\r\n\r\nA more detailed description of Sparse Matrix can be found on <a href=\"https://en.wikipedia.org/wiki/Sparse_matrix\">Wikipedia</a>.\r\n\r\n<h4>Sparse in ArrayFire</h4>\r\nIn ArrayFire, Sparse arrays use the same user facing object - af::array. This makes working with sparse arrays all the more seamless.\r\nWe have added a function <a href=\"http://arrayfire.org/docs/group__method__mat.htm#ga5945f95549bae25f75694a69d98ca584\">af::array::issparse()</a> that allows users to check if an array is sparse or not.\r\nAlthough externally, there is no difference between a sparse and dense array, internally, the sparse array is stored as 3 independent arrays along with other meta data.\r\nThese arrays, as well as other properties can be fetched using the <a href=\"http://arrayfire.org/docs/group__sparse__func.htm\">sparse functions</a>.\r\n\r\n<h4>Sparse Functionality</h4>\r\nAs of v3.4, the following features are supported for sparse arrays:\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm\">Creation for sparse arrays.</a>\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae4621e58bfaf6d80668309824fde86c0\">From 3 (data, row, column) sparse host or device memory vectors.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae9a843c7e93b05e6b090ca32b271ebe9\">From 3 (data, row, column) sparse af::arrays.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">From dense af::array.</a></li>\r\n<li>To create a sparse array from host or device dense array, first create a dense af::array, and then pass it to create sparse af::array from dense af::array.</li>\r\n</ul>\r\n</li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__convert__to.htm\">Storage format conversions.</a>\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">Dense to CSR/COO.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__dense.htm\">CSR/COO to dense.</a></li>\r\n</ul>\r\n</li>\r\n<li><a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">Sparse-Dense Matrix Multiplication (SPMM/SPMV)</a>\r\n<ul>\r\n<li>Sparse Matrix - Dense Matrix</li>\r\n<li>Sparse Matrix - Dense Vector</li>\r\n<li>Options for Sparse Matrix to be transposed.</li>\r\n</ul>\r\n</li>\r\n</ul>\r\n\r\nThe Sparse-Dense Matrix Multiplcation is a implemented as a part of <a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">af::matmul()</a>. This is part of making the usage of sparse arrays seamless with regular arrays.\r\n\r\nAs with all ArrayFire features, Sparse support is available across all backends and all architectures.\r\n\r\n<h4>Example</h4>\r\nThe example we shall use is a conjugate gradient solver. The inputs will be the same except that for the sparse conjugate gradient we will convert the A matrix to sparse.\r\n<pre>\r\n#include <arrayfire.h>\r\n#include <iostream>\r\n\r\nusing namespace af;\r\n\r\nstatic size_t dimension = 16 * 1024;\r\nstatic const int maxIter = 10;\r\nstatic const int sparsityFactor = 7;\r\n\r\nstatic array A;\r\nstatic array spA;  // Sparse A\r\nstatic array x0;\r\nstatic array b;\r\n\r\nvoid setupInputs()\r\n{\r\n    // Generate a random input: A\r\n    array T = randu(dimension, dimension, f32);\r\n    // Create 0s in input.\r\n    // Anything that is no divisible by sparsityFactor will become 0.\r\n    A = floor(T * 1000);\r\n    A = A * ((A % sparsityFactor) == 0) / 1000;\r\n    // Make it positive definite\r\n    A = transpose(A) + A + A.dims(0)*identity(A.dims(0), A.dims(0), f32);\r\n\r\n    // Make A sparse as spA\r\n    spA = sparse(A);\r\n\r\n    // Generate x0: Random guess\r\n    x0 = randu(A.dims(0), f32);\r\n\r\n    //Generate b\r\n    b = matmul(A, x0);\r\n\r\n    std::cout << \"Sparsity of A = \"\r\n              <<  100.f * (float)sparseGetNNZ(spA) / (float)spA.elements()\r\n              << std::endl;\r\n    std::cout << \"Memory Usage of A = \" <<  A.bytes() << \" bytes\" << std::endl;\r\n    std::cout << \"Memory Usage of spA = \"\r\n              << sparseGetValues(spA).bytes()\r\n               + sparseGetRowIdx(spA).bytes()\r\n               + sparseGetColIdx(spA).bytes()\r\n              << \" bytes\" << std::endl;\r\n}\r\n\r\nvoid conjugateGradient(const af::array in)\r\n{\r\n    array x = constant(0, b.dims(), f32);\r\n    array r = b - matmul(in, x);\r\n    array p = r;\r\n\r\n    for (int i = 0; i < maxIter; ++i) {\r\n        array Ap = matmul(in, p);\r\n        array alpha_num = dot(r, r);\r\n        array alpha_den = dot(p, Ap);\r\n        array alpha = alpha_num/alpha_den;\r\n        r = r - tile(alpha, Ap.dims())*Ap;\r\n        x = x + tile(alpha, Ap.dims())*p;\r\n        array beta_num = dot(r, r);\r\n        array beta = beta_num/alpha_num;\r\n        p = r + tile(beta, p.dims())*p;\r\n    }\r\n    array res = x0 - x;\r\n\r\n    std::cout<<\"Final difference in solutions:\\n\";\r\n    af_print(dot(res, res));\r\n}\r\n\r\nint main(int argc, char *argv[])\r\n{\r\n    af::info();\r\n    setupInputs();\r\n\r\n    std::cout << \"Verifying Dense Conjugate Gradient:\" << std::endl;\r\n    conjugateGradient(A);\r\n\r\n    std::cout << \"Verifying Sparse Conjugate Gradient:\" << std::endl;\r\n    conjugateGradient(spA);\r\n\r\n    return 0;\r\n}\r\n</pre>\r\n\r\n<h4>Performance and Memory Footprint</h4>\r\nUsing the CUDA backend on a Tesla K40c, the conjugate gradient function took a time of 64.94ms for dense matrix and 39.31ms for sparse Matrix. That is almost a 2x speedup! Add in the fact that not all computation in the conjugate gradient function is sparse, the speedup for sparse vs dense matrix multiplication is extremely good.\r\n\r\nWith a <code>sparsityFactor</code> of 7, the sparsity of A is about 26.38%. This means a tremendous amount of space and performance can be achieved by using sparse arrays. For a floating point matrix of size 16M x 16M, a dense matrix uses 1 GB of memory while the sparse array, stored in CSR format, only used 527 MB - halving the memory consumption!\r\n\r\n<table>\r\n  <tr>\r\n    <th>Elements</th>\r\n    <th>Matrix Size (Dense)</th>\r\n    <th>Matrix Size (Sparse)</th>\r\n    <th>CG Time (Dense)</th>\r\n    <th>CG Time (Sparse)</th>\r\n  </tr>\r\n  <tr>\r\n    <th>16M x 16M</th>\r\n    <th>1024 MB</th>\r\n    <th>527 MB</th>\r\n    <th>64.94ms</th>\r\n    <th>39.31ms</th>\r\n  </tr>\r\n  <tr>\r\n    <th>4M x 4M</th>\r\n    <th>64 MB</th>\r\n    <th>33 MB</th>\r\n    <th>4.752ms</th>\r\n    <th>3.935ms</th>\r\n  </tr>\r\n</table>\r\n\r\n<h4>Conclusion</h4>\r\nNothing excites us more than delivering on user requests for features and seeing them solve complex problems in diverse fields. Sparse arrays will now let users store more data on the GPU while allowing faster computation, thus allowing to solve for bigger, more complex problem.\r\n\r\nThe <a href=\"https://github.com/arrayfire/arrayfire\">ArrayFire GitHub</a> page is the place for all things ArrayFire. We look forward to your comments, feature requests and anything else that you may want to talk about.','Sparse Matrices in ArrayFire','','inherit','closed','closed','','9190-revision-v1','','','2016-09-20 18:55:04','2016-09-20 22:55:04','',9190,'http://sandbox.arrayfire.com/9190-revision-v1/',0,'revision','',0),(9353,18673,'2016-09-20 18:55:11','2016-09-20 22:55:11','There are a number of  additions and updates to image based features in the new v3.4 release of ArrayFire. Among the updates are:\r\n<ul>\r\n 	<li>New interpolation methods for several existing functions\r\n<ul>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__signal__func__approx1.htm\">approx1</a>, <a href=\"http://arrayfire.org/docs/group__signal__func__approx2.htm\">approx2</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__transform.htm\">transform</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__resize.htm\">resize</a></li>\r\n</ul>\r\n</li>\r\n 	<li>Functions for image <a href=\"http://arrayfire.org/docs/group__image__func__moments.htm\">moments</a></li>\r\n</ul>\r\nThis blog post will display some typical use cases for these new features.\r\n\r\n<span style=\"line-height: 1.5\">[custom_headline type=\"left\" level=\"h2\" looks_like=\"h2\"]Interpolation methods[/custom_headline]</span>\r\n\r\nArrayFire v3.4 implements several new <a href=\"http://arrayfire.org/docs/defines_8h.htm#ac0b7c1cfd1fb146314d96f17bbfffa68\"> interpolation methods </a>for 1-d and 2-d domains. The new interpolation methods for 1-d functions are:\r\n<ul>\r\n 	<li><strong>AF_INTERP_LINEAR_COSINE</strong></li>\r\n 	<li><strong>AF_INTERP_CUBIC</strong></li>\r\n</ul>\r\nand for 2-d functions are:\r\n<ul>\r\n 	<li><strong>AF_INTERP_BILINEAR_COSINE</strong></li>\r\n 	<li><strong>AF_INTERP_BICUBIC</strong></li>\r\n</ul>\r\nThe behavior of the <a href=\"http://arrayfire.org/docs/defines_8h.htm#ac0b7c1cfd1fb146314d96f17bbfffa68\">interpolation methods</a> can be seen in the following pictures.\r\n\r\n&nbsp;\r\n\r\n[caption id=\"attachment_9338\" align=\"aligncenter\" width=\"300\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_0.png\"><img class=\"size-medium wp-image-9338\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_0-300x271.png\" alt=\"Original data points\" width=\"300\" height=\"271\" /></a> Original data points[/caption]\r\n\r\n[caption id=\"attachment_9339\" align=\"aligncenter\" width=\"300\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_1.png\"><img class=\"size-medium wp-image-9339\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_1-300x267.png\" alt=\"Linear interpolation\" width=\"300\" height=\"267\" /></a> Linear interpolation[/caption]\r\n\r\n[caption id=\"attachment_9340\" align=\"aligncenter\" width=\"300\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_2.png\"><img class=\"size-medium wp-image-9340\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_2-300x252.png\" alt=\"Linear cosine interpolation\" width=\"300\" height=\"252\" /></a> Linear cosine interpolation[/caption]\r\n\r\n[caption id=\"attachment_9337\" align=\"aligncenter\" width=\"300\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_3.png\"><img class=\"size-medium wp-image-9337\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_3-300x255.png\" alt=\"Cubic interpolation\" width=\"300\" height=\"255\" /></a> Cubic interpolation[/caption]\r\n\r\nA common use for interpolation is image filtering. Given a coarse image, we can resample it to be smoother.\r\n<pre class=\"lang:c++ decode:true\" title=\"Interpolating images with approx2()\">af::array img = af::randu(7,7); //create a random image\r\n\r\n//define sample points for interpolation\r\naf::array Xs = af::seq(0, 6, 0.1f);\r\naf::array Ys = af::seq(0, 6, 0.1f);\r\nXs = af::tile(Xs, 1, Ys.dims(0));\r\nYs = af::tile(Ys.T(), Xs.dims(0));\r\n\r\n//interpolate based on specific method\r\naf::array img_bilinear = af::approx2(img, Xs, Ys, AF_INTERP_BILINEAR);\r\naf::array img_bilinearcos = af::approx2(img, Xs, Ys, AF_INTERP_BILINEAR_COSINE);\r\naf::array img_bicubic = af::approx2(img, Xs, Ys, AF_INTERP_BICUBIC_SPLINE);</pre>\r\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/Capture.png\"><img class=\"aligncenter size-full wp-image-9245\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/Capture.png\" alt=\"Capture\" width=\"1025\" height=\"243\" /></a>\r\n\r\nThe new interpolation methods further apply to several similar image transformation functions:\r\n<ul>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__resize.htm\">Resize</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__rotate.htm\">Rotate</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__scale.htm\">Scale</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__skew.htm\">Skew</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__transform.htm\">Transform</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__translate.htm\">Translate</a></li>\r\n</ul>\r\nDifferent interpolation methods greatly affect the final result of image processing operations. In general, bilinear and bilinear-cosine produce a good result when upscaling an image, however they often result in a blurry look. The bicubic method tends to sharpen the image resulting in slightly stronger borders and a better overall look. Below are some examples of this behavior using image rotation and image scaling.\r\n\r\n[caption id=\"attachment_9335\" align=\"alignleft\" width=\"150\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_0.png\"><img class=\"wp-image-9335 size-thumbnail\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_0-150x150.png\" alt=\"rot_0\" width=\"150\" height=\"150\" /></a> Image rotation with nearest neighbor sampling.[/caption]\r\n\r\n[caption id=\"attachment_9336\" align=\"alignleft\" width=\"150\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_1.png\"><img class=\"wp-image-9336 size-thumbnail\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_1-150x150.png\" alt=\"rot_1\" width=\"150\" height=\"150\" /></a> Image rotation with bilinear sampling.[/caption]\r\n\r\n[caption id=\"attachment_9329\" align=\"alignleft\" width=\"150\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_2.png\"><img class=\"wp-image-9329 size-thumbnail\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_2-150x150.png\" alt=\"rot_2\" width=\"150\" height=\"150\" /></a> Image rotation with bilinear-cosine sampling.[/caption]\r\n\r\n[caption id=\"attachment_9330\" align=\"alignleft\" width=\"150\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_3.png\"><img class=\"wp-image-9330 size-thumbnail\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_3-150x150.png\" alt=\"rot_3\" width=\"150\" height=\"150\" /></a> Image rotation with bicubic sampling.[/caption]\r\n\r\n[caption id=\"attachment_9331\" align=\"aligncenter\" width=\"300\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/s_0.png\"><img class=\"size-medium wp-image-9331\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/s_0-300x84.png\" alt=\"3x image scaling using nearest-neighbor sampling\" width=\"300\" height=\"84\" /></a> 3x image scaling using nearest-neighbor sampling[/caption]\r\n\r\n[caption id=\"attachment_9332\" align=\"aligncenter\" width=\"300\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/s_1.png\"><img class=\"size-medium wp-image-9332\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/s_1-300x85.png\" alt=\"3x image scaling using bilinear-cosine sampling\" width=\"300\" height=\"85\" /></a> 3x image scaling using bilinear-cosine sampling[/caption]\r\n\r\n[caption id=\"attachment_9334\" align=\"aligncenter\" width=\"300\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/s_2.png\"><img class=\"size-medium wp-image-9334\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/s_2-300x85.png\" alt=\"3x image scaling using bicubic sampling\" width=\"300\" height=\"85\" /></a> 3x image scaling using bicubic sampling[/caption]\r\n\r\n<span style=\"line-height: 1.5\">[custom_headline type=\"left\" level=\"h2\" looks_like=\"h2\"]Image moments[/custom_headline]</span>\r\n\r\nImage <a href=\"http://arrayfire.org/docs/group__image__func__moments.htm\">moments </a>are another new addition to ArrayFire v3.4. Image moments are weighted averages of pixels in an image which provide useful properties of an image. A common use of image moments is to find the center of mass or area (or gray level sum) of an image.\r\n\r\nCurrently, ArrayFire calculates all<a href=\"http://arrayfire.org/docs/defines_8h.htm#ad3ee8e32709131cf4126dfa26210aec0\"> first order moments</a>. Each moment can be returned individually or all first-order moments can be calculated at once. All moments can be found as follows:\r\n<pre class=\"lang:c++ decode:true\">af::array moments = af::moments(input_image, AF_MOMENT_FIRST_ORDER);</pre>\r\n<div class=\"fragment\">\r\n<div class=\"line\">Here is an example of how the shorthand versions might be used to find the area and center of mass of an image:</div>\r\n</div>\r\n<pre class=\"lang:default decode:true\" title=\"Calculating center of mass of grayscale image\">double m00, m01, m10;\r\naf::moments(&amp;m00, aimg, AF_MOMENT_M00);\r\naf::moments(&amp;m01, aimg, AF_MOMENT_M01);\r\naf::moments(&amp;m10, aimg, AF_MOMENT_M10);\r\n\r\ndouble area = m00;\r\ndouble x_center = m10 / m00;\r\ndouble y_center = m01 / m00;\r\n</pre>\r\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/com.png\"><img class=\"aligncenter wp-image-9246 size-medium\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/com-300x300.png\" alt=\"com\" width=\"300\" height=\"300\" /></a>\r\n\r\n[custom_headline type=\"left\" level=\"h2\" looks_like=\"h2\"]Download[/custom_headline]\r\n\r\nArrayFire v3.4 can be downloaded from these locations:\r\n<ul>\r\n 	<li><a href=\"http://sandbox.arrayfire.com/download/\">Binaries</a></li>\r\n 	<li><a href=\"https://github.com/arrayfire\">Source code</a></li>\r\n</ul>\r\n[custom_headline type=\"left\" level=\"h2\" looks_like=\"h2\"]Community[/custom_headline]\r\n\r\nArrayFire is continually improving through the addition of new image processing and computer vision functions. We look forward to hearing your feedback on our user group and encourage you to share your ArrayFire stories there to inspire and contribute back to our open source community.','New image-based features in ArrayFire v3.4','','inherit','closed','closed','','9166-revision-v1','','','2016-09-20 18:55:11','2016-09-20 22:55:11','',9166,'http://sandbox.arrayfire.com/9166-revision-v1/',0,'revision','',0),(9354,11,'2016-09-20 18:55:28','2016-09-20 22:55:28','With the release of <strong>ArrayFire v3.4</strong>, we have added a number of features that will help our vast range of users. Our users have been asking us for a way to reduce the memory footprint of their data on the GPU. And now we have answered our users\' requests.\r\nIntroducing Sparse Matrices in ArrayFire!\r\n\r\n<h4>Sparse Data Structure</h4>\r\nA sparse data structure is one where all the non-zero elements are not stored. Sparse matrices are useful when the number of zero-values elements are &gt;&gt; number of non-zero elements (sparsity of the matrix is high).\r\nThe sparse data structure in general is stored as 3 arrays.\r\n<ul>\r\n<li>A data or values array containing all the non-zero elements</li>\r\n<li>A vector for row indices (based on storage format).</li>\r\n<li>A vector for column indices (based on storage format).</li>\r\n</ul>\r\n\r\nThere are many ways to store sparse matrices, the most prominent of which are\r\n<ul>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_row_.28CSR.2C_CRS_or_Yale_format.29\">Compressed Sparse Row</a></li>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_column_.28CSC_or_CCS.29\">Compressed Storage Column</a></li>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Coordinate_list_.28COO.29\">Coordinate Format</a>.</li>\r\n</ul>\r\n\r\nArrayFire supports CSR and COO format, with CSC support coming in future releases.\r\n\r\nA more detailed description of Sparse Matrix can be found on <a href=\"https://en.wikipedia.org/wiki/Sparse_matrix\">Wikipedia</a>.\r\n\r\n<h4>Sparse in ArrayFire</h4>\r\nIn ArrayFire, Sparse arrays use the same user facing object - af::array. This makes working with sparse arrays all the more seamless.\r\nWe have added a function <a href=\"http://arrayfire.org/docs/group__method__mat.htm#ga5945f95549bae25f75694a69d98ca584\">af::array::issparse()</a> that allows users to check if an array is sparse or not.\r\nAlthough externally, there is no difference between a sparse and dense array, internally, the sparse array is stored as 3 independent arrays along with other meta data.\r\nThese arrays, as well as other properties can be fetched using the <a href=\"http://arrayfire.org/docs/group__sparse__func.htm\">sparse functions</a>.\r\n\r\n<h4>Sparse Functionality</h4>\r\nAs of v3.4, the following features are supported for sparse arrays:\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm\">Creation for sparse arrays.</a>\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae4621e58bfaf6d80668309824fde86c0\">From 3 (data, row, column) sparse host or device memory vectors.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae9a843c7e93b05e6b090ca32b271ebe9\">From 3 (data, row, column) sparse af::arrays.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">From dense af::array.</a></li>\r\n<li>To create a sparse array from host or device dense array, first create a dense af::array, and then pass it to create sparse af::array from dense af::array.</li>\r\n</ul>\r\n</li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__convert__to.htm\">Storage format conversions.</a>\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">Dense to CSR/COO.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__dense.htm\">CSR/COO to dense.</a></li>\r\n</ul>\r\n</li>\r\n<li><a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">Sparse-Dense Matrix Multiplication (SPMM/SPMV)</a>\r\n<ul>\r\n<li>Sparse Matrix - Dense Matrix</li>\r\n<li>Sparse Matrix - Dense Vector</li>\r\n<li>Options for Sparse Matrix to be transposed.</li>\r\n</ul>\r\n</li>\r\n</ul>\r\n\r\nThe Sparse-Dense Matrix Multiplcation is a implemented as a part of <a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">af::matmul()</a>. This is part of making the usage of sparse arrays seamless with regular arrays.\r\n\r\nAs with all ArrayFire features, Sparse support is available across all backends and all architectures.\r\n\r\n<h4>Example</h4>\r\nThe example we shall use is a conjugate gradient solver. The inputs will be the same except that for the sparse conjugate gradient we will convert the A matrix to sparse.\r\n<pre>\r\n#include <arrayfire.h>\r\n#include <iostream>\r\n\r\nusing namespace af;\r\n\r\nstatic size_t dimension = 16 * 1024;\r\nstatic const int maxIter = 10;\r\nstatic const int sparsityFactor = 7;\r\n\r\nstatic array A;\r\nstatic array spA;  // Sparse A\r\nstatic array x0;\r\nstatic array b;\r\n\r\nvoid setupInputs()\r\n{\r\n    // Generate a random input: A\r\n    array T = randu(dimension, dimension, f32);\r\n    // Create 0s in input.\r\n    // Anything that is no divisible by sparsityFactor will become 0.\r\n    A = floor(T * 1000);\r\n    A = A * ((A % sparsityFactor) == 0) / 1000;\r\n    // Make it positive definite\r\n    A = transpose(A) + A + A.dims(0)*identity(A.dims(0), A.dims(0), f32);\r\n\r\n    // Make A sparse as spA\r\n    spA = sparse(A);\r\n\r\n    // Generate x0: Random guess\r\n    x0 = randu(A.dims(0), f32);\r\n\r\n    //Generate b\r\n    b = matmul(A, x0);\r\n\r\n    std::cout << \"Sparsity of A = \"\r\n              <<  100.f * (float)sparseGetNNZ(spA) / (float)spA.elements()\r\n              << std::endl;\r\n    std::cout << \"Memory Usage of A = \" <<  A.bytes() << \" bytes\" << std::endl;\r\n    std::cout << \"Memory Usage of spA = \"\r\n              << sparseGetValues(spA).bytes()\r\n               + sparseGetRowIdx(spA).bytes()\r\n               + sparseGetColIdx(spA).bytes()\r\n              << \" bytes\" << std::endl;\r\n}\r\n\r\nvoid conjugateGradient(const af::array in)\r\n{\r\n    array x = constant(0, b.dims(), f32);\r\n    array r = b - matmul(in, x);\r\n    array p = r;\r\n\r\n    for (int i = 0; i < maxIter; ++i) {\r\n        array Ap = matmul(in, p);\r\n        array alpha_num = dot(r, r);\r\n        array alpha_den = dot(p, Ap);\r\n        array alpha = alpha_num/alpha_den;\r\n        r = r - tile(alpha, Ap.dims())*Ap;\r\n        x = x + tile(alpha, Ap.dims())*p;\r\n        array beta_num = dot(r, r);\r\n        array beta = beta_num/alpha_num;\r\n        p = r + tile(beta, p.dims())*p;\r\n    }\r\n    array res = x0 - x;\r\n\r\n    std::cout<<\"Final difference in solutions:\\n\";\r\n    af_print(dot(res, res));\r\n}\r\n\r\nint main(int argc, char *argv[])\r\n{\r\n    af::info();\r\n    setupInputs();\r\n\r\n    std::cout << \"Verifying Dense Conjugate Gradient:\" << std::endl;\r\n    conjugateGradient(A);\r\n\r\n    std::cout << \"Verifying Sparse Conjugate Gradient:\" << std::endl;\r\n    conjugateGradient(spA);\r\n\r\n    return 0;\r\n}\r\n</pre>\r\n\r\n<h4>Performance and Memory Footprint</h4>\r\nUsing the CUDA backend on a Tesla K40c, the conjugate gradient function took a time of 64.94ms for dense matrix and 39.31ms for sparse Matrix. That is almost a 2x speedup! Add in the fact that not all computation in the conjugate gradient function is sparse, the speedup for sparse vs dense matrix multiplication is extremely good.\r\n\r\nWith a <code>sparsityFactor</code> of 7, the sparsity of A is about 26.38%. This means a tremendous amount of space and performance can be achieved by using sparse arrays. For a floating point matrix of size 16M x 16M, a dense matrix uses 1 GB of memory while the sparse array, stored in CSR format, only used 527 MB - halving the memory consumption!\r\n\r\n<table>\r\n  <tr>\r\n    <th>Matrix Elements</th>\r\n    <th>Matrix Size (Dense)</th>\r\n    <th>Matrix Size (Sparse)</th>\r\n    <th>CG Time (Dense)</th>\r\n    <th>CG Time (Sparse)</th>\r\n  </tr>\r\n  <tr>\r\n    <th>16M x 16M</th>\r\n    <th>1024 MB</th>\r\n    <th>527 MB</th>\r\n    <th>64.94ms</th>\r\n    <th>39.31ms</th>\r\n  </tr>\r\n  <tr>\r\n    <th>4M x 4M</th>\r\n    <th>64 MB</th>\r\n    <th>33 MB</th>\r\n    <th>4.752ms</th>\r\n    <th>3.935ms</th>\r\n  </tr>\r\n</table>\r\n\r\n<h4>Conclusion</h4>\r\nNothing excites us more than delivering on user requests for features and seeing them solve complex problems in diverse fields. Sparse arrays will now let users store more data on the GPU while allowing faster computation, thus allowing to solve for bigger, more complex problem.\r\n\r\nThe <a href=\"https://github.com/arrayfire/arrayfire\">ArrayFire GitHub</a> page is the place for all things ArrayFire. We look forward to your comments, feature requests and anything else that you may want to talk about.','Sparse Matrices in ArrayFire','','inherit','closed','closed','','9190-revision-v1','','','2016-09-20 18:55:28','2016-09-20 22:55:28','',9190,'http://sandbox.arrayfire.com/9190-revision-v1/',0,'revision','',0),(9355,11,'2016-09-20 18:55:50','2016-09-20 22:55:50','With the release of <strong>ArrayFire v3.4</strong>, we have added a number of features that will help our vast range of users. Our users have been asking us for a way to reduce the memory footprint of their data on the GPU. And now we have answered our users\' requests.\r\nIntroducing Sparse Matrices in ArrayFire!\r\n\r\n<h4>Sparse Data Structure</h4>\r\nA sparse data structure is one where all the non-zero elements are not stored. Sparse matrices are useful when the number of zero-values elements are &gt;&gt; number of non-zero elements (sparsity of the matrix is high).\r\nThe sparse data structure in general is stored as 3 arrays.\r\n<ul>\r\n<li>A data or values array containing all the non-zero elements</li>\r\n<li>A vector for row indices (based on storage format).</li>\r\n<li>A vector for column indices (based on storage format).</li>\r\n</ul>\r\n\r\nThere are many ways to store sparse matrices, the most prominent of which are\r\n<ul>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_row_.28CSR.2C_CRS_or_Yale_format.29\">Compressed Sparse Row</a></li>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_column_.28CSC_or_CCS.29\">Compressed Storage Column</a></li>\r\n<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Coordinate_list_.28COO.29\">Coordinate Format</a>.</li>\r\n</ul>\r\n\r\nArrayFire supports CSR and COO format, with CSC support coming in future releases.\r\n\r\nA more detailed description of Sparse Matrix can be found on <a href=\"https://en.wikipedia.org/wiki/Sparse_matrix\">Wikipedia</a>.\r\n\r\n<h4>Sparse in ArrayFire</h4>\r\nIn ArrayFire, Sparse arrays use the same user facing object - af::array. This makes working with sparse arrays all the more seamless.\r\nWe have added a function <a href=\"http://arrayfire.org/docs/group__method__mat.htm#ga5945f95549bae25f75694a69d98ca584\">af::array::issparse()</a> that allows users to check if an array is sparse or not.\r\nAlthough externally, there is no difference between a sparse and dense array, internally, the sparse array is stored as 3 independent arrays along with other meta data.\r\nThese arrays, as well as other properties can be fetched using the <a href=\"http://arrayfire.org/docs/group__sparse__func.htm\">sparse functions</a>.\r\n\r\n<h4>Sparse Functionality</h4>\r\nAs of v3.4, the following features are supported for sparse arrays:\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm\">Creation for sparse arrays.</a>\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae4621e58bfaf6d80668309824fde86c0\">From 3 (data, row, column) sparse host or device memory vectors.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae9a843c7e93b05e6b090ca32b271ebe9\">From 3 (data, row, column) sparse af::arrays.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">From dense af::array.</a></li>\r\n<li>To create a sparse array from host or device dense array, first create a dense af::array, and then pass it to create sparse af::array from dense af::array.</li>\r\n</ul>\r\n</li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__convert__to.htm\">Storage format conversions.</a>\r\n<ul>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">Dense to CSR/COO.</a></li>\r\n<li><a href=\"http://arrayfire.org/docs/group__sparse__func__dense.htm\">CSR/COO to dense.</a></li>\r\n</ul>\r\n</li>\r\n<li><a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">Sparse-Dense Matrix Multiplication (SPMM/SPMV)</a>\r\n<ul>\r\n<li>Sparse Matrix - Dense Matrix</li>\r\n<li>Sparse Matrix - Dense Vector</li>\r\n<li>Options for Sparse Matrix to be transposed.</li>\r\n</ul>\r\n</li>\r\n</ul>\r\n\r\nThe Sparse-Dense Matrix Multiplcation is a implemented as a part of <a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">af::matmul()</a>. This is part of making the usage of sparse arrays seamless with regular arrays.\r\n\r\nAs with all ArrayFire features, Sparse support is available across all backends and all architectures.\r\n\r\n<h4>Example</h4>\r\nThe example we shall use is a conjugate gradient solver. The inputs will be the same except that for the sparse conjugate gradient we will convert the A matrix to sparse.\r\n<pre>\r\n#include <arrayfire.h>\r\n#include <iostream>\r\n\r\nusing namespace af;\r\n\r\nstatic size_t dimension = 16 * 1024;\r\nstatic const int maxIter = 10;\r\nstatic const int sparsityFactor = 7;\r\n\r\nstatic array A;\r\nstatic array spA;  // Sparse A\r\nstatic array x0;\r\nstatic array b;\r\n\r\nvoid setupInputs()\r\n{\r\n    // Generate a random input: A\r\n    array T = randu(dimension, dimension, f32);\r\n    // Create 0s in input.\r\n    // Anything that is no divisible by sparsityFactor will become 0.\r\n    A = floor(T * 1000);\r\n    A = A * ((A % sparsityFactor) == 0) / 1000;\r\n    // Make it positive definite\r\n    A = transpose(A) + A + A.dims(0)*identity(A.dims(0), A.dims(0), f32);\r\n\r\n    // Make A sparse as spA\r\n    spA = sparse(A);\r\n\r\n    // Generate x0: Random guess\r\n    x0 = randu(A.dims(0), f32);\r\n\r\n    //Generate b\r\n    b = matmul(A, x0);\r\n\r\n    std::cout << \"Sparsity of A = \"\r\n              <<  100.f * (float)sparseGetNNZ(spA) / (float)spA.elements()\r\n              << std::endl;\r\n    std::cout << \"Memory Usage of A = \" <<  A.bytes() << \" bytes\" << std::endl;\r\n    std::cout << \"Memory Usage of spA = \"\r\n              << sparseGetValues(spA).bytes()\r\n               + sparseGetRowIdx(spA).bytes()\r\n               + sparseGetColIdx(spA).bytes()\r\n              << \" bytes\" << std::endl;\r\n}\r\n\r\nvoid conjugateGradient(const af::array in)\r\n{\r\n    array x = constant(0, b.dims(), f32);\r\n    array r = b - matmul(in, x);\r\n    array p = r;\r\n\r\n    for (int i = 0; i < maxIter; ++i) {\r\n        array Ap = matmul(in, p);\r\n        array alpha_num = dot(r, r);\r\n        array alpha_den = dot(p, Ap);\r\n        array alpha = alpha_num/alpha_den;\r\n        r = r - tile(alpha, Ap.dims())*Ap;\r\n        x = x + tile(alpha, Ap.dims())*p;\r\n        array beta_num = dot(r, r);\r\n        array beta = beta_num/alpha_num;\r\n        p = r + tile(beta, p.dims())*p;\r\n    }\r\n    array res = x0 - x;\r\n\r\n    std::cout<<\"Final difference in solutions:\\n\";\r\n    af_print(dot(res, res));\r\n}\r\n\r\nint main(int argc, char *argv[])\r\n{\r\n    af::info();\r\n    setupInputs();\r\n\r\n    std::cout << \"Verifying Dense Conjugate Gradient:\" << std::endl;\r\n    conjugateGradient(A);\r\n\r\n    std::cout << \"Verifying Sparse Conjugate Gradient:\" << std::endl;\r\n    conjugateGradient(spA);\r\n\r\n    return 0;\r\n}\r\n</pre>\r\n\r\n<h4>Performance and Memory Footprint</h4>\r\nUsing the CUDA backend on a Tesla K40c, the conjugate gradient function took a time of 64.94ms for dense matrix and 39.31ms for sparse Matrix. That is almost a 2x speedup! Add in the fact that not all computation in the conjugate gradient function is sparse, the speedup for sparse vs dense matrix multiplication is extremely good.\r\n\r\nWith a <code>sparsityFactor</code> of 7, the sparsity of A is about 26.38%. This means a tremendous amount of space and performance can be achieved by using sparse arrays. For a floating point matrix of size 16M x 16M, a dense matrix uses 1 GB of memory while the sparse array, stored in CSR format, only used 527 MB - halving the memory consumption!\r\n\r\n<table>\r\n  <tr>\r\n    <th>Matrix Elements</th>\r\n    <th>Memory Size (Dense)</th>\r\n    <th>Memory Size (Sparse)</th>\r\n    <th>CG Time (Dense)</th>\r\n    <th>CG Time (Sparse)</th>\r\n  </tr>\r\n  <tr>\r\n    <th>16M x 16M</th>\r\n    <th>1024 MB</th>\r\n    <th>527 MB</th>\r\n    <th>64.94ms</th>\r\n    <th>39.31ms</th>\r\n  </tr>\r\n  <tr>\r\n    <th>4M x 4M</th>\r\n    <th>64 MB</th>\r\n    <th>33 MB</th>\r\n    <th>4.752ms</th>\r\n    <th>3.935ms</th>\r\n  </tr>\r\n</table>\r\n\r\n<h4>Conclusion</h4>\r\nNothing excites us more than delivering on user requests for features and seeing them solve complex problems in diverse fields. Sparse arrays will now let users store more data on the GPU while allowing faster computation, thus allowing to solve for bigger, more complex problem.\r\n\r\nThe <a href=\"https://github.com/arrayfire/arrayfire\">ArrayFire GitHub</a> page is the place for all things ArrayFire. We look forward to your comments, feature requests and anything else that you may want to talk about.','Sparse Matrices in ArrayFire','','inherit','closed','closed','','9190-revision-v1','','','2016-09-20 18:55:50','2016-09-20 22:55:50','',9190,'http://sandbox.arrayfire.com/9190-revision-v1/',0,'revision','',0),(9356,18673,'2016-09-20 19:02:45','2016-09-20 23:02:45','There are a number of  additions and updates to image based features in the new v3.4 release of ArrayFire. Among the updates are:\r\n<ul>\r\n 	<li>New interpolation methods for several existing functions\r\n<ul>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__signal__func__approx1.htm\">approx1</a>, <a href=\"http://arrayfire.org/docs/group__signal__func__approx2.htm\">approx2</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__transform.htm\">transform</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__resize.htm\">resize</a></li>\r\n</ul>\r\n</li>\r\n 	<li>Functions for image <a href=\"http://arrayfire.org/docs/group__image__func__moments.htm\">moments</a></li>\r\n</ul>\r\nThis blog post will display some typical use cases for these new features.\r\n\r\n<span style=\"line-height: 1.5\">[custom_headline type=\"left\" level=\"h2\" looks_like=\"h2\"]Interpolation methods[/custom_headline]</span>\r\n\r\nArrayFire v3.4 implements several new <a href=\"http://arrayfire.org/docs/defines_8h.htm#ac0b7c1cfd1fb146314d96f17bbfffa68\"> interpolation methods </a>for 1-d and 2-d domains. The new interpolation methods for 1-d functions are:\r\n<ul>\r\n 	<li><strong>AF_INTERP_LINEAR_COSINE</strong></li>\r\n 	<li><strong>AF_INTERP_CUBIC</strong></li>\r\n</ul>\r\nand for 2-d functions are:\r\n<ul>\r\n 	<li><strong>AF_INTERP_BILINEAR_COSINE</strong></li>\r\n 	<li><strong>AF_INTERP_BICUBIC</strong></li>\r\n</ul>\r\nThe behavior of the <a href=\"http://arrayfire.org/docs/defines_8h.htm#ac0b7c1cfd1fb146314d96f17bbfffa68\">interpolation methods</a> can be seen in the following pictures.\r\n\r\n&nbsp;\r\n\r\n[caption id=\"attachment_9338\" align=\"aligncenter\" width=\"300\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_0.png\"><img class=\"size-medium wp-image-9338\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_0-300x271.png\" alt=\"Original data points\" width=\"300\" height=\"271\" /></a> Original data points[/caption]\r\n\r\n[caption id=\"attachment_9339\" align=\"aligncenter\" width=\"300\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_1.png\"><img class=\"size-medium wp-image-9339\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_1-300x267.png\" alt=\"Linear interpolation\" width=\"300\" height=\"267\" /></a> Linear interpolation[/caption]\r\n\r\n[caption id=\"attachment_9340\" align=\"aligncenter\" width=\"300\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_2.png\"><img class=\"size-medium wp-image-9340\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_2-300x252.png\" alt=\"Linear cosine interpolation\" width=\"300\" height=\"252\" /></a> Linear cosine interpolation[/caption]\r\n\r\n[caption id=\"attachment_9337\" align=\"aligncenter\" width=\"300\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_3.png\"><img class=\"size-medium wp-image-9337\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_3-300x255.png\" alt=\"Cubic interpolation\" width=\"300\" height=\"255\" /></a> Cubic interpolation[/caption]\r\n\r\nA common use for interpolation is image filtering. Given a coarse image, we can resample it to be smoother.\r\n<pre class=\"lang:c++ decode:true\" title=\"Interpolating images with approx2()\">af::array img = af::randu(7,7); //create a random image\r\n\r\n//define sample points for interpolation\r\naf::array Xs = af::seq(0, 6, 0.1f);\r\naf::array Ys = af::seq(0, 6, 0.1f);\r\nXs = af::tile(Xs, 1, Ys.dims(0));\r\nYs = af::tile(Ys.T(), Xs.dims(0));\r\n\r\n//interpolate based on specific method\r\naf::array img_bilinear = af::approx2(img, Xs, Ys, AF_INTERP_BILINEAR);\r\naf::array img_bilinearcos = af::approx2(img, Xs, Ys, AF_INTERP_BILINEAR_COSINE);\r\naf::array img_bicubic = af::approx2(img, Xs, Ys, AF_INTERP_BICUBIC_SPLINE);</pre>\r\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/Capture.png\"><img class=\"aligncenter size-full wp-image-9245\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/Capture.png\" alt=\"Capture\" width=\"1025\" height=\"243\" /></a>\r\n\r\nThe new interpolation methods further apply to several similar image transformation functions:\r\n<ul>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__resize.htm\">Resize</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__rotate.htm\">Rotate</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__scale.htm\">Scale</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__skew.htm\">Skew</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__transform.htm\">Transform</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__translate.htm\">Translate</a></li>\r\n</ul>\r\nDifferent interpolation methods greatly affect the final result of image processing operations. In general, bilinear and bilinear-cosine produce a good result when upscaling an image, however they often result in a blurry look. The bicubic method tends to sharpen the image resulting in slightly stronger borders and a better overall look. Below are some examples of this behavior using image rotation and image scaling.\r\n\r\n[caption id=\"attachment_9331\" align=\"aligncenter\" width=\"300\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/s_0.png\"><img class=\"size-medium wp-image-9331\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/s_0-300x84.png\" alt=\"3x image scaling using nearest-neighbor sampling\" width=\"300\" height=\"84\" /></a> 3x image scaling using nearest-neighbor sampling[/caption]\r\n\r\n[caption id=\"attachment_9332\" align=\"aligncenter\" width=\"300\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/s_1.png\"><img class=\"size-medium wp-image-9332\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/s_1-300x85.png\" alt=\"3x image scaling using bilinear-cosine sampling\" width=\"300\" height=\"85\" /></a> 3x image scaling using bilinear-cosine sampling[/caption]\r\n\r\n[caption id=\"attachment_9334\" align=\"aligncenter\" width=\"300\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/s_2.png\"><img class=\"size-medium wp-image-9334\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/s_2-300x85.png\" alt=\"3x image scaling using bicubic sampling\" width=\"300\" height=\"85\" /></a> 3x image scaling using bicubic sampling[/caption]\r\n\r\n[block_grid type=\"four-up\"] [block_grid_item]\r\n\r\n[caption id=\"attachment_9335\" align=\"aligncenter\" width=\"150\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_0.png\"><img class=\"wp-image-9335 size-thumbnail\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_0-150x150.png\" alt=\"rot_0\" width=\"150\" height=\"150\" /></a> Image rotation with nearest neighbor sampling.[/caption]\r\n\r\n[/block_grid_item][block_grid_item]\r\n\r\n[caption id=\"attachment_9336\" align=\"aligncenter\" width=\"150\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_1.png\"><img class=\"wp-image-9336 size-thumbnail\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_1-150x150.png\" alt=\"rot_1\" width=\"150\" height=\"150\" /></a> Image rotation with bilinear sampling.[/caption]\r\n\r\n[/block_grid_item][block_grid_item]\r\n\r\n[caption id=\"attachment_9329\" align=\"aligncenter\" width=\"150\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_2.png\"><img class=\"wp-image-9329 size-thumbnail\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_2-150x150.png\" alt=\"rot_2\" width=\"150\" height=\"150\" /></a> Image rotation with bilinear-cosine sampling.[/caption]\r\n\r\n[/block_grid_item][block_grid_item]\r\n\r\n[caption id=\"attachment_9330\" align=\"aligncenter\" width=\"150\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_3.png\"><img class=\"wp-image-9330 size-thumbnail\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_3-150x150.png\" alt=\"rot_3\" width=\"150\" height=\"150\" /></a> Image rotation with bicubic sampling.[/caption]\r\n\r\n[/block_grid_item][block_grid_item]\r\n\r\n[/block_grid]\r\n\r\n<span style=\"line-height: 1.5\">[custom_headline type=\"left\" level=\"h2\" looks_like=\"h2\"]Image moments[/custom_headline]</span>\r\n\r\nImage <a href=\"http://arrayfire.org/docs/group__image__func__moments.htm\">moments </a>are another new addition to ArrayFire v3.4. Image moments are weighted averages of pixels in an image which provide useful properties of an image. A common use of image moments is to find the center of mass or area (or gray level sum) of an image.\r\n\r\nCurrently, ArrayFire calculates all<a href=\"http://arrayfire.org/docs/defines_8h.htm#ad3ee8e32709131cf4126dfa26210aec0\"> first order moments</a>. Each moment can be returned individually or all first-order moments can be calculated at once. All moments can be found as follows:\r\n<pre class=\"lang:c++ decode:true\">af::array moments = af::moments(input_image, AF_MOMENT_FIRST_ORDER);</pre>\r\n<div class=\"fragment\">\r\n<div class=\"line\">Here is an example of how the shorthand versions might be used to find the area and center of mass of an image:</div>\r\n</div>\r\n<pre class=\"lang:default decode:true\" title=\"Calculating center of mass of grayscale image\">double m00, m01, m10;\r\naf::moments(&amp;m00, aimg, AF_MOMENT_M00);\r\naf::moments(&amp;m01, aimg, AF_MOMENT_M01);\r\naf::moments(&amp;m10, aimg, AF_MOMENT_M10);\r\n\r\ndouble area = m00;\r\ndouble x_center = m10 / m00;\r\ndouble y_center = m01 / m00;\r\n</pre>\r\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/com.png\"><img class=\"aligncenter wp-image-9246 size-medium\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/com-300x300.png\" alt=\"com\" width=\"300\" height=\"300\" /></a>\r\n\r\n[custom_headline type=\"left\" level=\"h2\" looks_like=\"h2\"]Download[/custom_headline]\r\n\r\nArrayFire v3.4 can be downloaded from these locations:\r\n<ul>\r\n 	<li><a href=\"http://sandbox.arrayfire.com/download/\">Binaries</a></li>\r\n 	<li><a href=\"https://github.com/arrayfire\">Source code</a></li>\r\n</ul>\r\n[custom_headline type=\"left\" level=\"h2\" looks_like=\"h2\"]Community[/custom_headline]\r\n\r\nArrayFire is continually improving through the addition of new image processing and computer vision functions. We look forward to hearing your feedback on our user group and encourage you to share your ArrayFire stories there to inspire and contribute back to our open source community.','New image-based features in ArrayFire v3.4','','inherit','closed','closed','','9166-revision-v1','','','2016-09-20 19:02:45','2016-09-20 23:02:45','',9166,'http://sandbox.arrayfire.com/9166-revision-v1/',0,'revision','',0),(9357,4,'2016-09-21 13:02:52','2016-09-21 17:02:52','<span style=\"font-weight: 400;\">Pseudorandom number generators (PRNGs) are an integral part of many applications in statistics, modeling, and simulations. In ArrayFire v3.4, we introduce random number generation enhancements that improve speed, accuracy, storage, and unity among the ArrayFire backends.</span>\r\n\r\n<span style=\"font-weight: 400;\">Previously in ArrayFire v3.3, each ArrayFire backend used a different PRNG. In ArrayFire v3.4, each ArrayFire backend is able to select from among 3 different random number generators.</span>\r\n<table>\r\n<tbody>\r\n<tr>\r\n<td colspan=\"2\">ArrayFire v3.3 (platform specific)</td>\r\n<td rowspan=\"4\"></td>\r\n<td colspan=\"2\">ArrayFire v3.4 (all generators on all platforms)</td>\r\n</tr>\r\n<tr>\r\n<td>CUDA</td>\r\n<td>XORWOW</td>\r\n<td rowspan=\"3\">CUDA,\r\nOpenCL,\r\nCPU</td>\r\n<td rowspan=\"3\">Philox (CBRNG),\r\nThreefry (CBRNG),\r\nMersenne Twister</td>\r\n</tr>\r\n<tr>\r\n<td>OpenCL</td>\r\n<td>Threefry (CBRNG)</td>\r\n</tr>\r\n<tr>\r\n<td>CPU</td>\r\n<td>Mersenne Twister</td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n<span style=\"font-weight: 400;\">As see above, the XORWOW generator (which was only available for CUDA devices previously) has been replaced by the Philox generator which is available along with Threefry and Mersenne Twister on all ArrayFire supported devices. Philox and Threefry are</span><span style=\"font-weight: 400;\"> <a href=\"http://www.thesalmons.org/john/random123/papers/random123sc11.pdf\">Counter Based Random Number Generators</a> (CBRNGs). CBRNGs have two major advantages over PRNGs:</span>\r\n<ol>\r\n 	<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">Crush Resistance - CBRNGs pass rigorous and extensive statistical tests while Mersenne Twister and XORWOW are not as accurate.</span></li>\r\n 	<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">Smaller Storage Space - The amount of storage space required to maintain state of CBRNGs is minimal compared to Mersenne Twister.</span></li>\r\n</ol>\r\n<span style=\"font-weight: 400;\">For these reasons, ArrayFire v3.4 now includes both the Philox and Threefry CBRNGs. Mersenne Twister will continue to be maintained due to its ubiquity in technical computing.</span>\r\n<h4>Usage</h4>\r\n<span style=\"font-weight: 400;\">Users create a random engine object that contains the state of the engine. These objects are initialized by specifying the random engine type and seed. Then the objects are passed into the <span class=\"lang:default decode:true crayon-inline\">randu</span> and <span class=\"lang:default decode:true crayon-inline\">randn</span> functions which generate the sequence of random numbers.</span>\r\n<pre class=\"lang:c++ decode:true\">unsigned seed;\r\nunsigned elem;\r\naf::randomEngine rp(AF_RANDOM_ENGINE_PHILOX, seed);\r\naf::array R = af::randu(elem, f32, rp);</pre>\r\nAnother method to select a random engine algorithm is to set the default random engine type. If needed the desired seed can be specified as well. Subsequent calls to <span class=\"lang:default decode:true crayon-inline \">randu</span>  and <span class=\"lang:default decode:true crayon-inline \">randn</span>  will utilize the default random engine. These default random engines contain the state of the engine and can be reset or changed by the user.\r\n<pre class=\"lang:c++ decode:true \">unsigned seed;\r\nsetSeed(seed);\r\nsetDefaultRandomEngine(AF_RANDOM_ENGINE_THREEFRY);\r\n//Or AF_RANDOM_ENGINE_DEFAULT, AF_RANDOM_ENGINE_PHILOX, AF_RANDOM_ENGINE_MERSENNE\r\naf::array R = af::randu(elem, f32);//Utilizes threefry PRNG algorithm initialized with \'seed\'</pre>\r\nThe default random engine type is <span class=\"lang:default decode:true crayon-inline \">AF_RANDOM_ENGINE_PHILOX</span> . For further details on usage, please refer to the <a href=\"http://arrayfire.org/docs/group__random__mat.htm\">documentation</a>.\r\n<h4>Performance Numbers</h4>\r\n[caption id=\"attachment_9312\" align=\"aligncenter\" width=\"600\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/cuda.png\"><img class=\"wp-image-9312 size-full\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/cuda.png\" alt=\"OpenCL Backend Performance\" width=\"600\" height=\"371\" /></a> OpenCL Backend Performance[/caption]\r\n\r\n[caption id=\"attachment_9313\" align=\"aligncenter\" width=\"600\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/opencl.png\"><img class=\"wp-image-9313 size-full\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/opencl.png\" alt=\"CUDA Backend Performance\" width=\"600\" height=\"371\" /></a> CUDA Backend Performance[/caption]\r\n<h4>Future Plans</h4>\r\nXORWOW and MRG32K3a random engines will be added in the future. Apart from random engines, users can also benefit from the following random number distributions which are planned for inclusion in subsequent releases :\r\n<ul>\r\n 	<li class=\"task-list-item enabled hovered\">Exponential</li>\r\n 	<li class=\"task-list-item enabled\">Gamma</li>\r\n 	<li class=\"task-list-item enabled\">Chi-squared</li>\r\n 	<li class=\"task-list-item enabled\">Non-central Chi-squared</li>\r\n 	<li class=\"task-list-item enabled\">Poisson</li>\r\n</ul>\r\nWe look forward to your feedback on the improved random number generation and any feature requests to improve it further. Join us on our <a href=\"https://github.com/arrayfire/arrayfire\">GitHub</a> page, <a href=\"https://groups.google.com/forum/#!forum/arrayfire-users\">Google Group</a> or <a href=\"https://gitter.im/arrayfire/arrayfire?utm_source=badge&amp;utm_medium=badge&amp;utm_campaign=pr-badge&amp;utm_content=badge\">Gitter</a> to contribute to the discussion.','Random Number Generators in ArrayFire v3.4','','inherit','closed','closed','','9269-revision-v1','','','2016-09-21 13:02:52','2016-09-21 17:02:52','',9269,'http://sandbox.arrayfire.com/9269-revision-v1/',0,'revision','',0),(9358,4,'2016-09-21 13:03:15','2016-09-21 17:03:15','<span style=\"font-weight: 400;\">Pseudorandom number generators (PRNGs) are an integral part of many applications in statistics, modeling, and simulations. In ArrayFire v3.4, we introduce random number generation enhancements that improve speed, accuracy, storage, and unity among the ArrayFire backends.</span>\r\n\r\n<span style=\"font-weight: 400;\">Previously in ArrayFire v3.3, each ArrayFire backend used a different PRNG. In ArrayFire v3.4, each ArrayFire backend is able to select from among 3 different random number generators.</span>\r\n<table>\r\n<tbody>\r\n<tr>\r\n<td colspan=\"2\">ArrayFire v3.3 (platform specific)</td>\r\n<td rowspan=\"4\"></td>\r\n<td colspan=\"2\">ArrayFire v3.4 (all generators on all platforms)</td>\r\n</tr>\r\n<tr>\r\n<td>CUDA</td>\r\n<td>XORWOW</td>\r\n<td rowspan=\"3\">CUDA,\r\nOpenCL,\r\nCPU</td>\r\n<td rowspan=\"3\">Philox (CBRNG),\r\nThreefry (CBRNG),\r\nMersenne Twister</td>\r\n</tr>\r\n<tr>\r\n<td>OpenCL</td>\r\n<td>Threefry (CBRNG)</td>\r\n</tr>\r\n<tr>\r\n<td>CPU</td>\r\n<td>Mersenne Twister</td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n<span style=\"font-weight: 400;\">As seen above, the XORWOW generator (which was only available for CUDA devices previously) has been replaced by the Philox generator which is available along with Threefry and Mersenne Twister on all ArrayFire supported devices. Philox and Threefry are</span><span style=\"font-weight: 400;\"> <a href=\"http://www.thesalmons.org/john/random123/papers/random123sc11.pdf\">Counter Based Random Number Generators</a> (CBRNGs). CBRNGs have two major advantages over PRNGs:</span>\r\n<ol>\r\n 	<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">Crush Resistance - CBRNGs pass rigorous and extensive statistical tests while Mersenne Twister and XORWOW are not as accurate.</span></li>\r\n 	<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">Smaller Storage Space - The amount of storage space required to maintain state of CBRNGs is minimal compared to Mersenne Twister.</span></li>\r\n</ol>\r\n<span style=\"font-weight: 400;\">For these reasons, ArrayFire v3.4 now includes both the Philox and Threefry CBRNGs. Mersenne Twister will continue to be maintained due to its ubiquity in technical computing.</span>\r\n<h4>Usage</h4>\r\n<span style=\"font-weight: 400;\">Users create a random engine object that contains the state of the engine. These objects are initialized by specifying the random engine type and seed. Then the objects are passed into the <span class=\"lang:default decode:true crayon-inline\">randu</span> and <span class=\"lang:default decode:true crayon-inline\">randn</span> functions which generate the sequence of random numbers.</span>\r\n<pre class=\"lang:c++ decode:true\">unsigned seed;\r\nunsigned elem;\r\naf::randomEngine rp(AF_RANDOM_ENGINE_PHILOX, seed);\r\naf::array R = af::randu(elem, f32, rp);</pre>\r\nAnother method to select a random engine algorithm is to set the default random engine type. If needed the desired seed can be specified as well. Subsequent calls to <span class=\"lang:default decode:true crayon-inline \">randu</span>  and <span class=\"lang:default decode:true crayon-inline \">randn</span>  will utilize the default random engine. These default random engines contain the state of the engine and can be reset or changed by the user.\r\n<pre class=\"lang:c++ decode:true \">unsigned seed;\r\nsetSeed(seed);\r\nsetDefaultRandomEngine(AF_RANDOM_ENGINE_THREEFRY);\r\n//Or AF_RANDOM_ENGINE_DEFAULT, AF_RANDOM_ENGINE_PHILOX, AF_RANDOM_ENGINE_MERSENNE\r\naf::array R = af::randu(elem, f32);//Utilizes threefry PRNG algorithm initialized with \'seed\'</pre>\r\nThe default random engine type is <span class=\"lang:default decode:true crayon-inline \">AF_RANDOM_ENGINE_PHILOX</span> . For further details on usage, please refer to the <a href=\"http://arrayfire.org/docs/group__random__mat.htm\">documentation</a>.\r\n<h4>Performance Numbers</h4>\r\n[caption id=\"attachment_9312\" align=\"aligncenter\" width=\"600\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/cuda.png\"><img class=\"wp-image-9312 size-full\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/cuda.png\" alt=\"OpenCL Backend Performance\" width=\"600\" height=\"371\" /></a> OpenCL Backend Performance[/caption]\r\n\r\n[caption id=\"attachment_9313\" align=\"aligncenter\" width=\"600\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/opencl.png\"><img class=\"wp-image-9313 size-full\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/opencl.png\" alt=\"CUDA Backend Performance\" width=\"600\" height=\"371\" /></a> CUDA Backend Performance[/caption]\r\n<h4>Future Plans</h4>\r\nXORWOW and MRG32K3a random engines will be added in the future. Apart from random engines, users can also benefit from the following random number distributions which are planned for inclusion in subsequent releases :\r\n<ul>\r\n 	<li class=\"task-list-item enabled hovered\">Exponential</li>\r\n 	<li class=\"task-list-item enabled\">Gamma</li>\r\n 	<li class=\"task-list-item enabled\">Chi-squared</li>\r\n 	<li class=\"task-list-item enabled\">Non-central Chi-squared</li>\r\n 	<li class=\"task-list-item enabled\">Poisson</li>\r\n</ul>\r\nWe look forward to your feedback on the improved random number generation and any feature requests to improve it further. Join us on our <a href=\"https://github.com/arrayfire/arrayfire\">GitHub</a> page, <a href=\"https://groups.google.com/forum/#!forum/arrayfire-users\">Google Group</a> or <a href=\"https://gitter.im/arrayfire/arrayfire?utm_source=badge&amp;utm_medium=badge&amp;utm_campaign=pr-badge&amp;utm_content=badge\">Gitter</a> to contribute to the discussion.','Random Number Generators in ArrayFire v3.4','','inherit','closed','closed','','9269-revision-v1','','','2016-09-21 13:03:15','2016-09-21 17:03:15','',9269,'http://sandbox.arrayfire.com/9269-revision-v1/',0,'revision','',0),(9359,4,'2016-09-21 13:40:41','2016-09-21 17:40:41','<span style=\"font-weight: 400;\">Pseudorandom number generators (PRNGs) are an integral part of many applications in statistics, modeling, and simulations. In ArrayFire v3.4, we introduce random number generation enhancements that improve speed, accuracy, storage, and unity among the ArrayFire backends.</span>\r\n\r\n<span style=\"font-weight: 400;\">Previously in ArrayFire v3.3, each ArrayFire backend used a different PRNG. In ArrayFire v3.4, each ArrayFire backend is able to select from among 3 different random number generators.</span>\r\n<table>\r\n<tbody>\r\n<tr>\r\n<td colspan=\"2\">ArrayFire v3.3 (platform specific)</td>\r\n<td rowspan=\"4\"></td>\r\n<td colspan=\"2\">ArrayFire v3.4 (all generators on all platforms)</td>\r\n</tr>\r\n<tr>\r\n<td>CUDA</td>\r\n<td>XORWOW</td>\r\n<td rowspan=\"3\">CUDA,\r\nOpenCL,\r\nCPU</td>\r\n<td rowspan=\"3\">Philox (CBRNG),\r\nThreefry (CBRNG),\r\nMersenne Twister</td>\r\n</tr>\r\n<tr>\r\n<td>OpenCL</td>\r\n<td>Threefry (CBRNG)</td>\r\n</tr>\r\n<tr>\r\n<td>CPU</td>\r\n<td>Mersenne Twister</td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n<span style=\"font-weight: 400;\">As seen above, the XORWOW generator (which was only available for CUDA devices previously) has been replaced by the Philox generator which is available along with Threefry and Mersenne Twister on all ArrayFire supported devices. Philox and Threefry are</span><span style=\"font-weight: 400;\"> <a href=\"http://www.thesalmons.org/john/random123/papers/random123sc11.pdf\">Counter Based Random Number Generators</a> (CBRNGs). CBRNGs have two major advantages over PRNGs:</span>\r\n<ol>\r\n 	<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">Crush Resistance - CBRNGs pass rigorous and extensive statistical tests while Mersenne Twister and XORWOW are not as accurate.</span></li>\r\n 	<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">Smaller Storage Space - The amount of storage space required to maintain the state of CBRNGs is minimal compared to Mersenne Twister.</span></li>\r\n</ol>\r\n<span style=\"font-weight: 400;\">For these reasons, ArrayFire v3.4 now includes both the Philox and Threefry CBRNGs. Mersenne Twister will continue to be maintained due to its ubiquity in technical computing.</span>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Usage[/custom_headline]\r\n<span style=\"font-weight: 400;\">Users create a random engine object that contains the state of the engine. These objects are initialized by specifying the random engine type and seed. Then the objects are passed into the <span class=\"lang:default decode:true crayon-inline\">randu</span> and <span class=\"lang:default decode:true crayon-inline\">randn</span> functions which generate the sequence of random numbers.</span>\r\n<pre class=\"lang:c++ decode:true\">unsigned seed;\r\nunsigned elem;\r\naf::randomEngine rp(AF_RANDOM_ENGINE_PHILOX, seed);\r\naf::array R = af::randu(elem, f32, rp);</pre>\r\nAnother method to select a random engine algorithm is to set the default random engine type. If needed the desired seed can be specified as well. Subsequent calls to <span class=\"lang:default decode:true crayon-inline \">randu</span>  and <span class=\"lang:default decode:true crayon-inline \">randn</span>  will utilize the default random engine. These default random engines contain the state of the engine and can be reset or changed by the user.\r\n<pre class=\"lang:c++ decode:true \">unsigned seed;\r\nsetSeed(seed);\r\nsetDefaultRandomEngine(AF_RANDOM_ENGINE_THREEFRY);\r\n//Or AF_RANDOM_ENGINE_DEFAULT, AF_RANDOM_ENGINE_PHILOX, AF_RANDOM_ENGINE_MERSENNE\r\naf::array R = af::randu(elem, f32);//Utilizes threefry PRNG algorithm initialized with \'seed\'</pre>\r\nThe default random engine type is <span class=\"lang:default decode:true crayon-inline \">AF_RANDOM_ENGINE_PHILOX</span> because it has the best statistical properties among the available generators. For further details on usage, please refer to the <a href=\"http://arrayfire.org/docs/group__random__mat.htm\">documentation</a>.\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Performance Results[/custom_headline]\r\n\r\n[caption id=\"attachment_9312\" align=\"aligncenter\" width=\"600\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/cuda.png\"><img class=\"wp-image-9312 size-full\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/cuda.png\" alt=\"OpenCL Backend Performance\" width=\"600\" height=\"371\" /></a> OpenCL Backend Performance[/caption]\r\n\r\n[caption id=\"attachment_9313\" align=\"aligncenter\" width=\"600\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/opencl.png\"><img class=\"wp-image-9313 size-full\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/opencl.png\" alt=\"CUDA Backend Performance\" width=\"600\" height=\"371\" /></a> CUDA Backend Performance[/caption]\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Future Plans[/custom_headline]\r\n\r\nFuture releases of ArrayFire will include the XORWOW and MRG32K3a generators for all platforms.\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Download[/custom_headline]\r\n\r\nArrayFire v3.4 can be downloaded from these locations:\r\n<ul>\r\n 	<li><a href=\"http://sandbox.arrayfire.com/download/\">Binaries</a></li>\r\n 	<li><a href=\"https://github.com/arrayfire\">Source code</a></li>\r\n</ul>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Community[/custom_headline]\r\n\r\nArrayFire is continually improving through the addition of new random number generators and statistical functions. We welcome your feedback:\r\n<ul>\r\n 	<li>General discussion forums on the <a href=\"https://groups.google.com/forum/#!forum/arrayfire-users\">ArrayFire Google Group</a></li>\r\n 	<li>Live discussion chat on the <a href=\"https://gitter.im/arrayfire/arrayfire?utm_source=badge&amp;utm_medium=badge&amp;utm_campaign=pr-badge&amp;utm_content=badge\">ArrayFire Gitter</a></li>\r\n 	<li>Issue reports on the <a href=\"https://github.com/arrayfire/arrayfire/issues\">ArrayFire GitHub</a></li>\r\n</ul>\r\nFinally, as you find success with ArrayFire, we invite you to contribute a post to this blog to share with the broader community. Email <a href=\"mailto:scott@arrayfire.com\">scott@arrayfire.com</a> to contribute to this blog.','Random Number Generators in ArrayFire v3.4','','inherit','closed','closed','','9269-revision-v1','','','2016-09-21 13:40:41','2016-09-21 17:40:41','',9269,'http://sandbox.arrayfire.com/9269-revision-v1/',0,'revision','',0),(9360,4,'2016-09-21 13:40:44','2016-09-21 17:40:44','There are a number of  additions and updates to image based features in the new v3.4 release of ArrayFire. Among the updates are:\r\n<ul>\r\n 	<li>New interpolation methods for several existing functions\r\n<ul>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__signal__func__approx1.htm\">approx1</a>, <a href=\"http://arrayfire.org/docs/group__signal__func__approx2.htm\">approx2</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__transform.htm\">transform</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__resize.htm\">resize</a></li>\r\n</ul>\r\n</li>\r\n 	<li>Functions for image <a href=\"http://arrayfire.org/docs/group__image__func__moments.htm\">moments</a></li>\r\n</ul>\r\nThis blog post will display some typical use cases for these new features.\r\n\r\n<span style=\"line-height: 1.5;\">[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Interpolation methods[/custom_headline]</span>\r\n\r\nArrayFire v3.4 implements several new <a href=\"http://arrayfire.org/docs/defines_8h.htm#ac0b7c1cfd1fb146314d96f17bbfffa68\"> interpolation methods </a>for 1-d and 2-d domains. The new interpolation methods for 1-d functions are:\r\n<ul>\r\n 	<li><strong>AF_INTERP_LINEAR_COSINE</strong></li>\r\n 	<li><strong>AF_INTERP_CUBIC</strong></li>\r\n</ul>\r\nand for 2-d functions are:\r\n<ul>\r\n 	<li><strong>AF_INTERP_BILINEAR_COSINE</strong></li>\r\n 	<li><strong>AF_INTERP_BICUBIC</strong></li>\r\n</ul>\r\nThe behavior of the <a href=\"http://arrayfire.org/docs/defines_8h.htm#ac0b7c1cfd1fb146314d96f17bbfffa68\">interpolation methods</a> can be seen in the following pictures.\r\n\r\n&nbsp;\r\n\r\n[caption id=\"attachment_9338\" align=\"aligncenter\" width=\"300\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_0.png\"><img class=\"size-medium wp-image-9338\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_0-300x271.png\" alt=\"Original data points\" width=\"300\" height=\"271\" /></a> Original data points[/caption]\r\n\r\n[caption id=\"attachment_9339\" align=\"aligncenter\" width=\"300\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_1.png\"><img class=\"size-medium wp-image-9339\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_1-300x267.png\" alt=\"Linear interpolation\" width=\"300\" height=\"267\" /></a> Linear interpolation[/caption]\r\n\r\n[caption id=\"attachment_9340\" align=\"aligncenter\" width=\"300\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_2.png\"><img class=\"size-medium wp-image-9340\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_2-300x252.png\" alt=\"Linear cosine interpolation\" width=\"300\" height=\"252\" /></a> Linear cosine interpolation[/caption]\r\n\r\n[caption id=\"attachment_9337\" align=\"aligncenter\" width=\"300\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_3.png\"><img class=\"size-medium wp-image-9337\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_3-300x255.png\" alt=\"Cubic interpolation\" width=\"300\" height=\"255\" /></a> Cubic interpolation[/caption]\r\n\r\nA common use for interpolation is image filtering. Given a coarse image, we can resample it to be smoother.\r\n<pre class=\"lang:c++ decode:true\" title=\"Interpolating images with approx2()\">af::array img = af::randu(7,7); //create a random image\r\n\r\n//define sample points for interpolation\r\naf::array Xs = af::seq(0, 6, 0.1f);\r\naf::array Ys = af::seq(0, 6, 0.1f);\r\nXs = af::tile(Xs, 1, Ys.dims(0));\r\nYs = af::tile(Ys.T(), Xs.dims(0));\r\n\r\n//interpolate based on specific method\r\naf::array img_bilinear = af::approx2(img, Xs, Ys, AF_INTERP_BILINEAR);\r\naf::array img_bilinearcos = af::approx2(img, Xs, Ys, AF_INTERP_BILINEAR_COSINE);\r\naf::array img_bicubic = af::approx2(img, Xs, Ys, AF_INTERP_BICUBIC_SPLINE);</pre>\r\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/Capture.png\"><img class=\"aligncenter size-full wp-image-9245\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/Capture.png\" alt=\"Capture\" width=\"1025\" height=\"243\" /></a>\r\n\r\nThe new interpolation methods further apply to several similar image transformation functions:\r\n<ul>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__resize.htm\">Resize</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__rotate.htm\">Rotate</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__scale.htm\">Scale</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__skew.htm\">Skew</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__transform.htm\">Transform</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__translate.htm\">Translate</a></li>\r\n</ul>\r\nDifferent interpolation methods greatly affect the final result of image processing operations. In general, bilinear and bilinear-cosine produce a good result when upscaling an image, however they often result in a blurry look. The bicubic method tends to sharpen the image resulting in slightly stronger borders and a better overall look. Below are some examples of this behavior using image rotation and image scaling.\r\n\r\n[caption id=\"attachment_9331\" align=\"aligncenter\" width=\"300\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/s_0.png\"><img class=\"size-medium wp-image-9331\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/s_0-300x84.png\" alt=\"3x image scaling using nearest-neighbor sampling\" width=\"300\" height=\"84\" /></a> 3x image scaling using nearest-neighbor sampling[/caption]\r\n\r\n[caption id=\"attachment_9332\" align=\"aligncenter\" width=\"300\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/s_1.png\"><img class=\"size-medium wp-image-9332\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/s_1-300x85.png\" alt=\"3x image scaling using bilinear-cosine sampling\" width=\"300\" height=\"85\" /></a> 3x image scaling using bilinear-cosine sampling[/caption]\r\n\r\n[caption id=\"attachment_9334\" align=\"aligncenter\" width=\"300\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/s_2.png\"><img class=\"size-medium wp-image-9334\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/s_2-300x85.png\" alt=\"3x image scaling using bicubic sampling\" width=\"300\" height=\"85\" /></a> 3x image scaling using bicubic sampling[/caption]\r\n\r\n[block_grid type=\"four-up\"] [block_grid_item]\r\n\r\n[caption id=\"attachment_9335\" align=\"aligncenter\" width=\"150\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_0.png\"><img class=\"wp-image-9335 size-thumbnail\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_0-150x150.png\" alt=\"rot_0\" width=\"150\" height=\"150\" /></a> Image rotation with nearest neighbor sampling.[/caption]\r\n\r\n[/block_grid_item][block_grid_item]\r\n\r\n[caption id=\"attachment_9336\" align=\"aligncenter\" width=\"150\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_1.png\"><img class=\"wp-image-9336 size-thumbnail\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_1-150x150.png\" alt=\"rot_1\" width=\"150\" height=\"150\" /></a> Image rotation with bilinear sampling.[/caption]\r\n\r\n[/block_grid_item][block_grid_item]\r\n\r\n[caption id=\"attachment_9329\" align=\"aligncenter\" width=\"150\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_2.png\"><img class=\"wp-image-9329 size-thumbnail\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_2-150x150.png\" alt=\"rot_2\" width=\"150\" height=\"150\" /></a> Image rotation with bilinear-cosine sampling.[/caption]\r\n\r\n[/block_grid_item][block_grid_item]\r\n\r\n[caption id=\"attachment_9330\" align=\"aligncenter\" width=\"150\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_3.png\"><img class=\"wp-image-9330 size-thumbnail\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_3-150x150.png\" alt=\"rot_3\" width=\"150\" height=\"150\" /></a> Image rotation with bicubic sampling.[/caption]\r\n\r\n[/block_grid_item][block_grid_item]\r\n\r\n[/block_grid]\r\n\r\n<span style=\"line-height: 1.5;\">[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Image moments[/custom_headline]</span>\r\n\r\nImage <a href=\"http://arrayfire.org/docs/group__image__func__moments.htm\">moments </a>are another new addition to ArrayFire v3.4. Image moments are weighted averages of pixels in an image which provide useful properties of an image. A common use of image moments is to find the center of mass or area (or gray level sum) of an image.\r\n\r\nCurrently, ArrayFire calculates all<a href=\"http://arrayfire.org/docs/defines_8h.htm#ad3ee8e32709131cf4126dfa26210aec0\"> first order moments</a>. Each moment can be returned individually or all first-order moments can be calculated at once. All moments can be found as follows:\r\n<pre class=\"lang:c++ decode:true\">af::array moments = af::moments(input_image, AF_MOMENT_FIRST_ORDER);</pre>\r\n<div class=\"fragment\">\r\n<div class=\"line\">Here is an example of how the shorthand versions might be used to find the area and center of mass of an image:</div>\r\n</div>\r\n<pre class=\"lang:default decode:true\" title=\"Calculating center of mass of grayscale image\">double m00, m01, m10;\r\naf::moments(&amp;m00, aimg, AF_MOMENT_M00);\r\naf::moments(&amp;m01, aimg, AF_MOMENT_M01);\r\naf::moments(&amp;m10, aimg, AF_MOMENT_M10);\r\n\r\ndouble area = m00;\r\ndouble x_center = m10 / m00;\r\ndouble y_center = m01 / m00;\r\n</pre>\r\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/com.png\"><img class=\"aligncenter wp-image-9246 size-medium\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/com-300x300.png\" alt=\"com\" width=\"300\" height=\"300\" /></a>\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Download[/custom_headline]\r\n\r\nArrayFire v3.4 can be downloaded from these locations:\r\n<ul>\r\n 	<li><a href=\"http://sandbox.arrayfire.com/download/\">Binaries</a></li>\r\n 	<li><a href=\"https://github.com/arrayfire\">Source code</a></li>\r\n</ul>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Community[/custom_headline]\r\n\r\nArrayFire is continually improving through the addition of new image processing and computer vision functions. We welcome your feedback:\r\n<ul>\r\n 	<li>General discussion forums on the <a href=\"https://groups.google.com/forum/#!forum/arrayfire-users\">ArrayFire Google Group</a></li>\r\n 	<li>Live discussion chat on the <a href=\"https://gitter.im/arrayfire/arrayfire?utm_source=badge&amp;utm_medium=badge&amp;utm_campaign=pr-badge&amp;utm_content=badge\">ArrayFire Gitter</a></li>\r\n 	<li>Issue reports on the <a href=\"https://github.com/arrayfire/arrayfire/issues\">ArrayFire GitHub</a></li>\r\n</ul>\r\nFinally, as you find success with ArrayFire, we invite you to contribute a post to this blog to share with the broader community. Email <a href=\"mailto:scott@arrayfire.com\">scott@arrayfire.com</a> to contribute to this blog.','New image-based features in ArrayFire v3.4','','inherit','closed','closed','','9166-revision-v1','','','2016-09-21 13:40:44','2016-09-21 17:40:44','',9166,'http://sandbox.arrayfire.com/9166-revision-v1/',0,'revision','',0),(9362,4,'2016-09-21 17:09:35','2016-09-21 21:09:35','This post outlines the new graphics features available in ArrayFire v3.4:  Vector Fields, Overlays\n\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Vector Fields[/custom_headline]\n\nWe have added visualization support to render ArrayFire <a href=\"http://arrayfire.org/docs/array_8h_source.htm\" target=\"_blank\">array</a> objects as vector fields. An <a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/field.cpp\">example</a> of how to visualize vector fields is included in ArrayFire v3.4. A screenshot of this example\'s output in multi-view mode is shown below, showcasing both static and dynamic vector field rendering.\n\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/vfield.png\"><img class=\"alignnone size-large wp-image-9185\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/vfield-1024x550.png\" alt=\"vfield\" width=\"742\" height=\"399\" /></a>\n\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Overlays[/custom_headline]\n\nPreviously, each graph (such as plot, hist, scatter, etc) was rendered in its own window (or view). Overlaying graphs was not supported. ArrayFire v3.4 now support graph overlays.\n\nEach draw call in ArrayFire is either rendered to a whole window (single view) or to a view, which is a portion of the screen obtained in multiview mode.\n\nThe following image is an example of a graph overlay:\n\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/multipleplot.png\"><img class=\"alignnone size-full wp-image-9182\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/multipleplot.png\" alt=\"multipleplot\" width=\"1002\" height=\"828\" /></a>\n\nThe following graphs support overlay if they are of same type of object (2d/3d):\n<table width=\"100%\">\n<tbody>\n<tr>\n<td><a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/plot3.cpp\" target=\"_blank\">plot</a></td>\n<td><a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/plot3.png\"><img class=\"alignnone size-medium wp-image-9211\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/plot3-300x300.png\" alt=\"plot3\" width=\"300\" height=\"300\" /></a></td>\n</tr>\n<tr>\n<td><a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/histogram.cpp\" target=\"_blank\">histogram</a></td>\n<td><a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/histogram.png\"><img class=\"alignnone size-medium wp-image-9207\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/histogram-300x152.png\" alt=\"histogram\" width=\"300\" height=\"152\" /></a></td>\n</tr>\n<tr>\n<td><a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/surface.cpp\" target=\"_blank\">surface</a></td>\n<td><a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/surface.png\"><img class=\"alignnone size-medium wp-image-9212\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/surface-300x300.png\" alt=\"surface\" width=\"300\" height=\"300\" /></a></td>\n</tr>\n<tr>\n<td>vector fields</td>\n<td></td>\n</tr>\n</tbody>\n</table>\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Animations[/custom_headline]\n\nArrayFire v3.4 includes the following examples of producing fun animations based on simple equations: <a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/conway.cpp\" target=\"_blank\">conway</a>, <a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/conway_pretty.cpp\" target=\"_blank\">conway_pretty</a>, <a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/gravity_sim.cpp\" target=\"_blank\">gravity</a>, and <a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/fractal.cpp\" target=\"_blank\">julia fractal</a>.\n\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/conway.gif\"><img class=\"alignnone size-full wp-image-9227\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/conway.gif\" alt=\"conway\" width=\"256\" height=\"256\" /></a> <a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/conway_pretty.gif\"><img class=\"alignnone size-full wp-image-9228\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/conway_pretty.gif\" alt=\"conway_pretty\" width=\"256\" height=\"256\" /></a> <a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/gravity.gif\"><img class=\"alignnone size-full wp-image-9216\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/gravity.gif\" alt=\"gravity\" width=\"256\" height=\"256\" /></a> <a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/julia.gif\"><img class=\"alignnone size-full wp-image-9240\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/julia.gif\" alt=\"julia\" width=\"256\" height=\"256\" /></a>\n\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Behavioral Changes[/custom_headline]\n\nIn previous versions of ArrayFire, axes limits were computed automatically to fit the data. In ArrayFire v3.4, users are required to set axes limits, as shown below:\n<pre><code class=\"language-cpp\">\naf::Window myWindow(800, 800, \"2D Plot example: ArrayFire\");\n\narray Y;\nint sign = 1;\narray X = seq(-af::Pi, af::Pi, PRECISION);\narray noise = randn(X.dims(0))/5.f;\n\nmyWindow.grid(2, 1);\n// Set the axes limits to view (0,0), prior to v3.4 this was not needed\nmyWindow(0, 0).setAxesLimits(-2 * af::Pi, 2 * af::Pi, -1, 1, true);\n// Set the axes limits to view (0,0), prior to v3.4 this was not needed\nmyWindow(1, 0).setAxesLimits(-2 * af::Pi, 2 * af::Pi, -1.25, 1.25, true);\n\nfor (double val=0; !myWindow.close(); )\n{\n    Y = sin(X);\n\n    myWindow(0,0).plot(X, Y);\n    myWindow(1,0).scatter(X, Y + noise, AF_MARKER_POINT);\n\n    myWindow.show();\n\n    X = X + PRECISION * float(sign);\n    val += PRECISION * float(sign);\n\n    if (val&gt;af::Pi) {\n        sign = -1;\n    } else if (val&lt;-af::Pi) {\n        sign = 1;\n    }\n}\n</code></pre>\nThe code above produces the output below. You can find the complete example <a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/plot2d.cpp\" target=\"_blank\">here</a>.\n\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/plot2d.png\"><img class=\"alignnone size-large wp-image-9183\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/plot2d-1024x927.png\" alt=\"plot2d\" width=\"742\" height=\"672\" /></a>\n\n<strong>Miscellaneous Change</strong>\n\nRemoved <a href=\"http://glew.sourceforge.net/\">GLEW</a> and replaced with <a href=\"https://github.com/cginternals/glbinding\">glbinding</a>.\n\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Download[/custom_headline]\n\nArrayFire v3.4 can be downloaded from these locations:\n<ul>\n 	<li><a href=\"http://sandbox.arrayfire.com/download/\">Binaries</a></li>\n 	<li><a href=\"https://github.com/arrayfire\">Source code</a></li>\n</ul>\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Community[/custom_headline]\n\nArrayFire is continually improving through the addition of new visualization functions. We welcome your feedback:\n<ul>\n 	<li>General discussion forums on the <a href=\"https://groups.google.com/forum/#!forum/arrayfire-users\">ArrayFire Google Group</a></li>\n 	<li>Live discussion chat on the <a href=\"https://gitter.im/arrayfire/arrayfire?utm_source=badge&amp;utm_medium=badge&amp;utm_campaign=pr-badge&amp;utm_content=badge\">ArrayFire Gitter</a></li>\n 	<li>Issue reports on the <a href=\"https://github.com/arrayfire/arrayfire/issues\">ArrayFire GitHub</a></li>\n</ul>\nFinally, as you find success with ArrayFire, we invite you to contribute a post to this blog to share with the broader community. Email <a href=\"mailto:scott@arrayfire.com\">scott@arrayfire.com</a> to contribute to this blog.','Graphics Updates in ArrayFire v3.4','','inherit','closed','closed','','9172-autosave-v1','','','2016-09-21 17:09:35','2016-09-21 21:09:35','',9172,'http://sandbox.arrayfire.com/9172-autosave-v1/',0,'revision','',0),(9363,4,'2016-09-21 17:09:39','2016-09-21 21:09:39','This post outlines the new graphics features available in ArrayFire v3.4:  Vector Fields, Overlays\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Vector Fields[/custom_headline]\r\n\r\nWe have added visualization support to render ArrayFire <a href=\"http://arrayfire.org/docs/array_8h_source.htm\" target=\"_blank\">array</a> objects as vector fields. An <a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/field.cpp\">example</a> of how to visualize vector fields is included in ArrayFire v3.4. A screenshot of this example\'s output in multi-view mode is shown below, showcasing both static and dynamic vector field rendering.\r\n\r\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/vfield.png\"><img class=\"alignnone size-large wp-image-9185\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/vfield-1024x550.png\" alt=\"vfield\" width=\"742\" height=\"399\" /></a>\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Overlays[/custom_headline]\r\n\r\nPreviously, each graph (such as plot, hist, scatter, etc) was rendered in its own window (or view). Overlaying graphs was not supported. ArrayFire v3.4 now support graph overlays.\r\n\r\nEach draw call in ArrayFire is either rendered to a whole window (single view) or to a view, which is a portion of the screen obtained in multiview mode.\r\n\r\nThe following image is an example of a graph overlay:\r\n\r\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/multipleplot.png\"><img class=\"alignnone size-full wp-image-9182\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/multipleplot.png\" alt=\"multipleplot\" width=\"1002\" height=\"828\" /></a>\r\n\r\nThe following graphs support overlay if they are of same type of object (2d/3d):\r\n<table width=\"100%\">\r\n<tbody>\r\n<tr>\r\n<td><a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/plot3.cpp\" target=\"_blank\">plot</a></td>\r\n<td><a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/plot3.png\"><img class=\"alignnone size-medium wp-image-9211\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/plot3-300x300.png\" alt=\"plot3\" width=\"300\" height=\"300\" /></a></td>\r\n</tr>\r\n<tr>\r\n<td><a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/histogram.cpp\" target=\"_blank\">histogram</a></td>\r\n<td><a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/histogram.png\"><img class=\"alignnone size-medium wp-image-9207\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/histogram-300x152.png\" alt=\"histogram\" width=\"300\" height=\"152\" /></a></td>\r\n</tr>\r\n<tr>\r\n<td><a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/surface.cpp\" target=\"_blank\">surface</a></td>\r\n<td><a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/surface.png\"><img class=\"alignnone size-medium wp-image-9212\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/surface-300x300.png\" alt=\"surface\" width=\"300\" height=\"300\" /></a></td>\r\n</tr>\r\n<tr>\r\n<td>vector fields</td>\r\n<td></td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Animations[/custom_headline]\r\n\r\nArrayFire v3.4 includes the following examples of producing fun animations based on simple equations: <a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/conway.cpp\" target=\"_blank\">conway</a>, <a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/conway_pretty.cpp\" target=\"_blank\">conway_pretty</a>, <a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/gravity_sim.cpp\" target=\"_blank\">gravity</a>, and <a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/fractal.cpp\" target=\"_blank\">julia fractal</a>.\r\n\r\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/conway.gif\"><img class=\"alignnone size-full wp-image-9227\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/conway.gif\" alt=\"conway\" width=\"256\" height=\"256\" /></a> <a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/conway_pretty.gif\"><img class=\"alignnone size-full wp-image-9228\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/conway_pretty.gif\" alt=\"conway_pretty\" width=\"256\" height=\"256\" /></a> <a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/gravity.gif\"><img class=\"alignnone size-full wp-image-9216\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/gravity.gif\" alt=\"gravity\" width=\"256\" height=\"256\" /></a> <a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/julia.gif\"><img class=\"alignnone size-full wp-image-9240\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/julia.gif\" alt=\"julia\" width=\"256\" height=\"256\" /></a>\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Behavioral Changes[/custom_headline]\r\n\r\nIn previous versions of ArrayFire, axes limits were computed automatically to fit the data. In ArrayFire v3.4, users are required to set axes limits, as shown below:\r\n<pre><code class=\"language-cpp\">\r\naf::Window myWindow(800, 800, \"2D Plot example: ArrayFire\");\r\n\r\narray Y;\r\nint sign = 1;\r\narray X = seq(-af::Pi, af::Pi, PRECISION);\r\narray noise = randn(X.dims(0))/5.f;\r\n\r\nmyWindow.grid(2, 1);\r\n// Set the axes limits to view (0,0), prior to v3.4 this was not needed\r\nmyWindow(0, 0).setAxesLimits(-2 * af::Pi, 2 * af::Pi, -1, 1, true);\r\n// Set the axes limits to view (0,0), prior to v3.4 this was not needed\r\nmyWindow(1, 0).setAxesLimits(-2 * af::Pi, 2 * af::Pi, -1.25, 1.25, true);\r\n\r\nfor (double val=0; !myWindow.close(); )\r\n{\r\n    Y = sin(X);\r\n\r\n    myWindow(0,0).plot(X, Y);\r\n    myWindow(1,0).scatter(X, Y + noise, AF_MARKER_POINT);\r\n\r\n    myWindow.show();\r\n\r\n    X = X + PRECISION * float(sign);\r\n    val += PRECISION * float(sign);\r\n\r\n    if (val&gt;af::Pi) {\r\n        sign = -1;\r\n    } else if (val&lt;-af::Pi) {\r\n        sign = 1;\r\n    }\r\n}\r\n</code></pre>\r\nThe code above produces the output below. You can find the complete example <a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/plot2d.cpp\" target=\"_blank\">here</a>.\r\n\r\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/plot2d.png\"><img class=\"alignnone size-large wp-image-9183\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/plot2d-1024x927.png\" alt=\"plot2d\" width=\"742\" height=\"672\" /></a>\r\n\r\n<strong>Miscellaneous Change</strong>\r\n\r\nRemoved <a href=\"http://glew.sourceforge.net/\">GLEW</a> and replaced with <a href=\"https://github.com/cginternals/glbinding\">glbinding</a>.\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Download[/custom_headline]\r\n\r\nArrayFire v3.4 can be downloaded from these locations:\r\n<ul>\r\n 	<li><a href=\"http://sandbox.arrayfire.com/download/\">Binaries</a></li>\r\n 	<li><a href=\"https://github.com/arrayfire\">Source code</a></li>\r\n</ul>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Community[/custom_headline]\r\n\r\nArrayFire is continually improving through the addition of new visualization functions. We welcome your feedback:\r\n<ul>\r\n 	<li>General discussion forums on the <a href=\"https://groups.google.com/forum/#!forum/arrayfire-users\">ArrayFire Google Group</a></li>\r\n 	<li>Live discussion chat on the <a href=\"https://gitter.im/arrayfire/arrayfire?utm_source=badge&amp;utm_medium=badge&amp;utm_campaign=pr-badge&amp;utm_content=badge\">ArrayFire Gitter</a></li>\r\n 	<li>Issue reports on the <a href=\"https://github.com/arrayfire/arrayfire/issues\">ArrayFire GitHub</a></li>\r\n</ul>\r\nFinally, as you find success with ArrayFire, we invite you to contribute a post to this blog to share with the broader community. Email <a href=\"mailto:scott@arrayfire.com\">scott@arrayfire.com</a> to contribute to this blog.','Graphics Updates in ArrayFire v3.4','','inherit','closed','closed','','9172-revision-v1','','','2016-09-21 17:09:39','2016-09-21 21:09:39','',9172,'http://sandbox.arrayfire.com/9172-revision-v1/',0,'revision','',0),(9364,18607,'2016-09-21 17:12:17','2016-09-21 21:12:17','<span style=\"font-weight: 400;\">Pseudorandom number generators (PRNGs) are an integral part of many applications in statistics, modeling, and simulations. In ArrayFire v3.4, we introduce random number generation enhancements that improve speed, accuracy, storage, and unity among the ArrayFire backends.</span>\r\n\r\n<span style=\"font-weight: 400;\">Previously in ArrayFire v3.3, each ArrayFire backend used a different PRNG. In ArrayFire v3.4, each ArrayFire backend is able to select from among 3 different random number generators.</span>\r\n<table>\r\n<tbody>\r\n<tr>\r\n<td colspan=\"2\">ArrayFire v3.3 (platform specific)</td>\r\n<td rowspan=\"4\"></td>\r\n<td colspan=\"2\">ArrayFire v3.4 (all generators on all platforms)</td>\r\n</tr>\r\n<tr>\r\n<td>CUDA</td>\r\n<td>XORWOW</td>\r\n<td rowspan=\"3\">CUDA,\r\nOpenCL,\r\nCPU</td>\r\n<td rowspan=\"3\">Philox (CBRNG),\r\nThreefry (CBRNG),\r\nMersenne Twister</td>\r\n</tr>\r\n<tr>\r\n<td>OpenCL</td>\r\n<td>Threefry (CBRNG)</td>\r\n</tr>\r\n<tr>\r\n<td>CPU</td>\r\n<td>Mersenne Twister</td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n<span style=\"font-weight: 400;\">As seen above, the XORWOW generator (which was only available for CUDA devices previously) has been replaced by the Philox generator which is available along with Threefry and Mersenne Twister on all ArrayFire supported devices. Philox and Threefry are</span><span style=\"font-weight: 400;\"> <a href=\"http://www.thesalmons.org/john/random123/papers/random123sc11.pdf\">Counter Based Random Number Generators</a> (CBRNGs). CBRNGs have two major advantages over traditional PRNGs:</span>\r\n<ol>\r\n 	<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">Crush Resistance - CBRNGs pass rigorous and extensive statistical tests while Mersenne Twister and XORWOW are not as accurate.</span></li>\r\n 	<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">Smaller Storage Space - The amount of storage space required to maintain the state of CBRNGs is minimal compared to Mersenne Twister.</span></li>\r\n</ol>\r\n<span style=\"font-weight: 400;\">For these reasons, ArrayFire v3.4 now includes both the Philox and Threefry CBRNGs. Mersenne Twister will continue to be maintained due to its ubiquity in technical computing.</span>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Usage[/custom_headline]\r\n<span style=\"font-weight: 400;\">Users create a random engine object that contains the state of the engine. These objects are initialized by specifying the random engine type and seed. Then the objects are passed into the <span class=\"lang:default decode:true crayon-inline\">randu</span> and <span class=\"lang:default decode:true crayon-inline\">randn</span> functions which generate the sequence of random numbers.</span>\r\n<pre class=\"lang:c++ decode:true\">unsigned seed;\r\nunsigned elem;\r\naf::randomEngine rp(AF_RANDOM_ENGINE_PHILOX, seed);\r\naf::array R = af::randu(elem, f32, rp);</pre>\r\nAnother method to select a random engine algorithm is to set the default random engine type. If needed the desired seed can be specified as well. Subsequent calls to <span class=\"lang:default decode:true crayon-inline \">randu</span>  and <span class=\"lang:default decode:true crayon-inline \">randn</span>  will utilize the default random engine. These default random engines contain the state of the engine and can be reset or changed by the user.\r\n<pre class=\"lang:c++ decode:true \">unsigned seed;\r\nsetSeed(seed);\r\nsetDefaultRandomEngine(AF_RANDOM_ENGINE_THREEFRY);\r\n//Or AF_RANDOM_ENGINE_DEFAULT, AF_RANDOM_ENGINE_PHILOX, AF_RANDOM_ENGINE_MERSENNE\r\naf::array R = af::randu(elem, f32);//Utilizes threefry PRNG algorithm initialized with \'seed\'</pre>\r\nThe default random engine type is <span class=\"lang:default decode:true crayon-inline \">AF_RANDOM_ENGINE_PHILOX</span> because it has the best statistical properties among the available generators. For further details on usage, please refer to the <a href=\"http://arrayfire.org/docs/group__random__mat.htm\">documentation</a>.\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Performance Results[/custom_headline]\r\n\r\n[caption id=\"attachment_9312\" align=\"aligncenter\" width=\"600\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/cuda.png\"><img class=\"wp-image-9312 size-full\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/cuda.png\" alt=\"OpenCL Backend Performance\" width=\"600\" height=\"371\" /></a> OpenCL Backend Performance[/caption]\r\n\r\n[caption id=\"attachment_9313\" align=\"aligncenter\" width=\"600\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/opencl.png\"><img class=\"wp-image-9313 size-full\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/opencl.png\" alt=\"CUDA Backend Performance\" width=\"600\" height=\"371\" /></a> CUDA Backend Performance[/caption]\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Future Plans[/custom_headline]\r\n\r\nFuture releases of ArrayFire will include the XORWOW and MRG32K3a generators for all platforms.\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Download[/custom_headline]\r\n\r\nArrayFire v3.4 can be downloaded from these locations:\r\n<ul>\r\n 	<li><a href=\"http://sandbox.arrayfire.com/download/\">Binaries</a></li>\r\n 	<li><a href=\"https://github.com/arrayfire\">Source code</a></li>\r\n</ul>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Community[/custom_headline]\r\n\r\nArrayFire is continually improving through the addition of new random number generators and statistical functions. We welcome your feedback:\r\n<ul>\r\n 	<li>General discussion forums on the <a href=\"https://groups.google.com/forum/#!forum/arrayfire-users\">ArrayFire Google Group</a></li>\r\n 	<li>Live discussion chat on the <a href=\"https://gitter.im/arrayfire/arrayfire?utm_source=badge&amp;utm_medium=badge&amp;utm_campaign=pr-badge&amp;utm_content=badge\">ArrayFire Gitter</a></li>\r\n 	<li>Issue reports on the <a href=\"https://github.com/arrayfire/arrayfire/issues\">ArrayFire GitHub</a></li>\r\n</ul>\r\nFinally, as you find success with ArrayFire, we invite you to contribute a post to this blog to share with the broader community. Email <a href=\"mailto:scott@arrayfire.com\">scott@arrayfire.com</a> to contribute to this blog.','Random Number Generators in ArrayFire v3.4','','inherit','closed','closed','','9269-revision-v1','','','2016-09-21 17:12:17','2016-09-21 21:12:17','',9269,'http://sandbox.arrayfire.com/9269-revision-v1/',0,'revision','',0),(9365,4,'2016-09-21 17:36:37','2016-09-21 21:36:37','ArrayFire uses <a href=\"http://sandbox.arrayfire.com/performance-of-arrayfire-jit-code-generation/\">Just In Time</a> compilation to combine many light weight functions into a single kernel launch. This along with our <a href=\"http://arrayfire.org/docs/group__arrayfire__func.htm\">easy-to-use API</a> allows users to not only quickly prototype their algorithms, but also get the best out of the underlying hardware.\n\nThis feature has been a favorite among our users in the domains of finance and scientific simulation.\n\nThat said, ArrayFire v3.3 and earlier had a few limitations. Namely:\n<ul class=\"org-ul\">\n 	<li>Multiple outputs with inter-dependent variables were generating multiple kernels.</li>\n 	<li>The number of operations per kernel was fairly limited by default.</li>\n</ul>\nIn the latest release of ArrayFire, we addressed these issues to get some pretty impressive numbers.\n\nIn the rest of the post, we demonstrate the performance improvements using our <a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/financial/black_scholes_options.cpp\">BlackScholes example</a>. For this particular example, the improvements for the CUDA backend were <b>~8x</b> and OpenCL backend were between <b>~5x</b>.\n\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Performance Comparison between v3.4 and v3.3[/custom_headline]\n<div id=\"outline-container-sec-1\" class=\"outline-2\">\n<div id=\"text-1\" class=\"outline-text-2\">\n\n<a href=\"https://en.wikipedia.org/wiki/Black%25E2%2580%2593Scholes_model\">BlackScholes</a> is a commonly used mathematical model in financial markets.\n\nIt takes the following 5 inputs and generates 2 outputs.\n\nThe code to implement this model in ArrayFire can be seen below.\n<div class=\"org-src-container\">\n<pre class=\"src src-C++\">array cnd(array x)\n{\n    // constants\n    const float a1 =  0.254829592;\n    const float a2 = -0.284496736;\n    const float a3 =  1.421413741;\n    const float a4 = -1.453152027;\n    const float a5 =  1.061405429;\n    const float p  =  0.3275911;\n    const float sqrt2 = sqrt(2.0);\n\n    // Save the sign of x\n    array xSign = sign(x);\n\n    x = abs(x) / sqrt2;\n\n    // A&amp;S formula 7.1.26\n    array t = 1.0f / (1.0f + p*x);\n    array y = 1.0f + 0.5f * (((((a5*t + a4)*t) + a3)*t + a2)*t + a1)*t*exp(-x*x);\n\n    return xSign * y + !xSign * (1 - y); // equivalent of (x &gt;= 0) ? y : (1 - y);\n}\n\nstatic void black_scholes(array&amp; C, array&amp; P,\n                          const array&amp; S, const array&amp; X,\n                          const array&amp; R, const array&amp; V,\n                          const array&amp; T)\n{\n    // Inputs:\n    // S: Stock price\n    // X: Strike price\n    // R: Risk free rate of interest\n    // V: Volatility\n    // T: Time to maturity\n\n    // Outputs:\n    // C: Call option\n    // P: Put option\n\n    array d1 = log(S / X);\n    d1 = d1 + (R + (V*V)*0.5) * T;\n    d1 = d1 / (V*sqrt(T));\n\n    array d2 = d1 - (V*sqrt(T));\n\n    array cnd_d1 = cnd(d1);\n    array cnd_d2 = cnd(d2);\n\n    C = S * cnd_d1  - (X * exp((-R)*T) * cnd_d2);\n    P = X * exp((-R)*T) * (1 - cnd_d2) - (S * (1 - cnd_d1));\n}\n</pre>\n</div>\nThe <code>black_scholes</code> function contains a long chain of operations and generates two outputs. In older versions of ArrayFire, the Just-In-Time compiler of arrayfire was splitting up this single function call to multiple kernels.\n\nWe solved this in the latest release by:\n<ul class=\"org-ul\">\n 	<li>Enabling <code>af::eval</code> to hint ArrayFire\'s JIT compiler to combine inputs.</li>\n 	<li>Changing the heuristics to increase the number of operations per kernel.</li>\n</ul>\n<div class=\"org-src-container\">\n<pre class=\"src src-C++\">black_scholes(C, P, S, X, R, V, T);\neval(C, P); // Only change required for v3.4\n</pre>\n</div>\nThese changes resulted in dramatic performance improvements seen below. The following graph compares the number of call and put options being calculated per second on v3.3 and v3.4.\n\n&nbsp;\n\n<iframe width=\"650\" height=\"400\" seamless=\"\" frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1o4C5KZ4B5he3Ac8mJ9Z6Z_N-hlATeJwhP-j6XJtbEPA/pubchart?oid=927257953&amp;format=interactive\">\n</iframe>\n\n&nbsp;\n\nAs we can see, the performance has improved significantly. The absolute time taken when to calculate 2 million options is show below.\n\n&nbsp;\n\n<iframe width=\"513.5\" height=\"104.0141666666666\" seamless=\"\" frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1o4C5KZ4B5he3Ac8mJ9Z6Z_N-hlATeJwhP-j6XJtbEPA/pubchart?oid=1295781261&amp;format=interactive\"></iframe>\n\nNote:\n<ul class=\"org-ul\">\n 	<li>The additional gains made by CUDA are because of a <a href=\"https://github.com/arrayfire/arrayfire/pull/1534\">separate issue</a> involving CUDA JIT that was fixed.</li>\n 	<li>Only 24 threads of the 32 available were used for the Xeon CPU.</li>\n</ul>\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Performance Comparison to Native CUDA[/custom_headline]\n\n</div>\n</div>\n<div id=\"outline-container-sec-2\" class=\"outline-2\">\n<div id=\"text-2\" class=\"outline-text-2\">\n\nTo see how close we can get to the peak performance, we compared the performance of ArrayFire and CUDA on the new GTX 1080.\n\nTo do this we built ArrayFire 3.4 using the CUDA 8 RC and compared its performance with that of the example provided in <a href=\"http://docs.nvidia.com/cuda/cuda-samples/#black-scholes-option-pricing\">CUDA Samples</a>.\n\nThe results of this benchmark can be seen below.\n\n<iframe width=\"655.1414131325375\" height=\"323.5\" seamless=\"\" frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1o4C5KZ4B5he3Ac8mJ9Z6Z_N-hlATeJwhP-j6XJtbEPA/pubchart?oid=256763192&amp;format=interactive\"></iframe>\n\nThe results indicate ArrayFire v3.4 performs identical to the native kernel at larger simulation sizes, but there is a constant overhead causing issues at smaller sizes. This is one of the issues we want to address in future releases.\n\nNote: The <code>black_scholes</code> function had to be changed slightly as shown below to be similar to the sample provided by CUDA.\n\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Coming Soon[/custom_headline]\n\n</div>\n</div>\n<div id=\"outline-container-sec-3\" class=\"outline-2\">\n<div id=\"text-3\" class=\"outline-text-2\">\n\nWhile the new improvements to the JIT engine in ArrayFire can seamlessly generate kernels that perform close to handwritten native kernels on GPUs, there is room for improvement at smaller sizes. To see what lies ahead for ArrayFire JIT, please check out <a href=\"https://github.com/arrayfire/arrayfire/issues/1401\">this github issue</a>.\n\n</div>\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Download[/custom_headline]\n\nArrayFire v3.4 can be downloaded from these locations:\n<ul>\n 	<li><a href=\"http://sandbox.arrayfire.com/download/\">Binaries</a></li>\n 	<li><a href=\"https://github.com/arrayfire\">Source code</a></li>\n</ul>\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Community[/custom_headline]\n\nArrayFire is continually improving through the addition of new JIT enhancements. We welcome your feedback:\n<ul>\n 	<li>General discussion forums on the <a href=\"https://groups.google.com/forum/#!forum/arrayfire-users\">ArrayFire Google Group</a></li>\n 	<li>Live discussion chat on the <a href=\"https://gitter.im/arrayfire/arrayfire?utm_source=badge&amp;utm_medium=badge&amp;utm_campaign=pr-badge&amp;utm_content=badge\">ArrayFire Gitter</a></li>\n 	<li>Issue reports on the <a href=\"https://github.com/arrayfire/arrayfire/issues\">ArrayFire GitHub</a></li>\n</ul>\nFinally, as you find success with ArrayFire, we invite you to contribute a post to this blog to share with the broader community. Email <a href=\"mailto:scott@arrayfire.com\">scott@arrayfire.com</a> to contribute to this blog.\n\n&nbsp;\n\n</div>','Performance Improvements to JIT in ArrayFire v3.4','','inherit','closed','closed','','9252-autosave-v1','','','2016-09-21 17:36:37','2016-09-21 21:36:37','',9252,'http://sandbox.arrayfire.com/9252-autosave-v1/',0,'revision','',0),(9366,4,'2016-09-21 17:18:30','2016-09-21 21:18:30','ArrayFire uses <a href=\"http://sandbox.arrayfire.com/performance-of-arrayfire-jit-code-generation/\">Just In Time</a> compilation to combine many light weight functions into a single kernel launch. This along with our <a href=\"http://arrayfire.org/docs/group__arrayfire__func.htm\">easy-to-use API</a> allows users to not only quickly prototype their algorithms, but also get the best out of the underlying hardware.\r\n\r\nThis feature has been a favorite among our users in the domains of finance and scientific simulation.\r\n\r\nThat said, ArrayFire v3.3 and earlier had a few limitations. Namely:\r\n<ul class=\"org-ul\">\r\n 	<li>Multiple outputs with inter-dependent variables were generating multiple kernels.</li>\r\n 	<li>The number of operations per kernel was fairly limited by default.</li>\r\n</ul>\r\nIn the latest release of ArrayFire, we addressed these issues to get some pretty impressive numbers.\r\n\r\nIn the rest of the post, we demonstrate the performance improvements using our <a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/financial/black_scholes_options.cpp\">BlackScholes example</a>. For this particular example, the improvements for the CUDA backend were <b>~8x</b> and OpenCL backend were between <b>~5x</b>.\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Performance Comparison between v3.4 and v3.3[/custom_headline]\r\n<div id=\"outline-container-sec-1\" class=\"outline-2\">\r\n<div id=\"text-1\" class=\"outline-text-2\">\r\n\r\n<a href=\"https://en.wikipedia.org/wiki/Black%25E2%2580%2593Scholes_model\">BlackScholes</a> is a commonly used mathematical model in financial markets.\r\n\r\nIt takes the following 5 inputs and generates 2 outputs.\r\n\r\nThe code to implement this model in ArrayFire can be seen below.\r\n<div class=\"org-src-container\">\r\n<pre class=\"src src-C++\">array cnd(array x)\r\n{\r\n    // constants\r\n    const float a1 =  0.254829592;\r\n    const float a2 = -0.284496736;\r\n    const float a3 =  1.421413741;\r\n    const float a4 = -1.453152027;\r\n    const float a5 =  1.061405429;\r\n    const float p  =  0.3275911;\r\n    const float sqrt2 = sqrt(2.0);\r\n\r\n    // Save the sign of x\r\n    array xSign = sign(x);\r\n\r\n    x = abs(x) / sqrt2;\r\n\r\n    // A&amp;S formula 7.1.26\r\n    array t = 1.0f / (1.0f + p*x);\r\n    array y = 1.0f + 0.5f * (((((a5*t + a4)*t) + a3)*t + a2)*t + a1)*t*exp(-x*x);\r\n\r\n    return xSign * y + !xSign * (1 - y); // equivalent of (x &gt;= 0) ? y : (1 - y);\r\n}\r\n\r\nstatic void black_scholes(array&amp; C, array&amp; P,\r\n                          const array&amp; S, const array&amp; X,\r\n                          const array&amp; R, const array&amp; V,\r\n                          const array&amp; T)\r\n{\r\n    // Inputs:\r\n    // S: Stock price\r\n    // X: Strike price\r\n    // R: Risk free rate of interest\r\n    // V: Volatility\r\n    // T: Time to maturity\r\n\r\n    // Outputs:\r\n    // C: Call option\r\n    // P: Put option\r\n\r\n    array d1 = log(S / X);\r\n    d1 = d1 + (R + (V*V)*0.5) * T;\r\n    d1 = d1 / (V*sqrt(T));\r\n\r\n    array d2 = d1 - (V*sqrt(T));\r\n\r\n    array cnd_d1 = cnd(d1);\r\n    array cnd_d2 = cnd(d2);\r\n\r\n    C = S * cnd_d1  - (X * exp((-R)*T) * cnd_d2);\r\n    P = X * exp((-R)*T) * (1 - cnd_d2) - (S * (1 - cnd_d1));\r\n}\r\n</pre>\r\n</div>\r\nThe <code>black_scholes</code> function contains a long chain of operations and generates two outputs. In older versions of ArrayFire, the Just-In-Time compiler of arrayfire was splitting up this single function call to multiple kernels.\r\n\r\nWe solved this in the latest release by:\r\n<ul class=\"org-ul\">\r\n 	<li>Enabling <code>af::eval</code> to hint ArrayFire\'s JIT compiler to combine inputs.</li>\r\n 	<li>Changing the heuristics to increase the number of operations per kernel.</li>\r\n</ul>\r\n<div class=\"org-src-container\">\r\n<pre class=\"src src-C++\">black_scholes(C, P, S, X, R, V, T);\r\neval(C, P); // Only change required for v3.4\r\n</pre>\r\n</div>\r\nThese changes resulted in dramatic performance improvements seen below. The following graph compares the number of call and put options being calculated per second on v3.3 and v3.4.\r\n\r\n&nbsp;\r\n\r\n<iframe width=\"650\" height=\"400\" seamless=\"\" frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1o4C5KZ4B5he3Ac8mJ9Z6Z_N-hlATeJwhP-j6XJtbEPA/pubchart?oid=927257953&amp;format=interactive\">\r\n</iframe>\r\n\r\n&nbsp;\r\n\r\nAs we can see, the performance has improved significantly. The absolute time taken when to calculate 2 million options is show below.\r\n\r\n&nbsp;\r\n\r\n<iframe width=\"513.5\" height=\"104.0141666666666\" seamless=\"\" frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1o4C5KZ4B5he3Ac8mJ9Z6Z_N-hlATeJwhP-j6XJtbEPA/pubchart?oid=1295781261&amp;format=interactive\"></iframe>\r\n\r\nNote:\r\n<ul class=\"org-ul\">\r\n 	<li>The additional gains made by CUDA are because of a <a href=\"https://github.com/arrayfire/arrayfire/pull/1534\">separate issue</a> involving CUDA JIT that was fixed.</li>\r\n 	<li>Only 24 threads of the 32 available were used for the Xeon CPU.</li>\r\n</ul>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Performance Comparison to Native CUDA[/custom_headline]\r\n\r\n</div>\r\n</div>\r\n<div id=\"outline-container-sec-2\" class=\"outline-2\">\r\n<div id=\"text-2\" class=\"outline-text-2\">\r\n\r\nTo see how close we can get to the peak performance, we compared the performance of ArrayFire and CUDA on the new GTX 1080.\r\n\r\nTo do this we built ArrayFire 3.4 using the CUDA 8 RC and compared its performance with that of the example provided in <a href=\"http://docs.nvidia.com/cuda/cuda-samples/#black-scholes-option-pricing\">CUDA Samples</a>.\r\n\r\nThe results of this benchmark can be seen below.\r\n\r\n<iframe width=\"655.1414131325375\" height=\"323.5\" seamless=\"\" frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1o4C5KZ4B5he3Ac8mJ9Z6Z_N-hlATeJwhP-j6XJtbEPA/pubchart?oid=256763192&amp;format=interactive\"></iframe>\r\n\r\nThe results indicate ArrayFire v3.4 performs identical to the native kernel at larger simulation sizes, but there is a constant overhead causing issues at smaller sizes. This is one of the issues we want to address in future releases.\r\n\r\nNote: The <code>black_scholes</code> function had to be changed slightly as shown below to be similar to the sample provided by CUDA.\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Coming Soon[/custom_headline]\r\n\r\n</div>\r\n</div>\r\n<div id=\"outline-container-sec-3\" class=\"outline-2\">\r\n<div id=\"text-3\" class=\"outline-text-2\">\r\n\r\nWhile the new improvements to the JIT engine in ArrayFire can seamlessly generate kernels that perform close to handwritten native kernels on GPUs, there is room for improvement at smaller sizes. To see what lies ahead for ArrayFire JIT, please check out <a href=\"https://github.com/arrayfire/arrayfire/issues/1401\">this github issue</a>.\r\n\r\n</div>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Download[/custom_headline]\r\n\r\nArrayFire v3.4 can be downloaded from these locations:\r\n<ul>\r\n 	<li><a href=\"http://sandbox.arrayfire.com/download/\">Binaries</a></li>\r\n 	<li><a href=\"https://github.com/arrayfire\">Source code</a></li>\r\n</ul>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Community[/custom_headline]\r\n\r\nArrayFire is continually improving through the addition of new JIT enhancements. We welcome your feedback:\r\n<ul>\r\n 	<li>General discussion forums on the <a href=\"https://groups.google.com/forum/#!forum/arrayfire-users\">ArrayFire Google Group</a></li>\r\n 	<li>Live discussion chat on the <a href=\"https://gitter.im/arrayfire/arrayfire?utm_source=badge&amp;utm_medium=badge&amp;utm_campaign=pr-badge&amp;utm_content=badge\">ArrayFire Gitter</a></li>\r\n 	<li>Issue reports on the <a href=\"https://github.com/arrayfire/arrayfire/issues\">ArrayFire GitHub</a></li>\r\n</ul>\r\nFinally, as you find success with ArrayFire, we invite you to contribute a post to this blog to share with the broader community. Email <a href=\"mailto:scott@arrayfire.com\">scott@arrayfire.com</a> to contribute to this blog.\r\n\r\n</div>','Performance Improvements to JIT in ArrayFire v3.4','','inherit','closed','closed','','9252-revision-v1','','','2016-09-21 17:18:30','2016-09-21 21:18:30','',9252,'http://sandbox.arrayfire.com/9252-revision-v1/',0,'revision','',0),(9367,4,'2016-09-21 17:22:49','2016-09-21 21:22:49','<span style=\"font-weight: 400;\">Today we are pleased to announce the release of ArrayFire v3.4, our open source library of parallel computing functions supporting CUDA, OpenCL, and CPU host processing. This new version contains additions and updates that improve features and performance for applications in machine learning, computer vision, signal processing, statistics, finance, and more.</span>\r\n\r\n<span style=\"font-weight: 400;\">This release focuses on 5 major areas of the library that are common to many areas of mathematical, scientific, and financial computing:  sparse matrix operations, random number generation, image processing, just-in-time (JIT) compilation, and visualizations.</span>\r\n\r\n<strong>Major Updates and New Features</strong>\r\n<ul>\r\n 	<li>Sparse Matrix and BLAS (see blog post)\r\n<ul>\r\n 	<li>Support for CSR and COO storage types</li>\r\n 	<li>Sparse-Dense Matrix Multiplication and Matrix-Vector Multiplication</li>\r\n 	<li>Conversion to and from dense matrix to CSR and COO storage types</li>\r\n</ul>\r\n</li>\r\n 	<li>Support for Random Number Generator Engines (see blog post)\r\n<ul>\r\n 	<li>Philox</li>\r\n 	<li>Threefry</li>\r\n 	<li>Mersenne Twister</li>\r\n</ul>\r\n</li>\r\n 	<li>Image Processing (see blog post)\r\n<ul>\r\n 	<li>New interpolation functions</li>\r\n 	<li>Image moments</li>\r\n</ul>\r\n</li>\r\n 	<li>Faster JIT for CUDA and OpenCL (see blog post)\r\n<ul>\r\n 	<li>Performance improvements</li>\r\n 	<li>Support for evaluating multiple outputs in a single kernel</li>\r\n</ul>\r\n</li>\r\n 	<li>Improvements to Graphics (see blog post)\r\n<ul>\r\n 	<li>Using Forge v0.9.0</li>\r\n 	<li>Vector Field plotting functionality</li>\r\n 	<li>Multiple overlays on the same window are now possible</li>\r\n 	<li>New API to set axes limits for graphs</li>\r\n 	<li>New API for plot and scatter</li>\r\n</ul>\r\n</li>\r\n</ul>\r\nThere are numerous other improvements such as new interpolation methods, image moments, scan by key and so on. For the comprehensive list of improvements and bug fixes, please refer to our <a href=\"http://arrayfire.org/docs/releasenotes.htm\">release notes</a>.\r\n\r\n<span style=\"font-weight: 400;\">While we are happy for reaching this milestone, we are not complacent. Work on the next milestone is continuing on full steam and we hope to exceed the expectations of our community.</span>','ArrayFire v3.4 Official Release','','inherit','closed','closed','','9154-revision-v1','','','2016-09-21 17:22:49','2016-09-21 21:22:49','',9154,'http://sandbox.arrayfire.com/9154-revision-v1/',0,'revision','',0),(9368,4,'2016-09-21 17:24:26','2016-09-21 21:24:26','<span style=\"font-weight: 400;\">Today we are pleased to announce the release of ArrayFire v3.4, our open source library of parallel computing functions supporting CUDA, OpenCL, and CPU host processing. This new version contains additions and updates that improve features and performance for applications in machine learning, computer vision, signal processing, statistics, finance, and more.</span>\r\n\r\n<span style=\"font-weight: 400;\">This release focuses on 5 major areas of the library that are common to many areas of mathematical, scientific, and financial computing:  sparse matrix operations, random number generation, image processing, just-in-time (JIT) compilation, and visualizations.</span>\r\n\r\n<strong>Major Updates and New Features</strong>\r\n<ul>\r\n 	<li>Sparse Matrix and BLAS (see blog post)\r\n<ul>\r\n 	<li>Support for CSR and COO storage types</li>\r\n 	<li>Sparse-Dense Matrix Multiplication and Matrix-Vector Multiplication</li>\r\n 	<li>Conversion to and from dense matrix to CSR and COO storage types</li>\r\n</ul>\r\n</li>\r\n 	<li>Support for Random Number Generator Engines (see blog post)\r\n<ul>\r\n 	<li>Philox</li>\r\n 	<li>Threefry</li>\r\n 	<li>Mersenne Twister</li>\r\n</ul>\r\n</li>\r\n 	<li>Image Processing (see blog post)\r\n<ul>\r\n 	<li>New interpolation functions</li>\r\n 	<li>Image moments</li>\r\n</ul>\r\n</li>\r\n 	<li>Faster JIT for CUDA and OpenCL (see blog post)\r\n<ul>\r\n 	<li>Performance improvements</li>\r\n 	<li>Support for evaluating multiple outputs in a single kernel</li>\r\n</ul>\r\n</li>\r\n 	<li>Improvements to Graphics (see blog post)\r\n<ul>\r\n 	<li>Using Forge v0.9.0</li>\r\n 	<li>Vector Field plotting functionality</li>\r\n 	<li>Multiple overlays on the same window are now possible</li>\r\n 	<li>New API to set axes limits for graphs</li>\r\n 	<li>New API for plot and scatter</li>\r\n</ul>\r\n</li>\r\n</ul>\r\nFor the comprehensive list of new features and updates, please refer to our <a href=\"http://arrayfire.org/docs/releasenotes.htm\">release notes</a>.\r\n\r\n<span style=\"font-weight: 400;\">While we are happy for reaching this milestone, we are not complacent. Work on the next milestone is continuing on full steam and we hope to exceed the expectations of our community.</span>\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Download[/custom_headline]\r\n\r\nArrayFire v3.4 can be downloaded from these locations:\r\n<ul>\r\n 	<li><a href=\"http://sandbox.arrayfire.com/download/\">Binaries</a></li>\r\n 	<li><a href=\"https://github.com/arrayfire\">Source code</a></li>\r\n</ul>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Community[/custom_headline]\r\n\r\nArrayFire is continually improving through the addition of new functions and features. We welcome your feedback:\r\n<ul>\r\n 	<li>General discussion forums on the <a href=\"https://groups.google.com/forum/#!forum/arrayfire-users\">ArrayFire Google Group</a></li>\r\n 	<li>Live discussion chat on the <a href=\"https://gitter.im/arrayfire/arrayfire?utm_source=badge&amp;utm_medium=badge&amp;utm_campaign=pr-badge&amp;utm_content=badge\">ArrayFire Gitter</a></li>\r\n 	<li>Issue reports on the <a href=\"https://github.com/arrayfire/arrayfire/issues\">ArrayFire GitHub</a></li>\r\n</ul>\r\nFinally, as you find success with ArrayFire, we invite you to contribute a post to this blog to share with the broader community. Email <a href=\"mailto:scott@arrayfire.com\">scott@arrayfire.com</a> to contribute to this blog.\r\n\r\n&nbsp;','ArrayFire v3.4 Official Release','','inherit','closed','closed','','9154-revision-v1','','','2016-09-21 17:24:26','2016-09-21 21:24:26','',9154,'http://sandbox.arrayfire.com/9154-revision-v1/',0,'revision','',0),(9369,4,'2016-09-21 17:24:47','2016-09-21 21:24:47','<span style=\"font-weight: 400;\">Today we are pleased to announce the release of ArrayFire v3.4, our open source library of parallel computing functions supporting CUDA, OpenCL, and CPU host processing. This new version contains additions and updates that improve features and performance for applications in machine learning, computer vision, signal processing, statistics, finance, and more.</span>\r\n\r\n<span style=\"font-weight: 400;\">This release focuses on 5 major areas of the library that are common to many areas of mathematical, scientific, and financial computing:  sparse matrix operations, random number generation, image processing, just-in-time (JIT) compilation, and visualizations.</span>\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Major Updates and Features[/custom_headline]\r\n<ul>\r\n 	<li>Sparse Matrix and BLAS (see blog post)\r\n<ul>\r\n 	<li>Support for CSR and COO storage types</li>\r\n 	<li>Sparse-Dense Matrix Multiplication and Matrix-Vector Multiplication</li>\r\n 	<li>Conversion to and from dense matrix to CSR and COO storage types</li>\r\n</ul>\r\n</li>\r\n 	<li>Support for Random Number Generator Engines (see blog post)\r\n<ul>\r\n 	<li>Philox</li>\r\n 	<li>Threefry</li>\r\n 	<li>Mersenne Twister</li>\r\n</ul>\r\n</li>\r\n 	<li>Image Processing (see blog post)\r\n<ul>\r\n 	<li>New interpolation functions</li>\r\n 	<li>Image moments</li>\r\n</ul>\r\n</li>\r\n 	<li>Faster JIT for CUDA and OpenCL (see blog post)\r\n<ul>\r\n 	<li>Performance improvements</li>\r\n 	<li>Support for evaluating multiple outputs in a single kernel</li>\r\n</ul>\r\n</li>\r\n 	<li>Improvements to Graphics (see blog post)\r\n<ul>\r\n 	<li>Using Forge v0.9.0</li>\r\n 	<li>Vector Field plotting functionality</li>\r\n 	<li>Multiple overlays on the same window are now possible</li>\r\n 	<li>New API to set axes limits for graphs</li>\r\n 	<li>New API for plot and scatter</li>\r\n</ul>\r\n</li>\r\n</ul>\r\nFor the comprehensive list of new features and updates, please refer to our <a href=\"http://arrayfire.org/docs/releasenotes.htm\">release notes</a>.\r\n\r\n<span style=\"font-weight: 400;\">While we are happy for reaching this milestone, we are not complacent. Work on the next milestone is continuing on full steam and we hope to exceed the expectations of our community.</span>\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Download[/custom_headline]\r\n\r\nArrayFire v3.4 can be downloaded from these locations:\r\n<ul>\r\n 	<li><a href=\"http://sandbox.arrayfire.com/download/\">Binaries</a></li>\r\n 	<li><a href=\"https://github.com/arrayfire\">Source code</a></li>\r\n</ul>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Community[/custom_headline]\r\n\r\nArrayFire is continually improving through the addition of new functions and features. We welcome your feedback:\r\n<ul>\r\n 	<li>General discussion forums on the <a href=\"https://groups.google.com/forum/#!forum/arrayfire-users\">ArrayFire Google Group</a></li>\r\n 	<li>Live discussion chat on the <a href=\"https://gitter.im/arrayfire/arrayfire?utm_source=badge&amp;utm_medium=badge&amp;utm_campaign=pr-badge&amp;utm_content=badge\">ArrayFire Gitter</a></li>\r\n 	<li>Issue reports on the <a href=\"https://github.com/arrayfire/arrayfire/issues\">ArrayFire GitHub</a></li>\r\n</ul>\r\nFinally, as you find success with ArrayFire, we invite you to contribute a post to this blog to share with the broader community. Email <a href=\"mailto:scott@arrayfire.com\">scott@arrayfire.com</a> to contribute to this blog.\r\n\r\n&nbsp;','ArrayFire v3.4 Official Release','','inherit','closed','closed','','9154-revision-v1','','','2016-09-21 17:24:47','2016-09-21 21:24:47','',9154,'http://sandbox.arrayfire.com/9154-revision-v1/',0,'revision','',0),(9370,4,'2016-09-21 17:25:08','2016-09-21 21:25:08','<span style=\"font-weight: 400;\">Today we are pleased to announce the release of ArrayFire v3.4, our open source library of parallel computing functions supporting CUDA, OpenCL, and CPU processing. This new version contains additions and updates that improve features and performance for applications in machine learning, computer vision, signal processing, statistics, finance, and more.</span>\r\n\r\n<span style=\"font-weight: 400;\">This release focuses on 5 major areas of the library that are common to many areas of mathematical, scientific, and financial computing:  sparse matrix operations, random number generation, image processing, just-in-time (JIT) compilation, and visualizations.</span>\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Major Updates and Features[/custom_headline]\r\n<ul>\r\n 	<li>Sparse Matrix and BLAS (see blog post)\r\n<ul>\r\n 	<li>Support for CSR and COO storage types</li>\r\n 	<li>Sparse-Dense Matrix Multiplication and Matrix-Vector Multiplication</li>\r\n 	<li>Conversion to and from dense matrix to CSR and COO storage types</li>\r\n</ul>\r\n</li>\r\n 	<li>Support for Random Number Generator Engines (see blog post)\r\n<ul>\r\n 	<li>Philox</li>\r\n 	<li>Threefry</li>\r\n 	<li>Mersenne Twister</li>\r\n</ul>\r\n</li>\r\n 	<li>Image Processing (see blog post)\r\n<ul>\r\n 	<li>New interpolation functions</li>\r\n 	<li>Image moments</li>\r\n</ul>\r\n</li>\r\n 	<li>Faster JIT for CUDA and OpenCL (see blog post)\r\n<ul>\r\n 	<li>Performance improvements</li>\r\n 	<li>Support for evaluating multiple outputs in a single kernel</li>\r\n</ul>\r\n</li>\r\n 	<li>Improvements to Graphics (see blog post)\r\n<ul>\r\n 	<li>Using Forge v0.9.0</li>\r\n 	<li>Vector Field plotting functionality</li>\r\n 	<li>Multiple overlays on the same window are now possible</li>\r\n 	<li>New API to set axes limits for graphs</li>\r\n 	<li>New API for plot and scatter</li>\r\n</ul>\r\n</li>\r\n</ul>\r\nFor the comprehensive list of new features and updates, please refer to our <a href=\"http://arrayfire.org/docs/releasenotes.htm\">release notes</a>.\r\n\r\n<span style=\"font-weight: 400;\">While we are happy for reaching this milestone, we are not complacent. Work on the next milestone is continuing on full steam and we hope to exceed the expectations of our community.</span>\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Download[/custom_headline]\r\n\r\nArrayFire v3.4 can be downloaded from these locations:\r\n<ul>\r\n 	<li><a href=\"http://sandbox.arrayfire.com/download/\">Binaries</a></li>\r\n 	<li><a href=\"https://github.com/arrayfire\">Source code</a></li>\r\n</ul>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Community[/custom_headline]\r\n\r\nArrayFire is continually improving through the addition of new functions and features. We welcome your feedback:\r\n<ul>\r\n 	<li>General discussion forums on the <a href=\"https://groups.google.com/forum/#!forum/arrayfire-users\">ArrayFire Google Group</a></li>\r\n 	<li>Live discussion chat on the <a href=\"https://gitter.im/arrayfire/arrayfire?utm_source=badge&amp;utm_medium=badge&amp;utm_campaign=pr-badge&amp;utm_content=badge\">ArrayFire Gitter</a></li>\r\n 	<li>Issue reports on the <a href=\"https://github.com/arrayfire/arrayfire/issues\">ArrayFire GitHub</a></li>\r\n</ul>\r\nFinally, as you find success with ArrayFire, we invite you to contribute a post to this blog to share with the broader community. Email <a href=\"mailto:scott@arrayfire.com\">scott@arrayfire.com</a> to contribute to this blog.\r\n\r\n&nbsp;','ArrayFire v3.4 Official Release','','inherit','closed','closed','','9154-revision-v1','','','2016-09-21 17:25:08','2016-09-21 21:25:08','',9154,'http://sandbox.arrayfire.com/9154-revision-v1/',0,'revision','',0),(9371,4,'2016-09-21 17:25:20','2016-09-21 21:25:20','<span style=\"font-weight: 400;\">Today we are pleased to announce the release of ArrayFire v3.4, our open source library of parallel computing functions supporting CUDA, OpenCL, and CPU devices. This new version contains additions and updates that improve features and performance for applications in machine learning, computer vision, signal processing, statistics, finance, and more.</span>\r\n\r\n<span style=\"font-weight: 400;\">This release focuses on 5 major areas of the library that are common to many areas of mathematical, scientific, and financial computing:  sparse matrix operations, random number generation, image processing, just-in-time (JIT) compilation, and visualizations.</span>\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Major Updates and Features[/custom_headline]\r\n<ul>\r\n 	<li>Sparse Matrix and BLAS (see blog post)\r\n<ul>\r\n 	<li>Support for CSR and COO storage types</li>\r\n 	<li>Sparse-Dense Matrix Multiplication and Matrix-Vector Multiplication</li>\r\n 	<li>Conversion to and from dense matrix to CSR and COO storage types</li>\r\n</ul>\r\n</li>\r\n 	<li>Support for Random Number Generator Engines (see blog post)\r\n<ul>\r\n 	<li>Philox</li>\r\n 	<li>Threefry</li>\r\n 	<li>Mersenne Twister</li>\r\n</ul>\r\n</li>\r\n 	<li>Image Processing (see blog post)\r\n<ul>\r\n 	<li>New interpolation functions</li>\r\n 	<li>Image moments</li>\r\n</ul>\r\n</li>\r\n 	<li>Faster JIT for CUDA and OpenCL (see blog post)\r\n<ul>\r\n 	<li>Performance improvements</li>\r\n 	<li>Support for evaluating multiple outputs in a single kernel</li>\r\n</ul>\r\n</li>\r\n 	<li>Improvements to Graphics (see blog post)\r\n<ul>\r\n 	<li>Using Forge v0.9.0</li>\r\n 	<li>Vector Field plotting functionality</li>\r\n 	<li>Multiple overlays on the same window are now possible</li>\r\n 	<li>New API to set axes limits for graphs</li>\r\n 	<li>New API for plot and scatter</li>\r\n</ul>\r\n</li>\r\n</ul>\r\nFor the comprehensive list of new features and updates, please refer to our <a href=\"http://arrayfire.org/docs/releasenotes.htm\">release notes</a>.\r\n\r\n<span style=\"font-weight: 400;\">While we are happy for reaching this milestone, we are not complacent. Work on the next milestone is continuing on full steam and we hope to exceed the expectations of our community.</span>\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Download[/custom_headline]\r\n\r\nArrayFire v3.4 can be downloaded from these locations:\r\n<ul>\r\n 	<li><a href=\"http://sandbox.arrayfire.com/download/\">Binaries</a></li>\r\n 	<li><a href=\"https://github.com/arrayfire\">Source code</a></li>\r\n</ul>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Community[/custom_headline]\r\n\r\nArrayFire is continually improving through the addition of new functions and features. We welcome your feedback:\r\n<ul>\r\n 	<li>General discussion forums on the <a href=\"https://groups.google.com/forum/#!forum/arrayfire-users\">ArrayFire Google Group</a></li>\r\n 	<li>Live discussion chat on the <a href=\"https://gitter.im/arrayfire/arrayfire?utm_source=badge&amp;utm_medium=badge&amp;utm_campaign=pr-badge&amp;utm_content=badge\">ArrayFire Gitter</a></li>\r\n 	<li>Issue reports on the <a href=\"https://github.com/arrayfire/arrayfire/issues\">ArrayFire GitHub</a></li>\r\n</ul>\r\nFinally, as you find success with ArrayFire, we invite you to contribute a post to this blog to share with the broader community. Email <a href=\"mailto:scott@arrayfire.com\">scott@arrayfire.com</a> to contribute to this blog.\r\n\r\n&nbsp;','ArrayFire v3.4 Official Release','','inherit','closed','closed','','9154-revision-v1','','','2016-09-21 17:25:20','2016-09-21 21:25:20','',9154,'http://sandbox.arrayfire.com/9154-revision-v1/',0,'revision','',0),(9372,4,'2016-09-21 17:26:06','2016-09-21 21:26:06','<span style=\"font-weight: 400;\">Today we are pleased to announce the release of ArrayFire v3.4, our open source library of parallel computing functions supporting CUDA, OpenCL, and CPU devices. This new version of ArrayFire improves features and performance for applications in machine learning, computer vision, signal processing, statistics, finance, and more.</span>\r\n\r\n<span style=\"font-weight: 400;\">This release focuses on 5 major areas of the library that are common to many areas of mathematical, scientific, and financial computing:  sparse matrix operations, random number generation, image processing, just-in-time (JIT) compilation, and visualizations.</span>\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Major Updates and Features[/custom_headline]\r\n<ul>\r\n 	<li>Sparse Matrix and BLAS (see blog post)\r\n<ul>\r\n 	<li>Support for CSR and COO storage types</li>\r\n 	<li>Sparse-Dense Matrix Multiplication and Matrix-Vector Multiplication</li>\r\n 	<li>Conversion to and from dense matrix to CSR and COO storage types</li>\r\n</ul>\r\n</li>\r\n 	<li>Support for Random Number Generator Engines (see blog post)\r\n<ul>\r\n 	<li>Philox</li>\r\n 	<li>Threefry</li>\r\n 	<li>Mersenne Twister</li>\r\n</ul>\r\n</li>\r\n 	<li>Image Processing (see blog post)\r\n<ul>\r\n 	<li>New interpolation functions</li>\r\n 	<li>Image moments</li>\r\n</ul>\r\n</li>\r\n 	<li>Faster JIT for CUDA and OpenCL (see blog post)\r\n<ul>\r\n 	<li>Performance improvements</li>\r\n 	<li>Support for evaluating multiple outputs in a single kernel</li>\r\n</ul>\r\n</li>\r\n 	<li>Improvements to Graphics (see blog post)\r\n<ul>\r\n 	<li>Using Forge v0.9.0</li>\r\n 	<li>Vector Field plotting functionality</li>\r\n 	<li>Multiple overlays on the same window are now possible</li>\r\n 	<li>New API to set axes limits for graphs</li>\r\n 	<li>New API for plot and scatter</li>\r\n</ul>\r\n</li>\r\n</ul>\r\nFor the comprehensive list of new features and updates, please refer to our <a href=\"http://arrayfire.org/docs/releasenotes.htm\">release notes</a>.\r\n\r\n<span style=\"font-weight: 400;\">While we are happy for reaching this milestone, we are not complacent. Work on the next milestone is continuing on full steam and we hope to exceed the expectations of our community.</span>\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Download[/custom_headline]\r\n\r\nArrayFire v3.4 can be downloaded from these locations:\r\n<ul>\r\n 	<li><a href=\"http://sandbox.arrayfire.com/download/\">Binaries</a></li>\r\n 	<li><a href=\"https://github.com/arrayfire\">Source code</a></li>\r\n</ul>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Community[/custom_headline]\r\n\r\nArrayFire is continually improving through the addition of new functions and features. We welcome your feedback:\r\n<ul>\r\n 	<li>General discussion forums on the <a href=\"https://groups.google.com/forum/#!forum/arrayfire-users\">ArrayFire Google Group</a></li>\r\n 	<li>Live discussion chat on the <a href=\"https://gitter.im/arrayfire/arrayfire?utm_source=badge&amp;utm_medium=badge&amp;utm_campaign=pr-badge&amp;utm_content=badge\">ArrayFire Gitter</a></li>\r\n 	<li>Issue reports on the <a href=\"https://github.com/arrayfire/arrayfire/issues\">ArrayFire GitHub</a></li>\r\n</ul>\r\nFinally, as you find success with ArrayFire, we invite you to contribute a post to this blog to share with the broader community. Email <a href=\"mailto:scott@arrayfire.com\">scott@arrayfire.com</a> to contribute to this blog.\r\n\r\n&nbsp;','ArrayFire v3.4 Official Release','','inherit','closed','closed','','9154-revision-v1','','','2016-09-21 17:26:06','2016-09-21 21:26:06','',9154,'http://sandbox.arrayfire.com/9154-revision-v1/',0,'revision','',0),(9373,4,'2016-09-21 17:27:52','2016-09-21 21:27:52','<span style=\"font-weight: 400;\">Today we are pleased to announce the release of ArrayFire v3.4, our open source library of parallel computing functions supporting CUDA, OpenCL, and CPU devices. This new version of ArrayFire improves features and performance for applications in machine learning, computer vision, signal processing, statistics, finance, and more.</span>\r\n\r\n<span style=\"font-weight: 400;\">This release focuses on 5 major components of the library that are common to many areas of mathematical, scientific, and financial computing:  sparse matrix operations, random number generation, image processing, just-in-time (JIT) compilation, and visualizations.</span>\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Major Updates and Features[/custom_headline]\r\n<ul>\r\n 	<li>Sparse Matrix and BLAS (see blog post)\r\n<ul>\r\n 	<li>Support for CSR and COO storage types</li>\r\n 	<li>Sparse-Dense Matrix Multiplication and Matrix-Vector Multiplication</li>\r\n 	<li>Conversion to and from dense matrix to CSR and COO storage types</li>\r\n</ul>\r\n</li>\r\n 	<li>Support for Random Number Generator Engines (see blog post)\r\n<ul>\r\n 	<li>Philox</li>\r\n 	<li>Threefry</li>\r\n 	<li>Mersenne Twister</li>\r\n</ul>\r\n</li>\r\n 	<li>Image Processing (see blog post)\r\n<ul>\r\n 	<li>New interpolation functions</li>\r\n 	<li>Image moments</li>\r\n</ul>\r\n</li>\r\n 	<li>Faster JIT for CUDA and OpenCL (see blog post)\r\n<ul>\r\n 	<li>Performance improvements</li>\r\n 	<li>Support for evaluating multiple outputs in a single kernel</li>\r\n</ul>\r\n</li>\r\n 	<li>Improvements to Graphics (see blog post)\r\n<ul>\r\n 	<li>Using Forge v0.9.0</li>\r\n 	<li>Vector Field plotting functionality</li>\r\n 	<li>Multiple overlays on the same window are now possible</li>\r\n 	<li>New API to set axes limits for graphs</li>\r\n 	<li>New API for plot and scatter</li>\r\n</ul>\r\n</li>\r\n</ul>\r\nFor the comprehensive list of new features and updates, please refer to our <a href=\"http://arrayfire.org/docs/releasenotes.htm\">release notes</a>.\r\n\r\n<span style=\"font-weight: 400;\">While we are happy for reaching this milestone, we are not complacent. Work on the next milestone is continuing on full steam and we hope to exceed the expectations of our community.</span>\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Download[/custom_headline]\r\n\r\nArrayFire v3.4 can be downloaded from these locations:\r\n<ul>\r\n 	<li><a href=\"http://sandbox.arrayfire.com/download/\">Binaries</a></li>\r\n 	<li><a href=\"https://github.com/arrayfire\">Source code</a></li>\r\n</ul>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Community[/custom_headline]\r\n\r\nArrayFire is continually improving through the addition of new functions and features. We welcome your feedback:\r\n<ul>\r\n 	<li>General discussion forums on the <a href=\"https://groups.google.com/forum/#!forum/arrayfire-users\">ArrayFire Google Group</a></li>\r\n 	<li>Live discussion chat on the <a href=\"https://gitter.im/arrayfire/arrayfire?utm_source=badge&amp;utm_medium=badge&amp;utm_campaign=pr-badge&amp;utm_content=badge\">ArrayFire Gitter</a></li>\r\n 	<li>Issue reports on the <a href=\"https://github.com/arrayfire/arrayfire/issues\">ArrayFire GitHub</a></li>\r\n</ul>\r\nFinally, as you find success with ArrayFire, we invite you to contribute a post to this blog to share with the broader community. Email <a href=\"mailto:scott@arrayfire.com\">scott@arrayfire.com</a> to contribute to this blog.\r\n\r\n&nbsp;','ArrayFire v3.4 Official Release','','inherit','closed','closed','','9154-revision-v1','','','2016-09-21 17:27:52','2016-09-21 21:27:52','',9154,'http://sandbox.arrayfire.com/9154-revision-v1/',0,'revision','',0),(9374,4,'2016-09-21 17:28:42','2016-09-21 21:28:42','<span style=\"font-weight: 400;\">Today we are pleased to announce the release of ArrayFire v3.4, our open source library of parallel computing functions supporting CUDA, OpenCL, and CPU devices. This new version of ArrayFire improves features and performance for applications in machine learning, computer vision, signal processing, statistics, finance, and more.</span>\r\n\r\n<span style=\"font-weight: 400;\">This release focuses on 5 major components of the library that are common to many areas of mathematical, scientific, and financial computing:  sparse matrix operations, random number generation, image processing, just-in-time (JIT) compilation, and visualizations.</span>\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Major Updates and Features[/custom_headline]\r\n<ul>\r\n 	<li>Sparse Matrix and BLAS (see blog post)\r\n<ul>\r\n 	<li>Support for CSR and COO storage types</li>\r\n 	<li>Sparse-Dense Matrix Multiplication and Matrix-Vector Multiplication</li>\r\n 	<li>Conversion to and from dense matrix to CSR and COO storage types</li>\r\n</ul>\r\n</li>\r\n 	<li>Support for Random Number Generator Engines (see blog post)\r\n<ul>\r\n 	<li>Philox</li>\r\n 	<li>Threefry</li>\r\n 	<li>Mersenne Twister</li>\r\n</ul>\r\n</li>\r\n 	<li>Image Processing (see blog post)\r\n<ul>\r\n 	<li>New interpolation functions</li>\r\n 	<li>Image moments</li>\r\n</ul>\r\n</li>\r\n 	<li>Faster JIT for CUDA and OpenCL (see blog post)\r\n<ul>\r\n 	<li>Performance improvements</li>\r\n 	<li>Support for evaluating multiple outputs in a single kernel</li>\r\n</ul>\r\n</li>\r\n 	<li>Improvements to Graphics (see blog post)\r\n<ul>\r\n 	<li>Using Forge v0.9.0</li>\r\n 	<li>Vector Field plotting functionality</li>\r\n 	<li>Multiple overlays on the same window are now possible</li>\r\n 	<li>New API to set axes limits for graphs</li>\r\n 	<li>New API for plot and scatter</li>\r\n</ul>\r\n</li>\r\n</ul>\r\nFor the comprehensive list of new features and updates, please refer to our <a href=\"http://arrayfire.org/docs/releasenotes.htm\">release notes</a>.\r\n\r\n<span style=\"font-weight: 400;\">While we are happy for reaching this milestone, we are not complacent. Work on the next milestone has already commenced, and we hope to exceed the expectations of our community.</span>\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Download[/custom_headline]\r\n\r\nArrayFire v3.4 can be downloaded from these locations:\r\n<ul>\r\n 	<li><a href=\"http://sandbox.arrayfire.com/download/\">Binaries</a></li>\r\n 	<li><a href=\"https://github.com/arrayfire\">Source code</a></li>\r\n</ul>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Community[/custom_headline]\r\n\r\nArrayFire is continually improving through the addition of new functions and features. We welcome your feedback:\r\n<ul>\r\n 	<li>General discussion forums on the <a href=\"https://groups.google.com/forum/#!forum/arrayfire-users\">ArrayFire Google Group</a></li>\r\n 	<li>Live discussion chat on the <a href=\"https://gitter.im/arrayfire/arrayfire?utm_source=badge&amp;utm_medium=badge&amp;utm_campaign=pr-badge&amp;utm_content=badge\">ArrayFire Gitter</a></li>\r\n 	<li>Issue reports on the <a href=\"https://github.com/arrayfire/arrayfire/issues\">ArrayFire GitHub</a></li>\r\n</ul>\r\nFinally, as you find success with ArrayFire, we invite you to contribute a post to this blog to share with the broader community. Email <a href=\"mailto:scott@arrayfire.com\">scott@arrayfire.com</a> to contribute to this blog.\r\n\r\n&nbsp;','ArrayFire v3.4 Official Release','','inherit','closed','closed','','9154-revision-v1','','','2016-09-21 17:28:42','2016-09-21 21:28:42','',9154,'http://sandbox.arrayfire.com/9154-revision-v1/',0,'revision','',0),(9375,4,'2016-09-21 17:30:37','2016-09-21 21:30:37','With the release of <strong>ArrayFire v3.4</strong>, we have added a number of features that will help our vast range of users. Our users have been asking us for a way to reduce the memory footprint of their data on the GPU. And now we have answered our users\' requests.\r\nIntroducing Sparse Matrices in ArrayFire!\r\n<h4>Sparse Data Structure</h4>\r\nA sparse data structure is one where all the non-zero elements are not stored. Sparse matrices are useful when the number of zero-values elements are &gt;&gt; number of non-zero elements (sparsity of the matrix is high).\r\nThe sparse data structure in general is stored as 3 arrays.\r\n<ul>\r\n 	<li>A data or values array containing all the non-zero elements</li>\r\n 	<li>A vector for row indices (based on storage format).</li>\r\n 	<li>A vector for column indices (based on storage format).</li>\r\n</ul>\r\nThere are many ways to store sparse matrices, the most prominent of which are\r\n<ul>\r\n 	<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_row_.28CSR.2C_CRS_or_Yale_format.29\">Compressed Sparse Row</a></li>\r\n 	<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_column_.28CSC_or_CCS.29\">Compressed Storage Column</a></li>\r\n 	<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Coordinate_list_.28COO.29\">Coordinate Format</a>.</li>\r\n</ul>\r\nArrayFire supports CSR and COO format, with CSC support coming in future releases.\r\n\r\nA more detailed description of Sparse Matrix can be found on <a href=\"https://en.wikipedia.org/wiki/Sparse_matrix\">Wikipedia</a>.\r\n<h4>Sparse in ArrayFire</h4>\r\nIn ArrayFire, Sparse arrays use the same user facing object - af::array. This makes working with sparse arrays all the more seamless.\r\nWe have added a function <a href=\"http://arrayfire.org/docs/group__method__mat.htm#ga5945f95549bae25f75694a69d98ca584\">af::array::issparse()</a> that allows users to check if an array is sparse or not.\r\nAlthough externally, there is no difference between a sparse and dense array, internally, the sparse array is stored as 3 independent arrays along with other meta data.\r\nThese arrays, as well as other properties can be fetched using the <a href=\"http://arrayfire.org/docs/group__sparse__func.htm\">sparse functions</a>.\r\n<h4>Sparse Functionality</h4>\r\nAs of v3.4, the following features are supported for sparse arrays:\r\n<ul>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm\">Creation for sparse arrays.</a>\r\n<ul>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae4621e58bfaf6d80668309824fde86c0\">From 3 (data, row, column) sparse host or device memory vectors.</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae9a843c7e93b05e6b090ca32b271ebe9\">From 3 (data, row, column) sparse af::arrays.</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">From dense af::array.</a></li>\r\n 	<li>To create a sparse array from host or device dense array, first create a dense af::array, and then pass it to create sparse af::array from dense af::array.</li>\r\n</ul>\r\n</li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__sparse__func__convert__to.htm\">Storage format conversions.</a>\r\n<ul>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">Dense to CSR/COO.</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__sparse__func__dense.htm\">CSR/COO to dense.</a></li>\r\n</ul>\r\n</li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">Sparse-Dense Matrix Multiplication (SPMM/SPMV)</a>\r\n<ul>\r\n 	<li>Sparse Matrix - Dense Matrix</li>\r\n 	<li>Sparse Matrix - Dense Vector</li>\r\n 	<li>Options for Sparse Matrix to be transposed.</li>\r\n</ul>\r\n</li>\r\n</ul>\r\nThe Sparse-Dense Matrix Multiplcation is a implemented as a part of <a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">af::matmul()</a>. This is part of making the usage of sparse arrays seamless with regular arrays.\r\n\r\nAs with all ArrayFire features, Sparse support is available across all backends and all architectures.\r\n<h4>Example</h4>\r\nThe example we shall use is a conjugate gradient solver. The inputs will be the same except that for the sparse conjugate gradient we will convert the A matrix to sparse.\r\n<pre>#include \r\n#include \r\n\r\nusing namespace af;\r\n\r\nstatic size_t dimension = 16 * 1024;\r\nstatic const int maxIter = 10;\r\nstatic const int sparsityFactor = 7;\r\n\r\nstatic array A;\r\nstatic array spA;  // Sparse A\r\nstatic array x0;\r\nstatic array b;\r\n\r\nvoid setupInputs()\r\n{\r\n    // Generate a random input: A\r\n    array T = randu(dimension, dimension, f32);\r\n    // Create 0s in input.\r\n    // Anything that is no divisible by sparsityFactor will become 0.\r\n    A = floor(T * 1000);\r\n    A = A * ((A % sparsityFactor) == 0) / 1000;\r\n    // Make it positive definite\r\n    A = transpose(A) + A + A.dims(0)*identity(A.dims(0), A.dims(0), f32);\r\n\r\n    // Make A sparse as spA\r\n    spA = sparse(A);\r\n\r\n    // Generate x0: Random guess\r\n    x0 = randu(A.dims(0), f32);\r\n\r\n    //Generate b\r\n    b = matmul(A, x0);\r\n\r\n    std::cout &lt;&lt; \"Sparsity of A = \"\r\n              &lt;&lt;  100.f * (float)sparseGetNNZ(spA) / (float)spA.elements()\r\n              &lt;&lt; std::endl;\r\n    std::cout &lt;&lt; \"Memory Usage of A = \" &lt;&lt;  A.bytes() &lt;&lt; \" bytes\" &lt;&lt; std::endl;\r\n    std::cout &lt;&lt; \"Memory Usage of spA = \"\r\n              &lt;&lt; sparseGetValues(spA).bytes()\r\n               + sparseGetRowIdx(spA).bytes()\r\n               + sparseGetColIdx(spA).bytes()\r\n              &lt;&lt; \" bytes\" &lt;&lt; std::endl;\r\n}\r\n\r\nvoid conjugateGradient(const af::array in)\r\n{\r\n    array x = constant(0, b.dims(), f32);\r\n    array r = b - matmul(in, x);\r\n    array p = r;\r\n\r\n    for (int i = 0; i &lt; maxIter; ++i) {\r\n        array Ap = matmul(in, p);\r\n        array alpha_num = dot(r, r);\r\n        array alpha_den = dot(p, Ap);\r\n        array alpha = alpha_num/alpha_den;\r\n        r = r - tile(alpha, Ap.dims())*Ap;\r\n        x = x + tile(alpha, Ap.dims())*p;\r\n        array beta_num = dot(r, r);\r\n        array beta = beta_num/alpha_num;\r\n        p = r + tile(beta, p.dims())*p;\r\n    }\r\n    array res = x0 - x;\r\n\r\n    std::cout&lt;&lt;\"Final difference in solutions:\\n\";\r\n    af_print(dot(res, res));\r\n}\r\n\r\nint main(int argc, char *argv[])\r\n{\r\n    af::info();\r\n    setupInputs();\r\n\r\n    std::cout &lt;&lt; \"Verifying Dense Conjugate Gradient:\" &lt;&lt; std::endl;\r\n    conjugateGradient(A);\r\n\r\n    std::cout &lt;&lt; \"Verifying Sparse Conjugate Gradient:\" &lt;&lt; std::endl;\r\n    conjugateGradient(spA);\r\n\r\n    return 0;\r\n}\r\n</pre>\r\n<h4>Performance and Memory Footprint</h4>\r\nUsing the CUDA backend on a Tesla K40c, the conjugate gradient function took a time of 64.94ms for dense matrix and 39.31ms for sparse Matrix. That is almost a 2x speedup! Add in the fact that not all computation in the conjugate gradient function is sparse, the speedup for sparse vs dense matrix multiplication is extremely good.\r\n\r\nWith a <code>sparsityFactor</code> of 7, the sparsity of A is about 26.38%. This means a tremendous amount of space and performance can be achieved by using sparse arrays. For a floating point matrix of size 16M x 16M, a dense matrix uses 1 GB of memory while the sparse array, stored in CSR format, only used 527 MB - halving the memory consumption!\r\n<table>\r\n<tbody>\r\n<tr>\r\n<th>Matrix Elements</th>\r\n<th>Memory Size (Dense)</th>\r\n<th>Memory Size (Sparse)</th>\r\n<th>CG Time (Dense)</th>\r\n<th>CG Time (Sparse)</th>\r\n</tr>\r\n<tr>\r\n<th>16M x 16M</th>\r\n<th>1024 MB</th>\r\n<th>527 MB</th>\r\n<th>64.94ms</th>\r\n<th>39.31ms</th>\r\n</tr>\r\n<tr>\r\n<th>4M x 4M</th>\r\n<th>64 MB</th>\r\n<th>33 MB</th>\r\n<th>4.752ms</th>\r\n<th>3.935ms</th>\r\n</tr>\r\n</tbody>\r\n</table>\r\n<h4>Conclusion</h4>\r\nNothing excites us more than delivering on user requests for features and seeing them solve complex problems in diverse fields. Sparse arrays will now let users store more data on the GPU while allowing faster computation, thus allowing to solve for bigger, more complex problem.\r\n\r\nThe <a href=\"https://github.com/arrayfire/arrayfire\">ArrayFire GitHub</a> page is the place for all things ArrayFire. We look forward to your comments, feature requests and anything else that you may want to talk about.','Sparse Matrices in ArrayFire v3.4','','inherit','closed','closed','','9190-revision-v1','','','2016-09-21 17:30:37','2016-09-21 21:30:37','',9190,'http://sandbox.arrayfire.com/9190-revision-v1/',0,'revision','',0),(9376,4,'2016-09-21 22:13:21','2016-09-22 02:13:21','In ArrayFire v3.4, we have added support for sparse matrices, which greatly reduce the memory footprint on GPUs and accelerated devices for many applications.\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Sparse Data Structure[/custom_headline]\r\n\r\nA sparse data structure is one where all the non-zero elements are not stored. Sparse matrices are useful when the number of zero-values elements are much greater than the number of non-zero elements (i.e. the sparsity of the matrix is high). A sparse data structure is generally stored as 3 arrays:\r\n<ul>\r\n 	<li>A data or values array containing all the non-zero elements</li>\r\n 	<li>A vector for row indices (based on storage format)</li>\r\n 	<li>A vector for column indices (based on storage format)</li>\r\n</ul>\r\nThere are many ways to store sparse matrices, the most prominent of which are:\r\n<ul>\r\n 	<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_row_.28CSR.2C_CRS_or_Yale_format.29\">Compressed Sparse Row</a> (CSR)</li>\r\n 	<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_column_.28CSC_or_CCS.29\">Compressed Storage Column</a> (CSC)</li>\r\n 	<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Coordinate_list_.28COO.29\">Coordinate Format</a> (COO)</li>\r\n</ul>\r\nArrayFire supports CSR and COO format, with support for CSC coming in future releases.\r\n\r\nA more detailed description of sparse matrices can be found on <a href=\"https://en.wikipedia.org/wiki/Sparse_matrix\">Wikipedia</a>.\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Sparse Arrays in ArrayFire[/custom_headline]\r\n\r\nIn ArrayFire, sparse arrays use the same user-facing object, af::array. This makes working with sparse arrays very seamless. We have added a function <a href=\"http://arrayfire.org/docs/group__method__mat.htm#ga5945f95549bae25f75694a69d98ca584\">af::array::issparse()</a> that allows users to check if an array is sparse or not. Although externally, there is no difference between a sparse and dense array, internally, the sparse array is stored as 3 independent arrays along with other metadata. These arrays, as well as other properties, can be fetched using the <a href=\"http://arrayfire.org/docs/group__sparse__func.htm\">sparse functions</a>.\r\n\r\nIn v3.4, the following features are supported for sparse arrays:\r\n<ul>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm\">Creation for sparse arrays.</a>\r\n<ul>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae4621e58bfaf6d80668309824fde86c0\">From 3 (data, row, column) sparse host or device memory vectors.</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae9a843c7e93b05e6b090ca32b271ebe9\">From 3 (data, row, column) sparse af::arrays.</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">From dense af::array.</a></li>\r\n 	<li>To create a sparse array from a host or device dense array, first create a dense af::array, and then pass it to create sparse af::array.</li>\r\n</ul>\r\n</li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__sparse__func__convert__to.htm\">Storage format conversions.</a>\r\n<ul>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">Dense to CSR/COO.</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__sparse__func__dense.htm\">CSR/COO to dense.</a></li>\r\n</ul>\r\n</li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">Sparse-Dense Matrix Multiplication (SPMM/SPMV)</a>\r\n<ul>\r\n 	<li>Sparse Matrix - Dense Matrix</li>\r\n 	<li>Sparse Matrix - Dense Vector</li>\r\n 	<li>Options for Sparse Matrix to be transposed</li>\r\n</ul>\r\n</li>\r\n</ul>\r\nThe Sparse-Dense Matrix Multiplication is implemented within the same <a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">af::matmul()</a> function used for dense arrays.\r\n\r\nAs with all ArrayFire features, sparse support is available across all backends (CUDA, OpenCL, and CPU) and supported devices and architectures.\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Example[/custom_headline]\r\n\r\nIn this section, we showcase the difference in running a conjugate gradient solver with a dense array and a sparse array. The inputs are the same for each example. In the sparse case, the \"A\" matrix is converted to sparse.\r\n<pre class=\"\">#include \r\n#include \r\n\r\nusing namespace af;\r\n\r\nstatic size_t dimension = 16 * 1024;\r\nstatic const int maxIter = 10;\r\nstatic const int sparsityFactor = 7;\r\n\r\nstatic array A;\r\nstatic array spA;  // Sparse A\r\nstatic array x0;\r\nstatic array b;\r\n\r\nvoid setupInputs()\r\n{\r\n    // Generate a random input: A\r\n    array T = randu(dimension, dimension, f32);\r\n    // Create 0s in input.\r\n    // Anything that is no divisible by sparsityFactor will become 0.\r\n    A = floor(T * 1000);\r\n    A = A * ((A % sparsityFactor) == 0) / 1000;\r\n    // Make it positive definite\r\n    A = transpose(A) + A + A.dims(0)*identity(A.dims(0), A.dims(0), f32);\r\n\r\n    // Make A sparse as spA\r\n    spA = sparse(A);\r\n\r\n    // Generate x0: Random guess\r\n    x0 = randu(A.dims(0), f32);\r\n\r\n    //Generate b\r\n    b = matmul(A, x0);\r\n\r\n    std::cout &lt;&lt; \"Sparsity of A = \"\r\n              &lt;&lt;  100.f * (float)sparseGetNNZ(spA) / (float)spA.elements()\r\n              &lt;&lt; std::endl;\r\n    std::cout &lt;&lt; \"Memory Usage of A = \" &lt;&lt;  A.bytes() &lt;&lt; \" bytes\" &lt;&lt; std::endl;\r\n    std::cout &lt;&lt; \"Memory Usage of spA = \"\r\n              &lt;&lt; sparseGetValues(spA).bytes()\r\n               + sparseGetRowIdx(spA).bytes()\r\n               + sparseGetColIdx(spA).bytes()\r\n              &lt;&lt; \" bytes\" &lt;&lt; std::endl;\r\n}\r\n\r\nvoid conjugateGradient(const af::array in)\r\n{\r\n    array x = constant(0, b.dims(), f32);\r\n    array r = b - matmul(in, x);\r\n    array p = r;\r\n\r\n    for (int i = 0; i &lt; maxIter; ++i) {\r\n        array Ap = matmul(in, p);\r\n        array alpha_num = dot(r, r);\r\n        array alpha_den = dot(p, Ap);\r\n        array alpha = alpha_num/alpha_den;\r\n        r = r - tile(alpha, Ap.dims())*Ap;\r\n        x = x + tile(alpha, Ap.dims())*p;\r\n        array beta_num = dot(r, r);\r\n        array beta = beta_num/alpha_num;\r\n        p = r + tile(beta, p.dims())*p;\r\n    }\r\n    array res = x0 - x;\r\n\r\n    std::cout&lt;&lt;\"Final difference in solutions:\\n\";\r\n    af_print(dot(res, res));\r\n}\r\n\r\nint main(int argc, char *argv[])\r\n{\r\n    af::info();\r\n    setupInputs();\r\n\r\n    std::cout &lt;&lt; \"Verifying Dense Conjugate Gradient:\" &lt;&lt; std::endl;\r\n    conjugateGradient(A);\r\n\r\n    std::cout &lt;&lt; \"Verifying Sparse Conjugate Gradient:\" &lt;&lt; std::endl;\r\n    conjugateGradient(spA);\r\n\r\n    return 0;\r\n}\r\n</pre>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Performance and Memory Footprint[/custom_headline]\r\n\r\nUsing the CUDA backend on a Tesla K40c, the conjugate gradient function took 64.94ms for the dense matrix and 39.31ms for the sparse Matrix. That is almost a 2x speedup!\r\n\r\nWith a <code>sparsityFactor</code> of 7, the sparsity of A is about 26.38%. This means a tremendous amount of memory is conserved. For a floating point matrix of size 16M x 16M, a dense matrix uses 1 GB of memory while the sparse array, stored in CSR format, only uses 527 MB. That is a 2x savings on memory usage!\r\n<table>\r\n<tbody>\r\n<tr>\r\n<th>Matrix Elements</th>\r\n<th>Memory Size (Dense)</th>\r\n<th>Memory Size (Sparse)</th>\r\n<th>CG Time (Dense)</th>\r\n<th>CG Time (Sparse)</th>\r\n</tr>\r\n<tr>\r\n<th>16M x 16M</th>\r\n<th>1024 MB</th>\r\n<th>527 MB</th>\r\n<th>64.94ms</th>\r\n<th>39.31ms</th>\r\n</tr>\r\n<tr>\r\n<th>4M x 4M</th>\r\n<th>64 MB</th>\r\n<th>33 MB</th>\r\n<th>4.752ms</th>\r\n<th>3.935ms</th>\r\n</tr>\r\n</tbody>\r\n</table>\r\nSparse arrays will now let users store more data on the GPU while allowing faster computation, thus allowing to solve for bigger, more complex problems.\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Download[/custom_headline]\r\n\r\nArrayFire v3.4 can be downloaded from these locations:\r\n<ul>\r\n 	<li><a href=\"http://sandbox.arrayfire.com/download/\">Binaries</a></li>\r\n 	<li><a href=\"https://github.com/arrayfire\">Source code</a></li>\r\n</ul>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Community[/custom_headline]\r\n\r\nArrayFire is continually improving through the addition of new sparse matrix enhancements. We welcome your feedback:\r\n<ul>\r\n 	<li>General discussion forums on the <a href=\"https://groups.google.com/forum/#!forum/arrayfire-users\">ArrayFire Google Group</a></li>\r\n 	<li>Live discussion chat on the <a href=\"https://gitter.im/arrayfire/arrayfire?utm_source=badge&amp;utm_medium=badge&amp;utm_campaign=pr-badge&amp;utm_content=badge\">ArrayFire Gitter</a></li>\r\n 	<li>Issue reports on the <a href=\"https://github.com/arrayfire/arrayfire/issues\">ArrayFire GitHub</a></li>\r\n</ul>\r\nFinally, as you find success with ArrayFire, we invite you to contribute a post to this blog to share with the broader community. Email <a href=\"mailto:scott@arrayfire.com\">scott@arrayfire.com</a> to contribute to this blog.','Sparse Matrices in ArrayFire v3.4','','inherit','closed','closed','','9190-autosave-v1','','','2016-09-21 22:13:21','2016-09-22 02:13:21','',9190,'http://sandbox.arrayfire.com/9190-autosave-v1/',0,'revision','',0),(9377,4,'2016-09-21 17:46:41','2016-09-21 21:46:41','In ArrayFire v3.4, we have added a number of features that will help our vast range of users. Our users have been asking us for a way to reduce the memory footprint of their data on the GPU. And now we have answered our users\' requests. Introducing Sparse Matrices in ArrayFire!\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Sparse Data Structure[/custom_headline]\r\n\r\nA sparse data structure is one where all the non-zero elements are not stored. Sparse matrices are useful when the number of zero-values elements are &gt;&gt; number of non-zero elements (sparsity of the matrix is high).\r\nThe sparse data structure in general is stored as 3 arrays.\r\n<ul>\r\n 	<li>A data or values array containing all the non-zero elements</li>\r\n 	<li>A vector for row indices (based on storage format).</li>\r\n 	<li>A vector for column indices (based on storage format).</li>\r\n</ul>\r\nThere are many ways to store sparse matrices, the most prominent of which are\r\n<ul>\r\n 	<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_row_.28CSR.2C_CRS_or_Yale_format.29\">Compressed Sparse Row</a></li>\r\n 	<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_column_.28CSC_or_CCS.29\">Compressed Storage Column</a></li>\r\n 	<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Coordinate_list_.28COO.29\">Coordinate Format</a>.</li>\r\n</ul>\r\nArrayFire supports CSR and COO format, with CSC support coming in future releases.\r\n\r\nA more detailed description of Sparse Matrix can be found on <a href=\"https://en.wikipedia.org/wiki/Sparse_matrix\">Wikipedia</a>.\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Sparse Arrays in ArrayFire[/custom_headline]\r\n\r\nIn ArrayFire, Sparse arrays use the same user facing object - af::array. This makes working with sparse arrays all the more seamless. We have added a function <a href=\"http://arrayfire.org/docs/group__method__mat.htm#ga5945f95549bae25f75694a69d98ca584\">af::array::issparse()</a> that allows users to check if an array is sparse or not. Although externally, there is no difference between a sparse and dense array, internally, the sparse array is stored as 3 independent arrays along with other meta data. These arrays, as well as other properties can be fetched using the <a href=\"http://arrayfire.org/docs/group__sparse__func.htm\">sparse functions</a>.\r\n\r\nAs of v3.4, the following features are supported for sparse arrays:\r\n<ul>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm\">Creation for sparse arrays.</a>\r\n<ul>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae4621e58bfaf6d80668309824fde86c0\">From 3 (data, row, column) sparse host or device memory vectors.</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae9a843c7e93b05e6b090ca32b271ebe9\">From 3 (data, row, column) sparse af::arrays.</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">From dense af::array.</a></li>\r\n 	<li>To create a sparse array from host or device dense array, first create a dense af::array, and then pass it to create sparse af::array from dense af::array.</li>\r\n</ul>\r\n</li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__sparse__func__convert__to.htm\">Storage format conversions.</a>\r\n<ul>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">Dense to CSR/COO.</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__sparse__func__dense.htm\">CSR/COO to dense.</a></li>\r\n</ul>\r\n</li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">Sparse-Dense Matrix Multiplication (SPMM/SPMV)</a>\r\n<ul>\r\n 	<li>Sparse Matrix - Dense Matrix</li>\r\n 	<li>Sparse Matrix - Dense Vector</li>\r\n 	<li>Options for Sparse Matrix to be transposed.</li>\r\n</ul>\r\n</li>\r\n</ul>\r\nThe Sparse-Dense Matrix Multiplcation is a implemented as a part of <a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">af::matmul()</a>. This is part of making the usage of sparse arrays seamless with regular arrays.\r\n\r\nAs with all ArrayFire features, Sparse support is available across all backends and all architectures.\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Example[/custom_headline]\r\n\r\nThe example we shall use is a conjugate gradient solver. The inputs will be the same except that for the sparse conjugate gradient we will convert the A matrix to sparse.\r\n<pre class=\"\">#include \r\n#include \r\n\r\nusing namespace af;\r\n\r\nstatic size_t dimension = 16 * 1024;\r\nstatic const int maxIter = 10;\r\nstatic const int sparsityFactor = 7;\r\n\r\nstatic array A;\r\nstatic array spA;  // Sparse A\r\nstatic array x0;\r\nstatic array b;\r\n\r\nvoid setupInputs()\r\n{\r\n    // Generate a random input: A\r\n    array T = randu(dimension, dimension, f32);\r\n    // Create 0s in input.\r\n    // Anything that is no divisible by sparsityFactor will become 0.\r\n    A = floor(T * 1000);\r\n    A = A * ((A % sparsityFactor) == 0) / 1000;\r\n    // Make it positive definite\r\n    A = transpose(A) + A + A.dims(0)*identity(A.dims(0), A.dims(0), f32);\r\n\r\n    // Make A sparse as spA\r\n    spA = sparse(A);\r\n\r\n    // Generate x0: Random guess\r\n    x0 = randu(A.dims(0), f32);\r\n\r\n    //Generate b\r\n    b = matmul(A, x0);\r\n\r\n    std::cout &lt;&lt; \"Sparsity of A = \"\r\n              &lt;&lt;  100.f * (float)sparseGetNNZ(spA) / (float)spA.elements()\r\n              &lt;&lt; std::endl;\r\n    std::cout &lt;&lt; \"Memory Usage of A = \" &lt;&lt;  A.bytes() &lt;&lt; \" bytes\" &lt;&lt; std::endl;\r\n    std::cout &lt;&lt; \"Memory Usage of spA = \"\r\n              &lt;&lt; sparseGetValues(spA).bytes()\r\n               + sparseGetRowIdx(spA).bytes()\r\n               + sparseGetColIdx(spA).bytes()\r\n              &lt;&lt; \" bytes\" &lt;&lt; std::endl;\r\n}\r\n\r\nvoid conjugateGradient(const af::array in)\r\n{\r\n    array x = constant(0, b.dims(), f32);\r\n    array r = b - matmul(in, x);\r\n    array p = r;\r\n\r\n    for (int i = 0; i &lt; maxIter; ++i) {\r\n        array Ap = matmul(in, p);\r\n        array alpha_num = dot(r, r);\r\n        array alpha_den = dot(p, Ap);\r\n        array alpha = alpha_num/alpha_den;\r\n        r = r - tile(alpha, Ap.dims())*Ap;\r\n        x = x + tile(alpha, Ap.dims())*p;\r\n        array beta_num = dot(r, r);\r\n        array beta = beta_num/alpha_num;\r\n        p = r + tile(beta, p.dims())*p;\r\n    }\r\n    array res = x0 - x;\r\n\r\n    std::cout&lt;&lt;\"Final difference in solutions:\\n\";\r\n    af_print(dot(res, res));\r\n}\r\n\r\nint main(int argc, char *argv[])\r\n{\r\n    af::info();\r\n    setupInputs();\r\n\r\n    std::cout &lt;&lt; \"Verifying Dense Conjugate Gradient:\" &lt;&lt; std::endl;\r\n    conjugateGradient(A);\r\n\r\n    std::cout &lt;&lt; \"Verifying Sparse Conjugate Gradient:\" &lt;&lt; std::endl;\r\n    conjugateGradient(spA);\r\n\r\n    return 0;\r\n}\r\n</pre>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Performance and Memory Footprint[/custom_headline]\r\n\r\nUsing the CUDA backend on a Tesla K40c, the conjugate gradient function took a time of 64.94ms for dense matrix and 39.31ms for sparse Matrix. That is almost a 2x speedup! Add in the fact that not all computation in the conjugate gradient function is sparse, the speedup for sparse vs dense matrix multiplication is extremely good.\r\n\r\nWith a <code>sparsityFactor</code> of 7, the sparsity of A is about 26.38%. This means a tremendous amount of space and performance can be achieved by using sparse arrays. For a floating point matrix of size 16M x 16M, a dense matrix uses 1 GB of memory while the sparse array, stored in CSR format, only used 527 MB - halving the memory consumption!\r\n<table>\r\n<tbody>\r\n<tr>\r\n<th>Matrix Elements</th>\r\n<th>Memory Size (Dense)</th>\r\n<th>Memory Size (Sparse)</th>\r\n<th>CG Time (Dense)</th>\r\n<th>CG Time (Sparse)</th>\r\n</tr>\r\n<tr>\r\n<th>16M x 16M</th>\r\n<th>1024 MB</th>\r\n<th>527 MB</th>\r\n<th>64.94ms</th>\r\n<th>39.31ms</th>\r\n</tr>\r\n<tr>\r\n<th>4M x 4M</th>\r\n<th>64 MB</th>\r\n<th>33 MB</th>\r\n<th>4.752ms</th>\r\n<th>3.935ms</th>\r\n</tr>\r\n</tbody>\r\n</table>\r\nSparse arrays will now let users store more data on the GPU while allowing faster computation, thus allowing to solve for bigger, more complex problem.\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Download[/custom_headline]\r\n\r\nArrayFire v3.4 can be downloaded from these locations:\r\n<ul>\r\n 	<li><a href=\"http://sandbox.arrayfire.com/download/\">Binaries</a></li>\r\n 	<li><a href=\"https://github.com/arrayfire\">Source code</a></li>\r\n</ul>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Community[/custom_headline]\r\n\r\nArrayFire is continually improving through the addition of new sparse matrix enhancements. We welcome your feedback:\r\n<ul>\r\n 	<li>General discussion forums on the <a href=\"https://groups.google.com/forum/#!forum/arrayfire-users\">ArrayFire Google Group</a></li>\r\n 	<li>Live discussion chat on the <a href=\"https://gitter.im/arrayfire/arrayfire?utm_source=badge&amp;utm_medium=badge&amp;utm_campaign=pr-badge&amp;utm_content=badge\">ArrayFire Gitter</a></li>\r\n 	<li>Issue reports on the <a href=\"https://github.com/arrayfire/arrayfire/issues\">ArrayFire GitHub</a></li>\r\n</ul>\r\nFinally, as you find success with ArrayFire, we invite you to contribute a post to this blog to share with the broader community. Email <a href=\"mailto:scott@arrayfire.com\">scott@arrayfire.com</a> to contribute to this blog.','Sparse Matrices in ArrayFire v3.4','','inherit','closed','closed','','9190-revision-v1','','','2016-09-21 17:46:41','2016-09-21 21:46:41','',9190,'http://sandbox.arrayfire.com/9190-revision-v1/',0,'revision','',0);
INSERT INTO `wp_posts` VALUES (9378,4,'2016-09-21 18:01:55','2016-09-21 22:01:55','In ArrayFire v3.4, we have added support for sparse matrices, which greatly reduce the memory footprint on GPUs and accelerated devices for many applications.\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Sparse Data Structure[/custom_headline]\r\n\r\nA sparse data structure is one where all the non-zero elements are not stored. Sparse matrices are useful when the number of zero-values elements are much greater than the number of non-zero elements (i.e. the sparsity of the matrix is high). A sparse data structure is generally stored as 3 arrays:\r\n<ul>\r\n 	<li>A data or values array containing all the non-zero elements</li>\r\n 	<li>A vector for row indices (based on storage format)</li>\r\n 	<li>A vector for column indices (based on storage format)</li>\r\n</ul>\r\nThere are many ways to store sparse matrices, the most prominent of which are:\r\n<ul>\r\n 	<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_row_.28CSR.2C_CRS_or_Yale_format.29\">Compressed Sparse Row</a> (CSR)</li>\r\n 	<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_column_.28CSC_or_CCS.29\">Compressed Storage Column</a> (CSC)</li>\r\n 	<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Coordinate_list_.28COO.29\">Coordinate Format</a> (COO)</li>\r\n</ul>\r\nArrayFire supports CSR and COO format, with support for CSC coming in future releases.\r\n\r\nA more detailed description of sparse matrices can be found on <a href=\"https://en.wikipedia.org/wiki/Sparse_matrix\">Wikipedia</a>.\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Sparse Arrays in ArrayFire[/custom_headline]\r\n\r\nIn ArrayFire, sparse arrays use the same user-facing object, af::array. This makes working with sparse arrays very seamless. We have added a function <a href=\"http://arrayfire.org/docs/group__method__mat.htm#ga5945f95549bae25f75694a69d98ca584\">af::array::issparse()</a> that allows users to check if an array is sparse or not. Although externally, there is no difference between a sparse and dense array, internally, the sparse array is stored as 3 independent arrays along with other metadata. These arrays, as well as other properties, can be fetched using the <a href=\"http://arrayfire.org/docs/group__sparse__func.htm\">sparse functions</a>.\r\n\r\nIn v3.4, the following features are supported for sparse arrays:\r\n<ul>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm\">Creation for sparse arrays.</a>\r\n<ul>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae4621e58bfaf6d80668309824fde86c0\">From 3 (data, row, column) sparse host or device memory vectors.</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae9a843c7e93b05e6b090ca32b271ebe9\">From 3 (data, row, column) sparse af::arrays.</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">From dense af::array.</a></li>\r\n 	<li>To create a sparse array from a host or device dense array, first create a dense af::array, and then pass it to create sparse af::array.</li>\r\n</ul>\r\n</li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__sparse__func__convert__to.htm\">Storage format conversions.</a>\r\n<ul>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">Dense to CSR/COO.</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__sparse__func__dense.htm\">CSR/COO to dense.</a></li>\r\n</ul>\r\n</li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">Sparse-Dense Matrix Multiplication (SPMM/SPMV)</a>\r\n<ul>\r\n 	<li>Sparse Matrix - Dense Matrix</li>\r\n 	<li>Sparse Matrix - Dense Vector</li>\r\n 	<li>Options for Sparse Matrix to be transposed</li>\r\n</ul>\r\n</li>\r\n</ul>\r\nThe Sparse-Dense Matrix Multiplication is implemented within the same <a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">af::matmul()</a> function used for dense arrays.\r\n\r\nAs with all ArrayFire features, sparse support is available across all backends (CUDA, OpenCL, and CPU) and supported devices and architectures.\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Example[/custom_headline]\r\n\r\nIn this section, we showcase the difference in running a conjugate gradient solver with a dense array and a sparse array. The inputs are the same for each example. In the sparse case, the \"A\" matrix is converted to sparse.\r\n<pre class=\"\">#include \r\n#include \r\n\r\nusing namespace af;\r\n\r\nstatic size_t dimension = 16 * 1024;\r\nstatic const int maxIter = 10;\r\nstatic const int sparsityFactor = 7;\r\n\r\nstatic array A;\r\nstatic array spA;  // Sparse A\r\nstatic array x0;\r\nstatic array b;\r\n\r\nvoid setupInputs()\r\n{\r\n    // Generate a random input: A\r\n    array T = randu(dimension, dimension, f32);\r\n    // Create 0s in input.\r\n    // Anything that is no divisible by sparsityFactor will become 0.\r\n    A = floor(T * 1000);\r\n    A = A * ((A % sparsityFactor) == 0) / 1000;\r\n    // Make it positive definite\r\n    A = transpose(A) + A + A.dims(0)*identity(A.dims(0), A.dims(0), f32);\r\n\r\n    // Make A sparse as spA\r\n    spA = sparse(A);\r\n\r\n    // Generate x0: Random guess\r\n    x0 = randu(A.dims(0), f32);\r\n\r\n    //Generate b\r\n    b = matmul(A, x0);\r\n\r\n    std::cout &lt;&lt; \"Sparsity of A = \"\r\n              &lt;&lt;  100.f * (float)sparseGetNNZ(spA) / (float)spA.elements()\r\n              &lt;&lt; std::endl;\r\n    std::cout &lt;&lt; \"Memory Usage of A = \" &lt;&lt;  A.bytes() &lt;&lt; \" bytes\" &lt;&lt; std::endl;\r\n    std::cout &lt;&lt; \"Memory Usage of spA = \"\r\n              &lt;&lt; sparseGetValues(spA).bytes()\r\n               + sparseGetRowIdx(spA).bytes()\r\n               + sparseGetColIdx(spA).bytes()\r\n              &lt;&lt; \" bytes\" &lt;&lt; std::endl;\r\n}\r\n\r\nvoid conjugateGradient(const af::array in)\r\n{\r\n    array x = constant(0, b.dims(), f32);\r\n    array r = b - matmul(in, x);\r\n    array p = r;\r\n\r\n    for (int i = 0; i &lt; maxIter; ++i) {\r\n        array Ap = matmul(in, p);\r\n        array alpha_num = dot(r, r);\r\n        array alpha_den = dot(p, Ap);\r\n        array alpha = alpha_num/alpha_den;\r\n        r = r - tile(alpha, Ap.dims())*Ap;\r\n        x = x + tile(alpha, Ap.dims())*p;\r\n        array beta_num = dot(r, r);\r\n        array beta = beta_num/alpha_num;\r\n        p = r + tile(beta, p.dims())*p;\r\n    }\r\n    array res = x0 - x;\r\n\r\n    std::cout&lt;&lt;\"Final difference in solutions:\\n\";\r\n    af_print(dot(res, res));\r\n}\r\n\r\nint main(int argc, char *argv[])\r\n{\r\n    af::info();\r\n    setupInputs();\r\n\r\n    std::cout &lt;&lt; \"Verifying Dense Conjugate Gradient:\" &lt;&lt; std::endl;\r\n    conjugateGradient(A);\r\n\r\n    std::cout &lt;&lt; \"Verifying Sparse Conjugate Gradient:\" &lt;&lt; std::endl;\r\n    conjugateGradient(spA);\r\n\r\n    return 0;\r\n}\r\n</pre>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Performance and Memory Footprint[/custom_headline]\r\n\r\nUsing the CUDA backend on a Tesla K40c, the conjugate gradient function took 64.94ms for the dense matrix and 39.31ms for the sparse Matrix. That is almost a 2x speedup!\r\n\r\nWith a <code>sparsityFactor</code> of 7, the sparsity of A is about 26.38%. This means a tremendous amount of memory is conserved. For a floating point matrix of size 16M x 16M, a dense matrix uses 1 GB of memory while the sparse array, stored in CSR format, only uses 527 MB. That is a 2x savings on memory usage!\r\n<table>\r\n<tbody>\r\n<tr>\r\n<th>Matrix Elements</th>\r\n<th>Memory Size (Dense)</th>\r\n<th>Memory Size (Sparse)</th>\r\n<th>CG Time (Dense)</th>\r\n<th>CG Time (Sparse)</th>\r\n</tr>\r\n<tr>\r\n<th>16M x 16M</th>\r\n<th>1024 MB</th>\r\n<th>527 MB</th>\r\n<th>64.94ms</th>\r\n<th>39.31ms</th>\r\n</tr>\r\n<tr>\r\n<th>4M x 4M</th>\r\n<th>64 MB</th>\r\n<th>33 MB</th>\r\n<th>4.752ms</th>\r\n<th>3.935ms</th>\r\n</tr>\r\n</tbody>\r\n</table>\r\nSparse arrays will now let users store more data on the GPU while allowing faster computation, thus allowing to solve for bigger, more complex problems.\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Download[/custom_headline]\r\n\r\nArrayFire v3.4 can be downloaded from these locations:\r\n<ul>\r\n 	<li><a href=\"http://sandbox.arrayfire.com/download/\">Binaries</a></li>\r\n 	<li><a href=\"https://github.com/arrayfire\">Source code</a></li>\r\n</ul>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Community[/custom_headline]\r\n\r\nArrayFire is continually improving through the addition of new sparse matrix enhancements. We welcome your feedback:\r\n<ul>\r\n 	<li>General discussion forums on the <a href=\"https://groups.google.com/forum/#!forum/arrayfire-users\">ArrayFire Google Group</a></li>\r\n 	<li>Live discussion chat on the <a href=\"https://gitter.im/arrayfire/arrayfire?utm_source=badge&amp;utm_medium=badge&amp;utm_campaign=pr-badge&amp;utm_content=badge\">ArrayFire Gitter</a></li>\r\n 	<li>Issue reports on the <a href=\"https://github.com/arrayfire/arrayfire/issues\">ArrayFire GitHub</a></li>\r\n</ul>\r\nFinally, as you find success with ArrayFire, we invite you to contribute a post to this blog to share with the broader community. Email <a href=\"mailto:scott@arrayfire.com\">scott@arrayfire.com</a> to contribute to this blog.','Sparse Matrices in ArrayFire v3.4','','inherit','closed','closed','','9190-revision-v1','','','2016-09-21 18:01:55','2016-09-21 22:01:55','',9190,'http://sandbox.arrayfire.com/9190-revision-v1/',0,'revision','',0),(9379,6,'2016-09-21 21:16:02','2016-09-22 01:16:02','<span style=\"font-weight: 400;\">Today we are pleased to announce the release of ArrayFire v3.4, our open source library of parallel computing functions supporting CUDA, OpenCL, and CPU devices. This new version of ArrayFire improves features and performance for applications in machine learning, computer vision, signal processing, statistics, finance, and more.</span>\r\n\r\n<span style=\"font-weight: 400;\">This release focuses on 5 major components of the library that are common to many areas of mathematical, scientific, and financial computing:  sparse matrix operations, random number generation, image processing, just-in-time (JIT) compilation, and visualizations.</span>\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Major Updates and Features[/custom_headline]\r\n<ul>\r\n 	<li>Sparse Matrix and BLAS (see blog post)\r\n<ul>\r\n 	<li>Support for CSR and COO storage types</li>\r\n 	<li>Sparse-Dense Matrix Multiplication and Matrix-Vector Multiplication</li>\r\n 	<li>Conversion to and from dense matrix to CSR and COO storage types</li>\r\n</ul>\r\n</li>\r\n 	<li>Support for Random Number Generator Engines (see blog post)\r\n<ul>\r\n 	<li>Philox</li>\r\n 	<li>Threefry</li>\r\n 	<li>Mersenne Twister</li>\r\n</ul>\r\n</li>\r\n 	<li>Image Processing (see blog post)\r\n<ul>\r\n 	<li>New interpolation functions</li>\r\n 	<li>Image moments</li>\r\n</ul>\r\n</li>\r\n 	<li>Faster JIT for CUDA and OpenCL (see blog post)\r\n<ul>\r\n 	<li>Performance improvements</li>\r\n 	<li>Support for evaluating multiple outputs in a single kernel</li>\r\n</ul>\r\n</li>\r\n 	<li>Improvements to Graphics (see blog post)\r\n<ul>\r\n 	<li>Using Forge v0.9.0</li>\r\n 	<li>Vector Field plotting functionality</li>\r\n 	<li>Multiple overlays on the same window are now possible</li>\r\n 	<li>New API to set axes limits for graphs</li>\r\n 	<li>New API for plot and scatter</li>\r\n</ul>\r\n</li>\r\n</ul>\r\nFor the comprehensive list of new features and updates, please refer to our <a href=\"http://arrayfire.org/docs/releasenotes.htm\">release notes</a>.\r\n\r\n<span style=\"font-weight: 400;\">While we are happy for reaching this milestone, we are not complacent. Work on the next milestone has already commenced, and we hope to exceed the expectations of our community.</span>\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Download[/custom_headline]\r\n\r\nArrayFire v3.4 can be downloaded from these locations:\r\n<ul>\r\n 	<li><a href=\"http://sandbox.arrayfire.com/download/\">Binaries</a></li>\r\n 	<li><a href=\"https://github.com/arrayfire\">Source code</a></li>\r\n</ul>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Community[/custom_headline]\r\n\r\nArrayFire is continually improving through the addition of new functions and features. We welcome your feedback:\r\n<ul>\r\n 	<li>General discussion forums on the <a href=\"https://groups.google.com/forum/#!forum/arrayfire-users\">ArrayFire Google Group</a></li>\r\n 	<li>Live discussion chat on the <a href=\"https://gitter.im/arrayfire/arrayfire?utm_source=badge&amp;utm_medium=badge&amp;utm_campaign=pr-badge&amp;utm_content=badge\">ArrayFire Gitter</a></li>\r\n 	<li>Issue reports on the <a href=\"https://github.com/arrayfire/arrayfire/issues\">ArrayFire GitHub</a></li>\r\n</ul>\r\nFinally, as you find success with ArrayFire, we invite you to contribute a post to this blog to share with the broader community. Email <a href=\"mailto:scott@arrayfire.com\">scott@arrayfire.com</a> to contribute to this blog.\r\n\r\n&nbsp;','ArrayFire v3.4 Official Release','','inherit','closed','closed','','9154-autosave-v1','','','2016-09-21 21:16:02','2016-09-22 01:16:02','',9154,'http://sandbox.arrayfire.com/9154-autosave-v1/',0,'revision','',0),(9380,4,'2016-09-21 22:15:06','2016-09-22 02:15:06','There are a number of  additions and updates to image based features in the new v3.4 release of ArrayFire. Among the updates are:\r\n<ul>\r\n 	<li>New interpolation methods for several existing functions\r\n<ul>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__signal__func__approx1.htm\">approx1</a>, <a href=\"http://arrayfire.org/docs/group__signal__func__approx2.htm\">approx2</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__transform.htm\">transform</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__resize.htm\">resize</a></li>\r\n</ul>\r\n</li>\r\n 	<li>Functions for image <a href=\"http://arrayfire.org/docs/group__image__func__moments.htm\">moments</a></li>\r\n</ul>\r\nThis blog post will display some typical use cases for these new features.\r\n\r\n<span style=\"line-height: 1.5;\">[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Interpolation methods[/custom_headline]</span>\r\n\r\nArrayFire v3.4 implements several new <a href=\"http://arrayfire.org/docs/defines_8h.htm#ac0b7c1cfd1fb146314d96f17bbfffa68\"> interpolation methods </a>for 1-d and 2-d domains. The new interpolation methods for 1-d functions are:\r\n<ul>\r\n 	<li><strong>AF_INTERP_LINEAR_COSINE</strong></li>\r\n 	<li><strong>AF_INTERP_CUBIC</strong></li>\r\n</ul>\r\nand for 2-d functions are:\r\n<ul>\r\n 	<li><strong>AF_INTERP_BILINEAR_COSINE</strong></li>\r\n 	<li><strong>AF_INTERP_BICUBIC</strong></li>\r\n</ul>\r\nThe behavior of the <a href=\"http://arrayfire.org/docs/defines_8h.htm#ac0b7c1cfd1fb146314d96f17bbfffa68\">interpolation methods</a> can be seen in the following pictures.\r\n\r\n&nbsp;\r\n\r\n[caption id=\"attachment_9338\" align=\"aligncenter\" width=\"300\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_0.png\"><img class=\"size-medium wp-image-9338\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_0-300x271.png\" alt=\"Original data points\" width=\"300\" height=\"271\" /></a> Original data points[/caption]\r\n\r\n[caption id=\"attachment_9339\" align=\"aligncenter\" width=\"300\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_1.png\"><img class=\"size-medium wp-image-9339\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_1-300x267.png\" alt=\"Linear interpolation\" width=\"300\" height=\"267\" /></a> Linear interpolation[/caption]\r\n\r\n[caption id=\"attachment_9340\" align=\"aligncenter\" width=\"300\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_2.png\"><img class=\"size-medium wp-image-9340\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_2-300x252.png\" alt=\"Linear cosine interpolation\" width=\"300\" height=\"252\" /></a> Linear cosine interpolation[/caption]\r\n\r\n[caption id=\"attachment_9337\" align=\"aligncenter\" width=\"300\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_3.png\"><img class=\"size-medium wp-image-9337\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/ap1_3-300x255.png\" alt=\"Cubic interpolation\" width=\"300\" height=\"255\" /></a> Cubic interpolation[/caption]\r\n\r\nA common use for interpolation is image filtering. Given a coarse image, we can resample it to be smoother.\r\n<pre class=\"lang:c++ decode:true\" title=\"Interpolating images with approx2()\">af::array img = af::randu(7,7); //create a random image\r\n\r\n//define sample points for interpolation\r\naf::array Xs = af::seq(0, 6, 0.1f);\r\naf::array Ys = af::seq(0, 6, 0.1f);\r\nXs = af::tile(Xs, 1, Ys.dims(0));\r\nYs = af::tile(Ys.T(), Xs.dims(0));\r\n\r\n//interpolate based on specific method\r\naf::array img_bilinear = af::approx2(img, Xs, Ys, AF_INTERP_BILINEAR);\r\naf::array img_bilinearcos = af::approx2(img, Xs, Ys, AF_INTERP_BILINEAR_COSINE);\r\naf::array img_bicubic = af::approx2(img, Xs, Ys, AF_INTERP_BICUBIC_SPLINE);</pre>\r\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/Capture.png\"><img class=\"aligncenter size-full wp-image-9245\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/Capture.png\" alt=\"Capture\" width=\"1025\" height=\"243\" /></a>\r\n\r\nThe new interpolation methods further apply to several similar image transformation functions:\r\n<ul>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__resize.htm\">Resize</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__rotate.htm\">Rotate</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__scale.htm\">Scale</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__skew.htm\">Skew</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__transform.htm\">Transform</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__transform__func__translate.htm\">Translate</a></li>\r\n</ul>\r\nDifferent interpolation methods greatly affect the final result of image processing operations. In general, bilinear and bilinear-cosine produce a good result when upscaling an image, however they often result in a blurry look. The bicubic method tends to sharpen the image resulting in slightly stronger borders and a better overall look. Below are some examples of this behavior using image rotation and image scaling.\r\n\r\n[caption id=\"attachment_9331\" align=\"aligncenter\" width=\"300\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/s_0.png\"><img class=\"size-medium wp-image-9331\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/s_0-300x84.png\" alt=\"3x image scaling using nearest-neighbor sampling\" width=\"300\" height=\"84\" /></a> 3x image scaling using nearest-neighbor sampling[/caption]\r\n\r\n[caption id=\"attachment_9332\" align=\"aligncenter\" width=\"300\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/s_1.png\"><img class=\"size-medium wp-image-9332\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/s_1-300x85.png\" alt=\"3x image scaling using bilinear-cosine sampling\" width=\"300\" height=\"85\" /></a> 3x image scaling using bilinear-cosine sampling[/caption]\r\n\r\n[caption id=\"attachment_9334\" align=\"aligncenter\" width=\"300\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/s_2.png\"><img class=\"size-medium wp-image-9334\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/s_2-300x85.png\" alt=\"3x image scaling using bicubic sampling\" width=\"300\" height=\"85\" /></a> 3x image scaling using bicubic sampling[/caption]\r\n\r\n[block_grid type=\"four-up\"] [block_grid_item]\r\n\r\n[caption id=\"attachment_9335\" align=\"aligncenter\" width=\"150\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_0.png\"><img class=\"wp-image-9335 size-thumbnail\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_0-150x150.png\" alt=\"rot_0\" width=\"150\" height=\"150\" /></a> Image rotation with nearest neighbor sampling.[/caption]\r\n\r\n[/block_grid_item][block_grid_item]\r\n\r\n[caption id=\"attachment_9336\" align=\"aligncenter\" width=\"150\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_1.png\"><img class=\"wp-image-9336 size-thumbnail\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_1-150x150.png\" alt=\"rot_1\" width=\"150\" height=\"150\" /></a> Image rotation with bilinear sampling.[/caption]\r\n\r\n[/block_grid_item][block_grid_item]\r\n\r\n[caption id=\"attachment_9329\" align=\"aligncenter\" width=\"150\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_2.png\"><img class=\"wp-image-9329 size-thumbnail\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_2-150x150.png\" alt=\"rot_2\" width=\"150\" height=\"150\" /></a> Image rotation with bilinear-cosine sampling.[/caption]\r\n\r\n[/block_grid_item][block_grid_item]\r\n\r\n[caption id=\"attachment_9330\" align=\"aligncenter\" width=\"150\"]<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_3.png\"><img class=\"wp-image-9330 size-thumbnail\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/rot_3-150x150.png\" alt=\"rot_3\" width=\"150\" height=\"150\" /></a> Image rotation with bicubic sampling.[/caption]\r\n\r\n[/block_grid_item][block_grid_item]\r\n\r\n[/block_grid]\r\n\r\n<span style=\"line-height: 1.5;\">[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Image moments[/custom_headline]</span>\r\n\r\nImage <a href=\"http://arrayfire.org/docs/group__image__func__moments.htm\">moments </a>are another new addition to ArrayFire v3.4. Image moments are weighted averages of pixels in an image which provide useful properties of an image. A common use of image moments is to find the center of mass or area (or gray level sum) of an image.\r\n\r\nCurrently, ArrayFire calculates all<a href=\"http://arrayfire.org/docs/defines_8h.htm#ad3ee8e32709131cf4126dfa26210aec0\"> first order moments</a>. Each moment can be returned individually or all first-order moments can be calculated at once. All moments can be found as follows:\r\n<pre class=\"lang:c++ decode:true\">af::array moments = af::moments(input_image, AF_MOMENT_FIRST_ORDER);</pre>\r\n<div class=\"fragment\">\r\n<div class=\"line\">Here is an example of how the shorthand versions might be used to find the area and center of mass of an image:</div>\r\n</div>\r\n<pre class=\"lang:default decode:true\" title=\"Calculating center of mass of grayscale image\">double m00, m01, m10;\r\naf::moments(&amp;m00, aimg, AF_MOMENT_M00);\r\naf::moments(&amp;m01, aimg, AF_MOMENT_M01);\r\naf::moments(&amp;m10, aimg, AF_MOMENT_M10);\r\n\r\ndouble area = m00;\r\ndouble x_center = m10 / m00;\r\ndouble y_center = m01 / m00;\r\n</pre>\r\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/com.png\"><img class=\"aligncenter wp-image-9246 size-medium\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/com-300x300.png\" alt=\"com\" width=\"300\" height=\"300\" /></a>\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Download[/custom_headline]\r\n\r\nArrayFire v3.4 can be downloaded from these locations:\r\n<ul>\r\n 	<li><a href=\"http://sandbox.arrayfire.com/download/\">Binaries</a></li>\r\n 	<li><a href=\"https://github.com/arrayfire\">Source code</a></li>\r\n</ul>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Community[/custom_headline]\r\n\r\nArrayFire is continually improving through the addition of new image processing and computer vision functions. We welcome your feedback:\r\n<ul>\r\n 	<li>General discussion forums on the <a href=\"https://groups.google.com/forum/#!forum/arrayfire-users\">ArrayFire Google Group</a></li>\r\n 	<li>Live discussion chat on the <a href=\"https://gitter.im/arrayfire/arrayfire?utm_source=badge&amp;utm_medium=badge&amp;utm_campaign=pr-badge&amp;utm_content=badge\">ArrayFire Gitter</a></li>\r\n 	<li>Issue reports on the <a href=\"https://github.com/arrayfire/arrayfire/issues\">ArrayFire GitHub</a></li>\r\n</ul>\r\nFinally, as you find success with ArrayFire, we invite you to contribute a post to this blog to share with the broader community. Email <a href=\"mailto:scott@arrayfire.com\">scott@arrayfire.com</a> to contribute to this blog.','Image Processing Functions in ArrayFire v3.4','','inherit','closed','closed','','9166-revision-v1','','','2016-09-21 22:15:06','2016-09-22 02:15:06','',9166,'http://sandbox.arrayfire.com/9166-revision-v1/',0,'revision','',0),(9381,4,'2016-09-21 22:16:45','2016-09-22 02:16:45','ArrayFire uses <a href=\"http://sandbox.arrayfire.com/performance-of-arrayfire-jit-code-generation/\">Just In Time</a> compilation to combine many light weight functions into a single kernel launch. This along with our <a href=\"http://arrayfire.org/docs/group__arrayfire__func.htm\">easy-to-use API</a> allows users to not only quickly prototype their algorithms, but also get the best out of the underlying hardware.\r\n\r\nThis feature has been a favorite among our users in the domains of finance and scientific simulation.\r\n\r\nThat said, ArrayFire v3.3 and earlier had a few limitations. Namely:\r\n<ul class=\"org-ul\">\r\n 	<li>Multiple outputs with inter-dependent variables were generating multiple kernels.</li>\r\n 	<li>The number of operations per kernel was fairly limited by default.</li>\r\n</ul>\r\nIn the latest release of ArrayFire, we addressed these issues to get some pretty impressive numbers.\r\n\r\nIn the rest of the post, we demonstrate the performance improvements using our <a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/financial/black_scholes_options.cpp\">BlackScholes example</a>. For this particular example, the improvements for the CUDA backend were <b>~8x</b> and OpenCL backend were between <b>~5x</b>.\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Performance Comparison between v3.4 and v3.3[/custom_headline]\r\n<div id=\"outline-container-sec-1\" class=\"outline-2\">\r\n<div id=\"text-1\" class=\"outline-text-2\">\r\n\r\n<a href=\"https://en.wikipedia.org/wiki/Black%25E2%2580%2593Scholes_model\">BlackScholes</a> is a commonly used mathematical model in financial markets.\r\n\r\nIt takes the following 5 inputs and generates 2 outputs.\r\n\r\nThe code to implement this model in ArrayFire can be seen below.\r\n<div class=\"org-src-container\">\r\n<pre class=\"src src-C++\">array cnd(array x)\r\n{\r\n    // constants\r\n    const float a1 =  0.254829592;\r\n    const float a2 = -0.284496736;\r\n    const float a3 =  1.421413741;\r\n    const float a4 = -1.453152027;\r\n    const float a5 =  1.061405429;\r\n    const float p  =  0.3275911;\r\n    const float sqrt2 = sqrt(2.0);\r\n\r\n    // Save the sign of x\r\n    array xSign = sign(x);\r\n\r\n    x = abs(x) / sqrt2;\r\n\r\n    // A&amp;S formula 7.1.26\r\n    array t = 1.0f / (1.0f + p*x);\r\n    array y = 1.0f + 0.5f * (((((a5*t + a4)*t) + a3)*t + a2)*t + a1)*t*exp(-x*x);\r\n\r\n    return xSign * y + !xSign * (1 - y); // equivalent of (x &gt;= 0) ? y : (1 - y);\r\n}\r\n\r\nstatic void black_scholes(array&amp; C, array&amp; P,\r\n                          const array&amp; S, const array&amp; X,\r\n                          const array&amp; R, const array&amp; V,\r\n                          const array&amp; T)\r\n{\r\n    // Inputs:\r\n    // S: Stock price\r\n    // X: Strike price\r\n    // R: Risk free rate of interest\r\n    // V: Volatility\r\n    // T: Time to maturity\r\n\r\n    // Outputs:\r\n    // C: Call option\r\n    // P: Put option\r\n\r\n    array d1 = log(S / X);\r\n    d1 = d1 + (R + (V*V)*0.5) * T;\r\n    d1 = d1 / (V*sqrt(T));\r\n\r\n    array d2 = d1 - (V*sqrt(T));\r\n\r\n    array cnd_d1 = cnd(d1);\r\n    array cnd_d2 = cnd(d2);\r\n\r\n    C = S * cnd_d1  - (X * exp((-R)*T) * cnd_d2);\r\n    P = X * exp((-R)*T) * (1 - cnd_d2) - (S * (1 - cnd_d1));\r\n}\r\n</pre>\r\n</div>\r\nThe <code>black_scholes</code> function contains a long chain of operations and generates two outputs. In older versions of ArrayFire, the Just-In-Time compiler of arrayfire was splitting up this single function call to multiple kernels.\r\n\r\nWe solved this in the latest release by:\r\n<ul class=\"org-ul\">\r\n 	<li>Enabling <code>af::eval</code> to hint ArrayFire\'s JIT compiler to combine inputs.</li>\r\n 	<li>Changing the heuristics to increase the number of operations per kernel.</li>\r\n</ul>\r\n<div class=\"org-src-container\">\r\n<pre class=\"src src-C++\">black_scholes(C, P, S, X, R, V, T);\r\neval(C, P); // Only change required for v3.4\r\n</pre>\r\n</div>\r\nThese changes resulted in dramatic performance improvements seen below. The following graph compares the number of call and put options being calculated per second on v3.3 and v3.4.\r\n\r\n&nbsp;\r\n\r\n<iframe width=\"650\" height=\"400\" seamless=\"\" frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1o4C5KZ4B5he3Ac8mJ9Z6Z_N-hlATeJwhP-j6XJtbEPA/pubchart?oid=927257953&amp;format=interactive\">\r\n</iframe>\r\n\r\n&nbsp;\r\n\r\nAs we can see, the performance has improved significantly. The absolute time taken when to calculate 2 million options is show below.\r\n\r\n&nbsp;\r\n\r\n<iframe width=\"513.5\" height=\"104.0141666666666\" seamless=\"\" frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1o4C5KZ4B5he3Ac8mJ9Z6Z_N-hlATeJwhP-j6XJtbEPA/pubchart?oid=1295781261&amp;format=interactive\"></iframe>\r\n\r\nNote:\r\n<ul class=\"org-ul\">\r\n 	<li>The additional gains made by CUDA are because of a <a href=\"https://github.com/arrayfire/arrayfire/pull/1534\">separate issue</a> involving CUDA JIT that was fixed.</li>\r\n 	<li>Only 24 threads of the 32 available were used for the Xeon CPU.</li>\r\n</ul>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Performance Comparison to Native CUDA[/custom_headline]\r\n\r\n</div>\r\n</div>\r\n<div id=\"outline-container-sec-2\" class=\"outline-2\">\r\n<div id=\"text-2\" class=\"outline-text-2\">\r\n\r\nTo see how close we can get to the peak performance, we compared the performance of ArrayFire and CUDA on the new GTX 1080.\r\n\r\nTo do this we built ArrayFire 3.4 using the CUDA 8 RC and compared its performance with that of the example provided in <a href=\"http://docs.nvidia.com/cuda/cuda-samples/#black-scholes-option-pricing\">CUDA Samples</a>.\r\n\r\nThe results of this benchmark can be seen below.\r\n\r\n<iframe width=\"655.1414131325375\" height=\"323.5\" seamless=\"\" frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1o4C5KZ4B5he3Ac8mJ9Z6Z_N-hlATeJwhP-j6XJtbEPA/pubchart?oid=256763192&amp;format=interactive\"></iframe>\r\n\r\nThe results indicate ArrayFire v3.4 performs identical to the native kernel at larger simulation sizes, but there is a constant overhead causing issues at smaller sizes. This is one of the issues we want to address in future releases.\r\n\r\nNote: The <code>black_scholes</code> function had to be changed slightly as shown below to be similar to the sample provided by CUDA.\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Coming Soon[/custom_headline]\r\n\r\n</div>\r\n</div>\r\n<div id=\"outline-container-sec-3\" class=\"outline-2\">\r\n<div id=\"text-3\" class=\"outline-text-2\">\r\n\r\nWhile the new improvements to the JIT engine in ArrayFire can seamlessly generate kernels that perform close to handwritten native kernels on GPUs, there is room for improvement at smaller sizes. To see what lies ahead for ArrayFire JIT, please check out <a href=\"https://github.com/arrayfire/arrayfire/issues/1401\">this github issue</a>.\r\n\r\n</div>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Download[/custom_headline]\r\n\r\nArrayFire v3.4 can be downloaded from these locations:\r\n<ul>\r\n 	<li><a href=\"http://sandbox.arrayfire.com/download/\">Binaries</a></li>\r\n 	<li><a href=\"https://github.com/arrayfire\">Source code</a></li>\r\n</ul>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Community[/custom_headline]\r\n\r\nArrayFire is continually improving through the addition of new JIT enhancements. We welcome your feedback:\r\n<ul>\r\n 	<li>General discussion forums on the <a href=\"https://groups.google.com/forum/#!forum/arrayfire-users\">ArrayFire Google Group</a></li>\r\n 	<li>Live discussion chat on the <a href=\"https://gitter.im/arrayfire/arrayfire?utm_source=badge&amp;utm_medium=badge&amp;utm_campaign=pr-badge&amp;utm_content=badge\">ArrayFire Gitter</a></li>\r\n 	<li>Issue reports on the <a href=\"https://github.com/arrayfire/arrayfire/issues\">ArrayFire GitHub</a></li>\r\n</ul>\r\nFinally, as you find success with ArrayFire, we invite you to contribute a post to this blog to share with the broader community. Email <a href=\"mailto:scott@arrayfire.com\">scott@arrayfire.com</a> to contribute to this blog.\r\n\r\n&nbsp;\r\n\r\n</div>','Performance Improvements to JIT in ArrayFire v3.4','','inherit','closed','closed','','9252-revision-v1','','','2016-09-21 22:16:45','2016-09-22 02:16:45','',9252,'http://sandbox.arrayfire.com/9252-revision-v1/',0,'revision','',0),(9382,4,'2016-09-21 22:18:18','2016-09-22 02:18:18','<span style=\"font-weight: 400;\">Today we are pleased to announce the release of ArrayFire v3.4, our open source library of parallel computing functions supporting CUDA, OpenCL, and CPU devices. This new version of ArrayFire improves features and performance for applications in machine learning, computer vision, signal processing, statistics, finance, and more.</span>\r\n\r\n<span style=\"font-weight: 400;\">This release focuses on 5 major components of the library that are common to many areas of mathematical, scientific, and financial computing:  sparse matrix operations, random number generation, image processing, just-in-time (JIT) compilation, and visualizations.</span>\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Major Updates and Features[/custom_headline]\r\n<ul>\r\n 	<li>Sparse Matrix and BLAS (<a href=\"http://sandbox.arrayfire.com/sparse-matrices-in-arrayfire-v3-4/\">see blog post</a>)\r\n<ul>\r\n 	<li>Support for CSR and COO storage types</li>\r\n 	<li>Sparse-Dense Matrix Multiplication and Matrix-Vector Multiplication</li>\r\n 	<li>Conversion to and from dense matrix to CSR and COO storage types</li>\r\n</ul>\r\n</li>\r\n 	<li>Support for Random Number Generator Engines (<a href=\"http://sandbox.arrayfire.com/random-number-generators-in-arrayfire-v3-4/\">see blog post</a>)\r\n<ul>\r\n 	<li>Philox</li>\r\n 	<li>Threefry</li>\r\n 	<li>Mersenne Twister</li>\r\n</ul>\r\n</li>\r\n 	<li>Image Processing (<a href=\"http://sandbox.arrayfire.com/image-processing-functions-in-arrayfire-v3-4/\">see blog post</a>)\r\n<ul>\r\n 	<li>New interpolation functions</li>\r\n 	<li>Image moments</li>\r\n</ul>\r\n</li>\r\n 	<li>Faster JIT for CUDA and OpenCL (<a href=\"http://sandbox.arrayfire.com/performance-improvements-to-jit-in-arrayfire-v3-4/\">see blog post</a>)\r\n<ul>\r\n 	<li>Performance improvements</li>\r\n 	<li>Support for evaluating multiple outputs in a single kernel</li>\r\n</ul>\r\n</li>\r\n 	<li>Improvements to Graphics (<a href=\"http://sandbox.arrayfire.com/graphics-updates-in-arrayfire-v3-4/\">see blog post</a>)\r\n<ul>\r\n 	<li>Using Forge v0.9.0</li>\r\n 	<li>Vector Field plotting functionality</li>\r\n 	<li>Multiple overlays on the same window are now possible</li>\r\n 	<li>New API to set axes limits for graphs</li>\r\n 	<li>New API for plot and scatter</li>\r\n</ul>\r\n</li>\r\n</ul>\r\nFor the comprehensive list of new features and updates, please refer to our <a href=\"http://arrayfire.org/docs/releasenotes.htm\">release notes</a>.\r\n\r\n<span style=\"font-weight: 400;\">While we are happy for reaching this milestone, we are not complacent. Work on the next milestone has already commenced, and we hope to exceed the expectations of our community.</span>\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Download[/custom_headline]\r\n\r\nArrayFire v3.4 can be downloaded from these locations:\r\n<ul>\r\n 	<li><a href=\"http://sandbox.arrayfire.com/download/\">Binaries</a></li>\r\n 	<li><a href=\"https://github.com/arrayfire\">Source code</a></li>\r\n</ul>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Community[/custom_headline]\r\n\r\nArrayFire is continually improving through the addition of new functions and features. We welcome your feedback:\r\n<ul>\r\n 	<li>General discussion forums on the <a href=\"https://groups.google.com/forum/#!forum/arrayfire-users\">ArrayFire Google Group</a></li>\r\n 	<li>Live discussion chat on the <a href=\"https://gitter.im/arrayfire/arrayfire?utm_source=badge&amp;utm_medium=badge&amp;utm_campaign=pr-badge&amp;utm_content=badge\">ArrayFire Gitter</a></li>\r\n 	<li>Issue reports on the <a href=\"https://github.com/arrayfire/arrayfire/issues\">ArrayFire GitHub</a></li>\r\n</ul>\r\nFinally, as you find success with ArrayFire, we invite you to contribute a post to this blog to share with the broader community. Email <a href=\"mailto:scott@arrayfire.com\">scott@arrayfire.com</a> to contribute to this blog.\r\n\r\n&nbsp;','ArrayFire v3.4 Official Release','','inherit','closed','closed','','9154-revision-v1','','','2016-09-21 22:18:18','2016-09-22 02:18:18','',9154,'http://sandbox.arrayfire.com/9154-revision-v1/',0,'revision','',0),(9383,11,'2016-09-21 22:41:22','2016-09-22 02:41:22','In ArrayFire v3.4, we have added support for sparse matrices, which greatly reduce the memory footprint on GPUs and accelerated devices for many applications.\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Sparse Data Structure[/custom_headline]\r\n\r\nA sparse data structure is one where all the non-zero elements are not stored. Sparse matrices are useful when the number of zero-values elements are much greater than the number of non-zero elements (i.e. the sparsity of the matrix is high). A sparse data structure is generally stored as 3 arrays:\r\n<ul>\r\n 	<li>A data or values array containing all the non-zero elements</li>\r\n 	<li>A vector for row indices (based on storage format)</li>\r\n 	<li>A vector for column indices (based on storage format)</li>\r\n</ul>\r\nThere are many ways to store sparse matrices, the most prominent of which are:\r\n<ul>\r\n 	<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_row_.28CSR.2C_CRS_or_Yale_format.29\">Compressed Sparse Row</a> (CSR)</li>\r\n 	<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_column_.28CSC_or_CCS.29\">Compressed Storage Column</a> (CSC)</li>\r\n 	<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Coordinate_list_.28COO.29\">Coordinate Format</a> (COO)</li>\r\n</ul>\r\nArrayFire supports CSR and COO format, with support for CSC coming in future releases.\r\n\r\nA more detailed description of sparse matrices can be found on <a href=\"https://en.wikipedia.org/wiki/Sparse_matrix\">Wikipedia</a>.\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Sparse Arrays in ArrayFire[/custom_headline]\r\n\r\nIn ArrayFire, sparse arrays use the same user-facing object, af::array. This makes working with sparse arrays very seamless. We have added a function <a href=\"http://arrayfire.org/docs/group__method__mat.htm#ga5945f95549bae25f75694a69d98ca584\">af::array::issparse()</a> that allows users to check if an array is sparse or not. Although externally, there is no difference between a sparse and dense array, internally, the sparse array is stored as 3 independent arrays along with other metadata. These arrays, as well as other properties, can be fetched using the <a href=\"http://arrayfire.org/docs/group__sparse__func.htm\">sparse functions</a>.\r\n\r\nIn v3.4, the following features are supported for sparse arrays:\r\n<ul>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm\">Creation for sparse arrays.</a>\r\n<ul>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae4621e58bfaf6d80668309824fde86c0\">From 3 (data, row, column) sparse host or device memory vectors.</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae9a843c7e93b05e6b090ca32b271ebe9\">From 3 (data, row, column) sparse af::arrays.</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">From dense af::array.</a></li>\r\n 	<li>To create a sparse array from a host or device dense array, first create a dense af::array, and then pass it to create sparse af::array.</li>\r\n</ul>\r\n</li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__sparse__func__convert__to.htm\">Storage format conversions.</a>\r\n<ul>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">Dense to CSR/COO.</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__sparse__func__dense.htm\">CSR/COO to dense.</a></li>\r\n</ul>\r\n</li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">Sparse-Dense Matrix Multiplication (SPMM/SPMV)</a>\r\n<ul>\r\n 	<li>Sparse Matrix - Dense Matrix</li>\r\n 	<li>Sparse Matrix - Dense Vector</li>\r\n 	<li>Options for Sparse Matrix to be transposed</li>\r\n</ul>\r\n</li>\r\n</ul>\r\nThe Sparse-Dense Matrix Multiplication is implemented within the same <a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">af::matmul()</a> function used for dense arrays.\r\n\r\nAs with all ArrayFire features, sparse support is available across all backends (CUDA, OpenCL, and CPU) and supported devices and architectures.\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Example[/custom_headline]\r\n\r\nIn this section, we showcase the difference in running a conjugate gradient solver with a dense array and a sparse array. The inputs are the same for each example. In the sparse case, the \"A\" matrix is converted to sparse.\r\n<pre class=\"\">\r\n#include <arrayfire.h>\r\n#include <iostream>\r\n\r\nusing namespace af;\r\n\r\nstatic size_t dimension = 16 * 1024;\r\nstatic const int maxIter = 10;\r\nstatic const int sparsityFactor = 7;\r\n\r\nstatic array A;\r\nstatic array spA;  // Sparse A\r\nstatic array x0;\r\nstatic array b;\r\n\r\nvoid setupInputs()\r\n{\r\n    // Generate a random input: A\r\n    array T = randu(dimension, dimension, f32);\r\n    // Create 0s in input.\r\n    // Anything that is no divisible by sparsityFactor will become 0.\r\n    A = floor(T * 1000);\r\n    A = A * ((A % sparsityFactor) == 0) / 1000;\r\n    // Make it positive definite\r\n    A = transpose(A) + A + A.dims(0)*identity(A.dims(0), A.dims(0), f32);\r\n\r\n    // Make A sparse as spA\r\n    spA = sparse(A);\r\n\r\n    // Generate x0: Random guess\r\n    x0 = randu(A.dims(0), f32);\r\n\r\n    //Generate b\r\n    b = matmul(A, x0);\r\n\r\n    std::cout << \"Sparsity of A = \"\r\n              << 100.f * (float)sparseGetNNZ(spA) / (float)spA.elements()\r\n              << std::endl;\r\n    std::cout << \"Memory Usage of A = \" <<  A.bytes() << \" bytes\" << std::endl;\r\n    std::cout << \"Memory Usage of spA = \"\r\n              << sparseGetValues(spA).bytes()\r\n               + sparseGetRowIdx(spA).bytes()\r\n               + sparseGetColIdx(spA).bytes()\r\n              << \" bytes\" << std::endl;\r\n}\r\n\r\nvoid conjugateGradient(const af::array in)\r\n{\r\n    array x = constant(0, b.dims(), f32);\r\n    array r = b - matmul(in, x);\r\n    array p = r;\r\n\r\n    for (int i = 0; i < maxIter; ++i) {\r\n        array Ap = matmul(in, p);\r\n        array alpha_num = dot(r, r);\r\n        array alpha_den = dot(p, Ap);\r\n        array alpha = alpha_num/alpha_den;\r\n        r = r - tile(alpha, Ap.dims())*Ap;\r\n        x = x + tile(alpha, Ap.dims())*p;\r\n        array beta_num = dot(r, r);\r\n        array beta = beta_num/alpha_num;\r\n        p = r + tile(beta, p.dims())*p;\r\n    }\r\n    array res = x0 - x;\r\n\r\n    std::cout << \"Final difference in solutions:\\n\";\r\n    af_print(dot(res, res));\r\n}\r\n\r\nint main(int argc, char *argv[])\r\n{\r\n    af::info();\r\n    setupInputs();\r\n\r\n    std::cout << \"Verifying Dense Conjugate Gradient:\" << std::endl;\r\n    conjugateGradient(A);\r\n\r\n    std::cout << \"Verifying Sparse Conjugate Gradient:\" << std::endl;\r\n    conjugateGradient(spA);\r\n\r\n    return 0;\r\n}\r\n</pre>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Performance and Memory Footprint[/custom_headline]\r\n\r\nUsing the CUDA backend on a Tesla K40c, the conjugate gradient function took 64.94ms for the dense matrix and 39.31ms for the sparse Matrix. That is almost a 2x speedup!\r\n\r\nWith a <code>sparsityFactor</code> of 7, the sparsity of A is about 26.38%. This means a tremendous amount of memory is conserved. For a floating point matrix of size 16M x 16M, a dense matrix uses 1 GB of memory while the sparse array, stored in CSR format, only uses 527 MB. That is a 2x savings on memory usage!\r\n<table>\r\n<tbody>\r\n<tr>\r\n<th>Matrix Elements</th>\r\n<th>Memory Size (Dense)</th>\r\n<th>Memory Size (Sparse)</th>\r\n<th>CG Time (Dense)</th>\r\n<th>CG Time (Sparse)</th>\r\n</tr>\r\n<tr>\r\n<th>16M x 16M</th>\r\n<th>1024 MB</th>\r\n<th>527 MB</th>\r\n<th>64.94ms</th>\r\n<th>39.31ms</th>\r\n</tr>\r\n<tr>\r\n<th>4M x 4M</th>\r\n<th>64 MB</th>\r\n<th>33 MB</th>\r\n<th>4.752ms</th>\r\n<th>3.935ms</th>\r\n</tr>\r\n</tbody>\r\n</table>\r\nSparse arrays will now let users store more data on the GPU while allowing faster computation, thus allowing to solve for bigger, more complex problems.\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Download[/custom_headline]\r\n\r\nArrayFire v3.4 can be downloaded from these locations:\r\n<ul>\r\n 	<li><a href=\"http://sandbox.arrayfire.com/download/\">Binaries</a></li>\r\n 	<li><a href=\"https://github.com/arrayfire\">Source code</a></li>\r\n</ul>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Community[/custom_headline]\r\n\r\nArrayFire is continually improving through the addition of new sparse matrix enhancements. We welcome your feedback:\r\n<ul>\r\n 	<li>General discussion forums on the <a href=\"https://groups.google.com/forum/#!forum/arrayfire-users\">ArrayFire Google Group</a></li>\r\n 	<li>Live discussion chat on the <a href=\"https://gitter.im/arrayfire/arrayfire?utm_source=badge&amp;utm_medium=badge&amp;utm_campaign=pr-badge&amp;utm_content=badge\">ArrayFire Gitter</a></li>\r\n 	<li>Issue reports on the <a href=\"https://github.com/arrayfire/arrayfire/issues\">ArrayFire GitHub</a></li>\r\n</ul>\r\nFinally, as you find success with ArrayFire, we invite you to contribute a post to this blog to share with the broader community. Email <a href=\"mailto:scott@arrayfire.com\">scott@arrayfire.com</a> to contribute to this blog.','Sparse Matrices in ArrayFire v3.4','','inherit','closed','closed','','9190-autosave-v1','','','2016-09-21 22:41:22','2016-09-22 02:41:22','',9190,'http://sandbox.arrayfire.com/9190-autosave-v1/',0,'revision','',0),(9384,11,'2016-09-21 22:41:40','2016-09-22 02:41:40','In ArrayFire v3.4, we have added support for sparse matrices, which greatly reduce the memory footprint on GPUs and accelerated devices for many applications.\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Sparse Data Structure[/custom_headline]\r\n\r\nA sparse data structure is one where all the non-zero elements are not stored. Sparse matrices are useful when the number of zero-values elements are much greater than the number of non-zero elements (i.e. the sparsity of the matrix is high). A sparse data structure is generally stored as 3 arrays:\r\n<ul>\r\n 	<li>A data or values array containing all the non-zero elements</li>\r\n 	<li>A vector for row indices (based on storage format)</li>\r\n 	<li>A vector for column indices (based on storage format)</li>\r\n</ul>\r\nThere are many ways to store sparse matrices, the most prominent of which are:\r\n<ul>\r\n 	<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_row_.28CSR.2C_CRS_or_Yale_format.29\">Compressed Sparse Row</a> (CSR)</li>\r\n 	<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_column_.28CSC_or_CCS.29\">Compressed Storage Column</a> (CSC)</li>\r\n 	<li><a href=\"https://en.wikipedia.org/wiki/Sparse_matrix#Coordinate_list_.28COO.29\">Coordinate Format</a> (COO)</li>\r\n</ul>\r\nArrayFire supports CSR and COO format, with support for CSC coming in future releases.\r\n\r\nA more detailed description of sparse matrices can be found on <a href=\"https://en.wikipedia.org/wiki/Sparse_matrix\">Wikipedia</a>.\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Sparse Arrays in ArrayFire[/custom_headline]\r\n\r\nIn ArrayFire, sparse arrays use the same user-facing object, af::array. This makes working with sparse arrays very seamless. We have added a function <a href=\"http://arrayfire.org/docs/group__method__mat.htm#ga5945f95549bae25f75694a69d98ca584\">af::array::issparse()</a> that allows users to check if an array is sparse or not. Although externally, there is no difference between a sparse and dense array, internally, the sparse array is stored as 3 independent arrays along with other metadata. These arrays, as well as other properties, can be fetched using the <a href=\"http://arrayfire.org/docs/group__sparse__func.htm\">sparse functions</a>.\r\n\r\nIn v3.4, the following features are supported for sparse arrays:\r\n<ul>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm\">Creation for sparse arrays.</a>\r\n<ul>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae4621e58bfaf6d80668309824fde86c0\">From 3 (data, row, column) sparse host or device memory vectors.</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gae9a843c7e93b05e6b090ca32b271ebe9\">From 3 (data, row, column) sparse af::arrays.</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">From dense af::array.</a></li>\r\n 	<li>To create a sparse array from a host or device dense array, first create a dense af::array, and then pass it to create sparse af::array.</li>\r\n</ul>\r\n</li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__sparse__func__convert__to.htm\">Storage format conversions.</a>\r\n<ul>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__sparse__func__create.htm#gac0d367d8c9daf27b36a6981bbfcdb970\">Dense to CSR/COO.</a></li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__sparse__func__dense.htm\">CSR/COO to dense.</a></li>\r\n</ul>\r\n</li>\r\n 	<li><a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">Sparse-Dense Matrix Multiplication (SPMM/SPMV)</a>\r\n<ul>\r\n 	<li>Sparse Matrix - Dense Matrix</li>\r\n 	<li>Sparse Matrix - Dense Vector</li>\r\n 	<li>Options for Sparse Matrix to be transposed</li>\r\n</ul>\r\n</li>\r\n</ul>\r\nThe Sparse-Dense Matrix Multiplication is implemented within the same <a href=\"http://arrayfire.org/docs/group__blas__func__matmul.htm#ga63306b6ed967bd1055086db862fe885b\">af::matmul()</a> function used for dense arrays.\r\n\r\nAs with all ArrayFire features, sparse support is available across all backends (CUDA, OpenCL, and CPU) and supported devices and architectures.\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Example[/custom_headline]\r\n\r\nIn this section, we showcase the difference in running a conjugate gradient solver with a dense array and a sparse array. The inputs are the same for each example. In the sparse case, the \"A\" matrix is converted to sparse.\r\n<pre class=\"\">\r\n#include <arrayfire.h>\r\n#include <iostream>\r\n\r\nusing namespace af;\r\n\r\nstatic size_t dimension = 16 * 1024;\r\nstatic const int maxIter = 10;\r\nstatic const int sparsityFactor = 7;\r\n\r\nstatic array A;\r\nstatic array spA;  // Sparse A\r\nstatic array x0;\r\nstatic array b;\r\n\r\nvoid setupInputs()\r\n{\r\n    // Generate a random input: A\r\n    array T = randu(dimension, dimension, f32);\r\n    // Create 0s in input.\r\n    // Anything that is no divisible by sparsityFactor will become 0.\r\n    A = floor(T * 1000);\r\n    A = A * ((A % sparsityFactor) == 0) / 1000;\r\n    // Make it positive definite\r\n    A = transpose(A) + A + A.dims(0)*identity(A.dims(0), A.dims(0), f32);\r\n\r\n    // Make A sparse as spA\r\n    spA = sparse(A);\r\n\r\n    // Generate x0: Random guess\r\n    x0 = randu(A.dims(0), f32);\r\n\r\n    //Generate b\r\n    b = matmul(A, x0);\r\n\r\n    std::cout << \"Sparsity of A = \"\r\n              << 100.f * (float)sparseGetNNZ(spA) / (float)spA.elements()\r\n              << std::endl;\r\n    std::cout << \"Memory Usage of A = \" <<  A.bytes() << \" bytes\" << std::endl;\r\n    std::cout << \"Memory Usage of spA = \"\r\n              << sparseGetValues(spA).bytes()\r\n               + sparseGetRowIdx(spA).bytes()\r\n               + sparseGetColIdx(spA).bytes()\r\n              << \" bytes\" << std::endl;\r\n}\r\n\r\nvoid conjugateGradient(const af::array in)\r\n{\r\n    array x = constant(0, b.dims(), f32);\r\n    array r = b - matmul(in, x);\r\n    array p = r;\r\n\r\n    for (int i = 0; i < maxIter; ++i) {\r\n        array Ap = matmul(in, p);\r\n        array alpha_num = dot(r, r);\r\n        array alpha_den = dot(p, Ap);\r\n        array alpha = alpha_num/alpha_den;\r\n        r = r - tile(alpha, Ap.dims())*Ap;\r\n        x = x + tile(alpha, Ap.dims())*p;\r\n        array beta_num = dot(r, r);\r\n        array beta = beta_num/alpha_num;\r\n        p = r + tile(beta, p.dims())*p;\r\n    }\r\n    array res = x0 - x;\r\n\r\n    std::cout << \"Final difference in solutions:\\n\";\r\n    af_print(dot(res, res));\r\n}\r\n\r\nint main(int argc, char *argv[])\r\n{\r\n    af::info();\r\n    setupInputs();\r\n\r\n    std::cout << \"Verifying Dense Conjugate Gradient:\" << std::endl;\r\n    conjugateGradient(A);\r\n\r\n    std::cout << \"Verifying Sparse Conjugate Gradient:\" << std::endl;\r\n    conjugateGradient(spA);\r\n\r\n    return 0;\r\n}\r\n</pre>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Performance and Memory Footprint[/custom_headline]\r\n\r\nUsing the CUDA backend on a Tesla K40c, the conjugate gradient function took 64.94ms for the dense matrix and 39.31ms for the sparse Matrix. That is almost a 2x speedup!\r\n\r\nWith a <code>sparsityFactor</code> of 7, the sparsity of A is about 26.38%. This means a tremendous amount of memory is conserved. For a floating point matrix of size 16M x 16M, a dense matrix uses 1 GB of memory while the sparse array, stored in CSR format, only uses 527 MB. That is a 2x savings on memory usage!\r\n<table>\r\n<tbody>\r\n<tr>\r\n<th>Matrix Elements</th>\r\n<th>Memory Size (Dense)</th>\r\n<th>Memory Size (Sparse)</th>\r\n<th>CG Time (Dense)</th>\r\n<th>CG Time (Sparse)</th>\r\n</tr>\r\n<tr>\r\n<th>16M x 16M</th>\r\n<th>1024 MB</th>\r\n<th>527 MB</th>\r\n<th>64.94ms</th>\r\n<th>39.31ms</th>\r\n</tr>\r\n<tr>\r\n<th>4M x 4M</th>\r\n<th>64 MB</th>\r\n<th>33 MB</th>\r\n<th>4.752ms</th>\r\n<th>3.935ms</th>\r\n</tr>\r\n</tbody>\r\n</table>\r\nSparse arrays will now let users store more data on the GPU while allowing faster computation, thus allowing to solve for bigger, more complex problems.\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Download[/custom_headline]\r\n\r\nArrayFire v3.4 can be downloaded from these locations:\r\n<ul>\r\n 	<li><a href=\"http://sandbox.arrayfire.com/download/\">Binaries</a></li>\r\n 	<li><a href=\"https://github.com/arrayfire\">Source code</a></li>\r\n</ul>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Community[/custom_headline]\r\n\r\nArrayFire is continually improving through the addition of new sparse matrix enhancements. We welcome your feedback:\r\n<ul>\r\n 	<li>General discussion forums on the <a href=\"https://groups.google.com/forum/#!forum/arrayfire-users\">ArrayFire Google Group</a></li>\r\n 	<li>Live discussion chat on the <a href=\"https://gitter.im/arrayfire/arrayfire?utm_source=badge&amp;utm_medium=badge&amp;utm_campaign=pr-badge&amp;utm_content=badge\">ArrayFire Gitter</a></li>\r\n 	<li>Issue reports on the <a href=\"https://github.com/arrayfire/arrayfire/issues\">ArrayFire GitHub</a></li>\r\n</ul>\r\nFinally, as you find success with ArrayFire, we invite you to contribute a post to this blog to share with the broader community. Email <a href=\"mailto:scott@arrayfire.com\">scott@arrayfire.com</a> to contribute to this blog.','Sparse Matrices in ArrayFire v3.4','','inherit','closed','closed','','9190-revision-v1','','','2016-09-21 22:41:40','2016-09-22 02:41:40','',9190,'http://sandbox.arrayfire.com/9190-revision-v1/',0,'revision','',0),(9385,11,'2016-09-21 22:44:02','2016-09-22 02:44:02','This post outlines the new graphics features available in ArrayFire v3.4:  Vector Fields, Overlays\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Vector Fields[/custom_headline]\r\n\r\nWe have added visualization support to render ArrayFire <a href=\"http://arrayfire.org/docs/array_8h_source.htm\" target=\"_blank\">array</a> objects as vector fields. An <a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/field.cpp\">example</a> of how to visualize vector fields is included in ArrayFire v3.4. A screenshot of this example\'s output in multi-view mode is shown below, showcasing both static and dynamic vector field rendering.\r\n\r\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/vfield.png\"><img class=\"alignnone size-large wp-image-9185\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/vfield-1024x550.png\" alt=\"vfield\" width=\"742\" height=\"399\" /></a>\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Overlays[/custom_headline]\r\n\r\nPreviously, each graph (such as plot, hist, scatter, etc) was rendered in its own window (or view). Overlaying graphs was not supported. ArrayFire v3.4 now support graph overlays.\r\n\r\nEach draw call in ArrayFire is either rendered to a whole window (single view) or to a view, which is a portion of the screen obtained in multiview mode.\r\n\r\nThe following image is an example of a graph overlay:\r\n\r\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/multipleplot.png\"><img class=\"alignnone size-full wp-image-9182\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/multipleplot.png\" alt=\"multipleplot\" width=\"1002\" height=\"828\" /></a>\r\n\r\nThe following graphs support overlay if they are of same type of object (2d/3d):\r\n<table width=\"100%\">\r\n<tbody>\r\n<tr>\r\n<td><a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/plot3.cpp\" target=\"_blank\">plot</a></td>\r\n<td><a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/plot3.png\"><img class=\"alignnone size-medium wp-image-9211\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/plot3-300x300.png\" alt=\"plot3\" width=\"300\" height=\"300\" /></a></td>\r\n</tr>\r\n<tr>\r\n<td><a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/histogram.cpp\" target=\"_blank\">histogram</a></td>\r\n<td><a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/histogram.png\"><img class=\"alignnone size-medium wp-image-9207\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/histogram-300x152.png\" alt=\"histogram\" width=\"300\" height=\"152\" /></a></td>\r\n</tr>\r\n<tr>\r\n<td><a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/surface.cpp\" target=\"_blank\">surface</a></td>\r\n<td><a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/surface.png\"><img class=\"alignnone size-medium wp-image-9212\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/surface-300x300.png\" alt=\"surface\" width=\"300\" height=\"300\" /></a></td>\r\n</tr>\r\n<tr>\r\n<td>vector fields</td>\r\n<td></td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Animations[/custom_headline]\r\n\r\nArrayFire v3.4 includes the following examples of producing fun animations based on simple equations: <a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/conway.cpp\" target=\"_blank\">conway</a>, <a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/conway_pretty.cpp\" target=\"_blank\">conway_pretty</a>, <a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/gravity_sim.cpp\" target=\"_blank\">gravity</a>, and <a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/fractal.cpp\" target=\"_blank\">julia fractal</a>.\r\n\r\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/conway.gif\"><img class=\"alignnone size-full wp-image-9227\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/conway.gif\" alt=\"conway\" width=\"256\" height=\"256\" /></a> <a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/conway_pretty.gif\"><img class=\"alignnone size-full wp-image-9228\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/conway_pretty.gif\" alt=\"conway_pretty\" width=\"256\" height=\"256\" /></a> <a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/gravity.gif\"><img class=\"alignnone size-full wp-image-9216\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/gravity.gif\" alt=\"gravity\" width=\"256\" height=\"256\" /></a> <a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/julia.gif\"><img class=\"alignnone size-full wp-image-9240\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/julia.gif\" alt=\"julia\" width=\"256\" height=\"256\" /></a>\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Behavioral Changes[/custom_headline]\r\n\r\nIn previous versions of ArrayFire, axes limits were computed automatically to fit the data. In ArrayFire v3.4, users are required to set axes limits, as shown below:\r\n<pre><code class=\"language-cpp\">\r\naf::Window myWindow(800, 800, \"2D Plot example: ArrayFire\");\r\n\r\narray Y;\r\nint sign = 1;\r\narray X = seq(-af::Pi, af::Pi, PRECISION);\r\narray noise = randn(X.dims(0))/5.f;\r\n\r\nmyWindow.grid(2, 1);\r\n// Set the axes limits to view (0,0), prior to v3.4 this was not needed\r\nmyWindow(0, 0).setAxesLimits(-2 * af::Pi, 2 * af::Pi, -1, 1, true);\r\n// Set the axes limits to view (0,0), prior to v3.4 this was not needed\r\nmyWindow(1, 0).setAxesLimits(-2 * af::Pi, 2 * af::Pi, -1.25, 1.25, true);\r\n\r\nfor (double val=0; !myWindow.close(); )\r\n{\r\n    Y = sin(X);\r\n\r\n    myWindow(0,0).plot(X, Y);\r\n    myWindow(1,0).scatter(X, Y + noise, AF_MARKER_POINT);\r\n\r\n    myWindow.show();\r\n\r\n    X = X + PRECISION * float(sign);\r\n    val += PRECISION * float(sign);\r\n\r\n    if (val >af::Pi) {\r\n        sign = -1;\r\n    } else if (val < -af::Pi) {\r\n        sign = 1;\r\n    }\r\n}\r\n</code></pre>\r\nThe code above produces the output below. You can find the complete example <a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/plot2d.cpp\" target=\"_blank\">here</a>.\r\n\r\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/plot2d.png\"><img class=\"alignnone size-large wp-image-9183\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/plot2d-1024x927.png\" alt=\"plot2d\" width=\"742\" height=\"672\" /></a>\r\n\r\n<strong>Miscellaneous Change</strong>\r\n\r\nRemoved <a href=\"http://glew.sourceforge.net/\">GLEW</a> and replaced with <a href=\"https://github.com/cginternals/glbinding\">glbinding</a>.\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Download[/custom_headline]\r\n\r\nArrayFire v3.4 can be downloaded from these locations:\r\n<ul>\r\n 	<li><a href=\"http://sandbox.arrayfire.com/download/\">Binaries</a></li>\r\n 	<li><a href=\"https://github.com/arrayfire\">Source code</a></li>\r\n</ul>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Community[/custom_headline]\r\n\r\nArrayFire is continually improving through the addition of new visualization functions. We welcome your feedback:\r\n<ul>\r\n 	<li>General discussion forums on the <a href=\"https://groups.google.com/forum/#!forum/arrayfire-users\">ArrayFire Google Group</a></li>\r\n 	<li>Live discussion chat on the <a href=\"https://gitter.im/arrayfire/arrayfire?utm_source=badge&amp;utm_medium=badge&amp;utm_campaign=pr-badge&amp;utm_content=badge\">ArrayFire Gitter</a></li>\r\n 	<li>Issue reports on the <a href=\"https://github.com/arrayfire/arrayfire/issues\">ArrayFire GitHub</a></li>\r\n</ul>\r\nFinally, as you find success with ArrayFire, we invite you to contribute a post to this blog to share with the broader community. Email <a href=\"mailto:scott@arrayfire.com\">scott@arrayfire.com</a> to contribute to this blog.','Graphics Updates in ArrayFire v3.4','','inherit','closed','closed','','9172-revision-v1','','','2016-09-21 22:44:02','2016-09-22 02:44:02','',9172,'http://sandbox.arrayfire.com/9172-revision-v1/',0,'revision','',0),(9386,17729,'2016-09-22 00:20:38','2016-09-22 04:20:38','This post outlines the new graphics features available in ArrayFire v3.4:  Vector Fields, Overlays\n\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Vector Fields[/custom_headline]\n\nWe have added visualization support to render ArrayFire <a href=\"http://arrayfire.org/docs/array_8h_source.htm\" target=\"_blank\">array</a> objects as vector fields. An <a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/field.cpp\">example</a> of how to visualize vector fields is included in ArrayFire v3.4. A screenshot of this example\'s output in multi-view mode is shown below, showcasing both static and dynamic vector field rendering.\n\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/vfield.png\"><img class=\"alignnone size-large wp-image-9185\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/vfield-1024x550.png\" alt=\"vfield\" width=\"742\" height=\"399\" /></a>\n\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Overlays[/custom_headline]\n\nPreviously, each graph (such as plot, hist, scatter, etc) was rendered in its own window (or view). Overlaying graphs was not supported. ArrayFire v3.4 now support graph overlays.\n\nEach draw call in ArrayFire is either rendered to a whole window (single view) or to a view, which is a portion of the screen obtained in multiview mode.\n\nThe following image is an example of a graph overlay:\n\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/multipleplot.png\"><img class=\"alignnone size-full wp-image-9182\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/multipleplot.png\" alt=\"multipleplot\" width=\"1002\" height=\"828\" /></a>\n\nThe following graphs support overlay if they are of same type of object (2d/3d):\n<table width=\"100%\">\n<tbody>\n<tr>\n<td><a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/plot3.cpp\" target=\"_blank\">plot</a></td>\n<td><a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/plot3.png\"><img class=\"alignnone size-medium wp-image-9211\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/plot3-300x300.png\" alt=\"plot3\" width=\"300\" height=\"300\" /></a></td>\n</tr>\n<tr>\n<td><a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/histogram.cpp\" target=\"_blank\">histogram</a></td>\n<td><a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/histogram.png\"><img class=\"alignnone size-medium wp-image-9207\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/histogram-300x152.png\" alt=\"histogram\" width=\"300\" height=\"152\" /></a></td>\n</tr>\n<tr>\n<td><a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/surface.cpp\" target=\"_blank\">surface</a></td>\n<td><a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/surface.png\"><img class=\"alignnone size-medium wp-image-9212\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/surface-300x300.png\" alt=\"surface\" width=\"300\" height=\"300\" /></a></td>\n</tr>\n<tr>\n<td><a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/field.cpp\" target=\"_blank\">vector fields</a></td>\n<td><a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/vfield.png\"><img src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/vfield-300x161.png\" alt=\"vfield\" width=\"300\" height=\"161\" class=\"alignnone size-medium wp-image-9185\" /></a></td>\n</tr>\n</tbody>\n</table>\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Animations[/custom_headline]\n\nArrayFire v3.4 includes the following examples of producing fun animations based on simple equations: <a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/conway.cpp\" target=\"_blank\">conway</a>, <a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/conway_pretty.cpp\" target=\"_blank\">conway_pretty</a>, <a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/gravity_sim.cpp\" target=\"_blank\">gravity</a>, and <a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/fractal.cpp\" target=\"_blank\">julia fractal</a>.\n\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/conway.gif\"><img class=\"alignnone size-full wp-image-9227\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/conway.gif\" alt=\"conway\" width=\"256\" height=\"256\" /></a> <a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/conway_pretty.gif\"><img class=\"alignnone size-full wp-image-9228\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/conway_pretty.gif\" alt=\"conway_pretty\" width=\"256\" height=\"256\" /></a> <a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/gravity.gif\"><img class=\"alignnone size-full wp-image-9216\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/gravity.gif\" alt=\"gravity\" width=\"256\" height=\"256\" /></a> <a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/julia.gif\"><img class=\"alignnone size-full wp-image-9240\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/julia.gif\" alt=\"julia\" width=\"256\" height=\"256\" /></a>\n\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Behavioral Changes[/custom_headline]\n\nIn previous versions of ArrayFire, axes limits were computed automatically to fit the data. In ArrayFire v3.4, users are required to set axes limits, as shown below:\n<pre><code class=\"language-cpp\">\naf::Window myWindow(800, 800, \"2D Plot example: ArrayFire\");\n\narray Y;\nint sign = 1;\narray X = seq(-af::Pi, af::Pi, PRECISION);\narray noise = randn(X.dims(0))/5.f;\n\nmyWindow.grid(2, 1);\n// Set the axes limits to view (0,0), prior to v3.4 this was not needed\nmyWindow(0, 0).setAxesLimits(-2 * af::Pi, 2 * af::Pi, -1, 1, true);\n// Set the axes limits to view (0,0), prior to v3.4 this was not needed\nmyWindow(1, 0).setAxesLimits(-2 * af::Pi, 2 * af::Pi, -1.25, 1.25, true);\n\nfor (double val=0; !myWindow.close(); )\n{\n    Y = sin(X);\n\n    myWindow(0,0).plot(X, Y);\n    myWindow(1,0).scatter(X, Y + noise, AF_MARKER_POINT);\n\n    myWindow.show();\n\n    X = X + PRECISION * float(sign);\n    val += PRECISION * float(sign);\n\n    if (val >af::Pi) {\n        sign = -1;\n    } else if (val < -af::Pi) {\n        sign = 1;\n    }\n}\n</code></pre>\nThe code above produces the output below. You can find the complete example <a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/plot2d.cpp\" target=\"_blank\">here</a>.\n\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/plot2d.png\"><img class=\"alignnone size-large wp-image-9183\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/plot2d-1024x927.png\" alt=\"plot2d\" width=\"742\" height=\"672\" /></a>\n\n<strong>Miscellaneous Change</strong>\n\nRemoved <a href=\"http://glew.sourceforge.net/\">GLEW</a> and replaced with <a href=\"https://github.com/cginternals/glbinding\">glbinding</a>.\n\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Download[/custom_headline]\n\nArrayFire v3.4 can be downloaded from these locations:\n<ul>\n 	<li><a href=\"http://sandbox.arrayfire.com/download/\">Binaries</a></li>\n 	<li><a href=\"https://github.com/arrayfire\">Source code</a></li>\n</ul>\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Community[/custom_headline]\n\nArrayFire is continually improving through the addition of new visualization functions. We welcome your feedback:\n<ul>\n 	<li>General discussion forums on the <a href=\"https://groups.google.com/forum/#!forum/arrayfire-users\">ArrayFire Google Group</a></li>\n 	<li>Live discussion chat on the <a href=\"https://gitter.im/arrayfire/arrayfire?utm_source=badge&amp;utm_medium=badge&amp;utm_campaign=pr-badge&amp;utm_content=badge\">ArrayFire Gitter</a></li>\n 	<li>Issue reports on the <a href=\"https://github.com/arrayfire/arrayfire/issues\">ArrayFire GitHub</a></li>\n</ul>\nFinally, as you find success with ArrayFire, we invite you to contribute a post to this blog to share with the broader community. Email <a href=\"mailto:scott@arrayfire.com\">scott@arrayfire.com</a> to contribute to this blog.','Graphics Updates in ArrayFire v3.4','','inherit','closed','closed','','9172-autosave-v1','','','2016-09-22 00:20:38','2016-09-22 04:20:38','',9172,'http://sandbox.arrayfire.com/9172-autosave-v1/',0,'revision','',0),(9387,17729,'2016-09-22 00:22:00','2016-09-22 04:22:00','This post outlines the new graphics features available in ArrayFire v3.4:  Vector Fields, Overlays\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Vector Fields[/custom_headline]\r\n\r\nWe have added visualization support to render ArrayFire <a href=\"http://arrayfire.org/docs/array_8h_source.htm\" target=\"_blank\">array</a> objects as vector fields. An <a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/field.cpp\">example</a> of how to visualize vector fields is included in ArrayFire v3.4. A screenshot of this example\'s output in multi-view mode is shown below, showcasing both static and dynamic vector field rendering.\r\n\r\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/vfield.png\"><img class=\"alignnone size-large wp-image-9185\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/vfield-1024x550.png\" alt=\"vfield\" width=\"742\" height=\"399\" /></a>\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Overlays[/custom_headline]\r\n\r\nPreviously, each graph (such as plot, hist, scatter, etc) was rendered in its own window (or view). Overlaying graphs was not supported. ArrayFire v3.4 now support graph overlays.\r\n\r\nEach draw call in ArrayFire is either rendered to a whole window (single view) or to a view, which is a portion of the screen obtained in multiview mode.\r\n\r\nThe following image is an example of a graph overlay:\r\n\r\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/multipleplot.png\"><img class=\"alignnone size-full wp-image-9182\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/multipleplot.png\" alt=\"multipleplot\" width=\"1002\" height=\"828\" /></a>\r\n\r\nThe following graphs support overlay if they are of same type of object (2d/3d):\r\n<table width=\"100%\">\r\n<tbody>\r\n<tr>\r\n<td><a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/plot3.cpp\" target=\"_blank\">plot</a></td>\r\n<td><a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/plot3.png\"><img class=\"alignnone size-medium wp-image-9211\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/plot3-300x300.png\" alt=\"plot3\" width=\"300\" height=\"300\" /></a></td>\r\n</tr>\r\n<tr>\r\n<td><a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/histogram.cpp\" target=\"_blank\">histogram</a></td>\r\n<td><a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/histogram.png\"><img class=\"alignnone size-medium wp-image-9207\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/histogram-300x152.png\" alt=\"histogram\" width=\"300\" height=\"152\" /></a></td>\r\n</tr>\r\n<tr>\r\n<td><a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/surface.cpp\" target=\"_blank\">surface</a></td>\r\n<td><a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/surface.png\"><img class=\"alignnone size-medium wp-image-9212\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/surface-300x300.png\" alt=\"surface\" width=\"300\" height=\"300\" /></a></td>\r\n</tr>\r\n<tr>\r\n<td><a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/field.cpp\" target=\"_blank\">vector fields</a></td>\r\n<td><a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/vfield.png\"><img src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/vfield-300x161.png\" alt=\"vfield\" width=\"300\" height=\"161\" class=\"alignnone size-medium wp-image-9185\" /></a></td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Animations[/custom_headline]\r\n\r\nArrayFire v3.4 includes the following examples of producing fun animations based on simple equations: <a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/conway.cpp\" target=\"_blank\">conway</a>, <a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/conway_pretty.cpp\" target=\"_blank\">conway_pretty</a>, <a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/gravity_sim.cpp\" target=\"_blank\">gravity</a>, and <a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/fractal.cpp\" target=\"_blank\">julia fractal</a>.\r\n\r\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/conway.gif\"><img class=\"alignnone size-full wp-image-9227\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/conway.gif\" alt=\"conway\" width=\"256\" height=\"256\" /></a> <a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/conway_pretty.gif\"><img class=\"alignnone size-full wp-image-9228\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/conway_pretty.gif\" alt=\"conway_pretty\" width=\"256\" height=\"256\" /></a> <a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/gravity.gif\"><img class=\"alignnone size-full wp-image-9216\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/gravity.gif\" alt=\"gravity\" width=\"256\" height=\"256\" /></a> <a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/julia.gif\"><img class=\"alignnone size-full wp-image-9240\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/julia.gif\" alt=\"julia\" width=\"256\" height=\"256\" /></a>\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Behavioral Changes[/custom_headline]\r\n\r\nIn previous versions of ArrayFire, axes limits were computed automatically to fit the data. In ArrayFire v3.4, users are required to set axes limits, as shown below:\r\n<pre><code class=\"language-cpp\">\r\naf::Window myWindow(800, 800, \"2D Plot example: ArrayFire\");\r\n\r\narray Y;\r\nint sign = 1;\r\narray X = seq(-af::Pi, af::Pi, PRECISION);\r\narray noise = randn(X.dims(0))/5.f;\r\n\r\nmyWindow.grid(2, 1);\r\n// Set the axes limits to view (0,0), prior to v3.4 this was not needed\r\nmyWindow(0, 0).setAxesLimits(-2 * af::Pi, 2 * af::Pi, -1, 1, true);\r\n// Set the axes limits to view (0,0), prior to v3.4 this was not needed\r\nmyWindow(1, 0).setAxesLimits(-2 * af::Pi, 2 * af::Pi, -1.25, 1.25, true);\r\n\r\nfor (double val=0; !myWindow.close(); )\r\n{\r\n    Y = sin(X);\r\n\r\n    myWindow(0,0).plot(X, Y);\r\n    myWindow(1,0).scatter(X, Y + noise, AF_MARKER_POINT);\r\n\r\n    myWindow.show();\r\n\r\n    X = X + PRECISION * float(sign);\r\n    val += PRECISION * float(sign);\r\n\r\n    if (val >af::Pi) {\r\n        sign = -1;\r\n    } else if (val < -af::Pi) {\r\n        sign = 1;\r\n    }\r\n}\r\n</code></pre>\r\nThe code above produces the output below. You can find the complete example <a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/graphics/plot2d.cpp\" target=\"_blank\">here</a>.\r\n\r\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/plot2d.png\"><img class=\"alignnone size-large wp-image-9183\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/09/plot2d-1024x927.png\" alt=\"plot2d\" width=\"742\" height=\"672\" /></a>\r\n\r\n<strong>Miscellaneous Change</strong>\r\n\r\nRemoved <a href=\"http://glew.sourceforge.net/\">GLEW</a> and replaced with <a href=\"https://github.com/cginternals/glbinding\">glbinding</a>.\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Download[/custom_headline]\r\n\r\nArrayFire v3.4 can be downloaded from these locations:\r\n<ul>\r\n 	<li><a href=\"http://sandbox.arrayfire.com/download/\">Binaries</a></li>\r\n 	<li><a href=\"https://github.com/arrayfire\">Source code</a></li>\r\n</ul>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Community[/custom_headline]\r\n\r\nArrayFire is continually improving through the addition of new visualization functions. We welcome your feedback:\r\n<ul>\r\n 	<li>General discussion forums on the <a href=\"https://groups.google.com/forum/#!forum/arrayfire-users\">ArrayFire Google Group</a></li>\r\n 	<li>Live discussion chat on the <a href=\"https://gitter.im/arrayfire/arrayfire?utm_source=badge&amp;utm_medium=badge&amp;utm_campaign=pr-badge&amp;utm_content=badge\">ArrayFire Gitter</a></li>\r\n 	<li>Issue reports on the <a href=\"https://github.com/arrayfire/arrayfire/issues\">ArrayFire GitHub</a></li>\r\n</ul>\r\nFinally, as you find success with ArrayFire, we invite you to contribute a post to this blog to share with the broader community. Email <a href=\"mailto:scott@arrayfire.com\">scott@arrayfire.com</a> to contribute to this blog.','Graphics Updates in ArrayFire v3.4','','inherit','closed','closed','','9172-revision-v1','','','2016-09-22 00:22:00','2016-09-22 04:22:00','',9172,'http://sandbox.arrayfire.com/9172-revision-v1/',0,'revision','',0),(9390,17728,'2016-09-22 11:25:06','2016-09-22 15:25:06','ArrayFire uses <a href=\"http://sandbox.arrayfire.com/performance-of-arrayfire-jit-code-generation/\">Just In Time</a> compilation to combine many light weight functions into a single kernel launch. This along with our <a href=\"http://arrayfire.org/docs/group__arrayfire__func.htm\">easy-to-use API</a> allows users to not only quickly prototype their algorithms, but also get the best out of the underlying hardware.\r\n\r\nThis feature has been a favorite among our users in the domains of finance and scientific simulation.\r\n\r\nThat said, ArrayFire v3.3 and earlier had a few limitations. Namely:\r\n<ul class=\"org-ul\">\r\n 	<li>Multiple outputs with inter-dependent variables were generating multiple kernels.</li>\r\n 	<li>The number of operations per kernel was fairly limited by default.</li>\r\n</ul>\r\nIn the latest release of ArrayFire, we addressed these issues to get some pretty impressive numbers.\r\n\r\nIn the rest of the post, we demonstrate the performance improvements using our <a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/financial/black_scholes_options.cpp\">BlackScholes example</a>. For this particular example, the improvements for the CUDA backend were <b>~8x</b> and OpenCL backend were between <b>~5x</b>.\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Performance Comparison between v3.4 and v3.3[/custom_headline]\r\n<div id=\"outline-container-sec-1\" class=\"outline-2\">\r\n<div id=\"text-1\" class=\"outline-text-2\">\r\n\r\n<a href=\"https://en.wikipedia.org/wiki/Black%25E2%2580%2593Scholes_model\">BlackScholes</a> is a commonly used mathematical model in financial markets.\r\n\r\nIt takes the following 5 inputs and generates 2 outputs.\r\n\r\nThe code to implement this model in ArrayFire can be seen below.\r\n<div class=\"org-src-container\">\r\n<pre class=\"src src-C++\">array cnd(array x)\r\n{\r\n    // constants\r\n    const float a1 =  0.254829592;\r\n    const float a2 = -0.284496736;\r\n    const float a3 =  1.421413741;\r\n    const float a4 = -1.453152027;\r\n    const float a5 =  1.061405429;\r\n    const float p  =  0.3275911;\r\n    const float sqrt2 = sqrt(2.0);\r\n\r\n    // Save the sign of x\r\n    array xSign = sign(x);\r\n\r\n    x = abs(x) / sqrt2;\r\n\r\n    // A & S formula 7.1.26\r\n    array t = 1.0f / (1.0f + p*x);\r\n    array y = 1.0f + 0.5f * (((((a5*t + a4)*t) + a3)*t + a2)*t + a1)*t*exp(-x*x);\r\n\r\n    return xSign * y + !xSign * (1 - y); // equivalent of (x >= 0) ? y : (1 - y);\r\n}\r\n\r\nstatic void black_scholes(array& C, array& P,\r\n                          const array& S, const array& X,\r\n                          const array& R, const array& V,\r\n                          const array& T)\r\n{\r\n    // Inputs:\r\n    // S: Stock price\r\n    // X: Strike price\r\n    // R: Risk free rate of interest\r\n    // V: Volatility\r\n    // T: Time to maturity\r\n\r\n    // Outputs:\r\n    // C: Call option\r\n    // P: Put option\r\n\r\n    array d1 = log(S / X);\r\n    d1 = d1 + (R + (V*V)*0.5) * T;\r\n    d1 = d1 / (V*sqrt(T));\r\n\r\n    array d2 = d1 - (V*sqrt(T));\r\n\r\n    array cnd_d1 = cnd(d1);\r\n    array cnd_d2 = cnd(d2);\r\n\r\n    C = S * cnd_d1  - (X * exp((-R)*T) * cnd_d2);\r\n    P = X * exp((-R)*T) * (1 - cnd_d2) - (S * (1 - cnd_d1));\r\n}\r\n</pre>\r\n</div>\r\nThe <code>black_scholes</code> function contains a long chain of operations and generates two outputs. In older versions of ArrayFire, the Just-In-Time compiler of arrayfire was splitting up this single function call to multiple kernels.\r\n\r\nWe solved this in the latest release by:\r\n<ul class=\"org-ul\">\r\n 	<li>Enabling <code>af::eval</code> to hint ArrayFire\'s JIT compiler to combine inputs.</li>\r\n 	<li>Changing the heuristics to increase the number of operations per kernel.</li>\r\n</ul>\r\n<div class=\"org-src-container\">\r\n<pre class=\"src src-C++\">black_scholes(C, P, S, X, R, V, T);\r\neval(C, P); // Only change required for v3.4\r\n</pre>\r\n</div>\r\nThese changes resulted in dramatic performance improvements seen below. The following graph compares the number of call and put options being calculated per second on v3.3 and v3.4.\r\n\r\n&nbsp;\r\n\r\n<iframe width=\"650\" height=\"400\" seamless=\"\" frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1o4C5KZ4B5he3Ac8mJ9Z6Z_N-hlATeJwhP-j6XJtbEPA/pubchart?oid=927257953&amp;format=interactive\">\r\n</iframe>\r\n\r\n&nbsp;\r\n\r\nAs we can see, the performance has improved significantly. The absolute time taken when to calculate 2 million options is show below.\r\n\r\n&nbsp;\r\n\r\n<iframe width=\"513.5\" height=\"104.0141666666666\" seamless=\"\" frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1o4C5KZ4B5he3Ac8mJ9Z6Z_N-hlATeJwhP-j6XJtbEPA/pubchart?oid=1295781261&amp;format=interactive\"></iframe>\r\n\r\nNote:\r\n<ul class=\"org-ul\">\r\n 	<li>The additional gains made by CUDA are because of a <a href=\"https://github.com/arrayfire/arrayfire/pull/1534\">separate issue</a> involving CUDA JIT that was fixed.</li>\r\n 	<li>Only 24 threads of the 32 available were used for the Xeon CPU.</li>\r\n</ul>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Performance Comparison to Native CUDA[/custom_headline]\r\n\r\n</div>\r\n</div>\r\n<div id=\"outline-container-sec-2\" class=\"outline-2\">\r\n<div id=\"text-2\" class=\"outline-text-2\">\r\n\r\nTo see how close we can get to the peak performance, we compared the performance of ArrayFire and CUDA on the new GTX 1080.\r\n\r\nTo do this we built ArrayFire 3.4 using the CUDA 8 RC and compared its performance with that of the example provided in <a href=\"http://docs.nvidia.com/cuda/cuda-samples/#black-scholes-option-pricing\">CUDA Samples</a>.\r\n\r\nThe results of this benchmark can be seen below.\r\n\r\n<iframe width=\"655.1414131325375\" height=\"323.5\" seamless=\"\" frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1o4C5KZ4B5he3Ac8mJ9Z6Z_N-hlATeJwhP-j6XJtbEPA/pubchart?oid=256763192&amp;format=interactive\"></iframe>\r\n\r\nThe results indicate ArrayFire v3.4 performs identical to the native kernel at larger simulation sizes, but there is a constant overhead causing issues at smaller sizes. This is one of the issues we want to address in future releases.\r\n\r\nNote: The <code>black_scholes</code> function had to be changed slightly as shown below to be similar to the sample provided by CUDA.\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Coming Soon[/custom_headline]\r\n\r\n</div>\r\n</div>\r\n<div id=\"outline-container-sec-3\" class=\"outline-2\">\r\n<div id=\"text-3\" class=\"outline-text-2\">\r\n\r\nWhile the new improvements to the JIT engine in ArrayFire can seamlessly generate kernels that perform close to handwritten native kernels on GPUs, there is room for improvement at smaller sizes. To see what lies ahead for ArrayFire JIT, please check out <a href=\"https://github.com/arrayfire/arrayfire/issues/1401\">this github issue</a>.\r\n\r\n</div>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Download[/custom_headline]\r\n\r\nArrayFire v3.4 can be downloaded from these locations:\r\n<ul>\r\n 	<li><a href=\"http://sandbox.arrayfire.com/download/\">Binaries</a></li>\r\n 	<li><a href=\"https://github.com/arrayfire\">Source code</a></li>\r\n</ul>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Community[/custom_headline]\r\n\r\nArrayFire is continually improving through the addition of new JIT enhancements. We welcome your feedback:\r\n<ul>\r\n 	<li>General discussion forums on the <a href=\"https://groups.google.com/forum/#!forum/arrayfire-users\">ArrayFire Google Group</a></li>\r\n 	<li>Live discussion chat on the <a href=\"https://gitter.im/arrayfire/arrayfire?utm_source=badge&amp;utm_medium=badge&amp;utm_campaign=pr-badge&amp;utm_content=badge\">ArrayFire Gitter</a></li>\r\n 	<li>Issue reports on the <a href=\"https://github.com/arrayfire/arrayfire/issues\">ArrayFire GitHub</a></li>\r\n</ul>\r\nFinally, as you find success with ArrayFire, we invite you to contribute a post to this blog to share with the broader community. Email <a href=\"mailto:scott@arrayfire.com\">scott@arrayfire.com</a> to contribute to this blog.\r\n\r\n&nbsp;\r\n\r\n</div>','Performance Improvements to JIT in ArrayFire v3.4','','inherit','closed','closed','','9252-revision-v1','','','2016-09-22 11:25:06','2016-09-22 15:25:06','',9252,'http://sandbox.arrayfire.com/9252-revision-v1/',0,'revision','',0),(9391,17728,'2016-09-22 11:57:06','2016-09-22 15:57:06','ArrayFire uses <a href=\"http://sandbox.arrayfire.com/performance-of-arrayfire-jit-code-generation/\">Just In Time</a> compilation to combine many light weight functions into a single kernel launch. This along with our <a href=\"http://arrayfire.org/docs/group__arrayfire__func.htm\">easy-to-use API</a> allows users to not only quickly prototype their algorithms, but also get the best out of the underlying hardware.\r\n\r\nThis feature has been a favorite among our users in the domains of finance and scientific simulation.\r\n\r\nThat said, ArrayFire v3.3 and earlier had a few limitations. Namely:\r\n<ul class=\"org-ul\">\r\n 	<li>Multiple outputs with inter-dependent variables were generating multiple kernels.</li>\r\n 	<li>The number of operations per kernel was fairly limited by default.</li>\r\n</ul>\r\nIn the latest release of ArrayFire, we addressed these issues to get some pretty impressive numbers.\r\n\r\nIn the rest of the post, we demonstrate the performance improvements using our <a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/financial/black_scholes_options.cpp\">BlackScholes example</a>. For this particular example, the improvements for the CUDA backend were <b>~8x</b> and OpenCL backend were between <b>~5x</b>.\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Performance Comparison between v3.4 and v3.3[/custom_headline]\r\n<div id=\"outline-container-sec-1\" class=\"outline-2\">\r\n<div id=\"text-1\" class=\"outline-text-2\">\r\n\r\n<a href=\"https://en.wikipedia.org/wiki/Black%25E2%2580%2593Scholes_model\">BlackScholes</a> is a commonly used mathematical model in financial markets.\r\n\r\nIt takes the following 5 inputs and generates 2 outputs.\r\n\r\nThe code to implement this model in ArrayFire can be seen below.\r\n<div class=\"org-src-container\">\r\n<pre class=\"src src-C++\">array cnd(array x)\r\n{\r\n    // constants\r\n    const float a1 =  0.254829592;\r\n    const float a2 = -0.284496736;\r\n    const float a3 =  1.421413741;\r\n    const float a4 = -1.453152027;\r\n    const float a5 =  1.061405429;\r\n    const float p  =  0.3275911;\r\n    const float sqrt2 = sqrt(2.0);\r\n\r\n    // Save the sign of x\r\n    array xSign = sign(x);\r\n\r\n    x = abs(x) / sqrt2;\r\n\r\n    // A & S formula 7.1.26\r\n    array t = 1.0f / (1.0f + p*x);\r\n    array y = 1.0f + 0.5f * (((((a5*t + a4)*t) + a3)*t + a2)*t + a1)*t*exp(-x*x);\r\n\r\n    return xSign * y + !xSign * (1 - y); // equivalent of (x >= 0) ? y : (1 - y);\r\n}\r\n\r\nstatic void black_scholes(array& C, array& P,\r\n                          const array& S, const array& X,\r\n                          const array& R, const array& V,\r\n                          const array& T)\r\n{\r\n    // Inputs:\r\n    // S: Stock price\r\n    // X: Strike price\r\n    // R: Risk free rate of interest\r\n    // V: Volatility\r\n    // T: Time to maturity\r\n\r\n    // Outputs:\r\n    // C: Call option\r\n    // P: Put option\r\n\r\n    array d1 = log(S / X);\r\n    d1 = d1 + (R + (V*V)*0.5) * T;\r\n    d1 = d1 / (V*sqrt(T));\r\n\r\n    array d2 = d1 - (V*sqrt(T));\r\n\r\n    array cnd_d1 = cnd(d1);\r\n    array cnd_d2 = cnd(d2);\r\n\r\n    C = S * cnd_d1  - (X * exp((-R)*T) * cnd_d2);\r\n    P = X * exp((-R)*T) * (1 - cnd_d2) - (S * (1 - cnd_d1));\r\n}\r\n</pre>\r\n</div>\r\nThe <code>black_scholes</code> function contains a long chain of operations and generates two outputs. In older versions of ArrayFire, the Just-In-Time compiler of arrayfire was splitting up this single function call to multiple kernels.\r\n\r\nWe solved this in the latest release by:\r\n<ul class=\"org-ul\">\r\n 	<li>Enabling <code>af::eval</code> to hint ArrayFire\'s JIT compiler to combine kernels generating multiple outputs.</li>\r\n 	<li>Changing the heuristics to increase the number of operations per kernel.</li>\r\n</ul>\r\n<div class=\"org-src-container\">\r\n<pre class=\"src src-C++\">black_scholes(C, P, S, X, R, V, T);\r\neval(C, P); // Only change required for v3.4\r\n</pre>\r\n</div>\r\nThese changes resulted in dramatic performance improvements seen below. The following graph compares the number of call and put options being calculated per second on v3.3 and v3.4.\r\n\r\n&nbsp;\r\n\r\n<iframe width=\"650\" height=\"400\" seamless=\"\" frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1o4C5KZ4B5he3Ac8mJ9Z6Z_N-hlATeJwhP-j6XJtbEPA/pubchart?oid=927257953&amp;format=interactive\">\r\n</iframe>\r\n\r\n&nbsp;\r\n\r\nAs we can see, the performance has improved significantly. The absolute time taken when to calculate 2 million options is show below.\r\n\r\n&nbsp;\r\n\r\n<iframe width=\"513.5\" height=\"104.0141666666666\" seamless=\"\" frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1o4C5KZ4B5he3Ac8mJ9Z6Z_N-hlATeJwhP-j6XJtbEPA/pubchart?oid=1295781261&amp;format=interactive\"></iframe>\r\n\r\nNote:\r\n<ul class=\"org-ul\">\r\n 	<li>The additional gains made by CUDA are because of a <a href=\"https://github.com/arrayfire/arrayfire/pull/1534\">separate issue</a> involving CUDA JIT that was fixed.</li>\r\n 	<li>Only 24 threads of the 32 available were used for the Xeon CPU.</li>\r\n</ul>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Performance Comparison to Native CUDA[/custom_headline]\r\n\r\n</div>\r\n</div>\r\n<div id=\"outline-container-sec-2\" class=\"outline-2\">\r\n<div id=\"text-2\" class=\"outline-text-2\">\r\n\r\nTo see how close we can get to the peak performance, we compared the performance of ArrayFire and CUDA on the new GTX 1080.\r\n\r\nTo do this we built ArrayFire 3.4 using the CUDA 8 RC and compared its performance with that of the example provided in <a href=\"http://docs.nvidia.com/cuda/cuda-samples/#black-scholes-option-pricing\">CUDA Samples</a>.\r\n\r\nThe results of this benchmark can be seen below.\r\n\r\n<iframe width=\"655.1414131325375\" height=\"323.5\" seamless=\"\" frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1o4C5KZ4B5he3Ac8mJ9Z6Z_N-hlATeJwhP-j6XJtbEPA/pubchart?oid=256763192&amp;format=interactive\"></iframe>\r\n\r\nThe results indicate ArrayFire v3.4 performs identical to the native kernel at larger simulation sizes, but there is a constant overhead causing issues at smaller sizes. This is one of the issues we want to address in future releases.\r\n\r\nNote: The <code>black_scholes</code> function had to be changed slightly as shown below to be similar to the sample provided by CUDA.\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Coming Soon[/custom_headline]\r\n\r\n</div>\r\n</div>\r\n<div id=\"outline-container-sec-3\" class=\"outline-2\">\r\n<div id=\"text-3\" class=\"outline-text-2\">\r\n\r\nWhile the new improvements to the JIT engine in ArrayFire can seamlessly generate kernels that perform close to handwritten native kernels on GPUs, there is room for improvement at smaller sizes. To see what lies ahead for ArrayFire JIT, please check out <a href=\"https://github.com/arrayfire/arrayfire/issues/1401\">this github issue</a>.\r\n\r\n</div>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Download[/custom_headline]\r\n\r\nArrayFire v3.4 can be downloaded from these locations:\r\n<ul>\r\n 	<li><a href=\"http://sandbox.arrayfire.com/download/\">Binaries</a></li>\r\n 	<li><a href=\"https://github.com/arrayfire\">Source code</a></li>\r\n</ul>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Community[/custom_headline]\r\n\r\nArrayFire is continually improving through the addition of new JIT enhancements. We welcome your feedback:\r\n<ul>\r\n 	<li>General discussion forums on the <a href=\"https://groups.google.com/forum/#!forum/arrayfire-users\">ArrayFire Google Group</a></li>\r\n 	<li>Live discussion chat on the <a href=\"https://gitter.im/arrayfire/arrayfire?utm_source=badge&amp;utm_medium=badge&amp;utm_campaign=pr-badge&amp;utm_content=badge\">ArrayFire Gitter</a></li>\r\n 	<li>Issue reports on the <a href=\"https://github.com/arrayfire/arrayfire/issues\">ArrayFire GitHub</a></li>\r\n</ul>\r\nFinally, as you find success with ArrayFire, we invite you to contribute a post to this blog to share with the broader community. Email <a href=\"mailto:scott@arrayfire.com\">scott@arrayfire.com</a> to contribute to this blog.\r\n\r\n&nbsp;\r\n\r\n</div>','Performance Improvements to JIT in ArrayFire v3.4','','inherit','closed','closed','','9252-revision-v1','','','2016-09-22 11:57:06','2016-09-22 15:57:06','',9252,'http://sandbox.arrayfire.com/9252-revision-v1/',0,'revision','',0),(9392,17728,'2016-09-22 11:59:23','2016-09-22 15:59:23','ArrayFire uses <a href=\"http://sandbox.arrayfire.com/performance-of-arrayfire-jit-code-generation/\">Just In Time</a> compilation to combine many light weight functions into a single kernel launch. This along with our <a href=\"http://arrayfire.org/docs/group__arrayfire__func.htm\">easy-to-use API</a> allows users to not only quickly prototype their algorithms, but also get the best out of the underlying hardware.\r\n\r\nThis feature has been a favorite among our users in the domains of finance and scientific simulation.\r\n\r\nThat said, ArrayFire v3.3 and earlier had a few limitations. Namely:\r\n<ul class=\"org-ul\">\r\n 	<li>Multiple outputs with inter-dependent variables were generating multiple kernels.</li>\r\n 	<li>The number of operations per kernel was fairly limited by default.</li>\r\n</ul>\r\nIn the latest release of ArrayFire, we addressed these issues to get some pretty impressive numbers.\r\n\r\nIn the rest of the post, we demonstrate the performance improvements using our <a href=\"https://github.com/arrayfire/arrayfire/blob/devel/examples/financial/black_scholes_options.cpp\">BlackScholes example</a>. For this particular example, the improvements for the CUDA backend were <b>~8x</b> and OpenCL backend were between <b>~5x</b>.\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Performance Comparison between v3.4 and v3.3[/custom_headline]\r\n<div id=\"outline-container-sec-1\" class=\"outline-2\">\r\n<div id=\"text-1\" class=\"outline-text-2\">\r\n\r\n<a href=\"https://en.wikipedia.org/wiki/Black%25E2%2580%2593Scholes_model\">BlackScholes</a> is a commonly used mathematical model in financial markets.\r\n\r\nIt takes the following 5 inputs and generates 2 outputs.\r\n\r\nThe code to implement this model in ArrayFire can be seen below.\r\n<div class=\"org-src-container\">\r\n<pre class=\"src src-C++\">array cnd(array x)\r\n{\r\n    // constants\r\n    const float a1 =  0.254829592;\r\n    const float a2 = -0.284496736;\r\n    const float a3 =  1.421413741;\r\n    const float a4 = -1.453152027;\r\n    const float a5 =  1.061405429;\r\n    const float p  =  0.3275911;\r\n    const float sqrt2 = sqrt(2.0);\r\n\r\n    // Save the sign of x\r\n    array xSign = sign(x);\r\n\r\n    x = abs(x) / sqrt2;\r\n\r\n    // A & S formula 7.1.26\r\n    array t = 1.0f / (1.0f + p*x);\r\n    array y = 1.0f + 0.5f * (((((a5*t + a4)*t) + a3)*t + a2)*t + a1)*t*exp(-x*x);\r\n\r\n    return xSign * y + !xSign * (1 - y); // equivalent of (x >= 0) ? y : (1 - y);\r\n}\r\n\r\nstatic void black_scholes(array& C, array& P,\r\n                          const array& S, const array& X,\r\n                          const array& R, const array& V,\r\n                          const array& T)\r\n{\r\n    // Inputs:\r\n    // S: Stock price\r\n    // X: Strike price\r\n    // R: Risk free rate of interest\r\n    // V: Volatility\r\n    // T: Time to maturity\r\n\r\n    // Outputs:\r\n    // C: Call option\r\n    // P: Put option\r\n\r\n    array d1 = log(S / X);\r\n    d1 = d1 + (R + (V*V)*0.5) * T;\r\n    d1 = d1 / (V*sqrt(T));\r\n\r\n    array d2 = d1 - (V*sqrt(T));\r\n\r\n    array cnd_d1 = cnd(d1);\r\n    array cnd_d2 = cnd(d2);\r\n\r\n    C = S * cnd_d1  - (X * exp((-R)*T) * cnd_d2);\r\n    P = X * exp((-R)*T) * (1 - cnd_d2) - (S * (1 - cnd_d1));\r\n}\r\n</pre>\r\n</div>\r\nThe <code>black_scholes</code> function contains a long chain of operations and generates two outputs. In older versions of ArrayFire, the Just-In-Time compiler of arrayfire was splitting up this single function call to multiple kernels.\r\n\r\nWe solved this in the latest release by:\r\n<ul class=\"org-ul\">\r\n 	<li>Enabling <code>af::eval</code> to hint ArrayFire\'s JIT compiler to combine kernels generating multiple outputs.</li>\r\n 	<li>Changing the heuristics to increase the number of operations per kernel.</li>\r\n</ul>\r\n<div class=\"org-src-container\">\r\n<pre class=\"src src-C++\">black_scholes(C, P, S, X, R, V, T);\r\neval(C, P); // Only change required for v3.4\r\n</pre>\r\n</div>\r\nThese changes resulted in dramatic performance improvements seen below. The following graph compares the number of call and put options being calculated per second on v3.3 and v3.4.\r\n\r\n&nbsp;\r\n\r\n<iframe width=\"650\" height=\"400\" seamless=\"\" frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1o4C5KZ4B5he3Ac8mJ9Z6Z_N-hlATeJwhP-j6XJtbEPA/pubchart?oid=927257953&amp;format=interactive\">\r\n</iframe>\r\n\r\n&nbsp;\r\n\r\nAs we can see, the performance has improved significantly. The absolute time taken when to calculate 2 million options is show below.\r\n\r\n&nbsp;\r\n\r\n<iframe width=\"513.5\" height=\"104.0141666666666\" seamless=\"\" frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1o4C5KZ4B5he3Ac8mJ9Z6Z_N-hlATeJwhP-j6XJtbEPA/pubchart?oid=1295781261&amp;format=interactive\"></iframe>\r\n\r\nNote:\r\n<ul class=\"org-ul\">\r\n 	<li>The additional gains made by CUDA are because of a <a href=\"https://github.com/arrayfire/arrayfire/pull/1534\">separate issue</a> involving CUDA JIT that was fixed.</li>\r\n 	<li>Only 24 threads of the 32 available were used for the Xeon CPU.</li>\r\n</ul>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Performance Comparison to Native CUDA[/custom_headline]\r\n\r\n</div>\r\n</div>\r\n<div id=\"outline-container-sec-2\" class=\"outline-2\">\r\n<div id=\"text-2\" class=\"outline-text-2\">\r\n\r\nTo see how close we can get to the peak performance, we compared the performance of ArrayFire and CUDA on the new GTX 1080.\r\n\r\nTo do this we built ArrayFire 3.4 using the CUDA 8 RC and compared its performance with that of the example provided in <a href=\"http://docs.nvidia.com/cuda/cuda-samples/#black-scholes-option-pricing\">CUDA Samples</a>.\r\n\r\nThe results of this benchmark can be seen below.\r\n\r\n<iframe width=\"655.1414131325375\" height=\"323.5\" seamless=\"\" frameborder=\"0\" scrolling=\"no\" src=\"https://docs.google.com/spreadsheets/d/1o4C5KZ4B5he3Ac8mJ9Z6Z_N-hlATeJwhP-j6XJtbEPA/pubchart?oid=256763192&amp;format=interactive\"></iframe>\r\n\r\nThe results indicate ArrayFire v3.4 performs identical to the native kernel at larger simulation sizes, but there is a constant overhead causing issues at smaller sizes. This is one of the issues we want to address in future releases.\r\n\r\nNote: The <code>black_scholes</code> function had to be changed slightly to be similar to the sample provided by CUDA.\r\n\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Coming Soon[/custom_headline]\r\n\r\n</div>\r\n</div>\r\n<div id=\"outline-container-sec-3\" class=\"outline-2\">\r\n<div id=\"text-3\" class=\"outline-text-2\">\r\n\r\nWhile the new improvements to the JIT engine in ArrayFire can seamlessly generate kernels that perform close to handwritten native kernels on GPUs, there is room for improvement at smaller sizes. To see what lies ahead for ArrayFire JIT, please check out <a href=\"https://github.com/arrayfire/arrayfire/issues/1401\">this github issue</a>.\r\n\r\n</div>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Download[/custom_headline]\r\n\r\nArrayFire v3.4 can be downloaded from these locations:\r\n<ul>\r\n 	<li><a href=\"http://sandbox.arrayfire.com/download/\">Binaries</a></li>\r\n 	<li><a href=\"https://github.com/arrayfire\">Source code</a></li>\r\n</ul>\r\n[custom_headline type=\"left\" level=\"h3\" looks_like=\"h3\"]Community[/custom_headline]\r\n\r\nArrayFire is continually improving through the addition of new JIT enhancements. We welcome your feedback:\r\n<ul>\r\n 	<li>General discussion forums on the <a href=\"https://groups.google.com/forum/#!forum/arrayfire-users\">ArrayFire Google Group</a></li>\r\n 	<li>Live discussion chat on the <a href=\"https://gitter.im/arrayfire/arrayfire?utm_source=badge&amp;utm_medium=badge&amp;utm_campaign=pr-badge&amp;utm_content=badge\">ArrayFire Gitter</a></li>\r\n 	<li>Issue reports on the <a href=\"https://github.com/arrayfire/arrayfire/issues\">ArrayFire GitHub</a></li>\r\n</ul>\r\nFinally, as you find success with ArrayFire, we invite you to contribute a post to this blog to share with the broader community. Email <a href=\"mailto:scott@arrayfire.com\">scott@arrayfire.com</a> to contribute to this blog.\r\n\r\n&nbsp;\r\n\r\n</div>','Performance Improvements to JIT in ArrayFire v3.4','','inherit','closed','closed','','9252-revision-v1','','','2016-09-22 11:59:23','2016-09-22 15:59:23','',9252,'http://sandbox.arrayfire.com/9252-revision-v1/',0,'revision','',0),(9394,11,'2016-10-15 15:55:21','2016-10-15 19:55:21','[content_band inner_container=\"true\" no_margin=\"true\" border=\"none\" padding_top=\"10px\" padding_bottom=\"40px\"]\r\n[custom_headline type=\"left\" level=\"h2\"]\r\nArrayFire Installers\r\n[/custom_headline]\r\n[custom_headline type=\"left\" level=\"h5\"]\r\nEach download contains both CUDA and OpenCL files.\r\n[/custom_headline]\r\n[gap size=\"4%\"]\r\n\r\n[column type=\"one-fifth\"][button  block=\"true\" onclick=\"&lt;!-- Google Code for ArrayFire Download (Windows) Conversion Page\r\nIn your html page, add the snippet and call\r\ngoog_report_conversion when someone clicks on the\r\nchosen link or button. --&gt;\r\n&lt;script type=\"text/javascript\"&gt;\r\n/* &lt;!&amp;#91;CDATA&amp;#91; */\r\ngoog_snippet_vars = function() {\r\nvar w = window;\r\nw.google_conversion_id = 1021026651;\r\nw.google_conversion_label = \"18VJCJjPql4Q28Lu5gM\";\r\nw.google_remarketing_only = false;\r\n}\r\n// DO NOT CHANGE THE CODE BELOW.\r\ngoog_report_conversion = function(url) {\r\ngoog_snippet_vars();\r\nwindow.google_conversion_format = \"3\";\r\nwindow.google_is_call = true;\r\nvar opt = new Object();\r\nopt.onload_callback = function() {\r\nif (typeof(url) != \'undefined\') {\r\nwindow.location = url;\r\n}\r\n}\r\nvar conv_handler = window&amp;#91;\'google_trackConversion\'&amp;#93;;\r\nif (typeof(conv_handler) == \'function\') {\r\nconv_handler(opt);\r\n}\r\n}\r\n/* &amp;#93;&amp;#93;&gt; */\r\n&lt;/script&gt;\r\n&lt;script type=\"text/javascript\"\r\nsrc=\"//www.googleadservices.com/pagead/conversion_async.js\"&gt;\r\n&lt;/script&gt;\" class=\"center-text\" type=\"flat\" shape=\"rounded\" href=\"http://go.sandbox.arrayfire.com/windows-installer\" size=\"regular\" ]Windows, v3.4.1\r\n(64-bit)[/button][/column][column type=\"one-fifth\"][button block=\"true\" type=\"flat\"  shape=\"rounded\" href=\"http://go.sandbox.arrayfire.com/linux-x86_64-installer\" size=\"regular\" ]Linux, v3.4.1\r\n(64-bit)[/button][/column][column type=\"one-fifth\"][button block=\"true\" type=\"flat\"  shape=\"rounded\" href=\"http://go.sandbox.arrayfire.com/osx-installer\" size=\"regular\" ]Mac OSX, v3.4.1\r\n(64-bit)[/button][/column][column type=\"one-fifth\"][button block=\"true\" type=\"flat\"  shape=\"rounded\" href=\"http://sandbox.arrayfire.com/installer_archive/3.4.1/ArrayFire-v3.4.1_Linux_aarch64.sh\" size=\"regular\" ]Tegra X1 v3.4.1\r\n(64-bit)[/button][/column][column type=\"one-fifth\" last=\"true\"][button block=\"true\"  type=\"flat\"  shape=\"rounded\" href=\"http://go.sandbox.arrayfire.com/linux-armv7l-installer\" size=\"regular\"]Tegra K1 v3.4.1\r\n(32-bit)[/button][/column][gap size=\"9%\"]\r\n\r\nCheck out our GitHub repository to <a href=\"https://github.com/arrayfire/arrayfire\">build from the source</a>.\r\n\r\nLooking for an older installer? <a href=\"http://sandbox.arrayfire.com/installer_archive\">Try here</a>.[/content_band]\r\n<p style=\"text-align: left;\">[content_band inner_container=\"true\" no_margin=\"true\" border=\"none\" padding_top=\"10px\" padding_bottom=\"40px\" bg_color=\"#f1f1f1\"][column type=\"one-half\"][gap size=\"8%\"]<img class=\"aligncenter wp-image-5156\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/02/mobile_Exported1-e1395437261846.png\" alt=\"mobile_Exported\" width=\"233\" height=\"311\" />[/column][column type=\"one-half\" last=\"true\"]\r\n[custom_headline type=\"left\" level=\"h2\"]\r\nGreat. What now?\r\n[/custom_headline]</p>\r\n<p style=\"text-align: left;\">[gap size=\"5%\"]</p>\r\n<p style=\"text-align: left;\">Welcome to ArrayFire! You\'ve got an awesome library there—learn more about its features and functions on <a href=\"http://www.sandbox.arrayfire.com/docs/index.htm\">our documentation</a>.</p>\r\n<p style=\"text-align: left;\">In addition to our <a href=\"https://groups.google.com/forum/#!forum/arrayfire-users\">ArrayFire user group</a> and <a href=\"http://sandbox.arrayfire.com/blog/\">our blog</a>, we offer a plethora of <a href=\"http://sandbox.arrayfire.com/support/\">support options</a> to fit your needs.</p>\r\n<p style=\"text-align: left;\">Got a job that needs closer attention? Hire us! Our engineers are acceleration wizards. Check out our <a href=\"http://sandbox.arrayfire.com/consulting/\">consulting offerings</a>, or set up a free <a href=\"http://meetme.so/arrayfire\">technical consultation</a> with us today![/column]</p>\r\n<p style=\"text-align: left;\">[/content_band]</p>\r\n<p style=\"text-align: center;\"></p>','Download','','inherit','closed','closed','','5931-revision-v1','','','2016-10-15 15:55:21','2016-10-15 19:55:21','',5931,'http://sandbox.arrayfire.com/5931-revision-v1/',0,'revision','',0),(9395,11,'2016-10-21 17:55:03','0000-00-00 00:00:00','','installer_archive_s3','','draft','closed','closed','','','','','2016-10-21 17:55:03','2016-10-21 21:55:03','',0,'http://sandbox.arrayfire.com/?page_id=9395',0,'page','',0),(9396,11,'2016-10-21 17:55:03','2016-10-21 21:55:03','','installer_archive_s3','','inherit','closed','closed','','9395-revision-v1','','','2016-10-21 17:55:03','2016-10-21 21:55:03','',9395,'http://sandbox.arrayfire.com/9395-revision-v1/',0,'revision','',0),(9398,6,'2016-11-11 17:32:37','2016-11-11 22:32:37','<img src=\"http://www.advancedhpc.com/images/SC16.jpg\" alt=\"Image result for sc16 logo\" />\r\n\r\nSC16 is almost here! We\'re getting excited to heading to Salt Lake City, Utah, to be a part of this <a href=\"http://sc16.supercomputing.org/\">excellent conference</a>. It\'s a great place for soaking up HPC knowledge, getting inspired, and connecting with the brightest minds in the industry. Here\'s a quick run-down of where we\'ll be.\r\n\r\n<img class=\"alignright size-medium wp-image-6977\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/IMG_0737-300x225.jpg\" alt=\"IMG_0737\" width=\"300\" height=\"225\" />\r\n\r\n<strong>Visit our booth.</strong> We\'re booth #717 in the exhibit hall during exhibit hours November 15 - 17. We\'ll be showing off our latest demos and our engineers will be available for questions. Ask your questions, meet the team, or just bounce some ideas.\r\n\r\n<strong>Try our in-booth tutorials.</strong> Want to learn how to use <a href=\"http://sandbox.arrayfire.com/\">ArrayFire</a> to accelerate your code? Stop by and receive an in-booth tutorial from one of our ArrayFire experts. We’ll show you how the open source ArrayFire software library makes programming GPUs and other accelerators fast and easy.\r\n\r\n<b>Join us for a user meeting. </b>ArrayFire is more than a software library—it\'s an active community. Come to one of our user group meetings to get practical tips for working with ArrayFire and to network with other ArrayFire users. Visit our booth for more details.\r\n\r\n<strong>Set up individual meetings.</strong> Whether you live across the state or across the world, SC16 is a great time to connect one-on-one with us. Set up an appointment to discuss your work with an ArrayFire engineer at SC using our <a href=\"http://meetme.so/arrayfire\">meeting scheduler</a>.\r\n\r\nSee you in Salt Lake City!','ArrayFire at SC16','','publish','open','closed','','arrayfire-at-sc16','','','2016-11-11 17:32:37','2016-11-11 22:32:37','',0,'http://sandbox.arrayfire.com/?p=9398',0,'post','',0),(9399,6,'2016-11-11 17:28:24','2016-11-11 22:28:24','<img src=\"http://www.advancedhpc.com/images/SC16.jpg\" alt=\"Image result for sc16 logo\" />\r\n\r\nSC16 is almost here! We\'re getting excited to heading to Salt Lake City, Utah, to be a part of this <a href=\"http://sc16.supercomputing.org/\">excellent conference</a>. It\'s a great place for soaking up HPC knowledge, getting inspired, and connecting with the brightest minds in the industry. Here\'s a quick run-down of where we\'ll be.\r\n\r\n<img class=\"alignright size-medium wp-image-6977\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/IMG_0737-300x225.jpg\" alt=\"IMG_0737\" width=\"300\" height=\"225\" />\r\n\r\n<strong>Visit our booth.</strong> We\'re booth #717 in the exhibit hall during exhibit hours November 15 - 17. We\'ll be showing off our latest demos and our engineers will be available for questions. Ask your questions, meet the team, or just bounce some ideas.\r\n\r\n<strong>Try our in-booth tutorials.</strong> Want to learn how to use <a href=\"http://sandbox.arrayfire.com/\">ArrayFire</a> to accelerate your code? Stop by and receive an in-booth tutorial from one of our ArrayFire experts. We’ll show you how the open source ArrayFire software library makes programming GPUs and other accelerators fast and easy.\r\n\r\n<b>Join us for a user meeting. </b>ArrayFire is more than a software library—it\'s an active community. Come to one of our user group meetings to get practical tips for working with ArrayFire and to network with other ArrayFire users. Visit our booth for more details.\r\n\r\n<strong>Set up individual meetings.</strong> Whether you live across the state or across the world, SC16 is a great time to connect one-on-one with us. Set up an appointment to discuss your work with an ArrayFire engineer at SC using our <a href=\"http://meetme.so/arrayfire\">meeting scheduler</a>.\r\n\r\nSee you in Salt Lake City!','','','inherit','closed','closed','','9398-revision-v1','','','2016-11-11 17:28:24','2016-11-11 22:28:24','',9398,'http://sandbox.arrayfire.com/9398-revision-v1/',0,'revision','',0),(9400,6,'2016-11-11 17:32:32','2016-11-11 22:32:32','<img src=\"http://www.advancedhpc.com/images/SC16.jpg\" alt=\"Image result for sc16 logo\" />\r\n\r\nSC16 is almost here! We\'re getting excited to heading to Salt Lake City, Utah, to be a part of this <a href=\"http://sc16.supercomputing.org/\">excellent conference</a>. It\'s a great place for soaking up HPC knowledge, getting inspired, and connecting with the brightest minds in the industry. Here\'s a quick run-down of where we\'ll be.\r\n\r\n<img class=\"alignright size-medium wp-image-6977\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2014/10/IMG_0737-300x225.jpg\" alt=\"IMG_0737\" width=\"300\" height=\"225\" />\r\n\r\n<strong>Visit our booth.</strong> We\'re booth #717 in the exhibit hall during exhibit hours November 15 - 17. We\'ll be showing off our latest demos and our engineers will be available for questions. Ask your questions, meet the team, or just bounce some ideas.\r\n\r\n<strong>Try our in-booth tutorials.</strong> Want to learn how to use <a href=\"http://sandbox.arrayfire.com/\">ArrayFire</a> to accelerate your code? Stop by and receive an in-booth tutorial from one of our ArrayFire experts. We’ll show you how the open source ArrayFire software library makes programming GPUs and other accelerators fast and easy.\r\n\r\n<b>Join us for a user meeting. </b>ArrayFire is more than a software library—it\'s an active community. Come to one of our user group meetings to get practical tips for working with ArrayFire and to network with other ArrayFire users. Visit our booth for more details.\r\n\r\n<strong>Set up individual meetings.</strong> Whether you live across the state or across the world, SC16 is a great time to connect one-on-one with us. Set up an appointment to discuss your work with an ArrayFire engineer at SC using our <a href=\"http://meetme.so/arrayfire\">meeting scheduler</a>.\r\n\r\nSee you in Salt Lake City!','ArrayFire at SC16','','inherit','closed','closed','','9398-revision-v1','','','2016-11-11 17:32:32','2016-11-11 22:32:32','',9398,'http://sandbox.arrayfire.com/9398-revision-v1/',0,'revision','',0),(9403,4,'2017-02-08 15:32:46','2017-02-08 20:32:46','The <a href=\"https://www.catalysts.cc/en/projects/thrombotherm/\">Thrombotherm project</a> by Catalysts is developing a method to analyze blood platelets by means of cell microscopy in real time and to classify them according to their activation state. ArrayFire enabled faster overall research project times and real-time analysis on video data.\r\n\r\nThis project represents an enormous extension of thrombocyte diagnotics, especially through significantly accelerated analysis times. Faster analyses enabled university research collaborators from the University of Applied Sciences OÖ and the Ludwig Boltzmann Institute to shorten research project times.\r\n\r\nThe project has three main parts:\r\n<ol>\r\n 	<li><strong>Detect cell morphology in real-time</strong></li>\r\n</ol>\r\nThombotherm makes it possible to mathematically determine and categorize the cell boundaries by means of transmitted light microscopy. The software distinguishes between \"fried-egg”-shaped cells and \"spider”-shaped cells. This is used to assess the quality of blood platelets in the blink of an eye.\r\n\r\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/12/Fig1.png\"><img class=\"alignnone wp-image-9404 size-large\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/12/Fig1-1024x869.png\" alt=\"Fig1\" width=\"742\" height=\"630\" /></a>\r\n<ol start=\"2\">\r\n 	<li><strong>Real-time cluster analysis</strong></li>\r\n</ol>\r\nBy means of a fluorescence micrograph, Thrombotherm identifies cell clusters taking into account the diffraction phenomena that occur. This allows a quantitative real-time representation of the platelets in the present sample.\r\n\r\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/12/Fig2.png\"><img class=\"alignnone size-large wp-image-9407\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/12/Fig2-1024x869.png\" alt=\"Fig2\" width=\"742\" height=\"630\" /></a>\r\n<ol start=\"3\">\r\n 	<li><strong>Real-time analysis of the mobility of receptors</strong></li>\r\n</ol>\r\nThrombotherm also analyzes real-time image sequences. This allows following single molecules and their motion traces. Pixel resolution is not sufficiently accurate, so exact values are calculated with a point spread model. Complex modeling on real-time image sequences is computationally intensive, and ArrayFire enabled realtime computational performance results.\r\n\r\n[video width=\"2000\" height=\"700\" mp4=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/12/Thrombos_epoxy_blog_vid.mp4\" autoplay=\"true\" preload=\"auto\"][/video]\r\n<p style=\"text-align: center;\"><em>Complex calculation of moving images in real time enabled with ArrayFire</em></p>\r\nWith the help of ArrayFire, engineers were able to attain dramatic performance improvements on two of the core algorithmic steps for analyzing real-time image sequences.\r\n\r\nEngineers achieved a <strong>2451X</strong> and a <strong>529X</strong> speedup on both algorithms, respectively, when using ArrayFire over their standard CPU implementation. Significant performance improvements were also observed on both algorithms when using ArrayFire over their standard CUDA implementation using custom kernels - <strong>368X</strong> and <strong>95X</strong> speedups, respectively.\r\n\r\nThrough this completely new method of analyzing thrombocytes, laboratories can identify the mobility of cells, the compression of surface area markers, the morphology of cells, and their activation condition in real-time.','Thrombotherm:  Analyzing Blood Platelets with ArrayFire','','publish','open','closed','','thrombotherm-analyzing-blood-platelets-with-arrayfire','','','2017-02-08 15:32:46','2017-02-08 20:32:46','',0,'http://sandbox.arrayfire.com/?p=9403',0,'post','',0),(9404,4,'2016-12-19 13:01:03','2016-12-19 18:01:03','','Fig1','','inherit','open','closed','','fig1','','','2016-12-19 13:01:03','2016-12-19 18:01:03','',9403,'http://sandbox.arrayfire.com/wp-content/uploads/2016/12/Fig1.png',0,'attachment','image/png',0),(9405,4,'2016-12-19 13:01:21','2016-12-19 18:01:21','A few months ago Catalysts has started the Thrombotherm project. This project is organized in collaboration with the University of Applied Sciences OÖ and the Ludwig Boltzmann Institute. The main goal of  the project is developing a method to analyze blood platelets by means of cell microscopy in real time and to classify them according to their activation state.\r\n\r\nThe software (software plugin) developed by Catalysts represents an enormous extension of the diagnostics. This helped us to bring the research of our partners to the market and in the field of diagnostics of thrombocytes a significant acceleration of the analysis.\r\n\r\nThe success of this project depends on the time necessary to process gathered information. This is why we were very careful in choosing the right framework fast enough to make algorithms real-time. Reading all reviews and recommendations (NVidia etc.) we have decided to use ArrayFire. So far we are very satisfied with its performance.\r\n\r\nOur project has three main parts:\r\n<ol>\r\n 	<li>Detect cell morphology in real-time</li>\r\n</ol>\r\nThombotherm makes it possible to mathematically determine and categorize the cell boundaries by means of transmitted light microscopy. The software distinguishes between \"fried-egg”-shaped cells and \"spider”-shaped cells. This is used to assess the quality of blood platelets in the blink of an eye.\r\n\r\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/12/Fig1.png\"><img class=\"alignnone size-medium wp-image-9404\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/12/Fig1-300x255.png\" alt=\"Fig1\" width=\"300\" height=\"255\" /></a>\r\n<ol start=\"2\">\r\n 	<li>Real-time cluster analysis</li>\r\n</ol>\r\nBy means of a fluorescence micrograph, Thrombotherm identifies cell clusters taking into account the diffraction phenomena that occur. This allows a quantitative real-time representation of the platelets in the present sample.\r\n<ol start=\"3\">\r\n 	<li>Real-time analysis of the mobility of receptors</li>\r\n</ol>\r\nThrombotherm actually does not only analyse single images but also real-time image sequences. This allows following single molecules and their motion traces. Pixel resolution would not be sufficiently accurate in this case, so in order to receive exact values we are calculating with a point spread model. For doing a complex model calculation on the real-time image sequences preformant library- like ArrayFire- is invaluable.\r\n\r\n<em>Complex calculation of moving images in real time enabled with ArrayFire</em>\r\n\r\nWith the help of this completely new method of analyzing thrombocytes, laboratories can identify the mobility of cells, the compression of surface area markers, the morphology of cells and their activation condition in real time, all by using their existing hardware!','Thrombotherm:  Analyzing Blood Platelets with ArrayFire','','inherit','closed','closed','','9403-revision-v1','','','2016-12-19 13:01:21','2016-12-19 18:01:21','',9403,'http://sandbox.arrayfire.com/9403-revision-v1/',0,'revision','',0),(9406,4,'2016-12-19 13:01:41','2016-12-19 18:01:41','A few months ago Catalysts has started the Thrombotherm project. This project is organized in collaboration with the University of Applied Sciences OÖ and the Ludwig Boltzmann Institute. The main goal of  the project is developing a method to analyze blood platelets by means of cell microscopy in real time and to classify them according to their activation state.\r\n\r\nThe software (software plugin) developed by Catalysts represents an enormous extension of the diagnostics. This helped us to bring the research of our partners to the market and in the field of diagnostics of thrombocytes a significant acceleration of the analysis.\r\n\r\nThe success of this project depends on the time necessary to process gathered information. This is why we were very careful in choosing the right framework fast enough to make algorithms real-time. Reading all reviews and recommendations (NVidia etc.) we have decided to use ArrayFire. So far we are very satisfied with its performance.\r\n\r\nOur project has three main parts:\r\n<ol>\r\n 	<li>Detect cell morphology in real-time</li>\r\n</ol>\r\nThombotherm makes it possible to mathematically determine and categorize the cell boundaries by means of transmitted light microscopy. The software distinguishes between \"fried-egg”-shaped cells and \"spider”-shaped cells. This is used to assess the quality of blood platelets in the blink of an eye.\r\n\r\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/12/Fig1.png\"><img class=\"alignnone wp-image-9404 size-large\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/12/Fig1-1024x869.png\" alt=\"Fig1\" width=\"742\" height=\"630\" /></a>\r\n<ol start=\"2\">\r\n 	<li>Real-time cluster analysis</li>\r\n</ol>\r\nBy means of a fluorescence micrograph, Thrombotherm identifies cell clusters taking into account the diffraction phenomena that occur. This allows a quantitative real-time representation of the platelets in the present sample.\r\n<ol start=\"3\">\r\n 	<li>Real-time analysis of the mobility of receptors</li>\r\n</ol>\r\nThrombotherm actually does not only analyse single images but also real-time image sequences. This allows following single molecules and their motion traces. Pixel resolution would not be sufficiently accurate in this case, so in order to receive exact values we are calculating with a point spread model. For doing a complex model calculation on the real-time image sequences preformant library- like ArrayFire- is invaluable.\r\n\r\n<em>Complex calculation of moving images in real time enabled with ArrayFire</em>\r\n\r\nWith the help of this completely new method of analyzing thrombocytes, laboratories can identify the mobility of cells, the compression of surface area markers, the morphology of cells and their activation condition in real time, all by using their existing hardware!','Thrombotherm:  Analyzing Blood Platelets with ArrayFire','','inherit','closed','closed','','9403-revision-v1','','','2016-12-19 13:01:41','2016-12-19 18:01:41','',9403,'http://sandbox.arrayfire.com/9403-revision-v1/',0,'revision','',0),(9407,4,'2016-12-19 13:02:06','2016-12-19 18:02:06','','Fig2','','inherit','open','closed','','fig2','','','2016-12-19 13:02:06','2016-12-19 18:02:06','',9403,'http://sandbox.arrayfire.com/wp-content/uploads/2016/12/Fig2.png',0,'attachment','image/png',0),(9409,4,'2016-12-19 18:29:37','2016-12-19 18:29:37','','Thrombos_epoxy_blog_vid','','inherit','open','closed','','thrombos_epoxy_blog_vid','','','2016-12-19 18:29:37','2016-12-19 18:29:37','',9403,'http://sandbox.arrayfire.com/wp-content/uploads/2016/12/Thrombos_epoxy_blog_vid.mp4',0,'attachment','video/mp4',0),(9410,4,'2016-12-19 13:30:01','2016-12-19 18:30:01','A few months ago Catalysts has started the Thrombotherm project. This project is organized in collaboration with the University of Applied Sciences OÖ and the Ludwig Boltzmann Institute. The main goal of  the project is developing a method to analyze blood platelets by means of cell microscopy in real time and to classify them according to their activation state.\r\n\r\nThe software (software plugin) developed by Catalysts represents an enormous extension of the diagnostics. This helped us to bring the research of our partners to the market and in the field of diagnostics of thrombocytes a significant acceleration of the analysis.\r\n\r\nThe success of this project depends on the time necessary to process gathered information. This is why we were very careful in choosing the right framework fast enough to make algorithms real-time. Reading all reviews and recommendations (NVidia etc.) we have decided to use ArrayFire. So far we are very satisfied with its performance.\r\n\r\nOur project has three main parts:\r\n<ol>\r\n 	<li>Detect cell morphology in real-time</li>\r\n</ol>\r\nThombotherm makes it possible to mathematically determine and categorize the cell boundaries by means of transmitted light microscopy. The software distinguishes between \"fried-egg”-shaped cells and \"spider”-shaped cells. This is used to assess the quality of blood platelets in the blink of an eye.\r\n\r\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/12/Fig1.png\"><img class=\"alignnone wp-image-9404 size-large\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/12/Fig1-1024x869.png\" alt=\"Fig1\" width=\"742\" height=\"630\" /></a>\r\n<ol start=\"2\">\r\n 	<li>Real-time cluster analysis</li>\r\n</ol>\r\nBy means of a fluorescence micrograph, Thrombotherm identifies cell clusters taking into account the diffraction phenomena that occur. This allows a quantitative real-time representation of the platelets in the present sample.\r\n\r\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/12/Fig2.png\"><img class=\"alignnone size-large wp-image-9407\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/12/Fig2-1024x869.png\" alt=\"Fig2\" width=\"742\" height=\"630\" /></a>\r\n<ol start=\"3\">\r\n 	<li>Real-time analysis of the mobility of receptors</li>\r\n</ol>\r\nThrombotherm actually does not only analyse single images but also real-time image sequences. This allows following single molecules and their motion traces. Pixel resolution would not be sufficiently accurate in this case, so in order to receive exact values we are calculating with a point spread model. For doing a complex model calculation on the real-time image sequences preformant library- like ArrayFire- is invaluable.\r\n\r\n[video width=\"2000\" height=\"700\" mp4=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/12/Thrombos_epoxy_blog_vid.mp4\"][/video]\r\n\r\n<em>Complex calculation of moving images in real time enabled with ArrayFire</em>\r\n\r\nWith the help of this completely new method of analyzing thrombocytes, laboratories can identify the mobility of cells, the compression of surface area markers, the morphology of cells and their activation condition in real time, all by using their existing hardware!','Thrombotherm:  Analyzing Blood Platelets with ArrayFire','','inherit','closed','closed','','9403-revision-v1','','','2016-12-19 13:30:01','2016-12-19 18:30:01','',9403,'http://sandbox.arrayfire.com/9403-revision-v1/',0,'revision','',0),(9411,4,'2016-12-19 14:32:53','2016-12-19 19:32:53','The <a href=\"https://www.catalysts.cc/en/projects/thrombotherm/\">Thrombotherm project</a> by Catalysts is developing a method to analyze blood platelets by means of cell microscopy in real time and to classify them according to their activation state. ArrayFire enabled faster overall research project times and real-time analysis on video data.\r\n\r\nThis project represents an enormous extension of thrombocyte diagnotics, especially through significantly accelerated analysis times. Faster analyses enabled university research collaborators from the University of Applied Sciences OÖ and the Ludwig Boltzmann Institute to shorten research project times.\r\n\r\nThe project has three main parts:\r\n<ol>\r\n 	<li><strong>Detect cell morphology in real-time</strong></li>\r\n</ol>\r\nThombotherm makes it possible to mathematically determine and categorize the cell boundaries by means of transmitted light microscopy. The software distinguishes between \"fried-egg”-shaped cells and \"spider”-shaped cells. This is used to assess the quality of blood platelets in the blink of an eye.\r\n\r\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/12/Fig1.png\"><img class=\"alignnone wp-image-9404 size-large\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/12/Fig1-1024x869.png\" alt=\"Fig1\" width=\"742\" height=\"630\" /></a>\r\n<ol start=\"2\">\r\n 	<li><strong>Real-time cluster analysis</strong></li>\r\n</ol>\r\nBy means of a fluorescence micrograph, Thrombotherm identifies cell clusters taking into account the diffraction phenomena that occur. This allows a quantitative real-time representation of the platelets in the present sample.\r\n\r\n<a href=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/12/Fig2.png\"><img class=\"alignnone size-large wp-image-9407\" src=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/12/Fig2-1024x869.png\" alt=\"Fig2\" width=\"742\" height=\"630\" /></a>\r\n<ol start=\"3\">\r\n 	<li><strong>Real-time analysis of the mobility of receptors</strong></li>\r\n</ol>\r\nThrombotherm also analyzes real-time image sequences. This allows following single molecules and their motion traces. Pixel resolution is not sufficiently accurate, so exact values are calculated with a point spread model. Complex modeling on real-time image sequences is computationally intensive, and ArrayFire enabled realtime computational performance results.\r\n\r\n[video width=\"2000\" height=\"700\" mp4=\"http://sandbox.arrayfire.com/wp-content/uploads/2016/12/Thrombos_epoxy_blog_vid.mp4\" autoplay=\"true\" preload=\"auto\"][/video]\r\n<p style=\"text-align: center;\"><em>Complex calculation of moving images in real time enabled with ArrayFire</em></p>\r\nThrough this completely new method of analyzing thrombocytes, laboratories can identify the mobility of cells, the compression of surface area markers, the morphology of cells, and their activation condition in real-time.','Thrombotherm:  Analyzing Blood Platelets with ArrayFire','','inherit','closed','closed','','9403-revision-v1','','','2016-12-19 14:32:53','2016-12-19 19:32:53','',9403,'http://sandbox.arrayfire.com/9403-revision-v1/',0,'revision','',0),(9415,18672,'2016-12-20 14:32:32','2016-12-20 19:32:32','In last week\'s \"<a href=\"http://sandbox.arrayfire.com/tag/learning-arrayfire-from-scratch/\">Learning ArrayFire from scratch</a>\" post we discussed how to interface ArrayFire and CUDA to improve your productivity. This week we demonstrate how to mix ArrayFire and OpenCL in your application.\n\n<!--more-->\n\nMuch like with CUDA, ArrayFire controls almost all aspects of its execution on OpenCL platforms. ArrayFire manages its own context, queue, memory, and creates custom IDs for devices. As such, most of the interoperability functions focus on reducing potential synchronization conflicts between ArrayFire and OpenCL.\n\n<strong>NOTE</strong>: This blog post was written against ArrayFire 3.3.x. We intend to implement additional convenience functions in ArrayFire 3.4 that should simplify this process further. Be sure to <a href=\"http://arrayfire.org/docs/interop_cuda.htm\">check the documentation</a> for any updates in future releases.\n<h4>Basics</h4>\nIt is fairly straightforward to interface ArrayFire with your own custom OpenCL code. ArrayFire provides several functions to ease this process including:\n<table class=\"doxtable\">\n<tbody>\n<tr>\n<th>Function</th>\n<th>Purpose</th>\n</tr>\n<tr>\n<td><code>af::array(...)</code></td>\n<td>Construct an ArrayFire array from <code>cl_mem</code> references or <code>cl::Buffer</code> objects</td>\n</tr>\n<tr>\n<td><code>af::array.device()</code></td>\n<td>Obtain a pointer to the <code>cl_mem</code> reference (implies lock())</td>\n</tr>\n<tr>\n<td><code>af::array.lock()</code></td>\n<td>Removes ArrayFire\'s control of a <code>cl_mem</code> buffer</td>\n</tr>\n<tr>\n<td><code>af::array.unlock()</code></td>\n<td>Restore\'s ArrayFire\'s control over a <code>cl_mem</code> buffer</td>\n</tr>\n<tr>\n<td><code>afcl::getPlatform()</code></td>\n<td>Get ArrayFire\'s current <code>cl_platform</code></td>\n</tr>\n<tr>\n<td><code>af::getDevice()</code></td>\n<td>Get the current ArrayFire Device ID</td>\n</tr>\n<tr>\n<td><code>afcl::getDeviceId()</code></td>\n<td>Get ArrayFire\'s current <code>cl_device_id</code></td>\n</tr>\n<tr>\n<td><code>af::setDevice()</code></td>\n<td>Set ArrayFire\'s device from an ArrayFire device ID</td>\n</tr>\n<tr>\n<td><code>afcl::setDeviceId()</code></td>\n<td>Set ArrayFire\'s device from a <code>cl_device_id</code></td>\n</tr>\n<tr>\n<td><code>afcl::setDevice()</code></td>\n<td>Set ArrayFire\'s device from a <code>cl_device_id</code> and <code>cl_context</code></td>\n</tr>\n<tr>\n<td><code>afcl::getContext()</code></td>\n<td>Get ArrayFire\'s current <code>cl_context</code></td>\n</tr>\n<tr>\n<td><code>afcl::getQueue()</code></td>\n<td>Get ArrayFire\'s current <code>cl_command_queue</code></td>\n</tr>\n<tr>\n<td><code>afcl::getDeviceType()</code></td>\n<td>Get the current <code>afcl_device_type</code></td>\n</tr>\n</tbody>\n</table>\nAdditionally, the OpenCL backend permits the programmer to add and remove custom devices from the ArrayFire device manager. These permit you to attach ArrayFire directly to the OpenCL queue used by other portions of your application.\n<table class=\"doxtable\">\n<tbody>\n<tr>\n<th>Function</th>\n<th>Purpose</th>\n</tr>\n<tr>\n<td><code>afcl::addDevice()</code></td>\n<td>Add a new device to ArrayFire\'s device manager</td>\n</tr>\n<tr>\n<td><code>afcl::deleteDevice()</code></td>\n<td>Remove a device from ArrayFire\'s device manager</td>\n</tr>\n</tbody>\n</table>\nBelow we provide two worked examples on how ArrayFire can be integrated into new and existing projects.\n<h4>Adding custom OpenCL kernels to an existing ArrayFire application</h4>\nBy default, ArrayFire manages its own context, queue, memory, and creates custom IDs for devices. Thus there is some bookkeeping that needs to be done to integrate your custom OpenCL kernel.\n\nIf your kernels can share operate in the same queue as ArrayFire, you should:\n<ol type=\"1\">\n 	<li>Add an include for <code>af/opencl.h</code> to your project</li>\n 	<li>Obtain the OpenCL context, device, and queue used by ArrayFire</li>\n 	<li>Obtain <code>cl_mem</code> references to <code>af::array</code> objects</li>\n 	<li>Load, build, and use your kernels</li>\n 	<li>Return control of <code>af::array</code> memory to ArrayFire</li>\n</ol>\nNote, ArrayFire uses an in-order queue, thus when ArrayFire and your kernels are operating in the same queue, there is no need to perform any synchronization operations.\n\nThis process is best illustrated with a fully worked example:\n<div class=\"fragment\">\n<div class=\"line\">\n<pre class=\"lang:default decode:true \">#include &lt;arrayfire.h&gt;\n// 1. Add the af/opencl.h include to your project\n#include &lt;af/opencl.h&gt;\n \nint main() {\nsize_t length = 10;\n \n// Create ArrayFire array objects:\naf::array A = af::randu(length, f32);\naf::array B = af::constant(0, length, f32);\n \n// ... additional ArrayFire operations here\n \n// 2. Obtain the device, context, and queue used by ArrayFire\nstatic cl_context af_context = afcl::getContext();\nstatic cl_device_id af_device_id = afcl::getDeviceId();\nstatic cl_command_queue af_queue = afcl::getQueue();\n \n// 3. Obtain cl_mem references to af::array objects\ncl_mem * d_A = A.device&lt;cl_mem&gt;();\ncl_mem * d_B = B.device&lt;cl_mem&gt;();\n \n// 4. Load, build, and use your kernels.\n// For the sake of readability, we have omitted error checking.\nint status = CL_SUCCESS;\n \n// A simple copy kernel, uses C++11 syntax for multi-line strings.\nconst char * kernel_name = \"copy_kernel\";\nconst char * source = R\"(\nvoid __kernel\ncopy_kernel(__global float * gA, __global float * gB)\n{\nint id = get_global_id(0);\ngB[id] = gA[id];\n}\n)\";\n \n// Create the program, build the executable, and extract the entry point\n// for the kernel.\ncl_program program = clCreateProgramWithSource(af_context, 1, &amp;source, NULL, &amp;status);\nstatus = clBuildProgram(program, 1, &amp;af_device_id, NULL, NULL, NULL);\ncl_kernel kernel = clCreateKernel(program, kernel_name, &amp;status);\n \n// Set arguments and launch your kernels\nclSetKernelArg(kernel, 0, sizeof(cl_mem), d_A);\nclSetKernelArg(kernel, 1, sizeof(cl_mem), d_B);\nclEnqueueNDRangeKernel(af_queue, kernel, 1, NULL, &amp;length, NULL, 0, NULL, NULL);\n \n// 5. Return control of af::array memory to ArrayFire\nA.unlock();\nB.unlock();\n \n// ... resume ArrayFire operations\n \n// Because the device pointers, d_x and d_y, were returned to ArrayFire\'s\n// control by the unlock function, there is no need to free them using\n// clReleaseMemObject()\n \nreturn 0;\n}</pre>\n</div>\n</div>\nIf your kernels needs to operate in their own OpenCL queue, the process is essentially identical, except you need to instruct ArrayFire to complete its computations using the <code>af::sync()</code> function prior to launching your own kernel and ensure your kernels are complete using <code>clFinish</code> (or similar) commands prior to returning control of the memory to ArrayFire:\n<ol type=\"1\">\n 	<li>Add an include for <code>af/opencl.h</code> to your project</li>\n 	<li>Obtain the OpenCL context, device, and queue used by ArrayFire</li>\n 	<li>Obtain <code>cl_mem</code> references to <code>af::array</code> objects</li>\n 	<li>Instruct ArrayFire to finish operations using <code>af::sync()</code></li>\n 	<li>Load, build, and use your kernels</li>\n 	<li>Instruct OpenCL to finish operations using <code>clFinish()</code> or similar commands</li>\n</ol>\n 	<li>Return control of <code>af::array</code> memory to ArrayFire</li>\n<\n<h4>Adding ArrayFire to an existing OpenCL application</h4>\nAdding ArrayFire to an existing OpenCL application is slightly more involved and can be somewhat tricky due to several optimizations we implement. The most important are as follows:\n<ul>\n 	<li>ArrayFire assumes control of all memory provided to it.</li>\n 	<li>ArrayFire does not (in general) support in-place memory transactions.</li>\n</ul>\nWe will discuss the implications of these items below. To add ArrayFire to existing code you need to:\n<ol type=\"1\">\n 	<li>Add includes</li>\n 	<li>Instruct OpenCL to complete its operations using <code>clFinish</code> (or similar)</li>\n 	<li>Instruct ArrayFire to use the user-created OpenCL Context</li>\n 	<li>Create ArrayFire arrays from OpenCL memory objects</li>\n 	<li>Perform ArrayFire operations on the Arrays</li>\n 	<li>Instruct ArrayFire to finish operations using <code>af::sync()</code></li>\n 	<li>Obtain cl_mem references for important memory</li>\n 	<li>Continue your OpenCL application</li>\n</ol>\nTo create the af::array objects, you should use one of the following constructors:\n<div class=\"fragment\">\n<div class=\"line\">\n<pre class=\"lang:default decode:true \">// 1D - 3D af::array constructors\nstatic af::array array (dim_t dim0, cl_mem buf, af::dtype type, bool retain=false)\nstatic af::array array (dim_t dim0, dim_t dim1, cl_mem buf, af::dtype type, bool retain=false)\nstatic af::array array (dim_t dim0, dim_t dim1, dim_t dim2, cl_mem buf, af::dtype type, bool retain=false)\nstatic af::array array (dim_t dim0, dim_t dim1, dim_t dim2, dim_t dim3, cl_mem buf, af::dtype type, bool retain=false)\n \n// af::array constructor using a dim4 object\nstatic af::array array (af::dim4 idims, cl_mem buf, af::dtype type, bool retain=false)</pre>\n</div>\n</div>\n<em>NOTE</em>: With all of these constructors, ArrayFire\'s memory manager automatically assumes responsibility for any memory provided to it. If you are creating an array from a <code>cl::Buffer</code>, you should specify <code>retain=true</code> to ensure your memory is not deallocated if your <code>cl::Buffer</code> were to go out of scope. We use this technique in the example below. If you do not wish for ArrayFire to manage your memory, you may call the <code>array::unlock()</code> function and manage the memory yourself; however, if you do so, please be cautious not to call <code>clReleaseMemObj</code> on a <code>cl_mem</code> when ArrayFire might be using it!\n\nThe eight steps above are best illustrated using a fully-worked example. Below we use the OpenCL 2.0 C++ API and omit error checking to keep the code readable.\n<div class=\"fragment\">\n<div class=\"line\">\n<pre class=\"lang:default decode:true \">#include &lt;CL/cl2.hpp&gt;\n \n// 1. Add arrayfire.h and af/opencl.h to your application\n#include \"arrayfire.h\"\n#include \"af/opencl.h\"\n \n#include &lt;cstdio&gt;\n#include &lt;vector&gt;\n \nint main() {\n \n// Set up the OpenCL context, device, and queues\ncl::Context context(CL_DEVICE_TYPE_ALL);\nvector&lt;cl::Device&gt; devices = context.getInfo&lt;CL_CONTEXT_DEVICES&gt;();\ncl::Device device = devices[0];\ncl::CommandQueue queue(context, device);\n \n// Create a buffer of size 10 filled with ones, copy it to the device\nint length = 10;\nvector&lt;float&gt; h_A(length, 1);\ncl::Buffer cl_A(context, CL_MEM_READ_WRITE, length * sizeof(float), h_A.data());\n \n// 2. Instruct OpenCL to complete its operations using clFinish (or similar)\nqueue.finish();\n \n// 3. Instruct ArrayFire to use the user-created context\n// First, create a device from the current OpenCL device + context + queue\nafcl::addDevice(device(), context(), queue());\n// Next switch ArrayFire to the device using the device and context as\n// identifiers:\nafcl::setDevice(device(), context());\n \n// 4. Create ArrayFire arrays from OpenCL memory objects\naf::array af_A = afcl::array(length, cl_A(), f32, true);\n \n// 5. Perform ArrayFire operations on the Arrays\naf_A = af_A + af::randu(length);\n \n// NOTE: ArrayFire does not perform the above transaction using in-place memory,\n// thus the underlying OpenCL buffers containing the memory containing memory to\n// probably have changed\n \n// 6. Instruct ArrayFire to finish operations using af::sync\naf::sync();\n \n// 7. Obtain cl_mem references for important memory\ncl_A = *af_A.device&lt;cl_mem&gt;();\n \n// 8. Continue your OpenCL application\n \n// ...\n \nreturn 0;\n}</pre>\n</div>\n</div>\n<h4>Using multiple devices</h4>\nIf you are using ArrayFire and OpenCL with multiple devices be sure to use <code>afcl::addDevice</code> to add your custom context + device + queue to ArrayFire\'s device manager. This will let you switch ArrayFire devices using your current<code>cl_device_id</code> and <code>cl_context</code>.','ArrayFire - OpenCL Interoperability','','inherit','closed','closed','','8874-autosave-v1','','','2016-12-20 14:32:32','2016-12-20 19:32:32','',8874,'http://sandbox.arrayfire.com/8874-autosave-v1/',0,'revision','',0),(9416,18672,'2016-12-20 14:21:17','2016-12-20 19:21:17','In last week\'s \"<a href=\"http://sandbox.arrayfire.com/tag/learning-arrayfire-from-scratch/\">Learning ArrayFire from scratch</a>\" post we discussed how to interface ArrayFire and CUDA to improve your productivity. This week we demonstrate how to mix ArrayFire and OpenCL in your application.\r\n\r\n<!--more-->\r\n\r\nMuch like with CUDA, ArrayFire controls almost all aspects of its execution on OpenCL platforms. ArrayFire manages its own context, queue, memory, and creates custom IDs for devices. As such, most of the interoperability functions focus on reducing potential synchronization conflicts between ArrayFire and OpenCL.\r\n\r\n<strong>NOTE</strong>: This blog post was written against ArrayFire 3.3.x. We intend to implement additional convenience functions in ArrayFire 3.4 that should simplify this process further. Be sure to <a href=\"http://arrayfire.org/docs/interop_cuda.htm\">check the documentation</a> for any updates in future releases.\r\n<h4>Basics</h4>\r\nIt is fairly straightforward to interface ArrayFire with your own custom OpenCL code. ArrayFire provides several functions to ease this process including:\r\n<table class=\"doxtable\">\r\n<tbody>\r\n<tr>\r\n<th>Function</th>\r\n<th>Purpose</th>\r\n</tr>\r\n<tr>\r\n<td>af::array(...)</td>\r\n<td>Construct an ArrayFire array from cl_mem references or cl::Buffer objects</td>\r\n</tr>\r\n<tr>\r\n<td>af::array.device()</td>\r\n<td>Obtain a pointer to the cl_mem reference (implies lock())</td>\r\n</tr>\r\n<tr>\r\n<td>af::array.lock()</td>\r\n<td>Removes ArrayFire\'s control of a cl_mem buffer</td>\r\n</tr>\r\n<tr>\r\n<td>af::array.unlock()</td>\r\n<td>Restore\'s ArrayFire\'s control over a cl_mem buffer</td>\r\n</tr>\r\n<tr>\r\n<td>afcl::getPlatform()</td>\r\n<td>Get ArrayFire\'s current cl_platform</td>\r\n</tr>\r\n<tr>\r\n<td>af::getDevice()</td>\r\n<td>Get the current ArrayFire Device ID</td>\r\n</tr>\r\n<tr>\r\n<td>afcl::getDeviceId()</td>\r\n<td>Get ArrayFire\'s current cl_device_id</td>\r\n</tr>\r\n<tr>\r\n<td>af::setDevice()</td>\r\n<td>Set ArrayFire\'s device from an ArrayFire device ID</td>\r\n</tr>\r\n<tr>\r\n<td>afcl::setDeviceId()</td>\r\n<td>Set ArrayFire\'s device from a cl_device_id</td>\r\n</tr>\r\n<tr>\r\n<td>afcl::setDevice()</td>\r\n<td>Set ArrayFire\'s device from a cl_device_id and cl_context</td>\r\n</tr>\r\n<tr>\r\n<td>afcl::getContext()</td>\r\n<td>Get ArrayFire\'s current cl_context</td>\r\n</tr>\r\n<tr>\r\n<td>afcl::getQueue()</td>\r\n<td>Get ArrayFire\'s current cl_command_queue</td>\r\n</tr>\r\n<tr>\r\n<td>afcl::getDeviceType()</td>\r\n<td>Get the current afcl_device_type</td>\r\n</tr>\r\n</tbody>\r\n</table>\r\nAdditionally, the OpenCL backend permits the programmer to add and remove custom devices from the ArrayFire device manager. These permit you to attach ArrayFire directly to the OpenCL queue used by other portions of your application.\r\n<table class=\"doxtable\">\r\n<tbody>\r\n<tr>\r\n<th>Function</th>\r\n<th>Purpose</th>\r\n</tr>\r\n<tr>\r\n<td>afcl::addDevice()</td>\r\n<td>Add a new device to ArrayFire\'s device manager</td>\r\n</tr>\r\n<tr>\r\n<td>afcl::deleteDevice()</td>\r\n<td>Remove a device from ArrayFire\'s device manager</td>\r\n</tr>\r\n</tbody>\r\n</table>\r\nBelow we provide two worked examples on how ArrayFire can be integrated into new and existing projects.\r\n<h4>Adding custom OpenCL kernels to an existing ArrayFire application</h4>\r\nBy default, ArrayFire manages its own context, queue, memory, and creates custom IDs for devices. Thus there is some bookkeeping that needs to be done to integrate your custom OpenCL kernel.\r\n\r\nIf your kernels can share operate in the same queue as ArrayFire, you should:\r\n<ol type=\"1\">\r\n 	<li>Add an include for <code>af/opencl.h</code> to your project</li>\r\n 	<li>Obtain the OpenCL context, device, and queue used by ArrayFire</li>\r\n 	<li>Obtain cl_mem references to af::array objects</li>\r\n 	<li>Load, build, and use your kernels</li>\r\n 	<li>Return control of af::array memory to ArrayFire</li>\r\n</ol>\r\nNote, ArrayFire uses an in-order queue, thus when ArrayFire and your kernels are operating in the same queue, there is no need to perform any synchronization operations.\r\n\r\nThis process is best illustrated with a fully worked example:\r\n<div class=\"fragment\">\r\n<div class=\"line\">\r\n<pre class=\"lang:default decode:true \">#include &lt;arrayfire.h&gt;\r\n// 1. Add the af/opencl.h include to your project\r\n#include &lt;af/opencl.h&gt;\r\n \r\nint main() {\r\nsize_t length = 10;\r\n \r\n// Create ArrayFire array objects:\r\naf::array A = af::randu(length, f32);\r\naf::array B = af::constant(0, length, f32);\r\n \r\n// ... additional ArrayFire operations here\r\n \r\n// 2. Obtain the device, context, and queue used by ArrayFire\r\nstatic cl_context af_context = afcl::getContext();\r\nstatic cl_device_id af_device_id = afcl::getDeviceId();\r\nstatic cl_command_queue af_queue = afcl::getQueue();\r\n \r\n// 3. Obtain cl_mem references to af::array objects\r\ncl_mem * d_A = A.device&lt;cl_mem&gt;();\r\ncl_mem * d_B = B.device&lt;cl_mem&gt;();\r\n \r\n// 4. Load, build, and use your kernels.\r\n// For the sake of readability, we have omitted error checking.\r\nint status = CL_SUCCESS;\r\n \r\n// A simple copy kernel, uses C++11 syntax for multi-line strings.\r\nconst char * kernel_name = \"copy_kernel\";\r\nconst char * source = R\"(\r\nvoid __kernel\r\ncopy_kernel(__global float * gA, __global float * gB)\r\n{\r\nint id = get_global_id(0);\r\ngB[id] = gA[id];\r\n}\r\n)\";\r\n \r\n// Create the program, build the executable, and extract the entry point\r\n// for the kernel.\r\ncl_program program = clCreateProgramWithSource(af_context, 1, &amp;source, NULL, &amp;status);\r\nstatus = clBuildProgram(program, 1, &amp;af_device_id, NULL, NULL, NULL);\r\ncl_kernel kernel = clCreateKernel(program, kernel_name, &amp;status);\r\n \r\n// Set arguments and launch your kernels\r\nclSetKernelArg(kernel, 0, sizeof(cl_mem), d_A);\r\nclSetKernelArg(kernel, 1, sizeof(cl_mem), d_B);\r\nclEnqueueNDRangeKernel(af_queue, kernel, 1, NULL, &amp;length, NULL, 0, NULL, NULL);\r\n \r\n// 5. Return control of af::array memory to ArrayFire\r\nA.unlock();\r\nB.unlock();\r\n \r\n// ... resume ArrayFire operations\r\n \r\n// Because the device pointers, d_x and d_y, were returned to ArrayFire\'s\r\n// control by the unlock function, there is no need to free them using\r\n// clReleaseMemObject()\r\n \r\nreturn 0;\r\n}</pre>\r\n</div>\r\n</div>\r\nIf your kernels needs to operate in their own OpenCL queue, the process is essentially identical, except you need to instruct ArrayFire to complete its computations using the af::sync() function prior to launching your own kernel and ensure your kernels are complete using <code>clFinish</code> (or similar) commands prior to returning control of the memory to ArrayFire:\r\n<ol type=\"1\">\r\n 	<li>Add an include for <code>af/opencl.h</code> to your project</li>\r\n 	<li>Obtain the OpenCL context, device, and queue used by ArrayFire</li>\r\n 	<li>Obtain cl_mem references to af::array objects</li>\r\n 	<li>Instruct ArrayFire to finish operations using af::sync()</li>\r\n 	<li>Load, build, and use your kernels</li>\r\n 	<li>Instruct OpenCL to finish operations using clFinish() or similar commands.</li>\r\n</ol>\r\n<ol type=\"1\">\r\n 	<li>Return control of af::array memory to ArrayFire</li>\r\n</ol>\r\n<h4>Adding ArrayFire to an existing OpenCL application</h4>\r\nAdding ArrayFire to an existing OpenCL application is slightly more involved and can be somewhat tricky due to several optimizations we implement. The most important are as follows:\r\n<ul>\r\n 	<li>ArrayFire assumes control of all memory provided to it.</li>\r\n 	<li>ArrayFire does not (in general) support in-place memory transactions.</li>\r\n</ul>\r\nWe will discuss the implications of these items below. To add ArrayFire to existing code you need to:\r\n<ol type=\"1\">\r\n 	<li>Add includes</li>\r\n 	<li>Instruct OpenCL to complete its operations using clFinish (or similar)</li>\r\n 	<li>Instruct ArrayFire to use the user-created OpenCL Context</li>\r\n 	<li>Create ArrayFire arrays from OpenCL memory objects</li>\r\n 	<li>Perform ArrayFire operations on the Arrays</li>\r\n 	<li>Instruct ArrayFire to finish operations using af::sync()</li>\r\n 	<li>Obtain cl_mem references for important memory</li>\r\n 	<li>Continue your OpenCL application</li>\r\n</ol>\r\nTo create the af::array objects, you should use one of the following constructors:\r\n<div class=\"fragment\">\r\n<div class=\"line\">\r\n<pre class=\"lang:default decode:true \">// 1D - 3D af::array constructors\r\nstatic af::array array (dim_t dim0, cl_mem buf, af::dtype type, bool retain=false)\r\nstatic af::array array (dim_t dim0, dim_t dim1, cl_mem buf, af::dtype type, bool retain=false)\r\nstatic af::array array (dim_t dim0, dim_t dim1, dim_t dim2, cl_mem buf, af::dtype type, bool retain=false)\r\nstatic af::array array (dim_t dim0, dim_t dim1, dim_t dim2, dim_t dim3, cl_mem buf, af::dtype type, bool retain=false)\r\n \r\n// af::array constructor using a dim4 object\r\nstatic af::array array (af::dim4 idims, cl_mem buf, af::dtype type, bool retain=false)</pre>\r\n</div>\r\n</div>\r\n<em>NOTE</em>: With all of these constructors, ArrayFire\'s memory manager automatically assumes responsibility for any memory provided to it. If you are creating an array from a <code>cl::Buffer</code>, you should specify <code>retain=true</code> to ensure your memory is not deallocated if your <code>cl::Buffer</code> were to go out of scope. We use this technique in the example below. If you do not wish for ArrayFire to manage your memory, you may call the <code>array::unlock()</code> function and manage the memory yourself; however, if you do so, please be cautious not to call <code>clReleaseMemObj</code> on a <code>cl_mem</code> when ArrayFire might be using it!\r\n\r\nThe eight steps above are best illustrated using a fully-worked example. Below we use the OpenCL 2.0 C++ API and omit error checking to keep the code readable.\r\n<div class=\"fragment\">\r\n<div class=\"line\">\r\n<pre class=\"lang:default decode:true \">#include &lt;CL/cl2.hpp&gt;\r\n \r\n// 1. Add arrayfire.h and af/opencl.h to your application\r\n#include \"arrayfire.h\"\r\n#include \"af/opencl.h\"\r\n \r\n#include &lt;cstdio&gt;\r\n#include &lt;vector&gt;\r\n \r\nint main() {\r\n \r\n// Set up the OpenCL context, device, and queues\r\ncl::Context context(CL_DEVICE_TYPE_ALL);\r\nvector&lt;cl::Device&gt; devices = context.getInfo&lt;CL_CONTEXT_DEVICES&gt;();\r\ncl::Device device = devices[0];\r\ncl::CommandQueue queue(context, device);\r\n \r\n// Create a buffer of size 10 filled with ones, copy it to the device\r\nint length = 10;\r\nvector&lt;float&gt; h_A(length, 1);\r\ncl::Buffer cl_A(context, CL_MEM_READ_WRITE, length * sizeof(float), h_A.data());\r\n \r\n// 2. Instruct OpenCL to complete its operations using clFinish (or similar)\r\nqueue.finish();\r\n \r\n// 3. Instruct ArrayFire to use the user-created context\r\n// First, create a device from the current OpenCL device + context + queue\r\nafcl::addDevice(device(), context(), queue());\r\n// Next switch ArrayFire to the device using the device and context as\r\n// identifiers:\r\nafcl::setDevice(device(), context());\r\n \r\n// 4. Create ArrayFire arrays from OpenCL memory objects\r\naf::array af_A = afcl::array(length, cl_A(), f32, true);\r\n \r\n// 5. Perform ArrayFire operations on the Arrays\r\naf_A = af_A + af::randu(length);\r\n \r\n// NOTE: ArrayFire does not perform the above transaction using in-place memory,\r\n// thus the underlying OpenCL buffers containing the memory containing memory to\r\n// probably have changed\r\n \r\n// 6. Instruct ArrayFire to finish operations using af::sync\r\naf::sync();\r\n \r\n// 7. Obtain cl_mem references for important memory\r\ncl_A = *af_A.device&lt;cl_mem&gt;();\r\n \r\n// 8. Continue your OpenCL application\r\n \r\n// ...\r\n \r\nreturn 0;\r\n}</pre>\r\n</div>\r\n</div>\r\n<h4>Using multiple devices</h4>\r\nIf you are using ArrayFire and OpenCL with multiple devices be sure to use <code>afcl::addDevice</code> to add your custom context + device + queue to ArrayFire\'s device manager. This will let you switch ArrayFire devices using your current<code>cl_device_id</code> and <code>cl_context</code>.','ArrayFire - OpenCL Interoperability','','inherit','closed','closed','','8874-revision-v1','','','2016-12-20 14:21:17','2016-12-20 19:21:17','',8874,'http://sandbox.arrayfire.com/8874-revision-v1/',0,'revision','',0),(9417,18672,'2016-12-20 14:22:52','2016-12-20 19:22:52','In last week\'s \"<a href=\"http://sandbox.arrayfire.com/tag/learning-arrayfire-from-scratch/\">Learning ArrayFire from scratch</a>\" post we discussed how to interface ArrayFire and CUDA to improve your productivity. This week we demonstrate how to mix ArrayFire and OpenCL in your application.\r\n\r\n<!--more-->\r\n\r\nMuch like with CUDA, ArrayFire controls almost all aspects of its execution on OpenCL platforms. ArrayFire manages its own context, queue, memory, and creates custom IDs for devices. As such, most of the interoperability functions focus on reducing potential synchronization conflicts between ArrayFire and OpenCL.\r\n\r\n<strong>NOTE</strong>: This blog post was written against ArrayFire 3.3.x. We intend to implement additional convenience functions in ArrayFire 3.4 that should simplify this process further. Be sure to <a href=\"http://arrayfire.org/docs/interop_cuda.htm\">check the documentation</a> for any updates in future releases.\r\n<h4>Basics</h4>\r\nIt is fairly straightforward to interface ArrayFire with your own custom OpenCL code. ArrayFire provides several functions to ease this process including:\r\n<table class=\"doxtable\">\r\n<tbody>\r\n<tr>\r\n<th>Function</th>\r\n<th>Purpose</th>\r\n</tr>\r\n<tr>\r\n<td><pre>af::array(...)</pre></td>\r\n<td>Construct an ArrayFire array from cl_mem references or cl::Buffer objects</td>\r\n</tr>\r\n<tr>\r\n<td>af::array.device()</td>\r\n<td>Obtain a pointer to the cl_mem reference (implies lock())</td>\r\n</tr>\r\n<tr>\r\n<td>af::array.lock()</td>\r\n<td>Removes ArrayFire\'s control of a cl_mem buffer</td>\r\n</tr>\r\n<tr>\r\n<td>af::array.unlock()</td>\r\n<td>Restore\'s ArrayFire\'s control over a cl_mem buffer</td>\r\n</tr>\r\n<tr>\r\n<td>afcl::getPlatform()</td>\r\n<td>Get ArrayFire\'s current cl_platform</td>\r\n</tr>\r\n<tr>\r\n<td>af::getDevice()</td>\r\n<td>Get the current ArrayFire Device ID</td>\r\n</tr>\r\n<tr>\r\n<td>afcl::getDeviceId()</td>\r\n<td>Get ArrayFire\'s current cl_device_id</td>\r\n</tr>\r\n<tr>\r\n<td>af::setDevice()</td>\r\n<td>Set ArrayFire\'s device from an ArrayFire device ID</td>\r\n</tr>\r\n<tr>\r\n<td>afcl::setDeviceId()</td>\r\n<td>Set ArrayFire\'s device from a cl_device_id</td>\r\n</tr>\r\n<tr>\r\n<td>afcl::setDevice()</td>\r\n<td>Set ArrayFire\'s device from a cl_device_id and cl_context</td>\r\n</tr>\r\n<tr>\r\n<td>afcl::getContext()</td>\r\n<td>Get ArrayFire\'s current cl_context</td>\r\n</tr>\r\n<tr>\r\n<td>afcl::getQueue()</td>\r\n<td>Get ArrayFire\'s current cl_command_queue</td>\r\n</tr>\r\n<tr>\r\n<td>afcl::getDeviceType()</td>\r\n<td>Get the current afcl_device_type</td>\r\n</tr>\r\n</tbody>\r\n</table>\r\nAdditionally, the OpenCL backend permits the programmer to add and remove custom devices from the ArrayFire device manager. These permit you to attach ArrayFire directly to the OpenCL queue used by other portions of your application.\r\n<table class=\"doxtable\">\r\n<tbody>\r\n<tr>\r\n<th>Function</th>\r\n<th>Purpose</th>\r\n</tr>\r\n<tr>\r\n<td>afcl::addDevice()</td>\r\n<td>Add a new device to ArrayFire\'s device manager</td>\r\n</tr>\r\n<tr>\r\n<td>afcl::deleteDevice()</td>\r\n<td>Remove a device from ArrayFire\'s device manager</td>\r\n</tr>\r\n</tbody>\r\n</table>\r\nBelow we provide two worked examples on how ArrayFire can be integrated into new and existing projects.\r\n<h4>Adding custom OpenCL kernels to an existing ArrayFire application</h4>\r\nBy default, ArrayFire manages its own context, queue, memory, and creates custom IDs for devices. Thus there is some bookkeeping that needs to be done to integrate your custom OpenCL kernel.\r\n\r\nIf your kernels can share operate in the same queue as ArrayFire, you should:\r\n<ol type=\"1\">\r\n 	<li>Add an include for <code>af/opencl.h</code> to your project</li>\r\n 	<li>Obtain the OpenCL context, device, and queue used by ArrayFire</li>\r\n 	<li>Obtain cl_mem references to af::array objects</li>\r\n 	<li>Load, build, and use your kernels</li>\r\n 	<li>Return control of af::array memory to ArrayFire</li>\r\n</ol>\r\nNote, ArrayFire uses an in-order queue, thus when ArrayFire and your kernels are operating in the same queue, there is no need to perform any synchronization operations.\r\n\r\nThis process is best illustrated with a fully worked example:\r\n<div class=\"fragment\">\r\n<div class=\"line\">\r\n<pre class=\"lang:default decode:true \">#include &lt;arrayfire.h&gt;\r\n// 1. Add the af/opencl.h include to your project\r\n#include &lt;af/opencl.h&gt;\r\n \r\nint main() {\r\nsize_t length = 10;\r\n \r\n// Create ArrayFire array objects:\r\naf::array A = af::randu(length, f32);\r\naf::array B = af::constant(0, length, f32);\r\n \r\n// ... additional ArrayFire operations here\r\n \r\n// 2. Obtain the device, context, and queue used by ArrayFire\r\nstatic cl_context af_context = afcl::getContext();\r\nstatic cl_device_id af_device_id = afcl::getDeviceId();\r\nstatic cl_command_queue af_queue = afcl::getQueue();\r\n \r\n// 3. Obtain cl_mem references to af::array objects\r\ncl_mem * d_A = A.device&lt;cl_mem&gt;();\r\ncl_mem * d_B = B.device&lt;cl_mem&gt;();\r\n \r\n// 4. Load, build, and use your kernels.\r\n// For the sake of readability, we have omitted error checking.\r\nint status = CL_SUCCESS;\r\n \r\n// A simple copy kernel, uses C++11 syntax for multi-line strings.\r\nconst char * kernel_name = \"copy_kernel\";\r\nconst char * source = R\"(\r\nvoid __kernel\r\ncopy_kernel(__global float * gA, __global float * gB)\r\n{\r\nint id = get_global_id(0);\r\ngB[id] = gA[id];\r\n}\r\n)\";\r\n \r\n// Create the program, build the executable, and extract the entry point\r\n// for the kernel.\r\ncl_program program = clCreateProgramWithSource(af_context, 1, &amp;source, NULL, &amp;status);\r\nstatus = clBuildProgram(program, 1, &amp;af_device_id, NULL, NULL, NULL);\r\ncl_kernel kernel = clCreateKernel(program, kernel_name, &amp;status);\r\n \r\n// Set arguments and launch your kernels\r\nclSetKernelArg(kernel, 0, sizeof(cl_mem), d_A);\r\nclSetKernelArg(kernel, 1, sizeof(cl_mem), d_B);\r\nclEnqueueNDRangeKernel(af_queue, kernel, 1, NULL, &amp;length, NULL, 0, NULL, NULL);\r\n \r\n// 5. Return control of af::array memory to ArrayFire\r\nA.unlock();\r\nB.unlock();\r\n \r\n// ... resume ArrayFire operations\r\n \r\n// Because the device pointers, d_x and d_y, were returned to ArrayFire\'s\r\n// control by the unlock function, there is no need to free them using\r\n// clReleaseMemObject()\r\n \r\nreturn 0;\r\n}</pre>\r\n</div>\r\n</div>\r\nIf your kernels needs to operate in their own OpenCL queue, the process is essentially identical, except you need to instruct ArrayFire to complete its computations using the af::sync() function prior to launching your own kernel and ensure your kernels are complete using <code>clFinish</code> (or similar) commands prior to returning control of the memory to ArrayFire:\r\n<ol type=\"1\">\r\n 	<li>Add an include for <code>af/opencl.h</code> to your project</li>\r\n 	<li>Obtain the OpenCL context, device, and queue used by ArrayFire</li>\r\n 	<li>Obtain cl_mem references to af::array objects</li>\r\n 	<li>Instruct ArrayFire to finish operations using af::sync()</li>\r\n 	<li>Load, build, and use your kernels</li>\r\n 	<li>Instruct OpenCL to finish operations using clFinish() or similar commands.</li>\r\n</ol>\r\n<ol type=\"1\">\r\n 	<li>Return control of af::array memory to ArrayFire</li>\r\n</ol>\r\n<h4>Adding ArrayFire to an existing OpenCL application</h4>\r\nAdding ArrayFire to an existing OpenCL application is slightly more involved and can be somewhat tricky due to several optimizations we implement. The most important are as follows:\r\n<ul>\r\n 	<li>ArrayFire assumes control of all memory provided to it.</li>\r\n 	<li>ArrayFire does not (in general) support in-place memory transactions.</li>\r\n</ul>\r\nWe will discuss the implications of these items below. To add ArrayFire to existing code you need to:\r\n<ol type=\"1\">\r\n 	<li>Add includes</li>\r\n 	<li>Instruct OpenCL to complete its operations using clFinish (or similar)</li>\r\n 	<li>Instruct ArrayFire to use the user-created OpenCL Context</li>\r\n 	<li>Create ArrayFire arrays from OpenCL memory objects</li>\r\n 	<li>Perform ArrayFire operations on the Arrays</li>\r\n 	<li>Instruct ArrayFire to finish operations using af::sync()</li>\r\n 	<li>Obtain cl_mem references for important memory</li>\r\n 	<li>Continue your OpenCL application</li>\r\n</ol>\r\nTo create the af::array objects, you should use one of the following constructors:\r\n<div class=\"fragment\">\r\n<div class=\"line\">\r\n<pre class=\"lang:default decode:true \">// 1D - 3D af::array constructors\r\nstatic af::array array (dim_t dim0, cl_mem buf, af::dtype type, bool retain=false)\r\nstatic af::array array (dim_t dim0, dim_t dim1, cl_mem buf, af::dtype type, bool retain=false)\r\nstatic af::array array (dim_t dim0, dim_t dim1, dim_t dim2, cl_mem buf, af::dtype type, bool retain=false)\r\nstatic af::array array (dim_t dim0, dim_t dim1, dim_t dim2, dim_t dim3, cl_mem buf, af::dtype type, bool retain=false)\r\n \r\n// af::array constructor using a dim4 object\r\nstatic af::array array (af::dim4 idims, cl_mem buf, af::dtype type, bool retain=false)</pre>\r\n</div>\r\n</div>\r\n<em>NOTE</em>: With all of these constructors, ArrayFire\'s memory manager automatically assumes responsibility for any memory provided to it. If you are creating an array from a <code>cl::Buffer</code>, you should specify <code>retain=true</code> to ensure your memory is not deallocated if your <code>cl::Buffer</code> were to go out of scope. We use this technique in the example below. If you do not wish for ArrayFire to manage your memory, you may call the <code>array::unlock()</code> function and manage the memory yourself; however, if you do so, please be cautious not to call <code>clReleaseMemObj</code> on a <code>cl_mem</code> when ArrayFire might be using it!\r\n\r\nThe eight steps above are best illustrated using a fully-worked example. Below we use the OpenCL 2.0 C++ API and omit error checking to keep the code readable.\r\n<div class=\"fragment\">\r\n<div class=\"line\">\r\n<pre class=\"lang:default decode:true \">#include &lt;CL/cl2.hpp&gt;\r\n \r\n// 1. Add arrayfire.h and af/opencl.h to your application\r\n#include \"arrayfire.h\"\r\n#include \"af/opencl.h\"\r\n \r\n#include &lt;cstdio&gt;\r\n#include &lt;vector&gt;\r\n \r\nint main() {\r\n \r\n// Set up the OpenCL context, device, and queues\r\ncl::Context context(CL_DEVICE_TYPE_ALL);\r\nvector&lt;cl::Device&gt; devices = context.getInfo&lt;CL_CONTEXT_DEVICES&gt;();\r\ncl::Device device = devices[0];\r\ncl::CommandQueue queue(context, device);\r\n \r\n// Create a buffer of size 10 filled with ones, copy it to the device\r\nint length = 10;\r\nvector&lt;float&gt; h_A(length, 1);\r\ncl::Buffer cl_A(context, CL_MEM_READ_WRITE, length * sizeof(float), h_A.data());\r\n \r\n// 2. Instruct OpenCL to complete its operations using clFinish (or similar)\r\nqueue.finish();\r\n \r\n// 3. Instruct ArrayFire to use the user-created context\r\n// First, create a device from the current OpenCL device + context + queue\r\nafcl::addDevice(device(), context(), queue());\r\n// Next switch ArrayFire to the device using the device and context as\r\n// identifiers:\r\nafcl::setDevice(device(), context());\r\n \r\n// 4. Create ArrayFire arrays from OpenCL memory objects\r\naf::array af_A = afcl::array(length, cl_A(), f32, true);\r\n \r\n// 5. Perform ArrayFire operations on the Arrays\r\naf_A = af_A + af::randu(length);\r\n \r\n// NOTE: ArrayFire does not perform the above transaction using in-place memory,\r\n// thus the underlying OpenCL buffers containing the memory containing memory to\r\n// probably have changed\r\n \r\n// 6. Instruct ArrayFire to finish operations using af::sync\r\naf::sync();\r\n \r\n// 7. Obtain cl_mem references for important memory\r\ncl_A = *af_A.device&lt;cl_mem&gt;();\r\n \r\n// 8. Continue your OpenCL application\r\n \r\n// ...\r\n \r\nreturn 0;\r\n}</pre>\r\n</div>\r\n</div>\r\n<h4>Using multiple devices</h4>\r\nIf you are using ArrayFire and OpenCL with multiple devices be sure to use <code>afcl::addDevice</code> to add your custom context + device + queue to ArrayFire\'s device manager. This will let you switch ArrayFire devices using your current<code>cl_device_id</code> and <code>cl_context</code>.','ArrayFire - OpenCL Interoperability','','inherit','closed','closed','','8874-revision-v1','','','2016-12-20 14:22:52','2016-12-20 19:22:52','',8874,'http://sandbox.arrayfire.com/8874-revision-v1/',0,'revision','',0),(9418,18672,'2016-12-20 14:23:29','2016-12-20 19:23:29','In last week\'s \"<a href=\"http://sandbox.arrayfire.com/tag/learning-arrayfire-from-scratch/\">Learning ArrayFire from scratch</a>\" post we discussed how to interface ArrayFire and CUDA to improve your productivity. This week we demonstrate how to mix ArrayFire and OpenCL in your application.\r\n\r\n<!--more-->\r\n\r\nMuch like with CUDA, ArrayFire controls almost all aspects of its execution on OpenCL platforms. ArrayFire manages its own context, queue, memory, and creates custom IDs for devices. As such, most of the interoperability functions focus on reducing potential synchronization conflicts between ArrayFire and OpenCL.\r\n\r\n<strong>NOTE</strong>: This blog post was written against ArrayFire 3.3.x. We intend to implement additional convenience functions in ArrayFire 3.4 that should simplify this process further. Be sure to <a href=\"http://arrayfire.org/docs/interop_cuda.htm\">check the documentation</a> for any updates in future releases.\r\n<h4>Basics</h4>\r\nIt is fairly straightforward to interface ArrayFire with your own custom OpenCL code. ArrayFire provides several functions to ease this process including:\r\n<table class=\"doxtable\">\r\n<tbody>\r\n<tr>\r\n<th>Function</th>\r\n<th>Purpose</th>\r\n</tr>\r\n<tr>\r\n<td><code>af::array(...)</code></td>\r\n<td>Construct an ArrayFire array from cl_mem references or cl::Buffer objects</td>\r\n</tr>\r\n<tr>\r\n<td>af::array.device()</td>\r\n<td>Obtain a pointer to the cl_mem reference (implies lock())</td>\r\n</tr>\r\n<tr>\r\n<td>af::array.lock()</td>\r\n<td>Removes ArrayFire\'s control of a cl_mem buffer</td>\r\n</tr>\r\n<tr>\r\n<td>af::array.unlock()</td>\r\n<td>Restore\'s ArrayFire\'s control over a cl_mem buffer</td>\r\n</tr>\r\n<tr>\r\n<td>afcl::getPlatform()</td>\r\n<td>Get ArrayFire\'s current cl_platform</td>\r\n</tr>\r\n<tr>\r\n<td>af::getDevice()</td>\r\n<td>Get the current ArrayFire Device ID</td>\r\n</tr>\r\n<tr>\r\n<td>afcl::getDeviceId()</td>\r\n<td>Get ArrayFire\'s current cl_device_id</td>\r\n</tr>\r\n<tr>\r\n<td>af::setDevice()</td>\r\n<td>Set ArrayFire\'s device from an ArrayFire device ID</td>\r\n</tr>\r\n<tr>\r\n<td>afcl::setDeviceId()</td>\r\n<td>Set ArrayFire\'s device from a cl_device_id</td>\r\n</tr>\r\n<tr>\r\n<td>afcl::setDevice()</td>\r\n<td>Set ArrayFire\'s device from a cl_device_id and cl_context</td>\r\n</tr>\r\n<tr>\r\n<td>afcl::getContext()</td>\r\n<td>Get ArrayFire\'s current cl_context</td>\r\n</tr>\r\n<tr>\r\n<td>afcl::getQueue()</td>\r\n<td>Get ArrayFire\'s current cl_command_queue</td>\r\n</tr>\r\n<tr>\r\n<td>afcl::getDeviceType()</td>\r\n<td>Get the current afcl_device_type</td>\r\n</tr>\r\n</tbody>\r\n</table>\r\nAdditionally, the OpenCL backend permits the programmer to add and remove custom devices from the ArrayFire device manager. These permit you to attach ArrayFire directly to the OpenCL queue used by other portions of your application.\r\n<table class=\"doxtable\">\r\n<tbody>\r\n<tr>\r\n<th>Function</th>\r\n<th>Purpose</th>\r\n</tr>\r\n<tr>\r\n<td>afcl::addDevice()</td>\r\n<td>Add a new device to ArrayFire\'s device manager</td>\r\n</tr>\r\n<tr>\r\n<td>afcl::deleteDevice()</td>\r\n<td>Remove a device from ArrayFire\'s device manager</td>\r\n</tr>\r\n</tbody>\r\n</table>\r\nBelow we provide two worked examples on how ArrayFire can be integrated into new and existing projects.\r\n<h4>Adding custom OpenCL kernels to an existing ArrayFire application</h4>\r\nBy default, ArrayFire manages its own context, queue, memory, and creates custom IDs for devices. Thus there is some bookkeeping that needs to be done to integrate your custom OpenCL kernel.\r\n\r\nIf your kernels can share operate in the same queue as ArrayFire, you should:\r\n<ol type=\"1\">\r\n 	<li>Add an include for <code>af/opencl.h</code> to your project</li>\r\n 	<li>Obtain the OpenCL context, device, and queue used by ArrayFire</li>\r\n 	<li>Obtain cl_mem references to af::array objects</li>\r\n 	<li>Load, build, and use your kernels</li>\r\n 	<li>Return control of af::array memory to ArrayFire</li>\r\n</ol>\r\nNote, ArrayFire uses an in-order queue, thus when ArrayFire and your kernels are operating in the same queue, there is no need to perform any synchronization operations.\r\n\r\nThis process is best illustrated with a fully worked example:\r\n<div class=\"fragment\">\r\n<div class=\"line\">\r\n<pre class=\"lang:default decode:true \">#include &lt;arrayfire.h&gt;\r\n// 1. Add the af/opencl.h include to your project\r\n#include &lt;af/opencl.h&gt;\r\n \r\nint main() {\r\nsize_t length = 10;\r\n \r\n// Create ArrayFire array objects:\r\naf::array A = af::randu(length, f32);\r\naf::array B = af::constant(0, length, f32);\r\n \r\n// ... additional ArrayFire operations here\r\n \r\n// 2. Obtain the device, context, and queue used by ArrayFire\r\nstatic cl_context af_context = afcl::getContext();\r\nstatic cl_device_id af_device_id = afcl::getDeviceId();\r\nstatic cl_command_queue af_queue = afcl::getQueue();\r\n \r\n// 3. Obtain cl_mem references to af::array objects\r\ncl_mem * d_A = A.device&lt;cl_mem&gt;();\r\ncl_mem * d_B = B.device&lt;cl_mem&gt;();\r\n \r\n// 4. Load, build, and use your kernels.\r\n// For the sake of readability, we have omitted error checking.\r\nint status = CL_SUCCESS;\r\n \r\n// A simple copy kernel, uses C++11 syntax for multi-line strings.\r\nconst char * kernel_name = \"copy_kernel\";\r\nconst char * source = R\"(\r\nvoid __kernel\r\ncopy_kernel(__global float * gA, __global float * gB)\r\n{\r\nint id = get_global_id(0);\r\ngB[id] = gA[id];\r\n}\r\n)\";\r\n \r\n// Create the program, build the executable, and extract the entry point\r\n// for the kernel.\r\ncl_program program = clCreateProgramWithSource(af_context, 1, &amp;source, NULL, &amp;status);\r\nstatus = clBuildProgram(program, 1, &amp;af_device_id, NULL, NULL, NULL);\r\ncl_kernel kernel = clCreateKernel(program, kernel_name, &amp;status);\r\n \r\n// Set arguments and launch your kernels\r\nclSetKernelArg(kernel, 0, sizeof(cl_mem), d_A);\r\nclSetKernelArg(kernel, 1, sizeof(cl_mem), d_B);\r\nclEnqueueNDRangeKernel(af_queue, kernel, 1, NULL, &amp;length, NULL, 0, NULL, NULL);\r\n \r\n// 5. Return control of af::array memory to ArrayFire\r\nA.unlock();\r\nB.unlock();\r\n \r\n// ... resume ArrayFire operations\r\n \r\n// Because the device pointers, d_x and d_y, were returned to ArrayFire\'s\r\n// control by the unlock function, there is no need to free them using\r\n// clReleaseMemObject()\r\n \r\nreturn 0;\r\n}</pre>\r\n</div>\r\n</div>\r\nIf your kernels needs to operate in their own OpenCL queue, the process is essentially identical, except you need to instruct ArrayFire to complete its computations using the af::sync() function prior to launching your own kernel and ensure your kernels are complete using <code>clFinish</code> (or similar) commands prior to returning control of the memory to ArrayFire:\r\n<ol type=\"1\">\r\n 	<li>Add an include for <code>af/opencl.h</code> to your project</li>\r\n 	<li>Obtain the OpenCL context, device, and queue used by ArrayFire</li>\r\n 	<li>Obtain cl_mem references to af::array objects</li>\r\n 	<li>Instruct ArrayFire to finish operations using af::sync()</li>\r\n 	<li>Load, build, and use your kernels</li>\r\n 	<li>Instruct OpenCL to finish operations using clFinish() or similar commands.</li>\r\n</ol>\r\n<ol type=\"1\">\r\n 	<li>Return control of af::array memory to ArrayFire</li>\r\n</ol>\r\n<h4>Adding ArrayFire to an existing OpenCL application</h4>\r\nAdding ArrayFire to an existing OpenCL application is slightly more involved and can be somewhat tricky due to several optimizations we implement. The most important are as follows:\r\n<ul>\r\n 	<li>ArrayFire assumes control of all memory provided to it.</li>\r\n 	<li>ArrayFire does not (in general) support in-place memory transactions.</li>\r\n</ul>\r\nWe will discuss the implications of these items below. To add ArrayFire to existing code you need to:\r\n<ol type=\"1\">\r\n 	<li>Add includes</li>\r\n 	<li>Instruct OpenCL to complete its operations using clFinish (or similar)</li>\r\n 	<li>Instruct ArrayFire to use the user-created OpenCL Context</li>\r\n 	<li>Create ArrayFire arrays from OpenCL memory objects</li>\r\n 	<li>Perform ArrayFire operations on the Arrays</li>\r\n 	<li>Instruct ArrayFire to finish operations using af::sync()</li>\r\n 	<li>Obtain cl_mem references for important memory</li>\r\n 	<li>Continue your OpenCL application</li>\r\n</ol>\r\nTo create the af::array objects, you should use one of the following constructors:\r\n<div class=\"fragment\">\r\n<div class=\"line\">\r\n<pre class=\"lang:default decode:true \">// 1D - 3D af::array constructors\r\nstatic af::array array (dim_t dim0, cl_mem buf, af::dtype type, bool retain=false)\r\nstatic af::array array (dim_t dim0, dim_t dim1, cl_mem buf, af::dtype type, bool retain=false)\r\nstatic af::array array (dim_t dim0, dim_t dim1, dim_t dim2, cl_mem buf, af::dtype type, bool retain=false)\r\nstatic af::array array (dim_t dim0, dim_t dim1, dim_t dim2, dim_t dim3, cl_mem buf, af::dtype type, bool retain=false)\r\n \r\n// af::array constructor using a dim4 object\r\nstatic af::array array (af::dim4 idims, cl_mem buf, af::dtype type, bool retain=false)</pre>\r\n</div>\r\n</div>\r\n<em>NOTE</em>: With all of these constructors, ArrayFire\'s memory manager automatically assumes responsibility for any memory provided to it. If you are creating an array from a <code>cl::Buffer</code>, you should specify <code>retain=true</code> to ensure your memory is not deallocated if your <code>cl::Buffer</code> were to go out of scope. We use this technique in the example below. If you do not wish for ArrayFire to manage your memory, you may call the <code>array::unlock()</code> function and manage the memory yourself; however, if you do so, please be cautious not to call <code>clReleaseMemObj</code> on a <code>cl_mem</code> when ArrayFire might be using it!\r\n\r\nThe eight steps above are best illustrated using a fully-worked example. Below we use the OpenCL 2.0 C++ API and omit error checking to keep the code readable.\r\n<div class=\"fragment\">\r\n<div class=\"line\">\r\n<pre class=\"lang:default decode:true \">#include &lt;CL/cl2.hpp&gt;\r\n \r\n// 1. Add arrayfire.h and af/opencl.h to your application\r\n#include \"arrayfire.h\"\r\n#include \"af/opencl.h\"\r\n \r\n#include &lt;cstdio&gt;\r\n#include &lt;vector&gt;\r\n \r\nint main() {\r\n \r\n// Set up the OpenCL context, device, and queues\r\ncl::Context context(CL_DEVICE_TYPE_ALL);\r\nvector&lt;cl::Device&gt; devices = context.getInfo&lt;CL_CONTEXT_DEVICES&gt;();\r\ncl::Device device = devices[0];\r\ncl::CommandQueue queue(context, device);\r\n \r\n// Create a buffer of size 10 filled with ones, copy it to the device\r\nint length = 10;\r\nvector&lt;float&gt; h_A(length, 1);\r\ncl::Buffer cl_A(context, CL_MEM_READ_WRITE, length * sizeof(float), h_A.data());\r\n \r\n// 2. Instruct OpenCL to complete its operations using clFinish (or similar)\r\nqueue.finish();\r\n \r\n// 3. Instruct ArrayFire to use the user-created context\r\n// First, create a device from the current OpenCL device + context + queue\r\nafcl::addDevice(device(), context(), queue());\r\n// Next switch ArrayFire to the device using the device and context as\r\n// identifiers:\r\nafcl::setDevice(device(), context());\r\n \r\n// 4. Create ArrayFire arrays from OpenCL memory objects\r\naf::array af_A = afcl::array(length, cl_A(), f32, true);\r\n \r\n// 5. Perform ArrayFire operations on the Arrays\r\naf_A = af_A + af::randu(length);\r\n \r\n// NOTE: ArrayFire does not perform the above transaction using in-place memory,\r\n// thus the underlying OpenCL buffers containing the memory containing memory to\r\n// probably have changed\r\n \r\n// 6. Instruct ArrayFire to finish operations using af::sync\r\naf::sync();\r\n \r\n// 7. Obtain cl_mem references for important memory\r\ncl_A = *af_A.device&lt;cl_mem&gt;();\r\n \r\n// 8. Continue your OpenCL application\r\n \r\n// ...\r\n \r\nreturn 0;\r\n}</pre>\r\n</div>\r\n</div>\r\n<h4>Using multiple devices</h4>\r\nIf you are using ArrayFire and OpenCL with multiple devices be sure to use <code>afcl::addDevice</code> to add your custom context + device + queue to ArrayFire\'s device manager. This will let you switch ArrayFire devices using your current<code>cl_device_id</code> and <code>cl_context</code>.','ArrayFire - OpenCL Interoperability','','inherit','closed','closed','','8874-revision-v1','','','2016-12-20 14:23:29','2016-12-20 19:23:29','',8874,'http://sandbox.arrayfire.com/8874-revision-v1/',0,'revision','',0),