Today we are pleased to announce the release of ArrayFire v3.5, our open source
library of parallel computing functions supporting CUDA, OpenCL, and CPU
devices. This new version of ArrayFire improves features and performance for
applications in machine learning, computer vision, signal processing,
statistics, finance, and more.

This release focuses on thread-safety, support for simple sparse-dense
arithmetic operations, canny edge detector function, and a genetic algorithm
example. A complete list of ArrayFire v3.5 updates and new features are found
in the product <a href="http://arrayfire.org/docs/releasenotes.htm">Release Notes</a>.
<h1><strong>Thread Safety</strong></h1>
ArrayFire now supports threading programming models. This is not intended to
improve the performance since most of the parallelism is happening on the
device, but it does allow you to use multiple devices in an easy way.
With ArrayFire you had to use a for loop to get this functionality. Normally
this isn't a problem but when you have to work with host side transfers, this
becomes troublesome.

Let's suppose we wanted to perform a matrix multiply on multiple GPUs. Because
most ArrayFire functions are asynchronous this can be done in a for loop.
<pre class="lang:c++ decode:true" title="Simple multi-device">for (int i = 0; i &lt; nDevices; i++) {
    setDevice(i);
    array a = randu(1000, 1000);
    array b = constant(3, 1000, 1000);
    array c = matmul(a, b);
}</pre>
&nbsp;

This works well most of the time but this becomes an issue when you want to interact
with host side code in your application.

For example, this code attempts to perform a matrix multiply and then transfer
that data back to the CPU to do additional work. The ArrayFire code executes the
computation on different devices but they will not be queued at the same time
because the data transfer to the c_data array is a blocking operation.
<pre class="lang:c++ decode:true">for(int i = 0; i &lt; nDevices; i++) {
    setDevice(i)
    array a = randu(1000, 1000);
    array b = constant(3, 1000, 1000);
    array c = matmul(a, b);

    // Blocks until the previous operation has completed
    vector&lt;float&gt; c_data(c.elements());
    c.host(c_data.data());

    // Perform some host side side work

    array data(1000, 1000, c_data.data());
    // more work
}</pre>
In order for this to work correctly you would have had to split the loop so that
the matmul operation is queued first then in a separate loop the data is transfered
to the CPU. If more work needs to be performed, you will need to create a third loop
to iterate through the devices again.
<pre class="lang:c++ decode:true">vector&lt;array&gt; a(nDevices);
vector&lt;array&gt; b(nDevices);
vector&lt;array&gt; c(nDevices);
for(int i = 0; i &lt; nDevices; i++) {
    setDevice(i);
    a[i] = randu(1000, 1000);
    b[i] = constant(3, 1000, 1000);
    c[i] = matmul(a, b);
}

vector&lt;vector&lt;float&gt;&gt; c_data(nDevices)
for(int i = 0; i &lt; nDevices; i++) {
    setDevice(i)
    c_data[i] = vector&lt;float&gt;(c.elements());
    c[i].host(c_data[i].data());
}

// Perform some host side side work

for(int i = 0; i &lt; nDevices; i++) {
    setDevice(i);
    array data(1000, 1000, c_data.data());
    // more work
}</pre>
This will perform the operations on each of the GPUs on the machines as expected
but it is troublesome to program like this. With the thread-safety updates in 3.5
you can now execute the operations for each of the GPUs in separate threads.
<pre class="lang:c++ decode:true">vector&lt;thread&gt; threads;
for(int i = 0; i &lt; nDevices; i++) {
  threads.emplace_back([=]() {
    setDevice(i);
    array a = randu(1000, 1000);
    array b = constant(3, 1000, 1000);
    array c = matmul(a, b);

    // Blocks until the previous operation has completed
    vector&lt;float&gt; c_data(c.elements());
    c.host(c_data.data());

    // Perform some host side side work

    array data(1000, 1000, c_data.data());
    // more work
  });
}
</pre>
This is a much easier to implement using threads then the for loop approach.
<h1><strong>Canny edge detector</strong></h1>
We have added the Canny edge detection algorithm to ArrayFire. The Canny
algorithm is a multi-stage algorithm which can handle a wide variety of edges
The canny algorithm can automatically calculate the thresholds
using Otsu's method.
<h1><strong>Sparse-Dense arithmetic</strong></h1>
ArrayFire 3.4 introduced initial support for sparse arrays. With 3.5 we are
expanding the sparse arrays to allow for sparse-dense arithmetic.
<h1><strong>Support for CLBlast</strong></h1>
With ArrayFire 3.5 we are introducing support for a new OpenCL BLAS
implementation. It is developed by Cedric Nugteren and it is called CLBlast. It
has improved performance on NVIDIA hardware and features an autotuner support
devices that have not been explicitly been optimized. You can read more about
clBlast at the GitHub repository <a href="https://github.com/CNugteren/CLBlast">here</a>.
<h1><strong>Improved CUDA JIT engine</strong></h1>
The CUDA JIT engine has been refactored to use the NVRTC library. Previously the
JIT functionality was implemented using the NVVM library which made it difficult
to employ more advanced control flows. With NVRTC we can create more advanced
JIT functions to improve the performance.

As always we are working on improving the performance of all of our functions. This
has been another exciting update to ArrayFire and we have great plans for the next
release. Stay tuned!