Today we are pleased to announce the release of ArrayFire v3.5, our open source
library of parallel computing functions supporting CUDA, OpenCL, and CPU
devices. This new version of ArrayFire improves features and performance for
applications in machine learning, computer vision, signal processing,
statistics, finance, and more.

This release focuses on thread-safety, support for simple sparse-dense
arithmetic operations, canny edge detector function, and a genetic algorithm
example. A complete list of ArrayFire v3.5 updates and new features can be found
in the product <a href="http://arrayfire.org/docs/releasenotes.htm">Release Notes</a>.
# Thread Safety

ArrayFire now supports threading programming models. This is not intended to
improve the performance since most of the parallelism is happening on the
device, but it does allow you to easily use multiple devices in an easy way.
With ArrayFire you had to use a for loop to get this functionallity. Normally
this isn't a problem but when you have to work with host side transfers, this
becomes troublesome.

Lets suppose we wanted to perform a matrix multipley on muiltiple GPUs. Because
most ArrayFire functions are asynchronous this can be done in a for loop.

#+BEGIN_SRC cpp
for (int i = 0; i &lt; nDevices; i++) {
setDevice(i);
array a = randu(1000, 1000);
array b = constant(3, 1000, 1000);
array c = matmul(a, b);
}
#+END_SRC

This works well most of the time but this becomes an issue when you want to interact
with host side code in your application.

For example this code attempts to perform a matrix multiply and then transfer
that data back to the CPU to do additional work. The ArrayFire code execute the
computation on different devices but they will not be queued at the same time
because the data transfer to the c_data array is a blocking operation.

#+BEGIN_SRC cpp
for(int i = 0; i &lt; nDevices; i++) {
setDevice(i)
array a = randu(1000, 1000);
array b = constant(3, 1000, 1000);
array c = matmul(a, b);

// Blocks until the previous operation has completed
vector&lt;float&gt; c_data(c.elements());
c.host(c_data.data());

// Perform some host side side work

array data(1000, 1000, c_data.data());
// more work
}
#+END_SRC

In order for this to work correctly you would have had to split the loop so that
the matmul operation is queued first then in a separate loop the data is transfered
to the CPU. If more work needs to be performed, you will need to create a third loop
to iterate through the devices again.

#+BEGIN_SRC cpp
vector&lt;array&gt; a(nDevices);
vector&lt;array&gt; b(nDevices);
vector&lt;array&gt; c(nDevices);
for(int i = 0; i &lt; nDevices; i++) {
setDevice(i);
a[i] = randu(1000, 1000);
b[i] = constant(3, 1000, 1000);
c[i] = matmul(a, b);
}

vector&lt;vector&lt;float&gt;&gt; c_data(nDevices)
for(int i = 0; i &lt; nDevices; i++) {
setDevice(i)
c_data[i] = vector&lt;float&gt;(c.elements());
c[i].host(c_data[i].data());
}

// Perform some host side side work

for(int i = 0; i &lt; nDevices; i++) {
setDevice(i);
array data(1000, 1000, c_data.data());
// more work
}
#+END_SRC

This will perform the operations on each of the GPUs on the machines as expedted
but it is troublesome to program like this. With the thread-safety updates in 3.5
you can now execute the operations for each of the GPUs in separate threads.

#+BEGIN_SRC cpp

vector&lt;thread&gt; threads;
for(int i = 0; i &lt; nDevices; i++) {
threads.emplace_back([=]() {
setDevice(i);
array a = randu(1000, 1000);
array b = constant(3, 1000, 1000);
array c = matmul(a, b);

// Blocks until the previous operation has completed
vector&lt;float&gt; c_data(c.elements());
c.host(c_data.data());

// Perform some host side side work

array data(1000, 1000, c_data.data());
// more work
});
}
#+END_SRC

This is a much easier to implement using threads then the for loop approach.

* Canny edge detector

We have added the canny edge detection algoirthms to ArrayFire. The Canny
algoirthms is a multi-stage algorithm which is customizable to handle many types
of situations. The canny algorithm can automatically calculate the thresholds
using Otsu's method.

* Sparse Dense arithmetic

ArrayFire 3.4 introduced initial support for sparse arrays. With 3.5 we are
expanding the sparse arrays to allow for sparse dense arithmetic.

* Support for CLBlast

With ArrayFire 3.5 we are introducing support for a new OpenCL BLAS
implementation. It is developed by Cedric Nugteren and it is called CLBlast. It
has improved performance on NVIDIA hardware and features an autotuner support
devices that have not been explicitly been optimized. You can read more about
clBlast at the GitHub repository [[https://github.com/CNugteren/CLBlast][here]].

* Improved CUDA JIT engine

The CUDA JIT engine has been refactored to use the NVRTC library. Previously the
JIT functionallity was implemented using the NVVM library which made it difficult
to implement more advanced control flows. With NVRTC we can create more advanced
JIT functions to improve the performance.

As always we are working on improving the performance of all of our functions. This
has been another exciting release of ArrayFire and we have great plans for the next
release. Stay tuned!