<p>Programmers and Data Scientists want to take advantage of fast and parallel computational devices. Writing vectorized code is becoming a necessity to get the best performance out of the current generation parallel hardware and scientific computing software. However, writing vectorized code may not be intuitive immediately. Arrayfire provides many ways to vectorize a given code segment. In this tutorial, we will be presenting various ways to vectorize code using ArrayFire and the benefits and drawbacks associated with each method.</p>
<h2><a id="GenericDefault_vectorization_10"></a>Generic/Default vectorization</h2>
<p>By its very nature, Arrayfire is a vectorized library. Most functions operate on arrays as a whole – on all elements in parallel. Wherever possible, existing vectorized functions should be used opposed to manually indexing into arrays. For example, consider this valid, yet mislead code that attempts to increment each element of an array:</p>
<pre><code class="language-{.cpp}">af::<span class="hljs-built_in">af::array a = af::range(10); // [0,  9]
for(int i = 0; i &lt; a.dims(0); ++i)
{
    a(i) = a(i) + 1;         // [1, 10]
}
</code></pre>
<p>Instead, the existing vectorized Arrayfire overload of the + operator should have been used:</p>
<pre><code class="language-{.cpp}">af::array a = af::range(10);  // [0,  9]
a = a + 1;                    // [1, 10]
</code></pre>
<p>Most Arrayfire functions are vectorized. A small subset of these include:</p>
<p><strong>Arithmetic operations:</strong> +, -, *, / , %, &gt;&gt;, &lt;&lt;
<strong>Complex operations:</strong> real(), imag(), conj()…
<strong>Exponential and logarithmic functions:</strong> exp(), log(), expm1(), log1p()…
<strong>Hyperbolic functions:</strong> sinh(), cosh(), tanh()…
<strong>Logical operations:</strong> &amp;&amp;, ||, &lt;, &gt;, ==, !=
<strong>Numeric functions:</strong> abs(), floor(), round(), min(), max()…
<strong>Trigonometric functions:</strong> sin(), cos(), tan()…</p>
<p>Not only elementwise arithmetic operations are vectorized in Arrayfire.</p>
<p>Vector operations such as <strong>min()</strong> support vectorization:</p>
<pre><code class="language-{.cpp}">af::array arr = randn(100);
std::cout &lt;&lt; min(arr) &lt;&lt; std::endl;
</code></pre>
<p>Signal processing functions like <strong>convolve()</strong> support vectorization:</p>
<pre><code class="language-{.cpp}">float g_coef[] = { 1, 2, 1,
                   2, 4, 2,
                   1, 2, 1 };

af::array filter = 1.f/16 * af::array(3, 3, f_coef);

af::array signal = randu(WIDTH, HEIGHT, NUM);
af::array conv = convolve2(signal, filter);
</code></pre>
<p>Image processing functions such as <strong>rotate()</strong> support vectorization:</p>
<pre><code class="language-{.cpp}">af::array imgs = randu(WIDTH, HEIGHT, 100); // 100 (WIDTH x HEIGHT) images
af::array rot_imgs = rotate(imgs, 45); // 100 rotated images
</code></pre>
<p>One class of functions that does not support vectorization is the set of linear algebra functions. Using the built in vectorized operations should be the first and preferred method of vectorizing any code written with Arrayfire.</p>
<h2><a id="GFOR_Parallel_forloops_76"></a>GFOR: Parallel for-loops</h2>
<p>Another novel method of vectorization present in Arrayfire is the GFOR loop replacement construct. GFOR allows launching all iterations of a loop in parallel on the GPU or device, as long as the iterations are independent. While the standard for-loop performs each iteration sequentially, ArrayFire’s gfor-loop performs each iteration at the same time (in parallel). ArrayFire does this by tiling out the values of all loop iterations and then performing computation on those tiles in one pass. You can think of gfor as performing auto-vectorization of your code, e.g. you write a gfor-loop that increments every element of a vector but behind the scenes ArrayFire rewrites it to operate on the entire vector in parallel.</p>
<p>We can remedy our first example with GFOR:</p>
<pre><code class="language-{.cpp}">af::array a = af::range(10);
gfor(seq i, n)
    a(i) = a(i) + 1;</code></pre>
<p>To see another example, you could run an <strong>accum()</strong> on every slice of a matrix in a for-loop, or you could “vectorize” and simply do it all in one gfor-loop operation:</p>
<pre><code class="language-{.cpp}">for (int i = 0; i &lt; N; ++i)
   B(span,i) = accum(A(span,i)); // runs each accum() in sequence
gfor (seq i, N)
   B(span,i) = accum(A(span,i)); // runs N accums in parallel
</code></pre>
<p>However, returning to our previous vectorization technique, <strong>accum()</strong> is already vectorized and the operation could be completely replaced with merely:</p>
<pre><code class="language-{.cpp}">    B = accum(A);
</code></pre>
<p>It is best to vectorize computation as much as possible to avoid the overhead in both for-loops and gfor-loops. However, the gfor-loop construct is most effective in the narrow case of broadcast-style operations. Consider the case when we have a vector of constants that we wish to apply to a collection of variables, such as expressing the values of a linear combination for multiple vectors. The broadcast of one set of constants to many vectors works well with gfor-loops:</p>
<pre><code class="language-{.cpp}">const static int p=4, n=1000;
af::array consts = af::randu(p);
af::array var_terms = randn(p, n);

gfor(seq i, n)
    combination(span, i) = consts * var_terms(span, i);
</code></pre>
<p>Using GFOR requires following several rules and multiple guidelines for optimal performance. The details of this vectorization method can be found in the [GFOR documentation](\ref gfor).</p>
<h2><a id="Batching_127"></a>Batching</h2>
<p>The <strong>batchFunc()</strong> function allows the broad application of existing Arrayfire functions to multiple sets of data. Effectively, batchFunc() allows Arrayfire functions to execute in “batch processing” mode. In this mode, functions will find a dimension which contains “batches” of data to be processed and will parallelize the procedure. Consider the following example:</p>
<pre><code class="language-{.cpp}">af::array filter = randn(1, 5);
af::array weights = randu(5, 5);
</code></pre>
<p>We have a filter that we would like to apply to each of several weights vectors. The naive solution would be using a loop as we’ve seen before:</p>
<pre><code class="language-{.cpp}">af::array filtered_weights = constant(0, 5, 5);
for(int i=0; i&lt;weights.dims(1); ++i){
    filtered_weights.col(i) = filter * weights.col(i);
}
</code></pre>
<p>However we would like a vectorized solution. The following syntax begs to be used:</p>
<pre><code class="language-{.cpp}">af::array filtered_weights = filter * weights; // fails due to dimension mismatch
</code></pre>
<p>but it fails due to the (5x1), (5x5) dimension mismatch. Wouldn’t it be nice if Arrayfire could figure out along which dimension we intend to apply the batch operation? That is exactly what <strong>batchFunc()</strong> does! The signature of the function is:</p>
<p><code>array batchFunc(const array &amp;lhs, const array &amp;rhs, batchFunc_t func);</strong></code></p>
<p>where <strong>batchFunc_t<code><strong></strong> is a function pointer of the form:
<code>typedef array (*batchFunc_t) (const array &amp;lhs, const array &amp;rhs);</code></p>
<p>So, to use <strong>batchFunc()</strong>, we need to provide the function we will be applying as a batch operation. For illustration’s sake, let’s “implement” a multiplication function following the format.</p>
<pre><code class="language-{.cpp}">af::array my_mult (const af::array &amp;lhs, const af::array &amp;rhs){
    return lhs * rhs;
}</code></pre>
<p>Our final batch call is not much more difficult than the ideal syntax we imagined.</p>
<pre><code class="language-{.cpp}">af::array filtered_weights = batchFunc( filter, weights, my_mult );
</code></pre>
<p>The batch function will work with many previously mentioned vectorized Arrayfire functions. It can even work with a combination of those functions if they are wrapped inside a helper function matching the <strong>batchFunc_t</strong> signature. Unfortunately, the batch function cannot be used within a gfor() construct at this moment.</p>
<h2><a id="Advanced_Vectorization_181"></a>Advanced Vectorization</h2>
<p>We have seen the different methods Arrayfire provides to vectorize our code. Tying them all together is a slightly more involved process that needs to consider data dimensionality and layout, memory usage, nesting order, etc. An excellent example and discussion of these factors can be found in another blog post:</p>
<p><a href="http://sandbox.arrayfire.com/how-to-write-vectorized-code/">http://sandbox.arrayfire.com/how-to-write-vectorized-code/</a></p>
<p>It’s worth noting that the whole implementation discussed in the blog has since been transformed into a convenient <strong>af::nearestNeighbour()</strong> function. Before writing something from scratch, check that Arrayfire doesn’t already have an implementation. The default vectorized nature of Arrayfire and an extensive collection of functions will speed things up in addition to replacing dozens of lines of code!</p>

