<p>
ArrayFire uses <a href="http://sandbox.arrayfire.com/performance-of-arrayfire-jit-code-generation/">Just In Time</a> compilation to combine many light weight functions into a single kernel launch. This along with our <a href="http://arrayfire.org/docs/group__arrayfire__func.htm">easy-to-use API</a> allows users to not only quickly prototype their algorithms, but also get the best out of the underlying hardware.
</p>

<p>
This feature has been a favorite among our users in the domains of finance and scientific simulation.
</p>

<p>
That said, ArrayFire v3.3 and earlier had a few limitations. Namely:
</p>
<ul class="org-ul">
<li>Multiple outputs with inter-dependent variables were generating multiple kernels.
</li>
<li>The number of operations per kernel was fairly limited by default.
</li>
</ul>

<p>
In the latest release of ArrayFire, we addressed these issues to get some pretty impressive numbers.
</p>

<p>
In the rest of the post, we demonstrate the performance improvements using our <a href="https://github.com/arrayfire/arrayfire/blob/devel/examples/financial/black_scholes_options.cpp">BlackScholes example</a>. For this particular example, the improvements for the CUDA backend were <b>~8x</b> and OpenCL backend were between <b>~5x</b>.
</p>

<div id="outline-container-sec-1" class="outline-2">
<h3 id="sec-1">Performance Comparison to v3.3</h3>
<div class="outline-text-2" id="text-1">
<p>
<a href="https://en.wikipedia.org/wiki/Black%25E2%2580%2593Scholes_model">BlackScholes</a> is a commonly used mathematical model in financial markets.
</p>

<p>
It takes the following 5 inputs and generates 2 outputs.
</p>

<p>
The code to implement this model in arrayfire can be seen below.
</p>

<div class="org-src-container">

<pre class="src src-C++">array cnd(array x)
{
    // constants
    const float a1 =  0.254829592;
    const float a2 = -0.284496736;
    const float a3 =  1.421413741;
    const float a4 = -1.453152027;
    const float a5 =  1.061405429;
    const float p  =  0.3275911;
    const float sqrt2 = sqrt(2.0);

    // Save the sign of x
    array xSign = sign(x);

    x = abs(x) / sqrt2;

    // A&amp;S formula 7.1.26
    array t = 1.0f / (1.0f + p*x);
    array y = 1.0f + 0.5f * (((((a5*t + a4)*t) + a3)*t + a2)*t + a1)*t*exp(-x*x);

    return xSign * y + !xSign * (1 - y); // equivalent of (x &gt;= 0) ? y : (1 - y);
}

static void black_scholes(array&amp; C, array&amp; P,
                          const array&amp; S, const array&amp; X,
                          const array&amp; R, const array&amp; V,
                          const array&amp; T)
{
    // Inputs:
    // S: Stock price
    // X: Strike price
    // R: Risk free rate of interest
    // V: Volatility
    // T: Time to maturity

    // Outputs:
    // C: Call option
    // P: Put option

    array d1 = log(S / X);
    d1 = d1 + (R + (V*V)*0.5) * T;
    d1 = d1 / (V*sqrt(T));

    array d2 = d1 - (V*sqrt(T));

    array cnd_d1 = cnd(d1);
    array cnd_d2 = cnd(d2);

    C = S * cnd_d1  - (X * exp((-R)*T) * cnd_d2);
    P = X * exp((-R)*T) * (1 - cnd_d2) - (S * (1 - cnd_d1));
}
</pre>
</div>

<p>
The <code>black_scholes</code> function contains a long chain of operations and generates two outputs. In older versions of ArrayFire, the Just-In-Time compiler of arrayfire was splitting up this single function call to multiple kernels.
</p>

<p>
We solved this in the latest release by:
</p>
<ul class="org-ul">
<li>Enabling <code>af::eval</code> to hint ArrayFire's JIT compiler to combine inputs.
</li>
<li>Changing the heuristics to increase the number of operations per kernel.
</li>
</ul>

<div class="org-src-container">

<pre class="src src-C++">black_scholes(C, P, S, X, R, V, T);
eval(C, P); // Only change required for v3.4
</pre>
</div>



These changes resulted in dramatic performance improvements seen below. The following graph compares the number of call and put options being calculated per second on v3.3 and v3.4.

<br>

<iframe width="650" height="400" seamless frameborder="0" scrolling="no"
src="https://docs.google.com/spreadsheets/d/1o4C5KZ4B5he3Ac8mJ9Z6Z_N-hlATeJwhP-j6XJtbEPA/pubchart?oid=927257953&amp;format=interactive">
</iframe>

<br>

As we can see, the performance has improved significantly. The absolute time taken when to calculate 2 million options is show below.

<br>
<br>

<iframe width="513.5" height="104.0141666666666" seamless frameborder="0" scrolling="no" src="https://docs.google.com/spreadsheets/d/1o4C5KZ4B5he3Ac8mJ9Z6Z_N-hlATeJwhP-j6XJtbEPA/pubchart?oid=1295781261&amp;format=interactive"></iframe>

<p>
Note:
</p>
<ul class="org-ul">
<li>The additional gains made by CUDA are because of a <a href="https://github.com/arrayfire/arrayfire/pull/1534">separate issue</a> involving CUDA JIT that was fixed.
</li>
<li>Only 24 threads of the 32 available were used for the Xeon CPU.
</li>
</ul>
</div>
</div>

<div id="outline-container-sec-2" class="outline-2">
<h3 id="sec-2">Performance Comparison to Native CUDA</h3>
<div class="outline-text-2" id="text-2">
<p>
To see how close we can get to the peak performance, we compared the performance of ArrayFire and CUDA on the new GTX 1080.
</p>

<p>
To do this we built ArrayFire 3.4.0 using the CUDA 8 RC and compared its performance with that of the example provided in <a href="http://docs.nvidia.com/cuda/cuda-samples/#black-scholes-option-pricing">CUDA Samples</a>.
</p>


The results of this benchmark can be seen below.

<iframe width="655.1414131325375" height="323.5" seamless frameborder="0" scrolling="no" src="https://docs.google.com/spreadsheets/d/1o4C5KZ4B5he3Ac8mJ9Z6Z_N-hlATeJwhP-j6XJtbEPA/pubchart?oid=256763192&amp;format=interactive"></iframe>

<p>
The results indicate ArrayFire v3.4 performs identical to the native kernel at larger simulation sizes, but there is a constant overhead causing issues at smaller sizes. This is one of the issues we want to address in future releases.
</p>

<p>
Note: The <code>black_scholes</code> function had to be changed slightly as shown below to be similar to the sample provided by CUDA.
</p>
</div>
</div>

<div id="outline-container-sec-3" class="outline-2">
<h3 id="sec-3">Conclusion</h3>
<div class="outline-text-2" id="text-3">
<p>
The new improvements to the JIT engine in ArrayFire can seamlessly generate kernels that perform close to handwritten native kernels on GPUs. There is still scope for improvements at smaller sizes. To see what lies ahead for our Just-In-Time compiler, please check out <a href="https://github.com/arrayfire/arrayfire/issues/1401">this github issue</a>.
</p>
</div>
</div>
</div>
