<p>I will start with an assumption that whoever is reading this has knowledge of writing basic OpenCL code on desktop and knows how to use JNI for android.</p>
<p>My goal is to do some image processing on camera feed. There are couple of ways to generate c headers for jni.</p>
<ul>
<li>Use <a href="http://docs.oracle.com/javase/7/docs/technotes/tools/solaris/javah.html">javah</a> and generate the headers automatically based on the java code.<br /></li>
<li>Write it yourself and avoid typos!</li>
</ul>
<p>I took the second route. After a bit of playing with the tedious naming convention for jni functions, now i use the following macro to create my function declarations.</p>
<pre class="sourceCode c"><code class="sourceCode c"><span class="ot">#define DECLARE(returnType,fullClassName,func,...) \</span>
JNIEXPORT returnType JNICALL Java_##fullClassName##_##func(JNIEnv *env, jclass clazz,__VA_ARGS__);</code></pre>
<p>An example declaration of function that takes <a href="http://www.fourcc.org/yuv.php#NV21">NV21</a> format data as input and writes output to bitmap will be as follows:</p>
<pre class="sourceCode c"><code class="sourceCode c">DECLARE(<span class="dt">void</span>,com_example_CameraPreview,runfilter,jobject outBmp, jbyteArray inData, jint width, jint height)</code></pre>
<p>I like it this way because it makes my header more readable than it will be if generated using javah.</p>
<p>Unless you have multiple functions that share same number of input parameters and identical logic, it is hard to abstract the actual code using a macro. In anycase, i don't like macros with logic as it makes life hard to debug later. So, I had to write the full signature when implementing the function.</p>
<pre class="sourceCode c"><code class="sourceCode c">JNIEXPORT <span class="dt">void</span> JNICALL Java_com_example_CameraPreview_runfilter(
                JNIEnv      *env,
                jclass      clazz,
                jobject     outBmp,
                jByteArray  inData,
                jint        width,
                jint        height,
                jint        choice)
{
    <span class="co">// TODO: process the input and write the output</span>
}</code></pre>
<p>Click <a href="https://github.com/arrayfire/droidcl/blob/master/jni/processor.h">processor.h</a> to see the full source code for function declarations. I have included couple of extra android related headers which are required to read bitmap files and print debug messages from jni routines. The c-string pointed by macro <code>app_name</code> can be used to prune the output of command <code>adb logcat</code> to get only the messages that are associated with our jni routines. All you would need to do is to pass on the value of <code>app_name</code> to adb as an option as shown below.</p>
<pre class="sourceCode bash"><code class="sourceCode bash"><span class="kw">adb</span> logcat -s <span class="st">&quot;JNIProcessor&quot;</span></code></pre>
<p>This is quite useful when you are debugging problems that are related to accuracy of results.</p>
<p>Click <a href="https://github.com/arrayfire/droidcl/blob/master/jni/processor.cpp">processor.cpp</a> to see the full source code for function implementations. Following is some brief descriptions of the android/jni functions i used in the code. I have also added couple of functions of my own to make it look better and i hope it does :)</p>
<ul>
<li><code>AndroidBitmap_getInfo</code> - used to get bitmap details like width, height, stride, format.<br /></li>
<li><code>AndroidBitmap_lockPixels</code> - used to get a pointer to the memory location of image data. Once a lockPixels is called, it has to be followed by a unlockPixels at some point later in the code. Probably creates a copy of original bitmap data which the jvm needs to free up when we are done using it.<br /></li>
<li><code>GetPrimitiveArrayCritical</code> - As the name suggets, this is some kind of performance critical call. Hence, we should take care of the code that runs between a Get and Release calls runs as fast as possible and doesn't stall execution. As per <a href="http://docs.oracle.com/javase/6/docs/technotes/guides/jni/spec/functions.html">Oracle's java documenatation</a>, a VM may temporarily disable garbage collection when the native code is holding a pointer to an array obtained via GetPrimitiveArrayCritical. So, just be careful when you use this function.<br /></li>
<li><code>ReleasePrimitiveArrayCritical</code> - used to release the memory space requested by previous GetPrimitiveArrayCritical. call.<br /></li>
<li><code>AndroidBitmap_unlockPixels</code> - used to release memory space used by a previous lockPixels call.</li>
</ul>
<p>Listed below are few other auxilary functions that i needed to make the code look a little cleaner.</p>
<ul>
<li><code>file_contents</code> - for reading opencl kernel file from disk<br /></li>
<li><code>throwJavaException</code> - convenience function used to throw java exceptions from within jni routines.<br /></li>
<li><code>cb</code> - callback function provided when opencl kernels are built. When you actually look at this, you might wonder why did i do this part alone using c-interface of opencl. I just wanted to try it out, hence it happened so.<br /></li>
<li><code>Java_com_example_LiveFeatureActivity_compileKernels</code> - as you might have noticed, this is a jni routine i used to compile the opencl kernels only once during the app start up phase.<br /></li>
<li><code>helper</code> - used to abstract out opencl code from other android/jni code.</li>
</ul>
<h1 id="opencl">OpenCL</h1>
<p>My target was to run the application on Adreno GPU's for which Qualcomm provides OpenCL library, so the one problem was solved. But, where do i get the headers!! ? I initially thought i could use the OpenCL headers available on khronos website, however that turned out to be disaster. I faced OpenGL header inclusion compilation errors. Sony's opencl <a href="http://developer.sonymobile.com/knowledge-base/tutorials/android_tutorial/boost-the-performance-of-your-android-app-with-opencl/">example</a> helped me a bit here. Their example has modified verion of opencl headers that commented out all OpenGL related API of OpenCL, which is what we need exactly. Thanks to them, now i have both headers and lib file. What do i need to do now?</p>
<p>Two steps:</p>
<ul>
<li>Convert frame data provided by android camera from NV21 format to RGBA8888 format.<br /></li>
<li>Run laplacian edge detection operator on the RGBA8888 image data.</li>
</ul>
<h2 id="nv21-to-rgba888">NV21 to RGBA888</h2>
<p>Getting a grayscale image from NV21 format is cake walk, just read the first width*height bytes and you are done. As per the format of <a href="http://www.fourcc.org/yuv.php#NV21">NV21</a>, for each 2x2 sub-matrix of pixels there will be a pair of v, u in that order. Since 4 pixels are reading same u,v, i decided to use shared memory to reduce reading latency. Below code stub is core of this conversion kernel that i hope sped up the conversion in comparison to a pure c verion.</p>
<pre class="sourceCode c"><code class="sourceCode c"><span class="dt">int</span> inIdx= gy*im_width+gx;
<span class="dt">int</span> uvIdx= off + (gy/<span class="dv">2</span>)*im_width + (gx &amp; ~<span class="dv">1</span>);   <span class="co">// took me a while to figure this stupid formula</span>
<span class="dt">int</span> shlx = lx/<span class="dv">2</span>;
<span class="dt">int</span> shly = ly/<span class="dv">2</span>;
<span class="dt">int</span> shIdx= <span class="dv">2</span>*(shlx+shly*DSHRD_LEN);
<span class="kw">if</span>( gx%<span class="dv">2</span>==<span class="dv">0</span> &amp;&amp; gy%<span class="dv">2</span>==<span class="dv">0</span> ) {
    uvShrd[shIdx<span class="dv">+0</span>] = in[uvIdx<span class="dv">+0</span>];
    uvShrd[shIdx<span class="dv">+1</span>] = in[uvIdx<span class="dv">+1</span>];
}
<span class="dt">int</span> y   = (<span class="bn">0xFF</span> &amp; ((<span class="dt">int</span>)in[inIdx]));
<span class="co">//..... rest of code that does color space conversion</span></code></pre>
<p>Laplacian kernel is pretty straight forward. I brought the decoded frame into shared memory, calculated the result of laplacian operator using pixel values of all 8 neighbors from shared memory and finally write the result to output buffer. You can find the entire kernel source code <a href="https://github.com/arrayfire/droidcl/blob/master/assets/kernels.cl">here</a>.</p>
<h1 id="build-for-linux">Build (for Linux)</h1>
<p>blah blah blah, i can't use raw code!. Lets build it. Android-ndk provides a very useful tool <code>ndk-build</code>. We need to setup some stuff to run the build command <code>ndk-build</code>.</p>
<p>The entire example app is avaiable on github at <a href="https://github.com/arrayfire/droidcl">template</a>.</p>
<ol>
<li>Do <code>git clone https://github.com/arrayfire/droidcl</code><br /></li>
<li>Do <code>cd droidcl</code><br /></li>
<li>Ensure android-ndk installed and environment variable <code>ANDROID_NDK</code> is pointing to ndk installation root.<br /></li>
<li>Ensure environment variable <code>PATH</code> has the path <code>$ANDROID_NDK</code> appened to it.<br /></li>
<li>Do <code>ndk-build</code> from project root.<br /></li>
<li>If you have <code>eclipse</code> installed, then import the project from eclipse and run it as usually.<br /></li>
<li>If you do not have eclipse or don't want to use it, then i assume you might already know about apache-ant tool. Just do <code>ant debug install</code></li>
</ol>
<h2 id="android.mk">Android.mk</h2>
<p>Click <a href="https://github.com/arrayfire/droidcl/blob/master/jni/Android.mk">here</a> to see full content of Android.mk. There are couple of things that you need to do to use opencl on android apart from standard jni mk instructions.</p>
<ul>
<li>Ensure that you do <code>LOCAL_ARM_MODE := arm</code>, which enables you to build for 32-bit ARM architecture instead of the default 16-bit.<br /></li>
<li>Set the variable <code>LOCAL_CPPFLAGS</code> has been set to point to OpenCL headers.<br /></li>
<li>Add <code>libOpenCL.so</code> dependency to <code>LOCAL_LDLIBS</code> variable.<br />Note: In case you want a static library, do the following <code>include $(BUILD_STATIC_LIBRARY)</code> instead of <code>include $(BUILD_SHARED_LIBRARY)</code>.</li>
</ul>
<h2 id="application.mk">Application.mk</h2>
<p>This file is not mandatory i belive, but you may need it in cases where you want specify your requirements about the system libraries you want to use in your application. Click <a href="https://github.com/arrayfire/droidcl/blob/master/jni/Application.mk">here</a> to see full content of Application.mk. Declare that you want to use the static version of gnustl by doing <code>APP_STL := gnustl_static</code>. Also, set you target android api version using the variable <code>APP_PLATFORM</code>.</p>

[video width="1280" height="720" mp4="http://sandbox.arrayfire.com/wp-content/uploads/2014/05/ocl_demo.mp4"]Screen capture of the live app[/video]