A few weeks ago we added some computer vision functionality to our open source <a href="http://sandbox.arrayfire.com/download">ArrayFire GPU computing</a> library. Specifically, we implemented the <a href="http://www.springerlink.com/index/10.1007/11744023">FAST</a> feature extractor, <a href="http://www.springerlink.com/index/10.1007/978-3-642-15561-1">BRIEF</a> feature point descriptor, <a href="http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=6126544">ORB</a> multi-resolution scale invariant feature extractor, and a <a href="http://en.wikipedia.org/wiki/Hamming_distance">Hamming distance</a> function. When combined, these functions enable you to find features in videos (or images) and track them between successive frames.

Here is a quick video showing these functions executing on a few frames from the <a href="http://www.blender.org/">Blender Foundation</a>'s <a href="http://www.bigbuckbunny.org">Big Buck Bunny</a> video:

<iframe width="854" height="510" src="https://www.youtube.com/embed/sHHvORwgFQ4" frameborder="0" allowfullscreen="allowfullscreen"></iframe>

Now that you have seen the functionality in action, how do you use these in your code? Assuming that you have a suitable function that can load your video frames in to ArrayFire "array" objects, it's pretty simple:
<pre><code><span class="hljs-comment">// Run FAST on a frame and get the features:</span>
af::features fast_features = af::fast(frame, <span class="hljs-number">20</span>, <span class="hljs-number">9</span>, <span class="hljs-number">1</span>, <span class="hljs-number">0.05f</span>);
</code></pre>
The `af::features` object is a special structure which contains the `(x,y)` location of the features, their scores, and the number of features. To get access to this information simply use the `getX()`, `getY()`, `getScore()`, and `getNumFeatures()` functions:
<pre><code>af::<span class="hljs-built_in">array</span> x_pos = fast_features.getX();
af::<span class="hljs-built_in">array</span> y_pos = fast_features.getY();
af::<span class="hljs-built_in">array</span> scores = fast_features.getScore();
<span class="hljs-keyword">int</span> N = fast_features.getNumFeatures();
</code></pre>
ORB has a slightly different syntax:
<pre><code><span class="hljs-comment">// Run ORB on a frame and get the features and descriptors</span>
af::features orb_features;
af::<span class="hljs-built_in">array</span> orb_descriptors;
af::orb(orb_features, orb_descriptors, frame, <span class="hljs-number">20</span>, <span class="hljs-number">1000</span>, <span class="hljs-number">1.2</span>, <span class="hljs-number">1</span>, <span class="hljs-keyword">false</span>);</code></pre>
The features from `ORB` may be accessed in the same fashion as `FAST` described above. The `orb_descriptors` variable is a Nx8 array containing extracted descriptors (read the <a href="http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=6126544">ORB paper</a> for further details on this structure).

The Hamming matcher I mention in the video is almost ready to be committed to our upstream development branch, so it is also worth mentioning how it can be used. Assuming you have a list of features from two sources called `a_features` and `b_features` respectively, you can compute their Hamming distance using ArrayFire as follows:
<pre><code>af::<span class="hljs-built_in">array</span> idx;
af::<span class="hljs-built_in">array</span> distances;
af::<span class="hljs-built_in">array</span> a_features;
af::<span class="hljs-built_in">array</span> b_features;
af::hamming_matcher(idx, distances, a_features, b_features, <span class="hljs-number">0</span>, <span class="hljs-number">1</span>);</code></pre>
Then you may get the indicies of features whose distance is less than, say 50, using ArrayFire's `where` operator. After you have these inidices, you can pick out these near-matching features pretty easily:
<pre><code>// select features with distances less than <span class="hljs-number">50</span>:
af::array near_matches = where(dist &lt; <span class="hljs-number">50</span>);
// and now extract information about those features:
array a_feat_x = a_features.getX()(near_matches)
array a_feat_y = a_features.getY()(near_matches)
<span class="hljs-keyword">...</span></code></pre>
Because these are ArrayFire array objects, they exist in your GPU's (or other accelerator device's) memory. To get access to them within your host code, you need to get a copy of the data and place it somewhere in RAM:
<pre><code><span class="hljs-keyword">float</span> * features_x = a_feat_x.host&lt;<span class="hljs-keyword">float</span>&gt;();
<span class="hljs-keyword">float</span> * features_y = a_feat_y.host&lt;<span class="hljs-keyword">float</span>&gt;();</code></pre>