The increasing availability of high performance image processing hardware, such as GPUs, and quality video sensors has opened up exciting possibilities for video processing. One such area of interest is extracting moving targets from real-time video streams, useful for various computer vision applications such as recognition, tracking, video analysis, and surveillance. It is important to have video processing capabilities that can process such large amounts of data for real-time analysis, as conventional methods are computationally expensive.

&nbsp;

[caption id="attachment_1650" align="aligncenter" width="615" caption="Processing multiple subwindows in parallel on the GPU; each subwindow processing a moving target candidate"]<a href="http://blog.accelereyes.com/blog/wp-content/uploads/2011/08/ngc-gfor-hist-4-e1313092762974.png"><img class="size-full wp-image-1650" title="gfor hist() in action" src="http://blog.accelereyes.com/blog/wp-content/uploads/2011/08/ngc-gfor-hist-4-e1313092762974.png" alt="" width="615" height="377" /></a>[/caption]

&nbsp;

<a href="http://www.northropgrumman.com/" target="_blank">Northrop Grumman Corporation</a> (NYSE: NOC) is a leading global security company with expertise in areas such as unmanned systems, cybersecurity, C4ISR, and logistics. Recently, Northrop Grumman found success using LibJacket to achieve <span style="color: #ff0000;"><em>over <strong>4x</strong> speedup</em></span> of code detecting moving targets in sequenced video frames.

NGC had a video image sequence of size 2000x2000 that needed advanced temporal image processing performed on consecutive frames, but performance on the CPU was currently lacking. Fortunately for NGC, performing image processing on large images is a task well suited for GPUs, and made easy via <a href="http://www.accelereyes.com/libjacket_tour?idx=0" target="_blank">LibJacket</a>.

&nbsp;

[caption id="attachment_1651" align="aligncenter" width="634" caption="Mini example of a gfor histogram pipeline processing multiple subwindows"]<a href="http://blog.accelereyes.com/blog/wp-content/uploads/2011/08/ngc-gfor-hist-s.png"><img class="size-full wp-image-1651" title="example gfor hist() pipeline" src="http://blog.accelereyes.com/blog/wp-content/uploads/2011/08/ngc-gfor-hist-s.png" alt="" width="634" height="356" /></a>[/caption]

&nbsp;

One of the biggest performance boosts NGC gained while modifying their existing C++ code to use LibJacket, came from being able to make good use of LibJacket's <strong><a href="http://wiki.accelereyes.com/wiki/index.php/GFOR_Usage" target="_blank">gfor</a></strong> functionality.

To highlight one particular instance:
<blockquote>Consider the task of extracting hundreds of small subwindows from an image, computing the cumulative histogram for each subwindow, then searching each histogram to find a threshold for that area, all over a 2K x 2K image as fast as possible.</blockquote>
Using LibJacket's <em>gfor-enabled <a href="http://wiki.accelereyes.com/wiki/index.php/Hist" target="_blank">histogram</a></em>, instead of sequentially performing hundreds of small histograms on the CPU as before, all the subwindowed histograms are combined into a single kernel launch on the GPU to run in parallel. This simple change gained <span style="color: #ff0000;"><em>over <strong>8x</strong> speedup</em></span> for this stage of NGC's image processing pipeline, putting the goal of real-time processing within reach.

At the time of writing, about 90% of NGC's existing CPU moving target detection code has been successfully ported to LibJacket, totaling over 4x improvement in processing time so far. We're confidant that NGC will see their performance potential continue to grow.

&nbsp;

&nbsp;