Researchers at the Stanford Artificial Intelligence Laboratory (SAIL) have had more success (building on <a href="http://blog.accelereyes.com/blog/2011/04/09/feature_learning_architectures/">previous work</a>) using Jacket to speed up their algorithm.

In a paper at this year's CVPR 2011, entitled “<a href="http://ai.stanford.edu/~quocle/LeZouYeungNg11.pdf" target="_blank">Learning hierarchical invariant spatio-temporal features for action recognition with independent subspace analysis</a>”, they explain how their unsupervised feature learning algorithm competes with other algorithms that are hand crafted or use learned features.
<table width="576" cellspacing="0" cellpadding="7"><colgroup> <col width="100" /> <col width="101" /> <col width="101" /> <col width="101" /> <col width="100" /> </colgroup>
<tbody>
<tr valign="TOP">
<td bgcolor="#ffffff" width="100"></td>
<td bgcolor="#ffffff" width="101">
<p align="CENTER"><span style="color: #000000;"><strong>KTH</strong></span></p>
</td>
<td bgcolor="#ffffff" width="101">
<p align="CENTER"><span style="color: #000000;"><strong>Hollywood2</strong></span></p>
</td>
<td bgcolor="#ffffff" width="101">
<p align="CENTER"><span style="color: #000000;"><strong>UCF</strong></span></p>
</td>
<td bgcolor="#ffffff" width="100">
<p align="CENTER"><span style="color: #000000;"><strong>Youtube</strong></span></p>
</td>
</tr>
<tr valign="TOP">
<td bgcolor="#ffffff" width="100">
<p align="CENTER"><span style="color: #000000;"><em>Best published Results</em></span></p>
</td>
<td bgcolor="#ffffff" width="101">
<p align="CENTER"><span style="color: #000000;">92.1%</span></p>
</td>
<td bgcolor="#ffffff" width="101">
<p align="CENTER"><span style="color: #000000;">50.9%</span></p>
</td>
<td bgcolor="#ffffff" width="101">
<p align="CENTER"><span style="color: #000000;">85.6%</span></p>
</td>
<td bgcolor="#ffffff" width="100">
<p align="CENTER"><span style="color: #000000;">71.2%</span></p>
</td>
</tr>
<tr valign="TOP">
<td bgcolor="#ffffff" width="100">
<p align="CENTER"><span style="color: #000000;"><em>Stanford group Results</em></span></p>
</td>
<td bgcolor="#ffffff" width="101">
<p align="CENTER"><span style="color: #000000;">93.9%</span></p>
</td>
<td bgcolor="#ffffff" width="101">
<p align="CENTER"><span style="color: #000000;">53.3%</span></p>
</td>
<td bgcolor="#ffffff" width="101">
<p align="CENTER"><span style="color: #000000;">86.5%</span></p>
</td>
<td bgcolor="#ffffff" width="100">
<p align="CENTER"><span style="color: #000000;">75.8%</span></p>
</td>
</tr>
</tbody>
</table>
Testing their algorithm on four well-known benchmark datasets, they were able to achieve better performance than existing results that have been published so far.

For their training purposes, they used a multi-layered stacked convolutional ISA (Independent subspace analysis) network. An ISA is used for learning features from image patches without supervision.

[caption id="attachment_1489" align="aligncenter" width="404" caption="The architecture of an Independent Subspace Analysis network"]<a href="http://blog.accelereyes.com/blog/wp-content/uploads/2011/07/ISA.png"><img class="size-full wp-image-1489  " title="ISA" src="http://blog.accelereyes.com/blog/wp-content/uploads/2011/07/ISA.png" alt="" width="404" height="170" /></a>[/caption]
<p style="text-align: left;" align="CENTER"> The standard ISA algorithm however becomes computationally inefficient when the size of the image patches is scaled up. To overcome this problem, they developed a convolutional neural network which makes use of PCA and ISA at alternating levels. The output of the ISA at the a particular level was used to convolve a larger region of the image. The results of this convolution step were fed into the PCA layer for pre-processing before being passed on to the next ISA layer.</p>


[caption id="attachment_1492" align="aligncenter" width="406" caption="Architecture of a Stacked Convolutional ISA network"]<a href="http://blog.accelereyes.com/blog/wp-content/uploads/2011/07/ISA_Stacked.png"><img class="size-full wp-image-1492 " title="ISA_Stacked" src="http://blog.accelereyes.com/blog/wp-content/uploads/2011/07/ISA_Stacked.png" alt="" width="406" height="259" /></a>[/caption]
<p align="LEFT">Learning spatio-temporal features from video signals was done by using their model to learn features from 3D video blocks rather than 2D image samples.</p>


[caption id="attachment_1493" align="aligncenter" width="344" caption="Modified architecture for 3D video blocks"]<a href="http://blog.accelereyes.com/blog/wp-content/uploads/2011/07/ISA_3D.png"><img class="size-full wp-image-1493  " title="ISA_3D" src="http://blog.accelereyes.com/blog/wp-content/uploads/2011/07/ISA_3D.png" alt="" width="344" height="267" /></a>[/caption]
<p style="text-align: left;" align="CENTER">The trained networks appear to have learned features that are robust to translation, but sensitive to frequency and rotation (at the first level). The features learned at the second level appear to represent more complex shapes such as corners.</p>


[caption id="attachment_1494" align="aligncenter" width="356" caption="Features learnt from the Hollywood2 database at first and second layers"]<a href="http://blog.accelereyes.com/blog/wp-content/uploads/2011/07/Learned_Filters.png"><img class="size-full wp-image-1494 " title="Learned_Filters" src="http://blog.accelereyes.com/blog/wp-content/uploads/2011/07/Learned_Filters.png" alt="" width="356" height="214" /></a>[/caption]
<p align="LEFT"> The performance of their algorithm, as mentioned earlier, not beats the best published results in accuracy, but is also generally faster both while training and testing. The times taken for feature extraction of their algorithm are given below.</p>

<table width="779" cellspacing="0" cellpadding="7"><colgroup> <col width="177" /> <col width="178" /> <col width="177" /> </colgroup>
<tbody>
<tr valign="TOP">
<td style="text-align: left;" bgcolor="#ffffff" width="50%">
<p align="CENTER"><strong>Algorithm</strong></p>
</td>
<td style="text-align: left;" bgcolor="#ffffff" width="20%">
<p align="CENTER"><strong>Frames per Second</strong></p>
</td>
<td bgcolor="#ffffff" width="20%">
<p align="CENTER"><strong>Speedup</strong></p>
</td>
</tr>
<tr valign="TOP">
<td bgcolor="#ffffff">
<p align="left">HOG3D</p>
</td>
<td bgcolor="#ffffff">
<p align="CENTER">4.54</p>
</td>
<td bgcolor="#ffffff">
<p align="CENTER">1.0</p>
</td>
</tr>
<tr valign="TOP">
<td bgcolor="#ffffff">
<p align="left">Stacked ISA  (layer 1 only)</p>
</td>
<td bgcolor="#ffffff">
<p align="CENTER">7.14</p>
</td>
<td bgcolor="#ffffff">
<p align="CENTER">1.6</p>
</td>
</tr>
<tr valign="TOP">
<td bgcolor="#ffffff">
<p align="left">Stacked ISA  (flayers 1 and 2)</p>
</td>
<td bgcolor="#ffffff">
<p align="CENTER">2.27</p>
</td>
<td bgcolor="#ffffff">
<p align="CENTER">0.5</p>
</td>
</tr>
<tr valign="TOP">
<td style="text-align: left;" bgcolor="#ffffff">
<p align="left">Stacked ISA with Jacket  (layers 1 and 2)</p>
</td>
<td style="text-align: left;" bgcolor="#ffffff">
<p align="CENTER">10</p>
</td>
<td bgcolor="#ffffff">
<p align="CENTER">2.2</p>
</td>
</tr>
</tbody>
</table>
<p align="LEFT">As you can see, using Jacket for their algorithm (dominated by matrix vector and convolution problems), a <span style="color: #ff0000;">4.4X speedup</span> was achieved over the CPU implementation!</p>
<p align="LEFT">Special thanks to <a href="http://ai.stanford.edu/~quocle/">Quoc V. Le</a>, <a href="http://ai.stanford.edu/~wzou/">Will Y. Zou</a>, Serena Y. Yeung and <a href="http://www.cs.stanford.edu/people/ang/">Andrew Y. Ng</a> from SAIL for sharing their research. We have more success stories from their group. Keep an eye out for more blog posts!</p>