<p>A couple of weeks back, we did a <a href="http://sandbox.arrayfire.com/image-editing-using-arrayfire/">post</a> on a few image editing functions using ArrayFire library. Today, we shall be doing the second post in the series <code>Image Editing using ArrayFire</code>. We will be looking at the following operations today.</p>
<ul>
<li>Image distortion<br /></li>
<li>Noise addition<br /></li>
<li>Noise reduction<br /></li>
<li>Edge filters<br /></li>
<li>Boundary extraction<br /></li>
<li>Difference of gaussians</li>
</ul>
<p>Code and sample input/outputs corresponding to each operation are described below.</p>

<h4 id="image-distortion">Image distortion</h4>
<p>We will be looking at spread and pick filters in this section. Both of these filters are fundamentally the same, they replace each pixel in the original image with one of it's neighboring pixels. How the neighbor is chosen is essentially the difference between spread and pick. Both of these functions use a common function `getRandomNeighbor` to select neighboring pixels. The code snippet for `getRandomNeighbor` is given below.</p>
<pre>
array getRandomNeighbor(const array &in, int windW, int windH)
{
    array rnd = 2.0f*randu(in.dims(0), in.dims(1))-1.0f;
    array sx = seq(in.dims(0));
    array sy = seq(in.dims(1));
    array vx = tile(sx, 1, in.dims(1)) + floor(rnd*windW);
    array vy = tile(sy.T(), in.dims(0), 1) + floor(rnd*windH);
    array vxx = clamp(vx,0,in.dims(0));
    array vyy = clamp(vy,0,in.dims(1));
    array in2 = moddims(in, vx.elements(), 3);
    return moddims(in2(vyy*in.dims(0)+vxx, span),in.dims());
}
</pre>

<pre>
/**
 * randomly pick neighbor from given window size and replace the
 * current pixel with the randomly chosen color.
 * No new colors are introduced, unlike hurl.
 */
array spread(const array &in, int window_width, int window_height)
{
    return getRandomNeighbor(in,window_width,window_height);
}
</pre>
<div style = "text-align:center">
 <div style="display: inline-block; margin-right: 5px">
[caption id="attachment_6231" class="alignnone" width="300"]<a href="http://sandbox.arrayfire.com/wp-content/uploads/2014/07/lena512x512.jpg"><img src="http://sandbox.arrayfire.com/wp-content/uploads/2014/07/lena512x512-300x300.jpg" alt="lena512x512" width="300" height="300" class="alignnone size-medium wp-image-6231" /></a>Input image[/caption]
 </div>
 <div style="display: inline-block">
[caption id="attachment_6236" class="alignnone" width="300"]<a href="http://sandbox.arrayfire.com/wp-content/uploads/2014/07/spread.jpg"><img src="http://sandbox.arrayfire.com/wp-content/uploads/2014/07/spread-300x300.jpg" alt="spread" width="300" height="300" class="alignnone size-medium wp-image-6236" /></a>Image with spread applied[/caption]
 </div>
</div>
<pre>
/**
 * randomization - controls % of total number of pixels in the image
 * that will be effected by random noise
 * repeat - # of times the process is carried out on the previous steps output
 */
array pick(const array &in, int randomization, int repeat)
{
    int w = in.dims(0);
    int h = in.dims(1);
    float f = randomization/100.0f;
    int dim = (int)(f*w*h);
    array ret_val = in.copy();
    for(int i=0;i<repeat;++i) {
        array idxs  = (w*h)  * randu(dim);
        array rnd   = getRandomNeighbor(ret_val,1,1);
        array temp_src = moddims(rnd,w*h,3);
        array temp_dst = moddims(ret_val,w*h,3);
        temp_dst(idxs,span) = temp_src(idxs,span);
        ret_val = moddims(temp_dst,in.dims());
    }
    return ret_val;
}
</pre>
<div style = "text-align:center">
 <div style="display: inline-block; margin-right: 5px">
[caption id="attachment_6231" class="alignnone" width="300"]<a href="http://sandbox.arrayfire.com/wp-content/uploads/2014/07/lena512x512.jpg"><img src="http://sandbox.arrayfire.com/wp-content/uploads/2014/07/lena512x512-300x300.jpg" alt="lena512x512" width="300" height="300" class="alignnone size-medium wp-image-6231" /></a>Input image[/caption]
 </div>
 <div style="display: inline-block">
[caption id="attachment_6233" class="alignnone" width="300"]<a href="http://sandbox.arrayfire.com/wp-content/uploads/2014/07/pick.jpg"><img src="http://sandbox.arrayfire.com/wp-content/uploads/2014/07/pick-300x300.jpg" alt="pick" width="300" height="300" class="alignnone size-medium wp-image-6233" /></a>After Pick filter[/caption]
 </div>
</div>

<h4 id="noise-addition">Noise addition</h4>
<p>We will at look hurl noise now. Despite the fancy name, what the filter essentially does is replace some of the pixels of the image with random color. New colors are introduced into the resulting image. Therefore, giving a high `repeat` input value might result in an image that is completely different from the original.</p>
<pre>
/**
 * randomization - controls % of total number of pixels in the image
 * that will be effected by random noise
 * repeat - # of times the process is carried out on the previous steps output
 */
array hurl(const array &in, int randomization, int repeat)
{
    int w = in.dims(0);
    int h = in.dims(1);
    float f = randomization/100.0f;
    int dim = (int)(f*w*h);
    array ret_val = in.copy();
    array temp = moddims(ret_val,w*h,3);
    for(int i=0;i<repeat;++i) {
        array idxs = (w*h)  * randu(dim);
        array rndR = 255.0f * randu(dim);
        array rndG = 255.0f * randu(dim);
        array rndB = 255.0f * randu(dim);
        temp(idxs,0) = rndR;
        temp(idxs,1) = rndG;
        temp(idxs,2) = rndB;
    }
    ret_val = moddims(temp,in.dims());
    return ret_val;
}
</pre>
<div style = "text-align:center">
 <div style="display: inline-block; margin-right: 5px">
[caption id="attachment_6231" class="alignnone" width="300"]<a href="http://sandbox.arrayfire.com/wp-content/uploads/2014/07/lena512x512.jpg"><img src="http://sandbox.arrayfire.com/wp-content/uploads/2014/07/lena512x512-300x300.jpg" alt="lena512x512" width="300" height="300" class="alignnone size-medium wp-image-6231" /></a>Input Image[/caption]
 </div>
 <div style="display: inline-block">
[caption id="attachment_6230" class="alignnone" width="300"]<a href="http://sandbox.arrayfire.com/wp-content/uploads/2014/07/hurl.jpg"><img src="http://sandbox.arrayfire.com/wp-content/uploads/2014/07/hurl-300x300.jpg" alt="hurl" width="300" height="300" class="alignnone size-medium wp-image-6230" /></a>After adding hurl noise[/caption]
 </div>
</div>

<h4 id="noise-reduction">Noise reduction</h4>
<p>Noise reduction is typically achieved using smoothing filters. We will look at bilateral, median and Gaussian blur filters today. Our library has a built-in function(<a href="http://www.sandbox.arrayfire.com/docs/group__image__func__cpp__bilateral.htm">af::bilateral</a>) that readily does bilateral filtering on an image.</p> 
<div style = "text-align:center">
 <div style="text-align:left; display: inline-block">
[caption id="attachment_6230" class="alignnone" width="300"]<a href="http://sandbox.arrayfire.com/wp-content/uploads/2014/07/hurl.jpg"><img src="http://sandbox.arrayfire.com/wp-content/uploads/2014/07/hurl-300x300.jpg" alt="hurl" width="300" height="300" class="alignnone size-medium wp-image-6230" /></a>Input image[/caption]
 </div>
 <div style="text-align:center; display: inline-block;">
[caption id="attachment_6227" class="alignnone" width="300"]<a href="http://sandbox.arrayfire.com/wp-content/uploads/2014/07/bilateral.jpg"><img src="http://sandbox.arrayfire.com/wp-content/uploads/2014/07/bilateral-300x300.jpg" alt="bilateral" width="300" height="300" class="alignnone size-medium wp-image-6227" /></a>Bilateral smoothing[/caption]
 </div>
<div style="text-align:right; display: inline-block;">
[caption id="attachment_6237" class="alignnone" width="300"]<a href="http://sandbox.arrayfire.com/wp-content/uploads/2014/07/bilateral2.jpg"><img src="http://sandbox.arrayfire.com/wp-content/uploads/2014/07/bilateral2-300x300.jpg" alt="bilateral" width="300" height="300" class="alignnone size-medium wp-image-6237" /></a>Bilateral with more spatial and chromatic variance[/caption]
 </div>
</div>

<p>As you can see in the above images, bilateral is not quite effective with hurl noise. Increasing spatial radius might help a little but might result in loosing features as shown in the third picture. Next, we will look at Gaussian blur.</p>
<pre>
array gaussianblur(const array &in, int window_width, int window_height, int sigma)
{
    array g = gaussiankernel(window_width,window_height,sigma,sigma);
    return convolve(in,g);
}
</pre>
<div style = "text-align:center">
 <div style="display: inline-block; margin-right: 5px">
[caption id="attachment_6230" class="alignnone" width="300"]<a href="http://sandbox.arrayfire.com/wp-content/uploads/2014/07/hurl.jpg"><img src="http://sandbox.arrayfire.com/wp-content/uploads/2014/07/hurl-300x300.jpg" alt="hurl" width="300" height="300" class="alignnone size-mediuml wp-image-6230" /></a>Input Image[/caption]
 </div>
 <div style="display: inline-block">
[caption id="attachment_6229" class="alignnone" width="300"]<a href="http://sandbox.arrayfire.com/wp-content/uploads/2014/07/gblur.jpg"><img src="http://sandbox.arrayfire.com/wp-content/uploads/2014/07/gblur-300x300.jpg" alt="gblur" width="300" height="300" class="alignnone size-medium wp-image-6229" /></a>After applying Gaussian smoothing[/caption]
 </div>
</div>
Gaussian blur was able to remove noise to certain extent, but it is blurring out all important features equally and the noisy pixels participate in the convolution operation. Hence, Gaussian blur might not be a right candidate always.
<pre>
array medianfilter(const array &in, int window_width, int window_height)
{
    array ret_val(in.dims());
    ret_val(span,span,0) = medfilt(in(span,span,0),window_width,window_height);
    ret_val(span,span,1) = medfilt(in(span,span,1),window_width,window_height);
    ret_val(span,span,2) = medfilt(in(span,span,2),window_width,window_height);
    return ret_val;
}
</pre>
<div style = "text-align:center">
 <div style="display: inline-block; margin-right: 5px">
[caption id="attachment_6230" class="alignnone" width="300"]<a href="http://sandbox.arrayfire.com/wp-content/uploads/2014/07/hurl.jpg"><img src="http://sandbox.arrayfire.com/wp-content/uploads/2014/07/hurl-300x300.jpg" alt="hurl" width="300" height="300" class="alignnone size-medium wp-image-6230" /></a>Input Image[/caption]
 </div>
 <div style="display: inline-block">
[caption id="attachment_6232" class="alignnone" width="300"]<a href="http://sandbox.arrayfire.com/wp-content/uploads/2014/07/medfilt.jpg"><img src="http://sandbox.arrayfire.com/wp-content/uploads/2014/07/medfilt-300x300.jpg" alt="medfilt" width="300" height="300" class="alignnone size-medium wp-image-6232" /></a>After Median filter is applied[/caption]
 </div>
</div>
Median filter seems to do a good job for this type of noise.

<h4 id="edge-filters">Edge filters</h4>
<p>We will be looking at Sobel and Prewitt filters.</p>
<pre>
void prewitt(array &mag, array &dir, const array &in)
{
    float h1[] = { 1, 1, 1};
    float h2[] = {-1, 0, 1};

    // Find the gradients
    array Gy = convolve(3, h2, 3, h1, in)/6;
    array Gx = convolve(3, h1, 3, h2, in)/6;

    // Find magnitude and direction
    mag = hypot(Gx, Gy);
    dir = atan2(Gy, Gx);
}
</pre>
<div style = "text-align:center">
 <div style="display: inline-block; margin-right: 5px">
[caption id="attachment_6231" class="alignnone" width="300"]<a href="http://sandbox.arrayfire.com/wp-content/uploads/2014/07/lena512x512.jpg"><img src="http://sandbox.arrayfire.com/wp-content/uploads/2014/07/lena512x512-300x300.jpg" alt="lena512x512" width="300" height="300" class="alignnone size-medium wp-image-6231" /></a>Input Image[/caption]
 </div>
 <div style="display: inline-block">
[caption id="attachment_6234" class="alignnone" width="300"]<a href="http://sandbox.arrayfire.com/wp-content/uploads/2014/07/prewitt.jpg"><img src="http://sandbox.arrayfire.com/wp-content/uploads/2014/07/prewitt-300x300.jpg" alt="prewitt" width="300" height="300" class="alignnone size-medium wp-image-6234" /></a>Prewitt Gradient[/caption]
 </div>
</div>

<pre>
void sobel(array &mag, array &dir, const array &in)
{
    float h1[] = { 1, 2,  1};
    float h2[] = { 1, 0, -1};

    // Find the gradients
    array Gy = convolve(3, h2, 3, h1, in)/8;
    array Gx = convolve(3, h1, 3, h2, in)/8;

    // Find magnitude and direction
    mag = hypot(Gx, Gy);
    dir = atan2(Gy, Gx);
}
</pre>
<div style = "text-align:center">
 <div style="display: inline-block; margin-right: 5px">
[caption id="attachment_6231" class="alignnone" width="300"]<a href="http://sandbox.arrayfire.com/wp-content/uploads/2014/07/lena512x512.jpg"><img src="http://sandbox.arrayfire.com/wp-content/uploads/2014/07/lena512x512-300x300.jpg" alt="lena512x512" width="300" height="300" class="alignnone size-medium wp-image-6231" /></a>Input Image[/caption]
 </div>
 <div style="display: inline-block">
[caption id="attachment_6235" class="alignnone" width="300"]<a href="http://sandbox.arrayfire.com/wp-content/uploads/2014/07/sobel.jpg"><img src="http://sandbox.arrayfire.com/wp-content/uploads/2014/07/sobel-300x300.jpg" alt="sobel" width="300" height="300" class="alignnone size-medium wp-image-6235" /></a>Sobel Gradient[/caption]
 </div>
</div>

<h4 id="boundary-extraction">Boundary extraction</h4>
<p>The following code snippet extracts boundaries in an image using erosion morphological operator.</p>
<pre>
/**
 * dimensions of the mask control the thickness of the boundary that
 * will be extracted by the following function
 */
array boundary(const array &in, const array &mask)
{
    array ret_val = in - erode(in,mask);
    normalizeImage(ret_val);
    return ret_val;
}
</pre>
<div style = "text-align:center">
 <div style="display: inline-block; margin-right: 5px">
[caption id="attachment_6231" class="alignnone" width="300"]<a href="http://sandbox.arrayfire.com/wp-content/uploads/2014/07/lena512x512.jpg"><img src="http://sandbox.arrayfire.com/wp-content/uploads/2014/07/lena512x512-300x300.jpg" alt="lena512x512" width="300" height="300" class="alignnone size-medium wp-image-6231" /></a>Input Image[/caption]
 </div>
 <div style="display: inline-block">
[caption id="attachment_6236" class="alignnone" width="300"]<a href="http://sandbox.arrayfire.com/wp-content/uploads/2014/07/bdry.jpg"><img src="http://sandbox.arrayfire.com/wp-content/uploads/2014/07/bdry-300x300.jpg" alt="bdry" width="300" height="300" class="alignnone size-medium wp-image-6226" /></a>Difference of Gaussians[/caption]
 </div>
</div>

<h4 id="difference-of-gaussians">Difference of gaussians</h4>
<pre>
array diffOfGaussians(const array &in, int window_radius1, int window_radius2)
{
    array ret_val;
    int w1 = 2*window_radius1+1;
    int w2 = 2*window_radius2+1;
    array g1 = gaussiankernel(w1,w1);
    array g2 = gaussiankernel(w2,w2);
    ret_val = (convolve(in,g1)-convolve(in,g2));
    normalizeImage(ret_val);
    return ret_val;
}
</pre>
<div style = "text-align:center">
 <div style="display: inline-block; margin-right: 5px">
[caption id="attachment_6231" class="alignnone" width="300"]<a href="http://sandbox.arrayfire.com/wp-content/uploads/2014/07/lena512x512.jpg"><img src="http://sandbox.arrayfire.com/wp-content/uploads/2014/07/lena512x512-300x300.jpg" alt="lena512x512" width="300" height="300" class="alignnone size-medium wp-image-6231" /></a>[/caption]
 </div>
 <div style="display: inline-block">
[caption id="attachment_6231" class="alignnone" width="300"]<a href="http://sandbox.arrayfire.com/wp-content/uploads/2014/07/dog.jpg"><img src="http://sandbox.arrayfire.com/wp-content/uploads/2014/07/dog-300x300.jpg" alt="dog" width="300" height="300" class="alignnone size-medium wp-image-6228" /></a>[/caption]
 </div>
</div>

Last two sections use a utility function `normalizeImage` that basically re-scales the pixels values to [0-255] range.
<pre>
void normalizeImage(array &in)
{
    float min = af::min<float>(in);
    float max = af::max<float>(in);
    in = 255.0f*((in-min)/(max-min));
}
</pre>

<h4>Conclusion</h4>
All the functions from this post are updated and available through github repository located <a href="https://github.com/9prady9/wildfire">here</a>. Hope, you guys are enjoying this series.