Official Blog of Azilen

The Techie Explorations

[Part-1] Automation of Home Appliances using Raspberry Pi – Software Installation Guide

At Azilen, we believe in creating smart solutions for an interconnected and intelligent world. With emergence of IoT we try to achieve advanced connectivity and ensure efficiency through connected devices and network sensors.

Our recent work on the automation of home appliances through gesture recognition is one such example where we aim to control and automate home appliances such as air conditioners and televisions. We have achieved gesture recognition for implementing functionality like Turning On-Off, Increasing and decreasing the temperature for Air-conditioner and Turning On-Off for Television.

Through an extensive research, our team explored various feasible options for selecting the right platform and suitable libraries to make real time computer vision possible for gesture recognition. From Omega ,Chip, Aurduino ,Raspberry Pi amongst the platform and from BoofCV, OpenIMAJ, MATLAB, OpenCV and JavaCV among the libraries, the team selected Raspberry Pi and leading open source library ”Open CV” and its java port “JavaCV” as the right platform to make computer vision and image processing possible.

The library has more than 2500 optimized algorithms, which includes a comprehensive set of both classic and state-of-the-art computer vision and machine learning algorithms. These algorithms can be used to detect and recognize faces, identify objects, classify human actions in videos, track camera movements, track moving objects, etc.

This blog talks about the complete step by step guide to setup OpenCV and JavaCV on Raspberry Pi.

001

Note:

We assume that you have pre-installed latest Raspbian operating system on your Raspberry-Pi. If not please follow this link to install latest Raspbian on your Raspberry-Pi.

https://www.Raspberry Pi.org/downloads/noobs/

All required packages such as Maven, Ant, Git should be installed in the system.

Installation Guide
Please follow below steps to setup OpenCV and JavaCV on Raspberry Pi

Step-1: Update Raspbian

At the start you need compiler tools so update and upgrade the Raspberry Pi (Raspbian Linux) with latest available packages

Get all the compiler tools and video libraries using the below commands:
Open the terminal window and fire following commands one by one

sudo apt-get update
sudo apt-get upgrade

Step-2: Install Dependencies

Open CV requires few external dependencies for it to be able to run, therefore you need to install all the required dependencies in the following order:
Open the terminal window and fire following commands one by one

sudo apt-get install build-essential
sudo apt-get install cmake
sudo apt-get install pkg-config
sudo apt-get install libpng12-0 libpng12-dev libpng++-dev libpng3
sudo apt-get install libpnglite-dev
sudo apt-get install zlib1g-dbg zlib1g zlib1g-dev
sudo apt-get install pngtools libtiff4-dev libtiff4 libtiffxx0c2 libtiff-tools
sudo apt-get install libjpeg8 libjpeg8-dev libjpeg8-dbg libjpeg-progs
sudo apt-get install libav-tools libavcodec-dev libavcodec52 libavformat52 libavformat-dev
sudo apt-get install libgstreamer0.10-0-dbg libgstreamer0.10-0 libgstreamer0.10-dev
sudo apt-get install libxine2-ffmpeg libxine-dev libxine2-bin
sudo apt-get install libunicap2 libunicap2-dev
sudo apt-get install libdc1394-22-dev libdc1394-22 libdc1394-utils
sudo apt-get install swig
sudo apt-get install libv4l-0 libv4l-dev
sudo apt-get install python-numpy
sudo apt-get install libpython2.7 python-dev python2.7-dev
sudo apt-get install libgtk2.0-dev pkg-config

Note:
If some of the libraries fail to download, then try to look for a different version of the library using apt-cache search as some of these libraries are not in Raspbian repository anymore or are available with newer versions
For e.g. :

sudo apt-cache search ffmpeg

Step 3: Download OpenCV Source Code and Install

Now we have our dependencies installed, let’s grab the OpenCV archive version 2.4.13 from the official OpenCV repository.

(Note: As future versions of OpenCV are released, you can replace 3.1.0, 3.2.0 with the latest version number, but this guide is focused around version 2.4.13).

wget https://sourceforge.net/projects/OpenCVlibrary/files/OpenCV-unix/2.4.13/OpenCV-2.4.13.zip/download

  • Uncompress the downloaded source file
    Move into uncompressed directore and create a new directory named “release” inside it
    Run below commands to configure, compile and install OpenCV.Please note that compilation will take approximately 4-5 hours, so grab yourself a nice pizza and relax !

    cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local ..
    make
    sudo make install
    sudo idconfig

After cmake is done successfully, the jar file will be under release/bin/ and release/lib/ folder. Since we build version 2.4.13, the generated file will be named OpenCV-2413.jar and libOpenCV_java.so.

Both generated files will required in build path for any java based project with OpenCV. While libOpenCV_java.so is required for any C/C++ based project with OpenCV.

Note: If you get any Linker error or java.library.path issue ,you need to provide path for generated OpenCV jar file with java -jar cmd. using –Djava.library.path= ‘path to OpenCV libs’ or you can export LD_LIBRARY_PATH system variable with lib path. you can also add export command to .bashcr file as well

Step 4 : Compile JavaCV / javaCPP

Now that we have compiled and installed OpenCV 2.4.13 on our Raspberry Pi, we can compile and install JavaCV and JavaCPP on Raspberry Pi. In order to do that, you need to have Java, Maven and Git installed on the Raspberry Pi.

The compilation of JavaCV and javaCPP does not run with all versions of Java (such as Oracle JDK 8). It runs correctly with OpenJDK 7 which needs to be installed temporarily for this purpose if it’s not already installed. There might be multiple versions of Java installed. It needs to be ensured that OpenJDK 7 is the default selected Java version. Please run following commands on your terminal window to install OpenJDK.

Install OpenJDK 7 and select OpenJDK 7 as default Java version:

sudo apt-get install openjdk-7-jdk
sudo update-alternatives --config java
sudo update-alternatives --config javac

For JavaCV / javaCPP Maven is needed. The source will be downloaded using Git.

Once the source is downloded reset the sources to the old version 0.7 of JavaCV and javaCPP as they are the recent working ones for our purpose. Two modifications to the source files need to be performed to run on the Raspberry Pi and these will be done with a sed command. Afterwards the compilation can be started.

Run the following commands in terminal window: (Follow steps in order)

sudo apt-get install maven sed
git clone https://github.com/bytedeco/JavaCV.git
git clone https://github.com/bytedeco/javacpp.git
git --git-dir JavaCV/.git --work-tree=JavaCV reset --hard f3bb7aa577c0b427a88f927a2cea8ec505c31e22
git --git-dir javacpp/.git --work-tree=javacpp reset --hard 1104907b29ff7b42b562c5495672a51f9a46af7c
(above both cmd will reset JavaCV and javacpp to 0.7 version)
sed -r -i 's/link="OpenCV_core@.2.4", preload="tbb"\),/link="OpenCV_core@.2.4"\),/g' JavaCV/src/main/java/com/googlecode/JavaCV/cpp/OpenCV_core.java
sed -r -i 's/-mfloat-abi=softfp/-mfloat-abi=hard/g' javacpp/src/main/resources/com/googlecode/javacpp/properties/linux-arm.properties
mvn clean install -f javacpp/pom.xml -Dplatform.name=linux-arm (javacpp.jar generation)
mvn clean install -f JavaCV/pom.xml -Dplatform.name=linux-arm (JavaCV.jar and other required jar generation))

And that’s it. You have successfully installed JavaCV/JavaCPP on Raspberry Pi. Next step is to verify your installation.

Step 5: Testing your OpenCV / JavaCV Installation

Open your favorite editor on Raspberry Pi (VI/NANO/PageLeaf).

Create a new java file as shown below and save the file.

import org.OpenCV.core.Core;
import org.OpenCV.core.CvType;
import org.OpenCV.core.Mat;

public class JavaCVTest
{
public static void main( String[] args )
{
System.loadLibrary( Core.NATIVE_LIBRARY_NAME );
Mat mat = Mat.eye( 3, 3, CvType.CV_8UC1 );
System.out.println( "mat = " + mat.dump() );
}
}

Note: Raspbian ships latest Oracle JDK version with it. Please use command ‘whereis java’ to find out the location of the Oracle JDK and make it default java version now onwards.
Now compile this file using Oracle JDK version >= 7. Type following commands into your terminal to compile and run this java code.

javac –cp ‘path to generated JavaCV-2.4.13.jar’ JavaCVTest.java
java –cp ‘path to generated JavaCV-2.4.13.jar’ –Djava.library.path=’path to generated OpenCV-2.4.13.so’ JavaCVTest

The java code output must be as below:

img1

Congrats! You have a brand new, fresh install of OpenCV / JavaCV on your Raspberry Pi. And I’m sure you’re just itching to leverage your Raspberry Pi to build some awesome computer vision apps.

Conclusion

Our comprehensive approach can help implement various IoT solutions such as color detection, face recognition system, motion tracking, Object identification and Mobile robotics.

So, what’s next?

In our upcoming blog we aim to write about complete guide for automating air conditioner and television appliances for implementing features through gesture recognition.

Trackbacks

  1. [Part 2] Automation of Home Appliances using Raspberry Pi - Implementation of Hand Gesture Recognition | Official Blog of Azilen says:

    […] our recent blog we have talked about software installation guide to setup and begin automation of home appliances […]

  2. [Part 3] Automation of Home Appliances using Raspberry Pi – All Set to Automate the Home Appliances | Official Blog of Azilen says:

    […] Raspberry Pi. The first and second blogs of this series were more about technology set up such as Software Installation Guide and Implementation of Hand Gesture Recognition. Third blog is more about command configuration set […]

Leave a reply