Logistic Regression to classify flying aircrafts [2]
C++ Training and Testing code
I created a Google Code project for this example and it can be found here.
The code should be simple and self explanatory. It uses batch Logistic Regression by default to train but if you pass 1 as an argument, then it will use stochastic instead.
High level explanation of the difference between Stochastic and Batch mode: If we use Batch mode, then the gradient descent will make slower and smaller steps hence they are in the right direction. It will also merge very close to the minima. Stochastic, on the other hand, will make faster steps toward the mimima but it might not get as close as Batch to the goal.
Here is an example of the TesterMain ran on one image of an aircraft in the sky:
Input (click to enlarge)
Output (click to enlarge)
.
.
.
.
.
.
.
.
The testing part can be done in Actionscript and all it needs is the Thetas calculated from out training examples.
.
Logistic Regression to classify flying aircrafts [1]
Logistic Regression to detect Airplane Pixels
Background:
Logistic Regression falls into the supervised learning algorithms category ( algorithms used to train a software for a specific task where the training data is labeled ). The great thing about Logistic Regression is that it is simple to implement and its simplicity makes it very easy to debug.
The logistic regression is based on the sigmoid function:
This function has lots of nice features including:
You can learn more about Logistic Regression by clicking on the inline links. And now let’s talk about the problem we are trying to solve.
Problem:
Given an image or video of an airplane in the sky, how can we classify a pixel to be a sky or airplane one?
Solution:
The idea is simple; We will be traverse all the image pixels and plug the R,G and B into some hypothesis equation h(x) ( that we need to figure out). If the result is close to zero then we classify that pixel as a sky pixel and an aircraft pixel otherwise. This description implies that our equation should have an output from 0 to 1 which is how the sigmoid function is designed.
Let’s refer to a specific pixel by x, then the hypothesis function h(x) can be defined as:
where
and g is the sigmoid function.
Our goal is then to train some data to find the best values of Thetas.
Implementation
Training Part:
This could be done in Actionscript and have the code trace out the Thetas but it would take a very long time to complete the task. Much longer if we would do it in low level languages such as C++.
Since we are trying to do things the right way, let’s do the training part in C++ and the testing part in Flash.
If you don’t feel like training data and get your hands dirty with C++, feel free to use my training result (the best possible Thetas) straight into your AS code.
to be continued…
Ready, set, GO!
Abjuring my laziness and engaging in a love affair with my newly created blog.
