StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
text
Body
copied!<p>You are not going to find step-by-step instructions for writing an SVM. It's important to understand the theory and mathematics behind them because they play a large role in the algorithms and the algorithms are difficult to understand without that background.</p> <p>The best place to start would be with John Platt's <a href="http://research.microsoft.com/apps/pubs/default.aspx?id=69644">Sequential Minimal Optimization (SMO)</a> algorithm. I've implemented this paper before, however Platt assumes a great deal of background knowledge. A great introductory book is Lutz Hamel's book <a href="http://www.wiley.com/WileyCDA/WileyTitle/productCd-0470371927,descCd-description.html"><em>Knowledge Discovery with Support Vector Machines</em></a>. This book motivates everything, so you don't have to know or be current on statistics and optimization theory if you want to understand SVM.</p> <p>If you really just want to go super simple, then start with this <a href="http://math.unt.edu/~hsp0009/smo.pdf">Simplified SMO algorithm</a> from UNT to get your feet wet. It's the SMO algorithm without any of the optimizations.</p> <p>Platt's SMO algorithm is not the state of the art at this point, but its the basis for many of the currently used algorithms. Both <a href="http://www.csie.ntu.edu.tw/~cjlin/libsvm/">LibSVM</a> and <a href="http://svmlight.joachims.org/">SVMLight</a> have their source code available online. They are based on two different optimizations of the SMO concept.</p> <p>If you are okay with a higher level you can implement an SVM using a Convex optimization package like <a href="http://cvxr.com/cvx/">CVX</a>, which has interfaces in MatLab and other languages. In that case you would just be setting up the optimization problem at the core of SVM and having the solver find the solution for you. Implementing a QP solver from scratch is not something I would recommend trying.</p> <p>If you want to use an existing library, and when you say 'from scratch' you are referring to dealing with the data more than the SVM algorithm itself, you could look into existing implementations like those available in <a href="http://www.r-project.org/">R</a>. <a href="http://www.jstatsoft.org/v15/i09/paper">"Support Vector Machines in R"</a> goes over how to do that. Hamel's book also uses R.</p> <p>Handwritten digit recognition datasets have been used in numerous <a href="http://scholar.google.com/scholar?q=SVM%20MNIST&btnG=&hl=en&as_sdt=0,22">SVM papers</a> and used for <a href="http://www.sciencedirect.com/science/article/pii/S0031320303000852">classifier comparisons</a>.</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload