Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>It's difficult to distill complex mathematics to a simple level, but this example helps I believe.</p> <p>I apologize, it appears that Stack Overflow does not have support for math markup like <a href="http://scicomp.stackexchange.com">Computional Science</a> does, so you'll have to bear with some crappy text based equations. Vector <strong>x</strong> = [<em>x<sub>1</sub></em>, <em>x<sub>2</sub></em>], so <em>x<sub>1</sub></em> is the first component of the vector <strong>x</strong>.</p> <p>For simplicity, lets imagine you have just two attributes for each training example, so your data is two-dimensional. You have developed a mapping function to map these two attributes into a higher dimension of three attributes. Your mapping function is as follows:</p> <p>&Phi;(<strong>x</strong>) = [<em>x<sub>1</sub><sup>2</sup></em>, <em>x<sub>2</sub><sup>2</sup></em>, <em>x<sub>1</sub></em> &#42; <em>x<sub>2</sub></em>]</p> <p>In the SVM Lagrangian formula:</p> <p><img src="https://i.stack.imgur.com/jS5Tr.png" alt="Lagrangian SVM Formula"></p> <p>Each training example appears as an inner product with another training example (the above image shows this in the first equation). If you want to use your mapping function, you would plug it in for each training example.</p> <p>If you do it this way of course you will have to calculate &Phi;(<strong>x</strong>) for each training example explicitly, then calculate the inner product of the two vectors in your higher dimensional space. If we do this out for two vectors <strong>x</strong> and <strong>y</strong>, we would have:</p> <p>&Phi;(<strong>x</strong>) &#42; &Phi;(<strong>y</strong>) = [<em>x<sub>1</sub><sup>2</sup></em>, <em>x<sub>2</sub><sup>2</sup></em>, <em>x<sub>1</sub></em> &#42; <em>x<sub>2</sub></em>] &#42; [<em>y<sub>1</sub><sup>2</sup></em>, <em>y<sub>2</sub><sup>2</sup></em>, <em>y<sub>1</sub></em> &#42; <em>y<sub>2</sub></em>] = <em>x<sub>1</sub><sup>2</sup></em> &#42; <em>y<sub>1</sub><sup>2</sup></em> + <em>x<sub>2</sub><sup>2</sup></em> &#42; <em>y<sub>2</sub><sup>2</sup></em> + <em>x<sub>1</sub></em> &#42; <em>x<sub>2</sub></em> &#42; <em>y<sub>1</sub></em> &#42; <em>y<sub>2</sub></em></p> <p>Imagine instead you used the polynomial kernel <em>K</em>(<strong>x</strong> , <strong>y</strong>) = (<strong>x</strong> &#42; <strong>y</strong>)<sup><em>d</em></sup> with the degree of two, you would have:</p> <p><em>K</em>(<strong>x</strong>, <strong>y</strong>) = (<em>x<sub>1</sub></em> &#42; <em>y<sub>1</sub></em> + <em>x<sub>2</sub></em> &#42; <em>y<sub>2</sub></em>)<sup>2</sup> = <em>x<sub>1</sub><sup>2</sup></em> &#42; <em>y<sub>1</sub><sup>2</sup></em> + <em>x<sub>2</sub><sup>2</sup></em> &#42; <em>y<sub>2</sub><sup>2</sup></em> + <em>x<sub>1</sub></em> &#42; <em>x<sub>2</sub></em> &#42; <em>y<sub>1</sub></em> &#42; <em>y<sub>2</sub></em></p> <p>The kernel function allowed you to avoid calculating the higher dimensional space before calculating the inner product, but still resulted in the inner product of two vectors in that higher dimensional space. In this case we kept the example simple so we could do it out explicitly, but <a href="http://en.wikipedia.org/wiki/Mercer%27s_theorem" rel="nofollow noreferrer">Mercer's Theorem</a> shows that we can prove this is true for other functions without having to know the explicit mapping, as long as the function obeys <a href="http://www.svms.org/mercer/" rel="nofollow noreferrer">Mercer's condition</a>. You can see how the kernel parameter <em>d</em> affects the mapping significantly, <em>d = 3</em> would result in a completely different mapping, so changing the kernel parameters is modifying the higher-dimensional space.</p> <p>Since the higher-dimensional mapping is never used explicitly, only rather as a way to choose our optimal alpha's, we don't really need to know what it is, we can take advantage of it without having to calculate it.</p>
    singulars
    1. This table or related slice is empty.
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. VO
      singulars
      1. This table or related slice is empty.
    2. VO
      singulars
      1. This table or related slice is empty.
    3. VO
      singulars
      1. This table or related slice is empty.
    1. This table or related slice is empty.
 

Querying!

 
Guidance

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload