StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
text
Body
copied!<p>Twerdster posted some excellent code in his answer. So all credit go to him. I post this new answer, since comments don't allow for nice syntax colored code blocks, and i wanted to share some code. But if you like the code, please upvote Twerdster original answer.</p> <p>In Twerdster previous post he mentioned that the decode and encode might not work for all values.</p> <p>To further test this, and validate the result i made a java program. While porting the code i tried to stayed as close as possible to the shader code (therefore i implemented some helper functions). Note: I also use a store/load function to similate what happens when you write/read from a texture. </p> <p>I found out that:</p> <ol> <li>You need a special case for the zero</li> <li>You might also need special case for infinity, but i did not implement that to keep the shader simple (eg: faster)</li> <li>Because of rounding errors sometimes the result was wrong therefore: <ul> <li>subtract 1 from exponent when because of rounding the mantissa is not properly normalised (eg mantissa < 1)</li> <li>Change <code>float Mantissa = (exp2(- Exponent) * F);</code> to <code>float Mantissa = F/exp2(Exponent);</code> to reduce precision errors</li> <li>Use <code>float Exponent = floor(log2(F));</code> to calc exponent. (simplified by new mantissa check)</li> </ul></li> </ol> <p>Using these small modifications i got equal output on almost all inputs, and got only small errors between the original and encoded/decoded value when things do go wrong, while in Twerdster's original implementation rounding errors often resulted in the wrong exponent (thus the result being off by factor two).</p> <p>Please note that this is a Java test application which i wrote to test the algorithm. I hope this will also work when ported to the GPU. If anybody tries to run it on a GPU, please leave a comment with your experience. </p> <p>And for the code with a simple test to try different numbers until it failes.</p> <pre><code>import java.io.PrintStream; import java.util.Random; public class BitPacking { public static float decode32(float[] v) { float[] rgba = mult(255, v); float sign = 1.0f - step(128.0f,rgba[0])*2.0f; float exponent = 2.0f * mod(rgba[0],128.0f) + step(128.0f,rgba[1]) - 127.0f; if(exponent==-127) return 0; float mantissa = mod(rgba[1],128.0f)*65536.0f + rgba[2]*256.0f +rgba[3] + ((float)0x800000); return sign * exp2(exponent-23.0f) * mantissa ; } public static float[] encode32(float f) { float F = abs(f); if(F==0){ return new float[]{0,0,0,0}; } float Sign = step(0.0f,-f); float Exponent = floor(log2(F)); float Mantissa = F/exp2(Exponent); if(Mantissa < 1) Exponent -= 1; Exponent += 127; float[] rgba = new float[4]; rgba[0] = 128.0f * Sign + floor(Exponent*exp2(-1.0f)); rgba[1] = 128.0f * mod(Exponent,2.0f) + mod(floor(Mantissa*128.0f),128.0f); rgba[2] = floor(mod(floor(Mantissa*exp2(23.0f -8.0f)),exp2(8.0f))); rgba[3] = floor(exp2(23.0f)*mod(Mantissa,exp2(-15.0f))); return mult(1/255.0f, rgba); } //shader build-in's public static float exp2(float x){ return (float) Math.pow(2, x); } public static float[] step(float edge, float[] x){ float[] result = new float[x.length]; for(int i=0; i<x.length; i++) result[i] = x[i] < edge ? 0.0f : 1.0f; return result; } public static float step(float edge, float x){ return x < edge ? 0.0f : 1.0f; } public static float mod(float x, float y){ return x-y * floor(x/y); } public static float floor(float x){ return (float) Math.floor(x); } public static float pow(float x, float y){ return (float)Math.pow(x, y); } public static float log2(float x) { return (float) (Math.log(x)/Math.log(2)); } public static float log10(float x) { return (float) (Math.log(x)/Math.log(10)); } public static float abs(float x) { return (float)Math.abs(x); } public static float log(float x) { return (float)Math.log(x); } public static float exponent(float x) { return floor((float)(Math.log(x)/Math.log(10))); } public static float mantissa(float x) { return floor((float)(Math.log(x)/Math.log(10))); } //shorter matrix multiplication private static float[] mult(float scalar, float[] w){ float[] result = new float[4]; for(int i=0; i<4; i++) result[i] = scalar * w[i]; return result; } //simulate storage and retrieval in 4-channel/8-bit texture private static float[] load(int[] v) { return new float[]{v[0]/255f, v[1]/255f, v[2]/255f, v[3]/255f}; } private static int[] store(float[] v) { return new int[]{((int) (v[0]*255))& 0xff, ((int) (v[1]*255))& 0xff, ((int) (v[2]*255))& 0xff, ((int) (v[3]*255))& 0xff}; } //testing until failure, and some specific hard-cases separately public static void main(String[] args) { //for(float v : new float[]{-2097151.0f}){ //small error here for(float v : new float[]{3.4028233e+37f, 8191.9844f, 1.0f, 0.0f, 0.5f, 1.0f/3, 0.1234567890f, 2.1234567890f, -0.1234567890f, 1234.567f}){ float output = decode32(load(store(encode32(v)))); PrintStream stream = (v==output) ? System.out : System.err; stream.println(v + " ?= " + output); } //System.exit(0); Random r = new Random(); float max = 3200000f; float min = -max; boolean error = false; int trials = 0; while(!error){ float fin = min + r.nextFloat() * ((max - min) + 1); float fout = decode32(load(store(encode32(fin)))); if(trials % 10000 == 0) System.out.print('.'); if(trials % 1000000 == 0) System.out.println(); if(fin != fout){ System.out.println(); System.out.println("correct trials = " + trials); System.out.println(fin + " vs " + fout); error = true; } trials++; } } } </code></pre>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload