Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <blockquote> <p>Or did I get the connection between compression and entropy wrong?</p> </blockquote> <p>You're pretty close, but this last question is where the mistake was. If you're able to compress something into a form that was smaller than its original representation, it means that the original representation had at least some redundancy. <strong>Each bit in the message really wasn't conveying 1 bit of information.</strong></p> <p>Because redundant data does not contribute to the information content of a message, it also does not increase its entropy. Imagine, for example, a "random bit generator" that only returns the value "0". This conveys no information at all! (Actually, it conveys an <em>undefined</em> amount of information, because any binary message consisting of only one kind of symbol requires a division by zero in the entropy formula.)</p> <p>By contrast, had you simulated a large number of random coin flips, it would be very hard to reduce the size of this message by much. Each bit would be contributing close to 1 bit of entropy.</p> <p>When you compress data, you extract that redundancy. In exchange, you pay a one-time entropy price by having to devise a scheme that knows how to compress and decompress this data; that itself takes some information.</p> <blockquote> <p>However you can run-length encode it with something like 100 / 1 / 100 / 0 where it's number of bits to output followed by the bit. It seems like I have a representation smaller than the data. Especially if you increase the 100 to much larger number.</p> </blockquote> <p>To summarize, the fact that you could devise a scheme to make the <em>encoding of the data</em> smaller than the <em>original data</em> tells you something important. Namely, it says that <strong>your original data contained very little information</strong>.</p> <hr/> <h3>Further reading</h3> <p>For a more thorough treatment of this, including exactly how you'd calculate the entropy for any arbitrary sequence of digits with a few examples, check out <a href="http://www.michaelharold.com/downloads/The%20Relationship%20of%20Entropy%20to%20Message%20Size%20in%20Binary%20Symbol%20Strings.pdf" rel="nofollow noreferrer">this short whitepaper</a>.</p>
    singulars
    1. This table or related slice is empty.
    1. POShannon's entropy formula. Help my confusion
      singulars
      1. PTQuestion
    1. PTAnswer
    1. USJohn Feminella
    1. USJohn Feminella
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. POShannon's entropy formula. Help my confusion
      singulars
      1. PTQuestion
    1. This table or related slice is empty.
    1. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTAcceptedByOriginator
    1. COThanks. In terms of the formula itself, will the formula tell me this? The fact that the message contains little information? Please see my edit.
      singulars
      1. PO
      1. USBudric
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload