Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>Regarding your "Edit 3" ... I'll bite, although it's really a diversion from the original question.</p> <p>There's no reason you need Lists of arrays, since you're just breaking up the file into a continuous list of 4-byte sequences, looping through and processing each sequence, and then looping through and writing each sequence. You can do much better. <strong>NOTE:</strong> The implementation below does not check for or handle input files whose lengths are not exactly multiples of 4. I leave that as an exercise to you, if it is important.</p> <p>To directly address your comment, here is a single-array solution. We'll ditch the List objects, read the whole file into a single byte[] array, and then copy out 4-byte sections of that array to do your bit transforms, then put the result back. At the end we'll just slam the whole thing into the output file.</p> <pre><code>byte[] data; using (Stream fs = File.OpenRead("E:\\temp\\test.bmp")) { data = new byte[fs.Length]; fs.Read(data, 0, data.Length); } byte[] element = new byte[4]; for (int i = 0; i &lt; data.Length; i += 4) { Array.Copy(data, i, element, 0, element.Length); BitArray source = new BitArray(element); BitArray target = new BitArray(source.Length); target[26] = source[0]; target[31] = source[1]; // ... target[5] = source[31]; target.CopyTo(data, i); } using (Stream fs = File.OpenWrite("E:\\temp\\test_out.bmp")) { fs.Write(data, 0, data.Length); } </code></pre> <p>All of the ugly initial read code is gone since we're just using a single byte array. Notice I reserved a single 4-byte array before the processing loop to re-use, so we can save the garbage collector some work. Then we loop through the giant data array 4 bytes at a time and copy them into our working array, use that to initialize the BitArrays for your transforms, and then the last statement in the block converts the BitArray back into a byte array, and copies it directly back to its original location within the giant data array. This replaces <code>BitArrayToByteArray</code> method, since you did not provide it. At the end, writing is also easy since it's just slamming out the now-transformed giant data array.</p> <p>When I ran your original solution I got an OutOfMemory exception on my original test file of 100MB, so I used a 44MB file. It consumed 650MB in memory and ran in 30 seconds. The single-array solution used 54MB of memory and ran in 10 seconds. Not a bad improvement, and it demonstrates how bad holding onto millions of small array objects is.</p>
    singulars
    1. This table or related slice is empty.
    1. PORead file into ByteArrays of 4 bytes
      singulars
      1. PTQuestion
    1. PTAnswer
    1. USJustin Aquadro
    1. USJustin Aquadro
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PORead file into ByteArrays of 4 bytes
      singulars
      1. PTQuestion
    1. This table or related slice is empty.
    1. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTAcceptedByOriginator
    2. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    1. COAs an aside, you can get a 4x performance boost in release builds on top of that with a smarter bit-rearranging strategy. Since it's not relevant to the question, you can find it at http://pastebin.com/G8xWEEvG. You'll avoid creating millions of BitArray garbage objects and their overhead, and the compiler will likely optimize out the function call overhead for the bit shifting operations.
      singulars
      1. PO
      1. USJustin Aquadro
    2. COthank you! works like a charm with your solution.
      singulars
      1. PO
      1. USAlan Alvarez
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload