Databases
StackOverflow2013
Postsdbo
POInexplicable Mathematica7 DumpSave[] problem
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POInexplicable Mathematica7 DumpSave[] problem
    primarykey
    data
    text
    <p>I have a very large array of floating point image data:</p> <pre><code>In[25]:= Dimensions[daylightImgd] Out[25]= {18, 2002, 2989} In[26]:= daylightImgd[[1, 1]][[1 ;; 10]] Out[26]= {0.0122293, 0.0104803, 0.0103955, 0.0115533, 0.0118063, \ 0.0120648, 0.0122957, 0.011398, 0.0117426, 0.0119997} </code></pre> <p>I can save this whole image array to disk successfully using DumpSave a la:</p> <pre><code>DumpSave["thisWorks.mx", daylightImgd] </code></pre> <p>Dumping this giant (861 meg file) takes less than 10 seconds. If I downsample these images, a la:</p> <pre><code>downsample[image_, f_] := Module[{w, h}, h = Dimensions[image][[1]]; w = Dimensions[image][[2]]; Table[image[[i, j]], {i, 1, h, f}, {j, 1, w, f}]] In[26]:= daylightImgdDown = downsample[#, 4] &amp; /@ daylightImgd; In[27]:= Dimensions[daylightImgdDown] Out[27]= {18, 500, 748} In[28]:= daylightImgdDown[[1, 1]][[1 ;; 10]] Out[28]= {0.0122293, 0.0118063, 0.0117426, 0.0119349, 0.0109443, \ 0.0121632, 0.0121304, 0.00681408, 0.0101728, 0.00603242} </code></pre> <p>Then suddenly I can't dumpsave anymore; the thing hangs and spins forever -- or at least for many minutes, till I kill it, and maxes CPU:</p> <pre><code>In[31]:= DumpSave["broken.mx", daylightImgdDown]; (* Hangs forever *) </code></pre> <p>So far as I can determine, everything is as it should be: the downsampled images have the right dimensions; you can plot them via ArrayPlot and everything looks great; manually listing the first row looks fine. Everything, in short, appears the same as with the non-down-sampled images, yet on the much smaller dataset DumpSave hangs.</p> <p>Help?</p> <p><strong>UPDATE: Comments on Michael's answer</strong></p> <p>Wow. Thank you for the extremely thorough answer, which not only answered my question but taught me a bunch of peripheral stuff, too.</p> <p>For your reference, the issue of packed-ness was a little trickier than just replacing my downsample[] with one of yours. Since the array I was trying to dump is an array of 18 images - a 3d array, in other words - and since I'm applying the downsampling via the Map operator, the packed-ness of the 3d array is false (according to PackedArrayQ) using either of your downsample rewrites.</p> <p>However, if I take the output of those applications, and then pack the resultant 3d array, <em>then</em> the whole 3d array is packed, and <em>only then</em> can I DumpSave it. Weirdly, though, this final process of packing, while necessary for a successful DumpSave, barely seems to alter the size, as reported by ByteCount. Maybe this is easier in code:</p> <pre><code>In[42]:= downsample3[image_, f_] := Module[{w, h}, h = Dimensions[image][[1]]; w = Dimensions[image][[2]]; Developer`ToPackedArray@Table[image[[i, j]], {i, 1, h, f}, {j, 1, w, f}]] In[43]:= daylightImgdDown = downsample3[#, downsampleSize] &amp; /@ daylightImgd; In[44]:= ByteCount[daylightImgdDown] Out[44]= 53966192 In[45]:= Developer`PackedArrayQ[daylightImgdDown] Out[45]= False In[46]:= dd = Developer`ToPackedArray[daylightImgdDown]; In[47]:= Developer`PackedArrayQ[dd] Out[47]= True In[48]:= ByteCount[dd] Out[48]= 53963844 In[49]:= DumpSave["daylightImgdDown.mx", dd]; (* works now! *) </code></pre> <p>Again, thanks very much.</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USshanusmagnus
    1. USshanusmagnus
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POInexplicable Mathematica7 DumpSave[] problem
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POInexplicable Mathematica7 DumpSave[] problem
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. POInexplicable Mathematica7 DumpSave[] problem
      1. USTimo
      1. VTFavorite
    1. This table or related slice is empty.
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload