Databases
StackOverflow2013
Postsdbo
POOpen/read command in Tcl 8.5 for large files
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POOpen/read command in Tcl 8.5 for large files
    primarykey
    data
    text
    <p>Sorry if the title doesn't match my question well, I'm still unsure as to how I should put it.</p> <p>Anyway, I've been using Tcl/Tk on Windows (<code>wish</code>) for a while now and haven't encountered any problem on the script I wrote until recently. The script is supposed to break down a large txt file into smaller files that can be imported to excel (I'm talking about breaking down a file with maybe 25M lines which comes around 2.55 GB).</p> <p>My current script is something like that:</p> <pre><code>set data [open "file.txt" r] set data1 [open "File Part1.txt" w] set data2 [open "File Part2.txt" w] set data3 [open "File Part3.txt" w] set data4 [open "File Part4.txt" w] set data5 [open "File Part5.txt" w] set count 0 while {[gets $data line] != -1} { if {$count &gt; 4000000} { puts $data5 $line } elseif {$count &gt; 3000000} { puts $data4 $line } elseif {$count &gt; 2000000} { puts $data3 $line } elseif {$count &gt; 1000000} { puts $data2 $line } else { puts $data1 $line } incr count } close $data close $data1 close $data2 close $data3 close $data4 close $data5 </code></pre> <p>And I alter the numbers within the <code>if</code> to get the desired number of lines per file, or add/remove any <code>elseif</code> where required.</p> <p>The problem is, with the latest file I got, I end up with only about half the data (1.22 GB instead of 2.55 GB) and I was wondering if there was a line which told Tcl to ignore the limit that it can read. I tried to look for it, but I didn't find anything (or anything that I could understand well; I'm still quite the amateur at Tcl ^^;). Can anyone help me?</p> <p><strong>EDIT (update):</strong> I found a program to open large text files and managed to get a preview of the contents of the file directly. There are actually 16,756,263 lines. I changed the script to:</p> <pre><code>set data [open "file.txt" r] set data1 [open "File Part1.txt" w] set count 0 while {[gets $data line] != -1} { incr count } puts $data1 $count close $data close $data1 </code></pre> <p>to get where the script is blocking and it stopped here: <img src="https://i.stack.imgur.com/lhhmf.png" alt="enter image description here"></p> <p>There's a character that the text editor is not recognising in the middle line showing as a little square. I tried to use <code>fconfigure</code> like evil otto suggested but I'm afraid I don't quite understand how the <code>channelID</code>, <code>name</code> or <code>value</code> work exactly to escape that character. Um... help?</p> <p><strong>reEDIT</strong> : I managed to find out how <code>fconfigure</code> worked! Thanks evil otto! Um, I'm not sure how I can 'choose' your answer since it's a comment instead of a proper answer...</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USJerry
    1. USJerry
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POOpen/read command in Tcl 8.5 for large files
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POOpen/read command in Tcl 8.5 for large files
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. POOpen/read command in Tcl 8.5 for large files
      1. This table or related slice is empty.
      1. VTUpMod
    1. COIs 1.22 a combined size of resulting files? Is input file text or binary?
      singulars
      1. POOpen/read command in Tcl 8.5 for large files
      1. USpn8830
    2. COThat should work; large-file support was added in 8.4 and you're not trying to read all the data in at once…
      singulars
      1. POOpen/read command in Tcl 8.5 for large files
      1. USDonal Fellows
    3. COIs it possible there is any binary data in "file.txt"? Under windows, tcl will flag eof if it reads a `^Z` (the default `eofchar`) in a file. You can turn this off with `fconfigure` - see the docs for details.
      singulars
      1. POOpen/read command in Tcl 8.5 for large files
      1. USevil otto
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload