StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PORead a whole text file into a MATLAB variable at once
text
Body
copied!<p>I would like to read a (fairly big) log file into a MATLAB string cell in one step. I have used the usual:</p> <pre><code>s={}; fid = fopen('test.txt'); tline = fgetl(fid); while ischar(tline) s=[s;tline]; tline = fgetl(fid); end </code></pre> <p>but this is just slow. I have found that </p> <pre><code>fid = fopen('test.txt'); x=fread(fid,'*char'); </code></pre> <p>is way faster, but I get a <code>nx1</code> char matrix, <code>x</code>. I could try and convert <code>x</code> to a string cell, but then I get into char encoding hell; line delimiter seems to be \n\r, or 10 and 56 in <a href="http://en.wikipedia.org/wiki/ASCII" rel="nofollow noreferrer">ASCII</a> (I've looked at the end of the first line), but those two characters often don't follow each other and even show up solo sometimes.</p> <p>Is there an easy fast way to read an ASCII file into a string cell in one step, or convert <code>x</code> to a string cell?</p> <p>Reading via fgetl:</p> <pre><code>Code Calls Total Time % Time tline = lower(fgetl(fid)); 903113 14.907 s 61.2% </code></pre> <p>Reading via fread:</p> <pre><code>>> tic;for i=1:length(files), fid = open(files(i).name);x=fread(fid,'*char*1');fclose(fid); end; toc Elapsed time is 0.208614 seconds. </code></pre> <p>I have tested preallocation, and it does not help :(</p> <pre><code>files=dir('.'); tic for i=1:length(files), if files(i).isdir || isempty(strfind(files(i).name,'.log')), continue; end %# preassign s to some large cell array sizS = 50000; s=cell(sizS,1); lineCt = 1; fid = fopen(files(i).name); tline = fgetl(fid); while ischar(tline) s{lineCt} = tline; lineCt = lineCt + 1; %# grow s if necessary if lineCt > sizS s = [s;cell(sizS,1)]; sizS = sizS + sizS; end tline = fgetl(fid); end %# remove empty entries in s s(lineCt:end) = []; end toc </code></pre> <p>Elapsed time is 12.741492 seconds.</p> <p>Roughly 10 times faster than the original:</p> <pre><code>s = textscan(fid, '%s', 'Delimiter', '\n', 'whitespace', '', 'bufsize', files(i).bytes); </code></pre> <p>I had to set <code>'whitespace'</code> to <code>''</code> in order to keep the leading spaces (which I need for parsing), and 'bufsize' to the size of the file (the default 4000 threw a buffer overflow error).</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload