StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POPiping multiple file streams using Node.js
text
Body
copied!<p>I want to stream multiple files, one after each other, to the browser. To illustrate, think of having multiple CSS files which shall be delivered concatenated as one.</p> <p>The code I am using is:</p> <pre><code>var directory = path.join(__dirname, 'css'); fs.readdir(directory, function (err, files) { async.eachSeries(files, function (file, callback) { if (!endsWith(file, '.css')) { return callback(); } // (1) var currentFile = path.join(directory, file); fs.stat(currentFile, function (err, stats) { if (stats.isDirectory()) { return callback(); } // (2) var stream = fs.createReadStream(currentFile).on('end', function () { callback(); // (3) }); stream.pipe(res, { end: false }); // (4) }); }, function () { res.end(); // (5) }); }); </code></pre> <p>The idea is that I</p> <ol> <li>filter out all files that do not have the file extension <code>.css</code>.</li> <li>filter out all directories.</li> <li>proceed with the next file once a file has been read completely.</li> <li>pipe each file to the response stream without closing it.</li> <li>end the response stream once all files have been piped.</li> </ol> <p>The problem is that only the first <code>.css</code> file gets piped, and all remaining files are missing. It's as if (3) would directly jump to (5) after the first (4).</p> <p>The interesting thing is that if I replace line (4) with</p> <pre><code>stream.on('data', function (data) { console.log(data.toString('utf8')); }); </code></pre> <p>everything works as expected: I see multiple files. If I then change this code to</p> <pre><code>stream.on('data', function (data) { res.write(data.toString('utf8')); }); </code></pre> <p>all files expect the first are missing again.</p> <p>What am I doing wrong?</p> <p>PS: The error happens using Node.js 0.8.7 as well as using 0.8.22.</p> <p><strong>UPDATE</strong></p> <p>Okay, it works if you change the code as follows:</p> <pre><code>var directory = path.join(__dirname, 'css'); fs.readdir(directory, function (err, files) { var concatenated = ''; async.eachSeries(files, function (file, callback) { if (!endsWith(file, '.css')) { return callback(); } var currentFile = path.join(directory, file); fs.stat(currentFile, function (err, stats) { if (stats.isDirectory()) { return callback(); } var stream = fs.createReadStream(currentFile).on('end', function () { callback(); }).on('data', function (data) { concatenated += data.toString('utf8'); }); }); }, function () { res.write(concatenated); res.end(); }); }); </code></pre> <p>But: Why? Why can't I call <code>res.write</code> multiple times instead of first summing up all the chunks, and then write them all at once?</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload