StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POCreating a Node.js stream from two piped streams
text
Body
copied!<p>I'd like to combine two Node.js streams into one by piping them, if possible. I'm using <a href="http://nodejs.org/api/stream.html#stream_class_stream_transform">Transform</a> streams.</p> <p>In other words, I'd like my library to return <code>myStream</code> for people to use. For example they could write:</p> <pre class="lang-js prettyprint-override"><code>process.stdin.pipe(myStream).pipe(process.stdout); </code></pre> <p>And internally I'm using a third-party <code>vendorStream</code> that does some work, plugged into my own logic contained in <code>myInternalStream</code>. So what's above would translate to:</p> <pre class="lang-js prettyprint-override"><code>process.stdin.pipe(vendorStream).pipe(myInternalStream).pipe(process.stdout); </code></pre> <p>Can I do something like that? I've tried <code>var myStream = vendorStream.pipe(myInternalStream)</code> but that obviously doesn't work.</p> <p>To make an analogy with <code>bash</code>, let's say I want to write a program that checks if the letter <code>h</code> is present in the last line of some stream (<code>tail -n 1 | grep h</code>), I can create a shell script:</p> <pre class="lang-bsh prettyprint-override"><code># myscript.sh tail -n 1 | grep h </code></pre> <p>And then if people do:</p> <pre class="lang-bsh prettyprint-override"><code>$ printf "abc\ndef\nghi" | . myscript.sh </code></pre> <p>It just works.</p> <p>This is what I have so far:</p> <pre class="lang-js prettyprint-override"><code>// Combine a pipe of two streams into one stream var util = require('util') , Transform = require('stream').Transform; var chunks1 = []; var stream1 = new Transform(); var soFar = ''; stream1._transform = function(chunk, encoding, done) { chunks1.push(chunk.toString()); var pieces = (soFar + chunk).split('\n'); soFar = pieces.pop(); for (var i = 0; i < pieces.length; i++) { var piece = pieces[i]; this.push(piece); } return done(); }; var chunks2 = []; var count = 0; var stream2 = new Transform(); stream2._transform = function(chunk, encoding, done) { chunks2.push(chunk.toString()); count = count + 1; this.push(count + ' ' + chunk.toString() + '\n'); done(); }; var stdin = process.stdin; var stdout = process.stdout; process.on('exit', function () { console.error('chunks1: ' + JSON.stringify(chunks1)); console.error('chunks2: ' + JSON.stringify(chunks2)); }); process.stdout.on('error', process.exit); // stdin.pipe(stream1).pipe(stream2).pipe(stdout); // $ (printf "abc\nd"; sleep 1; printf "ef\nghi\n") | node streams-combine.js // Outputs: // 1 abc // 2 def // 3 ghi // chunks1: ["abc\nd","ef\nghi\n"] // chunks2: ["abc","def","ghi"] // Best working solution I could find var stream3 = function(src) { return src.pipe(stream1).pipe(stream2); }; stream3(stdin).pipe(stdout); // $ (printf "abc\nd"; sleep 1; printf "ef\nghi\n") | node streams-combine.js // Outputs: // 1 abc // 2 def // 3 ghi // chunks1: ["abc\nd","ef\nghi\n"] // chunks2: ["abc","def","ghi"] </code></pre> <p>Is this at all possible? Let me know if what I'm trying to do isn't clear.</p> <p>Thanks!</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload