Databases
StackOverflow2013
Postsdbo
PONodejs MySQL Query Blocks Request Module
Body
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PONodejs MySQL Query Blocks Request Module
    text
    copied!<p>I'm a nodejs newbie and getting used to all the async stuff.</p> <p>Using the: - Node-MySQL (https://github.com/felixge/node-mysql) - Request (https://github.com/mikeal/request) modules right now.</p> <p>I'm querying a big list of URLs (10,000 URLs) and trying to send them a HTTP GET request to get their HTTP statuses.</p> <p>It works but the problem is, when I run the code, when each row from the MySQL query is returned, the processURL function is called, it logs the 'BEFORE' and 'AFTER' just after the each URL value, (as far as I can understand) sends the HTTP request, but waits for the MySQL query to complete before returning the HTTP statuses.</p> <p>Simply, it can not check these URLs and return their results while the query is running. It waits for the query to complete to return the results of the HTTP requests.</p> <p>This looks meaningless to me as it runs the processURL function when each row is streamed. Here is the full code:</p> <pre><code>var mysql = require('mysql'); var connection = mysql.createConnection({ host : 'localhost', user : 'root', password : '', database : 'urlDB', }); var query = connection.query('SELECT * FROM urls LIMIT 10000'); query.on('error', function(err) { console.log(err); }) query.on('result', function(row) { processURL(row.urlValue, function() { }); console.log(row.urlValue); }) function processURL (url){ var request = require('request'); var startTime = new Date(); console.log('BEFORE'); request({ uri: url, method: 'GET', }, function(error, response, body) { console.log('INSIDE'); var endTime = new Date(); if (!error) { console.log(response.statusCode + ' Start: ' + startTime + ' End: ' + endTime); } else { console.log('timeout'); } }) console.log('AFTER'); } </code></pre> <p>The output is as below: www.google.com BEFORE AFTER www.yahoo.com BEFORE AFTER www.cnn.com BEFORE AFTER ...but no 'INSIDE' until the query completes.</p> <p>Any help is appreciated and thanks very much.</p>
 

Querying!

 
Guidance
An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload