Databases
StackOverflow2013
Postsdbo
POMultiple and Growing Outbound CURL Request
Body
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POMultiple and Growing Outbound CURL Request
    text
    copied!<p>Multi Outbound Request - Curl</p> <p>Here's the problem, I have several clients I have to CURL outbound requests to. Say for instance I currently have 20 clients and I send around 100~1000 requests per minute to each of them. I also use CURL multi however it seems to have a limitation of how many requests it can make at a time and it will also depend on the longest CURL response for the entire routine to complete.</p> <p>For instance I have the following clients:</p> <pre><code>Client 1 url: http://www.exampleclient1.com/process/ Client 2 url: http://www.exampleclient2.com/process/ </code></pre> <p>... and so on</p> <p>The main issue here is that I have a single script that does the job for every client. Say,</p> <pre><code>http://localhost/app/send/client1 &gt; will send out the pending outbound queue to client 1's url http://localhost/app/send/client2 &gt; will send out the pending outbound queue to client 2's url </code></pre> <p>... and so on</p> <p>The reason why I separated them is because there should be dedicated connections between clients and their latencies are different from each other. Some clients respond faster and have faster servers while some clients have slow servers or take more hops to reach.</p> <p>Here's my question: Is there a means to simplify this process? Because it's a hassle that everytime I have to add a client to my database I'll also have to add </p> <pre><code>http://localhost/send/newclient1 http://localhost/send/newclient2 . . . http://localhost/send/newclientn </code></pre> <p>to the list of cronjobs. Is it possible to put it in a single script instead so as my list of clients grow it will not affect the overall performance of the outbound CURL function I have?</p> <p>By the way, I'm using PHP, CURL. If there's a solution that recommends the use of another technology other than PHP for this, a linux queuing manager...etc, you're welcome to suggest.</p> <p>Thanks!</p>
 

Querying!

 
Guidance
An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload