StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
text
Body
copied!<p>Depending on your application queueing might not solve your problem at all. If you build up a queue your server will have less work to do at the same time because less concurrent threads are running.</p> <p><strong>When to queue?</strong></p> <p>Obviously i've to reverse the question: Why are you queuing? Propably because your servers just can't handle the amounts of incoming traffic. If we're talking about peaks which last only for a few seconds this is not a big deal: Build up a queue and process it in a few seconds.</p> <p><strong>A queue is just a bucket</strong></p> <p>As your question states this peaks last longer than a few seconds "several points throughout the year". So i assume that your servers are experiencing massive inrush of new requests for a time which is long enough that processing the queue would just result in timeouts on the clients. [FYI: Check out the bucket model in networking. These has the same issues to deal with]</p> <p>Let me give an easy example:</p> <p>It's a normal day in summer and you're offering cold lemonade on a street corner. You can handle up to 5 clients per minute and the peaks during lunch mean that about 7 clients are queuing up for your lemonade per minute. After one minute you've 2 clients waiting, after two minutes you have 4 clients waiting for you to process their orders [and so on].</p> <p>As the peak inrush just holds for let's say 10 minutes you're queue just reaches a maximum length of 20 clients.</p> <p>On another day it's <em>very</em> hot and your inrush peaks are not the usual 7 clients per minute, your inrush peaks up to 42 clients per minute. This means that already after one minute you've 37 clients waiting for your lemonade. It's obviously that - except it's the last lemonade on earth - your clients won't queue an hour for a cup of lemonade and will just leave.</p> <p>And this is the same problem concerning your peaks on the server: If you're continously experiencing more incoming connections than you can handle your queue will grow until it's full and then drop incoming clients. So all you're gaining with queuing is that you just delay the first dropped clients and put other clients on hold (which is also annoying).</p> <p>Going back the the real world example. How can this problem be solved here? It's quiet easy: speed up processing each client or get some clerks and open up a second, third,... queue to handle multiple clients at once and process all orders. As in real world and in programming: speeding up is not that easy. getting up multiple instances just for short periods of time is also quiet difficult. But clearly spoken: these are the only reliable solutions to handle such problems.</p> <p><strong>Using IIS and MVC: Not really queuing at the right place</strong></p> <p>[Queuing up on a single queue] You may build up a queue in your application and suspend all inrushing requests if you've reached your maximum concurrent active workers. But this doesn't mean that there is no overhead for all the inrushing tcp connections and asp sessions which are spawned and loaded on each request as IIS accepts new connections if there are free concurrent connections from the IIS' viewpoint.</p> <p><strong>Building a http proxy application</strong></p> <p>[Queuing up on a single or multiple queues] If you're building a custom proxy application you may be able decide yourself when you want to accept new incoming client connections (suspending the tcp listener or just not accepting new clients until your workload has enough resources to handle the new clients). Of course your proxy could spread the requests to several hosts and do some sort of load balancing - but this requires that you have additional hosts you can instantly fire up and handle the requests (this is what any loadbalancer would require from you: have a backend which may handle the workload). As this is an absolutely low level apporach i think it's not required to go into further detail and explain that there are plenty of risks when taking this road.</p> <p>Concerning some simple aspects to just reduce <em>some</em> parts of the workload: - Offload static contents like images to other servers / sites which are low cost. This reduces the workload on your primary server which do the hard, dynamic work. - Caching: Don't do things twice which have the same results. Check where you're application gets slow in high workloads and try to optimize them to be as fast as possible. - Work async / put long runners into sleep: If you've external requests (database) try to implement them in an async manner to the your threads be running as long as possible and then go to sleep to give the scheduler of your host the chance to process other pending work. Benchmark your threads and check how long the inclusive times of your critical methods are and where they are spending their time.</p> <p>Hope this helps to solve your problem or just get's you an idea what road to take.</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload