StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POAzure Service Bus - Two Way Communication Performance Challenge
text
Body
copied!<p>I need to establish a two-way communication between a Publisher and a Subscriber. This is to facilitate a front-end MVC3 application defining a Subscription with a Correlation Filter, and then placing a message onto a Topic. Finally, the MVC3 controller calls BeginReceive() on the SubscriptionClient, and awaits the response.</p> <p>The issue seems to be the creation and deletion of these Subscription objects. The overhead is enormous, and it slows the application to a crawl. This is not to mention the various limitations to work around, such as no more than 2000 Subscriptions on a Topic.</p> <p>What is the best practice for establishing this kind of two-way communication between a Publisher and Subscriber? We want the MVC3 app to publish a message and then wait for a response to that exact message (via the CorrelationId property and a CorrelationFilter). We already cache the NamespaceManager and MessagingFactory, as those are also prohibitively expensive, resource-wise, and also because we were advised that Service Bus uses an explicit provisioning model, where we are expected to pre-create most of these things during role startup.</p> <p>So, this leaves us with the challenge of correlating request to response, and having this tremendous overhead of the creation and deletion of Subscriptions. What better practice exists? Should we keep a cache of SubscriptionClients, and swap the Filter each time? What does everyone else do? I need to have a request throughput on the order of 5 to 10 thousand MVC3 requests per second through the Web Role cluster. We are already using AsyncController and employing the asynchronous BeginReceive() on SubscriptionClient. It appears to be the creation and deletion of the Subscriptions by the thousands that is choking the system at this point.</p> <p><strong>UPDATE1:</strong> Based on the great advice provided here, we have updated this solution to keep a cache of SubscriptionClient objects on each web role instance. Additionally, we have migrated to a MessageSession oriented approach.</p> <p>However, this is still not scaling. It seems that AcceptMessageSession() is a very expensive operation. Should MessageSession objects also be cached and re-used? Does each open MessageSession object consume a connection to the Service Bus? If so, is this counted against the Subscription's concurrent connection quota?</p> <p>Many thanks. I think we are getting there. Most of the example code on the web shows: Create Topic(), then CreateSubscription(), then CreateSubscriptionClient(), then BeginReceive() on the client, then teardown of all of the objects. All I can say is if you did this in real life, your server would be crushed, and you would max out on connections in no time.</p> <p>We need to put thousands of requests per second through this thing, and it is very apparent that these objects must be cached and reused heavily. So, is MessageSession yet another item to cache? I will have fun caching that, because we will have to implement a reference counting mechanism, where only one reference to the MessageSession can be given out at a time, since this is for http request-specific request/response, and we cannot have other subscribers using the MessageSession objects concurrently.</p> <p><strong>UPDATE2:</strong> OK, it is not feasible to cache MessageSession for re-use, because they only live as long as the LockDuration on the Subscription. This is a bummer, because the maximum LockDuration is 5 minutes. These appear to be for pub/sub of short duration, not for long-running distributed processes. It looks like we need to back to polling Azure Tables.</p> <p><strong>SUMMARY/COMMENTARY</strong> We tried to build on Service Bus because of the scale potential and its durability and delivery semantics. However, it seems that there are situations, high-volume request/response among them, that are not suited to it. The publishing part works great, and having competing consumers on the back-end is great, but having a front-end request wait on a defined, single-consumer response, does not scale well at all, because the MessageSessions take way too long to create via AcceptMessageSession() or BeginAcceptMessageSession(), and because they are not suited to caching.</p> <p>If someone has an alternative view, I would love to hear it.</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload