Databases
StackOverflow2013
Postsdbo
POAzure Table Storage transaction limitations
Body
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POAzure Table Storage transaction limitations
    text
    copied!<p>I'm running performance tests against ATS and its behaving a bit weird when using multiple virtual machines against the same table / storage account.</p> <p>The entire pipeline is non blocking (await/async) and using TPL for concurrent and parallel execution.</p> <p>First of all its very strange that with this setup i'm only getting about 1200 insertions. This is running on a L VM box, that is 4 cores + 800mbps.</p> <p>I'm inserting 100.000 rows with unique PK and unique RK, that should leverage the ultimate distribution.</p> <p>Even more deterministic behavior is the following.</p> <p>When I run 1 VM i get about 1200 insertions per second. When I run 3 VM i get about 730 on each insertions per second.</p> <p>Its quite humors to read the blog post where they are specifying their targets. <a href="https://azure.microsoft.com/en-gb/blog/windows-azures-flat-network-storage-and-2012-scalability-targets/" rel="nofollow noreferrer">https://azure.microsoft.com/en-gb/blog/windows-azures-flat-network-storage-and-2012-scalability-targets/</a> </p> <blockquote> <p>Single Table Partition– a table partition are all of the entities in a table with the same partition key value, and usually tables have many partitions. The throughput target for a single table partition is:</p> <p>Up to 2,000 entities per second</p> <p>Note, this is for a single partition, and not a single table. Therefore, a table with good partitioning, can process up to the 20,000 entities/second, which is the overall account target described above.</p> </blockquote> <p>What shall I do to be able to utilize the 20k per second, and how would it be possible to execute more than 1,2k per VM?</p> <p>--</p> <p>Update:</p> <p>I've now also tried using 3 storage accounts for each individual node and is still getting the performance / throttling behavior. Which i can't find a logical reason for.</p> <p>--</p> <p>Update 2:</p> <p>I've optimized the code further and now i'm possible to execute about 1550.</p> <p>--</p> <p>Update 3:</p> <p>I've now also tried in US West. The performance is worse there. About 33% lower.</p> <p>--</p> <p>Update 4:</p> <p>I tried executing the code from a XL machine. Which is 8 cores instead of 4 and the double amount of memory and bandwidth and got a 2% increase in performance so clearly this problem is not on my side..</p>
 

Querying!

 
Guidance
An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload