Databases
StackOverflow2013
Postsdbo
POGeneric class for performing mass-parallel queries. Feedback?
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POGeneric class for performing mass-parallel queries. Feedback?
    primarykey
    data
    text
    <p>I don't understand why, but there appears to be no mechanism in the client library for performing many queries in parallel for Windows Azure Table Storage. I've created a template class that can be used to save considerable time, and you're welcome to use it however you wish. I would appreciate however, if you could pick it apart, and provide feedback on how to improve this class.</p> <pre><code>public class AsyncDataQuery&lt;T&gt; where T: new() { public AsyncDataQuery(bool preserve_order) { m_preserve_order = preserve_order; this.Queries = new List&lt;CloudTableQuery&lt;T&gt;&gt;(1000); } public void AddQuery(IQueryable&lt;T&gt; query) { var data_query = (DataServiceQuery&lt;T&gt;)query; var uri = data_query.RequestUri; // required this.Queries.Add(new CloudTableQuery&lt;T&gt;(data_query)); } /// &lt;summary&gt; /// Blocking but still optimized. /// &lt;/summary&gt; public List&lt;T&gt; Execute() { this.BeginAsync(); return this.EndAsync(); } public void BeginAsync() { if (m_preserve_order == true) { this.Items = new List&lt;T&gt;(Queries.Count); for (var i = 0; i &lt; Queries.Count; i++) { this.Items.Add(new T()); } } else { this.Items = new List&lt;T&gt;(Queries.Count * 2); } m_wait = new ManualResetEvent(false); for (var i = 0; i &lt; Queries.Count; i++) { var query = Queries[i]; query.BeginExecuteSegmented(callback, i); } } public List&lt;T&gt; EndAsync() { m_wait.WaitOne(); m_wait.Dispose(); return this.Items; } private List&lt;T&gt; Items { get; set; } private List&lt;CloudTableQuery&lt;T&gt;&gt; Queries { get; set; } private bool m_preserve_order; private ManualResetEvent m_wait; private int m_completed = 0; private object m_lock = new object(); private void callback(IAsyncResult ar) { int i = (int)ar.AsyncState; CloudTableQuery&lt;T&gt; query = Queries[i]; var response = query.EndExecuteSegmented(ar); if (m_preserve_order == true) { // preserve ordering only supports one result per query lock (m_lock) { this.Items[i] = response.Results.Single(); } } else { // add any number of items lock (m_lock) { this.Items.AddRange(response.Results); } } if (response.HasMoreResults == true) { // more data to pull query.BeginExecuteSegmented(response.ContinuationToken, callback, i); return; } m_completed = Interlocked.Increment(ref m_completed); if (m_completed == Queries.Count) { m_wait.Set(); } } } </code></pre>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USAaron
    1. USAaron
    plurals
    1. This table or related slice is empty.
    1. PL
      singulars
      1. LTLinked
    2. PL
      singulars
      1. LTLinked
    3. PL
      singulars
      1. LTLinked
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POGeneric class for performing mass-parallel queries. Feedback?
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POGeneric class for performing mass-parallel queries. Feedback?
      1. USIgorek
      1. VTFavorite
    3. VO
      singulars
      1. POGeneric class for performing mass-parallel queries. Feedback?
      1. This table or related slice is empty.
      1. VTUpMod
    1. COAs a side note, this is running in production (on newscandy.com), and appears to work well for my usage, so far. But there seems to be much room for improvement.
      singulars
      1. POGeneric class for performing mass-parallel queries. Feedback?
      1. USAaron
    2. COFrom a quick look: Don't expose setters for collections, and don't expose concrete collection implementations.
      singulars
      1. POGeneric class for performing mass-parallel queries. Feedback?
      1. USR. Martinho Fernandes
    3. COGood advice. I agree implementation could use more consideration too. Anything as far as functional improvements? Be it for performance or function.
      singulars
      1. POGeneric class for performing mass-parallel queries. Feedback?
      1. USAaron
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload