StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
text
Body
copied!<p>If your worker roles write each finished job's details to a table using a PK of something like <code>(DateTime.MaxValue - DateTime.UtcNow).Ticks.ToString("d19")</code>, you will have a sorted list of the latest jobs that have been processed. Set your web role to poll the table like so:</p> <pre><code>var q = ctx.CreateQuery<LatestJobs>("jobstable") .Where(j => j.PartitionKey.CompareTo(LastIndexTime.GetReverseTicks()) < 0) .Take(1) .AsTableServiceQuery() if (q.Count() > 0) { //new jobs exist since last check... re-index. } </code></pre> <p>For worker roles that do the indexing work, this is great because they can write indiscriminately to the table without worry of conflict. For you, you also have an audit log of the jobs they are processing (assuming you put some details in there).</p> <p>However, you have one remaining problem: it sounds like you have 1 web role that updates the index. This one web role can of course poll this table on whatever frequency you choose (just track the LastIndexTime for searching later). Your issue is how to control concurrency of the web role(s) if you have more than one. Does each web role maintain it's own index or do you have one stored somewhere for all? Sorry, but I am not an expert in Lucene if that should be obvious.</p> <p>Anyhow, if you have multiple instances in your WebRole and a single index that all can see, you need to prevent multiple roles from updating the index over and over. You can do this through leasing the index (if stored in blob storage).</p> <p><strong>Update based on comment:</strong></p> <p>If each WebRole instance has its own index, then you don't have to worry about leasing. That is only if they are sharing a blob resource together. So, this technique should work fine as-is and your only potential obstacle is that the polling intervals for the web roles could be slightly out of sync, causing somewhat different results until all update (depending on which instance you hit). Poll every 30 seconds on the table and that will be your max out of sync. Each web role instance simply needs to track the last time it updated and do incremental searches from that point.</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload