Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>I think you're going to need something a little more complex to hit your reliability target.</p> <p>My understanding of your problem is that you have one producer and an unknown number of consumers all of which are different processes. Each consumer can NEVER miss any events.</p> <p>I'd like more clarification as to what missing an event means.</p> <p>i) if a consumer started to run and got to just before it waited on your notification method and an event occurred should it process it even though it wasn't quite ready at the point that the notification was sent? (i.e. when is a consumer considered to be active? when it starts or when it processes its first event)</p> <p>ii) likewise, if the consumer is processing an event and the code that waits on the next notification hasn't yet begun its wait (I'm assuming a <code>Wait -&gt; Process -&gt; Loop to Wait</code> code structure) then should it know that another event occurred whilst it was looping around?</p> <p>I'd assume that i) is a "not really" as it's a race between process start up and being "ready" and ii) is "yes"; that is notifications are, effectively, queued per consumer once the consumer is present and each consumer gets to consume all events that are produced whilst it's active and doesn't get to skip any.</p> <p>So, what you're after is the ability to send a stream of notifications to a set of consumers where a consumer is guaranteed to act on all notifications in that stream from the point where it acts on the first to the point where it shuts down. i.e. if the producer produces the following stream of notifications</p> <p>1 2 3 4 5 6 7 8 9 0</p> <p>and consumer a) starts up and processes 3, it should also process 4-0</p> <p>if consumer b) starts up and processes 5 but is shut down after 9 then it should have processed 5,6,7,8,9</p> <p>if consumer c) was running when the notifications began it should have processed 1-0</p> <p>etc.</p> <p>Simply pulsing an event wont work. If a consumer is not actively waiting on the event when the event is pulsed then it will miss the event so we will fail if events are produced faster than we can loop around to wait on the event again.</p> <p>Using a semaphore also wont work as if one consumer runs faster than another consumer to such an extent that it can loop around to the semaphore call before the other completes processing and if there's another notification within that time then one consumer could process an event more than once and one could miss one. That is you may well release 3 threads (if the producer knows there are 3 consumers) but you cant ensure that each consumer is released just the once. </p> <p>A ring buffer of events (tick counts) in shared memory with each consumer knowing the value of the event it last processed and with consumers alerted via a pulsed event should work at the expense of some of the consumers being out of sync with the ticks sometimes; that is if they miss one they will catch up next time they get pulsed. As long as the ring buffer is big enough so that all consumers can process the events before the producer loops in the buffer you should be OK.</p> <p>With the example above, if consumer d misses the pulse for event 4 because it wasn't waiting on its event at the time and it then settles into a wait it will be woken when event 5 is produced and since it's last processed counted is 3 it will process 4 and 5 and then loop back to the event...</p> <p>If this isn't good enough then I'd suggest something like <a href="http://research.microsoft.com/apps/pubs/default.aspx?id=68888" rel="nofollow noreferrer">PGM</a> via sockets to give you a reliable multicast; the advantage of this would be that you could move your consumers off onto different machines...</p>
    singulars
    1. This table or related slice is empty.
    1. POWhat are alternatives to Win32 PulseEvent() function?
      singulars
      1. PTQuestion
    1. PTAnswer
    1. USLen Holgate
    1. USLen Holgate
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. POWhat are alternatives to Win32 PulseEvent() function?
      singulars
      1. PTQuestion
    1. This table or related slice is empty.
    1. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTAcceptedByOriginator
    3. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    1. COThanks. You have described my situation very well. Indeed, this is exactly what I have now - a counter in shared memory, a ring buffer with event data, and a consumer that "knows" if it missed an event occasionally because it was not waiting when the PulseEvent occurred. I was "fishing" here on StackOverflow to see if I can do better. We miss about 1 or 2 events per week (running 24/7 at 20ms per event and about 10 consumers), which is nearly perfect. A "missed event" cannot be acted upon - it is only valid for a brief moment.
      singulars
      1. PO
      1. USBill
    2. COPGM would likely give you too much latency. Given that missed events cant be acted on I'm tempted to say that you're operating as intended. If the consumer isn't ready for a new event when one occurs and it can't catch up then surely all it can do is skip that event, which is what is happening.
      singulars
      1. PO
      1. USLen Holgate
    3. COThis video (http://channel9.msdn.com/shows/Going+Deep/Arun-Kishan-Farewell-to-the-Windows-Kernel-Dispatcher-Lock/) explains why Pulse is unreliable and deprecated.
      singulars
      1. PO
      1. USJeff Allen
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload