StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POUnittests testing timers fail indeterministically on TFS
text
Body
copied!<p>We do have a bunch of UnitTests for our projects which run during CheckIn on a TFS 2012 buildserver too. While locally they behave quite like expected, some weird behaviour occurrs with unittests using timers on TFS occassionally. Finally we wrote a class and a test to isolate that behaviour.</p> <p>The class:</p> <pre><code>public class TestTimerOnServer { Timer _MyTimer = new Timer(100); int _CountsToDo = 100; public int TimerElapsedCounter = 0; public TestTimerOnServer() { _MyTimer.Elapsed += _MyTimer_Elapsed; } public void StartCounting(int argNumberOfCounts, int argIntervalInMs) { _MyTimer.Stop(); _MyTimer.Interval = argIntervalInMs; TimerElapsedCounter = 0; _CountsToDo = argNumberOfCounts; _MyTimer.Start(); } void _MyTimer_Elapsed(object sender, System.Timers.ElapsedEventArgs e) { if (_CountsToDo > TimerElapsedCounter) TimerElapsedCounter++; else _MyTimer.Stop(); } } </code></pre> <p>The test running against it:</p> <pre><code>private void TestTimerServerBehaviour() { TestTimerOnServer ttos = new TestTimerOnServer(); ttos.StartCounting(10, 100); // should lead 1000 ms // we wait 1100 MS Thread.Sleep(1100); Assert.AreEqual(ttos.TimerElapsedCounter, 10,"We should have 10 samples, but only " + ttos.TimerElapsedCounter + " have been generated.!"); ttos.StartCounting(50, 100); // should lead 5000 ms // we wait 6000 MS Thread.Sleep(6000); Assert.AreEqual(ttos.TimerElapsedCounter, 50, "We should have 50 samples, but only " + ttos.TimerElapsedCounter + " have been generated.!"); } </code></pre> <p>Most of the time this test passes, occassionally it fails on TFS - firing the event just e.g. twice instead of 50 times. So the question is do we suffer a general unittest design flaw here or are the problems rather to be found in running the test by the TFS build service?</p> <p><strong>EDIT:</strong> The TFS is running on a VM - might this be the origin of our problems?</p> <p><strong>EDIT2 - final investigation results</strong> The problem obviously was the combination of Testing-Context and VMware. I wrote small application using the code posted by @allen, a test using the same code and another one to cause load on the cpu:</p> <pre><code>void CPULoader() { var action = new Action(() => { while (true);}); Parallel.Invoke(action, action, action, action, action, action, action, action); } </code></pre> <p>Quite primitive though efficient.</p> <p>On my hardware machine running the application and the CPU-Loader causes the timer event to occur a little later, e.g. 1100 ms instead of 1000. That's fine so far, Win7 is no realtime-system after all.</p> <p>On a VMware the some combination causes delays up to 5000 ms where there should be one second... - much worse of course, but still the event occurs.</p> <p>The real fun starts when running the test (I've packed the code into an infinite loop for that purpose) and then firing up the CPU-Loader. The whole system is very slow then until I stop loading the CPU and <em>the event does not occur before</em> in the test.</p> <p>Hope this can give someone some hints - we have fixed our tests by using a Timer abstraction and mocking the timing behaviour while unittesting. As a side effect the tests run faster now :)</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload