StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PODuplicate keys in Dictionary when using PhysicalAddress as the key
text
Body
copied!<p>So I have run into an interesting problem where I am getting duplicate keys in C# Dictionary when using a key of type PhysicalAddress. It is interesting because it only happens after a very long period of time, and I cannot reproduce it using the same code in a unit test on a completely different machine. I can reproduce it reliably on a Windows XP SP3 machine but only after letting it run for days at a time, and even then it only occurs once.</p> <p>Below is the code that I am using and beneath that is the log output for that part of the code.</p> <p>Code:</p> <pre><code>private void ProcessMessages() { IDictionary<PhysicalAddress, TagData> displayableTags = new Dictionary<PhysicalAddress, TagData>(); while (true) { try { var message = incomingMessages.Take(cancellationToken.Token); VipTagsDisappeared tagsDisappeared = message as VipTagsDisappeared; if (message is VipTagsDisappeared) { foreach (var tag in tagDataRepository.GetFromTagReports(tagsDisappeared.Tags)) { log.DebugFormat(CultureInfo.InvariantCulture, "Lost tag {0}", tag); RemoveTag(tag, displayableTags); } LogKeysAndValues(displayableTags); PublishCurrentDisplayableTags(displayableTags); } else if (message is ClearAllTags) { displayableTags.Clear(); eventAggregator.Publish(new TagReaderError()); } else if (message is VipTagsAppeared) { foreach (TagData tag in tagDataRepository.GetFromTagReports(message.Tags)) { log.DebugFormat(CultureInfo.InvariantCulture, "Detected tag ({0}) with Exciter Id ({1})", tag.MacAddress, tag.ExciterId); if (tagRules.IsTagRssiWithinThreshold(tag) && tagRules.IsTagExciterValid(tag)) { log.DebugFormat(CultureInfo.InvariantCulture, "Detected tag is displayable ({0})", tag); bool elementAlreadyExists = displayableTags.ContainsKey(tag.MacAddress); if (elementAlreadyExists) { displayableTags[tag.MacAddress].Rssi = tag.Rssi; } else { displayableTags.Add(tag.MacAddress, tag); } } else { log.DebugFormat(CultureInfo.InvariantCulture, "Detected tag is not displayable ({0})", tag); RemoveTag(tag, displayableTags); } } LogKeysAndValues(displayableTags); PublishCurrentDisplayableTags(displayableTags); } else { log.WarnFormat(CultureInfo.InvariantCulture, "Received message of unknown type {0}.", message.GetType()); } } catch (OperationCanceledException) { break; } } } private void PublishCurrentDisplayableTags(IDictionary<PhysicalAddress, TagData> displayableTags) { eventAggregator.Publish(new CurrentDisplayableTags(displayableTags.Values.Distinct().ToList())); } private void RemoveTag(TagData tag, IDictionary<PhysicalAddress, TagData> displayableTags) { displayableTags.Remove(tag.MacAddress); // Now try to remove any duplicates and if there are then log it out bool removalWasSuccesful = displayableTags.Remove(tag.MacAddress); while (removalWasSuccesful) { log.WarnFormat(CultureInfo.InvariantCulture, "Duplicate tag removed from dictionary: {0}", tag.MacAddress); removalWasSuccesful = displayableTags.Remove(tag.MacAddress); } } private void LogKeysAndValues(IDictionary<PhysicalAddress, TagData> displayableTags) { log.TraceFormat(CultureInfo.InvariantCulture, "Keys"); foreach (var physicalAddress in displayableTags.Keys) { log.TraceFormat(CultureInfo.InvariantCulture, "Address: {0}", physicalAddress); } log.TraceFormat(CultureInfo.InvariantCulture, "Values"); foreach (TagData physicalAddress in displayableTags.Values) { log.TraceFormat(CultureInfo.InvariantCulture, "Address: {0} Name: {1}", physicalAddress.MacAddress, physicalAddress.Name); } } </code></pre> <p>And process messages is used as follows:</p> <pre><code>Thread processingThread = new Thread(ProcessMessages); </code></pre> <p><em>GetFromTagReports Code</em></p> <pre><code>public IEnumerable<TagData> GetFromTagReports(IEnumerable<TagReport> tagReports) { foreach (var tagReport in tagReports) { TagData tagData = GetFromMacAddress(tagReport.MacAddress); tagData.Rssi = tagReport.ReceivedSignalStrength; tagData.ExciterId = tagReport.ExciterId; tagData.MacAddress = tagReport.MacAddress; tagData.Arrived = tagReport.TimeStamp; yield return tagData; } } public TagData GetFromMacAddress(PhysicalAddress macAddress) { TagId physicalAddressToTagId = TagId.Parse(macAddress); var personEntity = personFinder.ByTagId(physicalAddressToTagId); if (personEntity.Person != null && !(personEntity.Person is UnknownPerson)) { return new TagData(TagType.Person, personEntity.Person.Name); } var tagEntity = tagFinder.ByTagId(physicalAddressToTagId); if (TagId.Invalid == tagEntity.Tag) { return TagData.CreateUnknownTagData(macAddress); } var equipmentEntity = equipmentFinder.ById(tagEntity.MineSuiteId); if (equipmentEntity.Equipment != null && !(equipmentEntity.Equipment is UnknownEquipment)) { return new TagData(TagType.Vehicle, equipmentEntity.Equipment.Name); } return TagData.CreateUnknownTagData(macAddress); } </code></pre> <p><em>Where Physical Address is created</em></p> <pre><code>var physicalAddressBytes = new byte[6]; ByteWriter.WriteBytesToBuffer(physicalAddressBytes, 0, protocolDataUnit.Payload, 4, 6); var args = new TagReport { Version = protocolDataUnit.Version, MacAddress = new PhysicalAddress(physicalAddressBytes), BatteryStatus = protocolDataUnit.Payload[10], ReceivedSignalStrength = IPAddress.NetworkToHostOrder(BitConverter.ToInt16(protocolDataUnit.Payload, 12)), ExciterId = IPAddress.NetworkToHostOrder(BitConverter.ToInt16(protocolDataUnit.Payload, 14)) }; public static void WriteBytesToBuffer(byte[] oldValues, int oldValuesStartindex, byte[] newValues, int newValuesStartindex, int max) { var loopmax = (max > newValues.Length || max < 0) ? newValues.Length : max; for (int i = 0; i < loopmax; ++i) { oldValues[oldValuesStartindex + i] = newValues[newValuesStartindex + i]; } } </code></pre> <p>Note the following:</p> <ul> <li>Every 'Tag' in messages.Tags contains a 'new' PhysicalAddress.</li> <li>Each TagData that is returned is also 'new'.</li> <li>The 'tagRules' methods do not modify the passed in 'tag' in any way.</li> <li>Individual testing with trying to put two instances of a PhysicalAddress (that were created from the same bytes) into a Dictionary throws a 'KeyAlreadyExists' exception.</li> <li>I also tried TryGetValue and it produced the same result.</li> </ul> <p>Log output where everything was fine:</p> <pre><code>2013-04-26 18:28:34,347 [8] DEBUG ClassName - Detected tag (000CCC756081) with Exciter Id (0) 2013-04-26 18:28:34,347 [8] DEBUG ClassName - Detected tag is displayable (Unknown: ?56081) 2013-04-26 18:28:34,347 [8] TRACE ClassName - Keys 2013-04-26 18:28:34,347 [8] TRACE ClassName - Address: 000CCC755898 2013-04-26 18:28:34,347 [8] TRACE ClassName - Address: 000CCC756081 2013-04-26 18:28:34,347 [8] TRACE ClassName - Address: 000CCC755A27 2013-04-26 18:28:34,347 [8] TRACE ClassName - Address: 000CCC755B47 2013-04-26 18:28:34,347 [8] TRACE ClassName - Values 2013-04-26 18:28:34,347 [8] TRACE ClassName - Address: 000CCC755898 Name: Scotty McTester 2013-04-26 18:28:34,347 [8] TRACE ClassName - Address: 000CCC756081 Name: ?56081 2013-04-26 18:28:34,347 [8] TRACE ClassName - Address: 000CCC755A27 Name: JDTest1 2013-04-26 18:28:34,347 [8] TRACE ClassName - Address: 000CCC755B47 Name: 33 1 2013-04-26 18:28:34,347 [8] TRACE ClassName - Current tags: Scotty McTester, ?56081, JDTest1, 33 1 </code></pre> <p>Log output where we get a duplicate key:</p> <pre><code>2013-04-26 18:28:35,608 [8] DEBUG ClassName - Detected tag (000CCC756081) with Exciter Id (0) 2013-04-26 18:28:35,608 [8] DEBUG ClassName - Detected tag is displayable (Unknown: ?56081) 2013-04-26 18:28:35,608 [8] TRACE ClassName - Keys 2013-04-26 18:28:35,608 [8] TRACE ClassName - Address: 000CCC755898 2013-04-26 18:28:35,608 [8] TRACE ClassName - Address: 000CCC756081 2013-04-26 18:28:35,618 [8] TRACE ClassName - Address: 000CCC755A27 2013-04-26 18:28:35,618 [8] TRACE ClassName - Address: 000CCC755B47 2013-04-26 18:28:35,618 [8] TRACE ClassName - Address: 000CCC756081 2013-04-26 18:28:35,618 [8] TRACE ClassName - Values 2013-04-26 18:28:35,618 [8] TRACE ClassName - Address: 000CCC755898 Name: Scotty McTester 2013-04-26 18:28:35,618 [8] TRACE ClassName - Address: 000CCC756081 Name: ?56081 2013-04-26 18:28:35,648 [8] TRACE ClassName - Address: 000CCC755A27 Name: JDTest1 2013-04-26 18:28:35,648 [8] TRACE ClassName - Address: 000CCC755B47 Name: 33 1 2013-04-26 18:28:35,648 [8] TRACE ClassName - Address: 000CCC756081 Name: ?56081 2013-04-26 18:28:35,648 [8] TRACE ClassName - Current tags: Scotty McTester, ?56081, JDTest1, 33 1, ?56081 </code></pre> <p>Notice that everything is happening on a single thread (see the [8]) so there is no chance of the dictionary having been concurrently modified. The excerpts are from the same log and the same process instance. Also notice that in the second set of logs we end up with two keys that are the same!</p> <p>What I am looking into: I have changed PhysicalAddress to a string to see if I can eliminate that from the list of suspects.</p> <p>My questions are:</p> <ul> <li>Is there a problem that I'm not seeing in the code above?</li> <li>Is there a problem with the equality methods on PhysicalAddress? (That only error every now and then?)</li> <li>Is there a problem with the Dictionary?</li> </ul>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload