StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POPython and ADNS, falling in infinite loop somewhere
text
Body
copied!<p>I have written some code that queries adns. Problem with this code is that it gets stuck, how? Let me explain it:</p> <ul> <li>Say my dnslist is ["8.8.4.4", "8.8.8.8", "208.67.220.220", "208.67.222.222", "192.168.50.1"] </li> <li>It would pop a dns from the list and query againt it, now that means that DNS will be queried in reverse order</li> <li>No matter what i do, It never shows results from the dns it picked up first (in our case 192.168.50.1)</li> <li>I was not sure if that dns ever replied so <ul> <li>First i changed DNS list to contain just that last DNS Server and code executes fine</li> <li>Second i used the old list with 5 DNS servers except that the last one was managed by my so i could track if code even queries it or not, and to my surprise the query does take place.</li> <li>So query is made, we get result but that result is never inserted into resolved_hosts for some reason, and because that results is not inserted, its length will remain less than the length of dnslist, causing a infinite loop.</li> </ul></li> </ul> <p>What do you think could be causing this problem and how to solve it?</p> <h1>Code Execution Results</h1> <pre><code>Inside class's init' Data host www.yahoo.com dnslist length 5 intensity 1 Inside resolve() inside finished_resolving() Resolved : 0/5 Inside 'while not finished_resolving' Queue: 0/1 Launching Querying for www.yahoo.com/1 on 192.168.50.1 Queue: 1/1 Launching Querying for www.yahoo.com/1 on 208.67.222.222 inside collect_results() inside finished_resolving() Resolved : 0/5 Inside 'while not finished_resolving' ------------------------ CLIPPED ---------------- Inside 'while not finished_resolving' inside collect_results() Inside collect_results's for query in self.adns.completed() DNS used was208.67.222.222 Answered : (0, 'any-fp.wa1.b.yahoo.com', 1286169807, ('69.147.125.65', '67.195.160.76')) Resolved www.yahoo.com to 69.147.125.65 using 208.67.222.222 Resolved hosts count1 And they are: {'208.67.222.222': '69.147.125.65'} inside finished_resolving() Resolved : 1/5 Inside 'while not finished_resolving' Queue: 1/1 Launching Querying for www.yahoo.com/1 on 208.67.220.220 inside collect_results() inside finished_resolving() Resolved : 1/5 Inside 'while not finished_resolving' -------------------------- CLIPPED -------------------- inside collect_results() Inside collect_results's for query in self.adns.completed() DNS used was208.67.220.220 Answered : (0, 'any-fp.wa1.b.yahoo.com', 1286169790, ('67.195.160.76', '69.147.125.65')) Resolved www.yahoo.com to 67.195.160.76 using 208.67.220.220 Resolved hosts count2 And they are: {'208.67.222.222': '69.147.125.65', '208.67.220.220': '67.195.160.76'} inside finished_resolving() Resolved : 2/5 Inside 'while not finished_resolving' Queue: 1/1 Launching Querying for www.yahoo.com/1 on 8.8.8.8 inside collect_results() inside finished_resolving() Resolved : 2/5 Inside 'while not finished_resolving' -------------------------- CLIPPED -------------------- inside collect_results() Inside collect_results's for query in self.adns.completed() DNS used was8.8.8.8 Answered : (0, 'eu-fp.wa1.b.yahoo.com', 1286169758, ('87.248.122.122',)) Resolved www.yahoo.com to 87.248.122.122 using 8.8.8.8 Resolved hosts count3 And they are: {'208.67.222.222': '69.147.125.65', '208.67.220.220': '67.195.160.76', '8.8.8.8': '87.248.122.122'} inside finished_resolving() Resolved : 3/5 Inside 'while not finished_resolving' Queue: 1/1 Launching Querying for www.yahoo.com/1 on 8.8.4.4 inside collect_results() inside finished_resolving() Resolved : 3/5 Inside 'while not finished_resolving' -------------------- CLIPPED ------------------------------------- inside collect_results() Inside collect_results's for query in self.adns.completed() DNS used was8.8.4.4 Answered : (0, 'eu-fp.wa1.b.yahoo.com', 1286169757, ('87.248.122.122',)) Resolved www.yahoo.com to 87.248.122.122 using 8.8.4.4 Resolved hosts count4 And they are: {'208.67.222.222': '69.147.125.65', '208.67.220.220': '67.195.160.76', '8.8.8.8': '87.248.122.122', '8.8.4.4': '87.248.122.122'} inside finished_resolving() Resolved : 4/5 Inside 'while not finished_resolving' inside collect_results() inside finished_resolving() Resolved : 4/5 ---------------- CLIPPED ------------------------------- </code></pre> <p>(last block keeps repeating until system starts to hang up, load goes upto 24)</p> <h1>Code</h1> <p><strong>test.py</strong></p> <pre><code>import adns from time import time from async_dns import AsyncResolver dnslist2 = ["8.8.4.4", "8.8.8.8", "208.67.220.220", "208.67.222.222", "192.168.50.1"] #192.168.50.1 is a dns server i manage host = "www.yahoo.com" record = adns.rr.A intensity = len(dnslist2)/5+1 ar = AsyncResolver(dnslist2, host, record, intensity) start = time() resolved = ar.resolve() end = time() print "\n\n" for dns, ip in resolved.items(): if ip is None: print "%s could not resolv %s." % (dns, host) else: print "%s resolved it to %s : %s" % (dns, host, ip) print "\n\n----------------------------------------------------" print "It took %.2f seconds to query %d dns." % (end-start, len(dnslist)) print "----------------------------------------------------" </code></pre> <p><strong>async_dns.py</strong></p> <pre><code>#!/usr/bin/python # import sys import adns from time import time class AsyncResolver(object): def __init__(self, dnslist, host, record, intensity=10): """ dnslist: a list of dns used to resolve host : hostname to resolve record: recordtype to resolve intensity: how many hosts to resolve at once """ print "Inside class's init'" self.dnslist = dnslist self.host = host self.record = record if intensity >= len(dnslist) : self.intensity = len(dnslist)/5+1 else: self.intensity = intensity print "Data" print "host " + host print "dnslist length " + str(len(dnslist)) print "intensity " + str(intensity) def resolve(self): """ Resolves hosts and returns a dictionary of { 'dns': 'ip' }. """ print "Inside resolve()" host = self.host record = self.record resolved_hosts = {} active_queries = {} dns_queue = self.dnslist[:] def collect_results(): print "inside collect_results()" for query in self.adns.completed(): print "Inside collect_results's for query in self.adns.completed()" answer = query.check() dns = active_queries[query] print "DNS used was" + dns print "Answered : " + str(answer) del active_queries[query] if answer[0] == 0: #print "Inside answer[0] == 0 , ip:" + answer[3][0] ip = answer[3][0] resolved_hosts[dns] = ip print "Resolved %s to %s using %s" % (host, ip, dns) print "Resolved hosts count" + str(len(resolved_hosts)) print "And they are: " print str(resolved_hosts) print "\n" elif answer[0] == 101 and not record == adns.rr.CNAME: # CNAME if CNAME wasn't required' print "ooopppps, i got a CNAME, gotta find its A" print "\n" query = self.adns.submit(answer[1], adns.rr.A) active_queries[query] = dns else: resolved_hosts[dns] = None print "THIS COULD NOT BE RESOLVED" def finished_resolving(): print "inside finished_resolving()" print "Resolved : " + str(len(resolved_hosts)) + "/" + str(len(self.dnslist)) return len(resolved_hosts) == len(self.dnslist) while not finished_resolving(): print "Inside 'while not finished_resolving'" while dns_queue and len(active_queries) <= self.intensity: print "Queue: " + str(len(active_queries)) + "/" + str(self.intensity) dns = dns_queue.pop() self.adns = adns.init(adns.iflags.noautosys,sys.stderr,"nameserver "+dns) query = self.adns.submit(host, record) print "Launching Querying for " + host + "/" + str(record) + " on " + dns active_queries[query] = dns collect_results() return resolved_hosts </code></pre>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload