Databases
StackOverflow2013
Postsdbo
POHow to optimize use of querysets with lists
Body
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POHow to optimize use of querysets with lists
    text
    copied!<p>I have a model that has a couple million objects. Each object represents a call made/received by a company. </p> <p>To simplify things, let's say this model, Call, has these fields:<br> calldate, context, channel.</p> <p>My goal is to know the average # of calls made and received during each hour of the day of the month (load by hour). The catch is: I need to find this for <strong>port1</strong> and <strong>port2</strong> separately.</p> <p>As of now, my code works fine, except that it takes around 1 whole minute to give me the result for a range of 4 months and I it seems extremely inefficient.</p> <p>I've done some simple profiling and discovered that the <strong>extend</strong> is taking around 99% of the processing time: </p> <pre><code>queryset = Call.objects.filter(calldate__gte='SOME_DATE') port1, port2 = [],[] port1.extend(queryset.filter(context__icontains="e1-1")) port2.extend(queryset.filter(context__icontains="e1-2")) channels_in_port1 = ["Port/%d-2" % x for x in range(1,32)] channels_in_port2 = ["Port/%d-2" % x for x in range(32,63)] for i in channels_in_port1: port1.extend(queryset.filter(channel__icontains=i)) for i in channels_in_port2: port2.extend(queryset.filter(channel__icontains=i)) </code></pre> <p>port1 and port2 have around 150k objects combined now.</p> <p>As soon as I have all calls for <strong>port1</strong> and <strong>port2</strong>, I'm good to go. The rest of the code is basically some for loops for <strong>port1</strong> and <strong>port2</strong> that sums up and takes the average of calls according to the hour/day/month. Trivial stuff.</p> <p>I tried to avoid using any "extend" by using itertools.chain and chaining the querysets instead. However, that made the processing time shift to the part where I do the trivial for loops to calculate the load by hour.</p> <p>Any alternatives? Better ways to filter the queryset?<br> Thanks very much!!</p>
 

Querying!

 
Guidance
An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload