StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POHow to write a pager for Python iterators?
text
Body
copied!<p>I'm looking for a way to "page through" a Python iterator. That is, I would like to wrap a given iterator <b>iter</b> and <b>page_size</b> with another iterator that would would return the items from iter as a series of "pages". Each page would itself be an iterator with up to <b>page_size</b> iterations. </p> <p>I looked through <a href="http://docs.python.org/library/itertools.html" rel="nofollow noreferrer">itertools</a> and the closest thing I saw is <a href="http://docs.python.org/library/itertools.html#itertools.islice" rel="nofollow noreferrer">itertools.islice</a>. In some ways, what I'd like is the opposite of <a href="http://docs.python.org/library/itertools.html#itertools.chain" rel="nofollow noreferrer">itertools.chain</a> -- instead of chaining a series of iterators together into one iterator, I'd like to break an iterator up into a series of smaller iterators. I was expecting to find a paging function in itertools but couldn't locate one. </p> <p>I came up with the following pager class and demonstration. </p> <pre><code>class pager(object): """ takes the iterable iter and page_size to create an iterator that "pages through" iter. That is, pager returns a series of page iterators, each returning up to page_size items from iter. """ def __init__(self,iter, page_size): self.iter = iter self.page_size = page_size def __iter__(self): return self def next(self): # if self.iter has not been exhausted, return the next slice # I'm using a technique from # https://stackoverflow.com/questions/1264319/need-to-add-an-element-at-the-start-of-an-iterator-in-python # to check for iterator completion by cloning self.iter into 3 copies: # 1) self.iter gets advanced to the next page # 2) peek is used to check on whether self.iter is done # 3) iter_for_return is to create an independent page of the iterator to be used by caller of pager self.iter, peek, iter_for_return = itertools.tee(self.iter, 3) try: next_v = next(peek) except StopIteration: # catch the exception and then raise it raise StopIteration else: # consume the page from the iterator so that the next page is up in the next iteration # is there a better way to do this? # for i in itertools.islice(self.iter,self.page_size): pass return itertools.islice(iter_for_return,self.page_size) iterator_size = 10 page_size = 3 my_pager = pager(xrange(iterator_size),page_size) # skip a page, then print out rest, and then show the first page page1 = my_pager.next() for page in my_pager: for i in page: print i print "----" print "skipped first page: " , list(page1) </code></pre> <p>I'm looking for some feedback and have the following questions:</p> <ol> <li>Is there a pager already in <strong>itertools</strong> that serves a pager that I'm overlooking? </li> <li>Cloning self.iter 3 times seems kludgy to me. One clone is to check whether self.iter has any more items. I decided to go with <a href="https://stackoverflow.com/questions/1264319/need-to-add-an-element-at-the-start-of-an-iterator-in-python/1264331#1264331">a technique Alex Martelli suggested</a> (aware that he wrote of a <a href="https://stackoverflow.com/questions/1966591/hasnext-in-python-iterators/1967037#1967037">wrapping technique</a>). The second clone was to enable the returned page to be independent of the internal iterator (<strong>self.iter</strong>). Is there a way to avoid making 3 clones?</li> <li>Is there a better way to deal with the <strong>StopIteration</strong> exception beside catching it and then raising it again? I am tempted to not catch it at all and let it bubble up.</li> </ol> <p>Thanks! -Raymond</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload