StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
primarykey
Id
15992511
data
AcceptedAnswerId
0
AnswerCount
0
ClosedDate
CommentCount
0
CommunityOwnedDate
CreationDate
2013-04-13T20:23:01.353
FavoriteCount
0
LastActivityDate
2013-04-13T20:23:01.353
LastEditDate
LastEditorUserId
0
OwnerUserId
1867669
ParentId
15991702
PostTypeId
2
Score
0
ViewCount
0
LastEditorDisplayName
text
Body
After digging through the source a bit more and trying to understand more how <code>setvbuf</code> and <code>fread</code> work, I think I understand how <code>buffering</code> and <code>READAHEAD_BUFSIZE</code> relate to each other: when iterating through a file, a buffer of <code>READAHEAD_BUFSIZE</code> is filled on each line, but filling this buffer uses calls to <code>fread</code>, each of which fills a buffer of <code>buffering</code> bytes. Python's <code>read</code> is implemented as <a href="http://hg.python.org/cpython/file/84cd07899baf/Objects/fileobject.c#l1052" rel="nofollow">file_read</a>, which calls <a href="http://hg.python.org/cpython/file/84cd07899baf/Objects/fileobject.c#l2809" rel="nofollow">Py_UniversalNewlineFread</a>, passing it the number of bytes to read as <code>n</code>. <code>Py_UniversalNewlineFread</code> then eventually calls <code>fread</code> to read n bytes. When you iterate over a file, the function <a href="http://hg.python.org/cpython/file/84cd07899baf/Objects/fileobject.c#l2269" rel="nofollow">readahead_get_line_skip</a> is what retrieves a line. This function also calls <code>Py_UniversalNewlineFread</code>, passing <code>n = READAHEAD_BUFSIZE</code>. So this eventually becomes a call to <code>fread</code> for <code>READAHEAD_BUFSIZE</code> bytes. So now the question is, how many bytes does <code>fread</code> actually read from disk. If I run the following code in C, then 1024 bytes get copied into <code>buf</code> and 512 into <code>buf2</code>. (This might be obvious but never having used <code>setvbuf</code> before it was a useful experiment for me.) <pre><code>FILE *f = fopen("test.txt", "r"); void *buf = malloc(1024); void *buf2 = mallo(512); setvbuf(f, buf, _IOFBF, 1024); fread(buf2, 512, 1, f); </code></pre> So, finally, this suggests to me that when iterating over a file, at least <code>READAHEAD_BUF_SIZE</code> bytes are read from disk, but it might be more. I think that the first iteration of <code>for line in f</code> will read x bytes, where x is the smallest multiple of <code>buffering</code> that is greater than <code>READAHEAD_BUF_SIZE</code>. If anyone can confirm that this is what's actually going on, that would be great!
Tags
Title
singulars
PostAcceptedAnswerId
1. This table or related slice is empty.
PostParentId
1. POWhat is the difference between the buffering argument to open() and the hardcoded readahead buffer size used when iterating through a file?
 singulars
 PostTypePostTypeId
 PTQuestion
PostTypePostTypeId
1. PTAnswer
UserLastEditorUserId
1. This table or related slice is empty.
UserOwnerUserId
1. USEmily
plurals
PostLinksPostIdRelatedPostId
1. This table or related slice is empty.
PostLinksRelatedPostIdPostId
1. This table or related slice is empty.
PostsAcceptedAnswerId
1. This table or related slice is empty.
PostsParentIdCreationDate
1. This table or related slice is empty.
VotesPostIdCreationDate
1. This table or related slice is empty.
CommentsPostId
1. This table or related slice is empty.

Querying!

Guidance

A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.