StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
primarykey
Id
11127229
data
AcceptedAnswerId
0
AnswerCount
0
ClosedDate
CommentCount
2
CommunityOwnedDate
CreationDate
2012-06-20T19:59:02.637
FavoriteCount
0
LastActivityDate
2012-06-20T22:16:41.463
LastEditDate
2012-06-20T22:16:41.463
LastEditorUserId
583830
OwnerUserId
583830
ParentId
11055502
PostTypeId
2
Score
28
ViewCount
0
LastEditorDisplayName
text
Body
Have a look at the "Large memory and out-of-memory data" subsection of the <a href="http://cran.r-project.org/web/views/HighPerformanceComputing.html">high performance computing task view</a> on CRAN. <a href="http://cran.r-project.org/web/packages/bigmemory/index.html">bigmemory</a> and <a href="http://cran.r-project.org/web/packages/ff/index.html">ff</a> are two popular packages. For bigmemory (and the related <a href="http://cran.r-project.org/web/packages/biganalytics/index.html">biganalytics</a>, and <a href="http://cran.r-project.org/web/packages/bigtabulate/index.html">bigtabulate</a>), the <a href="http://www.bigmemory.org/">bigmemory website</a> has a few very good presentations, vignettes, and overviews from Jay Emerson. For ff, I recommend reading Adler Oehlschlägel and colleagues' excellent slide presentations on the <a href="http://ff.r-forge.r-project.org/">ff website</a>. Also, consider storing data in a database and reading in smaller batches for analysis. There are likely any number of approaches to consider. To get started, consdier looking through some of the examples in the <a href="http://cran.r-project.org/web/packages/biglm/index.html">biglm</a> package, as well as <a href="http://faculty.washington.edu/tlumley/tutorials/user-biglm.pdf">this presentation</a> from Thomas Lumley. And do investigate the other packages on the high-performance computing task view and mentioned in the other answers. The packages I mention above are simply the ones I've happened to have more experience with.
Tags
Title
singulars
PostAcceptedAnswerId
1. This table or related slice is empty.
PostParentId
1. PORecommended package for very large dataset processing and machine learning in R
 singulars
 PostTypePostTypeId
 PTQuestion
PostTypePostTypeId
1. PTAnswer
UserLastEditorUserId
1. USjthetzel
UserOwnerUserId
1. USjthetzel
plurals
PostLinksPostIdRelatedPostId
1. This table or related slice is empty.
PostLinksRelatedPostIdPostId
1. This table or related slice is empty.
PostsAcceptedAnswerId
1. PORecommended package for very large dataset processing and machine learning in R
 singulars
 PostTypePostTypeId
 PTQuestion
PostsParentIdCreationDate
1. This table or related slice is empty.
VotesPostIdCreationDate
1. VO
 singulars
 PostPostId
 PO
 UserUserId
 This table or related slice is empty.
 VoteTypeVoteTypeId
 VTUpMod
2. VO
 singulars
 PostPostId
 PO
 UserUserId
 This table or related slice is empty.
 VoteTypeVoteTypeId
 VTUpMod
3. VO
 singulars
 PostPostId
 PO
 UserUserId
 This table or related slice is empty.
 VoteTypeVoteTypeId
 VTAcceptedByOriginator
CommentsPostId
1. COEnded up using BigMemory and am generally quite pleased with it.
 singulars
 PostPostId
 PO
 UserUserId
 USJohn Robertson

Querying!

Guidance

A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.