StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
primarykey
Id
8678284
data
AcceptedAnswerId
0
AnswerCount
0
ClosedDate
CommentCount
2
CommunityOwnedDate
CreationDate
2011-12-30T10:02:13.517
FavoriteCount
0
LastActivityDate
2011-12-30T10:34:12.160
LastEditDate
2011-12-30T10:34:12.160
LastEditorUserId
57695
OwnerUserId
57695
ParentId
8677946
PostTypeId
2
Score
3
ViewCount
0
LastEditorDisplayName
text
Body
For really large blocks of memory, I tend to suggest using memory mapped files (perhaps this is what R does for you) You can do this in Java with some boiler plate code. Unfortunately Java doesn't directly support mapping more than 2 GB at a time, so you have to break it into sections. <pre><code>import sun.misc.Cleaner; import sun.nio.ch.DirectBuffer; import java.io.Closeable; import java.io.IOException; import java.io.RandomAccessFile; import java.nio.MappedByteBuffer; import java.nio.channels.FileChannel; import java.util.ArrayList; import java.util.List; public class LargeDoubleMatrix implements Closeable { private static final int MAPPING_SIZE = 1 << 30; private final RandomAccessFile raf; private final int width; private final int height; private final List<MappedByteBuffer> mappings = new ArrayList<MappedByteBuffer>(); public LargeDoubleMatrix(String filename, int width, int height) throws IOException { this.raf = new RandomAccessFile(filename, "rw"); try { this.width = width; this.height = height; long size = 8L * width * height; for (long offset = 0; offset < size; offset += MAPPING_SIZE) { long size2 = Math.min(size - offset, MAPPING_SIZE); mappings.add(raf.getChannel().map(FileChannel.MapMode.READ_WRITE, offset, size2)); } } catch (IOException e) { raf.close(); throw e; } } protected long position(int x, int y) { return (long) y * width + x; } public int width() { return width; } public int height() { return height; } public double get(int x, int y) { assert x >= 0 && x < width; assert y >= 0 && y < height; long p = position(x, y) * 8; int mapN = (int) (p / MAPPING_SIZE); int offN = (int) (p % MAPPING_SIZE); return mappings.get(mapN).getDouble(offN); } public void set(int x, int y, double d) { assert x >= 0 && x < width; assert y >= 0 && y < height; long p = position(x, y) * 8; int mapN = (int) (p / MAPPING_SIZE); int offN = (int) (p % MAPPING_SIZE); mappings.get(mapN).putDouble(offN, d); } public void close() throws IOException { for (MappedByteBuffer mapping : mappings) clean(mapping); raf.close(); } private void clean(MappedByteBuffer mapping) { if (mapping == null) return; Cleaner cleaner = ((DirectBuffer) mapping).cleaner(); if (cleaner != null) cleaner.clean(); } } </code></pre> has this test which sets the diagonal value. <pre><code>@Test public void getSetMatrix() throws IOException { long start = System.nanoTime(); final long used0 = usedMemory(); LargeDoubleMatrix matrix = new LargeDoubleMatrix("/tmp/ldm.test", 48*1000, 50*1000); for(int i=0;i<matrix.width();i++) matrix.set(i,i,i); for(int i=0;i<matrix.width();i++) assertEquals(i, matrix.get(i,i), 0.0); long time = System.nanoTime() - start; final long used = usedMemory() - used0; if (used==0) System.err.println("You need to use -XX:-UsedTLAB to see small changes in memory usage."); System.out.printf("Setting the diagonal took %,d ms, Heap used is %,d KB%n", time/1000/1000, used/1024); matrix.close(); } private long usedMemory() { return Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory(); } </code></pre> prints (when run with <code>-XX:-UseTLAB</code> ) <pre><code>Setting the diagonal took 60 ms, Heap used is 55 KB </code></pre> Only the pages actually used are created. The files appears to be very large, but the space allocated is based on use. <pre><code>$ ls -lh /tmp/ldm.test -rw-rw-r-- 1 peter peter 18G 2011-12-30 10:18 /tmp/ldm.test $ du -sh /tmp/ldm.test 222M /tmp/ldm.test </code></pre>
Tags
Title
singulars
PostAcceptedAnswerId
1. This table or related slice is empty.
PostParentId
1. POHandle large sized matrix in Java
 singulars
 PostTypePostTypeId
 PTQuestion
PostTypePostTypeId
1. PTAnswer
UserLastEditorUserId
1. USPeter Lawrey
UserOwnerUserId
1. USPeter Lawrey
plurals
PostLinksPostIdRelatedPostId
1. This table or related slice is empty.
PostLinksRelatedPostIdPostId
1. This table or related slice is empty.
PostsAcceptedAnswerId
1. This table or related slice is empty.
PostsParentIdCreationDate
1. This table or related slice is empty.
VotesPostIdCreationDate
1. This table or related slice is empty.
CommentsPostId
1. This table or related slice is empty.

Querying!

Guidance

A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.