Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>I'm just a student, newly joined to the community. Take what I say with a grain of salt.</p> <p>I'm not sure why everyone is so much as blinking at the ten thousand rows business when we're measuring modern personal computers' memory in gigabytes.</p> <p>Alright. I'm going to assume that what you're doing needs to be done in the browser, and so you can't switch to doing native code. In that case, looking for an alternative to Javascript won't get you much of anywhere. In the context of a browser, you're looking at an interpreted language. In terms of number of instructions the program ultimately has to run, the difference between one language or another is negligible in the face of how long it takes to be interpreted. Besides, Javascript has gotten nicely polished over the years.</p> <p>So never mind that. There's a much more important thing to consider here, and it applies no matter what you're programming in or on: The cache(s). <a href="http://igoro.com/archive/gallery-of-processor-cache-effects/" rel="nofollow">Igor Ostrovsky explains it beautifully</a>; read it until you grok it.</p> <p>So I'm <em>guessing</em> you have objects that would stringify to something like, <code>"obj1 = {field-1:'a', field-2:'b', ..., field-n:'n'}"</code>. And you can select a <code>field-i</code> to sort by. The trouble with this is that when you sort by <code>field-i</code>, you're loading the entirety of <code>obj1</code> into the cache, even though you don't need it. What you really want to do is load the <code>field-i</code>'s for<code>obj1, obj2, obj3, ..., objm</code> all at once. So you look at an array, stringifying to something like: <code>"field1 = [refToRow1, field1inRow1, refToRow2, field1inRow2, ..., refToRowM, field1inRowM]"</code>.</p> <p>You might not be able to do fit all M rows in the cache, after all <code>M==10000</code>! But you can group them together into chunks that you could reasonably expect a cache to manage. Anyone got a good number for this? Say, 64kB? So you figure for each <code>i</code> in <code>M</code> you've got a reference, and a field that's probably just a reference to a short string (it'd be better if you could have the string itself right there, but I don't think Javascript works that way). So that's 8B? 8B*1024 = 64kB? Hell, if that's right, you could fit it all into the cache in two chunks, which means you'd want to do it in 4.</p> <p>So now you've got a collection of smaller arrays, and you want to sort them. This is a classic application for <a href="http://en.wikipedia.org/wiki/B-tree" rel="nofollow">B-trees</a>. And while having a separate B-tree for each and every column in the table may seem like a lot, it's not.</p> <p>Okay, so that handles sorting. You tell it to sort by a column, and the truth is it's already sorted! You're just repopulating the visible table using a different b-tree. You still need to handle filtering, but that's fine. You do some cache juggling as you find something to display and follow the reference to get the other fields, but I'd still expect this to go fast since you're skipping over so many rows.</p> <p>Normally, I would say if you want to speed things up, look into multiprocessing. But I think browsers are still working to make that a thing with their Javascript implementations. Plus, while it would be well-suited for sorting, it would be a lot of effort to make it useful for the filtering part, and I expect you can do fine without.</p> <p>I hope this isn't too scatter-brained, and that it gives you some ideas. Good luck!</p>
    singulars
    1. This table or related slice is empty.
    1. POFast managing of tables with ~10k rows in Javascript
      singulars
      1. PTQuestion
    1. PTAnswer
    1. This table or related slice is empty.
    1. USAndrewK
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. CO"I'm not sure why everyone is so much as blinking at the ten thousand rows business when we're measuring modern personal computers' memory in gigabytes": I guess they are not blinking at *managing* 10k rows, but *displaying* 10k rows at a time. It just doesn't make sense, there is no way for a user to handle all that informations at once.
      singulars
      1. PO
      1. USAndrea Casaccia
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload