Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <blockquote> <p>most efficient (query performance and memory consumption)</p> </blockquote> <p>By this you probably mean something that is well balanced between the two. Also, I think that the data insertion must be fast.</p> <p>The easiest and maybe sufficient solution would be to use plain array IMO as it is most memory efficient non-compressed form you can store the data. Thus each array element contains the <code>timestamp, id and value</code>.</p> <p>When you query the data with two timestamps <code>begin</code> and <code>end</code>, you determine the locations of the timestamps in the array using <code>binary search</code>. Then, you traverse all the elements and fetch only the ones with id-s of data sources you are interested in. The array elements must be of course ordered by timestamps.</p> <ul> <li>The data takes O(n) memory, where the number of log entries is n.</li> <li>Data insertion is O(1)</li> <li>Data retrieval should be something like O(2*log(n) + n*m), where n is the number of elements. If you have more data sources you want to include in the query, then you can store the data source ID-s in a set, thus the complexity would be O(2*log(n) + n*log(m)). </li> </ul> <p>There are of course other possibilities that can involve storing the transactions in trees, hashtables or something that mixes these with lists to get more detailed balance between performance/memory consumption.</p> <p>Also, the problems arise, when the amount of logs is large. In that case, you should split the array into files and store the begin/end timestamps the files contain the logs. Then the implementation gets a little bit more complex.</p> <p>Hopefully this helps somehow you to decide the best data structure / implementation for your solution.</p>
    singulars
    1. This table or related slice is empty.
    1. POOptimal data structure for time and source dependent log data for fast browsing?
      singulars
      1. PTQuestion
    1. PTAnswer
    1. USTimo
    1. USTimo
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    1. COData retrieval O(2*log(n) + n*m)? I take it you're assuming a circular queue where you have to do two binary searches?
      singulars
      1. PO
      1. USJim Mischel
    2. CO@Dave: because timestamps arrive in order
      singulars
      1. PO
      1. USJason S
    3. CO@Jim Mischel. Binary search takes O(log(n)) in an array. You need to find the indices for both begin and end timestamps, thus 2*log(n) and to be more precise 2*log2(n), but in terms of big-oh notation it does not matter that much. The n comes from going through the data. The n times m comes because you need to check if for every transaction if it has the ID of data source you are interested in. Having m data sources, you may need to do n*m comparisons. If m is larger, you can use some more efficient method for deciding whether the transaction is interesting or not.
      singulars
      1. PO
      1. USTimo
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload