Databases
StackOverflow2013
Postsdbo
POData access pattern, combining push and pull?
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POData access pattern, combining push and pull?
    primarykey
    data
    text
    <p>I need some advice on what kind of pattern(s) I should use for pushing/pulling data into my application.</p> <p>I'm writing a rule-engine that needs to hold quite a large amount of data in-memory in order to be efficient enough. I have some rather conflicting requirements;</p> <ol> <li>It is not acceptable for the engine to always have to wait for a full pre-load of all data before it is functional.</li> <li>Only fetching and caching data on-demand will lead to the engine taking too long before it is running quickly enough.</li> <li>An external event can trigger the need for specific parts of the data to be reloaded.</li> </ol> <p>Basically, I think I need a combination of pushing and pulling data into the application.</p> <p>A simplified version of my current "pattern" looks like this (in psuedo-C# written in notepad):</p> <pre><code>// This interface is implemented by all classes that needs the data interface IDataSubscriber { void RegisterData(Entity data); } // This interface is implemented by the data access class interface IDataProvider { void EnsureLoaded(Key dataKey); void RegisterSubscriber(IDataSubscriber subscriber); } class MyClassThatNeedsData : IDataSubscriber { IDataProvider _provider; MyClassThatNeedsData(IDataProvider provider) { _provider = provider; _provider.RegisterSubscriber(this); } public void RegisterData(Entity data) { // Save data for later StoreDataInCache(data); } void UseData(Key key) { // Make sure that the data has been stored in cache _provider.EnsureLoaded(key); Entity data = GetDataFromCache(key); } } class MyDataProvider : IDataProvider { List&lt;IDataSubscriber&gt; _subscribers; // Make sure that the data for key has been loaded to all subscribers public void EnsureLoaded(Key key) { if (HasKeyBeenMarkedAsLoaded(key)) return; PublishDataToSubscribers(key); MarkKeyAsLoaded(key); } // Force all subscribers to get a new version of the data for key public void ForceReload(Key key) { PublishDataToSubscribers(key); MarkKeyAsLoaded(key); } void PublishDataToSubscribers(Key key) { Entity data = FetchDataFromStore(key); foreach(var subscriber in _subscribers) { subscriber.RegisterData(data); } } } // This class will be spun off on startup and should make sure that all data is // preloaded as quickly as possible class MyPreloadingThread { IDataProvider _provider; MyPreloadingThread(IDataProvider provider) { _provider = provider; } void RunInBackground() { IEnumerable&lt;Key&gt; allKeys = GetAllKeys(); foreach(var key in allKeys) { _provider.EnsureLoaded(key); } } } </code></pre> <p>I have a feeling though that this is not necessarily the best way of doing this.. Just the fact that explaining it seems to take two pages feels like an indication..</p> <p>Any ideas? Any patterns out there I should have a look at?</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USCodingInsomnia
    1. USCodingInsomnia
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POData access pattern, combining push and pull?
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POData access pattern, combining push and pull?
      1. This table or related slice is empty.
      1. VTDownMod
    3. VO
      singulars
      1. POData access pattern, combining push and pull?
      1. This table or related slice is empty.
      1. VTUpMod
    1. This table or related slice is empty.
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload