Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>I up-voted a couple of good answers, but came up with a solution with some changes to the back-end data and a new way of retrieving the data from Silverlight. Here is what is being done to address this:</p> <ol> <li>I am using beans to represent that large data graph. This removed <strong>a lot</strong> of transmission XML. I am only concerned with a subset of the data anyway, although its a rather significant subset. By flattening the data into a bean I think I have cut my serialized object size to about 20 - 25% of the original object graph.</li> <li>Almost all data on the back end will now have a field for the last time it was modified. I was able to get this for all the big data. There are a few pieces of data that won't have this, but the real problems of query performance and data aggregation were solved with this. As a general solution for others, it looks like this is rather simple to implement in a number of DBMSs.</li> <li>I am writing new APIs to retrieve data that has been updated after a provided DateTime. This allows me to query only for new and changed objects from the back-end system (this is the web service calling these APIs, and the Silverlight is calling the web service).</li> <li>Aggregate changes in the web service and detect if a portion of the datagraph has changed. For simplicity I just send the entire datagraph if anything has changed. This was actually the hardest part to figure out. A part of the datagraph could have a new updated time, but the core object of the graph has not been updated. I ended up having to write APIs to look for the changes of the sub-objects, and then API's to find the root objects based on those sub-objects (if they had been changed). An object graph can be returned with a root object (and actually much of the object graph) that has not been updated since the last poll. The web service logic is querying on small numbers of changes so even though the queries are not cheap individually, they will potentially only run a few times per poll. Even in very large installations of our product, this query loop will only run 10 or 20 times per polling cycle (see about my polling solution below). While our systems are very dynamic, not <em>that</em> much changes in 30 seconds. The web service call that handles all of this reacts the same to an initial load call as it does a polling. All it is concerned with is retrieving data newer than a given time. </li> <li>I wrote a collection that inherits from ObservableCollection that handles the querying and polling. The client code using this collection provides a delegate that queries the data. The date is returned asynchronously, and in pages. I haven't settled on a page size. It keeps re-querying for pages until the server returns a page that is smaller than the max page size. The collection is also provided information on how to determine the latest date of the newest object in the collection. It polls periodically for updates that are newer than the newest item in the collection. In reality this "latest date" is actually an object containing several dates of various parts of the original object graph. If an item returns from the server that are exists in the collection, the item in the collection is update with that returned data. I did this instead of inserting the new item and removing the old because it works in more databound situations.</li> </ol> <p>This pattern could be improved. I could send only deltas to Silverlight for changes. I could still try to use some sort of push technology. But this solution gives me one web service call that can return data for various cases. Polling is also very simple, and there is just one thing doing all of the data retrieval. There aren't a lot of moving parts. This handles object state changes both during the initial data load, and during polling, through the same mechanism. This also seems to scales well. The initial call seems to be the most expensive with subsequent calls running faster and faster. I would assume that this is because the data that is remaining on the back-end is getting smaller and smaller with each pass. </p> <p><strong>I still have one question about my implementation of this that <a href="https://stackoverflow.com/questions/1749549/using-silverlight-mvvm-with-prism-unity-and-need-to-detect-when-view-is-closed">I have posted here</a>.</strong></p> <p>Thanks for all of the suggestions. While I didn't heed all of the advice, several ideas either directly helped me or got my mind thinking down a different path on how to get this working.</p>
    singulars
    1. This table or related slice is empty.
    1. POIs there a design pattern for dealing with large datasets over the internet?
      singulars
      1. PTQuestion
    1. PTAnswer
    1. USCommunity
    1. USJason Jackson
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. POIs there a design pattern for dealing with large datasets over the internet?
      singulars
      1. PTQuestion
    1. This table or related slice is empty.
    1. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTAcceptedByOriginator
    1. COCan u please clarify/explain what do u mean by "beans".
      singulars
      1. PO
      1. USBuzz
    2. COBean is a common term for a persistent class that is intended for UI consumption. Standard business/model classes may be cumbersome for UI consumption, so a bean is populated with their values and used by the UI. In my case the business-tier on the server had a variety of model objects that have some rather rich object graphs. I translate these to beans for Silverlight consumption.
      singulars
      1. PO
      1. USJason Jackson
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload