Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>This does sound like you might need a data warehouse. You would update the data warehouse from the various sources, on whatever schedule was necessary. However, all the read-only transactions would come from the data warehouse, and would not require immediate calls to the various external sources.</p> <p>This assumes you don't need realtime access to the most up to date data. Even if you needed data accurate to within the past hour from a particular source, that only means you would need to update from that source every hour.</p> <p>You haven't said what platforms you're using. If you were using SQL Server 2005 or later, I would recommend SQL Server Integration Services (SSIS) for updating the data warehouse. It's made for just this sort of thing.</p> <p>Of course, depending on your platform choices, there may be alternatives that are more appropriate.</p> <hr> <p>Here are some resources on SSIS and data warehouses. I know you've stated you will not be using Microsoft products. I include these links as a point of reference: these are the products I was talking about above.</p> <ul> <li><a href="http://technet.microsoft.com/en-us/library/ms141263.aspx" rel="nofollow noreferrer">SSIS Overview</a></li> <li><a href="http://technet.microsoft.com/en-us/library/ms137795.aspx" rel="nofollow noreferrer">Typical Uses of Integration Services</a></li> <li><a href="http://technet.microsoft.com/en-us/library/ms141026.aspx" rel="nofollow noreferrer">SSIS Documentation Portal</a></li> <li><a href="http://msdn.microsoft.com/en-us/library/cc719165.aspx" rel="nofollow noreferrer">Best Practices for Data Warehousing with SQL Server 2008</a></li> </ul>
    singulars
    1. This table or related slice is empty.
    1. POPerformance problems with external data dependencies
      singulars
      1. PTQuestion
    1. PTAnswer
    1. USJohn Saunders
    1. USJohn Saunders
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. POPerformance problems with external data dependencies
      singulars
      1. PTQuestion
    1. This table or related slice is empty.
    1. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTAcceptedByOriginator
    1. COSounds like it can do what I am looking for, but Microsoft products is not an option. Would an ESB like ServiceMix or even RabbitMQ be any appropriate?
      singulars
      1. PO
      1. USLangali
    2. COI'm not familiar with those. I do recommend that you learn the kinds of things that SSIS can do - those are the features you should be looking for, in part. Some of your ongoing issues will be of getting at the data and cleaning it up. SSIS is an ETL tool that can do that. You could then have your 50,000 users would retrieve their data from the DW, optimized for read and query access, updated as frequently as required.
      singulars
      1. PO
      1. USJohn Saunders
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload