Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>You say that your system is capable of inserting 3000 records per second without indexes, but only about 100 with two additional non-clustered indexes. If 3k/s is the maximum throughput your I/O permits, adding two indexes should in theory reduces the throughput at about 1000-1500/sec. Instead you see a degradation 10 times worse. The proper solution and answer is 'It Dependts' and some serious troubleshooting and bottleneck identification would have to be carried out. With that in mind, if I was to venture a guess, I'd give two possible culprits:</p> <p>A. Th additional non-clustered indexes distribute the writes of dirty pages into more allocation areas. The solution would be to place the the clustered index and each non-clustered index into its own filegroup and place the three filegroups each onto separate LUNs on the RAID.</p> <p>B. The low selectivity of the non-clustered indexes creates high contention between reads and writes (key conflicts as well as <a href="http://rusanu.com/2009/05/29/lockres-collision-probability-magic-marker-16777215/" rel="noreferrer">%lockres% conflicts</a>) resulting in long lock wait times for both inserts and selects. Possible solutions would be using SNAPSHOTs with <a href="http://msdn.microsoft.com/en-us/library/bb522682.aspx" rel="noreferrer">read committed snapshot mode</a>, but I must warn about the danger of adding <em>lot</em> of IO in the <a href="http://msdn.microsoft.com/en-us/library/ms189050.aspx" rel="noreferrer">version store</a> (ie. in tempdb) on system that may already be under high IO stress. A second solution is using <a href="http://msdn.microsoft.com/en-us/library/ms175158.aspx" rel="noreferrer">database snapshots</a> for reporting, they cause lower IO stress and they can be better controlled (no tempdb version store involved), but the reporting is no longer on real-time data.</p> <p>I tend to believe B) as the likely cause, but I must again stress the need to proper investigation and proper root case analysis.</p> <p>'RAID10' is not a very precise description.</p> <ul> <li>How many spindles in the RAID 0 part? Are they short-striped?</li> <li>How many LUNs? </li> <li>Where is the database log located?</li> <li>Where is the database located? </li> <li>How many partitions? </li> <li>Where is tempdb located?</li> </ul> <p>As on the question whether relational databases are appropriate for something like this, yes, absolutely. There are many more factors to consider, recoverability, availability, toolset ecosystem, know-how expertise, ease of development, ease of deployment, ease of management and so on and so forth. Relational databases can easily handle your workload, they just need the proper tuning. 30 million inserts a day, 350 per second, is small change for a database server. But a 32bit 4GB RAM system hardly a database server, regardless the number of CPUs. </p>
    singulars
    1. This table or related slice is empty.
    1. PO30 million records a day, SQL Server can't keep up, other kind of database system needed?
      singulars
      1. PTQuestion
    1. PTAnswer
    1. USRemus Rusanu
    1. USRemus Rusanu
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO30 million records a day, SQL Server can't keep up, other kind of database system needed?
      singulars
      1. PTQuestion
    1. This table or related slice is empty.
    1. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    1. This table or related slice is empty.
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload