Databases
StackOverflow2013
Postsdbo
POIs it possible to accelerate (dynamic) LINQ queries using GPU?
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POIs it possible to accelerate (dynamic) LINQ queries using GPU?
    primarykey
    data
    text
    <p>I have been searching for some days for solid information on the possibility to accelerate LINQ queries using a GPU.</p> <p>Technologies I have "investigated" so far:</p> <ul> <li>Microsoft Accelerator </li> <li>Cudafy</li> <li>Brahma</li> </ul> <p>In short, would it even be possible at all to do an in-memory filtering of objects on the GPU?</p> <p>Let´s say we have a list of some objects and we want to filter something like:</p> <pre><code>var result = myList.Where(x =&gt; x.SomeProperty == SomeValue); </code></pre> <p>Any pointers on this one?</p> <p>Thanks in advance!</p> <p><strong>UPDATE</strong></p> <p>I´ll try to be more specific about what I am trying to achieve :)</p> <p>The goal is, to use any technology, which is able to filter a list of objects (ranging from ~50 000 to ~2 000 000), in the absolutely fastest way possible.</p> <p>The operations I perform on the data when the filtering is done (sum, min, max etc) is made using the built in LINQ-methods and is already fast enough for our application, so that´s not a problem.</p> <p>The bottleneck is "simply" the filtering of data.</p> <p><strong>UPDATE</strong></p> <p>Just wanted to add that I have tested about 15 databases, including MySQL (checking possible cluster approach / memcached solution), H2, HSQLDB, VelocityDB (currently investigating further), SQLite, MongoDB etc, and NONE is good enough when it comes to the speed of filtering data (of course, the NO-sql solutions do not offer this like the sql ones, but you get the idea) and/or the returning of the actual data.</p> <p>Just to summarize what I/we need:</p> <p>A database which is able to sort data in the format of 200 columns and about 250 000 rows in less than 100 ms.</p> <p>I currently have a solution with parallellized LINQ which is able (on a specific machine) to spend only <strong>nano</strong>-seconds on each row when filtering <strong>AND</strong> processing the result!</p> <p>So, we need like sub-<strong>nano</strong>-second-filtering on each row.</p> <ol> <li>Why does it seem that only in-memory LINQ is able to provide this?</li> <li>Why would this be impossible?</li> </ol> <p>Some figures from the logfile:</p> <pre><code>Total tid för 1164 frågor: 2579 </code></pre> <p>This is Swedish and translates:</p> <pre><code>Total time for 1164 queries: 2579 </code></pre> <p>Where the queries in this case are queries like:</p> <pre><code>WHERE SomeProperty = SomeValue </code></pre> <p>And those queries are all being done in parallell on 225639 rows.</p> <p>So, 225639 rows are being filtered in memory 1164 times in about 2.5 seconds.</p> <p>That´s 9,5185952917007032597107300413827e-9 seconds / row, <strong>BUT</strong>, that also <strong>includes</strong> the actual processing of the numbers! We do Count (not null), total count, Sum, Min, Max, Avg, Median. So, we have 7 operations on these filtered rows.</p> <p>So, we could say it´s actually <strong>7 times faster</strong> than the the databases we´ve tried, since we do <strong>NOT</strong> do any aggregation-stuff in those cases!</p> <p>So, in conclusion, why are the databases so poor at filtering data compared to in-memory LINQ filtering? Have Microsoft really done such a good job that it is impossible to compete with it? :)</p> <p>It makes sense though that in-memory filtering should be faster, but I don´t want a <strong>sense</strong> that it is faster. I want to <strong>know</strong> what is faster, and if it´s possible <strong>why</strong>.</p>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PTQuestion
    1. USJohan
    1. USJohan
    plurals
    1. This table or related slice is empty.
    1. PL
      singulars
      1. LTLinked
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POIs it possible to accelerate (dynamic) LINQ queries using GPU?
      1. USEoin Campbell
      1. VTFavorite
    2. VO
      singulars
      1. POIs it possible to accelerate (dynamic) LINQ queries using GPU?
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. POIs it possible to accelerate (dynamic) LINQ queries using GPU?
      1. This table or related slice is empty.
      1. VTUpMod
    1. COnot that I know of but +1 for the idea. I anticipate the answers.
      singulars
      1. POIs it possible to accelerate (dynamic) LINQ queries using GPU?
      1. USJodrell
    2. CO@Jodrell, thanks.
      singulars
      1. POIs it possible to accelerate (dynamic) LINQ queries using GPU?
      1. USJohan
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload