Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>Unfortunately their is no magic bullet which you could use to improve your performance. As always it will be a trade off between speed and memory. Also their are two sides which could lack on performance: The database site and the hdd drive i/o speed.</p> <p>So to gain speed i would in a first step improve the performance of the database query to ensure that it can return the names for searching fast enough. So ensure that your query is fast and also maybe uses (im MS SQL case) keywords like <code>READ SEQUENTIAL</code> in this case you will already retrieve the first results while the query is still running and you don't have to wait till the query finished and gave you the names as a big block.</p> <p>On the other hdd side you can either call <code>Directory.GetFiles()</code>, but this call would block till it iterated over all files and will give you back a big array containing all filenames. This would be the memory consuming path and take a while for the first search, but if you afterwards only work on that array you get speed improvements for all consecutive searches. Another approach would be to call <code>Directory.EnumerateFiles()</code> which would search the drive on the fly by every call and so maybe gain speed for the first search, but their won't happen any memory storage for the next search which improves memory footprint but costs speed, due to the fact that their is no array in your memory which could be searched. On the other hand the OS will also do some caching if detects that you iterate over the same files over and over again and some caching occurs on a lower level.</p> <p>So for the check on hdd site use <code>Directory.GetFiles()</code> if the returned array won't blow your memory and do all your searches on this (maybe put it into a <code>HashSet</code> to further improve performance if filename only or full path depends on what you get from your database) and in the other case use <code>Directory.EnumerateFiles()</code> and hope the best for some caching done be the OS.</p> <h3>Update</h3> <p>After re-reading your question and comments, as far as i understand you have a name like <code>1234567891_w.jpg</code> and you don't know which part of the name represents the directory part. So in this case you need to make an explicit search, cause iteration through all directories simply takes to much time. Here is some sample code, which should give you an idea on how to solve this in a first shot:</p> <pre><code>string rootDir = @"D:\RootDir"; // Iterate over all files reported from the database foreach (var filename in databaseResults) { var fullPath = Path.Combine(rootDir, filename); // Check if the file exists within the root directory if (File.Exists(Path.Combine(rootDir, filename))) { // Report that the file exists. DoFileFound(fullPath); // Fast exit to continue with next file. continue; } var directoryFound = false; // Use the filename as a directory var directoryCandidate = Path.GetFileNameWithoutExtension(filename); fullPath = Path.Combine(rootDir, directoryCandidate); do { // Check if a directory with the given name exists if (Directory.Exists(fullPath)) { // Check if the filename within this directory exists if (File.Exists(Path.Combine(fullPath, filename))) { // Report that the file exists. DoFileFound(fullPath); directoryFound = true; } // Fast exit, cause we looked into the directory. break; } // Is it possible that a shorter directory name // exists where this file exists?? // If yes, we have to continue the search ... // (Alternative code to the above one) ////// Check if a directory with the given name exists ////if (Directory.Exists(fullPath)) ////{ //// // Check if the filename within this directory exists //// if (File.Exists(Path.Combine(fullPath, filename))) //// { //// // Report that the file exists. //// DoFileFound(fullPath); //// // Fast exit, cause we found the file. //// directoryFound = true; //// break; //// } ////} // Shorten the directory name for the next candidate directoryCandidate = directoryCandidate.Substring(0, directoryCandidate.Length - 1); } while (!directoryFound &amp;&amp; !String.IsNullOrEmpty(directoryCandidate)); // We did our best but we found nothing. if (!directoryFound) DoFileNotAvailable(filename); } </code></pre> <p>The only furhter performance improvement i could think of, would be putting the directories found into a <code>HashSet</code> and before checking with <code>Directory.Exists()</code> use this to check for an existing directory, but maybe this wouldn't gain anything cause the OS already makes some caching in directory lookups and would then nearly as fast as your local cache. But for these things you simply have to measure your concrete problem.</p>
    singulars
    1. This table or related slice is empty.
    1. POFast way to check existance of a file in a directory with filename
      singulars
      1. PTQuestion
    1. PTAnswer
    1. USOliver
    1. USOliver
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. COYes it is MS SQL..but as the query will execute only once to fetch all the fileNames, I dont think that will be an area of much concern..I was thinking if we can have some effective logic to search a file in DIRECTORY and SUBDIRECTORY in one shot..
      singulars
      1. PO
      1. USJames
    2. CO@Learner: Both methods of `Directory` have an overload which takes an `SearchOption` which can be set to `SearchOption.AllDirectories`, leading to make a search on the given directory and ALL subdirectories.
      singulars
      1. PO
      1. USOliver
    3. COTrue..Can you give your thought on the way I found to address this, the answer I have added..Whether am I missing anything?
      singulars
      1. PO
      1. USJames
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload