Databases
StackOverflow2013
Postsdbo
POI want a clever algorithm for indexing a file directory...pointers?
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POI want a clever algorithm for indexing a file directory...pointers?
    primarykey
    data
    text
    <p>I have a directory of music on Ubuntu (.mp3, .wav, etc) files. This directory can have as many sub directories as it needs, no limits. I want to be able to make a music library out of it - that is, return list of songs based on filters of:</p> <p>1) membership to playlist 2) artist name 3) string search 4) name of song etc, etc</p> <p>However, if file names are changed, moved, or even added to my Music directory, I need to be able to reflect this is in my music organization engine - quickly! </p> <p>I originally thought to just monitor my directory with pyinotify, incron, or inotify. Unfortunately my directory is a Samba share and so monitoring file events <a href="https://stackoverflow.com/questions/6795847/notifier-on-linux-filesystem-for-samba-share">failed</a>. So my next guess was to simply recursively search the directory in python, and populate a SQL database. Then when updating, I would just look to see if anything has changed (scanning each subfolder to see if each song's name is in the database already, and if not adding it), and make UPDATEs accordingly. Unfortunately, this seems to be a terrible <code>O(n^2)</code> implementation - awful for a multi-terabyte music collection. </p> <p>A slightly better one might involve creating a tree structure in SQL, thus narrowing the possible candidates to search for a match at any given subfolder step to the size of that subfolder. Still seems inelegant. </p> <p>What design paradigms/packages can I use to help myself out? Obviously will involve lots of clever hash tables. I'm just looking for some pointers in the right direction for how to approach the problem. (Also I'm a complete junkie for optimization.)</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USCommunity
    1. USlollercoaster
    plurals
    1. PL
      singulars
      1. LTLinked
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    3. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POI want a clever algorithm for indexing a file directory...pointers?
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POI want a clever algorithm for indexing a file directory...pointers?
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. POI want a clever algorithm for indexing a file directory...pointers?
      1. This table or related slice is empty.
      1. VTUpMod
    1. COWhy would your original idea by O(n^2)? Assuming the database had a O(log n) index on the song name (which should be easy to arrange) it should be O(n log n).
      singulars
      1. POI want a clever algorithm for indexing a file directory...pointers?
      1. USandrewdski
    2. COIf you are so worried about complexity you could use a hash which is `O(n)` for `n` lookups/updates.
      singulars
      1. POI want a clever algorithm for indexing a file directory...pointers?
      1. USGabi Purcaru
    3. COI second the hashing idea; the only problem is, how do you decide a unique key to hash with? I wouldn't go with any of the normal file attributes, cause those can change. Maybe make a key based on samples from within the file?
      singulars
      1. POI want a clever algorithm for indexing a file directory...pointers?
      1. USmachine yearning
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload