Databases
StackOverflow2013
Postsdbo
POHow to handle large quantities of inter-related data
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POHow to handle large quantities of inter-related data
    primarykey
    data
    text
    <p>Currently I have a DB structure in MySQL with a few dozen tables with various foreign key linkages among them. All of the data is in files that I'm going to load in, so I'm hoping I can port the design over to a storage system that works with Drupal 7, since I can simply setup something (using the Feeds module?) to get my data in a way that Drupal 7 likes. The ultimate purpose of this is for lots of manual human revision, linking together entries in tables with relations and possibly revising some field data that looks wrong. So the whole goal is to make the human interface for viewing and editing (particularly adding) relations in Drupal 7. The question is, what is the proper way for the data to be stored so I have to write as little module code as needed?</p> <p>It seems to me that I would choose one of 3 modules to accomplish this task: </p> <ul> <li>Relation</li> <li>Entity Reference</li> <li>Data</li> </ul> <p>Relation and Entity Reference would allow me to store all of my data in nodes (entities?) in Drupal 7, so Drupal would have "native support" for handling all of the stuff. However, I expect there to be 100s of millions to low billions of nodes with perhaps up to ~3 relations to other nodes in each. How efficiently does Relation or Entity Reference handle this when referencing foreign data (and perhaps grabbing referenced data from that reference, and so on) with Views and the like? Can they support a node having a null reference, since many will be null until a user can set them (so I'd need a way to also have a view to find nodes with particular null references)?</p> <p>Data is another possibility, but it's in alpha and I wonder about its stability and efficiency. It also seems to me that having all of my data stored in an external MySQL database instead of in Drupal nodes defeats the entire purpose of using Drupal in the first place. Is my feeling on this correct?</p> <p>I'm having a difficult time nailing down what I would need to manage my content, which seems odd considering Drupal 7 is a CMS. I have to be missing something here, but I'm not sure what it is. What is the most mature module(s?) for handling/interfacing with this large quantity of inter-related data and being able to go through and have a user mostly setting up and managing the links (so "foreign keys") between "tables", along with perhaps field data review and revision? Are there any that would suffice?</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. This table or related slice is empty.
    1. USuser173342
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POHow to handle large quantities of inter-related data
      1. USuser173342
      1. VTBountyStart
    2. VO
      singulars
      1. POHow to handle large quantities of inter-related data
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. POHow to handle large quantities of inter-related data
      1. This table or related slice is empty.
      1. VTUpMod
    1. COWhy are you so sure you need to use Drupal?
      singulars
      1. POHow to handle large quantities of inter-related data
      1. USilanco
    2. COI don't know if Data will help in the process of turning things into entities. You probably want to get a good grasp of entities in general then use something like the entity construction kit to prep your different types of content then import each different type using feeds (Not sure if feeds works well with entities). The last step would be to create an interface in views (maybe using VBO as well) to let your users take care of the rest of the logistics.
      singulars
      1. POHow to handle large quantities of inter-related data
      1. USAntonio Torres
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload