StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POBest practice for importing a partial database dump into a rails app daily?
text
Body
copied!<p>The <a href="http://www.apple.com/itunes/affiliates/resources/documentation/itunes-enterprise-partner-feed.html" rel="nofollow">iTunes Enterprise Partner Feed</a> is "a data feed of the complete set of metadata from iTunes and the App Store" and "is available in two different formats - either as the files necessary to build a relational database or as stand-alone flat files that are country and media dependent."</p> <p>I need to consume the data from this feed (which is essentially exported into flat files) and allow linking of my own Model objects (User, Activity, etc.) to data provided by the feed (App, Developer, etc.) The data is provided as a weekly full export <strong>and</strong> a daily incremental export.</p> <p>I have two ideas for ways to implement this:</p> <ol> <li>Create all of the models in my rails app and write my own importer that will insert/update records directly into my app's database daily via cron using models I've created (App, Developer, etc.)</li> <li>Keep this database entirely separate and open up REST API that my own app will consume</li> </ol> <p>My naive approach with <strong>#1</strong> to keep everything in the Rails app is based on the need to be able to observe changes in the data I'm getting from the EPF. For example, if an App's description is updated, I want to be able to create an Activity object via an observer to track that update. </p> <p>On one hand <strong>#2</strong> feels like a better approach because it creates a standalone API into the data that can be consumed from several different apps I create. On the other hand, I'm just not sure how I'd accomplish the data change notifications without using observers directly on my own models. Or, how I would even consume the data in an object oriented way that could still be used with my own models. It feels like a lot of duplicate work to have to query the API for, say, an App's data, create a proper Active Record object for it and then save it just so it can be linked to one of my own models.</p> <p>Is there a standard way to do this that I'm totally missing? Any pointers?</p> <p><strong>EDIT:</strong> Rails engines sound interesting but it would mean that each app would still need to consume and insert the data separately. That doesn't sound so DRY. It sounds more and more like a REST API is the way to go. I just don't know how to bridge the gap from API to Active Record model.</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload