StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
text
Body
copied!<p>I think @karmi makes it right. However let me explain it a bit simpler. I needed to occasionally upgrade production schema with some new properties or analysis settings. I recently started to use the scenario described below to do live, constant load, zero-downtime index migrations. You can do that remotely.</p> <p>Here are steps:</p> <h2>Assumptions:</h2> <ul> <li>You have index <code>real1</code> and aliases <code>real_write</code>, <code>real_read</code> pointing to it,</li> <li>the client writes only to <code>real_write</code> and reads only from <code>real_read</code> ,</li> <li><code>_source</code> property of document is available.</li> </ul> <h2>1. New index</h2> <p>Create <code>real2</code> index with new mapping and settings of your choice.</p> <h2>2. Writer alias switch</h2> <p>Using following bulk query switch write alias.</p> <pre><code>curl -XPOST 'http://esserver:9200/_aliases' -d ' { "actions" : [ { "remove" : { "index" : "real1", "alias" : "real_write" } }, { "add" : { "index" : "real2", "alias" : "real_write" } } ] }' </code></pre> <p>This is atomic operation. From this time <code>real2</code> is populated with new client's data on all nodes. Readers still use old <code>real1</code> via <code>real_read</code>. This is eventual consistency.</p> <h2>3. Old data migration</h2> <p>Data must be migrated from <code>real1</code> to <code>real2</code>, however new documents in <code>real2</code> can't be overwritten with old entries. Migrating script should use <code>bulk</code> API with <code>create</code> operation (not <code>index</code> or <code>update</code>). I use simple Ruby script <a href="https://github.com/pgaertig/es-reindex" rel="nofollow noreferrer">es-reindex</a> which has nice E.T.A. status:</p> <pre><code>$ ruby es-reindex.rb http://esserver:9200/real1 http://esserver:9200/real2 </code></pre> <p><em>UPDATE 2017</em> You may consider new <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html" rel="nofollow noreferrer">Reindex API</a> instead of using the script. It has lot of interesting features like conflicts reporting etc.</p> <h2>4. Reader alias switch</h2> <p>Now <code>real2</code> is up to date and clients are writing to it, however they are still reading from <code>real1</code>. Let's update reader alias:</p> <pre><code>curl -XPOST 'http://esserver:9200/_aliases' -d ' { "actions" : [ { "remove" : { "index" : "real1", "alias" : "real_read" } }, { "add" : { "index" : "real2", "alias" : "real_read" } } ] }' </code></pre> <h2>5. Backup and delete old index</h2> <p>Writes and reads go to <code>real2</code>. You can backup and delete <code>real1</code> index from ES cluster.</p> <p>Done!</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload