StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PODrop keyspace hangs
text
Body
copied!<p>Problem: <code>drop keyspace MyKeyspace;</code> hangs.</p> <p><strong>Environment:</strong> This is an Ubuntu 12.04 64bit in virtualbox, running a single Cassandra instance (on a development machine). </p> <p>Cassandra is 1.1.6: </p> <pre><code>myuser@myhost:~$ /usr/bin/nodetool -h localhost version ReleaseVersion: 1.1.6 </code></pre> <p>Plenty of free disk space:</p> <pre><code>myuser@myhost:~$ df Filesystem 1K-blocks Used Available Use% Mounted on /dev/mapper/myhost-root 100232772 3100308 92112816 4% / udev 1016760 4 1016756 1% /dev tmpfs 410340 268 410072 1% /run none 5120 0 5120 0% /run/lock none 1025844 0 1025844 0% /run/shm /dev/sda1 233191 24999 195751 12% /boot </code></pre> <p>Machine is idle: </p> <pre><code>myuser@myhost:~$ uptime 21:24:50 up 3:46, 2 users, load average: 0.06, 0.04, 0.05 </code></pre> <p><strong>How I got there:</strong></p> <p>The machine was running another db, all fine for long time. Now I created a new keyspace MyKeyspace, and run a Java program to import data (using titan graph, but that shouldn't matter). After a couple thousand of records imported (a couple of MB only) the import program did not make progress anymore, and logged 6 times: </p> <pre><code>418455 [RetryService : myhost(192.168.1.241):9160] INFO com.netflix.astyanax.connectionpool.impl.CountingConnectionPoolMonitor - Reactivating myhost </code></pre> <p>Then my program (titan graph actually) gave up with:</p> <pre><code>com.thinkaurelius.titan.diskstorage.TemporaryStorageException: Temporary failure in storage backend </code></pre> <p>During all this time I was connected to <code>/usr/bin/cassandra-cli</code> and could successfully execute <code>show keyspaces;</code>.</p> <p>Then I decided to start over; drop the keyspace. That's where it hangs now, for hours. It doesn't respond to ctrl-c either. Meanwhile I'm able to log in by ssh, connect with cassandra-cli, and run <code>show keyspaces;</code>. The keyspace is still there. Also, my Java app can access that data store, but it's read only. Reading succeeds, but writes fail. It's just a timeout I get from the titan graph library when writing: </p> <pre><code>com.thinkaurelius.titan.core.TitanException: ID renewal thread on partition [2] did not complete in time. [60007 ms] </code></pre> <p>Any commands I could run to see what's going on? Should I report a bug? </p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload