Databases
StackOverflow2013
Postsdbo
POHow to Delete a 000000 file in S3 bucket in AWS using a hive script
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POHow to Delete a 000000 file in S3 bucket in AWS using a hive script
    primarykey
    data
    text
    <p>I've created a working hive script to backup data from dynamodb to a file in S3 bucket in AWS. A code snippet is shown below</p> <pre><code>INSERT OVERWRITE DIRECTORY '${hiveconf:S3Location}' SELECT * FROM DynamoDBDataBackup; </code></pre> <p>When I run the hive script it probably deletes the old file and creates a new file but if there are errors in the backup process I guess it rolls back to the old data because the file is still there when an error has occurred.</p> <p>Each day we want to make a backup but I need to know if an error has occurred so I want to delete the previous days backup first then create a backup. If it fails then there is no file in the folder which we can automatically detect.</p> <p>The filename gets automatically named 000000</p> <p>In my hive script I've tried unsuccesfully:</p> <pre><code>delete FILE '${hiveconf:S3Location}/000000' </code></pre> <p>and</p> <pre><code>delete FILE '${hiveconf:S3Location}/000000.0' </code></pre> <p>Perhaps the filename is wrong. I haven't set any permissions on the file.</p> <p>I've just tried this but fails at STORED</p> <pre><code>SET dynamodb.endpoint= ${DYNAMODBENDPOINT}; SET DynamoDBTableName = "${DYNAMODBTABLE}"; SET S3Location = ${LOCATION}; DROP TABLE IF EXISTS DynamoDBDataBackupPreferenceStore; CREATE TABLE IF NOT EXISTS DynamoDBDataBackupPreferenceStore(UserGuid STRING,PreferenceKey STRING,DateCreated STRING,DateEmailGenerated STRING,DateLastUpdated STRING,ReceiveEmail STRING,HomePage STRING,EmailFormat STRING,SavedSearchCriteria STRING,SavedSearchLabel STRING), STORED BY 'org.apache.hadoop.hive.dynamodb.DynamoDBStorageHandler' </code></pre> <p>LOCATION '${hiveconf:S3Location}', TBLPROPERTIES ("dynamodb.table.name" = ${hiveconf:DynamoDBTableName}, "dynamodb.column.mapping" = "UserGuid:UserGuid,PreferenceKey:PreferenceKey,DateCreated:DateCreated,DateEmailGenerated:DateEmailGenerated,DateLastUpdated:DateLastUpdated,ReceiveEmail:ReceiveEmail,HomePage:HomePage,EmailFormat:EmailFormat,SavedSearchCriteria:SavedSearchCriteria,SavedSearchLabel:SavedSearchLabel");</p>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PTQuestion
    1. USinsanepaul
    1. USinsanepaul
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. This table or related slice is empty.
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload