StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POSend large files to GAE Blobstore via API (from S3)
primarykey
Id
15970514
data
AcceptedAnswerId
0
AnswerCount
1
ClosedDate
CommentCount
3
CommunityOwnedDate
CreationDate
2013-04-12T11:43:55.767
FavoriteCount
0
LastActivityDate
2013-04-12T14:11:00.327
LastEditDate
2017-05-23T12:12:35.880
LastEditorUserId
-1
OwnerUserId
344877
ParentId
0
PostTypeId
1
Score
1
ViewCount
351
LastEditorDisplayName
text
Body
I'm trying to send large files (50MB-2GB) that I have stored in S3 using filepicker.io to the Google App Engine Blobstore (in Python). I would like this to happen without going through a form on the client browser as it defeats the project requirements (and often hangs with very large files). I tried multiple solutions, including: <ul> <li>trying to <a href="https://stackoverflow.com/questions/12149315/how-do-i-integrate-filepicker-io-with-google-app-engine-blobstore">load the file into GAE with urlfetch</a> (but <a href="https://stackoverflow.com/questions/6898736/upload-file-bigger-than-40mb-to-google-app-engine/6899157#6899157">GAE has a 32MB limit</a> for requests/responses)</li> <li>constructing a multi-part form in python and sending it to <code>blobstore.create_upload_url()</code> (can't transfer the file via url, and can't load it in the script because of the 32MB limit)</li> <li>using <a href="https://stackoverflow.com/questions/3948391/is-is-possible-to-read-a-file-from-s3-in-google-app-engine-using-boto">boto to read the file straight into the blobstore</a> (connection times out and logs show <code>encountered HTTPException exception</code> from boto that triggers <code>CancelledError: The API call logservice.Flush() was explicitly cancelled.</code> from GAE that crashes the process.</li> </ul> I am struggling to find a working solution. Any hints on how I could perform this transfer, or how to pass the file from s3 as a form attachment without loading it in python first (ie. just specifying its url) would be very much appreciated.
Tags
<python><google-app-engine><amazon-s3><blobstore><filepicker.io>
Title
Send large files to GAE Blobstore via API (from S3)
singulars
PostAcceptedAnswerId
1. This table or related slice is empty.
PostParentId
1. This table or related slice is empty.
PostTypePostTypeId
1. PTQuestion
UserLastEditorUserId
1. USCommunity
UserOwnerUserId
1. USergelo
plurals
PostLinksPostIdRelatedPostId
1. PL
 singulars
 LinkTypeLinkTypeId
 LTLinked
2. PL
 singulars
 LinkTypeLinkTypeId
 LTLinked
3. PL
 singulars
 LinkTypeLinkTypeId
 LTLinked
PostLinksRelatedPostIdPostId
1. PL
 singulars
 LinkTypeLinkTypeId
 LTLinked
PostsAcceptedAnswerId
1. This table or related slice is empty.
PostsParentIdCreationDate
1. PO
 singulars
 PostTypePostTypeId
 PTAnswer
VotesPostIdCreationDate
1. VO
 singulars
 PostPostId
 POSend large files to GAE Blobstore via API (from S3)
 UserUserId
 This table or related slice is empty.
 VoteTypeVoteTypeId
 VTUpMod
CommentsPostId

Querying!

Guidance

A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.