Databases
StackOverflow2013
Postsdbo
POWCF REST, streamed upload of files and httpRuntime maxRequestLength property
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POWCF REST, streamed upload of files and httpRuntime maxRequestLength property
    primarykey
    data
    text
    <p>I have created a simple WCF service to prototype file uploading. The service:</p> <pre><code>[ServiceContract] public class Service1 { [OperationContract] [WebInvoke(Method = "POST", UriTemplate = "/Upload")] public void Upload(Stream stream) { using (FileStream targetStream = new FileStream(@"C:\Test\output.txt", FileMode.Create, FileAccess.Write)) { stream.CopyTo(targetStream); } } } </code></pre> <p>It uses <code>webHttpBinding</code> with <code>transferMode</code> set to "Streamed" and <code>maxReceivedMessageSize</code>, <code>maxBufferPoolSize</code> and <code>maxBufferSize</code> all set to 2GB. <code>httpRuntime</code> has <code>maxRequestLength</code> set to 10MB. </p> <p>The client issues HTTP requests in the following way:</p> <pre><code>HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(@"http://.../Service1.svc/Upload"); request.Method = "POST"; request.SendChunked = true; request.AllowWriteStreamBuffering = false; request.ContentType = MediaTypeNames.Application.Octet; using (FileStream inputStream = new FileStream(@"C:\input.txt", FileMode.Open, FileAccess.Read)) { using (Stream outputStream = request.GetRequestStream()) { inputStream.CopyTo(outputStream); } } </code></pre> <p>Now, finally, what's wrong:</p> <p>When uploading the file 100MB big, the server returns HTTP 400 (Bad request). I've tried to enable WCF tracing, but it shows no error. When I increase <code>httpRuntime</code>.<code>maxRequestLength</code> to 1GB, the file gets uploaded without problems. The <a href="http://msdn.microsoft.com/sk-sk/library/e1f13641%28v=vs.100%29.aspx" rel="noreferrer">MSDN says</a> that <code>maxRequestLength</code> "<em>specifies the limit for the input stream buffering threshold, in KB</em>". </p> <p>This leads me to believe that the whole file (all 100MB of it) is first stored in "input stream buffer" and only then it is available to my <code>Upload</code> method on server. I can actually see that the size of file on server does not gradually increase (as I would expect), instead, in the moment it is created it is already 100MB big.</p> <p><strong>The question:</strong> How can I get this to work so that the "input stream buffer" is reasonably small (say, 1MB) and when it overflows, my <code>Upload</code> method gets called? In other words, I want the upload to be truly streamed without having to buffer the whole file anywhere.</p> <p><strong>EDIT:</strong> I now discovered the <code>httpRuntime</code> contains another setting that is relevant here - <code>requestLengthDiskThreshold</code>. It seems that when the input buffer grows beyond this threshold, it is no longer stored in memory, but instead, on filesystem. So at least the whole 100MB big file is not kept in memory (this is what I was most afraid of), however, I still would like to know whether there is some way to <strong>avoid this buffer altogether</strong>.</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USNikola Anusev
    1. USNikola Anusev
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POWCF REST, streamed upload of files and httpRuntime maxRequestLength property
      1. USNikola Anusev
      1. VTBountyStart
    2. VO
      singulars
      1. POWCF REST, streamed upload of files and httpRuntime maxRequestLength property
      1. USMike Payne
      1. VTFavorite
    3. VO
      singulars
      1. POWCF REST, streamed upload of files and httpRuntime maxRequestLength property
      1. This table or related slice is empty.
      1. VTUpMod
    1. CODon't have an answer off the top of my head, but this is doable, I've done it with 4096KB buffers and such. So, just for some reassurance really; I'll look into this when I'm done with work if I can.
      singulars
      1. POWCF REST, streamed upload of files and httpRuntime maxRequestLength property
      1. USGrant Thomas
    2. CO@GrantThomas Thanks for that, much appreciated.
      singulars
      1. POWCF REST, streamed upload of files and httpRuntime maxRequestLength property
      1. USNikola Anusev
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload