Databases
StackOverflow2013
Postsdbo
POWhy does a FTP send depend on its directory size?
Body
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POWhy does a FTP send depend on its directory size?
    text
    copied!<p>We have a Proxy that is taking messages from a JMS queue and sends them to an FTP folder. We discovered now, that the sending to the FTP is very slow when the target directory on the FTP already contains a lot of files. (i.e. when I have around 2000 files in a directory, it already takes several seconds)</p> <p>Here the code of our Proxy (get messages (plain-text) from a JMS and writes them to FTP):</p> <pre><code>&lt;?xml version="1.0" encoding="UTF-8"?&gt; &lt;proxy xmlns="http://ws.apache.org/ns/synapse" name="myProxy" statistics="disable" trace="disable" transports="jms"&gt; &lt;parameter name="transport.jms.Destination"&gt;myQueue&lt;/parameter&gt; &lt;parameter name="transport.jms.ConnectionFactory"&gt;myQueueConnectionFactory&lt;/parameter&gt; &lt;parameter name="transport.jms.DestinationType"&gt;queue&lt;/parameter&gt; &lt;parameter name="transport.jms.ContentType"&gt; &lt;rules&gt; &lt;jmsProperty&gt;contentType&lt;/jmsProperty&gt; &lt;default&gt;text/plain&lt;/default&gt; &lt;/rules&gt; &lt;/parameter&gt; &lt;target faultSequence="rollbackSequence"&gt; &lt;inSequence&gt; &lt;log level="custom"&gt; &lt;property name="STATUS" value="myProxy called"/&gt; &lt;/log&gt; &lt;property name="ClientApiNonBlocking" scope="axis2" action="remove"/&gt; &lt;property name="OUT_ONLY" value="true"/&gt; &lt;property name="transport.vfs.ReplyFileName" expression="fn:concat(get-property('SYSTEM_DATE','yyyyMMddHHmmss_SSS'), '_result.txt')" scope="transport"/&gt; &lt;send&gt; &lt;endpoint key="myFTPendpoint"/&gt; &lt;/send&gt; &lt;/inSequence&gt; &lt;/target&gt; </code></pre> <p></p> <p>And the FTPEndpoint lookes like this:</p> <pre><code>&lt;?xml version="1.0" encoding="UTF-8"?&gt; &lt;endpoint xmlns="http://ws.apache.org/ns/synapse" name="myFTPendpoint"&gt; &lt;address uri="vfs:ftp://USER:PASSWORD@SERVER.com/path/toSomewhere?vfs.passive=true"/&gt; &lt;/endpoint&gt; </code></pre> <p>My analysis for now:</p> <ol> <li>It is only slow when using FTP with VFS. When using the local file system - it is fast.</li> <li>The files are tiny - so it's not the upload time</li> <li>The network is fast</li> <li><strong>!Speed depends on the number of files already in the directory on the FTP!</strong></li> </ol> <p>Possible solutions?</p> <ul> <li>Fix the problem of the speed. Disable the directory listing?</li> <li>Workaraound: Create new folders at the output (that not one folder gets filled too much)</li> </ul> <p>Does someone also discovered the same issue? And how can the FTP speed to big directories be improved? Thanks for any help</p>
 

Querying!

 
Guidance
An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload