StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POHow can I ensure that my HttpClient 4.1 does not leak sockets?
text
Body
copied!<p>My server uses data from an internal web service to construct its response, on a per request basis. I'm using Apache HttpClient 4.1 to make the requests. Each initial request will result in about 30 requests to the web service. Of these, 4 - 8 will end up with sockets stuck in CLOSE_WAIT, which never get released. Eventually these stuck sockets exceed my ulimit and my process runs out of file descriptors.</p> <p>I don't want to just raise my ulimit (1024), because that will just mask the problem.</p> <p>The reason I've moved to HttpClient is that java.net.HttpUrlConnection was behaving the same way.</p> <p>I have tried moving to a SingleClientConnManager per request, and calling client.getConnectionManager().shutdown() on it, but sockets still end up stuck.</p> <p>Should I be trying to solve this so that I end up with 0 open sockets while there are no running requests, or should I be concentrating on request persistence and pooling?</p> <p>For clarity I'm including some details which may be relevant:</p> <p>OS: Ubuntu 10.10</p> <p>JRE: 1.6.0_22</p> <p>Language: Scala 2.8</p> <p>Sample code:</p> <pre><code>val cleaner = Executors.newScheduledThreadPool(1) private val client = { val ssl_ctx = SSLContext.getInstance("TLS") val managers = Array[TrustManager](TrustingTrustManager) ssl_ctx.init(null, managers, new java.security.SecureRandom()) val sslSf = new org.apache.http.conn.ssl.SSLSocketFactory(ssl_ctx, SSLSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER) val schemeRegistry = new SchemeRegistry() schemeRegistry.register(new Scheme("https", 443, sslSf)) val connection = new ThreadSafeClientConnManager(schemeRegistry) object clean extends Runnable{ override def run = { connection.closeExpiredConnections connection.closeIdleConnections(30, SECONDS) } } cleaner.scheduleAtFixedRate(clean,10,10,SECONDS) val httpClient = new DefaultHttpClient(connection) httpClient.getCredentialsProvider().setCredentials(new AuthScope(AuthScope.ANY), new UsernamePasswordCredentials(username,password)) httpClient } val get = new HttpGet(uri) val entity = client.execute(get).getEntity val stream = entity.getContent val justForTheExample = IOUtils.toString(stream) stream.close() </code></pre> <p>Test: netstat -a | grep {myInternalWebServiceName} | grep CLOSE_WAIT</p> <p>(Lists sockets for my process that are in CLOSE_WAIT state)</p> <p>Post comment discussion:</p> <p>This code now demonstrates correct usage.</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload