Databases
StackOverflow2013
Postsdbo
POUsing Pentaho Kettle, how do I automatically retry rest requests which fail due to connection hiccups?
Body
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POUsing Pentaho Kettle, how do I automatically retry rest requests which fail due to connection hiccups?
    text
    copied!<h2>How can we make Pentaho retry rest requests on connection errors?</h2> <p>We have a Pentaho BI system which, among numerous data sources, is querying a particular REST api for over 20k query variations each run.</p> <p>Predicatably, on most runs a few of these requests will fail due to connection hiccups. These usually manifest themselves in the Pentaho log as <code>org.pentaho.di.core.exception.KettleException</code> due to <code>javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake</code>, ultimately caused by <code>java.io.EOFException: SSL peer shut down incorrectly</code>.</p> <p>After scouring the internet and Pentaho forums, we cannot find any instructions on adding a simple retry approach in cases such as this. Hair is being torn out.</p> <p><i>Help me StackOverflow, you're my only hope!</i></p> <p><strong>Update 1:</strong> Stacktrace:</p> <pre><code>ERROR 29-11 11:02:17,659 - B - org.pentaho.di.core.exception.KettleException: Can not result from [https://&lt;DOMAIN&gt;/&lt;PATH&gt;?&lt;PARAMS&gt;] javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake at org.pentaho.di.trans.steps.rest.Rest.callRest(Rest.java:190) at org.pentaho.di.trans.steps.rest.Rest.processRow(Rest.java:385) at org.pentaho.di.trans.step.RunThread.run(RunThread.java:40) at java.lang.Thread.run(Thread.java:662) Caused by: com.sun.jersey.api.client.ClientHandlerException: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake at com.sun.jersey.client.apache.DefaultApacheHttpMethodExecutor.executeMethod(DefaultApacheHttpMethodExecutor.java:213) at com.sun.jersey.client.apache.ApacheHttpClientHandler.handle(ApacheHttpClientHandler.java:175) at com.sun.jersey.api.client.filter.HTTPBasicAuthFilter.handle(HTTPBasicAuthFilter.java:81) at com.sun.jersey.api.client.Client.handle(Client.java:648) at com.sun.jersey.api.client.WebResource.handle(WebResource.java:670) at com.sun.jersey.api.client.WebResource.get(WebResource.java:191) at org.pentaho.di.trans.steps.rest.Rest.callRest(Rest.java:141) ... 3 more Caused by: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:817) at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1138) at com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:632) at com.sun.net.ssl.internal.ssl.AppOutputStream.write(AppOutputStream.java:59) at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65) at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123) at org.apache.commons.httpclient.HttpConnection.flushRequestOutputStream(HttpConnection.java:827) at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.flushRequestOutputStream(MultiThreadedHttpConnectionManager.java:1525) at org.apache.commons.httpclient.HttpMethodBase.writeRequest(HttpMethodBase.java:1975) at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:993) at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:397) at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:170) at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:396) at com.sun.jersey.client.apache.DefaultApacheHttpMethodExecutor.executeMethod(DefaultApacheHttpMethodExecutor.java:210) ... 9 more Caused by: java.io.EOFException: SSL peer shut down incorrectly at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:333) at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:798) ... 22 more </code></pre>
 

Querying!

 
Guidance
An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload