StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POhtml form submission in python and php is simple, can a novice do it in java?
text
Body
copied!<p>I've made two versions of a script that submits a (https) web page form and collects the results. One version uses Snoopy.class in php, and the other uses urllib and urllib2 in python. Now I would like to make a java version.</p> <p>Snoopy makes the php version exceedingly easy to write, and it runs fine on my own (OS X) machine. But it allocated too much memory, and was killed at the same point (during curl execution), when run on the pair.com web hosting service. Runs fine on dreamhost.com web hosting service.</p> <p>So I decided to try a python version while I looked into what could cause the memory problem, and urllib and urllib2 made this very easy. The script runs fine. Gets about 70,000 database records, using several hundred form submissions, saving to a file of about 10MB, in about 7 minutes.</p> <p>Looking into how to do this with java, I get the feeling it will not be the same walk-in-the-park as it was with php and python. Is form submission in java not for mere mortals?</p> <p>I spent most of the day just trying to figure out how to set up Apache HttpClient. That is, before I gave up. If it takes me more than a few more days to sort that out, then it will be the subject of another question, I suppose.</p> <p>HttpClient innovation.ch does not support https.</p> <p>And WebClient looks like it will take me at least a few days to figure out.</p> <p>So, php and python versions were a breeze. Can a java version be made in a few simple lines as well? If not, I'll leave it for a later day since I'm only a novice. If so, can some kind soul please point me toward the light?</p> <p>Thanks.</p> <p>For comparison, the essential lines of code from the two versions:</p> <hr> <p>python version</p> <pre><code>import urllib import urllib2 submitVars['firstName'] = "John" submitVars['lastName'] = "Doe" submitUrl = "https URL of form action goes here" referer = "URL of referring web page goes here" submitVarsUrlencoded = urllib.urlencode(submitVars) req = urllib2.Request(submitUrl, submitVarsUrlencoded) req.add_header('Referer', referer) response = urllib2.urlopen(req) thePage = response.read() </code></pre> <hr> <p>php version</p> <pre><code>require('Snoopy.class.php'); $snoopy = new Snoopy; $submit_vars["first_name"] = "John"; $submit_vars["last_name"] = "Doe"; $submit_url = "https URL of form action goes here"; $snoopy->referer = "URL of referring web page goes here"; $snoopy->submit($submit_url,$submit_vars); $the_page = $snoopy->results; </code></pre>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload