Databases
StackOverflow2013
Postsdbo
POcannot download csv files from morningstar.com using authentication
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POcannot download csv files from morningstar.com using authentication
    primarykey
    data
    text
    <p>I have a premium account with morningstar and I tried to download a few csv files from the premium content area. For some reason I cannot get those premium content. For example, with premium account I can get 10 year financial statement data, but I've tried all the sample authentication java code from apache httpcomponents-client. All of them can only get me content that does not need authentication. How can I tell what authentication protocol morningtar is using and authenticate successfully? I tried the example code from org.apache.http.examples.client, including clientAuthentication.java, clientKerberosAuthentication.java, clientInteractiveAuthentication.java . If I log in in morningstar account in Chrome and paste this URL, I can get 10 years data csv, but if I access through java I only get 5 years data. Below are one of sample codes I tried. I didn't get exceptions or errors, but I only got 5 years data instead of 10.</p> <pre><code>import java.io.BufferedReader; import java.io.InputStreamReader; import org.apache.http.HttpEntity; import org.apache.http.HttpResponse; import org.apache.http.auth.AuthScope; import org.apache.http.auth.UsernamePasswordCredentials; import org.apache.http.client.methods.HttpGet; import org.apache.http.impl.client.DefaultHttpClient; import org.apache.http.util.EntityUtils; public class ClientAuthentication { public static void main(String[] args) throws Exception { DefaultHttpClient httpclient = new DefaultHttpClient(); try { httpclient.getCredentialsProvider().setCredentials( new AuthScope("morningstar.com", 443), new UsernamePasswordCredentials("xxx@gmail.com", "xxxx")); //anonymized this before posting to stackoverflow HttpGet httpget = new HttpGet("http://financials.morningstar.com/ajax/ReportProcess4CSV.html?t=aapl&amp;region=usa&amp;culture=en_US&amp;reportType=is&amp;period=12&amp;dataType=A&amp;order=asc&amp;columnYear=10&amp;rounding=3&amp;view=raw&amp;productCode=USA&amp;r=199209&amp;denominatorView=raw&amp;number=3"); System.out.println("executing request" + httpget.getRequestLine()); HttpResponse response = httpclient.execute(httpget); HttpEntity entity = response.getEntity(); BufferedReader in; in = new BufferedReader(new InputStreamReader(entity.getContent())); System.out.println("----------------------------------------"); System.out.println(response.getStatusLine()); if (entity != null) { System.out.println("Response content length: " + entity.getContentLength()); int linenum = 0; while (true){ String line = in.readLine(); if (line == null) break; linenum++; if (linenum&gt;1) System.out.println(line); } } EntityUtils.consume(entity); } finally { // When HttpClient instance is no longer needed, // shut down the connection manager to ensure // immediate deallocation of all system resources httpclient.getConnectionManager().shutdown(); } } } </code></pre>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PTQuestion
    1. USuser1030532
    1. USuser1030532
    plurals
    1. PL
      singulars
      1. LTLinked
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. COWhat have you tried? What errors are you getting? We need to see code and such.
      singulars
      1. POcannot download csv files from morningstar.com using authentication
      1. USCodeChimp
    2. COI tried the example code from org.apache.http.examples.client, including clientAuthentication.java, clientKerberosAuthentication.java, clientInteractiveAuthentication.java . The url to csv file is http://financials.morningstar.com/ajax/ReportProcess4CSV.html?t=aapl&region=usa&culture=en_US&reportType=is&period=12&dataType=A&order=asc&columnYear=10&rounding=3&view=raw&productCode=USA&r=199209&denominatorView=raw&number=3. If I log in in Chrome and paste this URL, I can get 10 years data csv, but if I access through java I only get 5 years data. thanks
      singulars
      1. POcannot download csv files from morningstar.com using authentication
      1. USuser1030532
    3. COCan you update your question with any relavent code, exceptions, configs, logs available (please wrap them as Code using the {} button). It would greatly improve your chances of getting a solid answer.
      singulars
      1. POcannot download csv files from morningstar.com using authentication
      1. USCodeChimp
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload