Databases
StackOverflow2013
Postsdbo
POPython requests result missing part of web page
Body
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POPython requests result missing part of web page
    text
    copied!<p>I'm trying to use requests to submit a form and download the resulting web page. The problem is that the downloaded page is missing a large part of the page I see when I fill in the form on a browser.</p> <p>I'm trying to view a seat map on aa.com. I get almost everything other than the actual seat map.</p> <pre><code>from requests import session def get_page(): payload = { 'airportLookupRequired': 'true', 'currentCodeForm': 'originAirport', 'currentCalForm': 'viewSeatsForm', 'flightNumber': '1132', 'departureMonth': '9', 'departureDay': '22', 'originAirport': 'dfw', 'destinationAirport': 'lga', 'cabin': 'coach', '_button_submit': 'GO' } with session() as c: c.post('https://www.aa.com/seatmap/viewSeatsAccess.do', payload) r = c.get('https://www.aa.com/seatmap/viewSeatsSubmit.do') print r.content.replace('/content/images', 'https://www.aa.com/content/images') get_page() </code></pre> <p>Any suggestions for fixing this or figuring out why it's not working would be appreciated.</p> <p>EDIT: using browser tools to look for AJAX requests, I see</p> <pre><code>Resource interpreted as Script but transferred with MIME type text/plain: "https://www.aa.com/dwr/interface/SeatMapAjaxUtils.js". viewSeatsSubmit.do:52 </code></pre> <p>Clicking on that leads to:</p> <pre><code>// Provide a default path to dwr.engine if (dwr == null) var dwr = {}; if (dwr.engine == null) dwr.engine = {}; if (DWREngine == null) var DWREngine = dwr.engine; if (SeatMapAjaxUtils == null) var SeatMapAjaxUtils = {}; SeatMapAjaxUtils._path = '/dwr'; SeatMapAjaxUtils.formatAmount = function(p0, p1, callback) { dwr.engine._execute(SeatMapAjaxUtils._path, 'SeatMapAjaxUtils', 'formatAmount', p0, p1, callback); } SeatMapAjaxUtils.getExitRowSeatSelectionResponse = function(p0, callback) { dwr.engine._execute(SeatMapAjaxUtils._path, 'SeatMapAjaxUtils', 'getExitRowSeatSelectionResponse', p0, callback); } </code></pre> <p>which appears to be the code to generate the information I want. I don't know how to proceed from there.</p>
 

Querying!

 
Guidance
An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload