Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>In the past I have used <a href="http://pdftohtml.sourceforge.net/" rel="noreferrer">pdftohtml</a> which can be used to generate xml, described <a href="http://blog.scraperwiki.com/2012/12/28/scraping-the-royal-society-membership-list/" rel="noreferrer">here</a>. The columns are generally fairly well separated so you could use the positioning to extract columns. </p> <p>I wrote a large part of <a href="http://blog.scraperwiki.com/2013/07/29/pdftables-a-python-library-for-getting-tables-out-of-pdf-files/" rel="noreferrer">pdftables</a>, apologies for the opaqueness! It works OK for some pages of the document you show, for example page 2 gives me the output at the bottom this reply. For other pages it falls over, on page 33, for example. The problem here is that there are two numbers under one column heading and they get stuck together by pdftables. The "COMARCA, CODI i, NOM EMA" columns don't get separated in either case. You can submit issues for pdftables on <a href="https://github.com/scraperwiki/pdftables" rel="noreferrer">GitHub</a>, I'm not working on it actively at the moment. It is available by pip install.</p> <p>If you wanted to go the commercial route then <a href="http://www.abbyy.com/" rel="noreferrer">Abbyy</a> FineReader is very good, they produce a <a href="http://ocrsdk.com/" rel="noreferrer">cloud SDK</a> which will give you 30 or so pages free. They have example code in multiple languages but their support isn't great.</p> <pre><code> 14 columns, 39 rows 0 1 2 3 4 5 6 7 8 9 10 11 12 13 ----------------------------------------------------------------------------------------------------- 0 | COMARCACODI i NOM EMA| GEN| FEB| MAR| ABR| MAI| JUN| JUL| AGO| SET| OCT| NOV| DES| ANY| 1 | VYNullesAlt Camp| 7,5| 5,5|10,9|12,3|16,7|21,6|22,3|24,4|20,1|15,9|11,0| 8,5|14,8| 2 | DQVila-rodonaAlt Camp| 7,9| 5,6|11,0|12,0|16,6|21,6|22,0|24,3|19,9|15,8|11,0| 8,6|14,7| 3 | Alt EmpordàU1Cabanes| 8,2| 6,5|11,7|12,6|17,5|22,0|23,1|24,4|20,4|16,6|11,8| 8,3|15,3| 4 | Alt EmpordàW1Castelló d'Empúries| 8,1| 6,4|11,6|12,9|17,0|21,1|22,0|23,4|20,1|16,4|12,1| 8,5|15,0| 5 | Alt EmpordàVZEspolla| 9,0| 6,7|12,4|12,7|17,8|22,0|23,3|24,8|20,9|16,7|12,0| 8,9|15,6| 6 | D6PortbouAlt Empordà| 9,6| 5,5|12,7|12,5|17,4|21,5|22,9|24,4|19,8|17,0|12,3|10,1|15,5| 7 | D4RosesAlt Empordà| 9,3| 7,2|13,0|13,6|18,2|22,6|23,9|25,7|21,3|17,5|13,2| 9,9|16,3| 8 | Alt EmpordàU2Sant Pere Pescador| 7,8| 6,3|11,5|12,9|16,8|21,2|22,2|23,6|20,2|16,5|12,3| 8,5|15,0| 9 | Alt EmpordàW2Torroella de Fluvià| 7,4| 6,0|11,2|12,6|16,4|21,2|22,3|23,7|19,9|16,1|11,7| 8,0|14,7| 10 | Alt EmpordàW3Ventalló| 7,3| 6,2|11,4|12,8|16,9|21,8|22,8|24,3|20,4|16,5|12,0| 8,1|15,1| 11 | Alt PenedèsWPCanaletes| 7,0| 5,2|11,3|11,9|16,7|21,5|22,0|24,2|19,7|15,6|10,7| 8,1|14,5| 12 | Alt PenedèsDIFont-rubí| 8,1| 6,2|12,0|11,9|16,9|21,8|22,0|24,4|20,0|15,9|11,4| 8,9|15,0| 13 | Alt PenedèsW4la Granada| 7,0| 5,5|11,2|12,6|17,2|21,9|22,4|24,3|20,0|16,0|11,1| 8,3|14,8| 14 | Alt PenedèsU3Sant Martí Sarroca| 6,4| 5,1|10,9|12,4|17,0|21,8|22,3|24,3|19,9|15,7|10,8| 8,0|14,6| 15 | Alt PenedèsWYSant Sadurní d'Anoia| 6,4| 5,1|11,0|12,8|17,6|22,6|23,2|25,0|20,5|16,2|10,9| 7,8|15,0| 16 | CDla Seu d'UrgellAlt Urgell| 3,6| 2,5| 8,5| 8,4|14,6|20,3|21,0|23,4|16,9|12,2| 7,0| 3,2|11,8| 17 | W5OlianaAlt Urgell| 2,0| 2,7| 9,8|10,2|16,8|23,0|22,9|25,6|19,1|13,9| 8,6| 3,1|13,2| 18 | Alt UrgellCJOrganyà| 2,6| 3,5| 9,8| 9,9|16,1|22,0|22,6|25,3|18,8|13,5| 8,2| 2,9|13,0| 19 | Alta RibagorçaZ2Boí (2.535 m)|-2,4|-7,5|-1,3|-3,4| 3,8| 8,6| 9,4|12,0| 6,3| 2,7|-1,1|-3,2| 2,0| 20 | Alta RibagorçaCTel Pont de Suert| 0,5| 1,6| 6,9| 7,9|14,1|18,0|19,1|20,4|15,7|10,7| 6,1| 1,3|10,2| 21 | CEels Hostalets de PierolaAnoia| 7,3| 5,5|11,7|12,1|17,4|22,4|22,9|25,2|20,3|16,2|11,1| 8,3|15,1| 22 | XBla LlacunaAnoia| 5,4| 3,3| 9,3|10,3|15,6|20,8|20,9|23,3|18,0|14,1| 9,1| 6,9|13,1| 23 | AnoiaXAla Panadella| 3,6| 1,7| 9,2| 8,7|14,9|20,5|20,4|23,2|17,2|13,3| 7,9| 5,1|12,2| 24 | H1ÒdenaAnoia| 5,1| 3,3| 9,4|11,5|16,3|21,7|22,5|24,6|19,4|15,2| 9,3| 6,0|13,7| 25 | WWArtésBages| 3,5| 2,8| 9,2|11,2|16,6|22,4|23,2|25,1|19,3|15,0| 9,1| 4,3|13,5| 26 | U4Castellnou de BagesBages| 4,8| 3,8|10,5|10,9|16,3|22,0|22,5|25,0|19,3|15,0| 9,6| 5,9|13,9| 27 | R1el Pont de VilomaraBages| 3,8| 3,1| 9,9|12,3|17,4|22,9|23,5|25,4|20,0|15,7| 9,7| 5,0|14,1| 28 | BagesWNMontserrat - Sant Dimes| 6,2| 3,3| 9,7| 8,6|14,8|19,5|19,5|22,4|16,9|13,5| 9,0| 7,1|12,6| 29 | CLSant Salvador de GuardiolaBages| 3,3| 2,8| 9,1|11,5|16,4|22,0|22,4|24,6|19,2|14,9| 9,1| 4,8|13,4| 30 | U5Prades - los HortalsBaix Camp| 2,8| 0,0| 6,4| 7,4|13,0|18,4|18,0|21,3|15,0|11,3| 6,5| 4,1|10,4| 31 | W6RiudomsBaix Camp| 9,7| 7,1|12,0|13,4|17,6|22,4|23,1|25,2|21,2|17,1|12,3|10,1|16,0| 32 | U6Vinyols i els ArcsBaix Camp|10,2| 7,6|12,0|13,8|17,6|22,5|24,0|25,9|22,3|18,2|13,2|11,1|16,6| 33 | Baix EbreU7Aldover|10,0| 8,5|13,2|14,8|19,7|24,6|25,2|27,1|22,7|18,3|12,9|11,1|17,4| 34 | DBel PerellóBaix Ebre| 8,7| 7,0|12,0|13,3|17,9|22,6|23,3|25,3|21,4|17,2|11,9|10,3|15,9| 35 | U9l'AldeaBaix Ebre| 9,9| 8,1|12,5|14,3|18,5|23,3|24,1|26,0|22,1|17,9|13,1|10,7|16,8| 36 | UAl'Ametlla de MarBaix Ebre| 9,6| 7,8|12,3|13,8|18,0|22,9|23,9|25,8|22,0|17,6|12,5|10,6|16,4| 37 | Baix EbreX5PN dels Ports| 3,4|-0,2| 6,5| 6,8|13,4|18,7|17,8|21,2|15,2|11,3| 6,1| 4,9|10,5| 38 | Baix EmpordàDOCastell d'Aro| 6,7| 5,1|10,6|12,0|16,2|20,9|21,8|23,8|20,1|16,3|12,2| 8,1|14,5| ----------------------------------------------------------------------------------------------------- </code></pre> <p>The unicode problems are down to my dev environment (Spyder).</p>
    singulars
    1. This table or related slice is empty.
    1. POScraping large pdf tables which span across multiple pages
      singulars
      1. PTQuestion
    1. PTAnswer
    1. This table or related slice is empty.
    1. USIan Hopkinson
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    1. CODear Ian, thank you! and sorry for late reply. But 1) how would your tool work on the page 5, when there are missing values in the table (for comarca Solsonès)? 2) is your tool able to join the table which is split accross multiple pages into one, coercing the corresponding columns? These problems are actually the essential thing for me, I can survive the "COMARCA, CODI i, NOM EMA" columns being merged.
      singulars
      1. PO
      1. USTMS
    2. COI tried the Cloud SDK you recommend, I registered but I absolutely don't know where to go from here - how to upload pages, recognize them etc. Please look at my updated question.
      singulars
      1. PO
      1. USTMS
    3. COSorry for delay - I thought I was getting notifications, and I'm not! The missing values are no problem, they will simply appear as empty cells in pdftables. Joining tables between pages would be a matter of programming (that's to say as currently configured it would extract a table for each page and how to join them would be a matter of further programming). It looks like there is the same number of columns on each page, and they just differ by their absolute location on the page - in which case joining up should be straightforward.
      singulars
      1. PO
      1. USIan Hopkinson
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload