Databases
StackOverflow2013
Postsdbo
POScraping Table with Nokogiri and need JSON output
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POScraping Table with Nokogiri and need JSON output
    primarykey
    data
    text
    <p>So, I have a table with multiple rows and columns.</p> <pre><code>&lt;table&gt; &lt;tr&gt; &lt;th&gt;Employee Name&lt;/th&gt; &lt;th&gt;Reg Hours&lt;/th&gt; &lt;th&gt;OT Hours&lt;/th&gt; &lt;/tr&gt; &lt;tr&gt; &lt;td&gt;Employee 1&lt;/td&gt; &lt;td&gt;10&lt;/td&gt; &lt;td&gt;20&lt;/td&gt; &lt;/tr&gt; &lt;tr&gt; &lt;td&gt;Employee 2&lt;/td&gt; &lt;td&gt;5&lt;/td&gt; &lt;td&gt;10&lt;/td&gt; &lt;/tr&gt; &lt;/table&gt; </code></pre> <p>There is also another table:</p> <pre><code>&lt;table&gt; &lt;tr&gt; &lt;th&gt;Employee Name&lt;/th&gt; &lt;th&gt;Revenue&lt;/th&gt; &lt;/tr&gt; &lt;td&gt;Employee 2&lt;/td&gt; &lt;td&gt;$10&lt;/td&gt; &lt;/tr&gt; &lt;tr&gt; &lt;td&gt;Employee 1&lt;/td&gt; &lt;td&gt;$50&lt;/td&gt; &lt;/tr&gt; &lt;/table&gt; </code></pre> <p>Notice that the employee order may be random between the tables.</p> <p>How can I use nokogiri to create a json file that has each employee as an object, with their total hours and revenue?</p> <p>Currently, I'm able to just get the individual table cells with some xpath. For example:</p> <pre><code>puts page.xpath(".//*[@id='UC255_tblSummary']/tbody/tr[2]/td[1]/text()").inner_text </code></pre> <p><strong>Edit:</strong></p> <p>Using the page-object gem and the link from @Dave_McNulla, I tried this piece of code just to see what I get:</p> <pre><code>class MyPage include PageObject table(:report, :id =&gt; 'UC255_tblSummary') def get_some_information report_element[1][2].text end end puts get_some_information </code></pre> <p>Nothing's being returned, however.</p> <p>Data: <a href="https://gist.github.com/anonymous/d8cc0524160d7d03d37b" rel="nofollow">https://gist.github.com/anonymous/d8cc0524160d7d03d37b</a></p> <p>There's a duplicate of the hours table. The first one is fine. The other table needed is the accessory revenue table. (I'll also need the activations table, but I'll try to merge that from the code that merges the hours and accessory revenue tables.</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USghal
    1. USghal
    plurals
    1. PL
      singulars
      1. LTLinked
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. COCan you modify the HTML to include classes for the table elements? Specifically, the <tr> tags could have an "employee" class and each <td> tag could have a class for what it is (e.g. "name", "revenue", etc.) This would help you match an employee name, then find it in the other HTML document, then either build JSON from the two, or merge them together before building your object.
      singulars
      1. POScraping Table with Nokogiri and need JSON output
      1. USCDub
    2. COThe HTML is not mine. Getting the xpath node is not the issue...I guess I'm more stuck on the Ruby part. I'm not sure how to iterate thought the rows and to merge the data between the two tables.
      singulars
      1. POScraping Table with Nokogiri and need JSON output
      1. USghal
    3. COIs there a reason you want to use Nokogiri over using Watir?
      singulars
      1. POScraping Table with Nokogiri and need JSON output
      1. USJustin Ko
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload