Databases
StackOverflow2013
Postsdbo
POWhat is the simplest way to parse an html file so that I can later enter it into a database?
Body
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POWhat is the simplest way to parse an html file so that I can later enter it into a database?
    text
    copied!<p>I have a ~2 MB <code>html</code> file that I need to parse that contains about 500 rows and about 70 columns. I need to clean it up obviously in a way that I can later enter it into a SQL Server database. I have parsed files in the past with Perl but never an <code>html</code> file and I was wondering if there are any modules I should check out before I do plain matching and formatting.</p> <p>A little update:</p> <pre><code>&lt;td class="tableHeaderDarkCenter"&gt;CUSIP/ISIN&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Stock Ticker&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;MLCC Code&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Bond Ticker&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Issuer Name&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Convertible Price(USD)&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Par Amount&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Coupon&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Maturity/Mandatory Conversion Date&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Outstanding Amt ($MM)&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Bonds/Shrs Outstanding&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Market Value($MM)&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Index Weight(%)&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;YTM(%)&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;YTP(%)&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Greater of YTM/YTP(%)&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Duration&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Currency&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Country&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Series&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Accrued Interest&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Current Yield(%)&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Yield Advantage(%)&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Moody Rating&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;S&amp;P Rating&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Avg. Rating&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Internal Rating&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Collateral Type&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Status&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Security Type&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Announce Date&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Issue Date&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;At-Issue Yield&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;At-Issue Prem&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Delta&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Gamma&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;RHO&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Theoretical Value&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Theoretical Discount (%)&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Cheap (%)&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Conversion Ratio&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Parity Cash Adjustment&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Payback&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Implied Volatility(%)&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Implied Spread&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Parity Delta&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Conversion Premium(%)&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Investment Value Premium(%)&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Investment Value(Bond floor)&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Price to Par&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Next Put Date&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Yrs to Put&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Yrs to Mat/Mand Conv Date&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Yrs to Mat/Yrs to Put&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Years to Call&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;1-Day Total Return(%)&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;1-WK Total Return(%)&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;MTD Total Return(%)&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;QTD Total Return(%)&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;YTD Total Return(%)&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Index Sector&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Industry&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;CS Sector L1&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;CS Sector L2&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;CS Sector L3&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;CS Sector L4&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;CS Sector L5&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;ML Sector L1&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;ML Sector L2&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;ML Sector L3&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;ML Sector L4&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;GIC Sector&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;GIC Industry Group&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;GIC Industry&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;GIC Sub Industry&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Bloomberg Sector&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Stock Price(USD)&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Stock Yield&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;1-Day Equity Total Return(%)&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;1-WK Equity Total Return(%)&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;MTD Equity Total Return(%)&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;QTD Equity Total Return(%)&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;YTD Equity Total Return(%)&lt;/td&gt; &lt;td class="tableHeaderDarkCenter"&gt;Eq Mkt Value($MM)&lt;/td&gt; </code></pre> <p>This is what occurs in the file and from the end of this until the end of the file are the corresponding values that map to the column names. Obviously a lot. I'm giving the <code>HTML::TableExtract</code> a shot, but I'm not sure if it is right for the situation.</p>
 

Querying!

 
Guidance
An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload