Databases
StackOverflow2013
Postsdbo
POExtract data from a txt file that contains data blocks and strings using python
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POExtract data from a txt file that contains data blocks and strings using python
    primarykey
    data
    text
    <p>I need to postprocess an output file from a model using python. The output file has a combination of data and strings. First, I want to separate the strings from data and then save the columns 0,1 and 2 from each output time (only data, with no strings) in a separate text file. So, for the below example, I will have 3 text files (for Time=0, Time=0.01, Time=0.04) each containing data from each output time without any header or any other strings in them. The short form of the output file from model looks like this: </p> <pre><code> ******* Program ****** ******* Program ****** ******* Program ****** Date: 26. 4. Time: 15:40:32 Units: L = cm , T = days , M = mmol Time: 0.000000 Node Depth Head Moisture HeadF MoistureF Flux [L] [L] [-] [L] [-] [L/T] 1 0.00 -1000.00 0.1088 -1000.00 0.002508 -0.562E-03 2 -0.04 -1000.00 0.1088 -1000.00 0.002508 -0.562E-03 3 -0.08 -1000.00 0.1088 -1000.00 0.002508 -0.562E-03 end Time: 0.010000 Node Depth Head Moisture HeadF MoistureF Flux [L] [L] [-] [L] [-] [L/T] 1 0.00 -666.06 0.1304 -14.95 0.139033 -0.451E-02 2 -0.04 -666.11 0.1304 -15.01 0.138715 -0.887E-02 3 -0.08 -666.35 0.1304 -15.06 0.138394 -0.174E-01 end Time: 0.040000 Node Depth Head Moisture HeadF MoistureF Flux [L] [L] [-] [L] [-] [L/T] 1 0.00 -324.87 0.1720 -12.30 0.157799 -0.315E-02 2 -0.04 -324.84 0.1720 -12.31 0.157724 -0.628E-02 3 -0.08 -324.83 0.1720 -12.32 0.157649 -0.125E-01 end </code></pre> <p>I found the following code from another question which was posted in stackoverflow earlier. Here is the link to that question: <a href="https://stackoverflow.com/questions/11712496/python-how-to-extract-specific-lines-of-data-from-a-big-text-file">enter link description here</a></p> <p>That problem is very similar to mine; however, I have problems modifying it to help solve my problem. How should I modify it for my problem? or should I use another strategy to approach this problem?</p> <pre><code>def parse_DPT(lines): DPT = [] while lines: line = lines.pop(0).lstrip() if line == ' ' or line.startswith('*'): continue if line.startswith('*'): lines.insert(0, line) break data = line.split(' ') # pick only columns 0, 1, 2 and # convert to appropiate numeric format # and append to list for current DPT and step DPT.append([int(data[0]), float(data[1]), float(data[2])]) return DPT raw = [] with open('NOD_INFTEST.txt') as nit: lines = nit.readlines() while lines: line = lines.pop(0) if line.startswith(''): if line.find('Time:') &gt; -1: raw.append(parse_DPT(lines)) from pprint import pprint for raw_step in zip(raw): print 'raw:' pprint(raw_step) </code></pre> <p>Here is the error message that I get from python:</p> <pre><code>'import sitecustomize' failed; use -v for traceback Traceback (most recent call last): File "C:\Users\Desktop\python test\p-test3.py", line 58, in &lt;module&gt; raw.append(parse_DPT(lines)) File "C:\Users\Desktop\python test\p-test3.py", line 35, in parse_DPT DPT.append([int(data[0]), float(data[1]), float(data[2])]) ValueError: invalid literal for int() with base 10: 'Units:' </code></pre>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USCommunity
    1. USMary Jane
    plurals
    1. PL
      singulars
      1. LTLinked
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. COWhat exactely are you trying to get as an end result? You say in the example below you have 3 files but I only see one. Where is the limit between them?
      singulars
      1. POExtract data from a txt file that contains data blocks and strings using python
      1. USIonut Hulub
    2. CO@IonutHulub Sorry if my explanation was confusing. By that sentence I meant that at the end, I want to have 3 separate text files that are created from the file that you can see above. Each section starts with the "Time: " and finishes with "end". I hope this helps with explanation.
      singulars
      1. POExtract data from a txt file that contains data blocks and strings using python
      1. USMary Jane
    3. COAnd I'm guessing you want file one to contain this:` 1 0.00 -1000.00 2 -0.04 -1000.00 3 -0.08 -1000.00` If not please provide the expected output for the example you gave.
      singulars
      1. POExtract data from a txt file that contains data blocks and strings using python
      1. USIonut Hulub
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload