Databases
StackOverflow2013
Postsdbo
POpython: read csv, execute command and write results to new vertical column
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POpython: read csv, execute command and write results to new vertical column
    primarykey
    data
    text
    <p>I am totally new to python, I read python's csv module is great for what Id like to do. Ive spent some time trying several different methods but have not yet been able to even create an array using the fourth (vertical) column.</p> <p>I have a four column csv file with hundreds of rows. Before I go on I should probably verify python can even accomplish all that Id like to do.</p> <ol> <li>read a csv FILE, </li> <li><p>executes COMMAND on fourth (vertical) column of FILE</p></li> <li><p>the COMMAND prints</p></li> <li><p>read each line for HEALTHY (from COMMAND)</p></li> <li><p>write HEALTHY on new fifth column to NEW_FILE with all five columns</p></li> <li>loop until first empty row of FILE</li> </ol> <p>example FILE (comma delimited in cell view)</p> <pre><code> HOST PLATFORM ARCH COMMAND server1 win x86_64 python '/root/server1.py' server2 linux x86_64 python '/root/server2.py' server3 linux x86_64 python '/root/server3.py' </code></pre> <p>example COMMAND</p> <pre><code> # python '/root/server1.py' -------------------- Error: Could not open /root/server1.py # python '/root/server2.py' -------------------- server2 p1 (NTFS) output1:100 output:200 HEALTHY:Yes -------------------- # python 'root/server3.py' -------------------- server3 p1 (linux) output1:100 output:200 HEALTHY:No server3 p2 (linux) output1:100 output:200 HEALTHY:Yes server3 p3 (swap) output1:100 output:200 HEALTHY:No -------------------- </code></pre> <p>if multiple lines of HEALTHY and all do not equal Yes, HEALTHY equals "No"</p> <p>if HEALTHY is not found on any lines, HEALTHY equals "Error Scanning"</p> <p>This is what I have so far</p> <pre><code> #!/usr/bin/python # import csv import subprocess # read csv file csv_file = open("my_list.csv", "rb") my_csv_reader = csv.reader(csv_file, delimiter=",") my_data_list = [] for row in my_csv_reader: print row my_data_list.append(row) csv_file.close() # write csv file csv_file = open("new_data.csv", "wb") my_csv_writer = csv.writer(csv_file, delimiter=",") for row in my_data_list: my_csv_writer.writerow(row) csv_file.close() # running commands, getting output # run COMMAND column from csv_file, use "python 'my_script.py'" for now # my_script.py only for now: print "HEALTHY:Yes" p = subprocess.Popen("python '/root/my_script.py'",stdout=subprocess.PIPE,stderr=subprocess.PIPE) output, errors = p.communicate() print output print errors </code></pre> <p>Executing the above:</p> <pre><code> # python '/root/this_script.py' ['HOST', 'PLATFORM', 'ARCH', 'COMMAND'] ['server1', 'win', 'x86_64', "python '/root/server1.py'"] ['server2', 'linux', 'x86_64', "python '/root/server2.py'"] ['server3', 'linux', 'x86_64', "python '/root/server3.py'"] Traceback (most recent call last): File "thisscript.py", line 24, in ? p = subprocess.Popen('python myscript1.py',stdout=subprocess.PIPE,stderr=subprocess.PIPE) File "/usr/lib64/python2.4/subprocess.py", line 550, in __init__ errread, errwrite) File "/usr/lib64/python2.4/subprocess.py", line 993, in _execute_child raise child_exception OSError: [Errno 2] No such file or directory </code></pre> <p><strong>Bonus:</strong><br> If I wanted to search the stdout/command output for something <strong>also</strong> (such as linux, swap, NTFS, etc, --third example command in question above) and append it to row[5], or next after it has already searched for [i]Healthy[/i]... Ive tried starting a new if statement but it appears to only append row[4], or the same row as when it does for [i]Healthy[/i].</p> <p>I also cant figure out how to to use an OR statement. Where </p> <pre><code> if 'Linux' OR 'swap' OR 'LVM' in stdout: writer.writerow(row + ['Linux']) # for multiple lines/partitions. elif 'BSD' in stdout: writer.writerow(row + ['BSD']) elif 'NTFS' in stdout: writer.writerow(row + ['Windows']) else: writer.writerow(row + ['Error Scanning']) </code></pre> <p>Last I have changed the COMMAND column to the PATH and modified the command to execute the PATH. Which is working. I'd like to execute a second command to fetch the filesize of PATH. Ive tried a couple methods.</p> <p>Thank you for your time. I hope this can all be done.</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USTommy
    1. USTommy
    plurals
    1. PL
      singulars
      1. LTLinked
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POpython: read csv, execute command and write results to new vertical column
      1. USjon
      1. VTFavorite
    2. VO
      singulars
      1. POpython: read csv, execute command and write results to new vertical column
      1. This table or related slice is empty.
      1. VTApproveEditSuggestion
    3. VO
      singulars
      1. POpython: read csv, execute command and write results to new vertical column
      1. This table or related slice is empty.
      1. VTUpMod
    1. COWhat happens when you try that code? Also, if that's your real input file, it's not comma delimited, because it doesn't have commas.
      singulars
      1. POpython: read csv, execute command and write results to new vertical column
      1. USThomas K
    2. CO@ThomasK I think he just prettified the CSV for us.
      singulars
      1. POpython: read csv, execute command and write results to new vertical column
      1. USKurzedMetal
    3. CORight thats an example cell view of a comma delimited csv.
      singulars
      1. POpython: read csv, execute command and write results to new vertical column
      1. USTommy
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload