Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>You cannot read from a file opened in append <code>&gt;&gt;</code> mode. It seems you've just renamed the <code>OUT</code> filehandle to <code>FP</code>, but that isn't enough.</p> <ul> <li>The original code included a <code>chdir $seculert_dir</code> – this makes sense because now we don't have to prefix our filenames with the directory all the time (it's the equivalent to the <code>cd</code> command in the shell).</li> <li>The original code didn't exactly use best practices. I wrapped the <code>scp</code> and <code>ssh</code> commands with versions that do a minimal amount of error checking.</li> <li>It is also unnecessary to invoke external tools for things we can do with Perl.</li> </ul> <p>Here is my updated code, that can read the data from simplistic CSV files:</p> <hr> <p>First, we start the preample. <code>use warnings</code> is more preferable than the <code>-w</code> switch.</p> <pre><code>#!/usr/bin/perl use strict; use warnings; my $seculert_dir = "!FIXME!"; chdir $seculert_dir; # go into that dir, so that we don't have to use prefixes my @csv_files = ("seculert.csv"); # this is a csv which will contains IP addresses # of one specific category for e.g malware. # We only use the 2nd column #ssh connection information my $qradar_console = '10.10.1.22'; my $qradar_ssh_key = "qr-id_dsa"; my $qradar_ssh_knownhosts = "known_hosts"; </code></pre> <p>That was our configuration. Next we fetch the conf from the server using the wrapped SCP command:</p> <pre><code># fetch the remotenet.conf from QRadar print STDERR "Fetching configuration from QRadar...\n"; scp("root\@$qradar_console:/store/configservices/staging/globalconfig/remotenet.conf" =&gt; '.'); </code></pre> <p>We now open a file where we put our changed configuration</p> <pre><code># write the changed conf here before uploading: open my $new_conf, "&gt;", "remotenet.conf.changed" or die qq(Can't open "remotenet.conf.changed" for writing: $!); </code></pre> <p>Now we open the old configuration, copy it over, but skip lines starting with <code>SECULERT</code>.</p> <pre><code># copy old conf over, delete lines starting with "SECULERT" open my $old_conf, "&lt;", "remotenet.conf" or die qq(Can't open "remotenet.conf" for reading: $!); while (&lt;$old_conf&gt;) { print {$new_conf} $_ unless /^SECULERT/; } close $old_conf; </code></pre> <p>Notice how I've used “lexical filoehandles” (<code>open my $fh, ...</code>). This avoids some problems, and is more modern than using barewords.</p> <p>Next, we loop through the CSV files. We open each one, then extract the 2nd column and print it with the other stuff into the changed configuration file.</p> <pre><code># append the data from the CSVs for my $csv_file (@csv_files) { my $source = 'BAD-IP-Addresses-LABEL'; my $type_description = 'honeypots-for-examnple'; open my $csv, "&lt;", $csv_file or die qq(Can't open "$csv_file" for reading: $!); while (my $line = &lt;$csv&gt;) { my (undef, $ip) = split /,/, $line; # we're only interested in the 2nd column # Based upon the format described below I want to render the csv # as written in print OUT statement. This format is important, because the # endsystem process the file (remotenet.conf) based upon the provided layout. # # Columns in the output: # 1 - Name # 2 - Sub-Name # 3 - IP Address # 4 - is colour, deprecated # 5 - database length, deprecated # 6 - asset weight, deprecated # 7 - an ID for the 'record' each unique name pair (first 2 columns) gets an ID print {$new_conf} "$source $type_description $ip #FF0000 0 90 29\n"; } } </code></pre> <p>Now we have all the information we want in the new configuration file and can upload it to the server:</p> <pre><code>close $new_conf; # copy the changed remotenet.conf back to QRadar scp('remotenet.conf.changed' =&gt; "root\@$qradar_console:/store/configservices/staging/globalconfig/remotenet.conf"); # Remove our SECULERT list and the newly pushed out qradar conf print STDERR "Cleaning up...\n"; unlink $_ or warn qq(Can't remove "$_": $!) for 'remotenet.conf', 'remotenet.conf.changed'; </code></pre> <p>Next, we run the deploy script:</p> <pre><code># QRadar magic -- run deploy script print STDERR "Deploying in QRadar...(takes time to complete)\n"; ssh("root\@$qradar_console", '/opt/qradar/upgrade/util/setup/upgrades/do_deploy.pl'); print STDERR "Complete!\n\n"; </code></pre> <p>Here are the wrappers for <code>scp</code> and <code>ssh</code>. They are <em>subroutines</em> (functions, procedures, or methods in other languages). The arguments are in the <code>@_</code> array from which we unpack them into variables with better names. The <code>system</code> command takes a name of a command and a list of arguments. Because this circumvents the shell, we don't have to think about shell escapes. <code>system</code> returns zero on sucess, so we use that to check for errors.</p> <pre><code># Wrappers for SSH and SCP commands sub scp { my ($from, $to) = @_; my $success = 0 == system 'scp', '-i' =&gt; $qradar_ssh_key, '-o' =&gt; "UserKnownHostsFile=$qradar_ssh_knownhosts", '-o' =&gt; "StrictHostKeyChecking=no", $from =&gt; $to; return $success if defined wantarray; # return failure when somebody checks for it die qq(Can't scp "$from" to "$to") if not $success; # die when failure, and nobody checks. } sub ssh { my ($host, $command) = @_; my $success = 0 == system 'ssh', '-i' =&gt; $qradar_ssh_key, '-o' =&gt; "UserKnownHostsFile=$qradar_ssh_knownhosts", '-o' =&gt; "StrictHostKeyChecking=no", $host, $command; return $success if defined wantarray; # return failure when somebody checks for it die qq(Can't ssh into "$host" for '$command') if not $success; # die when failure, and nobody checks. } </code></pre> <hr> <p>This code could still be improved, e.g. by using a proper CSV parser like <code>Text::CSV</code>, using better SSH bindings than simply wrapping command line programs, automated error checking with <code>autodie</code>, and a better handling of tempfiles.</p> <p>It seems you want different <code>$source</code> values for different files. For this, we should use a more complex data structure than <code>@csv_files</code> – I'd use a hash of arrays, e.g.</p> <pre><code>my %csv_files = ( 'malware.csv' =&gt; ['BAD-IP-Addresses-LABEL', 'honeypots-for-examnple'], 'bot.csv' =&gt; ['bot-net', 'top-10'], ); </code></pre> <p>This is a dictionary that maps keys (here filenames) to values (here the contents of two columns). Instead of looping over entries in an array, we would now loop over the keys in this hash:</p> <pre><code>for my $csv_file (keys %csv_files) { my ($source, $type_description) = @{ $csv_files{$csv_file} }; ... } </code></pre> <p>The expression <code>$csv_files{$csv_file}</code> accesses the entry called <code>$csv_file</code> in the hash <code>$csv_files</code>. This entry contains an array reference as value. The <code>@{…}</code> around that converts the array reference to an array, which we can unpack with a list assignment <code>my ($foo, $bar) = @array</code>.</p>
    singulars
    1. This table or related slice is empty.
    1. POMultiple csv file rendering by Perl using single file open handle?
      singulars
      1. PTQuestion
    1. PTAnswer
    1. USamon
    1. USamon
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. POMultiple csv file rendering by Perl using single file open handle?
      singulars
      1. PTQuestion
    1. This table or related slice is empty.
    1. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTAcceptedByOriginator
    3. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    1. COThanks amon I have revised the code with comments and new updates. Please have a look. I'm new to perl, I tried my best to answer your questions.
      singulars
      1. PO
      1. USuser3066819
    2. COI have no words to pay you back on the level of work you done with code. You are a magician ; perl magician:). I can't wait for monday to come as I want to run your code on production server I will update you about the results. With that one code, you have educated me lot about perl; thanks for everything amon!
      singulars
      1. PO
      1. USuser3066819
    3. COThis is to update ; infact pass my gratitude that i tested your script in production server; and it works like a charm:). The only minor adjustment there was related to `print {$new_conf} "$source $type_description $ip #FF0000 0 90 29\n";` what it would do that it would put a line break after ip address and rest of text upto 29 would come into separate line followed with another line break; I think csv considers last csv value as end of line. This causes the end-system to not render the remotnet.conf.To workaround i defined the format in csv file instead.
      singulars
      1. PO
      1. USuser3066819
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload