StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
text
Body
copied!<p>You should use <a href="http://search.cpan.org/~dlux/Parallel-ForkManager-0.7.5/ForkManager.pm" rel="nofollow">Parallel::ForkManager</a> off of CPAN. This (with a little <a href="https://metacpan.org/pod/Parallel%3a%3aForkManager" rel="nofollow">included explanation</a>) should allow you to fork each process and parse the files individually, in parallel. Also, be aware that Perl 5 has <a href="http://perldoc.perl.org/threads.html" rel="nofollow">threads</a>, but the performance gain will probably not be significant. </p> <p>The provided code on the linked page should do what you want, but I've posted it here for your convenience. As you can see, all it really does is create a new data structure for the management of the maximum number of allowed processes and for each new data piece (or file) forks and return the child, does the work, then terminates the process:</p> <pre><code>use Parallel::ForkManager; $pm = Parallel::ForkManager->new($MAX_PROCESSES); foreach $data (@all_data) { # Forks and returns the pid for the child: my $pid = $pm->start and next; ... do some work with $data in the child process ... $pm->finish; # Terminates the child process } </code></pre> <p>Be aware that you may want to use WINAPI on Windows if you want to create processes there, (as Parallel::ForkManager I believe uses Windows kernel level threading, though should still perform the task adequately). Perl also gives you the option of using <a href="http://search.cpan.org/~acalpini/Win32-API-0.41/API.pm" rel="nofollow">Win32::API</a>'s CreateProcess() function to do multiprocessing in Perl (provided you import it). There's also the option of the <a href="http://search.cpan.org/~mob/Forks-Super-0.68/lib/Forks/Super.pm" rel="nofollow">Forks::Super</a> package for multiprocessing, which works on Windows as well.</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload