Databases
StackOverflow2013
Postsdbo
POUnix sort command takes much longer depending on where it is executed?! (fastest from ProcessBuilder in program run from IDE, slowest from terminal)
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POUnix sort command takes much longer depending on where it is executed?! (fastest from ProcessBuilder in program run from IDE, slowest from terminal)
    primarykey
    data
    text
    <p>I have a java program that uses ProcessBuilder to call the unix sort command. When I run this code within my IDE (intelliJ) it only takes about a second to sort 500,000 lines. When I package it into an executable jar, and run that from the terminal it takes about 10 seconds. When I run the sort command myself from the terminal, it takes 20 seconds!</p> <p>Why the vast difference in performance and any way I can get the jar to execute with the same performance? Environment is OSX 10.6.8 and java 1.6.0_26. The bottom of the sort man page says "sort 5.93 November 2004"</p> <p>The command it is executing is:</p> <pre><code>sort -t' ' -k5,5f -k4,4f -k1,1n /path/to/imput/file -o /path/to/output/file </code></pre> <p>Note that when I run sort from the terminal I need to manually escape the tab delimiter and use the argument <code>-t$'\t'</code> instead of the actual tab (which I can pass to ProcessBuilder).</p> <p>Looking as <code>ps</code> everything seems the same except when run from IDE the sort command has a TTY of ?? instead of ttys000--but from <a href="https://stackoverflow.com/questions/7113770/what-does-tty-mean-in-the-unix-ps-command">this question</a> I don't think that should make a difference. Perhaps BASH is slowing me down? I am running out of ideas and want to close this 20x performance gap!</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USCommunity
    1. USZugwalt
    plurals
    1. PL
      singulars
      1. LTLinked
    2. PL
      singulars
      1. LTLinked
    1. PL
      singulars
      1. LTLinked
    2. PL
      singulars
      1. LTLinked
    3. PL
      singulars
      1. LTLinked
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POUnix sort command takes much longer depending on where it is executed?! (fastest from ProcessBuilder in program run from IDE, slowest from terminal)
      1. USChris Dennett
      1. VTFavorite
    2. VO
      singulars
      1. POUnix sort command takes much longer depending on where it is executed?! (fastest from ProcessBuilder in program run from IDE, slowest from terminal)
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. POUnix sort command takes much longer depending on where it is executed?! (fastest from ProcessBuilder in program run from IDE, slowest from terminal)
      1. USZugwalt
      1. VTBountyStart
    1. COwow .. I think I saw someone else ask this same question yesterday. http://stackoverflow.com/questions/7111127/why-is-my-application-running-faster-in-intellij-compared-to-command-line
      singulars
      1. POUnix sort command takes much longer depending on where it is executed?! (fastest from ProcessBuilder in program run from IDE, slowest from terminal)
      1. USKal
    2. CODo you know that you are running the same sort? Try an absolute path to the executable to be sure. If you have brew/macports/fink installed, it is possible that the sort from those packages are being run when it's slower.
      singulars
      1. POUnix sort command takes much longer depending on where it is executed?! (fastest from ProcessBuilder in program run from IDE, slowest from terminal)
      1. USergosys
    3. CO@Zugwalt : how 'wide' is each record, or put another way, how big is the overall file you are sorting. 500,000 of a normal record in 1 second sounds right for the the Unix systems I'm used to working on. 20 seconds seems insane. The sort will build its temp files in /tmp or /var/tmp dir (unless you are overriding it with `-D`). Maybe you can pickup a clue there by watching the processing. Otherwise, I'm thinking problems with disks, is your IDE writing tmpfiles to a different place than the standard /tmp/ or /var/tmp? Good luck.
      singulars
      1. POUnix sort command takes much longer depending on where it is executed?! (fastest from ProcessBuilder in program run from IDE, slowest from terminal)
      1. USshellter
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload