Databases
StackOverflow2013
Postsdbo
POInserting Rows in MySQL in Go very slow?
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POInserting Rows in MySQL in Go very slow?
    primarykey
    data
    text
    <p>So I have been rewriting an old PHP system to Go looking for some performance gains but I'm not get any. And the problem seems to be in the Inserts i'm doing into Mysql.</p> <p>So where PHP does some processing of a CSV file, does some hashing and inserts around 10k rows in MySQL it takes 40 seconds (unoptimized code).</p> <p>Now Go on the other hand stripped away of any processing and just the same inserting of 10k(empty) rows takes 110 seconds.</p> <p>Both tests are run on the same machine and I'm using the go-mysql-driver.</p> <p>Now for some Go code:</p> <p>This is extremely dumbed down code and this still takes almost 2 minutes, compared to PHP which does it in less then half.</p> <pre><code>db := GetDbCon() defer db.Close() stmt, _ := db.Prepare("INSERT INTO ticket ( event_id, entry_id, column_headers, column_data, hash, salt ) VALUES ( ?, ?, ?, ?, ?, ? )") for i := 0; i &lt; 10000; i++{ //CreateTicket(columns, line, storedEvent) StoreTicket(models.Ticket{int64(0), storedEvent.Id, int64(i), "", "", "", "", int64(0), int64(0)}, *stmt) } //Extra functions func StoreTicket(ticket models.Ticket, stmt sql.Stmt){ stmt.Exec(ticket.EventId, ticket.EntryId, ticket.ColumnHeaders, ticket.ColumnData, ticket.Hash, ticket.Salt) } func GetDbCon() (sql.DB) { db, _ := sql.Open("mysql", "bla:bla@/bla") return *db } </code></pre> <p><a href="http://min.us/l6pL9DsLSvTdz" rel="nofollow">Profiler result</a></p> <p>So is it my code, the go-mysql-driver or is this normal and is PHP just really fast in inserting records?</p> <p>==EDIT==</p> <p>As per requested, I have recorded both PHP and Go runs with tcpdump: The files:</p> <ul> <li><a href="https://dl.dropboxusercontent.com/u/1920297/golang10000.tcpdump" rel="nofollow">Go Tcpdump</a></li> <li><a href="https://dl.dropboxusercontent.com/u/1920297/golangRESULT.txt" rel="nofollow">Go Textdump</a></li> <li><a href="https://dl.dropboxusercontent.com/u/1920297/php10000.tcpdump" rel="nofollow">PHP Tcpdump</a></li> <li><a href="https://dl.dropboxusercontent.com/u/1920297/phpRESULT.txt" rel="nofollow">PHP Textdump</a></li> </ul> <p>I have a hard time reaching any conclusions comparing the two logs, both seem to be sending the same size packets back and forth. But with Go(~110) mysql seems to almost take twice as long to process the request then with PHP(~44), also Go seems to wait slightly longer before sending a new request again(the difference is minimal though).</p>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PTQuestion
    1. USRohan
    1. USRohan
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. VO
      singulars
      1. POInserting Rows in MySQL in Go very slow?
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POInserting Rows in MySQL in Go very slow?
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. POInserting Rows in MySQL in Go very slow?
      1. This table or related slice is empty.
      1. VTUpMod
    1. COSame machine is fine, but is it the same database and table? have you defined any extra indexes? Just a wild guess.
      singulars
      1. POInserting Rows in MySQL in Go very slow?
      1. USLitmus
    2. COThey operate on different databases with minor differences in naming(structure is identical), the indexes are exactly the same though and the databases are truncated before each test.
      singulars
      1. POInserting Rows in MySQL in Go very slow?
      1. USRohan
    3. COWhat Go version and MySQL driver are you using?
      singulars
      1. POInserting Rows in MySQL in Go very slow?
      1. USthwd
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload