StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POSlow PHP script which formats, sorts, and removes duplicates from list
text
Body
copied!<p>I have a PHP script that is called by a jQuery .post which will submit a list of email addresses, perform some operations on the list (remove duplicates, sort, format, etc.), and then store each into a MySQL database.</p> <p>The problem that I'm having is that it takes a really long time to perform this operation on a large list. I am testing with a list of 15,000 e-mail addresses and in 300 seconds (5 minutes), it only adds around 5,000 of the addresses.</p> <p>Is there something about my code that would take a long time to process? Here it is. I know that I'm doing a lot of formatting, but that is only because some of the e-mail addresses contain weird characters, etc.</p> <pre><code>// form posts $addresses = $_POST['email_addresses']; // cleanse and format $addresses = trim($addresses); $addresses = trim($addresses, "\xC2\xA0"); $addresses = str_replace(" ", "", $addresses); $addresses = preg_replace("/(^[\r\n]*|[\r\n]+)[\s\t]*[\r\n]+/", "\n", $addresses); $addresses = str_replace("\n", ",", $addresses); $addresses = preg_replace('/[^(\x20-\x7F)]*/','', $addresses); $addresses = strtolower($addresses); $array_addresses = explode(",", $addresses); // get unique values $unique_addresses = array(); foreach($array_addresses as $key => $value) { if(filter_var($value, FILTER_VALIDATE_EMAIL)){ $unique_addresses[$value] = $value; } } sort($unique_addresses); foreach($unique_addresses as $arr) { if ($insert_addresses_stmt = $mysqli->prepare("INSERT INTO email_addresses (lid, email_addresses) VALUES (?, ?)")) { $insert_addresses_stmt->bind_param("ss", $new_lid, $arr); $insert_addresses_stmt->execute(); $insert_addresses_stmt->close(); } } </code></pre>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload