StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
text
Body
copied!<p>Two things come to mind:</p> <ol> <li>Use string views instead of strings as the split result, saves a lot of allocations.</li> <li>If you know you're only going to be working with chars (in the [0,255] range), try using a bitset to test membership instead of <code>find</code>ing into the delimiter characters.</li> </ol> <p>Here's a quick attempt at applying these ideas:</p> <pre><code>#include <vector> #include <bitset> #include <iostream> #include <boost/algorithm/string/split.hpp> #include <boost/algorithm/string/classification.hpp> #include <boost/timer.hpp> using namespace std; size_t const N = 10000000; template<typename C> void test_custom(string const& s, char const* d, C& ret) { C output; bitset<255> delims; while( *d ) { unsigned char code = *d++; delims[code] = true; } typedef string::const_iterator iter; iter beg; bool in_token = false; for( string::const_iterator it = s.begin(), end = s.end(); it != end; ++it ) { if( delims[*it] ) { if( in_token ) { output.push_back(typename C::value_type(beg, it)); in_token = false; } } else if( !in_token ) { beg = it; in_token = true; } } if( in_token ) output.push_back(typename C::value_type(beg, s.end())); output.swap(ret); } template<typename C> void test_strpbrk(string const& s, char const* delims, C& ret) { C output; char const* p = s.c_str(); char const* q = strpbrk(p+1, delims); for( ; q != NULL; q = strpbrk(p, delims) ) { output.push_back(typename C::value_type(p, q)); p = q + 1; } output.swap(ret); } template<typename C> void test_boost(string const& s, char const* delims) { C output; boost::split(output, s, boost::is_any_of(delims)); } int main() { // Generate random text string text(N, ' '); for( size_t i = 0; i != N; ++i ) text[i] = (i % 2 == 0)?('a'+(i/2)%26):((i/2)%2?' ':'\t'); char const* delims = " \t[],-'/\\!\"§$%&=()<>?"; // Output strings boost::timer timer; test_boost<vector<string> >(text, delims); cout << "Time: " << timer.elapsed() << endl; // Output string views typedef string::const_iterator iter; typedef boost::iterator_range<iter> string_view; timer.restart(); test_boost<vector<string_view> >(text, delims); cout << "Time: " << timer.elapsed() << endl; // Custom split timer.restart(); vector<string> vs; test_custom(text, delims, vs); cout << "Time: " << timer.elapsed() << endl; // Custom split timer.restart(); vector<string_view> vsv; test_custom(text, delims, vsv); cout << "Time: " << timer.elapsed() << endl; // Custom split timer.restart(); vector<string> vsp; test_strpbrk(text, delims, vsp); cout << "Time: " << timer.elapsed() << endl; // Custom split timer.restart(); vector<string_view> vsvp; test_strpbrk(text, delims, vsvp); cout << "Time: " << timer.elapsed() << endl; return 0; } </code></pre> <p>Compiling this with Boost 1.46.1 using GCC 4.5.1 with the <code>-O4</code> flag enabled I get:</p> <ul> <li>Time: 5.951 (Boost.Split + vector)</li> <li>Time: 3.728 (Boost.Split + vector <li>Time: 1.662 (Custom split + vector)</li> <li>Time: 0.144 (Custom split + vector)</li> <li>Time: 2.13 (Strpbrk + vector)</li> <li>Time: 0.527 (Strpbrk + vector)</li> </ul> <p>NOTE: There's a slight difference in the output as empty tokens are dropped by my custom function. But you can adapt this code to your needs if you decide to use it.</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload