StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
text
Body
copied!<p>The problems with your code (at least second approach) are</p> <ul> <li>You are constantly using lowecase which is slow if called so many times</li> <li>If I saw correctly you are copying the whole remaining text back to the original source. This is sure to be extra slow for large strings (eg files)</li> </ul> <p>I have a tokenizer in my library. Its not the fastest or best but it should do (you can get it from <a href="http://www.cromis.net/blog/wp-content/plugins/download-monitor/download.php?id=CromisLibrary.zip" rel="nofollow">Cromis Library</a>, just use the units Cromis.StringUtils and Cromis.Unicode):</p> <pre><code>type TTokens = array of ustring; TTextTokenizer = class private FTokens: TTokens; FDelimiters: array of ustring; public constructor Create; procedure Tokenize(const Text: ustring); procedure AddDelimiters(const Delimiters: array of ustring); property Tokens: TTokens read FTokens; end; { TTextTokenizer } procedure TTextTokenizer.AddDelimiters(const Delimiters: array of ustring); var I: Integer; begin if Length(Delimiters) > 0 then begin SetLength(FDelimiters, Length(Delimiters)); for I := 0 to Length(Delimiters) - 1 do FDelimiters[I] := Delimiters[I]; end; end; constructor TTextTokenizer.Create; begin SetLength(FTokens, 0); SetLength(FDelimiters, 0); end; procedure TTextTokenizer.Tokenize(const Text: ustring); var I, K: Integer; Counter: Integer; NewToken: ustring; Position: Integer; CurrToken: ustring; begin SetLength(FTokens, 100); CurrToken := ''; Counter := 0; for I := 1 to Length(Text) do begin CurrToken := CurrToken + Text[I]; for K := 0 to Length(FDelimiters) - 1 do begin Position := Pos(FDelimiters[K], CurrToken); if Position > 0 then begin NewToken := Copy(CurrToken, 1, Position - 1); if NewToken <> '' then begin if Counter > Length(FTokens) then SetLength(FTokens, Length(FTokens) * 2); FTokens[Counter] := Trim(NewToken); Inc(Counter) end; CurrToken := ''; end; end; end; if CurrToken <> '' then begin if Counter > Length(FTokens) then SetLength(FTokens, Length(FTokens) * 2); FTokens[Counter] := Trim(CurrToken); Inc(Counter) end; SetLength(FTokens, Counter); end; </code></pre>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload