Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>Since you said that <code>texte</code> and <code>texteTotal</code> are strings, I have a couple of suggestions:</p> <p><strong>1. Always concatenate strings with the <a href="http://msdn.microsoft.com/en-us/library/kd1e4aey%28v=VS.85%29.aspx" rel="nofollow"><code>&amp;</code> operator</a>.</strong></p> <ul> In VBScript, there are two ways to concatenate (add together) two string variables: the & operator and the + operator. The + operator is normally used to add together two numeric values, but is retained for backwards compatibility with older versions of BASIC that did not have the & operator for strings. Because the & operator is available in VBScript, it is recommended that you always prefer using it to concatenate strings and reserve the + for adding together numeric values. This won't necessarily provide any speed increase, but it eliminates any ambiguity in your code and makes clear your original intention. This is especially important in VBScript where you're working with Variants. When you use the + operator on a string that may contain numeric values, you have no way of knowing whether it will add the two numeric values together or combine the two strings. </ul> <p><strong>2. Remember that string concatenation in VBScript has tremendous overhead and is very inefficient.</strong></p> <ul> Unlike VB.NET and Java, VBScript does not have a StringBuilder class to aid in the creation of large strings. Instead, whenever you repeatedly add things to the end of a string variable, VB will repeatedly copy that variable over and over again. When you're building a string in a loop like you do in the above code, this can really degrade performance as VB constantly allocates space for the new string variable and performs a copy. With each iteration of the loop, the concatenation becomes slower and slower (in geek-speak, you're dealing with an n&sup2; algorithm, where n = the number of concatenations). The problem gets even worse if the string exceeds 64K in size. VB can store small strings in a 64K cache, but if a string becomes larger than the cache, performance drops even more. This kind of thing is of course invisible to the programmer for simplicity, but if you're concerned about optimization, it's something to understand is happening in the background. <br><br> In light of the above information, let's revisit the two code samples that you posted. You said that `texte` is "not very big but there are hundreds [of] thousands [of] lines." That means you may easily run out of space in the 64K string cache, and eventually you may even run out of space in the RAM allocated to the script. The limit varies, but you can eventually get "Out of Memory" errors, depending on how large your string grows. Even if you're not anywhere near that point now, it's worth considering for the future. If someone goes back later to add functionality to the script, will they remember or bother to change the string concatenation algorithm? Probably not. <br><br> To prevent any "Out of Memory" errors from cropping up, you can simply stop keeping the text string in RAM and write it directly to a file instead. This makes even more sense in your case because that's what you're eventually going to do with the string anyway! <b>Why waste CPU cycles and space in memory by continually allocating and reallocating new string variables for each iteration of the loop when you could just write the value to the file and forget about it? I'd say that your first code sample is the simplest and preferred method to accomplish what you want.</b> <br><br> The only time that I would consider the second method is if you were dealing with file I/O that was extremely inefficient, such as a disk being accessed over a network connection. As long as you're writing to a local disk, you're not going to have any performance problems here. The other concern, pointed out by @astander, is that the first method leaves the file you're writing to open for a long period of time, placing a lock on that resource. While this is an important concern, I think its implication for your application is minimal, as I assume that you're creating and writing to your own private file that no other application is expected to be able to access. <br><br> However, despite my recommendation, the first method is still not the most optimized and efficient way to concatenate strings. The <i>best</i> way would be a VBScript implementation of a StringBuilder class that stores the string in memory in an array of bytes as it is created. Periodically, as you add text to the string, the concatenation class would allocate more space to the array to hold the additional text. This would be much faster than the native VBScript implementation of string concatenation because it would perform these reallocations far less often. Additionally, as the string you're building in memory grows larger, your StringBuilder class could flush the contents of the string in memory to the file, and start over with an empty string. You could use something like Francesco Balena's CString class (<a href="http://www.vbcode.com/asp/showsn.asp?theID=415" rel="nofollow">http://www.vbcode.com/asp/showsn.asp?theID=415</a>), or Microsoft's example (complete with some benchmarks and a further explanation) available here: <a href="http://support.microsoft.com/kb/170964" rel="nofollow">http://support.microsoft.com/kb/170964</a>. </ul>
    singulars
    1. This table or related slice is empty.
    1. POVBA I/O Performance
      singulars
      1. PTQuestion
    1. PTAnswer
    1. This table or related slice is empty.
    1. USCody Gray
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    1. COCody Thank you very much for your time. Here are a few more information: The Code Sample #1 is the current production code. To be more precise, this is one part of the whole process from 1)getting information from a DB #1 2)calculation with mathematics formulas (matrices, vectors) , N= BIG 3)copy results to txt files ( 10 k lignes + ) each. 4)psql queries to insert to Databases 5)restitution I am wondering if the copy to txt files is really necessary and costly compared to psql insertions? I like your idea of building a custom string class, do you think it can overtake the I/O performance?
      singulars
      1. PO
      1. USuser502080
    2. COSo the way it's set up now, the text file is acting as a temporary buffer for string concatenation? To be honest, the one area of Windows programming that I have almost zero experience in is databases, so I'm not really sure about the latency comparison between psql insertions and writing to a text file. My guess, though, is that database access is always going to be slower than any modern computer can read/write to a text file, unless your database server is caching the inputs. Except that you're still going to be bottlenecked by the speed of the database when you re-insert the results after
      singulars
      1. PO
      1. USCody Gray
    3. CO...you've finished all of the calculations. Maybe someone else who has more experience with psql/databases can jump in and help me out here? A custom string class is definitely a viable solution for speeding up string concat, but before you spend too much time and effort on benchmarking and optimizing this code it might be worth considering how difficult it would be to port the routine to a language like VB.NET with a built-in StringBuilder class. My suspicion is that the VBScript code you're using would port fairly painlessly to VB.NET, much easier than say a full VB6 project.
      singulars
      1. PO
      1. USCody Gray
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload