Databases
StackOverflow2013
Postsdbo
PORead in a file using a regular expression?
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PORead in a file using a regular expression?
    primarykey
    data
    text
    <p>This is tangentially related to <a href="https://stackoverflow.com/questions/12939597/returning-the-regular-expression-match-as-part-of-a-split-or-equivalent-functio">an earlier question of mine.</a></p> <p>Essentially, the solution in that question worked great, but now I need to adapt it to work in a much larger analysis application. Simply using <code>StreamReader.ReadToEnd()</code> is not acceptable, since some of the files I will be reading in are very, very large. If there's been a mistake and someone forgot to clean up, they can theoretically be gigabytes big. Obviously I can't just read to the end of that.</p> <p>Unfortunately, the normal read lines is also not acceptable, because some of the rows of data I am reading in contain stack traces - they obviously use <code>/r/n</code> in their formatting. Ideally, I would like to tell the program to read forward until it hits a match for a regex, which it then returns. Is there any functionality to do this in .net? If not, can I get some suggestions for how I'd go about writing it?</p> <p>Edit: To make it a bit easier to follow my question, here's a paste of some of the important parts of the adapted code:</p> <pre><code>foreach (var fileString in logpath.Select(log =&gt; new StreamReader(log)).Select(fileStream =&gt; fileStream.ReadToEnd())) { const string junkPattern = @"\[(?&lt;junk&gt;[0-9]*)\] \((?&lt;userid&gt;.{0,32})\)"; const string severityPattern = @"INFO|ERROR|FATAL"; const string datePattern = "^(?=[0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2},[0-9]{3})"; var records = Regex.Split(fileString, datePattern, RegexOptions.Multiline); foreach (var record in records.Where(x =&gt; string.IsNullOrEmpty(x) == false)) ...... </code></pre> <p>The problem lies in the Foreach. <code>.Select(fileStream =&gt; fileStream.ReadToEnd())</code> is gonna blow up memory badly, I just know it.</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USCommunity
    1. UStmesser
    plurals
    1. PL
      singulars
      1. LTLinked
    2. PL
      singulars
      1. LTLinked
    3. PL
      singulars
      1. LTLinked
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. PORead in a file using a regular expression?
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. PORead in a file using a regular expression?
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. PORead in a file using a regular expression?
      1. This table or related slice is empty.
      1. VTUpMod
    1. COAnd there you have one of the main reasons I don't really care for using RegEx for tasks like this. If you wrote a simple parser, you could simply adapt it to handle new lines.
      singulars
      1. PORead in a file using a regular expression?
      1. USJonathan Wood
    2. CO@JonathanWood The regexes are awesome for parsing out individual records after I have the entire record. I just pop out all the information I need and it goes straight into the associated fields. The problem is the file input in this case, it seems like it simply isn't flexible enough to give me a single record at a time with the file I've got. But that seems bogus, doesn't it? This cannot be an uncommon problem.
      singulars
      1. PORead in a file using a regular expression?
      1. UStmesser
    3. COSo a stack trace has /r/n. Why does that eliminate readline?
      singulars
      1. PORead in a file using a regular expression?
      1. USpaparazzo
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload