Databases
StackOverflow2013
Postsdbo
POhow to build a predicate/expression/rule/action engine
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POhow to build a predicate/expression/rule/action engine
    primarykey
    data
    text
    <p>Okay so first off, sorry for the TL;DR... </p> <p>2nd, some context about what I want to achieve.</p> <p>I have a script that processes a given file and generates a new file based on a defined set of rules. Each row in the file has a value and the script evaluates the value and generates other values based on that value and rules in general. The generated file contains the original value and the generated values in csv format. </p> <p>For example:</p> <p><strong>input file:</strong></p> <pre><code>row1value row2value </code></pre> <p><strong>output file:</strong></p> <pre><code>"row1value","generatedValue1","generatedValue2","generatedValue3" "row2value","generatedValue1","generatedValue2","generatedValue3" </code></pre> <p>For example, lets say the source file has a colon delimited value on each row, and the goal is to split the value by the colon and put each sub-value into the columns. This is what the "before" and "after" would look like:</p> <p><strong>input file:</strong></p> <pre><code>a:b:c some:random:value </code></pre> <p><strong>output file:</strong></p> <pre><code>"a:b:c","a","b","c" "some:random:value","some","random","value" </code></pre> <p>So to generate the rows, I might have:</p> <pre><code>function generateRow ($key) { // $key is the value for the current row being processed from the source file // $row is the array that will contain the columns to be inserted to the output file $row = array(); // generate the column values $row = explode(":",$key); // make the original value ($key) the first column array_unshift($row, $key); // return the array. some other function will fputcsv it return $row; } </code></pre> <p>Another example would be if the source file contains rows of values like this:</p> <p><code>[prefix]:[url]</code></p> <p>And the values for the columns in the output file would be:</p> <ul> <li>column1 = full original value (it will always be this) </li> <li>column2 = <code>[prefix]</code></li> <li>column3 = [url]</li> <li>column4 = parse [url] for the domain of the url</li> <li>column5 = parse [url] for the file extension </li> </ul> <p>So I write <code>generateRow()</code> to do these things. </p> <p>Okay so I have all this. It is all shiny and working great. I get requests to make new "processes" that receive their own file and generate a new file with columns based on defined rules. I just wanted to provide all this as backstory to put this into context so that y'all would more easily understand what I actually want.</p> <p>So the "problem" is that as it stands, the "process" is made by a coder (me) writing out <code>generateRow()</code> based on the specs. This isn't particularly difficult for myself or anybody else who knows php. But this is not "user friendly" in that certain stakeholders want to be in charge of creating this stuff on their own, but they are not coders.</p> <p>So my task now is to basically provide a web interface for <code>generateRow()</code>. So the way I see it, I basically need to make a form that dynamically generates form elements and possible options based on previously selected items. In other words, an expression engine. Or predicate engine. Or rule builder. Honestly I'm not 100% sure which, if any, of these terms is the most accurate, but I've been doing a lot of googling and reading and so far these are what I've come up with. </p> <p>So for example, the form would start out with asking the user to create a condition or assignment expression, </p> <p>[dropdown:if|set]</p> <p><strong>IF:</strong> If the user clicks "if" then another dropdown displays with a list of "variables" to check against. For example the system will have available as selects a reference to the other columns, any column from the source file (including the key), the import file name, or any custom variable a user previously made (see "set" below). The form will then show an "action" dropdown, that will display things like "is set","not set","greater than","regex",etc. If the user selects something like "is set" or "not set" then no other fields are output. But if the user selects for instance "greater than", then another dropdown will show, asking to select from the list of "variables", or an input field to enter in a value. Examples:</p> <pre><code>IF column1 "is set" /* check if column1 is set */ IF column1 contains "foobar" /* check if column1 contains "foobar" */ If column1 regex "^[a-z]+$" /* check if column1 contains only letters */ </code></pre> <p>Once this is defined, user can add "set" expressions inside it. For simplicity I don't think i need to nest conditions or use AND|OR to make compound conditions. </p> <p><strong>SET:</strong> So if user selects this option, the form will guide them through building an assignment expression. The first dropdown will hold the variables the user can assign values to, such as the columns for the output file, or temporary variables so that they can be referenced in other expressions. Examples:</p> <pre><code>SET [column1] [=] ["foobar"] SET [column2] [=] [column1] SET [column1] [regex] [userVar1] ["^[^:]+"] SET [userVar1] [explode] [key] [":"] </code></pre> <p>Well I'm not sure that's the "best" way to present it but hopefully you get the idea.</p> <p>Then I would save these expressions and then write php code to evaluate them; basically translate them into actual php code. I think I'm actually fine as far as that part: I would just use an interpreter pattern. </p> <p>But where I need help on is the whole expression "mapping" / "builder" part. Actually I think i can even swing the "builder" part, if I can get a "mapping" of possible expressions fleshed out. </p> <p><strong>So yeah, that's where the core of my question lays: how to map out possible expressions.</strong> Currently I've been playing around with trying to structure/map it as xml, but I can't seem to grasp how to do it, short of just hardcoding every possible path. Firstly, to me this seems kinda inefficient, like there should be a smarter way to do it. It can't possibly be easily scalable... let's say "process" #1's ruleset has 2 source file columns to draw from and 5 output file columns to generate.. and "process" #2's is 1 and 4? What about accounting for variable amount of user defined variables a user can set. </p> <p>So, does anybody have any tips or links to tuts explaining how to do this sort of thing? Or better yet a prefab (php) solution would be nice, but I haven't been able to find something yet.. </p> <p><strong>edit:</strong> here is an example of where I am now, to hopefully give a better understanding of my problem.</p> <p>For example, here's how it woulda sorta go down if i were to just hardcode a map (xml)</p> <pre><code>&lt;expressions&gt; &lt;expression type='if'&gt; &lt;variable name='column1'&gt; &lt;operator type='isset'&gt;&lt;/operator&gt; &lt;operator type='notset'&gt;&lt;/operator&gt; &lt;operator type='equals'&gt; &lt;variable name='column1' /&gt; &lt;variable name='column2' /&gt; &lt;variable name='column3' /&gt; &lt;/operator&gt; &lt;operator type='greaterThan'&gt; &lt;variable name='column1' /&gt; &lt;variable name='column2' /&gt; &lt;variable name='column3' /&gt; &lt;/operator&gt; &lt;operator type='lessThan'&gt; &lt;variable name='column1' /&gt; &lt;variable name='column2' /&gt; &lt;variable name='column3' /&gt; &lt;/operator&gt; &lt;/variable&gt; &lt;variable name='column2'&gt; &lt;operator type='equals'&gt; &lt;variable name='column1' /&gt; &lt;variable name='column2' /&gt; &lt;variable name='column3' /&gt; &lt;/operator&gt; &lt;operator type='greaterThan'&gt; &lt;variable name='column1' /&gt; &lt;variable name='column2' /&gt; &lt;variable name='column3' /&gt; &lt;/operator&gt; &lt;operator type='lessThan'&gt; &lt;variable name='column1' /&gt; &lt;variable name='column2' /&gt; &lt;variable name='column3' /&gt; &lt;/operator&gt; &lt;/variable&gt; &lt;/expressions&gt; </code></pre> <p>Couple notes:</p> <ol> <li>there are an arbitrary number of <code>variable</code> nodes. There will be one for each source file column, one for each output file column name, one for each variable the user creates on-the-fly, etc..</li> <li>there are a variable number of <code>operator</code> types. I just showed 3 examples, but so far I have like 20 that i've thought of</li> <li>Notice how some of the <code>operator</code> types would require additional form fields to be output on the form. For example, "if [column1] [isset]" vs. "if [column1] [greaterThan] [column2]"</li> <li>The <code>expression</code> node shows what the "if" expression would look like. There would also be a an <code>expression</code> node of type "set" which would have a different structure but in principle work the same. For example there could be an expression "[set] [column2] [equals] [column1]" or "[set] [column1] [regex] [sourceColumn1] [input value (regex)]"</li> </ol> <p>So right now I can just walk down the path to dynamically build menus based on what's available going down the tree. </p> <p>But as you can see, it seems like there's a lot of repetition, and also it would be a pita to throw new things into the mix (like a new operator). So the question here is, how would I structure this better? Or is it just not possible, and the only solution is to just hardcode every possibility? </p>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PTQuestion
    1. USslinkhi
    1. USslinkhi
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    3. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POhow to build a predicate/expression/rule/action engine
      1. USslinkhi
      1. VTBountyStart
    2. VO
      singulars
      1. POhow to build a predicate/expression/rule/action engine
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. POhow to build a predicate/expression/rule/action engine
      1. This table or related slice is empty.
      1. VTDownMod
    1. CObreak the problem down into pieces. write a generic row parser that has no hard-coded values in it, everything needed being passed as arguments. then, one-by-one, delete formal parameters and replace with a top-of-function var pointing to a form input's value instead of an argument, leaving the local name the same. ex (age, height){} would become (height){var age=ages.value;} after 1 step. you can hard-code form values/options at first to get it running, then figure out how to populate your form automatically based upon the input file. one step at a time...
      singulars
      1. POhow to build a predicate/expression/rule/action engine
      1. USdandavis
    2. CO@dandavis thanks for the response! I'm not quite sure I understand what you are suggesting..I think I already have a plan for parsing the expressions, and I'm pretty sure I can work out dynamically populating the form - assuming I have a map of what possible combinations there are. So what I'm actually stuck on is how to structure the map itself. The *easy* way would be to just hardcode all the possibilities somewhere (array/object, db struture, xml file).
      singulars
      1. POhow to build a predicate/expression/rule/action engine
      1. USslinkhi
    3. COBut all tedium aside, it's not a very scalable solution. If I think of a new "operator" to add, I'd have to go through and add it to every applicable branch. That doesn't really seem efficient to me. It seems the answer lies somewhere in having nodes backward and forward reference other nodes, but I've not quite worked out how to actually do that?
      singulars
      1. POhow to build a predicate/expression/rule/action engine
      1. USslinkhi
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload