StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
text
Body
copied!<p>There is no out-of-the-box or customize it for your needs solution of which I am aware to solve the problem you are facing.</p> <p>Looking at your problem, there are several distinct needs which standout:</p> <ul> <li>Test Tagging</li> <li>Test Execution</li> <li>Test Result Capture</li> </ul> <p>The first issue you need to address is how are you going to identify and track the tests that you can execute in a given environment, concurrently, etc. </p> <p>If you were using <a href="https://nose.readthedocs.org/en/latest/" rel="nofollow">nose</a> (i.e. is nicer testing for python), you would be able to use the <a href="https://nose.readthedocs.org/en/latest/plugins/attrib.html" rel="nofollow">Attribute selector plugin</a> to tag the tests with various attributes.</p> <p>nose would also assist you in your test execution and when coupled with test tagging, would allow you to run tests in parallel, etc. Using nose, you should be able to run an external executable and assert based on its status code.</p> <p>The final problem you face is how to capture test output in a proprietary format and translate it to a format that can be ingested to readily available tools. Again, I believe nose could help you out here. You could build a <a href="https://nose.readthedocs.org/en/latest/developing.html" rel="nofollow">nose plugin</a> that would take your proprietary format and translate it to XUnit format and report results that way.</p> <p>With all of the above in mind, here is how I would tackle this situation:</p> <ul> <li>Create a test wrapper class based on nose which <ul> <li>Can be tagged </li> <li>Execute a program and capture result output</li> <li>Translate that output to XUnit</li> </ul></li> <li>Create a wrapper for each test <ul> <li>Figure out how to automate this process, because it is going to be tedious</li> </ul></li> <li>Build a test execution harness, which <ul> <li>Spins Up one or more VMs</li> <li>Loads and runs a test wrapper</li> <li>Capture the results</li> </ul></li> </ul>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload