Databases
StackOverflow2013
Postsdbo
POWhich C++ libraries should I use for a large parallel computing number-crunching project exploiting third-party applications
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POWhich C++ libraries should I use for a large parallel computing number-crunching project exploiting third-party applications
    primarykey
    data
    text
    <p><strong>Introduction</strong></p> <p>I want to request a lot of advice on a new programming project I am going to start on my own. I am going to be very precise in what I would like to accomplish and in what my basic requirements are. Therefore this is going to be a long question. Please bear with me.</p> <p>I am going to split the question into five sections:</p> <ol> <li>Real-world problem</li> <li>Simulation problem</li> <li>Requirements and preferences</li> <li>Additional information</li> <li>List of advice requests</li> </ol> <p><strong>1. Real-world problem</strong></p> <p>Skyscrapers and large bridges suffer from dynamic wind loading. This means, when designed incorrectly they can collapse due to wind-induced vibrations (this actually happened in 1940: <a href="http://www.youtube.com/watch?v=3mclp9QmCGs">http://www.youtube.com/watch?v=3mclp9QmCGs</a>). To design such structures correctly, efficient number-crunching software is required for analysis and simulation.</p> <p><strong>2. Simulation problem</strong></p> <p>There exists a multitude of software capable of either simulating fluid flows or structural mechanics. Many have already been developed for over 30 years and are proven and mature technologies. Writing a multi-physical program capable of simulating both fluid flows and structural mechanics simultaneously from scratch, is therefore unwise. First of all, you would need years of development before reaching maturity and it is very hard to enter a world which has depended on specific software for over 30 years. But more important...why recreate when you can reuse? Instead of pursuing a monolithic approach, I prefer a partitioned approach where I can reuse existing simulation software.</p> <p>In the partitioned approach I will use software X to simulate flows and I will use software Y to simulate structures. Then I will write my own coupling algorithm which establishes communication between X and Y and uses them to simulate the multi-physical problem (e.g. wind-induced vibrations of skyscrapers or bridges). The reason I use X and Y and not actual software names is because X and Y are supposed to be black-boxes. In no way is my coupling algorithm to be dependent on the implementation of X and Y. The algorithm will only depend on the output of X and Y. This way an end-user can select which ever X or Y is available to them or which ever X or Y is capable of doing what the end-user wants to achieve.</p> <p>Because I want to use a black-box partitioned approach, software X knows nothing of Y and vice-versa. But how do I simulate the deformation of a bridge without knowing anything of the surrounding air flow and how to I know in which way the surrounding air flow is perturbed by the structure without knowing anything about its deformation? The answer is simple: start with a guess and use an iterative approach to converge to the correct solution. This approach is however very computationally expensive. To reduce the computational cost the coupling algorithm can be written in a clever way using very efficient technologies, not to be discussed here. All I would like to say is that some heavy linear algebra number-crunching is required.</p> <p><strong>3. Requirements and preferences</strong></p> <p>What I need to do is:</p> <ul> <li>establish communication between third-party open-source or proprietary software</li> <li>perform some heavy number-crunching (linear algebra)</li> <li>visualise results (2D / 3D plotting and animating)</li> <li>deliver an interactive analysis and development environment</li> <li>create an intuitive graphical user interface</li> </ul> <p>What I want my software to be:</p> <ul> <li>open-source</li> <li>cross-platform</li> <li>extendable through scripts and/or shared libraries</li> </ul> <p>What I am going to use:</p> <ul> <li>C++ for heavy number-crunching</li> <li>CPython for programming logic</li> <li>NumPy / SciPy for some number-crunching in CPython</li> <li>Matplotlib for results visualisation in CPython</li> </ul> <p><strong>4. Additional information</strong></p> <p>Facts:</p> <ul> <li>one-man project at start, grow to company if successful</li> <li>primary OS is a KDE-based Linux distribution</li> </ul> <p>Business model:</p> <ul> <li>Free software and basic documentation.</li> <li>Paid services and elaborate documentation.</li> </ul> <p><strong>5. List of advice requests</strong></p> <p>I want to do all number-crunching in C++ by writing many functions which individually perform just a tiny task. The program logic is to be contained in a CPython package which executes the entire simulation while relying on the C++ functions to perform the number-crunching. The C++ / CPython algorithm is to be extended with scripts written in CPython (using NumPy, SciPy, SymPy and Matplotlib) to generate and visualise results from raw numerical data. I want to be able to do parallel computing and I need to communicate with several third party open-source AND proprietary software.</p> <p>To accomplish all that I am going to need a whole bunch of existing libraries/packages/technologies etc. And to all relevant issues I know what I can use, however I do not know what I should use. The best solution is as always to try everything out and see what works best. However if any experienced user can weed out some of the more unlikely candidates I would gladly receive his or hers advice, suggestion, pro / con list on:</p> <ol> <li>Glueing C++ and CPython (e.g. CTypes, SIP, SWIG etc.)</li> <li>C++ linear algebra number-crunching library (e.g. Armadillo, Eigen, PETSc etc.)</li> <li>Graphical interface development library (e.g. Qt, GTK, wxWidgets etc.)</li> <li>Software communication and parallel computing (e.g. MPICH, OpenMPI, OpenMP etc.)</li> <li>CPython 2.7.x or CPython 3.x</li> </ol> <p>NOTE: I have summed some options above, but these are only exemplary and not a limitation to. I am open to everything as long as it is written in C, C++, Fortran or Python. Also I do not expect an answer in all five categories enlisted above from one individual. Let the collective knowledge of the community take care of that.</p> <p>I thank all contributors and wish you all the best of luck in your own endeavors.</p>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PTQuestion
    1. This table or related slice is empty.
    1. USAeronaelius
    plurals
    1. This table or related slice is empty.
    1. PL
      singulars
      1. LTLinked
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    3. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POWhich C++ libraries should I use for a large parallel computing number-crunching project exploiting third-party applications
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POWhich C++ libraries should I use for a large parallel computing number-crunching project exploiting third-party applications
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. POWhich C++ libraries should I use for a large parallel computing number-crunching project exploiting third-party applications
      1. USBlender
      1. VTFavorite
    1. CODon't know of any but giving you an up vote cause its cool to learn about structural engineering
      singulars
      1. POWhich C++ libraries should I use for a large parallel computing number-crunching project exploiting third-party applications
      1. USGreg
    2. COThank you Greg for the upvote.
      singulars
      1. POWhich C++ libraries should I use for a large parallel computing number-crunching project exploiting third-party applications
      1. USAeronaelius
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload