Databases
StackOverflow2013
Postsdbo
PO
Body
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    text
    copied!<p>Here is how to write N dimension arrays in HDF5 format</p> <p>It is much better to use the <a href="http://www.boost.org" rel="noreferrer">boost</a> <a href="http://www.boost.org/doc/libs/1_53_0/libs/multi_array/doc/user.html" rel="noreferrer"><code>multi_array</code></a> class. This is the equivalent of using <code>std::vector</code> rather than raw arrays: It does all the memory management for you and you can access elements as efficiently as raw arrays using familiar subscripting (e.g. <code>data[12][13] = 46</code>)</p> <p>Here is a short example:</p> <pre><code>#include &lt;algorithm&gt; #include &lt;boost/multi_array.hpp&gt; using boost::multi_array; using boost::extents; // dataset dimensions set at run time int NX = 5, NY = 6, NZ = 7; // allocate array using the "extents" helper. // This makes it easier to see how big the array is multi_array&lt;double, 3&gt; float_data(extents[NX][NY][NZ]); // use resize to change size when necessary // float_data.resize(extents[NX + 5][NY + 4][NZ + 3]); // This is how you would fill the entire array with a value (e.g. 3.0) std::fill_n(float_data.data(), float_data.num_elements(), 3.0) // initialise the array to some variables for (int ii = 0; ii != NX; ii++) for (int jj = 0; jj != NY; jj++) for (int kk = 0; kk != NZ; kk++) float_data[ii][jj][kk] = ii + jj + kk // write to HDF5 format H5::H5File file("SDS.h5", H5F_ACC_TRUNC); write_hdf5(file, "doubleArray", float_data ); </code></pre> <p>The last line calls a function which can write <code>multi_array</code>s of any dimension and any standard number type (<code>ints</code>, <code>chars</code>, <code>floats</code> etc). </p> <p>Here is code for <code>write_hdf5()</code>.</p> <p>First, we must map c++ types to HDF5 types (from the <code>H5</code> c++ api):</p> <pre><code>#include &lt;cstdint&gt; //!_______________________________________________________________________________________ //! //! map types to HDF5 types //! //! //! \author lg (04 March 2013) //!_______________________________________________________________________________________ template&lt;typename T&gt; struct get_hdf5_data_type { static H5::PredType type() { //static_assert(false, "Unknown HDF5 data type"); return H5::PredType::NATIVE_DOUBLE; } }; template&lt;&gt; struct get_hdf5_data_type&lt;char&gt; { H5::IntType type { H5::PredType::NATIVE_CHAR }; }; //template&lt;&gt; struct get_hdf5_data_type&lt;unsigned char&gt; { H5::IntType type { H5::PredType::NATIVE_UCHAR }; }; //template&lt;&gt; struct get_hdf5_data_type&lt;short&gt; { H5::IntType type { H5::PredType::NATIVE_SHORT }; }; //template&lt;&gt; struct get_hdf5_data_type&lt;unsigned short&gt; { H5::IntType type { H5::PredType::NATIVE_USHORT }; }; //template&lt;&gt; struct get_hdf5_data_type&lt;int&gt; { H5::IntType type { H5::PredType::NATIVE_INT }; }; //template&lt;&gt; struct get_hdf5_data_type&lt;unsigned int&gt; { H5::IntType type { H5::PredType::NATIVE_UINT }; }; //template&lt;&gt; struct get_hdf5_data_type&lt;long&gt; { H5::IntType type { H5::PredType::NATIVE_LONG }; }; //template&lt;&gt; struct get_hdf5_data_type&lt;unsigned long&gt; { H5::IntType type { H5::PredType::NATIVE_ULONG }; }; template&lt;&gt; struct get_hdf5_data_type&lt;long long&gt; { H5::IntType type { H5::PredType::NATIVE_LLONG }; }; template&lt;&gt; struct get_hdf5_data_type&lt;unsigned long long&gt; { H5::IntType type { H5::PredType::NATIVE_ULLONG }; }; template&lt;&gt; struct get_hdf5_data_type&lt;int8_t&gt; { H5::IntType type { H5::PredType::NATIVE_INT8 }; }; template&lt;&gt; struct get_hdf5_data_type&lt;uint8_t&gt; { H5::IntType type { H5::PredType::NATIVE_UINT8 }; }; template&lt;&gt; struct get_hdf5_data_type&lt;int16_t&gt; { H5::IntType type { H5::PredType::NATIVE_INT16 }; }; template&lt;&gt; struct get_hdf5_data_type&lt;uint16_t&gt; { H5::IntType type { H5::PredType::NATIVE_UINT16 }; }; template&lt;&gt; struct get_hdf5_data_type&lt;int32_t&gt; { H5::IntType type { H5::PredType::NATIVE_INT32 }; }; template&lt;&gt; struct get_hdf5_data_type&lt;uint32_t&gt; { H5::IntType type { H5::PredType::NATIVE_UINT32 }; }; template&lt;&gt; struct get_hdf5_data_type&lt;int64_t&gt; { H5::IntType type { H5::PredType::NATIVE_INT64 }; }; template&lt;&gt; struct get_hdf5_data_type&lt;uint64_t&gt; { H5::IntType type { H5::PredType::NATIVE_UINT64 }; }; template&lt;&gt; struct get_hdf5_data_type&lt;float&gt; { H5::FloatType type { H5::PredType::NATIVE_FLOAT }; }; template&lt;&gt; struct get_hdf5_data_type&lt;double&gt; { H5::FloatType type { H5::PredType::NATIVE_DOUBLE }; }; template&lt;&gt; struct get_hdf5_data_type&lt;long double&gt; { H5::FloatType type { H5::PredType::NATIVE_LDOUBLE }; }; </code></pre> <p>Then we can use a bit of template forwarding magic to make a function of the right type to output our data. Since this is template code, it needs to live in a header file if you are going to output HDF5 arrays from multiple source files in your programme:</p> <pre><code>//!_______________________________________________________________________________________ //! //! write_hdf5 multi_array //! //! \author leo Goodstadt (04 March 2013) //! //!_______________________________________________________________________________________ template&lt;typename T, std::size_t DIMENSIONS, typename hdf5_data_type&gt; void do_write_hdf5(H5::H5File file, const std::string&amp; data_set_name, const boost::multi_array&lt;T, DIMENSIONS&gt;&amp; data, hdf5_data_type&amp; datatype) { // Little endian for x86 //FloatType datatype(get_hdf5_data_type&lt;T&gt;::type()); datatype.setOrder(H5T_ORDER_LE); vector&lt;hsize_t&gt; dimensions(data.shape(), data.shape() + DIMENSIONS); H5::DataSpace dataspace(DIMENSIONS, dimensions.data()); H5::DataSet dataset = file.createDataSet(data_set_name, datatype, dataspace); dataset.write(data.data(), datatype); } template&lt;typename T, std::size_t DIMENSIONS&gt; void write_hdf5(H5::H5File file, const std::string&amp; data_set_name, const boost::multi_array&lt;T, DIMENSIONS&gt;&amp; data ) { get_hdf5_data_type&lt;T&gt; hdf_data_type; do_write_hdf5(file, data_set_name, data, hdf_data_type.type); } </code></pre>
 

Querying!

 
Guidance
An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload