StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POEffects of __attribute__((packed)) on nested array of structures?
text
Body
copied!<h3>The Problem</h3> <p>I'm working on sending a raw structure over a network to a known program on the other side, but have to worry about the silently introduced memory used for aligning the structures (other issues like endianness are covered). What I'm working with is something like:</p> <pre><code>typedef struct __attribute__((packed)) { uint16_t field1; uint16_t field2; uint16_t field3; } packed_1_s; typedef struct __attribute__((packed)) { uint16_t fieldA; uint16_t fieldB; packed_1_s structArray[10]; } packed_2_s; typedef struct __attribute__((packed)) { uint16_t fieldX; packed_2_s fieldY; uint8_t arrayZ[20]; } data_s; </code></pre> <p>I understand that normally the packed_1_s structure could/would have additional space allocated for every instance of the structure to fill it out to the compiler's favored size (dependent on the hardware it's being built for), and that favored size can be anywhere from 2 bytes to 64 bytes (most recently). Normally if I had a single instance of packed_1_s in packed_2_s there would be no problem, but I'm given to understand there's some differences when you try to put elements in an array. </p> <h3>Attempted Solutions</h3> <p>The gcc documentation seems to suggest that by simply including the packed attribute in the packed_2_s definition, the fields, even if they're arrays, will all be as tightly packed as possible and won't add space to the packed_2_s structure to align the elements of the array. The documentation on the align() attribute though suggests that arrays are handled differently than other fields and need the align/packed attribute set on the field directly for it to modify the extra spacing added to match the specified alignment (or lack thereof). I tried setting the packed attribute on both the structArray field, and when that didn't work, did a test by setting the packed attribute on arrayZ in the code above: </p> <pre><code>packed_1_s structArray[10] __attribute__((packed)); uint8_t arrayZ[20] __attribute__((packed)); </code></pre> <p>Both attempts gave me a compiler warning that the packed attribute wasn't understood in this context and would be skipped (good thing I build with "-Wall"). </p> <p>I hoped a way to get around the issue would be to use attribute align(1), indicating a desired alignment of 1 byte which is comparable to the packed attribute, but documentation says the align() attribute can only <em>increase</em> the alignment and packed should be used to <em>decrease</em> the alignment.</p> <h3>Considerations</h3> <p>From what I can determine from the GCC documentation, it seems like there's 3 major cases of additional memory being inserted.</p> <blockquote> <ol> <li>Structure contents may have additional memory allocated to the structure itself to change the spacing between fields.<br> Effectively, the definition of the memory map of the structure contents within the structure may change (though not the order of the elements).</li> <li>Structures may have additional memory allocated to them to fill them out to a more efficient overall size. This is generally intended so other variables coming after a declaration of one of their instances doesn't fall within the same "block" as the structure instance where a "block" is defined by the system/compiler.</li> <li>Arrays, whether within a structure or not, may have additional memory added to them to shift the elements to an efficient alignment.</li> </ol> </blockquote> <p>As far as I can tell, the packed attribute can be used to affect the structures and block the additional memory added in case 1 and 2 above, but there doesn't seem to be a way to handle case 3 above on my compiler(s). </p> <h3>The Question</h3> <p>Is there any way to guarantee that the data_s structure will have absolutely no additional space added to it or any of its sub-structures so I don't have compiler dependent shifts in the memory map? Am I misunderstanding the cases where the compiler can insert space to intentionally shift the memory map?</p> <h3>EDIT</h3> <p>I discussed some of the issues with my local guru and it sounds like I have some misunderstanding of case 3 above. The elements in the array don't have space inserted between them, but the additional space to guarantee they align correctly is added to the structure itself. Apparently this suggests things like "sizeof(structureOnlyContaining_uint32_t)" won't always return "4" since additional space may be added to align the uint32_t datatype on the compiler being used. The result is that there are really only 2 cases:</p> <blockquote> <ol> <li>Larger offsets in between fields in the memory map of the structure.<br> The space between the fields can be modified to align each field. This can be changed using the packed or align() attributes.</li> <li>Structure end padding. The size for a structure, as returned by sizeof(), can be modified so arrays of the structures end up correctly aligned for the system. This allows all systems to assume that the start of structures will always be aligned, causing issues if they aren't. This seems unaffected by the pack or align attributes.</li> </ol> </blockquote> <p>Because of the new case 2, elements of an array in a structure don't necessarily obey the packed or align() attributes specified on the structure, though the start of the array and the field immediately following the array do.</p> <p>My question is then about how to deal with the structArray in packed_2_s since the size of the array as a whole cannot be guaranteed purely by the packed attribute. Is there some way to guarantee the fixed size of the structArray field as a whole? It should be noted that I can't increase the size of the packed_1_s too much since the data_s struct needs to be kept as small as possible (its replacing Audio/Video data in a streaming scenario).</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload