Network Data Packing

Background

Network traffic consist of sending ints(4 bytes) and bools(1 byte) and we have a fixed amount for each package. Sending more variables requires either sending multiple packages or packing more than one variable into a single int. This page will tell about the latter, how to set it up in a readable way and how to avoid introducing bugs while doing so. Historically this has resulted in bugs multiple times, which is why readable approach is needed rather than manipulating bits directly.

C++ Fundamentals

Two keywords are needed for this: union and struct. They are written the same way, but they behave differently. Structs will store variables consequently while unions will store all variables in the same memory location.

union
{
    int A;
    int B;
};

A and B will be in the same memory location, meaning setting A to 0 will set B to 0. Vanilla doesn't use unions because in most cases this isn't useful.

struct
{
    int A;
    int B;
};

Here B is placed right after A. This means the struct takes up 8 bytes while the union takes up 4. However the struct allows A and B to work independently.

Union of structs

It is possible to make a union of structs, like:

union
{
    int iA;
    struct
    {
        short sA;
        short sB;
    };
    struct
    {
        char cA;
        char cB;
        char cC;
        char cD;
    };
};

This will result in the following memory layout:

int iA
short sA		short sB
char cA	char cB	char cC	char cD

The columns shows which variables shares memory.

Real example

struct NetworkDataTradeRouteInts
{
	NetworkDataTradeRouteInts(int a = 0) : iNetwork(a) {}

	union
	{
		int iNetwork;
		struct
		{
			unsigned short iImportLimitLevel;
			unsigned short iMaintainLevel;
		};
	};
};

BOOST_STATIC_ASSERT(sizeof(NetworkDataTradeRouteInts) == sizeof(int));

iNetwork
iImportLimitLevel		iMaintainLevel

When used iImportLimitLevel and iMaintainLevel are accessed directly for both reading and writing. When sending, iNetwork is used. When receiving through network, create a new NetworkDataTradeRouteInts instance with:

NetworkDataTradeRouteInts buffer(iData2);

This will grant access to buffer.iImportLimitLevel and buffer.iMaintainLevel.

BOOST_STATIC_ASSERT is used to cause a compile error if the struct is bigger than the int. If that's the case, then we know it won't work.

Use bit fields

By using bit fields all 32 bits can be directly accessed and even create 32 bools if needed, though a more realistic setup would be 3 ints of 10 bits and 2 bools.

Danger: the compiler can add padding when mixing data sizes

struct
{
    YieldTypes A_eYield : 16;
    short A_iAmount;
};

The compiler will see this as a 4 byte variable and a 2 byte variable and place them in different words (4 byte slots) before using the bit field. As a result this will use 8 bytes.

struct
{
    YieldTypes B_eYield : 16;
    int B_iAmount : 16;
};

This on the other hand will use 4 bytes as both YieldTypes and int will use 4 bytes, hence compatible for packing according to the compiler. Bool will also use 4 bytes unless bit fields is used.

iNetwork
A_eYield		padding #1		A_iAmount		padding #2
B_eYield		B_iAmount

Clearly the A approach won't work as A_iAmount is stored outside of iNetwork. Using #pragma pack won't help as it will remove padding #2 while keeping padding #1 in place.

For this reason if bool or enum types are in use, use ints with bit fields rather than shorts or char as that will make the compiler view all variables as 4 bit variables, hence compatible to be placed in the same 4 byte section.