How to think when writing pattern for similar records with different sizes

[bos4711]

I am reverse engineering a binary dataformat written by a program. In this program you can create 8 different 2D shapes and after digging deep in the datafile I have found the records for these shapes but I would need some help how to describe them as cleanly as possible.

Depending on which shape has been used, a u8 byte at position 0x44 in the record says which kind of shape it is. I have gotten this far:

enum PrimitiveType : u8 {
    Line    = 0x0,
    Box     = 0x1,
    Circle  = 0x2,
    Bezier  = 0x3,
    Text    = 0x4,
//  What?   = 0x5,    
    Symbol  = 0x6,
    Polygon = 0x7,
    Origin  = 0x8,
};

struct Primitive {
    padding[0x30];      // first 0x30 bytes are uninteresting
    PrimitiveType which_type @ 0x44;

    if (which_type == PrimitiveType::Line) {
        std::print("Line");
    } else {
        std::print("Not line");    // Add more here later
    }
};

Using the if-case I can select which 2D shape the record is supposed to be for. Basically, each record starts with bytes 0x02 ?? and everything up to - but not including - next 0x02 ?? OR an end of record marker 0xFF (no 0x02 ?? following). All shapes use this kind of record markers.

What I'm having problems with is how to define / work with the different sized payloads between these markers. A lot of structures are the same for all shapes (for example, colors u32[2] and modifiers u8[7 bits]) while some others are different. A line shape has two coordinates only, a Bezier shape has two coordinates for the end points and two coordinates for the drag handles, a polygon has a minimum of three coordinates (an u8 in the payload defines how many coordinates are expected to be found in the record), and so forth.

I can use a messy mix of functions and if / while to find all bits and pieces, but I have no idea how to create a clean struct depending on the u8 byte that tells me which shape it is and since the record can be of very varying size. Therefore I am asking for a sort of guidance or a ready example pattern I can study to learn how to do this.

[WerWolv]

Hey! Thing I'd do would be some variation of this:

struct Primitive {
    u8 marker;

    // Handle end marker
    if (marker == 0xFF)
        break;

    padding[0x2F];
    PrimitiveType type;

    // Handle primitive types
    match (type) {
        (PrimitiveType::Line): Line data;
        (PrimitiveType::Bezier): Bezier data;
    }
};

Primitive primitives[while(true)] @ 0x00;

That matches all primitives until it hits the 0xFF marker

[bos4711]

If I understand this correctly, there should be a separate struct for each shape (which handles all the peculiars of the shape in question), and then basically allocate this struct in the Primitive without dealing with the shape details in there?

I think that will work. I will definitely give it a try and see what I end up with.

[bos4711]

I think that solved basically everything.

Code (still messy, will clean-up after thorough testing):

struct Radius {
    s32 radius;
} [[format_read("radius_formatter")]];;

struct Color {
    u32 RGBA;
} [[format_read("color_formatter")]];

enum PrimitiveType : u8 {
    Line    = 0x0,
    Box     = 0x1,
    Circle  = 0x2,
    Bezier  = 0x3,
    Text    = 0x4,
//  What?   = 0x5,    
    Symbol  = 0x6,
    Polygon = 0x7,
    Origin  = 0x8,
};

fn _d(str foo) {
    std::print("In {}", foo);
};

struct Line {
    _d("Line");
};

struct Box {
    _d("Box");
};

struct Circle {
    _d("Circle");
    Coordinate center;
    s32 unknown[4];
    u8 PrimitiveType;
    u8 PrimitiveIdentifier; // should be 0x03
    u16 Marker1F; // should be 1F 00
    s32 unknown2[2];
    Color FG_color;
    Color BG_color;
    StylePrefix style;
    s32 unknown3[2];
    Radius radius; 
};

struct Bezier {
    _d("Bezier");
};

struct Text {
    _d("Text");
};

struct Symbol {
    _d("Symbol");
};

struct Polygon {
    _d("Polygon");
};

struct Origin {
    _d("Origin");
};

struct Primitive {
    u8 marker;
    
    if ( (marker != 0x20) && (marker != 0xFF) ) {
        // marker 0x20 = valid record start
        // marker 0xFF = valis record end
        std::print("Wrong primitive marker! 0x20 / 0xFF, but is {:#x}", marker);    
        break;    
    }
    
    if (marker == 0xFF)
        break;
    
    // padding[0x30];      // first 0x30 bytes are uninteresting
    PrimitiveType type @ ($ + 0x14);


    match (type) {
        (PrimitiveType::Line):
            Line data;
        (PrimitiveType::Box):
            Box data;
        (PrimitiveType::Circle):
            Circle data;
        (PrimitiveType::Bezier):
            Bezier data;
        (PrimitiveType::Text):
            Text data;
        (PrimitiveType::Symbol):
            Symbol data;
        (PrimitiveType::Polygon):
            Polygon data;
        (PrimitiveType::Origin):
            Origin data;
    }
};


Primitive primitive_at_0x2F @ 0x2F;

It find and fills basically all values for a Circle shape:

Thank you @WerWolv for the help, and thank you for this awesome software! I have used it only for a week but I am amazed how powerful it is.