Cycle over array of different types and sizes

[k0tran]

Hello everyone, I'm new to ImHex and PatternLanguage. Recently tried implementing RPM package format (http://ftp.rpm.org/max-rpm/s1-rpm-file-format-rpm-file-format.html#FTN.AEN14176) but I was unable to express some of RPM struct patterns.

Consider this working and completely correct example:

enum IndexType : u8 {
    _null = 0,
    _char = 1,
    _int8 = 2,
    _int16 = 3,
    _int32 = 4,
    _int64 = 5,
    _string = 6,
    _bin = 7,
};

struct Index {
    IndexType type;
    u8 offset;
    u8 count;
};

struct String {
    char s[];
};

struct StoreEntry<auto index, auto offset> {
    const u8 p = offset + index.offset;
    const u8 c = index.count;
    match (index.type) {
        (IndexType::_char): char data[c] @ p;
        (IndexType::_int8): s8 data[c] @ p;
        (IndexType::_int16): s16 data[c] @ p;
        (IndexType::_int32): s32 data[c] @ p;
        (IndexType::_int64): s64 data[c] @ p;
        (IndexType::_string): String data @ p;
        (IndexType::_bin): u8 data[c] @ p;
    }
};

struct Test {
    u8 n;
    Index i[n];
    
    const u8 offset = $;
    StoreEntry<i[0], offset> e1;
    StoreEntry<i[1], offset> e2;
    StoreEntry<i[2], offset> e3;
    StoreEntry<i[3], offset> e4;
};

Test t @ 0x00;

// Test hex:
// 04 01 00 02 05 02 01 07 0A 01 06 0B 01 61 61 00 00 00 00 00 00 00 00 00 73 74 72 00 00 00 00 00

TL;DR: we have array of (type, offset, amount). Next for each array element we go to offset and load type amount times

It works fine, but the trick is that there always n StoreEntry's so I tried to optimize it and ended up writing something like this:

struct Store<auto index, auto last, auto offset> {
    const u32 p = offset + index[last].offset;
    const u32 c = index[last].count;
    match (index[last].type) {
        (IndexType::_char): char data[c] @ p;
        (IndexType::_int8): s8 data[c] @ p;
        (IndexType::_int16): s16 data[c] @ p;
        (IndexType::_int32): s32 data[c] @ p;
        (IndexType::_int64): s64 data[c] @ p;
        (IndexType::_string): String data @ p;
        (IndexType::_bin): u8 data[c] @ p;
    }
    if (last == 0)
        break;
    Store<index, last - 1, offset> a;
};

// later call Store<i, n - 1, $> data;

It is not working and highlighting everything wrong. I honestly don't even know if it is possible to express this idea now. It is like using enums, but values are different types and you can't just slap a union and call it a day

[paxcut]

i think you are making it more complicated than it needs be:

• If there are n StoreEntry elements make an array of them and use the std::core::array_index() function to access each array item.

• For this example, the use of templates is unnecessary since all data is available one parent away.

• Finally, all data is contiguous so using the offsets is not needed which also saves you from having to rebase it.

The following resulting code produces the same patterns as the first example posted:

#pragma example 04 01 00 02 05 02 01 07 0A 01 06 0B 01 61 61 00 00 00 00 00 00 00 00 00 73 74 72 00 00 00 00 00
import std.core;

enum IndexType : u8 {
    _null = 0,
    _char = 1,
    _int8 = 2,
    _int16 = 3,
    _int32 = 4,
    _int64 = 5,
    _string = 6,
    _bin = 7,
};

struct Index {
    IndexType type;
    u8 offset;
    u8 count;
};

struct String {
    char s[];
};

struct StoreEntry {
    auto i = std::core::array_index();
    const u8 c = parent.index[i].count;
    match (parent.index[i].type) {
        (IndexType::_char): char data[c];
        (IndexType::_int8): s8 data[c];
        (IndexType::_int16): s16 data[c];
        (IndexType::_int32): s32 data[c];
        (IndexType::_int64): s64 data[c];
        (IndexType::_string): String data;
        (IndexType::_bin): u8 data[c];
    }
};

struct Test {
    u8 n;
    Index index[n];
    StoreEntry e[n];
};

Test t @ 0x00;

[k0tran]

All store is data contiguous and in this example it is in order, but it can also be unordered. For example

#pragma example 04 05 02 01 01 00 02 06 0B 01 07 0A 01 61 61 00 00 00 00 00 00 00 00 00 73 74 72 00 00 00 00 00

But I was able to achieve that introducing parent.offset into your solution, big thanks!

Only updated parts:

struct StoreEntry {
    auto i = std::core::array_index();
    auto c = parent.index[i].count;
    auto p = parent.offset + parent.index[i].offset;
    match (parent.index[i].type) {
        (IndexType::_char): char data[c] @ p;
        (IndexType::_int8): s8 data[c] @ p;
        (IndexType::_int16): s16 data[c] @ p;
        (IndexType::_int32): s32 data[c] @ p;
        (IndexType::_int64): s64 data[c] @ p;
        (IndexType::_string): String data @ p;
        (IndexType::_bin): u8 data[c] @ p;
    }
};

struct Test {
    u8 n;
    Index index[n];
    auto offset = $;
    StoreEntry e[n];
    $ += offset;
};

Leaving this if everyone will be interested

[paxcut]

There are advantages to not using the @ operator inside structs besides making the code more readable. Placing patterns inside structs generate calculated pointers that have zero size. If the data is contiguous then it is better to find the index associated with the current data by comparing the current address with the corrected offset instead of reading them in the order they appear.

[k0tran]

That is true, but I assumed that looping over 53 indexes over and over again to find next chuck is as much expensive as just placing them

Btw in a few minutes will open RPM pattern PR. Would really appreciate you look - I expect that some things will be required to fix