Issue with loading Union related archives

[Try]

More FYI, rather than actual issues. When OpenGothic load *.vdf from file system, game crashes with exception:

 for(auto& i:archives)
    inst->gothicAssets.merge(phoenix::vdf_file::open(i.name), false); // buffer_underflow here

Callstack:

0x00007ff67c54ed76: dbg::call_stack<64u>::collect(unsigned int) in C:\Users\mrsta\Downloads\opengothic_win\Gothic2Notr.exe
0x00007ff67c3cf1dc: CrashLog::dumpStack(char const*) in C:\Users\mrsta\Downloads\opengothic_win\Gothic2Notr.exe
0x00007ff67c3cf738: terminateHandler() in C:\Users\mrsta\Downloads\opengothic_win\Gothic2Notr.exe
0x00007fff739263f6: ZN10__cxxabiv111__terminateEPFvvE in C:\Users\mrsta\Downloads\opengothic_win\libstdc++-6.dll
0x00007fff73a22d43: ZSt9terminatev in C:\Users\mrsta\Downloads\opengothic_win\libstdc++-6.dll
0x00007fff73a2bf56: _cxa_throw in C:\Users\mrsta\Downloads\opengothic_win\libstdc++-6.dll
0x00007ff67c430819: phoenix::buffer::slice(unsigned long long, unsigned long long) const in C:\Users\mrsta\Downloads\opengothic_win\Gothic2Notr.exe
0x00007ff67c47a368: phoenix::vdf_entry::read(phoenix::buffer&, unsigned int) in C:\Users\mrsta\Downloads\opengothic_win\Gothic2Notr.exe
0x00007ff67c47aa5b: phoenix::vdf_file::open(phoenix::buffer&) in C:\Users\mrsta\Downloads\opengothic_win\Gothic2Notr.exe
0x00007ff67c47ac87: phoenix::vdf_file::open(std::filesystem::__cxx11::path const&) in C:\Users\mrsta\Downloads\opengothic_win\Gothic2Notr.exe
0x00007ff67c3b7c39: Resources::loadVdfs(std::vector<std::__cxx11::basic_string<char16_t, std::char_traits<char16_t>, std::allocator<char16_t> >, std::allocator<std::__cxx11::basic_string<char16_t, std::char_traits<char16_t>, std::allocator<char16_t> > > > const&) in C:\Users\mrsta\Downloads\opengothic_win\Gothic2Notr.exe

I've added simple try-catch handler to workaround issue, so here is the list of files that are failed to load:

unable to load archive: "D:/Games/Gothic II/Data/$Templates$/asmcl.dll"
unable to load archive: "D:/Games/Gothic II/Data/Union.vdf"
unable to load archive: "D:/Games/Gothic II/Data/$Templates$/ZUNIONUTILS.DLL"
unable to load archive: "D:/Games/Gothic II/Data/$Templates$/ZBINKFIX.DLL"
unable to load archive: "D:/Games/Gothic II/Data/$Templates$/ZMOUSEFIX.DLL"

Proposed change:
Can you add new dedicated type of exception, if *.vdf file signature does not match VDF_SIGNATURE_G* ?
Thanks!

[lmichaelis]

Hi, thanks I'll look at it. I might just implement a proper parsing routine if I can find documentation :)

[Try]

proper parsing routine

Not for this: ZUNIONUTILS.DLL :)

[lmichaelis]

@Try I've added the exception as requested. You can now catch phoenix::vdfs_signature_error for this specific error. Could you send over the VDF file that caused the error so I can investigate?

[Try]

Could you send over the VDF file that caused the error so I can investigate?

Unfortunately can't. This bug was reported on worldofplayer.ru forum, so don't have the file on my side.
Lets assume for now that issue was fixed :)

[lmichaelis]

I see. I'd still like to implement it so I'll keep this issue open :>

[Try]

Bump: got the Union.vdf file.
Union.zip

[lmichaelis]

Thanks, that helps out a lot!

I managed to find out, that it's a single entry in the VDF which is causing the problem: SYSTEM/ZBINKFIXOLD.DLL.

The entire archive looks like this:

_WORK
_WORK/DATA
_WORK/DATA/ANIMS
_WORK/DATA/ANIMS/_COMPILED
_WORK/DATA/ANIMS/_COMPILED/HUMANS_SKELETON-T_SPAWN.MAN
_WORK/DATA/ANIMS/_COMPILED/INVALID_SOURCE_FILE.MDL
_WORK/DATA/ANIMS/_COMPILED/INVALID_SOURCE_FILE.MDM
_WORK/DATA/ANIMS/_COMPILED/INVALID_SOURCE_FILE.MMB
_WORK/DATA/MESHES
_WORK/DATA/MESHES/_COMPILED
_WORK/DATA/MESHES/_COMPILED/INVALID_SOURCE_FILE.MRM
_WORK/DATA/SOUND
_WORK/DATA/SOUND/INVALID_SOURCE_FILE.OGG
_WORK/DATA/SOUND/INVALID_SOURCE_FILE.WAV
_WORK/DATA/TEXTURES
_WORK/DATA/TEXTURES/_COMPILED
_WORK/DATA/TEXTURES/_COMPILED/INVALID_SOURCE_FILE-C.TEX
SYSTEM
SYSTEM/ASMCL.DLL
SYSTEM/AUTORUN
SYSTEM/AUTORUN/MENU
SYSTEM/AUTORUN/MENU/ZUNIONMENU.D
SYSTEM/AUTORUN/UTILS
SYSTEM/AUTORUN/UTILS/ZUNIONTRIGGER.D
SYSTEM/AUTORUN/UTILS/ZUNIONVOB.D
SYSTEM/AUTORUN/ZBINKFIX.DLL
SYSTEM/AUTORUN/ZMOUSEFIX.DLL
SYSTEM/AUTORUN/ZPARSEREXTENDER.DLL
SYSTEM/AUTORUN/ZUNIONUTILS.DLL
SYSTEM/M3D
SYSTEM/M3D/MSSA3D.M3D
SYSTEM/M3D/MSSDS3D.M3D
SYSTEM/M3D/MSSDS3DH.M3D
SYSTEM/M3D/MSSDX7.M3D
SYSTEM/M3D/MSSEAX.M3D
SYSTEM/M3D/MSSRSX.M3D
SYSTEM/M3D/MSSSOFT.M3D
SYSTEM/OSTEAMWORKS.DLL
SYSTEM/STEAM_API.DLL
SYSTEM/ZBINKFIXOLD.DLL

That being said, there seems to be a lot of entropy in that file. When extracted as-is, all content is just random bytes and no structure can be found. The file tool reports a plain "data" format for all files in the archive.

I assume its some sort of compression or encryption but if it's compression I can't tell the type. I'll have to look into the Vdfs32g.dll Union ships with. That could take some time :)

Btw, I apologize for my absence this past week, I've been ill.

[Try]

Btw, I apologize for my absence this past week, I've been ill.

Welcome back :D

When extracted as-is, all content is just random bytes and no structure can be found.

Yes, most likely it's compressed. Yet it's probably not worth it to chaise after compression algorithm - just as long as it's not crashing it's fine.