Consider using EEST as input tests

[marioevz]

We could generate more tests using EEST tools.

The advantage is that we could write tests in parallel when developing tests for the EIP, and specifically write tests that use the new EIPs functionality to consume a ton of gas, and also mark these tests using a special marker to fill only tests that are relevant to odometer.

We can export tests to the well-known state test format, or we could maybe generate a blockchain test that combines all odometer-marked tests in one big blockchain, and then odometer can benchmark how long it takes for the client to consume the entire blockchain.

First one could work out-of-the-box with current EEST functionality, while the second one requires modifications to the framework but I would think it's not too much.

[marioevz]

Btw this issue is just to start discussion, not a direct request in any way start implementing this.

[raxhvl]

@kevaundray and I briefly discussed something very similar. Totally aligned with it.

[jochem-brouwer]

Hey, this is great! I would suggest to directly focus on supporting blockchain tests and not state tests. AFAIK, state tests only run one tx. This is especially "problematic" for benchmark tests once we introduce a tx gas limit like https://eips.ethereum.org/EIPS/eip-7825

This tool would also be highly beneficial for analysis of EIPs like https://eips.ethereum.org/EIPS/eip-7907 where we have to benchmark the performance impact of increasing the initcode size (and the increase in disk I/O on contracts being pulled from DB or querying stuff like code size (EXTCODESIZE) where we have to first pull the entire code from disk)

Maybe also a slight gotcha, we would like for these benchmarks (especially those heavy on disk I/O) to run on a database with mainnet size, to also account for low-level DB overhead for large databases. (I have almost no knowledge about how databases work, but I would assume that querying a big database is slower than a small database, and this should thus be accounted for in the analysis. A "naive" approach would likely be to just fill X key/value pairs with random data (?))