Skip to content

Conversation

@Red-GV
Copy link
Collaborator

@Red-GV Red-GV commented Jan 16, 2026

ITBench v2 will offer the same faults as v1. However, it will differentiate in a couple of ways:

  1. Consistent methods of accessing fault arguments: The arguments of each fault is defined within the fault library and validated with the scenario spec ahead of time. Meaning, that the arguments for each fault can be easily seen and understood from the docs over the previous need to check the argument_specs.yaml file that was a part of the faults role.
  2. argument_specs.yaml does not validate arguments: Covered mainly in point (1). However, for contributors, this means that this file will no longer be the point of contention for merge conflicts. Previously, multiple users adding faults around the same time caused annoyances via merge conflicts. By moving to the JSON library, this file will not be a source of headache for these types of scenarios.
  3. mandatory fault removal is unneeded: Many of the faults are "removed" via deleting namespace. Since fault removal occurs right before application removal, this meant that many of the faults removal processes just contained a message saying to remove the application. Thus the mandatory process for it has been dropped. Removing a fault is often needed for development processes versus actual benchmarking processes.
  4. live argument validation: Arguments were presumed to be correct within the environment. This has changed to now faults having asserts so that the live environment can be validated against.

The actual behavior of most faults remained unchanged. While the v2 injector and remover cannot interact with a v1 injector and remover, the actual implementation is largerly the same.

Faults will different behavior will be described below:

  • OTel Demo Flagd: Workloads will no longer be restarted via the injection. It is expected that v2 versions of these faults use the waiters in order to restart the necessary workloads. This allows us to not maintain a list of workloads to restart, which can change due to otel-demo being an application that still receives active development.

Red-GV added 12 commits January 16, 2026 11:02
Signed-off-by: Gerard Vanloo <gerard.vanloo@ibm.com>
Signed-off-by: Gerard Vanloo <gerard.vanloo@ibm.com>
…ation

Signed-off-by: Gerard Vanloo <gerard.vanloo@ibm.com>
…lementation

Signed-off-by: Gerard Vanloo <gerard.vanloo@ibm.com>
Signed-off-by: Gerard Vanloo <gerard.vanloo@ibm.com>
Signed-off-by: Gerard Vanloo <gerard.vanloo@ibm.com>
… implementation

Signed-off-by: Gerard Vanloo <gerard.vanloo@ibm.com>
Signed-off-by: Gerard Vanloo <gerard.vanloo@ibm.com>
Signed-off-by: Gerard Vanloo <gerard.vanloo@ibm.com>
Signed-off-by: Gerard Vanloo <gerard.vanloo@ibm.com>
…tion

Signed-off-by: Gerard Vanloo <gerard.vanloo@ibm.com>
…ntation

Signed-off-by: Gerard Vanloo <gerard.vanloo@ibm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants