Skip to content

Reorganize sample files used in tests. #43

@beporter

Description

@beporter

The codesniffer is unique in that it's not really a binary "yes/no" detector. In fact, it works in a strange kind of 3-way state in most cases:

"This must be X" vs "This must not be X" vs "This must be x^" vs "This must not be x^" vs "I don't care really."

Or using an example, you can write a sniff that dictates the opening brace must not appear "on the same line," but that doesn't necessarily preclude it from being two lines down:

if (...) { // fail
  //...
}

//vs

if (...) // pass
{
  //...
}

//vs

if (...) // pass(?)

{
  //...
}

Writing a rule in a "blacklist" fashion like this allows you to prevent a single bad behavior while not specifying a single preferred behavior. The alternative is a whitelist style rule that says, "this one way is accepted, all others are errors". Depending on what you want to allow and how flexible you want your standard to be, one or the other (or both) will be most appropriate in each case, but this makes testing hard.

In fact this typically means that good test cases have two sides to them: a file (or files) that fail a specific sniff as example(s) of breaking a rule, and other file(s) that demonstrates "proper" or "all acceptable" behavior that passes all sniffs.

To aid in understanding and reinforcing this dynamic, I'm suggesting grouping the sample files we test the ruleset against into folders that make their purpose a little more explicit: snifftests/files/must/ and snifftests/files/must_not/. There's more information in the README about the intended purpose of this, but the basic idea is that whitelist tests for rules that enforce a single specific style go in must/ and blacklist tests that ban a single specific behavior go in must_not/.

In fact, the README could probably stand to be updated to include a little more of this "theory" explanation too.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions