Discussion on Testing Your Test by Andres Sacco | Testμ 2024

Andres Sacco will explain how to measure test effectiveness using the Pitest tool. :test_tube:

Learn to evaluate test quality and address issues that arise when code changes but tests still pass.

Not registered yet? Don’t miss out—secure your free tickets and register now.

Already registered? Share your questions in the thread below :point_down:

Hi there,

If you couldn’t catch the session live, don’t worry! You can watch the recording here:

Here are some of the Q&As from this session:

How to we stop testers from writing contrived tests that don’t actually test anything, but trip the sensors of the automated coverage tools none-the-less?

Andres Sacco: Implement code reviews and testing standards, use robust test case design techniques, and regularly analyze test effectiveness to ensure tests provide real value.

How can developers incorporate mutation testing into their existing CI/CD pipelines?

Andres Sacco: Integrate mutation testing tools like Stryker into your CI/CD pipeline to run tests against mutated code and analyze the effectiveness of your tests as part of your build and deployment process.

Here are some unanswered questions that were asked in the session:

How can testers ensure that their tests are comprehensive and cover all necessary scenarios?

What techniques do you employ to evaluate the coverage and completeness of your test suite?

How can I make an affordable test team?

How do you make the transition as a tester to test AI apps from testing mobile/web apps?

How do we test teams so that we avoid testing ourselves? :grin:

What are the most reliable metrics or approaches to accurately measure the effectiveness of our tests?

Which are the top 5 metrics?

Are there instances where manual testing is the most effective approach to “test a test”?

Can failure injection testing be a useful way to test a test? If so, in what kinds of tests and use cases?

Can we stop testers from writing contrived tests?

Are there any additional future use cases we might see besides those presented here?

What are the limitations of mutation testing, and how can they be addressed?

What are the limitations of mutation testing?

What are the limitations and challenges of mutation testing?

Testers can ensure comprehensive coverage by using a combination of techniques like boundary testing, edge cases, exploratory testing, and analyzing historical bugs. Tools like code coverage analyzers (e.g., Pitest) also help identify untested areas of the code.

I recommend using mutation testing tools like Pitest, which helps evaluate the effectiveness of your test cases by introducing small code changes (mutants) and seeing if the tests catch them. Pair this with code coverage tools to ensure all branches are tested.