Discover the common root causes of test flakiness and learn what truly sets apart good test design from bad. This session is a must-attend for anyone looking to improve their testing strategy and achieve more reliable results. 
Don’t miss out—register now and join the live session 
Got questions? Drop them in the thread below and get them answered live! 
Hi there,
If you couldn’t catch the session live, don’t worry! You can watch the recording here:
Here are some of the Q&As from this session:
When your test suite is in a bad state, how do you prioritize which tests to fix first? What criteria should be used to make these decisions?
Filip: Prioritize fixing tests based on impact and frequency of failures. Focus on high-priority tests that cover critical functionality or core features. Consider the severity of failures and how they affect the overall quality. Use metrics like test execution frequency and failure rate to prioritize which tests to address first.
Once you’ve started fixing your tests, how do you ensure long-term health and prevent them from becoming unflaked again in the future?
Filip: Ensure long-term health by adopting best practices for test design, such as using mocking and stubbing to isolate tests. Implement regular maintenance and review practices to address issues proactively. Use test automation frameworks that provide robust handling of asynchronous operations and frequent updates to minimize flakiness.
In your view, what are the key design principles that separate effective test designs from those prone to flakiness?
Filip: Effective test designs are stable, isolated, and predictable. They use mocking to avoid dependencies on external systems and handle asynchronous operations gracefully. Avoid reliance on hard-coded values and ensure tests are data-driven where possible. Tests should also be reliable and have clear setup and teardown processes.
What are the most common signs that a test suite needs improvement?
Filip: Common signs include high flakiness rates, slow execution times, and frequent false positives or negatives. A test suite may also need improvement if it has high maintenance costs, low coverage of critical features, or if test results are rarely used for actionable insights.
What strategies do you recommend for teams to move from focusing on test flakiness to addressing app-level flakiness?
Filip: Address app-level flakiness by improving application stability and resolving underlying issues that cause flaky behavior. Focus on robust test designs that handle edge cases and asynchronous behavior well. Implement end-to-end monitoring to detect and address app-level issues before they impact tests.
How to utilize AI to reduce test flakiness?
Filip: Utilize AI by incorporating machine learning algorithms to analyze patterns and predict flaky tests. AI can help in identifying test environments or conditions that contribute to flakiness. Use AI-driven test optimization tools to dynamically adjust test execution strategies and handle variability more effectively.
Now, lets see some of the unanswered questions.
What are the most common signs that your test suite is in need of serious improvement or refactoring?
How does the concept of ‘app flakiness’ differ from ‘test flakiness,’ and why is it important for teams to shift their focus toward the stability of the application itself?
What are the most common environmental factors that contribute to test flakiness, particularly in CI environments, and how can these be mitigated?
How can we prioritize which tests to refactor or remove when faced with a large and unwieldy test suite?
How can you simplify and clean up overly complex or brittle tests to make them more maintainable?
What tools or techniques can help with systematically untangling a complex and unreliable test suite?
A chicken or the egg question: Should team culture or test flakiness be prioritized first? Or does it, by necessity, require both to be addressed at once?
What role does test coverage play in improving test quality, and how can we use coverage reports effectively?
Can chaos engineering uncover flakes? Perhaps more importantly, could that practice possibly be a cause of them?
How would you best suggest us to approach the QA best with your experience and mindset?
How could automation and AI tools can uncover instability in test suites and provide guidance for addressing it?
Best approaches for unfracking using AI ?
