Discussion on A Hybrid Approach to Performance Testing Discussion by Marie Cruz | I Testμ 2023

When we discuss performance testing, the usual thing we focus on is testing how well our backend servers handle loads. We tend to think that if we’ve improved our servers and databases, our application’s performance is good to go. :hammer_and_pick:

But this isn’t the whole story. Simply because our servers can provide a response to a certain request doesn’t guarantee that users can see that response on the actual webpage.

Tune in to this insightful session by Marie Cruz on frontend and backend performance testing and why a hybrid approach is needed to ensure your website is performant. :desktop_computer:

Still not registered? Hurry up and grab your free tickets: Register Now!

If you have already registered and up for the session, feel free to post your questions in the thread below :point_down:

Let’s see some of the questions and answers from the session!

Could you delve into the intricacies of how this approach intelligently combines virtual users and real user data to provide a more comprehensive performance analysis?

Marie: This approach involves a strategic combination of virtual users and real user data for a thorough performance analysis. When employing the k6 browser, the method entails scripting for controlled environments generating synthetic data. k6’s integration with Grafana Labs facilitates the utilization of complementary tools.

Striking a balance is crucial; relying solely on synthetic data is inadequate. Tools like the Chrome User Experience report, or Google Search Console are useful for quick overviews. Yet, real user data has limitations—it’s slower to reflect changes. This underscores the need for synthetic tools like k6 to gauge immediate impacts.

Can you please shed some light on using k6 for load testing of SQL or NoSQL databases?

Marie: Utilizing k6 for load testing of databases involves extensions designed for this purpose. On k6’s extensions page, available within k6 Studio, one can explore various sample extensions tailored for load testing different databases.

These extensions empower users to simulate and assess the impact of heavy loads on SQL or NoSQL databases. The community-contributed insights, along with documented scenarios, assist in devising effective load tests and evaluating database performance.

Is there any 1 to 1 migration possible for migrating existing JMeter test cases? If not, do you plan to incorporate the feature to onboard more users?

Marie: The objective of minimizing the resource footprint for browser-level replay is indeed a priority. We focus on gradually enhancing the capabilities of the browser API within k6. The ultimate aim is to align closely with the Playwright API, striving for compatibility. This alignment empowers users to replicate user journey flows primarily at the browser level.

The team is committed to refining this process and ensuring it’s seamless. Our GitHub repository hosts an open issue tracking the APIs still in development. Rest assured, our team is diligently working towards achieving Playwright API compatibility, significantly reducing reliance on protocol-level scripting for automated processes.

Here are some of the unanswered questions picked during the session:

What is an ideal/essential roadmap to start journey in performance testing?

In which test level do you prefer to perform the larger amount of performance testing?

How difficult is to visualize the testing results? and is there a possibility to have comparison between the multiple runs? How easy is to achieve this?

Is there any alerting available with k6?

How should we approach to APIs or business scenario which are very complex in performance testing.

Do you see xk6-browser becoming leverageable at some point to help detect outdated automation at protocol level?

Are there any ongoing efforts to reduce resource footprint for browser-level replay?

When running tests operating at both protocol-level and browser-level, what can we do to avoid doing twice the amount of automation work?

Can k6 provide analytics on resource utilization, such as server CPU and memory usage?

Why k6 extensions are not supported in k6 cloud? Its really blocker, since we need at least xsql extension?

Hi there,

If you couldn’t catch the session live, don’t worry! You can watch the recording here:

Additionally, we’ve got you covered with a detailed session blog:

Hello,

As I attended this session personally, I would love to answer our query on behalf of the speaker.

More concise version of the roadmap for starting a journey in performance testing:

1. Learn Basics: Understand performance testing fundamentals.

2. Choose Tools: Select a suitable testing tool.

3. Set Objectives: Define clear performance goals.

4. Design Scenarios: Create realistic test scenarios.

5. Scripting: Develop test scripts accurately.

6. Execute Tests: Run tests in various environments.

7. Analyze Results: Identify bottlenecks and issues.

8. Tune and Retest: Optimize and rerun tests.

9. Automation: Integrate tests into CI/CD pipelines.

10. Continuous Learning: Stay updated on industry trends.

11. Collaboration: Work closely with cross-functional teams.

12. Documentation: Maintain comprehensive records.

13. Feedback Loop: Gather user and stakeholder feedback.

Adapting this roadmap to your project’s needs is essential for success.

Hello,

Since I was actively involved in this enlightening session, I’m here to provide insights on behalf of the speaker.

Perform most of your performance testing at the system test level. This ensures you evaluate the overall application performance under realistic conditions, addressing issues before they impact users.

Hope this answers your query

Hi,

I was part of this informative session, and I would like to respond on behalf of the speaker.

Visualizing and comparing test results can be complex but manageable with the right tools like Grafana or Kibana. Setting up comparisons between runs may require scripting or specialized tools involving a learning curve.

Hope this answers your query

Hi,

I actively joined this session, and I would address your query on behalf of the speaker.

Yes, k6 provides alerting capabilities through its threshold feature. You can set performance thresholds for various metrics and receive alerts when those thresholds are breached, helping you proactively identify issues during testing.

Hope this answers your query

Hey,

I actively participated in this informative session and am pleased to share my perspective on behalf of the speaker.

When dealing with complex APIs or intricate business scenarios in performance testing, start by understanding the requirements, plan meticulous test scenarios, manage data effectively, and script precise user interactions. Generate varying loads, monitor system performance, and analyze results. Collaborate with teams to implement improvements iteratively, aiming for robust performance in complex scenarios.

Hope this helps you understand this query