How not to create a new test run when a failed test is retried in a build in lambda test

Product Support
1

I run all my tests in lambda test by creating a build. These tests can occassionally fail so i have added a retry logic. However, when a test is retried, a new test id is created in lambda test because of which now in the build i have both failed and succeeded(retry test succeeded now) tests. This is making the build to be shown as failed as there is a failed test however the test got succeeded on retry. Can someone help me how to deal with this scenario?

Testing Library: robotframework==7.1.1, robotframework-retryfailed==0.2.0(library used for retrying a test)

Language: Python

Attached screenshots for reference

Screenshot 2024-11-20 at 18.13.32
Screenshot 2024-11-20 at 18.13.321870×1076 178 KB
Screenshot 2024-11-20 at 18.13.52
Screenshot 2024-11-20 at 18.13.522492×398 38.6 KB

2

@LambdaTest could you please help me here

3

Hey @varun.dunga, Hope you are doing well!

Yes, you can implement the code using the method provided below to retry the test within the same session. This approach ensures that if a test fails, it will rerun within the same session, avoiding the creation of a new session. As a result, your build will only include the passed test IDs.

SAMPLE CODE:

*** Settings ***
Library    BuiltIn

*** Variables ***
${MAX_RETRIES}    3

*** Test Cases ***
Retry Failed Test In Same Session
    Retry Keyword    My Test Step    retries=${MAX_RETRIES}    retry_interval=5s

*** Keywords ***
My Test Step
    [Documentation]    A test step that may fail and needs retrying.
    Log    Performing test step...
    Should Be True    ${False}    # Simulates a failure

Please let us know if you have further query.

4

Hi @varun.dunga

Perhaps I missed this but which part is the sample code we’d need to implement?

We have the issue with the LambdaTest dashboard and we’d like to fix it. Our test framework is webdriver.io and we use javascript

5

Hey @Sherrylene.Gaucivirginmediao2,

Please find below our Robot sample repo link:

Also I have attached the retry sample code for the same:

*** Settings ***
Library  Selenium2Library
Library  BuiltIn

*** Variables ***
@{_tmp}
    ...  browserName: %{browserName},
    ...  platform: %{platform},
    ...  version: %{version},
    ...  name: RobotFramework Lambda Test

${BROWSER}          %{ROBOT_BROWSER}
${CAPABILITIES}     ${EMPTY.join(${_tmp})}
${KEY}              <YOUR_LAMBDATEST_USERNAME>:<YOUR_LAMBDATEST_ACCESS_KEY>
${REMOTE_URL}       https://${KEY}@hub.lambdatest.com/wd/hub
${MAX_RETRIES}      3

*** Test Cases ***
Retry Failed Test In Same Session
    Retry Keyword    My Test Step    retries=${MAX_RETRIES}    retry_interval=5s

LambdaTest Browser Test
    Open test browser
    [Teardown]    Close test browser

*** Keywords ***
My Test Step
    [Documentation]    A test step that may fail and needs retrying.
    Log    Performing test step...
    Should Be True    ${False}    # Simulates a failure

Open test browser
    Open browser  https://lambdatest.github.io/sample-todo-app/  browser=${BROWSER}
    ...  remote_url=${REMOTE_URL}
    ...  desired_capabilities=${CAPABILITIES}

Close test browser
    Close all browsers

Regards, Sarthak Tyagi

6

Hey @Sherrylene.Gaucivirginmediao2 ,

For your second issue related to webdriver.io, we will contact you over email.

Thanks & Regards, Sarthak Tyagi