LambdaTest Capabilities, Questions, and Clarifications for the Uninitiated

General Discussions TestAutomation
, ,
#1

Configuration Questions

#general-discussion #product-support #shell-scripting

I am trying to decide whether to use LambdaTest four our cloud based browser coverage testing (and later mobile testing). I have used SauceLabs and BrowserStack in the past so am pretty familiar with setting up projects.

LambdaTest has a great interface but the docs are missing a lot of information and don’t have many examples for how to pass values (i.e. like how to pass cypress environment variables or what glob rules are used for glob pattern expansion when specifying files to exclude).

I will post answers to my own questions once I hear back from technical support but wanted to reach out to the community too.

Are project files stored and cached so that future upload times are reduced?

Zipping up the whole project and sending it with every test run is wasteful. Does anyone know if LambdaTest stores the most recent project.zip file and checks if it’s identical to the incoming file? I would expect this is the case but it seems like the project is uploaded every time even with no changes to the project files.

I would think the test command does something like this:

  1. zip up project files (but not user ignored or Lambda ignored files)
  2. check SHA checksum value of project.zip
  3. compare SHA checksum value to last project.zip on the server
  4. if they are equal, skip uploading project.zip and upload tests.zip only

Is that what happens?

Are user ignored files output to the console when matches for the excludeFiles glob pattern are found? Can multiple globs be passed?

No matter what glob pattern I specify the output for ignored files stays the same - even when using a simple simple glob pattern like /dist/**. There are also no examples in the docs as to how to pass glob patterns and it’s not clear whether or not multiple patterns can be used.

LambdaTest errors if specifying a Node version above v16

Our package.json requires a specific version of node and I can’t get tests to run due to the Node version we require being above the Node version used by Cypress LambdaTest. Is there a way to specify the Node version that LambdaTest will use? For example by running nvm use v18.9.1 in a pre-run script?

install-npm-deps: npm install from package.json failed, due to : npm ERR! code EBADENGINE npm ERR! engine Unsupported engine npm ERR! engine Not compatible with your version of node/npm: redacted@0.0.584 npm ERR! notsup Not compatible with your version of node/npm: redacted@0.0.584 npm ERR! notsup Required: {"npm":">=8.6.0","node":">=18.6.0 <=18.9.1"} npm ERR! notsup Actual: {"npm":"8.19.2","node":"v16.18.1"} npm ERR! A complete log of this run can be found in: npm ERR! /Users/ltuser/.npm/_logs/2023-02-09T01_15_59_056Z-debug-0.log

Can LambdaTest be made to serve static files as part of a pre-run script?

When LambdaTest zips up our project it includes a minified, bundled version of our application (an ESM module). This is literally the only file that needs to be served in order to be able to run our Cypress tests against local html pages.

I want to know if it’s possible to run a pre-run script like so:

#!/usr/bin/env bash
echo "Changing into elements dist directory and starting http-server..."
cd dist/libs/elements/dist && npx http-server --gzip --cors -c-1

Which would install and run http-server so that the bundled local .html files can be used. This would be much easier than building and publishing changes to our staging server every time we want to run browser tests and is a much more attractive solution than reverse tunnelling into a local machine.

What is the execution order for running tests, installing node dependencies, and running pre and post run scripts?

So far my guess is this:

  1. developer runs lambdatest-cypress run
  2. lambdatest binary zips up the project and test files (ignoring default and user specified files)
  3. lambdatest uploads the project.zip and test.zip files
  4. lambdatest unzips the project files and then the test files on their server
  5. lambdatest runs any pre-run script(s) that have been uploaded
  6. lambdatest runs npm ci to install dependencies
  7. lambdatest runs npx cypress run which will run all cypress integration tests using the default settings
  8. lambdatest runs any post-run script(s) that have been uploaded

This might not be the case, I will update this thread once I have more information.

Any help is very much appreciated! Thanks.