Software Industry is changing at a very rapid pace and ‘Time to Market’ has become more critical than ever before.
‘Waterfall Model’ was one of the most prominent software models that were used by organizations, but it had a major drawback – testing was done at every stage of software development due to which highly critical bugs were left un-noticed till the software reached the production stage. Such a scenario could also lead to business losses, in case ‘buggy software’ is shipped to the end-customer, a problem that is present in every traditional ‘develop-first-then-test’ models.
Differences Between Continuous Testing and Shift-Left Testing
Now that we are aware of Shift-Left testing and Continuous testing, let’s have a look at some of the key differences between the two
1. Testing beyond UI: In Shift-Left testing, there is an increased amount of focus on UI & UX testing before the release. However, a web application or website is much more than impressive UI. Along with effective UI, performance also plays an important role in shaping up the product. As a developer, you have to ensure that transition between the pages is smooth/glitch free, the right mix of web & local components i.e. hybrid approach is used, CPU consumption is minimal, Thermal requirements are met, etc. These factors which go beyond the ‘look & feel’ of a website/web application are taken into consideration during Continuous testing. Hence, Continuous testing is more optimal since it takes into account ‘UI, UX, and performance metrics’ which are the key elements for any kind of product (website/web application)
2. More Real-time scenarios in testing: It is a known fact that bugs can occur at any stage of product development. This makes ‘product testing’ an integral part of the development life-cycle. Shift-Left testing focusses on testing at early stages i.e. pre-production stage of development, whereas Continuous testing focusses on pre-production to post-production part of the development cycle. There are some bugs that occur only when your system is being tested under extreme conditions and the possibility of unearthing such kind of issues at an early stage is unlikely since the ‘components/modules’ are still under design/development.
This is where Continuous testing can play a pivotal role in performing thorough testing of the product. Consider a mobile application (app); the app might perform well when it is executing under ‘the ideal conditions’ i.e. CPU cores are executing at the highest frequency, GPU is executing at a high frequency, etc.; whereas the performance would drastically vary if the testing is performed on mobile devices that have low configuration. Along with the performance, the look & feel of a web-app might vary from one device to another e.g. the manner in which a ‘page/component’ of a web-app is rendered & displayed on a ‘mobile device’ that has Android-O could be significantly different when the same page is being rendered on a ‘tablet device’ that has Android-O. There is a huge amount of fragmentation as far as mobile & tablet devices are concerned and it becomes more complicated with the different versions of OS’s that exist for the devices. In order to ensure that your web-app works perfectly fine on the designated devices and respective browsers, a thorough cross browser testing has to be performed; something that cannot be done on a web-app which is still in the pre-production phase. This is where Continuous testing plays a pivotal role since it takes real-time scenarios into consideration.
3. Increased learnings, more pro-activeness: Shift-Left testing empowers the testing/verification team since it provides an opportunity to the testers to work in close collaboration with the product managers, project managers, developers, other test engineers, etc. This helps in inculcating a ‘development mindset’ within the test team. Such an approach helps them in visualizing issues during the product planning & development phase. Due to this, testers can report issues at early stages of the product development cycle. They feel more motivated since there is a learning opportunity involved in each interaction. In some cases, the thought process of the test team can become ‘linear’ in approach.
- Continuous testing is performed on a more ‘polished version’ of a product since it is performed at every stage of product development & testing. Hence, the severity & type of issues that a tester comes across during the process of Continuous testing can be very different from those unearthed during Shift-Left testing. This makes the test team more informed about ‘critical issues’ and they can incorporate those learnings at the beginning of each testing cycle. The overall motivation & pro-activeness of the test team is more since there is more learning & they can utilize those learnings for the betterment of the product.
- Continuous Integration & Continuous Deployment: Today’s modern development practices make use of CI (Continuous Integration) and Continuous Deployment (CD). CI is a common practice that is used to simplify the overall ‘software release’ process. CD, on the other hand, ensures that important features & bug-fixes are shipped to the customer on a regular & timely basis. By implementing CI, important changes are merged to the ‘production branch’ on a regular basis & the same changes are shipped to the customer using CD.
- Before the changes are shipped to the end customer, they have to be tested and this is where Continuous Testing can play a vital role in the verification of those changes. As the product development reaches a more ‘mature phase’, the number of CI & CD cycles would also increase. In order to make sure that a quality product is shipped to the customer, continuous test cycles of Continuous testing should also be performed (at every stage of CI & CD).
Original Source: Continuous Testing Using Shift Left Testing Approach