Engineering quality in software

The diagram above displays the four possible long terms use cases for LLMs applied to software testing.

Universal test theory

Testing still lacks a general theory. It should be possible that assumptions, limitations and capabilities of test techniques and tools could be collected into a coherent and rigorous framework that informs and guides testers in choosing, composing and applying the test approaches that are the most adequate to their situation, thereof reducing dependency of success on tester's skills and background. In other words, better understand the essential and correct information needed for a testing activity.

Test-based modelling

It should be possible to construct a model carrying well-structured information needed for testing purposes; information such as functionality, assumptions about execution environment, possible input constraints, expected behavior for each input, etc. These models can be highly expressive and human-understandable by combining text, graphs and other forms into a multi-modal specification of the software to be tested.

100% automated testing

It should be possible to have an integrated test environment which monitors the progress of software development or maintenance to detect the need for testing and is able to autonomously manage the setting configuration, execution, test reporting as well as bug localization and repair where needed. It naturally follows that the whole process needs to be automated rather than individual processes in isolation.

Efficacy-maximized test engineering

Efficacy denotes both efficiency and effectiveness. It should be possible to engineer viable methods, processes, and tools to develop high-quality software in a cost-effective way.

To conclude

A more structured and well thought way of leveraging LLMs for software testing needs to be brought about and this article is a basic step towards that.