Test augmentation involves improving the existing test suites or the overall testing process. The process is iterative and is guided by an adequacy analysis based on quality attributes.
There are a few ways in which the augmentation can be accomplished.
One way is to evaluate the quality of the current test suite against the SUT (system under test). The adequacy analysis on the observed results guides the prompts that will be submitted to the LLM which will output a collection of new test cases. The quality of these new test cases is again evaluated (possibly also taking into account the original test suite). This generation and evaluation of test cases will be repeated iteratively until the expected adequacy criteria are met.
The adequacy criteria could refer to quality criteria such as test coverage, fault detection capability. In both cases, adequacy analysis results can be used to prompt LLM to generate test cases that cover a specific but not yet considered function or portion of the SUT.
Another way is to use LLM as a complement to existing automatic test case generation process e.g. an LLM can be used to generate new test cases once a traditinal SBST (search based software testing) has stalled after exhausting its coverage. These new test cases can be fed in to subsequent iterations of the SBST process.
LLMs can be used to improve the test process e.g. by enhancing the OpenAPI specifications of REST services, updating potentially obsolete test cases automatically, and predicting whether a test case can be flaky or not.
To concludeThese test augmentation would need to be assessed based on money-time cost, human-effort needed, correctness of the test cases generated, and bugs detected.