Engineering quality in software

High-level testing verifies the entire software system, from user interactions with the interface to API calls and database persistence operations such as insertions and deletions.

A, B

The requirements and context are specifications, system requirements, user requirements, bug reports.

C

Desired input-output pairs are used to fine-tune the model, whether it is to create the test scenarios (D) or the test data (F).

D

Test scenarios is the set of actions to be performed, which can be expressed in natural language or in a structured language such as Gherkin.

G, H

Examples of the desired output i.e. test cases from the required domain and/ or programming language are prompted to the LLM.

I, J

The final output are high level test cases with steps, inputs and desired outputs. The output is generated by iterating over the model multiple times, each time increasing the context provided.

To conclude

These high level test case generation techniques would need to be assessed based on money-time cost, human-effort needed, correctness of the test cases generated, and bugs detected.