Engineering quality in software

Test agents are lightweight clients that interact with the user and automate testing tasks, such as test suite generation, test configuration, test execution, security testing; or serve as an interface between humans and a broad range of testing tasks..

A, B

The test engineer provides the context to the agent, requesting the agent to perform a task or initiate a process with a reference to a project or its URI.

C

The test agent can prompt the LLM using custom prompts and retrieve the answers.

D

Alternatively, the request can be redirected to a more suitable agent, and waiting for its response.

G

The agent/ LLM's response can be either fed back to the test engineer or to separate actions such as generating, configuring and executing test cases..

To conclude

These agents will need to be evaluated based on context-dependent benchmarks e.g. when generating test cases, track the percentage of test cases which pass or build; when generating security test cases, track the number of successful exploits.