Winter Rant

Are you aware of research work or new products being created for software testing with AI? [Threads] [Web Archive]

Interestingly, Software Test Suite Generation is something that Software Engineering Research communities have been tackling for a while now! You have tools like Evosuite (evosuite.org) and Pex (microsoft.com/en-us…) that use tech like Genetic/Evolutionary algorithms or Symbolic Execution. The question always has been: would real human devs accept such tests in a pull request and even maintain those tests?

And do those tests accurately capture the original intent of the code-under-test? Results have been a case of mixed-bag. GenAI makes things interesting because it at least pretends to understand intent, although to what extent, I am not sure. I am not sure if it really understand intent. We know that tools like Code Llama and Github Co-pilot can generate test cases (like any other code); but I am curious to understand the applicability of such tests. And how we can push the envelope here.

It is proving to be a really exciting time to be a software engineer!