Initial Work (Post 1)

These posts explore one way to use AI technology to test complex software systems.

The posts are associated with the website http://TestingComplexSystems.com - a resource for software testing practitioners.

The above website makes two key assertions:  

  -- Complex software systems can be effectively tested using custom-built "test platforms." These platforms stress, exercise, and validate software in real time through randomized testing methods.

     Direct testing by an AI instance is currently infeasible due to the complex, high-speed and concurrent operations required. Thus, specialized test platforms are essential.

  -- To simplify and accelerate the creation of test platforms we advocate using a lightweight, Python-based "Global Datastore" (gDS). gDS provides the minimum needed to implement Python test platforms. gDS has consistently demonstrated practical advantages and efficiency in real-world, production environments.

This set of posts introduce a third assertion:

  -- It is feasible to train an AI instance to understand and utilize gDS concepts. Once trained the AI instance can help testing practitioners interactively design and construct test platforms for specific complex systems.

Through two case studies the posts validate that:

  -- gDS is sufficiently straightforward for current AI instances to comprehend and use.

  -- Human practitioners can effectively collaborate with AI instances to build powerful, customized test platforms.

We invite you to follow along as we explore this intersection of AI, Python, and software testing.

Using AI to use gDS to write a test platform to test a complex system is an act of faith. You must believe you can get the job done efficiently and that the effort will reward you by locating defects at a rate and of a kind that can't be done "by hand" or by any any other means available to you.

If you commit to writing a test platform using gDS you need to know how gDS operates and what DBMS complications and limitations are being circumvented by using it. gDS ignores SQL, transactions and persistence. It does make use of the "relational" concept and third-normal-form tables. It instantiates tables in shared memory to allow coordination and communication between the processes and threads used in the test platform. This leaves the way open for test practitioners to build test platforms that do what's necessary, the way it needs to be done, to test the target complex system.

At the core of this series of posts is how an AI instance can be taught to understand gDS's syntax and semantics well enough to allow a test practitioner to create a working test platform.

I am using the "Windsurf" Wave 5 IDE. It costs $15/month and uses an AI instance of your choice. Case Studies 4 and 5 use Windsurf and the Claude 3.5 Sonnet reasoning model.

Case Studies 4 and 5 start out with an "instruction file" describing how the gDS concept works. The file then describes an application schema and some directions about how the appliciation is to work. This instruction file is ingested by Windsurf where it creates and outputs running code.