Updated: 1/6/2025
Testing complex systems
“Testing a software system helps guarantee it meets all of the needs of its creators over its lifetime.”
(Scroll down)
Updated: 1/6/2025
Signed in as:
filler@godaddy.com
“Testing a software system helps guarantee it meets all of the needs of its creators over its lifetime.”
(Scroll down)
Testing a complex software system is essential to ensuring its reliability, its stability, and its long-term operation.
This website coins the term "test platform". A test platform is a custom (Python) program designed to automatically operate, stress and verify the system under test in ways no manual testing can achieve.
A test platform is designed to be used by QE, developers, DevOps, Systems Engineers and all the contributors to the system over its lifetime. Management can also use a test platform as a metric and as a guidepost.
This website's goal is to help the engineering staff and its testing practitioners build exemplary test platforms:
This website asserts that one way to test some complex computer systems is to automatically and continuously subject them to a random combination of functional, load, and stress operations, all the while observing and validating their proper behavior. This is often called "chaos testing".
Stress operations include hardware and software failures, configuration changes, upgrade and downgrade operations, site switchovers and failbacks. Chaos testing becomes more effective if the operation of the system under test is understood and the stress is engineered to target critical operations (e.g. fail a host during a downgrade operation).
Given the stressors above, validating the proper operation of the system should always involve looking for corrupt or missing data. (If not then, when?) This type of testing can also amplify the functional tests if they, or a subset of them are added to the test platform.
This website leaves it to the practitioners to decide if their system qualifies as "complex". It's also left to the practitioners to decide when to stop developing and running tests.
The term “test platform” is a broad concept, similar to “operating system”. In the context of this website, a test platform is a Python program running on a Linux system. The Python / Linux pairing provides specific, easy-to-use functionality not readily available in other language / OS pairings.
A test platform is designed to interface with, drive, load, stress (fail parts of), and verify the complex system being tested. These activities are executed concurrently and in random sequences. It also provides a capable UI to help its test and development users get the most out of working with it.
Given many activities running at the same time a test platform functions like an operating system in many ways. Building a test platform does require attention to some of the concepts found in operating systems. These skills can be learned as needed.
The logic behind developing, running and maintaining a test platform often requires some amount of buy-in by the engineering team. People look at the engineering budget and ask why they need to build and support a complex piece of testing code along with the initial complex system. My experience has been:
Refer to the pull-down menus at the top of the website screen:
https://github.com/talborough/testingComplexSystems
In total this website gives engineering staff something to analyze, something to run, something to draw upon as they size up and build the automation needed for their particular testing situation.
Open today | 09:00 am – 05:00 pm |
Copyright © 2024 Testing Complex Systems - All Rights Reserved.
Powered by GoDaddy