gDS Overview

Significant software systems are typically built around a database / data store / DBMS of some sort. The test platforms referenced in this website all use a Python-specific “global data store” called “gDS”.

gDS is unconvential. It's being presented because it works very well in test platforms. You can review its primary characteristics below. You can also use an alternative to gDS if you so choose.

See Case Study 1 for a tutorial on gDS.

gDS uses Python “list” and “dictionary” variables made global (located in shared memory) by the Python “multiprocessing” library. These variables are used just like any other Python variable. except that their values are seen by all the threads and processes created by the main Python code as it executes. It can be considered connectionless inter-process communication.

The Python “list” and “dictionary” variables can be combined into relationally organized tables and indices (in 3'rd-normal form if desired). 

Together with the naming conventions used in the tables and indices are navigated and dereferenced cleanly and easily with ordinary Python code.

A traditional DBMS uses a "transaction" to manage concurrent access to the data in the tables. gDS offers alternatives to managing concurrency:

1. A single "system lock" is used to provide atomic operations on the data in the tables.
2. Table data is memory-resident; loading data into it at the start does not take that long. Small amounts of "startup data" can be persisted if needed.
3. Data can be organized so that each thread has it's own row of data, reducing the need for locks.

These alternatives are not embedded in gDS but used in the general flow of control in the test platform. 

A code generator, “gDSCodeGen,” converts a gDS “.dd” (schema) file provided by the developer into Python code to define and help manipulate the tables defined by it. The “.py” code produced by gDSCodeGen is consumed by the test platform Python code when it starts up. The .py code can be reviewed separately to provide insight.