Creating, Populating, and Accessing gDS tables

Tables in gDS are built from collections of Python “lists.” Each Python list is a column of data in the table. For example:

person_Name = []
person_DOB = []
person_SSN = []
person_RowStatus = []

Python practitioners can see the above code contains no “special” Python syntax. The variables are not special in any way — it's just a plain-vanilla declaration of 4 ordinary lists.

In order to make lists/tables in shared memory, one does the following:

from multiprocessing import Manager
gDSMgr = Manager()

gPerson_Name = gDSMgr.list()
gPerson_DOB = gDSMgr.list()
gPerson_SSN = gDSMgr.list()
gPerson_RowStatus = gDSMgr.list()

The code above creates four list variables in shared memory. The variables do not need to be cited in a Python “global” statement to be used anywhere in the running program or its spawned threads and processes.

To add one “row” of data to the above table/set of lists, use the Python “append” method just like a regular list:

gPerson_Name.append("Tom")
gPerson_DOB.append("1/1/1970")
gPerson_SSN.append("100-20-0300")
gPerson_RowStatus.append(None)

The above table is considered to contain one row of data, and that data is said to reside at “offset zero.” To print one column of that one new row of data:

print (gPerson_SSN[0])

To print out all the data in the table above, one would write in Python:

for personRef in range(len(gPerson_RowStatus)):
print (gPerson_Name[personRef],
gPerson_DOB[personRef],
gPerson_SSN[personRef])

The existence of the RowStatus column is used to indicate the row has been completely filled in by whatever thread/process is adding the row. This convention is a must in an environment where there are multiple paths of execution. (More on this aspect of concurrency below.)

Referencing between gDS tables
==============================

Add two more tables and then two inter-table references to the above table definition:

# Add a Person to House table definition above
gPerson_gHouse_Ref = gDSMgr.list() # Person to House reference

# Add a House table definition
gHouse_StreetName = gDSMgr.list()
gHouse_StreetNumber = gDSMgr.list()
gHouse_gTown_Ref = gDSMgr.list() # House to Town reference

# Add a Town table definition
gTown_Name = gDSMgr.list()
gTown_ZipCode = gDSMgr.list()

Notice the columns with names ending in "_Ref." These columns link the table rows by citing the index of the target row in the target table. Note, there is no “magic” here; the Python code filling in the "_Ref" column value must enter the index of a valid row, or the link must be valid by the time the column's value is actually used.

To expand the "print all data" code from above, use the "_Ref" columns:

for personRef in range(len(gPerson_RowStatus)):

houseRef = gPerson_gHouse_Ref[personRef]
townRef = gHouse_gTown_Ref[houseRef]

print (gPerson_Name[personRef],
gPerson_DOB[personRef],
gPerson_SSN[personRef]),
gHouse_StreetName[houseRef],
gHouse_StreetNumber[houseRef],
gTown_Name[townRef],
gTown_ZipCode[townRef])

Note that rather than using intermediate values for the references, the code can be expanded out using cascading references such as:

gHouse_StreetName[gPerson_gHouse_Ref[personRef]]