ZODB and Python 3.6 (Zope Object Database)

NOSql solutions (object databases, graph databases, document databases, column store etc. ) have been around for several years becoming mature and stable. I’m discovering many NoSQL solutions I tried early in the trend; years ago, are now defunct.

One first to NOSql solutions was ZODB and several years later it still remains. I never got around to looking at ZODB until now and wanted to give it a try on Python 3.6. So, here are my notes.

I would like to point-out; I am assuming readers understand what the term NOSql means and have enough understanding of python to follow code examples. The ZODB documentation can be found at: ZODB.org and is what I used to piece together my notes and arrive at a working example.

Please keep in mind; this is an example only; not production ready code and is not meant for anything but educational purposes and generic notes.

The following steps assume you have Python 3.6 installed or are using a matching virtualenv and ZODB installed.

1. main.py

Create a main.py file or what ever name you like.
At the top add the necessary imports

import ZODB
import ZODB.FileStorage
from ZODB import DB
import transaction
import BTrees.OOBTree
from person import Person # this will be created later

2. Create a ZODB instance by adding a function

def create_empty_database():
    storage = ZODB.FileStorage.FileStorage('data.fs')
    db = ZODB.DB(storage)
    connection = db.open()
    root = connection.root

This function will create all the data files needed by ZODB in the current directory. You may name it what ever you want.

3. Add an object place holder to root container using this function.

def add_object_placeholder():
    object_name = "people"  # this is the name of the person object (class) container
    storage = ZODB.FileStorage.FileStorage('data.fs')
    db = ZODB.DB(storage)
    connection = db.open()
    root = connection.root
    setattr(root, object_name, BTrees.OOBTree.BTree())

This function opens the existing database and adds a container for our person class objects.

4. Create a function to add some objects of type person (class) to the people container.

def add_people():
    storage = ZODB.FileStorage.FileStorage('data.fs')
    db = ZODB.DB(storage)
    connection = db.open()
    root = connection.root
    root.people['person1'] = Person('Sally', 'Smith')
    root.people['person2'] = Person('Thomas', 'Thumb')
    root.people['person3'] = Person('John', 'Doe')

5. Create a function to read inserted objects

def read_people():
    storage = ZODB.FileStorage.FileStorage('data.fs')
    db = DB(storage)
    connection = db.open()
    root = connection.root()
    people = root['people']
    for person in people.values():
        first_name = person.first_name
        last_name = person.last_name

Note: This function doesn’t return anything on-purpose as I’m using my debugger to step through and view the variable values. You can complete the for-loop to build a return output if you want.

6. Create the person class object
To create the person object either create a class in a new .py file or put it in the main.py it doesn’t really matter but I prefer classes to belong in a separate file.

So, create a person.py file and at the top import the ZODB “persistent” and create the class definition like this:

import persistent
class Person(persistent.Persistent):
    def __init__(self, first_name, last_name):
        self.first_name = first_name
        self.last_name = last_name

7. Call and run the functions at the bottom of main.py in the following order:

read_people() # don't forget to set breakpoints so you can see the results.

So, at this point you should have two files (main.py & person.py) and a set of database files with records inserted.

Here is what I learned

  1. The database can be named anything.
  2. Containers are actually named attributes attached to the root object.
  3. The attribute name (of the container) does not need to match the objects it contains but I found it useful to use plural form. A people container to contain instances of person.
  4. It is not necessary to open the database in every function. It may be opened at a outer scope if needed.
  5. Using setattr() will allow you to dynamically create named containers (assumes you would re-factor the function to accept a parameter).
  6. Creation of the database does not require a transaction commit but adding named containers does.
  7. Not 100% sure on this one, but, it appears opening a connection does not read the data. Calling .values() does so you need an iterator.
  8. No SQL-like query language only pure python methods for filtering and querying.
  9. Many of the steps above can be re-factored to reduce the number of functions / methods. For example you do not need to first create an empty database as it will be automatically created if it doesn’t exist when you try to connect. This behavior is nearly identical to how SQLite creates new files on-the-fly.
  10. Each record is an instance object so no plumbing code is needed. This is what object databases are designed to do so no surprise here. This means an ORM (object relational mapper) is not needed.

For anyone new to Python or coming from C# / Java; ZODB is making use of python instance attributes to organize data containers attached to the root object. To clarify, in Python a class instance may have attributes removed / added after object initialization. This is not done in C# / Java, so it takes some getting used to in Python. As an example consider this C# code:

Person p = new Person();  
p.myNewProperty = SomeValue; // Compile time error

You cannot use the instance variable “p” to add attributes (properties), however in Python you can add p.x or p.y or p.z etc. as attributes and assign values.

Personally, I prefer not to use dynamically added attributes to class instances if I can avoid it; because, when I create a stand-alone class I hope to have modeled it well-enough to avoid it. I generally expect to see a complete object definition contained within the scope of the class definition not elsewhere in the code.

Examples and information from around the web:
Groups Google Zodb
2016 Plone Conf