SQLObject Developer Guide

Contents

• SQLObject Developer Guide
  • Development Installation
  • Architecture
    • Columns, validators and converters
  • Branch workflow
  • Style Guide
  • Testing
  • Documentation
  • Contributing

These are some notes on developing SQLObject.

Development Installation

First install FormEncode:

$ git clone git://github.com/formencode/formencode.git
$ cd formencode
$ sudo python setup.py develop

Then do the same for SQLObject:

$ git clone git clone git://github.com/sqlobject/sqlobject.git
$ cd sqlobject
$ sudo python setup.py develop

Or rather fork it and clone your fork. To develop a feature or a bugfix create a separate branch, push it to your fork and create a pull request to the original repo. That way CI will be triggered to test your code.

Voila! The packages are globally installed, but the files from the checkout were not copied into site-packages. See setuptools for more.

Architecture

There are three main kinds of objects in SQLObject: tables, columns and connections.

Tables-related objects are in sqlobject/main.py module. There are two main classes: SQLObject and sqlmeta; the latter is not a metaclass but a parent class for sqlmeta attribute in every class - the authors tried to move there all attributes and methods not directly related to columns to avoid cluttering table namespace.

Connections are instances of DBConnection class (from sqlobject/dbconnection.py) and its concrete descendants. DBConnection contains generic code for generating SQL, working with transactions and so on. Concrete connection classes (like PostgresConnection and SQLiteConnection) provide backend-specific functionality.

Columns, validators and converters

Columns are instances of classes from sqlobject/col.py. There are two classes for every column: one is for user to include into an instance of SQLObject, an instance of the other is automatically created by SQLObject’s metaclass. The two classes are usually named Col and SOCol; for example, BoolCol and SOBoolCol. User-visible classes, descendants of Col, seldom contain any code; the main code for a column is in SOCol descendants and in validators.

Every column has a list of validators. Validators validate input data and convert input data to python data and back. Every validator must have methods from_python and to_python. The former converts data from python to internal representation that will be converted by converters to SQL strings. The latter converts data from SQL data to python. Also please bear in mind that validators can receive None (for SQL NULL) and SQLExpression (an object that represents SQLObject expressions); both objects must be passed unchanged by validators.

Converters from sqlobject/converters.py aren’t visible to users. They are used behind the scene to convert objects returned by validators to backend-specific SQL strings. The most elaborated converter is StringLikeConverter. Yes, it converts strings to strings. It converts python strings to SQL strings using backend-specific quoting rules.

Let look into BoolCol as an example. The very BoolCol doesn’t have any code. SOBoolCol has a method to create BoolValidator and methods to create backend-specific column type. BoolValidator has identical methods from_python and to_python; the method passes None, SQLExpression and bool values unchanged; int and objects that have method __nonzero__ (__bool__ in Python 3) are converted to bool; other objects trigger validation error. Bool values that are returned by call to from_python will be converted to SQL strings by BoolConverter; bool values from to_python (is is supposed they are originated from the backend via DB API driver) are passed to the application.

Objects that are returned from from_python must be registered with converters. Another approach for from_python is to return an object that has __sqlrepr__ method. Such objects convert to SQL strings themselves, converters are not used.

Branch workflow

Initially SQLObject was being developed using Subversion. Even after switching to git development process somewhat preserves the old workflow.

The trunk, called master in git, is the most advanced and the most unstable branch. It is where new features are applied. Bug fixes are applied to oldstable and stable branches and are merged upward – from oldstable to stable and from stable to master.

Style Guide

Generally you should follow the recommendations in PEP 8, the Python Style Guide. Some things to take particular note of:

• With some exceptions sources must be pure ASCII. Including string literals and comments.
• With a few exceptions sources must be flake8-clean (and hence pep8-clean). Please consider using pre-commit hook installed by running flake8 --install-hook.

• No tabs. Not anywhere. Always indent with 4 spaces.

• We don’t stress too much on line length. But try to break lines up by grouping with parenthesis instead of with backslashes (if you can). Do asserts like:

  assert some_condition(a, b), (
      "Some condition failed, %r isn't right!" % a)
  

• But if you are having problems with line length, maybe you should just break the expression up into multiple statements.

• Blank lines between methods, unless they are very small and closely bound to each other.

• Never use the form condition and trueValue or falseValue. Break it out and use a variable.

• Careful of namespace pollution. SQLObject does allow for from sqlobject import * so names should be fairly distinct, or they shouldn’t be exported in sqlobject.__init__.

• We’re very picky about whitespace. There’s one and only one right way to do it. Good examples:

  short = 3
  longerVar = 4
  
  if x == 4:
      do stuff
  
  func(arg1='a', arg2='b')
  func((a + b)*10)
  

  Bad examples:

  short    =3
  longerVar=4
  
  if x==4: do stuff
  
  func(arg1 = 'a', arg2 = 'b')
  func(a,b)
  func( a, b )
  [ 1, 2, 3 ]
  

  To us, the poor use of whitespace seems lazy. We’ll think less of your code (justified or not) for this very trivial reason. We will fix all your code for you if you don’t do it yourself, because we can’t bear to look at sloppy whitespace.

• Use @@ to mark something that is suboptimal, or where you have a concern that it’s not right. Try to also date it and put your username there.

• Docstrings are good. They should look like:

  class AClass(object):
      """
      doc string...
      """
  

  Don’t use single quotes (‘’’). Don’t bother trying make the string less vertically compact.

  Not strictly required but reStructuredText format for docstrings is very much recommended.

• Comments go right before the thing they are commenting on.

• Methods never, ever, ever start with capital letters. Generally only classes are capitalized. But definitely never methods.

• mixedCase is preferred.

• Use cls to refer to a class. Use meta to refer to a metaclass (which also happens to be a class, but calling a metaclass cls will be confusing).

• Use isinstance instead of comparing types. E.g.:

  if isinstance(var, str): ...
  # Bad:
  if type(var) is StringType: ...
  

• Never, ever use two leading underscores. This is annoyingly private. If name clashes are a concern, use name mangling instead (e.g., _SO_blahblah). This is essentially the same thing as double-underscore, only it’s transparent where double underscore obscures.

• Module names should be unique in the package. Subpackages shouldn’t share module names with sibling or parent packages. Sadly this isn’t possible for __init__, but it’s otherwise easy enough.

• Module names should be all lower case, and probably have no underscores (smushedwords).

Testing

Tests are important. Tests keep everything from falling apart. All new additions should have tests.

Testing uses pytest, an alternative to unittest. It is available at http://pytest.org/ and https://pypi.org/project/pytest/. Read its getting started document for more.

To actually run the test, you have to give it a database to connect to. You do so with the option -D. You can either give a complete URI or one of several shortcuts like mysql (these shortcuts are defined in the top of tests/dbtest.py).

All the tests are modules in sqlobject/tests. Each module tests one kind of feature, more or less. If you are testing a module, call the test module tests/test_modulename.py – only modules that start with test_ will be picked up by pytest.

The “framework” for testing is in tests/dbtest. There’s a couple of important functions:

setupClass(soClass) creates the tables for the class. It tries to avoid recreating tables if not necessary.

supports(featureName) checks if the database backend supports the named feature. What backends support what is defined at the top of dbtest.

If you import * you’ll also get pytest’s version of raises, an inserts function that can create instances for you, and a couple miscellaneous functions.

If you submit a patch or implement a feature without a test, we’ll be forced to write the test. That’s no fun for us, to just be writing tests. So please, write tests; everything at least needs to be exercised, even if the tests are absolutely complete.

We now use Github Actions to run tests.

Documentation

Please write documentation. Documentation should live in the docs/ directory in reStructuredText format. We use Sphinx to convert docs to HTML.

Contributing

• Now de-facto stadard for good commit messages is required.
• Conventional commit subject liness are recommended.
• Markdown format for commit message bodies is recommended. Github-flavored Markdown is allowed.
• Commit messages must be pure ASCII. No fancy Unicode emojies, quotes, etc.