Chapter 23. How Veil2 Relates to the Original Veil

Before Veil2 there was Veil.

23.1. What is Veil?

Veil is a data security add-on for Postgres. It provides an API allowing you to control access to data at the row, or even column, level. Different users will be able to run the same query and see different results

Veil provides a set of components for dealing with bitmaps as variables, and for maintaining these variables in shared or session state.

It provides the necessary tools for you to build a secured database system, but:

  • the Veil variables cannot be stored in the database;

  • the Veil variables do not share in the transaction management of the underlying database;

  • session and connection management is not provided.

23.2. Where Did Veil2 Come From?

Although it was clear to the author that Veil had a serious deficiency in its lack of integration with Postgres' transaction management, it wasn't obvious how to resolve it.

The original Veil release was in 2011. One of the most useful datatypes that it provided was a hash of bitmaps. That is a set of bitmaps keyed by text strings.

The author could see no way to create a native Postgres datatype that could do the same thing, and so the original Veil continued as it was.

Eventually the author realized that the hstore module was just what was needed, and so set out to create a Postgres native bitmap type. This was pgbitmap. It worked pretty well and was simple and elegant. All of Veil could be replaced with pgbitmap, hstore and arrays, with all session objects being storable in temporary tables and shared objects in normal tables.

On further reflection, the author realized that some proper integration was needed, along with a decent framework for building VPDs using pgbitmap. It wouldn't be much work, and didn't need to contain much code.

As he tackled the problem of Veil2, he realized that much of what the user had to implement in a Veil-based system was going to be common to all users. And he had a SaaS project that he'd worked on as a guide to what was needed. So the project grew.

Ironically, as the implementation proceeded, the need for hstore never materialized.

And although it never became simple, the core concepts slowly crystallized, and the concept of a Relational Security System dropped out.

With a small number of relatively simple concepts that many developers, and even some PHBs could understand, it became clear that the underlying model was sound but that each implementation would be tricky and a lot of work. This was not something that most developers would be given the opportunity to build for themselves from a set of components. And given that the fundamental requirements were likely to be the same for many classes of problem, the direction for Veil2 was set.