Thursday, January 12, 2012

Lotus Notes: One of the First NoSQL Databases - No Really!

Depending on who you talk to, NoSQL means NO SQL at all or Not-Only SQL.  I like the latter meaning as I find it makes the technology more flexible and adoptable for those that have primarily worked with SQL databases.  Regardless of the meaning, there are over 100 databases that can be labeled as NoSQL.  These databases range from wide-column , document storage, and key-value tuple stores to object stores and graph databases, with several types in between.  Included in this list of 100+ NoSQL databases is Lotus Notes/Domino.

I started working with Lotus Notes in the early 90's, right after I started with HTML and VB.  Back then it a was a great tool to allow remote users to collaborate on common data, via replication.  Over the years (more than a decade) Notes evolved to include the Domino server-side technology, while including a host of other technology integration:  HTML, Java, C, C++, JavaScript, XML, Web Services, etc.  In fact, Lotus Notes was my springboard to move to Java.

To me, the two biggest strengths of Lotus Notes/Domino were its approach to non-structured data storage, and its replication technologies.  There are a whole host of other Domino features that make the platform very robust, such as agents, routing, and security, not to mention the robust HTTP stack.

Notes has been much maligned over the years as being mostly a productivity tool or email tool.  However, I was very happy working with Lotus Notes/Domino.  I think there are several reasons for the bad press surrounding Lotus Notes.  My top four reasons are below:
1.  IBM's incredible (dare I say colossal) mismanagement of the Lotus brand, especially in the wake of the WebSphere brand
2.  The lack of standards and design patterns, as well as documented ALM for Lotus Notes development (at one point I heard Notes referred to as a virus).
3.  The lack of forward momentum on modernizing client and web UI tools
4.  The lack of management understanding what Lotus Notes was and wasn't, while developers tried to make every solution a nail to be handled by the Notes hammer.

However much I lament the downward slide of Lotus Notes, there is truth to my NoSQL claim.

Notes stores data in documents, or more precisely, notes.  There are also design notes (forms, views, etc.) and administrative notes (ACL and replication). Based on that alone I would label it a document-based NoSQL database.

In Notes, the container model is quite simple.  Items contain data, notes (document notes) contain items, and database (NSF files) contain notes.  Items are of two types, summary and non-summary.  For the most part, only summary data can be displayed in a Notes view.  Non-summary data, while accessible at the note level, is generally not seen in a Notes view.  Notes views don't really contain document notes as much as they collect and index them.  Using views, designers can build views into Notes data based on the need of users and the data contained in document notes.  In this way, they are somewhat similar to views found in relational databases.

Non-summary items are those items usually bigger than 60K and/or composite, like rich text items.  These items can contain rich text, file attachments, images, multimedia, etc.

Notes forms are generally used to create, edit, and view document notes in the UI.  To that end, forms are designed for those functions with fields added to the forms to contain the data when document notes are displayed or edited in a form.  Forms contain fields, however, fields are not the same as items.  Items contain data, fields will temporarily show and allow editing of item data when the form is used to open a document note.  However, document notes can and do contain items that don't exist on any forms.  Multiple forms can be used to view or edit document notes.  In this fashion, forms are actually decoupled from document notes.  While document notes are item, and therefore data, containers, forms are merely the template used to view, edit, or create documents.

Forms are mostly UI concerns.  In fact, items contained in document notes can be, and often are, mutated, without forms.  Moreover, document notes can be created without ever using a form.

So, in the grand scheme of things, Lotus Notes, at its core, is a document-based NoSQL database for unstructured data.  You can store data within documents.  These documents can be related (or not) by data contained in items.  These items can be summary (simple) data or non-summary (composite) data.  Notes forms and views can be used to enter data (including tags and metadata) and view collections or related documents.  Notes replication ensures that documents contained in one database instance can be eventually synchronized with data in another Notes database with the same replica ID.

Given these features, which are comparable to many other NoSQL databases, Lotus Notes NoSQL is a reality, now.

3 comments:

  1. I'm another long-time Notes and Domino wiz and I've been looking over the NoSQL alternatives.

    Frankly, nothing even comes close.

    If (and there isn't enough emphasis on Earth for this) IBM had any sense, they'd position the Domino database server head to head with CouchDB and similar efforts, perhaps even aligning the APIs such that CouchDB could be the 'lite' version and Domino the industrial strength version with nice features like a real security model :)

    In case you don't know, the original designer of CouchDB rewrote Lotus' formula engine back in the mid-00's so he's very familiar with Notes and Domino.

    ReplyDelete
  2. Working with Notes since 3.3, I am familiar with Damien's work, I heard him speak at LS a few times, and talked with him in the labs. I did not know that he designed CouchDB. That is interesting news. I have long thought that Lotus Notes lacked the respect due it by RDBMS and OOP sycophants that did not understand the power and functionality in the platform.

    Thanks for the info!

    ReplyDelete
  3. Nice post. I felt a little nostalgic. I did work with Lotus Notes/Domino, I even made XPages applications. I do think Notes/Domino its a excelent productivity tool, but the lack of IBMs managements creates this horrible image that most of people have of Lotus Notes.
    Another thing that, I think, Notes lacks of is testing tools.
    I dont know, but maybe if IBM open source Lotus notes, maybe today we have more tools for development.

    ReplyDelete