MongoDB - Jongo and Morphia

When presented with the opportunity to store data into a database without schema restrictions, some of us are hesitant to leap into the NoSQL void.  I'm not.  I am tired of having to work with RDBMS restrictions for simple data changes in my applications.  I am tired of having to cram non-relational domain models into relational schemas.  I want an easier way to update my application data structures without having to constantly change my database, while still maintaining backwards compatibility, and providing minimal data transformation.  Lately I have been working with MongoDB and a couple of APIs (Jongo and Morphia) as well as MongoVUE.  What follows is my odyssey into MongoDB.

First I started with running MongoDB on my laptop.  On my Windows 7 machine I am using MongoDB 2.0.4 (mongodb-win32-x86_64-2.0.4).  The MongoDB daemon start-up script I am using is:

call mongod.exe --dbpath "C:/Tools/mongodb-win32-x86_64-2.0.4/data/db"

Once it is running, I can immediately connect to it with MongoVUE.  MongoVUE is a GUI that allows me to view the database, as well as edit data and other database level tasks.



The cool thing about MongoVUE is that most tasks that you execute in the GUI are read out in the log.  This has the added benefit of helping new comers to MongoDB more quickly learn the shell commands that are used to manage the database. Without the GUI, I would use the MongoDB shell.  In Windows this is launched by mongo.exe.

The Jongo API

Jongo allows Java developers to write programs that use the MongoDB Java Driver, while using MongoDB shell style JSON queries.  To see how it works, I will first use the MongoDB Java Driver API to insert an Employee object into the database.  First I start by connecting the database:




Next I get an Employee object.  Note:  MongoDriver is just the name of the Java class that I use to run this code.  I then serialize the Employee object into JSON via the FlexJSON API.  There is a MongoDB JSON API that includes a serializer, but it does not know how to serialize objects that are not intrinsic to MongoDB.  I wanted to serialize the object graph into JSON to experiment with how I would push JSON into MongoDB from my application.


Finally I parse the JSON into an Object and cast it to the com.mongodb.DBObject, and insert it into the "employees" collection.


To verify the insert I can view the employees collection in the MongoVUE GUI.  The interesting thing here is that the two date elements in the Employee document are serialized as longs and stored in the document store as numbers.  This seems counter intuitive to how dates should b e stored in MongoDB (ISO-Date).  In fact when working with the com.mongodb.util.JSON parser, I found a bug where longs were being improperly cast and truncated to ints.  This happened when the dates serialized were before 1/1/1970.  This bug is fixed in the MongoDB Java Driver version 2.8.0.  I am using version 2.7.3, so I had to fix it locally.


At this point the reader should notice that the Employee object contains two member objects:  Address and Department.  The JSON for this Employee object is seen below:


Now I can read the employee document using the Jongo API using a JSON query syntax:


The output is seen below:

The Morphia API

According Google Code, "Morphia is a lightweight, type-safe library for mapping Java objects to/from MongoDB."  Below is an example that I wrote to stuff a collection of Employee objects in the database as JSON documents.


Using Morphia requires a few Morphia annotations (@Entity, @Id, @Embedded) in the model objects (Employee, Address, Department).  In the Employee class I use all three.  The @Entity annotation is similar to the JPA @Entity.  The "employees" argument tells the Morphia API that this MongoDB document will be added to the employees collection.

The interesting thing about MongoDB collections is that if they do not yet exist in the database, they will be created in-line when documents are inserted.  This is also true of the database itself.  If the database specified by the API does not yet exist in the MongoDB environment, it too will be created on the first document insert.

The @Id annotation is needed for the for the "auto-generated" ObjectId field.  @Embedded is used on the Address and Department members to add them as embedded documents to the Employee document.  @Embedded is also on each of the Address and Department classes.







After execution I can again view the documents in MongoVUE.  In this Morphia insert, I have correctly mapped the java.util.Date to the ISODate MongoDB Date/Time data type.




Next I will investigate a scalable solution for MongoDB as well as Couchbase.

NVSS NFJS 2012 - Day 2

So today I started with a great session from Craig Walls about Spring Data (Templates and Repositories) focusing on integration with RDBMS, MongoDB, Neo4j, and Redis.  He seemed to be keen on Neo4j.  I am still looking into how the very schema-oriented graph database would work for my needs.

It was a very interesting session; I especially like the content on Spring Data Repositories.  I liked the convention of generating implementations by naming queries with method names.  If that is not good enough, or your methods don't adhere to what the Spring Data engine is expecting, you can use @Query annotations to specify the queries that Spring should implement.  This approach seems to still need a JPA provider, so for now, Hibernate will be used by our teams.  I am not 100% sold on the performance of Hibernate under the covers, but it does make implementation easy.

Next I sat through another MongoDB session with Ken Sipe.  I am partial to MongoDB, due to its schema-less approach and it's scalability.  Schema-less is a big departure from what most of us are use to.  I am eager to model one our applications into MongoDB collections of documents. I think that it gives most flexibility with the most web application performance.  Though I am still looking into Couchbase.  I am not necessarily worried about schema-less storage.  the biggest issue would be with data transformation needed to other service layers and other relational data stores.  I think we can manage that with service layers in our application.  I actually see schema-less storage with a schema-oriented services layers to be the best of both worlds.

I am still reviewing the limitations in MongoDB, like 24K namespaces.  Namespaces would include collections and indexes.  I should not bump into the collections limitations, but I am concerned about any limit on the number of documents in a single collection, or for that matter limitations on number of total documents in a database.

Finally, I ended the say with another Ken Sipe session on Web Application Security and ESAPI.

NVSS NFJS 2012 - Day 1

So I am up in Reston, VA for the Northern Virginia Software Symposium, No Fluff Just Stuff conference.  I came up here early for a Gradle session given by Tim Berglund.  I had looked at Gradle before, but his session was a good starter session for Gradle beginners.  We are already Maven and Groovy users, having moved from Ant.  Gradle lets you use Ant, Maven, Ivy, Java, Groovy, etc.  In fact, Gradle seems to elevate the art of writing builds to first-class programming, if it wasn't there already.  I will be looking into how we can better control our builds using Gradle, Maven, and Groovy.

In the afternoon I attended a few of Tim's sessions on NoSQL.  I have been researching several NoSQL solutions to date:  Cassandra, MongoDB, Couchbase.  After a survey of multiple NoSQL solutions, followed by a deep dive into Cassandra and Neo4j, I feel that I am better positioned to decide on the appropriate solution.

I really like the flexibility of schema-less storage in NoSQL databases like MongoDB.  However, my solutions need schema binding.  I am currently prototyping a solution that combines the schema-less, document-based storage of NoSQL with application layer schema validation and data transformation.  MongoDB and Couchbase seem to be good candidates due to there native handling of JSON.

One of the really cool things is that this year NFJS is not handing out paper session guides and agendas.  Instead, every attendee receives an iPad that they can use for the duration of the conference.  All the slides and conference documentation are on this iPad in a custom NFJS application.  It is convenient.

Speaking at TriJUG in May 2012

On May 21st I will be speaking at the May 2012 TriJUG meeting.  My talk will be based on my blog entry:  Dynamic Groovy Edges and Regression Isolation.  More info will follow as the date draws near.

PMI CVC PMP 2012 Spring Workshop - Framework Talk

I will be presenting on March 17th for the PMI Central Va Chapter during their Spring 2012 PMI Certification Workshop.  My topic will be the Project Management Framework.  My talk goes from 10:15 AM to 11:45 AM and touches on items found in sections 1 & 2, chapters 1, 2, & 3, of the PMBOK, version 4.

In my session I will be discussing these topics and more:

• Projects, Portfolios, and Program
• Process Groups, Knowledge Areas, and Processes
• PMO
• Project Life Cycle vs. Product Life Cycle
• Stakeholder Management
• Organizational Structure

Ever wondered how PMI keeps the PMP exam current and relevant?  This year we have also added information of the PMI Role Delineation Study and the Crosswalk.

This workshop is a great way to come up to speed for the PMP exam as well as gain valuable study tips from fellow project managers that have already passed the exam.  The workshop is also a great opportunity to gain the PDUs needed to maintain existing PMP certifications.  Best of all, attendees receive copies of all the slides presented at the workshop as well as other resources to help them study for the exam.