Thursday, May 9, 2013

Simple Pagination for MongoDB Queries Using QueryDSL

So, I have looked at several Java APIs for MongoDB integration; my favorite by far is Spring Data.  That said, Spring has API classes for paging and sorting.  Even with those classes, I find the QueryDSL API from mysema to be very functional.  To work with my Maven projects, I add the below plugin and dependencies:

...<plugin>
    <groupId>com.mysema.maven</groupId>
    <artifactId>maven-apt-plugin</artifactId>
    <version>1.0</version>
    <executions>
     <execution>
      <goals>
       <goal>process</goal>
      </goals>
      <configuration>
       <outputDirectory>target/generated-sources/java</outputDirectory>
       <processor>com.mysema.query.apt.QuerydslAnnotationProcessor</processor>
      </configuration>
     </execution>
    </executions>
   </plugin>
...
<dependency>
   <groupId>com.mysema.querydsl</groupId>
   <artifactId>querydsl-apt</artifactId>
   <version>2.2.3</version>
  </dependency>
  <dependency>
   <groupId>com.mysema.querydsl</groupId>
   <artifactId>querydsl-mongodb</artifactId>
   <version>2.2.3</version>
  </dependency>
...
Once my Maven configuration is set, I need to update the domain model objects that I will be using with QueryDSL.  Below is a snippet from my Employee model object with the @QueryEntity annotation added:


...
@QueryEntity
@Document(collection = "employees")
public class Employee extends Person {
...

Next I need to update the Spring Data Employee Repository interface to extend the QueryDslPredicateExecutor interface, type by my Employee model class.  At this point it is important to note that for the this integration, I had to switch from the annotated Mongo Repository definition to extending   the MongoRepository interface.  The @RepositoryDefinition annotation did not want to play nice with the QueryDslPredicateExecutor extension.


...
// @RepositoryDefinition(domainClass = Employee.class, idClass = String.class)
public interface EmployeeRepository extends MongoRepository<Employee, String>,
  QueryDslPredicateExecutor<Employee> {
...
Next, I need to update my EmployeeService interface and EmployeeServiceImpl implementation class to add a new method to access the generated find...() methods added for me by QueryDSL.


...
 public Page<Employee> findAllWithPages(int pageStart, int pageSize,
   Sort.Direction sortDirection, String sortField) {
  PageRequest pageRequest = new PageRequest(pageStart, pageSize,
    new Sort(Sort.Direction.ASC, "employeeId"));
  return this.employeeRepository.findAll(pageRequest);
 }
...
In this new method, I build a org.springframework.data.domain.PageRequest object and a org.springframework.data.domain.Sort object to pass into the newly provisioned findAll(..) method on the EmployeeRepository.  I did not write this new findAll(...) method, it was generated for me by QueryDSL and Spring Data.  Additionally, QueryDSL created the QEmployee class for me and placed it into target/generated-soutces/java in my Maven project.  QEmployee is a query type and is seen in its entirety below.


import static com.mysema.query.types.PathMetadataFactory.*;

import com.mysema.query.types.*;
import com.mysema.query.types.path.*;


/**
 * QEmployee is a Querydsl query type for Employee
 */
public class QEmployee extends EntityPathBase<Employee> {

    private static final long serialVersionUID = -236647047;

    public static final QEmployee employee = new QEmployee("employee");

    public final QPerson _super = new QPerson(this);

    public final SimplePath<Address> address = createSimple("address", Address.class);

    //inherited
    public final DateTimePath<java.util.Date> birthDate = _super.birthDate;

    public final SimplePath<Department> department = createSimple("department", Department.class);

    public final StringPath employeeId = createString("employeeId");

    //inherited
    public final StringPath firstName = _super.firstName;

    public final DateTimePath<java.util.Date> hireDate = createDateTime("hireDate", java.util.Date.class);

    //inherited
    public final StringPath id = _super.id;

    //inherited
    public final StringPath lastName = _super.lastName;

    //inherited
    public final StringPath middleName = _super.middleName;

    public final NumberPath<Integer> salary = createNumber("salary", Integer.class);

    public final StringPath title = createString("title");

    public QEmployee(String variable) {
        super(Employee.class, forVariable(variable));
    }

    public QEmployee(BeanPath<? extends Employee> entity) {
        super(entity.getType(), entity.getMetadata());
    }

    public QEmployee(PathMetadata<?> metadata) {
        super(Employee.class, metadata);
    }

}

Finally, below is the JUnit test that calls the pagination code generated for me.
package com.icfi.mongo;

import static org.junit.Assert.assertEquals;

import java.util.List;

import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.context.ApplicationContext;
import org.springframework.context.support.GenericXmlApplicationContext;
import org.springframework.data.domain.Page;
import org.springframework.data.domain.Sort;
import org.springframework.data.mongodb.core.MongoOperations;

import com.icfi.mongo.data.loaders.EmployeeShortLoader;
import com.icfi.mongo.data.model.Employee;
import com.icfi.mongo.services.EmployeeService;

public class PagingQueryTest {
 private static Logger log = LoggerFactory.getLogger(PagingQueryTest.class);

 private ApplicationContext ctx;
 MongoOperations mongoOps;
 List<Employee> employees;
 EmployeeService employeeService;

 @Before
 public void setup() {
  ctx = new GenericXmlApplicationContext("context/main.xml");
  mongoOps = (MongoOperations) ctx.getBean("mongoTemplate");
  employeeService = (EmployeeService) ctx.getBean("employeeService");
  EmployeeShortLoader.main(null);
 }

 @Test
 public void testPaging() {
  String[] lastNames = new String[] { "Stanfel", "Gustavson", "Lortz",
    "Marquardt", "Unno", "Savasere", "Spelt", "Wynblatt",
    "Danecki", "Weedman", "Hartvigsen", "Menhoudj", "Heyers",
    "Willoner", "Shumilov", "Zuberek", "Boguraev" };
  int pageCount = 10;
  int pageNumber = 0;
  String sortField = "employeeId";
  Sort.Direction sortOrder = Sort.Direction.ASC;

  Page<Employee> employeesPage = employeeService.findAllWithPages(
    pageNumber, pageCount, sortOrder, sortField);

  while (employeesPage.hasNextPage()) {

   assertEquals("List size is incorrect.", pageCount,
     employeesPage.getSize());

   log.info("Page Number = " + pageNumber);

   if (employeesPage.hasContent()) {
    log.info(employeesPage.getContent()
      .get(employeesPage.getSize() - 1).getLastName());

    assertEquals(
      "Last name was incorrect.",
      lastNames[pageNumber],
      employeesPage.getContent()
        .get(employeesPage.getSize() - 1).getLastName());
   }

   pageNumber++;

   employeesPage = employeeService.findAllWithPages(pageNumber,
     pageCount, sortOrder, sortField);
  }

  log.info("Page Number = " + pageNumber);

  employeesPage = employeeService.findAllWithPages(pageNumber, pageCount,
    sortOrder, sortField);

  log.info(employeesPage.getContent()
    .get(employeesPage.getContent().size() - 1).getLastName());

  assertEquals(
    "Last name was incorrect.",
    lastNames[pageNumber],
    employeesPage.getContent()
      .get(employeesPage.getContent().size() - 1)
      .getLastName());
 }

 @After
 public void tearDown() {
  this.mongoOps.getCollection("employees").drop();
 }
}

With this approach I have quickly added pagination to my MongoDB queries, while writing minimal code.

19 comments:

  1. Hi Jimmy,
    Nice tutorial, can you please post or send me the source code of this example tutorial, my email : akramakom@gmail.com

    Thanks, your help is appreciated.

    ReplyDelete
  2. in your code, you should have

    ...
    public Page findAllWithPages(int pageStart, int pageSize,
    Sort.Direction sortDirection, String sortField) {
    PageRequest pageRequest = new PageRequest(pageStart, pageSize,
    new Sort(Sort.Direction.ASC, sortField));
    return this.employeeRepository.findAll(pageRequest);
    }

    your hardcoding "employeId".... :)

    ReplyDelete
  3. This comment has been removed by the author.

    ReplyDelete
  4. I would like to have a service which can help me with code. I know good certified translation services near me, but not the one to help with coding. And it's so bad, because I don't really understand all that programming thing

    ReplyDelete
  5. The effectiveness of IEEE Project Domains depends very much on the situation in which they are applied. In order to further improve IEEE Final Year Project Domains practices we need to explicitly describe and utilise our knowledge about software domains of software engineering Final Year Project Domains for CSE technologies. This paper suggests a modelling formalism for supporting systematic reuse of software engineering technologies during planning of software projects and improvement programmes in Project Centers in Chennai for CSE.

    Spring Framework has already made serious inroads as an integrated technology stack for building user-facing applications. Spring Framework Corporate TRaining the authors explore the idea of using Java in Big Data platforms.
    Specifically, Spring Framework provides various tasks are geared around preparing data for further analysis and visualization. Spring Training in Chennai

    ReplyDelete
  6. Thanks for the wonderful share. Your article has proved your hard work and experience you have got in this field. Brilliant .i love it reading. tree removal loxahatchee

    ReplyDelete
  7. This post is good enough to make somebody understand this amazing thing, and I’m sure everyone will appreciate this interesting things.
    dumpster rental fort wayne

    ReplyDelete
  8. I have read your article, it is very informative and helpful for me.I admire the valuable information you offer in your articles. Thanks for posting it..septic tank pumping fort wayne

    ReplyDelete
  9. This is a great article thanks for sharing this informative information. I will visit your blog regularly for some latest post.tree trimming companies des moines

    ReplyDelete
  10. I think this is one of the most significant information for me. And i’m glad reading your article.dumpster rental cost fort collins

    ReplyDelete
  11. A superbly written article, if only all bloggers offered the same content as you, the internet would be a far better place. septic tank cleaning fort collins

    ReplyDelete
  12. Hello, I have browsed most of your posts. This post is probably where I got the most useful information for my research. Thanks for posting, maybe we can see more on this. Are you aware of any other websites on this subject?
    residential tree services fresno

    ReplyDelete
  13. A superbly written article, if only all bloggers offered the same content as you, the internet would be a far better place. dumpster rental companies bakersfield

    ReplyDelete
  14. This is a great article thanks for sharing this informative information. I will visit your blog regularly for some latest post.drain cleaning service bakersfield

    ReplyDelete
  15. You have a good point here!I totally agree with what you have said!! Thanks for sharing your views. hope more people will read this article!!! emergency tree services santa maria

    ReplyDelete
  16. I think this is one of the most significant information for me. And i’m glad reading your article. junk hauling services temecula

    ReplyDelete
  17. This is a great article thanks for sharing this informative information. I will visit your blog regularly for some latest post.grease trap pumping service temecula

    ReplyDelete
  18. I have read your article, it is very informative and helpful for me.I admire the valuable information you offer in your articles. Thanks for posting it..stump grinding roanoke

    ReplyDelete
  19. This post is good enough to make somebody understand this amazing thing, and I’m sure everyone will appreciate this interesting things.dumpster rental services norfolk

    ReplyDelete