API with NestJS #48. Definining indexes with MongoDB and Mongoose

September 20, 2021

This entry is part 48 of 180 in the API with NestJS

The bigger our database is, the more demanding our queries become in terms of computing power. A common way of tackling this problem is by creating indexes. In this article, we explore this concept and create indexes with MongoDB and Mongoose.

When performing a MongoDB query, the database must scan every document in a given collection to find matching documents. MongoDB can limit the number of records to inspect if we have an appropriate index in our database. Since it makes it easier to search for the documents in the database, indexes can speed up finding, updating, and deleting.

Under the hood, indexes are data structures that store a small part of the collection’s data in an easy-to-traverse way. It includes the ordered values of a particular field of the documents. It makes MongoDB indexes similar to indexes in databases such as PostgreSQL.

When we define indexes, MongoDB needs to store additional data to speed up our queries. But, unfortunately, it slows down our write queries. It also takes up more memory. Therefore, we need to create indexes sparingly.

Unique indexes

The unique index makes sure that we don’t store duplicate values. We can create it by passing unique: true to the @Prop decorator.

user.schema.ts

import { Prop, Schema, SchemaFactory } from '@nestjs/mongoose';

import { Document, ObjectId } from 'mongoose';

import { Transform } from 'class-transformer';

export type UserDocument = User & Document;

@Schema({

toJSON: {

getters: true,

virtuals: true,

})

export class User {

@Transform(({ value }) => value.toString())

_id: ObjectId;

@Prop({ unique: true })

email: string;

// ...

}

const UserSchema = SchemaFactory.createForClass(User);

It is important to know that MongoDB creates a unique index on the _id field when creating a collection. Therefore, we sometimes refer to it as the primary index. We take advantage of the above in the last part of this series, where we implement pagination and sort documents by the _id field.

When we sort documents using a field without an index, MongoDB performs sorting at query time. It takes time and resources to do that and makes our app response slower. However, having the right index can help us avoid sorting results at query time because the results are already sorted in the index. Therefore, we can return them immediately.

We need to keep in mind that making a property unique creates an index and slows down our write queries.

Implementing indexes with Mongoose

With MongoDB, we can also define secondary indexes that don’t make properties unique.

post.schema.ts

import { Prop, Schema, SchemaFactory } from '@nestjs/mongoose';

import { Document, ObjectId } from 'mongoose';

import { Transform, Type } from 'class-transformer';

export type PostDocument = Post & Document;

@Schema()

export class Post {

@Transform(({ value }) => value.toString())

_id: ObjectId;

@Prop({ index: true })

title: string;

// ...

}

export const PostSchema = SchemaFactory.createForClass(Post);

By doing the above, we speed up queries, such as when we look for a post with a specific title, for example. We also speed up queries where we sort posts by the title alphabetically.

Text indexes

MongoDB also implements text indexes that support search queries on string content. To define a text index, we need to use the index() method.

post.schema.ts

import { Prop, Schema, SchemaFactory } from '@nestjs/mongoose';

import { Document, ObjectId } from 'mongoose';

import { Transform } from 'class-transformer';

export type PostDocument = Post & Document;

@Schema()

export class Post {

@Transform(({ value }) => value.toString())

_id: ObjectId;

@Prop()

title: string;

// ...

}

const PostSchema = SchemaFactory.createForClass(Post);

PostSchema.index({ title: 'text' });

export { PostSchema };

When we set up a text index, we can take advantage of the $text operator. It performs a text search on the content of the fields indexed with a text index.

A collection can’t have more than one text index.

Let’s implement a feature of searching through our posts by adding a new query parameter.

post.controller.ts

import {

Controller,

Get,

Query,

UseInterceptors,

} from '@nestjs/common';

import PostsService from './posts.service';

import MongooseClassSerializerInterceptor from '../utils/mongooseClassSerializer.interceptor';

import { Post as PostModel } from './post.schema';

import { PaginationParams } from '../utils/paginationParams';

@Controller('posts')

@UseInterceptors(MongooseClassSerializerInterceptor(PostModel))

export default class PostsController {

constructor(private readonly postsService: PostsService) {}

@Get()

async getAllPosts(

@Query() { skip, limit, startId }: PaginationParams,

@Query('searchQuery') searchQuery: string,

) {

return this.postsService.findAll(skip, limit, startId, searchQuery);

}

// ...

}

We also need to add the $text query to the service.

post.service.ts

import { FilterQuery, Model } from 'mongoose';

import { Injectable } from '@nestjs/common';

import { InjectModel } from '@nestjs/mongoose';

import { Post, PostDocument } from './post.schema';

@Injectable()

class PostsService {

constructor(@InjectModel(Post.name) private postModel: Model<PostDocument>) {}

async findAll(

documentsToSkip = 0,

limitOfDocuments?: number,

startId?: string,

searchQuery?: string,

) {

const filters: FilterQuery<PostDocument> = startId

? {

_id: {

$gt: startId,

}

: {};

if (searchQuery) {

filters.$text = {

$search: searchQuery,

};

}

const findQuery = this.postModel

.find(filters)

.sort({ _id: 1 })

.skip(documentsToSkip)

.populate('author')

.populate('categories');

if (limitOfDocuments) {

findQuery.limit(limitOfDocuments);

}

const results = await findQuery;

const count = await this.postModel.count();

return { results, count };

}

// ...

}

export default PostsService;

Thanks to the above, MongoDB can search through the titles of our posts.

The $text query has more arguments, such as the $caseSensitive boolean. For more, check out the official documentation.

Compound indexes

The $text query searches through all of the fields indexed with the text index. With MongoDB, we can create compound indexes where the index structure holds references to multiple fields.

1	PostSchema.index({ title: 'text', content: 'text' });

Thanks to doing the above, the $text query will search both through the titles and contents of posts.

Besides the text indexes, we can also create regular compound indexes.

user.schema.ts

import { Prop, Schema, SchemaFactory } from '@nestjs/mongoose';

import { Document, ObjectId } from 'mongoose';

import { Transform } from 'class-transformer';

export type UserDocument = User & Document;

@Schema({

toJSON: {

getters: true,

virtuals: true,

})

export class User {

@Transform(({ value }) => value.toString())

_id: ObjectId;

@Prop()

firstName: string;

@Prop()

lastName: string;

// ...

}

const UserSchema = SchemaFactory.createForClass(User);

UserSchema.index({ firstName: 1, lastName: 1 });

export { UserSchema };

Doing the above creates a compound index on the firstName and lastName fields. It can speed queries such as the ones where we look for a user with a specific first name and last name.

By using 1, we create an ascending index. When we use -1, we create a descending index. The direction doesn’t matter for single key indexes because MongoDB can traverse the index in either direction. It can be significant for compound indexes, though. The official documentation and this Stackoverflow page provide a good explanation.

The @Prop({ index: true }) decorator creates an ascending index.

Summary

In this article, we’ve touched on the subject of indexes in MongoDB. We’ve explained different types of indexes, such as unique indexes, single-field indexes, and compound indexes. We’ve also learned about text indexes and implemented a search functionality using them. We also got to know that creating advantages can speed up some queries while slowing down others.

Series Navigation<< API with NestJS #47. Implementing pagination with MongoDB and MongooseAPI with NestJS #49. Updating with PUT and PATCH with MongoDB and Mongoose >>