API with NestJS #18. Exploring the idea of microservices

November 16, 2020

This entry is part 18 of 85 in the API with NestJS

As our application grows, maintaining it might become more challenging. One of the approaches that we can take is to structure our app with microservices.

You can find the code for the microservice we create in this article in this repository. The rest of the code can still be found in the repository of our monolithic application.

The overview of microservices

By implementing the microservice architecture, we break down our API into smaller, independent components. Having a separate codebase, and implementing a microservice separately, can make our application more scalable. Because we implement and deploy each microservice separately, it might be easier to handle feature releases and bug fixes.

Also, embracing microservices can be a good approach when choosing the right tool for the job. Your company can manage some microservices in Java and some in Node.js, for example.

Building our application with microservices also offers more flexibility in terms of deployment and managing resources between services. It might also be easier to manage database schema migration if we have multiple smaller databases instead of a monolithic one.

All of the above benefits come with some drawbacks, too. Even though each microservice is simpler than a monolithic application, managing multiple smaller apps is challenging. When updating a microservice, we need to make sure that we won’t break other services that depend on it.

We also need to adjust to microservices when developing. Communicating between services is not free and introduces a bit of latency. It can stack up as the number of dependencies between microservices grows. Also, testing microservices might prove not to be easy.

Implementing microservices with NestJS

Fortunately, NestJS has a set of tools prepared to make working with microservices easier. In this article, we craft a simple microservice for managing email subscriptions. Using the knowledge from this series, we create a way to store a list of email subscribers.

Creating a microservice

First, we create a brand new repository for our microservice. To start things up, we use NestJS CLI.

1	npm install -g @nestjs/cli

1	nest new nestjs-email-subscriptions

For starters, we connect to a Postgres database with Docker in the same way as in the second part of this series.

To avoid conflicts between databases of our main app and our microservice, you might choose a different set of ports in our docker-compose this time

First, let’s create an entity so that we can store our subscribers.

import { Column, Entity, PrimaryGeneratedColumn } from 'typeorm';

@Entity()

class Subscriber {

@PrimaryGeneratedColumn()

public id: number;

@Column({ unique: true })

public email: string;

@Column()

public name: string;

}

export default Subscriber;

Now, we can create a very straightforward service to add and list email subscribers.

import { Injectable } from '@nestjs/common';

import { InjectRepository } from '@nestjs/typeorm';

import Subscriber from './subscriber.entity';

import CreateSubscriberDto from './dto/createSubscriber.dto';

import { Repository } from 'typeorm';

@Injectable()

export class SubscribersService {

constructor(

@InjectRepository(Subscriber)

private subscribersRepository: Repository<Subscriber>,

) {}

async addSubscriber(subscriber: CreateSubscriberDto) {

const newSubscriber = await this.subscribersRepository.create(subscriber);

await this.subscribersRepository.save(newSubscriber);

return newSubscriber;

}

async getAllSubscribers() {

return this.subscribersRepository.find();

}

Once we have the core logic ready, we can start thinking about how we expose our microservice. For that, we can create a controller.

Using the TCP layer

This time, it is not a regular controller, though. Although we could create the API with HTTP, NestJS suggests a different approach. Instead of using HTTP, NestJS has its own abstraction over the TCP transport layer for microservices.

There are many suggested ways to connect to our NestJS microservices such as gRPC, but those are a topic for separate articles.

When bootstrapping multiple NestJS applications into microservice architecture, NestJS establishes the connection before the first call for a particular microservice. It later reuses it across each subsequent call, which couldn’t always be achieved with HTTP.

Also, using TCP differently allows us to achieve event-based communication if we want. This way, the client does not wait for the response from the microservice.

The first approach suggested by NestJS is the request-response message style. It is suitable when we want to exchange messages between services. To create a message based handle, we need the @MessagePattern() decorator.

import { Controller } from '@nestjs/common';

import { MessagePattern } from '@nestjs/microservices';

import CreateSubscriberDto from './dto/createSubscriber.dto';

import { SubscribersService } from './subscribers.service';

@Controller('subscribers')

export class SubscribersController {

constructor(

private readonly subscribersService: SubscribersService,

) {}

@MessagePattern({ cmd: 'add-subscriber' })

addSubscriber(subscriber: CreateSubscriberDto) {

return this.subscribersService.addSubscriber(subscriber);

}

@MessagePattern({ cmd: 'get-all-subscribers' })

getAllSubscribers() {

return this.subscribersService.getAllSubscribers();

}

Above, our handlers listen for messages that match the provided pattern. A pattern is a plain value that can be an object or a string, for example. NestJS sends them along with the data:

if { cmd: 'add-subscriber' }) is the pattern, the addSubscriber method is called
if { cmd: 'get-all-subscribers' } is the pattern, the getAllSubscribers method is called

We also need to create a simple SubscribersModule, so that we can add it to our AppModule.

import { Module } from '@nestjs/common';

import { TypeOrmModule } from '@nestjs/typeorm';

import Subscriber from './subscriber.entity';

import { SubscribersService } from './subscribers.service';

import { SubscribersController } from './subscribers.controller';

@Module({

imports: [TypeOrmModule.forFeature([Subscriber])],

providers: [SubscribersService],

exports: [],

controllers: [SubscribersController],

})

export class SubscribersModule {}

The last thing to do is to expose our microservice. To do that, let’s modify the default bootstrap function in the main.ts file of our service.

import { NestFactory } from '@nestjs/core';

import { AppModule } from './app.module';

import { MicroserviceOptions, Transport } from '@nestjs/microservices';

import { ConfigService } from '@nestjs/config';

async function bootstrap() {

const app = await NestFactory.create(AppModule);

const configService = app.get(ConfigService);

await app.connectMicroservice<MicroserviceOptions>({

transport: Transport.TCP,

options: {

port: configService.get('PORT'),

});

app.startAllMicroservices();

}

bootstrap();

Above we use an instance of the ConfigService. If you want to know how to deal with environment variables with NestJS, check out the second part of this series.

Adding a microservice to an existing monolith app

There might be a case in which we didn’t design our application with microservices in mind from the beginning. In this case, we would want to connect our monolithic app to a new microservice.

To do that, let’s create a SubscribersModule inside of our monolithic NestJS application.

import { Module } from '@nestjs/common';

import SubscribersController from './subscribers.controller';

import { ConfigModule, ConfigService } from '@nestjs/config';

import { ClientProxyFactory, Transport } from '@nestjs/microservices';

@Module({

imports: [ConfigModule],

controllers: [SubscribersController],

providers: [

{

provide: 'SUBSCRIBERS_SERVICE',

useFactory: (configService: ConfigService) => (

ClientProxyFactory.create({

transport: Transport.TCP,

options: {

host: configService.get('SUBSCRIBERS_SERVICE_HOST'),

port: configService.get('SUBSCRIBERS_SERVICE_PORT'),

}

})

inject: [ConfigService],

}

})

export class SubscribersModule {}

Above, we expect SUBSCRIBERS_SERVICE_HOST and SUBSCRIBERS_SERVICE_PORT to be provided in the .env file.

We also need a SubscribersController that communicates with our microservice.

import {

Body,

Controller,

Get,

Post,

UseGuards,

UseInterceptors,

ClassSerializerInterceptor, Inject,

} from '@nestjs/common';

import JwtAuthenticationGuard from '../authentication/jwt-authentication.guard';

import CreateSubscriberDto from './dto/createSubscriber.dto';

import { ClientProxy } from '@nestjs/microservices';

@Controller('subscribers')

@UseInterceptors(ClassSerializerInterceptor)

export default class SubscribersController {

constructor(

@Inject('SUBSCRIBERS_SERVICE') private subscribersService: ClientProxy,

) {}

@Get()

@UseGuards(JwtAuthenticationGuard)

async getSubscribers() {

return this.subscribersService.send({

cmd: 'get-all-subscribers'

}, '')

}

@Post()

@UseGuards(JwtAuthenticationGuard)

async createPost(@Body() subscriber: CreateSubscriberDto) {

return this.subscribersService.send({

cmd: 'add-subscriber'

}, subscriber)

}

We could also create a dedicated microservice instead of using the ClientProxy directly in the controller. This would give us the possibility to export it from the SubscribersModule and use in other places of our application.

Please note that even though we don’t need to send any data to get a list of subscribers, NestJS throws an error if we use null or undefined in subscribersService.send. Because of that, we pass an empty string here.

Above, we’ve created the following flow:

the user calls the /subscribers endpoint in our monolithic app,
our application calls the microservice to get the necessary data,
the microservice retrieves the data from its own database,
our main application responds with the data.

In our SubscribersController, only logged-in users can list and create subscribers. Although that’s the case, we didn’t implement any authentication mechanism inside of our microservice that deals with subscribers. This is because we intend it to be a private API. We don’t want to expose it to the world, and we want only our main application to be able to communicate with it. Such a firewall should be configured at the architecture level. We will cover this topic more when creating an API Gateway for an architecture designed with microservices in mind.

If you want to know how to implement JWT-based authentication, check out the third part of this series.

Using the event-based communication

Aside from using the @MessagePattern(), we can also implement event-based communication. This is fitting for cases in which we don’t want to wait for a response. We can do so in the case of creating new subscribers.

For starters, we want our microservice to listen for events instead of messages.

import { Controller } from '@nestjs/common';

import { EventPattern } from '@nestjs/microservices';

import CreateSubscriberDto from './dto/createSubscriber.dto';

import { SubscribersService } from './subscribers.service';

@Controller()

export class SubscribersController {

constructor(

private readonly subscribersService: SubscribersService,

) {}

@EventPattern({ cmd: 'add-subscriber' })

addSubscriber(subscriber: CreateSubscriberDto) {

return this.subscribersService.addSubscriber(subscriber);

}

// ...

}

Then, we need to emit the event from our monolithic app.

import {

Body,

Controller,

Post,

UseGuards,

UseInterceptors,

ClassSerializerInterceptor, Inject,

} from '@nestjs/common';

import JwtAuthenticationGuard from '../authentication/jwt-authentication.guard';

import CreateSubscriberDto from './dto/createSubscriber.dto';

import { ClientProxy } from '@nestjs/microservices';

@Controller('subscribers')

@UseInterceptors(ClassSerializerInterceptor)

export default class SubscribersController {

constructor(

@Inject('SUBSCRIBERS_SERVICE') private subscribersService: ClientProxy,

) {}

@Post()

@UseGuards(JwtAuthenticationGuard)

async createPost(@Body() subscriber: CreateSubscriberDto) {

return this.subscribersService.emit({

cmd: 'add-subscriber'

}, subscriber)

}

// ...

}

Even if this could be more performant, implementing the above might not be what we want in our case, though. Since we are not waiting for the response, we wouldn’t know if adding the subscriber succeeded. Even if it did, we wouldn’t receive details such as the id.

Summary

In this article, we’ve introduced the idea of microservices. This included creating a simple microservice and connecting it to an existing monolithic application. While this is a common use-case, we could also design architecture with microservices in mind from the beginning. This would include, for example, creating a dedicated API Gateway. We will cover that and other ways of communicating between services in the upcoming articles.

Series Navigation<< API with NestJS #17. Offset and keyset pagination with PostgreSQL and TypeORMAPI with NestJS #19. Using RabbitMQ to communicate with microservices >>

4 Comments

Oldest

Newest Most Voted

Inline Feedbacks

View all comments

Chris

1 year ago

I like your articles, they are really great to go beyond the nestjs documentation and to see more “real life” things

However, in this article, you are doing something that is not really correct from my point of view when you say “this is a private API and that you rely on a firewall at architecture level”.

I find this is a dangerous approach, more or less the same as providing a database access without a user/password and saying that you rely on architecture level to protect it.

So, in my opinion, when you create a microservice, you should integrate the HTTP layer with JWT security directly inside the microservice itself (as well as OpenApi to facilitate things later)

Optionnaly, you could also use gRPC (in adition to the HTTP layer), also with JWT security

I often (always) use the HTTP layer as a proxy to the gRPC service so I am 100% sure that both are always in sync.

That way, any service can call the microservice when they have a JWT : a browser / mobile with the HTTP layer, another internal service with the gRPC

Finally, you should emit events on a bus when you do something (i.e when you create/update/delete), also directly from the microservice and without providing details in the message itself

For ex, you can have bus.emit(‘post_created’, {post_id:1234})

That way, any service interested in your events can react BUT only if they have the proper auth to request your microservice.

This is more or less how AWS/Azure/… are made : you can use the service only if you have the proper role (and network access)
For ex : when you want to create a file in S3, the serrvice must have the permission to access the bucket with write privileges (it is not an anonymous unprotected access even if S3 bucket is not public)

My comment is longer than I expected …
I hope it will still be useful 🙂

Stephanie

10 months ago

Reply to Chris

Any Examples of implementing this into @nestjs as microservice, Chris?
It be really great to see it in action… also I think this is made for tutorial purposes.

We cannot walk until we can crawl, why learn about securing gRPC and bus before we learned about events? But it’s a good continue and I’m sure wanago will add it as a section here if you give him some help!

Kind regards, Stephanie

Last edited 10 months ago by Stephanie

Would it be possible too see a github repo for the microservice as well? Can’t really get mine working at the moment,

Peanut2001

8 months ago

Reply to Stephanie

You should create the new project NestJs and set up microservices, and set up microservices for this project. After, you can communicate with both