API with NestJS #19. Using RabbitMQ to communicate with microservices

JavaScript NestJS TypeScript Uncategorized

This entry is part 19 of 177 in the API with NestJS

NestJS subscribes to the idea that a microservice is an application that uses a different transport layer than HTTP. Therefore, in the previous part of this series, we’ve used TCP.

However, with NestJS, we have a broad selection of transporters that transmit messages across microservices. Each one of them is different, and it is worth exploring them before making a choice. Conveniently, Nest provides an abstraction for each transport layer, so it is easy to switch them without making significant changes to our code.

Introduction to RabbitMQ

The RabbitMQ is its fundamentals, a message broker that implements the Advanced Message Queuing Protocol (AMQP).

One of the most important concepts to understand is the producer (also called a publisher), whose job is to send the messages. The second important thing is the consumer that waits to receive messages.

Between the producer and the consumer is a queue. When the producer sends the message, it lands in the queue. The producer sends the messages through the exchange, which is a message routing agent.

We can bind the exchange to many different queues. When sending the message, the publisher can send the message to specified queues.

Finally, the consumer picks up the message from the queue and handles it.

Source: rabbitmq.com

Summing up the above:

  1. the producer sends a message to the exchange,
  2. the exchange receives the message and routes it to the desired queue,
  3. the consumer takes the message from the queue and consumes it.

Advantages of using RabbitMQ with microservices

With a message queue, we can send a message from one service to another without knowing if it can handle it. Therefore, we can separate our microservices more and make them less dependant on each other. CloudAMQP, in its article, also mentions that it helped them a lot in keeping the architecture flexible and fixing bugs. When a bug is found, a faulty microservice can be stopped and restarted after it is resolved. The queue of messages might pile quite a bit, but no data is lost.

Running RabbitMQ with Docker Compose

So far in this series, we’ve been using Docker Compose to work with Postgres and Elasticsearch. We can also easily use it to run an instance of RabbitMQ. Let’s add it to our file:

docker-compose.yml

Above, you can see that we specify the file. We do that to specify the default username and password. Let’s create it and add it to along with the file.

rabbitmq.env

.gitignore

In our configuration, we can see two ports exposed:

  • is the main RabbitMQ port that our NestJS application will use
  • is the port of the management plugin. It provides an interface that lets us manage and monitor our RabbitMQ instance

Using RabbitMQ with NestJS

In this article, we add RabbitMQ to the microservice built in the previous part of this series. Its job is to store email subscriptions.

Using RabbitMQ with microservices in NestJS is straightforward, thanks to the transporter built into the NestJS framework. We need to install some additional dependencies, though.

We also need additional environment variables in every part of our system that wants to connect to RabbitMQ.

Creating a microservice

Let’s start by configuring our microservice to use the above configuration.

main.ts

Queues in RabbitMQ can be durable or transient. The metadata of a durable queue is stored on the disk. If the queue is not durable, it is deleted during boot and would not survive a restart. It would delete not-consumed messages.

According to the RabbitMQ documentation, the performance does not differ in most cases.

After running our application, let’s visit . Here, we can see an interface that lets us interact with our RabbitMQ instance. The credentials are the same as we specified in the file.

When we visit the Queues tab, we can see our brand new queue that was automatically created.

The code from the previous part of this series would still work, and our controllers and services work the same.

Connecting to the microservice within the client

The other side of the connection is the client. It also needs to use environmental variables and connect to RabbitMQ.

subscribers.module.ts

Thanks to doing the above, we have a function connection with our microservice.

Replying to messages

When we start using our microservice, we notice that it can send a reply. This might make us wonder how does it happen. On the surface, we have a publisher that simply sends messages to the consumer.

If we take a look under the hood of NestJS, we can notice that it uses the queue. The framework sets is as a reply queue in the options passed to the library we’ve installed before. In the documentation of the Direct Reply-to feature, we can see that is a pseudo-queue that we can’t see in the RabbitMQ management interface.

This feature allows us to implement a request/reply pattern where a microservice can effortlessly respond.

Message acknowledgment

The important thing is to ensure that a message never gets lost. Since our messages are not guaranteed to reach our consumers or be successfully processed, we need a confirmation mechanism. Fortunately, RabbitMQ supports consumer acknowledgments and publisher confirms. This topic is covered in detail in the RabbitMQ documentation.

The consumer sends back an acknowledgment, stating that it received and processed the message. If the consumer fails to consume the message fully, RabbitMQ will re-queue it.

By default, NestJS handles acknowledgments automatically. We can do that manually, though. To do that, we need to pass the flag when creating a microservice.

To manually acknowledge a message, we need to access the context of the currently processed message.

There is more data available in the context. Let’s check out the   type:

Summary

In this article, we’ve implemented a microservice using RabbitMQ. We’ve gone through all of the basics of establishing communication with RabbitMQ and discussed the advantages of such an approach. We’ve also gone a little deeper and looked into the internals of NestJS to understand how it uses RabbitMQ under the hood a little better. This definitely gave us quite a bit of an overview of why we might want to use RabbitMQ and how.

Series Navigation<< API with NestJS #18. Exploring the idea of microservicesAPI with NestJS #20. Communicating with microservices using the gRPC framework >>
Subscribe
Notify of
guest
5 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Vikram Singh
Vikram Singh
3 years ago

This is very good stuff. can you please provide the git repo link for the whole code example? This will be very helpful

leonardo
leonardo
1 year ago
Reply to  Vikram Singh
Stephanie
Stephanie
2 years ago

what’s the code for the microservice – rabbitMQ?

xiafeng
xiafeng
2 years ago

Hi, I want to send messages to fanout exchange. If exchange doesn’t exist, need to create. But when I configure exchange, Transport.RMQ not working.

{
   provide: ‘EXAMPLE_SERVICE’,
   useFactory: (configService: ConfigService) => {
    const user = configService.get(‘RABBITMQ_USER’);
    const password = configService.get(‘RABBITMQ_PASSWORD’);
    const host = configService.get(‘RABBITMQ_HOST’);
    const queueName = configService.get(‘RABBITMQ_QUEUE_NAME’);
    const exchangeName = configService.get(‘RABBITMQ_EXCHANGE_NAME’);
    return ClientProxyFactory.create({
     transport: Transport.RMQ,
     options: {
      urls: [amqp://${user}:${password}@${host}],
      exchange: {
       name: exchangeName,
       type: ‘fanout’,
       echangeOpts: {
        durable: true,
       },
      },
     },
    });
   },
   inject: [ConfigService],
}

Last edited 2 years ago by xiafeng
gsoykan
gsoykan
2 years ago

Thank you so much! Can you provide some more information about how we can send delayed messages over the queue? Or can you give any leads about that? Best!