API with NestJS #142. A video chat with WebRTC and React

NestJS

This entry is part 142 of 158 in the API with NestJS

Let’s say two people want to video chat using our app. One solution would be for the first person to stream their camera feed to our server. Then, our application would pass this data to the other caller. Unfortunately, our application acting as a middleman could introduce a significant lag, especially if our server is far from the call participants. Fortunately, we can use WebRTC to connect our users directly without sending all of the data through our server. This improves performance and helps us achieve a smooth experience.

Even if the video and audio are not streamed through our application, we still need to build a server to allow the users to establish a connection with each other. In this article, we create a NestJS application to connect our users. Besides that, we also create a frontend app using React that manages and displays the video stream.

For the code of our backend application, check out this repository.
If you want to take a look at the frontend code, open this repository.

Capturing the camera

In this article, we gradually build both the frontend and backend applications. For the frontend part, we use Vite.

If you want to know more about Vite, check out React with Vite and TypeScript and its common challenges

Let’s start by capturing a stream from the local camera. To do that, we need to use the mediaDevices object.

useLocalCameraStream.tsx

To display it, we need to use the element. Since we need to attach the media stream to its property, we must use a React reference.

VideoFeed.tsx

The last step is to combine our hook with the component.

App.tsx

Creating video chat rooms

We want our users to be able to join a particular chat room and communicate in real time with the other person in the room. To do that, we need to use WebSockets.

If you want to know more about websockets, check out API with NestJS #26. Real-time chat with WebSockets

To use WebSockets in our NestJS application, we need to install a few additional libraries.

In this article, we use Socket.IO, because it is the most popular implementation of WebSockets for NestJS.

Let’s start with allowing the users to join a chat room with a particular name. In this straightforward implementation, we only handle a maximum of two people at once in the room.

chat.gateway.ts

Thanks to the above code, the user joins a room with the provided name as soon as the frontend application emits the message.

Dealing with Cross-Origin Resource Sharing

There is a solid chance that our frontend and backend applications are deployed to different origins. This can cause issues with Cross-Origin Resource Sharing when our frontend application tries to talk with the API.

If you want to know more about CORS, check out API with NestJS #117. CORS – Cross-Origin Resource Sharing

To deal with this, we need to define the URL of our frontend application on the backend side. A good way to do that is through environment variables.

.env

app.module.ts

To be able to provide the frontend URL dynamically, we should extend the built into NestJS.

websocket-adapter.ts

We can now use it in our function.

main.ts

Joining a room

One way of letting the user join a particular room is through URL path parameters. For example, if the users enter the URL, we assume that the room name is the .

The React Router is the most straightforward way to achieve that with React.

App.tsx

To connect to our API, we need its URL. The best way to store it is through environment variables.

.env

vite-env.d.ts

We need to use the library to establish the WebSocket connection.

socket.tsx

We can now connect to our NestJS application and join a room.

useChatConnection.tsx

Above, we emit the event as soon as we connect to the WebSocket and provide the name of the chat based on the URL.

VideoChatRoom.tsx

Connecting the people in the room

Once we have precisely two people in our chat room, we want one of them to initiate a connection. To be able to do that, we need to let one of the people know that the other person is ready. To do that, we can emit the event.

chat.gateway.ts

We need to start by creating an RTC Peer Connection. To do that, we need to provide the ICE server. In WebRTC, ICE is a framework used to facilitate the connection between the users. It helps us find the best path to connect two peers online. A part of the ICE framework is the STUN protocol used to discover the public address we need to establish the connection.

Fortunately, Google provides a public STUN server we can use.

usePeerConnection.tsx

Above, we create an instance of the and add video and audio tracks from the local stream created thanks to our webcam.

Let’s pass the object to the hook we created before.

VideoChatRoom.tsx

Sending the connection offer

The first step in establishing the WebRTC connection is sending the connection offer. To do that, we need to use the and methods.

Once the offer is created, we send it to our NestJS server through the event.

useOfferSending.tsx

We should send the offer as soon as we receive the event indicating that the other person in the chat room is ready.

useChatConnection.tsx

We now need to adjust our NestJS server to handle this event. Fortunately, it is very straightforward. As soon as our WebSocket server gets the , it sends it to the other person in the room.

chat.gateway.ts

Sending the answer

The other person in the room must prepare the answer as soon as they receive the offer. First, we need to call the function to acknowledge the offer. Then, we must create the answer and send it to our NestJS server through the WebSocket.

useSendingAnswer.tsx

We should configure our frontend application to send the answer once it receives the offer.

useChatConnection.tsx

We should configure our server to get the answer and pass it to the other person in the chat room.

chat.gateway.ts

Processing the answer

Finally, we need to process the answer to the offer. We need to start by acknowledging the answer through the method.

useAnswerProcessing.tsx

We need to do that as soon as we receive the event.

useChatConnection.tsx

When we receive the stream from the other person in the room, it causes the event to be dispatched. Let’s listen to it and store it in the state so that we can display it later.

usePeerConnection.tsx

The icecandidate event

The event in WebRTC is triggered when the local ICE agent discovers a new network candidate for establishing a peer-to-peer connection. This event provides the candidate’s information, which includes details like the IP address and port number. We must send this candidate to the peer we want to connect to. To do that, we need to send an event to our NestJS server.

usePeerConnection.tsx

Now, let’s handle the event by passing the data to the other person in the chat room.

chat.gateway.ts

The last step is to handle receiving the candidate.

useChatConnection.tsx

Displaying both video streams

We now have access to the video stream of the other person in the room through the variable. Let’s use it to render both videos.

VideoChatRoom.tsx

Summary

In this article, we’ve gone into detail about what Web Real-Time Communication (WebRTC) is and what problem it solves. We explained how it allows people to communicate in real-time directly through web browsers and keeps the role of the server to a minimum. We implemented a WebSocket server using NestJS and a frontend application with React to show how it works.

By implementing a real-life example, we got from the basics and progressed into more advanced aspects. Thanks to that, we established a solid understanding of WebRTC and used this knowledge in our code.

Series Navigation<< API with NestJS #141. Getting distinct records with Prisma and PostgreSQLAPI with NestJS #143. Optimizing queries with views using PostgreSQL and Kysely >>
Subscribe
Notify of
guest
0 Comments
Inline Feedbacks
View all comments