As our application grows, the amount of data we receive through our REST API grows as well. We often need to split the data into parts and load them separately to avoid performance issues. A common pattern is implementing infinite scrolling that loads more content when the user scrolls down.
In this article, we implement infinite scrolling with React using the intersection observer. We also learn how to implement End-to-End tests using Playwright to ensure our application works as expected.
Implementing infinite scrolling with React
In our article, we will use the JSONPlaceholder API to fetch a list of posts. We can use query parameters to specify how many elements we want to fetch:
|
1 |
https://jsonplaceholder.typicode.com/posts?_start=40&_end=60 |
For example, by fetching the posts from the above URL, we state that we want twenty posts, starting from the post with the number 40 and ending with the post with the number 60.
When we send the _end query parameter, the API attaches the X-Total-Count response header that tells us how many posts there are in total. This will be useful to determine whether we should stop trying to fetch more data.
Making the HTTP request
Let’s start by writing the function that makes the HTTP request to the REST API. It needs to determine the starting point of our data and how many posts we want to fetch. To make it easy to use, we can split the data into pages that users can fetch one by one.
To compose the request URL with the query parameters, we can use the URLSearchParams class provided by the browsers.
fetchPosts.tsx
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
export async function fetchPosts(pageNumber: number, elementsPerPage = 20) { const start = pageNumber * elementsPerPage; const end = start + elementsPerPage; const params = new URLSearchParams({ _start: String(start), _end: String(end), }); const response = await fetch( `https://jsonplaceholder.typicode.com/posts?${params.toString()}`, ); const totalNumberOfPosts = Number(response.headers.get('x-total-count')); const posts = await response.json(); return { posts, totalNumberOfPosts, }; } |
Fetching pages one by one
Now, let’s implement the logic of fetching the pages one by one. Let’s store the current page number and increase it every time the user fetches a set of posts. Every time additional posts are fetched, we add them to the posts array.
We can determine if all posts were fetched by comparing the value from the X-Total-Count header with the number of posts we have in the state.
usePostPages.tsx
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
import { useCallback, useRef, useState } from 'react'; import { fetchPosts } from './fetchPosts'; import { Post } from './Post'; export function usePostPages() { const [posts, setPosts] = useState<Post[]>([]); const currentPageNumber = useRef(0); const isLoading = useRef(false); const [totalNumberOfPosts, setTotalNumberOfPosts] = useState<null | number>( null, ); const loadNextPage = useCallback(async () => { if (isLoading.current) { return; } isLoading.current = true; const { posts, totalNumberOfPosts } = await fetchPosts(currentPageNumber.current); currentPageNumber.current = currentPageNumber.current + 1; setPosts((currentPosts) => { return [...currentPosts, ...posts]; }); setTotalNumberOfPosts(totalNumberOfPosts); isLoading.current = false; }, []); const areAllPostsFetched = posts.length === totalNumberOfPosts; return { loadNextPage, areAllPostsFetched, posts, }; } |
We store the currentPageNumber and isLoading as references because we don’t want to have to put them into the dependencies array in the useCallback.
Detecting that the user scrolled the page
Now, we need to fetch the data at the correct moment when the user scrolls the page to the bottom. One way of doing that is through the Intersection Observer API. With it, we can detect when an element becomes visible or hidden on our website.
Let’s display a list of posts and a loader at the bottom.
Posts.tsx
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
import { usePostsLoading } from './usePostsLoading'; export const Posts = () => { const { posts, loaderRef, areAllPostsFetched } = usePostsLoading(); return ( <div> {posts?.map((post) => ( <div key={post.id} data-testid="post"> <h3>{post.title}</h3> <p>{post.body}</p> </div> ))} {!areAllPostsFetched && ( <div ref={loaderRef} data-testid="loader"> Loading... </div> )} </div> ); }; |
The loading indicator is always rendered at the bottom of the list of posts, but it is not visible if the user does not scroll to the bottom. The goal is to fetch more data when the user scrolls to the bottom and the loading indicator becomes visible.
We don’t need to render the loading indicator if all posts have been fetched and there is no need to load more.
To detect when the user scrolled down and the loading indicator was visible, we can use the IntersectionObserver class, which receives a callback as an argument. It is called every time the visibility of the observed element changes.
usePostsLoading.tsx
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 |
import { useEffect, useRef } from 'react'; import { usePostPages } from './usePostPages'; export function usePostsLoading() { const loaderRef = useRef(null); const { posts, loadNextPage, areAllPostsFetched } = usePostPages(); useEffect(() => { const observer = new IntersectionObserver((entries) => { const target = entries.pop(); if (target?.isIntersecting) { loadNextPage(); } }); if (loaderRef.current) { observer.observe(loaderRef.current); } return () => { if (loaderRef.current) { observer.unobserve(loaderRef.current); } }; }, [loadNextPage]); return { posts, loaderRef, areAllPostsFetched, }; } |
We should clean up and stop observing the element when it is not necessary anymore.
Thanks to the above, we now have a fully working infinite scrolling functionality.
Writing End-to-End tests for the infinite scrolling feature with Playwright
Let’s start by writing a simple End-to-End test with Playwright that checks if the first twenty posts load without interacting with the user interface. We can achieve that by using the toHaveCount function, which ensures that a given number of elements can be found on the page.
Please notice that we added data-testid properties in the Posts component.
posts.test.tsx
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
import { expect, test } from '@playwright/test'; test.describe('The Posts page', () => { test.describe('when visited', () => { test.beforeEach(async ({ page }) => { await page.goto('/'); }); test('should contain twenty posts', async ({ page }) => { const posts = page.getByTestId('post'); await expect(posts).toHaveCount(20); }); }); }); |
We can use page.goto('/') to navigate to our website thanks to using the baseURL config and environment variables. If you want to know more, check out JavaScript testing #17. Introduction to End-to-End testing with Playwright
Now, we need to simulate the user scrolling down the page. The easiest way is to use the scrollIntoViewIfNeeded function built into Playwright. However, we must wait for the first twenty posts to load before scrolling down.
posts.test.tsx
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
import { expect, test } from '@playwright/test'; test.describe('The Posts page', () => { test.describe('when visited', () => { test.beforeEach(async ({ page }) => { await page.goto('/'); }); // ... test.describe('and when the user scrolls down', () => { test('should contain forty posts', async ({ page }) => { const posts = page.getByTestId('post'); await expect(posts).toHaveCount(20); await page.getByTestId('loader').scrollIntoViewIfNeeded(); await expect(posts).toHaveCount(40); }); }); }); }); |
Thanks to the scrollIntoViewIfNeeded function, we could scroll down so the loader would be visible. This triggers the second HTTP request and loads more posts.
Summary
In this article, we’ve implemented infinite scrolling using React and the Intersection Observer. With this approach, we improved the user experience and optimized the performance of our application by loading the data only if needed.
To ensure that everything works as expected, we wrote End-to-End tests using Playwright. To do that, we had to learn how to simulate the user scrolling down on our website. With all that, we helped our frontend application handle a lot of data without slowing down. We also made sure it was bug-free by writing E2E tests.
