When I started my Heal Together project, I knew there would be many challenges, but I was ready to take them head-on and improve as a developer. Because this is how you grow — you face challenges that seem impossible, and you conquer them. Before delving into the problems I encountered in my project and how I solved them, let me give you an overview of my project.

Heal Together is a social media website for mental health. Below, you can find the basic structures of the graph in the Neo4j database

Structures of the graph in the neo4j database

The problem I encountered was retrieving the comment details from the database and displaying the data on the frontend (React.js).

While initially retrieving data from a database might seem straightforward, Neo4j posed a unique challenge due to its structure. To provide an overview, there are 24 user nodes, 234 Mindful Moments (posts) nodes, and 914 comments, each with relationships between them. To tackle this, I implemented data pagination.

Unlocking Efficiency: Mastering Data Pagination for Seamless User Experiences

Data Pagination is the process of sending small chunks of data at a time. Every company dealing with large volumes of data needs to implement this technique. Later in this blog, I will discuss different types of Data Pagination techniques.

For example, let’s consider a scenario where a Mindful Moment has 20 comments on my website. This implies that not only does the system have to retrieve those 20 comments, but it also has to retrieve the associated 20 users by traversing the graph multiple times.

The main difference between standard databases like SQL and MongoDB and Neo4j lies in Neo4j’s nature as a graph database. Retrieving data from Neo4j involves not only fetching data but also traversing the graph structure. This traversal aspect requires careful consideration, distinguishing it from the more straightforward querying process of traditional databases.

One of the ways of performing Data Pagination is using scrolling. Since the tech stack of my project is MERN + Neo4j, I decided to use infinite scrolling for Data Pagination. Neo4j actually has two built-in functions, SKIP and LIMIT, for performing Data Pagination. Here was my code for implementing it with SKIP and LIMIT.

   const momentsQuery=` MATCH p=(l:Comments)-[:Respond_To]->(n2:\`Mindful Moments\`{Mindful_Moments:$MindfulMoments})
    MATCH n1=(n:Users)
    WHERE n.\`_id\` = REPLACE(l.User_Comment_id, ' ', '')
    RETURN l.Comments as comments ,n.username as username ,n.user_profile_pic as profile_pic`
    const result = await session.run(momentsQuery + ` SKIP ${skip} `+ `LIMIT ${limit}`, { MindfulMoments });

I was incrementing skip as the user scrolled. For example, initially, the skip value would be 0. Then, as the user scrolled, I incremented skip by 5, then 10, and so on. However, after implementing this approach, my site would crash after just 2 minutes.

Fatal Flaw: Unveiling the Crash Culprit Behind My Pagination Implementation

What happened? Why all of a sudden couldn’t you see all the comments for more than 2 minutes? Since this project wasn’t associated with any company and the sole reason for this project was to enhance my portfolio, I had to research on my own what could have been the problem.

My first thought was that there was something wrong with the query, and since I hadn’t used Neo4j’s built-in functions like UNWIND, that was the reason why this was happening. However, upon adding UNWIND to my query, the issue still wasn’t resolved.

I thought the main problem was with my pagination technique and the way I was using it. So I decided to see other ways to perform pagination

Revolutionizing Pagination: Unleashing the Power of Alternate Techniques

I had to start from scratch and explore alternative methods. I researched various types of pagination and stumbled upon cursor-based pagination. But what is cursor-based pagination? While my previous approach involved counting all the nodes through SKIP, cursor-based pagination operates differently. Instead of counting all the nodes, it returns an ID when data is retrieved from the server. This ID can then be used to fetch the remaining data.

For instance, in the example discussed throughout this blog, when we retrieve the first 5 comments along with their associated user data, we also receive an ID of 5. Using this ID, we can execute a code like the one shown below to fetch data where the ID is greater than 5:

    const momentsQuery = `
    MATCH (l:Comments)-[:Respond_To]->(n2:Mindful_Moments {Mindful_Moments:$MindfulMoments})
    where l.Comment_ID>$comment_id

    MATCH (n:Users)
    WHERE n._id = REPLACE(l.User_Comment_id, ' ', '')
    WITH l.Comments AS comments, n.username AS username, n.user_profile_pic AS profile_pic, l.Comment_ID AS comment_id
    RETURN comments, username, profile_pic, LAST(COLLECT(comment_id)) AS Comment_ID
    `;

By using this code, we can implement pagination without overloading our server. But one question arises: how can we keep track of IDs, considering they may vary significantly for different comments? I pondered this and devised a solution.

In the above images, you can see a portion of the data for comments. What I did was keep track of the count for every comment on a single post. For example, one post could have 5 comments, each assigned a unique ID as well as a sequential number to track their order, ranging from 1 to 5.

Browser Compatibility Woes: Navigating Pagination Pitfalls Across Chrome and Firefox

Although my solution worked in Chrome, it didn’t function properly in Firefox. My current theory is that there may be certain lines of JavaScript code that are not executing correctly in Firefox. However, I am confident that I will soon be able to resolve this error and achieve browser compatibility. I will update this blog once I’ve resolved the browser compatibility issue. InshaAllah.

Debugging CSS Rendering Issues in Firefox: A Case Study on Styled Components

The problem with Firefox began when I noticed that some CSS properties weren’t rendering correctly. This became apparent when I entered inspect mode in Firefox, where I observed a recurring warning that escalated significantly during data pagination. The warning stated: “The component styled… has been created dynamically. You may see this warning because you’ve called styled inside another component.” Initially, this confused me because I had organized my styled components into a separate file. However, upon further investigation, I realized I had omitted two styled components from this file. Every time I performed data pagination, these components were re-rendered dynamically, which was impacting performance. After I relocated these two styled components to the proper stylesheet, the number of warnings in Firefox reduced dramatically — from 6000 to 71. This significant reduction confirms that while Firefox may render some CSS properties differently, the main issue was related to how styled components were managed in my project.

Feel free to view my project GitHub repositories:https://github.com/ezaanamin/Heal-Together

Please note that the project is still in development