Reverse engineering CoinMarketCap API

Earlier, I discussed how to set up a TradingView webhook and send notifications to telegram for stocks.

The full workflow looks something like this:

• Before the market opens, send a telegram notification with stock data from TradingView, VIX Central, and CNN fear and greed index.

• When the market closes, receive the TradingView webhook and save the data in Redis.

Crypto telegram notification workflow

There is a similar workflow for crypto as well. Before the market opens, a telegram notification is sent with data from CoinMarketCap and the alternative.me fear and greed index. However, there is no TradingView webhook to receive data from.

Locating the data

Note: This post is for educational purpose only.

This is the page on CoinMarketCap where I found the data for the 24-hour price change by sectors.

This page is server-side rendered (SSR), so there is no API request for this data when you inspect the network tab.

CoinMarketCap uses NextJS, which stores the SSR data in window.__NEXT_DATA__, and the data we are interested in is located at props.pageProps.data.

You can also find it in the page source.

Finding the API request

Getting the data from HTML is fine, but having an API is better.

In Chrome's source tab, locate the Next.js bundle under the s2.coinmarketcap.com/cmc/_next/static/chunks/pages folder. There, you will find _app**.js and cryptocurrency-category-**.js.

_app**.js contains the application-level code that is bundled for every page.

cryptocurrency-category-**.js contains the code specific to this page.

Older version of cryptocurrency-category**.js

I still have an older copy of the JavaScript file for this page, which was retrieved sometime last year. The API call is found within a call to F().

getInitialProps is a NextJS lifecycle method that runs when the page is loaded.

Clicking into the function reveals that the endpoint is at /data-api/v3/sector/w/list.

Requesting data from the endpoint gives the outdated version of the top sectors by 24-hour price change, which doesn't match what is shown on the page.

Current version of cryptocurrency-category**.js

With the updated version of cryptocurrency-category**.js, the API call is at L().

Clicking into the function reveals that the endpoint is /data-api/v4/sector/m/full-list.

Requesting data from the endpoint provides the current list of top sectors by 24-hour price change, which is displayed on the page.

API domain

The above 2 requests referenced the javaApi endpoint, which can be found in _app**.js

Tips

NextJS knowledge

Since the page is written in NextJS, it's helpful to know where NextJS stores client-side data and understand the lifecycle method for fetching data.

Trial and error
Use the good old debugger to set a breakpoint, reload the page, and check if it gets triggered. If it does, there's a good chance it's the API request we're looking for.