6. JavaScript - promiseMap Utility Function

promiseMap is commonly used to batch-process a group of asynchronous tasks that only differ by the parameters passed in each time. However, we often want these tasks to run in a controlled pace, rather than all being triggered at once.

promiseMap(tasks, asyncFn, concurrency)

Why limit concurrency?

• To prevent server overload

• To avoid hitting browser concurrency limits (e.g., Chrome’s request limit per domain)

• To reduce the risk of API rate-limiting or account suspension

• To control resource consumption and reduce UI lag or performance issues

Typical Use Case Examples

These tasks typically share three common characteristics:

• They all rely on the same asynchronous function

• Each task differs only by its input parameters

• There is a need to control the number of tasks executing concurrently

Implementation of promiseMap

Our goal is to:

• Process each item in an input array by passing it to an asynchronous function

• Ensure that no more than N tasks run concurrently at any time

• Collect all results in an output array, preserving the original input order

function promiseMap(list, fn, concurrency = 1) {
    const results = []; // store the result of each promise
    let index = 0; // current index of the list
    let activeCount = 0; // record the number of now executing promises

    return new Promise((resolve, reject) => {
        function next() {
            if (index >= list.length && activeCount === 0) {
                // all promises task done
                return resolve(results)
            }

            // there are still tasks to be done and the number of now executing promises is 
            // less than concurrency
            while (activeCount < concurrency && index < list.length) {
                const currentIndex = index;
                const currentItem = list[index];
                index++; // move to the next item
                activeCount++; // increase the number of now executing promises

                Promise.resolve()
                .then(() => fn(currentItem))
                .then(result => {
                    results[currentIndex] = result; // store the result in the correct index
                    activeCount--;
                    next(); // start the next promise
                })
            }
        }
        next(); // start the first promise
    });
}

const urls = [
    'https://jsonplaceholder.typicode.com/posts/1',
    'https://jsonplaceholder.typicode.com/posts/2',
    'https://jsonplaceholder.typicode.com/posts/3',
    'https://jsonplaceholder.typicode.com/posts/4',
    'https://jsonplaceholder.typicode.com/posts/5'
 ];

function fetchUrl(url) {
    return new Promise((resolve) => {
        fetch(url)
            .then(response => response.json())
            .then(data => {
                console.log(`Fetched data from ${url}`); // this print order is not guaranteed, because of the response speed is different
                resolve(data);
             })
             .catch(error => {
                 console.error(`Error fetching ${url}:`, error);
                 resolve(null); // Resolve with null on error
             });
    })
}

promiseMap(urls, fetchUrl, 2).then((results) => {
    console.log(results);
});

// Output:
// Fetched data from https://jsonplaceholder.typicode.com/posts/1
// Fetched data from https://jsonplaceholder.typicode.com/posts/2
// Fetched data from https://jsonplaceholder.typicode.com/posts/4
// Fetched data from https://jsonplaceholder.typicode.com/posts/3
// Fetched data from https://jsonplaceholder.typicode.com/posts/5

// 0: {userId: 1, id: 1, title: 'sunt aut facere repellat ...
// 0: {userId: 1, id: 2, title: 'sunt aut facere repellat ...
// 0: {userId: 1, id: 3, title: 'sunt aut facere repellat ...
// 0: {userId: 1, id: 4, title: 'sunt aut facere repellat ...
// 0: {userId: 1, id: 5, title: 'sunt aut facere repellat ...

During the implementation of concurrency management, a scheduler function called next is at the core of controlling task execution. The promiseMap function returns a Promise that resolves once all tasks are completed.

Using Promise.resolve().then(...) makes your scheduling logic more generic, safe, and consistently asynchronous, while ensuring compatibility with both synchronous and asynchronous task functions.