Learning MongoDb with samples in Node Js

Introduction to NoSQL Databases (MongoDB)

1. Overview of NoSQL Databases

NoSQL databases, like MongoDB, handle a wide variety of data models and offer flexibility, scalability, and high availability. MongoDB uses a document-oriented data model, storing data in JSON-like documents.

2. Setting Up a MongoDB Database

Installation

1. Download MongoDB from the official website.

2. Follow the installation instructions for your operating system.

3. Start the MongoDB server by running mongod.

Creating a Database and Collections

In MongoDB, databases and collections are created implicitly when you first insert data into them.

// npm install mongodb --save
const { MongoClient } = require('mongodb');
const uri = "mongodb://localhost:27017";
// Create new instance of MongoDB
const client = new MongoClient(uri);

async function run() {
    try {
        // Connect to MongoDB
        await client.connect();

        // Create database and collections
        const database = client.db('school');
        const students = database.collection('students');
        const classes = database.collection('classes');
        const enrollment = database.collection('enrollment');

        console.log('Database and collections created');
    } finally {
        await client.close();
    }
}
run().catch(console.dir);

MongoDB Queries and Results

Common MongoDB Queries

1. Insert Documents

await students.insertMany([
            { _id: 1, name: 'Alice', age: 14, grade: '9' },
            { _id: 2, name: 'Bob', age: 15, grade: '10' },
            { _id: 3, name: 'Charlie', age: 13, grade: '8' }
        ]);

await classes.insertMany([
            { _id: 1, class_name: 'Math', teacher: 'Mr. Smith' },
            { _id: 2, class_name: 'Science', teacher: 'Ms. Johnson' }
        ]);

await enrollment.insertMany([
            { student_id: 1, class_id: 1 },
            { student_id: 2, class_id: 1 },
            { student_id: 2, class_id: 2 }
        ]);

2. Basic Find Query

const result = await students.find().toArray();
console.log(result);
// [
//     {
//         _id: new ObjectId('666300c8add741d7175c7de5'),
//         name: 'Alice',
//         age: 14,
//         grade: '9'
//     },
//     {
//         _id: new ObjectId('666300c8add741d7175c7de6'),
//         name: 'Bob',
//         age: 15,
//         grade: '10'
//     },
//     {
//         _id: new ObjectId('666300c8add741d7175c7de7'),
//         name: 'Charlie',
//         age: 13,
//         grade: '8'
//     }
// ]

3. Filtering Data

const result = await students.find({ age: { $gt: 13 } }).toArray();
console.log(result);
// [
//     {
//         _id: new ObjectId('666300c8add741d7175c7de5'),
//         name: 'Alice',
//         age: 14,
//         grade: '9'
//     },
//     {
//         _id: new ObjectId('666300c8add741d7175c7de6'),
//         name: 'Bob',
//         age: 15,
//         grade: '10'
//     }
// ]

• $gt: Stands for "greater than". This operator is used to filter documents where the field value is greater than the specified value.

4. Sorting Data

const result = await students.find().sort({ grade: 1 }).toArray();
console.log(result);
// [
//     {
//         _id: new ObjectId('666300c8add741d7175c7de6'),
//         name: 'Bob',
//         age: 15,
//         grade: '10'
//     },
//     {
//         _id: new ObjectId('666300c8add741d7175c7de7'),
//         name: 'Charlie',
//         age: 13,
//         grade: '8'
//     },
//     {
//         _id: new ObjectId('666300c8add741d7175c7de5'),
//         name: 'Alice',
//         age: 14,
//         grade: '9'
//     }
// ]

• 1: In the sort method, 1 specifies ascending order. Use -1 for descending order.

Joining Tables (Collections) with Aggregation

1. Inner Join: Fetch students along with the classes they are enrolled in

const result = await enrollment.aggregate([
    {
        $lookup: {
            from: 'students',
            localField: 'student_id',
            foreignField: '_id',
            as: 'student_info'
        }
    },
    {
        $lookup: {
            from: 'classes',
            localField: 'class_id',
            foreignField: '_id',
            as: 'class_info'
        }
    },
    { $unwind: '$student_info' },
    { $unwind: '$class_info' },
    {
        $project: {
            _id: 0,
            student_name: '$student_info.name',
            class_name: '$class_info.class_name'
        }
    }
]).toArray();
console.log(result);
// [
//     { student_name: 'Alice', class_name: 'Math' },
//     { student_name: 'Bob', class_name: 'Math' },
//     { student_name: 'Bob', class_name: 'Science' }
// ]

• $lookup: Performs a left outer join to a collection in the same database to filter in documents from the "joined" collection for processing.

  • from: The target collection to join.

  • localField: The field from the input documents.

  • foreignField: The field from the documents in the "from" collection.

  • as: The name of the new array field to add to the input documents.

• $unwind: Deconstructs an array field from the input documents to output a document for each element.

• $project: Reshapes each document in the stream by including, excluding, or adding new fields.

  • _id: 0 excludes the _id field.

  • $student_info.name: Refers to the name field in the student_info sub-document.

  • $class_info.class_name: Refers to the class_name field in the class_info sub-document.

Updating Data

1. Update a student's age

await students.updateOne(
    { name: 'Bob' },
    { $set: { age: 16 } }
);

• $set: Sets the value of a field in a document.

2. Update multiple fields

await students.updateOne(
    { name: 'Bob' },
    { $set: { age: 16, grade: '11' } }
);

• $set: Sets the value of multiple fields in a document.

Deleting Data

1. Delete a student

await students.deleteOne({ name: 'Charlie' });

2. Delete all students in a specific grade

await students.deleteMany({ grade: '9' });

3. Dropping Collections

await database.dropCollection('enrollment');
await database.dropCollection('students');
await database.dropCollection('classes');

Advanced MongoDB Queries

Subqueries

1. Find students who are enrolled in at least one class

const result = await students.find({
    _id: { $in: await enrollment.distinct('student_id') }
}).toArray();
console.log(result);
// [
//     { _id: 1, name: 'Alice', age: 14, grade: '9' },
//     { _id: 2, name: 'Bob', age: 15, grade: '10' }
// ]

• $in: Matches any of the values specified in an array. Here, it checks if _id is in the array of distinct student_ids from the enrollment collection.

2. Find classes with more than one student enrolled

const result = await classes.find({
    _id: {
        $in: await enrollment.aggregate([
            { $group: { _id: '$class_id', count: { $sum: 1 } } },
            { $match: { count: { $gt: 1 } } }
        ]).map(doc => doc._id).toArray()
    }
}).toArray();
console.log(result);
// [
//     { _id: 1, class_name: 'Math', teacher: 'Mr. Smith' },
//     { _id: 2, class_name: 'Science', teacher: 'Ms. Johnson' }
// ]

• $group: Groups documents by a specified identifier expression and applies an accumulator expression to each group.

  • _id: '$class_id': Groups by class_id.

  • count: { $sum: 1 }: Counts the number of documents in each group.

• $match: Filters the documents.

  • count: { $gt: 1 }: Filters groups where the count is greater than 1.

Union

1. List all student names and class names in one array

const studentNames = await students.distinct('name');
const classNames = await classes.distinct('class_name');
const allNames = studentNames.concat(classNames);
console.log(allNames);
// [ 'Bob', 'Math', 'Science' ]

Case Statements (Aggregation with $cond)

1. Categorize students by age group

const result = await students.aggregate([
    {
        $project: {
            name: 1,
            age_group: {
                $cond: {
                    if: { $lt: ['$age', 14] },
                    then: 'Junior',
                    else: {
                        $cond: {
                            if: { $lt: ['$age', 16] },
                            then: 'Intermediate',
                            else: 'Senior'
                        }
                    }
                }
            }
        }
    }
]).toArray();
console.log(result);
// [
//     {
//         _id: new ObjectId('666300c8add741d7175c7de6'),
//         name: 'Bob',
//         age_group: 'Senior'
//     }
// ]

• $project: Reshapes each document in the stream by including, excluding, or adding new fields.

  • name: 1: Includes the name field.

  • age_group: Adds a new field.

• $cond: Evaluates a boolean expression to return one of the two specified values.

  • if: The condition to evaluate.

  • then: The value to return if the condition is true.

  • else: The value to return if the condition is false.

• $lt: Checks if one value is less than another.

Nested Queries

1. Find the names of students who are enrolled in the same class as "Bob"

const bobId = (await students.findOne({ name: 'Bob' }))._id;
const bobClasses = await enrollment.distinct('class_id', { student_id: bobId });
const result = await students.find({
    _id: { $in: await enrollment.distinct('student_id', { class_id: { $in: bobClasses } }) }
}).toArray();
console.log(result);
// [
//     { _id: 1, name: 'Alice', age: 14, grade: '9' },
//     { _id: 2, name: 'Bob', age: 15, grade: '10' }
// ]

• $in: Matches any of the values specified in an array.

2. Find students with grades higher than the average grade

const avgGrade = await students.aggregate([
    { $group: { _id: null, avgGrade: { $avg: '$grade' } } }
]).toArray();
const result = await students.find({ grade: { $gt: avgGrade[0].avgGrade } }).toArray();
console.log(result);
// [
//     { _id: 2, name: 'Bob', age: 15, grade: '10' }
// ]

• $group: Groups documents by a specified identifier expression and applies an accumulator expression to each group.

  • _id: null: No specific grouping, so all documents are aggregated into one group.

  • avgGrade: { $avg: '$grade' }: Calculates the average grade.

• $avg: Computes the average of the numeric values that result from applying a specified expression to each document in a group.

Complex Join with Aggregation

1. Fetch the number of students in each class

const result = await classes.aggregate([
    {
        $lookup: {
            from: 'enrollment',
            localField: '_id',
            foreignField: 'class_id',
            as: 'enrollments'
        }
    },
    {
        $project: {
            class_name: 1,
            student_count: { $size: '$enrollments' }
        }
    }
]).toArray();
console.log(result);
// [
//     {
//         _id: new ObjectId('666300c8add741d7175c7de8'),
//         class_name: 'Math',
//         student_count: 0
//     },
//     {
//         _id: new ObjectId('666300c8add741d7175c7de9'),
//         class_name: 'Science',
//         student_count: 0
//     }
// ]

2. Fetch the class with the highest number of students

const result = await classes.aggregate([
    {
        $lookup: {
            from: 'enrollment',
            localField: '_id',
            foreignField: 'class_id',
            as: 'enrollments'
        }
    },
    {
        $project: {
            class_name: 1,
            student_count: { $size: '$enrollments' }
        }
    },
    { $sort: { student_count: -1 } },
    { $limit: 1 }
]).toArray();
console.log(result);
// [
//     {
//         _id: new ObjectId('666300c8add741d7175c7de8'),
//         class_name: 'Math',
//         student_count: 0
//     }
// ]

Change Stream (Triggers)

const pipeline = [
    { $match: { operationType: { $in: ['insert', 'update', 'delete'] } } }
];

const changeStream = students.watch(pipeline);

changeStream.on('change', async (next) => {
    const action = next.operationType.toUpperCase();
    const studentId = next.documentKey._id;

    await studentsHistory.insertOne({ action_time: new Date(), student_id: studentId, action: action });
    console.log(`Logged ${action} action for student_id: ${studentId}`);
});

• $match: Filters the documents.

  • operationType: Specifies the type of operation (insert, update, or delete).

• $in: Matches any of the values specified in an array.

Summary

In this comprehensive guide, we explored the concepts and practical implementations of MongoDB, a popular NoSQL database, by creating and manipulating collections that represent students, classes, enrollments, and students' history.

1. Collection Creation:

   • We created structured collections for students, classes, enrollment, and students_history. Each collection's fields and data types were defined, with unique identifiers generated by MongoDB's ObjectId.

2. Data Manipulation:

   • Insert Operations: Examples demonstrated how to insert documents into collections, including nested documents to represent relationships between students and classes.

   • Update Operations: The use of the $set operator to update specific fields within documents was showcased.

   • Delete Operations: Techniques for deleting specific documents based on conditions were provided.

3. Advanced Queries:

   • Nested Queries: We covered how to perform complex queries such as aggregations, sorting, filtering, and projecting specific fields from nested documents.

   • Aggregation Pipeline: The use of MongoDB's powerful aggregation framework to perform multi-stage data transformations and calculations was explained, with examples of $match, $group, $project, and $lookup stages.

4. Change Streams:

   • Simulating Triggers: We set up change streams to monitor real-time changes (inserts, updates, deletes) in the students collection and log these actions in the students_history collection, effectively simulating the behavior of SQL triggers.

Conclusion

This guide has provided a thorough understanding of MongoDB's capabilities, from basic operations to advanced queries and real-time change tracking. By mastering these techniques, you can effectively manage and manipulate data in a NoSQL database, leveraging MongoDB's flexibility and scalability.

Key takeaways include:

• The ability to design well-structured collections that represent real-world entities and their relationships.

• Proficiency in performing CRUD (Create, Read, Update, Delete) operations to manage data within MongoDB.

• The skills to construct complex queries using MongoDB's aggregation framework for sophisticated data analysis.

• Knowledge of change streams to monitor and react to real-time data changes, ensuring robust data integrity and auditing.

By applying these concepts and techniques, you can harness the full power of MongoDB to build efficient, scalable, and maintainable data-driven applications.