MongoDb - beginner to advance

Kumar HarshKumar Harsh
28 min read

Table of contents

Introduction to MongoDB and Mongo

Difference between MongoDB and Mongo?

MongoDB is a popular NoSQL database that allows for flexible and scalable data storage. In this blog post, we will cover the basics of MongoDB, how to get started, and some advanced features.

MongoDB

MongoDB is a document-based database that stores data in JSON-like documents.

Mongo

Mongo is the command-line interface (CLI) tool used to interact with MongoDB.

Downloading and Starting MongoDB

Downloading MongoDB

To download MongoDB, follow these steps:

  1. Go to the MongoDB download page and select the correct version for your operating system.

  2. Follow the installation instructions for your operating system.

Starting MongoDB

To start MongoDB, follow these steps:

Windows

  • Open a command prompt as an administrator.

  • Navigate to the MongoDB installation directory (usually C:\Program Files\MongoDB\Server\4.4\bin).

  • Run the command mongod.exe to start the MongoDB server.

macOS (via Homebrew)

  • Open a terminal.

  • Run the command brew services start mongodb to start the MongoDB server.

Linux

  • Open a terminal.

  • Run the command sudo systemctl start mongod to start the MongoDB server.

Importing JSON Data into MongoDB

To import JSON data into MongoDB, you can use the mongoimport command.

Example

Suppose we have a JSON file called data.json containing the following data:

[
  {
    "name": "John Doe",
    "age": 30
  },
  {
    "name": "Jane Doe",
    "age": 25
  }
]

To import this data into a MongoDB collection called users, run the following command:

mongoimport --db mydatabase --collection users --file data.json

Atomicity in MongoDB

Atomicity in MongoDB refers to the ability to perform multiple operations as a single, all-or-nothing unit. This ensures that either all operations succeed or none succeed.

Example

Suppose we want to update a document in the users collection and also insert a new document into the orders collection. We can use the $atomic operator to ensure that both operations succeed or fail together:

db.users.updateOne(
  { _id: ObjectId("...") },
  { $set: { name: "John Smith" } }
)

db.orders.insertOne(
  { userId: ObjectId("..."), total: 100 }
)
// Use $atomic to ensure both operations succeed or fail together
db.runCommand({
  update: "users",
  updates: [
    {
      q: { _id: ObjectId("...") },
      u: { $set: { name: "John Smith" } }
    }
  ],
  insert: "orders",
  documents: [
    { userId: ObjectId("..."), total: 100 }
  ],
  $atomic: true
})

Schema Validation in MongoDB

Schema validation in MongoDB allows you to define a schema for your documents and enforce it at the database level.

Example

Suppose we want to define a schema for the users collection that requires the name and age fields:

db.createCollection("users", {
  validator: {
    $jsonSchema: {
      required: ["name", "age"],
      properties: {
        name: {
          type: "string"
        },
        age: {
          type: "integer"
        }
      }
    }
  }
})

Embedded Documents in MongoDB

Embedded documents in MongoDB allow you to store multiple documents within a single document.

Example

Suppose we want to store a user's address as an embedded document:

db.users.insertOne({
  name: "John Doe",
  age: 30,
  address: {
    street: "123 Main St",
    city: "Anytown",
    state: "CA",
    zip: "12345"
  }
})

CRUD Operations in MongoDB

In this section, we will explore the different CRUD (Create, Read, Update, Delete) operations in MongoDB.

Delete in MongoDB

To delete a document in MongoDB, you can use the deleteOne or deleteMany method.

deleteOne

The deleteOne method deletes a single document that matches the specified filter.

db.users.deleteOne({ name: "John Doe" })

deleteMany

The deleteMany method deletes all documents that match the specified filter.

db.users.deleteMany({ age: { $gt: 30 } })

Find vs findOne

The find method returns a cursor that allows you to iterate over all documents that match the specified filter.

db.users.find({ age: { $gt: 30 } })

The findOne method returns a single document that matches the specified filter.

db.users.findOne({ name: "John Doe" })

insertOne vs insertMany

The insertOne method inserts a single document into the collection.

db.users.insertOne({ name: "John Doe", age: 30 })

The insertMany method inserts multiple documents into the collection.

db.users.insertMany([
  { name: "John Doe", age: 30 },
  { name: "Jane Doe", age: 25 },
  { name: "Bob Smith", age: 40 }
])

updateOne vs updateMany

The updateOne method updates a single document that matches the specified filter.

db.users.updateOne({ name: "John Doe" }, { $set: { age: 31 } })

The updateMany method updates all documents that match the specified filter.

db.users.updateMany({ age: { $gt: 30 } }, { $set: { age: 31 } })

Example Use Cases

Here are some example use cases for each of the CRUD operations:

  • Delete a user with the name "John Doe":
db.users.deleteOne({ name: "John Doe" })
  • Find all users with an age greater than 30:
db.users.find({ age: { $gt: 30 } })
  • Insert a new user with the name "Jane Doe" and age 25:
db.users.insertOne({ name: "Jane Doe", age: 25 })
  • Update the age of all users with an age greater than 30 to 31:
db.users.updateMany({ age: { $gt: 30 } }, { $set: { age: 31 } })

Note that these are just a few examples, and the actual use cases will depend on the specific requirements of your application.

Advanced MongoDB Features

In this section, we will explore some advanced features of MongoDB, including projections, ordered inserts, write concern, comparison operators, and logical operators.

Projections in MongoDB

Projections in MongoDB allow you to specify which fields to include or exclude from the result set.

Example

Suppose we have a collection called users with the following documents:

[
  {
    "_id": ObjectId("..."),
    "name": "John Doe",
    "age": 30,
    "address": {
      "street": "123 Main St",
      "city": "Anytown",
      "state": "CA",
      "zip": "12345"
    }
  },
  {
    "_id": ObjectId("..."),
    "name": "Jane Doe",
    "age": 25,
    "address": {
      "street": "456 Elm St",
      "city": "Othertown",
      "state": "NY",
      "zip": "67890"
    }
  }
]

To retrieve only the name and age fields, we can use the following projection:

db.users.find({}, { name: 1, age: 1 })

This will return the following result:

[
  {
    "_id": ObjectId("..."),
    "name": "John Doe",
    "age": 30
  },
  {
    "_id": ObjectId("..."),
    "name": "Jane Doe",
    "age": 25
  }
]

Ordered Option in Insert Command

The ordered option in the insert command allows you to specify whether the insert operation should be ordered or unordered.

Example

Suppose we have a collection called users and we want to insert multiple documents in a single operation:

db.users.insertMany([
  { name: "John Doe", age: 30 },
  { name: "Jane Doe", age: 25 },
  { name: "Bob Smith", age: 40 }
], { ordered: true })

If any of the inserts fail, the entire operation will be rolled back.

Write Concern in MongoDB

Write concern in MongoDB allows you to specify the level of acknowledgement required for a write operation.

Example

Suppose we have a collection called users and we want to insert a new document with a write concern of majority:

db.users.insertOne({ name: "John Doe", age: 30 }, { writeConcern: { w: "majority" } })

This will ensure that the write operation is acknowledged by a majority of the replica set members.

Comparison Operators

Comparison operators in MongoDB allow you to compare values in a query.

$eq

The $eq operator matches values that are equal to a specified value.

db.users.find({ age: { $eq: 30 } })

$ne

The $ne operator matches values that are not equal to a specified value.

db.users.find({ age: { $ne: 30 } })

$lt

The $lt operator matches values that are less than a specified value.

db.users.find({ age: { $lt: 30 } })

$gt

The $gt operator matches values that are greater than a specified value.

db.users.find({ age: { $gt: 30 } })

$lte

The $lte operator matches values that are less than or equal to a specified value.

db.users.find({ age: { $lte: 30 } })

$gte

The $gte operator matches values that are greater than or equal to a specified value.

db.users.find({ age: { $gte: 30 } })

$in

The $in operator matches values that are in an array of specified values.

db.users.find({ age: { $in: [30, 25, 40] } })

$nin

The $nin operator matches values that are not in an array of specified values.

db.users.find({ age: { $nin: [30, 25, 40] } })

Logical Operators in MongoDB

In this section, we will explore the logical operators in MongoDB, including $not, $and, $or, and $nor.

$not

The $not operator negates a condition.

db.users.find({ age: { $not: { $eq: 30 } } })

This will return all documents where the age field is not equal to 30.

$and

The $and operator combines multiple conditions with a logical AND.

db.users.find({ $and: [{ age: { $gt: 30 } }, { name: "John Doe" }] })

This will return all documents where the age field is greater than 30 and the name field is "John Doe".

$or

The $or operator combines multiple conditions with a logical OR.

db.users.find({ $or: [{ age: { $gt: 30 } }, { name: "Jane Doe" }] })

This will return all documents where the age field is greater than 30 or the name field is "Jane Doe".

$nor

The $nor operator combines multiple conditions with a logical NOR.

db.users.find({ $nor: [{ age: { $gt: 30 } }, { name: "John Doe" }] })

This will return all documents where the age field is not greater than 30 and the name field is not "John Doe".

$exists & $type Operators in MongoDB

In this section, we will explore the $exists and $type operators in MongoDB.

$exists

The $exists operator checks if a field exists in a document.

db.users.find({ address: { $exists: true } })

This will return all documents where the address field exists.

$type

The $type operator checks the type of a field in a document.

db.users.find({ age: { $type: "int" } })

This will return all documents where the age field is an integer.

Sort Data in MongoDB

In this section, we will explore how to sort data in MongoDB.

Sort by a Single Field

To sort data by a single field, you can use the sort() method.

db.users.find().sort({ age: 1 })

This will return all documents sorted by the age field in ascending order.

Sort by Multiple Fields

To sort data by multiple fields, you can use the sort() method with an array of fields.

db.users.find().sort({ age: 1, name: 1 })

This will return all documents sorted by the age field in ascending order, and then by the name field in ascending order.

Sort in Descending Order

To sort data in descending order, you can use the sort() method with a value of -1.

db.users.find().sort({ age: -1 })

This will return all documents sorted by the age field in descending order.

Example Use Cases

Here are some example use cases for the logical operators, $exists and $type operators, and sorting data:

  • Find all documents where the age field is greater than 30 and the name field is "John Doe":
db.users.find({ $and: [{ age: { $gt: 30 } }, { name: "John Doe" }] })
  • Find all documents where the address field exists:
db.users.find({ address: { $exists: true } })
  • Sort all documents by the age field in ascending order:
db.users.find().sort({ age: 1 })

Querying Arrays in MongoDB

In this section, we will explore how to query arrays in MongoDB using various comparison and logical operators.

Querying Arrays using Comparison Operators

MongoDB provides several comparison operators that can be used to query arrays.

$eq

The $eq operator matches values that are equal to a specified value.

db.users.find({ hobbies: { $eq: ["reading", "writing"] } })

This will return all documents where the hobbies field is an array containing the values "reading" and "writing".

$ne

The $ne operator matches values that are not equal to a specified value.

db.users.find({ hobbies: { $ne: ["reading", "writing"] } })

This will return all documents where the hobbies field is not an array containing the values "reading" and "writing".

$gt

The $gt operator matches values that are greater than a specified value.

db.users.find({ hobbies: { $gt: ["reading"] } })

This will return all documents where the hobbies field is an array containing values greater than "reading".

$lt

The $lt operator matches values that are less than a specified value.

db.users.find({ hobbies: { $lt: ["writing"] } })

This will return all documents where the hobbies field is an array containing values less than "writing".

$gte

The $gte operator matches values that are greater than or equal to a specified value.

db.users.find({ hobbies: { $gte: ["reading"] } })

This will return all documents where the hobbies field is an array containing values greater than or equal to "reading".

$lte

The $lte operator matches values that are less than or equal to a specified value.

db.users.find({ hobbies: { $lte: ["writing"] } })

This will return all documents where the hobbies field is an array containing values less than or equal to "writing".

Querying Arrays using Logical Operators

MongoDB provides several logical operators that can be used to query arrays.

$and

The $and operator combines multiple conditions with a logical AND.

db.users.find({ $and: [{ hobbies: { $eq: ["reading"] } }, { hobbies: { $eq: ["writing"] } }] })

This will return all documents where the hobbies field is an array containing both "reading" and "writing".

$or

The $or operator combines multiple conditions with a logical OR.

db.users.find({ $or: [{ hobbies: { $eq: ["reading"] } }, { hobbies: { $eq: ["writing"] } }] })

This will return all documents where the hobbies field is an array containing either "reading" or "writing".

$nor

The $nor operator combines multiple conditions with a logical NOR.

db.users.find({ $nor: [{ hobbies: { $eq: ["reading"] } }, { hobbies: { $eq: ["writing"] } }] })

This will return all documents where the hobbies field is not an array containing either "reading" or "writing".

$not

The $not operator negates a condition.

db.users.find({ hobbies: { $not: { $eq: ["reading"] } } })

This will return all documents where the hobbies field is not an array containing the value "reading".

Querying Arrays using $elemMatch

The $elemMatch operator allows you to specify a condition that must be met by at least one element in an array.

db.users.find({ hobbies: { $elemMatch: { $eq: "reading" } } })

This will return all documents where the hobbies field is an array containing at least one element that is equal to "reading".

Querying Arrays using $all

The $all operator allows you to specify an array of values that must be present in the array field.

db.users.find({ hobbies: { $all: ["reading", "writing"] } })

This will return all documents where the hobbies field is an array containing all of the values "reading" and "writing".

Example Use Cases

Here are some example use cases for querying arrays in MongoDB:

  • Find all documents where the hobbies field is an array containing the values "reading" and "writing":
db.users.find({ hobbies: { $eq: ["reading", "writing"] } })
  • Find all documents where the hobbies field is an array containing values greater than

Updating Documents in MongoDB

In this section, we will explore the various update operators in MongoDB, including $inc, $min, $max, $mul, $unset, $rename, and upsert.

$inc

The $inc operator increments the value of a field by a specified amount.

db.users.updateOne({ name: "John Doe" }, { $inc: { age: 1 } })

This will increment the age field of the document with the name "John Doe" by 1.

$min

The $min operator updates the value of a field to the minimum of the current value and a specified value.

db.users.updateOne({ name: "John Doe" }, { $min: { age: 30 } })

This will update the age field of the document with the name "John Doe" to the minimum of the current value and 30.

$max

The $max operator updates the value of a field to the maximum of the current value and a specified value.

db.users.updateOne({ name: "John Doe" }, { $max: { age: 40 } })

This will update the age field of the document with the name "John Doe" to the maximum of the current value and 40.

$mul

The $mul operator multiplies the value of a field by a specified value.

db.users.updateOne({ name: "John Doe" }, { $mul: { age: 2 } })

This will multiply the age field of the document with the name "John Doe" by 2.

$unset

The $unset operator removes a field from a document.

db.users.updateOne({ name: "John Doe" }, { $unset: { age: "" } })

This will remove the age field from the document with the name "John Doe".

$rename

The $rename operator renames a field in a document.

db.users.updateOne({ name: "John Doe" }, { $rename: { age: "yearsOld" } })

This will rename the age field to yearsOld in the document with the name "John Doe".

Upsert

Upsert is a combination of update and insert. If the document does not exist, it will be inserted. If it does exist, it will be updated.

db.users.updateOne({ name: "John Doe" }, { $set: { age: 30 } }, { upsert: true })

This will insert a new document with the name "John Doe" and age 30 if it does not exist. If it does exist, it will update the age to 30.

Example Use Cases

Here are some example use cases for the update operators in MongoDB:

  • Increment the age of a user by 1:
db.users.updateOne({ name: "John Doe" }, { $inc: { age: 1 } })
  • Update the age of a user to the minimum of the current value and 30:
db.users.updateOne({ name: "John Doe" }, { $min: { age: 30 } })
  • Remove the age field from a user document:
db.users.updateOne({ name: "John Doe" }, { $unset: { age: "" } })
  • Rename the age field to yearsOld in a user document:
db.users.updateOne({ name: "John Doe" }, { $rename: { age: "yearsOld" } })
  • Insert a new user document with the name "John Doe" and age 30 if it does not exist:
db.users.updateOne({ name: "John Doe" }, { $set: { age: 30 } }, { upsert: true })

Updating Nested Arrays in MongoDB: A Comprehensive Guide

MongoDB provides several operators for updating arrays, including $pop, $pull, $push, and $addToSet. In this blog post, we will explore how to use these operators to update nested arrays in MongoDB.

Prerequisites

  • MongoDB 3.6 or later

  • Basic understanding of MongoDB and its query language

Nested Arrays in MongoDB

Nested arrays are arrays that contain other arrays or objects as elements. For example:

{
  "_id" : ObjectId("..."),
  "name" : "John",
  "scores" : [
    {
      "subject" : "Math",
      "marks" : [90, 85, 95]
    },
    {
      "subject" : "Science",
      "marks" : [80, 90, 85]
    }
  ]
}

Updating Nested Arrays

$pop Operator

The $pop operator removes the first or last element of an array.

Example: Remove the last element of the marks array for the "Math" subject.

db.collection.updateOne(
  { _id: ObjectId("...") },
  { $pop: { "scores.0.marks": -1 } }
)

Result:

{
  "_id" : ObjectId("..."),
  "name" : "John",
  "scores" : [
    {
      "subject" : "Math",
      "marks" : [90, 85]
    },
    {
      "subject" : "Science",
      "marks" : [80, 90, 85]
    }
  ]
}

$pull Operator

The $pull operator removes all elements from an array that match a specified condition.

Example: Remove all scores less than 80 from the marks array for the "Science" subject.

db.collection.updateOne(
  { _id: ObjectId("...") },
  { $pull: { "scores.1.marks": { $lt: 80 } } }
)

Result:

{
  "_id" : ObjectId("..."),
  "name" : "John",
  "scores" : [
    {
      "subject" : "Math",
      "marks" : [90, 85, 95]
    },
    {
      "subject" : "Science",
      "marks" : [90, 85]
    }
  ]
}

$push Operator

The $push operator adds a new element to the end of an array.

Example: Add a new score of 92 to the marks array for the "Math" subject.

db.collection.updateOne(
  { _id: ObjectId("...") },
  { $push: { "scores.0.marks": 92 } }
)

Result:

{
  "_id" : ObjectId("..."),
  "name" : "John",
  "scores" : [
    {
      "subject" : "Math",
      "marks" : [90, 85, 95, 92]
    },
    {
      "subject" : "Science",
      "marks" : [80, 90, 85]
    }
  ]
}

$addToSet Operator

The $addToSet operator adds a new element to an array only if it does not already exist.

Example: Add a new subject "History" with a score of 88 to the scores array.

db.collection.updateOne(
  { _id: ObjectId("...") },
  { $addToSet: { scores: { subject: "History", marks: [88] } } }
)

Result:

{
  "_id" : ObjectId("..."),
  "name" : "John",
  "scores" : [
    {
      "subject" : "Math",
      "marks" : [90, 85, 95]
    },
    {
      "subject" : "Science",
      "marks" : [80, 90, 85]
    },
    {
      "subject" : "History",
      "marks" : [88]
    }
  ]
}

Indexing in MongoDB: A Comprehensive Guide

Indexing is a crucial aspect of database management that can significantly improve query performance. In MongoDB, indexing allows you to efficiently retrieve data by creating a data structure that facilitates quick lookup and retrieval of documents. In this blog post, we will explore the concept of indexing in MongoDB, its benefits, and how to create and manage indexes.

What is Indexing in MongoDB?

In MongoDB, an index is a data structure that improves the speed of data retrieval operations by providing a quick way to locate specific data. Indexes are built on top of a field or a set of fields in a collection, allowing MongoDB to efficiently locate and retrieve documents that match a query.

Benefits of Indexing in MongoDB

  1. Improved Query Performance: Indexing can significantly improve the performance of queries by reducing the number of documents that need to be scanned.

  2. Faster Data Retrieval: Indexes enable MongoDB to quickly locate and retrieve specific data, reducing the time it takes to execute queries.

  3. Reduced Load on the Server: By improving query performance, indexing can reduce the load on the server, allowing it to handle more requests and improve overall system performance.

Types of Indexes in MongoDB

  1. Single Field Index: An index created on a single field in a collection.

  2. Compound Index: An index created on multiple fields in a collection.

  3. Multikey Index: An index created on an array field in a collection.

  4. Text Index: An index created on a string field in a collection, used for text search.

  5. Hashed Index: An index created on a field in a collection, used for equality queries.

Creating Indexes in MongoDB

Single Field Index

To create a single field index, use the createIndex() method:

db.collection.createIndex({ field: 1 })

Example: Create an index on the name field in the users collection:

db.users.createIndex({ name: 1 })

Compound Index

To create a compound index, use the createIndex() method with multiple fields:

db.collection.createIndex({ field1: 1, field2: 1 })

Example: Create a compound index on the name and email fields in the users collection:

db.users.createIndex({ name: 1, email: 1 })

Multikey Index

To create a multikey index, use the createIndex() method with the multikey option:

db.collection.createIndex({ field: 1 }, { multikey: true })

Example: Create a multikey index on the scores field in the students collection:

db.students.createIndex({ scores: 1 }, { multikey: true })

Text Index

To create a text index, use the createIndex() method with the text option:

db.collection.createIndex({ field: "text" })

Example: Create a text index on the description field in the products collection:

db.products.createIndex({ description: "text" })

Hashed Index

To create a hashed index, use the createIndex() method with the hashed option:

db.collection.createIndex({ field: "hashed" })

Example: Create a hashed index on the id field in the users collection:

db.users.createIndex({ id: "hashed" })

Managing Indexes in MongoDB

Listing Indexes

To list all indexes in a collection, use the getIndexes() method:

db.collection.getIndexes()

Example: List all indexes in the users collection:

db.users.getIndexes()

Dropping Indexes

To drop an index, use the dropIndex() method:

db.collection.dropIndex({ field: 1 })

Example: Drop the index on the name field in the users collection:

db.users.dropIndex({ name: 1 })

Indexing in MongoDB: A Comprehensive Guide

Indexing is a crucial aspect of database management that can significantly improve query performance. In MongoDB, indexing allows you to efficiently retrieve data by creating a data structure that facilitates quick lookup and retrieval of documents. In this blog post, we will explore the concept of indexing in MongoDB, its benefits, and how to create and manage indexes.

What is Indexing in MongoDB?

In MongoDB, an index is a data structure that improves the speed of data retrieval operations by providing a quick way to locate specific data. Indexes are built on top of a field or a set of fields in a collection, allowing MongoDB to efficiently locate and retrieve documents that match a query.

Benefits of Indexing in MongoDB

  1. Improved Query Performance: Indexing can significantly improve the performance of queries by reducing the number of documents that need to be scanned.

  2. Faster Data Retrieval: Indexes enable MongoDB to quickly locate and retrieve specific data, reducing the time it takes to execute queries.

  3. Reduced Load on the Server: By improving query performance, indexing can reduce the load on the server, allowing it to handle more requests and improve overall system performance.

Types of Indexes in MongoDB

  1. Single Field Index: An index created on a single field in a collection.

  2. Compound Index: An index created on multiple fields in a collection.

  3. Multikey Index: An index created on an array field in a collection.

  4. Text Index: An index created on a string field in a collection, used for text search.

  5. Hashed Index: An index created on a field in a collection, used for equality queries.

Creating Indexes in MongoDB

Single Field Index

To create a single field index, use the createIndex() method:

db.collection.createIndex({ field: 1 })

Example: Create an index on the name field in the users collection:

db.users.createIndex({ name: 1 })

Compound Index

To create a compound index, use the createIndex() method with multiple fields:

db.collection.createIndex({ field1: 1, field2: 1 })

Example: Create a compound index on the name and email fields in the users collection:

db.users.createIndex({ name: 1, email: 1 })

Multikey Index

To create a multikey index, use the createIndex() method with the multikey option:

db.collection.createIndex({ field: 1 }, { multikey: true })

Example: Create a multikey index on the scores field in the students collection:

db.students.createIndex({ scores: 1 }, { multikey: true })

Text Index

To create a text index, use the createIndex() method with the text option:

db.collection.createIndex({ field: "text" })

Example: Create a text index on the description field in the products collection:

db.products.createIndex({ description: "text" })

Hashed Index

To create a hashed index, use the createIndex() method with the hashed option:

db.collection.createIndex({ field: "hashed" })

Example: Create a hashed index on the id field in the users collection:

db.users.createIndex({ id: "hashed" })

Managing Indexes in MongoDB

Listing Indexes

To list all indexes in a collection, use the getIndexes() method:

db.collection.getIndexes()

Example: List all indexes in the users collection:

db.users.getIndexes()

Dropping Indexes

To drop an index, use the dropIndex() method:

db.collection.dropIndex({ field: 1 })

Example: Drop the index on the name field in the users collection:

db.users.dropIndex({ name: 1 })

Aggregation in MongoDB: A Comprehensive Guide

Aggregation in MongoDB is a powerful feature that allows you to process and transform data in a collection. It provides a way to perform complex queries, group data, and calculate aggregate values. In this blog post, we will explore the different types of aggregation in MongoDB and provide examples for each.

What is Aggregation in MongoDB?

Aggregation in MongoDB is a pipeline-based framework that allows you to process data in a collection. It consists of a series of stages, each of which performs a specific operation on the data. The output of each stage is passed to the next stage, allowing you to perform complex transformations and calculations.

Types of Aggregation in MongoDB

1. $match Stage

The $match stage filters the documents in a collection based on a condition.

Example: Find all documents in the orders collection where the total is greater than 100.

db.orders.aggregate([
  {
    $match: {
      total: { $gt: 100 }
    }
  }
])

Result:

[
  {
    "_id" : ObjectId("..."),
    "total" : 150,
    "items" : [...]
  },
  {
    "_id" : ObjectId("..."),
    "total" : 200,
    "items" : [...]
  }
]

2. $project Stage

The $project stage transforms the documents in a collection by adding, removing, or modifying fields.

Example: Add a new field totalPrice to each document in the orders collection.

db.orders.aggregate([
  {
    $project: {
      totalPrice: { $multiply: ["$total", 1.1] }
    }
  }
])

Result:

[
  {
    "_id" : ObjectId("..."),
    "totalPrice" : 165
  },
  {
    "_id" : ObjectId("..."),
    "totalPrice" : 220
  }
]

3. $group Stage

The $group stage groups the documents in a collection by a field and calculates aggregate values.

Example: Group the documents in the orders collection by the customerId field and calculate the total amount spent by each customer.

db.orders.aggregate([
  {
    $group: {
      _id: "$customerId",
      totalAmount: { $sum: "$total" }
    }
  }
])

Result:

[
  {
    "_id" : ObjectId("..."),
    "totalAmount" : 500
  },
  {
    "_id" : ObjectId("..."),
    "totalAmount" : 1000
  }
]

4. $sort Stage

The $sort stage sorts the documents in a collection by a field.

Example: Sort the documents in the orders collection by the total field in descending order.

db.orders.aggregate([
  {
    $sort: {
      total: -1
    }
  }
])

Result:

[
  {
    "_id" : ObjectId("..."),
    "total" : 200
  },
  {
    "_id" : ObjectId("..."),
    "total" : 150
  }
]

5. $limit Stage

The $limit stage limits the number of documents returned in the result.

Example: Return only the top 5 documents in the orders collection with the highest total.

db.orders.aggregate([
  {
    $sort: {
      total: -1
    }
  },
  {
    $limit: 5
  }
])

Result:

[
  {
    "_id" : ObjectId("..."),
    "total" : 200
  },
  {
    "_id" : ObjectId("..."),
    "total" : 150
  },
  {
    "_id" : ObjectId("..."),
    "total" : 120
  },
  {
    "_id" : ObjectId("..."),
    "total" : 100
  },
  {
    "_id" : ObjectId("..."),
    "total" : 90
  }
]

6. $skip Stage

The $skip stage skips a specified number of documents in the result.

Example: Skip the first 5 documents in the orders collection and return the next 5 documents.

db.orders.aggregate([
  {
    $sort: {
      total: -1
    }
  },
  {
    $skip: 5
  },
  {
    $limit: 5
  }
])

Result:

[
  {
    "_id" : ObjectId("..."),
    "total" : 80
  },
  {
    "_id" : ObjectId("..."),
    "total" : 70
  },
  {
    "_id" : ObjectId("..."),
    "total" : 60
  },
  {
    "_id" : ObjectId("..."),
    "total" : 50
  },
  {
    "_id" : ObjectId("..."),
    "total" : 40
  }
]

$bucket and $lookup in MongoDB

In MongoDB, $bucket and $lookup are two powerful aggregation operators that can be used to perform complex data analysis and transformation.

$bucket Operator

The $bucket operator is used to categorize documents into different buckets based on a specified condition. It is commonly used to perform data analysis and create histograms.

Example: Use $bucket to categorize documents in the orders collection into different buckets based on the total field.

db.orders.aggregate([
  {
    $bucket: {
      groupBy: "$total",
      boundaries: [0, 100, 200, 300, 400],
      default: "Other",
      output: {
        count: { $sum: 1 }
      }
    }
  }
])

Result:

[
  {
    "_id": "0-100",
    "count": 10
  },
  {
    "_id": "100-200",
    "count": 20
  },
  {
    "_id": "200-300",
    "count": 30
  },
  {
    "_id": "300-400",
    "count": 40
  },
  {
    "_id": "Other",
    "count": 50
  }
]

$lookup Operator

The $lookup operator is used to perform a left outer join between two collections. It is commonly used to combine data from multiple collections.

Example: Use $lookup to join the orders collection with the customers collection based on the customerId field.

db.orders.aggregate([
  {
    $lookup: {
      from: "customers",
      localField: "customerId",
      foreignField: "_id",
      as: "customer"
    }
  }
])

Result:

[
  {
    "_id": ObjectId("..."),
    "total": 100,
    "customerId": ObjectId("..."),
    "customer": [
      {
        "_id": ObjectId("..."),
        "name": "John Doe",
        "email": "john.doe@example.com"
      }
    ]
  },
  {
    "_id": ObjectId("..."),
    "total": 200,
    "customerId": ObjectId("..."),
    "customer": [
      {
        "_id": ObjectId("..."),
        "name": "Jane Doe",
        "email": "jane.doe@example.com"
      }
    ]
  }
]

Note that the $lookup operator returns an array of documents from the customers collection that match the customerId field in the orders collection.

Using $bucket and $lookup Together

You can use $bucket and $lookup together to perform complex data analysis and transformation.

Example: Use $bucket to categorize documents in the orders collection into different buckets based on the total field, and then use $lookup to join the resulting documents with the customers collection.

db.orders.aggregate([
  {
    $bucket: {
      groupBy: "$total",
      boundaries: [0, 100, 200, 300, 400],
      default: "Other",
      output: {
        count: { $sum: 1 }
      }
    }
  },
  {
    $lookup: {
      from: "customers",
      localField: "customerId",
      foreignField: "_id",
      as: "customer"
    }
  }
])

Result:

[
  {
    "_id": "0-100",
    "count": 10,
    "customer": [
      {
        "_id": ObjectId("..."),
        "name": "John Doe",
        "email": "john.doe@example.com"
      }
    ]
  },
  {
    "_id": "100-200",
    "count": 20,
    "customer": [
      {
        "_id": ObjectId("..."),
        "name": "Jane Doe",
        "email": "jane.doe@example.com"
      }
    ]
  },
  {
    "_id": "200-300",
    "count": 30,
    "customer": [
      {
        "_id": ObjectId("..."),
        "name": "Bob Smith",
        "email": "bob.smith@example.com"
      }
    ]
  }
]

Note that the resulting documents contain the bucket information and the corresponding customer data.

Capped Collections in MongoDB

In MongoDB, a capped collection is a type of collection that has a fixed size and automatically removes the oldest documents when the collection reaches its maximum size. Capped collections are useful for storing data that needs to be kept for a limited amount of time, such as log data or real-time analytics data.

Creating a Capped Collection

To create a capped collection, you can use the createCollection method with the capped option set to true. You must also specify the size option to set the maximum size of the collection.

Example: Create a capped collection called logs with a maximum size of 100MB.

db.createCollection("logs", {
  capped: true,
  size: 100 * 1024 * 1024
})

Inserting Documents into a Capped Collection

You can insert documents into a capped collection using the insert method. When the collection reaches its maximum size, the oldest documents will be automatically removed to make room for new documents.

Example: Insert a document into the logs collection.

db.logs.insert({
  timestamp: new Date(),
  logLevel: "INFO",
  message: "This is a log message"
})

Querying a Capped Collection

You can query a capped collection using the find method. Because capped collections automatically remove the oldest documents, you may not be able to retrieve all documents that were inserted into the collection.

Example: Find all documents in the logs collection.

db.logs.find()

Using the $natural Operator

The $natural operator allows you to retrieve documents from a capped collection in the order they were inserted. This can be useful for retrieving the most recent documents.

Example: Find the 10 most recent documents in the logs collection.

db.logs.find().sort({ $natural: -1 }).limit(10)

Using the max Option

The max option allows you to specify the maximum number of documents to return from a capped collection.

Example: Find the 10 most recent documents in the logs collection, and return at most 10 documents.

db.logs.find().sort({ $natural: -1 }).limit(10).max(10)

Dropping a Capped Collection

To drop a capped collection, you can use the dropCollection method.

Example: Drop the logs collection.

db.logs.drop()

Note that dropping a capped collection will delete all documents in the collection, and the collection will no longer be available for use.

0
Subscribe to my newsletter

Read articles from Kumar Harsh directly inside your inbox. Subscribe to the newsletter, and don't miss out.

mongodb-basic-to-advancemongodb-commandsMongoDBmongoosemongodbshellmongo

Written by

Kumar Harsh
Kumar Harsh

Experienced Project Engineer skilled in developing and integrating Micro Frontend applications to enhance functionality and user experience in unified systems. Proficient in CI/CD pipeline automation using Bitbucket, Amazon S3, and CloudFront, streamlining deployments with efficient, scalable cloud infrastructure. Expertise in creating end-to-end automated testing with Cypress, integrating continuous quality checks into deployment workflows, and automating issue tracking through Jira to improve productivity. Skilled in React, Node.js, MongoDB, and cloud tools (AWS, Azure), with a strong foundation in backend/frontend development and DevOps practices.