Recently AWS has announced the preview release of Amazon OpenSearch Serverless, a new serverless option for Amazon OpenSearch Service that makes it easy for you to run large-scale search and analytics workloads without having to configure, manage, or scale OpenSearch clusters.

With Amazon OpenSearch Serverless, you can simply create a collection and start ingesting and querying data. The service automatically provisions and scales the underlying resources to deliver fast data ingestion and query responses for even the most demanding and unpredictable workloads.

Here are some of the key benefits of Amazon OpenSearch Serverless:

No cluster management: You don't have to worry about provisioning, configuring, or scaling OpenSearch clusters.
Automatic scaling: The service automatically scales the underlying resources based on your workload demands.
Fast data ingestion: Amazon OpenSearch Serverless can ingest data at up to 100 GB per second.
Fast query responses: Amazon OpenSearch Serverless can return query results in milliseconds.
Cost-effective: You only pay for the resources you use.

If you are looking for a simple, cost-effective way to run large-scale search and analytics workloads, then Amazon OpenSearch Serverless is a great option. To learn more.

Here are some examples of how you can use Amazon OpenSearch Serverless:

Log Analytics: Amazon OpenSearch Serverless can be used to collect, index, and analyze log data from your applications and infrastructure.
Web Search: Amazon OpenSearch Serverless can be used to build a search engine for your website or application.
Product Search: Amazon OpenSearch Serverless can be used to power a product search feature on your e-commerce website.
Fraud Detection: Amazon OpenSearch Serverless can be used to detect fraudulent activity in your data.

Getting started with Amazon OpenSearch Serverless

This tutorial walks you through the basic steps to get an Amazon OpenSearch Serverless collection up and running quickly.

If you are using an IAM user, you should have the following IAM role in order to create the OpenSearch Serverless Cluster
see Identity and Access Management for Amazon OpenSearch Serverless.

Create an IAM user With Admin Access.

Create a collection

A collection in Amazon OpenSearch Serverless is a logical grouping of one or more indexes that represent an analytics workload. OpenSearch Service automatically manages and tunes the collection, requiring minimal manual input.

For example, you could create a collection for your website's search feature. The collection would contain the indexes for all of the pages on your website, as well as the indexes for the search terms that users have entered. OpenSearch Service would automatically manage the size and performance of the collection, so you wouldn't have to worry about it.

To create an OpenSearch Serverless collection

Open the Amazon OpenSearch Service console at https://console.aws.amazon.com/aos/home.
Choose Collections in the left navigation pane and choose Create collection.

Provide a name for the collection,
Choose a collection type:
- Search – This collection type is designed for full-text search workloads. All search data is stored in hot storage to ensure fast query response times. This type of collection is a good choice for applications that require fast search performance, such as e-commerce websites, content management systems, and legal document searches.
- Time series – This collection type is designed for log analytics workloads. At least 24 hours of data is cached in hot storage, and the rest remains in warm storage. This type of collection is a good choice for applications that require fast access to recent log data, such as application monitoring, security analytics, and fraud detection.

For this tutorial, we are moving ahead with time-series

Under Encryption, choose an AWS KMS key to encrypt your data. OpenSearch Serverless notifies you if the collection name that you entered matches a pattern defined in an encryption policy. You can choose to keep this match or override it with unique encryption settings
For Access type, select public or VPC access. If you choose to enable access through a virtual private cloud (VPC)
For Data access, click on skip and configure later, we will configure it once we create the collection and click on create collection.
The Collection should look like the below image.

Create a data access policy Amazon OpenSearch Serverless:

Your collection data won't be accessible until you configure data access. Data access policies are separate from the IAM identity-based policy that you configured above. They allow users to access the actual data within a collection

You can grant access to data in OpenSearch Serverless by creating data access policies. Data access policies are JSON documents that define who has access to what data. You can create data access policies for individual collections or for entire domains.

To create a data access policy, you need to specify the following:

The principals who are allowed to access the data. Principals can be IAM roles, SAML identities, or IP addresses.
The actions that the principals are allowed to perform. Actions can include indexing data, searching data, and managing indexes.
The resources that the principals are allowed to access. Resources can be collections or indexes.

Once you have created a data access policy, you can attach it to a collection or domain. When you attach a data access policy to a collection or domain, the policy will be applied to all of the indexes in that collection or domain.

Data access control is a powerful feature that allows you to protect your data in Amazon OpenSearch Serverless. By creating data access policies, you can ensure that only authorized users have access to your data.

Here are some additional benefits of using data access control in Amazon OpenSearch Serverless:

You can easily grant or revoke access to data.
You can control who can index, search, and manage indexes.
You can protect your data from unauthorized access.

To create a data access policy

Choose Data access policies in the left navigation pane and choose Create access policy.
Name the policy data-policy, and Select Visual Editor (or you can also use the json) for the policy definition method.
Next, Select the rule, And enter the rule name "Rule 1"

Choose Add principals, IAM users and roles and select the user or role that you'll use to sign in to OpenSearch Dashboards and index data. Choose Save

Here i am selecting the IAM user "os-user" which we have created recently, and click on save

Under Index permissions, select all of the permissions.

This one is created using the Console, if you want to use the json code, then you can use the below json code:

OpenSearch Dashboards provides sample datasets that come with visualizations, dashboards, and other tools to help you explore Dashboards before you add your own data. To create indexes from this sample data, you need a data access policy that provides permissions to the dataset that you want to work with. The following policy uses a wildcard (*) to provide permissions to all three sample datasets.

To index and search data in the collection :

Now you can Sign in to AWS account using proviusly created IAM user and navigate to the Amazon OpenSearch Service, and click on collection, you will get the below page

Then, choose Collections in the left navigation pane and choose the "opensearch-serverless" collection to open its details page.

Choose the OpenSearch Dashboards URL for the collection. The URL takes the format https://collection-id.us-east-1.aoss.amazonaws.com/_dashboards.x

Next, if you want to explore the opensearch, you can upload some sample data, in this blog i have add the eCommerce order sample data.

You can open the dashboard inside the opensearch. This dashboard contains sample data for you to play with. You can view it, search it, and interact with the visualizations. For more information about OpenSearch Dashboards, check the docs

Conclusion:

Thank you for sticking to the end. In this blog, we have learned how we can create AWS OpenSearch serverless collectrion. So, This is really very quick and simple. If you like this blog, please share my blog and show your appreciation by giving thumbs-ups, and don’t forget to give me suggestions on how I can improve my future blogs that can suit your needs.

References:

https://docs.aws.amazon.com/opensearch-service/latest/developerguide/serverless-getting-started.html

Amazon OpenSearch Serverless: The Easy Way to Run Large-Scale Search and Analytics