One post tagged with "Snapshots"

In the world of data management and cloud computing, ensuring data security through regular backups is crucial. OpenSearch and Elasticsearch provide robust mechanisms to back up data using snapshots, offering several approaches to cater to different operational needs. This blog post will walk you through setting up and managing snapshots using AWS, with detailed steps for both beginners and advanced users.

Introduction to Snapshots in OpenSearch and Elasticsearch​

Snapshots are point-in-time backups of your OpenSearch or Elasticsearch data. By taking snapshots at regular intervals, you can ensure your data is always backed up, which is especially important in production environments. Snapshots can be scheduled to run automatically, whether hourly, daily, or at another preferred frequency, making it easy to maintain a stable backup routine.

Setting Up an OpenSearch Cluster on AWS​

Before diving into snapshot creation, its essential to set up an OpenSearch cluster. Here is how:

1. AWS Console Access: Begin by logging into your AWS Console and navigating to OpenSearch.
2. Cluster Creation: Create a new OpenSearch domain (essentially your cluster) using the "Easy Create" option. This option simplifies the setup process, especially for demonstration or learning purposes.
3. Instance Selection: For this setup, select a lower instance size if you are only exploring OpenSearch features and dont require high memory or compute power. For this demo, an m5.large instance with minimal nodes is sufficient.

Configuring the Cluster​

When configuring the cluster, adjust the settings according to your requirements:

1. Memory and Storage: Set minimal storage (e.g., 10 GB) to avoid unnecessary costs.
2. Node Count: Choose a single-node setup if you are only testing the system.
3. Access Control: For simplicity, keep public access open, though in production, you should configure a VPC and control access strictly.

Snapshot Architecture: AWS Lambda and S3 Buckets​

AWS provides a serverless approach to managing snapshots via Lambda and S3 buckets. Here is the basic setup:

1. Create an S3 Bucket: This bucket will store your OpenSearch snapshots.

2. Lambda Function for Snapshot Automation: Use AWS Lambda to automate the snapshot process. Configure the Lambda function to run daily or at a frequency of your choice, ensuring backups are consistent and reliable.

Writing the Lambda Code​

For the Lambda function, Python is a convenient choice, but you can choose other languages as well. The Lambda function will connect to OpenSearch, initiate a snapshot, and store it in the S3 bucket. Here is a simple breakdown of the code structure:

import boto3, os, time
import requests
from requests_aws4auth import AWS4Auth
from datetime import datetime
import logging

from requests.adapters import HTTPAdapter, Retry

# Set the global variables
# include https:// and trailing /
host = str(os.getenv('host'))
region = str(os.getenv('region','eu'))
s3Bucket = str(os.getenv('s3Bucket'))
s3_base_path = str(os.getenv('s3_base_path','daily'))
s3RepoName = str(os.getenv('s3RepoName'))
roleArn = str(os.getenv('roleArn'))

service = 'es'
credentials = boto3.Session().get_credentials()
awsauth = AWS4Auth(credentials.access_key, credentials.secret_key, region, service, session_token=credentials.token)

s3 = boto3.client('s3')

def lambda_handler(event, context):
    datestamp = datetime.now().strftime('%Y-%m-%dt%H:%M:%S')

    # Register repository
    # the Elasticsearch API endpoint

    path = '_snapshot/'+s3RepoName
    url = host + path

    snapshotName = 'snapshot-'+datestamp

    # Setting for us-east-1. Comment below if another region
    payload = {
    "type": "s3",
    "settings": {
        "bucket": s3Bucket,
        "base_path": s3_base_path, 
        "endpoint": "s3.amazonaws.com",
        "role_arn": roleArn
        }
    }

    headers = {"Content-Type": "application/json"}

    r = requests.put(url, auth=awsauth, json=payload, headers=headers)

    print(r.status_code)
    print(r.text)

    # Take snapshot - Even though this looks similar to above, but this code is required to take snapshot.    
    # Snapshot to take with datestamp concatanetated - this creates separate snapshots
    path = '_snapshot/'+s3RepoName+'/'+snapshotName

    url = host + path

    string = snapshotName
    bucket_name = s3Bucket
    
	
    s3 = boto3.resource("s3")
    s3.Bucket(bucket_name).put_object(Key=s3_path, Body=string)
    print(f"Created {s3_path}")
    ### Text File copying ends here

    while True:
        response = requests.put(url, auth=awsauth)
        status_code = response.status_code
        print("status_code == "+str(status_code))
        if status_code >= 500:
            # Hope it won't 500 a little later
            print("5xx thrown. Sleeping for 200 seconds.. zzzz...")
            time.sleep(200)
        else:
            print(f"Snapshot {snapshotName} successfully taken")
            break
    
    print(r.text)

1. Snapshot API Call: The code uses the OpenSearch API to trigger snapshot creation. You can customize the frequency to take snapshots.
2. Error Handling: In scenarios where snapshots take long, retries and error handling are implemented in to manage API call failures.
3. Permissions Setup: Grant your Lambda function the necessary permissions to access OpenSearch and the S3 bucket. This includes setting up roles and policies in AWS Identity and Access Management (IAM).
4. Invocation Permissions: Lambda function will need to have role that allows access to OpenSearch domain. The role should allow Lambda to upload snapshots to s3 bucket:

{
            "Effect": "Allow",
            "Action": [
                "s3:PutObject"
            ],
            "Resource": [
                "arn:aws:s3:::<BUCKET-NAME>/*"
            ]
        }

Creating an AWS Lambda Layer for the Requests Library​

To create a custom AWS Lambda layer specifically for the requests library, follow these steps. This guide will help you set up the requests package as a Lambda layer so it can be reused across multiple Lambda functions.

Follow these steps to create a custom AWS Lambda layer that includes the requests library.

Step 1: Prepare the Requests Dependency​

Since Lambda layers require dependencies to be packaged separately, we need to install the requests library in a specific structure.

Set Up a Local Directory​

Create a folder structure for installing the dependency.

mkdir requests-layer
cd requests-layer
mkdir python

Install the Requests Library​

Use pip to install requests into the python folder:

pip install requests -t python/

Verify Installation​

Check that the python directory contains the installed requests package:

ls python

You should see a folder named requests, confirming that the package was installed successfully.

Step 2: Create a Zip Archive of the Layer​

After installing the dependencies, zip the python directory:

zip -r requests-layer.zip python

This creates a requests-layer.zip file that you will upload as a Lambda layer.

Step 3: Upload the Layer to AWS Lambda​

1. Open the AWS Lambda Console
2. Go to the AWS Lambda Console.
3. Create a New Layer

4. Select Layers from the left-hand navigation.

5. Click Create layer.
6. Configure the Layer
7. Name: Provide a name like requests-layer.
8. Description: Optionally, describe the purpose of the layer.
9. Upload the .zip file: Choose the requests-layer.zip file you created.
10. Compatible runtimes: Choose the runtime(s) that match your Lambda function, such as Python 3.8, Python 3.9, or Python 3.10. 11.Create the Layer
11. Click Create to upload the layer.

Step 4: Add the Layer to Your Lambda Function​

1.Open Your Lambda Function 2.In the Lambda Console, open the Lambda function where you want to use requests. 3.Add the Layer 4.In the Layers section, click Add a layer. 5.Select Custom layers and choose the requests-layer. 6.Select the specific version (if there are multiple versions). 7.Click Add.

OpenSearch Dashboard Configuration​

The OpenSearch Dashboard (formerly Kibana) is your go-to for managing and monitoring OpenSearch. Here is how to set up your snapshot role in the dashboard:

1. Access the Dashboard: Navigate to the OpenSearch Dashboard using the provided domain link.
2. Role Setup: Go to the security settings and create a new role for managing snapshots. Grant this role permissions to access the necessary indices and S3 bucket. Following is the role that needs to be created:

Trust Policy:​

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": [
                    "opensearch.amazonaws.com",
                    "es.amazonaws.com"
                ]
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

Role Policy:​

{
   "Version": "2012-10-17",
   "Statement": [
       {
           "Effect": "Allow",
           "Action": "s3:ListBucket",
           "Resource": [
               "arn:aws:s3:::BUCKET-NAME"
           ]
       },
       {
           "Effect": "Allow",
           "Action": [
               "s3:*Object"
           ],
           "Resource": [
               "arn:aws:s3:::BUCKET-NAME/*"
           ]
       },
       {
           "Sid": "ESaccess",
           "Effect": "Allow",
           "Action": [
               "es:*"
           ],
           "Resource": [
               "arn:aws:es:eu-west-2:<ACCOUNT-NUMBER>:domain/*"
           ]
       }
   ]
}

3. Mapping the Role: Map the new role to your Lambda functions IAM role to ensure seamless access.

Setting Up Snapshot Policies in the Dashboard​

The OpenSearch Dashboard allows you to create policies for managing snapshots, making it easy to define backup schedules and retention periods. Here is how:

1. Policy Configuration: Define your backup frequency (daily, weekly, etc.) and the retention period for each snapshot.
2. Retention Period: Set the maximum number of snapshots to keep, ensuring that old snapshots are automatically deleted to save space.

3. Notification Channel: You can set up notifications (e.g., via Amazon SNS) to alert you if a snapshot operation fails.

Testing and Troubleshooting Your Snapshot Setup​

Once your setup is complete, it is time to test it:

1. Run a Test Snapshot: Trigger your Lambda function manually and check your S3 bucket for the snapshot data.
2. Verify Permissions: If you encounter errors, check your IAM roles and permissions. Snapshot failures often occur due to insufficient permissions, so make sure both the OpenSearch and S3 roles are configured correctly.
3. Monitor Logs: Use CloudWatch logs to review the execution of your Lambda function, which will help in troubleshooting any issues that arise.

Disaster Recovery and Restoring Snapshots​

In the unfortunate event of data loss or a disaster, restoring your data from snapshots is straightforward. Here is a simple guide:

1. New Cluster Setup: If your original cluster is lost, create a new OpenSearch domain.
2. Restore Snapshot: Use the OpenSearch API to restore the snapshot from your S3 bucket.
3. Cluster Health Check: Once restored, check the health of your cluster and validate that your data is fully recovered.

Conclusion​

Using AWS Lambda and S3 for snapshot management in OpenSearch provides a scalable and cost-effective solution for data backup and recovery. By setting up automated snapshots, you can ensure that your data is consistently backed up without manual intervention. With the additional security and monitoring tools provided by AWS, maintaining the integrity and availability of your OpenSearch data becomes a manageable task.

Explore the various options within AWS and OpenSearch to find the configuration that best fits your environment. And as always, remember to test your setup thoroughly to prevent unexpected issues down the line.

For more tips on OpenSearch, AWS, and other cloud solutions, subscribe to our newsletter and stay up-to-date with the latest in cloud technology! Ready to take your cloud infrastructure to the next level? Please reach out to us