3 posts tagged with "AWS CLI"

In today's fast-paced tech world, automating infrastructure setup is key to maximizing efficiency and reducing human error. One of the most reliable tools for this is AWS CloudFormation, which allows users to define their cloud resources and manage them as code. While AWS provides a Console for managing CloudFormation, the AWS Command Line Interface (CLI) is a powerful alternative that offers speed, control, and flexibility. In this blog, we'll walk you through setting up CloudFormation using AWS CLI, covering essential components like VPCs, KMS keys, and IAM roles.

1. Introduction to AWS CloudFormation​

Before diving into technical details, it's important to understand what AWS CloudFormation is and why it's so beneficial.

What is AWS CloudFormation?​

AWS CloudFormation is an Infrastructure-as-Code (IaC) service provided by AWS that allows you to model, provision, and manage AWS and third-party resources. You define your resources using template files (JSON or YAML) and deploy them via AWS CloudFormation, which takes care of the provisioning and configuration.

CloudFormation manages the entire lifecycle of your resources, from creation to deletion, allowing for automation and consistent environments.

Benefits of Using CloudFormation​

1. Automation: CloudFormation automates the entire infrastructure setup, from VPC creation to IAM role configuration, reducing manual work and errors.

2. Version Control: Treat your infrastructure like code. With CloudFormation, you can manage your infrastructure in repositories like Git, making it easy to version, track, and rollback changes.

3. Consistency: CloudFormation ensures that the same template can be used to create identical environments, such as development, staging, and production.

4. Cost Efficiency: With CloudFormation, resources can be automatically deleted when no longer needed, preventing unnecessary costs from unused resources.

2. Why Use AWS CLI Over the Console?​

AWS CLI vs Console: Which One is Better for You?​

The AWS Management Console offers an intuitive, visual interface for managing AWS resources, but it's not always the most efficient way to manage infrastructure, especially when it grows complex. Here's how AWS CLI compares:

Advantages of Using AWS CLI​

1. Automation: CLI commands can be scripted for automation, allowing you to run tasks without manually navigating through the Console.
2. Faster Setup: CLI allows you to automate stack creation, updates, and deletion, significantly speeding up the setup process.
3. Better Error Handling: You can incrementally update stacks and fix errors on the go with AWS CLI, making it easier to debug and manage resources.

3. Prerequisites​

Before we start building with CloudFormation, let’s go over the prerequisites.

Setting Up AWS CLI​

AWS CLI is a powerful tool that allows you to manage AWS services from the command line. To get started:

1. Install AWS CLI:

   • If you haven't already installed AWS CLI, you can do so by following the instructions from the official AWS documentation.

2. Verify Installation: After installation, verify that the AWS CLI is installed by typing the following command in your terminal:

   aws --version
   

   If successfully installed, the version information will be displayed.

Configuring AWS Profiles​

Before using AWS CLI to interact with your AWS account, you'll need to configure a profile:

aws configure

You'll be prompted to provide:

• AWS Access Key ID
• AWS Secret Access Key
• Default region name (e.g., us-west-2)
• Default output format (choose JSON)

This configuration will allow the CLI to authenticate and interact with your AWS account.

4. Step-by-Step Guide to AWS CloudFormation with AWS CLI​

Now that your CLI is set up, let us get into how to deploy AWS CloudFormation stacks using it.

Setting Up Your First CloudFormation Stack​

We will start with a simple example of how to create a CloudFormation stack. Suppose you want to create a Virtual Private Cloud (VPC).

1. Create a YAML Template: Save the following template in a file named vpc.yaml:

AWSTemplateFormatVersion: '2010-09-09'
Resources:
  MyVPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: 10.0.0.0/16
      Tags:
        - Key: Name
          Value: MyVPC

2. Deploy the Stack: To create the VPC, run the following command:

aws cloudformation create-stack --stack-name my-vpc-stack --template-body file://vpc.yaml --capabilities CAPABILITY_NAMED_IAM

This command will instruct CloudFormation to spin up a VPC using the specified template.

3. Check the Stack Status: To verify the status of your stack creation, use:

aws cloudformation describe-stacks --stack-name my-vpc-stack

Deploying a Virtual Private Cloud (VPC)​

A VPC is essential for defining your network infrastructure in AWS. Here’s how you can add more resources to your VPC, such as an Internet Gateway:

Resources:
  MyVPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: 10.0.0.0/16
      Tags:
        - Key: Name
          Value: MyVPC
  InternetGateway:
    Type: AWS::EC2::InternetGateway
  VPCGatewayAttachment:
    Type: AWS::EC2::VPCGatewayAttachment
    Properties:
      VpcId: !Ref MyVPC
      InternetGatewayId: !Ref InternetGateway

Deploy this using the same create-stack command.

Setting Up Security with KMS (Key Management Service)​

Next, we will add encryption keys for securing data:

1. KMS Template:

Resources:
  MyKMSKey:
    Type: AWS::KMS::Key
    Properties:
      Description: Key for encrypting data
      Enabled: true

2. Deploy KMS:

aws cloudformation create-stack --stack-name my-kms-stack --template-body file://kms.yaml --capabilities CAPABILITY_NAMED_IAM

Managing Access with IAM Roles​

IAM Roles allow secure communication between AWS services. Here’s an example of how to create an IAM role:

Resources:
  MyIAMRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service: ec2.amazonaws.com
            Action: sts:AssumeRole
      Path: /

Use the same create-stack command to deploy this.

5. Best Practices for AWS CloudFormation​

Use Nested Stacks​

Avoid large, monolithic stacks. Break them down into smaller, nested stacks for better manageability.

Resources:
  ParentStack:
    Type: AWS::CloudFormation::Stack
    Properties:
      TemplateURL: https://s3.amazonaws.com/path/to/nested-stack.yaml

Parameterization​

Use parameters to make your stacks reusable across different environments.

Parameters:
  InstanceType:
    Type: String
    Default: t2.micro
    Description: EC2 Instance Type

Exporting and Referencing Outputs​

Export important resource values for use in other stacks:

Outputs:
  VPCId:
    Value: !Ref MyVPC
    Export:
      Name: VPCId

Incremental Stack Updates​

Always update your stacks incrementally to avoid failures.

aws cloudformation update-stack --stack-name my-stack --template-body file://updated-template.yaml

6. Advanced CloudFormation Features​

Handling Dependencies and Stack Failures​

Use the DependsOn attribute to specify dependencies between resources to avoid issues with resource creation order.

Custom Resource Creation​

For advanced use cases, you can create custom resources by using Lambda functions or CLI.

7. Conclusion and Next Steps​

By using AWS CloudFormation with AWS CLI, you can automate your infrastructure, reduce errors, and scale your environment effortlessly. Continue learning by experimenting with more complex templates, incorporating advanced features like stack sets, and automating further with scripts.

Code shown in the video can be accessed from https://github.com/arinatechnologies/cloudformation

Pre-requisites​

• Ensure your SSO User has CodeCommit access.
• You should have GitBash CLI installed on your machine.
• Able to install Python preferably 3.12 version.
• Able to install other software such as git-remote-codecommit.

Setup AWS Profile​

The second main feature we want to enable is AWS SSO login from the AWS Command Line Interface (AWS CLI) on our local machine.

aws configure sso
SSO start URL [None]: https://<sso-name>.awsapps.com/start/#
SSO region [None]:us-east-1

You will be redirected to your default browser. Or copy the link provided in your browser and ensure the code provided matches what is shown in CLI.

In case you have access to more than 1 account, when you return to the CLI, you must choose your account.

There are 2 AWS accounts available to you.
> AdministratorAccess, <email> (<Account1>)
> AdministratorAccess, <email2> (Account2)

Choose the account with your CodeCommit repository.

Next, you see the permissions sets available to you in the account you just picked.

You now see the options for the profile you’re creating for these AWS SSO permissions:

CLI default client Region [None]: us-east-1<ENTER>
CLI default output format [None]: json<ENTER>
CLI profile name [<Account1>-Developer]: Dev-profile<ENTER>

Note: In GitBash, if you get an error such as:

http://aws.amazon.com/cli
http://aws.amazon.com/cli
https://asg-infra.awsapps.com/start/#/console?account_id=735360830536&role_name=AdministratorAccess

You can run the command from CMD or another WSL.

Git Bash Setup​

Python Installation​

To install Python on Git Bash, follow these steps:

1. Download Python:

   • Visit the official Python downloads page.
   • Choose the latest version of Python for your operating system (Windows) and download the installer.

2. Install Python:

   • Run the Installer:
     • Locate the downloaded installer file and double-click to run it.
   • Customize Installation:
     • Check the box that says "Add Python to PATH". This is crucial as it allows you to use Python from the command line.
   • Click on "Customize installation" for more options if needed.
   • Choose Optional Features:
     • You can leave the default options checked. Click "Next".
   • Advanced Options:
     • Leave the default options or adjust as needed. Click "Install".

3. Verify Python Installation:

   • Open Git Bash:
   • Type the following command and press Enter:

     python --version
     

   • You should see the version of Python that you installed.

4. Install pip:

   • pip usually comes bundled with Python, but if it's not available, you can install it manually.

     curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
     python get-pip.py
     

   • Verify pip Installation:

     pip --version
     

Installing git-remote-codecommit​

To install git-remote-codecommit in Git Bash:

1. Install with the following code:

   pip install git-remote-codecommit
   

2. For some operating systems, you might need to run:

   sudo pip install git-remote-codecommit
   

3. Clone the code from one of your repositories:

   git clone codecommit://<profile name>@<CodeCommit repo name>
   

   Example:

   git clone codecommit://AdministratorAccess-735360830536@asg-admin
   

Reconnect if Session Expired​

If your SSO session expires, follow these steps to reconnect:

1. Run the following command in Git Bash or another WSL:

   aws sso login --sso-session <session name>
   

2. If you have forgotten the session name, you can find it in C:\Users\<UserName>\.aws\config.

3. Follow the steps where a URL will open and accept as shown.

GitHub Desktop Setup​

1. Ensure git-remote-codecommit is installed in Git Bash CLI as described above.
2. Follow the instructions provided to use GitHub Desktop.

Visual Studio Code Setup​

1. Ensure git-remote-codecommit is installed in Git Bash CLI as described above.
2. Follow the provided highlights. Ready to take your cloud infrastructure to the next level? Please reach out to us Contact Us
   

Introduction​

This approach of using EC2 Endpoint compared to a Bastion host not only streamlines the connection process but also saves costs associated with maintaining a bastion host.

The Traditional Method​

Typically, to connect to an EC2 instance in a private subnet, you would use a bastion host. The process involves:

1. Setting up a bastion host in a public subnet.
2. Connecting to the bastion host from your local machine.
3. Using the bastion host to access the EC2 instance in the private subnet.

While this method is effective, it can be cumbersome and costly.

The New Approach: EC2 Instance Connect Endpoint​

AWS recently introduced a new service called EC2 Instance Connect Endpoint. This service allows you to connect directly to your EC2 instance in a private subnet without the need for a bastion host.

Here's how to set it up:

Step-by-Step Guide​

Set Up Your VPC and Subnets​

First, create a Virtual Private Cloud (VPC) with both public and private subnets:

1. Public Subnet: This is where you will create the EC2 Instance Connect Endpoint.
2. Private Subnet: This is where your EC2 instance will reside.
3. Delete Default VPC: Start by deleting the default VPC in your AWS region to avoid conflicts.
4. Create a New VPC:
   1. Navigate to the VPC console.
   2. Choose “VPC and more” and give it a suitable name (e.g., "YouTubeDemoVPC").
   3. Create one public subnet and one private subnet.
   4. No need for an S3 gateway in this setup.

Step 2: Launching the EC2 Instance​

1. Launch an EC2 Instance in the Private Subnet:

   1. Go to the EC2 console and launch a new instance.
   2. Choose the private subnet for your instance.
   3. Select an instance type (e.g., t-nano).
   4. Create a key pair or use an existing one for SSH access.

2. Set Up Security Groups:

   1. Create a new security group allowing all traffic within the VPC.
   2. Ensure the security group is attached to your EC2 instance.

Step 3: Creating the EC2 Instance Endpoint​

1. Navigate to the VPC console and select “Endpoints.”

2. Create a new endpoint and attach it to the VPC.

3. Assign a security group to the endpoint that allows all traffic.

4. Configure Security Groups:

   1. Ensure the public security group for the endpoint allows inbound traffic from your local IP.

Step 4: Connecting via AWS CLI​

1. Install and Configure AWS CLI:

   1. Ensure AWS CLI is installed on your local machine.
   2. Configure AWS CLI with your access keys.

2. Connect to the EC2 Instance:

   1. Use the following command with the correct private key and endpoint details to connect to the EC2 instance.

   ssh -i <keyName>.pem <username>@<instance-id> -o ProxyCommand="aws ec2-instance-connect open-tunnel --instance-id <instance-id>" --profile <profile-name>
   

Step 5: Connecting via WinSCP​

1. Download and Install WinSCP:

   1. Install WinSCP for a graphical interface to manage files on your EC2 instance.

2. Set Up WinSCP:

1. Open WinSCP and configure a new session.

   1. Enter the private IP of the EC2 instance.

   2. Use the private key for authentication.

   3. Set up a proxy command to use the EC2 endpoint.

      1. Click Advanced --> Connection --> Proxy --> Local and enter following command:

      aws ec2-instance-connect open-tunnel --instance-id <instance-id>
      

2. Connect and Manage Files:

1. Save the session settings and connect.
   1. You should now be able to manage files on your EC2 instance via WinSCP.

Troubleshooting Common Issues​

1. Permission Denied Errors:

2. Ensure your private key file has the correct permissions (chmod 400 your-key.pem).

3. Verify the security group rules allow inbound traffic from your IP.

4. Endpoint Initialization Issues:

5. Check the VPC and subnet configurations.

6. Ensure the endpoint is associated with the correct security group.

Benefits of Using EC2 Instance Endpoints​

1. Cost Savings: Avoid additional costs associated with running a bastion host.
2. Simplicity: Simplify the connection process by eliminating the need for an intermediary host.
3. Security: Maintain secure access to instances in private subnets without exposing a bastion host.

Conclusion​

Using EC2 instance endpoints is a powerful and cost-effective way to manage instances in private subnets. This guide has provided a comprehensive walkthrough of setting up and connecting to an EC2 instance without a bastion host, utilizing both AWS CLI and WinSCP. Implementing this approach can streamline your workflow and reduce costs, making your cloud infrastructure more efficient and manageable.

🔚 Call to Action​

Choosing the right platform depends on your organizations needs. For more insights, subscribe to our newsletter for insights on cloud computing, tips, and the latest trends in technology. or follow our video series on cloud comparisons.

Interested in having your organization setup on cloud? If yes, please contact us and we'll be more than glad to help you embark on cloud journey.

💬 Comment below:
Which tool is your favorite? What do you want us to review next?