Anup Deshpande – Projects AWS Web application deployment using AWS PAAS & Beanstalk

Web application deployment using AWS PAAS & Beanstalk

July 4, 2024July 4, 2024| DMAnupDMAnup| 0 Comments| 2:40 PM
Categories:

Introduction

Welcome to the first article in my DevOps & SRE learning series. Here, I’ll share my experiences and solutions using the STAR format (Situation, Task, Action, Result). Today, we’ll explore how to migrate an existing web application to AWS PAAS and Beanstalk.

Situation

The client needed to:
– Shift their Tomcat-based web application and its components from physical servers to a managed platform.
– Minimize infrastructure maintenance.
– Focus more on application enhancements.
– Lower latency for users across the globe.

Task

Provide a Proof of Concept (POC) demonstrating:
– AWS Beanstalk for Tomcat instance replacement, load balancing, and auto-scaling.
– AWS RDS, Elasticache, and Amazon MQ as managed services.
– AWS Cloudfront implementation.

Action

Given below are the detailed steps for the entire execution to create all the components. All the below script were executed from awscli which is configured on my machine.

Security Group for the Backend components:

– Dedicated Security Group for all the backend components
– RDS DB, MQ & Elasticache needs to be created
– Self access where all these components can interact internally
– An inbound rule is added from the self security group id once security grp is created.

Show/Hide Script


#!/bin/bash
# Author: Anup Deshpande
# Date: 25-June-2024
# Dedicated Security Group for all the backend components
# RDS DB, MQ & Elasticache  needs to be created
# Self access where all these components can interact internally 
# An inbound rule is added from the self security group id once security grp is created.

# Set the region
REGION="ap-south-1"

# Create the security group and get its ID
SECURITY_GROUP_ID=$(aws ec2 create-security-group \
    --group-name dmanup-demo-aws-paas-backend-secgrp \
    --description "Security group for backend of AWS PaaS demo" \
    --vpc-id $(aws ec2 describe-vpcs --query 'Vpcs[0].VpcId' --output text --region $REGION) \
    --region $REGION \
    --query 'GroupId' \
    --output text)

# Add an inbound rule to allow all traffic from within the same security group
aws ec2 authorize-security-group-ingress \
    --group-id $SECURITY_GROUP_ID \
    --protocol -1 \
    --port all \
    --source-group $SECURITY_GROUP_ID \
    --region $REGION

# Output the created security group ID
echo "Created Security Group ID: $SECURITY_GROUP_ID"

Amazon RDS:

– Script to create & setup the AWS RDS DB with MySQL
– Creates a separate subnet & parameter group
– Creates & stores the credentials into Secrets Manager
– Once DB is fully available a temp Ubuntu based EC2 instance is created
– IAM role to connect to the DB is used to run the sql for setting up the required tables.
– EC2 Public IP is added into the Inbound rule of the RDS DB Sec Grp
– Post the successful tables creation, the temp EC2 instance & its associated IAM role,instance profile, inbound rule for EC2 inst from DB Sec Grp is also removed.

Show/Hide Script


#!/bin/bash
# Author: Anup Deshpande
# Date: 25-June-2024
# Script to create & setup the AWS RDS DB with MySQL
# Creates a separate subnet & parameter group
# Creates & stores the credentials into Secrets Manager
# Once DB is fully available a temp Ubuntu based EC2 instance is created
# IAM role to connect to the DB is used to run the sql for setting up the required tables.
# EC2 Public IP is added into the Inbound rule of the RDS DB Sec Grp 
# Post the successful tables creation, the temp EC2 instance & its associated IAM role,   
# instance profile, inbound rule for EC2 inst from DB Sec Grp is also removed.

set -e

# Set the region
REGION="ap-south-1"

echo "Setting region to $REGION."

# Fetch the default VPC ID
VPC_ID=$(aws ec2 describe-vpcs --query 'Vpcs[0].VpcId' --output text --region $REGION)
echo "Fetched VPC ID: $VPC_ID."

# Fetch the first two subnet IDs in the default VPC
SUBNET_IDS=$(aws ec2 describe-subnets --filters "Name=vpc-id,Values=$VPC_ID" --query 'Subnets[0:2].SubnetId' --output text --region $REGION)
echo "Fetched Subnet IDs: $SUBNET_IDS."

# Create the RDS subnet group with the fetched subnet IDs
aws rds create-db-subnet-group \
    --db-subnet-group-name dmaup-aws-paas-demo-db-subnt \
    --db-subnet-group-description "Subnet group for AWS PaaS demo RDS DB" \
    --subnet-ids $SUBNET_IDS \
    --region $REGION
echo "Created RDS subnet group."

# Parameter Group Creation
aws rds create-db-parameter-group \
    --db-parameter-group-name dmaup-aws-paas-demo-rds-db-param-grp \
    --db-parameter-group-family mysql8.0 \
    --description "Parameter group for AWS PaaS demo RDS DB" \
    --region $REGION
echo "Created RDS parameter group."

# Fetch the security group ID for the given security group name
SECURITY_GROUP_ID=$(aws ec2 describe-security-groups --filters "Name=group-name,Values=dmanup-demo-aws-paas-backend-secgrp" --query "SecurityGroups[0].GroupId" --output text --region $REGION)
echo "Fetched Security Group ID: $SECURITY_GROUP_ID."

# Check if the secret already exists
SECRET_NAME="RDSDB_Credentials1"
if aws secretsmanager describe-secret --secret-id $SECRET_NAME --region $REGION > /dev/null 2>&1; then
# Fetch the RDS password and endpoint from AWS Secrets Manager
   RDS_PASSWORD=$(aws secretsmanager get-secret-value --secret-id $SECRET_NAME --region $REGION --query "SecretString" --output text | sed -e 's/^.*"password":"\([^"]*\)".*$/\1/')
    echo "Secret $SECRET_NAME already exists."
else
    # Generate a secure password with allowed characters
    RDS_PASSWORD=$(openssl rand -base64 16 | tr -dc 'a-zA-Z0-9!#$%&()*+,-.:;<=>?@[]^_`{|}~' | head -c 16)
    echo "Generated RDS password."

    # Store RDS credentials in AWS Secrets Manager
    aws secretsmanager create-secret --name $SECRET_NAME --description "RDS MySQL DB credentials" \
        --secret-string "{\"username\":\"admin\",\"password\":\"$RDS_PASSWORD\"}" --region $REGION
    echo "Stored RDS credentials in AWS Secrets Manager."
fi


# Create the RDS instance using the fetched security group ID
aws rds create-db-instance \
    --db-instance-identifier dmaup-aws-paas-demo-rdsdb \
    --db-instance-class db.t3.micro \
    --engine mysql \
    --master-username admin \
    --master-user-password $RDS_PASSWORD \
    --allocated-storage 20 \
    --db-subnet-group-name dmaup-aws-paas-demo-db-subnt \
    --vpc-security-group-ids $SECURITY_GROUP_ID \
    --db-parameter-group-name dmaup-aws-paas-demo-rds-db-param-grp \
    --backup-retention-period 0 \
    --no-storage-encrypted \
    --no-multi-az \
    --no-publicly-accessible \
    --region $REGION \
    --no-auto-minor-version-upgrade \
    --port 3306 \
    --db-name accounts \
    --engine-version 8.0 \
    --storage-type gp2
echo "Created RDS instance."

# Wait until the RDS instance is available
aws rds wait db-instance-available --db-instance-identifier dmaup-aws-paas-demo-rdsdb --region $REGION
echo "RDS instance is now available."

# Get the RDS endpoint
RDS_ENDPOINT=$(aws rds describe-db-instances --db-instance-identifier dmaup-aws-paas-demo-rdsdb --region $REGION --query "DBInstances[0].Endpoint.Address" --output text)
echo "Fetched RDS endpoint: $RDS_ENDPOINT."

# Check if the IAM role already exists
ROLE_NAME="EC2AccessSecretsManagerRole"
if aws iam get-role --role-name $ROLE_NAME --region $REGION 2>/dev/null; then
    echo "Role $ROLE_NAME already exists."
else
    # Create IAM Role for EC2 to access Secrets Manager and RDS
    aws iam create-role --role-name $ROLE_NAME --assume-role-policy-document '{
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Principal": {
            "Service": "ec2.amazonaws.com"
          },
          "Action": "sts:AssumeRole"
        }
      ]
    }' --region $REGION
    echo "Created IAM role $ROLE_NAME."

    # Attach policy to the role to allow access to Secrets Manager and RDS
    aws iam attach-role-policy --role-name $ROLE_NAME --policy-arn arn:aws:iam::aws:policy/AmazonRDSFullAccess --region $REGION
    aws iam attach-role-policy --role-name $ROLE_NAME --policy-arn arn:aws:iam::aws:policy/SecretsManagerReadWrite --region $REGION
    echo "Attached policies to IAM role."

    # Create instance profile and attach role
    aws iam create-instance-profile --instance-profile-name $ROLE_NAME --region $REGION
    aws iam add-role-to-instance-profile --instance-profile-name $ROLE_NAME --role-name $ROLE_NAME --region $REGION
    echo "Created instance profile and attached role."

    # Wait for the instance profile to be available
    sleep 10
fi

# Fetch the latest Ubuntu AMI ID dynamically
AMI_ID=$(aws ec2 describe-images --owners amazon --filters "Name=name,Values=ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-*" "Name=state,Values=available" --query "Images | sort_by(@, &CreationDate) | [-1].ImageId" --output text --region $REGION)
echo "Fetched Ubuntu AMI ID: $AMI_ID."

# Your local machine's IP address
LOCAL_IP=$(curl -s http://checkip.amazonaws.com/)
echo "Fetched local IP: $LOCAL_IP."

# Create an EC2 instance in the same security group as RDS
INSTANCE_TYPE="t3.micro"
KEY_NAME="Temp_Key_Pair"
KEY_PATH="/d/devops_articles/aws-paas-beanstalk/vprofile-project/src/main/resources/Temp_Key_Pair.pem"

# Check if the key pair already exists
if ! aws ec2 describe-key-pairs --key-name $KEY_NAME --region $REGION 2>/dev/null; then
    # Create a new key pair and save it to a file
    aws ec2 create-key-pair --key-name $KEY_NAME --query 'KeyMaterial' --output text --region $REGION > $KEY_PATH
    chmod 400 $KEY_PATH
    echo "Created key pair $KEY_NAME."
fi

INSTANCE_ID=$(aws ec2 run-instances \
    --image-id $AMI_ID \
    --instance-type $INSTANCE_TYPE \
    --key-name $KEY_NAME \
    --security-group-ids $SECURITY_GROUP_ID \
    --subnet-id $(aws ec2 describe-subnets --filters "Name=vpc-id,Values=$VPC_ID" --query 'Subnets[0].SubnetId' --output text --region $REGION) \
    --iam-instance-profile Name=$ROLE_NAME \
    --query 'Instances[0].InstanceId' \
    --output text \
    --region $REGION)
echo "Created EC2 instance with ID: $INSTANCE_ID."

# Wait for the instance to be in running state
aws ec2 wait instance-running --instance-ids $INSTANCE_ID --region $REGION
echo "EC2 instance is now running."

# Get the public IP of the instance
EC2_PUBLIC_IP=$(aws ec2 describe-instances --instance-ids $INSTANCE_ID --query 'Reservations[0].Instances[0].PublicIpAddress' --output text --region $REGION)
echo "Fetched EC2 public IP: $EC2_PUBLIC_IP."

# Function to check if a security group rule exists
rule_exists() {
    local group_id=$1
    local protocol=$2
    local port=$3
    local cidr=$4
    aws ec2 describe-security-group-rules --filters "Name=group-id,Values=$group_id" "Name=ip-protocol,Values=$protocol" "Name=from-port,Values=$port" "Name=to-port,Values=$port" "Name=cidr,Values=$cidr" --region $REGION --query 'SecurityGroupRules[0].SecurityGroupRuleId' --output text
}

# Add the EC2 instance public IP to the RDS security group inbound rules if it doesn't exist
if [ -z "$(rule_exists $SECURITY_GROUP_ID tcp 3306 $EC2_PUBLIC_IP/32)" ]; then
    aws ec2 authorize-security-group-ingress \
        --group-id $SECURITY_GROUP_ID \
        --protocol tcp \
        --port 3306 \
        --cidr $EC2_PUBLIC_IP/32 \
        --region $REGION
    echo "Authorized inbound rule for EC2 IP on port 3306."
fi

# Add an inbound rule to allow SSH from your local machine if it doesn't exist
if [ -z "$(rule_exists $SECURITY_GROUP_ID tcp 22 $LOCAL_IP/32)" ]; then
    aws ec2 authorize-security-group-ingress \
        --group-id $SECURITY_GROUP_ID \
        --protocol tcp \
        --port 22 \
        --cidr $LOCAL_IP/32 \
        --region $REGION
    echo "Authorized inbound rule for local IP on port 22."
fi

# Copy the SQL file to the EC2 instance
scp -o StrictHostKeyChecking=no -i $KEY_PATH /d/devops_articles/aws-paas-beanstalk/vprofile-project/src/main/resources/db_backup.sql ubuntu@$EC2_PUBLIC_IP:/home/ubuntu/
echo "Copied SQL file to EC2 instance."

# SSH into the EC2 instance and run MySQL commands
ssh -o StrictHostKeyChecking=no -i $KEY_PATH ubuntu@$EC2_PUBLIC_IP << EOF
    sudo apt update -y
    sudo apt install -y mysql-client-8.0 jq awscli

    # Fetch the RDS password and endpoint from AWS Secrets Manager
    SECRET=\$(aws secretsmanager get-secret-value --secret-id RDSDB_Credentials1 --region $REGION --query "SecretString" --output text)
    RDS_PASSWORD=\$(echo \$SECRET | jq -r '.password')
    RDS_ENDPOINT=$RDS_ENDPOINT

    # Create a temporary MySQL option file
    MYSQL_OPTIONS_FILE=\$(mktemp)
    cat << EOL > \$MYSQL_OPTIONS_FILE
[client]
user=admin
password=\$RDS_PASSWORD
host=\$RDS_ENDPOINT
port=3306
database=accounts
EOL

    # Run the SQL script using the MySQL option file
    mysql --defaults-extra-file=\$MYSQL_OPTIONS_FILE < /home/ubuntu/db_backup.sql
    echo "Executed SQL file."

    # Verify table creation
    mysql --defaults-extra-file=\$MYSQL_OPTIONS_FILE -e "SHOW TABLES;"
    echo "Verified table creation."

    # Remove the temporary MySQL option file
    rm -f \$MYSQL_OPTIONS_FILE
EOF
echo "Completed operations on EC2 instance."


# Terminate the EC2 instance
aws ec2 terminate-instances --instance-ids $INSTANCE_ID --region $REGION
echo "Terminated EC2 instance."

# Wait for the instance to be terminated
aws ec2 wait instance-terminated --instance-ids $INSTANCE_ID --region $REGION
echo "EC2 instance is terminated."

# Remove the EC2 instance public IP from the RDS security group inbound rules
aws ec2 revoke-security-group-ingress \
    --group-id $SECURITY_GROUP_ID \
    --protocol tcp \
    --port 3306 \
    --cidr $EC2_PUBLIC_IP/32 \
    --region $REGION
echo "Revoked inbound rule for EC2 IP on port 3306."

# Remove the inbound rule for SSH access from your local machine
aws ec2 revoke-security-group-ingress \
    --group-id $SECURITY_GROUP_ID \
    --protocol tcp \
    --port 22 \
    --cidr $LOCAL_IP/32 \
    --region $REGION
echo "Revoked inbound rule for local IP on port 22."

# Clean up IAM role and instance profile
aws iam remove-role-from-instance-profile --instance-profile-name $ROLE_NAME --role-name $ROLE_NAME --region $REGION
aws iam delete-instance-profile --instance-profile-name $ROLE_NAME --region $REGION
aws iam detach-role-policy --role-name $ROLE_NAME --policy-arn arn:aws:iam::aws:policy/AmazonRDSFullAccess --region $REGION
aws iam detach-role-policy --role-name $ROLE_NAME --policy-arn arn:aws:iam::aws:policy/SecretsManagerReadWrite --region $REGION
aws iam delete-role --role-name $ROLE_NAME --region $REGION
echo "Cleaned up IAM role and instance profile."

echo "RDS instance creation with accounts table is complete."

Amazon MQ Broker

- Script to create & setup the AWS Rabbit MQ Broker.
- Creates & stores the credentials into Secrets Manager

Show/Hide Script


#!/bin/bash
# Author: Anup Deshpande
# Date: 25-June-2024
# Script to create & setup the AWS Rabbit MQ Broker
# Creates & stores the credentials into Secrets Manager

set -e

# Set the region
REGION="ap-south-1"

echo "Setting region to $REGION."

# Fetch the default VPC ID
VPC_ID=$(aws ec2 describe-vpcs --query 'Vpcs[0].VpcId' --output text --region $REGION)
echo "Fetched VPC ID: $VPC_ID."

# Fetch the first subnet ID in the default VPC
SUBNET_ID=$(aws ec2 describe-subnets --filters "Name=vpc-id,Values=$VPC_ID" --query 'Subnets[0].SubnetId' --output text --region $REGION)
echo "Fetched Subnet ID: $SUBNET_ID"


# Fetch the security group ID for the given security group name
SECURITY_GROUP_ID=$(aws ec2 describe-security-groups --filters "Name=group-name,Values=dmanup-demo-aws-paas-backend-secgrp" --query "SecurityGroups[0].GroupId" --output text --region $REGION)
echo "Fetched Security Group ID: $SECURITY_GROUP_ID."

# Check if the secret already exists
SECRET_NAME="RabbitMQ_Credentials"
if aws secretsmanager describe-secret --secret-id $SECRET_NAME --region $REGION > /dev/null 2>&1; then
    # Fetch the RabbitMQ password from AWS Secrets Manager
    MQ_PASSWORD=$(aws secretsmanager get-secret-value --secret-id $SECRET_NAME --region $REGION --query "SecretString" --output text | sed -e 's/^.*"password":"\([^"]*\)".*$/\1/')
    echo "Secret $SECRET_NAME already exists."
else
    # Generate a complex password
    MQ_PASSWORD=$(openssl rand -base64 20 | tr -dc 'a-zA-Z0-9!#$%&()*+,-./:;<=>?@[]^_`{|}~' | head -c 20)
    echo "Generated RabbitMQ password."

    # Store RabbitMQ credentials in Secrets Manager
    aws secretsmanager create-secret --name $SECRET_NAME --description "RabbitMQ credentials" \
        --secret-string "{\"username\":\"rabbitmq\",\"password\":\"$MQ_PASSWORD\"}" --region $REGION
    echo "Stored RabbitMQ credentials in AWS Secrets Manager."
fi

# Create the Amazon MQ broker
aws mq create-broker \
    --broker-name dmanup-aws-paas-demo-mq-broker \
    --engine-type RabbitMQ \
    --engine-version 3.8.22 \
    --host-instance-type mq.t3.micro \
    --deployment-mode SINGLE_INSTANCE \
    --no-publicly-accessible \
    --subnet-ids $SUBNET_ID \
    --security-groups $SECURITY_GROUP_ID \
    --users "[{\"Username\":\"rabbitmq\",\"Password\":\"$MQ_PASSWORD\"}]" \
    --region $REGION \
    --auto-minor-version-upgrade
echo "Created Amazon MQ broker."

BROKER_ID=$(echo $BROKER_RESPONSE | jq -r '.BrokerId')
echo "Broker ID: $BROKER_ID"

# Wait until the MQ broker is available
echo "Waiting for the MQ broker to become available..."
while true; do
    BROKER_STATUS=$(aws mq describe-broker --broker-id $BROKER_ID --region $REGION --query 'BrokerState' --output text)
    echo "Current Broker Status: $BROKER_STATUS"
    if [ "$BROKER_STATUS" == "RUNNING" ]; then
        break
    fi
    sleep 30
done
echo "MQ broker is now available."

# Check the broker status
BROKER_STATUS=$(aws mq describe-broker --broker-id $BROKER_ID --region $REGION --query 'BrokerState' --output text)
echo "Final Broker Status: $BROKER_STATUS"

Amazon Elasticache Creation

- Script to create & setup the AWS Amazon Elasticache
- Creates a separate subnet & parameter group
- Create the ElastiCache cluster with the specified configuration

Show/Hide Script


#!/bin/bash
# Author: Anup Deshpande
# Date: 25-June-2024
# Script to create & setup the AWS Amazon Elasticache
# Creates a separate subnet & parameter group
# Create the ElastiCache cluster with the specified configuration

set -e

# Set the region
REGION="ap-south-1"

echo "Setting region to $REGION."

# Fetch the default VPC ID
VPC_ID=$(aws ec2 describe-vpcs --query 'Vpcs[0].VpcId' --output text --region $REGION)
echo "Fetched VPC ID: $VPC_ID."

# Fetch the subnet IDs for all availability zones in the default VPC
SUBNET_IDS=$(aws ec2 describe-subnets --filters "Name=vpc-id,Values=$VPC_ID" --query 'Subnets[*].SubnetId' --output text --region $REGION)
echo "Fetched Subnet IDs: $SUBNET_IDS."

# Create the subnet group with the fetched subnet IDs
aws elasticache create-cache-subnet-group \
    --cache-subnet-group-name dmanup-aws-paas-demo-memcached-subgrp \
    --cache-subnet-group-description "Subnet group for AWS PaaS demo Memcached" \
    --subnet-ids $SUBNET_IDS \
    --region $REGION
echo "Created Cache Subnet Group: dmanup-aws-paas-demo-memcached-subgrp."

# Create the parameter group for the latest version of Memcached
aws elasticache create-cache-parameter-group \
    --cache-parameter-group-name dmanup-aws-paas-demo-memcached-paragrp \
    --cache-parameter-group-family memcached1.6 \
    --description "Parameter group for AWS PaaS demo Memcached" \
    --region $REGION
echo "Created Cache Parameter Group: dmanup-aws-paas-demo-memcached-paragrp."

# Fetch the security group ID for the given security group name
SECURITY_GROUP_ID=$(aws ec2 describe-security-groups --filters "Name=group-name,Values=dmanup-demo-aws-paas-backend-secgrp" --query "SecurityGroups[0].GroupId" --output text --region $REGION)
echo "Fetched Security Group ID: $SECURITY_GROUP_ID."

# Create the ElastiCache cluster with the specified configuration
aws elasticache create-cache-cluster \
    --cache-cluster-id dmanup-aws-pass-demo-elasticache-svc \
    --engine memcached \
    --engine-version 1.6.12 \
    --cache-node-type cache.t3.micro \
    --num-cache-nodes 1 \
    --cache-parameter-group-name dmanup-aws-paas-demo-memcached-paragrp \
    --cache-subnet-group-name dmanup-aws-paas-demo-memcached-subgrp \
    --security-group-ids $SECURITY_GROUP_ID \
    --region $REGION
echo "Created ElastiCache Cluster: dmanup-aws-pass-demo-elasticache-svc."

# Wait until the ElastiCache cluster is available
echo "Waiting for the ElastiCache cluster to become available..."
while true; do
    CLUSTER_STATUS=$(aws elasticache describe-cache-clusters --cache-cluster-id dmanup-aws-pass-demo-elasticache-svc --region $REGION --query 'CacheClusters[0].CacheClusterStatus' --output text)
    echo "Current Cluster Status: $CLUSTER_STATUS"
    if [ "$CLUSTER_STATUS" == "available" ]; then
        break
    fi
    sleep 30
done
echo "ElastiCache cluster is now available."

# Output the created cache cluster ID
echo "Created Cache Cluster ID: dmanup-aws-pass-demo-elasticache-svc"

Updating Application Properties with the infrastructure component values

- For this application to run, it needs to have the application.properties file to have the values of the components it uses, such as RDS DB,MQ Broker & Elasticache during the processing of the application.
- As all the infrastructure components are created, we will fetch their component names, credentials from the AWS.
- Script will fetch all the required values from the AWS such as RDS, Elasticache & Rabbit MQ endpoints and the required credentails from the Secrets Manager
- After backing up the existing file, all the details are retrieved and the application.properties file is updated with the components values, credentials.
- Once this script is executed successfully, the build can be created using any sutiable tool. I used maven to build artificat. Once the war file was generated, we can make use of it in our beanstalk creation part.
- In the next step we will have the required war file(vprofile-v2.war) with the updated application properties

Show/Hide Script


#!/bin/bash
# Author: Anup Deshpande
# Date: 25-June-2024
# Script will fetch all the required values from the AWS such as RDS, Elasticache & #Rabbit MQ
# endpoints and the required credentails from the Secrets Manager
# Once all the details are retrieved, the application.properties file is updated after taking a 
# backup of the existing file.

set -e

# Set region
REGION="ap-south-1"

# Fetch the RDS endpoint and credentials
RDS_ENDPOINT=$(aws rds describe-db-instances \
    --db-instance-identifier dmaup-aws-paas-demo-rdsdb \
    --query 'DBInstances[0].Endpoint.Address' \
    --region $REGION \
    --output text)
RDS_PORT=$(aws rds describe-db-instances \
    --db-instance-identifier dmaup-aws-paas-demo-rdsdb \
    --query 'DBInstances[0].Endpoint.Port' \
    --region $REGION \
    --output text)
RDS_SECRET_NAME="RDSDB_Credentials1"
RDS_USERNAME=$(aws secretsmanager get-secret-value --secret-id $RDS_SECRET_NAME --region $REGION --query "SecretString" --output text | jq -r '.username')
RDS_PASSWORD=$(aws secretsmanager get-secret-value --secret-id $RDS_SECRET_NAME --region $REGION --query "SecretString" --output text | jq -r '.password')

# Fetch the ElastiCache endpoint
ELASTICACHE_ENDPOINT=$(aws elasticache describe-cache-clusters \
    --cache-cluster-id dmanup-aws-pass-demo-elasticache-svc \
    --query 'CacheClusters[0].ConfigurationEndpoint.Address' \
    --region $REGION \
    --output text)
ELASTICACHE_PORT=$(aws elasticache describe-cache-clusters \
    --cache-cluster-id dmanup-aws-pass-demo-elasticache-svc \
    --query 'CacheClusters[0].ConfigurationEndpoint.Port' \
    --region $REGION \
    --output text)

# Fetch the RabbitMQ endpoint and credentials
RABBITMQ_SECRET_NAME="RabbitMQ_Credentials"
RABBITMQ_ENDPOINT=$(aws mq describe-broker \
    --broker-id $(aws mq list-brokers --query "BrokerSummaries[?BrokerName=='dmanup-aws-paas-demo-mq-broker'].BrokerId" --region $REGION --output text) \
    --query 'BrokerInstances[0].Endpoints[0]' \
    --region $REGION \
    --output text)
RABBITMQ_PORT=5672
RABBITMQ_USERNAME=$(aws secretsmanager get-secret-value --secret-id $RABBITMQ_SECRET_NAME --region $REGION --query "SecretString" --output text | jq -r '.username')
RABBITMQ_PASSWORD=$(aws secretsmanager get-secret-value --secret-id $RABBITMQ_SECRET_NAME --region $REGION --query "SecretString" --output text | jq -r '.password')

# Path to the application.properties file
PROPERTIES_FILE="/d/devops_articles/aws-paas-beanstalk/vprofile-project/src/main/resources/application.properties"

# Backup the original properties file
cp $PROPERTIES_FILE ${PROPERTIES_FILE}.bak

# Update the application.properties file
cat < $PROPERTIES_FILE
#JDBC Configuration for Database Connection
jdbc.driverClassName=com.mysql.jdbc.Driver
jdbc.url=jdbc:mysql://$RDS_ENDPOINT:$RDS_PORT/accounts?useUnicode=true&characterEncoding=UTF-8&zeroDateTimeBehavior=convertToNull
jdbc.username=$RDS_USERNAME
jdbc.password=$RDS_PASSWORD

#Memcached Configuration For Active and StandBy Host
#For Active Host
memcached.active.host=$ELASTICACHE_ENDPOINT
memcached.active.port=$ELASTICACHE_PORT
#For StandBy Host
memcached.standBy.host=127.0.0.2
memcached.standBy.port=11211

#RabbitMq Configuration
rabbitmq.address=$RABBITMQ_ENDPOINT
rabbitmq.port=$RABBITMQ_PORT
rabbitmq.username=$RABBITMQ_USERNAME
rabbitmq.password=$RABBITMQ_PASSWORD

#Elasticsearch Configuration
elasticsearch.host=192.168.1.85
elasticsearch.port=9300
elasticsearch.cluster=vprofile
elasticsearch.node=vprofilenode
EOL

# Print success message
echo "application.properties file updated successfully."

Elastic Beanstalk Application

- Creates the following required for the beanstalk application
- Key pair file, S3 Bucket to store the artifact, IAM Role for Elastic Beanstalk Service
- Instance Profile for Elastic Beanstalk EC2 Instances
- Attaching the AWS managed policies to the instance role
- Create an Elastic Beanstalk application
- Upload the application code to S3
- Create an application version with all the required configuration by creating options.json
- Waits till the beanstalk application is fully up & available.

Show/Hide Script


#!/bin/bash
# Creates the following required for the beanstalk application
# Key pair file, S3 Bucket to store the artifact, IAM Role for Elastic Beanstalk Service
# Instance Profile for Elastic Beanstalk EC2 Instances
# Attaching the AWS managed policies to the instance role
# Create an Elastic Beanstalk application
# Upload the application code to S3
# Create an application version with all the required configuration by creating options.json
# Waits till the beanstalk application is fully up & available.

set -e

# Set the region
REGION="ap-south-1"

export AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query "Account" --output text)
KEY_PAIR_NAME="dmanup-aws-pass-demo-keypair"
KEY_PAIR_FILE="$KEY_PAIR_NAME.pem"

# Check if the key pair already exists
if aws ec2 describe-key-pairs --key-names $KEY_PAIR_NAME --region $REGION 2>&1 | grep -q 'InvalidKeyPair.NotFound'; then
    # Create the key pair
    aws ec2 create-key-pair --key-name $KEY_PAIR_NAME --region $REGION --query 'KeyMaterial' --output text > $KEY_PAIR_FILE
    chmod 400 $KEY_PAIR_FILE
    echo "Created key pair: $KEY_PAIR_NAME"
else
    echo "Key pair already exists: $KEY_PAIR_NAME"
fi

# Set the local and destination paths
LOCAL_PATH="/d/devops_articles/aws-paas-beanstalk/vprofile-project/target/vprofile-v2.war"
S3_BUCKET="dmanup-aws-paas-demo-bucket"
S3_KEY="vprofile-v2.war"
DESTINATION_PATH="s3://$S3_BUCKET/$S3_KEY"

echo "Setting region to $REGION."

# Fetch the default VPC ID
DEFAULT_VPC_ID=$(aws ec2 describe-vpcs \
    --filters Name=isDefault,Values=true \
    --query "Vpcs[0].VpcId" \
    --region $REGION \
    --output text)
echo "Fetched Default VPC ID: $DEFAULT_VPC_ID."

# Fetch the subnet IDs for the default VPC
SUBNET_IDS=$(aws ec2 describe-subnets \
    --filters Name=vpc-id,Values=$DEFAULT_VPC_ID \
    --query "Subnets[*].SubnetId" \
    --region $REGION \
    --output text | tr '\t' ',')
echo "Fetched Subnet IDs: $SUBNET_IDS."

# Check if the S3 bucket already exists
if aws s3 ls "s3://$S3_BUCKET" 2>&1 | grep -q 'NoSuchBucket'; then
    # Create an S3 bucket (if not already created)
    aws s3 mb s3://$S3_BUCKET --region $REGION
    echo "Created S3 bucket: $S3_BUCKET."
else
    echo "S3 bucket already exists: $S3_BUCKET."
fi

# Check if the IAM Role already exists
if aws iam get-role --role-name aws-elasticbeanstalk-service-role 2>&1 | grep -q 'NoSuchEntity'; then
    # Create an IAM Role for Elastic Beanstalk Service
    aws iam create-role --role-name aws-elasticbeanstalk-service-role \
      --assume-role-policy-document '{
        "Version": "2012-10-17",
        "Statement": [
          {
            "Effect": "Allow",
            "Principal": {
              "Service": "elasticbeanstalk.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
          }
        ]
      }'
    echo "Created IAM Role: aws-elasticbeanstalk-service-role."
else
    echo "IAM Role already exists: aws-elasticbeanstalk-service-role."
fi

# Attach the AWS managed policies to the service role
aws iam attach-role-policy --role-name aws-elasticbeanstalk-service-role \
  --policy-arn arn:aws:iam::aws:policy/service-role/AWSElasticBeanstalkEnhancedHealth
aws iam attach-role-policy --role-name aws-elasticbeanstalk-service-role \
  --policy-arn arn:aws:iam::aws:policy/AWSElasticBeanstalkManagedUpdatesCustomerRolePolicy
echo "Attached policies to IAM Role: aws-elasticbeanstalk-service-role."

# Check if the Instance Profile already exists
if aws iam get-instance-profile --instance-profile-name aws-elasticbeanstalk-ec2-role 2>&1 | grep -q 'NoSuchEntity'; then
    # Create an Instance Profile for Elastic Beanstalk EC2 Instances
    aws iam create-instance-profile --instance-profile-name aws-elasticbeanstalk-ec2-role
    echo "Created Instance Profile: aws-elasticbeanstalk-ec2-role."
else
    echo "Instance Profile already exists: aws-elasticbeanstalk-ec2-role."
fi

# Check if the IAM Role already exists
if aws iam get-role --role-name aws-elasticbeanstalk-ec2-role 2>&1 | grep -q 'NoSuchEntity'; then
    # Create a role for the instance profile
    aws iam create-role --role-name aws-elasticbeanstalk-ec2-role \
      --assume-role-policy-document '{
        "Version": "2012-10-17",
        "Statement": [
          {
            "Effect": "Allow",
            "Principal": {
              "Service": "ec2.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
          }
        ]
      }'
    echo "Created IAM Role: aws-elasticbeanstalk-ec2-role."
else
    echo "IAM Role already exists: aws-elasticbeanstalk-ec2-role."
fi

# Attach the AWS managed policies to the instance role
aws iam attach-role-policy --role-name aws-elasticbeanstalk-ec2-role \
  --policy-arn arn:aws:iam::aws:policy/AWSElasticBeanstalkWebTier
aws iam attach-role-policy --role-name aws-elasticbeanstalk-ec2-role \
  --policy-arn arn:aws:iam::aws:policy/AWSElasticBeanstalkWorkerTier
echo "Attached policies to IAM Role: aws-elasticbeanstalk-ec2-role."

# Check if the role is already attached to the instance profile
if aws iam list-instance-profiles-for-role --role-name aws-elasticbeanstalk-ec2-role --query 'InstanceProfiles[?InstanceProfileName==`aws-elasticbeanstalk-ec2-role`]' --output text | grep -q 'aws-elasticbeanstalk-ec2-role'; then
    echo "Role already attached to Instance Profile: aws-elasticbeanstalk-ec2-role."
else
    # Attach the role to the instance profile
    aws iam add-role-to-instance-profile --instance-profile-name aws-elasticbeanstalk-ec2-role --role-name aws-elasticbeanstalk-ec2-role
    echo "Attached Role to Instance Profile: aws-elasticbeanstalk-ec2-role."
fi

# Create an Elastic Beanstalk application
aws elasticbeanstalk create-application \
    --application-name dmanup-aws-paas-demo-app \
    --description "Demo application for AWS PaaS" \
    --region $REGION
echo "Created Elastic Beanstalk Application: dmanup-aws-paas-demo-app."

# Upload the application code to S3
aws s3 cp $LOCAL_PATH $DESTINATION_PATH --region $REGION
echo "Uploaded application code to S3: $DESTINATION_PATH."

# Create an application version
aws elasticbeanstalk create-application-version \
    --application-name dmanup-aws-paas-demo-app \
    --version-label v1 \
    --source-bundle S3Bucket=$S3_BUCKET,S3Key=$S3_KEY \
    --region $REGION
echo "Created Application Version: v1."

# Create the configuration options file
cat < options.json
[
  {
    "Namespace": "aws:autoscaling:launchconfiguration",
    "OptionName": "InstanceType",
    "Value": "t3.micro"
  },
  {
    "Namespace": "aws:autoscaling:launchconfiguration",
    "OptionName": "EC2KeyName",
    "Value": "dmanup-aws-pass-demo-keypair"
  },
  {
    "Namespace": "aws:elasticbeanstalk:environment",
    "OptionName": "EnvironmentType",
    "Value": "LoadBalanced"
  },
  {
    "Namespace": "aws:autoscaling:asg",
    "OptionName": "MinSize",
    "Value": "2"
  },
  {
    "Namespace": "aws:autoscaling:asg",
    "OptionName": "MaxSize",
    "Value": "3"
  },
  {
    "Namespace": "aws:autoscaling:trigger",
    "OptionName": "MeasureName",
    "Value": "NetworkOut"
  },
  {
    "Namespace": "aws:autoscaling:trigger",
    "OptionName": "Statistic",
    "Value": "Average"
  },
  {
    "Namespace": "aws:autoscaling:trigger",
    "OptionName": "Unit",
    "Value": "Bytes"
  },
  {
    "Namespace": "aws:autoscaling:trigger",
    "OptionName": "Period",
    "Value": "300"
  },
  {
    "Namespace": "aws:autoscaling:trigger",
    "OptionName": "BreachDuration",
    "Value": "300"
  },
  {
    "Namespace": "aws:ec2:vpc",
    "OptionName": "VPCId",
    "Value": "$DEFAULT_VPC_ID"
  },
  {
    "Namespace": "aws:ec2:vpc",
    "OptionName": "Subnets",
    "Value": "$SUBNET_IDS"
  },
  {
    "Namespace": "aws:elasticbeanstalk:environment:process:default",
    "OptionName": "StickinessEnabled",
    "Value": "true"
  },
  {
    "Namespace": "aws:elasticbeanstalk:environment:process:default",
    "OptionName": "StickinessLBCookieDuration",
    "Value": "86400"
  },
  {
    "Namespace": "aws:elasticbeanstalk:environment:process:default",
    "OptionName": "HealthCheckPath",
    "Value": "/login"
  },
  {
    "Namespace": "aws:elasticbeanstalk:healthreporting:system",
    "OptionName": "SystemType",
    "Value": "basic"
  },
  {
    "Namespace": "aws:elasticbeanstalk:application",
    "OptionName": "Application Healthcheck URL",
    "Value": "/login"
  },
  {
    "Namespace": "aws:elasticbeanstalk:managedactions",
    "OptionName": "ManagedActionsEnabled",
    "Value": "false"
  },
  {
    "Namespace": "aws:elasticbeanstalk:command",
    "OptionName": "DeploymentPolicy",
    "Value": "Rolling"
  },
  {
    "Namespace": "aws:elasticbeanstalk:command",
    "OptionName": "BatchSizeType",
    "Value": "Fixed"
  },
  {
    "Namespace": "aws:elasticbeanstalk:command",
    "OptionName": "BatchSize",
    "Value": "1"
  },
  {
    "Namespace": "aws:elasticbeanstalk:sns:topics",
    "OptionName": "Notification Endpoint",
    "Value": "anupde@gmail.com"
  },
  {
    "Namespace": "aws:elasticbeanstalk:environment",
    "OptionName": "ServiceRole",
    "Value": "aws-elasticbeanstalk-service-role"
  },
  {
    "Namespace": "aws:autoscaling:launchconfiguration",
    "OptionName": "IamInstanceProfile",
    "Value": "aws-elasticbeanstalk-ec2-role"
  }
]
EOL
echo "Created Configuration Options file: options.json."

# Create an Elastic Beanstalk environment
export AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query "Account" --output text)
SOLUTION_STACK_NAME=$(aws elasticbeanstalk list-available-solution-stacks --region ap-south-1 \
    | jq -r '.SolutionStacks[] | select(contains("64bit Amazon Linux 2") and contains("Corretto 11") and contains("Tomcat 8.5"))')

aws elasticbeanstalk create-environment \
    --application-name dmanup-aws-paas-demo-app \
    --environment-name dmanup-aws-paas-demo-env \
    --solution-stack-name "$SOLUTION_STACK_NAME" \
    --option-settings file://options.json \
    --version-label v1 \
    --region $REGION \
    --tier "Name=WebServer,Type=Standard,Version=1.0"
echo "Created Elastic Beanstalk Environment: dmanup-aws-paas-demo-env."

# Wait until the environment is available
echo "Waiting for the Elastic Beanstalk environment to become available..."
while true; do
    ENV_STATUS=$(aws elasticbeanstalk describe-environments \
        --application-name dmanup-aws-paas-demo-app \
        --environment-names dmanup-aws-paas-demo-env \
        --region $REGION \
        --query 'Environments[0].Status' \
        --output text)
    echo "Current Environment Status: $ENV_STATUS"
    if [ "$ENV_STATUS" == "Ready" ]; then
        break
    fi
    sleep 30
done
echo "Elastic Beanstalk environment is now available."

# Check if the application was created successfully
APP_STATUS=$(aws elasticbeanstalk describe-applications \
    --application-names dmanup-aws-paas-demo-app \
    --region $REGION \
    --query 'Applications[0].ApplicationName' \
    --output text)

echo "Application Status: $APP_STATUS"

# Check if the environment was created successfully
ENV_STATUS=$(aws elasticbeanstalk describe-environments \
    --application-name dmanup-aws-paas-demo-app \
    --environment-names dmanup-aws-paas-demo-env \
    --region $REGION \
    --query 'Environments[0].Status' \
    --output text)

echo "Environment Status: $ENV_STATUS"

Allowing Beanstalk Appl to connect to Backend Components

- Once the Elastic Beanstalk application is created,separate instance level security group is created and this needs to have access(inbound) to the Backend Security Group
- that is catering for the DB,MQ & Elasticache components.

Show/Hide Script


#!/bin/bash
# Once the Elastic Beanstalk application is created, 
# separate instance level security group is created and this needs to have access(inbound) to the Backend Security Group
# that is catering for the DB,MQ & Elasticache components. 


# Set the region
REGION="ap-south-1"
ENV_NAME="dmanup-aws-paas-demo-env"

# Fetch the Auto Scaling group name associated with the Elastic Beanstalk environment
AUTO_SCALING_GROUP_NAME=$(aws elasticbeanstalk describe-environment-resources \
    --environment-name $ENV_NAME \
    --region $REGION \
    --query 'EnvironmentResources.AutoScalingGroups[0].Name' \
    --output text)
echo "AUTO_SCALING_GROUP_NAME: $AUTO_SCALING_GROUP_NAME"

# Fetch the instance ID from the Auto Scaling group
INSTANCE_ID=$(aws autoscaling describe-auto-scaling-groups \
    --auto-scaling-group-names $AUTO_SCALING_GROUP_NAME \
    --region $REGION \
    --query 'AutoScalingGroups[0].Instances[0].InstanceId' \
    --output text)
echo "INSTANCE_ID: $INSTANCE_ID"

# Fetch the security group ID associated with the instance
INSTANCE_SECURITY_GROUP_ID=$(aws ec2 describe-instances \
    --instance-ids $INSTANCE_ID \
    --region $REGION \
    --query 'Reservations[0].Instances[0].SecurityGroups[0].GroupId' \
    --output text)
echo "INSTANCE_SECURITY_GROUP_ID: $INSTANCE_SECURITY_GROUP_ID"

# Backend security group ID
BACKEND_SECURITY_GROUP_ID=$(aws ec2 describe-security-groups \
    --filters Name=group-name,Values=dmanup-demo-aws-paas-backend-secgrp \
    --query 'SecurityGroups[0].GroupId' \
    --region $REGION \
    --output text)
echo "BACKEND_SECURITY_GROUP_ID: $BACKEND_SECURITY_GROUP_ID"

# Add inbound rule to allow MySQL (RDS) traffic on port 3306
aws ec2 authorize-security-group-ingress \
    --group-id $BACKEND_SECURITY_GROUP_ID \
    --protocol tcp \
    --port 3306 \
    --source-group $INSTANCE_SECURITY_GROUP_ID \
    --region $REGION
echo "Added inbound rule for MySQL (RDS) traffic on port 3306."

# Add inbound rule to allow Memcached (ElastiCache) traffic on port 11211
aws ec2 authorize-security-group-ingress \
    --group-id $BACKEND_SECURITY_GROUP_ID \
    --protocol tcp \
    --port 11211 \
    --source-group $INSTANCE_SECURITY_GROUP_ID \
    --region $REGION
echo "Added inbound rule for Memcached (ElastiCache) traffic on port 11211."

# Add inbound rule to allow RabbitMQ (Amazon MQ) traffic on port 5672
aws ec2 authorize-security-group-ingress \
    --group-id $BACKEND_SECURITY_GROUP_ID \
    --protocol tcp \
    --port 5672 \
    --source-group $INSTANCE_SECURITY_GROUP_ID \
    --region $REGION
echo "Added inbound rule for RabbitMQ (Amazon MQ) traffic on port 5672."

# Verification
# Check the updated inbound rules for the backend security group
aws ec2 describe-security-groups \
    --group-ids $BACKEND_SECURITY_GROUP_ID \
    --region $REGION \
    --query 'SecurityGroups[0].IpPermissions' \
    --output json

Result

The entire infrastructure was successfully deployed using shell scripts, with no manual intervention via the AWS Console. This ensured:
- Security: No hardcoded credentials.
- Automation: Seamless updates to application properties.
- Efficiency: Minimal maintenance with managed services.

Conclusion

This POC showcases the efficiency of using AWS PAAS and Beanstalk for web application deployment. Stay tuned for more articles in this series, where I’ll dive deeper into other DevOps and SRE solutions. By migrating to AWS PAAS and Beanstalk, we estimated a cost savings calculation as per below. Please note this was based on assumptions of the size of the instances & the price & the associated esimated PAAS service costs. This was not a direct cost savings, this was presented as a potential cost savings.


Cost Savings:

- On-Premise/Cloud VMs (3 years): $71,700.
- AWS Managed Services (3 years): $4,078.08.
- Total Savings: $67,621.92 over 3 years.
Operational Efficiency:
- Reduced Maintenance: Minimal maintenance overhead with managed services.
- Scalability: Automatic scaling and load balancing with Elastic Beanstalk.
- Security: Enhanced security using AWS Secrets Manager and managed services.
- Focus on Development: Freed up resources to focus more on application enhancements.

Leave a Reply

Your email address will not be published. Required fields are marked *

Related Post