Popular Posts

Monday, June 6, 2016

Building a sub $300/month Oracle RAC on AWS - Part I

NOTE: The idea and inspiration of building a cheap Oracle RAC come from Amazon’s documentation (https://aws.amazon.com/articles/7455908317389540)
This setup is best for testing or non-prod as we would not build network and storage resiliency.
This would be a 3 part series where we discuss different components of AWS that are needed to be setup to enable us to install Oracle RAC on AWS. This would involve setting up networking, shared storage and RAC hosts/nodes. Which would eventually lead to installation of the Oracle grid and database software and RAC database creation.
We would use command-line and UI during this setup.
The AWS UI can be quite intimidating, the main landing page looks like:
Let’s setup the command-line interface for AWS:
  • Installing AWS command-line utility
    curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip"
    unzip awscli-bundle.zip
    sudo ./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws
  • Test the installation
    aws --version
  • Setup credentials
    Create file $HOME/.aws/credentials and add your aws_access_key_id & aws_secret_access_key
    [default]
    aws_access_key_id = <your access key>
    aws_secret_access_key=<your secret access key>

    You should receive these when your user was created or you can ask you AWS admin to generate and provide these for your ID. More details can be found at: http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSGettingStartedGuide/AWSCredentials.html
  • Configure default AWS region
$ aws configure
 AWS Access Key ID [****************XXXX]:
 AWS Secret Access Key [****************xXxX]:
 Default region name [us-west-2]:
 Default output format [None]:

Part 1:  Network Infrastructure Setup

Let’s begin with building the network infrastructure needed in AWS to setup Oracle RAC.

Setup VPC

VPC or Virtual Private Cloud create a logical isolated network for you on which you have total control. Think of this as your personal data center. You can add hosts, storage, access permissions over the resources (EC2, S3 etc.) provisioned in the VPC. Detailed documentation can be found here:  https://aws.amazon.com/vpc/

Command-line:
$ aws ec2 create-vpc --cidr-block 10.0.0.0/16
{
   "Vpc": {
       "InstanceTenancy": "default",
       "State": "pending",
       "VpcId": "vpc-ebad038f",
       "CidrBlock": "10.0.0.0/16",
       "DhcpOptionsId": "dopt-9ea35ffb"
   }
}


Let’s add a name to this VPC so that it’s easy to identify.
$ aws ec2 create-tags --tags Key=Name,Value=OracleRAC-VPC --resources vpc-ebad038f
$ aws ec2 describe-vpcs --vpc-ids vpc-ebad038f
{
   "Vpcs": [
       {
           "VpcId": "vpc-ebad038f",
           "InstanceTenancy": "default",
           "Tags": [
               {
                   "Value": "OracleRAC-VPC",
                   "Key": "Name"
               }
           ],
           "State": "available",
           "DhcpOptionsId": "dopt-9ea35ffb",
           "CidrBlock": "10.0.0.0/16",
           "IsDefault": false
       }
   ]
}

Enable DNS and DNS Hostname support
$ aws ec2 modify-vpc-attribute --vpc-id vpc-ebad038f --enable-dns-support
$ aws ec2 modify-vpc-attribute --vpc-id vpc-ebad038f --enable-dns-hostnames

Setup Internet Gateway

To enable us to connect to your newly created VPC from outside/internet we need to setup an internet gateway and enable our VPC to make use of that.

Create internet gateway:
$ aws ec2 create-internet-gateway
{
   "InternetGateway": {
       "Tags": [],
       "InternetGatewayId": "igw-0e13286b",
       "Attachments": []
   }
}

Connect the VPC to the internet gateway:
$ aws ec2 attach-internet-gateway --internet-gateway-id igw-0e13286b --vpc-id vpc-ebad038f

Verify:
$ aws ec2 describe-internet-gateways --internet-gateway-ids igw-0e13286b
{
   "InternetGateways": [
       {
           "Tags": [],
           "InternetGatewayId": "igw-0e13286b",
           "Attachments": [
               {
                   "State": "available",
                   "VpcId": "vpc-ebad038f"
               }
           ]
       }
   ]
}

We need to enable routing traffic from the VPC to the internet gateway, for that we need to update the routing table of our VPC with the appropriate rule. Details of AWS routing table can be found at: http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Route_Tables.html

Get routing table details from our VPC:
$ aws ec2 describe-route-tables --filters "Name=vpc-id,Values=vpc-ebad038f"
{
   "RouteTables": [
       {
           "Associations": [
               {
                   "RouteTableAssociationId": "rtbassoc-fbd0e99f",
                   "Main": true,
                   "RouteTableId": "rtb-719b0d15"
               }
           ],
           "RouteTableId": "rtb-719b0d15",
           "VpcId": "vpc-ebad038f",
           "PropagatingVgws": [],
           "Tags": [],
           "Routes": [
               {
                   "GatewayId": "local",
                   "DestinationCidrBlock": "10.0.0.0/16",
                   "State": "active",
                   "Origin": "CreateRouteTable"
               }
           ]
       }
   ]
}

Now add the route/rule to the routing table:
$ aws ec2 create-route --route-table-id rtb-719b0d15 --destination-cidr-block 0.0.0.0/0 --gateway-id igw-0e13286b
{
   "Return": true
}

Setup Subnet

Create a new subnet for Oracle RAC with the VPC:
$ aws ec2 create-subnet --vpc-id vpc-ebad038f --cidr-block 10.0.0.0/24
{
   "Subnet": {
       "VpcId": "vpc-ebad038f",
       "CidrBlock": "10.0.0.0/24",
       "State": "pending",
       "AvailabilityZone": "us-west-2c",
       "SubnetId": "subnet-dac3dd83",
       "AvailableIpAddressCount": 251
   }
}

Tag the subnet with an identifiable name:
$ aws ec2 create-tags --resources subnet-dac3dd83 --tags Key=Name,Value=OracleRAC-Subnet

Verify:
$ aws ec2 describe-subnets --subnet-ids subnet-dac3dd83
{
   "Subnets": [
       {
           "VpcId": "vpc-ebad038f",
           "Tags": [
               {
                   "Value": "OracleRAC-Subnet",
                   "Key": "Name"
               }
           ],
           "CidrBlock": "10.0.0.0/24",
           "MapPublicIpOnLaunch": false,
           "DefaultForAz": false,
           "State": "available",
           "AvailabilityZone": "us-west-2c",
           "SubnetId": "subnet-dac3dd83",
           "AvailableIpAddressCount": 251
       }
   ]
}

Setup Placement Group

A placement group is a logical grouping of instances within a single Availability Zone. Using placement groups enables applications to participate in a low-latency, 10 Gigabits per second (Gbps) network.
We create a placement to group all the Oracle RAC instances logically together to take benefit of the faster network and lower latency.

$ aws ec2 create-placement-group --group-name OracleRAC-PlacementGroup --strategy cluster

$ aws ec2 describe-placement-groups
{
   "PlacementGroups": [
       {
           "GroupName": "OracleRAC-PlacementGroup",
           "State": "available",
           "Strategy": "cluster"
       }
]
}

Setup DNS

We need to setup DNS to be able to identify the various components of Oracle RAC by names.
Oracle RAC would include 2 hosts, storage, SCAN name, virtual names, private names. We need to setup DNS using AWS Route 53 service. We need to first create a private hosted zone using Route 53. Details can be found at: http://docs.aws.amazon.com/Route53/latest/DeveloperGuide/hosted-zones-private.html

Create Private Hosted Zone:
$ aws route53 create-hosted-zone --name oracleraczone.net --vpc VPCRegion=us-west-2,VPCId=vpc-ebad038f --caller-reference OracleRAC-2016-05-04
{
   "ChangeInfo": {
       "Status": "PENDING",
       "SubmittedAt": "2016-05-04T21:39:19.615Z",
       "Id": "/change/C1EMZL7SEBW9FX"
   },
   "HostedZone": {
       "ResourceRecordSetCount": 2,
       "CallerReference": "OracleRAC-2016-05-04",
       "Config": {
           "PrivateZone": true
       },
       "Id": "/hostedzone/Z26IJOYZR20BT8",
       "Name": "oracleraczone.net."
   },
   "Location": "https://route53.amazonaws.com/2013-04-01/hostedzone/Z26IJOYZR20BT8",
   "VPC": {
       "VPCId": "vpc-ebad038f",
       "VPCRegion": "us-west-2"
   }
}

Create DNS record for Oracle RAC:
To create the DNS names for hosts, storage, scan etc for Oracle RAC we need to create a JSON documents with these relations which will be used AWS Route 53  to create the required DNS entries:
Create a file as oraclerac.json with the following text:

{   
  "Changes": [
    {"Action": "UPSERT",
      "ResourceRecordSet": {
        "Name": "oraclerac-scan.oracleraczone.net","Type": "A","TTL": 300,
        "ResourceRecords": [{"Value": "10.0.0.31"},{"Value": "10.0.0.32"},{"Value": "10.0.0.33"}]
      }
    },
    {"Action": "UPSERT",
      "ResourceRecordSet": {
        "Name": "oraclerac-scan1.oracleraczone.net","Type": "A","TTL": 300,
        "ResourceRecords": [{"Value": "10.0.0.31"}]
      }
    },
    {"Action": "UPSERT",
      "ResourceRecordSet": {
        "Name": "oraclerac-scan2.oracleraczone.net","Type": "A","TTL": 300,
        "ResourceRecords": [{"Value": "10.0.0.32"}]
      }
    },
    {"Action": "UPSERT",
      "ResourceRecordSet": {
        "Name": "oraclerac-scan3.oracleraczone.net","Type": "A","TTL": 300,
        "ResourceRecords": [{"Value": "10.0.0.33"}]
      }
    },
    {"Action": "UPSERT",
      "ResourceRecordSet": {
        "Name": "racnode01-i.oracleraczone.net","Type": "A","TTL": 300,
        "ResourceRecords": [{"Value": "10.1.0.21"}]
      }
    },
    {"Action": "UPSERT",
      "ResourceRecordSet": {
        "Name": "racnode01-v.oracleraczone.net","Type": "A","TTL": 300,
        "ResourceRecords": [{"Value": "10.0.0.21"}]
      }
    },
    {"Action": "UPSERT",
      "ResourceRecordSet": {
        "Name": "racnode01.oracleraczone.net","Type": "A","TTL": 300,
        "ResourceRecords": [{"Value": "10.0.0.11"}]
      }
    },
    {"Action": "UPSERT",
      "ResourceRecordSet": {
        "Name": "racnode02-i.oracleraczone.net","Type": "A","TTL": 300,
        "ResourceRecords": [{"Value": "10.1.0.22"}]
      }
    },
    {"Action": "UPSERT",
      "ResourceRecordSet": {
        "Name": "racnode02-v.oracleraczone.net","Type": "A","TTL": 300,
        "ResourceRecords": [{"Value": "10.0.0.22"}]
      }
    },
    {"Action": "UPSERT",
      "ResourceRecordSet": {
        "Name": "racnode02.oracleraczone.net","Type": "A","TTL": 300,
        "ResourceRecords": [{"Value": "10.0.0.12"}]
      }
    },
    {"Action": "UPSERT",
      "ResourceRecordSet": {
        "Name": "storage01.oracleraczone.net","Type": "A","TTL": 300,
        "ResourceRecords": [ {"Value": "10.0.0.51"}]
      }
    }
  ]
}

Now lets upload these details using AWS Route 53 CLI:

$ aws route53 change-resource-record-sets --hosted-zone-id Z26IJOYZR20BT8 --change-batch file://oraclerac.json
{
   "ChangeInfo": {
       "Status": "PENDING",
       "SubmittedAt": "2016-05-04T21:51:04.752Z",
       "Id": "/change/C1QJM0W9873VAZ"
   }
}

Verify if the DNS records have been updated:
$ aws route53 list-resource-record-sets --hosted-zone-id Z26IJOYZR20BT8
{
   "ResourceRecordSets": [
       {
           "ResourceRecords": [
               {
                   "Value": "ns-1536.awsdns-00.co.uk."
               },
               {
                   "Value": "ns-0.awsdns-00.com."
               },
               {
                   "Value": "ns-1024.awsdns-00.org."
               },
               {
                   "Value": "ns-512.awsdns-00.net."
               }
           ],
           "Type": "NS",
           "Name": "oracleraczone.net.",
           "TTL": 172800
       },
       {
           "ResourceRecords": [
               {
                   "Value": "ns-1536.awsdns-00.co.uk. awsdns-hostmaster.amazon.com. 1 7200 900 1209600 86400"
               }
           ],
           "Type": "SOA",
           "Name": "oracleraczone.net.",
           "TTL": 900
       },
……

Output is truncated due to its length.

This finally completes the network setup for Oracle RAC.

Let’s just setup the security for our VPC, which would enable us to ssh to the Oracle RAC hosts be create later and call it a day :).

Setup Security Group

A security group acts as a virtual firewall that controls the traffic for one or more instances. When you launch an instance, you associate one or more security groups with the instance. More detail at: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-network-security.html

To enable communication between the hosts and to enable access control to the instances we create we need create a security group. Later we can create rules which would enable us to ssh to our instances.

$ aws ec2 create-security-group --group-name OracleRACSecurityGroup --description "Communication with/between Oracle RAC instances" --vpc-id vpc-ebad038f
{
   "GroupId": "sg-eca3a58b"
}

Enable traffic from outside to our VPC:

$ aws ec2 authorize-security-group-ingress --group-id sg-eca3a58b --protocol -1 --port -1 --source-group sg-eca3a58b

Lastly, lets generate an RSA private key using which we would be able to ssh to the EC2 instances once they are created.

$ aws ec2 create-key-pair --key-name OracleRACKeyPair
{
   "KeyMaterial": "-----BEGIN RSA PRIVATE KEY-----\blabblabblabblabblabblabblabblabblab/wGsoRXVTyK4PFw+vCXZp2nJQ82C1pcV+fooXR4GLhV\nTTCLdx92a7C3N8MFJm/blabblabblabblabblabblabblabblab\blabblabblabblabblabblab+qespZjSj+h71K2\ntIsp9jKXq993NlVLpv4CoGFhh9baCcE+LOAdw/uE8+fwPmLOzC7FizD/xWHAxC2s1KbwcRrKXdF……………………………
……………………………
……………………………
……………………………
HCINJ1mY5Pr31YKB5UrogSBewQotRKYcydIhkClqDcCxwEZ\blabblabblabblabblabblabblabblabblab/H6LgNF4AsBYpIEnHyd6PVG53YATwKBgHlhMP3d+jVE/blabblabblabblabblabblab\nTloLWbn9UqzGqQ3vcIlvWXcRs/NUg+rFipol17JhTvNOJm89di574fOSyHihLYpbsySZ+7W+YBtW\blabblabblab+5nUE4VOPKfqSVcAqth6rRBzmxXsinne7kqrXFbmea9\n-----END RSA PRIVATE KEY-----",
   "KeyName": "OracleRACKeyPair",
   "KeyFingerprint": "100:c3:2f:r1:e8:07:51:55:9d:z3:49:d2:3f:4e:a9:01:ac:97:52:10"
}

Copy from -----BEGIN RSA PRIVATE KEY----- through -----END RSA PRIVATE KEY----- into a file on your local host called OracleRACKey.pem. Change the permissions on the file to “600”.
$ chmod 600 OracleRACKey.pem

All of the above can be achieved using GUI. Adding screenshots for reference:
From the dashboard, under the Networking Section, select “VPC”.

Note: Anytime to get to the main dashboard, click the orange cube on the top left of the page.
Click on “Start VPC Wizard”:

Click Select.


Update VPC Name, Zone, Subnet Name and then click “Create VPC”.

This creates the VPC, Internet Gateway & Subnet, in just a couple of clicks. Now to create the hosted zone and DNS.
From the main AWS dashboard, Click Route 53.

Click on Hosted Zones.

Click on Create Hosted Zone.


Update the Domain Name, Comment, select Type as Private and then select the VPC from the drop down. Click “Create”.







Now add DNS entries for instances, private and virtual IP etc. Eg.
Click “Create Record Set”



Click “Create”.
Similarly add entries for each instance, private, virtual and storage interfaces. Refer to JSON file created earlier.




Creating Security Group
From the main dashboard, Click “EC2”.

 Click on “Security Groups” from the left menu under “Network & Security”.




Click “Create Security Group”.






Update Group Name, Description, Select the correct VPC. Add rule to ssh. Click “Create”.

Posted by Apun Hiran at 5:55 AM  

Labels:

0 Comments:

Post a Comment

Subscribe to: Post Comments (Atom)