Thursday, April 18, 2019

How to create EC2 instances using Terraform in Windows | Terraform Files for provisioning EC2 instance on AWS Cloud

Terraform can provision resources on any cloud platform. We will see how we can use Terraform to provision EC2 instance using Windows laptop. Please do the below steps for provisioning EC2 instances on AWS:


Login to AWS console, click on username and go to My security credentials.
 Continue on security credentials, click on access keys

Create a new access key if you don't have one. Make sure you download the keys in your local machine.

Pre-requisites:

Perform below commands in Windows laptop where you have installed Terraform:

First setup your access keys, secret keys and region code locally.
aws configure


Open Git bash and enter below commands:

cd c:

mkdir project-terraform
cd project-terraform

Create Terraform Files
start notepad variables.tf


variable "aws_region" {
       description = "The AWS region to create things in." 
       default     = "us-east-2
}

variable "key_name" { 
    description = " SSH keys to connect to ec2 instance" 
    default     =  "myJune2021Key
}

variable "instance_type" { 
    description = "instance type for ec2" 
    default     =  "t2.micro" 
}

variable "security_group" { 
    description = "Name of security group" 
    default     = "my-jenkins-security-group-2022" 
}

variable "tag_name" { 
    description = "Tag Name of for Ec2 instance" 
    default     = "my-ec2-instance" 
variable "ami_id" { 
    description = "AMI for Ubuntu Ec2 instance" 
    default     = "ami-0b9064170e32bde34
}
save above file as "variables.tf" under c:\project-terraform folder

start notepad main.tf

provider "aws" {
  region = var.aws_region
}

resource "aws_vpc" "main" {
  cidr_block = "172.16.0.0/16"
  instance_tenancy = "default"
  tags = {
    Name = "main"
  }
}

#Create security group with firewall rules
resource "aws_security_group" "jenkins-sg-2022" {
  name        = var.security_group
  description = "security group for jenkins"

  ingress {
    from_port   = 8080
    to_port     = 8080
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

 ingress {
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

 # outbound from Jenkins server
  egress {
    from_port   = 0
    to_port     = 65535
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags= {
    Name = var.security_group
  }
}

resource "aws_instance" "myFirstInstance" {
  ami           = var.ami_id
  key_name = var.key_name
  instance_type = var.instance_type
  vpc_security_group_ids = [aws_security_group.jenkins-sg-2022.id]
  tags= {
    Name = var.tag_name
  }
}


save above file as "main.tf" under c:\project-terraform folder.
If you dont provide double quote, it will be .txt in the end.

Now execute the below command:
terraform init
* provider.aws: version = "~> 1.22"
Terraform has been successfully initialized!

and then execute the below command
terraform plan
the above command will show how many resources will be added.
Plan: 3 to add, 0 to change, 0 to destroy.

Execute the below command
terraform apply
Plan: 3 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

Apply complete! Resources: 3 added, 0 changed, 0 destroyed.
 
List Resources created by Terraform
Execute the below command to view list of the resources created by Terraform.
terraform state list
The above command will list three resources created.


Now login to AWS EC2 console, you would see the new instances up and running.

How to push Terraform files into GitHub
Pre-requisites:
First create SSH keys and upload public keys into GitHub by executing below command:
 
ssh-keygen
This should generate both public and private keys. Now navigate to c:\users\your_username\.ssh folder and open id_rsa.pub(pubkic key) file probably in notepad and copy the whole content of it and upload into GitHub. 

Create Remote repo in GitHub
Create a new repo with below name, make sure it is a private repo. Also do not click on initialize this repository with a README option.

 Note down the remote url as highlighted below:


All Terraform files should be checked into version control systems such as GitHub, BitBucket or GitLab. Let us see how to push code changes into GitHub. Make sure you are in the directory where Terraform files are created.

Initialize the directory first
Once you uploaded make sure you are in the directory where you had created tf files.
git init

The above command will create local git repository.
Now add terraform files.
git add *.tf

You need to setup your email address and user name.
git config --global user.email "your@email.com"

git config --global user.name "your_userid"
 
git commit -m "added terraform files"
Copy the below green highligted url from above screenshots circled in red.
git remote add origin git@github.com:userid/myInfraRepo.git

Now push the code into GitHub
git push -u origin master

Now Login to GitHub to view the Terraform files


Note:
If you have any issues in uploading tf files, you may not have created ssh-keys and uploaded into GitHub. Create ssh keys using ssh-keygen command:
ssh-keygen
This should generate both public and private keys.
navigate to c:\users\your_username\.ssh folder and open id_rsa.pub(pubkic key) file and copy the content of it and upload into GitHub. 

Monday, April 15, 2019

Install Sonarqube on Ubuntu - How to install SonarQube on Ubuntu 16.0.4 - How to setup Sonar on Ubuntu?

Please find steps for installing SonarQube on Ubuntu EC2. Make sure port 9000 is opened in security group(firewall rule).

SonarQube is java based tool along with back end - back end can be MySQL, Oracle or PostgreSQL. We will use Postgres for set up on Ubuntu.

Let us start with java install (skip java install if you already have it installed)

1. Java steps 

sudo apt-get update
sudo apt-get install default-jdk -y

Verify Java Version

java -version

openjdk version "1.8.0_191"
OpenJDK Runtime Environment (build 1.8.0_191-8u191-b12-2ubuntu0.16.04.1-b12)
OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)

2. Postgres Installation

1. sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt/ `lsb_release -cs`-pgdg main" >> /etc/apt/sources.list.d/pgdg.list'


 
2. sudo wget -q https://www.postgresql.org/media/keys/ACCC4CF8.asc -O - | sudo apt-key add -



3. sudo apt-get -y install postgresql postgresql-contrib








4. sudo systemctl start postgresql
5. sudo systemctl enable postgresql

Login as postgres user now
7. sudo su - postgres

8. Now create a user below
createuser sonar

9. Switch to sql shell by entering
psql







Execute the below three lines (one by one)

ALTER USER sonar WITH ENCRYPTED password 'password';
CREATE DATABASE sonar OWNER sonar;
\q






type exit to come out of postgres user.




3. Now install SonarQube Web App

sudo wget https://binaries.sonarsource.com/Distribution/sonarqube/sonarqube-6.4.zip


sudo apt-get -y install unzip
sudo unzip sonarqube-6.4.zip -d /opt





sudo mv /opt/sonarqube-6.4 /opt/sonarqube -v




Modify sonar.properties file
sudo vi /opt/sonarqube/conf/sonar.properties
uncomment the below lines by removing # and add values highlighted yellow
sonar.jdbc.username=sonar
sonar.jdbc.password=password







Next, uncomment the below line, removing #
sonar.jdbc.url=jdbc:postgresql://localhost/sonar





Press escape, and enter :wq! to come out of the above screen.

Create Sonar as a service

Execute the below command:
sudo vi /etc/systemd/system/sonar.service











add the below code in green color:
[Unit]
Description=SonarQube service
After=syslog.target network.target

[Service]
Type=forking

ExecStart=/opt/sonarqube/bin/l
inux-x86-64/sonar.sh start
ExecStop=/opt/sonarqube/bin/li
nux-x86-64/sonar.sh stop

User=root
Group=root
Restart=always

[Install]
WantedBy=multi-user.target


sudo systemctl enable sonar
sudo systemctl start sonar
sudo systemctl status sonar
type q now to come out of this mode.
Now execute the below command to see if Sonarqube is up and running. This may take a few minutes.

tail -f /opt/sonarqube/logs/sonar.log

Make sure you get the below message that says sonarqube is up..
Now access sonarQube UI by going to browser and enter public dns name with port 9000

Click here to learn how to integrate SonarQube with Jenkins.

Please watch above steps in myYouTube video as well:

Tuesday, April 9, 2019

Build Pipelines in Azure DevOps | How to build Azure Build pipelines to migrate Java WebApp to Azure Cloud

Building pipelines in Azure DevOps is really easy, you can migrate your web application code from any where into Azure Cloud by using Azure pipelines.

We are going to migrate Java Web App that was setup in Azure Git into Azure Cloud by creating a WebApp in Azure.

Pre-requisites:

You need to create WebApp in Azure Cloud. WebApp is an App service (mostly Platform as a Service) provide by Azure Cloud to migrate any web applications.

Once you sign in to Azure Portal, you need to create an app service which is a WebApp.

Steps for Creating WebApp in Azure Portal

1. Now login to https://portal.azure.com
2. Click on App services


3.Click on + Add or click on Create app service



Click on Web App.

Choose Free Azure subscription

Create a new resource group(for first time if you create an app service, otherwise you can use existing group name)
Enter App service name(it should be unique)
Publish as Code
Run time stack as Java 11 --> Tomcat9.0
Operating System as Linux
Region as South Central US
Enter LinuxPlan name
Choose SKU and size as given below:
Choose DEV/Test and use 1 GB memory



Click on Apply
Now click on Apply & Create
This will take bit time to create the app service.

Once WebApp is created, go resources, click on WebApp name and click on the URL.



You should see the app service home page some thing like below:



Since our WebApp is up and running, now we can start creating pipelines to migrate to Azure cloud.

2. Creating pipelines in Azure Devops

1. Now go to your visual studio project page --> https://dev.azure.com/
Select the Project dashboard you already created.

Click on Pipelines, Builds.


2. Click on New pipeline and click on use the classic editor to create a pipeline without YAML

3. Select source repository as Azure Repo Git and click continue

4. Since our application is Java stack, type Java and Choose Azure WebApp for Java


5. Click on pipeline name and re-name per your naming standard

6. Modify maven goal as clean install and also choose POM.xml by clicking on three dots ...


7. Leave the value as it is for Copy Files to staging folder

8. Leave the value as it is for Publish Artifact: Drop
9. Click on Stop Azure WebApp step, Enter Azure WebApp details - where  you would like to deploy your app in Azure. Select Free trial subscription from the drop down. Click on Authorize button.
Make sure you disable popup blocker.


10. Do the same in Deploy Azure WebApp & Start Azure WebApp steps.

Enable Webhooks in Pipeline

11. Click on Triggers and then enable check box for Continuous Integration

12. Now click on Save and Queue

13. If your configurations are correct, you should be able to see the build success and able to access in the Azure. Click on Pipelines and pipeline name

Now you will see that Azure DevOps would have started build.

14. Make sure build is success, it should have all green like below.




After successful build, you can check the output by accessing below URL:

https://myAzureWebAppUrl/MyAwesomeApp


How to Configure GitHub Advanced Security for Azure DevOps | How to Perform Security scan for Azure Repos using GitHub Advanced Security

GitHub Advanced Security for Azure DevOps brings the  secret scanning, dependency scanning  and  CodeQL code scanning  solutions already ava...