Provision Ubuntu 22.0.4 EC2 Instance | How to create EC2 instance in AWS console | Launch Ubuntu 22.0.4 instance in AWS

How to create new EC2 instance in AWS console using new UI experience?

What is EC2 instance? 

It is virtual server provided by AWS. We will be using this EC2 to setup both Jenkins and Tomcat. Please follow the below steps to create an EC2 instance.

Watch in YouTube for a demo:

Steps:
1: Login to AWS console by clicking this link -->  https://aws.amazon.com/console/

click on All services, Click on Compute -->  Click on EC2

2. Click on Launch instance

3. Enter Name as EC2 and enter 2 as number of instances (one for Jenkins and another for Tomcat)

4. Select Ubuntu

 and choose Ubuntu server 22.0.4 as AMI

5. Enter t2.small as instance type

6. Click on Create new Key Pair

7. Choose the existing key pair if you have one, otherwise create new one, give some name as myJenkinsKey. Make sure you download the key in your local machine. Please do NOT give space or any character while naming the key.

8. Under Network settings, Click Edit

Add port range as 8080 and select AnyWhere as Source Type, that should enter 0.0.0.0/0 as Source

9. Configure Storage

Enter 15 GB as root volume 

And then make sure in Summary, values appear as below:

10. Click on Launch Instance.

Click on View instances

Now you should be able to view instances in AWS console. Now you can re-name as Jenkins-EC2 and Tomcat-EC2

Connect to EC2 instance from local machine:
Please click the below link to understand the steps for connecting to EC2 instance from your local machine - windows or Apple laptop.

http://www.cidevops.com/2018/02/how-to-connect-to-ec2-instance-from.html

Install Jenkins on Ubuntu 22.0.4 | Setup Jenkins on AWS EC2 Ubuntu instance | How to setup Jenkins in Ubuntu EC2 instance?

Jenkins is an open source, continuous integration/continuous delivery and deployment (CI/CD) automation software DevOps tool written in the Java programming language. It is used to implement CI/CD workflows, called pipelines. 

Please follow the steps to install Java, Jenkins, Maven on Ubuntu 22.0.4 instance. Jenkins, Maven are Java based applications, so we need to install Java first. 

Pre-requisites:

• EC2 instance is up and running
• port 8080 is open in firewall rule
• connect to EC2 instance using Git bash or item or any SSH terminal

Steps to install Jenkins:

Change Host Name to Jenkins
sudo hostnamectl set-hostname Jenkins

Perform update first
sudo apt update

Install Java 11

sudo apt install default-jdk -y

 

Once install java, enter the below command

Verify Java Version

java -version

Maven Installation
Maven is a popular build tool used for building Java applications. Please click here to learn more about Maven. You can install Maven by executing below command:

sudo apt install maven -y

you can type mvn --version
you should see the below output.

Now lets start Jenkins installation

Jenkins Setup

Add Repository key to the system

curl -fsSL https://pkg.jenkins.io/debian/jenkins.io-2023.key | sudo tee \

  /usr/share/keyrings/jenkins-keyring.asc > /dev/null

Append debian package repo address to the system

echo deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] \

  https://pkg.jenkins.io/debian binary/ | sudo tee \

  /etc/apt/sources.list.d/jenkins.list > /dev/null

Update Ubuntu package

sudo apt update

Install Jenkins

sudo apt install jenkins -y

The above screenshot should confirm that Jenkins is successfully installed.

Access Jenkins in web browser

Now Go to AWS console. Click on EC2, click on running instances link. Select the checkbox of EC2 you are installing Java and Jenkins. Click on Action. Copy the value from step 4 that says --> Connect to your instance using its Public DNS:

Now go to browser. enter public dns name or public IP address with port no 8080.

http://EC2_public_dns_name:8080

Unlock Jenkins
You may get screen, enter the below command in Git bash( Ubuntu console)

Get the initial password from the below file
sudo cat /var/lib/jenkins/secrets/initialAdminPassword

Copy the password and paste in the browser.

Then click on install suggested plug-ins. 

Also create user name and password.

enter everything as admin. at least user name as admin password as admin

Click on Save and Finish. Click on start using Jenkins. Now you should see a screen like below:

That's it. You have setup Jenkins successfully. Please watch the steps in our YouTube channel:

Top DevOps Skills | Skills required to become a DevOps engineer | Complete Roadmap to become a DevOps Engineer

Complete Roadmap to become a successful DevOps Engineer

We all know how DevOps is trending right now. And we know where it is going. Let's get to know what skills will make you a successful a DevOps engineer.

Top DevOps skills

1. Linux knowledge and scripting - basic troubleshooting, intermediate scripting, looking at the logs

2. Experience in Git, GitHub, Bitbucket or any version control systems such as SVN, TFVC

3. Experience in Continuous Integrations tools such as Jenkins, TeamCity, Circle CI 

4. Experience in Infrastructure automation tools such as Terraform, AWS cloud formation

5. Experience in Configuration Management tools such as Ansible, Puppet or Chef

6. Experience in scripting languages such as YAML, groovy, Ruby, Python and Shell

7. Experience in containers such as Docker, Kubernetes and Helm

8. Experience in Monitoring tools such as Prometheus, Grafana

9. Ability to troubleshoot in case builds, deployments failure.

10. Any cloud knowledge and experience - AWS, Azure and Google cloud

Soft skills employers are looking:

These days employers are not only looking for strong technical skills but also looking "soft skills" which are essentials to become successful in IT. If you think if you are lagging on any of these skills, no worries. All these skills can be developed and improved over period of time by practicing.

1. Open minded

2. Willingness to learn new skills

3. Communication

4. Approachable

5. "Get it done" attitude

6. Being adaptable. 

How to setup monitoring on AKS Cluster using Prometheus and Grafana | Setup monitoring on AKS Cluster using Prometheus and Grafana | Monitor Azure Kubernetes Service(AKS) with Prometheus and Grafana

How to setup monitoring on AKS Cluster using Prometheus and Grafana?

What is Azure Log Analytics Workspace?

Azure Log Analytics workspace is a central place where you can monitor and manage all the Azure Monitor logs collected from one or more Azure resources such as Azure Virtual machine or Azure Kubernetes Service. You can think of the workspace as a folder where all your monitoring data is stored.

What is Prometheus?

• Prometheus is an open source monitoring and alerting tool.
• Provides out-of-the-box monitoring capabilities for the Kubernetes container orchestration platform. It can monitor servers and databases as well.
• Collects and stores metrics as time-series data, recording information with a timestamp 
• It is based on pull and collects metrics from targets by scraping metrics HTTP endpoints.

What is Grafana?

• Grafana is an open source visualization and analytics tool. 
• It allows you to query, visualize, alert on, and explore your metrics no matter where they are stored.

Installation Method:

The are are many ways you can setup Prometheus and Grafana. You can install in following ways:

1. Create all configuration files of both Prometheus and Grafana and execute them in right order.

2. Prometheus Operator - to simplify and automate the configuration and management of the Prometheus monitoring stack running on a Kubernetes cluster

3. Helm chart (Recommended) - Using helm to install Prometheus Operator including Grafana

Why to use Helm?

Helm is a package manager for Kubernetes. Helm simplifies the installation of all components in one command. Install using Helm is recommended as you will not be missing any configuration steps and very efficient. 

Pre-requisites:

• Azure subscription, click here if you don't have one.
• Install Azure CLI on your local machine
• Install Helm3

Create AKS Cluster

Make sure you are login to Azure portal first.

az login

enter your Microsoft credentials.

Create a resource group first

az group create --name myResourceGroup --location southcentralus

Create Log Analytics workspace

az monitor log-analytics workspace create --resource-group myResourceGroup
                                           --workspace-name my-loganalytics-workspace \
                                           --query id \
                                           -o tsv)

Output of the above command will display log analytics Id which is needed for next command while creating AKS cluster

Create AKS cluster with 2 worker nodes with Monitoring Enabled

az aks create --resource-group myResourceGroup --name myAKSCluster --node-count 2 --enable-addons monitoring --workspace-resource-id /subscriptions/XXXXX/resourceGroups/myResourceGroup/providers/Microsoft.OperationalInsights/workspaces/aks-loganalytics-workspace

Verify all the resources are created in Azure Portal

Click on resource group name, you will see AKS cluster, log analytics workspace

Display Details of Cluster

az aks show --name myAKSCluster --resource-group myResourceGroup

The above command will display Cluster details.

Connect to the cluster

az aks get-credentials --resource-group myResourceGroup --name myAKSCluster --overwrite-existing

To verify the connection to your cluster, use the kubectl get command to return a list of the cluster nodes.

kubectl get nodes

 

 

Implementation steps

We need to add the Helm Stable Charts for your local client. Execute the below command:

helm repo add stable https://charts.helm.sh/stable

# Add prometheus Helm repo

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts

# Search for newly installed repositories

helm repo list

helm search repo prometheus-community

Prometheus and grafana helm chart moved to kube prometheus stack

Create Prometheus namespace
kubectl create namespace prometheus

Install kube-prometheus-stack

Below is helm command to install kube-prometheus-stack. The helm repo kube-stack-prometheus (formerly prometheus-operator) comes with a grafana deployment embedded.

helm install stable prometheus-community/kube-prometheus-stack -n prometheus

Lets check if prometheus and grafana pods are running already

kubectl get pods -n prometheus

kubectl get svc -n prometheus

This confirms that prometheus and grafana services have been created successfully using Helm. To access them, you will need to run the following commands:

Login to Grafana and Prometheus

Expose Prometheus

# Port forward the Prometheus service

kubectl port-forward -n prometheus prometheus-stable-kube-prometheus-sta-prometheus-0 9090

Now Open localhost:9090 in browser to access Prometheus

Expose Grafana

# Port forward the Grafana service

kubectl port-forward  -n prometheus stable-grafana-XXX 3000

Now Open localhost:3000 in browser to access Grafana

UserName: admin

Password: prom-operator

Create a Service Principal and Assign Role

We need to create Service principal(SPN) and assign Monitoring Reader Role on the AKS Cluster ResourceGroup. Execute below command to 

First Let's get Resource Group ID 

az group show --name  myResourceGroup --query id --output tsv

output of the above command will be resource group ID. Create service principal now:

az ad sp create-for-rbac --role="Log Analytics Reader" --scopes="/subscriptions/xxxxxx-xxxx-xxxx-xxxxx/resourceGroups/myResourceGroup"

Creating 'Log Analytics Reader' role assignment under scope '/subscriptions/XXX/resourceGroups/myResourceGroup'

{
  "appId": "xxx",
  "displayName": "azure-cli-2023-04-12-22-06-28",
  "password": "xxx",
  "tenant": "xxx"
}

Note all the information above and save it.

Create Data Source in Grafana

Go to Grafana, configuration and click Data Sources. Click on Add Data Source and search for Azure Monitor.

Enter tenant id, app id and secret information. click on Load subscriptions. Click on Save and Test.

Create Dashboard in Grafana

In Grafana, we can create various kinds of dashboards as per our needs.

How to Create Azure Monitor For Containers Dashboard?

Click '+' button on left panel and select ‘Import’.

Enter 10956 dashboard id under Grafana.com Dashboard.

Click ‘Load’.

Click ‘Import’.

This will show monitoring dashboard for all cluster nodes

Clean up Resources

Let's see how to clean up the resources that were created. We can use the az group delete command to remove the resource group, AKS cluster, and all related resources. 

az group delete --name myResourceGroup --yes --no-wait