Thursday, August 26, 2021

How to send approval notification from Jenkins Pipeline to Slack | Trigger Jenkins Job using Webhooks from Slack

How to integrate Slack with Jenkins using webhooks? How to send approval notification from Jenkins to Slack and approve and trigger job from Slack to Jenkins. We need to create webhooks in Slack to send messages to Slack channel.



Pre-requisites:
1. Jenkins is up and running
2. Slack account is is created and channel is created.

Steps to integrate Slack with Jenkins using Webhooks

Create Webhooks in Slack
2. Click on Create an app

3. Select from scratch


4. Name the app and choose the workspace

Now click on incoming webhooks

Enable it by clicking on


Now select the channel where you like to post messages

Now copy the Webhook url which we will be using in Jenkins pipeline:



Now go to Jenkins and create this pipeline:

import groovy.json.JsonOutput

pipeline {
    agent any
    tools {
        maven 'Maven3'
    }
    environment {
        jenkins_server_url = "http://ec2-3-129-59-179.us-east-2.compute.amazonaws.com:8090"
        notification_channel = 'aug-2021-weekday-batch'
        slack_url = 'https://hooks.slack.com/services/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'
        
    }

   stages {
       
    stage('Cloning Git') {
            steps {
                checkout([$class: 'GitSCM', branches: [[name: '*/main']], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: '8b429648-98dd-446a-80c6-120f9eab44fe', url: 'https://github.com/akannan1087/myAug21WeekendRepo']]])     
            }
        }
    stage ('Build') {
      steps {
      sh 'mvn clean install -f MyWebApp/pom.xml'
      }
    }

    stage ('DEV Deploy') {
      steps {
      echo "deploying to DEV Env "
      deploy adapters: [tomcat9(credentialsId: '341bc648-7df9-4b41-b520-9872cd2d8097', path: '', url: 'http://ec2-18-222-124-177.us-east-2.compute.amazonaws.com:8080/')], contextPath: null, war: '**/*.war'
      }
    }
    stage('QA approve') {
        steps {
          notifySlack("Do you approve QA deployment? $jenkins_server_url/job/$JOB_NAME", notification_channel, [])
            input 'Do you approve QA deployment?'
            }
        }
        
    stage ('QA Deploy') {
      steps {
      echo "deploying to QA Env "
      deploy adapters: [tomcat9(credentialsId: '341bc648-7df9-4b41-b520-9872cd2d8097', path: '', url: 'http://ec2-18-222-124-177.us-east-2.compute.amazonaws.com:8080/')], contextPath: null, war: '**/*.war'
      }
    }
    
    }
}

def notifySlack(text, channel, attachments) {

    def payload = JsonOutput.toJson([text: text,
        channel: channel,
        attachments: attachments
    ])

    sh "curl -X POST --data-urlencode \'payload=${payload}\' ${slack_url}"
}


Tuesday, August 24, 2021

Automate Docker builds using Jenkins - Dockerize PHP App | Containerize PHP App | Build Docker images using Jenkins

We will learn how to automate Docker builds using Jenkins and Deploy PHP based Docker container into EKS cluster. I have already created a repo with source code + Dockerfile. The repo also have Jenkinsfile for automating the following:


- Automating builds
- Automating Docker image creation
- Automating Docker image upload into AWS ECR
- Automating Docker container Deployment into EKS Cluster



Pre-requisites:
1. Amazon EKS Cluster is setup and running. Click here to learn how to create Amazon EKS cluster.
3. Setup Jenkins slave, install docker in it.
4. Docker, Docker pipeline and Kubernetes Deploy plug-ins are installed in Jenkins
5. Create an IAM role with Administrator policy, attach to Jenkins EC2 instance


Step #1 - Create Credentials for Kubernetes Cluster
Go to your Jenkins where you have installed Docker as well. Go to credentials -->
Click on Add Credentials, use Kubernetes configuration from drop down.


execute the below command to get kubeconfig info, copy the entire content of the file:
sudo cat ~/.kube/config


Enter ID as K8S and choose enter directly and paste the above file content and save.
Step #2 - set a clusterrole as cluster-admin

By default, clusterrolebinding has system:anonymous set which blocks the cluster access. Execute the following command to set a clusterrole as cluster-admin which will give you the required access.

kubectl create clusterrolebinding cluster-system-anonymous --clusterrole=cluster-admin --user=system:anonymous

Step # 3 - Create a pipeline in Jenkins, name can be anything


Step # 4 - Copy the pipeline code from below
Make sure you change red highlighted values below:
Your docker user id should be updated.
your registry credentials ID from Jenkins from step # 1 should be copied

pipeline {
    agent any 
      environment {
        registry = "acct_id.dkr.ecr.us-east-2.amazonaws.com/
your_ecr_repo"
    }

    
    stages {
        stage('Cloning Git') {
            steps {
                checkout([$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: '', url: 'https://bitbucket.org/ananthkannan/phprepo/']]])       
            }
        }
    
    // Building Docker images
    stage('Building image') {
      steps{
        script {
          dockerImage = docker.build registry
        }
      }
    }
    
  // Uploading Docker images into AWS ECR
    stage('Pushing to ECR') {
     steps{  
         script {
                sh 'aws ecr get-login-password --region us-east-2 | docker login --username AWS --password-stdin 
acct_id.dkr.ecr.us-east-2.amazonaws.com'
                sh 'docker push 
acct_id.dkr.ecr.us-east-2.amazonaws.com/your_ecr_repo:latest'
         }
        }
      }
    
      stage ('K8S Deploy') {
       
                kubernetesDeploy(
                    configs: 'phpK8SDeploy.yaml',
                    kubeconfigId: 'K8S',
                    enableConfigSubstitution: true
                    )           
                  
        }

  }
}  


Step # 5- Click on Build - Build the pipeline
Once you create the pipeline and changes values per your Docker user id and credentials ID, click on 

Step # 6 - Verify deployments to K8S

kubectl get pods


kubectl get deployments


kubectl get services

Steps # 7 - Access PHP App
Once build is successful, go to browser and enter http://public_dns_name:8086
You should see page like below:



Thursday, August 19, 2021

Create Freestyle job in Jenkins | How to create Freestyle job in Jenkins to automate build and deployment to Tomcat

Jenkins is popular open source Continuous integration tool. It was written entirely in Java. Jenkins is a self-contained automation server used for automating builds, tests and deployment.

See below the steps for configuring Jenkins to automate the build and deployment for the project we already set up in GitHub. Let us see how to configure Jenkins to automate build and deployment:


pre-requisites:(please do this first before you start)

1. Make sure you configure maven installation under Jenkins-->manage Jenkins-> Global Tool Configuration. under maven installation. enter Maven3 as name, enter path of maven installation --> /usr/share/maven and uncheck install automatically option.

2. Also install SonarQube scanner,  deploy to container Jacoco plugins under Jenkins --> Manage Jenkins --> Manage plug-ins

Click on Available, type Sonarqube, select SonarQube scanner. Click on Install without restart.

SonarQube

Deploy to container


JaCoCo

Click on without restart.

3. Generate Personal Access Token(PSA) if you are using GitHub. Click here to learn how to generate that in GitHub and you can use your token as password.

steps to automate MyWebApp project in Jenkins:

1. Login to Jenkins. Click on New item.

2. Enter an item name --> select Free style project.
enter name as myFirstAutomateJob. click OK.
3. under source code mgmt, click git. enter your source code GitHub URL
Click on your repo, Copy the url from the browser. Paste the url as Repository URL below.


under credentials --> click Add- > select Jenkins -->  enter your GitHub userid and your PSA - Personal access token as password. You would have created PSA from pre-requisites (# 3) 
Add description as my SCM credentials.
Enter main as branch specifier or which ever branch you want to check out.

4. select that from drop down.
5. under build trigger click on poll scm, enter this value to check

for every 2 mins --> H/02 * * * *

6. Build --> Add build step --> invoke top level maven targets -->

select Maven3 from drop down and goal as clean install


7. Click on advanced, enter the path of POM file as --> MyWebApp/pom.xml


8. click on Add post build action, select Record Jacoco Code coverage report
  
9. click on Add post build action, select deploy war/ear to container.

      for WAR/EAR files enter as 
          **/*.war


in WAR/EAR files, leave context path empty

   10. click on Add container , select Tomcat 9.x

   11. click on add credentials, enter tomcat as user name and password as password.
      select it from drop down.
 


13. tomcat url should be --> http://your_public_dns_name:8080


click Apply, click Save
click on build now..It should build.
if there is any error, please check the console output. Most of the common error would be checking the path of Maven installation, valid credentials for GitHub or Tomcat. Also make sure you install the plug-ins.

After successful deployment, please make sure you check the output in Tomcat by going to browser and enter below URL

You should see Hello World!!!


This is how you automate the builds and deployments using Jenkins and migrate applications to AWS.

You can watch the video of this lab here:

  

stderr: remote: Support for password authentication was removed on August 13, 2021. Please use a personal access token instead | Fix for this issue | How to Create Personal Access Token in GitHub?

How to Create Personal Access Token in GitHub?

Fix for Support for password authentication was removed on August 13, 2021

How to Fix for the above error by creating Personal Access Token:GitHub removed password authentication support from August 13, 2021 instead it recommends to use either OAuth or Personal Access Token.

Create Personal Access Token in GitHub

Go to GitHub.com--> Settings

Go to Developer Settings 
Go to Personal Access tokens --> Generate new token
Enter name for the token, choose no expiration if you don't want your token to expire.


Select repo and click generate token

Now copy the token and use this as password.

Watch here on how to do this on YouTube Channel:

Friday, August 6, 2021

Ansible playbook to install Apache on Ubuntu | How to install Apache Using Ansible on Ubuntu 22.0.4

Find the playbook for installing Apache on Ubuntu using Ansible:

Click here if you would like learn how to create a new Ubuntu EC2 instance using Ansible Playbook.

Pre-requisites:

Setup Ansible on EC2 instance with Boto.

Install Apache using Ansible Playbook

1. Login to Ansible machine. Create SSH keys in Ansible host machine by executing the below command:

ssh-keygen
2. Copy the public keys(id_rsa.pub) from ansible node each node in /home/ubuntu/.ssh/authorized_keys file. Execute the below command on Ansible and copy the content:
 sudo cat ~/.ssh/id_rsa.pub

3. Now login to target node, execute the below command to open the file
sudo vi /home/ubuntu/.ssh/authorized_keys
type shift A and then enter. now 
    and paste the key in the above file. please do not delete any existing keys.

4. go back to ansible mgmt node, make sure you are able to ssh from ansible mgmt node after copying the keys above:
ssh private_ip_of_target_node
now type exit to come out of the target node.

5. Now in ansible mgmt node, now make changes in /etc/ansible/hosts file to include the node you will be installing software. Make sure you add public IP address as highlighted below in green color:
sudo vi /etc/ansible/hosts
[Apache_Group]  
XX.XX.XX.XX ansible_ssh_user=ubuntu ansible_ssh_private_key_file=~/.ssh/id_rsa  ansible_python_interpreter=/usr/bin/python3

6. make changes in playbooks as given below,
cd ~/playbooks

sudo vi installApache.xml
Copy the below content

---
- hosts: Apache_Group
  become: true
  tasks:
    - name: Install apcahe
      apt: name=apache2 state=present update_cache=yes

    - name: ensure apache started
      service: name=apache2 state=started enabled=yes


7. now execute the playbook by running the below command:
sudo ansible-playbook installApache.xml
8. This should install Apache on the nodes and should bring the apache up and running.
9. Now enter public ip address or public dns name of target server by in the browser to see home page of Apache running.


Please watch this step on YouTube channel:

How to setup Jenkins on Ubuntu using Ansible playbook | Setup Java, Jenkins, Maven on Ubuntu EC2 using Ansible Playbook

Here below are the playbooks for installing Java, Jenkins, Maven on Ubuntu EC2 instance using Ansible. You need to install Java first (first link below) and then do the steps in the second link for installing Jenkins, third link for installing Maven.

Click here if you would like to create a new Ubuntu EC2 instance using Ansible Playbook.

You can watch this lab on YouTube:

Tuesday, August 3, 2021

DevSecOps Coaching Program Model Information | AWS Cloud Azure Cloud DevOps Coaching Program Model Information

(More New Topics..New CICD tool GitHub Actions, Helm included !!)

Here is the coaching model:
  • Total 12 weeks of coaching program
  • 2 sessions per weekend/week (every session lasts for 2 hours generally)
  • Fast paced
  • Purely lab oriented(hands on)
  • ~80% hands on, ~20% theory
  • 50+ lab exercises, also few bonus lab exercises(~60 total lab exercises in 10 weeks)
  • Wiki page for troubleshooting needs
  • Collaborative way of learning by having separate WhatsApp group per batch.
  • Coach’s website(www.cidevops.com & www.coachdevops.com) also has instructions
  • YouTube channel
  • Experience assistant coach is also available to help resolve lab related issues
  • Help in resume preparation
  • Help in interview preparation
  • Interview Coaching & tips
  • DevOps concepts and interview notes will be provided
About Coach Ananth(or known as Coach AK):
  • TOGAF certified architect
  • 23 years of professional IT experience
  • 9+ years in practicing DevOps and Cloud computing.
  • Comprehensive hands on knowledge on Git, BitBucket, GitHub, Jenkins, Maven, SonarQube, Nexus, Artifactory, Slack, Terraform, Ansible, Docker, Helm, Prometheus and Kubernetes on AWS and Azure cloud platforms.
  • Many students already placed in reputed companies from the coaching program successfully.
  • Working as a Sr.DevOps Coach/Architect in a top IT services company in US.
  • Unique program...less theory, more hands on work on AWS and Azure 
  • Resume preparation will be done with candidates personally.
  • One-to-one Interview coaching.
  • Coaching is purely hands on with job relevant.
  • Coached about 2000+ people successfully.
  • Completed seven years of coaching anniversary past August 2024.
Useful Links:
    • Click here for the coaching schedule
    • Click here to know more about the program
    • Click here to know about the facts about the program
    Just believe!! Anything is possible!!

    How to Configure GitHub Advanced Security for Azure DevOps | How to Perform Security scan for Azure Repos using GitHub Advanced Security

    GitHub Advanced Security for Azure DevOps brings the  secret scanning, dependency scanning  and  CodeQL code scanning  solutions already ava...