How to Configure GitHub Advanced Security for Azure DevOps | How to Perform Security scan for Azure Repos using GitHub Advanced Security

GitHub Advanced Security for Azure DevOps brings the secret scanning, dependency scanning and CodeQL code scanning solutions already available for GitHub users and natively integrates them into Azure DevOps to protect your Azure Repos and Pipelines. 

Pre-requisites:

• Azure DevOps account
• Enable GitHub advanced Security billing in ADO
• Application code, I am using Java project to scan

How to Set up dependency scanning?

Dependency scanning is a pipeline-based scanning tool. Results are aggregated per repository. It's recommended that you add the dependency scanning task to all the pipelines you'd like to be scanned.

Add the task Advanced Security Dependency Scanning task (AdvancedSecurity-Dependency-Scanning@1) directly to your YAML pipeline file or select the Advanced Security Dependency Scanning task from the task assistant.

How to Set up code scanning

Code scanning is also a pipeline-based scanning tool where results are aggregated per repository.

Add the tasks in the following order:

1. Advanced Security Initialize CodeQL (AdvancedSecurity-Codeql-Init@1)
2. Your custom build steps
3. Advanced Security Perform CodeQL Analysis (AdvancedSecurity-Codeql-Analyze@1)

Pipeline YAML Code for scanning Java code using GitHub Advanced Security

trigger:

- main

resources:

- repo: self

variables:

tag: '$(Build.BuildId)'

stages:

- stage: Build

displayName: Build image

jobs:

- job: Build

displayName: Build

pool:

vmImage: ubuntu-latest

steps:

- task: AdvancedSecurity-Codeql-Init@1

inputs:

languages: 'java'

- task: Maven@4

inputs:

mavenPomFile: 'pom.xml'

goals: 'install'

publishJUnitResults: true

testResultsFiles: '**/surefire-reports/TEST-*.xml'

javaHomeOption: 'JDKVersion'

mavenVersionOption: 'Default'

mavenAuthenticateFeed: false

effectivePomSkip: false

sonarQubeRunAnalysis: false

- task: AdvancedSecurity-Dependency-Scanning@1

- task: AdvancedSecurity-Codeql-Analyze@1

Where to see the Scan resule in Azure DevOps?

What is GitHub Advanced Security for Azure DevOps | How to Enable GitHub Advanced Security for Azure DevOps ?

GitHub Advanced Security for Azure DevOps brings the secret scanning, dependency scanning and CodeQL code scanning solutions already available for GitHub users and natively integrates them into Azure DevOps to protect your Azure Repos and Pipelines.

These scanning tools will natively embed automated security checks into the Azure DevOps platform, allowing developers to secure their code, secrets and supply chain without leaving their workflow.

Azure DevOps Advanced Security provides below security features to help organizations identify and address security vulnerabilities in their development processes.

• Secret Scanning push protection: check if code pushes include commits that expose secrets such as credentials
• Secret Scanning repo scanning: scan your repository and look for exposed secrets that were committed accidentally
• Dependency Scanning – search for known vulnerabilities in open source dependencies (direct and transitive)
• Code Scanning – use CodeQL static analysis engine to identify code-level application vulnerabilities such as SQL injection and authentication bypass.

Scope of GitHub Advanced Security for Azure DevOps

• only available for Git repositories
• only available for Azure DevOps services, not available in Azure DevOps Server(old TFS) 

Enable GitHub Advanced Security

You can enable Advanced Security at the organization, project, or repository level.

Repository-level onboarding

1. Go to your Project settings for your Azure DevOps project.
2. Select Repos > Repositories.
3. Select the repository you want to enable Advanced Security for.
4. Select Enable and Begin billing to activate Advanced Security. A shield icon now appears in the repository view for any repository with Advanced Security enabled.

Project-level onboarding

1. Go to your Project settings for your Azure DevOps project.
2. Select Repos.
3. Select the Settings tab.
4. Select Enable all and see an estimate for the number of active committers for your project appear.
5. Select Begin billing to activate Advanced Security for every existing repository in your project.
6. Optionally, select Automatically enable Advanced Security for new repositories so that any newly created repositories have Advanced Security enabled upon creation.

Organization-level onboarding

1. Go to your Organization settings for your Azure DevOps organization.
2. Select Repositories.
3. Select Enable all and see an estimate for the number of active committers for your organization appear.
4. Select Begin billing to activate Advanced Security for every existing repository in each project in your organization.
5. Optionally, select Automatically enable Advanced Security for new repositories so that any newly created projects have Advanced Security enabled upon creation.

Setup Secret Scanning 

Secret scanning push protection and repository scanning are automatically enabled when you turn on Advanced Security. You can enable or disable secret push protection from the repository settings page.

As mentioned, secret scanning repository scanning is automatically kicked off upon enabling Advanced Security for a selected repository.

DevOps Bootcamp Dec 2024 Schedule | DevOps & AWS Azure Cloud Coaching by Coach AK | DevOps and Cloud Computing Online Classes

Lot of new topics covered like GitHub Actions, Helm, Prometheus and Grafana..)

The DevOps requirements in the IT market space is expected to grow by 35% by 2024. Getting a DevOps education now is a great investment into your future, which will pay off very fast!

You are in the right place to kick start your career in DevOps. DevOps is one of the top and hot IT skills right now. Currently almost all the employers are struggling to get right resources in their teams who can do the DevOps and automation work..You could be that person by attending this coaching program.

DevOps Coaching schedule - Dec 2024 (promotions are on, please contact Coach AK)

Date	Time	Type	When?
Dec 1st	11:35 AM CST - 01:30 PM CST on Saturdays 01:30 PM CST - 03:30 PM CST on Sundays	Weekends	Sat/Sundays
Dec 3rd	6:00 to 8:00 PM CST	Weekdays	Tuesdays/Thursdays

DevOps Coaching Highlights:

Comprehensive hands on knowledge on Git, GitHub, Jenkins, Maven, SonarQube, Nexus, Terraform, Ansible, Docker, Kubernetes, Helm, Prometheus, Docker registry, AWS and Azure cloud platform.

To join DevOps Coaching classes, please contact Coach AK below:

Contact no# : +1 (469)733-5248

WhatsApp #: +1 (469)733-5248

Email id: contact.devopscoaching@gmail.com
Contact Name: Coach AK

Deploy Python App into Kubernetes Cluster using kubectl Jenkins Pipeline | Containerize Python App and Deploy into EKS Cluster | Kubectl Deployment using Jenkins

We will learn how to automate Docker builds using Jenkins and Deploy into Kubernetes Cluster in AWS Cloud. We will use kubectl command to deploy Docker images into EKS cluster. We will use Python based application. I have already created a repo with source code + Dockerfile. The repo also have Jenkinsfile for automating the following:

- Automating builds using Jenkins
- Automating Docker image creation
- Automating Docker image upload into Elastic container registry
- Automating Deployments to Kubernetes Cluster using kubectl CLI plug-in

Pre-requisites:
1. EKS Cluster is setup and running. Click here to learn how to create EKS cluster.

2. Jenkins Master is up and running.

3. Install Docker in Jenkins.

4. Docker, Docker pipeline and Kubectl CLI plug-ins are installed in Jenkins

5. ECR repo created to store docker images.

The Code for this video is here:

Make sure you fork my repo https://github.com/akannan1087/myPythonDockerRepo

and make necessary changes in eks-deploy-from-ecr.yaml file after you fork into your account.

Step #1 - Create Credentials for connecting to EKS cluster using Kubeconfig

Go to Jenkins UI, click on Credentials -->

Click on Global credentials

Click on Add Credentials

use secret file from drop down.

execute the below command to login as jenkins user.

sudo su - jenkins

you should see the nodes running in EKS cluster.

kubectl get nodes

Create namespace to deploy containers

kubectl create namespace python-app-ns

kubectl get ns

Execute the below command to get kubeconfig info, copy the entire content of the file:

cat /var/lib/jenkins/.kube/config

Open your text editor or notepad, copy and paste the entire content and save in a file.

We will upload this file.

Enter ID as K8S and choose File and upload the file and save.

Step # 2 - Create a pipeline in Jenkins

Create a new pipeline job.

Step # 3 - Copy the pipeline code from below
Make sure you change values as per your settings highlighted in yellow below:

pipeline {

    agent any

    environment {

        registry = "account_id.dkr.ecr.us-east-1.amazonaws.com/coachak/my-docker-repo"

    }

    stages {

        stage('checkout') {

            steps {

                checkout([$class: 'GitSCM', branches: [[name: '*/master']], extensions: [], userRemoteConfigs: [[url: 'https://github.com/akannan1087/myPythonDockerRepo']]])

            }

        }

        

        stage ("build image") 

        {

            steps {

                script {

                    dockerImage = docker.build registry

                      dockerImage.tag("$BUILD_NUMBER")

                    }

                }

        }

        

        stage ("upload ECR") {

            steps {

                script {

                    sh "aws ecr get-login-password --region us-east-2 | docker login --username AWS --password-stdin account_id.dkr.ecr.us-east-2.amazonaws.com"

                sh 'docker push account_id.dkr.ecr.us-east-1.amazonaws.com/coachak/my-docker-repo:$BUILD_NUMBER'

                }

            }

        }

        

    // Avoid latest tag image and pass build ID dynamically from Jenkins pipeline

       stage('K8S Deploy') {

        steps{   

            script {

                withKubeConfig([credentialsId: 'K8S', serverUrl: '']) {

                echo "Current build number is: ${env.BUILD_ID}"

               // Replace the placeholders in the deployment.yaml file 

                sh """ 

                sed -i 's/\${BUILD_NUMBER}/${env.BUILD_ID}/g' k8s-deployment.yaml

                """ 

                sh ('kubectl apply -f  k8s-deployment.yaml -n springboot-app-ns')

                }

            }

        }

       }

    }    

}

Step # 4 - Build the pipeline

Step # 5 - Verify deployments to EKS

kubectl get pods

kubectl get deployments

kubectl get services

Steps # 6 - Access Python App in K8S cluster

Once deployment is successful, go to browser and enter above load balancer URL 

You should see page like below:

Deploy Springboot Microservices App into Amazon EKS Cluster using Jenkins Pipeline and Kubectl CLI Plug-in | Containerize Springboot App and Deploy into EKS Cluster using Jenkins Pipeline

 We will learn how to create CICD pipeline to deploy springboot microservices using Jenkins pipeline into EKS Cluster with help of Kubernetes CLI plug-in.

We will use Springboot Microservices based Java application. I have already created a repo with source code + Dockerfile. The repo also have Jenkinsfile for automating the following:

- Automating builds using Jenkins Pipeline
- Automating Docker image creation and tagging
- Automating Docker image upload into AWS ECR
- Automating Docker Containers Deployments to Kubernetes Cluster

 

Watch steps in YouTube channel:

Same Code for this video is here:

Make sure you fork my repo https://github.com/akannan1087/springboot-app

Pre-requisites:
1. Amazon EKS Cluster is setup and running. Click here to learn how to create Amazon EKS cluster.

2. Create ECR repo in AWS

3. Jenkins Master is up and running

4. Docker installed on Jenkins instance 

5. Docker, Docker pipeline and Kubernetes CLI plug-ins are installed in Jenkins

Install Kubernetes CLI plug-in:

6. Install kubectl on your instance

Step # 1 - Create Maven3 variable under Global tool configuration in Jenkins

Make sure you create Maven3 variable under Global tool configuration. 

Step #2 - Create Credentials for connecting to Kubernetes Cluster using kubeconfig

Click on Add Credentials, use Kubernetes configuration from drop down.

use secret file from drop down.

execute the below command to login as jenkins user.

sudo su - jenkins

you should see the nodes running in EKS cluster.

kubectl get nodes

Create namespace to deploy the containers

kubectl create namespace springboot-app-ns

kubectl get ns

Execute the below command to get kubeconfig info, copy the entire content of the file:

cat /var/lib/jenkins/.kube/config

Open your text editor or notepad, copy and paste the entire content and save in a file.

We will upload this file.

Enter ID as K8S and choose File and upload the file and save.

Enter ID as K8S and choose enter directly and paste the above file content and save.

Step # 3 - Create a pipeline in Jenkins

Create a new pipeline job.

Step # 4 - Copy the pipeline code from below
Make sure you change red highlighted values below as per your settings:
Your docker user id should be updated.
your registry credentials ID from Jenkins from step # 1 should be copied

pipeline {

   tools {

        maven 'Maven3'

    }

    agent any

    environment {

        registry = "account_id.dkr.ecr.us-east-1.amazonaws.com/coachak/my-docker-repo"

    }

   

    stages {

        stage('Cloning Git') {

            steps {

                checkout([$class: 'GitSCM', branches: [[name: '*/main']], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: '', url: 'https://github.com/akannan1087/springboot-app']]])     

            }

        }

      stage ('Build') {

          steps {

            sh 'mvn clean install'           

            }

      }

    // Building Docker images

    stage('Building image') {

      steps{

        script {

          dockerImage = docker.build registry 

          dockerImage.tag("$BUILD_NUMBER")

        }

      }

    }

   

    // Uploading Docker images into AWS ECR

    stage('Pushing to ECR') {

     steps{  

         script {

                sh 'aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin account_id.dkr.ecr.us-east-1.amazonaws.com'

                sh 'docker push account_id.dkr.ecr.us-east-1.amazonaws.com/coachak/my-docker-repo:$BUILD_NUMBER'

         }

        }

      }

    // Avoid latest tag image and pass build ID dynamically from Jenkins pipeline

       stage('K8S Deploy') {

        steps{   

            script {

                withKubeConfig([credentialsId: 'K8S', serverUrl: '']) {

                echo "Current build number is: ${env.BUILD_ID}"

               // Replace the placeholders in the deployment.yaml file 

                sh """ 

                sed -i 's/\${BUILD_NUMBER}/${env.BUILD_ID}/g' eks-deploy-k8s.yaml

                """ 

                sh ('kubectl apply -f  eks-deploy-k8s.yaml -n springboot-app-ns')

                }

            }

        }

       }

    }

}

Step # 5 - Build the pipeline

Once you create the pipeline and changes values per your configuration, click on Build now:

Step # 6 - Verify deployments to K8S

kubectl get deployments -n springboot-app-ns

kubectl get pods -n springboot-app-ns

kubectl get services -n springboot-app-ns

If you see any errors after deploying the pods, you can check the pod logs.

kubectl logs <pod_name> -n spring-app-ns

Steps # 7 - Access SpringBoot App in K8S cluster

Once build is successful, go to browser and enter master or worker node public ip address along with port number mentioned above

http://loadbalancer_ip_address

You should see page like below:

Note:

Make sure you fork my repo https://github.com/akannan1087/springboot-app

and make changes in eks-deploy-k8s.yaml to pull Docker image from your AWS ECR repo.