CloudBees Jenkins Certified Engineer

Jenkins-Logo

Last week I Passed my CloudBees Jenkins Certified Engineer Exam.
It was not an easy exam by any stretch. The few of the questions were very long. I took all the time to finish the exam.

While preparing for the exam, I created a Gitbook. Folks preparing for the exam will find it very useful.

https://www.gitbook.com/book/muralibala/cloudbees-certified-jenkins-engineer-2017/details

Other very useful resources:

jeanne’s experiences with the jenkins certification beta exam


https://memefected.gitbooks.io/studying-for-the-jenkins-engineer-certification/content/
https://ankiweb.net/shared/info/1993369597

Getting started with Jenkins Rest API

Jenkins-Logo

Before we can access Jenkins API, we need grab the API key associated with your account.

  • Login into your Jenkins instance.
  • Click on Manage Jenkins
  • Scroll down to find Manage Users and click on the link.
  • Under your username, click on the gear.
  • Under API token, click on the “Show API Token…”
  • Other way to get to the Configure link is using this URL : http://[Jenkins Instance]:8080/me/configure
  • Copy the API token, you would need this information while making API calls to Jenkins Rest API
  • Now your are ready to test the API and run few command.

Oh! One other thing you need to do before you could curl up. Navigate to Manage Jenkins>>Configure Global Security and uncheck the box “Prevent Cross Site Request Forgery exploits”. I don’t know why but you need to uncheck this before you could run the POST commands below. Some useful information here – http://stackoverflow.com/questions/16738441/how-to-request-for-crumb-issuer-for-jenkins

Lets open the Shell or command prompt. I am using SampleFreeStyleJob for the Jenkins Project.

Trigger a build:
curl -X POST http://[Jenkins_Instance]:8080/job/SampleFreeStyleJob/build --user [USER_NAME]:[API_TOKEN]

Retrieve a project config.xml file
curl http://[Jenkins_Instance]:8080/job/SampleFreeStyleJob/config.xml --user [USER_NAME]:[API_TOKEN]

Disable a project
curl -X POST http://[Jenkins_Instance]:8080/job/SampleFreeStyleJob/disable --user [USER_NAME]:[API_TOKEN]

Enable a project
curl -X POST http://[Jenkins_Instance]:8080/job/SampleFreeStyleJob/enable --user [USER_NAME]:[API_TOKEN]

More Examples

> curl http://[Jenkins_Instance]:8080/api/json?pretty=true --user [USER_NAME]:[API_TOKEN]
> curl -g http://[Jenkins_Instance]:8080/api/json?pretty=true&tree=jobs[name,color] --user [USER_NAME]:[API_TOKEN]
> curl -g http://[Jenkins_Instance]:8080/job/[Job Name]/config.xml -o config.xml [USER_NAME]:[API_TOKEN]

Install Docker on AWS

aws-docker

First thing first. Launch an instance with the Amazon Linux AMI and Connect to your instance.

Install Docker

Update the installed packages and package cache on your instance.

sudo yum update -y

Install Docker and Start the service.

sudo yum install -y docker
sudo service docker start

Add the ec2-user to the docker group so you can execute Docker commands without using sudo.

sudo usermod -a -G docker ec2-user

Install Docker Compose


sudo curl -L "https://github.com/docker/compose/releases/download/1.11.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

Apply executable permissions to the binary

sudo chmod +x /usr/local/bin/docker-compose

Log out and log back in again to pick up the new docker group permissions.


docker-compose --version
docker --version

References:
http://docs.aws.amazon.com/AmazonECS/latest/developerguide/docker-basics.html
https://docs.docker.com/compose/install/

Installing SonarQube on AWS EC2 and integration with Jenkins

Sonar-qube

We are going to install SonarQube on AWS EC2 instance. If you have not already done so, please login to your AWS account and launch an EC2 instance with RHEL OS. For more information on how to launch an instance, please read this post.

Step 1: Create a new RDS database for SonarQube (you may use an existing MySQL instance)

  • Go to RDS Instances
  • Launch a new DB instance
  • Select MySQL
  • Select Dev/Test
  • Click Next
  • Select the instance class (t2.micro should be good for testing)
  • Multi-AZ Deployment – Select No
  • Enter instance name
  • Enter master username and password
  • Click Next
  • Enter Database name – sonar
  • Click on Launch Instance
  • Once the database instance is ready, connect to your instance using MySQL Workbench (or any other client for that matter) and run the following commands

    CREATE USER 'sonar' IDENTIFIED BY 'sonar';
    GRANT ALL ON sonar.* TO 'sonar'@'%' IDENTIFIED BY 'sonar';
    GRANT ALL ON sonar.* TO 'sonar'@'localhost' IDENTIFIED BY 'sonar';
    FLUSH PRIVILEGES;

    Step 2: Now launch and connect to your AWS EC2 instance.

    Update your instance. It’s always a good habit.

    sudo yum update

    Get Sonar repository

    sudo wget -O /etc/yum.repos.d/sonar.repo http://downloads.sourceforge.net/project/sonar-pkg/rpm/sonar.repo

    Install Sonar package

    sudo yum install sonar

    Step 3: Configure SonarQube properties

    sudo vi /opt/sonar/conf/sonar.properties

    Locate following lines in the file and update the username and password:

    sonar.jdbc.username=sonar
    sonar.jdbc.password=sonar

    Scroll down to the MySQL section and update the Database Endpoint.

    #----- MySQL 5.6 or greater
    # Only InnoDB storage engine is supported (not myISAM).
    # Only the bundled driver is supported. It can not be changed.
    sonar.jdbc.url=jdbc:mysql://[REPLACE_WITH_RDS_ENDPOINT]:3306/sonar?useUnicode=true&characterEncoding=utf8&rewriteBatchedStatements=true&useC onfigs=maxPerformance

    Save the file (Esc):wq!

    Step 4: Getting Sonar up & running

    Start Sonar and make sure it starts automatically at system startup.

    sudo service sonar start
    sudo chkconfig sonar on

    Navigate to the SonarQube Url http://[SonarQube-Instance-IP]:9000 (Default port 9000) to confirm it is running.

    If you don’t see the site getting loaded check the logs.

    sudo cat /opt/sonar/logs/sonar.log

    If you don’t see lines at the end like these then you may have to dig through to find the issue.

    2017.02.01 19:48:22 INFO app[][o.s.p.m.Monitor] Process[ce] is up
    2017.02.01 19:48:22 INFO app[][o.s.application.App] SonarQube is up

    Step 5: Integrating SonarQube with Jenkins

    Create Auth Token
    You will need the Token information from SonarQube when you configure Jenkins with SonarQube. So let’s get started.

    • Connect to your SonarQube instance at http://[SonarQube-Instance-IP]:9000/users
    • Click on Create User.
    • Enter Jenkins for Login and Name.
    • Enter Password and click ok.
    • Under Groups, click on the token image. Click on All and check sonar-administrators box.
    • Under Tokens column, click on Update token. token
    • Enter token name and click on Generate.
    • Copy token. Make sure you copy it now, you won’t be able to see it again!

    Configure SonarQube Server

    • Login to your Jenkins instance with administrative user.
    • Navigate to Manage Jenkins >> Configure System
    • Under “SonarQube Servers” section, add the SonarQube Server URL and an Auth Token to access it.

    Now, the last thing we need to do is configure SonarQube Scanner

    • Navigate to Manage Jenkins >> Global Tool Configuration
    • Under “SonarQube Scanner”, click SonarQube Scanner installations
    • Enter Name
    • Check Install Automatically
    • Install from Maven Central. Pick the latest version.
    • Click Save.

    That’s it! Good luck

    Top 5 Jenkins CI Tips you should know.

    Jenkins-Logo

    1. Tell Jenkins to run a specific project on a particular slave node

    Set the “Restrict where this job can be run” check box in your job configuration and specify the name of your slave. If you add more slaves later, you can set labels for each slave and specify those in your job configs.

    2. Restart Jenkins manually

    You can use either of the following commands:

    http://(jenkins_url)/safeRestart – Allows all running jobs to complete. New jobs will remain in the queue to run after the restart is complete.

    http://(jenkins_url)/restart – Forces a restart without waiting for builds to complete.

    You can even use the SafeRestart Plugin. Super useful tool.

    Finally, via CLI:

    • sudo service jenkins start – To start the Jenkins
    • sudo service jenkins stop – To stop the Jenkins
    • sudo service jenkins restart – To restart the Jenkins
    • sudo service jenkins status – To know the status of Jenkins

    3. Locked out of Jenkins

    If you don’t have a lot of other configuration that you’d like to save, you can just delete %JENKINS_HOME%/config.xml and restart Jenkins to disable security. Otherwise, edit config.xml and set the values inside the useSecurity tags to false, then restart Jenkins.

    I had this exact issue today on my Windows Jenkins server. Just removing the XML file and restarting did not work for me. So I had to:

    • Stop the service.
    • Check Task Manager to ensure the process is gone.
    • Edit the Config.XML file and change the useSecurity false (or you could delete the config.xml file).
    • Start the service again.

    4. Default shell environment valuables
    To get the list of all the variables that are available to shell scripts:

    http://(jenkins_url)/env-vars.html

    5. Jenkins Directory Structure

    jenkins-directory_structure

    Ref: https://wiki.jenkins-ci.org/display/JENKINS/Administering+Jenkins

    Installing Jenkins on AWS EC2

    Jenkins-Logo

    Jenkins is an open source automation server software that allows continuous integration. Read more about it here.

    We are going to install Jenkins on AWS EC2 instance. If you have not already done so, please login into your AWS account and launch an EC2 instance with RHEL OS. For more information on how to launch an instance, please read this post.

    1) Install the latest stable packages, then reboot.

    sudo yum update -y

    2) Before you can install Jenkins, you need to setup a JVM.

    sudo yum install java-1.8.0-openjdk.x86_64

    3) After the installation, you can confirm it by running the following command:

    java -version

    Jenkins require Java 1.6 or more than. If Java version is less than 1.6 than we have to upgrade the Java. Follow the instruction here on installing Java

    4) Install wget

    sudo yum install wget -y

    5) Install Jenkins

    sudo wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkins-ci.org/redhat-stable/jenkins.repo
    sudo rpm --import http://pkg.jenkins-ci.org/redhat-stable/jenkins-ci.org.key
    sudo yum install jenkins -y

    6) Start the Jenkins service and set it to run at boot time:

    sudo systemctl start jenkins.service
    sudo systemctl enable jenkins.service

    6) Install firewalld (if not already installed)

    sudo yum install firewalld -y

    8) After install unmask, enable and start the firewall with below commands

    sudo systemctl unmask firewalld
    sudo systemctl enable firewalld
    sudo systemctl start firewalld

    9) In order to allow visitors access to Jenkins, you need to allow inbound traffic on port 8080. You can either open port 8080 in AWS Console or run the following command:

    sudo firewall-cmd --zone=public --permanent --add-port=8080/tcp
    sudo firewall-cmd --reload
    sudo firewall-cmd --list-all

    10) Finally, visit the following address from your web browser to confirm your installation:

    http://[your-jenkins-server-IP:8080]

    or by running the following command

    sudo netstat -ntulp | grep 8080

    If the site does not load, please check the Jenkins log file for more information

    Linux
    By default, logs are here /var/log/jenkins/jenkins.log (unless customized in /etc/default/jenkins (for *.deb) or via /etc/sysconfig/jenkins (for */rpm)

    Windows
    By default, logs are here %JENKINS_HOME%/jenkins.out and %JENKINS_HOME%/jenkins.err (unless customized in %JENKINS_HOME%/jenkins.xml)

    More information can be found here.

    Moving forward, you will need the Jenkins Initial Password to get started:

    sudo cat /var/lib/jenkins/secrets/initialAdminPassword

    Oh! And one last thing – If you wish to change Jenkins HTTP port number from the default port 8080, you need to change the file /etc/sysconfig/jenkins

    sudo vi /etc/sysconfig/jenkins

    Good luck!

    Getting Started with OpenShift Origin on Windows

    OpenShift
    First thing first. Install Docker.

    Once Docker is running, add an insecure registry of 172.30.0.0/16:

    • 1) Right click on the Docker icon in the notification area and select Settings…
    • 2) Click on Docker Daemon in the settings dialog
    • 3) Edit the Docker daemon configuration by adding “172.30.0.0/16” to the “insecure-registries”: setting { “registry-mirrors”: [], “insecure-registries”: [ “172.30.0.0/16” ] }
    • 4) Click on Apply and Docker will restart.

    docker-daemon-openshift

    Once you have successfully done that it is time install the OpenShift client.

    • Download OpenShift client.
    • Extract the content to a folder and place it in your path.
    • That’s it. You are all set.

    Now let’s get started hacking your first OpenShift project.

    Open a Command window as Administrator and run:
    oc cluster up

    You will see the following if all went ok.
    OpenShift-getting_started

    Here are some handy OpenShift CLI commands.

    This in no way a complete list of all the OC command. For complete list and OpenShift reference, please click here and here.

    Cluster Up
    oc cluster up

    Cluster Down
    oc cluster down

    Return information about the current session
    oc whoami

    Display client and server versions
    oc version

    End the current server session
    oc logout

    Start a new session
    oc login -u system:admin

    Create new Project
    oc new-project sample-demo \
    --description="This is an example sample project" \
    --display-name="Sample OpenShift Demo"

    Delete a Project
    oc delete project sample-demo

    Switch to a project
    oc project

    Show an overview of the current project
    oc status

    Display existing projects
    oc projects

    View all projects you have access to
    oc get projects

    See a list of all services in the current project
    oc get svc

    Describe a deployment configuration in detail
    oc describe dc nodeapp-1-aqtx7

    Create an application using a remote Git repository
    Before proceeding, make sure have Git installed on your workstation.
    oc new-app https://github.com/your_user/your_git_repo.git

    Specify a Git branch to use by appending #dev to the end of the URL
    oc new-app https://github.com/your_user/your_git_repo.git#dev

    Add labels to the created objects.
    oc new-app https://github.com/your_user/your_git_repo.git -l name=hello-world

    Create an application from the DockerHub MySQL image
    oc new-app mysql

    Create multiple applications from a source repository and DockerHub image
    oc new-app https://github.com/your_user/your_git_repo.git mysql

    Deploying multiple images together in a single pod
    oc new-app nginx+mysql

    And finally, help about any command
    oc help

    Docker Cheat Sheet

    Here is a list of some basic Docker Commands to help you get started with Docker. This helped me a lot and still does from time to time. Note, this not meant to be a complete Docker reference. For complete list, please visit docs.docker.com

    Containers

    Pull a base image.
    docker pull ubuntu

    Start a container
    docker run --name docker-nginx -p 80:80 -d nginx
    docker run --name docker-nginx -p 80:80 -d -v /app/html:/usr/share/nginx/html nginx

    For list of running containers
    docker ps

    For list of all containers
    docker ps - a

    Restart a Container.
    docker restart container_id

    Show all running process in a Container.
    docker top container_id

    Stop container
    docker stop container_id

    Remove container
    docker rm container_id>

    Kill containers and remove them
    docker stop $(docker ps -aq)
    docker rm $(docker ps -aq)
    docker rm $(docker kill $(docker ps -aq))

    Clean up old containers that are weeks old (via Stackoverflow)
    docker ps --filter "status=exited" | grep 'weeks ago' | awk '{print $1}' | xargs --no-run-if-empty docker rm

    Print the last 100 lines of a container’s logs
    docker logs --tail 100 container_id

    Create a new bash process inside the container and connect it to the terminal
    docker exec -it container_id bash

    Docker Container link
    Docker run -d -p 5000:5000 --link redis dockerapp:v0.3

    Docker Images

    Remove all images
    docker rmi $(docker images -q) -f
    docker rmi $(docker images -qf "dangling=true")

    Remove all images except “my-image”
    You could use grep to remove all except my-image and ubuntu
    docker rmi $(docker images | grep -v ‘ubuntu\|my-image’ | awk {‘print $3’})

    Save a running container as an image
    docker commit -m "commit message" -a "author" container_id username/image_name:tag

    Other Docker Commands

    Run commands inside an existing container
    docker exec -it [containerID] bash

    Push to Images to Docker
    docker tag 2f98ca2e63ab docker-nginx:1.01
    docker login --username muralibala
    docker push docker-nginx:1.01

    To find Ip address
    docker machine ls

    List the networks
    docker network ls

    List the volumes
    docker volume ls

    Amazon Web Service (AWS) Service Limits that you should know

    While preparing for my AWS Developer Certification, I prepared this AWS Service limits (the most important ones) for my own benefit and now would like to share this with you. Please note that this is in no way an exhaustive list of all the service limits. For a complete list, please visit Amazon Web Service (AWS) Service Limits site.

    S3

    Max buckets per Account100
    Max object size5TB
    Multi-part upload required for Files of size>5GB
    Bucket OwnershipNon-transferable

    SQS

    Max Message Size256KB
    Min Message Retention Period1 minute
    Max Message Retention Period14 Days
    Default visibility time out30 Secs
    Min visibility time out0 seconds
    Max visibility time out12 Hours
    Max Inflight Messages12000
    Min Receive Message Time out0 seconds
    Max Receive Message Time out20 seconds

    DynamoDb

    Max tables per region256
    Max Local Secondary Index5
    Max Global Secondary Index5
    Min Partition Key Length1 byte
    Max Partition Key Length2048 bytes
    Max Sort Key Length1 byte
    Min Sort Key Length1024 bytes
    Max Item Size (attribute name and value)400KB

    SWF

    Max open workflow executions100,000 per domain
    Max workflow execution time1 year
    Max workflow execution history size 25,000 events
    Max child workflow executions1,000 per workflow execution
    Workflow execution idle time limit1 year
    Workflow retention time limit90 days
    Maximum pollers per task list100 per host, per tasklist
    Max task execution time1 year
    Max time SWF will keep a task in the queue1 year
    Max open activity tasks1,000 per workflow execution
    Max open timers1,000 per workflow execution

    VPC

    Max VPC per region5
    Max Subnets per VPC200
    Max Customer gateways per region50
    Max Internet gateways per region5
    Max Elastic IP per region per account5
    Max Connections per region50
    Max Route tables per region200
    Max Security groups per region500

    CloudFormation

    Max Stacks per account20
    Max Parameters per template60
    Max Mappings per template100
    Max Resources per template200
    Max Outputs per template60

    RDS

    Max DB instances per account40
    Max Reserved instances per account40
    Max total storage for all DB instances per account100TB
    Max Read replicas per master5