jenkins+gitlab+harbor部署CICD-526互联

jenkins相关操作

docker部署jenkins

docker run --name jenkins -d -p 8080:8080 -p 50000:50000 --privileged=true --restart=always -v /jenkins/home:/var/jenkins_home -v /var/run/docker.sock:/var/run/docker.sock -v /usr/bin/docker:/usr/bin/docker jenkins/jenkins:2.387.2-lts

打开jenkins访问页面需要下载的插件

Publish Over SSH
GitLab Plugin
Git Parameter Plug-In

配置k8s可以执行yaml的主机

配置webhook，作用是一旦gitlab仓库代码被提交立马给通知jenkins发送CICD的请求

1.关闭对于project的认证

下载安装maven并放到jenkins所映射的宿主机目录下，然后制定maven home

下载jdk并解压到jenkins所映射的宿主机目录下，然后制定jdk的home目录

创建jenkins pipline流水线项目，点击build when....目的是一旦发现gitlab有代码提交jenkins就执行pipline任务

http://xxxx需要记住，一会要在gitlab中配置

制定gitlab仓库的url,并指定分支，然后pipline脚本指定为jenkinsfile

gitlab相关操作

在gtilab上配置webhook，配置好之后点击test测试一下返回200代表ok

gitlab上这2个地方打钩，否则webhook会失败

在gitlab仓库里面除了有java代码以外，还需要有

1.dockerfiile

2.部署pod的yaml文件

3.Jenkinsfile，注意开头字符必须是大写，否则检测不到Jenkins文件

Jenkinsfile文件内容如下：

需要注意的是如何用publish over ssh插件把部署文件传过去之后，要cd的话必须制定绝对路径

pipeline {
    agent any
    environment {
        image="core.harbor.domain:80/myharbor/myjava:latest"
        harbor="core.harbor.domain:80"
    }
    stages {
        stage('拉去git代码') {
            steps {
                checkout scmGit(branches: [[name: '*/main']], extensions: [], userRemoteConfigs: [[url: 'http://192.168.1.101/root/myjava.git']])
            }
        }
        stage('maven构建项目') {
            steps {
                sh '/var/jenkins_home/maven3.9.1/bin/mvn clean package -DskipTests'
            }
        }
        stage('dockerfile制作镜像') {
            steps {
                sh '''
                docker login -uadmin -pHarbor12345 $harbor
                cp target/*.jar ./docker
                cd docker
                docker build -t $image ./
                '''
            }
        }
        stage('推送镜像到harbor') {
            steps {
                sh 'docker push $image'
            }
        }
        stage('通过publish over ssh通知目标服务器拉取镜像并部署') {
            steps {
                sshPublisher(publishers: [sshPublisherDesc(configName: 'test01', transfers: [sshTransfer(cleanRemote: false, excludes: '', execCommand: '''cd /jenkins_data/mypipline01 && kubectl apply -f a.yaml && kubectl rollout restart deploy myjava
''', execTimeout: 120000, flatten: false, makeEmptyDirs: false, noDefaultExcludes: false, patternSeparator: '[, ]+', remoteDirectory: 'mypipline01', remoteDirectorySDF: false, removePrefix: '', sourceFiles: '**/*.yaml')], usePromotionTimestamp: false, useWorkspaceInPromotion: false, verbose: false)])
            }
        }

    }
}

yaml部署文件内容如下：

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: myjava
  name: myjava
  namespace: default
spec:
  replicas: 3
  selector:
    matchLabels:
      app: myjava
  template:
    metadata:
      labels:
        app: myjava
    spec:
      imagePullSecrets:
        - name: harbor
      containers:
      - image: core.harbor.domain:80/myharbor/myjava:latest
        imagePullPolicy: Always
        name: myjavaweb
        ports:
        - containerPort: 8080
          protocol: TCP
          name: http

dockerfile内容如下：

FROM java:openjdk-8u111
WORKDIR /usr/local
COPY demo2-0.0.1-SNAPSHOT.jar ./
CMD java -jar demo2-0.0.1-SNAPSHOT.jar

部署ingress-nginx（我的k8s版本是1.22）

kubectl create -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.1.0/deploy/static/provider/clouddeploy.yaml

查看ingress controller是否正常

[root@ceph1 ~]# kubectl get pods -n ingress-nginx
NAME                                      READY   STATUS      RESTARTS      AGE
ingress-nginx-admission-create--1-vmgcm   0/1     Completed   0             8d
ingress-nginx-admission-patch--1-7nbcr    0/1     Completed   2             8d
ingress-nginx-controller-54bfb9bb-f4wf6   1/1     Running     8 (19h ago)   8d

[root@ceph1 ~]# kubectl get svc -n ingress-nginx
NAME                                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx-controller             NodePort    10.107.14.210   <none>        80:31485/TCP,443:30185/TCP   8d
ingress-nginx-controller-admission   ClusterIP   10.99.113.251   <none>        443/TCP                      8d

然后部署deploy和svc

[root@master myself]# cat mypod.yaml 
apiVersion: v1
kind: Service
metadata:
  name: service-nginx
  namespace: default
spec:
  selector:
    app: mynginx
  ports:
  - name: http
    port: 80
    targetPort: 80

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mydepoy
  namespace: default
spec:
  replicas: 5
  selector:
    matchLabels:
      app: mynginx
  template:
    metadata:
      labels:
        app: mynginx
    spec:
      containers:
      - name: mycontainer
        image: lizhaoqwe/nginx:v1
        imagePullPolicy: IfNotPresent
        ports:
        - name: nginx 
          containerPort: 80

View Code

编写ingress规则

[root@ceph1 ~]# cat ingress.yaml 
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: service-nginx-ingress
  namespace: default
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /    
spec:
  ingressClassName: nginx    #制定ingressclassname，可以用kubectl get ingressclass查看
  rules:
  - host: mynginx.fengzi.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service: 
            name: service-nginx  #制定svc的名称
            port: 
              number: 80

查看ingress

[root@ceph1 ~]# kubectl get ingress 
NAME                    CLASS   HOSTS                  ADDRESS         PORTS   AGE
harbor-ingress          nginx   core.harbor.domain     10.107.14.210   80      21h
harbor-ingress-notary   nginx   notary.harbor.domain   10.107.14.210   80      21h
service-nginx-ingress   nginx   mynginx.fengzi.com     10.107.14.210   80      8d

有ingress之后再要访问服务就需要域名了，所以，在找一台机器搭建nginx，配置文件内容如下：

harbor的配置文件内容如下：

[root@hadoop1 conf.d]# cat core.harbor.domain.conf 
upstream myk8s {
    server 192.168.1.101:31485 weight=1;
    server 192.168.1.102:31485 weight=1;
    server 192.168.1.103:31485 weight=1;
}

server {
    listen       80;
    server_name  core.harbor.domain;
    location / {
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $remote_addr;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_connect_timeout 5s;
        client_max_body_size 0;
        proxy_pass http://myk8s;
    }    

}

View Code

gitlab的配置内容如下：

[root@hadoop1 conf.d]# cat mygitlab.fengzi.com.conf 
upstream mygitlab {
    server 192.168.1.101:80 weight=1;
}

server {
    listen       80;
    server_name  mygitlab.fengzi.com;
    location / {
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $remote_addr;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_connect_timeout 5s;
        client_max_body_size 0;
        proxy_pass http://mygitlab;
    }    

}

View Code

部署gitlab

curl -fsSL https://packages.gitlab.cn/repository/raw/scripts/setup.sh | /bin/bash

EXTERNAL_URL="http://0.0.0.0" yum install -y gitlab-jh

部署harbor

helm add repo harbor https://helm.goharbor.io

helm pull harbor/harbor

cd harbor
vim values.yaml

修改values.yaml文件

1.如果不需要https访问，需要改成false

2.如果没有pvc，修改为false

3.配置harbor域名

一般externalURL的域名与上面core的一致

4.制定classname，也就是kubectl get ingressclass的name

5.部署harbor

helm install harbor ../harbor

全都完成以后推送代码后，最终的效果如下：

jenkins gitlab harbor cicd

jenkins环境gitlab harbor

tool-gitlab-cicd

gitlab-cicd

gitlab-cicd gitlab cicd

tool-gitlab-cicd gitlab tool cicd

gitlab cicd

流水线流水gitlab cicd

流程jenkins docker cicd

jenkins gitlab