k8s-gitlab-runner实现go项目的自动化发布

gitlab构建CI/CD

准备

docker部署gitlab

通过docker-compose启动gitlab

version: '3' 
services: 
  gitlab: 
    image: 'gitlab/gitlab-ce:latest' 
    restart: always 
    hostname: '1.1.1.1' 
    environment: 
      TZ: 'Asia/Shanghai' 
      GITLAB_OMNIBUS_CONFIG: | 
        external_url 'http://1.1.1.1:9001' 
        gitlab_rails['gitlab_shell_ssh_port'] = 1022 
        unicorn['port'] = 8888 
        nginx['listen_port'] = 9001 
    ports: 
      - '9001:9001' 
      - '443:443' 
      - '1022:22' 
    volumes: 
      - ./config:/etc/gitlab 
      - ./data:/var/opt/gitlab 
      - ./losg:/var/log/gitlab 

使用二进制部署gitlab-runner

可参考官方的安装方式Install GitLab Runner manually on GNU/Linux

gitlab-runner注册

安装完成之后使用注册命令注册

$ gitlab-runner register 

然后会提示输入gitlab的地址以及token信息

地址信息和token我们在下面可以看到

根据提示输入信息,需要注意的是里面的tags就是我们编写.gitlab-ci.yml对应填写的tag

之后在gitlab的runner中就可以看到我们注册的gitlab-runner了

配置Variables

在gitlab中可以配置我们gitlab-runner需要的变量，比如我们的docker-hub的密码，gitlab的账号密码等信息

简单先来个测试

先来个简单的gitlab-ci.yml测试下

stages: 
  - test 
  - build 
 
variables: 
  GOPROXY: https://goproxy.cn 
 
lint: 
  stage: test 
  script: 
    - echo "hello world lint" 
  only: 
    - branches 
  tags: 
    - golang-runner 
 
test: 
  stage: test 
  script: 
    - echo "hello world test" 
  only: 
    - branches 
  cache: 
    key: "bazel" 
    paths: 
      - .cache 
  tags: 
    - golang-runner 

开始构建

通过helm和bazel实现在gitlab-runner中k8s应用的自动编译，发布。

镜像推送到docker-hub中,gitlab-runner中的helm需要配置好，这里我是用了helm默认初始化的charts结构来发布应用

gitlab-ci.yml

stages: 
  - test 
  - build 
  - deploy 
 
variables: 
  GOPROXY: https://goproxy.cn 
 
lint: 
  stage: test 
  script: 
    - export GO_PROJECT_PATH="/home/gitlab-runner/goWork/src" 
    - mkdir -p $GO_PROJECT_PATH 
    - ln -s $(pwd) $GO_PROJECT_PATH/test 
    - cd $GO_PROJECT_PATH/test 
    - bash build/lint.sh 
  only: 
    - branches 
  tags: 
    - golang-runner 
 
test: 
  stage: test 
  script: 
    - go mod vendor 
    - bash build/bazel-test.sh 
  only: 
    - branches 
  cache: 
    key: "bazel" 
    paths: 
      - .cache 
  tags: 
    - golang-runner 
 
build: 
  stage: build 
  before_script: 
    - url_host=`git remote get-url origin | sed -e "s/http:\/\/gitlab-ci-token:.*@//g"` 
    - git remote set-url origin "http://$GIT_ACCESS_USER:$GIT_ACCESS_PASSWORD@${url_host}" 
    - git config user.name $GIT_ACCESS_USER 
    - git config user.email $GIT_ACCESS_EMAIL 
    - git fetch --tags --force 
  script: 
    - docker login -u $DOCKER_ACCESS_USER -p $DOCKER_ACCESS_PASSWORD 
    - go mod vendor 
    - bash build/bazel-build.sh 
  only: 
    - master 
  cache: 
    key: "bazel" 
    paths: 
      - .cache 
  tags: 
    - golang-runner 
 
 
 
include: '/.gitlab/deploy.yaml' 

镜像打包

test::util:build_docker_images() { 
  local docker_registry=$1 
  local docker_tag=$2 
  local base_image="alpine:3.7" 
 
  query=$(test::util::find_changes) 
 
  if [ "$query" == "" ]; then 
    test::util::log "no change and exit..." 
    exit 0 
  fi 
 
 
  for b in ${query}; do 
    b=${b//\/\/src/"/src"} 
 
    if [[ $b == *test* ]] 
        then 
        continue 
    fi 
 
    local binary_file_path=$(test::util::find_binary "$b") 
    local binary_name=$(test::util::get_binary_name "$b") 
    local docker_build_path="dockerbuild/${binary_name}" 
    local docker_file_path="${docker_build_path}/Dockerfile" 
    local docker_image_tag="${docker_registry}/${binary_name}:${docker_tag}" 
 
 
    test::util::log "Starting docker build for image: ${binary_name}" 
    ( 
      rm -rf "${docker_build_path}" 
      mkdir -p "${docker_build_path}" 
      cp "${binary_file_path}" "${docker_build_path}/${binary_name}" 
      cat <<EOF >"${docker_file_path}" 
FROM ${base_image} 
COPY ${binary_name} /usr/local/bin/${binary_name} 
RUN mkdir /lib64 && ln -s /lib/libc.musl-x86_64.so.1 /lib64/ld-linux-x86-64.so.2 
RUN sed -i 's/dl-cdn.alpinelinux.org/mirrors.ustc.edu.cn/g' /etc/apk/repositories \ 
  && apk update --no-cache \ 
ENTRYPOINT ["/usr/local/bin/${binary_name}"] 
EOF 
      docker build -q -t "${docker_image_tag}" "${docker_build_path}" 
      docker push ${docker_image_tag} 
    ) 
 
 
  cat <<EOF >>".gitlab/deploy.yaml" 
${binary_name}: 
  stage: deploy 
  script: 
    - bash build/deploy.sh ${docker_registry} ${binary_name} ${docker_tag} 
  only: 
    - tags 
  when: manual 
  environment: 
    name: test 
  tags: 
    - golang-runner 
EOF 
 
done 
 
  test::util::log "Docker builds done" 
} 

整体的处理思路是

1、通过bazel构建go项目。

2、构建的时候找到有改动的项目，编译，打包镜像，生成deploy脚本。

3、打上tag，推到gitlab中。

4、最后通过手动触发项目的deploy，通过helm发布对应的项目到k8s中。

项目的地址gitlab-runner构建go项目

最后附上deploy的截图

对应的项目已经部署上去了

$ kubectl get pods -n test 
NAME                               READY   STATUS    RESTARTS   AGE 
demo1-main-test-654fbd4df6-vlhwr   1/1     Running   0          27m 
demo2-main-test-58ccfd4f8-hjcdf    1/1     Running   0          25m 
demo3-main-test-6897ff9496-mhm5n   1/1     Running   0          27m 

遇到的报错

go: writing go.mod cache: mkdir /home/goWork: permission denied

给用户gitlab-runner添加最用目录的执行权限

sudo chown -R $(whoami):gitlab-runner /Users/zhushuyan/go/pkg && sudo chmod -R g+rwx /Users/zhushuyan/go/pkg 

需要预备的知识点

这里主要使用到了helm和bazel

helm使用

bazel使用

本文参考链接：https://www.cnblogs.com/ricklz/p/14920184.html