Методические материалы Лохтурова Вячеслава

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Инструмент GitLab ======

  * [[https://ru.wikipedia.org/wiki/GitLab - Википедия]]

  * [[https://youtu.be/n_21ya2MoKg|Youtube. RomNero. GitLab. Devops система]]
  * [[https://qna.habr.com/q/498925|GitLab бесплатен? gitlab-ee vs gitlab-ce]]

===== Установка =====

  * RAM от 4Gb



==== Установка из репозитория ====

  * [[https://about.gitlab.com/install/|Install self-managed GitLab]]
  * Доступно из РФ: [[https://packages.gitlab.com/gitlab/gitlab-ce]]

<code>
server# apt-get install -y curl ca-certificates perl

server# curl https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.deb.sh | bash

server# time EXTERNAL_URL="http://$(hostname)" apt-get install gitlab-ce
...
real    122m54.883s  !!! Загрузка может прерываться, надо повторять команду !!!
..
</code>

==== Установка через docker-compose ====

  * [[https://docs.gitlab.com/ee/install/docker.html#install-gitlab-using-docker-compose|Install GitLab using Docker Compose]]
  * [[https://hub.docker.com/r/gitlab/gitlab-ce/tags/|gitlab/gitlab-ce tags (versions)]]

  * [[Технология Docker]]
  * [[Технология Docker#docker-compose]]
  * [[#Включение TLS]]

<code>
# cat docker-compose.yml
</code><code>
services:
  gitlab:
#    image: 'gitlab/gitlab-ce:latest'
    image: 'gitlab/gitlab-ce:18.6.2-ce.0'
#    userns_mode: 'host'
#    privileged: true
    restart: always
    hostname: 'server.corpX.un'
    environment:
      GITLAB_ROOT_PASSWORD: "strongpassword"
      GITLAB_OMNIBUS_CONFIG: |
        prometheus_monitoring['enable'] = false
        gitlab_rails['registry_enabled'] = true
        gitlab_rails['registry_host'] = "server.corpX.un"
        external_url 'https://server.corpX.un'
        registry_external_url 'https://server.corpX.un:5000'
        gitlab_rails['registry_port'] = "5050"
        registry['registry_http_addr'] = "0.0.0.0:5050"
        #gitlab_rails['env'] = { 'SKIP_RAISE_ON_INITIALIZE_CONNECTIONS' => 'true' }
    ports:
      - '443:443'
      - '2222:22'
      - '5000:5000'
    volumes:
      - '/etc/gitlab:/etc/gitlab'
      - vol1:/var/opt/gitlab
    shm_size: '256m'
volumes:
  vol1:
</code><code>
# docker-compose up -d

# docker-compose logs -f

### docker-compose stop
</code>
==== Установка через Ansible Role ====

  * [[https://galaxy.ansible.com/ui/repo/published/hifis/toolkit/content/role/gitlab/]]



===== Подключение =====

==== Подключение к Web интерфейсу =====

  * http://server.corpX.un/

==== Подключение через API ====

=== проект ===

  * Токен доступа: Settings -> Access Tokens ([[https://docs.gitlab.com/ee/user/project/settings/project_access_tokens.html|Project access tokens]]), в примере достаточно role: Reporter, Scopes: api
  * Номер проекта: Settings -> General ([[https://stackoverflow.com/questions/39559689/where-do-i-find-the-project-id-for-the-gitlab-api|Where do I find the project ID for the GitLab API?]])
  * [[https://stackoverflow.com/questions/56943327/how-to-download-a-single-file-from-gitlab|How to download a single file from GitLab?]]

<code>
root@node1,2,3:~# curl "http://server.corpX.un/api/v4/projects/2/repository/files/docker-compose.yml/raw" | tee docker-compose.yml

  или, для НЕ публичных проектов
root@node1,2,3:~# curl --header "PRIVATE-TOKEN: NNNNNNNNNNNNNNNNNNNNN" "http://server.corpX.un/api/v4/projects/4/repository/files/docker-compose.yml/raw?ref=master" | tee docker-compose.yml
</code>

  * [[Сервис Ansible#ansible-pull]]

<code>
client1:~/ansible-pull-gpo# cat readme.md
</code><code>
sudo -i

export BR=main; bash <(curl -s http://gate.corpX.un/api/v4/projects/1/repository/files/start.sh/raw?ref=$BR)
</code>

=== REST API интерфейс ===

  * Search or go to... -> Profile -> Personal access tokens -> Add new token -> api -> Create token

== Добавление пользователя ==

<code>
client1$ cat adduser.sh
</code><code>
GITLAB_URL="https://gitlab.corpX.un"
PRIVATE_TOKEN=NNNNNNNNNNNN

USER_EMAIL="user1@corpX.un"
USER_PASSWORD="Pa\$\$w0rd"
USER_USERNAME="user1"
USER_NAME="Your Name"

curl --header "PRIVATE-TOKEN: $PRIVATE_TOKEN" \
     --header "Content-Type: application/json" \
     --request POST \
     --data "{ \"email\": \"$USER_EMAIL\", \"password\": \"$USER_PASSWORD\", \"username\": \"$USER_USERNAME\", \"name\": \"$USER_NAME\", \"admin\": true }" \
     "$GITLAB_URL/api/v4/users" -k
</code>
===== Настройка =====

==== Файл конфигурации ====
<code>
# cat /etc/gitlab/gitlab.rb
</code><code>
...
external_url 'http://server.corpX.un'
...
</code>

==== Проверка конфигурации и перезапуск ====

<code>
### docker exec -it root_web_1 bash
### kubectl -n my-gitlab-ns exec -ti pods/gitlab-<TAB> -- bash

# ###gitlab-ctl status
# ###gitlab-rake gitlab:check
# ###gitlab-ctl tail sidekiq

# gitlab-ctl show-config

# time gitlab-ctl reconfigure
...
real    2m34.726s
...
</code>
==== Настройка отправки почты ====

  * [[Утилита msmtp#sendmail без mta]]
<code>
root@gitlab:/# gitlab-rails console
irb(main):001> Notify.test_email('student@corp15.un', 'Message Subject', 'Message Body').deliver_now
</code>
==== GitLab Docker Registry ====

  * [[https://docs.gitlab.com/ee/administration/packages/container_registry.html|The Container Registry is automatically enabled and available on your GitLab domain, port 5050 if you’re using the built-in Let’s Encrypt integration]]
  * [[https://sysadmintalks.ru/insecure-gitlab-registry/|Настройка работы Gitlab с registry без ssl - Sysadmin]]
<code>
# cat /etc/gitlab/gitlab.rb
</code><code>
...
registry_external_url 'http://server.corpX.un'
gitlab_rails['registry_enabled'] = true
gitlab_rails['registry_host'] = "server.corpX.un"
gitlab_rails['registry_port'] = "5000"
registry['registry_http_addr'] = "server.corpX.un:5000"
...
</code>

  * [[#Проверка конфигурации и перезапуск]]
==== GitLab Grafana ====
<code>
# cat /etc/gitlab/gitlab.rb
</code><code>
...
grafana['http_addr'] = '0.0.0.0'
...
</code>

  * [[#Проверка конфигурации и перезапуск]]
==== GitLab Prometheus ====

<code>
# cat /etc/gitlab/gitlab.rb
</code><code>
...
prometheus_monitoring['enable'] = false
...
</code>

  * [[#Проверка конфигурации и перезапуск]]

<code>
# time rm -rf /var/opt/gitlab/prometheus/data/*
</code>

==== Включение TLS ====

  * [[https://docs.gitlab.com/omnibus/settings/ssl.html#configure-https-manually|Configure HTTPS manually]]
  * [[https://www.techbeatly.com/configure-custom-ssl-to-secure-gitlab-server/|Configure Custom SSL to Secure GitLab Server]]

<code>
mkdir -p /etc/gitlab/ssl/

cp wild.crt -v /etc/gitlab/ssl/$(hostname).crt
cp wild.key -v /etc/gitlab/ssl/$(hostname).key

# cat /etc/gitlab/gitlab.rb
</code><code>
...
external_url 'https://server.corpX.un'
...
# nginx['ssl_certificate'] = "/etc/gitlab/ssl/#{node['fqdn']}.crt"
# nginx['ssl_certificate_key'] = "/etc/gitlab/ssl/#{node['fqdn']}.key"
...
letsencrypt['enable'] = false
...
</code>

  * [[#Проверка конфигурации и перезапуск]]

==== Управление пользователями ====

=== Внутренние пользователи ===

  * Username - login, Name - ФИО

<code>
# cat /etc/gitlab/initial_root_password
</code>

  * [[https://stackoverflow.com/questions/60062065/gitlab-initial-root-password|gitlab initial root password reset]]

<code>
# gitlab-rake "gitlab:password:reset[root]"
</code>

=== Использование LDAP ===

  * [[https://docs.gitlab.com/ee/administration/auth/ldap/index.html|Integrate LDAP with GitLab]]
  * [[Установка и настройка OpenLDAP]]
  * [[Хранение учетных записей UNIX в LDAP]] !!! с атрибутом почты и паролем
  * [[Решение FreeIPA]]

<code>
# cat /etc/gitlab/gitlab.rb
</code><code>
...
gitlab_rails['ldap_enabled'] = true

gitlab_rails['ldap_servers'] = YAML.load <<-'EOS'
  main:
    label: 'LDAP'
    host: 'server.corpX.un'
#    host: 'server2.corpX.un'
#    host: '192.168.X.10'
    port: 389
#    uid: 'uid'
    uid: 'sAMAccountName'
#    bind_dn: 'cn=admin,dc=corpX,dc=un'
#    password: 'secret'
#    bind_dn: 'uid=admin,cn=users,cn=accounts,dc=corpX,dc=un'
#    password: 'strongpassword'
    bind_dn: 'cn=Administrator,cn=Users,dc=corpX,dc=un'
    password: 'Pa$$w0rd'
    encryption: 'plain'
#    active_directory: false
    active_directory: true
    base: 'dc=corpX,dc=un'
EOS
...
</code>

  * [[#Проверка конфигурации и перезапуск]]
===== GitLab Runner =====

  * [[https://stackoverflow.com/questions/32933174/use-gitlab-ci-to-run-tests-locally|Use GitLab CI to run tests locally?]]
==== Установка из пакета ====

  * [[https://docs.gitlab.com/runner/install/linux-manually.html|Install GitLab Runner manually on GNU/Linux]]
  * [[https://val.bmstu.ru/unix/Git/gitlab-runner_amd64.deb]] (17.3.1-1) 

<code>
# wget http://gate.isp.un/unix/Git/gitlab-runner_amd64.deb               # Version: 17.3.1-1

# dpkg -i gitlab-runner_amd64.deb

## Может занять 2 часа ## 
# ###curl -LJO "https://s3.dualstack.us-east-1.amazonaws.com/gitlab-runner-downloads/v18.6.2/deb/gitlab-runner-helper-images.deb"
# ###curl -LJO "https://s3.dualstack.us-east-1.amazonaws.com/gitlab-runner-downloads/v18.6.2/deb/gitlab-runner_amd64.deb"
# ###dpkg -i gitlab-runner-helper-images.deb gitlab-runner_amd64.deb
</code>
==== Регистрация ====
<code>
# gitlab-runner register --help

# export CI_SERVER_URL=http://server.corpX.un

# gitlab-runner register
...
Enter the GitLab instance URL: http://server.corpX.un
Enter the registration token: ...
...
Enter tags for the runner: dhcptest, dhcpdeploy
  или
Enter tags for the runner: openvpn1deploy   или     ansible
...
Enter an executor: shell
...
</code>

или

<code>
# gitlab-runner register -n --executor "shell" -u http://server.corpX.un -r "NNNNNNNNNNNNNNNNNNNNNNNNNNNN"
</code>

или по инструкции в "New instance runner"
<code>
# gitlab-runner register -n --executor "shell" -u http://server.corpX.un -t "NNNNNNNNNNNNNNNNNNNNNNNNNNNN"
</code>
Перезапускать не нужно

<code>
# gitlab-runner verify
</code>

<code>
# cat /etc/gitlab-runner/config.toml
log_level = "debug"
...
</code><code>
# systemctl restart gitlab-runner
</code>

==== Установка в виде контейнера ====

  * [[https://habr.com/ru/companies/cloud4y/articles/710782/|Использование Docker in Docker в GitLab]]

<code>
gate:~### docker stop gitlab-runner; docker rm gitlab-runner
gate:~### rm /srv/gitlab-runner/config/config.toml

gate:~# docker run -d --name gitlab-runner --restart always \
  -v /srv/gitlab-runner/config:/etc/gitlab-runner \
  -v /var/run/docker.sock:/var/run/docker.sock \
  gitlab/gitlab-runner:v18.6.2
</code>

=== Регистрация DooD ===

  * Включаем Docker [[Технология Docker#Insecure Private Registry]]

<code>
gate:~# docker run --rm -v /srv/gitlab-runner/config:/etc/gitlab-runner gitlab/gitlab-runner:v18.6.2 register \
  --non-interactive \
  --url "http://server.corpX.un/" \
  --registration-token "NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN" \
  --executor "docker" \
  --docker-image "docker:stable" \
  --docker-volumes /var/run/docker.sock:/var/run/docker.sock \
  --description "dood-runner"
</code>

=== Регистрация DinD ===

  * Можно отключить Docker [[Технология Docker#Insecure Private Registry]]

<code>
gate:~# docker run --rm -v /srv/gitlab-runner/config:/etc/gitlab-runner gitlab/gitlab-runner:v18.6.2 register \
  --non-interactive \
  --url "http://server.corpX.un/" \
  --token "NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN" \
  --executor "docker" \
  --docker-image "docker:stable" \
  --docker-privileged \
  --description "dind-runner"
</code><code>
gate:~# docker volume ls

gate:~# docker volume inspect ...
</code>

=== TLS для DooD и DinD ===
<code>
server# scp /opt/freeipa-data/etc/ipa/ca.crt client1:/srv/gitlab-runner/config/

client1# docker run --rm -v /srv/gitlab-runner/config:/etc/gitlab-runner gitlab/gitlab-runner:v18.6.2 register \
...
  --url "https://gitlab.corpX.un/" \
  --tls-ca-file "/etc/gitlab-runner/ca.crt" \
...
</code>
===== GitLab CI/CD =====

  * [[https://docs.gitlab.com/ee/ci/examples/#cicd-templates|CI/CD templates]]
  * [[https://medium.com/@ryzmen/gitlab-fast-pipelines-stages-jobs-c51c829b9aa1|GitLab: understanding pipelines, stages, jobs and organising them efficiently for speed and feedback loop]]
  * [[https://stackoverflow.com/questions/64725914/how-to-disable-auto-pipelines-in-gitlab|How to disable auto pipelines in gitlab]]

==== Пример shell make ====

<code>
IDE GitLab->New File: .gitlab-ci.yml

или

CI/CD -> Editor -> Configure Pipelines

или

Build -> Pipeline editor -> Configure Pipelines
</code><code>
#stages:
#  - build
#  - test
#  - deploy

test1-job:
  stage: test
  script:
    - echo $(date) "Do test dhcpd" >> /tmp/Bash.gitlab-ci.log
    - make test
  tags:
    - dhcptest

deploy1-job:
  stage: deploy
  script:
    - echo $(date) "Do deploy dhcpd" >> /tmp/Bash.gitlab-ci.log
    - sudo make install
  tags:
    - dhcpdeploy
</code>

==== Пример shell ansible ====

  * [[https://asyncdrink.com/blog/gitlab-ci-limit-branch|Limit Gitlab CI pipelines to specific branches]]
  * [[https://stackoverflow.com/questions/52169219/get-branch-name-in-gitlab-ci|Get Branch name in gitlab ci]]

<code>
Settings -> CI/CD -> Variables -> Add variable -> Masked and hidden ... ANS_V_SEC или VAULT_ADDR/VAULT_TOKEN

снять Protect variable
  ИЛИ
Settings -> Repository -> Protected Branches - Add Protected Branch -> test
</code>
<code>
~/openvpn1# cat .gitlab-ci.yml
  или
Build -> Pipeline editor -> Configure Pipelines
</code><code>
stages:
  - deploy

deploy_test:
  stage: deploy
  script:
    - ansible-playbook openvpn1.yaml -i inventory.yaml -e "variable_host=test_nodes"
#    - echo $ANS_V_SEC | ansible-playbook openvpn1.yaml -i inventory2.yaml --vault-password-file=/bin/cat -e "variable_host=test_nodes"
#    - sops exec-file --no-fifo inventory.yaml 'ansible-playbook openvpn1.yaml -i {} -e "variable_host=test_nodes"'
  tags:
    - ansible
  only:
    - test

deploy_prod:
  stage: deploy
  script:
    - ansible-playbook openvpn1.yaml -i inventory.yaml
#    - echo $ANS_V_SEC | ansible-playbook openvpn1.yaml -i inventory2.yaml --vault-password-file=/bin/cat
#    - sops exec-file --no-fifo inventory.yaml 'ansible-playbook openvpn1.yaml -i {}'
  tags:
    - ansible
  only:
    - main
</code>

==== Пример shell docker ====

  * Технология Docker [[Технология Docker#Предоставление прав непривилегированным пользователям]]

  * [[https://docs.gitlab.com/ee/ci/docker/using_docker_build.html|Use Docker to build Docker images]]
  * [[https://docs.gitlab.com/ee/ci/variables/predefined_variables.html|Predefined variables reference]]
  * [[https://docs.gitlab.com/ee/ci/variables/#add-a-cicd-variable-to-a-project|Add a CI/CD variable to a project]]


<code>
# Можно назначить в GitLab (Settings -> CI/CD -> Variables)
# export MY_CI_REGISTRY=server.corpX.un:5000 
# export MY_CI_REGISTRY_IMAGE=student/webd
# или использовать встроенные CI_REGISTRY и CI_REGISTRY_IMAGE 
# поскольку используем этот же проект GitLab как Registry

# в GitLab будет установлено автоматически после git commit -m "ver 1.2" и git push
# export CI_COMMIT_MESSAGE="ver 1.2"   
</code>

<code>
gitlab-runner@server:~/webd$ cat build.sh
</code><code>
#!/bin/sh

VER="$(echo $CI_COMMIT_MESSAGE | sed 's/[^a-zA-Z0-9\.]//g')"

# needed once
# docker login -u $MY_CI_REGISTRY_USER -p $MY_CI_REGISTRY_PASSWORD $MY_CI_REGISTRY
# docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY

docker build -t webd webd

#docker run --rm -e MYMODE=TEST webd || exit 1

#docker tag webd $MY_CI_REGISTRY/$MY_CI_REGISTRY_IMAGE:$VER
#docker tag webd $MY_CI_REGISTRY/$MY_CI_REGISTRY_IMAGE
docker tag webd $CI_REGISTRY_IMAGE:$VER
docker tag webd $CI_REGISTRY_IMAGE

# previously need: docker login ...

#docker push $MY_CI_REGISTRY/$MY_CI_REGISTRY_IMAGE:$VER
#docker push $MY_CI_REGISTRY/$MY_CI_REGISTRY_IMAGE
docker push $CI_REGISTRY_IMAGE:$VER
docker push $CI_REGISTRY_IMAGE
</code><code>
gitlab-runner@server:~/webd$ cat .gitlab-ci.yml
</code><code>
stages:
  - lintertest
  - build
#  - deploy

lintertest1:
  stage: lintertest
  script:
#    - echo $(date) "Do a test webd here" >> /tmp/Bash.gitlab-ci.log
    - shellcheck webd/webd
  tags:
    - shellcheck

build1:
  stage: build
  script:
#    - echo $(date) "Do a build webd here" >> /tmp/Bash.gitlab-ci.log
#    - env | tee -a /tmp/Bash.gitlab-ci.log
    - sh build.sh
  tags:
    - webdbuild

#deploy1:
#  stage: deploy
#  script:
#    - sh deploy.sh
#  tags:
#    - webddeploy

### OR .gitlab-ci.yml for gowebd-k8s project running from another pipeline ###

#deploy1:
#  stage: deploy
#  variables:
#    HELM_NAMESPACE: "my-ns"
#  rules:
#    - if: '$CI_PIPELINE_SOURCE == "pipeline" && $VER'
#  script:
#    - env
#    - envsubst < my-webd-deployment-env.yaml | kubectl apply -f - -n my-ns
#    - helm upgrade -i my-webd webd-chart/ --set=image.tag=$VER --create-namespace
</code>

==== Пример shell Kubernetes ====

<code>
kube1:~/gowebd-k8s# cat .gitlab-ci.yml
</code><code>
stages:
  - deploy

#variables:
#  HELM_NAMESPACE: "my-ns"

trigger-deploy:
  stage: deploy
  rules:
    - if: '$CI_PIPELINE_SOURCE == "pipeline" && $VER'
  script:
    - env
    - envsubst < my-webd-deployment-env.yaml | kubectl apply -f - -n my-ns
#    - helm upgrade -i my-webd webd-chart/ --set=image.tag=$VER --create-namespace
  tags:
    - k8s-deploy

manual-deploy:
  stage: deploy
  when: manual
  variables:
    VER: "$MY_WEBD_VER"  # New Pipeline or Settings->CI/CD->Variables
  script:
    - env
    - envsubst < my-webd-deployment-env.yaml | kubectl apply -f - -n my-ns
#    - helm upgrade -i my-webd webd-chart/ --set=image.tag=$VER --create-namespace
  tags:
    - k8s-deploy
</code>

== старый вариант ==
<code>
gitlab-runner@server:~/webd$ cp my-webd-deployment.yaml my-webd-deployment-env.yaml
  или
gitlab-runner@server:~/gowebd-k8s$ scp root@node1:my-webd-deployment.yaml my-webd-deployment-env.yaml

gitlab-runner@server:~/webd$ cat my-webd-deployment-env.yaml
</code><code>
...
        image: server.corpX.un:5000/student/webd:$VER
...
</code><code>
# в GitLab будет устанавлено автоматически
gitlab-runner@gate:~/webd$ export CI_COMMIT_MESSAGE="ver 1.2"

gitlab-runner@gate:~/webd$ cat deploy.sh
</code><code>
#!/bin/sh

#alias kubectl='minikube kubectl --'

kubectl apply -f my-webd-deployment.yaml -n my-ns

#export VER="$(echo $CI_COMMIT_MESSAGE | sed 's/[^a-zA-Z0-9\.]//g')"

#envsubst < my-webd-deployment-env.yaml | kubectl apply -f - -n my-ns

kubectl apply -f my-webd-service.yaml -n my-ns


#export HELM_NAMESPACE=my-ns
#helm upgrade --install my-webd webd-chart/ --set=image.tag=$VER --create-namespace
</code><code>
gitlab-runner@server:~/$ kubectl describe replicaset.apps/my-webd-NNNNNNNNNNN -n my-ns
</code>
==== Пример CI с использованием контейнеров ====

  * [[https://akyriako.medium.com/build-golang-docker-images-with-gitlab-ci-pipelines-2117f8505350|Build Golang Docker images with GitLab CI Pipelines]]
  * [[https://blog.callr.tech/building-docker-images-with-gitlab-ci-best-practices/|Best practices for building docker images with GitLab CI]]

  * [[https://habr.com/ru/articles/764568/|Настройка CI/CD для Gitlab-репозитория: схемы и гайд по шагам (на примере Spring Boot-приложения)]]

  * [[https://stackoverflow.com/questions/63693061/how-to-run-a-script-from-file-in-another-project-using-include-in-gitlab-ci|How to run a script from file in another project using include in GitLab CI?]]
  * [[https://medium.com/@captain_sparrow/gitlab-%D1%82%D1%80%D0%B8%D0%B3%D0%B3%D0%B5%D1%80%D1%8B-%D0%B8-%D0%B4%D0%BB%D1%8F-%D0%BA%D0%B0%D0%BA%D0%B8%D1%85-%D1%82%D0%B5%D1%81%D1%82%D0%BE%D0%B2-%D0%B8%D1%85-%D1%81%D1%82%D0%BE%D0%B8%D1%82-%D0%B8%D1%81%D0%BF%D0%BE%D0%BB%D1%8C%D0%B7%D0%BE%D0%B2%D0%B0%D1%82%D1%8C-afa04f8c78a7|Gitlab триггеры и для каких тестов их стоит использовать?]]

  * [[https://earthly.dev/blog/docker-vs-buildah-vs-kaniko/|Container Image Build Tools: Docker vs. Buildah vs. kaniko]]
  * [[https://docs.gitlab.com/ee/ci/docker/using_kaniko.html|Use kaniko to build Docker images]]
  * [[https://eng.d2iq.com/blog/a-tale-of-two-container-image-tools-skopeo-and-crane/|A Tale of Two Container Image Tools: Skopeo and Crane]]

<code>
student@client1:~/gowebd$ cat .gitlab-ci.yml
</code><code>
stages:
#  - lint
#  - prebuildtest
  - build
#  - scan
#  - test
  - push
#  - deploy

#variables:
#  DOCKER_TLS_CERTDIR: ""
#  KANIKO_REGISTRY_MIRROR: "mirror.gcr.io"

#services:
#  - name: docker:20.10.24-dind
#    command:
#      [
#        '--insecure-registry=server.corpX.un:5000','--registry-mirror=https://mirror.gcr.io',
#      ]

before_script:
  - env
#  - docker info
  - echo -n $CI_REGISTRY_PASSWORD | docker login -u $CI_REGISTRY_USER --password-stdin $CI_REGISTRY

Build:
  stage: build
#  image:
#    name: gcr.io/kaniko-project/executor:v1.9.0-debug
#    entrypoint: [""]
  script:
    - docker pull $CI_REGISTRY_IMAGE:latest || true
    - >
      docker build
      --pull
      --cache-from $CI_REGISTRY_IMAGE:latest
      --tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA .
    - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA

#    - echo "{\"auths\":{\"${CI_REGISTRY}\":{\"auth\":\"$(printf "%s:%s" "${CI_REGISTRY_USER}" "${CI_REGISTRY_PASSWORD}" | base64 | tr -d '\n')\"}}}" > /kaniko/.docker/config.json
#    - /kaniko/executor
#      --insecure --skip-tls-verify
#      --context "${CI_PROJECT_DIR}"
#      --dockerfile "${CI_PROJECT_DIR}/Dockerfile"
#      --destination "${CI_REGISTRY_IMAGE}:${CI_COMMIT_SHA}"

#  except:
#    - tags
#  tags:
#    - build

Push latest:
#  image:
#    name: gcr.io/go-containerregistry/crane:debug
#    entrypoint: [""]
  variables:
    GIT_STRATEGY: none
  stage: push
  only:
    - main
  script:
    - docker pull $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
    - docker tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA $CI_REGISTRY_IMAGE:latest
    - docker push $CI_REGISTRY_IMAGE:latest

#    - crane auth login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
#    - crane --insecure cp $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA $CI_REGISTRY_IMAGE:latest
#  tags:
#    - build

Push tag:
#  image:
#    name: gcr.io/go-containerregistry/crane:debug
#    entrypoint: [""]
  variables:
    GIT_STRATEGY: none
  stage: push
  only:
    - tags
  script:
    - docker pull $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
    - docker tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_NAME
    - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_NAME

#   - crane auth login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
#   - crane --insecure cp $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_NAME
#  tags:
#    - build

### Not work in DooD
#Lint test:
#  stage: lint
#  script:
#    - pwd
#    - docker run --rm -v $(pwd):/app -w /app golangci/golangci-lint:v1.62.2 golangci-lint run --timeout=10m
#  except:
#    - tags
#  tags:
#    - build

#semgrep:
#  stage: lint
#  image: semgrep/semgrep
#  script: semgrep --config=auto --error
#  variables:
#    SEMGREP_SRC_DIRECTORY: $CI_PROJECT_DIR
#    #https_proxy: http://gate.isp.un:3128/
#    #no_proxy: localhost,127.0.0.1,isp.un,corpX.un
#  except:
#    - tags
#  tags:
#    - build

#Unit test:
#  stage: prebuildtest
#  script:
#    - export PYTHON_KEYRING_BACKEND=keyring.backends.null.Keyring
#    - poetry install
#    - poetry run python3 -m unittest
#  except:
#    - tags
#  tags:
#    - build

#Smoke test:
#  stage: test
#  script:
#    - MY_ID=$(docker run -d --rm $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA)
#    - MY_IP=$(docker inspect -f '{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}' $MY_ID)
#    - docker run --rm alpine/curl -sS $MY_IP
#    - docker stop $MY_ID
#  except:
#    - tags
#  tags:
#    - build

#trivy:
#  stage: scan
#  image:
#    name: ghcr.io/aquasecurity/trivy
#    entrypoint: [""] 
#  variables:
#    #TRIVY_USERNAME: "$CI_REGISTRY_USER"
#    #TRIVY_PASSWORD: "$CI_REGISTRY_PASSWORD"
#    TRIVY_REGISTRY: "$CI_REGISTRY"
#    TRIVY_CACHE_DIR: ".trivycache/"
#  script:
#    - trivy image --exit-code 1 --severity HIGH --insecure $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
#  cache:
#    paths:
#      - ".trivycache/"
#  except:
#    - tags
#  tags:
#    - build

#Deploy:
#  variables:
#    VER: "$CI_COMMIT_REF_NAME"
#  stage: deploy
#  only:
#    - tags
#  trigger:
#    project: student/gowebd-k8s
</code>
===== Сервер OpenID =====

  * [[https://github.com/zmartzone/mod_auth_openidc/wiki/GitLab-OAuth2]]
  * [[Сервис HTTP#Управление доступом к HTTP серверу с использованием OpenID аутентификации]]
  * Admin Area-> Applications

<code>
Name: test-cgi
Redirect URI: http://gate.corpX.un/cgi-bin/test-cgi  !!! Если URL каталога, то без финального "/" !!!
Trusted: Yes
Confidential: Yes
Scopes: openid

Application ID: ...
Secret: ...
Callback URL = Redirect URI
</code>

===== Клиент OpenID =====

  * [[https://docs.gitlab.com/ee/administration/auth/oidc.html|You can use GitLab as a client application with OpenID Connect as an OmniAuth provider]]
  * [[https://gitlab.com/gitlab-org/gitlab/-/issues/196193|use self-signed to integate gitlab with keycloak but see error: certificate verify failed (self signed certificate))]]
  * [[https://forum.gitlab.com/t/using-keycloak-as-sso-for-gitlab-with-pre-existing-users-no-autocreate/67833|Using Keycloak as SSO for Gitlab with pre-existing users (no autocreate)]]

<code>
# cp server.crt /etc/gitlab/trusted-certs/
  или
# cp ca.crt /etc/gitlab/trusted-certs/

# cat /etc/gitlab/gitlab.rb
</code><code>
...
gitlab_rails['omniauth_providers'] = [
  {
    name: "openid_connect",
    label: "Keycloak",
    args: {
      name: "openid_connect",
      scope: ["openid", "profile", "email"],
      response_type: "code",
      issuer:  "https://keycloak.corpX.un/realms/corpX",
      client_auth_method: "query",
      discovery: true,
      uid_field: "preferred_username",
      pkce: true,
      client_options: {
        identifier: "any-client",
        secret: "anystring",
#        redirect_uri: "https://gitlab.corpX.un/users/auth/openid_connect/callback"
#        redirect_uri: "https://gate.corpX.un/users/auth/openid_connect/callback"
      }
    }
  }
]
...
</code>

  * [[#Проверка конфигурации и перезапуск]]
  * User -> Profile -> Account -> Select a service to sign in with -> Keycloak
Методические материалы Лохтурова Вячеслава

User Tools

Site Tools

Page Tools
