This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
инструмент_gitlab [2022/07/13 09:05] val [Пример docker] |
инструмент_gitlab [2024/04/16 08:14] val [Установка из пакета] |
||
---|---|---|---|
Line 1: | Line 1: | ||
====== Инструмент GitLab ====== | ====== Инструмент GitLab ====== | ||
+ | |||
+ | * [[https://ru.wikipedia.org/wiki/GitLab - Википедия]] | ||
* [[https://youtu.be/n_21ya2MoKg|Youtube. RomNero. GitLab. Devops система]] | * [[https://youtu.be/n_21ya2MoKg|Youtube. RomNero. GitLab. Devops система]] | ||
Line 5: | Line 7: | ||
===== Установка ===== | ===== Установка ===== | ||
+ | |||
+ | * RAM от 4Gb | ||
+ | |||
+ | ==== Если нужен почтовый сервер ==== | ||
+ | |||
+ | <code> | ||
+ | server# time ansible-playbook conf/ansible/roles/mail.yml | ||
+ | real 2m57.922s | ||
+ | |||
+ | # cat /etc/apache2/ports.conf | ||
+ | </code><code> | ||
+ | ... | ||
+ | Listen 81 | ||
+ | ... | ||
+ | </code><code> | ||
+ | server# service apache2 restart | ||
+ | </code><code> | ||
+ | http://server.corpX.un:81/mail/ | ||
+ | </code> | ||
+ | |||
+ | ==== Установка из репозитория ==== | ||
* [[https://about.gitlab.com/install/|Install self-managed GitLab]] | * [[https://about.gitlab.com/install/|Install self-managed GitLab]] | ||
* Доступно из РФ: [[https://packages.gitlab.com/gitlab/gitlab-ce]] | * Доступно из РФ: [[https://packages.gitlab.com/gitlab/gitlab-ce]] | ||
- | * [[https://www.techbeatly.com/configure-custom-ssl-to-secure-gitlab-server/|Configure Custom SSL to Secure GitLab Server]] | + | * [[http://gate.isp.un/unix/Git/gitlab-ce_16.3.3-ce.0_amd64.deb]] |
+ | |||
+ | <code> | ||
+ | server# apt-get install -y curl ca-certificates perl | ||
+ | |||
+ | server# curl https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.deb.sh | bash | ||
+ | |||
+ | server# time EXTERNAL_URL="http://$(hostname)" apt-get install gitlab-ce | ||
+ | ... | ||
+ | real 38m49.787s !!! Загрузка может прерываться, надо повторять команду !!! | ||
+ | .. | ||
+ | </code> | ||
+ | |||
+ | ==== Установка через docker-compose ==== | ||
+ | |||
+ | * [[https://docs.gitlab.com/ee/install/docker.html#install-gitlab-using-docker-compose|Install GitLab using Docker Compose]] | ||
+ | * [[Технология Docker]] | ||
+ | * [[Технология Docker#docker-compose]] | ||
+ | |||
+ | <code> | ||
+ | # cat docker-compose.yml | ||
+ | </code><code> | ||
+ | version: '3.6' | ||
+ | services: | ||
+ | web: | ||
+ | image: 'gitlab/gitlab-ce:latest' | ||
+ | # image: 'gitlab/gitlab-ce:16.7.4-ce.0' | ||
+ | restart: always | ||
+ | hostname: 'server.corpX.un' | ||
+ | environment: | ||
+ | GITLAB_ROOT_PASSWORD: "strongpassword" | ||
+ | GITLAB_OMNIBUS_CONFIG: | | ||
+ | prometheus_monitoring['enable'] = false | ||
+ | gitlab_rails['registry_enabled'] = true | ||
+ | gitlab_rails['registry_host'] = "server.corpX.un" | ||
+ | external_url 'http://server.corpX.un' | ||
+ | registry_external_url 'http://server.corpX.un' | ||
+ | gitlab_rails['registry_port'] = "5000" | ||
+ | registry['registry_http_addr'] = "server.corpX.un:5000" | ||
+ | # external_url 'https://server.corpX.un' | ||
+ | # registry_external_url 'https://server.corpX.un:5000' | ||
+ | # gitlab_rails['registry_port'] = "5050" | ||
+ | # registry['registry_http_addr'] = "server.corpX.un:5050" | ||
+ | ports: | ||
+ | - '80:80' | ||
+ | # - '443:443' | ||
+ | - '2222:22' | ||
+ | - '5000:5000' | ||
+ | volumes: | ||
+ | - '/etc/gitlab:/etc/gitlab' | ||
+ | - '/srv/gitlab/logs:/var/log/gitlab' | ||
+ | - '/srv/gitlab/data:/var/opt/gitlab' | ||
+ | shm_size: '256m' | ||
+ | </code><code> | ||
+ | # ### cat /etc/gitlab/ssl/gitlab.bmstu.ru.{crt,key} | ||
+ | |||
+ | # docker-compose up -d | ||
+ | |||
+ | # docker logs root_web_1 -n 10 -f | ||
+ | |||
+ | ### docker-compose stop | ||
+ | ### rm -r /srv/gitlab/ /etc/gitlab/ | ||
+ | </code> | ||
===== Подключение ===== | ===== Подключение ===== | ||
- | * [[https://stackoverflow.com/questions/60062065/gitlab-initial-root-password|gitlab initial root password]] | + | ==== Подключение к Web интерфейсу ===== |
+ | |||
+ | * http://server.corpX.un/ | ||
+ | |||
+ | ==== Подключение через API ==== | ||
+ | |||
+ | * Токен доступа: Settings -> Access Tokens ([[https://docs.gitlab.com/ee/user/project/settings/project_access_tokens.html|Project access tokens]]), в примере достаточно role: Reporter, Scopes: api | ||
+ | * Номер проекта: Settings -> General ([[https://stackoverflow.com/questions/39559689/where-do-i-find-the-project-id-for-the-gitlab-api|Where do I find the project ID for the GitLab API?]]) | ||
+ | * [[https://stackoverflow.com/questions/56943327/how-to-download-a-single-file-from-gitlab|How to download a single file from GitLab?]] | ||
<code> | <code> | ||
- | # cat /etc/gitlab/initial_root_password | + | root@node1,2,3:~# curl "http://server.corpX.un/api/v4/projects/2/repository/files/docker-compose.yml/raw?ref=master" | tee docker-compose.yml |
+ | |||
+ | или, для НЕ публичных проектов | ||
+ | root@node1,2,3:~# curl --header "PRIVATE-TOKEN: NNNNNNNNNNNNNNNNNNNNN" "http://server.corpX.un/api/v4/projects/4/repository/files/docker-compose.yml/raw?ref=master" | tee docker-compose.yml | ||
</code> | </code> | ||
+ | * [[Сервис Ansible#ansible-pull]] | ||
+ | <code> | ||
+ | client1:~/ansible-pull-gpo# cat readme.md | ||
+ | </code><code> | ||
+ | sudo -i | ||
+ | export BR=main; bash <(curl -s http://gate.corp13.un/api/v4/projects/1/repository/files/start.sh/raw?ref=$BR) | ||
+ | </code> | ||
===== Настройка ===== | ===== Настройка ===== | ||
Line 26: | Line 129: | ||
# cat /etc/gitlab/gitlab.rb | # cat /etc/gitlab/gitlab.rb | ||
</code><code> | </code><code> | ||
- | # gitlab-ctl reconfigure | + | ... |
+ | external_url 'http://server.corpX.un' | ||
+ | ... | ||
+ | </code> | ||
+ | |||
+ | ==== Проверка конфигурации и перезапуск ==== | ||
+ | |||
+ | <code> | ||
+ | ### docker exec -it root_web_1 bash | ||
+ | |||
+ | # gitlab-ctl show-config | ||
+ | |||
+ | # time gitlab-ctl reconfigure | ||
+ | ... | ||
+ | real 2m34.726s | ||
+ | ... | ||
</code> | </code> | ||
==== GitLab Docker Registry ==== | ==== GitLab Docker Registry ==== | ||
+ | |||
+ | * [[https://docs.gitlab.com/ee/administration/packages/container_registry.html|The Container Registry is automatically enabled and available on your GitLab domain, port 5050 if you’re using the built-in Let’s Encrypt integration]] | ||
+ | * [[https://sysadmintalks.ru/insecure-gitlab-registry/|Настройка работы Gitlab с registry без ssl - Sysadmin]] | ||
<code> | <code> | ||
# cat /etc/gitlab/gitlab.rb | # cat /etc/gitlab/gitlab.rb | ||
Line 35: | Line 156: | ||
... | ... | ||
registry_external_url 'http://server.corpX.un' | registry_external_url 'http://server.corpX.un' | ||
- | ... | ||
gitlab_rails['registry_enabled'] = true | gitlab_rails['registry_enabled'] = true | ||
- | ... | ||
gitlab_rails['registry_host'] = "server.corpX.un" | gitlab_rails['registry_host'] = "server.corpX.un" | ||
- | ... | + | gitlab_rails['registry_port'] = "5000" |
registry['registry_http_addr'] = "server.corpX.un:5000" | registry['registry_http_addr'] = "server.corpX.un:5000" | ||
... | ... | ||
</code> | </code> | ||
+ | * [[#Проверка конфигурации и перезапуск]] | ||
==== GitLab Grafana ==== | ==== GitLab Grafana ==== | ||
<code> | <code> | ||
Line 52: | Line 172: | ||
... | ... | ||
</code> | </code> | ||
+ | |||
+ | * [[#Проверка конфигурации и перезапуск]] | ||
+ | ==== GitLab Prometheus ==== | ||
+ | |||
+ | <code> | ||
+ | # cat /etc/gitlab/gitlab.rb | ||
+ | </code><code> | ||
+ | ... | ||
+ | prometheus_monitoring['enable'] = false | ||
+ | ... | ||
+ | </code> | ||
+ | |||
+ | * [[#Проверка конфигурации и перезапуск]] | ||
+ | |||
+ | <code> | ||
+ | # time rm -rf /var/opt/gitlab/prometheus/data/* | ||
+ | </code> | ||
+ | |||
+ | ==== Включение TLS ==== | ||
+ | |||
+ | * [[https://docs.gitlab.com/omnibus/settings/ssl.html#configure-https-manually|Configure HTTPS manually]] | ||
+ | * [[https://www.techbeatly.com/configure-custom-ssl-to-secure-gitlab-server/|Configure Custom SSL to Secure GitLab Server]] | ||
+ | |||
+ | <code> | ||
+ | mkdir /etc/gitlab/ssl/ | ||
+ | |||
+ | cp wild.crt -v /etc/gitlab/ssl/$(hostname).crt | ||
+ | cp wild.key -v /etc/gitlab/ssl/$(hostname).key | ||
+ | |||
+ | # cat /etc/gitlab/gitlab.rb | ||
+ | </code><code> | ||
+ | ... | ||
+ | external_url 'https://server.corpX.un' | ||
+ | ... | ||
+ | # nginx['ssl_certificate'] = "/etc/gitlab/ssl/#{node['fqdn']}.crt" | ||
+ | # nginx['ssl_certificate_key'] = "/etc/gitlab/ssl/#{node['fqdn']}.key" | ||
+ | ... | ||
+ | letsencrypt['enable'] = false | ||
+ | ... | ||
+ | </code> | ||
+ | |||
+ | * [[#Проверка конфигурации и перезапуск]] | ||
+ | |||
==== Управление пользователями ==== | ==== Управление пользователями ==== | ||
+ | |||
+ | === Внутренние пользователи === | ||
+ | |||
+ | * Username - login, Name - ФИО | ||
+ | |||
+ | <code> | ||
+ | # cat /etc/gitlab/initial_root_password | ||
+ | </code> | ||
+ | |||
+ | * [[https://stackoverflow.com/questions/60062065/gitlab-initial-root-password|gitlab initial root password reset]] | ||
+ | |||
+ | <code> | ||
+ | # gitlab-rake "gitlab:password:reset[root]" | ||
+ | </code> | ||
=== Использование LDAP === | === Использование LDAP === | ||
Line 70: | Line 247: | ||
label: 'LDAP' | label: 'LDAP' | ||
host: 'server.corpX.un' | host: 'server.corpX.un' | ||
+ | # host: 'server2.corpX.un' | ||
port: 389 | port: 389 | ||
- | # port: 636 | + | # uid: 'uid' |
- | uid: 'uid' | + | uid: 'sAMAccountName' |
- | bind_dn: 'cn=admin,dc=corpX,dc=un' | + | # bind_dn: 'cn=admin,dc=corpX,dc=un' |
- | password: 'secret' | + | # password: 'secret' |
+ | bind_dn: 'cn=Administrator,cn=Users,dc=corpX,dc=un' | ||
+ | password: 'Pa$$w0rd' | ||
encryption: 'plain' | encryption: 'plain' | ||
- | # encryption: 'simple_tls' | + | # active_directory: false |
- | active_directory: false | + | active_directory: true |
- | base: 'ou=People,dc=corpX,dc=un' | + | base: 'dc=corpX,dc=un' |
+ | EOS | ||
... | ... | ||
</code> | </code> | ||
+ | * [[#Проверка конфигурации и перезапуск]] | ||
===== GitLab Runner ===== | ===== GitLab Runner ===== | ||
+ | |||
+ | ==== Установка из пакета ==== | ||
* [[https://docs.gitlab.com/runner/install/linux-manually.html|Install GitLab Runner manually on GNU/Linux]] | * [[https://docs.gitlab.com/runner/install/linux-manually.html|Install GitLab Runner manually on GNU/Linux]] | ||
+ | * [[https://val.bmstu.ru/unix/Git/gitlab-runner_amd64.deb]] (15.0.0) | ||
- | * [[https://medium.com/@ryzmen/gitlab-fast-pipelines-stages-jobs-c51c829b9aa1|GitLab: understanding pipelines, stages, jobs and organising them efficiently for speed and feedback loop]] | + | <code> |
+ | # wget http://gate.isp.un/unix/Git/gitlab-runner_amd64.deb | ||
- | * [[https://val.bmstu.ru/unix/Git/gitlab-runner_amd64.deb]] (15.0.0) | + | ##2 часа## curl -LJO "https://gitlab-runner-downloads.s3.amazonaws.com/latest/deb/gitlab-runner_amd64.deb" |
+ | # dpkg -i gitlab-runner_amd64.deb | ||
+ | </code> | ||
+ | ==== Регистрация ==== | ||
<code> | <code> | ||
- | root@node2:~# gitlab-runner register | + | # gitlab-runner register --help |
- | shell или docker | + | |
- | root@node2:~# cat /etc/gitlab-runner/config.toml | + | # export CI_SERVER_URL=http://server.corpX.un |
+ | |||
+ | # gitlab-runner register | ||
+ | ... | ||
+ | Enter the GitLab instance URL: http://server.corpX.un | ||
+ | Enter the registration token: ... | ||
+ | ... | ||
+ | Enter tags for the runner: dhcptest, dhcpdeploy | ||
+ | или | ||
+ | Enter tags for the runner: openvpn1deploy | ||
+ | ... | ||
+ | Enter an executor: shell | ||
+ | ... | ||
+ | </code> | ||
+ | |||
+ | или | ||
+ | |||
+ | <code> | ||
+ | # gitlab-runner register -n --executor "shell" -u http://server.corpX.un -r "NNNNNNNNNNNNNNNNNNNNNNNNNNNN" | ||
+ | </code> | ||
+ | |||
+ | Перезапускать не нужно | ||
+ | |||
+ | <code> | ||
+ | # cat /etc/gitlab-runner/config.toml | ||
log_level = "debug" | log_level = "debug" | ||
... | ... | ||
</code><code> | </code><code> | ||
- | root@node2:~# systemctl start gitlab-runner | + | # systemctl restart gitlab-runner |
+ | |||
+ | # gitlab-runner verify | ||
</code> | </code> | ||
+ | |||
+ | ==== Установка в виде контейнера ==== | ||
+ | |||
+ | * [[https://habr.com/ru/companies/cloud4y/articles/710782/|Использование Docker in Docker в GitLab]] | ||
+ | |||
+ | <code> | ||
+ | gate:~### docker stop gitlab-runner; docker rm gitlab-runner | ||
+ | gate:~### rm /srv/gitlab-runner/config/config.toml | ||
+ | |||
+ | gate:~# docker run -d --name gitlab-runner --restart always \ | ||
+ | -v /srv/gitlab-runner/config:/etc/gitlab-runner \ | ||
+ | -v /var/run/docker.sock:/var/run/docker.sock \ | ||
+ | gitlab/gitlab-runner:latest | ||
+ | </code> | ||
+ | |||
+ | === Регистрация DooD === | ||
+ | |||
+ | * Включаем Docker [[Технология Docker#Insecure Private Registry]] | ||
+ | |||
+ | <code> | ||
+ | gate:~# docker run --rm -v /srv/gitlab-runner/config:/etc/gitlab-runner gitlab/gitlab-runner register \ | ||
+ | --non-interactive \ | ||
+ | --url "http://server.corpX.un/" \ | ||
+ | --registration-token "NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN" \ | ||
+ | --executor "docker" \ | ||
+ | --docker-image "docker:stable" \ | ||
+ | --docker-volumes /var/run/docker.sock:/var/run/docker.sock \ | ||
+ | --description "dood-runner" | ||
+ | </code> | ||
+ | |||
+ | === Регистрация DinD === | ||
+ | |||
+ | * Можно отключить Docker [[Технология Docker#Insecure Private Registry]] | ||
+ | |||
+ | <code> | ||
+ | gate:~# docker run --rm -v /srv/gitlab-runner/config:/etc/gitlab-runner gitlab/gitlab-runner register \ | ||
+ | --non-interactive \ | ||
+ | --url "http://server.corpX.un/" \ | ||
+ | --registration-token "NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN" \ | ||
+ | --executor "docker" \ | ||
+ | --docker-image "docker:stable" \ | ||
+ | --docker-privileged \ | ||
+ | --description "dind-runner" | ||
+ | </code><code> | ||
+ | gate:~# docker volume ls | ||
+ | |||
+ | gate:~# docker volume inspect ... | ||
+ | </code> | ||
+ | |||
===== GitLab CI/CD ===== | ===== GitLab CI/CD ===== | ||
* [[https://docs.gitlab.com/ee/ci/examples/#cicd-templates|CI/CD templates]] | * [[https://docs.gitlab.com/ee/ci/examples/#cicd-templates|CI/CD templates]] | ||
+ | * [[https://medium.com/@ryzmen/gitlab-fast-pipelines-stages-jobs-c51c829b9aa1|GitLab: understanding pipelines, stages, jobs and organising them efficiently for speed and feedback loop]] | ||
* [[https://stackoverflow.com/questions/64725914/how-to-disable-auto-pipelines-in-gitlab|How to disable auto pipelines in gitlab]] | * [[https://stackoverflow.com/questions/64725914/how-to-disable-auto-pipelines-in-gitlab|How to disable auto pipelines in gitlab]] | ||
Line 109: | Line 373: | ||
<code> | <code> | ||
- | student@gate:~/dhcp$ cat .gitlab-ci.yml | + | IDE GitLab->New File: .gitlab-ci.yml |
+ | |||
+ | или | ||
+ | |||
+ | CI/CD -> Editor -> Configure Pipelines | ||
+ | |||
+ | или | ||
+ | |||
+ | Build -> Pipeline editor -> Configure Pipelines | ||
</code><code> | </code><code> | ||
- | test1: | + | #stages: |
+ | # - build | ||
+ | # - test | ||
+ | # - deploy | ||
+ | |||
+ | test1-job: | ||
stage: test | stage: test | ||
script: | script: | ||
Line 119: | Line 396: | ||
- dhcptest | - dhcptest | ||
- | deploy1: | + | deploy1-job: |
stage: deploy | stage: deploy | ||
script: | script: | ||
Line 126: | Line 403: | ||
tags: | tags: | ||
- dhcpdeploy | - dhcpdeploy | ||
- | |||
</code> | </code> | ||
==== Пример shell ansible ==== | ==== Пример shell ansible ==== | ||
+ | |||
+ | * [[https://asyncdrink.com/blog/gitlab-ci-limit-branch|Limit Gitlab CI pipelines to specific branches]] | ||
+ | * [[https://stackoverflow.com/questions/52169219/get-branch-name-in-gitlab-ci|Get Branch name in gitlab ci]] | ||
<code> | <code> | ||
- | root@server:~/openvpn1# cat .gitlab-ci.yml | + | Administrator@Ra-master ~/openvpn1 (test) |
+ | λ touch .gitlab-ci.yml | ||
+ | или | ||
+ | Build -> Pipeline editor -> Configure Pipelines | ||
</code><code> | </code><code> | ||
deploy_test: | deploy_test: | ||
Line 141: | Line 423: | ||
tags: | tags: | ||
- openvpn1deploy | - openvpn1deploy | ||
- | rules: | + | only: |
- | - if: ($CI_COMMIT_BRANCH == "test" && $CI_COMMIT_REF_NAME == "test") | + | - test |
deploy_prod: | deploy_prod: | ||
Line 151: | Line 433: | ||
tags: | tags: | ||
- openvpn1deploy | - openvpn1deploy | ||
- | rules: | + | only: |
- | - if: ($CI_COMMIT_BRANCH == "main" && $CI_COMMIT_REF_NAME == "main") | + | # - master |
+ | # - main | ||
</code> | </code> | ||
Line 158: | Line 441: | ||
* Технология Docker [[Технология Docker#Предоставление прав непривилегированным пользователям]] | * Технология Docker [[Технология Docker#Предоставление прав непривилегированным пользователям]] | ||
+ | |||
+ | * [[https://docs.gitlab.com/ee/ci/docker/using_docker_build.html|Use Docker to build Docker images]] | ||
+ | * [[https://docs.gitlab.com/ee/ci/variables/predefined_variables.html|Predefined variables reference]] | ||
+ | * [[https://docs.gitlab.com/ee/ci/variables/#add-a-cicd-variable-to-a-project|Add a CI/CD variable to a project]] | ||
+ | |||
<code> | <code> | ||
- | vagrant@node1:~/project1$ cat .gitlab-ci.yml | + | # Надо назначить в GitLab (Settings -> CI/CD -> Variables) |
+ | export MY_CI_REGISTRY=server.corpX.un:5000 | ||
+ | export MY_CI_REGISTRY_IMAGE=student/webd | ||
+ | # Можно использовать встроенные CI_REGISTRY и CI_REGISTRY_IMAGE | ||
+ | # поскольку используем этот же проект GitLab как registry | ||
+ | |||
+ | # в GitLab будет устанавлено автоматически | ||
+ | export CI_COMMIT_MESSAGE="ver 1.2" | ||
+ | </code> | ||
+ | |||
+ | <code> | ||
+ | gitlab-runner@server:~/webd$ cat build.sh | ||
</code><code> | </code><code> | ||
- | test1: | + | #!/bin/sh |
- | stage: test | + | |
+ | VER="$(echo $CI_COMMIT_MESSAGE | sed 's/[^a-zA-Z0-9\.]//g')" | ||
+ | |||
+ | # need only one time | ||
+ | # docker login -u $MY_CI_REGISTRY_USER -p $MY_CI_REGISTRY_PASSWORD $MY_CI_REGISTRY | ||
+ | # docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY | ||
+ | |||
+ | docker build -t webd webd | ||
+ | #docker build --no-cache -t webd webd | ||
+ | |||
+ | docker tag webd $MY_CI_REGISTRY/$MY_CI_REGISTRY_IMAGE:$VER | ||
+ | docker tag webd $MY_CI_REGISTRY/$MY_CI_REGISTRY_IMAGE | ||
+ | #docker tag webd $CI_REGISTRY_IMAGE:$VER | ||
+ | #docker tag webd $CI_REGISTRY_IMAGE | ||
+ | |||
+ | # previously need: docker login ... | ||
+ | |||
+ | docker push $MY_CI_REGISTRY/$MY_CI_REGISTRY_IMAGE:$VER | ||
+ | docker push $MY_CI_REGISTRY/$MY_CI_REGISTRY_IMAGE | ||
+ | #docker push $CI_REGISTRY_IMAGE:$VER | ||
+ | #docker push $CI_REGISTRY_IMAGE | ||
+ | </code><code> | ||
+ | gitlab-runner@server:~/webd$ cat .gitlab-ci.yml | ||
+ | </code><code> | ||
+ | stages: | ||
+ | - lintertest | ||
+ | - build | ||
+ | # - deploy | ||
+ | |||
+ | lintertest1: | ||
+ | stage: lintertest | ||
script: | script: | ||
- | - echo $(date) "Do a test here" >> /tmp/Bash.gitlab-ci.log | + | - echo $(date) "Do a test webd here" >> /tmp/Bash.gitlab-ci.log |
- shellcheck webd/webd | - shellcheck webd/webd | ||
tags: | tags: | ||
- shellcheck | - shellcheck | ||
- | deploy1: | + | build1: |
- | stage: deploy | + | stage: build |
script: | script: | ||
- | - echo $(date) "Do your deploy here" >> /tmp/Bash.gitlab-ci.log | + | - echo $(date) "Do a build webd here" >> /tmp/Bash.gitlab-ci.log |
- | - docker-compose up -d | + | # - env | tee /tmp/Bash.gitlab-ci.log |
+ | - sh build.sh | ||
tags: | tags: | ||
- | - webddeploy | + | - webdbuild |
+ | |||
+ | #deploy1: | ||
+ | # stage: deploy | ||
+ | # script: | ||
+ | # - echo $(date) "Do your deploy webd to k8s here" >> /tmp/Bash.gitlab-ci.log | ||
+ | # - sh deploy.sh | ||
+ | # tags: | ||
+ | # - webddeploy | ||
+ | |||
+ | ### OR .gitlab-ci.yml for gowebd-k8s project running from another pipeline ### | ||
+ | |||
+ | #deploy1: | ||
+ | # stage: deploy | ||
+ | # variables: | ||
+ | # HELM_NAMESPACE: "my-ns" | ||
+ | # rules: | ||
+ | # - if: '$CI_PIPELINE_SOURCE == "pipeline" && $VER' | ||
+ | # script: | ||
+ | # - env | ||
+ | # - envsubst < my-webd-deployment-env.yaml | kubectl apply -f - -n my-ns | ||
+ | # - helm upgrade -i my-webd webd-chart/ --set=image.tag=$VER --create-namespace | ||
</code> | </code> | ||
- | ==== Пример Kubernetes ==== | + | ==== Пример shell Kubernetes ==== |
<code> | <code> | ||
- | gitlab-runner@gate:~/webd$ cat build.sh | + | gitlab-runner@server:~/webd$ cp my-webd-deployment.yaml my-webd-deployment-env.yaml |
- | </code><code> | + | или |
- | #!/bin/sh | + | gitlab-runner@server:~/gowebd-k8s$ scp root@node1:my-webd-deployment.yaml my-webd-deployment-env.yaml |
- | VER="$(echo $CI_COMMIT_MESSAGE | sed 's/[^a-zA-Z0-9\.]//g')" | + | gitlab-runner@server:~/webd$ cat my-webd-deployment-env.yaml |
- | + | ||
- | docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD http://$CI_REGISTRY | + | |
- | docker build -t $CI_REGISTRY/$CI_REGISTRY_IMAGE:$VER webd | + | |
- | docker push $CI_REGISTRY/$CI_REGISTRY_IMAGE:$VER | + | |
- | </code><code> | + | |
- | gitlab-runner@gate:~/webd$ cat my-webd-deployment-env.yaml | + | |
</code><code> | </code><code> | ||
... | ... | ||
- | image: server.corp13.un:5000/student/webd:$VER | + | image: server.corpX.un:5000/student/webd:$VER |
+ | ... | ||
</code><code> | </code><code> | ||
+ | # в GitLab будет устанавлено автоматически | ||
+ | gitlab-runner@gate:~/webd$ export CI_COMMIT_MESSAGE="ver 1.2" | ||
+ | |||
gitlab-runner@gate:~/webd$ cat deploy.sh | gitlab-runner@gate:~/webd$ cat deploy.sh | ||
</code><code> | </code><code> | ||
#!/bin/sh | #!/bin/sh | ||
- | export VER="$(echo $CI_COMMIT_MESSAGE | sed 's/[^a-zA-Z0-9\.]//g')" | + | #alias kubectl='minikube kubectl --' |
+ | |||
+ | kubectl apply -f my-webd-deployment.yaml -n my-ns | ||
+ | |||
+ | #export VER="$(echo $CI_COMMIT_MESSAGE | sed 's/[^a-zA-Z0-9\.]//g')" | ||
+ | |||
+ | #envsubst < my-webd-deployment-env.yaml | kubectl apply -f - -n my-ns | ||
+ | |||
+ | kubectl apply -f my-webd-service.yaml -n my-ns | ||
- | #envsubst < my-webd-deployment-env.yaml | kubectl apply -f - | ||
- | #kubectl apply -f my-webd-service.yaml,my-webd-ingress.yaml | ||
#export HELM_NAMESPACE=my-ns | #export HELM_NAMESPACE=my-ns | ||
- | #helm test my-webd && helm upgrade my-webd webd-chart/ --set=image.tag=$VER || helm install my-webd webd-chart/ --set=image.tag=$VER | + | #helm upgrade --install my-webd webd-chart/ --set=image.tag=$VER --create-namespace |
+ | </code><code> | ||
+ | gitlab-runner@server:~/$ kubectl describe replicaset.apps/my-webd-NNNNNNNNNNN -n my-ns | ||
+ | </code> | ||
+ | ==== Пример CI с использованием контейнеров ==== | ||
+ | |||
+ | * [[https://akyriako.medium.com/build-golang-docker-images-with-gitlab-ci-pipelines-2117f8505350|Build Golang Docker images with GitLab CI Pipelines]] | ||
+ | * [[https://blog.callr.tech/building-docker-images-with-gitlab-ci-best-practices/|Best practices for building docker images with GitLab CI]] | ||
+ | |||
+ | * [[https://stackoverflow.com/questions/63693061/how-to-run-a-script-from-file-in-another-project-using-include-in-gitlab-ci|How to run a script from file in another project using include in GitLab CI?]] | ||
+ | * [[https://medium.com/@captain_sparrow/gitlab-%D1%82%D1%80%D0%B8%D0%B3%D0%B3%D0%B5%D1%80%D1%8B-%D0%B8-%D0%B4%D0%BB%D1%8F-%D0%BA%D0%B0%D0%BA%D0%B8%D1%85-%D1%82%D0%B5%D1%81%D1%82%D0%BE%D0%B2-%D0%B8%D1%85-%D1%81%D1%82%D0%BE%D0%B8%D1%82-%D0%B8%D1%81%D0%BF%D0%BE%D0%BB%D1%8C%D0%B7%D0%BE%D0%B2%D0%B0%D1%82%D1%8C-afa04f8c78a7|Gitlab триггеры и для каких тестов их стоит использовать?]] | ||
+ | |||
+ | * [[https://earthly.dev/blog/docker-vs-buildah-vs-kaniko/|Container Image Build Tools: Docker vs. Buildah vs. kaniko]] | ||
+ | * [[https://docs.gitlab.com/ee/ci/docker/using_kaniko.html|Use kaniko to build Docker images]] | ||
+ | * [[https://eng.d2iq.com/blog/a-tale-of-two-container-image-tools-skopeo-and-crane/|A Tale of Two Container Image Tools: Skopeo and Crane]] | ||
+ | |||
+ | <code> | ||
+ | student@client1:~/gowebd$ cat .gitlab-ci.yml | ||
</code><code> | </code><code> | ||
- | gitlab-runner@gate:~/webd$ cat .gitlab-ci.yml | ||
stages: | stages: | ||
- | - lintertest | ||
- build | - build | ||
- | - deploy | + | - push |
+ | # - deploy | ||
- | lintertest1: | + | #variables: |
- | stage: lintertest | + | # DOCKER_TLS_CERTDIR: "" |
- | script: | + | |
- | - echo $(date) "Do a test here" >> /tmp/Bash.gitlab-ci.log | + | |
- | - shellcheck webd/webd | + | |
- | tags: | + | |
- | - shellcheck | + | |
- | build1: | + | #services: |
+ | # - name: docker:dind | ||
+ | # command: | ||
+ | # [ | ||
+ | # '--insecure-registry=server.corpX.un:5000', | ||
+ | # ] | ||
+ | |||
+ | before_script: | ||
+ | - env | ||
+ | # - docker info | ||
+ | - echo -n $CI_REGISTRY_PASSWORD | docker login -u $CI_REGISTRY_USER --password-stdin $CI_REGISTRY | ||
+ | |||
+ | Build: | ||
stage: build | stage: build | ||
+ | # image: | ||
+ | # name: gcr.io/kaniko-project/executor:v1.9.0-debug | ||
+ | # entrypoint: [""] | ||
script: | script: | ||
- | - echo $(date) "Do a build here" >> /tmp/Bash.gitlab-ci.log | + | - docker pull $CI_REGISTRY_IMAGE:latest || true |
- | - ./build.sh | + | - > |
- | tags: | + | docker build |
- | - webddeploy | + | --pull |
+ | --cache-from $CI_REGISTRY_IMAGE:latest | ||
+ | --tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA | ||
+ | . | ||
+ | - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA | ||
- | deploy1: | + | # - echo "{\"auths\":{\"${CI_REGISTRY}\":{\"auth\":\"$(printf "%s:%s" "${CI_REGISTRY_USER}" "${CI_REGISTRY_PASSWORD}" | base64 | tr -d '\n')\"},\"$CI_DEPENDENCY_PROXY_SERVER\":{\"auth\":\"$(printf "%s:%s" ${CI_DEPENDENCY_PROXY_USER} "${CI_DEPENDENCY_PROXY_PASSWORD}" | base64 | tr -d '\n')\"}}}" > /kaniko/.docker/config.json |
- | stage: deploy | + | # - /kaniko/executor |
+ | # --insecure --skip-tls-verify | ||
+ | # --context "${CI_PROJECT_DIR}" | ||
+ | # --dockerfile "${CI_PROJECT_DIR}/Dockerfile" | ||
+ | # --destination "${CI_REGISTRY_IMAGE}:${CI_COMMIT_SHA}" | ||
+ | |||
+ | Push latest: | ||
+ | # image: | ||
+ | # name: gcr.io/go-containerregistry/crane:debug | ||
+ | # entrypoint: [""] | ||
+ | variables: | ||
+ | GIT_STRATEGY: none | ||
+ | stage: push | ||
+ | only: | ||
+ | - main | ||
script: | script: | ||
- | - echo $(date) "Do your deploy k8s here" >> /tmp/Bash.gitlab-ci.log | + | - docker pull $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA |
- | - ./deploy.sh | + | - docker tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA $CI_REGISTRY_IMAGE:latest |
- | tags: | + | - docker push $CI_REGISTRY_IMAGE:latest |
- | - webddeploy | + | |
+ | # - crane auth login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY | ||
+ | # - crane --insecure cp $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA $CI_REGISTRY_IMAGE:latest | ||
+ | |||
+ | Push tag: | ||
+ | # image: | ||
+ | # name: gcr.io/go-containerregistry/crane:debug | ||
+ | # entrypoint: [""] | ||
+ | variables: | ||
+ | GIT_STRATEGY: none | ||
+ | stage: push | ||
+ | only: | ||
+ | - tags | ||
+ | script: | ||
+ | - docker pull $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA | ||
+ | - docker tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_NAME | ||
+ | - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_NAME | ||
+ | |||
+ | # - crane auth login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY | ||
+ | # - crane --insecure cp $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_NAME | ||
+ | |||
+ | #Deploy: | ||
+ | # variables: | ||
+ | # VER: "$CI_COMMIT_REF_NAME" | ||
+ | # stage: deploy | ||
+ | # only: | ||
+ | # - tags | ||
+ | # trigger: | ||
+ | # project: student/gowebd-k8s | ||
</code> | </code> | ||
+ | ===== Сервер OpenID ===== | ||
- | ===== Сервер OAuth2 ===== | + | * [[https://github.com/zmartzone/mod_auth_openidc/wiki/GitLab-OAuth2]] |
+ | * [[Сервис HTTP#Управление доступом к HTTP серверу с использованием OpenID аутентификации]] | ||
+ | * Admin Area-> Applications | ||
- | * !!! URL без финального "/" !!! | + | <code> |
+ | Name: test-cgi | ||
+ | Redirect URI: http://gate.corp13.un/cgi-bin/test-cgi !!! Если URL каталога, то без финального "/" !!! | ||
+ | Trusted: Yes | ||
+ | Confidential: Yes | ||
+ | Scopes: openid | ||
+ | |||
+ | Application ID: ... | ||
+ | Secret: ... | ||
+ | Callback URL = Redirect URI | ||
+ | </code> | ||
+ | |||
+ | ===== Клиент OpenID ===== | ||
+ | |||
+ | * [[https://docs.gitlab.com/ee/administration/auth/oidc.html|You can use GitLab as a client application with OpenID Connect as an OmniAuth provider]] | ||
+ | * [[https://gitlab.com/gitlab-org/gitlab/-/issues/196193|use self-signed to integate gitlab with keycloak but see error: certificate verify failed (self signed certificate))]] | ||
+ | * [[https://forum.gitlab.com/t/using-keycloak-as-sso-for-gitlab-with-pre-existing-users-no-autocreate/67833|Using Keycloak as SSO for Gitlab with pre-existing users (no autocreate)]] | ||
<code> | <code> | ||
- | Admin Area-> Applications-> val-auth-test | + | # cp server.crt /etc/gitlab/trusted-certs/ |
- | Callback URL: https://val.bmstu.ru/auth-test | + | или |
- | Trusted: Yes | + | # cp ca.crt /etc/gitlab/trusted-certs/ |
+ | |||
+ | # cat /etc/gitlab/gitlab.rb | ||
+ | </code><code> | ||
+ | ... | ||
+ | gitlab_rails['omniauth_providers'] = [ | ||
+ | { | ||
+ | name: "openid_connect", # do not change this parameter | ||
+ | label: "Keycloak", # optional label for login button, defaults to "Openid Connect" | ||
+ | args: { | ||
+ | name: "openid_connect", | ||
+ | scope: ["openid", "profile", "email"], | ||
+ | response_type: "code", | ||
+ | # issuer: "https://keycloak.example.com/realms/myrealm", | ||
+ | issuer: "https://keycloak.corpX.un/realms/corpX", | ||
+ | client_auth_method: "query", | ||
+ | discovery: true, | ||
+ | uid_field: "preferred_username", | ||
+ | pkce: true, | ||
+ | client_options: { | ||
+ | # identifier: "<YOUR CLIENT ID>", | ||
+ | identifier: "any-client", | ||
+ | # secret: "<YOUR CLIENT SECRET>", | ||
+ | secret: "anystring", | ||
+ | # redirect_uri: "https://gitlab.example.com/users/auth/openid_connect/callback" | ||
+ | redirect_uri: "https://gate.corpX.un/users/auth/openid_connect/callback" | ||
+ | } | ||
+ | } | ||
+ | } | ||
+ | ] | ||
+ | ... | ||
</code> | </code> | ||
+ | |||
+ | * [[#Проверка конфигурации и перезапуск]] | ||
+ | * User -> Profile -> Account -> Select a service to sign in with -> Keycloak | ||
+ |