Differences

This shows you the differences between two versions of the page.

--- инструмент_gitlab [2023/09/17 07:02]
val [Пример shell docker]
+++ инструмент_gitlab [2025/03/18 09:14] (current)
val [Пример shell Kubernetes]
@@ Line 10: / Line 10: @@
   * RAM от 4Gb
-==== Если нужен почтовый сервер ====
-<code>
-server# time ansible-playbook conf/ansible/roles/mail.yml
-real    2m57.922s
-# cat /etc/apache2/ports.conf
-</code><code>
-...
-Listen 81
-...
-</code><code>
-server# service apache2 restart
-</code><code>
-http://server.corpX.un:81/mail/
-</code>
 ==== Установка из репозитория ====
@@ Line 31: / Line 16: @@
   * [[https://about.gitlab.com/install/|Install self-managed GitLab]]
   * Доступно из РФ: [[https://packages.gitlab.com/gitlab/gitlab-ce]]
-  * [[http://gate.isp.un/unix/Git/gitlab-ce_16.3.3-ce.0_amd64.deb]]
 <code>
@@ Line 38: / Line 22: @@
 server# curl https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.deb.sh | bash
-server# time EXTERNAL_URL="http://server.corpX.un" apt-get install gitlab-ce
+server# time EXTERNAL_URL="http://$(hostname)" apt-get install gitlab-ce
 ...
-real    38m49.787s
+real    38m49.787s  !!! Загрузка может прерываться, надо повторять команду !!!
 ..
 </code>
@@ Line 47: / Line 31: @@
   * [[https://docs.gitlab.com/ee/install/docker.html#install-gitlab-using-docker-compose|Install GitLab using Docker Compose]]
+  * [[https://hub.docker.com/r/gitlab/gitlab-ce/tags/|gitlab/gitlab-ce tags (versions)]]
   * [[Технология Docker]]
   * [[Технология Docker#docker-compose]]
@@ Line 57: / Line 43: @@
   web:
     image: 'gitlab/gitlab-ce:latest'
+#    image: 'gitlab/gitlab-ce:16.7.4-ce.0'
     restart: always
     hostname: 'server.corpX.un'
@@ Line 62: / Line 49: @@
       GITLAB_ROOT_PASSWORD: "strongpassword"
       GITLAB_OMNIBUS_CONFIG: |
-        external_url 'http://server.corpX.un'
+        prometheus_monitoring['enable'] = false
-#        gitlab_rails['initial_root_password'] = 'strongpassword'
+        gitlab_rails['registry_enabled'] = true
-#        registry_external_url 'http://server.corpX.un'
+        gitlab_rails['registry_host'] = "server.corpX.un"
-#        gitlab_rails['registry_enabled'] = true
+        external_url 'https://server.corpX.un'
-#        gitlab_rails['registry_host'] = "server.corpX.un"
+        registry_external_url 'https://server.corpX.un:5000'
-#        gitlab_rails['registry_port'] = "5000"
+        gitlab_rails['registry_port'] = "5050"
-#        registry['registry_http_addr'] = "server.corpX.un:5000"
+        registry['registry_http_addr'] = "server.corpX.un:5050"
-#        prometheus_monitoring['enable'] = false
     ports:
-      - '80:80'
+      - '443:443'
-#      - '443:443'
       - '2222:22'
       - '5000:5000'
-      - '5050:5050'
     volumes:
       - '/etc/gitlab:/etc/gitlab'
@@ Line 81: / Line 65: @@
       - '/srv/gitlab/data:/var/opt/gitlab'
     shm_size: '256m'
+    logging:
+      driver: "json-file"
+      options:
+        max-size: "2048m"
 </code><code>
 # docker-compose up -d
@@ Line 89: / Line 77: @@
 ### rm -r /srv/gitlab/ /etc/gitlab/
 </code>
+==== Установка через Ansible Role ====
+  * [[https://galaxy.ansible.com/ui/repo/published/hifis/toolkit/content/role/gitlab/]]
 ===== Подключение =====
@@ Line 102: / Line 93: @@
   * [[https://stackoverflow.com/questions/56943327/how-to-download-a-single-file-from-gitlab|How to download a single file from GitLab?]]
 <code>
+root@node1,2,3:~# curl "http://server.corpX.un/api/v4/projects/2/repository/files/docker-compose.yml/raw" | tee docker-compose.yml
+  или, для НЕ публичных проектов
 root@node1,2,3:~# curl --header "PRIVATE-TOKEN: NNNNNNNNNNNNNNNNNNNNN" "http://server.corpX.un/api/v4/projects/4/repository/files/docker-compose.yml/raw?ref=master" | tee docker-compose.yml
 </code>
@@ Line 113: / Line 107: @@
 sudo -i
-export BR=main; bash <(curl -s http://gate.corp13.un/api/v4/projects/2/repository/files/start.sh/raw?ref=$BR)
+export BR=main; bash <(curl -s http://gate.corp13.un/api/v4/projects/1/repository/files/start.sh/raw?ref=$BR)
 </code>
 ===== Настройка =====
@@ Line 188: / Line 182: @@
 <code>
+mkdir -p /etc/gitlab/ssl/
+cp wild.crt -v /etc/gitlab/ssl/$(hostname).crt
+cp wild.key -v /etc/gitlab/ssl/$(hostname).key
+# cat /etc/gitlab/gitlab.rb
+</code><code>
 ...
 external_url 'https://server.corpX.un'
 ...
-nginx['ssl_certificate'] = "/etc/gitlab/ssl/server.crt"
+# nginx['ssl_certificate'] = "/etc/gitlab/ssl/#{node['fqdn']}.crt"
-nginx['ssl_certificate_key'] = "/etc/gitlab/ssl/server.key"
+# nginx['ssl_certificate_key'] = "/etc/gitlab/ssl/#{node['fqdn']}.key"
 ...
 letsencrypt['enable'] = false
 ...
 </code>
+  * [[#Проверка конфигурации и перезапуск]]
 ==== Управление пользователями ====
@@ Line 230: / Line 233: @@
     label: 'LDAP'
     host: 'server.corpX.un'
+#    host: 'server2.corpX.un'
     port: 389
-    uid: 'uid'
+#    uid: 'uid'
-#    uid: 'sAMAccountName'
+    uid: 'sAMAccountName'
-    bind_dn: 'cn=admin,dc=corpX,dc=un'
+#    bind_dn: 'cn=admin,dc=corpX,dc=un'
-    password: 'secret'
+#    password: 'secret'
-#    bind_dn: 'cn=Administrator,cn=Users,dc=corpX,dc=un'
+    bind_dn: 'cn=Administrator,cn=Users,dc=corpX,dc=un'
-#    password: 'Pa$$w0rd'
+    password: 'Pa$$w0rd'
     encryption: 'plain'
-    active_directory: false
+#    active_directory: false
-#    active_directory: true
+    active_directory: true
     base: 'dc=corpX,dc=un'
-...
 EOS
 ...
@@ Line 249: / Line 252: @@
 ===== GitLab Runner =====
+  * [[https://stackoverflow.com/questions/32933174/use-gitlab-ci-to-run-tests-locally|Use GitLab CI to run tests locally?]]
 ==== Установка из пакета ====
   * [[https://docs.gitlab.com/runner/install/linux-manually.html|Install GitLab Runner manually on GNU/Linux]]
-  * [[https://val.bmstu.ru/unix/Git/gitlab-runner_amd64.deb]] (15.0.0)
+  * [[https://val.bmstu.ru/unix/Git/gitlab-runner_amd64.deb]] (16.10.0)
 <code>
-# apt install gitlab-runner   # достаточно для shell executor но не отображает команды ci/cd в gitlab
+# wget http://gate.isp.un/unix/Git/gitlab-runner_amd64.deb               # Version: 17.3.1-1
-или
+##2 часа## curl -LJO "https://gitlab-runner-downloads.s3.amazonaws.com/latest/deb/gitlab-runner_amd64.deb"
-# wget http://gate.isp.un/unix/Git/gitlab-runner_amd64.deb
-##2 часа## curl -LJO "https://gitlab-runner-downloads.s3.amazonaws.com/latest/deb/gitlab-runner_amd64.deb"
 # dpkg -i gitlab-runner_amd64.deb
 </code>
@@ Line 276: / Line 278: @@
 Enter tags for the runner: dhcptest, dhcpdeploy
   или
-Enter tags for the runner: openvpn1deploy
+Enter tags for the runner: openvpn1deploy   или     ansible
 ...
 Enter an executor: shell
@@ Line 285: / Line 287: @@
 <code>
-# gitlab-runner register -n --executor "shell" -u http://server.corp13.un -r "NNNNNNNNNNNNNNNNNNNNNNNNNNNN"
+# gitlab-runner register -n --executor "shell" -u http://server.corpX.un -r "NNNNNNNNNNNNNNNNNNNNNNNNNNNN"
 </code>
+или по инструкции в "New instance runner"
+<code>
+# gitlab-runner register -n --executor "shell" -u http://server.corpX.un -t "NNNNNNNNNNNNNNNNNNNNNNNNNNNN"
+</code>
 Перезапускать не нужно
+<code>
+# gitlab-runner verify
+</code>
 <code>
@@ Line 296: / Line 306: @@
 </code><code>
 # systemctl restart gitlab-runner
-# gitlab-runner verify
 </code>
@@ Line 305: / Line 313: @@
 <code>
+gate:~### docker stop gitlab-runner; docker rm gitlab-runner
 gate:~### rm /srv/gitlab-runner/config/config.toml
@@ Line 336: / Line 345: @@
   --non-interactive \
   --url "http://server.corpX.un/" \
-  --registration-token "NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN" \
+  --token "NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN" \
   --executor "docker" \
   --docker-image "docker:stable" \
@@ Line 347: / Line 356: @@
 </code>
+=== TLS для DooD и DinD ===
+<code>
+# cp wild.crt /srv/gitlab-runner/config/
+docker run --rm -v /srv/gitlab-runner/config:/etc/gitlab-runner gitlab/gitlab-runner register \
+...
+  --url "https://server.corpX.un/" \
+  --tls-ca-file "/etc/gitlab-runner/wild.crt" \
+...
+</code>
 ===== GitLab CI/CD =====
@@ Line 396: / Line 414: @@
 <code>
 Administrator@Ra-master ~/openvpn1 (test)
 λ touch .gitlab-ci.yml
+  или
+Build -> Pipeline editor -> Configure Pipelines
 </code><code>
+stages:
+  - deploy
 deploy_test:
   stage: deploy
   script:
-    - echo $(date) "Deploy TEST openvpn1" >> /tmp/Bash.gitlab-ci.log
     - ansible-playbook openvpn1.yaml -i inventory.yaml -e "variable_host=test_nodes"
   tags:
     - openvpn1deploy
+#    - ansible
   only:
     - test
@@ Line 412: / Line 434: @@
   stage: deploy
   script:
-    - echo $(date) "Deploy PROD openvpn1" >> /tmp/Bash.gitlab-ci.log
     - ansible-playbook openvpn1.yaml -i inventory.yaml
   tags:
     - openvpn1deploy
+#    - ansible
   only:
 #    - master
@@ Line 431: / Line 453: @@
 <code>
-# Надо назначить в GitLab (Settings -> CI/CD -> Variables)
+# Можно назначить в GitLab (Settings -> CI/CD -> Variables)
-export MY_CI_REGISTRY=server.corpX.un:5000
+# export MY_CI_REGISTRY=server.corpX.un:5000
-export MY_CI_REGISTRY_IMAGE=student/webd
+# export MY_CI_REGISTRY_IMAGE=student/webd
-# Можно использовать встроенные CI_REGISTRY и CI_REGISTRY_IMAGE
+# или использовать встроенные CI_REGISTRY и CI_REGISTRY_IMAGE
-# поскольку используем этот же проект GitLab как registry
+# поскольку используем этот же проект GitLab как Registry
-# в GitLab будет устанавлено автоматически
+# в GitLab будет установлено автоматически после git commit -m "ver 1.2" и git push
-export CI_COMMIT_MESSAGE="ver 1.2"
+# export CI_COMMIT_MESSAGE="ver 1.2"
 </code>
@@ Line 448: / Line 470: @@
 VER="$(echo $CI_COMMIT_MESSAGE | sed 's/[^a-zA-Z0-9\.]//g')"
-# need only one time
+# needed once
 # docker login -u $MY_CI_REGISTRY_USER -p $MY_CI_REGISTRY_PASSWORD $MY_CI_REGISTRY
 # docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
 docker build -t webd webd
-#docker build --no-cache -t webd webd
-docker tag webd $MY_CI_REGISTRY/$MY_CI_REGISTRY_IMAGE:$VER
+#docker run --rm -e MYMODE=TEST webd || exit 1
-docker tag webd $MY_CI_REGISTRY/$MY_CI_REGISTRY_IMAGE
-#docker tag webd $CI_REGISTRY_IMAGE:$VER
+#docker tag webd $MY_CI_REGISTRY/$MY_CI_REGISTRY_IMAGE:$VER
-#docker tag webd $CI_REGISTRY_IMAGE
+#docker tag webd $MY_CI_REGISTRY/$MY_CI_REGISTRY_IMAGE
+docker tag webd $CI_REGISTRY_IMAGE:$VER
+docker tag webd $CI_REGISTRY_IMAGE
 # previously need: docker login ...
-docker push $MY_CI_REGISTRY/$MY_CI_REGISTRY_IMAGE:$VER
+#docker push $MY_CI_REGISTRY/$MY_CI_REGISTRY_IMAGE:$VER
-docker push $MY_CI_REGISTRY/$MY_CI_REGISTRY_IMAGE
+#docker push $MY_CI_REGISTRY/$MY_CI_REGISTRY_IMAGE
-#docker push $CI_REGISTRY_IMAGE:$VER
+docker push $CI_REGISTRY_IMAGE:$VER
-#docker push $CI_REGISTRY_IMAGE
+docker push $CI_REGISTRY_IMAGE
-</code><code>
-gitlab-runner@server:~/webd$ sh build.sh
-...
-Step...
- ---> Using cache
-...
-..............: Pushed
-...
-..............: Layer already exists
-...
-latest: digest...
 </code><code>
 gitlab-runner@server:~/webd$ cat .gitlab-ci.yml
@@ Line 488: / Line 500: @@
   stage: lintertest
   script:
-    - echo $(date) "Do a test webd here" >> /tmp/Bash.gitlab-ci.log
+#    - echo $(date) "Do a test webd here" >> /tmp/Bash.gitlab-ci.log
     - shellcheck webd/webd
   tags:
@@ Line 496: / Line 508: @@
   stage: build
   script:
-    - echo $(date) "Do a build webd here" >> /tmp/Bash.gitlab-ci.log
+#    - echo $(date) "Do a build webd here" >> /tmp/Bash.gitlab-ci.log
-#    - env | tee /tmp/Bash.gitlab-ci.log
+#    - env | tee -a /tmp/Bash.gitlab-ci.log
     - sh build.sh
   tags:
@@ Line 505: / Line 517: @@
 #  stage: deploy
 #  script:
-#    - echo $(date) "Do your deploy webd to k8s here" >> /tmp/Bash.gitlab-ci.log
 #    - sh deploy.sh
 #  tags:
@@ Line 526: / Line 537: @@
 ==== Пример shell Kubernetes ====
+<code>
+kube1:~/gowebd-k8s# cat .gitlab-ci.yml
+</code><code>
+stages:
+  - deploy
+#variables:
+#  HELM_NAMESPACE: "my-ns"
+trigger-deploy:
+  stage: deploy
+  rules:
+    - if: '$CI_PIPELINE_SOURCE == "pipeline" && $VER'
+  script:
+    - env
+    - envsubst < my-webd-deployment-env.yaml | kubectl apply -f - -n my-ns
+#    - helm upgrade -i my-webd webd-chart/ --set=image.tag=$VER --create-namespace
+  tags:
+    - k8s-deploy
+manual-deploy:
+  stage: deploy
+  when: manual
+  variables:
+    VER: "$MY_WEBD_VER"  # New Pipeline or Settings->CI/CD->Variables
+  script:
+    - env
+    - envsubst < my-webd-deployment-env.yaml | kubectl apply -f - -n my-ns
+#    - helm upgrade -i my-webd webd-chart/ --set=image.tag=$VER --create-namespace
+  tags:
+    - k8s-deploy
+</code>
+== старый вариант ==
 <code>
 gitlab-runner@server:~/webd$ cp my-webd-deployment.yaml my-webd-deployment-env.yaml
@@ Line 543: / Line 588: @@
 </code><code>
 #!/bin/sh
+#alias kubectl='minikube kubectl --'
 kubectl apply -f my-webd-deployment.yaml -n my-ns
@@ Line 555: / Line 602: @@
 #export HELM_NAMESPACE=my-ns
 #helm upgrade --install my-webd webd-chart/ --set=image.tag=$VER --create-namespace
+</code><code>
+gitlab-runner@server:~/$ kubectl describe replicaset.apps/my-webd-NNNNNNNNNNN -n my-ns
 </code>
 ==== Пример CI с использованием контейнеров ====
@@ Line 573: / Line 621: @@
 </code><code>
 stages:
+#  - lint
   - build
+#  - test
   - push
 #  - deploy
@@ Line 584: / Line 634: @@
 #    command:
 #      [
-#        '--insecure-registry=server.corp13.un:5000',
+#        '--insecure-registry=server.corpX.un:5000',
 #      ]
@@ Line 607: / Line 657: @@
     - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
-#    - echo "{\"auths\":{\"${CI_REGISTRY}\":{\"auth\":\"$(printf "%s:%s" "${CI_REGISTRY_USER}" "${CI_REGISTRY_PASSWORD}" | base64 | tr -d '\n')\"},\"$CI_DEPENDENCY_PROXY_SERVER\":{\"auth\":\"$(printf "%s:%s" ${CI_DEPENDENCY_PROXY_USER} "${CI_DEPENDENCY_PROXY_PASSWORD}" | base64 | tr -d '\n')\"}}}" > /kaniko/.docker/config.json
+#    - echo "{\"auths\":{\"${CI_REGISTRY}\":{\"auth\":\"$(printf "%s:%s" "${CI_REGISTRY_USER}" "${CI_REGISTRY_PASSWORD}" | base64 | tr -d '\n')\"}}}" > /kaniko/.docker/config.json
 #    - /kaniko/executor
 #      --insecure --skip-tls-verify
@@ Line 613: / Line 663: @@
 #      --dockerfile "${CI_PROJECT_DIR}/Dockerfile"
 #      --destination "${CI_REGISTRY_IMAGE}:${CI_COMMIT_SHA}"
+#  except:
+#    - tags
+#  tags:
+#    - build
 Push latest:
@@ Line 630: / Line 685: @@
 #    - crane auth login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
 #    - crane --insecure cp $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA $CI_REGISTRY_IMAGE:latest
+#  tags:
+#    - build
 Push tag:
@@ Line 647: / Line 704: @@
 #   - crane auth login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
 #   - crane --insecure cp $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_NAME
+#  tags:
+#    - build
+### Not work in DooD
+#Lint test:
+#  stage: lint
+#  script:
+#    - pwd
+#    - docker run --rm -v $(pwd):/app -w /app golangci/golangci-lint:v1.62.2 golangci-lint run --timeout=10m
+#  except:
+#    - tags
+#Smoke test:
+#  stage: test
+#  script:
+#    - MY_ID=$(docker run -d --rm $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA)
+#    - MY_IP=$(docker inspect -f '{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}' $MY_ID)
+#    - docker run --rm alpine/curl -sS $MY_IP
+#    - docker stop $MY_ID
+#  except:
+#    - tags
 #Deploy:
@@ Line 657: / Line 736: @@
 #    project: student/gowebd-k8s
 </code>
-===== Сервер OAuth2 =====
+===== Сервер OpenID =====
   * [[https://github.com/zmartzone/mod_auth_openidc/wiki/GitLab-OAuth2]]
-  * [[Сервис HTTP#Управление доступом к HTTP серверу с использованием OAuth2 аутентификации]]
+  * [[Сервис HTTP#Управление доступом к HTTP серверу с использованием OpenID аутентификации]]
   * Admin Area-> Applications
@@ Line 666: / Line 745: @@
 Name: test-cgi
 Redirect URI: http://gate.corp13.un/cgi-bin/test-cgi  !!! Если URL каталога, то без финального "/" !!!
-Trusted: yes
+Trusted: Yes
-Confidential: ?
+Confidential: Yes
-Scopes: ?
+Scopes: openid
 Application ID: ...
@@ Line 675: / Line 754: @@
 </code>
+===== Клиент OpenID =====
+  * [[https://docs.gitlab.com/ee/administration/auth/oidc.html|You can use GitLab as a client application with OpenID Connect as an OmniAuth provider]]
+  * [[https://gitlab.com/gitlab-org/gitlab/-/issues/196193|use self-signed to integate gitlab with keycloak but see error: certificate verify failed (self signed certificate))]]
+  * [[https://forum.gitlab.com/t/using-keycloak-as-sso-for-gitlab-with-pre-existing-users-no-autocreate/67833|Using Keycloak as SSO for Gitlab with pre-existing users (no autocreate)]]
+<code>
+# cp server.crt /etc/gitlab/trusted-certs/
+  или
+# cp ca.crt /etc/gitlab/trusted-certs/
+# cat /etc/gitlab/gitlab.rb
+</code><code>
+...
+gitlab_rails['omniauth_providers'] = [
+  {
+    name: "openid_connect", # do not change this parameter
+    label: "Keycloak", # optional label for login button, defaults to "Openid Connect"
+    args: {
+      name: "openid_connect",
+      scope: ["openid", "profile", "email"],
+      response_type: "code",
+#     issuer:  "https://keycloak.example.com/realms/myrealm",
+      issuer:  "https://keycloak.corpX.un/realms/corpX",
+      client_auth_method: "query",
+      discovery: true,
+      uid_field: "preferred_username",
+      pkce: true,
+      client_options: {
+#        identifier: "<YOUR CLIENT ID>",
+        identifier: "any-client",
+#        secret: "<YOUR CLIENT SECRET>",
+        secret: "anystring",
+#        redirect_uri: "https://gitlab.example.com/users/auth/openid_connect/callback"
+        redirect_uri: "https://gate.corpX.un/users/auth/openid_connect/callback"
+      }
+    }
+  }
+]
+...
+</code>
+  * [[#Проверка конфигурации и перезапуск]]
+  * User -> Profile -> Account -> Select a service to sign in with -> Keycloak

Методические материалы Лохтурова Вячеслава

User Tools

Site Tools

Differences

Page Tools