This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
сервис_victoriametrics [2026/05/23 08:09] val [Метрики] |
сервис_victoriametrics [2026/06/10 15:38] (current) val [Подключение Grafana] |
||
|---|---|---|---|
| Line 10: | Line 10: | ||
| * [[https://github.com/VictoriaMetrics/helm-charts/tags]] | * [[https://github.com/VictoriaMetrics/helm-charts/tags]] | ||
| - | * [[Система Kubernetes#kube-state-metrics]] | + | kube-state-metrics |
| + | |||
| + | * [[Система Kubernetes#kube-state-metrics]] собирает информацию о состоянии объектов внутри кластера Kubernetes (поды, deployments, namespaces) | ||
| + | |||
| + | node-exporter | ||
| + | |||
| + | * prometheus-node-exporter собирает информацию о состоянии узлов кластера | ||
| <code> | <code> | ||
| (venv1) server# ansible all -f 4 -m apt -a 'pkg=prometheus-node-exporter state=present update_cache=true' -i /root/kubespray/inventory/mycluster/hosts.yaml | (venv1) server# ansible all -f 4 -m apt -a 'pkg=prometheus-node-exporter state=present update_cache=true' -i /root/kubespray/inventory/mycluster/hosts.yaml | ||
| root@9f823e41e1c4:/kubespray# ansible all -f 4 -m apt -a 'pkg=prometheus-node-exporter state=present update_cache=true' -i /inventory/inventory.ini | root@9f823e41e1c4:/kubespray# ansible all -f 4 -m apt -a 'pkg=prometheus-node-exporter state=present update_cache=true' -i /inventory/inventory.ini | ||
| + | </code> | ||
| + | victoriametrics | ||
| + | |||
| + | <code> | ||
| kube1# helm repo add vm https://victoriametrics.github.io/helm-charts/ | kube1# helm repo add vm https://victoriametrics.github.io/helm-charts/ | ||
| kube1# helm repo update | kube1# helm repo update | ||
| Line 25: | Line 35: | ||
| </code><code> | </code><code> | ||
| ... | ... | ||
| - | size: 16Gi #replace to 6 | + | size: 16Gi # replace to 6 |
| ... | ... | ||
| scrape: | scrape: | ||
| Line 54: | Line 64: | ||
| cmder> kubectl port-forward svc/vmsingle-victoria-metrics-single-server 8428 -n vm | cmder> kubectl port-forward svc/vmsingle-victoria-metrics-single-server 8428 -n vm | ||
| </code> | </code> | ||
| + | |||
| + | ===== Визуализация ===== | ||
| * Сервис Grafana в [[Сервис Grafana#Kubernetes]] | * Сервис Grafana в [[Сервис Grafana#Kubernetes]] | ||
| Line 60: | Line 72: | ||
| * [[Сервис Prometheus#prometheus-alertmanager]] (ссылки на правила) | * [[Сервис Prometheus#prometheus-alertmanager]] (ссылки на правила) | ||
| + | * [[Web интерфейс к почте#Roundcube]] | ||
| <code> | <code> | ||
| (venv1) server# ansible-playbook /root/conf/ansible/roles/mail.yml | (venv1) server# ansible-playbook /root/conf/ansible/roles/mail.yml | ||
| + | </code> | ||
| + | ==== Настройка ==== | ||
| + | <code> | ||
| kube1:~/vm# helm show values vm/victoria-metrics-alert --version 0.37.0 > vm-alert-values.yaml | kube1:~/vm# helm show values vm/victoria-metrics-alert --version 0.37.0 > vm-alert-values.yaml | ||
| Line 90: | Line 104: | ||
| ... | ... | ||
| - name: KubestateExporter | - name: KubestateExporter | ||
| + | ... | ||
| + | # -- Vertical Pod Autoscaler | ||
| ... | ... | ||
| alertmanager: | alertmanager: | ||
| Line 112: | Line 128: | ||
| - to: 'student@corpX.un' | - to: 'student@corpX.un' | ||
| send_resolved: true | send_resolved: true | ||
| + | |||
| + | # -- Alertmanager extra templates | ||
| ... | ... | ||
| </code><code> | </code><code> | ||
| - | kube1:~/vm# helm upgrade -i vma vm/victoria-metrics-alert -f vm-alert-values.yaml -n vm --version 0.37.0 | + | kube1:~/vm# helm template vma vm/victoria-metrics-alert -f vm-alert-values.yaml -n vm --version 0.37.0 |
| + | kube1:~/vm# helm upgrade -i vma vm/victoria-metrics-alert -f vm-alert-values.yaml -n vm --version 0.37.0 | ||
| + | </code> | ||
| + | ==== Отдадка ==== | ||
| + | <code> | ||
| kube1:~/vm# kubectl -n vm exec -ti pods/vma-victoria-metrics-alert-server-<TAB> -- sh | kube1:~/vm# kubectl -n vm exec -ti pods/vma-victoria-metrics-alert-server-<TAB> -- sh | ||
| </code><code> | </code><code> | ||
| Line 122: | Line 144: | ||
| 1 root 0:00 /vmalert-prod --datasource.url=http://vmsingle-victoria-metrics-single-server:8428 --envflag.enable --envflag.prefix=VM_ --httpListenAddr=:8880 --loggerFormat=json --notifier.url=http://vma-victoria-metrics-alert-alertmanager.vm.svc.cluster.local.:9093 --rule=/config/alert-rules.yaml | 1 root 0:00 /vmalert-prod --datasource.url=http://vmsingle-victoria-metrics-single-server:8428 --envflag.enable --envflag.prefix=VM_ --httpListenAddr=:8880 --loggerFormat=json --notifier.url=http://vma-victoria-metrics-alert-alertmanager.vm.svc.cluster.local.:9093 --rule=/config/alert-rules.yaml | ||
| ... | ... | ||
| + | |||
| / # cat /config/alert-rules.yaml | / # cat /config/alert-rules.yaml | ||
| ... | ... | ||
| - | </code><code> | ||
| - | cmder> kubectl -n vm port-forward svc/vma-victoria-metrics-alert-server 8880 | ||
| kube1:~/vm# kubectl -n vm exec -ti pods/vma-victoria-metrics-alert-alertmanager-<TAB> -- sh | kube1:~/vm# kubectl -n vm exec -ti pods/vma-victoria-metrics-alert-alertmanager-<TAB> -- sh | ||
| Line 131: | Line 152: | ||
| /alertmanager $ cat /config/alertmanager.yaml | /alertmanager $ cat /config/alertmanager.yaml | ||
| ... | ... | ||
| - | </code><code> | + | </code> |
| + | ==== Подключение ==== | ||
| + | <code> | ||
| + | cmder> kubectl -n vm port-forward svc/vma-victoria-metrics-alert-server 8880 | ||
| cmder> kubectl -n vm port-forward svc/vma-victoria-metrics-alert-alertmanager 9093 | cmder> kubectl -n vm port-forward svc/vma-victoria-metrics-alert-alertmanager 9093 | ||
| </code> | </code> | ||
| - | * Используем [[Система Kubernetes#Базовые объекты k8s]] для стресс тестирования из [[Анализ производительности системы]] | + | ==== Тестирование ==== |
| - | * [[Команда dd]] для нехватки места на диске | + | |
| + | * [[Управление сервисами в Linux#Управление юнитами Systemd]] systemctl reset-failed | ||
| + | * Используем [[Система Kubernetes#Manifest]] развертывания с последующим стресс тестированием из [[Анализ производительности системы]] | ||
| + | * [[Команда dd]] для воспроизведения нехватки места на диске | ||
| ===== Журналы ===== | ===== Журналы ===== | ||
| Line 149: | Line 177: | ||
| </code><code> | </code><code> | ||
| ... | ... | ||
| - | size: 10Gi | + | size: 10Gi # replace to 5 |
| ... | ... | ||
| vector: | vector: | ||
| Line 167: | Line 195: | ||
| <code> | <code> | ||
| Grafana -> Connections -> | Grafana -> Connections -> | ||
| - | Add new connection: VictoriaLogs -> | + | Add new connection: VictoriaLogs |
| - | + | ||
| - | Install -> New Datasources | + | |
| - | http://vls-victoria-logs-single-server:9428 -> | + | |
| - | Explore -> LogQL: _time:5m | + | Install -> New Datasources -> VictoriaLogs |
| + | URL: http://vls-victoria-logs-single-server:9428 | ||
| - | Builder-> Filter: kubernetes.pod_name = my-debian | + | Explore -> victoriametrics-logs-datasources |
| + | Code (LogsQL): kubernetes.pod_labels.app: "my-debian" _time: 5m | ||
| </code> | </code> | ||