User Tools
сервис_victoriametrics
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
сервис_victoriametrics [2026/04/20 11:16] val |
сервис_victoriametrics [2026/05/25 09:24] (current) val [Подключение Grafana] |
||
|---|---|---|---|
| Line 4: | Line 4: | ||
| * [[https://docs.victoriametrics.com/sd_configs/|vmagent and single-node VictoriaMetrics supports the following Prometheus-compatible service discovery]] | * [[https://docs.victoriametrics.com/sd_configs/|vmagent and single-node VictoriaMetrics supports the following Prometheus-compatible service discovery]] | ||
| - | |||
| * [[https://docs.victoriametrics.com/guides/k8s-monitoring-via-vm-cluster/|Kubernetes monitoring with VictoriaMetrics Cluster]] | * [[https://docs.victoriametrics.com/guides/k8s-monitoring-via-vm-cluster/|Kubernetes monitoring with VictoriaMetrics Cluster]] | ||
| - | |||
| * [[https://docs.victoriametrics.com/guides/k8s-monitoring-via-vm-single/|Kubernetes monitoring via VictoriaMetrics Single]] | * [[https://docs.victoriametrics.com/guides/k8s-monitoring-via-vm-single/|Kubernetes monitoring via VictoriaMetrics Single]] | ||
| - | |||
| * [[https://docs.victoriametrics.com/scrape_config_examples/]] | * [[https://docs.victoriametrics.com/scrape_config_examples/]] | ||
| - | * [[Система Kubernetes#kube-state-metrics]] | + | * [[https://github.com/VictoriaMetrics/helm-charts/tags]] |
| + | |||
| + | * [[Система Kubernetes#kube-state-metrics]] собирает информацию о состоянии объектов внутри кластера Kubernetes (поды, узлы, deployments, namespaces) | ||
| <code> | <code> | ||
| (venv1) server# ansible all -f 4 -m apt -a 'pkg=prometheus-node-exporter state=present update_cache=true' -i /root/kubespray/inventory/mycluster/hosts.yaml | (venv1) server# ansible all -f 4 -m apt -a 'pkg=prometheus-node-exporter state=present update_cache=true' -i /root/kubespray/inventory/mycluster/hosts.yaml | ||
| Line 19: | Line 18: | ||
| kube1# helm repo add vm https://victoriametrics.github.io/helm-charts/ | kube1# helm repo add vm https://victoriametrics.github.io/helm-charts/ | ||
| kube1# helm repo update | kube1# helm repo update | ||
| + | kube1# helm search repo vm | ||
| - | kube1:~/vm# cat guide-vmsingle-values.yaml | + | kube1:~/vm# helm show values vm/victoria-metrics-single --version 0.35.0 > vmsingle-values.yaml |
| + | |||
| + | kube1:~/vm# cat vmsingle-values.yaml | ||
| </code><code> | </code><code> | ||
| ... | ... | ||
| + | size: 16Gi # replace to 6 | ||
| + | ... | ||
| + | scrape: | ||
| + | enabled: true | ||
| + | ... | ||
| + | # End of COPY | ||
| - job_name: kube-state-metrics | - job_name: kube-state-metrics | ||
| kubernetes_sd_configs: | kubernetes_sd_configs: | ||
| Line 40: | Line 48: | ||
| - kube2.corpX.un:9100 | - kube2.corpX.un:9100 | ||
| - kube3.corpX.un:9100 | - kube3.corpX.un:9100 | ||
| + | ... | ||
| </code><code> | </code><code> | ||
| - | kube1:~/vm# helm upgrade -i vmsingle vm/victoria-metrics-single -f guide-vmsingle-values.yaml -n vm --create-namespace | + | kube1:~/vm# helm upgrade -i vmsingle vm/victoria-metrics-single -f vmsingle-values.yaml -n vm --create-namespace --version 0.35.0 |
| - | + | ||
| - | kube1:~/vm# kubectl scale --replicas 1 statefulset vmsingle-victoria-metrics-single-server -n vm | + | |
| cmder> kubectl port-forward svc/vmsingle-victoria-metrics-single-server 8428 -n vm | cmder> kubectl port-forward svc/vmsingle-victoria-metrics-single-server 8428 -n vm | ||
| Line 56: | Line 63: | ||
| <code> | <code> | ||
| (venv1) server# ansible-playbook /root/conf/ansible/roles/mail.yml | (venv1) server# ansible-playbook /root/conf/ansible/roles/mail.yml | ||
| + | </code> | ||
| + | ==== Настройка ==== | ||
| + | <code> | ||
| + | kube1:~/vm# helm show values vm/victoria-metrics-alert --version 0.37.0 > vm-alert-values.yaml | ||
| - | kube1:~/vm# helm show values vm/victoria-metrics-alert > vm-alert-values.yaml | + | $ wget -qO - https://raw.githubusercontent.com/samber/awesome-prometheus-alerts/master/dist/rules/host-and-hardware/node-exporter.yml | sed 's/^/ /' |
| - | + | ||
| - | $ wget -qO - https://raw.githubusercontent.com/samber/awesome-prometheus-alerts/master/dist/rules/host-and-hardware/node-exporter.yml | sed 's/^/ /' | + | |
| - | $ wget -qO - https://raw.githubusercontent.com/samber/awesome-prometheus-alerts/master/dist/rules/kubernetes/kubestate-exporter.yml | sed 's/^/ /' | + | $ wget -qO - https://raw.githubusercontent.com/samber/awesome-prometheus-alerts/master/dist/rules/kubernetes/kubestate-exporter.yml | sed 's/^/ /' |
| kube1:~/vm# cat vm-alert-values.yaml | kube1:~/vm# cat vm-alert-values.yaml | ||
| Line 72: | Line 81: | ||
| # url: "http://vmsingle-victoria-metrics-single-server.vm.svc.cluster.local:8428" | # url: "http://vmsingle-victoria-metrics-single-server.vm.svc.cluster.local:8428" | ||
| # url: "http://vmcluster-victoria-metrics-cluster-vmselect.vm.svc.cluster.local:8481/select/0/prometheus/" | # url: "http://vmcluster-victoria-metrics-cluster-vmselect.vm.svc.cluster.local:8481/select/0/prometheus/" | ||
| - | ... | ||
| - | notifier: | ||
| - | alertmanager: | ||
| - | url: "http://alertmanager:9093" | ||
| ... | ... | ||
| config: | config: | ||
| Line 82: | Line 87: | ||
| groups: | groups: | ||
| - name: NodeExporter | - name: NodeExporter | ||
| + | ... | ||
| + | - alert: HostFilesystemDeviceError | ||
| + | expr: 'node_filesystem_device_error{fstype!~"^(fuse.*|tmpfs|cifs|nfs)", mountpoint!~".*/kubelet.*"} == 1' | ||
| ... | ... | ||
| - name: KubestateExporter | - name: KubestateExporter | ||
| - | rules: | ||
| - | - alert: KubernetesContainerOomKiller | ||
| - | expr: (kube_pod_container_status_restarts_total - kube_pod_container_status_restarts_total offset 10m >= 1) and ignoring (reason) min_over_time(kube_pod_container_status_last_terminated_reason{reason="OOMKilled"}[10m]) == 1 | ||
| - | for: 0m | ||
| - | labels: | ||
| - | severity: warning | ||
| - | annotations: | ||
| - | summary: Kubernetes Container oom killer (instance {{ $labels.instance }}) | ||
| - | description: "Container {{ $labels.container }} in pod {{ $labels.namespace }}/{{ $labels.pod }} has been OOMKill" | ||
| ... | ... | ||
| alertmanager: | alertmanager: | ||
| Line 117: | Line 116: | ||
| ... | ... | ||
| </code><code> | </code><code> | ||
| - | kube1:~/vm# helm upgrade -i vma vm/victoria-metrics-alert -f vm-alert-values.yaml -n vm | + | kube1:~/vm# helm upgrade -i vma vm/victoria-metrics-alert -f vm-alert-values.yaml -n vm --version 0.37.0 |
| + | </code> | ||
| + | ==== Отдадка ==== | ||
| + | <code> | ||
| kube1:~/vm# kubectl -n vm exec -ti pods/vma-victoria-metrics-alert-server-<TAB> -- sh | kube1:~/vm# kubectl -n vm exec -ti pods/vma-victoria-metrics-alert-server-<TAB> -- sh | ||
| </code><code> | </code><code> | ||
| + | / # ps|cat | ||
| + | ... | ||
| + | 1 root 0:00 /vmalert-prod --datasource.url=http://vmsingle-victoria-metrics-single-server:8428 --envflag.enable --envflag.prefix=VM_ --httpListenAddr=:8880 --loggerFormat=json --notifier.url=http://vma-victoria-metrics-alert-alertmanager.vm.svc.cluster.local.:9093 --rule=/config/alert-rules.yaml | ||
| + | ... | ||
| / # cat /config/alert-rules.yaml | / # cat /config/alert-rules.yaml | ||
| ... | ... | ||
| - | </code><code> | + | </code> |
| + | ==== Подключение ==== | ||
| + | <code> | ||
| cmder> kubectl -n vm port-forward svc/vma-victoria-metrics-alert-server 8880 | cmder> kubectl -n vm port-forward svc/vma-victoria-metrics-alert-server 8880 | ||
| Line 134: | Line 141: | ||
| </code> | </code> | ||
| + | ==== Тестирование ==== | ||
| + | |||
| + | * [[Управление сервисами в Linux#Управление юнитами Systemd]] systemctl reset-failed | ||
| * Используем [[Система Kubernetes#Базовые объекты k8s]] для стресс тестирования из [[Анализ производительности системы]] | * Используем [[Система Kubernetes#Базовые объекты k8s]] для стресс тестирования из [[Анализ производительности системы]] | ||
| * [[Команда dd]] для нехватки места на диске | * [[Команда dd]] для нехватки места на диске | ||
| Line 143: | Line 153: | ||
| <code> | <code> | ||
| - | kube1:~/vm# helm show values vm/victoria-logs-single > vls-values.yaml | + | kube1:~/vm# helm show values vm/victoria-logs-single --version 0.12.0 > vls-values.yaml |
| kube1:~/vm# cat vls-values.yaml | kube1:~/vm# cat vls-values.yaml | ||
| </code><code> | </code><code> | ||
| + | ... | ||
| + | size: 10Gi # replace to 5 | ||
| ... | ... | ||
| vector: | vector: | ||
| Line 153: | Line 165: | ||
| ... | ... | ||
| </code><code> | </code><code> | ||
| - | kube1:~/vm# helm upgrade -i vls oci://ghcr.io/victoriametrics/helm-charts/victoria-logs-single -f vls-values.yaml -n vm | + | kube1:~/vm# helm upgrade -i vls vm/victoria-logs-single -f vls-values.yaml -n vm --version 0.12.0 |
| cmder$ kubectl port-forward svc/vls-victoria-logs-single-server 9428 -n vm | cmder$ kubectl port-forward svc/vls-victoria-logs-single-server 9428 -n vm | ||
| Line 169: | Line 181: | ||
| http://vls-victoria-logs-single-server:9428 -> | http://vls-victoria-logs-single-server:9428 -> | ||
| - | Explore -> LogQL: _time:5m | + | Explore -> LogsQL: kubernetes.pod_labels.app: "my-debian" _time: 5m |
| - | + | ||
| - | Builder-> Filter: kubernetes.pod_name = my-debian | + | |
| </code> | </code> | ||
сервис_victoriametrics.1776673006.txt.gz · Last modified: 2026/04/20 11:16 by val
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
