User Tools
решение_ceph
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
решение_ceph [2025/04/06 11:10] val [Решение Ceph] |
решение_ceph [2026/10/19 19:27] (current) val [MON] |
||
|---|---|---|---|
| Line 22: | Line 22: | ||
| <code> | <code> | ||
| nodeN# apt install ceph | nodeN# apt install ceph | ||
| + | |||
| + | nodeN# ceph -v | ||
| </code> | </code> | ||
| Line 80: | Line 82: | ||
| <code> | <code> | ||
| - | debian11_12/ubuntu# ceph mon enable-msgr2 | + | node3# ceph mon enable-msgr2 |
| - | debian11_12/ubuntu# ceph config set mon mon_warn_on_insecure_global_id_reclaim false | + | |
| - | debian11_12/ubuntu# ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false | + | node3# ceph config set mon mon_warn_on_insecure_global_id_reclaim false |
| + | node3# ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false | ||
| </code><code> | </code><code> | ||
| # ceph -s | # ceph -s | ||
| Line 94: | Line 97: | ||
| MON_DOWN 1/3 mons down, quorum node4,node5 | MON_DOWN 1/3 mons down, quorum node4,node5 | ||
| mon.node3 (rank 0) addr 192.168.X.3:6789/0 is down (out of quorum) | mon.node3 (rank 0) addr 192.168.X.3:6789/0 is down (out of quorum) | ||
| + | </code> | ||
| + | === Замена MON === | ||
| + | <code> | ||
| + | node3# ceph mon remove node4 | ||
| + | |||
| + | node3# ceph mon getmap -o /tmp/monmap | ||
| + | node3# monmaptool --print /tmp/monmap | ||
| + | node3# monmaptool --add node7 192.168.13.7 --fsid a7f64266-0894-4f1e-a635-d0aeaca0e993 /tmp/monmap | ||
| + | node3# scp /tmp/monmap node7:/tmp/monmap | ||
| + | |||
| + | node3# cat /etc/ceph/ceph.conf | ||
| + | ... | ||
| + | mon initial members = node3,node7,node5 | ||
| + | mon host = 192.168.13.3,192.168.13.7,192.168.13.5 | ||
| + | ... | ||
| + | |||
| + | node3# | ||
| + | scp /etc/ceph/ceph.conf node5:/etc/ceph/ | ||
| + | scp /etc/ceph/ceph.conf node6:/etc/ceph/ | ||
| + | scp /etc/ceph/ceph.conf node7:/etc/ceph/ | ||
| + | |||
| + | node7# | ||
| + | sudo -u ceph mkdir /var/lib/ceph/mon/ceph-node7 | ||
| + | sudo -u ceph ceph-mon --mkfs -i node7 --monmap /tmp/monmap | ||
| + | systemctl enable ceph-mon@node7 --now | ||
| + | |||
| + | sudo -u ceph mkdir /var/lib/ceph/mgr/ceph-node7 | ||
| + | systemctl enable ceph-mgr@node7 --now | ||
| </code> | </code> | ||
| + | * [[Технология KVM#Использование Ceph RBD в качестве диска]] | ||
| + | * [[Сервис iSCSI]] | ||
| ==== MGR ==== | ==== MGR ==== | ||
| Line 117: | Line 150: | ||
| # ceph -s | # ceph -s | ||
| - | # ps ax | grep ceph | + | debian13# ceph health detail |
| + | debian13# ceph health mute MGR_MODULE_DEPENDENCY | ||
| </code> | </code> | ||
| + | |||
| ==== OSD ==== | ==== OSD ==== | ||
| Line 160: | Line 195: | ||
| <code> | <code> | ||
| + | ceph osd tree | grep down | ||
| + | |||
| ceph osd safe-to-destroy osd.0 | ceph osd safe-to-destroy osd.0 | ||
| OSD(s) 0 are safe to destroy without reducing data durability. | OSD(s) 0 are safe to destroy without reducing data durability. | ||
| Line 196: | Line 233: | ||
| Total PGs = OSDs * 100 / pool_size | Total PGs = OSDs * 100 / pool_size | ||
| - | 5*100/3 = 166,6... | + | 4*100/3 = 133,3... |
| </code><code> | </code><code> | ||
| - | # ceph osd pool create test-pool1 180 | + | # ceph osd pool create test-pool1 128 |
| - | + | ||
| - | debian11_12/ubuntu# ceph osd pool create test-pool1 128 | + | |
| - | или | + | |
| - | debian11_12/ubuntu# ceph config set global mon_warn_on_pool_pg_num_not_power_of_two false | + | |
| # ceph -s | # ceph -s | ||
| </code><code> | </code><code> | ||
| ... | ... | ||
| - | pgs: 180 active+clean | + | health: HEALTH_WARN |
| + | 1 pool(s) do not have an application enabled | ||
| + | ... | ||
| + | pgs: 128 active+clean | ||
| + | ... | ||
| </code><code> | </code><code> | ||
| # ceph pg dump | # ceph pg dump | ||
| Line 215: | Line 252: | ||
| === Изменение параметров POOL === | === Изменение параметров POOL === | ||
| <code> | <code> | ||
| - | ceph osd pool set test-pool1 pg_num 250 | + | ceph osd pool set test-pool1 pg_num 256 |
| - | ceph osd pool set test-pool1 pgp_num 250 | + | ceph osd pool set test-pool1 pgp_num 160 ... 192 ... 224 ... 256 |
| ceph osd pool get test-pool1 size | ceph osd pool get test-pool1 size | ||
| Line 232: | Line 269: | ||
| <code> | <code> | ||
| - | # ceph df | + | node3# ceph df |
| - | # rbd pool init test-pool1 | + | node3# rbd pool init test-pool1 |
| - | # rbd create -p test-pool1 rbd1 --size 1G | + | node3# ceph -s |
| - | # rbd list test-pool1 | + | </code> |
| - | # rbd info test-pool1/rbd1 | + | * Технология KVM [[Технология KVM#Использование Ceph RBD в качестве диска]] |
| - | </code><code> | + | |
| - | # rbd resize --size 3584M test-pool1/rbd1 | + | |
| - | node3# bash -c ' | + | <code> |
| + | |||
| + | node3# rbd create -p test-pool1 rbd1 --size 1G | ||
| + | |||
| + | node3# rbd list test-pool1 | ||
| + | |||
| + | node3# rbd info test-pool1/rbd1 | ||
| + | |||
| + | node3# rbd status test-pool1/rbd1 | ||
| + | |||
| + | node3# rbd lock list test-pool1/rbd1 | ||
| + | |||
| + | node3# rbd device map test-pool1/rbd1 | ||
| + | /dev/rbd0 | ||
| + | |||
| + | # mount -t ntfs3 /dev/rbd0p1 /mnt/ | ||
| + | |||
| + | # umount /mnt/ | ||
| + | |||
| + | # rbd device unmap /dev/rbd0 | ||
| + | </code> | ||
| + | |||
| + | * Изменение размера блочного устройствва | ||
| + | |||
| + | <code> | ||
| + | node3# rbd resize --size 3584M test-pool1/rbd1 | ||
| + | |||
| + | node3# ###bash -c ' | ||
| systemctl restart tgt | systemctl restart tgt | ||
| ssh node4 systemctl restart tgt | ssh node4 systemctl restart tgt | ||
| ssh node5 systemctl restart tgt | ssh node5 systemctl restart tgt | ||
| ' | ' | ||
| + | |||
| + | node3# ### rbd rm test-pool1/rbd1 | ||
| </code> | </code> | ||
решение_ceph.1743927059.txt.gz · Last modified: 2025/04/06 11:10 by val
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
