From 4ae6ae60ce7eec0ce13028a90e005f1bc5ddb000 Mon Sep 17 00:00:00 2001 From: Dan van der Ster Date: Tue, 22 Apr 2025 13:41:30 -0700 Subject: [PATCH] blog: fix old pre formatting markdown Fix preformatted markdown in a couple nice old blog posts. Signed-off-by: Dan van der Ster --- .../index.md | 77 +++-- .../index.md | 309 ++++++++++-------- 2 files changed, 213 insertions(+), 173 deletions(-) diff --git a/src/en/news/blog/2017/new-luminous-cephfs-subtree-pinning/index.md b/src/en/news/blog/2017/new-luminous-cephfs-subtree-pinning/index.md index 008419515..c89ae367e 100644 --- a/src/en/news/blog/2017/new-luminous-cephfs-subtree-pinning/index.md +++ b/src/en/news/blog/2017/new-luminous-cephfs-subtree-pinning/index.md @@ -15,26 +15,28 @@ In Luminous, [multiple active metadata servers](http://ceph.com/community/new-lu You can view the current subtree divisions of the file system by querying the admin socket of each MDS (on the host each MDS is operating on): - $ ceph fs status - cephfs - 0 clients - ======== - +------+--------+-----+---------------+-------+-------+ - | Rank | Stat e | MDS | Activity | dns | inos | - +------+--------+-----+---------------+-------+-------+ - | 0 | active | b | Reqs: 0 /s | 0 | 0 | - | 1 | active | c | Reqs: 0 /s | 0 | 0 | - | 2 | active | a | Reqs: 0 /s | 0 | 0 | - +------+--------+-----+---------------+-------+-------+ - +-------------------+----------+-------+-------+ - | Pool | type | used | avail | - +-------------------+----------+-------+-------+ - | cephfs_metadata | metadata | 4098 | 9554M | - | cephfs_data | data | 0 | 9554M | - +-------------------+----------+-------+-------+ - $ bin/ceph daemon mds.a get subtrees | jq '.[] | [.dir.path, .auth_first]' - ["~mds2", 2] - ["", 0] - ["/tmp", 2] +``` +$ ceph fs status +cephfs - 0 clients +======== ++------+--------+-----+---------------+-------+-------+ +| Rank | Stat e | MDS | Activity | dns | inos | ++------+--------+-----+---------------+-------+-------+ +| 0 | active | b | Reqs: 0 /s | 0 | 0 | +| 1 | active | c | Reqs: 0 /s | 0 | 0 | +| 2 | active | a | Reqs: 0 /s | 0 | 0 | ++------+--------+-----+---------------+-------+-------+ ++-------------------+----------+-------+-------+ +| Pool | type | used | avail | ++-------------------+----------+-------+-------+ +| cephfs_metadata | metadata | 4098 | 9554M | +| cephfs_data | data | 0 | 9554M | ++-------------------+----------+-------+-------+ +$ bin/ceph daemon mds.a get subtrees | jq '.[] | [.dir.path, .auth_first]' +["~mds2", 2] +["", 0] +["/tmp", 2] +``` The “” subtree is the root of the file system (“/”) and is always managed by rank 0. The “/tmp” subtree is being managed by rank 2. (A subtree path beginning with “~” is an internal subtree and not part of the file system hierarchy.) @@ -48,15 +50,17 @@ For example, it can prevent a directory from splitting into multiple subtrees an Pinning a directory to a particular rank is done by setting an extended attribute: - $ setfattr -n ceph.dir.pin -v 2 /mnt/cephfs/tmp - +``` +$ setfattr -n ceph.dir.pin -v 2 /mnt/cephfs/tmp +``` This has the effect of preventing the CephFS directory “/tmp” from being split into smaller subtrees and also pinning “/tmp” to rank 2 (if that rank exists). Once this is done, you may query the rank 2 MDS to see its subtree map: - $ ceph daemon mds.b get subtrees | jq '.[] | [.dir.path, .auth_first, .export_pin]' - ["", 0, -1] - ["~mds0", 0, -1] - ["/tmp", 2, 2] - +``` +$ ceph daemon mds.b get subtrees | jq '.[] | [.dir.path, .auth_first, .export_pin]' +["", 0, -1] +["~mds0", 0, -1] +["/tmp", 2, 2] +``` Here we can see that “/tmp” has its export_pin set to 2 and rank 2 is authoritative (auth_first). (N.B. a pinned directory is only shipped to its rank if it is not empty.) @@ -65,15 +69,16 @@ Here we can see that “/tmp” has its export_pin set to 2 and rank 2 is author You may also have a hierarchy of pins. This means a child directory can have a pin set which overrides the pin of a parent. So we may have: - $ setfattr -n ceph.dir.pin -v 0 /mnt/cephfs/users/ - $ setfattr -n ceph.dir.pin -v 1 /mnt/cephfs/users/joe/ - $ ceph daemon mds.b get subtrees | jq '.[] | [.dir.path, .auth_first, .export_pin]' - ["", 0, -1] - ["~mds0", 0, -1] - ["/tmp", 2, 2] - ["/users/joe", 1, 1] - ["/users", 0, 0] - +``` +$ setfattr -n ceph.dir.pin -v 0 /mnt/cephfs/users/ +$ setfattr -n ceph.dir.pin -v 1 /mnt/cephfs/users/joe/ +$ ceph daemon mds.b get subtrees | jq '.[] | [.dir.path, .auth_first, .export_pin]' +["", 0, -1] +["~mds0", 0, -1] +["/tmp", 2, 2] +["/users/joe", 1, 1] +["/users", 0, 0] +``` The “/users” subtree sets a “default” pin for itself and its children (home directories) to rank 0. However, “/users/joe” has a pin to rank 1 which overrides the “/users” pin. ### Future Direction diff --git a/src/en/news/blog/2018/new-mimic-centralized-configuration-management/index.md b/src/en/news/blog/2018/new-mimic-centralized-configuration-management/index.md index be9cb8989..5acf920df 100644 --- a/src/en/news/blog/2018/new-mimic-centralized-configuration-management/index.md +++ b/src/en/news/blog/2018/new-mimic-centralized-configuration-management/index.md @@ -51,11 +51,15 @@ has done. In most cases, a small ceph.conf is still necessary in order to identify who the monitors are. For example, a typical minimal ceph.conf file might be: -> mon host = mon-a.foo.com, mon-b.foo.com, mon-c.foo.com +``` +mon host = mon-a.foo.com, mon-b.foo.com, mon-c.foo.com +``` or better yet -> mon host = ceph-mons.foo.com +``` +mon host = ceph-mons.foo.com +``` where _ceph-mons_ is a DNS entry with multiple A records (one for each monitor). This allows the number and identities of monitors to change over time @@ -75,45 +79,51 @@ priority order that Ceph uses to set options is: Typing _ceph config -h_ will summarize the set of commands available: -> $ ceph config -h -> \[...\] -> config assimilate-conf Assimilate options from a conf, and return a -> new, minimal conf file -> config dump Show all configuration option(s) -> config get {} Show configuration option(s) for an entity -> config help Describe a configuration option -> config log {} Show recent history of config changes -> config reset Revert configuration to previous state -> config rm Clear a configuration option for one or more -> entities -> config set Set a configuration option for one or more -> entities -> config show {} Show running configuration -> config show-with-defaults Show running configuration (including compiled- -> in defaults) +``` +$ ceph config -h +[...] +config assimilate-conf Assimilate options from a conf, and return a + new, minimal conf file +config dump Show all configuration option(s) +config get {} Show configuration option(s) for an entity +config help Describe a configuration option +config log {} Show recent history of config changes +config reset Revert configuration to previous state +config rm Clear a configuration option for one or more + entities +config set Set a configuration option for one or more + entities +config show {} Show running configuration +config show-with-defaults Show running configuration (including compiled- + in defaults) +``` A good place to start is simply dumping the cluster configuration: -> $ ceph config dump -> WHO MASK LEVEL OPTION VALUE RO -> global advanced mon\_pg\_warn\_min\_per\_osd 3 -> global advanced osd\_pool\_default\_min\_size 1 -> global advanced osd\_pool\_default\_size 1 -> mon advanced mon\_allow\_pool\_delete true -> ... +``` +$ ceph config dump +WHO MASK LEVEL OPTION VALUE RO +global advanced mon_pg_warn_min_per_osd 3 +global advanced osd_pool_default_min_size 1 +global advanced osd_pool_default_size 1 + mon advanced mon_allow_pool_delete true +... +``` We can set an option in the following way: -> $ ceph config set osd debug\_ms 1 -> $ ceph config dump -> WHO MASK LEVEL OPTION VALUE RO -> global advanced mon\_pg\_warn\_min\_per\_osd 3 -> global advanced osd\_pool\_default\_min\_size 1 -> global advanced osd\_pool\_default\_size 1 -> mon advanced mon\_allow\_pool\_delete true -> ... -> osd advanced debug\_ms 1 -> ... +``` +$ ceph config set osd debug_ms 1 +$ ceph config dump +WHO MASK LEVEL OPTION VALUE RO +global advanced mon_pg_warn_min_per_osd 3 +global advanced osd_pool_default_min_size 1 +global advanced osd_pool_default_size 1 + mon advanced mon_allow_pool_delete true +... + osd advanced debug_ms 1 +... +``` Note that this and nothing more is all that is necessary to make the change: any daemons or clients in the system to which this option applies will be @@ -126,24 +136,28 @@ daemons or clients the option applies to, and can match either a CRUSH location example, we could set a higher debug level that applies only to OSDs that are backed by SSDs (and that are reported by the _ceph osd crush tree_ command): -> $ ceph config set osd/class:ssd debug\_ms 2 -> $ ceph config dump -> WHO MASK LEVEL OPTION VALUE RO -> ... -> osd advanced debug\_ms 1 -> osd class:ssd advanced debug\_ms 2 -> ... +``` +$ ceph config set osd/class:ssd debug_ms 2 +$ ceph config dump +WHO MASK LEVEL OPTION VALUE RO +... + osd advanced debug_ms 1 + osd class:ssd advanced debug_ms 2 +... +``` Instead of dumping the entire config database, you can inspect the config for a individual daemon in the system. For example: -> $ ceph config set osd.0 debug\_osd 10 -> $ ceph config get osd.0 -> WHO MASK LEVEL OPTION VALUE RO -> osd class:ssd advanced debug\_ms 2/2 -> osd.0 advanced debug\_osd 10/10 -> global advanced mon\_pg\_warn\_min\_per\_osd 3 -> ... +``` +$ ceph config set osd.0 debug_osd 10 +$ ceph config get osd.0 +WHO MASK LEVEL OPTION VALUE RO +osd class:ssd advanced debug_ms 2/2 +osd.0 advanced debug_osd 10/10 +global advanced mon_pg_warn_min_per_osd 3 +... +``` This output tells you which options and values apply the daemon, as well as where the option is coming from (is it set globally, is it set for this daemon @@ -151,12 +165,14 @@ specifically, etc.). Naturally, a config entry can also be cleared: -> $ ceph config rm osd/class:ssd debug\_ms -> $ ceph config get osd.0 -> WHO MASK LEVEL OPTION VALUE RO -> osd advanced debug\_ms 1/1 -> global advanced mon\_pg\_warn\_min\_per\_osd 3 -> ... +``` +$ ceph config rm osd/class:ssd debug_ms +$ ceph config get osd.0 +WHO MASK LEVEL OPTION VALUE RO +osd advanced debug_ms 1/1 +global advanced mon_pg_warn_min_per_osd 3 +... +``` ## Enforced configuration schema @@ -167,24 +183,28 @@ is globally known. If you try to set something that doesn't make sense, you'll get an informative error message and you will not affect the existing configuration. For example: -> $ ceph config set osd.10 debug\_osd very\_high -> Error EINVAL: error parsing value: value must take the form N or N/M, where N and M are integers -> $ ceph config set osd.10 bluestore\_compression\_mode 1 -> Error EINVAL: error parsing value: '1' is not one of the permitted values: none, passive, aggressive, force +``` +$ ceph config set osd.10 debug_osd very_high +Error EINVAL: error parsing value: value must take the form N or N/M, where N and M are integers +$ ceph config set osd.10 bluestore_compression_mode 1 +Error EINVAL: error parsing value: '1' is not one of the permitted values: none, passive, aggressive, force +``` The schema for a particular option can be queried with a help command: -> $ ceph config help bluestore\_compression\_mode -> bluestore\_compression\_mode - Default policy for using compression when pool does not specify -> (std::string, advanced) -> Default: none -> Possible values: none passive aggressive force -> Can update at runtime: true -> -> 'none' means never use compression. 'passive' means use compression when clients hint that data -> is compressible. 'aggressive' means use compression unless clients hint that data is not -> compressible. This option is used when the per-pool property for the compression mode is not -> present. +``` +$ ceph config help bluestore_compression_mode +bluestore_compression_mode - Default policy for using compression when pool does not specify + (std::string, advanced) + Default: none + Possible values: none passive aggressive force + Can update at runtime: true + +'none' means never use compression. 'passive' means use compression when clients hint that data +is compressible. 'aggressive' means use compression unless clients hint that data is not +compressible. This option is used when the per-pool property for the compression mode is not +present. +``` You'll notice that that _advanced_ is on the second line. All options are divided into three categories: basic, advanced, and dev. The dev options are @@ -197,11 +217,13 @@ Ceph work. Some numeric options include a minimum and maximum value, and will accept suffixes like K (kilo) or M (mega) for large values: -> $ ceph config set mon mon\_data\_size\_warn 100G -> $ ceph config get mon.a -> WHO MASK LEVEL OPTION VALUE RO -> mon advanced mon\_data\_size\_warn 107374182400 -> ... +``` +$ ceph config set mon mon_data_size_warn 100G +$ ceph config get mon.a +WHO MASK LEVEL OPTION VALUE RO +mon advanced mon_data_size_warn 107374182400 +... +``` Note that whether 'K' means 1000 or 1024 depends on the configuration option in question: some are based on SI units (base-10) and some on IEC units (base-2, @@ -214,15 +236,17 @@ local ceph.conf, operator override) there is a _show_ command that returns the active configuration options as reported by any daemon in the system. For example: -> $ ceph daemon mgr.x config set debug\_mgr 10 # manual override of config option -> $ ceph config set mgr.x ms\_type simple # set an option normally -> $ ceph config show mgr.x -> NAME VALUE SOURCE OVERRIDES IGNORES -> debug\_mgr 10/10 override mon\[20/20\] -> debug\_mon 20/20 mon -> debug\_ms 1/1 file -> ms\_type async+posix default mon -> ... +``` +$ ceph daemon mgr.x config set debug_mgr 10 # manual override of config option +$ ceph config set mgr.x ms_type simple # set an option normally +$ ceph config show mgr.x +NAME VALUE SOURCE OVERRIDES IGNORES +debug_mgr 10/10 override mon[20/20] +debug_mon 20/20 mon +debug_ms 1/1 file +ms_type async+posix default mon +... +``` The NAME and VALUE columns tell you which options and values are currently in effect. SOURCE tells you where the value came from: "override" came from our @@ -239,17 +263,21 @@ can take effect only when the daemon is restarted, such as _ms\_type_ (which controls which message passing implementation is used). You can also see that this is a read-only value from the RO column in _config get_ command results: -> $ ceph config get mgr.x -> WHO MASK LEVEL OPTION VALUE RO -> mgr advanced debug\_mgr 20/20 \* -> mgr advanced ms\_type simple \* +``` +$ ceph config get mgr.x +WHO MASK LEVEL OPTION VALUE RO +mgr advanced debug_mgr 20/20 * +mgr advanced ms_type simple * +``` You'll also notice that the help result for _ms\_type_ tells us the same thing: -> $ ceph config help ms\_type -> ... -> Default: async+posix -> Can update at runtime: false +``` +$ ceph config help ms_type +... + Default: async+posix + Can update at runtime: false +``` ## Configuration change history @@ -262,15 +290,17 @@ Ceph's new configuration management provides a simple version of that capability. Every configuration change in the system is recorded and easily viewable: -> $ ceph config log -> --- 15 --- 2018-06-13 15:02:46.176060 --- -> - mgr.x/ms\_type = simple -> + mgr.x/ms\_type = async -> --- 14 --- 2018-06-13 14:52:51.877714 --- -> + mgr.x/ms\_type = simple -> --- 13 --- 2018-06-13 14:45:33.988326 --- -> + mon/mon\_data\_size\_warn = 107374182400 -> ... +``` +$ ceph config log +--- 15 --- 2018-06-13 15:02:46.176060 --- +- mgr.x/ms_type = simple ++ mgr.x/ms_type = async +--- 14 --- 2018-06-13 14:52:51.877714 --- ++ mgr.x/ms_type = simple +--- 13 --- 2018-06-13 14:45:33.988326 --- ++ mon/mon_data_size_warn = 107374182400 +... +``` The output is meant to be familiar to anyone familiar with _diff_ output, where "+" lines indicate a new configuration entry and "-" lines indicate a removed @@ -280,19 +310,20 @@ The configuration of the system can be reverted to a previous state by using the numeric identifier preceding each change record. For example, to undo our changes to _ms\_type_, -> $ ceph config reset 13 -> $ ceph config log -> --- 16 --- 2018-06-13 15:05:10.960659 --- reset to 13 --- -> - mgr.x/ms\_type = async -> --- 15 --- 2018-06-13 15:02:46.176060 --- -> - mgr.x/ms\_type = simple -> + mgr.x/ms\_type = async -> --- 14 --- 2018-06-13 14:52:51.877714 --- -> + mgr.x/ms\_type = simple -> --- 13 --- 2018-06-13 14:45:33.988326 --- -> + mon/mon\_data\_size\_warn = 107374182400 -> ... - +``` +$ ceph config reset 13 +$ ceph config log +--- 16 --- 2018-06-13 15:05:10.960659 --- reset to 13 --- +- mgr.x/ms_type = async +--- 15 --- 2018-06-13 15:02:46.176060 --- +- mgr.x/ms_type = simple ++ mgr.x/ms_type = async +--- 14 --- 2018-06-13 14:52:51.877714 --- ++ mgr.x/ms_type = simple +--- 13 --- 2018-06-13 14:45:33.988326 --- ++ mon/mon_data_size_warn = 107374182400 +... +``` (The net effect of resetting to 13 is that the ms\_type entry is removed, even though it had two intermediate values since then.) Because the _reset_ command is a configuration change like any other, it can be undone with another reset @@ -312,25 +343,27 @@ existing config file as input and generates a (hopefully shorter) config file for output that contains any options that could not be assimilated. For example: -> $ cat ceph.conf -> \[global\] -> mon host = foo.ceph.com -> \[osd.1\] -> debug\_osd = 0/0 -> \[mds.a\] -> mds invalid option = this option does not exist -> -> $ ceph config assimilate-conf -i ceph.conf -o ceph.conf.new -> \[global\] -> mon\_host = foo.ceph.com -> -> \[mds.a\] -> mds\_invalid\_option = this option does not exist -> -> $ ceph config get osd.1 -> WHO MASK LEVEL OPTION VALUE RO -> osd.1 advanced debug\_osd 0/0 -> ... +``` +$ cat ceph.conf +[global] +mon host = foo.ceph.com +[osd.1] +debug_osd = 0/0 +[mds.a] +mds invalid option = this option does not exist + +$ ceph config assimilate-conf -i ceph.conf -o ceph.conf.new +[global] + mon_host = foo.ceph.com + +[mds.a] + mds_invalid_option = this option does not exist + +$ ceph config get osd.1 +WHO MASK LEVEL OPTION VALUE RO +osd.1 advanced debug_osd 0/0 +... +``` In this simple example, only the _debug\_osd_ option for osd.1 was imported. _mon\_host_ was left behind (it's needed for bootstrapping) and @@ -341,10 +374,12 @@ process is to run an assimilate command (like the one above) on each host, which will incorporate settings into the cluster's configuration database and leave behind only the bootstrap-related options on each host. For example: -> $ cd /etc/ceph -> $ ceph config assimilate-conf -i ceph.conf -o ceph.conf.new -> $ cat ceph.conf.new # make sure it looks okay! -> $ mv ceph.conf.new ceph.conf +``` +$ cd /etc/ceph +$ ceph config assimilate-conf -i ceph.conf -o ceph.conf.new +$ cat ceph.conf.new # make sure it looks okay! +$ mv ceph.conf.new ceph.conf +``` This will work in the majority of cases. But if you're assimilating a configuration file that changes any settings mentioned in the input (which