Merging multiple config files together needs to know newest (highest)
timestamp of merged files. Persistent cache file is being used
only in case, the config file is older then .cache file.
Add verbose message when we will not obtain devices from udev
(i.e. testing is using different udev dir, and the log was
giving misleading info about using udev)
Add proper error message if zalloc from pull would have failed.
Assign fid as the last step before returning VG.
Make the format reader for 'lvm1' and 'pool' equal to 'lvm2' format reader.
It has caused memory corruption to lvmetad as it later calls
destroy_instance() to allocated fid. This patch should fix problems
with crashing test lvmetad-lvm1.sh.
Sharing char* with field has a problem in error path,
when we allocate event, but fail to allocate timeout string.
Instead of creating complicated error paths to resolve
it individually stop using unions, and let the resource
to be released in a simple _free_message().
Peter Rajnoha [Fri, 19 Apr 2013 10:17:53 +0000 (12:17 +0200)]
udev: also autoactivate on coldplug
Commit 756bcabbfe297688ba240a880bc2b55265ad33f0 fixed autoactivation
to not trigger on each uevent for a PV that appeared in the system
most notably the events that are triggered artificially (udevadm
trigger or as the result of the WATCH udev rule being applied that
consequently generates CHANGE uevents). This fixed a situation in
which VGs/LVs were activated when they should not.
BUT we still need to care about the coldplug used at boot to
retrigger the ADD events - the "udevadm trigger --action=add"!
For non-DM-based PVs, this is already covered as for these we
run the autoactivation on ADD event only.
However, for DM-based PVs, we still need to run the
autoactivation even for the artificial ADD event, reusing
the udev DB content from previous proper CHANGE event that
came with the DM device activation.
Simply, this patch fixes a situation in which we run extra
"udevadm trigger --action=add" (or echo add > /sys/block/<dev>/uevent)
for DM-based PVs (cryptsetup devices, multipath devices, any
other DM devices...).
Without this patch, while using lvmetad + autoactivation,
any VG/LV that has a DM-based PV and for which we do not
call the activation directly, the VG/LV is not activated.
For example a VG with an LV with root FS on it which is directly
activated in initrd and then missing activation of the rest
of the LVs in the VG because of unhandled uevent retrigger on
boot after switching to root FS (the "coldplug").
(No WHATS_NEW here as this fixes the commit mentioned
above and which was not released yet.)
Tony Asleson [Tue, 16 Apr 2013 08:42:03 +0000 (10:42 +0200)]
config_def_check: fix memory leak
There is no need to strdup a key when inserting into
the hash table as the table allocates memory and copies
the string. This was causing memory to be lost.
Jonathan Brassow [Mon, 15 Apr 2013 18:59:46 +0000 (13:59 -0500)]
RAID: Add writemostly/writebehind support for RAID1
'lvchange' is used to alter a RAID 1 logical volume's write-mostly and
write-behind characteristics. The '--writemostly' parameter takes a
PV as an argument with an optional trailing character to specify whether
to set ('y'), unset ('n'), or toggle ('t') the value. If no trailing
character is given, it will set the flag.
Synopsis:
lvchange [--writemostly <PV>:{t|y|n}] [--writebehind <count>] vg/lv
Example:
lvchange --writemostly /dev/sdb1:y --writebehind 512 vg/raid1_lv
The last character in the 'lv_attr' field is used to show whether a device
has the WriteMostly flag set. It is signified with a 'w'. If the device
has failed, the 'p'artial flag has priority.
Example ("nosync" raid1 with mismatch_cnt and writemostly):
[~]# lvs -a --segment vg
LV VG Attr #Str Type SSize
raid1 vg Rwi---r-m 2 raid1 500.00m
[raid1_rimage_0] vg Iwi---r-- 1 linear 500.00m
[raid1_rimage_1] vg Iwi---r-w 1 linear 500.00m
[raid1_rmeta_0] vg ewi---r-- 1 linear 4.00m
[raid1_rmeta_1] vg ewi---r-- 1 linear 4.00m
Example (raid1 with mismatch_cnt, writemostly - but failed drive):
[~]# lvs -a --segment vg
LV VG Attr #Str Type SSize
raid1 vg rwi---r-p 2 raid1 500.00m
[raid1_rimage_0] vg Iwi---r-- 1 linear 500.00m
[raid1_rimage_1] vg Iwi---r-p 1 linear 500.00m
[raid1_rmeta_0] vg ewi---r-- 1 linear 4.00m
[raid1_rmeta_1] vg ewi---r-p 1 linear 4.00m
A new reportable field has been added for writebehind as well. If
write-behind has not been set or the LV is not RAID1, the field will
be blank.
Example (writebehind is set):
[~]# lvs -a -o name,attr,writebehind vg
LV Attr WBehind
lv rwi-a-r-- 512
[lv_rimage_0] iwi-aor-w
[lv_rimage_1] iwi-aor--
[lv_rmeta_0] ewi-aor--
[lv_rmeta_1] ewi-aor--
Example (writebehind is not set):
[~]# lvs -a -o name,attr,writebehind vg
LV Attr WBehind
lv rwi-a-r--
[lv_rimage_0] iwi-aor-w
[lv_rimage_1] iwi-aor--
[lv_rmeta_0] ewi-aor--
[lv_rmeta_1] ewi-aor--
The original code also handled len==1, which the new code doesn't.
Press <TAB> in the lvm shell to get a list of the possible
flag completions for a single hyphen.
Jonathan Brassow [Fri, 12 Apr 2013 16:30:04 +0000 (11:30 -0500)]
CLEAN-UP: Better string checking to avoid substring matches
Commit 9fd7ac7d035f0b2f8dcc3cb19935eb181816bd76 introduced a way a
method of avoiding reading from mirrors with a device failure. If
a device was found to be dead, the mapping table was checked for
'handle_errors' or 'block_on_error'. These strings were checked for
in the table string via 'strstr', which could also match on strings
like, 'no_handle_errors' or 'no_block_on_error'. No such strings
exist, but we don't want to have problems in the future if they do.
So, we check for ' <string>{'\0'|' '}'.
Move common code for changing activation state from
vgchange and lvchange to one function.
Fix the order of checks - so we always implicitelly
activate snapshots and thin volumes in exclusive mode,
and we do not allow local deactivation for them.
Jonathan Brassow [Thu, 11 Apr 2013 20:57:14 +0000 (15:57 -0500)]
RAID: Revert previous commit that allowed identical table loads.
Revert commit 31c24dd9f2ad7b5f7913a18c9f11a00d7b3474a1. This commit
was used to force a RAID device-mapper table to be loaded into the
kernel despite the fact that it was identical to the one already
loaded. The effect allowed a RAID array with a transiently failed
device to refresh and reintegrate the failed device. This operation
is better done in the kernel on a 'resume'. Since,
'lvchange --refresh' already performs a suspend/resume cycle, the
above commit is not needed once the kernel change is made. Reverting
the commit removes an unnecessary (at least for now) change to the
device-mapper interface.
Jonathan Brassow [Thu, 11 Apr 2013 20:33:59 +0000 (15:33 -0500)]
RAID: Add scrubbing support for RAID LVs
New options to 'lvchange' allow users to scrub their RAID LVs.
Synopsis:
lvchange --syncaction {check|repair} vg/raid_lv
RAID scrubbing is the process of reading all the data and parity blocks in
an array and checking to see whether they are coherent. 'lvchange' can
now initaite the two scrubbing operations: "check" and "repair". "check"
will go over the array and recored the number of discrepancies but not
repair them. "repair" will correct the discrepancies as it finds them.
'lvchange --syncaction repair vg/raid_lv' is not to be confused with
'lvconvert --repair vg/raid_lv'. The former initiates a background
synchronization operation on the array, while the latter is designed to
repair/replace failed devices in a mirror or RAID logical volume.
Additional reporting has been added for 'lvs' to support the new
operations. Two new printable fields (which are not printed by
default) have been added: "syncaction" and "mismatches". These
can be accessed using the '-o' option to 'lvs', like:
lvs -o +syncaction,mismatches vg/lv
"syncaction" will print the current synchronization operation that the
RAID volume is performing. It can be one of the following:
- idle: All sync operations complete (doing nothing)
- resync: Initializing an array or recovering after a machine failure
- recover: Replacing a device in the array
- check: Looking for array inconsistencies
- repair: Looking for and repairing inconsistencies
The "mismatches" field with print the number of descrepancies found during
a check or repair operation.
The 'Cpy%Sync' field already available to 'lvs' will print the progress
of any of the above syncactions, including check and repair.
Finally, the lv_attr field has changed to accomadate the scrubbing operations
as well. The role of the 'p'artial character in the lv_attr report field
as expanded. "Partial" is really an indicator for the health of a
logical volume and it makes sense to extend this include other health
indicators as well, specifically:
'm'ismatches: Indicates that there are discrepancies in a RAID
LV. This character is shown after a scrubbing
operation has detected that portions of the RAID
are not coherent.
'r'efresh : Indicates that a device in a RAID array has suffered
a failure and the kernel regards it as failed -
even though LVM can read the device label and
considers the device to be ok. The LV should be
'r'efreshed to notify the kernel that the device is
now available, or the device should be 'r'eplaced
if it is suspected of failing.
Jonathan Brassow [Wed, 10 Apr 2013 21:47:04 +0000 (16:47 -0500)]
mirror: Fix overly-concerning warning on mirror up-convert failure.
Attempting to up-convert an inactive mirror when there is insufficient
space leads to the following message:
Unable to allocate extents for mirror(s).
ABORTING: Failed to remove temporary mirror layer inactive_mimagetmp_3.
Manual cleanup with vgcfgrestore and dmsetup may be required.
This is caused by a failure to execute the 'deactivate_lv' function in
the error condition. The deactivate returns an error because the LV is
already inactive. This patch checks if the LV is activate and calls
deactivate_lv only if it is. This allows the error cleanup code to work
properly in this condition.
It wasn't that big of a deal anyway, since there was no previous vg_commit
that needed to be reverted. IOW, no harm was done if the allocation failed.
The message was scary and useless.
RAID: Capture new RAID kernel sync_action status fields
I've updated the dm_status_raid structure and dm_get_status_raid()
function to make it handle the new kernel status fields that will
be coming in dm-raid v1.5.0. It is backwards compatible with the
old status line - initializing the new fields to '0'. The new
structure is also more amenable to future changes. It includes a
'reserved' field that is currently initialized to zero but could
be used to hold flags describing new features. It also now uses
pointers for the character strings instead of attempting to allocate
their space along with the structure (causing the size of the
structure to be variable). This allows future fields to be appended.
The new fields that are available are:
- sync_action : shows what the sync thread in the kernel is doing
(idle, frozen, resync, recover, check, repair, or
reshape)
- mismatch_count: shows the number of discrepancies which were
found or repaired by a "check" or "repair"
process, respectively.
Peter Rajnoha [Mon, 25 Mar 2013 13:30:40 +0000 (14:30 +0100)]
pv_write: clean up non-orphan format1 PV write
...to not pollute the common and format-independent code in the
abstraction layer above.
The format1 pv_write has common code for writing metadata and
PV header by calling the "write_disks" fn and when rewriting
the header itself only (e.g. just for the purpose of changing
the PV UUID) during the pvchange operation, we had to tweak
this functionality for the format1 case and we had to assign
the PV the orphan state temporarily.
This patch removes the need for this format1 tweak and it calls
the write_disks with appropriate flag indicating whether this is
a PV write call or a VG write call, allowing for metatada update
for the latter one.
Also, a side effect of the former tweak was that it effectively
invalidated the cache (even for the non-format1 PVs) as we
assigned it the orphan state temporarily just for the format1
PV write to pass.
Also, that tweak made it difficult to directly detect whether
a PV was part of a VG or not because the state was incorrect.
Also, it's not necessary to backup and restore some PV fields
when doing a PV write:
Peter Rajnoha [Tue, 19 Mar 2013 13:17:53 +0000 (14:17 +0100)]
cleanup: remove unused 'pv_by_path' fn
The pv_by_path might be also dangerous to use as it does not
count with any other metadata areas but the ones found on the PV
itself. If metadata was not found on the PV referenced by the path,
it returned no PV though it might have been referenced by metadata
elsewhere (on other PVs...).
Peter Rajnoha [Tue, 19 Mar 2013 13:13:44 +0000 (14:13 +0100)]
vgextend: do not allow PV with 0 MDAs to be added while already in a VG
If extending a VG and including a PV with 0 MDAs that was already
a part of a VG, the vgextend allowed that PV to be added and we
ended up *with one PV in two VGs*!
The vgextend code used the 'pv_by_path' fn that returned a PV for
a given path. However, when the PV did not have any metadata areas,
the fn just returned a PV without any reference to existing VG.
Consequently, any checks for the existing VG failed.
[0] raw/~ # vgcreate vg2 /dev/sdc
Volume group "vg2" successfully created
Before this patch (incorrect):
[0] raw/~ # vgextend vg2 /dev/sda
Volume group "vg2" successfully extended
With this patch (correct):
[0] raw/~ # vgextend vg2 /dev/sda
Physical volume '/dev/sda' is already in volume group 'vg1'
Unable to add physical volume '/dev/sda' to volume group 'vg2'.
Peter Rajnoha [Tue, 19 Mar 2013 13:05:19 +0000 (14:05 +0100)]
metadata: add 'allow_orphan' arg to find_pv_by_name fn
Before, the find_pv_by_name call always failed if the PV found was orphan.
However, we might use this function even for a PV that is not part of any VG.
This patch adds 'allow_orphan' arg to find_pv_by_name fn that allows that.
The only callers of the underscored variants were their wrappers
without the underscore. No other part of the code referenced the
underscored variants.
Zdenek Kabelac [Mon, 11 Mar 2013 08:49:10 +0000 (09:49 +0100)]
thin: rework lvconvert
Usage of layer was not the best plan here - for proper devices stack
we have to keep correct reference in volume_group structure and
make the new thin pool LV appear as a new volume.
Peter Rajnoha [Tue, 12 Mar 2013 11:37:24 +0000 (12:37 +0100)]
dmsetup: fix 'splitname -o' to not fail if used without '-c'
This was a regression introduced with e33fd978a8a56228677c14612fce99b3b3588125
(libdm v1.02.68/lvm2 v2.02.89) with the introduction of new output
fields blkdevname and blkdevs_used for ls and deps dmsetup commands.
A new common '_process_options' fn was added with that commit, but the
fn was called prematurely which then broke processing of
'dmsetup splitname -o' which should implicitly use '-c' option
and this was failing after the commit:
alatyr/~ $ dmsetup splitname -o lv_name /dev/mapper/vg_data-test
Option not recognised: lv_name
Couldn't process command line.
The '-c' had to be used for correct operation:
alatyr/~ $ dmsetup splitname -c -o lv_name /dev/mapper/vg_data-test
LV
test
Now fixed to work as it did before:
alatyr/~ $ dmsetup splitname -o lv_name /dev/mapper/vg_data-test
LV
test
test: Fix the way in_sync calculates the sync field location
'in_sync' was using the last field in the RAID status output as
the location for the sync ratio field. The sync ratio may not always
be the last field, but it will always be the 7th field. So we switch
to using the absolute value rather than computing the last field
index number.
RAID: Code changes missing from previous commit (bbc6378)
Previous commit included changes to WHATSNEW, but the code changes
were missing. Here is the description from the previous commit:
commit bbc6378b73246d7ef00c16274fea38118531ae95
Author: Jonathan Brassow <jbrassow@redhat.com>
Date: Thu Feb 21 11:31:36 2013 -0600
RAID: Make 'lvchange --refresh' restore transiently failed RAID PVs
A new function (dm_tree_node_force_identical_table_reload) was added to
avoid the suppression of identical table reloads. This allows RAID LVs
to reload the on-disk superblock information that contains which devices
have failed and the bitmaps. If the failed device has returned, this has
the effect of restoring the device and initiating recovery. Without this
patch, the user had to completely deactivate their RAID LV and re-activate
it in order to restore the failed device. Now they simply need to
suspend and resume (which is done by 'lvchange --refresh').
The identical table suppression is only avoided if the LV is not PARTAIL
(i.e. all of it's devices can be seen and read by LVM) and the kernel
status of the array contains failed devices. In other words, the function
will only be called in the case where we may have success in restoring
a failed device in the array.
Peter Rajnoha [Tue, 5 Mar 2013 17:21:13 +0000 (18:21 +0100)]
dumpconfig: add --withcomments and --withversions switch
lvm dumpconfig [--withcomments] [--withversions]
The --withcomments causes the comments to appear on output before each
config node (if they were defined in config_settings.h).
The --withversions causes a one line extra comment to appear on output
before each config node with the version information in which the
configuration setting first appeared.
Peter Rajnoha [Tue, 5 Mar 2013 17:02:13 +0000 (18:02 +0100)]
config: add support for enhanced config node output
There's a possibility to interconnect the dm_config_node with an
ID, which in our case is used to reference the configuration
definition ID from config_settings.h. So simply interconnecting
struct dm_config_node with struct cfg_def_item.
This patch also adds support for enhanced config node output besides
existing "output line by line". This patch adds a possibility to
register a callback that gets called *before* the config node is
processed line by line (for example to include any headers on output)
and *after* the config node is processed line by line (to include any
footers on output). Also, it adds the config node reference itself
as the callback arg in addition to have a possibility to extract more
information from the config node itself if needed when processing the
output callback (e.g. the key name, the id, or whether this is a
section or a value etc...).
If the config node from lvm.conf/--config tree is recognized and valid,
it's always coupled with the config node definition ID from
config_settings.h:
For example if the dm_config_node *cn holds "devices/dev" configuration,
then the cn->id holds "devices_dev_CFG" ID from config_settings.h, -1 if
not found in config_settings.h and 0 if matching has not yet been done.
To support the enhanced config node output, a new structure has been
defined in libdevmapper to register it:
struct dm_config_node_out_spec {
dm_config_node_out_fn prefix_fn; /* called before processing config node lines */
dm_config_node_out_fn line_fn; /* called for each config node line */
dm_config_node_out_fn suffix_fn; /* called after processing config node lines */
};
(so in comparison to existing callbacks for config node output, it has
an extra dm_config_node *cn arg in addition)
This patch also adds these functions to libdevmapper:
- dm_config_write_node_out
- dm_config_write_one_node_out
...which have exactly the same functionality as their counterparts
without the "out" suffix. The "*_out" functions adds the extra hooks
for enhanced config output (prefix_fn and suffix_fn mentioned above).
One can still use the old interface for config node output, this is
just an enhancement for those who'd like to modify the output more
extensively.
This patch adds above-mentioned args to lvm dumpconfig and it maps them
to creation and writing out a configuration tree of a specific type
(see also previous commit):
- current maps to CFG_TYPE_CURRENT
- default maps to CFG_TYPE_DEFAULT
- missing maps to CFG_TYPE_MISSING
- new maps to CFG_TYPE_NEW
If --type is not defined, dumpconfig defaults to "--type current"
which is the original behaviour of dumpconfig before all these changes.
The --validate option just validates current configuration tree
(lvm.conf/--config) and it writes a simple status message:
"LVM configuration valid" or "LVM configuration invalid"
Peter Rajnoha [Tue, 5 Mar 2013 16:36:10 +0000 (17:36 +0100)]
config: use config checks and add support for creating trees from config definition (config_def_create_tree fn)
Configuration checking is initiated during config load/processing
(_process_config fn) which is part of the command context
creation/refresh.
This patch also defines 5 types of trees that could be created from
the configuration definition (config_settings.h), the cfg_def_tree_t:
- CFG_DEF_TREE_CURRENT that denotes a tree of all the configuration
nodes that are explicitly defined in lvm.conf/--config
- CFG_DEF_TREE_MISSING that denotes a tree of all missing
configuration nodes for which default valus are used since they're
not explicitly used in lvm.conf/--config
- CFG_DEF_TREE_DEFAULT that denotes a tree of all possible
configuration nodes with default values assigned, no matter what
the actual lvm.conf/--config is
- CFG_DEF_TREE_NEW that denotes a tree of all new configuration nodes
that appeared in given version
- CFG_DEF_TREE_COMPLETE that denotes a tree of the whole configuration
tree that is used in LVM2 (a combination of CFG_DEF_TREE_CURRENT +
CFG_DEF_TREE_MISSING). This is not implemented yet, it will be added
later...
The function that creates the definition tree of given type:
Where the "spec" specifies the tree type to be created:
struct config_def_tree_spec {
cfg_def_tree_t type; /* tree type */
uint16_t version; /* tree at this LVM2 version */
int ignoreadvanced; /* do not include advanced configs */
int ignoreunsupported; /* do not include unsupported configs */
};
This tree can be passed to already existing functions that write
the tree on output (like we already do with cmd->cft).
There is a new lvm.conf section called "config" with two new options:
- config/checks which enables/disables checking (enabled by default)
- config/abort_on_errors which enables/disables aborts on any type of
mismatch found in the config (disabled by default)
Peter Rajnoha [Tue, 5 Mar 2013 16:14:18 +0000 (17:14 +0100)]
config: add support for configuration check (config_def_check fn)
Add support for configuration checking - type checking and recognition
of registered configuration settings that LVM2 understands and also
check the structure of the configuration. Log error on any mismatch
found.
A hash over all allowed configuration paths is created which helps
with matching the exact configuration (lvm.conf/--config tree) with
the configuration item definition from config_settings.h in an
efficient and one-step way.
Two more helper flags are introduced for each configuration definition
item:
- CFG_USED which marks the item as being used (lvm.conf/--config)
This helps with identifying missing configuration options
(and for which defaults were used) when traversing the tree later.
- CFG_VALID which denotes that the item has already been checked and
it was found valid. This improves performance, so if the check
is called once again on the same tree which was not reloaded, we
can just return the state from previous check (with a possibility
to force the check if needed).
The new function that config.h exports and which is going to be used
to perform the configuration checking is:
int config_def_check(struct cmd_context *cmd, int force, int skip, int suppress_messages)
Peter Rajnoha [Tue, 5 Mar 2013 16:10:30 +0000 (17:10 +0100)]
libdevmapper: add dm_config_value_is_bool function
Export this functionality from libdevmapper just for
convenience and general use when reading boolean values
which could be defined either in a numeric way with 0/1
or by using strings with "true"/"false", "yes"/"no",
"on"/"off", "y"/"n".
So we're referring to the named configuration ID instead
of passing the configuration path and the default value
is taken from central config definition in config_settings.h
automatically.
Peter Rajnoha [Tue, 5 Mar 2013 15:49:42 +0000 (16:49 +0100)]
config: add structs to represent config definition and register config_settings.h content
This patch adds basic structures that encapsulate the config_settings.h
content - it takes each item and puts it in structures:
- cfg_def_type_t to define config item type
- cfg_def_value_t to define config item (default) value
- flags used to define the nature and use of the config item:
- CFG_NAME_VARIABLE for items with variable names (e.g. tags)
- CFG_ALLOW_EMPTY for items where empty value is allowed
- CFG_ADVANCED for items which are considered as "advanced settings"
- CFG_UNSUPPORTED for items which are not officially supported
(config options mostly for internal use and testing/debugging)
- cfg_def_item_t to encapsulate the whole definition of the config
definition itself
Each config item is referenced by named ID, e.g. "devices_dir_CFG"
instead of directly typing the path "devices/dir" as it was before.
This patch also adds cfg_def_get_path helper function to get the
config setting path up to the root for given config ID
(it returns the path in form of "abc/def/.../xyz" where the "abc"
is the topmost element).