Device-mapper loopback target

Overview

The dm-loop target provides integrated loopback device support for device-mapper. This is similar to the standard Linux /dev/loop in that it allows a regular file to be treated as if it were a block device. With dm-loop, it is possible to create far more than the 256 loop devices possible with the regular loop driver. The upper limit varies between systems depending on available memory. Test runs regularly use >2000 loop devices and we have reports of 10s of thousands of devices being successfully created on a system with 32GiB of RAM.

Implementation

The loop target supports two different types of loopback device mapping. The target automatically selects the type to use when a device is created based on the characteristics of the backing file. In the case that the file is stored on a local block-device backed file system dm-loop will attempt to use block mapped mode. For networked file systems, file systems without a block device or backing files containing "holes" (i.e. sparse files) the target will automatically fall back to file mapped mode.

Block mapping

Block mapping aims to give better performance for the common case of a loop file stored on a local, block-device backed file system. This is achived using a technique similar to that currently in use for Linux swap file support. When a loop device is first created the on-disk layout is determined and a table of extents is built. These extents map loop device sectors to their physical location on the host device. This means the when I/O passes through the loop device the target just needs to find the correct extent in the table and re-map the I/O request to the new sector and device. This avoids making calls to the file system layer (VFS) during I/O and avoids memory allocation during I/O. Extents are stored in a linear table and lookup is implemented using a simple binary search.

Due to the extra work being done mapping all extents in the file, a dm-loop device will take longer to setup than a traditional /dev/loop device. This is one-off cost at setup time and allows dm-loop to achieve higher throughput and more predictable behavior once the device has been initialised. The exact time taken depends on the file size, file sysyem type and the degree of fragmentation present.

File mapping

File mapping makes use of the same approach as the regular loopback driver. That is, we receive I/O requests for the loop device and submit them via a kernel thread to the VFS layer using traditional I/O calls (read, write etc.). This has the advantage that it should work with any file system type supported by the Linux VFS (including networked file systems), but has some drawbacks that may affect performance and scalability. This is because it is hard to predict what a file system may attempt to do when an I/O request is submitted; for example, it may need to allocate memory to handle the request and the loopback driver has no control over this. Particularly under low-memory or intensive I/O scenarios this can lead to out of memory (OOM) problems or deadlocks as the kernel tries to make memory available to the VFS layer while satisfying a request from the block layer.

Development status

The loop target is not currently merged in the mainstream kernel but we hope to be in a position to submit the patch for review and possible inclusion in the near future. The current patches have been in development for about 9 months and have recieved fairly heavy testing on a variety of hosts and file system types. There are no serious issues known to the developers at the moment but it is wise to treat dm-loop as you would any development code.

Development patches are regularly posted to the dm-devel mailing list and are also available as attachments to this wiki page:

The patches should apply cleanly to a vanilla 2.6.20 kernel. If using a later git tree, building dm-loop will produce a deprecation warning for the function invalidate_inode_pages. The following patch converts dm-loop to use the invalidate_mapping_pages function directly.

Older kernels will need addititonal patches that are now merged into the mainstream kernel releases. Please contact the dm-loop maintainers if you need to use an older kernel and are unsure what patches you need.

Table format

The table format is straightforward: 

loop <file path> <offset>

The offset parameter describes a byte-offset into the file at which the loop mapping will begin. Currently this offset must be a multiple of the file system block size.

The table format can be used directly to create loop devices in the normal way: 

# echo 0 2048 loop /tmp/img0 0 | dmsetup create loop0

or using the --table option available in recent device-mapper releases: 

# dmsetup create loop0 --table "0 2048 loop /tmp/img0 0"

Managing loop devices with dmlosetup

Simplified support for creating dm-loop devices is available in current versions of dmsetup by creating a symbolic link to the dmsetup binary with the name "dmlosetup" or "losetup". When dmsetup is called this way it takes similar options to the standard losetup command. To create a device, just supply a name (loopN) and the path to the image file. Devices may be removed using the -d option. For example: 

# dmlosetup loop0 /tmp/img0
# dmlosetup -d loop0

Reporting problems with dm-loop

The dm-loop target is still under development and we appreciate your feedback and problem reports. Please send problem reports to the dm-devel mailing list, CC'ing breeves AT redhat DOT com. When reporting bugs against dm-loop, please try to include the following information:

If we are unable to resolve your issue with this information, we may ask you to collect further details from the system to aid debugging (for e.g. the on-disk layout of any problematic files).

TODO: upload scripts to grab this stuff.

None: DMLoop (last edited 2008-01-10 19:42:37 by localhost)