[cygwin] DD bug fails to wipe last 48 sectors of a disk

[Brian Inglis]

On 2020-12-28 19:41, Jason Pyeron wrote:
> On Monday, December 28, 2020 7:46 PM, Hashim Aziz wrote:
>> On 23 June 2020 8:33 PM, Brian Inglis wrote:
>>> I don't have the facilities to test, and there appear to be *NO* Windows 
>>> documentation details on error condition handling, but my suspicion is
>>> that Unix reads and writes fail only *AFTER* reading or writing at the
>>> end of the device, but Windows reads and writes extents may be checked
>>> and failed *BEFORE* reading or writing any data near the end of the
>>> device.
>>> If the actual Windows error code returned is generic, Cygwin would need
>>> to pre-check the device size as Windows does, and reduce read and write
>>> sizes to the allowed maximum at the end of the device.

>> That's very helpful, thank you. Do you know if any more work has been done 
>> to attempt to fix this bug, and whether it's likely to be fixed anytime
>> soon? It's crazy that such a commonly used command leaves so much data
>> unwiped unbeknown to so many users, it's a very serious security hole and
>> the sooner it can be fixed the better.

> Have you tried iflag=fullblock ? This causes special handling.

>> I didn't previously see this email, but the point is that this is a bug - 
>> dd should not require first making calculations based on the size of each
>> drive or using the smallest possible block size (and hence taking a
>> ridiculous amount of time) in order to do what

> Do you have any metrics that it is faster, by any meaningful amount? If so I 
> would be very interested in mitigating it, but I suspect not the actual
> case.

>> it's meant to do. It should always wipe the last sector of the drive
>> regardless, just as it does on other UNIX-like systems. This is why this
>> behaviour is a bug that needs to be fixed.

> This does not appear to be a bug, but user error. Per the DD source "Some
> devices require alignment on a sector or page boundary"
> DD has never "dealt with error handling" except when conversion were in play.
> When no conversions are in play it

>              {
>                /* Write any partial block. */
>                exit_status = EXIT_FAILURE;
>                break;
>              }

> On windows the block devices require respecting block device boundaries, any 
> change would be an upstream patch - not a Cygwin patch.

Your dd output appears to be ambiguous, relative to your claim that the last 48 
sectors are not written, and may appear to indicate that all sectors of the 
drive may have been written, assuming that you mean 512 byte sectors.

> 1000182120448 bytes (1.0 TB, 931 GiB) copied, 8284 s, 121 MB/s

1000182120448 == 238462*4*1024^2

> dd: error writing '/dev/sda': No space left on device
> 238468+0 records in

1000207286272 == 238468*4*1024^2

> 238467+0 records out

1000203091968 == 238467*4*1024^2

> 1000204861440 bytes (1.0 TB, 932 GiB) copied, 8284.89 s, 121 MB/s

1000204861440 == 238467*4*1024^2 + 27*64*1024

None of these numbers +/-48*512 bytes, which have odd factors, make a lot of 
sense as a disk size.

Could you please state explicitly, how many bytes/sectors/blocks/pages/clusters 
of what size you expect to get written, and how many 
bytes/sectors/blocks/pages/clusters of what size are actually written?

If anyone has access to a Linux system which has write access to a Windows drive 
over the network (e.g. Samba, NFS) where this can be reproduced, we can try to 
take this upstream, get their take, suggest an incremental reseek and write half 
buffer size patch, if they agree this is an issue and could be tackled in this 
manner.

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.
[Data in binary units and prefixes, physical quantities in SI.]