grep treating my text files as binary!

[Eric Blake]

On 12/25/2014 11:41 AM, Thomas Wolff wrote:
> I've read the POSIX definition of "binary file" that was quoted in the
> grep bug already,
> and if I remember correctly (or how this is abbreviated here...) it does
> not mention character encoding or locale.

Ah, but it does.

http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_397

"Text File
A file that contains characters organized into zero or more lines."

http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_87

"Character
A sequence of one or more bytes representing a single graphic symbol or
control code.

Note:
    This term corresponds to the ISO C standard term multi-byte
character, where a single-byte character is a special case of a
multi-byte character. Unlike the usage in the ISO C standard, character
here has no necessary relationship with storage space, and byte is used
when storage space is discussed.

    See the definition of the portable character set in Portable
Character Set for a further explanation of the graphical representations
of (abstract) characters, as opposed to character encodings."

If you have a file that contains byte sequences that are not characters
in the current locale, then that file is NOT a text file.  It might be a
mostly-text file, and it might even be a text file if you switch to the
correct locale, but the point is that the POSIX definition of text file
is that character encoding errors in the current locale MAKE a file
binary, at which point behavior is unspecified.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 604 bytes
Desc: OpenPGP digital signature
URL: <http://cygwin.com/pipermail/cygwin/attachments/20150102/ea1a46f7/attachment.sig>