This is the mail archive of the
mailing list for the Cygwin project.
Re: grep treating my text files as binary!
- From: Warren Young <wyml at etr-usa dot com>
- To: The Cygwin Mailing List <cygwin at cygwin dot com>
- Date: Fri, 26 Dec 2014 19:00:38 -0700
- Subject: Re: grep treating my text files as binary!
- Authentication-results: sourceware.org; auth=none
- References: <XnsA40D81CA1FAA8davidrayninfocouk at 80 dot 91 dot 229 dot 13> <549B4258 dot 5050509 at redhat dot com> <XnsA40DECB2AE256davidrayninfocouk at 80 dot 91 dot 229 dot 13> <549C5A6B dot 2000509 at towo dot net>
On Dec 25, 2014, at 11:41 AM, Thomas Wolff <firstname.lastname@example.org> wrote:
> In any case the argument is quite artificial since the new behaviour
> hits many files that are in fact text files.
Please define the term “text file” in a way that allows a C programmer to write a program that automatically does the correct thing for all members of the class “text file” without involving locales, or an equivalent mechanism.
Just because you, the human, can use your superior pattern matching skills to see that a file is a “text file” doesn’t mean that a relatively dumb program like grep(1) can. You can’t expect someone to build an AI system into grep(1) just to get automatic locale detection.
If grep runs into a byte sequence that makes it think it is not legal for your current locale, it must treat the file as raw bytes, unless you give it -a.
If you don’t like this behavior, say “alias grep=grep -a” in your ~/.bashrc, and forget the change ever happened. It’ll be on you when some non-text file gets treated as text and grep spams your terminal with binary garbage, though.
This isn’t really a Cygwin problem. It just happens to affect it more than other *ix systems because there are two sets of rules on the same system, and they may conflict. But, if I go and copy a UTF-16LE file to a Linux box, I’m not going to complain to the grep bug list when grep doesn’t automatically do the right thing with it while $LANG contains UTF-8.
Ultimately, the proper solution is to use UTF-8 on all systems you use Cygwin on. Many Unicode-aware native Windows programs that deal with text files can cope with UTF-8.
If you have one of those that demands UTF-16LE, iconv(1) can make the conversion for you.
Problem reports: http://cygwin.com/problems.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple