textmode for stdout, what is "correct" now?

[Michael Haubenwallner]

On 2/14/19 5:20 PM, Corinna Vinschen wrote:
> On Feb 14 16:23, Michael Haubenwallner wrote:
>> Hi,
>>
>> so I find myself struggling with textmode versus binmode for stdio again.
>>
>> Running the openssl command (from within the apps/ build directory here) does
>> yield different results regarding carriage return depending on the version:
>>
>> $ ./apps/openssl version
>> OpenSSL 1.0.2p  14 Aug 2018
>> $ ./apps/openssl x509 -hash -noout -in /etc/pki/tls/cert.pem | xxd
>> 00000000: 6139 3464 3039 6535 0a                   a94d09e5.
>>
>>
>> $ ./apps/openssl version
>> OpenSSL 1.1.0j  20 Nov 2018
>> $ ./apps/openssl x509 -hash -noout -in /etc/pki/tls/cert.pem | xxd
>> 00000000: 6139 3464 3039 6535 0d0a                 a94d09e5..
>>
>> Some subsequent shell script does create wrong symlink filenames
>> (with embedded CR) when used with openssl-1.1.x.
>>
>> The commit that changed this behaviour in openssl-1.1 is:
>> https://github.com/openssl/openssl/commit/bdd58d98467e9f0f6635c1628e1eae304383afb1
>>
>> >From an openssl developer's point of view, I can understand to set
>> textmode when the intent is to output some text, and to set
>> binmode when the intent is to output some binary data.
> 
> How do you create \r\n in this case?  The upstream patch never
> adds the explicit 't' flag.  It only adds 'b' or nothing.  So
> the output should be \n only unless you write to a file on a
> text mode mount.  What am I missing?

Down the line in their BIO module they do use setmode(fd, O_TEXT),
which is the one that does introduce the \r, as far as I know.

The backtrace in openssl-1.1.1a in this use case is:

https://github.com/openssl/openssl/blob/OpenSSL_1_1_1-stable/crypto/bio/bss_file.c#L257 in file_ctrl()
https://github.com/openssl/openssl/blob/OpenSSL_1_1_1-stable/crypto/bio/bio_lib.c#L528 in BIO_ctrl()
https://github.com/openssl/openssl/blob/OpenSSL_1_1_1-stable/crypto/bio/bss_file.c#L104 in BIO_new_fp()
https://github.com/openssl/openssl/blob/OpenSSL_1_1_1-stable/apps/apps.c#L2471 in dup_bio_out()
https://github.com/openssl/openssl/blob/OpenSSL_1_1_1-stable/apps/apps.c#L2624 in bio_open_default_()
https://github.com/openssl/openssl/blob/OpenSSL_1_1_1-stable/apps/apps.c#L2652 in bio_open_default()
https://github.com/openssl/openssl/blob/OpenSSL_1_1_1-stable/apps/x509.c#L595 in x509_main()
https://github.com/openssl/openssl/blob/OpenSSL_1_1_1-stable/apps/openssl.c#L564 in do_cmd()
https://github.com/openssl/openssl/blob/OpenSSL_1_1_1-stable/apps/openssl.c#L183 in main()

> 
>> Question now is: These days, what is the correct way to handle this?
>>
>> Is it up to openssl to not use textmode at all with Cygwin (what if
>> used from within some cmd.exe?), or is it up to the shell script to
>> explicitly drop any carriage return here?
> 
> They way openssl does it looks right.  Off the top of my head I don't 
> grok where you get the \r from in the example above.

So a "correct" and portable way in shell scripts really is something like this?
$ var=`openssl ... | tr -d "\\r"`
or even (with bash or similar):
$ var=$(openssl ...); var=${var%$'\r'}

Thanks!
/haubi/

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple