This is the mail archive of the cygwin mailing list for the Cygwin project.

Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: sort utility goes berzerk (x86_64)

On 2017-11-28 14:37, Doug Henderson wrote:

On 2017-11-25 14:23, Houder wrote:
> Hi,
> Anyone seeing this as well? sort goes berzerk on my system when piped
> into
> head (or less) when it is fed with a 'specially prepared' input file.
>  - only happens on x86_64
>  - does not happen for 'LC_COLLATE=C sort tt | head'
> 'specially prepared' input file? (see bottom of post).

Anyone ** NOT ** seeing this?

Q4. After you kill a hung sort, are there files left over in /tmp? I
am wondering if the problem files are too big to do an in-core sort,
and something goes wrong when it switches to a temp file sort, but not
for smaller or larger input files where it correctly predicts which
type of sort it needs to do. Possibly using the "--buffer-size=N"
option will change the behavior, making smaller input files fail and
failing ones work.

Preliminary result:

# tt = 8150 lines
sort -S 1024 tt | head # 1024 * 1024 = 1M
sort -S 1152 tt | head # + 128
sort -S 1280 tt | head # + 256 <==== fail (1226?)

# ttt = 5050 lines
sort -S 512 ttt | head # 512 * 1024 = .5M
sort -S 640 ttt | head # + 128
sort -S 768 ttt | head # + 256 <==== fail (760?)

These numbers are correct for --parallel=8 (the eight "cores" that are
in my machine)

Specifying a smaller number of cores, also make the above numbers grow

All of the above is about tuning the performance of sort; however it is
not an explanation for the way in which sort fails.

sort should not fail in the way it does; it does not on "Cygwin/x86" on
my machine; it does not on Linux (Fedora 24) (same machine).


Problem reports:
Unsubscribe info:

Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]