1.4.5 gnu sort 2.0.16 extremely slow under Win98 (tiny default buffer)
Patrick Tufts
zippy@cs.brandeis.edu
Tue Feb 19 11:03:00 GMT 2002
I looked at the temp files sort creates and noticed that they were around
700 bytes (bytes, not kilobytes) in size. The normal behavior of sort is to
create temp files of around 16kB in size.
When I use the --buffer-size option to force a large buffer,
like --buffer-size=200000 (200MB) sort completes in seconds.
I'm running sort like this:
cat <47k line file> | perl -ne "<script that returns 1000 lines from the
source file>" | sort -k 1nr,1 -k 2,2 -o outputfile
The perl script on its own takes a second or less to run - it's a simple
filter.
--Pat
Cygnus Win95/NT Configuration Diagnostics
Current System Time: Tue Nov 20 17:19:00 2001
Windows 98 SE Ver 4.10 build 67766446 A=20
Path: C:\bin\lang\cygwin\usr\local\bin
C:\bin\lang\cygwin\bin
C:\bin\lang\cygwin\bin
c:\WINDOWS
c:\WINDOWS\COMMAND
c:\BIN\APPS\PGP
SysDir: C:\WINDOWS\SYSTEM
WinDir: C:\WINDOWS
HOME =3D `C:\bin\lang\cygwin\home\ptufts'
MAKE_MODE =3D `unix'
PWD =3D `/cygdrive/c/Windows/'
USER =3D `ptufts'
BLASTER =3D `A220 I5 D1'
CMDLINE =3D `bash --login -i'
COMSPEC =3D `C:\WINDOWS\COMMAND.COM'
OLDPWD =3D `/cygdrive/c'
PROMPT =3D `$p$g'
PS1 =3D `\[\033]0;\w\007
\033[32m\]\u@\h \[\033[33m\w\033[0m\]
$ '
SHLVL =3D `1'
TEMP =3D `c:\WINDOWS\TEMP'
TERM =3D `cygwin'
TMP =3D `c:\WINDOWS\TEMP'
WINBOOTDIR =3D `C:\WINDOWS'
WINDIR =3D `C:\WINDOWS'
_ =3D `/usr/bin/cygcheck'
HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\Menu=
Order\Start Menu\&Programs\Cygnus Solutions
(default) =3D (unsupported type)
HKEY_CURRENT_USER\Software\Cygnus Solutions
HKEY_CURRENT_USER\Software\Cygnus Solutions\Cygwin
HKEY_CURRENT_USER\Software\Cygnus Solutions\Cygwin\mounts v2
(default) =3D `/cygdrive'
cygdrive flags =3D 0x00000020
HKEY_CURRENT_USER\Software\Cygnus Solutions\Cygwin\Program Options
HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions
HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin
HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2
HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2\/
(default) =3D `C:/bin/lang/cygwin/'
flags =3D 0x00000008
HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2\/usr/bin
(default) =3D `C:/bin/lang/cygwin/bin'
flags =3D 0x00000008
HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2\/usr/lib
(default) =3D `C:/bin/lang/cygwin/lib'
flags =3D 0x00000008
HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\Program Options
a: fd N/A N/A =20
c: hd FAT32 4848Mb 98% CP UN HARDDRIVE
. /cygdrive user textmode,noumount
C:/bin/lang/cygwin/ / system textmode
C:/bin/lang/cygwin/bin /usr/bin system textmode
C:/bin/lang/cygwin/lib /usr/lib system textmode
Found: C:\bin\lang\cygwin\bin\bash.exe
Found: C:\bin\lang\cygwin\bin\cat.exe
Not Found: cpp (good!)
Found: C:\bin\lang\cygwin\bin\find.exe
Found: c:\WINDOWS\COMMAND\find.exe
Warning: C:\bin\lang\cygwin\bin\find.exe hides =
c:\WINDOWS\COMMAND\find.exe
Not Found: gcc
Not Found: gdb
Not Found: ld
Found: C:\bin\lang\cygwin\bin\ls.exe
Not Found: make
Found: C:\bin\lang\cygwin\bin\sh.exe
18k 2000/10/23 C:\bin\lang\cygwin\bin\cyggdbm.dll - os=3D4.0 =
img=3D1.0 sys=3D4.0
"cyggdbm.dll" v0.0 ts=3D2000/10/22 19:26
45k 2001/04/25 C:\bin\lang\cygwin\bin\cygform5.dll - os=3D4.0 =
img=3D1.0 sys=3D4.0
"cygform5.dll" v0.0 ts=3D2001/4/24 22:28
26k 2001/04/25 C:\bin\lang\cygwin\bin\cygmenu5.dll - os=3D4.0 =
img=3D1.0 sys=3D4.0
"cygmenu5.dll" v0.0 ts=3D2001/4/24 22:27
156k 2001/04/25 C:\bin\lang\cygwin\bin\cygncurses++5.dll - os=3D4.0 =
img=3D1.0 sys=3D4.0
"cygncurses++5.dll" v0.0 ts=3D2001/4/24 22:29
226k 2001/04/25 C:\bin\lang\cygwin\bin\cygncurses5.dll - os=3D4.0 =
img=3D1.0 sys=3D4.0
"cygncurses5.dll" v0.0 ts=3D2001/4/24 22:17
15k 2001/04/25 C:\bin\lang\cygwin\bin\cygpanel5.dll - os=3D4.0 =
img=3D1.0 sys=3D4.0
"cygpanel5.dll" v0.0 ts=3D2001/4/24 22:27
34k 2001/09/30 C:\bin\lang\cygwin\bin\cygform6.dll - os=3D4.0 =
img=3D1.0 sys=3D4.0
"cygform6.dll" v0.0 ts=3D2001/9/29 19:43
19k 2001/09/30 C:\bin\lang\cygwin\bin\cygmenu6.dll - os=3D4.0 =
img=3D1.0 sys=3D4.0
"cygmenu6.dll" v0.0 ts=3D2001/9/29 19:43
175k 2001/09/30 C:\bin\lang\cygwin\bin\cygncurses++6.dll - os=3D4.0 =
img=3D1.0 sys=3D4.0
"cygncurses++6.dll" v0.0 ts=3D2001/9/29 19:45
201k 2001/09/30 C:\bin\lang\cygwin\bin\cygncurses6.dll - os=3D4.0 =
img=3D1.0 sys=3D4.0
"cygncurses6.dll" v0.0 ts=3D2001/9/29 19:42
12k 2001/09/30 C:\bin\lang\cygwin\bin\cygpanel6.dll - os=3D4.0 =
img=3D1.0 sys=3D4.0
"cygpanel6.dll" v0.0 ts=3D2001/9/29 19:43
17k 2001/06/28 C:\bin\lang\cygwin\bin\cyghistory4.dll - os=3D4.0 =
img=3D1.0 sys=3D4.0
"cyghistory4.dll" v0.0 ts=3D2001/1/6 20:34
21k 2001/06/28 C:\bin\lang\cygwin\bin\cyghistory5.dll - os=3D4.0 =
img=3D1.0 sys=3D4.0
"cyghistory5.dll" v0.0 ts=3D2001/6/27 19:27
108k 2001/06/28 C:\bin\lang\cygwin\bin\cygreadline4.dll - os=3D4.0 =
img=3D1.0 sys=3D4.0
"cygreadline4.dll" v0.0 ts=3D2001/1/6 20:34
135k 2001/06/28 C:\bin\lang\cygwin\bin\cygreadline5.dll - os=3D4.0 =
img=3D1.0 sys=3D4.0
"cygreadline5.dll" v0.0 ts=3D2001/6/27 19:27
49k 2001/02/03 C:\bin\lang\cygwin\bin\cygz.dll - os=3D4.0 img=3D1.0 =
sys=3D4.0
"cygz.dll" v0.0 ts=3D2001/2/3 12:35
21k 2001/06/20 C:\bin\lang\cygwin\bin\cygintl.dll - os=3D4.0 =
img=3D1.0 sys=3D4.0
"cygintl.dll" v0.0 ts=3D2001/6/20 10:09
119k 2001/06/06 C:\bin\lang\cygwin\bin\cygjpeg6b.dll - os=3D4.0 =
img=3D1.0 sys=3D4.0
"cygjpeg6b.dll" v0.0 ts=3D2001/6/5 21:27
714k 2001/11/14 C:\bin\lang\cygwin\bin\cygwin1.dll - os=3D4.0 =
img=3D1.0 sys=3D4.0
"cygwin1.dll" v0.0 ts=3D2001/11/13 20:16
Cygwin DLL version info:
DLL version: 1.3.5
DLL epoch: 19
DLL bad signal mask: 19005
DLL old termios: 5
DLL malloc env: 28
API major: 0
API minor: 47
Shared data: 3
DLL identifier: cygwin1
Mount registry: 2
Cygnus registry name: Cygnus Solutions
Cygwin registry name: Cygwin
Program options name: Program Options
Cygwin mount registry name: mounts v2
Cygdrive flags: cygdrive flags
Cygdrive prefix: cygdrive prefix
Cygdrive default prefix:=20
Build date: Tue Nov 13 23:16:25 EST 2001
CVS tag: cygwin-1-3-5-2
Shared id: cygwin1S3
Cygwin Package Information
Last downloaded files to: C:\My Documents
Last downloaded files from: =
ftp://ftp.nas.nasa.gov/mirrors/cygwin.com/pub/cygwin
Package Version =20
ash 20011018-1 =20
bash 2.05a-1 =20
clear 1.0 =20
crypt 1.0-1 =20
cygrunsrv 0.94-2 =20
cygwin 1.3.5-3 =20
diff 0.0 =20
fileutils 4.1-1 =20
findutils 0.0 =20
gdbm 1.8.0-3 =20
gettext 0.10.38-2 =20
grep 2.4.2-1 =20
gzip 1.3-1 =20
inetutils 1.3.2-15 =20
jpeg 6b-4 =20
less 358-3 =20
libncurses5 5.2-1 =20
libncurses6 5.2-2 =20
login 1.4-2 =20
lynx 2.8.4-1 =20
ncftp 3.0.2-2 =20
ncurses 5.2-7 =20
openssh 3.0p1-1 =20
openssl 0.9.6b-1 =20
perl 5.6.1-2 =20
readline 4.2-3 =20
regex 4.4 =20
sed 3.02-1 =20
sh-utils 2.0-2 =20
squid 2.4-STABLE20010508 =20
tar 1.13.19-1 =20
termcap 20010825-1 =20
terminfo 5.2-1 =20
textutils 2.0.16-1 =20
vim 6.0.93-1 =20
w32api 1.1-1 =20
wget 1.7-1 =20
which 1.5-1 =20
zlib 1.1.3-6 =20
Use -h to see help about each section
--
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Bug reporting: http://cygwin.com/bugs.html
Documentation: http://cygwin.com/docs.html
FAQ: http://cygwin.com/faq/
More information about the Cygwin
mailing list