xz archiver has a problem compressing a large volume in multithread mode
Tetsuji Rai via cygwin
Wed Nov 15 05:41:00 GMT 2017
On 11/14/2017 12:37, L A Walsh wrote:
> Tetsuji Rai via cygwin wrote:
>> Hi all,
>> I have many music files, so I often archive music directory with "tar
>> -cJvf music.tar.xz music" with environment variable XZ_OPTS="-T 3" (I
>> have 4 threaded first generation i7).Â Â Â The archive is about 30GB.
>> Until some weeks ago it worked fine.Â Â However since I updated to the
>> latest version of cygwin64 some weeks ago, I have a trouble extracting
>> from that archive at some point.Â "tar -xJvf music.tar.xz" says "archive
>> file is corrupt" or something and quits.Â Â T
> Â Â Have you tried it with one xz thread?...
> BTW to compress/decompress w/xz, I usually use
> Â tar caf /tmp/archive.txz /music_archiveÂ Â to compress and
> Â tar xaf /tmp/archive.txzÂ Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â to uncompress
> (but those should be unrelated to your problem).
> I just tried "-T 3" on my version of tar and with "xz"
> and it doesn't make a difference in time, but does result
> in slightly larger file sizes -- which is characteristic
> of it using multiple threads.Â However, tar only produces
> 1 thread, so it never gets a chance to run things in multiple
> Also --
> Besides, music isn't "real compressible"Â -- especially
> if you are compressing mp3 or flac, but the general purpose
> compression algorithms don't do a great job of compressing
> Compressing mp3->mp3+xz saved about 2.8%
> Compressing flac->flac+xz saved about 0.82% (<less than 1%)
> Compressing raw waves:
> xz files were 32% LARGER than flac.
> So what type of music are you compressing?
> BTW -- sending this to you in addition to list, since
> the list has been flakey for me recently -- seems to
> have a bug rejecting my domain.
Now I found single compression doesn't harm at all.Â Multithreaded
compression is the problem.Â Â Actually, compression of mp3 files doesn't
mean anything.....so I will just use tar for archive.Â Â Â It's much faster :)
Problem reports: http://cygwin.com/problems.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
More information about the Cygwin