14562 2012-09-07 23:49:00 +0000 threaded programs with x32 abi randomly crash with arena.c:661: heap_trim: Assertion `p->size == (0|0x1)' failed 2012-09-26 18:33:18 +0000 1 1 1 Unclassified glibc malloc 2.16 All All RESOLVED FIXED https://bugs.gentoo.org/show_bug.cgi?id=394175 P2 normal 2.16 1 vapier hjl.tools drepper.fsp dschepler hjl.tools 57306 vapier 2012-09-07 23:49:32 +0000 test case: - use glibc-2.16 with x32 abi - get a bunch of files (like ~150; perl man pages are a sample input) - run `bzip2 -k <files>` - run `pbzip2 -k -f <files> -p4` see crash: pbzip2: arena.c:661: heap_trim: Assertion `p->size == (0|0x1)' failed. this has happened with a other threaded programs (git & squashfs); see gentoo bug for some more details 57307 hjl.tools 2012-09-08 00:42:46 +0000 hjl/x32/release/2.15 branch is OK. 57308 hjl.tools 2012-09-08 01:03:14 +0000 Copy malloc from hjl/x32/release/2.15 branch makes the problem goes away. 57310 hjl.tools 2012-09-08 04:03:06 +0000 The bug is triggered by MALLOC_ALIGNMENT=16. 57312 hjl.tools 2012-09-08 14:52:34 +0000 663 assert(p->size == (0|PREV_INUSE)); /* must be fencepost */ (gdb) p p $34 = (mchunkptr) 0xf55feff8 (gdb) p isn't 16-byte aligned. 57313 hjl.tools 2012-09-08 15:30:13 +0000 The size of top chunk must be a multiple of MALLOC_ALIGNMENT. But _int_new_arena has /* Set up the top chunk, with proper alignment. */ ptr = (char *)(a + 1); misalign = (unsigned long)chunk2mem(ptr) & MALLOC_ALIGN_MASK; if (misalign > 0) ptr += MALLOC_ALIGNMENT - misalign; top(a) = (mchunkptr)ptr; set_head(top(a), (((char*)h + h->size) - ptr) | PREV_INUSE); It doesn't check size requirement. 57314 hjl.tools 2012-09-08 20:43:47 +0000 I am testing this patch: diff --git a/malloc/arena.c b/malloc/arena.c index f88b41d..ac5afc4 100644 --- a/malloc/arena.c +++ b/malloc/arena.c @@ -655,14 +655,22 @@ heap_trim(heap_info *heap, size_t pad) mchunkptr top_chunk = top(ar_ptr), p, bck, fwd; heap_info *prev_heap; long new_size, top_size, extra; + unsigned long misalign; /* Can this heap go away completely? */ while(top_chunk == chunk_at_offset(heap, sizeof(*heap))) { prev_heap = heap->prev; p = chunk_at_offset(prev_heap, prev_heap->size - (MINSIZE-2*SIZE_SZ)); + /* fencepost must be properly aligned. */ + misalign = ((unsigned long) p) & MALLOC_ALIGN_MASK; + if (misalign > 0) + { + p = (mchunkptr)(((unsigned long) p) & ~MALLOC_ALIGN_MASK); + misalign = MALLOC_ALIGNMENT - misalign; + } assert(p->size == (0|PREV_INUSE)); /* must be fencepost */ p = prev_chunk(p); - new_size = chunksize(p) + (MINSIZE-2*SIZE_SZ); + new_size = chunksize(p) + (MINSIZE-2*SIZE_SZ) + misalign; assert(new_size>0 && new_size<(long)(2*MINSIZE)); if(!prev_inuse(p)) new_size += p->prev_size; 57315 hjl.tools 2012-09-08 20:48:37 +0000 This is the right one: diff --git a/malloc/arena.c b/malloc/arena.c index f88b41d..8d52109 100644 --- a/malloc/arena.c +++ b/malloc/arena.c @@ -655,14 +655,18 @@ heap_trim(heap_info *heap, size_t pad) mchunkptr top_chunk = top(ar_ptr), p, bck, fwd; heap_info *prev_heap; long new_size, top_size, extra; + unsigned long misalign; /* Can this heap go away completely? */ while(top_chunk == chunk_at_offset(heap, sizeof(*heap))) { prev_heap = heap->prev; p = chunk_at_offset(prev_heap, prev_heap->size - (MINSIZE-2*SIZE_SZ)); + /* fencepost must be properly aligned. */ + misalign = ((unsigned long) p) & MALLOC_ALIGN_MASK; + p = (mchunkptr)(((unsigned long) p) & ~MALLOC_ALIGN_MASK); assert(p->size == (0|PREV_INUSE)); /* must be fencepost */ p = prev_chunk(p); - new_size = chunksize(p) + (MINSIZE-2*SIZE_SZ); + new_size = chunksize(p) + (MINSIZE-2*SIZE_SZ) + misalign; assert(new_size>0 && new_size<(long)(2*MINSIZE)); if(!prev_inuse(p)) new_size += p->prev_size; 57316 hjl.tools 2012-09-08 20:58:39 +0000 This one is better: diff --git a/malloc/arena.c b/malloc/arena.c index f88b41d..4727d1e 100644 --- a/malloc/arena.c +++ b/malloc/arena.c @@ -654,15 +654,18 @@ heap_trim(heap_info *heap, size_t pad) unsigned long pagesz = GLRO(dl_pagesize); mchunkptr top_chunk = top(ar_ptr), p, bck, fwd; heap_info *prev_heap; - long new_size, top_size, extra; + long new_size, top_size, extra, misalign; /* Can this heap go away completely? */ while(top_chunk == chunk_at_offset(heap, sizeof(*heap))) { prev_heap = heap->prev; p = chunk_at_offset(prev_heap, prev_heap->size - (MINSIZE-2*SIZE_SZ)); + /* fencepost must be properly aligned. */ + misalign = ((long) p) & MALLOC_ALIGN_MASK; + p = (mchunkptr)(((unsigned long) p) & ~MALLOC_ALIGN_MASK); assert(p->size == (0|PREV_INUSE)); /* must be fencepost */ p = prev_chunk(p); - new_size = chunksize(p) + (MINSIZE-2*SIZE_SZ); + new_size = chunksize(p) + (MINSIZE-2*SIZE_SZ) + misalign; assert(new_size>0 && new_size<(long)(2*MINSIZE)); if(!prev_inuse(p)) new_size += p->prev_size; 57329 hjl.tools 2012-09-09 18:29:59 +0000 Please try hjl/pr14562/2.16 branch. 57337 vapier 2012-09-10 05:42:25 +0000 (In reply to comment #9) seems to be the same fix as in comment #8 which i've been running locally. it fixed the known bugs i was hitting, and haven't seen any new ones related to this. there are other bugs, but those are porting issues :). 57344 dschepler 2012-09-10 17:59:14 +0000 (In reply to comment #9) > Please try hjl/pr14562/2.16 branch. As I already commented on the x32 list, for me this fixes among other things: gij running ecj, and the GraphicsMagick, perl and pixman testsuites. 57576 hjl.tools 2012-09-25 00:20:53 +0000 Fixed in 2.17 by http://sourceware.org/git/?p=glibc.git;a=commit;h=ced6f16ee919d12725840d43d007f1cfd67118df 57624 hjl.tools 2012-09-26 18:33:18 +0000 Fixed in 2.16.1 by http://sourceware.org/git/?p=glibc.git;a=commit;h=bbe53ed2c311b7281837b6f3f24ddeb8f3d65697