This is the mail archive of the
cygwin-patches
mailing list for the Cygwin project.
Re: Extending /proc/*/maps
On 12/05/2011 12:55 PM, Corinna Vinschen wrote:
struct heap
{
heap *next;
- void *base;
+ unsigned heap_id;
+ uintptr_t base;
+ uintptr_t end;
+ unsigned long flags;
};
We don't actually need the end pointer: we're trying to match an
No, we need it. The heaps consist of reserved and committed memory
blocks, as well as of shareable and non-shareable blocks. Thus you
get multiple VirtualQuery calls per heap, thus you have to check for
the address within the entire heap(*).
The code which queries heap_info already deals with allocations that
take multiple VirtualQuery calls to traverse, and only calls into
heap_info for the base of any given allocation. These bases are
identified because they have the same value for both mb.AllocationBase
and mb.BaseAddress (except with unallocated regions, for which
AllocationBase is undefined). This is how I compute the reported
offsets. Further, adjacent regions having the same base and whose
permissions resolve to the same rwxp bits are coalesced and reported as
a single region. For example, several parts of a rebased dll will have
copy-on-write regions (those where no rewrites took place), but these
are collapsed because rwxp can't express copy-on-write (dlls can have up
to 2x more VirtualQuery calls than is reflected in the output of
/proc/*/maps).
In other words, it shouldn't matter how the heap has sliced up a given
region, as long as the base address of each region is properly
identified by some heap block having non-zero flags&2 (see caveat
below). If no such block exists, then the code in
heap_info::fill_if_match would need to test whether [heap_base,heap_end)
overlaps [alloc_base,alloc_end) because alloc_base will never fall
inside any block on the list.
CAVEAT: According to MSDN's documentation for HeapWalk, "a heap consists
of one or more regions of virtual memory, each with a unique region
index." During heap walking, wFlags is set to PROCESS_HEAP_REGION
whenever "the heap element is located at the beginning of a region of
contiguous virtual memory in use by the heap." However, "large block
regions" (those allocated directly via VirtualAlloc) do not set the
PROCESS_HEAP_REGION flag. If this PROCESS_HEAP_REGION flag corresponds
to your (flags & 2), then we won't report any large blocks (however,
it's documented to have the value 0x0001, with 0x0002 being
PROCESS_HEAP_UNCOMMITTED_RANGE).
heap *heaps;
This is a misnomer now -- it's really a list of heap regions/blocks.
I don't think so. The loop stores only the blocks which constitute
the original VirtualAlloc'ed memory regions. They are not the real
heap blocks returned by Heap32First/Heap32Next. These are filtered
out by checking for flags& 2 (**).
Sorry, I cut too much context out of the diff. That's heap_info::heaps,
which indeed refers to heap regions which we identified by checking
flags&2 (that's why it needs the heap_id inside it now, where it didn't
before) (++)
+ heap *h = (heap *) cmalloc (HEAP_FHANDLER, sizeof (heap));
+ *h = (heap) {heaps, hcnt, barray[bcnt].Address,
+ barray[bcnt].Address + barray[bcnt].Size,
+ harray->Heaps[hcnt].Flags};
+ heaps = h;
Given that the number of heap blocks is potentially large, I think
Not really. See (**). 3 heaps -> 3 blocks, or only slightly more
if a growable heap got expanded.
See (++). When I point my essentially-identical version of the code at
emacs, it reports 8 heaps, each with 2-4 regions. The list traversed by
fill_on_match has ~20 entries.
Do you actually encounter requests which fall inside a heap region
rather than at its start?
Yes, see (*). And have a look into the output of my code in
contrast to what's printed by the currently version from CVS.
I'll have to test out the patch -- my local version which checks only
allocation bases reports additional regions (those not part of the
original heap allocation), but if there are yet other regions which I'm
missing and yours finds then I'll stand corrected about (++).
I have not seen this in my experiments,
and if you have it is almost certainly a bug in format_process_maps'
allocation traversal.
No, see (*).
Also, are there ever shareable-but-not-mem_mapped segments?
Yes, there are. 2 of the 3 default heaps are marked as shareable, but
one of them is only partially shared. Of course I don't understand the
reason behind this and how this is accomplished, given that the user
code can't even set a flag "shareable" at HeapCreate time.
Hmm. Weird. I'll respond further after testing your patch and comparing
its output against my local version of the same code.
Ryan