Bug 6008 - support larger on-memory buffer
Summary: support larger on-memory buffer
Status: RESOLVED FIXED
Alias: None
Product: systemtap
Classification: Unclassified
Component: translator (show other bugs)
Version: unspecified
: P2 normal
Target Milestone: ---
Assignee: Masami Hiramatsu
URL:
Keywords:
Depends on: 5645
Blocks:
  Show dependency treegraph
 
Reported: 2008-03-31 15:58 UTC by Masami Hiramatsu
Modified: 2008-04-30 16:22 UTC (History)
0 users

See Also:
Host:
Target:
Build:
Last reconfirmed:


Attachments
remove the size limitation of buffer (808 bytes, patch)
2008-04-08 20:33 UTC, Masami Hiramatsu
Details | Diff
relayfs patch: use vmalloc to allocate *page array (690 bytes, patch)
2008-04-08 20:36 UTC, Masami Hiramatsu
Details | Diff
extend size limitation of buffer to 4095MB (807 bytes, patch)
2008-04-25 20:41 UTC, Masami Hiramatsu
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Masami Hiramatsu 2008-03-31 15:58:17 UTC
systemtap allows users to allocate more memories (ex. 2GB) for trace buffers
which can keep the trace data for a longer time on main memory.

Current trace buffer size is limited less than 64 MB per cpu even if we have a
plenty of memory (tens of GB) and a large memory space in kernel(especially
64bit arch). Therefore, we aren't able to log trace data for long enough.

Current limitation comes from relayfs which allocates an array of (struct page*)
for total allocating pages by kmalloc. However, usually, since the size of
kmalloc is limited, relayfs can not allocate the array if specified buffer size
is over (max_kmalloc_size/sizeof(struct page*))*page_size (=128KB/8B*4KB = 64MB
on x86-64). So, I think we can solve this by using vmalloc instead of kmalloc
when the array size is over one page_size.
Comment 1 Masami Hiramatsu 2008-04-08 20:33:26 UTC
Created attachment 2634 [details]
remove the size limitation of buffer

This patch removes current 64MB upper limitation from systemtap.
Comment 2 Masami Hiramatsu 2008-04-08 20:36:25 UTC
Created attachment 2648 [details]
relayfs patch: use vmalloc to allocate *page array

with this patch, relayfs uses vmalloc to allocate *page array if the array is
bigger than one page, otherwise, it uses kcalloc.
Comment 3 Masami Hiramatsu 2008-04-25 20:41:53 UTC
Created attachment 2719 [details]
extend size limitation of buffer to 4095MB

Since the buffer size is an "unsigned int" variable, we can't specify more than
4095 to -s option.
Comment 4 Masami Hiramatsu 2008-04-30 16:12:15 UTC
(In reply to comment #2)
> Created an attachment (id=2648)
> relayfs patch: use vmalloc to allocate *page array
> 
> with this patch, relayfs uses vmalloc to allocate *page array if the array is
> bigger than one page, otherwise, it uses kcalloc.
> 

this feature has been merged to linus tree(2.6.25-git15).
Comment 5 Masami Hiramatsu 2008-04-30 16:22:17 UTC
Patch Committed.