Differences between revisions 11 and 12
Revision 11 as of 2015-01-19 20:17:18
Size: 3416
Editor: FChE
Comment:
Revision 12 as of 2015-01-19 22:56:53
Size: 4245
Editor: FChE
Comment:
Deletions are marked like this. Additions are marked like this.
Line 7: Line 7:
||error message ||exhausted resource ||||possible remedy ||
||<style="text-align:center;" |2>MAXACTION exceeded ||<style="text-align:center;" |2>time -- number of statements executed by probe handler ||||recompile with larger -DMAXACTION=NN or -DMAXACTION_INTERRUPTIBLE=NN ||
||||impose limits on iteration - foreach (... limit NN) or break) ||
||<style="text-align:center;" |2>MAXNESTING exceeded ||<style="text-align:center;" |2>time/space -- function recursion too deep ||||reduce recursion ||
||||recompile with larger -DMAXNESTING=NN ||
||division by zero ||time/space -- universe is too small to represent exact value ||||try dividing by one instead ||
||<style="text-align:center;" |3>aggregation overflow ||<style="text-align:center;" |3>space -- number of distinct index tuples in aggregation array ||||recompile with larger -DMAXMAPENTRIES=NN ||
||||declare that array only with larger size -- global my_array[NN] ||
||||delete unneeded elements, or use % array global declaration suffix ||
||Array overflow ||space -- number of distinct index tuples in ordinary array ||||as above ||
||string truncation ||space -- strings are silently limited to a maximum length ||||recompile with larger -DMAXSTRINGLEN=NN ||
||<style="text-align:center;" |6>WARNING: ... skipped probes: NN ||<style="text-align:center;" |3>time -- cross-processor contention over global script variables ||||reduce number of globals ||
||||maybe recompile with larger -DMAXTRYLOCK=NN ||
||||use aggregates and @count() instead of integer counters ||
||space -- too many return probes may be pending ||||add larger .maxaction(NN) to .return probe point ||
||<style="text-align:center;" |2>space -- too little kernel stack available for probe handler run ||||move probe points higher in the call stack ||
||||risky, but maybe recompile with smaller -DMINSTACKSPACE=NN ||
||<style="text-align:center;" |3>probe overhead exceeded threshold ||<style="text-align:center;" |3>time -- probe handlers are taking too high fraction of total real time ||||shrink probe handler code ||
||||recompile with larger -DSTP_OVERLOAD_THRESHOLD ||
||||disable this heuristic with -DSTP_NO_OVERLOAD ||
||<style="text-align:center;" |2>There were NN transport failures ||<style="text-align:center;" |2>space -- amount of data printed by script exceeded staprun's capability to copy it to the output file ||||print less stuff ||
||||run with larger trace buffers: stap -s NN ||
||<|4>out-of-memory during or after startup||<|4>space -- large arrays?||||recompile with smaller -DMAXSTRINGLEN, if string indexes||
||||recompile with smaller -DMAXMAPENTRIES or global foo[SIZE]||
||||join index columns into single string with sprintf()||
||||use smaller kernel boot maxcpus=NNN if kernel reserves excess hotplug slots||
||error message ||exhausted resource ||||<style="text-align:center">possible remedy ||
||<style="text-align:center;" |2>MAXACTION exceeded ||<style="text-align:center;" |2>time -- number of statements executed by probe handler ||||<style="text-align:center">recompile with larger -DMAXACTION=NN or -DMAXACTION_INTERRUPTIBLE=NN ||
||||<style="text-align:center">impose limits on iteration - foreach (... limit NN) or break) ||
||<style="text-align:center;" |2>MAXNESTING exceeded ||<style="text-align:center;" |2>time/space -- function recursion too deep ||||<style="text-align:center">reduce recursion ||
||||<style="text-align:center">recompile with larger -DMAXNESTING=NN ||
||division by zero ||time/space -- universe is too small to represent exact value ||||<style="text-align:center">try dividing by one instead ||
||<style="text-align:center;" |3>aggregation overflow ||<style="text-align:center;" |3>space -- number of distinct index tuples in aggregation array ||||<style="text-align:center">recompile with larger -DMAXMAPENTRIES=NN ||
||||<style="text-align:center">declare that array only with larger size -- global my_array[NN] ||
||||<style="text-align:center">delete unneeded elements, or use % array global declaration suffix ||
||Array overflow ||space -- number of distinct index tuples in ordinary array ||||<style="text-align:center">as above ||
||string truncation ||space -- strings are silently limited to a maximum length ||||<style="text-align:center">recompile with larger -DMAXSTRINGLEN=NN ||
||<style="text-align:center;" |6>WARNING: ... skipped probes: NN ||<style="text-align:center;" |3>time -- cross-processor contention over global script variables ||||<style="text-align:center">reduce number of globals ||
||||<style="text-align:center">maybe recompile with larger -DMAXTRYLOCK=NN ||
||||<style="text-align:center">use aggregates and @count() instead of integer counters ||
||space -- too many return probes may be pending ||||<style="text-align:center">add larger .maxaction(NN) to .return probe point ||
||<style="text-align:center;" |2>space -- too little kernel stack available for probe handler run ||||<style="text-align:center">move probe points higher in the call stack ||
||||<style="text-align:center">risky, but maybe recompile with smaller -DMINSTACKSPACE=NN ||
||<style="text-align:center;" |3>probe overhead exceeded threshold ||<style="text-align:center;" |3>time -- probe handlers are taking too high fraction of total real time ||||<style="text-align:center">shrink probe handler code ||
||||<style="text-align:center">recompile with larger -DSTP_OVERLOAD_THRESHOLD ||
||||<style="text-align:center">disable this heuristic with -DSTP_NO_OVERLOAD ||
||<style="text-align:center;" |2>There were NN transport failures ||<style="text-align:center;" |2>space -- amount of data printed by script exceeded staprun's capability to copy it to the output file ||||<style="text-align:center">print less stuff ||
||||<style="text-align:center">run with larger trace buffers: stap -s NN ||
||<style="text-align:center" |5>out-of-memory during or after startup ||<style="text-align:center" |5>space -- large arrays? ||||<style="text-align:center">recompile with smaller -DMAXSTRINGLEN, if string indexes ||
||||<style="text-align:center">recompile with smaller -DMAXMAPENTRIES or global foo[SIZE] ||
||||<style="text-align:center">join index columns into single string with sprintf() ||
||||<style="text-align:center">use smaller kernel boot maxcpus=NNN if kernel reserves excess hotplug slots ||
||||use scalar (++) rather than aggregate (<<<1; @count) arrays||

Exhausted resources

Systemtap attempts to assure that probe handlers run within strict time/space limits. A typical script producing a modest amount of trace data will not encounter these limits, but maybe your script does. What can you do?

A blunt workaround is to use the --suppress-handler-errors flag. It lets most errors be quietly absorbed by the systemtap runtime, so the script will soldier on and on, until it runs exit(), or a user interrupts it. It'll print a single warning message then.

It depends on what resource ran out.

error message

exhausted resource

possible remedy

MAXACTION exceeded

time -- number of statements executed by probe handler

recompile with larger -DMAXACTION=NN or -DMAXACTION_INTERRUPTIBLE=NN

impose limits on iteration - foreach (... limit NN) or break)

MAXNESTING exceeded

time/space -- function recursion too deep

reduce recursion

recompile with larger -DMAXNESTING=NN

division by zero

time/space -- universe is too small to represent exact value

try dividing by one instead

aggregation overflow

space -- number of distinct index tuples in aggregation array

recompile with larger -DMAXMAPENTRIES=NN

declare that array only with larger size -- global my_array[NN]

delete unneeded elements, or use % array global declaration suffix

Array overflow

space -- number of distinct index tuples in ordinary array

as above

string truncation

space -- strings are silently limited to a maximum length

recompile with larger -DMAXSTRINGLEN=NN

WARNING: ... skipped probes: NN

time -- cross-processor contention over global script variables

reduce number of globals

maybe recompile with larger -DMAXTRYLOCK=NN

use aggregates and @count() instead of integer counters

space -- too many return probes may be pending

add larger .maxaction(NN) to .return probe point

space -- too little kernel stack available for probe handler run

move probe points higher in the call stack

risky, but maybe recompile with smaller -DMINSTACKSPACE=NN

probe overhead exceeded threshold

time -- probe handlers are taking too high fraction of total real time

shrink probe handler code

recompile with larger -DSTP_OVERLOAD_THRESHOLD

disable this heuristic with -DSTP_NO_OVERLOAD

There were NN transport failures

space -- amount of data printed by script exceeded staprun's capability to copy it to the output file

print less stuff

run with larger trace buffers: stap -s NN

out-of-memory during or after startup

space -- large arrays?

recompile with smaller -DMAXSTRINGLEN, if string indexes

recompile with smaller -DMAXMAPENTRIES or global foo[SIZE]

join index columns into single string with sprintf()

use smaller kernel boot maxcpus=NNN if kernel reserves excess hotplug slots

use scalar (++) rather than aggregate (<<<1; @count) arrays

See also TipSkippedProbesOptimization.


Tips

None: TipExhaustedResourceErrors (last edited 2015-01-19 22:56:53 by FChE)