This is the mail archive of the
systemtap@sourceware.org
mailing list for the systemtap project.
Re: precompiled probing scenarios
Frank Ch. Eigler wrote:
Hi -
On Fri, Oct 06, 2006 at 03:33:19PM -0500, David Smith wrote:
[...]
Hmm. Are we hashing the input script? If so, how does this work with
probe wildcards? For example, let's say I probe "kernel.function("*")".
We compile and cache this module. I then plug in a bunch of
additional hardware, which causes several extra modules to be loaded. I
then run stap again with the exact same input script. [...]
kernel.function("*") should match exactly what was there before.
Probes on module("*").FOO would be redefined to mean something like
"all modules that we know at translation time that *might* exist, that
also happen to be *loaded* at run time. This aspect of wildcard
expansion would thus take place at run time rather than translate
time. It just so happens that the same module might probe a greater
or lesser number of modules on an actual system. With that proviso,
a script-source-level hash still seems to work.
Here's a patch that implements a cache for systemtap. It works somewhat
like ccache (which is unsurprising since I based it on ccache), but
simpler. (Note that if you want to apply the patch and compile it,
you'll need to regenerate Makefile.in.)
Here's an outline of what happens.
By default, the cache is stored under ~/.stap_cache. This can be
overridden using the new 'SYSTEMTAP_CACHE' environment variable.
At the end of pass 2, a hash is generated (in find_hash(), which is in
hash.cxx:70). The hash is computed using the following data:
- kernel version and arch
- any user-specified macros
- runtime path
- gcc's path, size, and mtime
- stap's version and compile date
- pass 2 script output
The hash is then used to compute a hash pathname.
Still at the end of pass 2, we now check to see if we can find a cache
hit (get_from_cache(), cache.cxx:42). If the hash pathname exists, we
copy it from the cache to the temporary directory. We then skip to pass
5 to run the module.
If a cache hit wasn't found, we proceed normally. At the end of pass 4,
we save the compiled module and its C file in the cache (add_to_cache(),
cache.cxx:17).
By default, cache support is on. To turn it off, you can either set
SYSTEMTAP_CACHE to a non-directory ("SYSTEMTAP_CACHE=/dev/null stap ..")
or use the '-m' option to change the module name (we must turn off cache
support because we have to name the module 'stap_{hash}'). We could add
a command line option to disable the cache if needed.
Note that currently several tests in the testsuite fail after a first
run to seed the cache because they don't expect to see the skip from
pass 2 to pass 5.
Stuff left to do:
- Testing, testing, testing
- Correct module handling (which Frank outlined above)
- Set a maximum cache size and expire old modules. Is this needed?
- Documentation.
--
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)
Index: Makefile.am
===================================================================
RCS file: /cvs/systemtap/src/Makefile.am,v
retrieving revision 1.57
diff -u -p -u -p -r1.57 Makefile.am
--- Makefile.am 18 Oct 2006 21:23:53 -0000 1.57
+++ Makefile.am 19 Oct 2006 19:44:57 -0000
@@ -15,7 +15,8 @@ dist_man_MANS = stap.1 stapprobes.5 stap
bin_PROGRAMS = stap staprun
stap_SOURCES = main.cxx \
parse.cxx staptree.cxx elaborate.cxx translate.cxx \
- tapsets.cxx buildrun.cxx loc2c.c
+ tapsets.cxx buildrun.cxx loc2c.c hash.cxx mdfour.c \
+ cache.cxx util.cxx
stap_LDADD = @stap_LIBS@
stap_CXXFLAGS = -Werror $(AM_CXXFLAGS)
Index: cache.cxx
===================================================================
RCS file: cache.cxx
diff -N cache.cxx
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ cache.cxx 19 Oct 2006 19:44:57 -0000
@@ -0,0 +1,101 @@
+#include "session.h"
+#include "cache.h"
+#include "util.h"
+#include <cerrno>
+#include <string>
+
+extern "C" {
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+}
+
+using namespace std;
+
+
+void
+add_to_cache(systemtap_session& s)
+{
+ string module_src_path = s.tmpdir + "/" + s.module_name + ".ko";
+ if (s.verbose)
+ clog << "Copying " << module_src_path << " to " << s.hash_path << endl;
+ if (copy_file(module_src_path.c_str(), s.hash_path.c_str()) != 0)
+ {
+ cerr << "Copy failed (\"" << module_src_path << "\" to \""
+ << s.hash_path << "\"): " << strerror(errno) << endl;
+ return;
+ }
+
+ string c_dest_path = s.hash_path;
+ if (c_dest_path.rfind(".ko") == (c_dest_path.size() - 3))
+ c_dest_path.resize(c_dest_path.size() - 3);
+ c_dest_path += ".c";
+
+ if (s.verbose)
+ clog << "Copying " << s.translated_source << " to " << c_dest_path
+ << endl;
+ if (copy_file(s.translated_source.c_str(), c_dest_path.c_str()) != 0)
+ {
+ cerr << "Copy failed (\"" << s.translated_source << "\" to \""
+ << c_dest_path << "\"): " << strerror(errno) << endl;
+ }
+}
+
+
+bool
+get_from_cache(systemtap_session& s)
+{
+ string module_dest_path = s.tmpdir + "/" + s.module_name + ".ko";
+ string c_src_path = s.hash_path;
+ int fd_module, fd_c;
+
+ if (c_src_path.rfind(".ko") == (c_src_path.size() - 3))
+ c_src_path.resize(c_src_path.size() - 3);
+ c_src_path += ".c";
+
+ // See if module exists
+ fd_module = open(s.hash_path.c_str(), O_RDONLY);
+ if (fd_module == -1)
+ {
+ // It isn't in cache.
+ return false;
+ }
+
+ // See if C file exists.
+ fd_c = open(c_src_path.c_str(), O_RDONLY);
+ if (fd_c == -1)
+ {
+ // The module is there, but the C file isn't. Cleanup and
+ // return.
+ close(fd_module);
+ unlink(s.hash_path.c_str());
+ return false;
+ }
+
+ // Copy the cached module to the destination
+ if (copy_file(s.hash_path.c_str(), module_dest_path.c_str()) != 0)
+ {
+ cerr << "Copy failed (\"" << s.hash_path << "\" to \""
+ << module_dest_path << "\"): " << strerror(errno) << endl;
+ close(fd_module);
+ close(fd_c);
+ return false;
+ }
+
+ // Copy the cached C file to the destination
+ if (copy_file(c_src_path.c_str(), s.translated_source.c_str()) != 0)
+ {
+ cerr << "Copy failed (\"" << c_src_path << "\" to \""
+ << s.translated_source << "\"): " << strerror(errno) << endl;
+ unlink(module_dest_path.c_str());
+ close(fd_module);
+ close(fd_c);
+ return false;
+ }
+
+ close(fd_module);
+ close(fd_c);
+ clog << "Using cached result \"" << s.hash_path << "\" as \""
+ << module_dest_path << "\"" << endl;
+ return true;
+}
Index: cache.h
===================================================================
RCS file: cache.h
diff -N cache.h
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ cache.h 19 Oct 2006 19:44:57 -0000
@@ -0,0 +1,2 @@
+void add_to_cache(systemtap_session& s);
+bool get_from_cache(systemtap_session& s);
Index: hash.cxx
===================================================================
RCS file: hash.cxx
diff -N hash.cxx
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ hash.cxx 19 Oct 2006 19:44:57 -0000
@@ -0,0 +1,154 @@
+// Copyright (C) Andrew Tridgell 2002 (original file)
+// Copyright (C) 2006 Red Hat Inc. (systemtap changes)
+//
+// This program is free software; you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation; either version 2 of the License, or
+// (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program; if not, write to the Free Software
+// Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+
+#include "config.h"
+#include "session.h"
+#include "hash.h"
+#include "util.h"
+#include <sstream>
+#include <iomanip>
+#include <cerrno>
+
+extern "C" {
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <unistd.h>
+}
+
+using namespace std;
+
+void
+hash::start()
+{
+ mdfour_begin(&md4);
+}
+
+
+void
+hash::add(const unsigned char *buffer, size_t size)
+{
+ mdfour_update(&md4, buffer, size);
+}
+
+
+void
+hash::result(string& r)
+{
+ ostringstream rstream;
+ unsigned char sum[16];
+
+ mdfour_update(&md4, NULL, 0);
+ mdfour_result(&md4, sum);
+
+ for (int i=0; i<16; i++)
+ {
+ rstream << hex << setfill('0') << setw(2) << (unsigned)sum[i];
+ }
+ rstream << "_" << setw(0) << dec << (unsigned)md4.totalN;
+ r = rstream.str();
+}
+
+
+// Grabbed from linux/module.h kernel include.
+#define MODULE_NAME_LEN (64 - sizeof(unsigned long))
+
+void
+find_hash (systemtap_session& s, const string& script)
+{
+ hash h;
+ int nlevels = 2;
+ struct stat st;
+
+ // We use a N level subdir for the cache path. Let N be adjustable.
+ const char *s_n;
+ if ((s_n = getenv("SYSTEMTAP_NLEVELS")))
+ {
+ nlevels = atoi(s_n);
+ if (nlevels < 1) nlevels = 1;
+ if (nlevels > 8) nlevels = 8;
+ }
+
+ // Hash kernel release and arch.
+ h.add(s.kernel_release);
+ h.add(s.architecture);
+
+ // Hash user-specified arguments (that change the generated module).
+ for (unsigned i = 0; i < s.macros.size(); i++)
+ h.add(s.macros[i]);
+
+ // Hash runtime path (that gets added in as "-I path").
+ h.add(s.runtime_path);
+
+ // Hash compiler path, size, and mtime. We're just going to assume
+ // we'll be using gcc, which should be correct most of the time.
+ string gcc_path;
+ if (find_executable("gcc", gcc_path))
+ {
+ if (stat(gcc_path.c_str(), &st) == 0)
+ {
+ h.add(gcc_path);
+ h.add(st.st_size);
+ h.add(st.st_mtime);
+ }
+ }
+
+ // Hash the systemtap version and compile date.
+ h.add(VERSION);
+ h.add(DATE);
+
+ // Add in pass 2 script output.
+ h.add(script);
+
+ // Use a N level subdir for the cache path to reduce the impact on
+ // filesystems which are slow for large directories.
+ string hashdir = s.cachedir;
+ string result;
+ h.result(result);
+
+ for (int i = 0; i < nlevels; i++)
+ {
+ hashdir += string("/") + result[i];
+ if (create_dir(hashdir.c_str()) != 0)
+ {
+ cerr << "Warning: failed to create cache directory (\""
+ << hashdir + "\"): " << strerror(errno) << endl;
+ cerr << "Disabling cache support." << endl;
+ s.use_cache = false;
+ return;
+ }
+ }
+
+ // Update module name to be 'stap_{hash start}'. '{hash start}'
+ // must not be too long. This shouldn't happen, since the maximum
+ // size of a hash is 32 fixed chars + 1 (for the '_') + a max of 11.
+ s.module_name = "stap_" + result;
+ if (s.module_name.size() >= (MODULE_NAME_LEN - 1))
+ s.module_name.resize(MODULE_NAME_LEN - 1);
+
+ // 'ccache' would use a hash path of something like:
+ // s.hash_path = hashdir + "/" + result.substr(nlevels);
+ // which would look like:
+ // ~/.stap_cache/A/B/CDEFGHIJKLMNOPQRSTUVWXYZABCDEF_XXX
+ //
+ // We're using the following so that the module can be used straight
+ // from the cache if desired. This ends up looking like this:
+ // ~/.stap_cache/A/B/stap_ABCDEFGHIJKLMNOPQRSTUVWXYZABCDEF_XXX.ko
+ s.hash_path = hashdir + "/" + s.module_name + ".ko";
+
+ // Update C source name with new module_name.
+ s.translated_source = string(s.tmpdir) + "/" + s.module_name + ".c";
+}
Index: hash.h
===================================================================
RCS file: hash.h
diff -N hash.h
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ hash.h 19 Oct 2006 19:44:57 -0000
@@ -0,0 +1,34 @@
+#include <string>
+
+extern "C" {
+#include <string.h>
+#include <mdfour.h>
+}
+
+class hash
+{
+private:
+ struct mdfour md4;
+
+public:
+ hash() { start(); }
+
+ void start();
+
+ void add(const unsigned char *buffer, size_t size);
+ void add(const int x) { add((const unsigned char *)&x, sizeof(x)); }
+ void add(const long x) { add((const unsigned char *)&x, sizeof(x)); }
+ void add(const long long x) { add((const unsigned char *)&x, sizeof(x)); }
+ void add(const unsigned int x) { add((const unsigned char *)&x, sizeof(x)); }
+ void add(const unsigned long x) { add((const unsigned char *)&x,
+ sizeof(x)); }
+ void add(const unsigned long long x) { add((const unsigned char *)&x,
+ sizeof(x)); }
+ void add(const char *s) { add((const unsigned char *)s, strlen(s)); }
+ void add(const std::string& s) { add((const unsigned char *)s.c_str(),
+ s.length()); }
+
+ void result(std::string& r);
+};
+
+void find_hash (systemtap_session& s, const std::string& script);
Index: main.cxx
===================================================================
RCS file: /cvs/systemtap/src/main.cxx,v
retrieving revision 1.53
diff -u -p -u -p -r1.53 main.cxx
--- main.cxx 28 Sep 2006 01:48:59 -0000 1.53
+++ main.cxx 19 Oct 2006 19:44:57 -0000
@@ -15,6 +15,9 @@
#include "translate.h"
#include "buildrun.h"
#include "session.h"
+#include "hash.h"
+#include "cache.h"
+#include "util.h"
#include <iostream>
#include <fstream>
@@ -113,6 +116,65 @@ stringify(T t)
}
+static void
+printscript(systemtap_session& s, ostream& o)
+{
+ if (s.globals.size() > 0)
+ o << "# globals" << endl;
+ for (unsigned i=0; i<s.globals.size(); i++)
+ {
+ vardecl* v = s.globals[i];
+ v->printsig (o);
+ o << endl;
+ }
+
+ if (s.functions.size() > 0)
+ o << "# functions" << endl;
+ for (unsigned i=0; i<s.functions.size(); i++)
+ {
+ functiondecl* f = s.functions[i];
+ f->printsig (o);
+ o << endl;
+ if (f->locals.size() > 0)
+ o << " # locals" << endl;
+ for (unsigned j=0; j<f->locals.size(); j++)
+ {
+ vardecl* v = f->locals[j];
+ o << " ";
+ v->printsig (o);
+ o << endl;
+ }
+ if (s.verbose)
+ {
+ f->body->print (o);
+ o << endl;
+ }
+ }
+
+ if (s.probes.size() > 0)
+ o << "# probes" << endl;
+ for (unsigned i=0; i<s.probes.size(); i++)
+ {
+ derived_probe* p = s.probes[i];
+ p->printsig (o);
+ o << endl;
+ if (p->locals.size() > 0)
+ o << " # locals" << endl;
+ for (unsigned j=0; j<p->locals.size(); j++)
+ {
+ vardecl* v = p->locals[j];
+ o << " ";
+ v->printsig (o);
+ o << endl;
+ }
+ if (s.verbose)
+ {
+ p->body->print (o);
+ o << endl;
+ }
+ }
+}
+
int
main (int argc, char * const argv [])
{
@@ -140,6 +202,7 @@ main (int argc, char * const argv [])
s.target_pid = 0;
s.merge=true;
s.perfmon=0;
+ s.use_cache = true;
const char* s_p = getenv ("SYSTEMTAP_TAPSET");
if (s_p != NULL)
@@ -159,6 +222,20 @@ main (int argc, char * const argv [])
else
s.runtime_path = string(PKGDATADIR) + "/runtime";
+ const char* s_c = getenv ("SYSTEMTAP_CACHE");
+ if (s_c != NULL)
+ s.cachedir = s_c;
+ else
+ s.cachedir = get_home_directory() + string("/.stap_cache");
+ if (create_dir(s.cachedir.c_str()) == 1)
+ {
+ const char* e = strerror (errno);
+ cerr << "Warning: failed to create cache directory (\"" << s.cachedir
+ << "\"): " << e << endl;
+ cerr << "Disabling cache support." << endl;
+ s.use_cache = false;
+ }
+
while (true)
{
int grc = getopt (argc, argv, "hVMvtp:I:e:o:R:r:m:kgc:x:D:bs:u");
@@ -216,6 +293,8 @@ main (int argc, char * const argv [])
case 'm':
s.module_name = string (optarg);
+ cerr << "Warning: using '-m' disables cache support." << endl;
+ s.use_cache = false;
break;
case 'r':
@@ -339,6 +418,10 @@ main (int argc, char * const argv [])
clog << "Created temporary directory \"" << s.tmpdir << "\"" << endl;
}
+ // Create the name of the C source file within the temporary
+ // directory.
+ s.translated_source = string(s.tmpdir) + "/" + s.module_name + ".c";
+
struct tms tms_before;
times (& tms_before);
struct timeval tv_before;
@@ -469,62 +552,7 @@ main (int argc, char * const argv [])
rc = semantic_pass (s);
if (rc == 0 && s.last_pass == 2)
- {
- if (s.globals.size() > 0)
- cout << "# globals" << endl;
- for (unsigned i=0; i<s.globals.size(); i++)
- {
- vardecl* v = s.globals[i];
- v->printsig (cout);
- cout << endl;
- }
-
- if (s.functions.size() > 0)
- cout << "# functions" << endl;
- for (unsigned i=0; i<s.functions.size(); i++)
- {
- functiondecl* f = s.functions[i];
- f->printsig (cout);
- cout << endl;
- if (f->locals.size() > 0)
- cout << " # locals" << endl;
- for (unsigned j=0; j<f->locals.size(); j++)
- {
- vardecl* v = f->locals[j];
- cout << " ";
- v->printsig (cout);
- cout << endl;
- }
- if (s.verbose)
- {
- f->body->print (cout);
- cout << endl;
- }
- }
-
- if (s.probes.size() > 0)
- cout << "# probes" << endl;
- for (unsigned i=0; i<s.probes.size(); i++)
- {
- derived_probe* p = s.probes[i];
- p->printsig (cout);
- cout << endl;
- if (p->locals.size() > 0)
- cout << " # locals" << endl;
- for (unsigned j=0; j<p->locals.size(); j++)
- {
- vardecl* v = p->locals[j];
- cout << " ";
- v->printsig (cout);
- cout << endl;
- }
- if (s.verbose)
- {
- p->body->print (cout);
- cout << endl;
- }
- }
- }
+ printscript(s, cout);
times (& tms_after);
gettimeofday (&tv_after, NULL);
@@ -540,6 +568,38 @@ main (int argc, char * const argv [])
cerr << "Pass 2: analysis failed. "
<< "Try again with more '-v' (verbose) options."
<< endl;
+ // Generate hash. There isn't any point in generating the hash
+ // if last_pass is 2, since we'll quit before using it.
+ else if (s.last_pass != 2 && s.use_cache)
+ {
+ ostringstream o;
+ unsigned saved_verbose;
+
+ // Make sure we're in verbose mode, so that printscript()
+ // will output function/probe bodies.
+ saved_verbose = s.verbose;
+ s.verbose = 3;
+
+ // Print script to 'o'
+ printscript(s, o);
+
+ // Restore original verbose mode setting.
+ s.verbose = saved_verbose;
+
+ // Generate hash
+ find_hash (s, o.str());
+
+ // See if we can use a cached module.
+ if (get_from_cache(s))
+ {
+ // If our last pass isn't 5, we're done (since passes 3 and
+ // 4 just generate what we just pulled out of the cache).
+ if (s.last_pass < 5) goto cleanup;
+
+ // Short-circuit to pass 5.
+ goto pass_5;
+ }
+ }
if (rc || s.last_pass == 2) goto cleanup;
@@ -548,7 +608,6 @@ main (int argc, char * const argv [])
times (& tms_before);
gettimeofday (&tv_before, NULL);
- s.translated_source = string(s.tmpdir) + "/" + s.module_name + ".c";
rc = translate_pass (s);
if (rc == 0 && s.last_pass == 3)
@@ -590,11 +649,17 @@ main (int argc, char * const argv [])
cerr << "Pass 4: compilation failed. "
<< "Try again with more '-v' (verbose) options."
<< endl;
+ else if (s.use_cache)
+ {
+ // Update cache.
+ add_to_cache(s);
+ }
// XXX: what to do if rc==0 && last_pass == 4? dump .ko file to stdout?
if (rc || s.last_pass == 4) goto cleanup;
// PASS 5: RUN
+pass_5:
times (& tms_before);
gettimeofday (&tv_before, NULL);
// NB: this message is a judgement call. The other passes don't emit
Index: mdfour.c
===================================================================
RCS file: mdfour.c
diff -N mdfour.c
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ mdfour.c 19 Oct 2006 19:44:57 -0000
@@ -0,0 +1,218 @@
+/*
+ a implementation of MD4 designed for use in the SMB authentication protocol
+ Copyright (C) Andrew Tridgell 1997-1998.
+
+ This program is free software; you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation; either version 2 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program; if not, write to the Free Software
+ Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+*/
+
+#include <stdio.h>
+#include <string.h>
+#include "mdfour.h"
+
+/* NOTE: This code makes no attempt to be fast!
+
+ It assumes that a int is at least 32 bits long
+*/
+
+static struct mdfour *m;
+
+#define MASK32 (0xffffffff)
+
+#define F(X,Y,Z) ((((X)&(Y)) | ((~(X))&(Z))))
+#define G(X,Y,Z) ((((X)&(Y)) | ((X)&(Z)) | ((Y)&(Z))))
+#define H(X,Y,Z) (((X)^(Y)^(Z)))
+#define lshift(x,s) (((((x)<<(s))&MASK32) | (((x)>>(32-(s)))&MASK32)))
+
+#define ROUND1(a,b,c,d,k,s) a = lshift((a + F(b,c,d) + M[k])&MASK32, s)
+#define ROUND2(a,b,c,d,k,s) a = lshift((a + G(b,c,d) + M[k] + 0x5A827999)&MASK32,s)
+#define ROUND3(a,b,c,d,k,s) a = lshift((a + H(b,c,d) + M[k] + 0x6ED9EBA1)&MASK32,s)
+
+/* this applies md4 to 64 byte chunks */
+static void
+mdfour64(uint32_t *M)
+{
+ uint32_t AA, BB, CC, DD;
+ uint32_t A,B,C,D;
+
+ A = m->A; B = m->B; C = m->C; D = m->D;
+ AA = A; BB = B; CC = C; DD = D;
+
+ ROUND1(A,B,C,D, 0, 3); ROUND1(D,A,B,C, 1, 7);
+ ROUND1(C,D,A,B, 2, 11); ROUND1(B,C,D,A, 3, 19);
+ ROUND1(A,B,C,D, 4, 3); ROUND1(D,A,B,C, 5, 7);
+ ROUND1(C,D,A,B, 6, 11); ROUND1(B,C,D,A, 7, 19);
+ ROUND1(A,B,C,D, 8, 3); ROUND1(D,A,B,C, 9, 7);
+ ROUND1(C,D,A,B, 10, 11); ROUND1(B,C,D,A, 11, 19);
+ ROUND1(A,B,C,D, 12, 3); ROUND1(D,A,B,C, 13, 7);
+ ROUND1(C,D,A,B, 14, 11); ROUND1(B,C,D,A, 15, 19);
+
+
+ ROUND2(A,B,C,D, 0, 3); ROUND2(D,A,B,C, 4, 5);
+ ROUND2(C,D,A,B, 8, 9); ROUND2(B,C,D,A, 12, 13);
+ ROUND2(A,B,C,D, 1, 3); ROUND2(D,A,B,C, 5, 5);
+ ROUND2(C,D,A,B, 9, 9); ROUND2(B,C,D,A, 13, 13);
+ ROUND2(A,B,C,D, 2, 3); ROUND2(D,A,B,C, 6, 5);
+ ROUND2(C,D,A,B, 10, 9); ROUND2(B,C,D,A, 14, 13);
+ ROUND2(A,B,C,D, 3, 3); ROUND2(D,A,B,C, 7, 5);
+ ROUND2(C,D,A,B, 11, 9); ROUND2(B,C,D,A, 15, 13);
+
+ ROUND3(A,B,C,D, 0, 3); ROUND3(D,A,B,C, 8, 9);
+ ROUND3(C,D,A,B, 4, 11); ROUND3(B,C,D,A, 12, 15);
+ ROUND3(A,B,C,D, 2, 3); ROUND3(D,A,B,C, 10, 9);
+ ROUND3(C,D,A,B, 6, 11); ROUND3(B,C,D,A, 14, 15);
+ ROUND3(A,B,C,D, 1, 3); ROUND3(D,A,B,C, 9, 9);
+ ROUND3(C,D,A,B, 5, 11); ROUND3(B,C,D,A, 13, 15);
+ ROUND3(A,B,C,D, 3, 3); ROUND3(D,A,B,C, 11, 9);
+ ROUND3(C,D,A,B, 7, 11); ROUND3(B,C,D,A, 15, 15);
+
+ A += AA; B += BB;
+ C += CC; D += DD;
+
+ A &= MASK32; B &= MASK32;
+ C &= MASK32; D &= MASK32;
+
+ m->A = A; m->B = B; m->C = C; m->D = D;
+}
+
+static void
+copy64(uint32_t *M, const unsigned char *in)
+{
+ int i;
+
+ for (i=0;i<16;i++)
+ M[i] = (in[i*4+3]<<24) | (in[i*4+2]<<16) |
+ (in[i*4+1]<<8) | (in[i*4+0]<<0);
+}
+
+static void
+copy4(unsigned char *out,uint32_t x)
+{
+ out[0] = x&0xFF;
+ out[1] = (x>>8)&0xFF;
+ out[2] = (x>>16)&0xFF;
+ out[3] = (x>>24)&0xFF;
+}
+
+void
+mdfour_begin(struct mdfour *md)
+{
+ md->A = 0x67452301;
+ md->B = 0xefcdab89;
+ md->C = 0x98badcfe;
+ md->D = 0x10325476;
+ md->totalN = 0;
+ md->tail_len = 0;
+}
+
+
+static void
+mdfour_tail(const unsigned char *in, int n)
+{
+ unsigned char buf[128];
+ uint32_t M[16];
+ uint32_t b;
+
+ m->totalN += n;
+
+ b = m->totalN * 8;
+
+ memset(buf, 0, 128);
+ if (n) memcpy(buf, in, n);
+ buf[n] = 0x80;
+
+ if (n <= 55)
+ {
+ copy4(buf+56, b);
+ copy64(M, buf);
+ mdfour64(M);
+ }
+ else
+ {
+ copy4(buf+120, b);
+ copy64(M, buf);
+ mdfour64(M);
+ copy64(M, buf+64);
+ mdfour64(M);
+ }
+}
+
+void
+mdfour_update(struct mdfour *md, const unsigned char *in, int n)
+{
+ uint32_t M[16];
+
+ m = md;
+
+ if (in == NULL)
+ {
+ mdfour_tail(md->tail, md->tail_len);
+ return;
+ }
+
+ if (md->tail_len)
+ {
+ int len = 64 - md->tail_len;
+ if (len > n) len = n;
+ memcpy(md->tail+md->tail_len, in, len);
+ md->tail_len += len;
+ n -= len;
+ in += len;
+ if (md->tail_len == 64)
+ {
+ copy64(M, md->tail);
+ mdfour64(M);
+ m->totalN += 64;
+ md->tail_len = 0;
+ }
+ }
+
+ while (n >= 64)
+ {
+ copy64(M, in);
+ mdfour64(M);
+ in += 64;
+ n -= 64;
+ m->totalN += 64;
+ }
+
+ if (n)
+ {
+ memcpy(md->tail, in, n);
+ md->tail_len = n;
+ }
+}
+
+
+void
+mdfour_result(struct mdfour *md, unsigned char *out)
+{
+ m = md;
+
+ copy4(out, m->A);
+ copy4(out+4, m->B);
+ copy4(out+8, m->C);
+ copy4(out+12, m->D);
+}
+
+
+void
+mdfour(unsigned char *out, const unsigned char *in, int n)
+{
+ struct mdfour md;
+ mdfour_begin(&md);
+ mdfour_update(&md, in, n);
+ mdfour_update(&md, NULL, 0);
+ mdfour_result(&md, out);
+}
Index: mdfour.h
===================================================================
RCS file: mdfour.h
diff -N mdfour.h
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ mdfour.h 19 Oct 2006 19:44:57 -0000
@@ -0,0 +1,39 @@
+/*
+ Unix SMB/Netbios implementation.
+ Version 1.9.
+ a implementation of MD4 designed for use in the SMB authentication protocol
+ Copyright (C) Andrew Tridgell 1997-1998.
+
+ This program is free software; you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation; either version 2 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program; if not, write to the Free Software
+ Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+*/
+
+#include <stdint.h>
+
+struct mdfour
+{
+ uint32_t A, B, C, D;
+ uint32_t totalN;
+ unsigned char tail[64];
+ unsigned tail_len;
+};
+
+void mdfour_begin(struct mdfour *md);
+void mdfour_update(struct mdfour *md, const unsigned char *in, int n);
+void mdfour_result(struct mdfour *md, unsigned char *out);
+void mdfour(unsigned char *out, const unsigned char *in, int n);
+
+
+
+
Index: session.h
===================================================================
RCS file: /cvs/systemtap/src/session.h,v
retrieving revision 1.11
diff -u -p -u -p -r1.11 session.h
--- session.h 11 Oct 2006 14:56:09 -0000 1.11
+++ session.h 19 Oct 2006 19:44:57 -0000
@@ -85,6 +85,11 @@ struct systemtap_session
int buffer_size;
unsigned perfmon;
+ // Cache data
+ bool use_cache;
+ std::string cachedir;
+ std::string hash_path;
+
// temporary directory for module builds etc.
// hazardous - it is "rm -rf"'d at exit
std::string tmpdir;
Index: util.cxx
===================================================================
RCS file: util.cxx
diff -N util.cxx
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ util.cxx 19 Oct 2006 19:44:57 -0000
@@ -0,0 +1,199 @@
+// Copyright (C) Andrew Tridgell 2002 (original file)
+// Copyright (C) 2006 Red Hat Inc. (systemtap changes)
+//
+// This program is free software; you can redistribute it and/or
+// modify it under the terms of the GNU General Public License as
+// published by the Free Software Foundation; either version 2 of the
+// License, or (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful, but
+// WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.See the GNU
+// General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program; if not, write to the Free Software
+// Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+
+#include "util.h"
+#include <stdexcept>
+#include <cerrno>
+
+extern "C" {
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <pwd.h>
+#include <unistd.h>
+#include <stdlib.h>
+#include <fcntl.h>
+}
+
+using namespace std;
+
+
+// Return current users home directory or die.
+const char *
+get_home_directory(void)
+{
+ const char *p = getenv("HOME");
+ if (p)
+ return p;
+
+ struct passwd *pwd = getpwuid(getuid());
+ if (pwd)
+ return pwd->pw_dir;
+
+ throw runtime_error("Unable to determine home directory");
+ return NULL;
+}
+
+
+// Copy a file. The copy is done via a temporary file and atomic
+// rename.
+int
+copy_file(const char *src, const char *dest)
+{
+ int fd1, fd2;
+ char buf[10240];
+ int n;
+ string tmp;
+ char *tmp_name;
+ mode_t mask;
+
+ // Open the src file.
+ fd1 = open(src, O_RDONLY);
+ if (fd1 == -1)
+ return -1;
+
+ // Open the temporary output file.
+ tmp = dest + string(".XXXXXX");
+ tmp_name = (char *)tmp.c_str();
+ fd2 = mkstemp(tmp_name);
+ if (fd2 == -1)
+ {
+ close(fd1);
+ return -1;
+ }
+
+ // Copy the src file to the temporary output file.
+ while ((n = read(fd1, buf, sizeof(buf))) > 0)
+ {
+ if (write(fd2, buf, n) != n)
+ {
+ close(fd2);
+ close(fd1);
+ unlink(tmp_name);
+ return -1;
+ }
+ }
+ close(fd1);
+
+ // Set the permissions on the temporary output file.
+ mask = umask(0);
+ fchmod(fd2, 0666 & ~mask);
+ umask(mask);
+
+ // Close the temporary output file. The close can fail on NFS if
+ // out of space.
+ if (close(fd2) == -1)
+ {
+ unlink(tmp_name);
+ return -1;
+ }
+
+ // Rename the temporary output file to the destination file.
+ unlink(dest);
+ if (rename(tmp_name, dest) == -1)
+ {
+ unlink(tmp_name);
+ return -1;
+ }
+
+ return 0;
+}
+
+
+// Make sure a directory exists.
+int
+create_dir(const char *dir)
+{
+ struct stat st;
+ if (stat(dir, &st) == 0)
+ {
+ if (S_ISDIR(st.st_mode))
+ return 0;
+ errno = ENOTDIR;
+ return 1;
+ }
+
+ if (mkdir(dir, 0777) != 0 && errno != EEXIST)
+ return 1;
+
+ return 0;
+}
+
+
+void
+tokenize(const string& str, vector<string>& tokens,
+ const string& delimiters = " ")
+{
+ // Skip delimiters at beginning.
+ string::size_type lastPos = str.find_first_not_of(delimiters, 0);
+ // Find first "non-delimiter".
+ string::size_type pos = str.find_first_of(delimiters, lastPos);
+
+ while (pos != string::npos || lastPos != string::npos)
+ {
+ // Found a token, add it to the vector.
+ tokens.push_back(str.substr(lastPos, pos - lastPos));
+ // Skip delimiters. Note the "not_of"
+ lastPos = str.find_first_not_of(delimiters, pos);
+ // Find next "non-delimiter"
+ pos = str.find_first_of(delimiters, lastPos);
+ }
+}
+
+
+// Find an executable by name in $PATH.
+bool
+find_executable(const char *name, string& retpath)
+{
+ const char *p;
+ string path;
+ vector<string> dirs;
+ struct stat st1, st2;
+
+ if (*name == '/')
+ {
+ retpath = name;
+ return true;
+ }
+
+ p = getenv("PATH");
+ if (!p)
+ return false;
+ path = p;
+
+ // Split PATH up.
+ tokenize(path, dirs, string(":"));
+
+ // Search the path looking for the first executable of the right name.
+ for (vector<string>::iterator i = dirs.begin(); i != dirs.end(); i++)
+ {
+ string fname = *i + "/" + name;
+ const char *f = fname.c_str();
+
+ // Look for a normal executable file.
+ if (access(f, X_OK) == 0
+ && lstat(f, &st1) == 0
+ && stat(f, &st2) == 0
+ && S_ISREG(st2.st_mode))
+ {
+ // Found it!
+ retpath = fname;
+ return true;
+ }
+ }
+
+ return false;
+}
Index: util.h
===================================================================
RCS file: util.h
diff -N util.h
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ util.h 19 Oct 2006 19:44:57 -0000
@@ -0,0 +1,13 @@
+#include <string>
+#include <vector>
+
+const char *get_home_directory(void);
+
+int copy_file(const char *src, const char *dest);
+
+int create_dir(const char *dir);
+
+void tokenize(const std::string& str, std::vector<std::string>& tokens,
+ const std::string& delimiters);
+
+bool find_executable(const char *name, std::string& retpath);