[ltc-perf] draft of nfs event hook
Xue Peng Li
xuepengl@cn.ibm.com
Fri Aug 11 01:59:00 GMT 2006
Hi folks,
I am working on NFS tapsets recently which could be able to probe
both NFS server side and client side operations. As the first step, I am
working on tapsets about NFS client side. This tapset includes three
levels, i.e. file operation, address space operation, client-side
procedures stubs(nfs_proc function). I have finished the tapset about
fop(file operation) and aop (address space operation). I resues the
Tom's vfs.stp to avoid duplicate work.
I will be very happy if you can take a look at it. Feel free to tell
me if you have any questions/suggestions/comments.
Thanks.
Li Guanglei wrote:
> Hi,
>
> The NFS trace hooks we are working on will be part of the trace hooks
> of LKET, which is a system trace tool and we mainly use it for
> performance analysis.
>
> LKET is a dynamic trace facility based on SystemTap. It is actually
> implemented as SystemTap's tapsets library and it has been integrated
> into SystemTap already. For more info of LKET, you can refer to:
>
> http://sourceware.org/systemtap/man5/lket.5.html
>
> When we started working on NFS trace hooks, we realized it is not an
> easy task. Although we use NFS in daily work but we don't have much
> knowledge about the NFS protocol details and its implementation inside
> the Kernel. So I divided the work into two steps. At the first step I
> need get a list of trace points. And at the second step I need to make
> sure what trace data is available for each trace hook. In a short, the
> trace data available for each hook will be derived from the arguments of
> the kernel functions being probed.
>
> We read through the Kernel source code and chose some functions to be
> instrumented. We will trace the entry of these functions and if
> necessary, the return of them will also be traced. The following is the
> list of these functions, please take a review:
>
> ==================== Client Side ==========================
>
> <1> nfs directory operations
>
> All functions from nfs_dir_operations:
>
> const struct file_operations nfs_dir_operations = {
> .llseek = nfs_llseek_dir,
> .read = generic_read_dir,
> .readdir = nfs_readdir,
> .open = nfs_opendir,
> .release = nfs_release,
> .fsync = nfs_fsync_dir,
> };
>
> <2> nfs file operations
>
> All functions from nfs_file_operations:
>
> const struct file_operations nfs_file_operations = {
> .llseek = nfs_file_llseek,
> .read = do_sync_read,
> .write = do_sync_write,
> .aio_read = nfs_file_read,
> .aio_write = nfs_file_write,
> .mmap = nfs_file_mmap,
> .open = nfs_file_open,
> .flush = nfs_file_flush,
> .release = nfs_file_release,
> .fsync = nfs_fsync,
> .lock = nfs_lock,
> .flock = nfs_flock,
> .sendfile = nfs_file_sendfile,
> .check_flags = nfs_check_flags,
> };
>
> <3> nfs address space operations:
> All functions from nfs_file_aops:
>
> struct address_space_operations nfs_file_aops = {
> .readpage = nfs_readpage,
> .readpages = nfs_readpages,
> .set_page_dirty = __set_page_dirty_nobuffers,
> .writepage = nfs_writepage,
> .writepages = nfs_writepages,
> .prepare_write = nfs_prepare_write,
> .commit_write = nfs_commit_write,
> .invalidatepage = nfs_invalidate_page,
> .releasepage = nfs_release_page,
> #ifdef CONFIG_NFS_DIRECTIO
> .direct_IO = nfs_direct_IO,
> #endif
> };
>
> <4> NFS RPC procedures:
>
> All functions from nfs_v[2,3,4]_clientops:
> I only list the nfs_v3 rpc procedures:
> struct nfs_rpc_ops nfs_v3_clientops = {
> .version = 3, /* protocol version */
> .dentry_ops = &nfs_dentry_operations,
> .dir_inode_ops = &nfs3_dir_inode_operations,
> .file_inode_ops = &nfs3_file_inode_operations,
> .getroot = nfs3_proc_get_root,
> .getattr = nfs3_proc_getattr,
> .setattr = nfs3_proc_setattr,
> .lookup = nfs3_proc_lookup,
> .access = nfs3_proc_access,
> .readlink = nfs3_proc_readlink,
> .read = nfs3_proc_read,
> .write = nfs3_proc_write,
> .commit = nfs3_proc_commit,
> .create = nfs3_proc_create,
> .remove = nfs3_proc_remove,
> .unlink_setup = nfs3_proc_unlink_setup,
> .unlink_done = nfs3_proc_unlink_done,
> .rename = nfs3_proc_rename,
> .link = nfs3_proc_link,
> .symlink = nfs3_proc_symlink,
> .mkdir = nfs3_proc_mkdir,
> .rmdir = nfs3_proc_rmdir,
> .readdir = nfs3_proc_readdir,
> .mknod = nfs3_proc_mknod,
> .statfs = nfs3_proc_statfs,
> .fsinfo = nfs3_proc_fsinfo,
> .pathconf = nfs3_proc_pathconf,
> .decode_dirent = nfs3_decode_dirent,
> .read_setup = nfs3_proc_read_setup,
> .read_done = nfs3_read_done,
> .write_setup = nfs3_proc_write_setup,
> .write_done = nfs3_write_done,
> .commit_setup = nfs3_proc_commit_setup,
> .commit_done = nfs3_commit_done,
> .file_open = nfs_open,
> .file_release = nfs_release,
> .lock = nfs3_proc_lock,
> .clear_acl_cache = nfs3_forget_cached_acls,
> };
>
> The LKET already has syscall and iosyscall trace hooks. So with the
> above trace hooks, LKET could trace different layer of NFS operations:
> --> Syscall
> --> struct file_operations
> --> struct address_space_operations
> --> struct nfs_rpc_ops
>
> ======================= Server Side =============================
>
> <1> nfsd_dispatch
> This is the NFS dispatching function sit on top of RPC.
>
> <2> NFS RPC procedures:
>
> For NFSv4, it will be nfsd4_proc_compound
>
> For NFSv2, NFSv3, it will be the functions from nfsd_procedures[2,3]
>
> Here is a list for NFSv3, NFSv2 are almost the same:
> nfsd3_proc_null,
> nfsd3_proc_getattr,
> nfsd3_proc_setattr,
> nfsd3_proc_lookup,
> nfsd3_proc_access,
> nfsd3_proc_readlink,
> nfsd3_proc_read,
> nfsd3_proc_write,
> nfsd3_proc_create,
> nfsd3_proc_mkdir,
> nfsd3_proc_symlink,
> nfsd3_proc_mknod,
> nfsd3_proc_remove,
> nfsd3_proc_rmdir,
> nfsd3_proc_rename,
> nfsd3_proc_link,
> nfsd3_proc_readdir,
> nfsd3_proc_readdirplus,readdirplus,
> nfsd3_proc_fsstat,
> nfsd3_proc_fsinfo,
> nfsd3_proc_pathconf,
> nfsd3_proc_commit,
>
> <3> NFSD file VFS operations
>
> The functions nfsd_xxx from "fs/nfsd/vfs.c"
>
> With the above server side trace hooks, LKET could trace NFS operations
> at different layer:
>
> nfsd_dispatch -->
> --> NFS RPC Procedures
> --> NFS VFS file operations
>
>
> What I didn't list about NFS operations includes authentication, NFSv4
> callback and RPC(I prefer to use a separate set of trace hooks for RPC).
> I am not sure if these operations are also required to be traced. If I
> missed some important functions or I listed some redundant functions,
> please feel free to let me know. Any comments will be highly appreciated.
>
> Thanks.
>
> The following is from Li Xuepeng posted on nfs@lists.sourceforge.net
> which involved some implementations details and its trace point lists is
> a subset of the above.
>
> - Guanglei
>
> Xue Peng Li ??:
>> Hi folks,
>>
>> I am working on NFS trace hooks for SystemTap/LKET. These trace
>> hooks could be used for performance analyzing which will trace both
>> NFS client and server side activities.
>>
>> At the first step I need make sure that the trace hooks I defined
>> are appropriate and every trace hook probes the right places inside
>> the Kernel. So I will be appreciated if you could help me review the
>> following trace hooks.
>>
>>
>> Thanks
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: nfs.stp
URL: <http://sourceware.org/pipermail/systemtap/attachments/20060811/5455557e/attachment.ksh>
More information about the Systemtap
mailing list