No OneTemporary
Actions

Size

33 KB

Referenced Files

None

Subscribers

None

View Options

	diff --git a/askant/INSTALL b/askant/INSTALL
	new file mode 100644
	index 000000000..ce26b8255
	--- /dev/null
	+++ b/askant/INSTALL
	@@ -0,0 +1,42 @@
	+
	+ How To Install Askant
	+ ---------------------
	+
	+o Build Dependencies
	+ - Python
	+ - libgfs2 (part of the Red Hat cluster suite as a static lib)
	+
	+o Dependencies
	+ - Python
	+ - blktrace and blkparse with appropriate kernel config options enabled and
	+ debugfs mounted
	+
	+o Intro
	+ Askant uses Python's distutils for its standard installation procedure so the
	+ setup.py script provides installation routines. Note that it doesn't provide
	+ uninstallation routines so the best way to install it is to obtain a package
	+ for your particular distribution and install that.
	+
	+o Installation
	+ To install askant to the default location on your system, run as root:
	+
	+ ./setup.py install
	+
	+ If you wish to install it to a different root directory (e.g. /var/mychroot)
	+ or prefix directory (e.g. /usr/local instead of /usr) see:
	+
	+ ./setup.py install --help
	+
	+ To find out everything else setup.py can do:
	+
	+ ./setup.py --help-commands
	+
	+o Use Without Installing
	+ To use askant without installing it (e.g. if you want to test patches
	+ quickly) it is necessary to carry out the build stage to compile the required
	+ plugins. Do:
	+
	+ ./setup.py build
	+ cd build/lib*
	+ python askant/askant.py
	+
	diff --git a/askant/PLUGINAPI b/askant/PLUGINAPI
	new file mode 100644
	index 000000000..4c2ecebc7
	--- /dev/null
	+++ b/askant/PLUGINAPI
	@@ -0,0 +1,65 @@
	+
	+ Writing File System Plugins for Askant
	+ --------------------------------------
	+
	+0. Contents
	+
	+ 1. Intro
	+ 2. API
	+ 3. Reference
	+
	+
	+1. Intro
	+
	+In order to gather file system data with which to enhance the data provided by
	+blktrace, there must be specific code written for each file system type we wish
	+to support. This is where file system plugins come along. The plugins must
	+basically traverse an on-disk file system structure on a block device and report
	+back their findings through a callback handed to them by askant. They must also
	+expose a function which stops their traversal loop in case askant wishes to stop
	+their execution prematurely.
	+
	+
	+2. API
	+
	+A file system plugin must be written as a Python module. This means that C, C++
	+and Python are suitable languages to write them in. They must expose a number of
	+Python functions to askant:
	+
	+parsefs(dev)
	+
	+ This function is called when askant wants the plugin to parse the file
	+ system on the device specified by the device, dev.
	+
	+get_block_size()
	+
	+ This function is called to ascertain the file system block size which is
	+ required in order to map the disk sector numbers provided by blktrace to
	+ file system block numbers. It takes no arguments and should return a Python
	+ integer value.
	+
	+set_report_hook(func)
	+
	+ This function is called to pass in a report function which the plugin should
	+ call to report the details of a file system block. func is a function which
	+ accepts four arguments in this order:
	+
	+ Block number: a Python long integer
	+ Block type: a Python string
	+ Block number of parent: a Python long integer
	+ Block file name: a Python string (should be "" for non-inode blocks)
	+
	+handle_sigint()
	+
	+ This function should make the plugin stop its parsing loop. This allows it
	+ to be interrupted if askant catches a signal, for example. It takes no
	+ arguments and its return value is not used.
	+
	+
	+3. Reference
	+
	+o See fsplugins/gfs2/gfs2module.c for an example of writing a Python fs plugin
	+ module in C.
	+
	+o http://www.python.org/doc/ext/intro.html shows how to extend Python with C or
	+ C++.
	diff --git a/askant/README b/askant/README
	new file mode 100644
	index 000000000..74e32193f
	--- /dev/null
	+++ b/askant/README
	@@ -0,0 +1,74 @@
	+
	+ Askant - A Block I/O Analysis Tool
	+ ----------------------------------
	+
	+0. Contents
	+
	+ 1. Intro
	+ 2. Commands and their options
	+ 3. Examples
	+
	+
	+1. Intro
	+
	+Askant allows you to gather file system block I/O data for performance analysis.
	+It uses blktrace to trace block operations and adds file system specific data by
	+parsing fs block information straight from the storage device. To do this it
	+invokes functionality from file system parser plugins which traverse the on-disk
	+structures and report details of the blocks they encounter through callbacks.
	+
	+Usage: askant <command> [options...]
	+
	+o See 'askant --commands' for a full list of commands.
	+
	+
	+2. Commands and their options
	+
	+ dumpfs
	+
	+ Dumps the block data provided by an fs plugin from the given file system.
	+
	+ -d The block device which contains the file system under scrutiny
	+ -t The type of the file system, i.e. the name of the fs plugin to load
	+ -o Output file (optional). Stdout is used if '-' or omitted.
	+ (Note that the output file should be on a different device to the one
	+ specified with -d.)
	+
	+ unlinks
	+
	+ Does a dumpfs and runs blktrace until interrupted. It then blkparses the
	+ blktrace data and adds the block data. This is suitable for tracing I/O
	+ operations during an unlink test run.
	+
	+ -d The block device which contains the file system under scrutiny
	+ -t The type of the file system, i.e. the name of the fs plugin to load
	+ -g The path to the debugfs mount point, defaults to /sys/kernel/debug
	+ -D The directory in which to store the block dumps and blktrace files.
	+
	+ creates
	+
	+ Runs blktrace until interrupted. It then does a dumpfs and blkparses the
	+ blktrace data and adds the block data. This is suitable for tracing I/O
	+ operations during a file creation test run.
	+
	+ -d The block device which contains the file system under scrutiny
	+ -t The type of the file system, i.e. the name of the fs plugin to load
	+ -g The path to the debugfs mount point, defaults to /sys/kernel/debug
	+ -D The directory in which to store the block dumps and blktrace files.
	+
	+3. Examples
	+
	+o Parse the file system and dump information about each block to stdout e.g.:
	+
	+ # askant dumpfs -t gfs2 -d /dev/sda3
	+
	+o Trace 'unlinks' by dumping block information before the blktrace and matching
	+ block data to provide a more detailed blkparse output e.g.:
	+
	+ # askant unlinks -t gfs2 -d /dev/sda3 -D /tmp/dir -g /mnt/debugfs
	+
	+o Trace 'creates' by dumping block information after the blktrace and matching
	+ block data to provide a more detailed blkparse output e.g.:
	+
	+ # askant creates -t gfs2 -d /dev/sda3 -D /tmp/dir -g /mnt/debugfs
	+
	diff --git a/askant/askant/__init__.py b/askant/askant/__init__.py
	new file mode 100644
	index 000000000..e69de29bb
	diff --git a/askant/askant/about.py b/askant/askant/about.py
	new file mode 100644
	index 000000000..3d40a746e
	--- /dev/null
	+++ b/askant/askant/about.py
	@@ -0,0 +1,5 @@
	+"""
	+Version information for askant.
	+"""
	+
	+version = "0.0.1"
	diff --git a/askant/askant/askant.py b/askant/askant/askant.py
	new file mode 100644
	index 000000000..a4376b6c5
	--- /dev/null
	+++ b/askant/askant/askant.py
	@@ -0,0 +1,24 @@
	+"""
	+The main entry point for askant.
	+"""
	+
	+import sys
	+import commands
	+
	+def main():
	+
	+ """Run askant"""
	+
	+ cmds = commands.Commands()
	+ cmds.register_command(commands.DumpFSCommand())
	+ cmds.register_command(commands.UnlinksCommand())
	+ cmds.register_command(commands.CreatesCommand())
	+
	+ try:
	+ cmds.process_argv(sys.argv)
	+ except commands.NoFSPluginException, p:
	+ print >>sys.stderr, "Plugin not found: %s" %p
	+
	+
	+if __name__ == '__main__':
	+ main()
	diff --git a/askant/askant/blktrace.py b/askant/askant/blktrace.py
	new file mode 100644
	index 000000000..fdf02194b
	--- /dev/null
	+++ b/askant/askant/blktrace.py
	@@ -0,0 +1,93 @@
	+"""
	+Wrapper classes for running blktrace and blkparse and using their output
	+"""
	+
	+import os
	+import sys
	+import time
	+import signal
	+from subprocess import Popen
	+from subprocess import PIPE
	+from sysfs import Sysfs
	+
	+class BlktraceException(Exception):
	+
	+ def __init__(self, val, msg):
	+ Exception.__init__(self)
	+ self.val = val
	+ self.msg = msg
	+
	+ def __str__(self):
	+ return self.msg
	+
	+class Blktrace:
	+
	+ def __init__(self, dev, debugfs='/sys/kernel/debug'):
	+ self.dev = dev
	+ self.debugfs = debugfs
	+ self.sysfs = Sysfs(dev)
	+ self.btpid = -1
	+
	+ def handle_sigint(self, sig, frame):
	+ if self.btpid >= 0:
	+ os.kill(self.btpid, signal.SIGTERM)
	+
	+ def trace(self, tracefile):
	+
	+ if not self.dev:
	+ raise Exception("No device specified.")
	+
	+ btargs = ['blktrace',
	+ '-d', self.dev,
	+ '-r', self.debugfs,
	+ '-o', tracefile]
	+
	+ blktrace = Popen(btargs, bufsize=1, stdout=PIPE,
	+ stderr=open('/dev/null','w'))
	+ self.btpid = blktrace.pid
	+ btres = None
	+ while btres is None:
	+ time.sleep(1)
	+ btres = blktrace.poll()
	+
	+ self.btpid = -1
	+ if btres:
	+ raise BlktraceException(btres,
	+ 'blktrace exited with code ' + str(btres))
	+
	+
	+
	+ def parse(self, tracefile, getblk):
	+
	+ if not self.dev:
	+ raise Exception("No device specified.")
	+
	+ offset = self.sysfs.get_partition_start_sector()
	+
	+ bpargs = ['blkparse', '-i', tracefile]
	+ blkparse = Popen(bpargs, bufsize=1, stdout=PIPE)
	+
	+ bpres = None
	+ while bpres is None:
	+ output = blkparse.stdout.readline().strip()
	+ if output:
	+ chunks = output.split()
	+ try:
	+ # chunks[7] is the sector number
	+ blk = list(getblk(int(chunks[7])))
	+ print "%s %s %s" %\
	+ (' '.join(blk[0:3]),\
	+ output.strip(),
	+ blk[3].strip())
	+ except KeyError:
	+ pass
	+ except ValueError:
	+ pass
	+
	+ bpres = blkparse.poll()
	+
	+ if bpres:
	+ raise BlktraceException(bpres,
	+ 'blkparse exited with code ' + str(bpres))
	+
	+
	diff --git a/askant/askant/commands.py b/askant/askant/commands.py
	new file mode 100644
	index 000000000..ad4639f67
	--- /dev/null
	+++ b/askant/askant/commands.py
	@@ -0,0 +1,333 @@
	+"""Command line parsing and processing for askant"""
	+
	+import fs
	+import os
	+import sys
	+import time
	+import about
	+import signal
	+import tempfile
	+from sysfs import Sysfs, SysfsException
	+from blktrace import Blktrace, BlktraceException
	+from optparse import OptionParser, make_option
	+
	+class NoFSPluginException(Exception):
	+
	+ """A generic Exception class for when file system plugins are missing"""
	+
	+ pass # Nothing special about this exception
	+
	+class Commands:
	+
	+ """Provides a convenient interface to the askant commands"""
	+
	+ def __init__(self):
	+
	+ self.commands = {}
	+ self.parser = OptionParser(usage="%prog COMMAND [options]",
	+ version=about.version)
	+
	+
	+ def register_command(self, command):
	+ self.commands[str(command)] = command
	+
	+ def __lookup_command(self, command):
	+ try:
	+ return self.commands[command]
	+ except KeyError, k:
	+ return DefaultCommand()
	+
	+ def __parse_command(self, command, argv):
	+ self.parser.prog = "%s %s" %\
	+ (self.parser.get_prog_name(), str(command))
	+ self.parser.set_usage(command.get_usage())
	+ optlst = command.get_options()
	+ for o in optlst:
	+ self.parser.add_option(o)
	+ self.parser.set_description(command.get_help())
	+
	+ if str(command) != "":
	+ argv = argv[1:]
	+
	+ return self.parser.parse_args(argv)
	+
	+ def process_argv(self, argv):
	+ try:
	+ command = self.__lookup_command(argv[1])
	+ except IndexError, i:
	+ # This exits
	+ self.parser.error("No command specified. Try --help")
	+
	+ options, args = self.__parse_command(command, argv[1:])
	+
	+ command.do_command(options, args, self)
	+
	+class Command:
	+ def __init__(self):
	+ self.options = []
	+
	+class DefaultCommand(Command):
	+ def __str__(self):
	+ return ""
	+
	+ def get_options(self):
	+ self.options.append(make_option("-c","--commands",
	+ action="store_true",
	+ help="Lists available commands"))
	+ return self.options
	+
	+ def get_usage(self):
	+ return "%prog COMMAND [options] [args]"
	+
	+ def get_help(self):
	+ return "Askant is a tool for tracing I/O activity in Linux "\
	+ "and adding extra file system context by "\
	+ "parsing file system data directly."
	+
	+ def do_command(self, options, args, base):
	+ if options.commands:
	+ base.parser.print_usage()
	+ print "Available commands:"
	+ keys = base.commands.keys()
	+ keys.sort()
	+ for k in keys:
	+ print " %10s %s" %\
	+ (k, base.commands[k].get_help())
	+ print ""
	+ print "For command-specific help, use "\
	+ "%sCOMMAND --help" % base.parser.get_prog_name()
	+ else:
	+ base.parser.error("Command not found. Try --commands")
	+
	+
	+class FSCommandTemplate(Command):
	+ def __init__(self):
	+ self.fsmod = None
	+ self.block_table = {}
	+ self.block_func = self.__report_hook
	+ self.sector_size = None
	+ self.offset = None
	+ self.options = [
	+ make_option("-d","--device", metavar="DEVICE",
	+ help="The device to analyse."),
	+ ]
	+
	+ def load_fs_plugin(self, fsname):
	+ try:
	+ self.fsmod = __import__("fs." + fsname,
	+ globals(), locals(), ["fs"])
	+ except ImportError, i:
	+ raise NoFSPluginException(fsname)
	+
	+ def parse_fs(self, dev, hook=None):
	+ if hook is None:
	+ hook = self.block_func
	+
	+ sfs = Sysfs(dev)
	+ offset = sfs.get_partition_start_sector()
	+ sector_size = sfs.get_dev_sector_size()
	+
	+ self.fsmod.set_report_hook(hook)
	+
	+ try:
	+ sfs = Sysfs(dev)
	+ self.offset = sfs.get_partition_start_sector()
	+ self.sector_size = sfs.get_dev_sector_size()
	+ signal.signal(signal.SIGINT, self.fsmod.handle_sigint)
	+ self.blk_size = self.fsmod.get_block_size(dev)
	+ self.fsmod.parsefs(dev)
	+ signal.signal(signal.SIGINT, signal.SIG_DFL)
	+ except IOError, i:
	+ print >>sys.stderr, str(i)
	+ sys.exit(1)
	+
	+ def set_outfile(self, outfile):
	+ self.outfile = outfile
	+
	+ def trace(self, options, dump_before):
	+
	+ sfs = Sysfs(options.device)
	+ self.offset = sfs.get_partition_start_sector()
	+ self.sector_size = sfs.get_dev_sector_size()
	+
	+ tmstamp = time.strftime('%Y%m%d%H%M%S')
	+ self.block_func = self.__report_hook
	+ try:
	+ self.outfile = open(os.path.join(options.outdir, "blocks-%s" %
	+ tmstamp), 'w')
	+ except IOError, e:
	+ print >>sys.stderr, str(e)
	+ sys.exit(1)
	+
	+ if dump_before:
	+ print >>sys.stderr, "Gathering block data..."
	+ self.parse_fs(options.device)
	+ self.outfile.close()
	+
	+ if options.debugfs:
	+ bt = Blktrace(options.device, options.debugfs)
	+ else:
	+ bt = Blktrace(options.device)
	+
	+ os.chdir(options.outdir) # Blktrace doesn't use absolute paths
	+ print >>sys.stderr, "Tracing. Hit Ctrl-C to end..."
	+ try:
	+ signal.signal(signal.SIGINT, bt.handle_sigint)
	+ bt.trace(tmstamp)
	+ signal.signal(signal.SIGINT, signal.SIG_DFL)
	+ except BlktraceException, b:
	+ signal.signal(signal.SIGINT, signal.SIG_DFL)
	+ print >>sys.stderr, str(b)
	+ sys.exit(1)
	+
	+ if not dump_before:
	+ print >>sys.stderr, "Gathering block data..."
	+ self.parse_fs(options.device)
	+ self.outfile.close()
	+
	+ print >>sys.stderr, "Matching blocks..."
	+ blockdb = open(self.outfile.name, 'r')
	+ blocks = {}
	+ for l in blockdb.readlines():
	+ s = l.split('\t')
	+ blocks[int(s[0])] = tuple(s[1:])
	+
	+ bt.parse(os.path.join(options.outdir, tmstamp), blocks.__getitem__)
	+ blockdb.close()
	+
	+ def __report_hook(self, blk, type, parent, fn):
	+ if not fn:
	+ fn = ""
	+ try:
	+ self.outfile.write("%d\t%d\t%s\t%d\t\"%s\"\n" % (
	+ ((blk * self.blk_size)/self.sector_size) + self.offset,
	+ blk, type, parent, fn))
	+ except Exception, e:
	+ print str(e)
	+
	+ def __dict_hook(self, blk, type, parent, fn):
	+ self.block_table[
	+ ((blk * self.blk_size)/self.sector_size)+self.offset] =\
	+ (blk, type, parent, fn)
	+
	+
	+class DumpFSCommand(FSCommandTemplate):
	+ def __str__(self):
	+ return "dumpfs"
	+
	+ def get_options(self):
	+ self.options.append(make_option("-t","--type", metavar="FSTYPE",
	+ help="The type of file system on the device."))
	+ self.options.append(make_option("-o","--output", metavar="FILE",
	+ default="-",
	+ help="File to write output to or '-' for stdout "
	+ "(default)"))
	+ return self.options
	+
	+ def get_usage(self):
	+ return "%prog [options]"
	+
	+ def get_help(self):
	+ return "Dump fs block information in TSV format."
	+
	+ def do_command(self, options, args, base):
	+
	+ if not options.device:
	+ base.parser.error("No device specified. Use -d.")
	+
	+ if not options.type:
	+ base.parser.error("No file system type specified. "
	+ "Use -t.")
	+ try:
	+ if options.output != "-":
	+ self.set_outfile(open(options.output, 'w'))
	+ else:
	+ self.set_outfile(sys.stdout)
	+ except IOError, e:
	+ print >>sys.stderr,\
	+ "Unable to open file for writing: %s" %\
	+ options.output
	+ return
	+ except KeyError:
	+ pass
	+
	+ self.load_fs_plugin(options.type)
	+ try:
	+ self.parse_fs(options.device)
	+ except SysfsException, s:
	+ base.parser.error(s)
	+
	+class UnlinksCommand(FSCommandTemplate):
	+ def __str__(self):
	+ return "unlinks"
	+
	+ def get_options(self):
	+ self.options.append(make_option("-t","--type", metavar="FSTYPE",
	+ help="The type of file system on the device."))
	+ self.options.append(make_option("-g","--debugfs", metavar="PATH",
	+ default="/sys/kernel/debug",
	+ help="The path to the debugfs mountpoint. Default "
	+ "/sys/kernel/debug"))
	+ self.options.append(make_option("-D","--outdir", metavar="DIR",
	+ help="Directory to write output to. Required."))
	+ return self.options
	+
	+ def get_usage(self):
	+ return "%prog [options]"
	+
	+ def get_help(self):
	+ return "Trace block I/O activity during unlink tests."
	+
	+ def do_command(self, options, args, base):
	+ if not options.device:
	+ base.parser.error("No device specified. Use -d.")
	+
	+ self.device = options.device
	+
	+ if not options.type:
	+ base.parser.error("No file system type specified. "
	+ "Use -t.")
	+ if not options.outdir:
	+ base.parser.error("No output directory specified. "
	+ "Use -D.")
	+
	+ self.load_fs_plugin(options.type)
	+ self.trace(options, dump_before=True)
	+
	+class CreatesCommand(FSCommandTemplate):
	+ def __str__(self):
	+ return "creates"
	+
	+ def get_options(self):
	+ self.options.append(make_option("-t","--type", metavar="FSTYPE",
	+ help="The type of file system on the device."))
	+ self.options.append(make_option("-g","--debugfs", metavar="PATH",
	+ default="/sys/kernel/debug",
	+ help="The path to the debugfs mountpoint. Default "
	+ "/sys/kernel/debug"))
	+ self.options.append(make_option("-D","--outdir", metavar="DIR",
	+ help="Directory to write output to. Required."))
	+ return self.options
	+
	+ def get_usage(self):
	+ return "%prog [options]"
	+
	+ def get_help(self):
	+ return "Trace block I/O activity during create tests."
	+
	+ def do_command(self, options, args, base):
	+ if not options.device:
	+ base.parser.error("No device specified. Use -d.")
	+
	+ self.device = options.device
	+
	+ if not options.type:
	+ base.parser.error("No file system type specified. "
	+ "Use -t.")
	+ if not options.outdir:
	+ base.parser.error("No output directory specified. "
	+ "Use -D.")
	+
	+ self.load_fs_plugin(options.type)
	+ self.trace(options, dump_before=False)
	+
	diff --git a/askant/askant/fs/__init__.py b/askant/askant/fs/__init__.py
	new file mode 100644
	index 000000000..e69de29bb
	diff --git a/askant/askant/sysfs.py b/askant/askant/sysfs.py
	new file mode 100644
	index 000000000..12b7a3bc0
	--- /dev/null
	+++ b/askant/askant/sysfs.py
	@@ -0,0 +1,86 @@
	+"""
	+Provides access to sysfs data pertaining to storage partitions.
	+"""
	+
	+import os
	+import os.path
	+
	+class SysfsException(Exception):
	+ """
	+ An exception which is raised when things go wrong with Sysfs.
	+ """
	+ pass # No functionality to add over Exception class
	+
	+class Sysfs:
	+ """
	+ Provides access to sysfs data pertaining to storage partitions.
	+ """
	+ def __init__(self, partition, sysfs_path='/sys'):
	+ """
	+ Instantiate a Sysfs object. partition should be a path to a
	+ disc partition device e.g. /dev/sda3.
	+ """
	+ self.partition = partition
	+ self.device = self.__partition2parent()
	+ self.sysfs_path = sysfs_path
	+ line = self.__firstline(os.path.join(
	+ sysfs_path,'block',
	+ os.path.split(self.device)[1],
	+ os.path.split(self.partition)[1],
	+ 'start'))
	+ self.partition_start = int(line)
	+ line = self.__firstline(os.path.join(
	+ sysfs_path,'block',
	+ os.path.split(self.device)[1],
	+ 'queue',
	+ 'hw_sector_size'))
	+ self.dev_sector_size = int(line)
	+ line = self.__firstline(os.path.join(
	+ sysfs_path,'block',
	+ os.path.split(self.device)[1],
	+ os.path.split(self.partition)[1],
	+ 'size'))
	+ self.partition_size = int(line)
	+
	+ def __firstline(self, fname):
	+ """
	+ Read the first line of a file
	+ """
	+ try:
	+ fobj = open(fname, 'r')
	+ except IOError:
	+ raise SysfsException(
	+ 'Could not open file "%s" for reading. Please '
	+ 'check your kernel is 2.6.25 or later and '
	+ 'sysfs is mounted.' % fname)
	+ line = fobj.readline()
	+ fobj.close()
	+ return line
	+
	+ def __partition2parent(self):
	+ """
	+ Return the name of a partition device's parent device
	+ e.g. /dev/sda3 -> /dev/sda
	+ """
	+ # This may need thinking about a bit more -
	+ # nothing in life can be this simple.
	+ return self.partition.rstrip('0123456789')
	+
	+
	+ def get_partition_start_sector(self):
	+ """
	+ Look up the start sector of the partition.
	+ """
	+ return self.partition_start
	+
	+ def get_dev_sector_size(self):
	+ """
	+ Look up the size of the device's sectors.
	+ """
	+ return self.dev_sector_size
	+
	+ def get_partition_size(self):
	+ """
	+ Look up the size of the partition.
	+ """
	+ return self.partition_size
	diff --git a/askant/fsplugins/__init__.py b/askant/fsplugins/__init__.py
	new file mode 100644
	index 000000000..e69de29bb
	diff --git a/askant/fsplugins/gfs2/gfs2.c b/askant/fsplugins/gfs2/gfs2.c
	new file mode 100644
	index 000000000..bac9c3509
	--- /dev/null
	+++ b/askant/fsplugins/gfs2/gfs2.c
	@@ -0,0 +1,405 @@
	+/**
	+ * main.c - Functions for parsing the on-disk structure of a GFS2 fs.
	+ * Written by Andrew Price
	+ */
	+
	+#include <stdio.h>
	+#include <stdlib.h>
	+#include <stdarg.h>
	+#include <unistd.h>
	+#include <sys/types.h>
	+#include <sys/stat.h>
	+#include <string.h>
	+#include <fcntl.h>
	+#include <libgfs2.h>
	+#include <errno.h>
	+
	+void (report_func)(long int blk, char type, long int parent, char *fn);
	+
	+/* Define how the block information is reported */
	+#define report(b,p,t,n) \
	+ ((*report_func)((long int)b, t, (long int)p,n))
	+/* (printf("%lu\t%s\t%lu\t%s\n", (long int)b, t, (long int)p, f)) */
	+#define report_data(b,p) report(b,p,"D","")
	+#define report_leaf(b,p) report(b,p,"L","")
	+#define report_indir(b,p) report(b,p,"i","")
	+#define report_inode(b,p,n) report(b,p,"I",n)
	+
	+struct blk_extended {
	+ uint64_t parent_blk;
	+ char *fname;
	+ char *blk;
	+};
	+
	+/* FIXME: A static length block stack is most likely a stupid idea */
	+struct blkstack {
	+ int top;
	+ struct blk_extended blocks[1024];
	+};
	+
	+struct blkstack blk_stack;
	+struct gfs2_sb sb;
	+uint32_t blk_size;
	+off_t max_seek;
	+int fd;
	+int flag_stop;
	+
	+/* prog_name and print_it() are needed to satisfy externs in libgfs2 */
	+char *prog_name = "askant";
	+
	+void print_it(const char label, const char fmt, const char *fmt2, ...)
	+{
	+ va_list args;
	+
	+ va_start(args, fmt2);
	+ printf("%s = ", label);
	+ vprintf(fmt, args);
	+ printf("\n");
	+ va_end(args);
	+}
	+
	+/**
	+ * Push a block onto the stack
	+ */
	+static void push_blk(char blk, uint64_t parent, char fn)
	+{
	+ struct blk_extended eblk;
	+
	+ eblk.blk = blk;
	+ eblk.parent_blk = parent;
	+ eblk.fname = fn;
	+
	+ blk_stack.top++;
	+ blk_stack.blocks[blk_stack.top] = eblk;
	+}
	+
	+/**
	+ * Initialise the block stack
	+ */
	+static int blk_stack_init(void)
	+{
	+ blk_stack.top = -1;
	+ return 1;
	+}
	+
	+/**
	+ * Pop a block from the stack
	+ */
	+static struct blk_extended pop_blk(void)
	+{
	+ return blk_stack.blocks[blk_stack.top--];
	+}
	+
	+/**
	+ * Read the GFS2 superblock into the global sb variable.
	+ * Returns 1 on error, 0 on success.
	+ */
	+static int read_gfs2_sb(void)
	+{
	+ off_t offsetsb;
	+ off_t offsetres;
	+ unsigned char buffer[GFS2_BASIC_BLOCK];
	+ ssize_t readsz;
	+
	+ offsetsb = GFS2_SB_ADDR * GFS2_BASIC_BLOCK;
	+ offsetres = lseek(fd, offsetsb, SEEK_SET);
	+
	+ if (offsetres != offsetsb) {
	+ fprintf(stderr, "Could not seek to sb location on device.\n");
	+ return 0;
	+ }
	+
	+ readsz = read(fd, buffer, GFS2_BASIC_BLOCK);
	+ if (readsz != GFS2_BASIC_BLOCK) {
	+ fprintf(stderr, "Could not read superblock.\n");
	+ return 0;
	+ }
	+
	+ gfs2_sb_in(&sb, (char *)buffer);
	+ if (check_sb(&sb)) {
	+ fprintf(stderr, "Not a GFS2 filesystem.\n");
	+ return 0;
	+ }
	+
	+ return 1;
	+}
	+
	+/**
	+ * Read a block.
	+ * blk_offset must be a block number.
	+ * The returned pointer must be free'd.
	+ */
	+static char *read_gfs2_blk(off_t blk_offset)
	+{
	+ off_t offset;
	+ off_t offsetres;
	+ ssize_t readsz;
	+ char *buffer;
	+
	+ buffer = (char *)malloc(blk_size);
	+ if (!buffer) {
	+ fprintf(stderr, "Could not allocate memory for block.\n");
	+ return NULL;
	+ }
	+
	+ offset = blk_offset * blk_size;
	+
	+ offsetres = lseek(fd, offset, SEEK_SET);
	+ if (offsetres != offset) {
	+ fprintf(stderr,
	+ "Could not seek to block location: %lu error: %s\n",
	+ (long int)blk_offset, strerror(errno));
	+ return NULL;
	+ }
	+
	+ readsz = read(fd, buffer, blk_size);
	+ if (readsz != blk_size) {
	+ fprintf(stderr, "Could not read block: %lu\n",
	+ (long int)blk_offset);
	+ return NULL;
	+ }
	+
	+ return buffer;
	+}
	+
	+/**
	+ * Look at indirect pointers from a starting point in a block.
	+ */
	+static void do_indirect(char *start, uint16_t height, uint64_t parent)
	+{
	+ uint64_t ptr;
	+ unsigned int i;
	+ char *blk;
	+
	+ for (i = 0; i < blk_size; i += sizeof(uint64_t)) {
	+ ptr = be64_to_cpu((uint64_t )(start + i));
	+ if (ptr > 0 && ptr < (max_seek / blk_size)) {
	+ if (height == 1) {
	+ report_data(ptr, parent);
	+ } else if (height > 1) {
	+ blk = read_gfs2_blk(ptr);
	+ if (blk) {
	+ report_indir(ptr, parent);
	+ do_indirect(blk, height - 1, ptr);
	+ free(blk);
	+ }
	+ }
	+ } else {
	+ break;
	+ }
	+ }
	+}
	+
	+/**
	+ * Parse count number of dirents from a starting point in a block.
	+ */
	+static void do_dirents(char dirents, char end, uint64_t parent, uint64_t gparent)
	+{
	+ struct gfs2_dirent dirent;
	+ char *di_blk;
	+ char *fname;
	+
	+ while (dirents < end) {
	+ gfs2_dirent_in(&dirent, dirents);
	+ if (dirent.de_inum.no_addr &&
	+ dirent.de_inum.no_addr != parent &&
	+ dirent.de_inum.no_addr != gparent &&
	+ dirent.de_name_len > 0 &&
	+ dirent.de_name_len <= GFS2_FNAMESIZE) {
	+
	+ fname = (char *)malloc(dirent.de_name_len + 1);
	+ if (!fname) {
	+ break;
	+ }
	+
	+ memcpy(fname, dirents + sizeof(struct gfs2_dirent),
	+ dirent.de_name_len);
	+ fname[dirent.de_name_len] = '\0';
	+
	+ di_blk = read_gfs2_blk(dirent.de_inum.no_addr);
	+ if (di_blk) {
	+ push_blk(di_blk, parent, fname);
	+ }
	+ }
	+ dirents += dirent.de_rec_len;
	+ }
	+}
	+
	+/**
	+ * Examine the dirents in a leaf block.
	+ * If the leaf is chained, do the chained leaves too.
	+ */
	+static void do_leaf(char *blk, uint64_t parent, uint64_t gparent)
	+{
	+ struct gfs2_leaf leaf;
	+
	+ while (blk) {
	+ gfs2_leaf_in(&leaf, blk);
	+ do_dirents(blk + sizeof(struct gfs2_leaf), blk + blk_size,
	+ parent, gparent);
	+ free(blk);
	+ if (!leaf.lf_next) {
	+ break;
	+ }
	+
	+ blk = read_gfs2_blk(leaf.lf_next);
	+ }
	+}
	+
	+/**
	+ * Parse leaf pointer data and examine the
	+ * dirents in the destination leaf blocks.
	+ */
	+static void do_leaves(char *start, uint64_t parent, uint64_t gparent)
	+{
	+ uint64_t ptr;
	+ uint64_t prev;
	+ unsigned int i;
	+ char *blk;
	+
	+ prev = 0;
	+ for (i = 0; i < blk_size - sizeof(struct gfs2_dinode);
	+ i += sizeof(uint64_t)) {
	+ ptr = be64_to_cpu((uint64_t )(start + i));
	+
	+ if (ptr >= (max_seek / blk_size)) {
	+ break;
	+ }
	+
	+ if (ptr && ptr != prev) {
	+ blk = read_gfs2_blk(ptr);
	+ if (blk) {
	+ report_leaf(ptr, parent);
	+ do_leaf(blk, parent, gparent);
	+ }
	+ prev = ptr;
	+ }
	+ }
	+}
	+
	+/**
	+ * Parse inode data from a block
	+ */
	+static void do_inode_blk(char blk, uint64_t parent, char fname)
	+{
	+ struct gfs2_dinode di;
	+ char *data;
	+
	+ gfs2_dinode_in(&di, blk);
	+ report_inode(di.di_num.no_addr, parent, fname);
	+
	+ data = (char )((struct gfs2_dinode )blk + 1);
	+
	+ if (di.di_height > 0) {
	+ /* Indirect pointers */
	+ do_indirect(data, di.di_height, di.di_num.no_addr);
	+ } else if (S_ISDIR(di.di_mode) && !(di.di_flags & GFS2_DIF_EXHASH)) {
	+ /* Stuffed directory */
	+ do_dirents(data, blk + blk_size, di.di_num.no_addr, parent);
	+ } else if (S_ISDIR(di.di_mode) &&
	+ (di.di_flags & GFS2_DIF_EXHASH) &&
	+ !(di.di_height)) {
	+ /* Directory, has hashtable, height == 0 */
	+ do_leaves(data, di.di_num.no_addr, parent);
	+ }
	+
	+ /* free previously stacked block */
	+ free(fname);
	+ free(blk);
	+}
	+
	+/**
	+ * Get the root dir block and parse the fs
	+ * using a stack to keep track of the unvisited
	+ * inode blocks.
	+ */
	+static void parse_fs(void)
	+{
	+ struct gfs2_inum *root_dir_inum;
	+ struct gfs2_inum *master_dir_inum;
	+ struct blk_extended blk;
	+ char *root_blk;
	+ char *master_blk;
	+
	+ flag_stop = 0;
	+
	+ root_dir_inum = &(sb.sb_root_dir);
	+ master_dir_inum = &(sb.sb_master_dir);
	+
	+ root_blk = read_gfs2_blk(root_dir_inum->no_addr);
	+ master_blk = read_gfs2_blk(master_dir_inum->no_addr);
	+ if (!root_blk \|\| !master_blk) {
	+ return;
	+ }
	+
	+ push_blk(root_blk, root_dir_inum->no_addr, NULL);
	+ while (blk_stack.top >= 0 && !flag_stop) {
	+ blk = pop_blk();
	+ do_inode_blk(blk.blk, blk.parent_blk, blk.fname);
	+ }
	+
	+ push_blk(master_blk, master_dir_inum->no_addr, NULL);
	+ while (blk_stack.top >= 0 && !flag_stop) {
	+ blk = pop_blk();
	+ /* TODO: Examine each block's magic number instead of assuming
	+ * they're inodes. Omitted for now due to time constraints and
	+ * the number of GFS2_METATYPE_*s which need catering for.
	+ */
	+ do_inode_blk(blk.blk, blk.parent_blk, blk.fname);
	+ }
	+}
	+
	+/**
	+ * Raise a flag to stop the parse loop cleanly
	+ */
	+void gfs2_stop(void)
	+{
	+ flag_stop = 1;
	+}
	+
	+/**
	+ * Parse a gfs2 file system on a given device
	+ */
	+int gfs2_parse(char dev, void (func)(long int b, char t, long int p, char f))
	+{
	+ report_func = func;
	+
	+ if (!blk_stack_init()) {
	+ return 0;
	+ }
	+
	+ if ((fd = open(dev, O_RDONLY)) < 0) {
	+ return 0;
	+ }
	+
	+ if (!read_gfs2_sb()) {
	+ close(fd);
	+ return 0;
	+ }
	+
	+ blk_size = sb.sb_bsize;
	+ max_seek = lseek(fd, 0, SEEK_END);
	+
	+ parse_fs();
	+
	+ close(fd);
	+
	+ return 1;
	+}
	+
	+/**
	+ * Return the block size of the gfs2 file system on a given device.
	+ */
	+uint32_t gfs2_block_size(char *dev)
	+{
	+ if ((fd = open(dev, O_RDONLY)) < 0) {
	+ return 0;
	+ }
	+
	+ if (!read_gfs2_sb()) {
	+ close(fd);
	+ return 0;
	+ }
	+ close(fd);
	+ return sb.sb_bsize;
	+}
	diff --git a/askant/fsplugins/gfs2/gfs2.h b/askant/fsplugins/gfs2/gfs2.h
	new file mode 100644
	index 000000000..6c72eb087
	--- /dev/null
	+++ b/askant/fsplugins/gfs2/gfs2.h
	@@ -0,0 +1,3 @@
	+uint32_t gfs2_block_size(char *dev);
	+int gfs2_parse(char dev, void (func)(long int b, char t, long int p, char f));
	+void gfs2_stop(void);
	diff --git a/askant/fsplugins/gfs2/gfs2module.c b/askant/fsplugins/gfs2/gfs2module.c
	new file mode 100644
	index 000000000..c6a0584e1
	--- /dev/null
	+++ b/askant/fsplugins/gfs2/gfs2module.c
	@@ -0,0 +1,104 @@
	+#include <Python.h>
	+
	+#include "gfs2.h"
	+
	+static PyObject *report_func = NULL;
	+
	+void report_func_wrapper(long int blk, char type, long int parent, char fn)
	+{
	+ PyObject *arglist;
	+ PyObject *result;
	+
	+ arglist = Py_BuildValue("lsls", blk, type, parent, fn);
	+ if (!arglist) {
	+ return;
	+ }
	+ result = PyEval_CallObject(report_func, arglist);
	+ Py_DECREF(arglist);
	+ if (!result) {
	+ return;
	+ }
	+ Py_DECREF(result);
	+}
	+
	+static PyObject gfs2_set_report_hook(PyObject self, PyObject *args)
	+{
	+ PyObject *result = NULL;
	+ PyObject *temp;
	+
	+ if (PyArg_ParseTuple(args, "O:set_callback", &temp)) {
	+ if (!PyCallable_Check(temp)) {
	+ PyErr_SetString(PyExc_TypeError, "parameter must be callable");
	+ return NULL;
	+ }
	+ Py_XINCREF(temp);
	+ Py_XDECREF(report_func);
	+ report_func = temp;
	+
	+ Py_INCREF(Py_None);
	+ result = Py_None;
	+ }
	+ return result;
	+}
	+
	+static PyObject gfs2_parsefs(PyObject self, PyObject *args)
	+{
	+ char *dev;
	+
	+ if (!PyArg_ParseTuple(args, "s:parsefs", &dev)) {
	+ return NULL;
	+ }
	+
	+ if (!gfs2_parse(dev, &report_func_wrapper)) {
	+ return PyErr_SetFromErrno(PyExc_IOError);
	+ }
	+ Py_INCREF(Py_None);
	+
	+ return Py_None;
	+}
	+
	+static PyObject gfs2_get_block_size(PyObject self, PyObject *args)
	+{
	+ char *dev;
	+ uint32_t blksize;
	+
	+ if (!PyArg_ParseTuple(args, "s:get_block_size", &dev)) {
	+ return NULL;
	+ }
	+
	+ blksize = gfs2_block_size(dev);
	+ if (!blksize) {
	+ return PyErr_SetFromErrno(PyExc_IOError);
	+ }
	+
	+ PyObject *size = Py_BuildValue("I", blksize);
	+ if (!size) {
	+ return NULL;
	+ }
	+ Py_INCREF(size);
	+ return size;
	+}
	+
	+static PyObject gfs2_handle_sigint(PyObject signum, PyObject *frame)
	+{
	+ gfs2_stop();
	+ Py_INCREF(Py_None);
	+ return Py_None;
	+}
	+
	+static PyMethodDef GFS2Methods[] = {
	+ {"set_report_hook", gfs2_set_report_hook, METH_VARARGS,
	+ "Specify a hook function through which to report blocks."},
	+ {"parsefs", gfs2_parsefs, METH_VARARGS,
	+ "Parses the given block device as a GFS2 file system."},
	+ {"get_block_size", gfs2_get_block_size, METH_VARARGS,
	+ "Returns the file system block size."},
	+ {"handle_sigint", gfs2_handle_sigint, METH_VARARGS,
	+ "Handles signal SIGINT."},
	+ {NULL, NULL, 0, NULL}
	+};
	+
	+PyMODINIT_FUNC initgfs2(void)
	+{
	+ (void)Py_InitModule("gfs2", GFS2Methods);
	+}
	diff --git a/askant/scripts/askant b/askant/scripts/askant
	new file mode 100755
	index 000000000..3375b0c96
	--- /dev/null
	+++ b/askant/scripts/askant
	@@ -0,0 +1,6 @@
	+#! /usr/bin/env python
	+
	+from askant import askant
	+
	+if __name__ == '__main__':
	+ askant.main()
	diff --git a/askant/setup.py b/askant/setup.py
	new file mode 100755
	index 000000000..78daff24c
	--- /dev/null
	+++ b/askant/setup.py
	@@ -0,0 +1,18 @@
	+#!/usr/bin/env python
	+
	+from distutils.core import setup, Extension
	+from askant import about
	+
	+setup(name="askant",
	+ version=about.version,
	+ description="File system performance analysis tool",
	+ author="Andrew Price",
	+ author_email="andy@andrewprice.me.uk",
	+ url="http://andrewprice.me.uk/projects/askant",
	+ packages = ['askant','askant.fs'],
	+ ext_modules = [Extension("askant.fs.gfs2",
	+ sources = ["fsplugins/gfs2/gfs2module.c","fsplugins/gfs2/gfs2.c"],
	+ libraries = ["gfs2"],
	+ include_dirs=['../gfs2/libgfs2', '../gfs2/include', '../make'],
	+ library_dirs=['../gfs2/libgfs2'])],
	+ scripts = ['scripts/askant'])

File Metadata

Mime Type: text/x-diff
Expires: Sat, Nov 23, 11:04 AM (1 d, 18 h)
Storage Engine: blob
Storage Format: Raw Data
Storage Handle: 1018599
Default Alt Text: (33 KB)

No OneTemporaryActions

View Options

File Metadata

Event Timeline

No OneTemporary
Actions