API Reference¶
drgn_tools.bitops module¶
drgn_tools.block module¶
Helpers for block layers.
No kernel modules are required since uek built in all io schduler modules.
- drgn_tools.block.blkdev_name(bdev)¶
Return block device name
- drgn_tools.block.blkdev_ro(bdev)¶
Check whether
struct block_device *is read only
- drgn_tools.block.blkdev_size(bdev)¶
Return block device size
- drgn_tools.block.dump_inflight_io(prog, diskname='all')¶
Dump all inflight io from all disks
- drgn_tools.block.for_each_badblocks(bb)¶
List all bad blocks
- drgn_tools.block.for_each_hw_queue(q)¶
List all hw queues of some request_queue
- drgn_tools.block.for_each_hwq_pending_rq(hwq)¶
List pending requests from
struct blk_mq_hw_ctx *
- drgn_tools.block.for_each_mq_pending_request(q)¶
List pending requests from
struct request_queue *that supports mq
- drgn_tools.block.for_each_request_queue(prog)¶
List all request_queue in the system.
- drgn_tools.block.for_each_sbitmap_set_bit(sb)¶
List set bit in struct sbitmap
- drgn_tools.block.for_each_sq_elevator_rq(q)¶
List request in elevator of legacy request_queue
- drgn_tools.block.for_each_sq_pending_request(q)¶
List pending requests from legacy
struct request_queue
- drgn_tools.block.for_each_tag_bt_set_bit(bt)¶
List all used tag from struct blk_mq_bitmap_tags
- drgn_tools.block.for_each_tag_pending_rq(tags)¶
List all pending request from struct blk_mq_tags
- drgn_tools.block.for_each_tag_pending_rq_uek4(tags)¶
List all pending request from struct blk_mq_tags
It only works for the kernel where struct sbitmap_queue doesn’t exist.
- drgn_tools.block.get_inflight_io_nr(prog, disk)¶
Get inflight io number from some disk
- drgn_tools.block.is_mq(q)¶
Check whether request queue support multiple queue
- drgn_tools.block.print_block_devs_info(prog)¶
Prints the block device information
- drgn_tools.block.request_target(rq)¶
Get the target disk of io request
- drgn_tools.block.rq_flags(rq)¶
Get request operation flags
- drgn_tools.block.rq_op(rq)¶
Get request operation name
- drgn_tools.block.rq_op_ef295ecf(rq)¶
Get request operation name
This only works for kernel which is newer than commit ef295ecf090d (“block: better op and flags encoding”) like uek5.
- drgn_tools.block.rq_op_old(rq)¶
Get request operation name for kernel which is older than commit ef295ecf(uek4)
- drgn_tools.block.rq_pending_time_ns(rq)¶
Get io pending time in ns
- drgn_tools.block.show_rq_issued_cpu(rq)¶
Get the cpu that request was issued from, if cpu is offline, it will be marked in the output with “offline”. For sq, “-” will be returned.
drgn_tools.bt module¶
- drgn_tools.bt.bt(task_or_prog, cpu=None, pid=None, show_vars=False, show_absent=False, retframes=False, indent=0)¶
Format a crash-like stack trace.
This formats a stack trace reminiscent of (but not strictly identical to) the crash “bt” command. The function can be called in several ways, to maximize flexibility. The first argument may be a task struct or pt_regs object, or it may be a
drgn.Threadrepresenting a task. Finally, it can be adrgn.Program, in which case you need to provide a CPU or PID number as an argument. Here are some examples:>>> task = get_some_task_struct(prog) >>> bt(task) ... >>> bt(prog.thread(1)) ... >>> bt(prog, cpu=0) ...
Not all of crash’s bt features are yet implemented, but there is one feature which already surpasses crash’s implementation: printing variable values. When enabled, at each stack frame there will be a listing of each local variable or function arg, and its value. The value may be “absent” if it was optimized out or if the compiler/debuginfo is not able to provide enough information to retrieve it.
This helper also mitigates some issues seen with drgn’s built-in stack trace functionality: sometimes, the stack trace is truncated (typically at a page fault or IRQ boundary). This helper will detect this and “expand” the stack trace by searching for the last
struct pt_regsvariable in it. The helper will print all relevant stack traces. Seeexpand_traces(),expand_frames(), andbt_frames()for ways to use this logic without needing to print the stack trace.- Parameters:
task_or_prog (
Union[Object,Thread,Program]) – Either a task struct pointer, adrgn.Threadobject, or adrgn.Program.cpu (
Optional[int]) – The CPU number to backtrace (only used whentask_or_progis adrgn.Program). Mutually exclusive withpid.pid (
Optional[int]) – The PID to backtrace (only used whentask_or_progis adrgn.Program). Mutually exclusive withcpu.show_vars (
bool) – Whether to enable formatting variables for each frame.show_absent (
bool) – When show_vars=True, this can further expand the output to include absent variables. Normally there’s no reason to see this, since absent variables have no information.retframes (
bool) – When true, returns a list of stack frames.indent (
int) – Number of spaces to indent all output lines
- Return type:
- Returns:
A list of the stack frames which were printed. This can be useful for accessing the variables out of the frames interactively. If you’re writing a script that needs to access frames, you may want to consider the other functions in this module, which do not print the frames.
- drgn_tools.bt.bt_frames(task_or_prog, cpu=None, pid=None)¶
Return the stack frames that
bt()would printThis takes mostly the same args as
bt(), but it doesn’t print anything. It just returns the frames.
- drgn_tools.bt.bt_has(prog, funcname, task=None)¶
Search for tasks whose stack contains a single function
This function is identical to bt_has_any(), but takes only one function argument.
- drgn_tools.bt.bt_has_any(prog, funcs, task=None)¶
Search for tasks whose stack contains the given functions
For each task on the system, examine their stack trace and search for a list of functions. For each task containing such a function on the stack, return a tuple containing a pointer to the task, and the stack frame containing the function call.
This can be an expensive operation, since there may be many running tasks, and unwinding all of them may take some time. As a result, when running against a core dump, this function will cache information in order to improve runtime.
- drgn_tools.bt.expand_frames(trace)¶
Return the frames of an expanded stack trace, flattened to a single list.
This is almost the same as
expand_traces(), except that it returns the frames in a single list, which is easier to access if you have runbt()and you are looking to simply access the correct frame index.- Parameters:
trace (
StackTrace) – A stack trace to expand- Return type:
- Returns:
A list of stack frames
- drgn_tools.bt.expand_traces(trace)¶
Given a stack trace, return all stack segments we can find.
A stack segment would be something like an interrupt stack or NMI. It is possible to have multiple kernel stacks: for instance, a system call, which is interrupted, and then a NMI. Sometimes drgn doesn’t get all of these in one stack trace, and we need to search for additional stack segments by finding a pt_regs variable on the stack. This function returns as many as possible.
- Return type:
- Parameters:
trace (StackTrace)
- drgn_tools.bt.find_pt_regs(trace)¶
Given a stack trace, return the last pt_regs variable found, if any.
- Return type:
- Parameters:
trace (StackTrace)
- drgn_tools.bt.frame_name(prog, frame)¶
Return a suitable name for a stack frame
- Return type:
- Parameters:
prog (Program)
frame (StackFrame)
- drgn_tools.bt.is_pt_regs(type_)¶
Determine whether a type refers to struct pt_regs, (pointer or direct)
- drgn_tools.bt.print_frames(prog, trace, show_vars=False, show_absent=False, start_idx=0, indent=0)¶
Print stack frames using the drgn-tools (crash-like) format
- Parameters:
prog (
Program) – Program - necessary because a list of frames has no reference to the program they are from.trace (
Union[StackTrace,List[StackFrame]]) – The stack trace or list of frames to printshow_vars (
bool) – True if you want to show variablesshow_absent (
bool) – True if you further want to show absent variablesstart_idx (
int) – Where to start counting the frame indices fromindent (
int) – How many spaces to indent the output
- Return type:
drgn_tools.buddyinfo module¶
Helpers for dumping details about the per-zone buddy page allocator
- drgn_tools.buddyinfo.get_per_zone_buddyinfo(zone)¶
Pages are managed in memory blocks: each memory zone has an array
zone->free_areathat tracks blocks of all orders. This function parses and returns a list that records numbers of free blocks.- Parameters:
zone (
Object) –struct zone *of the target zone- Returns:
A list that records numbers of memory blocks of all orders
drgn_tools.cli module¶
The drgn_tools CLI entry point.
This is not intended to be anything “wild & crazy”. It is just the normal drgn REPL, but with helpers to automatically find the vmlinux and modules.
drgn_tools.cmdline module¶
Helpers for command line
- drgn_tools.cmdline.get_cmdline(prog)¶
Returns the kernel command line
drgn_tools.config module¶
Configuration support
Ideally drgn-tools shouldn’t require much in the way of configuration, but some things like debuginfo fetching are better if they can be configured.
- drgn_tools.config.get_config()¶
Return drgn-tools configuration information
- Return type:
drgn_tools.corelens module¶
Run analysis helper code and output to stdout or a directory
- class drgn_tools.corelens.CorelensModule¶
Bases:
ABCThe base class for all corelens modules.
In order to make a helper executable by corelens, you need to create your own sub-class of this one. At a minimum, you must define the field
nameand provide arun()method. Here is a minimal example of a hello world module:class HelloWorld(CorelensModule): name = "hello" def run(self, prog, args): print("Hello, world!")
However, further fields can be set in order to document debuginfo and kernel module expectations, and to handle command line arguments.
- add_args(parser)¶
Set command line arguments for the module
Use the
argparselibrary to add any arguments your program accepts. The arguments may be provided on the command line, or come from thedefault_argsfield.- Params parser:
argument parser to configure
- Return type:
- Parameters:
parser (ArgumentParser)
- property debuginfo_kmods: List[str]¶
A list of kmod names (or fnmatch patterns) we need debuginfo for
This field, when specified, is a list of strings. Each string could be a kmod name, or a
fnmatchpattern which matches several kmods.Unlike
skip_unless_have_kmod, we don’t require that the kmods here are loaded. Instead, we just require that if the modules are loaded, they must have debuginfo present.This can be useful for a subsystem that has several modules which may use it, e.g. the virtio subsystem. Each virtio device has its own kmod implementing it (such as
virtio_blk). There should be no specific requirement that a particular virtio device has its module loaded, but just that all the virtio modules have debuginfo ready.
- property default_args: List[List[str]]¶
When specified, this list contains arguments to be passed by default to the module, when corelens is run in report mode (-a).
Since corelens supports executing the same module multiple times, it may be desirable to run the same module multiple times in different configurations. Thus, each element of this list is a sub-list containing command line arguments. For example, suppose module “mod” should be run as below during a report:
corelens /proc/kcore -M mod –arg one -M mod –arg two
Then you could specify these arguments as:
default_args = [[”–arg”, “one”], [”–arg”, “two”]]
In the more common case, where a module should be run just once with specified arguments, you would use:
default_args = [[”–specified”, “arguments”]]
As a special case, when this is omitted, it is set to the empty list
[], which is a short-hand for[[]], meaning to run module once with no additional arguments.
- run(prog, args)¶
Required: a function which executes the module
This function will be called to execute the module. Use the normal print() function, which writes to
sys.stdout, to print information. When output is directed to a file,sys.stdoutwill be updated, so output redirection is transparent to the function.- Parameters:
prog (
Program) – Kernel being debuggedargs (
Namespace) – Parsed command line arguments (seeadd_args())
- Return type:
- property run_when: str¶
Specify when this corelens module should be run in reports
always: whenever -a or -A are specified verbose: whenever -A is specified never: never run by -a or -A. Can still be run via -M
- property skip_unless_have_kmod: str | None¶
If specified, skip this when the kmod is not present in the kernel
This field, when specified, is a string naming a kernel module (referred to as kmod in these docs to avoid confusing with “corelens module”). This kmod is expected to signify the presence of the subsystem that this module is dealing with. If the kmod is not loaded in the kernel, then this module is skipped by corelens. If the kmod is loaded, but no debuginfo is present, the module is skipped and an error is raised.
- drgn_tools.corelens.all_corelens_modules()¶
- Return type:
- drgn_tools.corelens.default_corelens_modules(verbose=False)¶
Return the default corelens modules to run for reports
- Parameters:
verbose (
bool) – select the more verbose report preset (-A)- Return type:
- drgn_tools.corelens.make_runner(prog)¶
Return a helper, “cl”, which executes corelens commands against a program
The
run()function requires passing theprogas its first argument, which is a bit clunky when trying to run a command. So, this function can return a helper that is bound to the specific program, so that you can simply runcl("command")without including theprogargument. This is intended for interactive environments.
- drgn_tools.corelens.run(prog, cl_cmd)¶
Run a single corelens command
The string
cl_cmdis split using common shell lexing rules, and the first token is used as the name of the corelens module. The remaining tokens are used as arguments to that corelens module. The module is executed againstprog.
drgn_tools.cpuinfo module¶
Helper to view cpuinfo data
- drgn_tools.cpuinfo.aarch64_get_cpu_info(prog)¶
Helper to get cpuinfo data for aarch64
- drgn_tools.cpuinfo.check_smt_enabled(prog)¶
Checks if SMT (Simultaneous Multithreading) is enabled
- drgn_tools.cpuinfo.get_gds_mitigation(prog, cpu_caps_bugs)¶
Extracts Mitigation for GDS
- drgn_tools.cpuinfo.get_itlb_multihit_mitigation(prog, cpu_caps_bugs)¶
Extracts Mitigation for itlb_multihit
- drgn_tools.cpuinfo.get_l1tf_mitigation(prog, cpu_caps_bugs)¶
Extracts Mitigation for L1TF
- drgn_tools.cpuinfo.get_mds_mitigation(prog, cpu_caps_bugs)¶
Extracts Mitigation for MDS
- drgn_tools.cpuinfo.get_meltdown_mitigation(prog, cpu_caps_bugs)¶
Extracts Mitigation for Meltdown
- drgn_tools.cpuinfo.get_mmio_stale_data_mitigation(prog, cpu_caps_bugs)¶
Extracts Mitigation for mmio_stale_data and mmio_unknown
- drgn_tools.cpuinfo.get_retbleed_mitigation(prog, cpu_caps_bugs)¶
Extracts Mitigation for Retbleed
- drgn_tools.cpuinfo.get_spectre_v1_mitigation(prog, cpu_caps_bugs)¶
Extracts Mitigation for Spectre_V1
- drgn_tools.cpuinfo.get_spectre_v2_mitigation(prog, cpu_caps_bugs)¶
Extracts Mitigation for Spectre_V2
- drgn_tools.cpuinfo.get_srbds_mitigation(prog, cpu_caps_bugs)¶
Extracts Mitigation for SRBDS
- drgn_tools.cpuinfo.get_srso_mitigation(prog, cpu_caps_bugs)¶
Extracts Mitigation for SRSO
- drgn_tools.cpuinfo.get_ssb_mitigation(prog, cpu_caps_bugs)¶
Extracts Mitigation for spec_store_bypass
- drgn_tools.cpuinfo.get_taa_mitigation(prog, cpu_caps_bugs)¶
Extracts Mitigation for tsx_async_abort
- drgn_tools.cpuinfo.print_cpu_info(prog)¶
Prints the cpuinfo data
- drgn_tools.cpuinfo.x86_get_cpu_info(prog)¶
Helper to get cpuinfo data for x86
- drgn_tools.cpuinfo.x86_get_cpu_mitigations(prog)¶
Helper to get mitigations for vulnerabilities for x86
- drgn_tools.cpuinfo.x86_get_per_node_cpulist(prog)¶
Parse per-node cpumask and return human readable representation of it. The output is a list of cpulist; each cpulist is a list of CPUs with ranges. For example, [“0-31”, “32-63”].
drgn_tools.debuginfo module¶
The drgn_tools.debuginfo module provides the APIs for finding debuginfo.
Please note: this file is quite special. Currently, it is not just the
drgn_tools.debuginfo module: it is also a file which is copied and pasted
into the /share/linuxtools/bin directory in order to allow people & bash scripts
to easily find and extract debuginfo, while ensuring that they also make the
appropriate updates to the access.db file.
In the future, the CRASH scripts will be updated to call drgn-tools directly to do this. But in the meantime, this quick solution is easier for testing, and allows me to avoid having to deal with OL8 drgn-tools deployment yet. The consequence here is that we can only use the standard library: no imports from drgn or drgn-tools or third-party modules.
- class drgn_tools.debuginfo.KernelVersion(version, version_tuple, release, release_tuple, ol_version, ol_update, arch, original, extraversion1, extraversion2, is_uek, uek_version, is_ueknext)¶
Bases:
NamedTuple- Parameters:
-
ol_update:
Optional[int]¶ The Oracle Linux distribution update.
Note that this is not provided by UEK kernel versions. It is, however, provided by the regular kernel package.
- classmethod parse(original)¶
Parse the given kernel release string and return a
KernelVersion:>>> KernelVersion.parse('4.14.35-2047.516.2.4.el7uek.x86_64') KernelVersion(version='4.14.35', version_tuple=(4, 14, 35), release='2047.516.2.4', release_tuple=(2047, 516, 2, 4), ol_version=7, ol_update=None, arch='x86_64', original='4.14.35-2047.516.2.4.el7uek.x86_64', extraversion1='', extraversion2='', is_uek=True, uek_version=5, is_ueknext=False)
- Parameters:
original (
str) – The kernel’s release string- Return type:
- Returns:
A
KernelVersionwith fields parsed
- drgn_tools.debuginfo.fetch_debuginfo(uname, modules)¶
Fetch debuginfo in a potentially expensive way
Assuming that
find_debuginfo()has failed, we can assume that debuginfo is not easily available locally. However, we may be able to “fetch” it and extract it from a remote source. This is usually costly: it will take some time to download and process the debuginfo. This function may use different strategies depending on the user’s configuration.The result may be incomplete: out-of-tree modules likely can’t be found, and it’s of course possible that nothing can be found.
- drgn_tools.debuginfo.find_debuginfo(prog_or_release, mod, dinfo_path=None)¶
Search for debuginfo (either module or regular debuginfo)
This function searches for a given module’s debuginfo in a list of paths. It returns the path of a match, if found. The debuginfo paths are determined as follows:
Files within
$PWD/$RELEASEare considered, if it exists. Otherwise, files within$PWDare considered.Files in the directory
$DEBUGINFO_BASE/$RELEASEare considered, for each colon-separated path in$DEBUGINFO_BASE, if it exists.Files in
/usr/lib/debug/lib/modules/$RELEASEand./usr/lib/debug/lib/modules/$RELASE, if either exist.Files in
/share/linuxrpm/vmlinux_repo/{64,32}/$RELEASEare searched.
The directories may be searched in one of two ways. For directories whose full paths contain the string
lib/modules, we assume that the directory was created by installing the RPM, or by extracting the RPM directly. This means that the module debuginfo may be in a subdirectory, and so we use a recursive search through subdirectories. For directories which do not contain the stringlib/modules, our search is not recursive. This is mainly to improve performance: listing directories is slow on network filesystems, and there’s a chance that directories like$PWDwill contain a lot of subdirectories.Finally, it is important to note that this function is lenient on module names. It should be called with the original module name, but it will match a module file whose name has had hyphens replaced by underscore. This ensures it can match files extracted by
fetch_debuginfo().
drgn_tools.dentry module¶
Helpers for dentries.
- drgn_tools.dentry.count_dentries_in_hashtable(prog)¶
Count the total number of dentries in hashtable
- drgn_tools.dentry.count_negative_dentries_in_hashtable(prog)¶
Count the number of negative dentries in hashtable
- drgn_tools.dentry.count_unused_dentries_in_hashtable(prog)¶
Count the number of unused dentries in hashtable
- drgn_tools.dentry.count_used_dentries_in_hashtable(prog)¶
Count the number of used dentries in hashtable
- drgn_tools.dentry.d_count(dentry)¶
Count the number of references of a dentry
- drgn_tools.dentry.dentry_for_each_child(dentry)¶
Iterate over every child of a dentry
- drgn_tools.dentry.dentry_is_negative(dentry)¶
Check if a dentry is negative
- drgn_tools.dentry.dentry_is_unused(dentry)¶
Check if a dentry is unused
- drgn_tools.dentry.dentry_is_used(dentry)¶
Check if a dentry is used
- drgn_tools.dentry.dentry_path_any_mount(dentry)¶
Like dentry_path(), but don’t require a path/mount. Just pick one arbitrarily
- drgn_tools.dentry.dentry_path_first_mount(dentry)¶
Return the full path of a dentry from the root of its filesystem, arbitrarily selecting the first mountpoint of its superblock.
The Linux kernel uses
list_add_tail()when creating new mountpoints. So, the first mount will be the oldest and thus it is likely the most relevant or most “expected” mount point, as opposed to bind-mounts or mounts from non-init filesystem namespaces. In short, this is very likely to get you a useful filesystem path for diagnostics, without needing to have astruct vfsmount *.
- drgn_tools.dentry.for_each_dentry_in_hashtable(prog)¶
Get all the dentries in dentry hashtable
- drgn_tools.dentry.for_each_negative_dentry_in_hashtable(prog)¶
Get only the negative dentries in dentry hashtable
- drgn_tools.dentry.for_each_unused_dentry_in_hashtable(prog)¶
Get only the unused dentries in dentry hashtable
- drgn_tools.dentry.for_each_used_dentry_in_hashtable(prog)¶
Get only the used dentries in dentry hashtable
- drgn_tools.dentry.list_dentries_in_hashtable(prog, limit)¶
List all dentries in hashtable along with their stats
- drgn_tools.dentry.list_negative_dentries_in_hashtable(prog, limit)¶
List negative dentries in hashtable along with their stats
- drgn_tools.dentry.list_unused_dentries_in_hashtable(prog, limit)¶
List unused dentries in hashtable along with their stats
- drgn_tools.dentry.list_used_dentries_in_hashtable(prog, limit)¶
List used dentries in hashtable along with their stats
- drgn_tools.dentry.ls(prog, directory, count=False)¶
Print dentry children, like the ls command :type directory:
str:param directory: directory to print children of :type count:bool:param count: when true, only print counts (not the full contents)
- drgn_tools.dentry.print_dentry_table(dentries, refcount=True)¶
Prints a table of dentries
- drgn_tools.dentry.sb_first_mount_point(sb)¶
Return the first mountpoint of the superblock
A single filesystem instance can be mounted at several locations, so the super_block has a list of instances. When iterating over the dentry cache, we want the full path and don’t care too much about _which_ path we get. We just want to have a valid filesystem path. So return any arbitrary mount point, the first one in the list. If the list is empty (unlikely except during an unmount race) or if we are at the root filesystem, return None.
drgn_tools.dm module¶
Helpers for device mapper devices.
- class drgn_tools.dm.DmFlagsBits(value)¶
Bases:
BitNumberFlagsClass to convert preprocessor definitions to enum
drgn can’t get the value of preprocessor definitions. This is only appliable to the kernel starting 8ae126660fdd which was merged by v4.10
- BLOCK_IO_FOR_SUSPEND = 0¶
- DEFERRED_REMOVE = 6¶
- DELETING = 4¶
- EMULATE_ZONE_APPEND = 9¶
- FREEING = 3¶
- FROZEN = 2¶
- NOFLUSH_SUSPENDING = 5¶
- POST_SUSPENDING = 8¶
- SUSPENDED = 1¶
- SUSPENDED_INTERNALLY = 7¶
- class drgn_tools.dm.DmFlagsBitsOld(value)¶
Bases:
BitNumberFlagsonly appliable to kernel older than v4.10
- BLOCK_IO_FOR_SUSPEND = 0¶
- DEFERRED_REMOVE = 7¶
- DELETING = 4¶
- FREEING = 3¶
- FROZEN = 2¶
- MERGE_IS_OPTIONAL = 6¶
- NOFLUSH_SUSPENDING = 5¶
- SUSPENDED = 1¶
- SUSPENDED_INTERNALLY = 8¶
- drgn_tools.dm.dm_table(dm)¶
return the
struct dm_table *There were two definitions of
struct dm_tablebefore commit 1d3aa6f683b1(“dm: remove dummy definition of ‘struct dm_table’”) which was included in v4.10, specify file for the correct symbol.
- drgn_tools.dm.for_each_dm(prog)¶
- drgn_tools.dm.for_each_dm_hash(prog)¶
- drgn_tools.dm.for_each_dm_rbtree(prog)¶
drgn_tools.ext4_dirlock module¶
Help detect hung by ext4 direcotry inode lock
Exadata customers run into bug 32016306 many times. The symptom of the bug is that some process is reading one ext4 directory with inode lock held, some other processes are trying to access the same directory and get hung by inode lock of that directory. In most of the cases, that directory is some db trace direcotory under /u01 in exadata system since db tracing is enabled, once CRS detects those hung db processes, it will evict the system, sometimes due to some unknown reason, CRS eviction doesn’t happen, instead the system just hung or panic by hung task panic.
The following is an example of the culprit process that causes hung. The other processs will be hung by rwsem write block or mutex depending on kernel verison.
PID: 106479 TASK: ffff9bcb1c8c5f00 CPU: 46 COMMAND: "ohasd.bin"
#0 [ffffb54e916efa10] __schedule at ffffffffb988ff6c
#1 [ffffb54e916efab0] schedule at ffffffffb9890586
#2 [ffffb54e916efac8] io_schedule at ffffffffb9890a06
#3 [ffffb54e916efae0] bit_wait_io at ffffffffb9890fb1
#4 [ffffb54e916efaf8] __wait_on_bit at ffffffffb9890a96
#5 [ffffb54e916efb38] out_of_line_wait_on_bit at ffffffffb9890b51
#6 [ffffb54e916efb90] __wait_on_buffer at ffffffffb92d1432
#7 [ffffb54e916efba8] ext4_bread at ffffffffc019024e [ext4]
#8 [ffffb54e916efbd0] __ext4_read_dirblock at ffffffffc01a75f4 [ext4]
#9 [ffffb54e916efc28] htree_dirblock_to_tree at ffffffffc01a7ecb [ext4]
#10 [ffffb54e916efce0] ext4_htree_fill_tree at ffffffffc01a928b [ext4]
#11 [ffffb54e916efdc0] ext4_readdir at ffffffffc0173bea [ext4]
#12 [ffffb54e916efe80] iterate_dir at ffffffffb92aa1b8
#13 [ffffb54e916efec8] sys_getdents at ffffffffb92aaaf8
#14 [ffffb54e916eff28] do_syscall_64 at ffffffffb9003ca9
#15 [ffffb54e916eff50] entry_SYSCALL_64_after_hwframe at ffffffffb9a001b1
Since ext4_readdir() reads each dir block synchronized, so either of the
following reason can lead to this hung.
1. The size of that ext4 inode direcotry is large
2. The volume hosting the direcotry has high io latency
This helper will dump the following output if hung detected. From the metric it report we can tell what’s causing the hung. The “Lastrun2now” of the first lock waiter can tell how long this hung has been there, any hung time close or over 30s could lead to CRS eviction. If direcotory size is large, please clean up it, if that direcotry doesn’t has much files in it, please recreate the directory, ext4 may not free some direcotry blocks even all dentry in it are already removed. If it’s db trace direcotry, please also disable db tracing to avoid that directory get filled up again.
If directory size is not large, but hung time is long, then it’s probably underlying disk volume is slow. For example in the following output, lock owner stays in D status 259ms, run the helper again with call trace enabled, if it is waiting io done, then that means the underlying disk volume has long I/O latency, you should review iostat from oswatcher for more details.
>>> ext4_dirlock.ext4_dirlock_scan(prog)
Directory : /u01/app/grid/diag/crs/lrlupxa5adm02vm02/crs/trace
Volume : dm-2
dentry : 0xffff9bd0bd8a3a40
inode : 0xffff9bd9d09611e8
Size : 21835776
Blocks : 42728
Inode Lock : Command Pid Status Lastrun2now(ms)
Lock owner : ohasd.bin 106479 D 259
Lock waiter :
[0] : ocssd.bin 120176 D 31128
[1] : gipcd.bin 110130 D 22267
[2] : cssdagent 118193 D 14643
[3] : cssdmonitor 118117 D 14074
[4] : ohasd.bin 106260 D 603
Please note even though I mention a lot of exadata/db/crs, it doesn’t mean this helper only works there, it can help detect ext4 directory hung in other systems also.
This helper will only work with vmcore, because it requires unwinding the stack trace of each process for searching the hung, which is not supported by live system. Also debuginfo is always required for this helper because it will grab variable from the stack frame.
- drgn_tools.ext4_dirlock.ext4_dirlock_scan(prog, stacktrace=False)¶
Scan processes hung by ext4 directory inode lock
The inode lock is
struct mutexin uek4 andstruct rw_semaphorein uek5+
drgn_tools.file module¶
Helpers for file cache.
- drgn_tools.file.file_page_dump(file)¶
Get page pointers associated with a file object
- drgn_tools.file.filecache_dump(prog, top_n, page_limit, fs_types=None, skip_fs_types=None)¶
Dump the filecache stats including the pages, size, filesystem type and filepath. Dump NUMA stats as well if numa set to True.
- Parameters:
top_n (
int) – The largest <top_n> files to be dumpedpage_limit (
int) – Only files with the number of pages greater than <page_limit> are dumpedfs_types (
Optional[List[str]]) – File system types to dump. None to dump allskip_fs_types (
Optional[List[str]]) – File system types to skip. None to skip nothingprog (Program)
- Return type:
- drgn_tools.file.for_each_file_system_page_in_pagecache(fst)¶
Walk through pages of page cache in a file system
- drgn_tools.file.for_each_inode_page_in_pagecache(inode)¶
Walk through pages of page cache in an inode
- drgn_tools.file.for_each_superblock_page_in_pagecache(sb)¶
Walk through pages of page cache in a superblock
drgn_tools.irq module¶
Helpers related to kernel irq management framework under kernel/irq.
- drgn_tools.irq.for_each_in_use_irq_num(prog)¶
Iterate through all inuse irq numbers.
- drgn_tools.irq.for_each_irq_desc(prog)¶
Iterate through all allocated irq descriptors.
- drgn_tools.irq.get_irq_affinity(prog, irq)¶
Get
struct cpumaskfor given irq’s cpu affinity
- drgn_tools.irq.get_irq_affinity_list(prog, irq)¶
Get affinity of a given cpu.
- drgn_tools.irq.irq_has_action(prog, irq)¶
Check if a given irq has handler(s) registered or not.
- drgn_tools.irq.irq_in_use(prog, irq)¶
Check if a given irq number is in use or not. An irq number is considered to be in use by the kernel, if the kernel has allocated a irq descriptor for it. The irq may not yet have any registered irq handlers.
- drgn_tools.irq.irq_name_to_desc(prog, name)¶
Get
struct irq_desc *for irq handler of given name
- drgn_tools.irq.irq_to_desc(prog, irq)¶
Get
struct irq_desc *for given irq number
- drgn_tools.irq.print_all_irqs(prog, ignore_zero=False)¶
Print number, name,
struct irq_desc *and current affinity for all irqs in use.
- drgn_tools.irq.print_irq_affinity(prog, irq)¶
Print cpu affinity of specified irq.
- drgn_tools.irq.print_irqs_affinities(prog)¶
Print cpu affinities for all irqs in use.
- drgn_tools.irq.show_cpu_irq_name_stats(prog, cpu, irq_name)¶
Show irq stats of a cpu for irqs whose handler have specified name or for irqs whose handler names begin with specified string.
- drgn_tools.irq.show_cpu_irq_num_stats(prog, cpu, irq)¶
Show irq stats of a cpu for a given irq number
- drgn_tools.irq.show_cpu_irq_stats(prog, cpu)¶
Show irq stats for specified cpu.
- drgn_tools.irq.show_each_cpu_irq_stats(prog)¶
Show irq stats for each cpu.
- drgn_tools.irq.show_irq_name_stats(prog, irq_name)¶
Show irq stats for irqs whose handler have specified name or for irqs whose handler names begin with specified string.
- drgn_tools.irq.show_irq_num_stats(prog, irq)¶
Show stats for a given irq number
drgn_tools.itertools module¶
- drgn_tools.itertools.count(it)¶
Count the contents of any iterator (consumes it)
drgn_tools.lock module¶
corelens lock should support:
Should find process stuck in mutex or semaphore.
Find the contested mutex
List all waiters on that lock and the time of wait
Target Mutex API list for the lock contentation:
void mutex_lock(struct mutex *lock);
void mutex_lock_nested(struct mutex *lock, unsigned int subclass);
int mutex_lock_interruptible_nested(struct mutex *lock,
unsigned int subclass);
int mutex_lock_interruptible(struct mutex *lock);
int atomic_dec_and_mutex_lock(atomic_t *cnt, struct mutex *lock);
Try variants of mutex are ignored as they will not block. The common function
used in all these api’s is: __mutex_lock() and is sufficant to trap all
block by mutexes.
For semaphores, There is no owners, and waiters generally have the common
functions as: __down_common and __down depending upon releases. So
trapping these two function is sufficient to check the semaphore waiters.
- drgn_tools.lock.scan_lock(prog, stack, time=None, pid=None)¶
Scan tasks for Mutex and Semaphore
- drgn_tools.lock.scan_mutex_lock(prog, stack, time=None, pid=None)¶
Scan for mutex and show details
- drgn_tools.lock.scan_osq_node(prog, verbosity=0)¶
Show CPUs spinning to grab sleeping lock(s).
- drgn_tools.lock.scan_rwsem_lock(prog, stack, time=None, pid=None)¶
Scan for read-write(rw) semphores
- drgn_tools.lock.scan_sem_lock(prog, stack, time=None, pid=None)¶
Scan for semaphores
- drgn_tools.lock.show_rwsem_lock(prog, frame_list, seen_rwsems, stack, time=None, pid=None)¶
Show rw_semaphore details
drgn_tools.locking module¶
Helper for linux kernel locking
- class drgn_tools.locking.RwsemStateCode(value)¶
Bases:
EnumAn enumeration.
- ANONYMOUSLY_OWNED = 'anonymously owned (owned by reader(s) or writer owner has not yet set the owner field)'¶
- OWNER_TYPE_UNKNOWN = 'owned by owner of unknown type'¶
- READER_OWNED = 'reader owned'¶
- UNLOCKED = 'unlocked'¶
- WRITER_OWNED = 'writer owned'¶
- drgn_tools.locking.for_each_mutex_waiter(prog, mutex)¶
List task waiting on the mutex
- drgn_tools.locking.for_each_rwsem_waiter(prog, rwsem)¶
List task waiting on the rw semaphore
- drgn_tools.locking.for_each_rwsem_waiter_entity(rwsem)¶
Find rwsem_waiter(s) for given rwsem
- drgn_tools.locking.for_osq_owner_and_each_spinner(osq)¶
Given an osq, find its owner and all spinners
MCS/OSQ locks are unique in the sense that for these locks both the owners and waiters spin, albeit on different things. The owner spins, usually to optimistically grab a sleeping lock but the waiters spin on some per-cpu entity.
- drgn_tools.locking.get_osq_owner_cpu(osq)¶
Get owner cpu of an osq.
- drgn_tools.locking.get_rwsem_info(rwsem, callstack=0)¶
Get information about given rwsem. This consists of type of owner,
struct task_struct *, pid(s) and type of waiter(s)
- drgn_tools.locking.get_rwsem_owner(rwsem)¶
Find owner of given rwsem
- Parameters:
rwsem (
Object) –struct rw_semaphore *- Return type:
- Returns:
type of owner and
struct task_struct *if owner can be found, NULL otherwise
- drgn_tools.locking.get_rwsem_spinners_info(rwsem, callstack=0)¶
Get a summary of rwsem spinners. The summary consists of
struct task_struct *, pid, CPU, state and spin time
- drgn_tools.locking.get_rwsem_waiter_type(rwsem_waiter)¶
Find type of an rwsem waiter
- drgn_tools.locking.get_rwsem_waiters_info(rwsem, callstack=0)¶
Get a summary of rwsem waiters. The summary consists of
struct task_struct *, pid and type of waiters
- drgn_tools.locking.is_rwsem_reader_owned(rwsem)¶
Check if rwsem is reader owned or not
- drgn_tools.locking.is_rwsem_writer_owned(rwsem)¶
Check if rwsem is writer owned or not
- drgn_tools.locking.mutex_is_locked(lock)¶
Check if a mutex is locked or not
- drgn_tools.locking.mutex_owner(prog, mutex)¶
Get mutex owner
- drgn_tools.locking.osq_is_locked(osq)¶
Check if an osq is locked or not
- drgn_tools.locking.rwsem_has_spinner(rwsem)¶
Check if rwsem has optimistic spinners or not
- drgn_tools.locking.show_lock_waiter(prog, task, index, stacktrace)¶
Show lock waiter
- drgn_tools.locking.tail_osq_node_to_spinners(osq_node)¶
Given an osq_node, find owner and all spinners of same osq
MCS/OSQ locks are unique in the sense that for these locks both the owners and waiters spin, albeit on different things. The owner spins, usually to optimistically grab a sleeping lock but the waiters spin on some per-cpu entity.
drgn_tools.logging module¶
Helpers for logging with some added context info.
- class drgn_tools.logging.PrependedLoggerAdapter(*args, **kwargs)¶
Bases:
LoggerAdapter- process(message, kwargs)¶
Process the logging message and keyword arguments passed in to a logging call to insert contextual information. You can either manipulate the message itself, the keyword args or both. Return the message and kwargs modified (or not) to suit your needs.
Normally, you’ll only need to override this one method in a LoggerAdapter subclass for your specific needs.
- Return type:
Tuple[str,MutableMapping[str,str]]- Parameters:
message (str)
kwargs (MutableMapping[str, Any])
drgn_tools.lsmod module¶
- drgn_tools.lsmod.for_each_module_use(source_list_addr)¶
Provide the list of
struct module_useas an iterable object
- drgn_tools.lsmod.print_module_parameters(prog)¶
Prints each loaded module and its parameter values
drgn_tools.md module¶
Helpers for soft raid
- drgn_tools.md.for_each_md(prog)¶
List all soft raid disk in the system
- drgn_tools.md.for_each_md_pv(prog, mddev)¶
List all physical disks of the md
- drgn_tools.md.md_pending_writes(prog, mddev)¶
Get pending write IO request number from
struct mddev *
- drgn_tools.md.md_pv_badblocks(prog, mddev)¶
- drgn_tools.md.md_pv_name(rdev)¶
Get the name of one physical disk for the md
- drgn_tools.md.md_pv_pending_io(rdev)¶
Get pending IO request from
struct md_rdev *Only raid level 1,4,5,6,10 supports this
- drgn_tools.md.raid1_nr_value(nr)¶
Get nr_pending/waiting value
Those fields are changed by 824e47daddbfc from “int” to atomic *
- drgn_tools.md.show_md(prog)¶
Dump md info in the system
- drgn_tools.md.show_md_pv(prog, mddev)¶
Dump the physical disk specific info
- drgn_tools.md.show_md_pv_badblocks(prog, mddev)¶
- drgn_tools.md.show_raid1_info(prog, mddev)¶
Show raid1 only info
drgn_tools.meminfo module¶
Helpers for dumping memory usage information and statistics
- drgn_tools.meminfo.get_all_meminfo(prog)¶
Collect detailed memory statistics items that match /proc/meminfo.
drgn_tools.mm module¶
Helpers for examining the memory management subsystem.
- class drgn_tools.mm.AddrKind(value)¶
Bases:
EnumAn enumeration of possible virtual memory address kinds.
The kernel can have a huge variety of kinds of memory: percpu data, memory for device I/O, read-only data, and more. Understanding what kind of memory address you’re dealing with can help with a variety of debugging tasks.
- BSS = 'bss'¶
An address from the vmlinux / core kernel’s BSS data segment.
These are static data structures which aren’t initialized to a non-zero value. As such, they are placed into a separate section and their memory is zero’d at initialization.
- DATA = 'data'¶
An address from the vmlinux / core kernel’s RW data segment.
These are static data structures which are initialized to some non-zero value, and not declared const.
- DIRECT_MAP = 'direct map'¶
An address from the direct mapping of virtual addresses to physical.
A lot of kernel addresses (such as memory allocated via the slab allocator) fall under this category.
- INITTEXT = 'inittext'¶
An address from the vmlinux / core kernel’s __init text segment.
These are functions decorated with the “__init” macro. Their memory is freed after initializing the kernel. However, their symbols remain, and the virtual addresses used to refer to them don’t get reused. Thus, you could see these addresses in a booted system, though it would be quite unlikely.
- MODULE = 'module'¶
An address from the kernel module range.
Modules have a reserved range of memory into which their code and data pages are mapped.
- PERCPU = 'percpu'¶
An address from the vmlinux / core kernel which refers to percpu data.
Please note that as of now, we can only detect static percpu variables from the core kernel. Modules and dynamically allocated percpu variables are harder to detect, and are in the works.
- RODATA = 'rodata'¶
An address from the vmlinux / core kernel’s RO data segment.
These are static data structures which are declared const.
- TEXT = 'text'¶
An address from the vmlinux / core kernel’s text segment.
Most code falls under this category. However, some __init code is discarded after boot, and is part of INITTEXT.
- UNKNOWN = 'unknown'¶
An address which we could not categorize.
- USER = 'user'¶
Userspace memory.
- VMALLOC = 'vmalloc'¶
An address from vmalloc.
The vmalloc subsystem serves a variety of purposes (vmalloc allocator, vmap, ioremap). It allows the kernel to allocate large ranges of virtually contiguous, but physically discontiguous memory, as well as map physical I/O memory addresses into the kernel range. It also allows the kernel to allocate memory with “guard pages” that protect against over/under flow.
These addresses may be returned by vmalloc(), or setup by vmap() or ioremap(). A common example of this category of memory is kernel stacks, and another quite common example is memory-mapped I/O ranges.
- VMEMMAP = 'vmemmap'¶
An address from virtual memory map section.
The memory map is an array of “struct page” that describes all physical memory. When CONFIG_SPARSEMEM_VMEMMAP is enabled, the memory mapping is in a separate virtual address range, which allows “holes” in the memory mapping to be unmapped, saving a significant portion of memory when there are lots of holes in the physical address space.
What this means is that, when CONFIG_SPARSEMEM_VMEMMAP is enabled, all
struct page *addresses are within the vmemmap region. If you see a struct page, you know it should be in the vmemmap, and if you see a vmemmap address, you know it must be a struct page. On UEK, VMEMMAP is always enabled, however in general, it is configuration specific.
- classmethod categorize(prog, addr)¶
Given a memory address, tell what kind of memory it refers to.
Please note that right now, this is x86_64-specific and restricted to the configurations used by UEK.
- Parameters:
prog (
Program) – program we’re debuggingaddr (
IntegerLike) – address to categorize
- Return type:
- drgn_tools.mm.check_freelists_at_cpu(prog, cpu)¶
- drgn_tools.mm.totalram_pages(prog)¶
The value of totalram_pages is used to show MemTotal in /proc/meminfo.
It was defined as
unsigned long totalram_pages, and then changed toatomic_long_t _totalram_pagesin commit ca79b0c211af (“mm: convert totalram_pages and totalhigh_pages variables to atomic”)Return this value as a drgn unsigned long.
drgn_tools.module module¶
- class drgn_tools.module.KernelModule(obj)¶
Bases:
objectProvides a more “object-oriented” interface to the module helpers.
This class wraps an object of type
struct module *, and adds methods which can be used to interact with it. For example:>>> km = KernelModule.find(prog, "nf_nat") >>> km KernelModule(nf_nat) >>> km.address_region() ModuleLayout(base=Object(prog, 'void *', address=0xffffffffc0878580), total_size=40960, text_size=16384, ro_size=20480, ro_after_init_size=24576) >>> km.init_region() >>> km.percpu_region() (Object(prog, 'void *', address=0xffffffffc0878688), 0) >>> hex(km.address_region().base + 123) '0xffffffffc087207b' >>> KernelModule.lookup_address(prog, 0xffffffffc087207b) KernelModule(nf_nat)
- Parameters:
obj (Object)
- address_region()¶
Return the core region of the module. See
module_address_region().- Return type:
- Returns:
Layout of the “core” (non-init) region
- classmethod all(prog)¶
Get an iterator of KernelModule helpers for each loaded module
- Parameters:
prog (
Program) – Program being debugged- Return type:
- Returns:
Iterable of kernel module helpers
- exports()¶
Return a list of exported (with or without GPL) symbol name & address
- classmethod find(prog, name)¶
Return a KernelModule helper for
nameif present- Parameters:
- Return type:
- Returns:
Iterable of kernel module helpers
- find_debuginfo()¶
Search for a debuginfo file matching this module.
- get_symbol(addr)¶
Lookup a symbol by address.
This function is intended to be used when the DWARF debuginfo for a module is not available, but when the debuginfo for vmlinux is available. If the module’s debuginfo is loaded, this should not be used; instead, just rely on drgn’s built-in symbol lookup behavior.
Note that this function relies on the list of symbols returned by
unified_symbols(), the most complete list available.- Parameters:
addr (
IntegerLike) – Address to lookup- Return type:
- Returns:
A symbol name, if found
- have_debuginfo()¶
Determine whether debuginfo is loaded for this module.
- Return type:
- Returns:
True if debuginfo exists for this module
- init_region()¶
Return the init memory region of the module.
- Return type:
- is_oot()¶
Return true if the module is out of tree.
This is determined using the module.taints field: modules which are tainted OOT_MODULE, PROPRIETARY_MODULE, or UNSIGNED_MODULE, could not have come from our debuginfo RPMs, and so they are considered out of tree. Unfortunately, OOT_MODULE itself is not enough: for some reason, it seems that not all out-of-tree modules get that taint applied.
- Return type:
- load_debuginfo(extract=False, need_dwarf=False)¶
If the module’s debuginfo is not loaded, find and load it
- classmethod lookup_address(prog_or_addr, addr=None)¶
Lookup the module containing this address and return the KernelModule helper for it, if found. See
address_to_module().- Return type:
- Parameters:
addr (IntegerLike | None)
- params()¶
Return a dictionary of the parameters of the module, see
module_params()
- percpu_region()¶
Return the percpu memory region of the module, see
module_percpu_region().
- symbols()¶
Return a list of ELF symbol objects via kallsyms, and their names.
This function is intended to be used when the DWARF debuginfo for a module is not available, but when the debuginfo for vmlinux is available. If the module’s debuginfo is loaded, this should not be used; instead, just rely on drgn’s built-in symbol lookup behavior.
Note
Please note that this only returns the symbols from the module’s kallsyms table, and does not include any symbols from the exported symbol list. This is usually ok, but sometimes the exports are useful. The
`unified_symbols()interface includes the rest, but they are not represented as ELF symbols, instead just as names, addresses, and sizes.
- unified_symbols()¶
Return a sorted list of all symbols we can find for a module.
This function is intended to be used when the DWARF debuginfo for a module is not available, but when the debuginfo for vmlinux is available. If the module’s debuginfo is loaded, this should not be used; instead, just rely on drgn’s built-in symbol lookup behavior.
- class drgn_tools.module.ModuleLayout(base, total_size, text_size, ro_size, ro_after_init_size)¶
Bases:
NamedTupleRepresents a module’s layout in memory.
Module memory layout is organized into three sections. First is the text section, which is read-only (RO). Next is the RO data section, which is usually protected with no-execute (NX) permissions. Next is additional data which becomes RO after init, and finally is the RW data. The below diagram from the kernel source code demonstrates this layout (note that for clarity, we refer to
sizeastotal_size).General layout of module is: [text] [read-only-data] [ro-after-init] [writable data] text_size -----^ ^ ^ ^ ro_size ------------------------| | | ro_after_init_size -----------------------------| | size -----------------------------------------------------------|
- contains(address)¶
- Return type:
- Parameters:
address (IntegerLike)
- class drgn_tools.module.ParamInfo(name, kernel_param, type_name, value)¶
Bases:
NamedTupleContains information about a kernel module parameter
- drgn_tools.module.address_to_module(prog_or_addr, addr=None)¶
Try to find the module corresponding to a memory address.
Search for the given address in the list of loaded kernel modules. If it is within the address range corresponding to a kernel module, return that module. This function searches the module’s core (normal memory) region, the module’s init region (if present) and the module’s percpu region (if present). Note that it’s impossible to detect memory allocated by a particular kernel module: this function only deals with static data.
This helper performs a linear search of the list of modules, which could grow quite large. As a result, the performance may suffer on repeated lookups.
- drgn_tools.module.ensure_debuginfo(prog, modules)¶
Ensure that the modules listed are loaded in the kernel and have debuginfo available. If the modules are not present in the kernel or the debuginfo cannot be loaded, return an error message.
- drgn_tools.module.find_module(prog, name)¶
Return the module with the given name :type name:
Union[str,bytes] :param name: Module name :rtype:Optional[Object] :returns: if found,struct module *
- drgn_tools.module.for_each_module(prog)¶
Get all loaded kernel module objects in kernel
- drgn_tools.module.load_module_debuginfo(prog, modules=None, extract=False, strict=False, quiet=False)¶
Load all available debuginfo for all modules, with optional extraction
This function uses the normal search paths to find debuginfo (See find_debuginfo() for details). For modules whose debuginfo is not found, if extract is True, it attempts to extract the debuginfo from the vmlinux repo. When strict is true, it raises an error if not all module debuginfo could be found and loaded.
This function ignores all modules which are tainted as OOT_MODULE, PROPRIETARY_MODULE, or UNSIGNED_MODULE. These modules will not be found in the debuginfo RPMs so there’s no point in looking.
- Parameters:
modules (
Union[None,List[str],List[KernelModule]]) – list of modules to load. The list may be provided as a list of strings, or a list ofKernelModuleobjects – but they cannot be mixed! If not provided, we fall back to loading all modules.extract (
bool) – when true, attempt to extract debuginfostrict (
bool) – when true, raise an exception if we couldn’t load all modulesquiet (
bool) – when true, silence output regarding missing or OOT modulesprog (Program)
- Return type:
- drgn_tools.module.module_address_region(mod)¶
Lookup the core memory region of a module.
Given a
struct module *, return the address and length of its code and data. This region ignores the “__init” data of the module; see- Func:
module_init_region()to find that.- Parameters:
mod (
Object) – Object of typestruct module *- Return type:
- Returns:
A tuple representing the address and size of the memory, along with the size of various protection zones within.
- drgn_tools.module.module_build_id(mod)¶
Return the build ID (as a hex string) for this module.
- drgn_tools.module.module_exports(module)¶
Return a list of names and addresses from the exported symbols
Kernel modules may have various fields like
syms,gpl_syms, etc. These fields correspond to exported symbols, that is, the symbols for which there was anEXPORT_SYMBOL()macro declared. The exported symbols are the only ones which may be used by other modules.This function returns names and addresses for each exported symbol. It includes all symbols available, regardless of license. The symbols are returned in sorted order by increasing address. Note that size information is not provided by the kernel, and so it is not returned here.
- drgn_tools.module.module_init_region(mod)¶
Lookup the init memory region of a module.
Given a
struct module *, return the address and length of the__initmemory regions. This memory is typically freed after the module is loaded, so under most circumstances, this will return None.- Parameters:
mod (
Object) – Object of typestruct module *- Return type:
- Returns:
A tuple representing the layout of the init memory
- drgn_tools.module.module_params(mod)¶
Return a dictionary of kernel module parameters
- drgn_tools.module.module_percpu_region(mod)¶
Lookup the percpu memory region of a module.
Given a
struct module *, return the address and the length of the percpu memory region. Modules may have a NULL percpu region, in which case(void *)NULLis returned. Rarely, on kernels withoutCONFIG_SMP, there is no percpu region at all, and this function returnsNone.
- drgn_tools.module.module_symbols(module)¶
Return a list of ELF symbols for a module via kallsyms.
Kernel modules may have a
module_kallsymsfield which contains ELF symbol objects describing all kallsyms symbols. This function accesses this symbol information.Returns a list of objects of type
Elf_Sym. This object is a typedef to an architecture specific type (either 64 or 32 bits), either of which contain the same fields – see elf(5) for their definition. Since thest_namefield is merely an index and can’t be interpreted without the string table, this helper returns a tuple of the decoded name, and the symbol object.
- drgn_tools.module.module_unified_symbols(module)¶
Unify all sources of module symbols and return basics: name, value, length.
There are multiple possible sources of module symbol information: kallsyms, exports, etc. This function unifies them all and attempts to give just basic info. Note that in some cases, we have to infer the symbol length. This helper does that as best it can.
drgn_tools.mounts module¶
- drgn_tools.mounts.get_mountinfo(prog)¶
Get all the mount points from the vmcore
drgn_tools.nfs_tools module¶
Helpers for NFS (client-side only) and SUNRPC subsystem.
- drgn_tools.nfs_tools.nfsshow(prog, max_tasks=10, filename=None)¶
Display summary information on NFS client side. The information includes major structures such as nfs_client, nfs_server, rpc_clnt, rpc_xprt and rpc_task.
- drgn_tools.nfs_tools.show_nfs_clients(prog)¶
Display summary information of the NFS layer on the NFS client side. The information includes major data structures such as nfs_client, nfs_server and corresponding RPC transport.
- Parameters:
prog – drgn pogram
- Return type:
- Returns:
None
- drgn_tools.nfs_tools.show_rpc_clnts(prog)¶
Display info of all rpc_clnt in the system.
- drgn_tools.nfs_tools.show_rpc_tasks(prog, max_tasks=10)¶
Display the content of all struct rpc_task.
- Parameters:
- Return type:
- Returns:
None
- Example:
show_rpc_tasks(prog, 20) Print 1st 20 rpc_tasks show_rpc_tasks(prog -1) Print all rpc_tasks
drgn_tools.numastat module¶
Helpers for dumping memory usage information for each NUMA node
- drgn_tools.numastat.get_per_node_meminfo(prog, node)¶
Collect detailed memory statistics for a NUMA node. Results are expected to be similar to outputs produced by node_read_meminfo(…) in drivers/base/node.c.
drgn_tools.nvme module¶
- drgn_tools.nvme.for_each_nvme_disk(prog)¶
Return each NVMe device
- drgn_tools.nvme.show_ctrl_info(prog)¶
Display information about each NVMe controller
- drgn_tools.nvme.show_firmware_info(prog)¶
Display NVMe firmware information
- drgn_tools.nvme.show_msi_mask(prog)¶
Display if the MSI has been masked for each NVMe queue
- drgn_tools.nvme.show_ns_info(prog)¶
Display namepsace information for each NVMe device
- drgn_tools.nvme.show_queue_info(prog)¶
Display details of various allocated NVMe queues Value of -1 indicates parameter does not exist in that version
drgn_tools.partition module¶
Helper to print partition information
- class drgn_tools.partition.PartInfo(major, minor, name, start_sect, nr_sects, ro, obj)¶
Bases:
NamedTuplePartition info, from either
struct block_deviceorstruct hd_struct- Parameters:
- drgn_tools.partition.get_partinfo_from_blkdev_struct(part)¶
Collect partition information from
struct block_deviceReturns a list with partition information for the given partition.
- drgn_tools.partition.get_partinfo_from_hd_struct(part)¶
Collects partition information from
struct hd_structReturns a list with partition information for the given partition.
drgn_tools.printk module¶
Additional helpers for printk utilities
- drgn_tools.printk.dmesg(prog, pager=None)¶
Display the kernel log in a pager
The pager is selected in the following manner. First, if the pager argument is provided, that is used. Second, if the
PAGERenvironment variable is defined, that is used. Finally, the fallback value oflessis used.
drgn_tools.rds module¶
Helpers for examining rds related info.
- class drgn_tools.rds.RdsIbConnInfo(i_cm_id_val, rdma_cm_state_val, ib_cm_id_val, ib_cm_state_val)¶
Bases:
NamedTuple
- drgn_tools.rds.be16_to_host(prog, value)¶
Convert 16 byte value from big endian to host order
- drgn_tools.rds.be32_to_host(prog, value)¶
Convert 32 byte value from big endian to host order
- drgn_tools.rds.be64_to_host(prog, value)¶
Convert 64 byte value from big endian to host order
- drgn_tools.rds.ensure_mlx_core_ib_debuginfo(prog, dev_name)¶
Ensure that the correct mlx[5/4]_core debuginfo is present.
- drgn_tools.rds.fields_to_list(fields)¶
Create a list of fields from a given string
- drgn_tools.rds.for_each_rds_conn(prog, laddr=None, faddr=None, tos=None, states=None)¶
Provide the list of
struct rds_connectionfrom the conn hash list as an iterable object
- drgn_tools.rds.for_each_rds_ib_conn(dev, laddr=None, faddr=None, tos=None, states=None)¶
Provide the list of
struct rds_ib_connectionas an iterable object
- drgn_tools.rds.for_each_rds_ib_device(prog)¶
Provide the list of
struct rds_ib_deviceas an iterable object
- drgn_tools.rds.for_each_rds_ib_ipaddr(dev)¶
Provide the list of
struct rds_ib_ipaddras an iterable object
- drgn_tools.rds.for_each_rds_sock(prog)¶
Provide the list of
struct rds_sockas an iterable object
- drgn_tools.rds.get_connection_uptime(conn)¶
Return the time the connection was in the “UP” state..
- drgn_tools.rds.ib_cm_state(cm_id)¶
Get the IB CM state for a given RDS connection.
- drgn_tools.rds.print_mr_list_head_info(prog, list_head, pool_name, list_name)¶
Print the specific MR list info for busy_list or clean_list
- drgn_tools.rds.rdma_cm_state(rdma_id_priv)¶
Get the RDMA CM state for a given RDS connection.
- drgn_tools.rds.rds_conn_info(prog, laddr=None, faddr=None, tos=None, state=None, ret=False, outfile=None, report=False)¶
Display all RDS connections
- Parameters:
prog (
Program) – drgn programladdr (
Optional[str]) – comma separated string list of LOCAL-IP. Ex: ‘192.168.X.X, 10.211.X.X, …’faddr (
Optional[str]) – comma separated string list of REMOTE-IP. Ex: ‘192.168.X.X, 10.211.X.X, …’tos (
Optional[str]) – comma separated string list of TOS. Ex: ‘0, 3, …’state (
Optional[str]) – comma separated string list of conn states. Ex ‘RDS_CONN_UP, CONNECTING, …’ret (
bool) – If true the function returns thestruct rds_ib_connectionlist and None if the arg is falsereport (
bool) – Open the file in append mode. Used to generate a report of all the functions in the rds module.
- Return type:
- Returns:
A List of
struct rds_ib_connectionthat match the filters provided or None
- drgn_tools.rds.rds_conn_path_state(conn)¶
Get the connection state for a given RDS connection.
- drgn_tools.rds.rds_dev_info(prog, ret=False, outfile=None, report=False)¶
Print the IB device info
- Parameters:
- Return type:
- Returns:
A List of
struct rds_ib_deviceor None
- drgn_tools.rds.rds_get_ib_conn_info(ic)¶
Get the
struct rds_ib_connectionspecific info- Parameters:
ic (
Object) –struct rds_ib_connectionObject- Return type:
- Returns:
A namedtuple of type RdsIbConnInfo
- drgn_tools.rds.rds_get_mr_list_info(prog_or_obj, dev_ptr=None)¶
Print the MR list info
- drgn_tools.rds.rds_get_stats(prog, rds_stat_list, fields=None)¶
Get the rds stats and counters from the stat list provided.
- drgn_tools.rds.rds_ib_conn_ring_info(prog_or_obj, ic_ptr=None)¶
Display the ring info for a particular RDS connection.
- drgn_tools.rds.rds_inet_ntoa(addr_obj)¶
Convert addr from sin6_addr to string
- drgn_tools.rds.rds_info_verbose(prog, laddr=None, faddr=None, tos=None, fields=None, ret=False, outfile=None, report=False)¶
Print the rds conn stats similar to rds-info -Iv
- Parameters:
prog (
Program) – drgn programladdr (
Optional[str]) – comma separated string list of LOCAL-IP. Ex: ‘192.168.X.X, 10.211.X.X, …’faddr (
Optional[str]) – comma separated string list of REMOTE-IP. Ex: ‘192.168.X.X, 10.211.X.X, …’tos (
Optional[str]) – comma separated string list of TOS. Ex: ‘0, 3, …’fields (
Optional[str]) – List of comma separated fields to display. It also supports substring matching for the fields provided. Ex: ‘Recv_alloc_ctr, Cache Allocs, Tx, …’ret (
bool) – If true the function returns thestruct rds_ib_connectionlist and None if the arg is falsereport (
bool) – Open the file in append mode. Used to generate a report of all the functions in the rds module.
- Return type:
- Returns:
A List of
struct rds_ib_connectionthat match the filters provided or None
- drgn_tools.rds.rds_ip_state_list(dev)¶
The number of connections and its state associated to a rds_ib_device
- drgn_tools.rds.rds_print_msg_queue(prog, queue='All', laddr=None, raddr=None, tos=None, lport=None, rport=None, ret=False, outfile=None, report=False)¶
Print the rds msg queue similar rds-info -srt
- Parameters:
prog (
Program) – drgn programladdr (
Optional[str]) – comma separated string list of LOCAL-IP. Ex: ‘192.168.X.X, 10.211.X.X, …’raddr (
Optional[str]) – comma separated string list of REMOTE-IP. Ex: ‘192.168.X.X, 10.211.X.X, …’tos (
Optional[str]) – comma separated string list of TOS. Ex: ‘0, 3, …’lport (
Optional[str]) – comma separated string list of lport. Ex: 2259, 36554, …’rport (
Optional[str]) – comma separated string list of rport. Ex: 2259, 36554, …’report (
bool) – Open the file in append mode. Used to generate a report of all the functions in the rds module.queue (str)
ret (bool | None)
- Prarm queue:
The msg queue to be displaed. Ex: ‘send’, ‘recv’ , ‘retrans’, ‘All’. Default: ‘All’
- Return type:
- Returns:
None
- drgn_tools.rds.rds_print_recv_msg_queue(prog, laddr=None, raddr=None, tos=None, lport=None, rport=None, ret=False, outfile=None, report=False)¶
Print the rds recv msg queue similar rds-info -r
- Parameters:
prog (
Program) – drgn programladdr (
Optional[str]) – comma separated string list of LOCAL-IP. Ex: ‘192.168.X.X, 10.211.X.X, …’raddr (
Optional[str]) – comma separated string list of REMOTE-IP. Ex: ‘192.168.X.X, 10.211.X.X, …’tos (
Optional[str]) – comma separated string list of TOS. Ex: ‘0, 3, …’lport (
Optional[str]) – comma separated string list of lport. Ex: 2259, 36554, …’rport (
Optional[str]) – comma separated string list of rport. Ex: 2259, 36554, …’report (
bool) – Open the file in append mode. Used to generate a report of all the functions in the rds module.ret (bool | None)
- Return type:
- Returns:
None
- drgn_tools.rds.rds_print_send_retrans_msg_queue(prog, queue, laddr=None, raddr=None, tos=None, lport=None, rport=None, ret=False, outfile=None, report=False)¶
Print the rds send or retransmit msg queue similar rds-info -st
- Parameters:
prog (
Program) – drgn programladdr (
Optional[str]) – comma separated string list of LOCAL-IP. Ex: ‘192.168.X.X, 10.211.X.X, …’raddr (
Optional[str]) – comma separated string list of REMOTE-IP. Ex: ‘192.168.X.X, 10.211.X.X, …’tos (
Optional[str]) – comma separated string list of TOS. Ex: ‘0, 3, …’lport (
Optional[str]) – comma separated string list of lport. Ex: 2259, 36554, …’rport (
Optional[str]) – comma separated string list of rport. Ex: 2259, 36554, …’report (
bool) – Open the file in append mode. Used to generate a report of all the functions in the rds module.queue (str)
ret (bool | None)
- Prarm queue:
The msg queue to be displaed. Ex: ‘send’, ‘recv’
- Return type:
- Returns:
None
- drgn_tools.rds.rds_sk_rcvbuf(rs)¶
Extract the rcvbuf from the socket
- drgn_tools.rds.rds_sk_sndbuf(rs)¶
Extract the sndbuf from the socket
- drgn_tools.rds.rds_sock_info(prog, ret=False, outfile=None, report=False)¶
Print the rds socket info similar to rds-tools -k
- Parameters:
- Return type:
- Returns:
A List of
struct rds_sockor None
- drgn_tools.rds.rds_stats(prog, fields=None, outfile=None, report=False)¶
Print the RDS stats and counters.
- Parameters:
prog (
Program) – drgn programfields (
Optional[str]) – List of comma separated fields to print. It also supports substring matching for the fields provided. Ex: ‘conn_reset, ib_tasklet_call, send, …’report (
bool) – Open the file in append mode. Used to generate a report of all the functions in the rds module.
- Return type:
- Returns:
None
- drgn_tools.rds.report(prog, outfile=None)¶
Generate a report of RDS related data. This functions runs all the functions in the module and saves the results to the output file provided.
drgn_tools.runq module¶
drgn_tools.scsi module¶
Helper to print scsi hosts
- drgn_tools.scsi.for_each_scsi_host(prog)¶
Iterates through all scsi hosts and returns a iterator. :rtype:
Iterator[Object] :returns: a iterator ofstruct Scsi_Host *
- drgn_tools.scsi.host_module_name(shost)¶
Fetch the module name associated with the scsi host. returns: the module name string.
drgn_tools.slabinfo module¶
Helper to view slabinfo data
- class drgn_tools.slabinfo.SlabCacheInfo(cache, objsize, allocated, total, nr_slabs, ssize, name)¶
Bases:
NamedTupleDescribes summary information about a slab cache
- Parameters:
- drgn_tools.slabinfo.collect_node_info(cache)¶
Parse through each node to collect per-node slab data
- drgn_tools.slabinfo.get_kmem_cache_slub_info(cache)¶
Get slab information for given slab cache
- Parameters:
cache (
Object) –struct kmem_cachedrgn object- Return type:
- Returns:
a
SlabCacheInfowith statistics about the cache
- drgn_tools.slabinfo.kmem_cache_percpu(cache)¶
Count the number of cpu_slab pages for all nodes.
- drgn_tools.slabinfo.kmem_cache_pernode(cache, nodeid)¶
Get number of slabs, objects and partial object per node traverse though the partial list of node and count the number of partial objects
- drgn_tools.slabinfo.kmem_cache_slub_info(cache)¶
For given kmem_cache object, parse through each cpu and get number of total slabs and free objects
- drgn_tools.slabinfo.print_slab_info(prog)¶
Helper to print slab information
- drgn_tools.slabinfo.slub_get_cpu_freelist_cnt(cpu_freelist, slub_helper)¶
Get number of elements in percpu freelist
drgn_tools.smp module¶
Decode CSD/CFD info
- drgn_tools.smp.dump_cfd_at_all_cpus(prog)¶
Dump per-cpu
cfd_dataThere is a per-cpu
struct call_function_dataobject (cfd_data) for each cpu and cpus can use the underlying per-cpu csd to send csd requests.This helper dumps CSDs within
cfd_dataof each cpu.
- drgn_tools.smp.dump_pending_csd_for_all_cpus(prog)¶
Dump
call_single_queuelist of all CPUs
- drgn_tools.smp.dump_smp_ipi_objects(prog)¶
Dump objects of smp ipi subsystem :rtype:
Noneper cpu
call_single_queuelist of each CPUper cpu
cur_csdof each CPUper cpu
cur_csd_funcof each CPU
- Parameters:
prog (Program)
- Return type:
None
- drgn_tools.smp.dump_smp_ipi_state(prog)¶
Dump state of SMP IPI subsystem :rtype:
NoneWhich CPUs have pending smp ipis, what they are doing and whether they have interrupts disabled.
Which CPUs are waiting in csd lock, at what place and for how long.
Is there any pending csd for an offline CPU?
- Parameters:
prog (Program)
- Return type:
None
- drgn_tools.smp.for_each_call_single_queue(prog)¶
Iterate over list of per-cpu
call_single_queueobjects
- drgn_tools.smp.for_each_cfd_data(prog)¶
Iterate over per-cpu
cfd_data
- drgn_tools.smp.for_each_cur_csd(prog)¶
Iterate over all CSDs pointed to by per-cpu
cur_csd
- drgn_tools.smp.for_each_cur_csd_func(prog)¶
Iterate over all functions pointed to by per-cpu
cur_csd_func
- drgn_tools.smp.is_call_single_queue_empty(prog, cpu)¶
Return whether
call_single_queueof a CPU is empty or not.
- drgn_tools.smp.is_cur_csd_pending(prog, cpu)¶
Return whether cur_csd.func of a CPU is still under execution.
For kernels that have
cur_csd_funcandcur_csd, before starting execution of csd function at destination CPU, both of these per-cpu variables are updated and just after executing csd function cur_csd is made NULL.So for a CPU if
cur_csd_funcis same ascur_csd.func, that would mean that the CPU is in the middle of executing csd function or it has just finished the execution of csd function but has noy yet updated cur_csd to NULL.If
cur_csd.funcandcur_csd_funcare not same,cur_csd_funcgives last csd function that this CPU executed.
drgn_tools.sys module¶
Helper to view sysinfo data
- drgn_tools.sys.get_mem(prog)¶
Returns the total memory of the system, as a human-readable string
- drgn_tools.sys.get_sysinfo(prog)¶
Helper to get sysinfo of system
- drgn_tools.sys.loadavg_str(prog)¶
Return system load averaged over 1, 5 and 15 minutes as a string.
- drgn_tools.sys.print_sysinfo(prog)¶
Prints the sysinfo of the system
drgn_tools.sysctl module¶
Helpers for sysctl.
- drgn_tools.sysctl.get_data_entry(prog, ct)¶
Extract the data of a ct a control table entry value given procname.
- drgn_tools.sysctl.get_sysctl_table(prog)¶
Get the sysctl table.
- Return type:
- Returns:
sysctl table as a read-only dictionary, where keys are procnames and values are their data
- Parameters:
prog (Program)
- drgn_tools.sysctl.look_up_sysctl_entry(prog, procname)¶
look up sysctl entry of given procname
drgn_tools.table module¶
- class drgn_tools.table.FixedTable(header, widths=None, **kwargs)¶
Bases:
TableCreated an aligned, formatted table with fixed column widths
This is a variant of the
Tableclass, and it is designed to be nearly a drop-in replacement. UnlikeTable, the column widths are “fixed”; that is they are set up during initialization. This means that each row can be printed immediately, which is great for letting users know that progress is being made. The trade-off is that if any row contains a column value which is too wide, the row will become unaligned. This is makes output less visually appealing, but frequently it can be avoided by designing the table carefully.The column widths can be specified in one of two ways. First, you may specify a list of widths to this constructor. In that case, the table header is immediately printed, and the given widths are used exactly as provided. Alternatively, you can omit the
widthsargument. When the first row is printed, column widths are calculated using the width of each value in the row (and also the header width) to ensure everything fits. This is an especially useful behavior if your table data is already fixed-width anyway: you don’t need to hand-calculate widths and risk them going out of date.The other behaviors of this class are identical to
Table. Please note that even though this table tends to print rows immediately, it is still good practice to use callwrite()once finished adding rows. This makes it trivial to swap out the implementation forTableif desired.- Parameters:
- add_row(fields)¶
Add a row to the table (it is immediately printed)
- class drgn_tools.table.Table(header, outfile=None, report=False)¶
Bases:
objectCreate an aligned, formatted table
This helper makes it simple to create a text table which is aligned to your requirements, and whose values are formatted with whatever string formatter you’d like. The table will be written to stdout by default, but can be written to a custom output file if you prefer.
To create the table, you need to specify all the columns. Each column is specified by a string which contains the column name, and optionally a colon (“:”) followed by a format string. You can prefix the format string with a “<” or “>” to control the justification of the column (it is stripped from the format string). By default, columns are left justified and formatted using
format(value, '')which is typically the same asstr(). Here are some example column specifiers:“TIME:>.3f” - a column named “TIME”, right justified
“NAME” - a column named “NAME”, left justified, formatted by str()
“PTR:016x” - a 16-digit hexadecimal value, 0-filled
Please note that this function will store all rows until
write()is called. This way, it can determine the expected column widths for all rows, and align them accordingly. If you’d like your table rows to be printed as they are created (e.g. if producing the output takes a long time, and you’d like the user to see output as it becomes available), then you could useFixedTable.- Parameters:
- add_row(fields)¶
Add a row to the table (values expressed as a list)
- row(*fields)¶
Add a row to the table (values expressed as positional args)
- drgn_tools.table.print_dictionary(dictionary, outfile=None, report=False)¶
Align and print the data
- drgn_tools.table.print_table(fields, outfile=None, report=False)¶
Print a given nested list as table, given that the first list is the column headers.
drgn_tools.taint module¶
Contains definitions for kernel taint values
- class drgn_tools.taint.Taint(value)¶
Bases:
IntEnumKernel and module taint flags
These flags are not recorded in any enum type, only preprocessor definitions, since they need to be used in assembly listings in the kernel. Record them here. They can be found at
include/linux/panic.hor for older kernels,include/linux/kernel.h.- AUX = 16¶
- BAD_PAGE = 5¶
- CPU_OUT_OF_SPEC = 2¶
- CRAP = 10¶
- DIE = 7¶
- FIRMWARE_WORKAROUND = 11¶
- FLAGS_COUNT = 18¶
- FORCED_MODULE = 1¶
- FORCED_RMMOD = 3¶
- LIVEPATCH = 15¶
- MACHINE_CHECK = 4¶
- OOT_MODULE = 12¶
- OVERRIDDEN_ACPI_TABLE = 8¶
- PROPRIETARY_MODULE = 0¶
- RANDSTRUCT = 17¶
- SOFTLOCKUP = 14¶
- UNSIGNED_MODULE = 13¶
- USER = 6¶
- WARN = 9¶
drgn_tools.task module¶
Helpers related to accessing fields of task_struct in a compatible way.
The struct task_struct is a constantly evolving struct, and it can be
impacted by many configuration options. This module aims to help paper over
those many differences so your code will run on a variety of kernel versions
and configurations.
- class drgn_tools.task.TaskRss(rss_file, rss_anon, rss_shmem)¶
Bases:
NamedTupleRepresent’s a task’s resident set size in pages. See task_rss().
- drgn_tools.task.check_arg_type(arg)¶
Check the filter type of the argument
- drgn_tools.task.count_tasks_in_state(prog, state)¶
Count all tasks in a given state.
- drgn_tools.task.for_each_task_in_group(task, include_self=False)¶
Iterate over all tasks in the thread group
Or, in the more common userspace terms, iterate over all threads of a process.
- drgn_tools.task.for_each_task_in_state(prog, state)¶
Iterate over all tasks in a given state.
- drgn_tools.task.format_nanosecond_duration(nanosecs)¶
- drgn_tools.task.get_command(task)¶
- drgn_tools.task.get_current_run_time(prog, cpu)¶
Get running duration of the current task on some cpu
- drgn_tools.task.get_ppid(task)¶
- drgn_tools.task.get_runq_lag(prog, cpunum)¶
Get time lag of given CPU’s runq clock, relative to most recent runq timestamp
A CPU’s runq clock may be lagging due to reasons such as dyntick idle, missing sched tick updates etc. This helper can be used in cases where we need to make sure that a cpu’s runq clock is uptodate.
- drgn_tools.task.get_task_arrival_time(task)¶
Get a task’s arrival time on cpu
A task’s arrival time is only updated when the task is put ON a cpu via context_switch.
- drgn_tools.task.get_task_rss(task, cache)¶
Return the task’s resident set size (RSS) in pages
The task’s RSS is the number of pages which are currently resident in memory. The RSS values can be broken down into anonymous pages (not bound to any file), file pages (those associated with memory mapped files), and shared memory pages (those which aren’t associated with on-disk files, but belonging to shared memory mappings). This function returns a tuple containing each category, but the common behavior is to use the “total” value which sums them up.
- Parameters:
task (
Object) –struct task_struct *for which to compute RSScache (
Optional[Dict[int,TaskRss]]) – if provided, we can use this to cache the mapping of “mm_struct” to RSS. This helps avoid re-computing the RSS value for processes with many threads, but note that it could result in out of date values on a live system.
- Return type:
- Returns:
the file, anon, and shmem page values
- drgn_tools.task.get_vmem(task)¶
Return virtual memory size of the task
- drgn_tools.task.is_group_leader(t)¶
Check if a task is thread group leader.
- drgn_tools.task.is_kthread(t)¶
Check if a task is kernel thread.
- drgn_tools.task.is_user(t)¶
Check if a task is user thread.
- drgn_tools.task.nanosecs_to_secs(nanosecs)¶
Convert from nanosecs to secs
- drgn_tools.task.runq_clock(prog, cpu)¶
Get clock of cpu runqueue
struct rq
- drgn_tools.task.show_taskinfo(prog, tasks)¶
Display task information.
- drgn_tools.task.show_tasks_last_runtime(tasks)¶
Display task information in their last arrival order.
- drgn_tools.task.task_cpu(task)¶
Return the CPU on which a task is running.
This is an equivalent to the kernel function
task_cpu(), but it covers a wide variety of variations in kernel version and configuration. It would be a bit impractical to spell out all the variants, but essentially, if there’s a “cpu” field instruct task_struct, then we can just use that. Otherwise, we need to get it from thethread_info.
- drgn_tools.task.task_lastrun2now(task)¶
Get the duration from task last run timestamp to now
The return duration will cover task’s last run time on cpu and also the time staying in current status, usually the time slice for task on cpu will be short, so this can roughly tell how long this task has been staying in current status. For task status in “RU” status, if it’s still on cpu, then this return the duration time this task has been running, otherwise it roughly tell how long this task has been staying in runqueue.
- drgn_tools.task.task_state(task)¶
Get a task’s state, which is combination of both its runnable and exit states. Since runnable and exit states are exclusive and occupy different bit positions, they will not interfere with each other. But we don’t want to miss out on corner cases or bugs.
- drgn_tools.task.task_thread_info(task)¶
Return a task’s
thread_infoThis is an equivalent to the kernel function / inline / macro
task_thread_info(), but it must cover a wide variety of versions and configurations.
drgn_tools.util module¶
- class drgn_tools.util.SimpleProgress(desc, total_bytes, quiet=False, update_every=0.5, notty_update_every=5.0)¶
Bases:
objectA simple download progress reporter
- Parameters:
- complete()¶
- drgn_tools.util.cpumask_to_cpulist(cpumask)¶
Get list of CPUs, present in a cpumask.
- drgn_tools.util.download_file(url, f, quiet=True, desc='Downloading')¶
- drgn_tools.util.enum_flags_str(prog, t, flags)¶
Convert enum flag bit to string
- drgn_tools.util.enum_name_get(enum, value, default=None)¶
Given an enum class and a value, return its name, or a default.
- drgn_tools.util.get_uts(prog)¶
Get system and version info
- drgn_tools.util.has_member(obj, name)¶
Return true if a given object has a member with the given name. :type obj:
Object:param obj: Drgn object to check :type name:str:param name: string member name to check :rtype:bool:returns: whether the object has a member by that name
- drgn_tools.util.kernel_version(prog)¶
Returns the kernel version as a tuple (major, minor, patch)
This is not the full release string, and it shouldn’t be confused with the UEK-specific parsing that is present in
drgn_tools.debuginfo.KernelVersion. It simply corresponds to the upstream major, minor, and patch versions, which typically (but not always) remain constant over a distribution kernel’s releases.Given a kernel version, especially the major.minor version alone, there is no guarantee about whether a commit is necessarily present or not. The linux-stable process regularly backports commits from newer releases into older ones, especially when they have a Fixes tag. Distributions like UEK also backport certain changes, regardless of whether they were included in stable releases.
This should be used only as a last resort for helper compatibility. At each usage of this function, a comment should be in place describing (a) the exact git commit SHA which introduces the change, and which kernel version the change appears in. (b) Why couldn’t the change in behavior be handled by detecting changes to variables or types? (c) Address whether there is a risk that stable/distro kernels may have a bakckport of the commit, which couldn’t be detected via a simple kernel version comparison.
- drgn_tools.util.per_cpu_owner(name, val)¶
Given a per-cpu variable/pointer, return CPU to which this per-cpu variable/pointer belongs.
- drgn_tools.util.percpu_ref_sum(prog, ref)¶
Get the sum of percpu reference count from
struct percpu_ref
- drgn_tools.util.redirect_stdout(filename, append=False)¶
Redirect standard output to a file within the body of the context manager
This context manager is a safe way to run code and capture its results in a new file. Here’s how to use it properly:
>>> with redirect_stdout("myfile.txt"): print("hello world")
In the above example, the file
myfile.txtwill now contain the string “hello worldn”.
- drgn_tools.util.to_binary_units(num, units)¶
Format a number as a simple human-readable number with units
- drgn_tools.util.type_exists(prog, type_)¶
Check whether some type exists in drgn
- drgn_tools.util.type_has_member(prog, typ, name)¶
Return true if a given object has a member with the given name. :type typ:
str:param typ: type name to check :type name:str:param name: string member name to check :rtype:bool:returns: whether the object has a member by that name
- drgn_tools.util.type_lookup_conflict(prog, name, module, filenames)¶
Lookup a type which has conflicting definitions, with DWARF or CTF
Unfortunately, DWARF and CTF handle conflicting type definitions in different ways, and drgn can’t handle them uniformly. With DWARF, drgn allows us to provide the filename which contains the definition of the desired type. However, CTF doesn’t contain filename information. It uses module names to resolve these conflicts. This function smooths over those problems.
- Parameters:
prog (
Program) – Program whose types we are looking upname (
str) – the type name to look upmodule (
str) – the name of the kernel module containing the definitionfilenames (
List[str]) – one or more lists of filenames containing the definition (in case the filenames have changed over different kernel versions)
- Return type:
- Returns:
the Type associated with
name, or else raisesLookupError
drgn_tools.virtio module¶
Helpers for dumping virtio device information
- class drgn_tools.virtio.VirtioQueueInfo(vq_name, vq_address, vring_addr, vring_avail_addr, vring_used_addr, nr_ent, last_used_idx, avail_idx_shadow, avail_idx, used_idx, event)¶
Bases:
tupleRepresents information about a virtqueue (or vring)
Since there are multiple formats that a vring_virtqueue could take depending on kernel version, this namedtuple represents common elements. See
get_vring_info()for more information.- avail_idx¶
Alias for field number 8
- avail_idx_shadow¶
Alias for field number 7
- event¶
Alias for field number 10
- last_used_idx¶
Alias for field number 6
- nr_ent¶
Alias for field number 5
- used_idx¶
Alias for field number 9
- vq_address¶
Alias for field number 1
- vq_name¶
Alias for field number 0
- vring_addr¶
Alias for field number 2
- vring_avail_addr¶
Alias for field number 3
- vring_used_addr¶
Alias for field number 4
- drgn_tools.virtio.for_each_device_virtio(prog, virtio_mods, vd_name=None)¶
Generate
struct device *list for all virtio device
- drgn_tools.virtio.for_each_virtio(prog, virtio_mods, vd_name=None)¶
Generate
struct virtio_device *object list from core
- drgn_tools.virtio.for_each_vring_by_vdev(vdev)¶
Return all
struct vring_virtqueue *of astruct virtio_device *
- drgn_tools.virtio.get_virtio_dev_vrings(vdevs, vd_name=None)¶
Get the vrings for the given virtio devices.
- drgn_tools.virtio.get_vring_info(vring)¶
Given a
struct vring_virtqueue *, get important info out of it.The vring_virtqueue differs across kernel versions, so the VirtioQueueInfo structure is used to represent the important information related to the virtqueue. Depending on the present fields of the struct, this function chooses the correct implementation and fills this structure out.
- Parameters:
vring (
Object) – Object of typestruct vring_virtqueue *- Return type:
- Returns:
Common vring/virtqueue info
- drgn_tools.virtio.load_virtio_mods(prog)¶
Lookup the list of virtio-related kernel modules and ensure debuginfo is loaded for each one.
- drgn_tools.virtio.virtio_devices_show(dev_list, vd_name=None, header=True)¶
Print virtio devices information to stdout
- drgn_tools.virtio.virtio_show(prog, show_vq=False, vd_name=None)¶
Show virtio devices info.
- drgn_tools.virtio.vrings_show(vrings, header=True, key=None)¶
Print all vrings
- Parameters:
- Return type:
drgn_tools.workqueue module¶
Workqueue¶
The drgn.helpers.linux.workqueue module provides helpers for working with the
Linux workqueues.
- drgn_tools.workqueue.find_worker_executing_work(work)¶
Find the worker that is current executing the specified work
- drgn_tools.workqueue.find_workqueue(prog, name)¶
Find workqueue with the given name
- drgn_tools.workqueue.for_each_cpu_worker_pool(prog, cpu)¶
Iterate over all worker_pool(s) of a CPU
- drgn_tools.workqueue.for_each_pending_work(prog)¶
Iterate over all pending work items (work_struct)
- drgn_tools.workqueue.for_each_pending_work_in_pool(pool)¶
Iterate over all works pending in a worker_pool
- drgn_tools.workqueue.for_each_pending_work_of_pwq(pwq)¶
Iterate over all pending works of a pool_workqueue
- drgn_tools.workqueue.for_each_pending_work_on_cpu(prog, cpu)¶
Iterate over all works pending in a CPU’s worker_pools
- drgn_tools.workqueue.for_each_pool(prog)¶
Iterate over all worker_pools in the system.
- drgn_tools.workqueue.for_each_pool_worker(pool)¶
Iterate over all workers in a worker_pool
- drgn_tools.workqueue.for_each_pwq(workqueue)¶
Iterate over all pool_workqueues(pwq) of a specified workqueue
- drgn_tools.workqueue.for_each_worker(prog)¶
Iterate over all workers in a system
- drgn_tools.workqueue.for_each_workqueue(prog)¶
Iterate over all workqueues in the system.
- drgn_tools.workqueue.get_work_pool(work)¶
Get worker_pool associated with a work
- drgn_tools.workqueue.get_work_pwq(work)¶
Get pool_workqueue associated with a work
- drgn_tools.workqueue.is_task_a_worker(prog, pid)¶
Check if specified task is a worker thread.
- drgn_tools.workqueue.print_workqueue_names(prog)¶
Print the name and
struct workqueue_struct *value of all workqueues.
- drgn_tools.workqueue.show_all_workqueues(prog, showidle=False)¶
Dump state of all workqueues and worker_pools
- drgn_tools.workqueue.show_one_worker_pool(worker_pool)¶
Dump a worker_pool :type worker_pool:
Object:param worker_pool:struct worker_pool *.
- drgn_tools.workqueue.show_one_workqueue(workqueue)¶
Dump a workqueue :type workqueue:
Object:param workqueue:struct workqueue_struct *.
- drgn_tools.workqueue.show_pwq(pwq)¶
Dump a pool_workqueue
