Scratch/riel/edge visibility - #85
Merged
rikvanriel merged 7 commits intoAug 30, 2026
Merged
Conversation
Three walks over file scope each decided for themselves what a preprocessor
conditional does to the code it holds. Two listed four node kinds by name and
so dropped everything under #elifdef and #elifndef; the third matched every
preproc_ node. The narrow spelling is what hid 345 initcalls, 16,223 dev_dbg
call sites and 213 syscall bodies, each found and fixed separately.
is_conditional_group() answers the question once. at_file_scope() and
file_scope_sequences() are the two shapes its callers need: whether a node sits
at file scope, and the runs of siblings there, which is what identifies a body
a macro opens.
No rows move on Linux 0595459f, which uses neither directive:
$ grep -rn "^#[[:space:]]*elifdef\|^#[[:space:]]*elifndef" \
--include=*.c --include=*.h . | wc -l
0
$ regress.sh
functions 1054808 -> 1054808 +0
registrations 849839 -> 849839 +0
call_edges 3205369 -> 3205369 +0
fields 872037 -> 872037 +0
VERDICT: ok
A tree that uses them needs no fourth fix. Both tests fail with the four-kind
list restored: a SYSCALL_DEFINE1 body under #elifdef is not indexed, and an
initcall written there is not recorded.
Assisted-by: claw:claude-opus-5
Signed-off-by: Rik van Riel <riel@surriel.com>
A declaration a macro builds is not parseable C. `TRAILING_OVERLAP(...)` is
one: the grammar ends the function at that line and makes the statements below
it children of the translation unit, where they belong to no function. The
function records the calls above the break, none of those below it, and says
nothing about the difference.
$ semcode -q "calls intel_security_freeze"
'intel_security_freeze' directly calls 1 functions:
1. nvdimm_provider_data
drivers/acpi/nfit/intel.c:119 calls three. Counting braces from the body's
opening one finds the end the parser could not. Calls are already attributed by
byte range, so the range is the whole fix. Strings, character literals and
comments are skipped, since a brace inside any of them is text; the count stops
at the next function definition, so an unbalanced brace inside a conditional
cannot swallow the rest of the file.
$ semcode -q "calls intel_security_freeze"
'intel_security_freeze' directly calls 3 functions:
1. nvdimm_ctl
2. nvdimm_provider_data
3. test_bit
A `return` or an `if` cannot appear outside a function, so each one at file
scope marks a body that ended early. Linux 0595459f has 1,176 of them holding
687 call sites, 451 outside tools/.
$ regress.sh --allow call_edges,functions
~ call_edges 3205369 -> 3205518 +149
functions 1054808 -> 1054808 +0
registrations 849839 -> 849839 +0
fields 872037 -> 872037 +0
VERDICT: ok
-- kernel: idempotent
149 edges rather than 687 sites because a function records each callee once
however often it calls it. No function is added and no other table moves: the
recovered edges land on functions that were already indexed, and a body the
parser read whole keeps the end it had, which is what the second test pins.
Assisted-by: claw:claude-opus-5
Signed-off-by: Rik van Riel <riel@surriel.com>
A name in C is not one thing. The Linux tree defines pr_warn nine times, and a
callee query returned the callees of whichever definition a heuristic preferred
-- not a header, then the longest body -- with nothing said about the choice:
$ semcode -q "calls pr_warn"
'pr_warn' directly calls 4 functions:
1. fprintf
2. va_end
3. va_start
4. vfprintf
Those belong to arch/x86/tools/insn_decoder_test.c:48. No kernel caller reaches
that definition; the one a kernel caller does reach, include/linux/printk.h:563,
calls printk. The answer was not incomplete, it was about someone else.
14,739 names in Linux 0595459f have several definitions that disagree about
what the name calls:
distinct names: 840658
more than one row: 165263
more than one row recording calls: 20882
and those rows disagree about what it calls: 14739
get_function_callees_by_definition returns each definition with the file and
line it was read from, and the calls command prints one group per definition.
Which one a call site reaches depends on that file and on the configuration the
tree is built with; the index knows neither, and the reader knows both, so the
answer names the file rather than guessing.
A prototype is not a second definition. Nearly every exported function has one,
and counting it would make almost every name ambiguous, so a row is classified
by its own stored text: no braces, no leading #, ends in a semicolon. Testing
whether the row records calls would have been cheaper and wrong -- a definition
that calls nothing is a second answer, and a different one.
An edited file answers for itself. The working directory is what the reader is
looking at, so a definition in a dirty file replaces the committed row for that
file rather than being reported beside it.
Call chains keep the single-definition path: walking every definition of every
name multiplies a chain by the ambiguity at each step, and a chain is read as
one path. callers, func and callchain still collapse ambiguity the way calls
used to.
$ regress.sh # against the previous commit
functions 1054808 -> 1054808 +0
call_edges 3205518 -> 3205518 +0
VERDICT: ok
-- kernel: idempotent
Nothing in the index changes: this is what a query does with what is already
stored.
Assisted-by: claw:claude-opus-5
Signed-off-by: Rik van Riel <riel@surriel.com>
#define printk(fmt, ...) printk_index_wrap(_printk, fmt, ##__VA_ARGS__)
Nothing in Linux calls _printk by name -- callers write printk, and the macro
hands _printk to printk_index_wrap, which calls it. Handovers were read only
outside macro bodies, so the index recorded none of this and the query ended
the search:
$ semcode -q "callers _printk"
4 functions directly call '_printk':
no_printk, __printf, _fat_msg, _btrfs_printk
5,729 functions call it in a built kernel. The four are the ones that happen to
spell the name.
A macro body is already re-parsed to find what it calls, what it installs in a
struct member and what it dispatches through; a handover is the same kind of
fact and is now read from the same parse. A macro's own parameter is excluded,
as it is for registrations: `#define call_it(fn, x) helper(fn, x)` names no
function.
argument_functions 1921278 -> 2052229 +130951
call_edges 3205518 -> 3205518 +0
functions 1054808 -> 1054808 +0
VERDICT: ok
-- kernel: idempotent
27,023 more distinct names are recorded as handed to a call, and 900 more
functions have a handover as the only way in that the index knows about.
`callers` now reports that way in rather than reporting silence. This is the
half that matters for reading: 49,699 functions in this tree are reached only
by being handed over, and the answer for every one of them was "No functions
call it", which reads as dead code and ends the search.
$ semcode -q "callers _printk"
Handed over: it is handed to another call as an argument, so a caller of
that reaches it:
printk_index_wrap() argument 0 at include/linux/printk.h:511 in printk
=== Direct Callers ===
4 functions directly call '_printk': ...
The named call and its file and line are what let a reader continue: `callers
printk` is the next question, and it has an answer.
Assisted-by: claw:claude-opus-5
Signed-off-by: Rik van Riel <riel@surriel.com>
An edge the index cannot record is still an edge. Printing the callees it has
and stopping there says the rest are not there.
#define printk_index_wrap(_p_func, fmt, ...) _p_func(fmt, ##__VA_ARGS__)
The call is real; its callee is whatever the invocation passed. Recording
`_p_func` as the callee names a function no tree defines, and 1,752 such edges
were stored across Linux 0595459f, each pointing at nothing. Dropping them
without saying anything would replace a wrong answer with a silence, which is
worse: a wrong answer gets checked and a silence gets believed.
The unresolved_edges table stores what is known, the mechanism that blocks the
edge, and where to look:
$ semcode -q "calls printk_index_wrap"
Unresolved: a call here goes to _p_func (parameter 0), which this file does
not name: definition include/linux/printk.h:481
'printk_index_wrap' directly calls 1 functions:
1. __printk_index_emit
Locations are a list with a role each, not one file and line. A macro needs the
definition that hides the call and the invocation that supplied the name; a
hardware interrupt needs the IDT entry that installs the handler as well as the
assembly stub. Committing to a single pair of columns would make every consumer
re-derive the rest, and each would derive it differently. `kind` is an open
string namespaced by language for the same reason in the other direction: the
row shape holds across C, Rust and Python, and a vocabulary common to macros,
trait objects and attribute lookup would describe none of them.
unresolved_edges 0 -> 1862 +1862
unresolved_edges:c:macro_parameter_call 0 -> 1862 +1862
call_edges 3205518 -> 3203766 -1752
functions 1054808 -> 1054808 +0
VERDICT: ok
-- kernel: idempotent
1,862 rows against 1,752 removed edges: a macro that calls two of its own
parameters records both.
Schema version 9. An index written by 8 holds the call to the parameter as a
call to a name nothing defines, so it is refused rather than read.
Assisted-by: claw:claude-opus-5
Signed-off-by: Rik van Riel <riel@surriel.com>
Indexing a tree fails outright:
ERROR semcode::git_range: Inserter 0 failed to insert registrations: lance
error: Invalid user input: Ambiguous merge inserts are prohibited: multiple
source rows match the same target row on (file_path = "kernel/fork.c",
git_file_hash = "f0e2e13...", byte_start = 77208, target = "set_tid")
The body a macro opens is not a function_definition. Anything inside it is
therefore found by two walks: the one over the function the macro defines, and
the one that collects what no function encloses. `kargs.set_tid = set_tid` in
`SYSCALL_DEFINE2(clone3, ...)` was recorded twice, at kernel/fork.c:3060, once
in `sys_clone3` and once with no enclosing function:
$ semcode -q "registrations set_tid"
2 places install it:
1. kernel_clone_args::set_tid at kernel/fork.c:3060 [assignment]
2. kernel_clone_args::set_tid at kernel/fork.c:3060 in sys_clone3
A row is keyed by its place in a file, so the pair is one fact and a duplicate,
and the database refuses the batch rather than storing it. Whether it refuses
depends on which rows land together, which is why a tree indexes cleanly one
day and not the next.
Keeping the row that names what encloses the fact leaves one:
registrations 1088591 -> 1088366 -225
dispatch_sites 98351 -> 98334 -17
on Linux at 0a0d1d55dad5. Handovers get the same treatment, though this tree
has no duplicate among them.
Assisted-by: claw:claude-opus-5
Signed-off-by: Rik van Riel <riel@surriel.com>
meta-cla
Bot
added
the
CLA Signed
Indexing a tree fails outright:
ERROR semcode::git_range: Inserter 0 failed to insert dispatch sites: lance
error: Invalid user input: Ambiguous merge inserts are prohibited: multiple
source rows match the same target row on (file_path = "fs/aio.c",
git_file_hash = "f57fa21...", byte_start = 58144, target = "")
A file unchanged between two commits is analysed under each, so a batch built
from several commits holds the same row twice. Once the row is in the table --
after the first commit that named it was indexed -- those two source rows both
match one target row, and merge_insert refuses the batch rather than the
duplicate. Everything batched alongside it is lost with it, and whether that
happens depends on which commits' rows share a batch, so the same tree indexes
cleanly one day and not the next.
The rows are identical, so which is kept does not matter; that one is kept does.
Applied to every table whose rows are keyed by a place in a file or by content:
dispatch sites, registrations, handovers, unresolved edges, globals, functions
and object macros.
The tests hold the row first and then insert a batch containing it twice, which
is the order that fails. Without this they report the error above verbatim.
Assisted-by: claw:claude-opus-5
Signed-off-by: Rik van Riel <riel@surriel.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
syscalls, more functions called through macros, and breadcrumbs for auditors when edges do not resolve cleanly