Can Jev beat a compiler's handwritten rules?

A compiler spends a surprising amount of time guessing. Is this branch usually taken? Is this function worth inlining? Will this loop run enough times to justify making it larger?

Profile-guided optimization i.e running the program a bunch of times and checking the actual numbers, can answer these questions, but most programs are compiled without a representative profile. In that case, the compiler falls back to rules written by people.

That sounds like a classification problem. I ran three experiments to see whether Jev could make better guesses:

The answer to the title is not yet. Jev's predictions carry useful information, but these experiments did not establish a faster SQLite binary.

What the compiler is guessing

Inlining

Inlining replaces a function call with a copy of the function's body:

static int square(int x) { return x * x; }

int sum_squares(const int *a, int n) {
    int s = 0;
    for (int i = 0; i < n; i++)
        s += square(a[i]);      /* a call on every iteration */
    return s;
}

After inlining square, the loop contains no call:

int sum_squares(const int *a, int n) {
    int s = 0;
    for (int i = 0; i < n; i++)
        s += a[i] * a[i];       /* body copied in place */
    return s;
}

The call is gone, and the optimizer can now see the multiply inside the loop. That lets it vectorize the loop, which it could not do across a call.

Inline too little and the program does extra work. Small helpers in hot loops pay for a call and return on every iteration, and the optimizer cannot see through them.

Inline too much and the binary grows. Every call site gets its own copy of the body. Hot code stops fitting in the instruction cache, and large functions run out of registers and spill to the stack.

GCC balances this with a cost model. It estimates how much each inlining decision grows the code and saves in time, then compares the result to tunable limits such as --param max-inline-insns-single and --param max-inline-insns-auto. The limits are hand-tuned numbers.

Hot and cold code

Consider a familiar error path:

char *buf = malloc(100);
if (buf == NULL)
    return SQLITE_NOMEM;

/* main work */

The compiler sees a control-flow graph rather than indented C:

                         ┌──────────────────┐
                         │ buf = malloc(100)│
                         └────────┬─────────┘
                                  │
                           buf == NULL?
                             ╱          ╲
                        yes ╱            ╲ no
                           ▼              ▼
                  ┌────────────────┐  ┌─────────────┐
                  │ return NOMEM   │  │  main work  │
                  │   COLD PATH    │  │   HOT PATH  │
                  └────────────────┘  └─────────────┘

If allocation normally succeeds, the return is cold and the main work is hot. The compiler wants to place the common instructions next to each other and move rare error handling out of the way. The hot path then fits into fewer instruction-cache lines and the CPU fetches less code it will not run.

Guessing before the program runs

The compiler never sees your input. With profile-guided optimization (PGO) you run an instrumented build on a representative workload, and the measured counts guide the next build. A normal gcc -O2 build has no counts, so GCC guesses from the shape of the code. Its rules live in gcc/predict.def. A few of them:

When several rules match the same branch, GCC combines them, either by taking the first strong match or by Dempster-Shafer combination. It then propagates the probabilities through the control-flow graph to estimate how often each block runs. Programmers can override all of this with __builtin_expect, usually wrapped in likely() and unlikely() macros.

These values have always been static. A person picked each number, and every branch that matches a rule gets the same number in every program. Each one is still a guess. What if we showed Jev the code around a branch instead and let it predict the outcome for that specific branch?

Jev as a classifier

A language model's distribution is over vocabulary tokens:

Jev's distribution is over classes, or actions:

For example, give Jev a support issue and the actions a team could take:

input:    How do I triage this issue? User state is invalid after login.
actions:  ping the frontend team | ping the backend team | ping the infra team

Jev returns one probability per action, something like:

ping the frontend team   0.21
ping the backend team    0.64
ping the infra team      0.15

Nothing is generated. The answer is always one of the listed actions, with a probability attached.

Every experiment below uses the same pattern. The input is a window of LLVM IR, and the actions are the possible answers, such as hot or cold. In the hotness experiment, reading the answer looked like this:

p_hot = response["answers"]["hot"]["noul"]
prediction = "hot" if p_hot >= 0.5 else "cold"

Compilers need many narrow decisions like this, not paragraphs of prose.

Experiment 1: inlining

The first question was the original one for the whole project. At every call site the compiler decides "inline or not" using the cost model above. Could Jev predict which choice makes the binary faster?

Example input and output

This is a made-up example, shown as C for readability. The actual input was the callee's LLVM IR and nearby call-site code. The probabilities and timings below are illustrative, not experimental results.

static int square(int x) { return x * x; }

int sum_squares(const int *a, int n) {
    int total = 0;
    for (int i = 0; i < n; i++)
        total += square(a[i]);  /* candidate call site */
    return total;
}

Question to Jev: Would inlining this call make the program faster?
Illustrative output:
  inline       0.80
  do not inline 0.20

Separate timing test on our workload:
  forced inline:       95 ms
  forced out of line: 100 ms
  label: inline, if the difference survives repeated measurements

Jev predicts a choice, not a run time. We score that choice against the winner of the timing test. GCC's normal inlining choice is scored against the same winner. The example looks straightforward; the real measurements often did not have a stable winner.

Setup

For each candidate call site, I built two binaries. In one, the callee was forced inline. In the other, it was forced out of line:

/* arm A */
static inline __attribute__((always_inline)) int helper(/* ... */) { /* ... */ }

/* arm B */
static __attribute__((noinline)) int helper(/* ... */) { /* ... */ }

I timed both arms with hyperfine on a fixed workload, and the faster arm became the label. Jev saw the callee's IR plus the call-site context and predicted which arm would win. The labeling rule was strict:

label = the faster arm
keep the site only if:
  all 3 independent timing rounds agree on which arm is faster, and
  |relative difference| > max(1%, 2 × standard error)

It was all Noise

First pass: PolyBench with clang -O3. Of 93 candidate call sites, 25 had a timing winner. Only 1 of those 25 cleared its own 2σ noise bar. The effects were 2–7%, while run-to-run timing varied by up to 28%. Most candidates were init_array and print_array helpers that run outside the timed kernel, so their inlining barely mattered.

Rebuilding flips the answer. On SQLite with clang, 27 sites had a clear winner. Then I rebuilt SQLite from scratch and timed those 27 sites again:

Rebuilding SQLite changes its code layout. Functions move, and loop heads land on different cache-line boundaries. That shift was worth several percent of run time, more than the 1–3% effect of the inlining decision I was trying to measure. A trustworthy sample of 4 sites is not a dataset.

Last pass: SQLite with GCC -O2. I hoped a less aggressive compiler would leave bigger effects to measure. The effects were bigger (median 4.6%, against about 2.5% under clang), but no more stable: only 16 of 40 sites gave the same answer in all three rounds. On the 13 sites with a clear winner, GCC's own choice was right 9 times. Jev was right 7 times, the same as always guessing the majority class.

I stopped the inlining experiment. Jev did not clearly fail; the labels were the problem. A stopwatch cannot label a 2% effect when rebuilding the binary moves run time by more than that.

Experiment 2: branch hotness

The fix for noisy labels was to stop timing and start counting:

Does this conditional branch execute often?

So I used a profile from an instrumented SQLite build as the answer key. Jev still saw code without the workload's queries, data, or measured frequencies. It made a static guess, much as a compiler does without a profile. The test was whether that guess matched the profile-derived hot/cold label, not whether Jev could predict an exact execution count.

Example input and output

This is a made-up example, shown as C rather than LLVM IR. The output probabilities and profile numbers are illustrative.

int sum_rows(const int *rows, int n) {
    int total = 0;
    for (int i = 0; i < n; i++) {
        if (rows[i] > 0)        /* branch being classified */
            total += rows[i];
    }
    return total;
}

Question to Jev: Does this branch execute at least 50 times per workload run?
Illustrative output:
  yes  0.85
  no   0.15

Separate profile, across 20 workload runs:
  true executions:    200
  false executions:   400
  total checks:       600

Loop rule: inside a loop → hot
Answer key: 600 / 20 = 30 checks per run → cold
Jev predicts yes → incorrect on this example

Every time the program reaches the if, it checks the condition and gets either true or false. In this example, 200 true answers plus 400 false answers means 600 checks. Hotness counts all those checks, not just the true answers. Jev's 0.85 is confidence in the yes/no answer, not a predicted count or a fraction of true outcomes.

Jev can see a loop, but without the workload it cannot know n or how often the function is called. The same limitation applies to the loop rule. We test both guesses against what happened on this workload.

Setup

I ran the same SQLite workload 20 times with profiling enabled. The profiler recorded how often the program reached each condition. These measurements were the answer key, not input to Jev.

For this test, I defined hot as at least 50 evaluations per run on average: 1,000 across the 20 runs. Below that was cold. Jev saw the nearby code and guessed whether the condition met that cutoff. It did not predict an exact count.

I tested 1,247 conditions selected from 3,934, randomly taking at most three per function so large functions would not dominate. The random seed was fixed so the sample could be reproduced.

For each condition, the rule is simple. Here, true_checks and false_checks mean how many times the condition answered true or false across all 20 runs:

total_checks = true_checks + false_checks  # how many times we reach this condition
hot = total_checks >= 50 * 20              # 20 runs of the same SQL workload

I applied this cutoff separately to each of the 1,247 conditions to give it a hot or cold label.

The extracted profile values include two added counts per condition. I checked that this small adjustment changes none of the hot/cold labels in the sample.

The actual code input was LLVM IR: up to 25 lines leading to the condition and six lines from each of its two paths. The C example above shows the same idea in a more familiar form.

I evaluated two input versions. The code-only version contained the IR window and function/block identifiers, without an introductory program description or a heuristic hint. Function symbols still identify SQLite code. The original version added this context:

You are looking at one conditional branch in the LLVM IR of a C program (sqlite).
The program runs a fixed SQL workload. Predict execution behavior from static code only.

Heads-up: branches inside loops execute many times; branches in error-handling or
one-time initialization paths execute rarely.

Both versions used the same question: "This branch executes at least 50 times per workload run."

Our custom baseline

The comparison was deliberately simple:

hot  if the branch's basic block is part of a cycle in the control-flow graph
cold otherwise

In plain terms: code in a loop is hot, and code outside a loop is cold. I computed cycles with Tarjan's strongly connected components algorithm. This is our implementation of one simple rule, not GCC's production predictor. The other baseline always guesses cold, the more common class. Both baselines use the same branches and labels as Jev.

Results

A branch counted as hot at the 1,000-check cutoff. That gave 415 hot and 832 cold branches. All 1,247 calls in the code-only run succeeded. Jev predicts hot when its probability is at least 0.5.

Code-only Jev beats the loop rule by 13.2 accuracy points, but beats always guessing cold by only 2.6 points. AUC measures how well the scores separate hot from cold branches: 0.716 means a randomly selected hot branch gets a higher score than a randomly selected cold branch about 71.6% of the time, counting ties as half. It is not an execution-count prediction.

On this workload, Jev did better than our naive hotness heuristic. I plan to test it against GCC's full predictor at some point; that comparison has not been run yet.

Experiment 3: branch direction

Hotness asks how often the program reaches an if and evaluates its condition. Direction asks which answer that condition usually gives once it is reached: true or false.

Take if (rows[i] > 0) from the previous example. Suppose one workload run processes 100 rows, but only one is positive. The program evaluates the condition 100 times, so it is hot by our 50-check cutoff. Yet the condition is false 99% of the time. A hot condition can usually be false.

Now suppose a run processes only 10 rows, all positive. The condition is cold by our cutoff because it is evaluated only 10 times, but its answer is always true. Frequency and direction are separate measurements.

__builtin_expect hints at direction: which answer the compiler should expect when it reaches the condition. It does not say how many times the condition will be evaluated. Experiment 3 tests whether Jev can predict that direction, then whether using its predictions as hints makes SQLite faster.

Example input and output

This is a made-up example, shown as C rather than LLVM IR. The probabilities and profile numbers are illustrative.

char *buf = malloc(100);
if (buf == NULL)              /* branch being classified */
    return SQLITE_NOMEM;
/* continue with normal work */

Question to Jev: When this branch executes, is its condition true
                at least 9 out of 10 times?
Illustrative output:
  yes  0.08
  no   0.92

Separate profile:
  true edge:     5
  false edge:  995
  true fraction: 5 / 1,000 = 0.5%
  label: no, this condition is usually false

Jev's 0.08 is the probability that the condition is usually true, not a prediction that it is true on 8% of executions. Unlike Experiment 2, we score the direction rather than total frequency. We kept only branches whose profiles leaned at least 90% one way, so "no" means usually false within this selected sample. The classification baseline always answers "no"; it is not GCC's full predictor. We then separately test whether Jev's directions, supplied as compiler hints, make SQLite faster.

Setup

Same branches, same IR, same counts as Experiment 2. I kept the 942 branches that went the same way at least 90% of the time: 335 usually true and 607 usually false. The proposition was:

When this branch executes, its condition is true almost every time (at least 9 out of 10 executions)

Results

The AUC says Jev ranks direction reasonably well. The accuracy gain over the majority guess is only 3.4 points, and the per-class breakdown shows why:

Jev answered "usually true" for only 13.8% of branches, while 35.6% really were. It spots branches that are rarely true, and mostly misses the ones that are almost always true.

Putting the hints into a real build

The real test is whether the predictions make a binary faster. I wrapped each predicted condition in SQLite's source with __builtin_expect. For example:

/* before */
if (rc != SQLITE_OK) goto abort_due_to_error;

/* after: Jev predicts the condition is usually false */
if (__builtin_expect((rc != SQLITE_OK), 0)) goto abort_due_to_error;

I built four versions with GCC -O2:

Each arm ran the fixed SQL workload on a pinned core, in 5 independent hyperfine rounds of 40 runs.

The chart shows how much longer each version took than stock GCC. Higher means slower, not better. Zero means the same run time as stock; a positive value means a slowdown, and a negative value means a speedup. Profile-based hints were 5.44% slower, random hints were 4.61% slower, and Jev's hints stayed close to stock with no consistent improvement.

Perfect hints made SQLite slower. So did random hints. Jev's arm stayed close to stock but moved around between rounds, so I don't read anything into it.

I may come back to this and dig deeper someday. One guess: branch hints change code layout, and the layout change can have side effects that cost more than a correct hint saves.

So, can Jev compile?

Not yet, maybe in the future TypeSafe AI will allow general finetuning and we can revisit this, there already are other decision models in the market, maybe I'll try finetuning them myself and then re-running these experiments.