Imagine you are watching a recording of a single molecule as it switches between a few shapes. You want to know whether something is driving it, such as a molecular motor burning fuel, or whether it is simply settled at equilibrium. The data arrive one step at a time, so you run a standard statistical test every so often and plan to stop once the answer looks convincing.
Suppose the test is designed to raise a false alarm only 5% of the time. In our simulations, checking it as the data came in and stopping when it looked convincing raised the false-alarm rate to between 49% and 60%.
The test was working as designed. Its 5% promise assumes a single look at a sample size fixed in advance, and repeated checking breaks that assumption. This note summarizes a paper from FriendMachine’s research on this problem, for one question from physics. It describes a test that stays valid however often you check it, and a limit on how quickly any such test can reach a verdict.
Is this system at equilibrium?
Many systems can be recorded as a sequence of states, such as a molecule switching shapes or a machine changing modes. The paper treats the simplest case: a system with finitely many states, all of which you can see, whose next step depends only on its current state. A basic question about such a system is whether it is at equilibrium.
At equilibrium, a condition called detailed balance holds. Every transition happens as often in one direction as in the reverse, so a film of the system played backward would look statistically the same. When detailed balance is broken, something is driving the system. Physicists usually measure how strongly it is driven by the entropy production rate, written σ.
Most work asks how large σ is. We studied a question that comes first: from data examined as they arrive, can you decide that detailed balance is broken at all, without fooling yourself?
Why checking repeatedly breaks a test
Our baseline was the likelihood-ratio test, a classical test that asks how much better the data fit an unrestricted model than the best-fitting equilibrium model. Like most classical tests, it promises a 5% false-alarm rate at one planned sample size. Each extra check is another chance for random noise to cross the threshold, and those chances add up.
We simulated 1,000 recordings of 20,000 steps for each of several systems that were truly at equilibrium, so every alarm was false. On the fully observed systems the theory covers, the likelihood-ratio test behaved as designed when used once, with 4.4–5.7% false alarms. Checked 10 times along the way, it flagged 29.1–35.5% of recordings. Checked at 221 points from step 100 onward, it flagged 49.4–59.8%.
Tests you can check after every observation
Statisticians have a family of tests built for exactly this habit. These anytime-valid tests keep their false-alarm guarantee no matter when or how often you look, and no matter when you decide to stop.
One way to build such a test is as a bet. Start with $1 of imaginary money and bet on the data as they arrive. If the system really is at equilibrium, the bets are fair, so on average your money cannot grow. You declare equilibrium broken only if your money reaches $20. A classical result in probability, Ville’s inequality, says that in a fair game your money reaches 20 times its starting value with probability at most 1 in 20. That gives a 5% false-alarm rate that holds however often you check.
The money also measures evidence. Counting how many times it has doubled gives evidence in bits, so reaching $20 takes about 4.3 bits (log₂ 20). When the system is out of equilibrium, a well-designed betting strategy grows the money steadily, and the useful question becomes how many bits it can gain per step.
A speed limit below the entropy production
We call the fastest rate at which evidence can accumulate ρ, measured in bits per step. It is a statistical distance (a Kullback–Leibler divergence rate) from the observed system to the nearest equilibrium system. That nearest system is the average of the observed system and its own time reversal, the system whose recordings look like the original’s played backward.
No valid test can, on average, gather evidence faster than ρ bits per step. Since a 5% test needs about 4.3 bits, detection takes at least roughly 4.3/ρ steps. This lower bound is a special case of a general result for sequential tests on this kind of data by Sethi et al. (2026). Our contribution is a closed-form expression for ρ in the equilibrium question, and its relation to entropy production. ρ is at most half of σ, and near equilibrium it is about a quarter of σ. It also stays finite when σ is infinite, which happens when some transition occurs in only one direction.
The limit is lower than σ because our question is harder. Evidence accumulates at rate σ for a narrower question: does this system differ from its own time reversal? We ask whether the system matches any equilibrium system, so the data must rule out all of them, including the one closest to the truth, and evidence against that closest one accumulates more slowly.
The loop test
Knowing the limit, the next question is how close a practical test can get to it. The paper’s main practical proposal is the loop test. Pick a reference state that the system visits often. Each time the system leaves that state and returns, it has traced a loop. At equilibrium, every loop is as likely to be traced in one direction as in the reverse, so each loop’s direction is a fair coin. The test bets on loop directions, using what it has learned from earlier loops. It needs no optimization, and it can be checked after every step.
Looking only at loop direction throws away some information, though less than you might expect. Near equilibrium, for ρ up to 0.02 bits per step, loop directions carried 92–100% of the available evidence. At ρ around 0.06 they carried about 80%. Far from equilibrium they carried as little as 34%, but those systems were still detected within about a hundred steps.
We also compared the loop test with an idealized classical test that is told in advance exactly how much data it will need, which experimenters rarely know. For the five test systems with ρ below 0.07 bits per step, the loop test reached a verdict in 80% of recordings within 1.7–1.9 times that idealized sample size. On those same five systems, the paper’s other anytime-valid test, built from likelihoods, needed 5.5–8.7 times as much.
Where the guarantees stop
These results hold under specific conditions, and some questions remain open.
- The theory covers fully observed systems with finitely many states, where the next step depends only on the current state. On three systems seen through noise or with merged states, the loop test’s false-alarm rate was 2.6–4.5%, within target, but no theorem guarantees that.
- The tests detect that equilibrium is broken. They don’t say by how much, and building a confidence range for the entropy production is an open problem.
- The loop test’s 5% guarantee holds when it is checked at every step. A stronger property, that your expected money at the moment you stop never exceeds the $1 you started with, does not hold for stopping rules that act partway through a loop. An adversarial review by separate AI-agent sessions, run as a consistency check and not as peer review, found such a rule.
- The experiments are small and synthetic. All systems had at most five states, none came from real recordings, and the efficiency figures used one choice of reference state.
The paper, code and seeded scripts that regenerate the experiments are in the Evidence-Holonomy repository.