49 Technical Signals, Zero Passed

The previous article tested nine chart pattern types and left behind an honest list of gaps: indicators were computed but never tested, market regime was computed but never entered a test cell, and Fibonacci was never attempted. It also contained this sentence:

These will be done separately — one piece tested, one article written. … Whether the answer comes back "yes" or "no," it gets written up the same way.

This is the first instalment of that promise. All three gaps are filled, with candlestick patterns added.

The result: 49 signal types, 1,779,160 events, 11,025 test cells, and zero passed FDR correction.

But the zero is not what this article is about. Before arriving at it, I first arrived at 5 "significant findings," four of which were Fibonacci retracement levels. All of them were false, and what caught them was a control run I could have skipped.

The awkward part is why they appeared: I was following my own written specification, the specification was wrong, and following it made the result worse.

This is a research and methodology article. It describes how a test was designed, what it found, and where the boundaries of that finding lie. It is not a recommendation regarding any security or any method of analysis, and it is not investment advice.

1. What this instalment adds

Against the gap list from last time:

That is 49 event types across seven families. Every one is a mechanical rule, not a human reading a chart:

By timeframe: 1,440,900 daily, 284,673 weekly, 53,587 monthly.

Parameters were frozen before any statistics ran, exactly as in the previous article. Worth stating precisely: freezing the indicator parameters changed no values at all — the 5/10/20/120-day averages, and the stochastic, RSI, MACD and Bollinger settings, were already in the config file before this research question was posed. There was no window in which to try parameter sets and keep the flattering one.

How Fibonacci was made testable

Fibonacci was the hardest of the group, because it is the most subjective: which high and which low you measure the retracement from is itself a judgement. Two people drawing the same stretch of price can arrive at completely different 0.618 levels.

The approach was to hand that degree of freedom to the ZigZag pivots frozen in the previous article: the endpoints are chosen by a fixed rule, not by a person. The cost is recorded in the limitations — the conclusion is now tied to that ZigZag parameter set. But without that step Fibonacci cannot be tested at all; it stays permanently inside the "you drew it wrong" loop.

Multi-timeframe conditioning

"A golden cross on the daily, but check the weekly trend first" is a common way to talk about signals in practice. This time it became a conditioning dimension: the weekly or monthly trend was expanded into four regime states, and every daily signal was tested separately within each.

Three combinations: daily × weekly, daily × monthly, weekly × monthly.

2. Test design: same as before, so only what is new

Three benchmark layers (market-adjusted, unconditional, matched control), p-values from a randomization test, confidence intervals from a studentized bootstrap, Benjamini–Hochberg FDR at q = 0.10 — all of that is covered in Section 2 of the previous article and is not repeated here.

One new decision is worth stating: indicator signals form their own FDR family, corrected separately from the 354 chart-pattern cells.

Separate correction is the more permissive choice, so the cost was quantified: under combined correction, zero cells flip — none move from non-significant to significant, and none move the other way.

There is a counterintuitive point buried in that check. Combined correction is not uniformly more conservative. A BH q-value depends on both the denominator and the rank, so folding in a batch of more significant cells can make a given cell's q smaller. Which means "flips" must be checked in both directions — my first version of that check only looked one way, and another session caught it.

3. Results

Five studies, all zero:

★ The shuffled control re-runs everything with event dates scrambled — a smoke test. It also returns zero, meaning the pipeline does not mistake pure noise for a finding. (Lesson from last time: passing this test does not mean the method is correct. Section 4 is exactly the class of problem it cannot catch.)

Across all cells:

The closest cell to significance has a q of 0.100 — it misses the threshold by 0.0002. That near-miss is where the next section's error begins.

4. The 5 false discoveries I nearly published

This section is why the article exists.

The project's architecture document specifies a randomization resample count of B = 10,000. The code was actually running 2,000. That is a specification-versus-implementation mismatch, the kind of thing that should be fixed — so I raised it to the specified 10,000 and re-ran.

The answer changed:

Five cells passed FDR correction. Four were Fibonacci retracement levels (0.382 and 0.500 support); the fifth was a Bollinger upper-band reversal.

It looked exactly like a finding. The effect sizes were not small (−3.5% to −7.4%), the direction was consistent (all negative — "support does not support"), and Fibonacci is the most talked-about item in the set. Had I stopped and reported there, it would have been a very persuasive error.

What caught it was the control

I had also re-run the shuffled control at B = 10,000. That is the null world — the event dates are scrambled, so by construction there is nothing there to find.

The null control produced exactly as many significant cells. Empirical FDR: 100%.

Those 5 cells were not a discovery. They were an artifact of the method.

Why: more resampling made the precision fake

The cause is in how the randomization works. As the previous article explained, the null distribution is rebuilt by shifting every event by the same offset — which is what preserves the clustering of events onto shared days (the very error that article had to correct).

But a uniform shift has a built-in ceiling: the offset can only take integer values within a bounded range, so the number of distinct null realizations is finite. Drawing more times simply redraws the same realizations.

Measured:

At 10,000 draws only 36.4% are distinct. The nominal p-value therefore overstates its precision by 2.74× — the denominator says 10,000 while the independent information is barely a third of that. The p-values come out too small, and cross the threshold.

This is the counterintuitive part: more resampling made the answer worse. The usual intuition — that more draws can only sharpen an estimate — does not hold for this null model.

So the specification itself was wrong: B = 10,000 is the wrong number for this kind of randomization, and following it manufactured false discoveries. It has been returned to 2,000 and the specification document corrected. (The bootstrap's 10,000 is unaffected; that is a different resampling procedure.)

This is the second instance of one class of error

Section 6 of the previous article documented a retracted intermediate conclusion: 16 significant cells, all produced by a broken null distribution (event clustering was destroyed, making the distribution 2.63× too narrow).

This time the mechanism is different — too few distinct realizations, inflating nominal precision — but the class is the same:

When the null distribution itself is built wrong, every number on the report looks normal, and the smoke test passes.

Both times. Neither was a data problem, a signal-definition problem, or look-ahead bias. Both times, the ruler was wrong.

And both times, what caught it was the same move: not a test, but measuring the ruler itself. The first time that meant measuring the width of the null distribution. This time it meant counting what fraction of the draws were distinct.

If one sentence survives from this article, let it be this: when your method produces a finding, run the same method over data that cannot contain one.

5. The shape of the result: weak, diffuse, unattributable

Having a shuffled control as an empirical null makes possible something a bare zero cannot do: check whether the real data shows any excess over the null at looser thresholds.

The real data does exceed the null. Thirty-five cells more at q ≤ 0.20, one hundred and ninety-two more at p ≤ 0.05. That is not nothing.

But it has to be read very carefully:

First, the threshold is frozen and I did not move it. The headline conclusion stands on the pre-registered q = 0.10. The table above is for understanding the shape of the result, not for picking a threshold — "change the cutoff and there's a finding" is precisely what this whole apparatus exists to prevent.

Second, the excess cannot be attributed to any single signal type. It does not concentrate in Fibonacci, or in moving averages, or in any one family or timeframe. It is diffuse.

Third, and most usefully: this shape is itself evidence that the test is not blind. If the machinery could see nothing at all, real and shuffled data would look identical at every threshold. They do not. So there is discriminating power — the signal is simply too weak to clear a line drawn in advance.

My reading of the shape: if something is there, it looks more like a very thin layer of predictability spread across everything than like any particular signal working. And how thin? Thin enough that the median hit rate is 50.2% and the median effect size is 0.30% — before any transaction costs.

6. How solid is this zero

A test that cannot detect anything and a world with nothing to detect produce identical reports. The previous article settled that with a measurement: inject a 1% effect, detection rate 100%.

That measurement was not re-run for this study, so I will not claim the same number. Three things can be said, at decreasing strength:

One: the sample is far larger than in the previous article. 1,779,160 events in the development tier, with the median reliable cell in the tens of thousands. The same effect size is easier to detect in a larger sample.

Two: but sample size does not translate directly into power. Indicator signals fire at more than two orders of magnitude the density of chart patterns, holding periods overlap heavily, and the effective sample size is far below the nominal one. Serial dependence is handled by the bootstrap and the uniform-shift randomization; the overlap itself still reduces power.

Three: and there is evidence this test runs conservative. The shuffled control's p-values have a median of 0.606 (a properly calibrated test would centre on 0.5), and only 3.02% fall below 0.05 against a nominal 5%. The direction is safe — it under-reports false positives — but it also means actual power is below nominal.

So the honest statement is: this zero holds, but it stands on softer ground than the previous one. That article had a measured power curve; this one does not. What it has instead is a very large sample and the indirect evidence in Section 5 that the machinery can see something, just not enough.

Turning that soft footing into a hard one requires re-running the effect-injection measurement on this signal set. That is the next thing to do — not something this article can claim.

7. What this does and does not establish

One: the scope is still limited, just wider than before. What was tested is 49 mechanically defined signal types, one frozen parameter set, on S&P 500 large caps, across three timeframes. Different parameters, different signal definitions, different markets or asset classes could all give different answers.

Two: usefulness still was not measured. Information content was. No transaction costs, no slippage, no position sizing, no risk management. The sentence from last time still applies: "no detectable information" and "loses money when traded" are different statements.

Three: the two articles together cover this much: nine chart pattern types plus 49 indicator and signal types plus multi-timeframe conditioning — 11,379 test cells, none of which cleared a pre-registered threshold. That is the widest claim I can currently make, and it is still some distance from "technical analysis does not work." That would be a proposition about how a person synthesizes judgement, and everything measured here is a single mechanical rule. Section 7 of the previous article is entirely about that distinction.

If you only quote one sentence

Forty-nine mechanically defined technical signals, on S&P 500 large caps between 2000 and 2017, show no detectable information at a pre-registered threshold once market drift, stock-specific characteristics, and same-day same-sector moves are controlled for.

Do not shorten it to "stochastics don't work" or "moving averages don't work" — what was tested is specific rules under specific definitions, not every use of those tools. And do not shorten it to "indicators can't make money" — this study did not measure whether anything makes money.

Problems with this study

1. Survivorship bias. yfinance lacks delisted stocks; only 674 of the 985-ticker universe have price data. The conclusion leans toward companies that survived, and that direction makes signals look better than they were.
2. Power was not measured (Section 6). The effect-injection test was not re-run for this signal set.
3. High signal density with heavy holding-period overlap, so effective sample size is below nominal.
4. Candlestick context definitions are subjective. A hammer and a hanging man are the same shape, separated by the net change over the preceding five closes. That definition is in the frozen config, but it is not the only reasonable one.
5. Fibonacci is tied to the ZigZag parameters (Section 1).
6. No transaction cost model.
7. Vendors restate historical data.

The final holdout remains unopened

Data from 2022 onward stays locked for the duration, openable exactly once, enforced in code. Zero significant cells in development means there is no candidate to carry forward, so that single opportunity was not spent.

The same precision as last time applies: signal detection ran across the full history (signals are a deterministic function of price and never touch forward returns); what is locked is the statistics layer. The real protection remains that the parameters were frozen before any statistics ran.

8. Why the 5 cells are the point

In hindsight this article contains two results.

One was expected: 49 signal types, 11,025 cells, none passing. It is consistent with the weak-form efficient market hypothesis. Unsurprising, and not especially interesting.

The other was not: before that zero, the same data, the same person, the same afternoon produced 5 significant findings, four of them Fibonacci. All false. And they appeared not because I cut a corner — the opposite. They appeared because I brought an implementation into line with its specification.

The lesson is not "specifications can be wrong." It is something more troublesome: you cannot avoid this class of error by working harder. Those 5 cells passed every check already in place. The smoke test passed. Parameters were frozen. There was no look-ahead. The multiple-testing correction was applied. They were generated inside the method.

The only thing that caught them was something strictly unnecessary — running the same method over data that could not possibly contain a finding, to see whether it would "find" something anyway.

That habit will never help you find a method that works. It only tells you, at the moment you think you have found one, that you have not.

In this field, that is the more valuable of the two.

---

Study window 1 January 2000 to 31 December 2017. Universe is point-in-time S&P 500 membership; daily, weekly, and monthly bars. Every figure is recomputed by the analysis pipeline from raw result files rather than transcribed by hand.