Ranking systems and the bucket test

A screen answers yes or no. A ranking answers "how good, compared with everyone else", for every stock at once. Each stock gets a score from 0 to 100, 100 best, and a strategy buys from the top. Ranking is how most quantitative stock selection works, from Greenblatt's Magic Formula to institutional multi-factor models, and it is the core of Portfolio123, which this design follows closely.

Why rank instead of thresholds

A screen with five thresholds treats a stock that misses one of them by a hair the same as one that misses all five, and a stock that clears every threshold by a mile the same as one that barely clears them. The thresholds also interact: tighten one and the survivors change in ways that are hard to predict, and the number of survivors swings from month to month. A ranking uses every stock's full position on every factor, lets a strength on one offset a weakness on another, and always produces exactly the number of names you want to hold. Thresholds still have a job: hard exclusions (no stocks under $5, no financials) belong in the universe or the buy rules.

Percentile ranks and direction

A factor node takes one numeric field and turns it into a percentile among the stocks being ranked. You choose the direction: higher is better (ROIC, margins, momentum) or lower is better (P/E, leverage, volatility). Ties share their average position, the best stock scores 100 and the worst $100/n$:

Percentiles throw away the size of the differences and keep only the order. That is a feature: one company with a P/E of 400 cannot distort the scores of everyone else, and a P/E percentile and a ROIC percentile live on the same scale, so they can be averaged.

Composite trees and weights

A ranking system is a tree. Its leaves are factors; its branches are composites that combine their children. A composite takes the weighted mean of its children's scores and re-ranks that mean to a percentile again:

Weights are relative among siblings: 60 and 40 mean the same as 3 and 2, and all zero means equal. The effective weight of a leaf is the product of the shares down the tree: a factor with half its composite, in a composite with 40% of the root, carries 20% of the final rank. The re-rank matters because averaging pulls scores towards the middle (a stock is rarely best on everything), so without it a composite's scores would bunch between 30 and 70. A composite can instead keep the raw weighted mean (summation); the Core systems do, because that reproduces the factor composites on the screener exactly.

Scope: universe, sector or industry

Each factor is ranked within a scope. Universe compares every stock with every other. Sector and industry compare a company only with its peers, the same idea as the relative fields in screening. Use the narrow scopes for anything whose normal level differs by industry: valuation multiples, margins, asset turnover, leverage. Use the universe for things that mean the same everywhere: price momentum, volatility, growth (arguably). A group with fewer than 3 valid values, or a company without a group, falls back to the universe percentile; the run reports how many stocks fell back per factor, and a company's rank breakdown says which scope was used.

Conditional nodes

A conditional node routes each stock to one of two branches by a screener condition, for example "sector is Financials": banks are then scored with price to book and return on equity, everyone else with EV/EBITDA and ROIC. Each branch is ranked only among the stocks routed to it, and the two are re-ranked together. A condition whose field is missing counts as false.

NA policies: what a missing value scores

Missing data is common: no P/E for a loss-maker, no dividend growth for a company that pays none. How a missing value scores changes results more than most people expect; Portfolio123 measured 5 to 8 points of annual return from this one switch. Stoxa has three policies, set per system and overridable per run.

• Negative (default for new systems): missing values tie just below the worst valid stock. A stock is punished for missing data, sharply when few are missing, mildly when many are.
• Neutral: missing values get the midpoint of the valid scores, $100\,(k+1)/2k$ for $k$ valid values, so they reach neither the top nor the bottom. Use it when you will also look at the bottom of the ranking.
• Skip: a missing factor drops out of its parent's average and the other children's weights are rescaled; a stock with no valid input at all gets no rank. This is how the screener's factor composites work.

The bucket test: does the ranking work?

A ranking is a claim that higher-ranked stocks do better. The rank performance test checks it directly. On every rebalance date it ranks the universe, sorts the stocks into equal-sized buckets (deciles by default, bucket 1 the lowest ranks) and holds each bucket equal-weighted until the next date, on point-in-time data and with dividends. Then it reports each bucket's annualised return, its excess over the benchmark, its volatility and Sharpe ratio, and statistics across buckets:

• the Spearman correlation between bucket number and annual return (1 = every bucket beats the one below it),
• the slope of annual return on bucket number (return per step up the ranking) and the top-minus-bottom spread,
• all of these for the whole period and separately for each half.

A ranking that works shows a ladder: returns rise from bucket 1 to bucket 10, not perfectly but clearly, with a positive slope in both halves. A ranking that only works at the top (bucket 10 strong, the rest flat) is fragile: the result rests on a few dozen stocks. Noise looks like a jagged profile with a Spearman near zero, or a ladder in one half that disappears in the other. With ten buckets, a perfect Spearman of 1 arises by chance in 1 of 3.6 million random orderings, but a Spearman of 0.5 is not rare at all; the halves are the first, cheapest check against luck.

The trap: tuning weights to the past

Every time you change a weight, look at the bucket test and keep the change because the result improved, you spend some of the test's ability to tell you anything. Try fifty variations and the best one will look excellent by chance alone. Harvey, Liu and Zhu counted over 300 published factors and argued that a new one should clear a t-statistic of about 3, not the usual 2; Bailey and López de Prado showed how quickly the probability of backtest overfitting rises with the number of trials. Practical defences: decide the factors from an economic reason before testing, keep weights round (equal, or 2:1), prefer few factors, check both halves, and treat the period after you finished tuning as the only real test.

The eight built-in systems

The built-ins are read-only templates; copy one to change it.

• Core: Value. P/E, price to free cash flow, EV/EBITDA, price to sales and price to book, all lower is better, equal weights, universe scope. Cheapness measured five ways, so no single accounting quirk dominates.
• Core: Quality. ROIC, ROE, gross margin and free-cash-flow margin (higher is better) and net debt to EBITDA (lower). Profitable, cash-generative, conservatively financed businesses, the ingredients of Asness, Frazzini and Pedersen's quality factor and Novy-Marx's gross profitability.
• Core: Momentum. 6- and 12-month price returns, higher is better. Jegadeesh and Titman's medium-term momentum, without the usual one-month skip.
• Core: Growth. Revenue growth and EPS growth, higher is better. Historical growth only; there are no analyst estimates in Stoxa.
• Core: Low Risk. One-year volatility and net debt to EBITDA (lower is better) and interest coverage (higher). Price risk and balance-sheet risk together.
• Core: Combination. The five composites above with equal weight. No single style has to work every year; blending styles that are not highly correlated smooths the long droughts each one has. Its scores equal the screener's composite_score.
• Core: Value (industry-relative). The same five multiples ranked within the industry, re-ranked, NA policy negative. Finds the cheap company in every industry instead of the cheapest industries.
• Quality at a Reasonable Price. The Quality composite at 60% and the Value composite at 40%, both re-ranked, NA policy negative. Good businesses first, without paying any price for them.

The six Core systems use universe scope, equal weights, summation and the skip policy, which is what makes them reproduce the screener's factor scores to floating point.