Bluesky Benchmaxxing

One simulated week of English Bluesky (Sep 18–25, 2026). Three losses, each at two timescales. Lower is better.

Current best

Like-count Poisson · how many likes
Poisson deviance of each post's like count against its expected count ÷ the group's usual likes · 0 = perfect · ≈ 0.4–0.6 = off by 2×
—
2-day: —

Scores over time · headline timescale

Likes scored once on the whole week; followers scored day by day.

Liker Brier Follower CRPS Like-count Poisson bar height = e−loss, so a lower loss is a taller bar (1.0 = perfect) · grey = not reported * faded = Liker Brier from before Nov 6, 2026, divided by each post's real likes: lower than now for the same forecast (naming no one 0.600 then, 1.12 now)

Scores over time · 2-day windows

The same three losses on every 2-day stretch of the week (days 1–2, 2–3, … 6–7), averaged over the six stretches.

All submissions, best first

One week of English Bluesky, Sep 18–25, 2026 (days 1–7 are UTC days, day 1 = Sep 18). Everything the held-out accounts did that week and real news-account posts are copied into the simulation at the real moment; the rest of the network is simulated, with no required number of runs: each loss scores the chances and counts an entry reports. Older entries report averages over 8 runs, which adds some sampling noise. Three losses: who liked each post (Loss 1), how many followers each account gained and lost (Loss 2), and how many likes each post got (Loss 3). Each is reported at its headline timescale and again on 2-day windows, so every entry has six numbers. Lower is better, 0 is perfect, none is capped.

Loss 1 — Liker Brier (likes)

For every held-out post: did each person like it, and how sure was the simulation? Scored once per post at the end of the week.

per post
\( \displaystyle S_q \;=\; \dfrac{1}{c_g} \sum_{u \in U_q} \big(p_{u,q} - y_{u,q}\big)^2 \)
overall
\( \displaystyle L_1 \;=\; \dfrac{1}{|G|}\sum_{g \in G}\; \dfrac{1}{|A_g|}\sum_{a \in A_g}\; \dfrac{1}{|Q_a|}\sum_{q \in Q_a} S_q \)
\(q\)
a held-out post (replies count too); \(Q_a\) the posts of account \(a\)
\(U_q\)
everyone who liked \(q\) in reality or has \(p_{u,q} > 0\). Anyone else contributes exactly 0
\(p_{u,q}\)
the simulation's probability that person \(u\) likes \(q\), any number from 0 to 1
\(y_{u,q}\)
1 if \(u\) really liked \(q\), else 0
\((p-y)^2\)
Brier score for that person. Misses \((1-p)^2\) over real likers, false alarms \(p^2\) over the rest
\(c_g\)
the usual likes per post before the week of the group \(g\) that \(q\)'s account is in: the average of \(c_a\) over the group's accounts. \(c_a\) is likes received ÷ posts made (replies count) over Aug 28–Sep 17, 2026, floored at 1; 1 if the account made no posts then
\(G,\;A_g\)
the sampling groups and the held-out accounts in each; every group weighs the same

Reading it: 0 = every liker right with full certainty · 1.12 = named no one · 2.25 = right count, all the wrong people.

Why squared: plain distance \(|p-y|\) is minimised by rounding every person to 0 or 1, which throws away the uncertainty. Squaring charges a full 1 for confidently wrong and 0.09 for saying 30% to someone who didn't like it, so the best report is the true chance. Why divide by \(c_g\): each group is its own test of the simulation, and groups of large accounts would otherwise outweigh the rest. One number per group puts the tests on one scale and divides nothing away inside a group, so missing every liker on a 2,000-like post costs 400 times as much as on a 5-like post. \(c_g\) comes from before the week and is the same for every entry, so the best report stays the true chance, apart from a small effect of groups being partly defined by the week's likes.

Loss 2 — Follower CRPS (followers)

For every held-out account and day: how many followers it gained and lost, split by the follower's size. The exact person is not scored (under 1% per person, too rare to score); the kind of account is.

log counts
\( \displaystyle y \;=\; \log(1 + \text{real count}) \qquad x_k \;=\; \log(1 + \text{forecast count } k) \)
per series
\( \displaystyle \operatorname{CRPS}(x, y) \;=\; \dfrac{1}{K}\sum_{k=1}^{K} |x_k - y| \;-\; \dfrac{1}{2K(K-1)} \sum_{k \neq j} |x_k - x_j| \)
per day
\( \displaystyle D_{a,d} \;=\; \sum_{s \in S} \operatorname{CRPS}\big(x^{(s)}_{a,d},\, y^{(s)}_{a,d}\big) \)
overall
\( \displaystyle L_2 \;=\; \dfrac{1}{|G|}\sum_{g \in G}\; \dfrac{1}{|A_g|}\sum_{a \in A_g}\; \dfrac{1}{7}\sum_{d=1}^{7} D_{a,d} \)
\(S\)
six series per account per day: new followers from accounts with <1K, 1K–10K, >10K followers, and unfollows in the same three bands (size read at the start of the week)
\(y,\;x_k\)
real and simulated counts on a log scale, so a distance of 0.69 means "off by 2×" whether the day had 10 followers or 1,000
first term
average distance of the forecast counts from reality
second term
half the average distance between those counts: the credit for honest spread
\(K(K-1)\)
pairs over \(k \neq j\), the "fair" form, unbiased at any number of counts \(K \ge 2\); a single count scores plain \(|x-y|\)
\(d\)
day of the week, 1–7; \(D_{a,d}\) the six CRPS values added

Reading it: stays in log units · 0 = every daily count exactly right · 0.69 ≈ typically off by 2× · 2.3 ≈ off by 10×. When all counts agree, CRPS is plain \(|x-y|\).

Why subtract half the spread: the first term alone is minimised by collapsing all counts onto one value. Subtracting half the spread makes a forecast shaped like reality the best one; subtracting all of it would let a scattered forecast score below zero. Why log, not percent: percent error is lopsided and caps at 1, so a 10× miss and a 1,000× miss look the same; log keeps counting, which is where the growth accounts live.

Loss 3 — Like-count Poisson (likes, how many)

For every held-out post: how many likes did it get, against the simulation's expected count? Loss 1 checks who liked; this checks how many. Scored once per post at the end of the week.

floor
\( \displaystyle \tilde\lambda_q \;=\; \begin{cases} \max\!\big(\lambda_q,\, \tfrac{1}{16}\big) & y_q > 0 \\ \lambda_q & y_q = 0 \end{cases} \)
per post
\( \displaystyle D_q \;=\; 2\Big(\, y_q \log \dfrac{y_q}{\tilde\lambda_q} \;-\; y_q \;+\; \tilde\lambda_q \Big) \qquad (0 \log 0 = 0) \)
overall
\( \displaystyle L_3 \;=\; \dfrac{1}{|G|}\sum_{g \in G}\; \dfrac{1}{|A_g|}\sum_{a \in A_g}\; \dfrac{1}{|Q_a|}\sum_{q \in Q_a} \dfrac{D_q}{c_g} \)
\(y_q\)
likes \(q\) really got by the end of the week
\(\lambda_q\)
the simulation's expected like count, any number \(\ge 0\)
\(\tilde\lambda_q\)
\(\lambda_q\) with a floor of \(\tfrac{1}{16}\), used only when the post really got likes. A liked post forecast at zero pays a large but finite price; a correct zero still costs nothing
\(D_q\)
Poisson deviance: twice the gap in Poisson log-likelihood between the simulation's rate and the perfect rate \(y_q\). It is 0 only when \(\lambda_q = y_q\); for a post with no real likes it is \(2\lambda_q\)
\(c_g,\;G,\;A_g,\;Q_a\)
as in Loss 1: the group's usual likes per post; groups weigh the same, accounts within a group weigh the same, posts within an account weigh the same

Reading it: 0 = every post's expected count exactly right · for a post at the group's usual level, ≈ 0.61 = predicted 2× too many, ≈ 0.39 = predicted half as many · ≈ 13 = 10× too many, ≈ 2.8 = a tenth as many.

Why Poisson deviance: it is the Poisson log-likelihood shifted so a perfect count scores 0, and its best report is the true expected count, so a simulator cannot gain by shading its counts up or down. It is the count companion to Loss 1: a simulation that gives a post the right number of likes from the wrong people does well here and badly there. Why divide by \(c_g\): deviance grows with the size of the count, so groups of large accounts would otherwise outweigh the rest; inside a group, a bad miss on a big post still costs more than on a small one. As in Loss 1, the best report stays the true expected count, apart from the same small effect.

The 2-day timescale

Every loss is also scored on 2-day windows of the same forecast and averaged over the windows. Seven days give six windows, one starting on each of days 1–6, so no day is dropped: \(W_1\) = days 1–2, \(W_2\) = days 2–3, …, \(W_6\) = days 6–7. Days 2–6 fall in two windows, days 1 and 7 in one. The headline likes losses only look at the week's totals; the 2-day versions also check when the likes came. The headline follower loss looks at single days; the 2-day version forgives a follow landing a day early or late but still checks the 2-day totals.

Loss 1, per window
\( \displaystyle S_{q,w} \;=\; \dfrac{1}{c_g} \sum_{u \in U_{q,w}} \big(p_{u,q,w} - y_{u,q,w}\big)^2 \)
Loss 1, 2-day
\( \displaystyle L_1^{\text{2d}} \;=\; \dfrac{1}{|G|}\sum_{g \in G}\; \dfrac{1}{|A_g|}\sum_{a \in A_g}\; \dfrac{1}{|W_a|}\sum_{w \in W_a}\; \dfrac{1}{|Q_{a,w}|}\sum_{q \in Q_{a,w}} S_{q,w} \)
Loss 2, per window
\( \displaystyle D_{a,w} \;=\; \sum_{s \in S} \operatorname{CRPS}\big(x^{(s)}_{a,w},\, y^{(s)}_{a,w}\big) \)
Loss 2, 2-day
\( \displaystyle L_2^{\text{2d}} \;=\; \dfrac{1}{|G|}\sum_{g \in G}\; \dfrac{1}{|A_g|}\sum_{a \in A_g}\; \dfrac{1}{6}\sum_{w=1}^{6} D_{a,w} \)
Loss 3, 2-day
\( \displaystyle L_3^{\text{2d}} \;=\; \dfrac{1}{|G|}\sum_{g \in G}\; \dfrac{1}{|A_g|}\sum_{a \in A_g}\; \dfrac{1}{|W_a|}\sum_{w \in W_a} \dfrac{1}{|Q_{a,w}|}\sum_{q \in Q_{a,w}} \dfrac{D_{q,w}}{c_g} \)
\(W_w\)
window \(w\) = days \(w\) and \(w{+}1\), \(w = 1,\dots,6\)
\(Q_{a,w}\)
account \(a\)'s held-out posts made before the end of \(W_w\) (posts that exist during the window, older ones included)
\(W_a\)
the windows in which account \(a\) has at least one such post
likes, 2-day
only likes made inside \(W_w\) count: \(y_{u,q,w}\), \(p_{u,q,w}\) and \(U_{q,w}\) are Loss 1's quantities restricted to likes in the window; \(D_{q,w}\) is Loss 3's deviance on the window's real and simulated like counts, same floor; both use the same \(c_g\)
followers, 2-day
each of the six series is summed over the window's two days, then \(\log(1+\cdot)\) and fair CRPS exactly as in Loss 2
empty windows
a post with no likes in a window, real or simulated, scores 0 there and still counts in the average, the same for every simulation

Bars on the leaderboard

Bar height is \(e^{-\text{loss}}\): 1.0 for perfect, 0.5 at a loss of 0.69, 0.25 at 1.39. Entries are ordered by the average of \(e^{-\text{loss}}\) over all six numbers, \(\tfrac{1}{6}\sum_{i=1}^{3}\big(e^{-L_i} + e^{-L_i^{\text{2d}}}\big)\), highest first. Entries submitted before Sep 30, 2026 have only \(L_1\) and \(L_2\) at the headline timescale, with \(L_1\) divided by each post's real likes; they are listed after the complete entries, ordered by \(\tfrac{1}{2}\big(e^{-L_1} + e^{-L_2}\big)\). Their \(L_1\) is marked * and drawn faded, and left out of the best Liker Brier; naming no one scores 0.600 that way and 1.12 now.