One simulated week of English Bluesky (Sep 18–25, 2026, 8 futures). Two losses, lower is better.
One week of English Bluesky, Sep 18–25, 2026. Everything the held-out accounts did that week and real news-account posts are copied into the simulation at the real moment; the rest of the network is simulated, \(K = 8\) times with different seeds. Both losses are proper scores: a simulator does best by reporting its true uncertainty. Lower is better, 0 is perfect, neither is capped.
Loss 1 — Liker Brier (likes)
For every held-out post: did each person like it, and how sure was the simulation? Scored once per post at the end of the week.
per person
\( \displaystyle p_{u,q} \;=\; \dfrac{1}{K}\sum_{k=1}^{K} \mathbf{1}\big[\,u \text{ liked } q \text{ in future } k\,\big] \)
per post
\( \displaystyle S_q \;=\; \dfrac{1}{\max(L_q,\,1)} \sum_{u \in U_q} \big(p_{u,q} - y_{u,q}\big)^2 \)
overall
\( \displaystyle L_1 \;=\; \dfrac{1}{|G|}\sum_{g \in G}\; \dfrac{1}{|A_g|}\sum_{a \in A_g}\; \dfrac{1}{|Q_a|}\sum_{q \in Q_a} S_q \)
- \(q\)
- a held-out post (replies count too); \(Q_a\) the posts of account \(a\)
- \(U_q\)
- everyone who liked \(q\) in reality or in at least one future. Anyone else contributes exactly 0
- \(p_{u,q}\)
- share of the 8 futures in which person \(u\) liked \(q\): the simulation's probability, in eighths
- \(y_{u,q}\)
- 1 if \(u\) really liked \(q\), else 0
- \((p-y)^2\)
- Brier score for that person. Misses \((1-p)^2\) over real likers, false alarms \(p^2\) over the rest
- \(L_q\)
- the post's real like count; \(\max(L_q,1)\) only avoids dividing by zero
- \(G,\;A_g\)
- the sampling groups and the held-out accounts in each; every group weighs the same
Reading it: 0 = every liker right in every future · 1 = produced no likes at all · 2 = right count, all the wrong people.
Why squared: plain distance \(|p-y|\) is minimised by rounding every person to 0 or 1, which throws away the uncertainty the futures exist to capture. Squaring charges a full 1 for confidently wrong and 0.09 for saying 30% to someone who didn't like it, so the best report is the true chance. Why divide by \(L_q\): otherwise one 2,000-like post outweighs 400 five-like posts. Dividing by reality only (never by anything the simulation controls) keeps it ungameable.
Loss 2 — Follower CRPS (followers)
For every held-out account and day: how many followers it gained and lost, split by the follower's size. The exact person is not scored (under 1% per person, 8 futures can't see it); the kind of account is.
log counts
\( \displaystyle y \;=\; \log(1 + \text{real count}) \qquad x_k \;=\; \log(1 + \text{count in future } k) \)
per series
\( \displaystyle \operatorname{CRPS}(x, y) \;=\; \dfrac{1}{K}\sum_{k=1}^{K} |x_k - y| \;-\; \dfrac{1}{2K(K-1)} \sum_{k \neq j} |x_k - x_j| \)
per day
\( \displaystyle D_{a,d} \;=\; \sum_{s \in S} \operatorname{CRPS}\big(x^{(s)}_{a,d},\, y^{(s)}_{a,d}\big) \)
overall
\( \displaystyle L_2 \;=\; \dfrac{1}{|G|}\sum_{g \in G}\; \dfrac{1}{|A_g|}\sum_{a \in A_g}\; \dfrac{1}{7}\sum_{d=1}^{7} D_{a,d} \)
- \(S\)
- six series per account per day: new followers from accounts with <1K, 1K–10K, >10K followers, and unfollows in the same three bands (size read at the start of the week)
- \(y,\;x_k\)
- real and simulated counts on a log scale, so a distance of 0.69 means "off by 2×" whether the day had 10 followers or 1,000
- first term
- average distance of the 8 futures from reality
- second term
- half the average distance between futures: the credit for honest spread
- \(K(K-1)\)
- pairs over \(k \neq j\), the "fair" form, unbiased at any number of futures
- \(d\)
- day of the week, 1–7; \(D_{a,d}\) the six CRPS values added
Reading it: stays in log units · 0 = every daily count exactly right · 0.69 ≈ typically off by 2× · 2.3 ≈ off by 10×. When all 8 futures agree, CRPS is plain \(|x-y|\).
Why subtract half the spread: the first term alone is minimised by collapsing all 8 futures onto one value. Subtracting half the spread makes a forecast shaped like reality the best one; subtracting all of it would let a scattered forecast score below zero. Why log, not percent: percent error is lopsided and caps at 1, so a 10× miss and a 1,000× miss look the same; log keeps counting, which is where the growth accounts live.
Bars on the leaderboard
Bar height is \(e^{-\text{loss}}\): 1.0 for perfect, 0.5 at a loss of 0.69, 0.25 at 1.39. Entries are ordered by \(\tfrac{1}{2}\big(e^{-L_1} + e^{-L_2}\big)\), highest first.