Who's to Blame for This One: Assigning Blame in a Mahjong Game Log

Origin Link to heading

After dealing into a fourth place, I wanted to curse the RNG, but honestly I wasn’t sure: was this hand simply unplayable, or did I just play it badly? The trouble with mahjong is that there’s a thick layer of variance between the outcome and the process — a single game can’t settle anything, but a person only ever gets to play one game at a time.

誰だ / dareda exists to solve exactly this. It pulls the wall out of a game log, has an AI replay the same hand dozens of times under the exact same deal, and gives you a distribution of finishing ranks:

座2 同水平自打分布  n=30  平均顺位 2.30
    1位  20.0%  ████████
    2位  40.0%  ████████████████
    3位  30.0%  ████████████
    4位  10.0%  ████

实际 4 位；同水平自打期望 2.30 位。
  判断：偏背 —— 你的同水平只有 10% 会打到 4 位或更差

In plain terms: replaying this hand at your own skill level, you’d average 2.3rd place, and only 10% of the time would you end up dealing into fourth like you actually did. This hand was bad luck, not bad play. (Rage quietly justified.)

What It Actually Runs Link to heading

Under the hood is Mortal (an open-source riichi mahjong AI) and its libriichi engine. dareda patches in a way to pin down the wall convention so the same hand can be replayed over and over. Data is scraped from Tenhou/Majsoul packet captures, and the very first thing it does after parsing is verify: check, game by game, that the reconstructed deal is identical to the original — if it isn’t, nothing downstream is allowed to run. Every conclusion after that rests on the premise that “we’re replaying the same hand.”

On top of that sit three distinct experiments, and it’s easy to blur them together, so here’s the breakdown:

“Skill level” isn’t a guess: it’s measured by running an EV-loss check against each player’s real decisions in the original hand (the expected gap versus Mortal’s optimal play), then inverting that into a sampling temperature — higher temperature means more randomness, i.e. weaker play. So the opponents in self-luck aren’t three full-strength AIs, but three players who are “exactly as mediocre as they were that night.”

Finally, diagnose compresses all of this into one identity:

实际名次 − 2.5 = (牌与座次) + (打法) + (运气残差)

The key is that you need two baselines. The expected value in self-luck already bakes in both the tiles and the play, so only by also running an “all four seats at equal skill” baseline can you separate “how good was this hand” from “how many places did your play earn you.” Using self-luck alone as a proxy for “were the tiles good” would fold play quality into the tile quality. The three terms sum to exactly the left-hand side by construction — the test enforces this identity — and precisely because it’s an identity, the sum itself is not independent evidence.

A Real Report Link to heading

Below is a full run: an east-south match played out to south-4 (K = 9 hands), hero seated at position 2, ending with 4,300 points, in 4th place. The same game log was run through two different Mortal model weights (150k and 298k) just to see whether the conclusions held up.

The one-line verdict: good tiles, mediocre play, and this hand’s luck really was bad.

• Tiles-and-seat +0.33, play −0.13, luck −1.70, summing to −1.50 — dropping from a neutral 2.5 to an actual 4th place, with luck accounting for the vast majority of it.
• Switching to the 298k weights, the same −1.50 splits into +0.40 / −0.03 / −1.87 — the verdict is still “unlucky,” and the probability of landing in 4th is still 10%. Both sets of weights agree on the direction, so this conclusion holds.
• An even more humbling detail: hero’s EV loss was the 2nd-lowest of the four seats (and the lowest with the 298k weights) — while the seat that played the sloppiest game, seat 0, with an EV loss of 0.488, ended up in 3rd.

The report also has a section specifically on discounts that should be applied, e.g. the median trajectory fidelity of counterfactual was only 51% (the opponents derail quickly), so the specific numbers in that section shouldn’t be read as point estimates; the full-mortal tier of montecarlo --sensitivity is a deterministic single replay, so showing 100% 4th place means “that one deterministic trajectory landed in 4th,” not “100% confidence.” What most needs spelling out in a report like this is usually exactly this kind of caveat — not the big headline numbers.

At the end there’s a full time and memory ledger: every command was run through run_step.py, with per-process rusage sampled via os.wait4. The full set of 11 steps took 577.6 s (9.6 minutes) altogether, with peak resident memory of 440 MB for a single step.

(If it feels cramped inside the iframe, open this page on its own.)

A Few Changes, and the Reasoning Behind Them Link to heading

The tool grew organically over time, and a few changes are worth calling out on their own, because they’re all about the difference between “the number computes” and “the number is right.”

Luck can’t be defined by a mean difference. The original rule was: flag as “unlucky/lucky” if actual rank minus expected rank (at your own skill level) exceeded ±0.2 places. The problem is the self-play distribution is often bimodal, so the mean lands in a region nobody actually lands in — which meant a routine 4th place often got read as “catastrophically unlucky.” It was later changed to a percentile: luck_q = the median rank of the actual result within your own distribution, i.e. “what fraction of your own replays did you beat.” ≤0.25 counts as unlucky, ≥0.75 as lucky, and the middle is just tough luck either way. The luck star rating is now drawn from this same quantity, so it no longer disagrees with the verdict text.

Removed the “swap in a full-strength AI” column. There used to be a column showing “you’re swapped for full-strength Mortal, opponents unchanged,” meant to represent a ceiling on skill. In practice it wasn’t a ceiling: on a fixed wall, Mortal actually played more aggressively — winning more, dealing in more — and the average rank came out roughly the same as the real skill level. A column that misleads users with its numbers should be removed even if the computation itself is correct — as a bonus, it also cut compute by a third.

Strength measurements now skip forced decisions. Draws-and-discards after declaring riichi, or responses where only one legal move exists, aren’t choices. Counting them toward the agreement rate is free score-padding; counting them toward EV loss dilutes it with zero-loss decisions. After excluding them, hero’s agreement rate dropped from 68% to 66%, and EV loss rose from 0.337 to 0.362 — still the best at the table — while the opponent who re-declared riichi had their EV loss rise from 0.656 to 0.741, stripping out an inflated number. The trigger was a very specific grievance: the 18,000-point deal-in from the hand where the user dealt into fourth was a forced draw-and-discard after riichi, and shouldn’t have been counted against the player’s decision-making.

Drew out the distribution. The label “dealt into 4th = bad luck” looks like a cop-out on its own. Adding a bar chart of rank probabilities to the results made the logic self-evident — you can literally see how thin that 10% slice is. Making the black box transparent turned out to matter more than computing a few extra numbers.

Cut runtime from 26 minutes to 9.6 minutes, through three structural fixes — none of them “tuning a constant”: montecarlo’s cross-trajectory loop used to be a bare for loop (CPU usage was only 106%, while every other command ran around 330%); wrapping it in an off-the-shelf process pool took a single command from 351s to 135s, with each trajectory seeded by base_seed + i, and the distributions for jobs=1 and jobs=4 matching term-for-term; the full-mortal tier of --sensitivity has no randomness source, so running 20 replicas was computing the same single game 20 times — collapsed to n=1; self-luck was already strictly a subset of what diagnose computes (that same Monte Carlo run already computes the distribution for all four seats, it just only kept hero’s copy), so simply keeping all four was free; adding --with-selfluck on top meant recomputing everything a second time — running both accounted for about 17% of the total analysis time.

West round got blocked by a single line of code for a long while. If no one reaches 30,000 by the end of south-4, the match rolls into west-1, but KyokuReplay’s upper bound on kyoku was hardcoded to 8, so anything past that reported “kyoku out of range: 8.” Digging in, it turned out west round was actually already supported end to end — Board already derived bakaze = E + kyoku/4, and the observation encoding already clamped west-round hands onto south-round conventions. In other words, west round had been blocked by that one line the whole time, not genuinely unsupported. Bumping the bound to 12 fixed it.

There’s also a batch of purely engineering gotchas, written up in the README’s “Troubleshooting” section: Windows’ 260-character path limit can cause pip install torch to stop partway through the install, then throw a completely unrelated No module named 'torchgen' error afterward; building the extension without setting PYO3_PYTHON links it against the system Python, which throws symbol not found when imported inside a venv — and the error message gives no hint at all that the interpreter is linked wrong. The packet-capture script was also switched from “press D to download” to a floating button in the bottom-right corner — the old raw hotkey would misfire whenever you typed a “d” into any text field inside Majsoul.

Three Caveats Link to heading

• Only meant for reviewing games that have already finished. Do not use it on an ongoing match.
• Capturing a game log requires modifying the Majsoul client in your browser, which may violate the terms of service and carries a risk of account suspension. Use at your own discretion — a throwaway account is recommended.
• The project ships no AI model and provides no model weights (the install script fetches them from a third party automatically). Licensed AGPL-3.0.

One more thing worth mentioning — the line in the report that says “the sloppiest player at the table was seat 0, and he came in 3rd” — that’s probably the most honest thing this tool is good for: it will just as readily tell you that you genuinely played worse than the person across the table, and luck simply carried you that night.