so i was working on a game. the game has haystacks sometimes. if you set fire to the haystack, there’s a one in six chance that when it burns away, it turns out to contain a treasure chest! you can get some loot
i implement this. much later, QA reports a bug: the chance of a treasure chest spawning isn’t one in six. uhhh ok. i have a look and can’t repro. i look at the code and it’s the same “random chance” code used everywhere, so if there was anything wrong with it, this wouldn’t be where it came up. i have a bit more of a think and then send it back, CNR.
the bug bounces back and forth for 2 weeks as the tester keeps saying it’s still not one in six. i am down a few rabbit holes thinking about, could we be not spawning the chest even though the result comes in true? is some other class destroying chests? are some levels containing instances of the haystack that are busted in some way? much to think about
eventually i’m like, ok, what exactly are you seeing in your repro. old mate says “i burn 6 haystacks. sometimes there’s no treasure chest at all. sometimes i burn 6 haystacks and get more than one treasure chest. so that’s not a 1 in 6 chance”
i now have a new, much more challenging task: explain this without embarassing anyone
in reply to @joewintergreen’s post:
I knew where this was going basically from the go but the end still hit me like a throat punch
I guessed by paragraph 2 but was like “No. It can’t be.” I was wrong.
thinking about how some games (don’t remember which) will intentionally fudge their ‘random’ elements to cater to this perception of probability (e.g. if something is a 1 in 6 chance, and the first five rolls are false, then there’s a 100% chance of the sixth roll being true). wondering if it was all maybe a big mistake.
i could definitely imagine being exasperated enough to do this just to not get assigned “bugs” like this
That’s basically how Dota 2’s Pseudo Random Distribution works. Increment some percent chance value until it procs or is over 100% (forcing the proc) and then reset. It’s a cool system.
Huh. That’s pretty much exactly how I’m implementing weapon malfunctions of all things in a game I’m working on. Shouldn’t be surprised it’s been done before, honestly!
(logic of course being: a bunch of statistically improbable jams back to back feels unfair and frustrating, but they still should happen frequently enough to be a problem that makes decisions like “should i really cheap out on 50 year old surplus ammo” something to think about)
Remember to include a minimum no-jam period after each double-jam if you want to avoid long statistically improbable bad luck streaks. You both want to ensure the jam happens eventually (the cost of cheap ammo) and avoid too much streakiness, but don’t want to eliminate bad streaks – so within each ammo quality I’d recommend setting up a maximum jam streak you want it to be allowed have, and if it hits that, protecting it from further jams until the chances feel more evened out, then starting the chance cycle on it again. For example: “questionable ex-Soviet surplus 7.62,” jam chance 10% (builds up to guaranteed 10th shot jam), streak ceiling 3/grace period 5 (if it jams 3x in a row, the next 5 shots will go to 0% jam, then the 6th after the streak will return to base jam chance).
I’m sure Path of Exile does that for evasion chance (i remember reading about that sometime in the past).
It feels extra fair in an environment where players are trying to min-max their build and compare different defensive measures (max life, resistances, armor, block, evasion etc.) and if any was pure rng prone to unlucky streaks, minmaxers would just not use it at all. Wouldn’t be fun if every meta build was life+armor and 1/3 of skill tree was gathering dust.
different domain, but this reminds me of (on a database engine) having to have the conversation over and over again that “not enough memory to execute query” means… that there’s not enough memory to execute the query. no, it is not a bug that you can’t do a group-by on 2 billion records on a docker instance with less RAM than an average Minecraft server
Tetris has a well thought out reason for that: it ensures the game never enters an unwinnable state (a bunch of S pieces, then a bunch of Z pieces, then a bunch of S pieces…). That’s not necessarily a concern for random drops in an action game.
Yup, it’s like shuffling a deck with 6 times x cards (smallest 6-card deck means it’s not possible to get the same result more than twice in a row) instead of using dice.
I know that at least one digital version of Catan lets you pick between both styles – roll based or shuffled results. Which is super nice to see, because i kinda prefer dice-based in that game and being forced to use the other one would feel off to me.
deflated sigh of resignation
reminded of a bit in the game “Stories: The Path of Destinies” where in one path our protagonist gets his hands on a doomsday weapon which he’s warned has a “1 in 128 chance” of destroying the world if used
having no grasp of probability, he blithely marches on the enemy wielding it freely, claiming that he’ll be done with it long before he “enters the danger zone”
more than one person chews him out on his fallacious logic (it’s a 1 in 128 chance every time he uses it; it doesn’t work like a deck of cards), and sure enough, the last time he uses it is the last time he uses it; it rips a hole in reality and brings the world to an end
“1 in 128. What are the odds?”
yeah…
i bet you’re right
Or practical examples calling out other possible expected behavior:
Ah, I see how “1 in 6” was ambiguous and could have been documented better. We don’t enforce this event happening once each six tries* or periodically (i.e. “once every six hours”) so it should ideally behave like rolling a fair 6-sided die (so, rarely, you will roll two 1s in a row, or you will not get a 1 in 12 rolls. These should seem both rare but possible to the player)
* presumably this is the kind of thing where game developers are sometimes obliged to do an “iTunes shuffle random” to feel more correct to players?
“by design”, close task
as infuriating as this is! (it is) there’s probably a fair bit of milage in conversations around RNG and so on in games and how they’re presented to the player.
FOR EXAMPLE- monster rancher 2 (or one, if you lived in the PAL badlands) has a ‘bit’ where in battles the to-hit chance is displayed as a percentage onscreen. given that attack damage ranges from ‘devastating’ to ‘tickled’ it can feel extremely bullshit for your monster to miss five attacks in a row with a 90% hit chance for the enemy monster to one hit KO you with an attack with a 10% hit chance. Even if it is statistically possible.
I remember a Jake Solomon interview where he referenced the exact same thing. Players in XCOM would take a 50/50 shot, miss and expect the next 50/50 shot to hit. And were very frustrated when it didn’t.
Firaxis ended up implementing an under the hood system where the more misses a player gets in a row the higher chance their next shot will hit, unbeknownst to the player.
Couldn’t find the exact interview but this one mentions it: https://www.gamedeveloper.com/design/jake-solomon-explains-the-careful-use-of-randomness-in-i-xcom-2-i-
A friend has a similar story about a seat-neighbor on a plane trip talking about how weird it is that celebrity deaths seem to cluster together. He responded that it’d probably be weirder if they died on a regular schedule…
pizza party at my place for all the ppl who knew exactly where this was going
Magic word of the day: quasirandom, which is basically randomness that conforms to this sort of expectation. Apparently financial modelling people like it for some reason… I met a guy once who made and sold high performance hardware quasirandomness sources to hedge funds.
ohmygod Hhh I remembered a GMTK video that was all about randomness and there was a part where Mark showed a ton of angry players going to the Steam forums of XCOM and complaining that they missed 6 shots in a row that all had a 90% chance to hit or something like that.
Eventually, I’ll end up making a game with random chances in it, and what I’m thinking I’ll do is secretly roll two random numbers back to back and average them together. I swear I heard someone mention this strategy before, but I can’t remember from where. It’s reasonable to say that this should massage numbers towards the middle of the total number range, making it less likely to miss high chance shots or hit low chance ones (fitting the player’s broken understanding of probability better).
I’d also be tempted to have a value in the background that tracks the player’s luck and secretly gives them an advantage/disadvantage on their rolls depending on what kind of luck they’ve been having.
Another option for randomness is to roll a random number within a range and then decrement that value each time a specific event occurs, and only do the special thing whenever that value hits zero. Stops the special thing from happening too frequently, or being too spaced out.
Eventually, I’ll end up making a game with random chances in it, and what I’m thinking I’ll do is secretly roll two random numbers back to back and average them together. I swear I heard someone mention this strategy before, but I can’t remember from where.
i think that’s how fire emblem does it
yep! Different Fire Emblem games use different variants – https://serenesforest.net/general/true-hit/ explains the system the older ones use (exactly as described above), https://fireemblemwiki.org/wiki/True_hit talks about a ‘weakened’ version of the system they use in later games.
whoa, neat! i had no idea about the hybrid version.
Neat! I had been wondering what the actual chances were with two averaged random numbers, so that chart is super helpful!
“I’d also be tempted to have a value in the background that tracks the player’s luck and secretly gives them an advantage/disadvantage on their rolls depending on what kind of luck they’ve been having”
Reminds me of the L4D AI Director which tracks per-player and per-squad “stress” via a bunch of factors like “how long since you’ve seen health” and figures the most dramatic still-winnable moment to throw in a boss
I’ve noticed that has RPG-style mechanics have progressed in computer games, the concept of the miss has been deprecated, in favor of the critical.
An old-school game might give an enemy 30 HP, and your attack has a 50% to hit to do 2-8 points of damage, so you can expect your enemy to go down in about six hits.
A modern game might give an enemy 2,000 HP, and your attack always hits to do 20-40 points of damage with a 50% chance to critical for 700-900 points of damage, so you can expect your enemy to go down in about six hits. Same effective results, but the player feels better that “there’s no misses”.
This is actually a pretty hard problem in probability! Like, just hard to think about. Probably the most respectful way to explain this is to show a normal distribution? Then you’re not calling them an idjit or whatever you’re just having a conversation about math.
I thought there was a thing called the “Chernoff bound” that was a specific way of measuring “the more trials you do, the more likely your 1/N probability is to get exactly N successes”, but looking it up on Google now it might be something more complicated. Math Wikipedia used to be more readable than it is now.
By the way, in addition to Strawberry Fox’s proposal above to make your random numbers seem subjectively “more random” by making the distribution more “lumpy”, another interesting approach is to do something like the “7bag” approach in modern Tetris, which has the effective result of forcing probability to act like “intuitive” probability rather than “real” probability. For example, in the haystack case, instead of calculating a 1/6 probability, every 6 haystacks you generate you could pick a number between 1 and 6, and for example if your random number is 4 then you put a single treasure chest under the 4th haystack. I’m not sure this is as fun as true probability, but it would result in nice clean “works as expected” properties like “if there are 36 haystacks there are exactly 6 chests”.
Yeah, probability is just hard to think about. I’m too humbled by the Monty Hall problem to think this guy was an idiot
This is agony.
a friend elsewhere linked this post and suggested that video games may be partially responsible for teaching bad models of probability
we do find that plausible, though we feel that the primary blame rests with education policymakers, not with game studios
but… yeah. a video game is a collection of hyper-real simulacra, to a degree that Baudrillard must have been astonished by in his later years. lots of fake things masquerading as real things and taking a moment to congratulate the player on the fake thing
age old saying wins again: half of all people are dumber than you think the average person is.
my sympathies to all the game devs in the audience saying “I knew exactly where this was going”, but as a non-gamedev let me say this did not go where I expected, which was a needle in a haystack punchline
FFXIV is a great example of the devs using full randomness on chances. Just ask any gatherer that’s heard the muffled noise of failing a 99% gather chance, or a melder who just had a 95% fail rate cause a stack of 100+ materia to explode into smoke because each one rolled a fail.
Deeply frustrating, but I appreciate that it really is random chance.
Optimist: “This glass is half-full.”
Pessimist: “This glass is half-empty.”
Gamer: “There’s something wrong with the RNG, we got three half-empty glasses in a row.”
AAAAAAAA
On a debug toggle, literally render a plastic D6 rolling around on the screen each time and if it rolls a 6 have there be treasure?
as someone doing a lot of QA right now this is giving me the heebie-jeebies EXTRA oh my god
Leave a Reply