PDA

View Full Version : Percentiles completely innacurate



MrEntity
03-10-2013, 01:05 AM
So in basically every game I've played I've attempted to make attacks on an 80 or 90 percentile. More often than not they miss. In my complaining about this mid-match I have learned that many many people have come across this flaw.

The random number generator in this game really needs to be looked into, fixed, or replaced.

I know I'm not alone on this, I just hop the devs notice.

SunAngel
03-10-2013, 01:26 AM
I haven't had any issues with this personally. I've had most of my 90% shots hit, and I've seen 40% shots hit by my opponents. I'm fairly certain it's just bad RNG, which isn't fixable unless completely removed from the game.

raven2134
03-10-2013, 01:31 AM
First, there's a difference between percent and percentile. Percent is straightforward, it is a portion of a whole 9/10, 90%, the concept being talked about is within the 90%. Percentile on the other hand is slightly different. This refers to being part of what is beyond that mentioned boundary, i.e. when you say I am in the 90th percentile, you are part of the top 10%.

Second, there needs to be a leveling of expectation and proper understanding of statistics and probability. Remember that even if 90% can be expressed as 9/10 times something will happen, in reality probabilities occur based on large number theory. Meaning, the probability figure which we abstract actually only emerges over hundreds or thousands of cases. Meaning an individual may not see this happen 9/10, because somewhere within that sample, someone else has hit 30/30. Does this mean the random number generator is broken? Not really, this is closer to "real"/"real life."

Third, neither does this mean that any event is actually influenced by the previous or succeeding event. Each event is independent, meaning it does not matter what happened before (hit/miss) or what will happen after (hit/miss). So even if you do miss on a 90% shot, this does not mean the next one will hit.

Now, the issue isn't with the probabilities or the random number generator...you would get exactly the same behavior using appropriate dice. The issue is actually whether in-game probabilities and player's perception and expectations need to be leveled or brought closer together.

netnazgul
03-10-2013, 03:27 AM
The main issue with hit/miss mechanic here is people taking these percentage attacks when there is no need for that. Most of the times you would get more profit breaking the armor.

The only moments I think are viable are when either breaking armor can't really change anything (0wp archer with 1 armor break), or you have a good probability of one-shot-kill that will give you a huge advantage (killing thrasher with lots of wp left) and will not reflect badly onto you if you miss.

MrEntity
03-10-2013, 04:31 AM
Okay, yeah I used the word percentile incorrectly - I meant percentages. But I wrote that in a hurry, And trust me, after spending 6 years in university learning physics and math I'm well aware of how probabilities work.

The issue is - I have, multiple times, proceeded through matches never taking a shot below 70% likelihood, and missed AT LEAST half of them. Almost everyone I've talked to has had similar experiences - often.

I think I'll just have to keep a log over the next couple days so I can show you how skewed the ratio is.

netnazgul
03-10-2013, 04:55 AM
I had almost all my 90% shots hit (as far as I don't remember them miss) if it counts to your statistics.

Tirean
03-10-2013, 05:58 AM
Most of my high % shots hit but I use the act of telling the game not to miss when I do them! that seems to work :D

pix
03-10-2013, 05:59 AM
Simple confirmation bias. You remember easily when your expectations are violated by an unlikely miss, and forget all the times the probabilities were accurate.

raven2134
03-10-2013, 06:19 AM
MrEntity, even if you kept the log, there is insufficient data taking from you alone to determine whether in fact the probabilities are skewed. We would need similar logs from many other players in order to accurately estimate the mean success chance of the specific action taken or prove the alternate hypothesis. Or well, your log could also suffice but would require hundreds of entries to ascertain this with sufficient confidence.

SeraphimLoki
03-10-2013, 06:40 AM
I've had the same probem. I just stopped shooting anything below 90%. Cuz most of my 80% was a miss.
There is a problem with that and no matter how long or how smart your posts will be it wont change it.

Solon64
03-10-2013, 08:07 AM
I've had the same probem. I just stopped shooting anything below 90%. Cuz most of my 80% was a miss.
There is a problem with that and no matter how long or how smart your posts will be it wont change it.

Sticking your fingers in your ears and going "LaLaLaLa Can't hear you" is not an appropriate nor mature response to a discussion about percents.

If you feel that the percents are incorrect and that "80 or 90% shots miss more often than 20 or 10% of the time," then that is a valid concern, but you must of course back up your argument with valid evidence that the random number generator is, in fact, not working correctly, and/or is biased on a bell curve. Anecdotal evidence (that is, evidence from your personal experience or in other words your "feeling" or "hunch" about it) is not evidence at all.

Provide logs of your combats (noting, of course, that you'll need a LARGE sample size) or else no one is going to take you seriously. The burden of truth is on you, buddy.

trisenk
03-10-2013, 08:09 AM
I've had the same probem. I just stopped shooting anything below 90%. Cuz most of my 80% was a miss.
There is a problem with that and no matter how long or how smart your posts will be it wont change it.
Maybe all your hits went to me, because I don't remember missing any 80-90% shots* since official launch. So no, there's no problem, and topics like this are normal for every strategy game with visible percentages, due to mentioned above confirmation bias.

Unless Stoic wrote their own RNG (which may have bugs), I'm pretty sure that it's fine.

(*) I only use them in archer-on-archer action

erom
03-10-2013, 09:49 AM
I have not noticed any problems in this regard and use high-risk shots often. If you really believe the RNG to be faulty, start collecting data.

tnankie
03-10-2013, 08:35 PM
I am an advocate of removing the random aspects of the game.

However, I do not believe there is anything wrong with the random number generator (other than its presence in the game at all :)).

Or as I proposed previously display the results of the upcoming 5 RNG rolls, pretend that in this universe warriors are attuned to the winds of chance.

Oh and don't worry, my opponents usually get their 90-60% hits...so it cant be broken :(

RobertTheScott
03-10-2013, 09:41 PM
Lots of psychologists have pointed out that we humans expect regular, even distributions of luck, whereas true randomness always looks much more unfair, and has longer streaks of unexpected results. I have a feeling this is what is happening. (And I say this as someone who missed about 5 of my last 6 50%-chance-or-greater shots. I also say this as someone who has gone long streaks of hitting rather than missing.)

netnazgul
03-11-2013, 01:39 AM
Most of my high % shots hit but I use the act of telling the game not to miss when I do them! that seems to work :D

you should move your mouse cursor clockwise, that works for me

mouton
03-11-2013, 08:24 AM
Lots of psychologists have pointed out that we humans expect regular, even distributions of luck, whereas true randomness always looks much more unfair, and has longer streaks of unexpected results. I have a feeling this is what is happening.
Yup, it happens in every game where RNG plays a visible role. Seen the same complaints in X-Com forums, to give a close-ish example. Heh, screw games, people can't handle true chaos in real life. They make up all kinds of "reasons" for why something happened, ranging from divine intervention to them not being nice to accident victims the day before.

Unless someone can prove, by analyzing the game's code, that the RNG engine itself is faulty, we have nothing to talk about here.

MrEntity
03-11-2013, 08:20 PM
My sample size so far is not really big enough to offer legitimate proof, but the skew seems to be leaning towards vague correctness so far. I retract my drunken annoyance. I suppose I just went through a rash of ill-luck. And then had almost everyone I played with complain similarly. Confirmation bias indeed.

I repeat that THIS SAMPLE SIZE IS NOT LARGE ENOUGH TO CONFIRM OR DENY MY ASSERTION. I'm going to continue logging data. Just thought people might want to see what I've collected so far.
Hit | Miss | Resultant percentage of hits
90 - 14 - 3 - 79
80 - 11 - 5 - 55
70 - 12 - 7 - 42
60 - 13 - 5 - 62
50 - 7 - 5 - 39
40 - 4 - 1 - 75


(edit - it auto-edited my spaces and stuff so the table looked odd. I inserted hyphens to increase legibility.)

franknarf
03-11-2013, 09:59 PM
Go ahead and use a code block for monospaced font, if you want, MrE:



60%: 13 5 62%
50%: 7 5 39%

bwoneill
03-11-2013, 10:14 PM
Hit | Miss | Resultant percentage of hits
90 - 14 - 3 - 79
80 - 11 - 5 - 55
70 - 12 - 7 - 42
60 - 13 - 5 - 62
50 - 7 - 5 - 39
40 - 4 - 1 - 75
The expected results of a binomial (hit or miss) distribution with a probability to hit of p after N samples is Np+/-sqrt(Np*(1-p)) hits (68% confidence level).

For a 90% chance to hit and 17 samples, you would expect to hit 15.3+/-1.2.
For 80% and 16 samples: 12.8+/-1.6.
For 70% and 19 samples: 13.3+/-2.0.
For 60% and 18 samples: 10.8+/-2.1.
For 50% and 12 samples: 6.0+/-1.7.
For 40% and 5 samples: 2.0+/-1.1

Move along folks, nothing to see here. Statistics still works.

Grits
03-12-2013, 04:02 AM
Interesting stuff… Be curious to see further recordings of your statistics Mr. In my experience the percentages have been pretty close to my expectations. I like this aspect of our NG in the game. It adds just a little bit of excitement and Unpredictability. And, it's totally avoidable if that's your play style.

I can definitely see the argument for taking orangey out of the game completely. It would make it a little bit more skill base and fair but on the other hand it would lose some of that fun risk reward factor.

Ps
Siri has a funny way of writing RNG haha

hreinnbeno
03-12-2013, 06:09 AM
So in basically every game I've played I've attempted to make attacks on an 80 or 90 percentile. More often than not they miss. In my complaining about this mid-match I have learned that many many people have come across this flaw.

The random number generator in this game really needs to be looked into, fixed, or replaced.

I know I'm not alone on this, I just hop the devs notice.
You are a father of 8 girls and you are getting your 9th child. What are the chances it will be a girl it is 50%. Everytime it is 9/10. But you could be the lucky guy to get the 1/10 every time.

erom
03-12-2013, 09:15 AM
orangey

Siri has a funny way of writing RNG haha
That's beautiful.

tnankie
03-13-2013, 05:48 PM
You are a father of 8 girls and you are getting your 9th child. What are the chances it will be a girl it is 50%. Everytime it is 9/10. But you could be the lucky guy to get the 1/10 every time.

Possibly, Bayesian statistics would disagree with you I suspect.

I'd be looking at the father myself. Are they a fighter pilot? (High G-forces tend to kill more male than female sperm). You could be a simple statistical anomaly, or their could be environmental factors that are seriously skewing the underlying probabilities.

Grits
03-13-2013, 09:28 PM
Sorry about the self promotion but if you want to see some ridiculously tense percentage shots, check the end of this game out...

http://www.twitch.tv/gritsz/b/377463382

John
03-14-2013, 11:46 PM
Love it.

It is amazing how strongly psychological effects influence ones perception of randomness. It has become a running joke inside the studio, because _everybody_ including ourselves, are subject to those effects from time to time. Something went wrong --> "Damn you faulty random number generator!!" :)


The expected results of a binomial (hit or miss) distribution with a probability to hit of p after N samples is Np+/-sqrt(Np*(1-p)) hits (68% confidence level).

For a 90% chance to hit and 17 samples, you would expect to hit 15.3+/-1.2.
For 80% and 16 samples: 12.8+/-1.6.
For 70% and 19 samples: 13.3+/-2.0.
For 60% and 18 samples: 10.8+/-2.1.
For 50% and 12 samples: 6.0+/-1.7.
For 40% and 5 samples: 2.0+/-1.1

Move along folks, nothing to see here. Statistics still works.