Incorrect global player scores

Case number:671071-989253
Topic:Game: Other
Opened by:CharlieFortsCon...
Status:Closed
Type:Bug
Opened on:Wednesday, February 9, 2011 - 23:01
Last modified:Wednesday, March 30, 2011 - 14:41

Evening FoldCentral - It may be time to update the global scores again? The 4 month rolling window suggests the last scoring puzzle should be 369 (ended Oct 12th) whereas the webpage scores seem to be reaching back to 362 (Sept 27th), although there is an element of doubt there as well.

New batteries for the UnderGrad in the Basement perhaps? :)

Cheers
CFC

(Wed, 02/09/2011 - 23:01  |  14 comments)


Vincero's picture
User offline. Last seen 2 hours 34 min ago. Offline
Joined: 02/14/2010
Topic: Game: Social » Game: Other

Rank lowered , score inaccurately computed. Lost more than 300 points.

Mark-'s picture
User offline. Last seen 29 weeks 5 days ago. Offline
Joined: 08/05/2009
Groups: Contenders

'Rank lowered'

All your results from the beginner puzzles do not count any more. They are now outside outside of the 4 month rolling window. They were where you had the majority of your points from. You have been scoring less points on the later puzzles.

'Score Inaccurately Computed'

You have 127pts solo. Adding up your results, this takes us back to between 18th October and 3rd November. It's difficult to say exactly where the 4 month window ends, because you missed playing a few puzzles during that time. If we say it was somewhere inbetween those dates, then it's pretty close to 4 months.

'Lost more than 300 points'

I lost a lot more than that. Everyone 'lost' points. It happens from time to time when the global scores are adjusted for the 4 month window.

Just to recap: global scores only count for a rolling 4 months. If you stop playing for 4 months your score will be zero when you come back.

Hope this explains some of the changes to your score.

Changing priority. This doesn't affect game-play
Leaving open. Because I really don't know how accurate the scores are now. I'll leave that up to people who keep an eye on them.

Mark

Vincero's picture
User offline. Last seen 2 hours 34 min ago. Offline
Joined: 02/14/2010

"Come back"? I never left. Have played every single month since August.

Joined: 06/17/2010

I think here should be some information how far back points are counted and recount should be made every puzzle close. imo :)

Vincero's picture
User offline. Last seen 2 hours 34 min ago. Offline
Joined: 02/14/2010

MARK:

"COME BACK"? I NEVER LEFT. HAVE PLAYED EVERY SINGLE MONTH SINCE AUGUST PAST. PERCHANCE THIS EXPLAINS WHY I NO LONGER SEE ONE OF MY BUDDIES IN CHAT SINCE DECEMBER. HE QUIT THE GAME!!

[I THOUGHT I WAS BEING PENALIZED BECAUSE I WAS BITCHING ABOUT HAVING TO SWITCH CONTROL FROM FRENCH TO ENGLISH TO ACCOMMODATE DECIMAL POINT]. WTF????

SOME FOLDIT DEVOTEES WILL FEEL COMPELLED TO EXIT TO ETERNA.

Joined: 11/14/2009
Groups: None

Hey, SuperNova, we're going to look into this as soon as we can. There is lots of stuff that's not working exactly right at this moment. There are just a few of us. One thing for sure, though, we'd never penalize anybody for feedback about Foldit. In fact we really appreciate it because it helps us find and fix bugs. Thanks for your patience.

-IM

Mark-'s picture
User offline. Last seen 29 weeks 5 days ago. Offline
Joined: 08/05/2009
Groups: Contenders

@SuperNova

You misunderstand me. I did not say you had left. I said 'IF' you stop playing for 4 months your score will be zero when you come back. It was an example. To try and help you understand. Which failed miserably.

Everyones global score should be 4 months of puzzle scores, going backwards from the last puzzle. Anything further back than that should not count anymore. It's the same for everyone.

I apologise if you misunderstood me but please don't shout. It's not very nice.

Mark

infjamc's picture
User offline. Last seen 2 years 41 weeks ago. Offline
Joined: 02/20/2009
Groups: Contenders

SuperNova:

I agree with you that the current system can be improved. After all, although the theoretical implementation is that global scores should be recalculated after each puzzle closes (to include only those from the last four months), it seems to me that what's actually being done is recalculation at irregular intervals.

Hence, if the admins have waited for too long before a recalculation has been done, there is a sudden drop in global scores that can surprise a lot of players; my guess is that this is unfortunately what has happened to you. Based on your "lost more than 300 points" observation, it seems that the admins probably updated the global score calculations two months too late. In other words, your September and October 2010 puzzles, which accounted for 286 points, were included by mistake.

For your information, here are two more data points from the Contenders to demonstrate how the 4-month rolling window works:
1. I also observed a major drop in rank myself, from the teens to #27. The calculation does look correct, though, as the last puzzle being counted being #372 (which closed on October 19, 2010). Why did this happen? It turns out that my performance between September and October 2010 was much better than recently-- I was averaging more than 60 points per puzzle during that period, versus about 44 in the last four months. The difference would account for about 400 points, which is consistent with the difference between the 10-20 and 20-30 ranges.

2. And then there's mimi, who occupied the #1 spot for a while. Well, she dropped to #16 in no time due to being on vacation. How did this happen? It turns out that she was averaging somewhere between 70-80 points per puzzle, and staying out of the last nine puzzles accounted for more than 600 potential points that could be earned if the hiatus didn't take place. How big is this difference? As the soloist scores suggest, 14 places.

==> The bottom line is that the system is theoretically fair in the sense that it requires steady participation and performance to keep your rank; the problem, though, is that it not being strictly enforced. [Admins: It would be nice to redesign the website so that scores are automatically re-calculated rather than having to do this manually...]

Vincero's picture
User offline. Last seen 2 hours 34 min ago. Offline
Joined: 02/14/2010

I appreciate your taking the extended moment to reply Infjamc.

My questions are these:

1. Has the 4-month rolling window been in place since the earliest release? If so, then, yes, the system is theoretically fair. If not, then the earliest FoldIt players are benefitting from accumulated points that laters players are disallowed to accrue. Not intending to diminish their skills, I'm saying that none other than the front-runners will ever have an opportunity to ever be a front-runner, superior scripting skills or none. {Not referencing me; I'm just learning lua script].

2. Steady participation and INCREASED performance is my aim, not steady performance. I need my stats to reflect that for me to perceive some that I've advanced my skills base, even if ever so slightly. Now, I'm bumped back down to the kiddie pool [<150 points] but I'm not entitled to engage in the beginner-level games. I'm neither a newbie nor a veteran. Participation and performance need to align more in tandem for me to perceive that I am increasing in knowledge.

3. My earliest engagement as a player was last February, with my first account. Being wholly inundated with work-related projects, I did not play. How does it look that a full year later, I've attained only 127 points and rank in the 400s. Yes, I must UP my game and learn scripting. That I'm committed to anyway. But if ranks and scores did not matter, most players would not maintain their devotion to the game. That goes against human nature, evidenced in labs and out. I'm seeking my level best and require something to show for that. Points - like pounds and dollars and yen and francs - do count; cumulative points count more.

Vincero's picture
User offline. Last seen 2 hours 34 min ago. Offline
Joined: 02/14/2010

We should be able to keep all points accrued. If a lapse occurs, our respective ranks would reflect that ANYWAY, sans interventions and tampering.

mimi's picture
User offline. Last seen 32 weeks 4 days ago. Offline
Joined: 11/17/2008
Groups: Contenders

As Infjamc mentioned, I'm not playing at present because I'm away travelling the world.
I have every expectation that by the time I get back I will have lost most, if not all, of my current points. I may well be back to playing <150 pt level puzzles. It will be a challenge to see if I can get back up to my previous level.

If players kept all their accumulated points then new players would never be able to compete with those who have been around for the last three years or more. The four month cut off makes life interesting and allows "newbies" to play on an equal footing with the veterans.

The scoring mechanism has always appeared to be a bit "erratic". It would be nice to see an automatic recalculation on a 120 day basis - or whatever - that way we could all easily check whether there seems to be a problem with our own scores.

infjamc's picture
User offline. Last seen 2 years 41 weeks ago. Offline
Joined: 02/20/2009
Groups: Contenders

Re: SuperNova

"Steady participation and INCREASED performance is my aim, not steady performance. I need my stats to reflect that for me to perceive some that I've advanced my skills base, even if ever so slightly."

Well, in theory, this will eventually happen... but it takes time. In your case, for example, the sudden drop was unfortunately due to a scoring gap between the <15 / <150 puzzles and the "regular" puzzles: your <15 and <150 puzzles accounted for 245 points (30.63 pts/puzzle over 8 puzzles) within the span of one month, while your non-beginner puzzles afterward accounted for 170 points over 26 puzzles (6.5 pts/puzzle). Hence, once the September 2010 puzzles dropped out of the equation, a huge drop in rank was unfortunately inevitable even though your skills are most likely actually progressing.

==> Don't be discouraged, though. The fact is that the "beginners only" puzzle are not a fair comparison because it is much easier to be ranked high there. After all, it's obviously much harder to score at the same rate as in the <15 and <150 puzzles when the competition pool is much larger. This fact is further strengthened by the way that score is calculated (relative position raised to the 7th power), which arguably rewards a small number of very good performances over a steady above-average performance. For example, ranking in the 75th percentile over 10 puzzles would yield 14 points per puzzle (for 140 points), while ranking in the 95th percentile over 2 puzzles and in the 60th percentile over the other 8 would yield 70 pts/puzzle over the two top finishes and 3 pts/puzzle for the other eight, for a total of 164 points.

Bottom line: unless you are consistently performing significantly above average, your improvement in skills could very easily be hidden by the seemingly low score. For example, your profile page indicated that you're getting 6.65 pts/puzzle over 30 puzzles during the last four months. While this number might look low at first glance, the fact is that this translates a 68 percentile performance (about half a standard deviation above average). Statistically speaking, this means that your skills are definitely improving.

[By the way: My experience is that scripting, while useful, is not necessary or even sufficient for success. Some tactics require significant human judgment and cannot be easily automated without programming a full AI (good luck with that) anyway. Plus, there's always the option of downloading the recipes and scripts that others have created and combining them with your manual folding strategies...]

Joined: 05/03/2009
Groups: Contenders

The 4 month score window is probably the fairest way of 'levelling the playing field' between so-called veterans and newer players, as it reflects contribution over a sustained period of activity. As players hang around and improve, they naturally find their level, and older scores fall off the back of the window.

The notion of these web-page scores for group, evo and solo occasionally needing a recalculation has come up before (http://fold.it/portal/node/986302). There does also seem to be an issue with <15 and <150 contributing towards current group scores (when they shouldnt). To be fair, they historically do seem to be trimmed away, but it just underlines the need for a recalculation.

For clarification, is the 4 month window calculated by month, or is it 120 days (as mimi mentioned earlier)?
It would be great to set the record straight here in fine detail, once and for all - ""for example...""

Today is the 15th Feb, the window is 10 days long, so it stretches from the 6th to the 15th. This means the first scoring puzzle is 397 which ended on the 7th, and 396 is dropped, as it finished on the 5th which is 11 days ago, and therefore the first puzzle outside of the window.

Thanks
CFC

Joined: 05/03/2009
Groups: Contenders
Status: Open » Closed

duplicated here
http://fold.it/portal/node/989490

Sitemap

Developed by: UW Center for Game Science, UW Institute for Protein Design, Northeastern University, Vanderbilt University Meiler Lab, UC Davis
Supported by: DARPA, NSF, NIH, HHMI, Amazon, Microsoft, Adobe, RosettaCommons