23 replies [Last post]
Angus's picture
User offline. Last seen 3 days 14 hours ago. Offline
Joined: 06/04/2008
Groups: Beta Folders

What it the scoring formula used to award points after puzzles are closed?

In the puzzles closed recently -

41st place = 18 points
32nd place = 30 points
49th place = 46 points
53rd place = 10 points
59th place = 15 points

In the interest of openness, perhaps the formula or chart could be posted by someone from the project staff? (I'm not interested in guesses by outsiders.)

Angus

Joined: 05/24/2008
points.

You get one point for turning up,

One point for logging in properly,

One point for playing a puzzle, perhaps giving other people points is you access anyone elses work without the permission of the others...

BUT - Minus points are taken off for anything you write in a forum or chat room that makes people curse, or annoy others who are playing the same game with the same rules and dont complain or try and work out something thats evolving.

Leaving you with Negative overall score Angus of -47 ( including this last post.)

(this is a Non Endorsed commical commercial)

j/k

Joined: 05/12/2008
Groups: None
Empirical observation about points

Based on fitting a curve to my own scores .....

"Percentile rank" is 1 - (your rank divided by total number of players in that game)
That is, players with higher scores and lower rank numbers get higher percentile rank, because they're in the higher percentile of players.

Points = 0.019 * exp(8.83*percentile rank)
The fit is good, with R-squared of .9917

If you want to go the other way, from points to percentile rank, it's
Percentile rank = 0.114 ln(points) + 0.45
Since it's the same data with axes reversed, the fit is just as good, with R-squared of 0.9917

In other words, the points decrease exponentially as you go lower (bigger numbers) in rank. The rate at which the points decrease depends on the number of players in the game.

Approximately the lower-ranked half of the players get one point.

This, by the way, is why it's so annoying to independent players for group members to pile in at the last minute and collect high ranks for small incremental improvements.

Joined: 05/12/2008
Groups: None
Empirical observation about points

Based on fitting a curve to my own scores .....

"Percentile rank" is 1 - (your rank divided by total number of players in that game)
That is, players with higher scores and lower rank numbers get higher percentile rank, because they're in the higher percentile of players.

Points = 0.019 * exp(8.83*percentile rank)
The fit is good, with R-squared of .9917

If you want to go the other way, from points to percentile rank, it's
Percentile rank = 0.114 ln(points) + 0.45
Since it's the same data with axes reversed, the fit is just as good, with R-squared of 0.9917

In other words, the points decrease exponentially as you go lower (bigger numbers) in rank. The rate at which the points decrease depends on the number of players in the game.

Approximately the lower-ranked half of the players get one point.

This, by the way, is why it's so annoying to independent players for group members to pile in at the last minute and collect high ranks for small incremental improvements.

Joined: 05/24/2008
Angus has been throwing sand in everyones faces.

Im sorry Anjen!

Im sorry you too have now had to experience my inability to cope with the continious whinging Angus has made "an only child" habit of, while so many others are "playing the same game" and have injoyed the sportsmanship of everyone else and not forgetting the Hidden added features or little gems that makes foldit "evolve" during its most rigious testing period.

While I respect your voice and dedication to making something more usable and understandible, friendly and competitive, his dribble is NOT. ( thus my talking shit, how it gets a response from yourself and his does not?! hmmm. )

Whining at any chance is not sportmanship, it's selfish and constructed in a way that shows disrespect to those who are following the SAME everything else he is experiencing. we voice it in our group chat so noone else can be effected by it. He's effected me, so my little j/k on tat last post should be removed and replaced by.

Angus, get out of our sandpit, go and play by yourself with Bionic where noone can read your excuses for being number 1.

Sorry Anjen, ive been here for months, I can only take so much; like yourself.

RBMI.
aka Aotearoa/Marijuana
Protein Mercenary.

Angus's picture
User offline. Last seen 3 days 14 hours ago. Offline
Joined: 06/04/2008
Groups: Beta Folders
Whining?

I don't see asking for the points award formula any form of whining (or "whinging", whatever that is).

It's a simple request for information from the project that should have been posted on the site before it ever went live.

I did ask for replies only from the project staff, but you - Marijuana or Aotearoa or whichever of your multiple personalities is currently present - decided that you couldn't resist crapping in the thread. Most of your drivel is totally incomprehensible, so why not toddle on back to your private team chat room and bore those folks for a while. Oh, and learn to spell and use proper punctuation and capitalization if you're going to continue to make a fool of yourself in public.

Jwb52z's picture
User offline. Last seen 1 year 4 weeks ago. Offline
Joined: 05/11/2008
Groups: None
Ugh....

Why these two constantly expect every I to be dotted and every T to be crossed from the get go is beyond me. Everything shouldn't be expected to be given just because you think it should, or anyone else for that matter. "Proper conduct" for a person or a project is in the eye of the beholder, really. Your idea of orderly would mean alot of things never get off the ground to be useful because someone, or a group, wants to look polished and keep up appearances. Well-run does not mean perfection down to the last syllable or page. There's a difference between, "I'd like to know how it works", which is reasonable and "This should have been done first thing" which is a demanding thing to say and not reasonable in all cases.

Angus's picture
User offline. Last seen 3 days 14 hours ago. Offline
Joined: 06/04/2008
Groups: Beta Folders
*ANY* answer from the project

would be an unexpected bonus at this point. The people running this are supposed to be intelligent IT types. It shouldn't be too much of a leap of logic for them to figure out that if they are going to award points that someone would ask "How are the points awarded?".

It's not like we're asking for the source code to Rosetta.

Joined: 07/23/2008
Groups: None
It's quite possible there is

It's quite possible there is no objective scoring function. Keep in mind that the reason they're asking people to fold proteins is that it's hard for computers to do. This means that the computer may not have "solved" the protein you're working on and therefore doesn't have a ground-truth from which to compute a good score.

Angus's picture
User offline. Last seen 3 days 14 hours ago. Offline
Joined: 06/04/2008
Groups: Beta Folders
Not "Score", but "Points"

I was referring to the "points" that are assigned after each puzzle is finished. It's based loosely on ranking, but seems to vary quite a bit from puzzle to puzzle.

Joined: 05/12/2008
Groups: None
I say again ...

Points decrease exponentially with ranking. To compare puzzles, you have to normalize for the number of players.

If you don't believe my data (above), run your own numbers.

Angus's picture
User offline. Last seen 3 days 14 hours ago. Offline
Joined: 06/04/2008
Groups: Beta Folders
There's got to be more...

Looking at the last closed puzzle, #65, there are a number of people with the same score who got different points assigned (because they were assigned rank randomly?) , then there are some with the same score who got the same number of points. So - rank doesn't seem to work in this case.

We can pursue this SWAGuessing forever - wouldn't it be easier for the project staff to just post the answer? What happened to the new, more frequent updates mentioned in the "Civility" thread? Haven't seen anything yet...

admin's picture
User offline. Last seen 1 day 8 hours ago. Offline
Joined: 11/10/2007
Groups: vi users
Explanation

Good guesses, shepherd. It's actually not exponential, but power of 7.

Angus, the people with the same score may not internally have the same score. What is displayed to you is a rounded score, some digits are chopped off, and these may be different. When people really have the same score, they also get the same number of points.

The formula for points on a puzzle is based on the total number of players in the puzzle, and your own rank. If there are less than 10, you get no points.

As Marijuana said, you get one point for showing up.

Points = Max(1, Ceil( 1 - (Rank - 1)/(NumPlayers - 1) )^7 ) * 100)

Feel free to put this in a convenient place on the wiki.

Diderot's picture
User offline. Last seen 7 years 18 weeks ago. Offline
Joined: 05/09/2008
Will do, admin!

...but what's a Ceil?

admin's picture
User offline. Last seen 1 day 8 hours ago. Offline
Joined: 11/10/2007
Groups: vi users
Ceil

Sorry. To quote ceil(3):
"ceiling function: smallest integral value not less than argument"

Diderot's picture
User offline. Last seen 7 years 18 weeks ago. Offline
Joined: 05/09/2008
I still don't understand.

I'm sorry but this still doesn't make sense to me. A celi is described as the "smallest integral value not less than argument." But what does THAT mean?

Joined: 11/10/2007
Groups: Window Group
I believe it means basically

I believe it means basically round up.

Diderot's picture
User offline. Last seen 7 years 18 weeks ago. Offline
Joined: 05/09/2008
Round up from what?

Which argument are we talking about?

And is it "integral" (suggesting calculus not shown in the formula), or "integer" (which I understand)?

I think I understand, but these are why I was confused.

admin's picture
User offline. Last seen 1 day 8 hours ago. Offline
Joined: 11/10/2007
Groups: vi users
More info

It rounds up between the parentheses of the Ceil function.

Ceil(4.1) = 5. Ceil(4) = 4. Ceil(1 - 1/2) = 1.

Integral as a noun means what you think of in calculus; as an adjective here it means integer.

Joined: 05/10/2008
Thank you Shepherd and

Thank you Shepherd and Admin. I think it would be better to have your score depend, not on you rank, but on how far you are from the protein's best solution.

For instance, best result = 100 points, median result = 30 points...

That way, if a team is at first place at some point in the contest, it is still in its interest to try and find a much better solution (because this will lower everyone else's final score).

The ultimate goal of Fold.it being to find better solutions!

Al-Lexx's picture
User offline. Last seen 9 years 16 weeks ago. Offline
Joined: 09/11/2008
Groups: Russian team
Mistake?

>Points = Max(1, Ceil( 1 - (Rank - 1)/(NumPlayers - 1) )^7 ) * 100)

There are five ")" but only four "(". WTF? I'm confused. 

Joined: 08/15/2008
Groups: None
ah, the missing ( is why i

ah, the missing ( is why i never got this formula right.
its actually Points = Max(1, Ceil(( 1 - (Rank - 1)/(NumPlayers - 1) )^7 * 100))

Joined: 11/13/2008
Groups: None
jeez.

wow calm down, he's just kidding. Freak

admin's picture
User offline. Last seen 1 day 8 hours ago. Offline
Joined: 11/10/2007
Groups: vi users
No flaming

If you continue trolling, I'll block your account.

Sitemap

Developed by: UW Center for Game Science, UW Institute for Protein Design, Northeastern University, Vanderbilt University Meiler Lab, UC Davis
Supported by: DARPA, NSF, NIH, HHMI, Amazon, Microsoft, Adobe, RosettaCommons