Are scores of today comparable with scores of 2008?

Case number:671071-995826
Topic:Game: Other
Opened by:Bruno Kestemont
Opened on:Tuesday, August 27, 2013 - 11:09
Last modified:Wednesday, May 14, 2014 - 15:19

Puzzle 688 is a re-post of 72
(by the way: 72 was created in 2009 and expired in 2008 ?!).

At this old time, the top score was 8889.

Now, after a few hours, I see that the top score is already higher (currently 8896).

Are these scores comparable? Which would mean that the tools and hardware are today very very more efficient?

(Tue, 08/27/2013 - 11:09  |  5 comments)

spmm's picture
User offline. Last seen 19 weeks 1 day ago. Offline
Joined: 08/05/2010
Groups: Void Crushers

It may also be interesting to see if the whole 'field' gets to that high score, probably not. Whilst some players may get a higher score it will be interesting to compare the two cohorts.

jflat06's picture
User offline. Last seen 3 hours 13 min ago. Offline
Joined: 09/29/2010
Groups: Window Group

They are probably not comparable. We have changed score functions a number of times, and the Rosetta folks have made improvements to the existing score functions that change the score values. Unless both puzzles have the same setup, the energy isn't likely to be comparable.

The more important metric is the quality of the structures produced in terms of RMSD and GDT, which are comparisons of how 'close' player solutions are to the native. I believe that for most of these revisiting puzzles, the results are much better than the past version.

Joined: 09/21/2011
Groups: Void Crushers

Question: have enough old puzzles been played to may conclude about the tool changes?
Especially because all those all puzzles have a known native that is easy to find?

bertro's picture
User offline. Last seen 24 weeks 3 days ago. Offline
Joined: 05/02/2011
Groups: Beta Folders

A possible way to get an idea would be to re-open an old puzzle and if it does, compare your current in client top score to the previously recorded score on the foldit site one for the same solution/track.

Joined: 09/24/2012
Groups: Go Science
Status: Open » Closed



Developed by: UW Center for Game Science, UW Institute for Protein Design, Northeastern University, Vanderbilt University Meiler Lab, UC Davis
Supported by: DARPA, NSF, NIH, HHMI, Amazon, Microsoft, Adobe, Boehringer Ingelheim, RosettaCommons