3 replies [Last post]
georg137's picture
User offline. Last seen 30 min 17 sec ago. Offline
Joined: 08/07/2010
Groups: Contenders

Has anyone else in the user community been wondering about this statement, which we see a lot? Usefulness of recent additions can be confounded with other factors, such as the gain in experience by the core foldit user community, the change in composition and skill of the teams and the improvements in processors. Are player ratings used to normalize the data? There have been very many "recent additions" over the years... some of them are not very recent anymore. Is there data that can be shared about the usefulness of recent additions? It's time to share.

It would be great to see some charts and metrics showing the evolution of foldit successes, usefulness, raw performance, user statistics, etc. As a guide The Visual Display of Quantitative Information, a beautiful text by Edward Tufte would be a good source for finding a format.

jeff101's picture
User offline. Last seen 4 hours 30 min ago. Offline
Joined: 04/20/2012
Groups: Go Science
Plot Foldit score vs rmsd from native/best for Revisit Puzzles

For the Revisit Puzzles, I would like to see plots with Foldit score on the vertical axis and rmsd from native on the horizontal axis. If the native is not known, you could plot the rmsd from the best-scoring solution instead. By rmsd I mean the root-mean-squared deviation between the positions of the alpha-carbons for a given solution and the native or best-scoring solution.

See http://fold.it/portal/node/2000880 and its links for more details.

Joined: 09/24/2012
Groups: Go Science
Look at this

Locioiling is tracking and publishing all old puzzles and revisits and top players and scores on the wiki here:

http://foldit.wikia.com/wiki/Revisiting_puzzle_results

bkoep's picture
User offline. Last seen 17 hours 50 min ago. Offline
Joined: 11/15/2012
Groups: None
Good question!

Unfortunately, we haven't been able to dig into this data to the extent that you suggest. Typically, we'll just summarize the puzzle data in a form similar to that described by jeff101 above. We simply plot every model with respect to score and "distance-to-native" to get a sense of how Foldit players are performing as a whole. We've posted plots like this before; they look something like this.

These plots can tell us a few things: How closely to the native did Foldit players sample? Were the closest solutions also the best scoring? Did Foldit players find other low-energy solutions? These give us a picture, in broad strokes, of how well the Foldit score function represents the true energy landscape.

We wouldn't be able to draw any conclusions about player performance from this kind of analysis, but we can look for red flags that indicate problems with the score function and Foldit tools. Is a distant decoy model (megahelix?) scoring exceptionally well? Are Foldit players exploring models less broadly than they have in the past?

I agree it would be cool to look at this data in more depth, and hopefully we will be able to at some point! We're always to happy to see that Foldit players are invested in the scientific aspects of the game!

Sitemap

Developed by: UW Center for Game Science, UW Institute for Protein Design, Northeastern University, Vanderbilt University Meiler Lab, UC Davis
Supported by: DARPA, NSF, NIH, HHMI, Microsoft, Adobe, RosettaCommons