Protein Puzzle Score Distribution

Case number:671071-988112
Topic:Game: Other
Opened by:srssmith92
Status:Open
Type:Suggestion
Opened on:Saturday, July 31, 2010 - 00:46
Last modified:Thursday, October 9, 2014 - 19:09

Would it be possible to display a graph and statistics of the puzzle score distribution as the end of each puzzle.

Make this data available on the webpage as a rollup and at each puzzle's web page.

Also keep a running statistics of things like, standard deviation of winner score, how 1 standard deviation is changing from puzzle to puzzle, and other stuff like that.

(Sat, 07/31/2010 - 00:46  |  10 comments)


Tlaloc's picture
User offline. Last seen 10 weeks 3 days ago. Offline
Joined: 08/04/2008
Groups: Mojo Risin'

It would also be cool to see the final exploration map of a puzzle on the web page.

Joined: 07/02/2008
Groups: ExPitneyFolders

Showing data such as number of players participating in puzzle, average score for a puzzle, etc.

These stats would provide the trend data to see if scripts have raised the average score and how the best scores compare to the average.

jeff101's picture
User offline. Last seen 1 day 2 hours ago. Offline
Joined: 04/20/2012
Groups: Go Science

I like the above ideas. It would also be neat if for each puzzle each player could receive by e-mail a png file with a customized graph in it. Each graph could show Foldit scores on the vertical axis and alpha-carbon RMSD values on the horizontal axis. The RMSD values could be found between each solution and the solution with the best Foldit score on the puzzle (or if the true native structure for the puzzle is known, show the RMSD's between each solution and the native structure). The graph could color the player's own solutions in black, the rest of his/her group's solutions in dark grey, and all other Foldit players' solutions in light grey. This would show the diversity of solutions all players are obtaining for each puzzle and give each player and group an idea of their contribution to this.

Within the Foldit client for each puzzle, there could be a button that you have to press to sign up for this e-mail. This way, if you don't play a puzzle, you won't receive the e-mail for the puzzle. Also, if you don't want to receive the e-mail, you just don't click the button to sign up for the e-mail.

An extra e-mail like the above could be automatically sent to each group's manager. This graph could show his/her group's solutions in dark grey, and all other Foldit players' solutions in light grey.

Finally, in the comments for each puzzle, you could post the same kind of graph but use one color (like light grey or green) and show all Foldit players' solutions on this graph.

jeff101's picture
User offline. Last seen 1 day 2 hours ago. Offline
Joined: 04/20/2012
Groups: Go Science

If you make the graphs as in the previous comment, you could also include points for the starting model(s). If it is a multi-start puzzle, you could put numbers like 1,2,3 for starting models 1,2,3 on the plot. You could also mark points like those in http://fold.it/portal/node/1998460 on the plots.

Thanks!

jeff101's picture
User offline. Last seen 1 day 2 hours ago. Offline
Joined: 04/20/2012
Groups: Go Science

The alpha-carbon RMSD (root-mean-square-deviation) is one way to measure the similarity between 2 solutions. The smaller the RMSD value, the more similar the 2 solutions are.

Ideally, the Foldit score reflects how well-folded the solution is, and you would expect the native structure to have the best Foldit score.

The alpha-carbon RMSD value seems reasonable for puzzles with fixed amino acid sequences. I'm not sure what measure of similarity would be best for puzzles that allow mutations, let you add or delete residues, or let you move residues to different parts of the protein. Anyone have any suggestions?

jeff101's picture
User offline. Last seen 1 day 2 hours ago. Offline
Joined: 04/20/2012
Groups: Go Science

Another idea for these graphs is to shade/color a player's solos differently from his/her evos. One could also mark on these graphs any solutions the player "shared with scientists".

jeff101's picture
User offline. Last seen 1 day 2 hours ago. Offline
Joined: 04/20/2012
Groups: Go Science

It isn't easy to get a top-scoring Foldit solution. Letting players see the score distribution as in http://fold.it/portal/node/988112#comment-28798 can show them if they were on the right track. It can also show that each player's contribution to Foldit is more than his/her top score on a particular puzzle. Each player is in fact exploring part of the protein's wide range of configurations, and sometimes solutions that didn't score best turn out to be the most interesting. Some methods might be better for this exploration of a protein's configuration space than others, and seeing the score distribution can help players adjust their methods to do this exploration more effectively.

jeff101's picture
User offline. Last seen 1 day 2 hours ago. Offline
Joined: 04/20/2012
Groups: Go Science

With such graphs, players could ask questions like "Which explores configuration space better: DRW or GAB?" and do experiments to get the answers.

spmm's picture
User offline. Last seen 40 weeks 6 days ago. Offline
Joined: 08/05/2010
Groups: Void Crushers

If I have understood the OP correctly this only shows scores against other contestants not scores against the 'native' or show if we are getting better at what counts ie correctly folding proteins.

jeff101's picture
User offline. Last seen 1 day 2 hours ago. Offline
Joined: 04/20/2012
Groups: Go Science

http://fold.it/portal/node/1998460 is more along the lines you mention here, but http://fold.it/portal/node/988112#comment-28798 above does discuss what to do if the native structure is known.

Sitemap

Developed by: UW Center for Game Science, UW Institute for Protein Design, Northeastern University, Vanderbilt University Meiler Lab, UC Davis
Supported by: DARPA, NSF, NIH, HHMI, Amazon, Microsoft, Adobe, RosettaCommons