Puzzle 107: a curious dilema

Case number:671071-784909
Topic:Game: Other
Opened by:xiando
Opened on:Tuesday, December 30, 2008 - 18:50
Last modified:Sunday, February 15, 2009 - 00:27

In puzzle #107, the top solution presented by group AHAP was approximately 1 point above the solo top score (although had I not experienced a problem with the server which forced me to continually log in and out to update my score for the last several hours of competition that value would have either dropped to a few tenths of a difference or a higher value than the top group entry)

In any case, the submission presented by the group and that which I submitted solo differ little in score, yet they are substantially different in content.

I am hoping that the researchers can and will comment on the significant disparity in protein models in light of the nearly identical scores for that puzzle. In fact, the difference in the final tabulated scores is 0.01%, ie, virtual statistical insignificance, and the model is stable to within a few thousandths of a percent (pulls followed by a wiggle result in return to best score to within first or second decimal accuracy)

thank you

(Tue, 12/30/2008 - 18:50  |  10 comments)

Joined: 10/19/2008
Groups: Oma Gawd

Still waiting for a comment from Baker people qualified on the science to answer the issue. It's been a month now.

axcho's picture
User offline. Last seen 8 years 10 weeks ago. Offline
Joined: 12/18/2007
Groups: None
Status: Open » Open

Okay, I sent this out to the Foldit list, so hopefully you'll get a reply sooner or later.

Joined: 05/09/2008
Groups: None

Hello xiando,

You bring up one of the main problems in protein structure prediction.

Unfortunately, unless you are very close to the correct native solution, it is often the case that our energy functions assign similar scores to very different models.

You can think of this as trying to climb to the top of Mt. Everest (maximizing your foldit score) without knowing the exact altitude of Everest and only having an altimeter.
Let's say that everyone starts in the same place, in Death Valley for example (with a very low score), and you get all the way to the top of the Mont Blanc. You've gotten as high as you can, you can't get any higher (you've maxed out your foldit score and there doesn't seem to be anyway to get a higher score by making any small moves), but sadly you are very far from Mt. Everest.
Someone else may have ended up in the Andes, on a mountain with the exact same altitude as the Mont Blanc (give or take a few decimal points), and they too are stuck at the top and can't get any higher... but sadly they aren't any closer to Everest than you.

I don't mean to be depressing or imply that nobody was near the correct solution, but rather the problem is that there is no way of knowing who is closer when all that you are told is your altitude.

It may be the case that one of you was actually in the Himalayas very close to Everest, but stuck at the top of a peak with the same altitude as the Mont Blanc (where just one major move in the protein would have gotten the correct solution) but when all that you see is the foldit score... it's very tough for you to tell who is closer.

We hope that the new all-hands feature will help with these cases, where will we share the top scoring solutions to see if everyone can improve upon each others solutions... getting us all a little closer to Everest!

Joined: 10/19/2008
Groups: Oma Gawd

"Unfortunately, unless you are very close to the correct native
solution, it is often the case that our energy functions assign similar
scores to very different models"

Since the claim was made that this program would ultimately be used for the purposes of working unknown solutions, rather than known solutions in which the known solution can be harnessed by "those in the know" using the PDB or similar to taylor their output, you are defacto confirming that your basic model for gameplay  (compact, hide hydrophobics, and resolve clashes) is flawed and that results cannot be considered as being valid themselves by using the play-model or scoring system offered, as it was not thought out with enough consideration to elements that can distort the score in favor of completely non viable solutions..

In the case of 107, since AHAP used the protein database to initiate the model they submitted, it is likely the correct solution. That there's was the "correct" solution I suspected already, and I'm not surprised. My submission was in part based on the hypothesis that we are being deceived about what actions should result in a valid fold and that the scoring system in place as of today is a broken model.

Thank you for the confirmation of my hypothesis... I thought as much.

I was wrong folks. It IS just a game. All that oher stuff is just plot thickener. doesn't change that I will or won't play,  just that the nature of its worth is called into question.

lol, no wonder your team has been so silent on the questions asked. it's all fakery.

Joined: 12/08/2008
Groups: None


It is only flouride in the tooth paste not a conspriracy.

Joined: 05/13/2008
Groups: Boinc.be

In my opinion there is something wrong in your conclusion taking: it will hopefully be used to address the problem of folding unknown structures, but that is: 'better than with computers alone', and not 'the exact structure'. The goal is to go as close as possible, i.e. the best score. If humans can have better scores than simulations, than so be it and foldit was a success. But getting to the exact native structure..Even when we speak about 'known structures', that does not mean it is the exact one or the correct one. ('if correct' means the one it has in nature). But I'm going off topic here..

Joined: 10/19/2008
Groups: Oma Gawd

nah...just a simple deception by children barely out of highschool trying to prove themselves as more than they really are..i expect as much from members of the Playstation generation.

Problem is, they've advertised it to the world as much more than it really is.

I will treat it as what it is from now on, no more thinking that we're doing valid science. The results of puzzle 107 prove it to be nothing more than poppycock, and the answer that the unknown and *un-verified "biochem" expert confirms the hypothesis. I've has suspicions for quite a while and have a selfish reason for "need to know"., ie, is the premise accurate or purposefully misleading...I now know the latter to be true.

* credentials and name to verify claim

admin's picture
User offline. Last seen 2 weeks 3 days ago. Offline
Joined: 11/10/2007
Groups: vi users

Assigning topic to beta_helix, so he can be updated of followups to his reply. I am sorry that you have to be bothered with insults and trivialities, but I thought it better that such wild accusations not remain unanswered in a public place. I can remove you if you like (or you can too, when you post a reply, by assigning the issue to "Anonymous" or "admin").

Joined: 10/19/2008
Groups: Oma Gawd

I figured you'd play this card...you want tit for tat...I can play too.

Don't address it, silence criticism.

admin's picture
User offline. Last seen 2 weeks 3 days ago. Offline
Joined: 11/10/2007
Groups: vi users
Status: Open » Closed

beta_helix has addressed the original question in a blog post, as he wanted to give it higher visibility.


Developed by: UW Center for Game Science, UW Institute for Protein Design, Northeastern University, Vanderbilt University Meiler Lab, UC Davis
Supported by: DARPA, NSF, NIH, HHMI, Amazon, Microsoft, Adobe, RosettaCommons