9 replies [Last post]
Joined: 12/06/2008
Groups: Contenders

I was trying to figure out why the top dozen or so players on "174: More Core Design" were scoring five, six, and seven hundred points more than me. I may not be a top-seeded folder, but I'm no slouch at this game, either.

I spent my "peekaboo" points to get an idea of what the best designers are doing. HOLY COW. Doctor Frankenstein didn't create anything as gruesome as I see here.

Why have so many of you changed the protein from one with seven sheets to seemingly random structures composed mostly of helices? Yes, it appears these freakish designs are producing high scores. But aren't y'all defeating the purpose of this particular puzzle by ripping apart the "core" and gluing it back together in ways that can't possibly exist in real life?

Looks like the scoring algorithms aren't as robust as they could be. I vote for trashing this puzzle and all scores attached to it, and re-submitting it with the secondary structures locked, as was done in other design puzzles.

Susanne's picture
User offline. Last seen 6 years 21 weeks ago. Offline
Joined: 11/09/2008
Groups: SETI.Germany

Thanks Boots,
I wondered where I had gone wrong and had already written off getting anywhere near the top scores. I have only 1 Oracle point left so didn't get to see the top results. Maybe there should be 2 prizes in this one? As if it wasn't bad enough being thrown out of the game during mutate thinking it was my unreliable internet connection (some mutates have had half a dozen starts), but I read in feedback that they fixed that now. The 'subgroup' will just have to have its own competition with this one :)

Joined: 11/20/2008
Groups: None
Boots said: Looks like the

Boots said:
Looks like the scoring algorithms aren't as robust as they could be. I vote for trashing this puzzle and all scores attached to it, and re-submitting it with the secondary structures locked, as was done in other design puzzles.

I guess this does show a major flaw, so the puzzle served at least one purpose.
Trashing it would be my idea as well, but instead of re-submitting it with the secondary structures locked all the way, I wonder if there's if it's possible to only lock the structure function?

Not being able to change the structure does give a lot more room to work on a puzzle than locking up the backbone, so that would be more fun!

Joined: 11/10/2007
Groups: Window Group
You are right, thanks for

You are right, thanks for pointing this out! On this puzzle (as in the previous "core design" puzzles) we want the backbone to be able to move a bit, unlike the "interface design" puzzles where we want it pretty much unchangeable. I would like to take this puzzle down early, and then post a new one with some constraints (like in level 5-4, and used in the CASP puzzles) to keep the backbone closer to where it starts.

Joined: 09/17/2008
Groups: None
my $.02

I understand exploring, and I understand gaming for points. Still, it is kind of lame that those of us who tried to stay true to the purpose of the puzzle (in spite of knowing how people were attaining such high scores) will be punished in the global rankings (and our teams as well) if we don't spend useless hours redesigning the core out of our proteins in a way that games for points and is irrelevant to the research goals of the puzzle.

Joined: 06/24/2008
Groups: Void Crushers
A shame

I have been very busy at work. I only read the forum this morning. I am a bit angry that I wasted my time working this puzzle.

g_s's picture
User offline. Last seen 1 year 10 weeks ago. Offline
Joined: 05/09/2008
Groups: None
I was tempted

I was at 10733 and couldn't figure out why there were so many scores 700 points higher than me just like Boots. I then saw someone post a screenshot of their work that was just a long Helix. I was tempted to see if that was possible and in about 5 minutes I had gotten 500 more points. Sorry I tried it now, but can't the fold.it guys just look at all the top solutions and see which ones aren't following the guide and invalidate those points which would include mine?

More importantly looking at the scores for the new puzzle 176 Quest to the Native 9 is this still happening?

beta_helix's picture
User offline. Last seen 1 hour 36 sec ago. Offline
Joined: 05/09/2008
Groups: None
Quest to the Native 9 concerns


Quest to the Native 9 is based on Rosetta Decoy 15 which had the following top soloist scores:

Rank Player Group Score Points
1 infjamc 47 12 GoFolders 10,536 100
2 aap 34 9 Richard Dawkins Foundation 10,505 97


Both these players got very close to the native topology that is shown as the guide in Quest to the Native 9,
so it does not surprise me that currently aap has the top score of 10,589 when seeing the native (since the native guide has a score of 10846)

I can look at the current top solutions if you like, just to make sure!

Joined: 11/08/2008
Groups: Contenders
Though I won't claim to know

Though I won't claim to know a lot about protein design other than what I've learned through this game, a some of the high scoring but not-single-helix solutions may also be classified as abominations of science.

Because the game gives high scores to tryptophanes, histidines, and phenylalanines, it is extremely easy to use these to get relatively good scores that are still close to the original protein shape. However, using so many of these amino acids is highly unusual, perhaps even unusual to the "How could I have created such a monstrosity?" degree.

A consideration of this scoring phenomenon might be in order.

possu's picture
User offline. Last seen 11 years 44 weeks ago. Offline
Joined: 11/16/2007
Groups: None
don't get discouraged + problem analysis

Here's a little bit more description on puzzle 174. The template given is a de novo model. This means that the protein structure you see on the screen doesn't exist in nature. The general topology does, but not the exact structure. This is a protein build from scratch by Prof. Brian Kuhlman's lab at Univ North Carolina, and we are using Foldit to try to improve it. There will likely be more puzzles like it, but hopefully with improved setup to circumvent the problem we see with this one.

But please don't get discouraged. The goal of having this puzzle is to actually have an improved design so we can experimentally test the best ones in the lab. It will not be cancelled so people who get high scores will still have the credit.

For people who sticked with the core design objective, I think the reward is still high because we will look through those answers and test the good ones in the lab. It would be very exciting if the answers actually work as designed. We'll let you know if it did. (unfortunately, I don't think we can test the ones turned into helices).

The problem with this puzzle is that the entire protein is turned into poly-alanine and this opens up ways to gain score by turning the structure into helices. (alanines like to be in helical conformation). The reason for turning both the surface and core residues into alanines is to reduce artificial scores people can get by making contacts with surface residues. It's a core packing design, so the score should emphasize on just the interactions in the core. We tested a few puzzles with this setup and haven't had serious problems. But I think partly due to the novel nature of the structure, the existing scaffold may not "lock things in place" as well as the ones with known crystal structures.

The results from this puzzle are actually very interesting. We now learned that design puzzles need to be setup very differently from, say, quest to the native. And we will need new ways to make sure surface residues stay surface. We can simply lock up the structure to make sure things don't change too much, but I personally would prefer that people do explore for better answers. In retrospect, the poly alanine setup masks regions of the protein from players but it doesn't do the job of making sure the properties of those positions stay. All of the not mutatable positions in this puzzle are actually something else other than alanine; it would be very difficult to turn the structure into helices if those identities were assigned. We learn from this, and I hope this response makes it clear for everybody why the problem arises and why we setup it up this way.

We already have a few new ideas for getting around this. I hope you've had fun exploring this ultra cool project (building a de novo beta fold has never been done before). As I said, we will still look through the answers, so your efforts aren't really lost.



Developed by: UW Center for Game Science, UW Institute for Protein Design, Northeastern University, Vanderbilt University Meiler Lab, UC Davis
Supported by: DARPA, NSF, NIH, HHMI, Amazon, Microsoft, Adobe, Boehringer Ingelheim, RosettaCommons