Protein Design Partition Tournament

We are announcing a friendly protein design tournament for Foldit players! This tournament is something of an experiment, and we are not sure exactly how it will unfold. Participation in the tournament is completely voluntary, and we will continue to post regular Foldit puzzles during the tournament, for those who do not wish to participate. There will be no prizes (except for bragging rights and a rare new Foldit Achievement), and we cannot guarantee that any scientific results will arise from the tournament. The main purpose of the tournament is to have fun folding proteins, and to inspire Foldit players to think differently about protein design; but we do also think the tournament could lead to higher quality protein designs.

Unlike our regular design puzzles, in which players compete to design proteins with the best absolute energy, this tournament is designed to reward proteins with the best energy landscape. For more discussion about energy landscapes, see parts one and two of this blog series.

The tournament will take place in two phases, over the next 6 weeks.

Phase One: Defense

The first phase of the tournament will take the form of a Foldit design puzzle, similar to the regular Monomer Design puzzles that are posted every week. Players will have two weeks to craft their best protein design, which will have to defend itself in Phase Two. To enter the tournament, players must share their chosen design with scientists, using the Upload for Scientists button in the Save Solution menu. When you save your solution, give it the title ‘Tournament Submission’ in the Save Solution dialog box. Every player is allowed one tournament submission. If a player submits multiple solutions, only the most recent solution will be accepted. For the sake of competition logistics, team play will not be allowed in the tournament; only soloist solutions will be accepted.

The Phase One design puzzle will include two objectives:
Residue Count: Designs may contain 70-100 residues, at a cost of 32 points per residue.
Secondary Structure: Designs may be up to 10% α-helix.* Additional helices will be penalized at 10 points per residue.

*We will accept Phase One submissions regardless of their secondary structure content. However, we’d like to discourage players from submitting helical bundles and ferredoxin-like folds (a.k.a. "surfing hotdogs") that typically score well in regular design puzzles. Rather, this is a chance for players to showcase designs that aren’t normally competitive in regular design puzzles.

Phase Two: Offense

Twenty Foldit player submissions will be selected to advance to Phase Two:
Five will be the five top-scoring submissions from Phase One.
Five will be hand-selected by the Foldit team, on merits of creativity and plausibility.
Ten will be chosen at random from the remaining submissions.

For each of the 20 selections, we will create a special Partition Contest using the selected protein design. Each contest will be set up as prediction puzzle, similar to the regular De-novo Freestyle puzzles, except that the starting structure will be the fully-folded design. All 20 Partition Contests will be open to the entire Foldit community and will remain online for four weeks, during which time the selected designs will be vulnerable to “challenge.” Any Foldit player can challenge a design by joining its Partition Contest and attempting to refold the design into another high-scoring decoy structure.

The Phase Two contests will include an RMSD Objective: All solutions must differ from the starting model with an RMSD of at least 2.5 Å.

Scoring

Ultimately, each design in the tournament will be evaluated by its partition function (described in the previous blog post), based on the decoys found by challengers in the Phase Two contests.

By challenging a design and finding a high-scoring decoy, you show that your opponent's sequence does not have 100% probability of adopting the folded structure, and that its partition function must be shared with your decoy structure. You effectively stake a claim in the partition function of that design; the higher the score of your decoy, the larger your claim in the opponent's partition function.

A player may make multiple challenges against a single design; in some cases, it may be more effective to make many moderate-scoring challenges rather than a single high-scoring challenge. In order to calculate the partition function for a design, we will cluster all of the contest solutions to identify representative states. Then, we’ll use the partition function to determine the probability of each state.

Unfortunately, we cannot calculate the partition function on the fly, so players can only estimate how well a design is resisting challenge by following the Contest leaderboards. However, we will post weekly updates throughout Phase Two, with updated partition functions for all 20 Contests.

The champion of the tournament will be the protein design with the highest probability, as determined by its partition function.

There will also be Achievements for the most effective challengers, who are able to stake the greatest claims in the partition functions of their opponents.

Finally, we’d like to point out that, while players may be tempted to aim for a high-ranking design in Phase One, what really counts is how well each design can withstand challenges in Phase Two. If you design a high-scoring protein in Phase One, but its sequence is also compatible with many high-scoring decoy structures, then in Phase Two challengers will easily find high-scoring decoys and stake large claims in your design’s partition function.

The Phase One design puzzle is online now! Happy folding!

( Posted by  bkoep 74 525  |  Thu, 08/30/2018 - 20:54  |  22 comments )
2
frood66's picture
User offline. Last seen 31 min 14 sec ago. Offline
Joined: 09/20/2011
Groups: Marvin's bunch
partition

Well explained blogs - whilst many vets will already understand the concept.

But why would the first puzzle (an experiment as u say) be run on 2 bases? either u want guys to create solutions that may pass yr partition test - or U don't. Please don't be lazy - create this puzzle purely for partition (and a separate one for those not interested in the experiment) It makes little sense as it stands and can only cause confusion for newbs.....and right now newbs (and retention here) is probably a strong argument for changing this. In short - this system makes no sense (with respect)

I and others believe this approach is flawed.

bkoep's picture
User offline. Last seen 5 hours 45 min ago. Offline
Joined: 11/15/2012
Groups: None
Thanks for the input!

We considered posting a completely separate puzzle for the tournament, and it probably would work just as well as the current dual-purpose puzzle. The main reason for this setup (i.e. allowing tournament submissions from a regular puzzle) is that we thought it would lead to more participation in the tournament.

If any players are on the fence about joining the tournament, then they can play the Phase One puzzle like a normal design puzzle, for now. If they change their mind at any point and decide to make a tournament submission, then they can easily draw from the work they've already invested in normal play—they won't have to start over in a completely new puzzle.

Alternatively, if we had made an secondary, optional puzzle specifically for the tournament, then any participating players would have to commit time and effort in an experimental puzzle worth zero points. Even then, there is still the possibility that their submission would not be selected for Phase Two of the tournament. We were afraid this would dissuade some players from participating in the tournament at all.

I don't think the current setup is too confusing for new players, and we will continue with the dual-purpose puzzle. If there is a particular point of confusion among players, please let us know! We can always clarify the rules, and amend the puzzle description if necessary!

jeff101's picture
User offline. Last seen 11 hours 58 sec ago. Offline
Joined: 04/20/2012
Groups: Go Science
Any helix restrictions in Phase Two?

It looks like Phase One penalizes structures with more than 10% a-helix content. Will Phase Two do this as well? Seems like in nature, the protein can always sample some helical conformations as it folds, so having the same 10% a-helix restriction in Phase Two would not be a good test. I think it would be better to let players use as much a-helix content as they want in Phase Two.

bkoep's picture
User offline. Last seen 5 hours 45 min ago. Offline
Joined: 11/15/2012
Groups: None
No helix restrictions in Phase Two

Good question! There will be no secondary structure restrictions in the Phase Two contests. The only purpose of helix restrictions in Phase One is to discourage players from designing the types of folds that we've already mastered.

Like you say, a protein may explore the entire energy landscape, so helix-rich conformations are perfectly valid decoys. It is up to the protein designer to pick a sequence that disfavors helical decoys!

jeff101's picture
User offline. Last seen 11 hours 58 sec ago. Offline
Joined: 04/20/2012
Groups: Go Science
Can players opt not to share solutions from Phase One?

If a player or group gets a solution in Phase One that they'd rather not share with everyone else, can they opt not to have their solution used in Phase Two? Will only solutions with `Tournament Submission’ as their Title be allowed to be starting points in the Phase Two Contests? Can evo's in Phase One be saved with the Title `Tournament Submission’ so they have the chance to be used in Phase Two?

bkoep's picture
User offline. Last seen 5 hours 45 min ago. Offline
Joined: 11/15/2012
Groups: None
Yes!

Absolutely! Only scientist-shared solutions titled "Tournament Submission" will be considered for Phase Two of the tournament. Regular solutions, and scientist-shares with different titles, will not be shared with the rest of the community in Phase Two.

Evolved solutions will not be considered for Phase Two. We know this isn't ideal (Foldit players are great at working in teams), but we want to make sure all players have an equal shot at Phase Two selections, and we didn't have a good solution for handling mixed or ambiguous attribution in Evolver designs.

jeff101's picture
User offline. Last seen 11 hours 58 sec ago. Offline
Joined: 04/20/2012
Groups: Go Science
Can a player "challenge" his/her own design?

If a player's design from Phase One is chosen for Phase Two,
can that player play the Phase Two contest featuring his/her
own design from Phase One?

bkoep's picture
User offline. Last seen 5 hours 45 min ago. Offline
Joined: 11/15/2012
Groups: None
Sure!

A player may challenge their own design in Phase Two, if they like. This could only ever "weaken" the partition function of that design, so it would not be a very good strategy for winning the tournament. But every challenge contributes to our predictions about the design's energy landscape, so there would still be some scientific value in challenging your own design!

Susume's picture
User offline. Last seen 8 hours 42 min ago. Offline
Joined: 10/02/2011
Challenge your own in Phase 1

A player could challenge their own design in Phase 1, to try to figure out how easy it is to challenge, before they pick one to submit. Can you easily rearrange part of it and get a similar score? If you can, others can as well.

Joined: 09/24/2012
Groups: Go Science
How to select the One solution to submit ?

I don' know which strategy is the best for submitting.

I understand that my criteria should be:

-one of my best scoring designs
-one of my most original design ? (no surfing hotdog etc)
-and still similar to something existing in the nature

But I've absolutely no idea which one has a chance to fold "uniquely" enough (to have a good partition).

My feeling is that sheet-based designs will be easily challenged by helix-based alternatives in phase 2.

jeff101's picture
User offline. Last seen 11 hours 58 sec ago. Offline
Joined: 04/20/2012
Groups: Go Science
Watch out for MALEK residues?

Maybe pick solutions with few helix-favoring MALEK
(methionine, alanine, leucine, glutamate, or lysine)
residues. Maybe pick solutions with alternating
hydrophilic & hydrophobic residues (like sheets often
have) rather than ones with patterns that helices
often have.

jeff101's picture
User offline. Last seen 11 hours 58 sec ago. Offline
Joined: 04/20/2012
Groups: Go Science
Include helix-breaking and sheet-forming residues

https://en.wikipedia.org/wiki/Protein_secondary_structure
says proline and glycine tend to be helix-breakers
while tryptophan, tyrosine, phenylalanine, isoleucine,
valine, and threonine prefer to adopt β-strand conformations.

jeff101's picture
User offline. Last seen 11 hours 58 sec ago. Offline
Joined: 04/20/2012
Groups: Go Science
Will Phase Two let us use the Rebuild Tool?

Often Design Puzzles don't let us use the
Rebuild Tool, yet other puzzles do. Will
Phase Two let us use the Rebuild Tool?

bkoep's picture
User offline. Last seen 5 hours 45 min ago. Offline
Joined: 11/15/2012
Groups: None
Yes

The Rebuild tool will be enabled in the Phase Two contests; however, it has been disabled in the Phase One design puzzle.

Susume's picture
User offline. Last seen 8 hours 42 min ago. Offline
Joined: 10/02/2011
Negative Design?

Is improving the partition function the same as negative design?

bkoep's picture
User offline. Last seen 5 hours 45 min ago. Offline
Joined: 11/15/2012
Groups: None
Basically, yes

In protein engineering, "negative design" is any kind of design that is meant to destabilize a particular state (to increase the absolute energy of that state), rather than stabilize that state (decrease its energy). Usually this is used to bias the partition function towards another desired state. In our tournament, for example, you could try to bias the partition function toward your design by anticipating potential decoys, and changing your design sequence to decrease the score (increase the energy) of those decoys. In this case you would carry out "negative design" of the decoy states.

Negative design can also be useful for improving the specificity of protein interactions. If you design your protein to bind some target, but you discover that your protein also binds to other targets (and causes unwanted side effects, maybe), you could use negative design to eliminate off-target binding. For example, see Figures 2 and 6 of this paper.

On the other hand, there are applications where you might want to reduce the bias in a protein's partition function, so that your protein can exchange between two states easily. Suppose you want your protein to act like a switch, so that it changes state in response to subtle changes in the environment—or maybe you just want to study how a protein exchanges between two folds. See this paper for an extreme example of this kind of negative design.

jeff101's picture
User offline. Last seen 11 hours 58 sec ago. Offline
Joined: 04/20/2012
Groups: Go Science
Player Names in Contest Names?

Will you list which players' designs from Phase One are chosen for Phase Two? Will each Phase Two contest title include the name of the player who designed the contest's starting structure?

Thanks again,
Jeff

bkoep's picture
User offline. Last seen 5 hours 45 min ago. Offline
Joined: 11/15/2012
Groups: None
Yes

The title of each Phase Two contest will include the username of the player who designed the target.

jeff101's picture
User offline. Last seen 11 hours 58 sec ago. Offline
Joined: 04/20/2012
Groups: Go Science
When will Phase Two begin?

Do you know when Phase Two will begin? Will you make a big announcement when it starts? Will Phase Two be a bunch of Science Puzzles or will it be a bunch of Contests?

bkoep's picture
User offline. Last seen 5 hours 45 min ago. Offline
Joined: 11/15/2012
Groups: None
Preparing Phase Two puzzles now

Check back on Monday!

Joined: 03/05/2015
Groups: Gargleblasters
All good questions... I'm wondering too.

I'm eager to see how this works... I like the idea and enjoyed the first round, was interesting to work with the patterns that never scored well in monomers and play around with "off-book" ideas.

Joined: 09/24/2012
Groups: Go Science
Exciting

It's quite exciting to see and challenge all these puzzles together :)

Get Started: Download
  Windows    OSX    Linux  
Windows
(7/8/10)
OSX
(10.7 or later)
Linux
(64-bit)

Are you new to Foldit? Click here.

Are you a student? Click here.

Are you an educator? Click here.
Search
Only search fold.it
Recommend Foldit
User login
Soloists
Evolvers
Groups
Topics
Top New Users
Sitemap

Developed by: UW Center for Game Science, UW Institute for Protein Design, Northeastern University, Vanderbilt University Meiler Lab, UC Davis
Supported by: DARPA, NSF, NIH, HHMI, Amazon, Microsoft, Adobe, RosettaCommons