Partition Tournament Update: Final Week
We are now in the final week of the Protein Design Partition Tournament! We apologize for the delayed update; some unrelated technical problems kept us from running our weekly analysis yesterday. Thank you for your patience!
Today we were able to analyze the results from all current partition puzzles, plus the two regular puzzles that were posted last week: 1580: Partition Puzzle: fiendish_ghoul and 1582: Partition Puzzle: wisky. As you can see in the summary results below, the regular puzzles resulted in many additional solutions, and more thorough exploration of the energy landscapes for the designs by wisky and fiendish_ghoul.
As of 18:00 GMT October 10, the rankings are as follows:
In the final week, we will post regular puzzles for the top four designs in the tournament. Like last week's puzzles, each of these regular puzzles will be available for four days, and players will be able to load solutions from the original Partition Puzzles. Yesterday, we released the first of these, with 1583: Partition Puzzle: LociOiling. Today, we are releasing 1584: Partition Puzzle: actiasluna, and will follow up the next two days with puzzles for matosfran and Galaxie.
Do your best to find high-scoring decoys in these final puzzles! This is your last chance to explore the energy landscapes for these designs before we calculate the final partition functions to determine the tournament winner!
On Tuesday, October 16 at 23:00 GMT, the last of these puzzles will close, along with all remaining Partition Puzzles. On Wednesday, October 17 we will post the final results of the tournament.
Partition Puzzles Summary
Below are the energy landscapes and partition functions for all remaining Partition Puzzles, as of 18:00 GMT October 10. For an explanation of these plots, see this previous blog post.
( Posted by bkoep 83 566 | Wed, 10/10/2018 - 21:54 | 1 comment )
Partition Tournament Update: More Challenges
We are now two weeks into Phase Two of the Protein Design Partition Tournament! In the Partition Puzzles, challengers continue to find low energy decoys, revealing more of the energy landscape for each design. The rankings have changed significantly since last week, and we expect them to keep shifting over the next two weeks of the tournament!
As of 18:00 GMT October 1, the rankings are as follows:
This week we will start reposting some of the Partition Puzzles as regular Foldit puzzles. We will focus on puzzles that still have a high partition score, but that have not seen very much activity from challengers. We hope that more people will attempt the regular puzzles, and explore more of the energy landscape for these designs. Any players that have participated in a Partition Puzzle will be able to load their work if it is reposted as a regular puzzle.
Unfortunately, another design has fallen out of competition. Partition Puzzle: alcor29 has dropped to a score of 0.00, after challengers found a decoy structure with energy lower than the design, by about 3 kcal/mol (+30 Foldit points). We will be closing this puzzle early, so that challengers can focus on the remaining Partition Puzzles.
Taking a closer look at the decoys from this puzzle, we see that the α-helix and β-hairpin in the foreground can pack in different orientations with the larger β-sheet in the back. We loaded the original design into Foldit with a Core Existence Objective (which was not available in the original design puzzle), and we can see that this design falls short of the typical "30% core requirement" that we like to see in protein designs. In fact, only about 18% of residues in this design contribute to the protein core. The implication is that there are not very many interactions between the different parts of this protein, allowing them to refold in different orientations with minimal consequences.
This result highlights that a substantial, tightly-packed core is essential for a well-folded protein. Good core packing, with interactions between all parts of the protein, helps to ensure that there is one (and only one) stable configuration for the protein!
Partition Puzzles Summary
Below are the energy landscapes and partition functions for the remaining 16 Partition Puzzles, as of 18:00 GMT October 1. For an explanation of these plots, see this previous blog post.
Keep up the good work! There has been a lot of progress since last week's results, but there are surely more high-scoring decoys to be discovered! Check back next Tuesday, October 9 for the next tournament update!
( Posted by bkoep 83 566 | Mon, 10/01/2018 - 23:17 | 1 comment )
Partition Tournament Update: Initial Results
Phase Two of the Protein Design Partition Tournament has been underway for one week, now. Thank you, to all the Foldit players that have been participating! All 20 Partition Puzzles have seen initial challenges, and already we’re seeing that some designs are more resistant than others.
To recap: the Partition Puzzles will be evaluated by their partition functions, which we explained in a previous blog post. In order to calculate the partition function for a puzzle, we cluster all of players solutions to identify decoy states. The relative energy of those decoy states tells us how the ensemble will be distributed over those states.
In order to rank the tournament designs, we’re using a "Partition Score” to summarize how well each design resists challenges from other players. This score measures how much of the partition function has been claimed by challengers, on a logarithmic scale, where a higher score indicates a more stable protein. For example, a Partition Score of 12.0 means that challengers have claimed 10-12 (one trillionth) of the partition function; likewise, we could say the design is about 1012 times more likely to fold into the designed structure than all of the decoys combined.
As of 18:00 GMT September 24, the rankings are as follows:
Three puzzles have a Partition Score of 0.00, meaning that challengers have found decoys that score even better than the original design. We are closing these puzzles early, so that challengers can focus on the other Partition Puzzles. Below, we'll take a closer look at the solutions from these puzzles, and what features might have led to their demise. If you want to skip ahead, the summary updates for all Partition Puzzles are at the bottom of this blog post.
Protein design pitfalls
First, in Partition Puzzle: MicElephant, we see that challengers have found a decoy with an energy that is lower than the design by about 6 kcal/mol (+60 Foldit points).
Taking a closer look at the decoy structure, we see that challengers have refolded one of the β-strands into a helix. In the original design, this strand was designed with all polar residues. Without any hydrophobic residues to be buried, this section of the protein will happily flop around in solution. You might think that it would be bad to break the ladder-like hydrogen bonds that were designed between this strand and its partner—however, these surface residues can compensate by making new hydrogen bonds with the surrounding water.
Next, in Partition Puzzle: ManVsYard, challengers have found a decoy that is lower in energy by about 4 kcal/mol (+40 Foldit points), as well as a second decoy with an energy that is about the same as the designed structure.
The low-scoring decoy exposes the short helix that was packed at the top of the β-strands. Again, we see that this section was designed with entirely polar residues, which are happy to interact with the surrounding water instead of the protein. In the second decoy, challengers have completely refolded a β-strand into an α-helix. This is a little surprising, since the designed residues in this section form an alternating pattern of polar and non-polar residues (which normally favors β-strands). However, this strand may not have been packed tightly enough in the protein core; it is able to make equally favorable interactions in the protein core when refolded as a helix.
Finally, in Partition Puzzle: manu8170, challengers found two decoys that are lower in energy by about 5 kcal/mol (+50 Foldit points).
The decoys differ from the design structure in a couple regions. First we see that the terminal helix can be unwound, and is "fraying" at the end. In the original design, these end residues (again, all polar residues) were floating off in solvent, and did not make very strong interactions with the rest of the protein; these residues can be unfolded relatively easy. We also see that this design has long loops that are made mostly of polar residues, and do not form regular secondary structure. Such polar, unstructured loops are expected to be flexible and disordered in solution; challengers were able to refold the loops while keeping the rest of the structure intact.
The clear message from these early results is that long stretches of polar residues can be easily unfolded. The burial of non-polar residues is the main driving force of protein folding, and it's important that every region of a protein contributes to its core!
Partition Puzzles Summary (September 24)
Below are the energy landscapes and partition functions for the remaining 17 Partition Puzzles, as of 18:00 GMT September 24. For an explanation of these plots, see this previous blog post.
Some puzzles have seen very little activity, and their rankings may be artificially boosted due to a lack of challengers. There is still plenty of opportunity to find high-scoring decoys in your opponent's Partition Puzzles!bkoep 83 566 | Mon, 09/24/2018 - 21:18 | 6 comments )
Partition Tournament: Phase Two
Thanks to all the players that submitted designs for the Protein Design Partition Tournament! At the bottom of this post are the submissions that were selected for Phase Two of the tournament.
We’ve created 20 new Partition Puzzles for these selections, where the starting structure for each puzzle is the original design. Each partition puzzle has an RMSD objective, which requires all solutions to have RMSD > 2.5 Å relative to the starting structure. This means that players will have to significantly change the starting structure in order for scores to register. The starting solution is marked with secondary structure predictions from PSIPRED. In some cases the PSIPRED predictions do not agree with the designed structure, and might suggest ways that challengers can refold the starting structure. Remember, the tournament submissions will ultimately be evaluated by how well they withstand challenges, as measured by their partition functions.
These puzzles are available in the regular puzzle menu and anyone can play them! They are listed as “Expert” difficulty; the only reason for this is so they are sorted below other regular puzzles (since Foldit sorts the Puzzle Menu by difficulty), to reduce clutter for players who do not wish to participate. The puzzles will remain online for four weeks, and expire on 23:00 GMT October 15.
We do not expect everyone to play all 20 partition puzzles! These puzzles are all “voluntary”, for now, and each puzzle is worth zero points. However, as the tournament progresses, at least a couple of these puzzles will be re-released as regular puzzles (for points), to spur more challenges from the greater Foldit community. Sharing will be enabled in the re-released puzzles, so anyone who participates early in a partition puzzle will have a leg up if it is re-released later as a regular puzzle. We will also present new Achievements, at the end of the tournament, to those players that are most successful in challenging others’ designs.
We encourage all players to check out a couple partition puzzles below, and at least poke around with some of the designs you find most interesting. By playing the partition puzzles you reveal the energy landscapes of these proteins, which helps us determine whether the designs are likely to fold as intended. We also hope that, by playing these puzzles, players might be able to learn from the designs of others. What features of a protein design make it easy or difficult to challenge? This is something of an open question in protein design research, and we think Foldit players could provide some insight about this aspect of protein design.
Check out the partition puzzles, which are linked below!
Randomly-selected designsbkoep 83 566 | Mon, 09/17/2018 - 22:07 | 0 comments )
Protein Design Partition Tournament
We are announcing a friendly protein design tournament for Foldit players! This tournament is something of an experiment, and we are not sure exactly how it will unfold. Participation in the tournament is completely voluntary, and we will continue to post regular Foldit puzzles during the tournament, for those who do not wish to participate. There will be no prizes (except for bragging rights and a rare new Foldit Achievement), and we cannot guarantee that any scientific results will arise from the tournament. The main purpose of the tournament is to have fun folding proteins, and to inspire Foldit players to think differently about protein design; but we do also think the tournament could lead to higher quality protein designs.
Unlike our regular design puzzles, in which players compete to design proteins with the best absolute energy, this tournament is designed to reward proteins with the best energy landscape. For more discussion about energy landscapes, see parts one and two of this blog series.
The tournament will take place in two phases, over the next 6 weeks.
Phase One: Defense
The first phase of the tournament will take the form of a Foldit design puzzle, similar to the regular Monomer Design puzzles that are posted every week. Players will have two weeks to craft their best protein design, which will have to defend itself in Phase Two. To enter the tournament, players must share their chosen design with scientists, using the Upload for Scientists button in the Save Solution menu. When you save your solution, give it the title ‘Tournament Submission’ in the Save Solution dialog box. Every player is allowed one tournament submission. If a player submits multiple solutions, only the most recent solution will be accepted. For the sake of competition logistics, team play will not be allowed in the tournament; only soloist solutions will be accepted.
The Phase One design puzzle will include two objectives:
Residue Count: Designs may contain 70-100 residues, at a cost of 32 points per residue.
Secondary Structure: Designs may be up to 10% α-helix.* Additional helices will be penalized at 10 points per residue.
*We will accept Phase One submissions regardless of their secondary structure content. However, we’d like to discourage players from submitting helical bundles and ferredoxin-like folds (a.k.a. "surfing hotdogs") that typically score well in regular design puzzles. Rather, this is a chance for players to showcase designs that aren’t normally competitive in regular design puzzles.
Phase Two: Offense
Twenty Foldit player submissions will be selected to advance to Phase Two:
Five will be the five top-scoring submissions from Phase One.
Five will be hand-selected by the Foldit team, on merits of creativity and plausibility.
Ten will be chosen at random from the remaining submissions.
For each of the 20 selections, we will create a special Partition Contest using the selected protein design. Each contest will be set up as prediction puzzle, similar to the regular De-novo Freestyle puzzles, except that the starting structure will be the fully-folded design. All 20 Partition Contests will be open to the entire Foldit community and will remain online for four weeks, during which time the selected designs will be vulnerable to “challenge.” Any Foldit player can challenge a design by joining its Partition Contest and attempting to refold the design into another high-scoring decoy structure.
The Phase Two contests will include an RMSD Objective: All solutions must differ from the starting model with an RMSD of at least 2.5 Å.
Ultimately, each design in the tournament will be evaluated by its partition function (described in the previous blog post), based on the decoys found by challengers in the Phase Two contests.
By challenging a design and finding a high-scoring decoy, you show that your opponent's sequence does not have 100% probability of adopting the folded structure, and that its partition function must be shared with your decoy structure. You effectively stake a claim in the partition function of that design; the higher the score of your decoy, the larger your claim in the opponent's partition function.
A player may make multiple challenges against a single design; in some cases, it may be more effective to make many moderate-scoring challenges rather than a single high-scoring challenge. In order to calculate the partition function for a design, we will cluster all of the contest solutions to identify representative states. Then, we’ll use the partition function to determine the probability of each state.
Unfortunately, we cannot calculate the partition function on the fly, so players can only estimate how well a design is resisting challenge by following the Contest leaderboards. However, we will post weekly updates throughout Phase Two, with updated partition functions for all 20 Contests.
The champion of the tournament will be the protein design with the highest probability, as determined by its partition function.
There will also be Achievements for the most effective challengers, who are able to stake the greatest claims in the partition functions of their opponents.
Finally, we’d like to point out that, while players may be tempted to aim for a high-ranking design in Phase One, what really counts is how well each design can withstand challenges in Phase Two. If you design a high-scoring protein in Phase One, but its sequence is also compatible with many high-scoring decoy structures, then in Phase Two challengers will easily find high-scoring decoys and stake large claims in your design’s partition function.
The Phase One design puzzle is online now! Happy folding!( Posted by bkoep 83 566 | Thu, 08/30/2018 - 20:54 | 26 comments )