Partition Tournament Final Results
Our Protein Design Partition Tournament concluded yesterday. In the final week, we ran four regular puzzles for the tournament front-runners, so that the greater Foldit community could help us explore the energy landscapes for those designs. Below, we present the tournament results and discuss the final outcomes.
We will host a Science Chat next Tuesday, October 23 at 21:00 GMT. We'll be happy to discuss any questions about the tournament, as well as any other recent Foldit activity! Leave your questions in the comment section below!
The final rankings for the tournament are as follows:
The champion of the tournament is Galaxie, whose design resisted challengers more effectively than any other design. No challenger was able to find a decoy with energy comparable to the design structure; the highest scoring decoy has a probability of only 10-19 (one in ten billion billion)!
Galaxie's winning design
Congratulations, Galaxie! Galaxie will receive the brand new Partition Tournament achievement (coming soon!). Galaxie was also an exceptional challenger in Phase Two, and found the highest-scoring decoy state for 7 different targets!
Two other players will receive the Partition Tournament achievement, for their outstanding contributions as challengers in Phase Two of the tournament:
The most prolific challenger was robgee, who found 20 decoy states for 14 different targets—more than any other challenger!
The most venturous challenger was Mike Cassidy, who found a decoy state for Partition Puzzle (B): MicElephant with an RMSD of 13.5 Å (off the charts—literally!).
Recall that the initial motivation for this tournament concerned our protein design strategy. In a typical Foldit design puzzle, players optimize the absolute energy (the Foldit score) of their design. However, the success of a protein design depends not on its absolute energy, but on its energy landscape. In theory, a design with a mediocre Foldit score can still have an excellent energy landscape, and might be expected to fold up well in the lab. Inversely, a high-scoring Foldit design could have a problematic energy landscape (with other high-scoring decoys), and would be expected to fold poorly. The Partition Tournament was set up to evaluate Foldit players' designs based on their energy landscapes, instead of their absolute energy.
Note that, of the five top-scoring tournament submissions selected for Phase Two of the tournament (from matosfran, actiasluna, phi16, Galaxie, and fiendish_ghoul) all five performed very well in the tournament. None of these designs were overtaken by a high-scoring decoy. This is encouraging, because it suggests that our normal strategy (i.e. optimizing for absolute energy) is an effective shortcut for finding favorable energy landscapes.
However we also see that many of the lower-scoring submissions still performed well in the tournament, and maintained high partition scores. These designs were able to resist challenges from other Foldit players, and seem to have favorable energy landscapes. This indicates that Foldit players may have protein design skills that are not captured by our energy function. If we focus solely on optimizing absolute energy, then we're probably going to miss out on some perfectly good protein designs.
These are exciting results, and we're very keen to keep exploring protein design with Foldit players! Nevertheless, we should recognize a significant limitation of the tournament results.
Most challengers tended to stay very close to the designed structure. In many of the energy landscapes below, we see the majority of solutions (usually a dense cluster of black dots) right around the 2.5 Å RMSD cutoff. The average decoy had an RMSD of only 4.0 Å.
This was also clear in our weekly tournament updates (see here and here), where we looked more closely at some players' high-scoring decoys. In most cases, the decoys were largely identical to the design structure, and only small, localized regions of the design were refolded by challengers.
These low-RMSD decoys only inform us about how these designs could partially unfold. The challengers have revealed how certain regions of the protein might unfold locally. However, it doesn't tell us much about the stability of the protein core, and how the protein might be able to refold globally. For this we would want to see many more solutions that are very different from the design structure, and that explore the energy landscape distant from the designed structure.
If we were to repeat this tournament, we would need to find ways to encourage more explorational challenges. We could imagine applying a bonus that scales with RMSD (as has already been suggested by some of you), by concealing the designed structure from challengers, or some other means to encourage broader exploration of the energy landscape.
Many of the tournament submissions are excellent designs, and these tournament results are promising, but unfortunately we can't be confident enough to test these in the lab until we have a better picture of the global energy landscape. For that, we'll submit these designs to Rosetta@home for ab initio structure prediction, which is very effective for finding decoys that are globally-refolded.
In summary, the results from the Protein Design Partition Tournament support the idea that our regular design strategy—though not perfect—is still effective for finding designs with favorable energy landscapes.
Foldit players were very effective in this tournament at finding locally-unfolded decoy states. In the weekly updates we saw exactly how this happened, and proposed ways to avoid designs that can unfold locally. Specifically, designers should focus on building a substantial, deeply-integrated core that involves all regions of the protein, and avoid long stretches of completely polar residues. Otherwise, parts of your protein may be able to unfold locally while maintaining a high score.
Partition Puzzles Summary
Below are the final energy landscapes and partition functions for all the Partition Puzzles from Phase Two of the tournament. For an explanation of these plots, see this previous blog post. Thanks to all those who participated in the tournament!
( Posted by bkoep 113 1277 | Wed, 10/17/2018 - 22:48 | 3 comments )