The New Grand Challenge
The release of "The New Challenge" puzzle signifies the start of a new sequence of puzzles aimed at determining the specific ways in which human puzzle solving can outperform best known automated methods. These puzzles are a result of the analysis of all game play results since we introduced the game in May. The puzzles we will be unrolling from now on are specifically selected to confirm certain hypothesis about the nature of problems where the collective "game brain" will outperform other search methods.
The CASP8 results showed us that protein exploration through foldit puzzles is a promising new paradigm for 3D structural exploration. It did not, however, point out the how and why people are able to get such good solutions. The "grand challenge" puzzles will hopefully provide that information. Should the results prove to be interesting, we will shortly afterward submit a research paper reporting our findings.
So let the Grand Challenge begin! We're counting on you.
We've got some very exciting news! The table below shows our placement in CASP8 puzzles. Click on the Target column to go to the respective CASP8 page showing results. The rank column shows Foldit solution rank according to the default CASP8 sorting criteria (GDT_TS). Entries column shows the number of submissions from biochemistry labs worldwide. The puzzle column shows the corresponding foldit puzzle.
|Target||Rank||Entries||Puzzle||Group / User|
|TR389||3||71||TS389_5||Another Hour Another Point|
|TR389||7||71||TS389_1||Another Hour Another Point|
|TR461||2||83||TS461_1||Another Hour Another Point|
|TR461||19||83||TS461_5||Another Hour Another Point|
|TR469||2||74||TS469_5||Another Hour Another Point|
|TR469||3||74||TS469_4||Another Hour Another Point|
|TR488||1||77||TS488_1||Another Hour Another Point|
|TR488||3||77||TS488_3||Another Hour Another Point|
|TR488||26||77||TS488_5||Another Hour Another Point|
|TS492||2||527||TS492_3||Another Hour Another Point|
|TS492||96||527||TS492_5||Another Hour Another Point|
Looking at the numbers, it appears that Foldit players did amazingly well! You placed in top 3 in a number of puzzles and it seems that we apparently won one of the puzzles. Congratulations, Foldit players! You're amzing.
Looking beyond numbers, the preliminary conclusion by biochemist is that foldit players are on par, but not better than protein folding experts at trying to solve the same problem with all tools available to them. It also appears that foldit outperformed all fully automated server submissions. Hopefully over time foldit can do even better, but being able to produce solutions of same quality as experts means that the top science research can now also be done outside of labs by game players, significantly speeding up the process of scientific advancement! As developers, we are truly inspired by this.
Of course, we didn't do well on all puzzles, but even this has been very useful to us as we're evolving the game further to significantly improve over these findings.
Some caveats that should be mentioned:
- there is a considerable amount of discussion over what is the best way to score the quality of the solution, and GDT_TS is just one of them that CASP8 sorts their solutions by. The ranking would be slightly different with different metrics, but the overall conclusion should not change.
- it is possible that the specific starting point including all the constraint bands played a significant role in the success of foldit solutions, and we may have gotten a particularly good one from the Baker lab. To understand this better, we'll be introducing new puzzles (more on that tomorrow).
- foldit did not have the sequence allignment tool that is fundamental for protein prediction. Scientists use sequence alighment a lot to find the best score. For next CASP challenge we will make sure that these tools are also part of the game, which should significantly enable the gameplay and the resulting proteins.
Welcome to Developers Blog
This is the place where we will describe some of the outcomes and results of your folding work, provide a glimpse of the future challenges, and developments, and in general give you a better sense of where we are and where the foldit hopes to go in the future.
Today, I'd like to draw your attention to the evolution videos that we will be creating for each finished puzzle. The videos show how best performing group has evolved the protein to it's final state. The individual contributor of each improvement is displayed at the bottom. The duration of each individual transformation is proportional to the amount of score change. We fould these videos very useful, and we hope that the will be particularly useful for players looking for intuition and strategy that would lead to higher scores. You can find them on most older puzzle pages (for example see the Turkey puzzle). There is a also a video link on the puzzle list page http://fold.it/portal/puzzles?page=1. Currently there is only one video per puzzle, but over time we will have videos for the top 3 groups and top 3 solo solutions.
Let us know what you think by posting comments below.