CASP11 starts next month!

The CASP organizers have shaken things up a bit for the upcoming CASP11 experiment!

One of the major changes is that for each prediction that we submit, we will be required to provide how confident we are with each residue in the chain.
For example, we might report for our model1 submission of a particular target (with length 181) that residues 1-80 are 100% accurate, but residues 81-85 are 50% accurate and 86-181 are 80% accurate. Then for model2 we might have completely different confidence values: residues 1-20 are 50% accurate, 21-180 are 70% accurate, and we have a confidence of 25% for the last residue, 181.
The organizers have told us that predictions with fewer than 2 different confidence values will be rejected, so this is going to add another layer of complexity to CASP!

The more exciting changes are the new initiatives in CASP11:

Modeling based on predicted contacts,
Modeling based on simulated sparse experimental data,

and
Structure prediction based on only the correct predicted contacts

The Assisted Structure Prediction Category was first introduced at the last CASP, but unfortunately Foldit was unable to participate in this new category because the puzzle load of the regular CASP10 targets was already too much.
Since this year they are expanding this category to 3 different rounds (up to 4 rounds if they able to include real cross-linking data for certain targets) we want to make sure Foldit is able to participate in these.

Our plan for CASP11 is to post Foldit puzzles for as many of these Assisted Structure Prediction targets as we can, in addition to the targets in the Refinement Category.

For the Template-Based Modeling (TBM) targets, Foldit has never been able to participate in all the TBM targets, but the organizers have added a new initiative for CASP11:

Evaluation of models based on a perfect alignment to the template.

Instead of giving you 3-5 different alignments (that may or may not be correct) in the Alignment Tool like in previous CASPs, the organizers will provide us with the perfect alignment!
Our plan for the TBM targets in CASP11 is to participate in these cases where we can give you the correct alignment.

This just leaves the Free Modeling (FM) category (where there are no templates), which is of course the most difficult category at CASP.

We are happy to announce that the best-performing FM group at CASP10, keasar, will be teaming up with Foldit at CASP11 via WeFold. You can see how well keasar performed at CASP10 in the pdf of the official rankings

The table below shows all the CASP10 groups that submitted at least one model deemed good enough by the CASP10 assessors of the FM category. At the top, with 8 different targets, is keasar followed by everyone else below (including FOLDIT and 2 of the Foldit teams):

What the keasar team is able to do very well is pick out the best server models (denoted by the corresponding green squares: the server models used by keasar for each target that they won) but the keasar team isn’t as good at improving these server models. This is why we are excited about joining forces with keasar by creating a WeFold team where keasar will provide us with their best server picks, and we’ll post those as Foldit puzzles for you to improve even further!
This is our plan for the FM targets.

We also hope to collaborate with other WeFold groups for all the non-FM puzzles that we post, to see if they are able to pick out your predictions better than we can.

In summary, our plan for CASP11 is for Foldit to participate in the Refinement Category, all the new CASP11 categories, and the FM category via WeFold.

Speaking of categories, there will be a new Foldit category for CASP11 (and perhaps a sub-category for puzzles using contacts) http://fold.it/portal/categories

( Posted by  beta_helix 79 1450  |  Fri, 04/04/2014 - 18:20  |  4 comments )
5
Joined: 04/15/2012
Groups: Beta Folders
Cool...

Glad that keasar will be joining in, as it sounded like choosing models was something we've had trouble with (or I could be making things up :P).

Might we get some contact puzzles to practice with? We usually don't get many, and I don't thing any have been done since NewChapter (correct me if I'm wrong, I haven't been all too active).

beta_helix's picture
User offline. Last seen 21 hours 41 min ago. Offline
Joined: 05/09/2008
Groups: None
Practice Contact Puzzles

The first of these is now posted: http://fold.it/portal/node/997414
We plan to post of many different Contact Puzzles before CASP11 begins (as you are correct that we haven't posted any since the NewChapter update).

Susume's picture
User offline. Last seen 14 hours 43 min ago. Offline
Joined: 10/02/2011
How much should we change

How much should we change the keasar-provided server models? Will these be like refinement puzzles? In the past for free modelling we were encouraged to submit something that was different enough from the server model to be recognizably novel, rather than just a copy of the server model.

Also, will these be multi-start, or just the top keasar pick for each puzzle?

beta_helix's picture
User offline. Last seen 21 hours 41 min ago. Offline
Joined: 05/09/2008
Groups: None
Great question!

These will most likely be multi-start puzzles, where keasar will provide us with their 5 submissions, and you will try to improve them.

Given keasar's 5 selections, there are two different aspects that are interesting:

1) If keasar is indeed able to consistently select the best 5 server models, are Foldit players able to pick out which of the 5 is the closest to the native? This would be a very interesting result if all of you converge onto the same model with your top-scoring Foldit predictions given a multi-start puzzle.
Then we would have a really cool pipeline where given a bunch of server models, keasar+Foldit via WeFold would be able to tell biologists which server model is the best!

2) If keasar is indeed able to consistently select the best 5 server models, are Foldit players able to significantly improve those models even further?
Of course this would be an even more exciting result, but it is also more of a gamble... since by trying to submit something different enough from keasar's starting model, it could result in making the server models much worse.

We will bring these questions up to the WeFold team, as they will be the ones making the final selection for the keasar+Foldit predictions, and let you know what the plan is.

Thanks again for the great question!

Get Started: Download
  Windows    OSX    Linux  
Windows
(7/8/10)
OSX
(10.7 or later)
Linux
(64-bit)

Are you new to Foldit? Click here.

Are you a student? Click here.

Are you an educator? Click here.
Search
Only search fold.it
Recommend Foldit
User login
Soloists
Evolvers
Groups
Topics
PlayerPuzzleScore
Top New Users
Sitemap

Developed by: UW Center for Game Science, UW Institute for Protein Design, Northeastern University, Vanderbilt University Meiler Lab, UC Davis
Supported by: DARPA, NSF, NIH, HHMI, Amazon, Microsoft, Adobe, RosettaCommons