Speaking of CASP X

Case number:699969-997318
Topic:General
Opened by:spmm
Status:Closed
Type:Suggestion
Opened on:Wednesday, March 26, 2014 - 08:10
Last modified:Sunday, April 20, 2014 - 09:49

I am and perhaps many others are, very out of practice doing the types of puzzles we had in CASP IX, and I haven't touched the Alignment Tool for a very long time.
I appreciate you are still calibrating and was hoping we could start to get a few simple exercises for a warm up?

(Wed, 03/26/2014 - 08:10  |  9 comments)


brow42's picture
User offline. Last seen 6 days 4 hours ago. Offline
Joined: 09/19/2011
Groups: None

Scientists should note that the overwhelming sentiment of the community is that we will not be testing many starts on multistart puzzles, and those that do will only do so for, say, a day or less for each, and thus won't be optimizing their scores.

beta_helix's picture
User offline. Last seen 6 days 17 hours ago. Offline
Joined: 05/09/2008
Groups: None

Thank you both for your feedback about this, especially since CASP 11 starts a little over a month from now! http://predictioncenter.org/casp11/index.cgi

I will write up a blogpost about our plans for CASP 11, including Foldit's participation in the 2nd WeFold collaboration.

I am still waiting for the final edit of the WeFold paper to be posted, it's still in the "Accepted Article" section of the Proteins website: http://onlinelibrary.wiley.com/doi/10.1002/prot.24538/abstract .
Attached is that version of the WeFold paper if you are interested, since you are all co-authors! :-)

beta_helix's picture
User offline. Last seen 6 days 17 hours ago. Offline
Joined: 05/09/2008
Groups: None

CASP11 UPDATE:
The organizers have just posted a detailed description of the CASP11 experiment on their webpage: http://predictioncenter.org/casp11/index.cgi

They have made some significant changes to the experiment by launching additional prediction challenges and introducing important changes to how they will assess predictions.

I will address these in my blogpost, which should be posted by the end of this week.

spmm's picture
User offline. Last seen 41 weeks 3 days ago. Offline
Joined: 08/05/2010
Groups: Void Crushers

Thanks beta, will we have a new category for CASP11? So everyone starts at the same place, no advantage or disadvantage.

brow42's picture
User offline. Last seen 6 days 4 hours ago. Offline
Joined: 09/19/2011
Groups: None

I expected our CASP scores to be zeroed out at the start of each casp....the other categories have limited windows too.

beta_helix's picture
User offline. Last seen 6 days 17 hours ago. Offline
Joined: 05/09/2008
Groups: None

Blog is now up: http://fold.it/portal/blog
New CASP11 category as well: http://fold.it/portal/node/997398
The CASP category has been renamed to CASP10: http://fold.it/portal/node/992762

jeff101's picture
User offline. Last seen 13 hours 42 min ago. Offline
Joined: 04/20/2012
Groups: Go Science

http://fold.it/portal/node/997397 says the following:

"One of the major changes is that for each prediction that we submit, we will be required to provide how confident we are with each residue in the chain. For example, we might report for our model1 submission of a particular target (with length 181) that residues 1-80 are 100% accurate, but residues 81-85 are 50% accurate and 86-181 are 80% accurate. Then for model2 we might have completely different confidence values: residues 1-20 are 50% accurate, 21-180 are 70% accurate, and we have a confidence of 25% for the last residue, 181. The organizers have told us that predictions with fewer than 2 different confidence values will be rejected, so this is going to add another layer of complexity to CASP!"

This leads me to ask the following:

For teams that submit their own results in CASP11, how do you suggest we estimate these confidence values? Will the Foldit Team provide us with these confidence values or the tools necessary to find them?

beta_helix's picture
User offline. Last seen 6 days 17 hours ago. Offline
Joined: 05/09/2008
Groups: None

That is a great question, jeff101, and one that we have been discussing with those teams that submitted their own models in CASP10.
We have not come up with an answer yet, but as the time constraints during CASP are already difficult enough to deal with, we are leaning towards providing those teams with our suggested confidence values.

If there are any other Foldit Groups that would like to submit their own Foldit predictions in CASP11, please PM both bkoep & I so we can tell you more about the process.

Be forewarned, however, that submitting your own predictions to CASP involves a great deal of work, under very difficult time constraints (because there is never much time between when a Foldit puzzle closes and when that CASP target is due). You only have that very short window of time to do the analysis of which 5 top-scoring models from your Foldit Group you want to select!
Although hopefully the Share with Scientists button will help out with that a little bit (if as you are playing the puzzle you know you want to select a model that might not be the best-scoring).

If your group is considering submitting your own models to CASP11, you might want to ask some of the players who did it in CASP10 for their opinion. They will be able to describe the time commitment required, etc.

spmm's picture
User offline. Last seen 41 weeks 3 days ago. Offline
Joined: 08/05/2010
Groups: Void Crushers
Status: Open » Closed

tx beta please see blog posts for further info

Sitemap

Developed by: UW Center for Game Science, UW Institute for Protein Design, Northeastern University, Vanderbilt University Meiler Lab, UC Davis
Supported by: DARPA, NSF, NIH, HHMI, Amazon, Microsoft, Adobe, RosettaCommons