CASP10 update

Now that CASP10 is over, we've been able to analyze your Foldit predictions for the CASP10 targets whose natives have been released so far.

We won't know how well we all did overall until the CASP10 meeting in early December, which is when the assessors present their results. Until then, we can look at the natives that have come out already and at least see how Foldit compared to the Baker Lab's human CASP team (since we don't have the human submissions for any of the other groups that competed in CASP10).

Most of the targets whose natives have been released are from the refinement category, and we've noticed some very remarkable Foldit results!

Below are two plots for the same target, TR663, with all your Foldit predictions shown in green.
The black dotted line represents the starting model given to you in the puzzle (it's shown as a line because it scored so poorly, it would be off the plot).
In blue represents the native after wiggling/shaking with Rosetta (so closer to these blue points is where you want to be on both plots).

The left plot is using the GDT metric: http://en.wikipedia.org/wiki/Global_distance_test
The right plot is using the RMSD metric: http://tinyurl.com/74p4qtn

On the GDT plot you want to be as close to the right as you can (1.0 would be a perfect match to the native).
On the RMSD plot you want to be as close to the left as you can (0 would be a perfect match to the native).

In red are the 5 submissions by the Baker Human CASP10 Team, and in gold are the 3 submissions that we submitted using your Foldit predictions (the other two submissions were from the other TR663 Foldit puzzle).

You can see that many of your Foldit predictions improved the starting structure significantly, but we were very bad a picking these models out!
You'll also notice that our human Baker Lab CASP10 team was also unable to submit models that were significantly closer to the native than the initial refinement model.

You're probably wondering what the purple dot represents.
It is a Foldit prediction that was submitted by one of the three Foldit Groups that submitted their own predictions to CASP10!
Not only was the Foldit Group able to greatly improve the starting model, but they were also able to select that structure (out of all the top-scoring solutions from their Foldit Group) and submit it!

This is very exciting and it was not an isolated case for just TR633!

We always knew that Foldit players were amazing at folding, and now you are showing that you can excel at model selection as well (which, as you can see, is a very tough problem!).

We want to thank you all for your hard work during CASP10. We realize the protein size and puzzle load got a bit crazy at times, and we really appreciate your patience with it.
For all of you that just survived your first CASP, congratulations and welcome to the club!
For all of you who had been through it before (so you knew what you were getting into) and still put in so much time & effort this past summer: we can't thank you enough. Thanks for all the many hours of folding, as well as all the help you gave to the new players that had never gone through this before. We really appreciate all of you answering the many CASP-related questions and pointing players towards the relevant CASP-related websites.

Keep up the great folding everyone, we hope you all got a bit of rest after CASP10 and are ready for some exciting design puzzles that are coming soon!

( Posted by  beta_helix 122 3289  |  Fri, 09/21/2012 - 23:22  |  3 comments )
0
harp's picture
User offline. Last seen 27 weeks 3 days ago. Offline
Joined: 10/20/2011
Thanks for this Beta, very

Thanks for this Beta, very interesting and can't wait to see what Dec. will bring.

Joined: 11/03/2011
I have some questions 1/if

I have some questions
1/if you want to be at the extreme right of the gdt plot
aren't there a few foldit solutions which were not submitted
but were significantly even better than the purple dot (tvdl's ??)
would it be interesting to identify these?

2/if optimisation criterium is based on internal energy
that means that everybody is working to be as down as possible
in the gdt plot.Isn't there a contradiction with the fact
to be as right as possible?

Renton's picture
User offline. Last seen 1 year 44 weeks ago. Offline
Joined: 04/18/2010
Does this mean fold.it players/software are improving ?

If we compare the results over the last four years, against the other (international)teams folding and designing software for folding proteins prediction; has the fold.it software measured up to the needs of the community?? does the human approach still out perform the computers ?

Get Started: Download
  Windows    OSX    Linux  
Windows
(XP/Vista/7)
OSX
(Intel 10.5 or later)
Linux
(64-bit)

Are you new to Foldit? Click here.

Are you an educator? Click here.
Search
Only search fold.it
Recommend Foldit
User login
Soloists
Evolvers
Groups
Topics
Top New Users
Sitemap

Supported by: UW Center for Game Science, UW Department of Computer Science and Engineering, UW Baker Lab, DARPA, NSF, HHMI, Microsoft, and Adobe