6 replies [Last post]
Susume's picture
User offline. Last seen 4 days 19 hours ago. Offline
Joined: 10/02/2011


Google's AI DeepMind (competing in CASP under the name A7D) has placed first in the 2018 CASP protein structure prediction tournament, which brings together the top protein prediction teams from all over the world every 2 years. This year's CASP rankings can be viewed at http://predictioncenter.org/casp13/index.cgi - Rankings links are on the right.

The foldit community as a whole and some foldit teams competed in CASP 10 (2012) and CASP 11 (2014).

Joined: 05/19/2017
Groups: None
CASP Participation

I was just reading about this on Twitter; exciting news indeed! Machine learning and pattern recognition still has ways to go but the results have the potential to usher in a new era.

CASP in general is something I've been really curious about and I'm wondering if there are any plans to have Foldit involvment in 2020. I joined too late to get in on any of the CASP action you mentioned before, and I'd love to know what it's like to compete with the cutting edge.

spvincent's picture
User offline. Last seen 7 hours 32 min ago. Offline
Joined: 12/07/2007
Groups: Contenders
Foldit didn't do particularly

Foldit didn't do particularly well in previous CASP competitions and it was a lot of extra work on the part of the Baker lab so I suspect it's unlikely we'll be participating in the future.

Joined: 05/19/2017
Groups: None

That's quite the shame but I can see why there'd be a lot of hurdles to get this sort of thing running. Thanks for filling me in.

beta_helix's picture
User offline. Last seen 13 hours 55 min ago. Offline
Joined: 05/09/2008
Groups: None
Foldit in CASP

Here are the final results for CASP13 (showing how DeepMind, whose CASP id is "A7D", outperformed every other group overall): http://predictioncenter.org/casp13/zscores_final.cgi

The major factor with Foldit's participation in CASP is that we are severely limited by protein size.
The majority of CASP targets are not <180 residues, so because of this we are unable to participate in most targets.

The other issue is that even if Foldit could handle all the targets, it would simply be too many puzzles.
We would end up with either ~10 puzzles up at a time, or we could only post puzzles up for a few days each.

Because of this, we tried to strategically select which CASP targets Foldit would "compete" in. (It's not supposed to be a competition... it's an "experiment" ;-)
The trouble with doing this, is that the CASP organizers only rank groups that participate in every target in a particular category (which makes sense, otherwise someone could just "pick and choose all the easy targets")

So, for Foldit, the CASP category where all of you performed the best was Refinement (many of the targets were relatively small) and Foldit outperformed all groups on many targets.

Check out Table II in: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4249725/ for CASP10
Supplementary Figure 2 in: https://media.nature.com/original/nature-assets/nsmb/journal/v18/n10/extref/nsmb.2119-S1.pdf for CASP9
and the most recent WeFold paper for CASP11: https://www.nature.com/articles/s41598-018-26812-8

One thing I am certain of, if there is ever a CASP category for fitting Electron Density... nobody could touch you!
It's not that far fetched, they have a "Data Assisted category" that was expanded this year to include sparse NMR data and FRET: http://predictioncenter.org/casp13/zscores_final_assisted.cgi
.... so maybe density maps will factor in soon!

jeff101's picture
User offline. Last seen 3 hours 44 min ago. Offline
Joined: 04/20/2012
Groups: Go Science
Please keep posting ED Puzzles:

I like that you wrote "One thing I am certain of, if there is ever a
CASP category for fitting Electron Density... nobody could touch you!"

I hope you will keep giving us ED Puzzles to do. They feel very real,
and I finally feel like I can do them successfully. If you can also
make the "Center Protein on Density" button fit the structure better
into the ED cloud or implement https://fold.it/portal/node/2006218
I would appreciate it.

Susume's picture
User offline. Last seen 4 days 19 hours ago. Offline
Joined: 10/02/2011
David Baker's comments on why DeepMind did so well

Developed by: UW Center for Game Science, UW Institute for Protein Design, Northeastern University, Vanderbilt University Meiler Lab, UC Davis
Supported by: DARPA, NSF, NIH, HHMI, Amazon, Microsoft, Adobe, Boehringer Ingelheim, RosettaCommons