Has the validity of 'Foldit Alphafold' been scientifically examined ?

Case number:699969-2012475
Topic:General
Opened by:Bletchley Park
Status:Open
Type:Suggestion
Opened on:Saturday, December 11, 2021 - 18:10
Last modified:Tuesday, December 28, 2021 - 11:42

Has there been a scientific challenge to the validity of predictions that are issued by 'Foldit Alphafold' ?

How can we be sure that the predictions this model gives are accurate representations of true chances for success ? We receive bad scores for designs that score well in Foldit's energy function, until now the standard measure in Rosetta. Does that mean that Rosetta is also wrong ?

If there is (going to be) a paper on this topic, please tell us. I would also imagine that Foldit players are credited as they contributed to the models that are used in 'Foldit Alphafold'.

(Sat, 12/11/2021 - 18:10  |  2 comments)


bkoep's picture
User offline. Last seen 1 day 3 hours ago. Offline
Joined: 11/15/2012
Groups: Foldit Staff

We have not conducted any new experiments to test Foldit designs that were created with the AlphaFold tool (but that will change soon!).

Right now the strongest data we have for AlphaFold predictions is retrospective. When we look at previous experiment results, we see that AlphaFold predicted structures are slightly more accurate than design models for successful designs, and we see that AlphaFold confidence is a good predictor of design success. For details see our previous blog post about AlphaFold and Foldit.

Foldit's score function is not perfect for protein design, and this is to be expected. (For a longer discussion about this, see our blog post on the problem of protein design.) We should also expect AlphaFold to have its own weaknesses. Right now, the most promising designs have a high AlphaFold confidence and satisfy all of the Foldit Objectives and have a reasonable baseline Foldit score.

Rest assured that Foldit players will be credited on any research that uses the work of Foldit players. Proper attribution is an essential part of academic research. And, generally speaking, researchers are excited to include Foldit players on their scientific papers because it increases the appeal of a paper to a wider audience.

It is perhaps worth reiterating that we are not using Foldit solutions to make any changes to DeepMind's original AlphaFold 2.0 algorithm. The Foldit team is not retraining the AlphaFold neural net in any way. We are only using the neural net to provide Foldit players with feedback, so that Foldit players can create more successful protein designs.

Joined: 05/19/2009
Groups: Contenders

Thank you for clarifying, in particular the section "It is perhaps worth reiterating that we are not using Foldit solutions to make any changes to DeepMind's original AlphaFold 2.0 algorithm. The Foldit team is not retraining the AlphaFold neural net in any way. We are only using the neural net to provide Foldit players with feedback, so that Foldit players can create more successful protein designs.".

I thought you had retrained AF with past foldit solutions, but as I read in the text above, that is not the case.

Regards, BP

Sitemap

Developed by: UW Center for Game Science, UW Institute for Protein Design, Northeastern University, Vanderbilt University Meiler Lab, UC Davis
Supported by: DARPA, NSF, NIH, HHMI, Amazon, Microsoft, Adobe, Boehringer Ingelheim, RosettaCommons