What is the best way to use AlphaFold in Foldit puzzles?

Case number:845813-2011997
Topic:Game: Tools
Opened by:jeff101
Opened on:Saturday, August 28, 2021 - 17:14
Last modified:Tuesday, September 7, 2021 - 05:00

At https://fold.it/portal/node/2011929#comment-45091 argyrw wrote:

question for deep mind

Hi! we want only statistic 80% in the confidence and similarity or
we want max points of the foldit with that similarity and confidence.
it's necessary have max points or the important is that statistic?

argyrw has been asking similar questions in Group and Vet Chat,
but I don't know how to answer these questions. We have already
done 4 puzzles with AlphaFold, and another will finish on Aug.31.
Can you tell if we are using AlphaFold correctly? argyrw says it
is easy to get solutions with high confidence and similarity, but
they often give low Foldit scores. Is it better to focus on Foldit
scores, or is it better to go for high confidence and similarity?
My sense is that we want high Foldit scores for structures that
are very similiar to what AlphaFold would predict, so I guess if
your high-scoring Foldit solution gives low similarity vs what
AlphaFold predicts, you should try mutating the protein until
AlphaFold's prediction matches it better. This may mean using
more malek residues in sections you want to be helical. It may
also mean doing conservative mutations, like replacing certain
hydrophobic residues with other hydrophobics and replacing
certain hydrophilic ones with other hydrophilic ones.

I would appreciate it if Foldit players and staff would respond
to this Feedback with what they think is the best strategy to
use. Also, if someone on the Foldit staff would examine argyrw's
results and message argyrw directly with advice for doing better,
I would appreciate it.


(Sat, 08/28/2021 - 17:14  |  7 comments)

jeff101's picture
User offline. Last seen 1 day 8 hours ago. Offline
Joined: 04/20/2012
Groups: Go Science

Below are links to the Aug 6, 13, & 20 Foldit Newsletters. 
If you scroll down to the bottom of each one, there are 
sections with tips for using AlphaFold.


My guess is that as more Newsletters are added in the future, 
these links will change (like try page=2 or page=3 instead).

spvincent's picture
User offline. Last seen 14 hours 1 min ago. Offline
Joined: 12/07/2007
Groups: Contenders

Maybe it depends on the puzzle. If we're looking for novel folds as perhaps in 2027, then maybe it wouldn't be expected that AlphaFold would give a good similarity as it wouldn't have anything like that in its training set.

georg137's picture
User offline. Last seen 3 days 15 hours ago. Offline
Joined: 08/07/2010
Groups: Contenders

By submitting our solutions to Alphafold for evaluation, we are providing Deep Mind with extremely high quality training data for their AI. There is no way to determine if our solutions are receiving any point benefit, but more to the point, Deep Mind is harvesting a lot of excellent human thinking/creativity regarding protein folding (which they need to do in order to prune a googol of branches that go nowhere). It would be nice if Deep Mind shared its strategy with the people who are the basis for it.

bkoep's picture
User offline. Last seen 1 day 13 hours ago. Offline
Joined: 11/15/2012
Groups: Foldit Staff

This is not true! Foldit solutions are NOT shared with DeepMind. When you use Foldit to submit a solution for AlphaFold evaluation, those solutions stay on the Foldit server where only the Foldit team can access them.

When DeepMind released the source code for AlphaFold, that allowed us to install AlphaFold on our Foldit server, to use it for academic research however we like. We have even made some minor tweaks to our installation of AlphaFold so that we can skip unnecessary steps and run AlphaFold more efficiently on batch Foldit solutions. Because DeepMind has openly released their algorithm, we can use the algorithm without exposing Foldit solutions.

The Foldit team has no connection with DeepMind and we do not share any data with DeepMind.

Sorry for the confusion! I can see how this might not have been clear, especially since the Foldit button is called "DeepMind AlphaFold" (we simply wanted to give clear credit to the DeepMind team for their algorithm, and avoid any misconceptions that the Foldit team had designed the algorithm).

georg137's picture
User offline. Last seen 3 days 15 hours ago. Offline
Joined: 08/07/2010
Groups: Contenders

Thanks, bkoep. The exclamation points and bolded sentence dispel the murkiness. But why not partner with Deep Mind? It is actually surprising that there is no relationship. Given Deep Mind's successful track record with AlphaZero & AlphaGo and given Foldit's wealth of human folding experience, they are extremely well-positioned for collaboration.

bkoep's picture
User offline. Last seen 1 day 13 hours ago. Offline
Joined: 11/15/2012
Groups: Foldit Staff

Possibly, but I'm not sure DeepMind is interested with human experience. Human experience was an essential part of their original AlphaGo program, but the field of AI seems to be moving towards reinforcement learning techniques that do not require any human input (e.g. AlphaZero and AlphaFold).

That is not to say that there is nothing to be learned from human intuition and problem solving. Rather, I think AI researchers have recognized that human-independent technology is generally more valuable than technology that depends on human experience -- especially for problems that humans have not been able to solve, or for which there is no human input available.

georg137's picture
User offline. Last seen 3 days 15 hours ago. Offline
Joined: 08/07/2010
Groups: Contenders

You are probably right, bkoep. Still, DeepMind cannot be completely uninterested. There is a strong argument that all experience is self-aware experience. AlphaZero became a powerful Go player after playing Go with itself millions of times, but humans taught it the rules and set up the environment and provided the initial weightings of critical neurons. Humans invented AlphaZero and Go (Humans did not invent proteins, so that point may lose some of its edge).

The bottom line is that these AI's do not "know" they are playing chess or Go. Peel away the layers and there is always a human experience kernel or seedstock. Hopefully, the better the kernel, the better the outcome.

Jeff101, I apologize for drifting off topic.


Developed by: UW Center for Game Science, UW Institute for Protein Design, Northeastern University, Vanderbilt University Meiler Lab, UC Davis
Supported by: DARPA, NSF, NIH, HHMI, Amazon, Microsoft, Adobe, Boehringer Ingelheim, RosettaCommons