The New Grand Challenge

The release of "The New Challenge" puzzle signifies the start of a new sequence of puzzles aimed at determining the specific ways in which human puzzle solving can outperform best known automated methods.  These puzzles are a result of the analysis of all game play results since we introduced the game in May.  The puzzles we will be unrolling from now on are specifically selected to confirm certain hypothesis about the nature of problems where the collective "game brain" will outperform other search methods. 

The CASP8 results showed us that protein exploration through foldit puzzles is a promising new paradigm for 3D structural exploration.  It did not, however, point out the how and why people are able to get such good solutions.  The "grand challenge" puzzles will hopefully provide that information.  Should the results prove to be interesting, we will shortly afterward submit a research paper reporting our findings.

So let the Grand Challenge begin!  We're counting on you.

( Posted by  zoran 82 2411  |  Sun, 12/21/2008 - 04:36  |  9 comments )
Joined: 05/26/2008
Groups: None
Thanks for the update

Looking forward to seeing what you make of the results. Have not been as active lately with the holidays, family, and work all taking most of my time, but since you are starting a new challenge, will definitely try to sneak in some hours wherever possible.

I was curious if you have any plans to enter into other competitions similar to CASP? I think the opportunity to compete against others outside of FoldIt is a highly motivational factor in both attracting new players, and coaxing better results from existing ones. I've seen references to a couple of them from time to time, so I know there are others out there, though I couldn't name any of them without spending some time on Google.

zoran's picture
User offline. Last seen 5 years 6 weeks ago. Offline
Joined: 11/10/2007
Groups: Window Group
future challenges

Great, spread the word about the challenge. To crack some of the challenges coming up in January i'm pretty sure we'll need all human brain power available.

CASP is unfortunately every other year, so we'll have to wait on the next prediction challenge. In 2 years, we're hoping to be competitive on every challenge.

On the other hand, we should have an AIDS vaccine challenge early in 09. I'll talk a bit more about this in another blog post.

xiando's picture
User offline. Last seen 3 years 6 weeks ago. Offline
Joined: 10/19/2008
Groups: Oma Gawd
My only concern

Is that with the application in its present state, there is an over emphasis on the need for isolated wiggle for solution.

I have never gotten so many points from isolated wiggle in the past, and in fact it represents a bulk of points coming from the last several puzzles, (30-40 points or more per backbone walk) since some change was made to the algorithms during the last major update that appears to have also compromised the shake command, leading to Shake rotations one or more steps away from their previously more ideal clash-minimized rotations (where I believe all these points in isolated wiggles are coming from).

My concern is simple. I would be uncomfortable to see a developer evaluation of human solving techniques skewed towards Rosetta style solution simply because the application is "guiding" the player into overuse of isolated wiggles, instead of using intuition and stable tools to come up with more "quantum" thinking solutions. That would, imo, be nothing more than the analog to a poll designed to prove an apriori assumption about those methods, rather than the discovery of new information about intuitive human methods of solution.

I would not want to discount the possibility that I have just "run out of steam". this is a reasonable possibility and one that has been on my mind. However, I keep wondering why the change in my ability corresponds so well with the last major update and the emergence of a heavy need for my use of isolated wiggle in puzzle solutions to gain points (compact the proteins), and my need for manually checking each and ever y shake to see whether side chains under going rotations as a result of its use have actually rotated to the "correct" position, so that I am left doubtful that the former possibility is an actuality, instead the outcome of an accidental or purposeful change in the fundamental nature of the application from seeking quantum thinking to  human-guided algorithmic solutions (like an interactive version of the automated "plug-and-chug application presented in Rosetta)

Joined: 05/19/2008
Groups: None
I tend to agree

with Xiando. Reaching a decent score takes much much more time recently than it did before, simple because intuition is more and more replaced with manual hard work of local wiggling the backbone. And the support for this in the app is very limited and did not improve since the beginning, regardless of the countless requests of automated back bone walking.

Trying new structures once we found a local minimum that seems far from the global minimum, in other words resetting the puzzle, is not supported, since the requested "New best" function is not implemented, regardless of myriads of requests for that. I'd rather tweak the current one to the limits, hoping a miracle.

As a suggestion to improve efficiency of human folding: It would be wonderful if I can leave my computer on while I'm away and the application would run a modified Rosetta code to determine if my solve is a dead end street, compared to others, or to a goal. This could tell if my current solve has a fundamental structural flaw that prevents me getting high enough score. Computer Aided Folding; CAF if you will. :)

Another suggestion: being able to play with hidden parameters of the app might improve human efficiency as well. For example, wiggle can take the the protein off of a local minimum, but the potential barrier it can jump through is a predefined value. I'd like to adjust it.

To improve humans: We should be able to learn not only from our successes and failures, but others' results as well. Previous winner solves should be published for the community. Just think it over: a beginner can NEVER see what the best solves look like, other than the recent and sporadic animations, which is useless, I think. So how could they figure the whole thing? Reinvent everything from the beginning? Only a very few people can do that, and it takes time. And even less people are actually willing to do it.

xiando's picture
User offline. Last seen 3 years 6 weeks ago. Offline
Joined: 10/19/2008
Groups: Oma Gawd
Regarding the energy barrier

I agree wholeheartedly regarding youre comments about the energy barrier

I once posted on the very same issue, but one of the at-the-time researchers , someone who went by the screename "Veritas" told me that I didn't understand the science and what I was asking for was impossible due to the discrete nature of folding. I tend to disagree with that particular characterization and left the post with little more than a clear mental picture of having spoken to an ignorant or otherwise stupid student, but sometimes not having a degree in molecular biology puts one at a distinct disadvantage when attempting to use a silly little applied physics degree, and a life long pursuit of a host of sciences and over 20 years of serving the academic research community to explain that one already "gets it" and "let's get beyond a narrow and pedantic conceptualization, since you siimply can't hide what is obvious to a seasoned multi-versed scientist).

There are a host of  program variables, that were they avaiable to the player would lend themselves to a richer ability to fold while impinging minimally (or not at all) on the basic paradigm of this project, ie, human rather than automated Rosetta style soultion, since after all, one must remember that while proteins *are being solved, their specific solution is tertiary to the fundamental aim of the project, ie, the study and characterization of the "human" element of intuition, in part towards direct solution and imo, in large part towards eventual improvement in automated code by detailed study of our human soluton methods. (note that since the project has been somewhat vague in its discourse since inception, I have intuited much of my understanding of this from its and pieces of leading, though muyrky statements in text, chat, and forum postings by the various officers of the project)

IMHO, more thought should be directed towards a configuration menu as used in most higher level applications. A basic menu can be provided for use by less inclined individuals,  but a five or six entry "general Options" configuratio menu is a bit terse, to say the least. If one drastic change were made to the program, I would say that that would be it.  I personally don't want to see it made avaiable for a rework of Rosetta, but I do believe very strongly that the paradgim can be maitained while opening up degrees of freedom pursuant towards the goals so far stated.

zoran's picture
User offline. Last seen 5 years 6 weeks ago. Offline
Joined: 11/10/2007
Groups: Window Group
what's different with the new puzzles


see my blog post for the possible difference in the game play of the recent puzzles.

Joined: 05/19/2008
Groups: None
One more thing

We don't know if there is a scientific value of trying to improve the score beyond the best when the puzzle is closed.

To put the solves in the global arena where anyone can try to improve anyone else's solution could do this. We have no means to improve the winning team's solve, however our method of folding might improve it... for the benefit of the science and human ego.

zoran's picture
User offline. Last seen 5 years 6 weeks ago. Offline
Joined: 11/10/2007
Groups: Window Group
All-hands stage


we are indeed planning to establish the all-hands stage after the initial puzzle challenge expires as soon as next week.  You must be reading our mind.  In this stage, everyone is effectively an evolver placed into one meta-group containing all players.  you get to choose any of the best N solutions to start with and you work on solving it.  our hope is that this will enable us to improve the overall time spent on evolution which seems to have decreased over time.

we agree with you on enabling greater knowledge spread by providing more hindsight information.  our first step was the evolution videos. we will also allow players to tackle any past challenges by downloading the best solutions and trying to improve them.

we're also planning a whole lot of features that should significantly strengthen the social evolution of the game, specifically, allowing players to create/trade/evaluate macro tools, issues and hints, and even create new levels that depict specific wisdom nugget that should be useful to others.  One of our primary research objectives in the future design of the game is to create a framework that would enable players to have the primary role in the improvement of the game across many levels.  I'll write a more detailed blog entry on this.

spvincent's picture
User offline. Last seen 7 hours 7 min ago. Offline
Joined: 12/07/2007
Groups: Contenders
Control over wiggle

Following up on DH's suggestion above, I'd like to see players have more control over wiggle backbone: specifically the ability to control how fast it approaches its goal and how much of a scoring decrease is permitted at each step. Wiggling looks like a simulated annealing process (if it isn't, it can perhaps usefully be thought of as one), so the first suggestion above could be implemented by having a slider, similar to that seen for bands, that would control the 'temperature change' permitted at each step in the wiggle process. The second suggestion, control over the amount of 'uphill' movement permitted at any step already seems to implemented after a fashion since it seems that local wiggle often results in a score decrease whereas global wiggle never does. Let's have more control this function, which everyone uses all the time. At the very least it would make walking the backbone a lot less mechanical, as players could experiment with different wiggle parameters.

Get Started: Download
  Windows    OSX    Linux  
(10.7 or later)

Are you new to Foldit? Click here.

Are you a student? Click here.

Are you an educator? Click here.
Only search
Recommend Foldit
User login
Top New Users

Developed by: UW Center for Game Science, UW Institute for Protein Design, Northeastern University, Vanderbilt University Meiler Lab, UC Davis
Supported by: DARPA, NSF, NIH, HHMI, Microsoft, Adobe, RosettaCommons