Speed Folding -- Suggesting New Puzzle Type

Case number:671071-991666
Topic:Game: Other
Opened by:phi16
Status:Open
Type:Suggestion
Opened on:Saturday, January 28, 2012 - 17:23
Last modified:Thursday, July 19, 2012 - 14:45

Normal puzzles such as Denovo, Exploration and Mutate are played with a time limit. As soon as play begins, a countdown timer starts counting down to the allowed time limit. Puzzles may be played over and over. Players find winning strategies through use of the most efficient, economical and fastest pathways to the highest score.

Collective intelligence will guide players as they search for more economical, efficient scripts and better, manual work. Just as speed chess exercises the chess player's ability to see the bird's eye view and a rearview mirror, timed fold.it games will help players and teams to streamline their work and focus on essentials.

For example, teams now share info on which scripts they are using. Out of that comes a recommendation of four or five scripts to be run for each game for best results. However, the first script may have had to run for eight hours. The second script only worked for the first half, most found the third script useless and the fourth script produced only two points. Much effort is wasted trying to reproduce these recommendations. With a time limit, players will focus more on results within a time frame. Players may discover what proportions of shaking the backbone and wiggling the sidechains really makes a difference in any particular situation.

Typical play goes something like this:

0-5000 - score during the initial shake down period (5000)
5000 - 8500 - score during initial rebuilding, remodeling phase (3500)
8500 - 10500 - score during relaxing, getting the kinks out phase (2000)
10500 - 11500 - compression, wiggle/shake (1000)
11500 - 12000 - sidechain work, walking (500)
12000 - 12100 - final walk (100)

Why spend any time at all on the last phase, worth 100 points, if so much more can be achieved in the initial phases? A timed play would encourage more of this type of thinking and stress the importance of good beginnings. Of course, at some point, each phase maxes out and its time for the next bit of business. But we have learned that these sections should each be done in their correct proportions.

I personally recommend a 4 hour game. But there are good arguments for playing longer and shorter games. Currently, we are playing 7 day games which are wasteful and don't encourage this type of strategizing.

After a puzzle has been played, players examine results and discuss how their play may be improved. Improved play will come in the form of different order of played scripts, different manual operations and revised scripts.

(Sat, 01/28/2012 - 17:23  |  29 comments)


Joined: 06/24/2008
Groups: Void Crushers

I like this.

tokens's picture
User offline. Last seen 20 weeks 19 hours ago. Offline
Joined: 11/28/2011

If this is implemented it should take into account that people start on a puzzle at different times of the day. One idea might be to have the timer reset each time the puzzle is reset. Of course that might change things a bit. Then people will have all the time they want to find out what changes of topology work in the beginning. And it will become more about finding the optimal order to do moves/scripts in.

One downside to your suggestion is that it might make a difference how fast a computer you have.

Joined: 08/30/2011

i don't like to be stressed in anything i'm doing. so i wouldn't like this new type of puzzle.
I think we don't have enough time to discover new routes. I have to decide quite early which of my solutions i want to develop further and all the others i had will be lost. and i don't think that with a really short time limit people are gonna be better folders. the experienced people know what to do and will rock these puzzles and the unexperienced onces will be lost and they cannot even improve because they have no time to figure out where their problem was.

spmm's picture
User offline. Last seen 37 weeks 1 day ago. Offline
Joined: 08/05/2010
Groups: Void Crushers

4 hours is really not a lot of time for a denovo - particularly a large one - this may also increase people in groups just copying the first high solution - I think we all try to become faster more efficient folders anyway.
Discussion afterwards - is this in a group or globally? If globally the best solutions should also be available - to interact with for every folder - you can learn a lot by interacting with a good fold.

I don't really understand what you mean by wasteful.
Perhaps you could try it in a large team like AD and see how it goes.

beta_helix's picture
User offline. Last seen 23 hours 26 min ago. Offline
Joined: 05/09/2008
Groups: None

We have discussed this many times and the problem of different computer speeds always comes up.
There is simply no fair way of doing this when people have very different computers.

An alternative would be to allow a maximum number of moves:

The puzzle would be up for a few days (or as long as usual) but you'd only be allowed to perform a fixed set of moves. Once you hit that number you wouldn't be able to do anything else (or your score would no longer count, like the conditions).

If you reset the puzzle, you would get that same number of moves to try again.

This idea is another form of Exploration: you search one area of conformational space, then when you run out of moves you can try searching the same area (using less moves) but eventually you'd have to search a different region of space.

There would be a few details to work out (as it would surely limit the number of recipe moves) but it might be interesting.

infjamc's picture
User offline. Last seen 2 years 41 weeks ago. Offline
Joined: 02/20/2009
Groups: Contenders

Would it be possible to recycle some of the aspects of the "duel" feature from earlier versions for this? After all, a duel puzzle is essentially a simple form of this (with 20 moves).

Joined: 10/11/2011
Groups: None

Different computer speeds, different time zones, would make the idea beneficial to only a few players.
The rest wouldnt stand a chance.

Beta's idea of maximum number of moves would give a more level "playing field", it would almost be reminiscent of Chess.

Joined: 06/17/2010

Cool idea.
It should be doable this way:
1. Puzzle posted for 4-7 days
2. Time when we can work on it: 4hrs
3. Puzzle CAN NOT be play offline
4. Game time counted: we play one hour - one hour counted, we can quit and continue next day

This way no time zone or "cant play 4hrs in row" problem will not affect anyone.

Maybe multi start puzzle, each start has separate time count.

brow42's picture
User offline. Last seen 1 week 2 days ago. Offline
Joined: 09/19/2011
Groups: None

I would prefer not to see scripts used at all. One can easily do a blue fuze by hand, but a script that samples 10 rebuilds and evaluates them all is vastly more powerful than a human simply because of all the bookkeeping (and humans only have 3 quicksaves).

infjamc's picture
User offline. Last seen 2 years 41 weeks ago. Offline
Joined: 02/20/2009
Groups: Contenders

Well, this is exactly why I said that if recipes/scripts are to be allowed, they should be counted for the same number of moves as their manual folding equivalent. Let's take your hypothetical script for example-- I would expect the script to require at least the following:

Set recent best
Repeat 10 times: rebuild for 1 iteration
Load recent best

==> That's 12 moves already. If we add in the needed shake and wiggle, the protocol becomes the following:

Set recent best
Repeat 10 times: rebuild for 1 iteration, then shake and wiggle
Restore recent best

==> That's 32 moves. If this were a duel puzzle with a 20-move limit, the script cannot even be completed; instead, one would have to stop at model #6 before spending the last and 20th move on "restore very best." And we haven't even added anything advanced that would be difficult (or cumbersome) to replicate manually.
.

The bottom line is that I don't see the usage of recipes/scripts in such puzzles as a problem because those who use automated methods will hit the move limit very quickly. And I can say this from my personal experience from my last 3 years of playing foldit: automated methods that are very complicated (e.g. overnight scripts) are generally much less cost-effective from a "score gained per moves made" ratio standpoint relative to manual folding. Since this is the case, I would expect players who rely on scripts to actually be at a disadvantage in puzzles with a move limit.

Joined: 06/17/2010

I would love to work on non-script puzzle :)
I probably end in inverted top 10 but ... :P

Joined: 10/11/2011
Groups: None

Surely it would be easier to implement as a "No scripts" type of puzzle.
Having to calculate the value of script moves etc, would be too time consuming for it to be set up I would imagine

X number of moves..no scripts. One chance only. Once you have done your moves, thats it, over and out.

A puzzle would need to be up for so many days( 3, 5 etc) to allow for times zones, people working etc.

infjamc's picture
User offline. Last seen 2 years 41 weeks ago. Offline
Joined: 02/20/2009
Groups: Contenders

There's no need to calculate the number of script moves. Just use a mechanics similar to that of duel puzzles, so that the number of moves would be incremented by one when any of the following occurs:

* Dragging any piece
* Shake
* Wiggle all / backbone / side chains
* Undo
* Restore recent/very best
* Rebuild (one iteration)
* Freezing or unfreezing

infjamc's picture
User offline. Last seen 2 years 41 weeks ago. Offline
Joined: 02/20/2009
Groups: Contenders

Looks like I forgot the following:

* Applying the alignment would also count as one move for each accepted iteration.
* If loading solution is allowed, that should also count as one move.

(The bottom line is that anything that generates a point on the undo graph would count as one move. That would require very little to no redesign.)

phi16's picture
User offline. Last seen 23 hours 12 min ago. Offline
Joined: 12/23/2008

If players recognize that scripts are using an enormous number of moves for a disproportionately small number of points, compared with manual play they will naturally gravitate to manual play in a puzzle which only allows a finite number of moves.

beta_helix's picture
User offline. Last seen 23 hours 26 min ago. Offline
Joined: 05/09/2008
Groups: None

if this were implemented (and we've talked about it in our meetings for a while, but it never makes it to the top of our priority queue) it would most likely allow scripts, but as many people have mentioned: using scripts would quickly use up most of your moves so they would most likely not be used.

infjamc's picture
User offline. Last seen 2 years 41 weeks ago. Offline
Joined: 02/20/2009
Groups: Contenders

It seems to me that the number cannot be either too high or too low:

* If the limit is too high, it wouldn't affect things much other than discouraging overnight rebuild scripts.

* If the limit is too low, not much exploration could be done even manually. Plus, this could encourage "Hail Mary pass"-type moves (i.e. tactics that have a low probability of succeeding but a high payoff if it succeeds) in the attempt of maximizing the score-gain-to-moves-made ratio and generate a lot of models that are mostly incorrect.

==> So, it seems to me that the optimal move limit would be somewhere in the 100 to 10000 range...

beta_helix's picture
User offline. Last seen 23 hours 26 min ago. Offline
Joined: 05/09/2008
Groups: None
Status: Open » Open

yes, it would require some testing to find out the optimal range.

phi16's picture
User offline. Last seen 23 hours 12 min ago. Offline
Joined: 12/23/2008

There are three other aspects to speed folding that are in keeping with learning how proteins fold by using the collective intelligence of the players and not just about the 'game.'

First, teamwork should and would be an important consideration of speed folding. Because the game becomes a reasonably shorter version of itself, the team is able to discuss best results and improve upon them. It then is not about the play so much as it is the replay. I can see teams trying the puzzle over and over, to keep within the time (move) limit and get better and better results. In the process, perhaps there will be insights as to what is making a difference to the outcome and what is not. I'm reminded of the scene from the movie Apollo 13 where a fellow astronaut on the ground is trying to figure out how to make a CO2 scrubber from the remaining parts and duct tape, fire the retro rockets, and use the little fuel that they have left in the proper sequence to get them down safely. He tries the sequence of moves over and over and over until he figured out how to do it with the remaining fuel. According to the movie version of events, he saved the day.

The second aspect to why we should try speed folding is the possibility that there is going to be new scripts written just for this type of game. And, like protein folding in nature, where folding is done over and over until an advantage is achieved and the new variation is kept instead of discarded, scripts can be rewritten after the game has been played, again and again, eliminated superfluous moves. This revisiting and streamlining will also help to teach us what works and what does not.

The third possibility is that players, themselves, figure out simplified play. For example, let's say that at the end of a run, it is seen that a certain helix is now at a new location. In the first run, the helix got there through trial and error and 100,000 wiggles and shakes, banding and pulls, of a Compressor script. Knowing the advantageous end position, folders may be able to band and pull the helix into its correct position in just a few moves. It is this revisiting that is to be encouraged in speed folding. To do that, I would emphasize the shorter game variation.

Joined: 12/06/2008
Groups: Contenders

And what are we gaining by all this?

Remember, kids, we're trying to determine the correct (and usually unknown) structures of real-life proteins. I don't see "speed folding" as something moving us in that direction.

jeff101's picture
User offline. Last seen 1 hour 26 min ago. Offline
Joined: 04/20/2012
Groups: Go Science

I think one goal of Foldit is to develop more efficient methods for finding the best structures. I think "speed folding" puzzles would help achieve this goal.

jeff101's picture
User offline. Last seen 1 hour 26 min ago. Offline
Joined: 04/20/2012
Groups: Go Science

If you don't believe me, see http://fold.it/portal/info/science#whygame where it says:

"Can humans really help computers fold proteins?

We’re collecting data to find out if humans' pattern-recognition and puzzle-solving abilities make them more efficient than existing computer programs at pattern-folding tasks. If this turns out to be true, we can then teach human strategies to computers and fold proteins faster than ever!"

You can also read papers like the following:

Algorithm discovery by protein folding game players. Firas Khatib, Seth Cooper, Michael D. Tyka, Kefan Xu, Ilya Makedon, Zoran Popović, David Baker, and Foldit Players. In Proceedings of the National Academy of Sciences (2011).

http://www.pnas.org/content/early/2011/11/02/1115898108.full.pdf+html

Its abstract says:

"Benchmark calculations show that the new algorithm independently discovered by scientists and by Foldit players outperforms previously published methods. Thus, online scientific game frameworks have the potential not only to solve hard scientific problems, but also to discover and formalize effective new strategies and algorithms."

Joined: 09/19/2011
Groups: Beta Folders

I have zero interest in speed folding.

phi16's picture
User offline. Last seen 23 hours 12 min ago. Offline
Joined: 12/23/2008

Hi smilingone. Thanks for commenting. Your comment is noted and understood. So that I'm not being misunderstood, let me respond that I wouldn't want a 'who can work fastest 'speeding' contest, either. What I'm hoping will happen is that people fold more efficiently which means ending scripts sooner, or running scripts without excessive amounts of repeated processes which they were doing now simply because they can, or more manual folding without scripts or finding more efficient scripts. Folding is not about doing things quickly.

In this 'race' it will mean folders are using shorter runs, switching more often, making the right choices about which scripts to run and thinking more globally about what they are doing. It's an experiment. We shall see.

phi16

B_2's picture
User offline. Last seen 4 years 21 weeks ago. Offline
Joined: 11/29/2008
Groups: None

And the client crashes so often, maybe we should have a "how many moves can you do before a crash" contest?

Joined: 04/15/2012
Groups: Beta Folders

Why not? :)

infjamc's picture
User offline. Last seen 2 years 41 weeks ago. Offline
Joined: 02/20/2009
Groups: Contenders

Such a contest would have to separate the users by OS. Otherwise, Windows users would have too much of an advantage. :-)

Joined: 06/17/2010

You meant mac ones? ;]

Joined: 05/19/2009
Groups: Contenders

I reject the idea of speed folding. period.

Sitemap

Developed by: UW Center for Game Science, UW Institute for Protein Design, Northeastern University, Vanderbilt University Meiler Lab, UC Davis
Supported by: DARPA, NSF, NIH, HHMI, Amazon, Microsoft, Adobe, RosettaCommons