4 replies [Last post]
Joined: 06/06/2013
Groups: Gargleblasters

You’ve finished the tutorials. What next? I think we should reward mastery of basic handfolding skills, and give points for progression at various levels of complexity. Things to consider would be the levels below. Fellow folders should pipe up or tell me where I’m off point.

Add more levels for new players where they can score points. The old timers can sit these out –or just be sure they earn no points. Players should pass out of a level when they have mastered each element. Awards for those who can master each level! I’m not sure how many skills should be in each level, but took a stab below. Correct me where I am wrong, missing something, or suggest which levels could be better defined.

Level 1 we teach people basic fold shapes like 2 and 3 plane proteins (two plane is surfing hot dogs or two sheet planes), a Koga ideal and a ferodoxin fold, a helix cluster. Showing someone how to see and fold a sheet plane at the core might be a useful “no belt” video to go with this. Lay out each step without assuming the watcher knows how to do anything. Show stubs to see how sheets line up. Gifted people will move through this quickly. We should also teach folders to look at backbone, clashing, voids and exposed parts of a fold to see how it can be improved. I’d start these puzzles with a given SS in a string, or even partly wonky with a few pieces to put into place. Perhaps some of both. There are plenty of existing folds to use as basis for these puzzles. Award for Level One is a Sheet Plane badge – sheet with wings. Earn award when get an acceptable fold for each basic shape just like tutorials have a minimum score.

Level 2 lets players learn how to spot foldable SS, so start with a non-mutatable string. Teach that sheets tend to alternate hydrophobes and hydrophiles, especially if not on the edge. We know that a glycine is likely to be part of a loop between SS elements, and that helices are two or three to one hydrophiles/phobes. We know certain patterns of proteins will likely be in a helix or sheet. Think of this as reverse blueprint, and use common loops like a GD or GN for sheet to sheet so folders learn to spot them. Basically these puzzles are smallish de novo folds of the shapes listed above. Can just lift some existing solutions to fold. Award for Level Two is a Surfing Hot Dog badge. Breed of dog to be selected by developers

Level 3 teaches some tricky bits. Barrels, a transmembrane protein (inside out barrel), cysteine bonds. QTTN if you plan to continue this format. These should complete the basic shapes. Need to learn that need at least 7 sheets for a barrel, cysteine bond is done from tip of side chain to tip of side chain and that angle must work for things to bond. Award with a barrel badge, or a keg :)

Level 4 is time to mutate. With basic shapes mastered, can teach players to build a shape and mutate to fit. At this point, the monomer puzzles will be within reach. Work through varying types of filters – must be no more than 50% helix, can be all helix, ideal loops, etc. Must have a core. Award extra points for pi sandwiches(aromatics lined up in the core). Teach how to check with RAMA map or Blueprint. Stick to small monomers of a fixed size for this. Also, teach H bonding to improve fold stability and points. Badge is Ice Cream sandwich filled with pi.

Level 5 plus should be open to all. Time to learn how to dock a ligand, mutate and/or change the SS around a ligand, work ED, work contact maps, work symmetry. Many of us long timers still can’t do all these things, so this might be what is currently “intermediate” and open to all. Congratulate on moving to intermediate with a fun and geeky badge. Or perhaps a badge for each type of puzzle and throw out the long time folders who can do some of these things in their sleep (anyone in top 50 or so). Require a minimum number of points as mastery of each puzzle type. Make the puzzles workable as a teaching tool. So make sure the contact map has some neat diagonal sections that are obviously helix and is mostly complete, put some long aromatic side chain sections into the ED so players can spot them. Better folders can use these to hone skills, but not earn points.

Current revisits should be intermediate. Time for the messy world of real proteins. Similarly, monomer, H bond and symmetry puzzles can fit this label, though perhaps it needs a new name. Not sure if contact maps go here or next level up. I think it depends on the person. Novice points might also be appropriate for players below the top 50.

Alphatoxin, other large “challenge” puzzles, ED need to be classified advanced. I think fewer than 20 players can really work the ED puzzles so this is just good labelling. I would not lock this or the prior level out to any player – rather use the labels as truth in advertising. Sort of like Angry Birds – ED is a hard level to pass while the other levels are easier. I’d also put any large protein into the advanced class as some people are working these puzzles on antique equipment (myself amongst them).

Please offer suggestions. Recommendations from new players in particular would be helpful. How do we keep this fun for everyone? Sometimes our new players jolt us out of a rut and bring fresh and useful insight into the game for all of us, so you are far more important than you realize.

Happy folding

Joined: 09/24/2012
Groups: Go Science
Proposed method to guide assessing difficulty

In the puzzle menu, all puzzles are now labeled as "intermediate". This is not conform to our feeling. I liked to see, when I was a beginner, "intermediate", "advanced" etc.). If I remembered well, I even was not allowed to play "advanced" puzzles when I was beginner: this gives an incentive to work harder.

A categories achievement system would be ok (for each kind of specific puzzles, e.g. "master designer" if more than x design puzzles above rank y).

In order to answer these concerns, I suggest a method to help define the difficulty of a puzzle.

I took some statistics for recent case puzzles in the table bellow (puzzle case numbers are respectively 1541 1531 1543 1516 1533 1516).
The table shows the percentiles of the players and their score relative to best score. For example, bellow left, 75% of the players reached a score above 98% of the best score. Orange color shows which percentile reached above 90% of the score of the best score. The yellow lines (median and 0.75 percentile) and the orange spots give a proposed measure of an ex-post difficulty of a puzzle.

This gives proposed default label as follows:
Revisiting = Basic
De Novo, Monomer design, and complex second rounds (contacts etc): Intermediate
Symmetry and De Novo contacts or HBonds: advanced
Electron Density, Hand Fold, Pilot (Sketchbook), Hbonds and other Drugs: Expert

Puzzle levelsPuzzle levels

Joined: 05/19/2017
Groups: None
A rebalance is definitely in order

I definitely think that not all science puzzles in rotation are created equal and the ideas shared so far are pretty viable. Revisiting puzzles are bread n' butter easy, De Novo needs a little bit of thought, and H-bond networks are just flying over my head.

A tutorial rework is definitely in the cards. I know a little bit about it, but it's not quite ready in multiple respects. These are definitely good points to note.

An experience system has been a mainstay of progression systems in other games and I think I mentioned something like that before, but I wouldn't know how to furnish it with suitable milestones and rewards.

bkoep's picture
User offline. Last seen 11 hours 14 min ago. Offline
Joined: 11/15/2012
Groups: Foldit Staff
Excellent ideas!

Thanks all for the suggestions! I think you're right that Foldit could benefit from some more attention to user experience—particularly in assimilating new players and getting them comfortable with more difficult and complex puzzles. We're continuing to test new ideas for the tutorials, but there's certainly room to improve the post-tutorial experience as well.

One relatively easy improvement would be a better use of the existing "Difficulty" ratings. We're adding two new difficulty ratings for Foldit puzzles: "Novice" and "Expert", which expand on the existing "Intermediate" and "Advanced" ratings. The "Beginner" difficulty rating will be reserved for puzzles in the Beginner category (for players with <150 points). I like Bruno's method of determining puzzle difficulty, so we'll try to use a similar scale for future puzzles. The new ratings are ready to go on the website, but unfortunately the Foldit client also has to be updated, so we will need to wait for the next release before we can start using the new ratings.

As always, your suggestions are extremely helpful! Please keep them coming!

Joined: 09/24/2012
Groups: Go Science
Light / heavy or fingering/bruteforce puzzles

Great, you implemented it !

This "expert" signaling is actually a mix between fingering and expertise, and computer power.

Would you added something like 1 to 3 "computer icons" and/or 1 to 3 "fingers icons" in order to inform the player about the basic resources needed for this kind of puzzle?
(for hand, you need free time and fingering, for computers, you need hardware).

Or would it be too much as information ?


Developed by: UW Center for Game Science, UW Institute for Protein Design, Northeastern University, Vanderbilt University Meiler Lab, UC Davis
Supported by: DARPA, NSF, NIH, HHMI, Amazon, Microsoft, Adobe, Boehringer Ingelheim, RosettaCommons