46 replies [Last post]
Joined: 04/24/2014
Groups: None

Hi everybody!

We were going over the Foldit Survey puzzle results and turns out it has been pretty useful, especially looking at trends over time! If you didn’t get to put your thoughts in that way (and a bunch of you did, thank you very much), unfortunately that survey is closed.

We asked the survey takers, just like last year, what your favorite and least favorite puzzle categories were, provided you had to pick just one. It’s a tough choice.. or is it? What struck as immediately as interesting was not only did our respondents pick “electron density” for "least favorite" two years in a row, this year there was a huge jump in doing so, making it a runaway choice compared to the other categories.

Seriously, what’s up? That’s some pretty strong dislike, and I’m sure some of you probably feel the same way but didn’t get to the survey for whatever reason (don’t make that error over our demographic survey though). We want to know where the problems are with electron density puzzles, so we can address them as much as possible given our resources. We also want to know what's working for you on those. We have some theories and speculation what’s causing these numbers, but we don’t want to influence the discussion and so we’re opening this thread let you tell us your thoughts.

Please note that this discussion is restricted to electron density puzzles only. This is not a discussion about other puzzle types, bugs, or general game feedback/suggestions (and those will likely be removed, as well as anything not keeping in spirit of the community guidelines, obviously).

jeff101's picture
User offline. Last seen 12 hours 41 min ago. Offline
Joined: 04/20/2012
Groups: Go Science
Many have posted Feedbacks about ED Puzzles
Joined: 04/24/2014
Groups: None
This is a good start for

This is a good start for context to the discussion (and we have seen those links), thanks for digging them up. Which of these suggestions are your favorite?

jeff101's picture
User offline. Last seen 12 hours 41 min ago. Offline
Joined: 04/20/2012
Groups: Go Science
More Feedbacks about ED Puzzles:
jeff101's picture
User offline. Last seen 12 hours 41 min ago. Offline
Joined: 04/20/2012
Groups: Go Science
My own personal favorite ideas for ED Puzzles:

https://fold.it/portal/node/993959#comment-32465
proposes a way to fix the "Align Protein to Density"
or "Center Protein on Density" button using 3 xyz
coordinates to translate the protein, 3 Euler angles
to rotate the protein, and an optimization algorithm
like the Nelder-Mead Simplex Direct Search Method
to vary the 3 xyz coordinates and the 3 Euler angles.

https://fold.it/portal/node/2002670#comment-33044
https://fold.it/portal/node/2001368#comment-31396
are about filling continuous parts of the ED with color,
like in a paint program, as if the ED were a mold and
we were squirting a colored liquid into it.

https://fold.it/portal/node/2001368#comment-32121
have options to show the calculated ED for the
present protein conformation so you could compare
the calculated ED with the given experimental ED.

https://fold.it/portal/node/2001368#comment-31397
https://fold.it/portal/node/2001368#comment-31401
let players fold the protein without the electron
density affecting its score. This could be by
confining the electron density to a few unit cells
and having the rest of space have zero electron
density. It could also be by letting players turn
on/off the electron density score as they fold.

http://fold.it/portal/node/993956
Rav3n_pl's "old idea of alignment editor. It would be
1000% easier to draw shapes on ED how we see them then
thread protein into that shape. I wish to draw helices,
sheets, then add loops and thread protein into it."

http://fold.it/portal/node/993956#comment-20594
having hoops (fixed points in space that the protein
must slide through in a particular order).

bertro's picture
User offline. Last seen 16 hours 28 min ago. Offline
Joined: 05/02/2011
Groups: Beta Folders
What I would wish for ED

What I would wish for ED puzzles is more control of the viewing planes.

In particular being able to lock one axe (or have a wheel like tool keeping one axe fixed but ajustable) while playing with the mouse in that 2d plane.

Also better control of the Z-planes near and far (better cutoff)

actiasluna's picture
User offline. Last seen 1 hour 14 min ago. Offline
Joined: 03/05/2015
Groups: Gargleblasters
ED puzzles

These are still mysterious to me.
I can't seem to put the dots where I want them (always a "mile off") So some way of orienting the dot where you want it that is more precise as a marker would be one wish.

MAYBE a higher contrast puzzle color scheme that would show up better against the cloud... or highlight colors specifically for ED that show your higher ED scoring segments, not just aggregated scoring.

An in-game key to shapes to look for in the cloud similar to that diagram circulated during the contest.

I agree about improving navigation through slices of the cloud.

"Snap to cloud" --- is that even a possibility? (like snap to grid in graphics programs) if you get close enough to a cloud shape.

Better control of the cloud alpha and threshold (1-100 "dials" or numeric entry)

Definitely fix the center/align. Would align puzzle to highest ED score aggregate be possible?

(ideas, I got 'em. Thanks for the official forum to post them!)

Susume's picture
User offline. Last seen 1 day 21 hours ago. Offline
Joined: 10/02/2011
Practice puzzles

I think it would help to have some easier ED puzzles for people to practice on - the only ones we get are high-stakes puzzles, when struggling with the tools is most frustrating.

Maybe every third revisit puzzle could have an ED cloud added to it. Give us the protein partially folded (not straight chain) just like they are now, but have a cloud next to the protein to fit it into. The revisit proteins are generally solved, so the ED data ought to be available.

If you are going to make some of the suggested improvements to the ED tools, letting us try the changes out on a revisit and offer feedback would be better than introducing tool changes on a high-stakes puzzle.

actiasluna's picture
User offline. Last seen 1 hour 14 min ago. Offline
Joined: 03/05/2015
Groups: Gargleblasters
agreed on all counts...

and definitely the second, partially folded proteins would be beneficial, the building from a straight chain while also minding ED is super time-consuming particularly when also not so great at using the tools. Working on smaller "revisit" puzzles is an excellent idea for ED practice and gathering more feedback.

Joined: 06/06/2013
Groups: Gargleblasters
ED puzzles need training process & smaller size

I agree that all the tools suggested sound highly useful. That said, I still can not "see" alignment, no matter how much time Susume & Frood have spent trying to help me work them. I can't tell a backbone from a sidechain often, "see" the helices, etc. I have watched the black belt videos several times, but just can't get there.
Secondly, all the ED puzzle this year have been monster sized. I can't do anything monster sized, never mind a monster sized ED. I do not have the computer capacity to do it, nor the time & ability to sit for hours at a stretch to work these large puzzles
If this is the preferred tool, we need a "training program". Start with some small ones with really good resolution. Let us work several before moving on to bigger ones or ones where some or all of the protein can not be presented well. Maybe even keep those puzzles for those of us who have never been able to do an ED. My only decent score was to stick 4 helices into the cloud once. I was embarrassed to score a top 10 on that one. FoldIt should be too, as it showed that the scoring system outweighed a useful answer
Absent training that I can follow and use as a foundation, I will open these ED puzzles only to evo a team mates solution when it is competitive. I know the science may be going there, but the citizen part of citizen science may not be able to follow

spmm's picture
User offline. Last seen 28 weeks 11 hours ago. Offline
Joined: 08/05/2010
Groups: Void Crushers
ED practice

Agree with Susume regarding practice puzzles, we get so few ED ones I forget how to use the tools and interpret the clouds.
The great thing about ED with decent resolution is certaintity. Also agree with smaler ones for practice.

The markers are useful but may be more useful to me if they were rods as I find them difficult to position in 3D space when I am marking big rings.

Practice puzzles with tryp and other hydro rings are easier to get a grip on because they are easier to see in the cloud.
I really like ED and would like to see more of them, certainly prefer a better resolution.

gitwut's picture
User offline. Last seen 28 weeks 3 days ago. Offline
Joined: 05/18/2012
Groups: Contenders
De-novos with a penalty

Like Skippy, I just don't see anything in the density clouds, no matter how hard I try. The existing tools are useless for me, so I turn the cloud visualization off and play them like De-novos. De-novos are challenging enough, but having to deal with the ED "tug" makes them even more so. So yes, I dread each one that comes around.

Having the ability to toggle the "tug" off and on would be a good place to start.

Joined: 09/24/2012
Groups: Go Science
small and easy ones first

I agree with getting smaller and easier ones to start (EDs are very slow for computers when viewing the density).
Also with some tryptophan inside (these are AAs that we have chance to identify in the cloud).
With no gap if possible (parts that are not in any density cloud) for de novos.
Any puzzles that are already for a big part in the density cloud are ok.
My main problem is to find a (density) starting point for a de novo.
Simple structures at start (e.g. only helices or only sheets), no more that 2 levels above each other in 3D (e.g. something like the design puzzles).
Revisiting (with a picture of the solution as help) or old design solutions that we have to put in a cloud.

Joined: 09/21/2011
Groups: Void Crushers
Give us a way to move the dots

Personally I really like ED puzzles, they are my favorite.
But the really frustrating part is that I cant position the dots in the places where I want them.
That takes a lot of time.
I would really like to be able to put them inside the ED, not on the surface.
We really need a way to select one and move it around, just like we can with the protein parts.

Joined: 09/21/2011
Groups: Void Crushers
Give us a slice tool on the ED

Now we can do it a bit with foreground and background moving, but that is not really easy.
When we would be able to have a slice of the density and control the thickness and the placement of the ED slice, in combination with placement of dots in the middle of the slice things might be easier.

Susume's picture
User offline. Last seen 1 day 21 hours ago. Offline
Joined: 10/02/2011
Trim and slice

Maybe two sliders on the ED menu for front cutaway and back cutaway, rather than the clumsy key-mouse combos. Also move the trim to a slider on the ED menu that stays where you put it. The RGB sliders could move to a secondary menu as they are less often changed.

The threshold slider is much more useful since jflat adjusted its scale.

Joined: 09/21/2011
Groups: Void Crushers
foreground and background trimming

My preference would be just to slice the ED cloud, because I would like to be able to see more of the protein itself. And being able to put the dots in the middle of the slice.

Joined: 05/26/2008
Groups: Hold My Beer
I really like ED puzzles, but

I really like ED puzzles, but rarely have the time to do them. What I think would really help would take a bit of doing but..

1) being able to control the color of sections

2) being able to show/hide sections easier

What I think would be very helpful is to be able to create basics 3d shapes (sphere and cylinder primarily) and to be able to stretch them in 3d space to define an area (semi transparent with node points to manipulate them on their axis, and arrows to move them along axis)

Then have a drop down list with all your defined areas (undefined area, sphere 1, sphere 2, cone 1, etc...)

For each if these areas you could :
Name the area
Show/hide ED data in the area
Change color of the ED data in that area
Show/hide the area boundary (once set it may obscure view)

Then all ED data not in a defined area would have the same options as well

I think this would give a tremendous amount of flexibility in manipulating data.

In addition players who are looking for more manageable planes could simply move cubes around and show/hide any area(s) they wish

Joined: 09/24/2012
Groups: Go Science
Seing spheres + adding "band to density"

I agree with Steven if I understood him wel.

My problem is eliminated all "noise" views when working on a small part, trying to fit it into the density.

Shift+q and slices are ok but it still gives too much to view.

I would like to be able to focus only on a "sphere" or "cylinder" around where I'm working on, but independent spheres for protein and density.

For example, seeing a large portion of the protein, but limiting the cloud only to a small sphere around the segments I'm working on.

Then also beeing able to "freeze" or "glue" a portion into a density (no possible move in the cloud for this part). It's frustrating fitting well a portion in the density then seeing it moving away on first wiggle all (and adding a lot of bands in space takes time).

Other Idea: "band to density" that would always link a segment to the same density position (not to neutral space).

Susume's picture
User offline. Last seen 1 day 21 hours ago. Offline
Joined: 10/02/2011
Coot interface

The University of Michigan undergrads used a program called Coot to build a protein into an ED cloud. Maybe looking at the Coot interface for viewing the cloud would yield some useful ideas.

toshiue's picture
User offline. Last seen 2 hours 42 min ago. Offline
Joined: 01/31/2016
Groups: Go Science
COOT
alwen's picture
User offline. Last seen 1 week 5 days ago. Offline
Joined: 10/03/2011
I wonder if part of the

I wonder if part of the change was the drop in regular players. In the past there were often 130 players online when I was on, now it's 70-90.

Maybe the 40-60 players who liked ED (I like them!) have drifted away.

Joined: 04/24/2014
Groups: None
Keep in mind

These are the results from players who filled out the puzzle survey. Not everyone who is active is inclined to fill out the survey (being an optional thing).

Fun fact: More people filled out the puzzle survey than last year's puzzle survey.

jeff101's picture
User offline. Last seen 12 hours 41 min ago. Offline
Joined: 04/20/2012
Groups: Go Science
I like ED puzzles

I think ED puzzles are important and wish I were better at them.
The last few I have done very poorly on as a soloist.
It seems the best I can do is evo my teammates' ED solutions.

frood66's picture
User offline. Last seen 3 hours 46 min ago. Offline
Joined: 09/20/2011
Groups: Marvin's bunch
I like ED puzzles - I believe

I like ED puzzles - I believe them to be a great opportunity for players to build on existing science. I also accept that , for many, it can be very difficult to see things even after careful viewing selection. I feel sure that in many cases this comes down to practice and clear instruction. Eyesight is also an issue - I rarely work for more than 20 mins at a go. (I principally use handwork)

I do, however, have one proviso: The resolution needs to be reasonable. If I cannot spot an aromatic when it's in the core, I don't bother to continue. When I have to ask if a side chain is methionine or Tyrosine I know that I am wasting my time (but that's me)

I agree with many comments above - particularly with Susume's re better tutorials and sliders. I think it would be better to have the depth of field tools on a nice small window of it's own. I rarely change the other 'main' viewing sliders once I've set them up. It is very annoying when any slider resets to default.

I do not like the fact that the ED cloud covers all space - this appears pointless to me at a number of levels.

The one thing I would love to see is an eraser for the rendered cloud area. In many (but not all) posted puzzles, parts of adjacent molecules are inside the working (rendered) area - I want to be able to remove these at an early stage. The ability to create one's own edited cloud as a template that can be recalled at anytime would be great! To be clear - I am not suggesting permanent erasing of the cloud data. I can see probs with this re threshold setting - but if any 'template cloud' created was reloaded with the 'original' setting - would that not work? Taken further, this ability could answer many of the viewing issues raised above.

frood66's picture
User offline. Last seen 3 hours 46 min ago. Offline
Joined: 09/20/2011
Groups: Marvin's bunch
I forgot this thought

Much about ED puzzles is due to lack of 'easy' practice.

I suggest some ED puzzles on short proteins - perhaps 60 segs or so. Start with high resolution - and I do mean high. These would be rather like QTTN puzzles - but using an ED cloud instead.

I'm pretty confident that many players would be turned on to this over a little time. I believe that the mystery has to be removed - this appeals to me and others as a simple, straightforward way to achieve this

Joined: 09/24/2012
Groups: Go Science
The new Nature paper is encouraging

A pity that ED are so hard to work, now that we know that it's there that we can be most useful for science.

Joined: 04/24/2014
Groups: None
We know that ED puzzles are

We know that ED puzzles are hard/not fun/etc. to work with, and that's part of the reason we've started this thread! The feedback so far has been very useful for helping us sort though the issues, and several recurring themes bring out the most important suggestions from players to help decide and prioritize improvements. Not being a coder myself, I can't speak to the feasibility of certain suggestions, or the timeline in which they could be implemented, but do be assured the team is watching this thread diligently!

Joined: 09/24/2012
Groups: Go Science
Here some suggestions by Batz

Batz said this in PM to our group and bkoep when we discussed the way we achieved the result of the paper. I think it's interesting for this threat (and sure Batz would agree that I quote this). May be other high level density players share this opinion:

"In order to improve the results of Electron Density Puzzles, I would like to point out some issues with Foldit, since I recently observed several things during the last puzzle.

1) Especially aromatic residues like Histidines do not really fit well within the density. Are there enough rotamers available for these residues in Foldit? Because it seems that other rotamers are preferred even though I tweaked the residue in its right place. After wiggling and shaking the aromatic ring was always offset again...

2) How much is the weight of the density in the scoring function? I noticed especially for Arginines and Lysines, that they do not like to be in their densities but rather form hydrogen bonds with other residues or the backbone. Therefore, it seems that the scoring term for hydrogen bonding is higher weighted than the term for the ED.

2b) Which belongs to the previous point. It's water molecules. Electron densities at that resolution always show water molecules within or next to the protein. This is another reason why in Foldit residues like Arginines do rather form hydrogen bonds with e.g. Glutamates although in reality a water molecule is in between these two residues.

3) At the end a really good thing, the Remix Tool! Especially for loops, I could much better sample several conformations and see which ones fit better into the density. It is much more suitable than the rebuild tool, it always shows the score and it is really fast. This makes it a perfect tool for these puzzles.

Now for those who use scripts, did you notice some similar problems, that residues do not fit quite well into their densities?"

Joined: 09/24/2012
Groups: Go Science
Relevant extract rom the paper

This is very true:

"Editing of nearly complete structures could form a base of easier Foldit puzzles for new players, allowing players to practice their model-building skills before moving to more difficult de novo model-building puzzles. To further improve the capability for Foldit players to aid crystallographers, ongoing development will make it possible for the players to add or remove residues with insufficient electron density, and have these changes accurately reflected in the Foldit score"

Joined: 03/30/2013
Groups: Go Science
ED puzzles and slab width

I find navigating thru the density frustrating. I would like to have better control over the slab width, and maybe some way to navigate along the z axis. I believe this has been mentioned by other people too.

My second desire would be some different way of treating fragments in the scoring. If I can fit 10 residues here and 10 residues there, and maybe 15 somewhere else, and I don't yet know how to make loops to connect them, I am likely to have a global score of -999999. This is also frustrating.

alcor29's picture
User offline. Last seen 2 hours 15 min ago. Offline
Joined: 11/16/2012
Maybe more time needed, etc.

I like EDs. They take a lot of work however. If a lot more time were allotted, in addition to all the improvements mentioned above, perhaps more people would try to work on them

Joined: 04/24/2014
Groups: None
"More time" is vague

More time is vague. How much time is more time? :)

Susume's picture
User offline. Last seen 1 day 21 hours ago. Offline
Joined: 10/02/2011
2-3 weeks

How about 2 weeks for <150 residues, and 3 weeks for >150 residues?

Joined: 09/24/2012
Groups: Go Science
7-40 days

-revisiting almost solved puzzles: 7 days
-easy short puzzles: 10 days

-flat heavy de novo: 30-40 days

Background: We needed the all 45 days to build the puzzle of the paper.

First you must find some free time to start the puzzle (several hours are needed to explore the density). This could make you wait a couple of days before starting playing.

Then you spent >1 hour to find the tryptophans or other starting point inside of the cloud and on the puzzle. Your time is over for next day.

Then another day, you start walking in the cloud. This is extremely slow (at least on my computer) when the cloud is visible on screen. One hour a day, 15 hours needed for me = 15 days.

Then you refine the all structure.

Total 20 days to finish the full hand design

Then you can start to share to team and to use recipes. Cloud view disabled to save time.

But some time, you have to manually correct your density (and find the 1 free hour for it). Total + 10 days = 30.

If there are disordered loops, add 10 days.

Total 30-40.

alcor29's picture
User offline. Last seen 2 hours 15 min ago. Offline
Joined: 11/16/2012
more time...

I think that should be polled or discussed with the general membership. (I myself can't do EDs anymore due to physical limitation.)

Joined: 04/24/2014
Groups: None
That of course, is always a

That of course, is always a good idea, however as you did bring up the idea, you clearly have some preferences!

It's cool to have opinions! That's what the thread is for.

alcor29's picture
User offline. Last seen 2 hours 15 min ago. Offline
Joined: 11/16/2012
preference

Well, I have no preference since I can't do them at all any more due to an arm difficulty no matter how much time I had. So I guess I'll withdraw the suggestion. My apologies.

alcor29's picture
User offline. Last seen 2 hours 15 min ago. Offline
Joined: 11/16/2012
more time

I hadn't seen Susume's answer. Her specs seem like an improvement which might invite more appreciation and participation.

tokens's picture
User offline. Last seen 10 weeks 6 days ago. Offline
Joined: 11/28/2011
Small denovos

It was really nice to see puzzle 1287. However I would also like to see small denovos with an ED cloud to make the task a bit more difficult while still being easier than the "real" ED puzzles.

alwen's picture
User offline. Last seen 1 week 5 days ago. Offline
Joined: 10/03/2011
One thing I noticed while

One thing I noticed while working on 1287 is how much changing a sidechain's position (rotamer) can affect the fit to density.

We could use some better tools for changing rotamers, other than scripts. As far as I know, the only way to change rotamers without a script is the < (left arrow) in Selection Interface, and sometimes this is crashy.

Pulling on a sidechain changes the rotamer, but not with any sort of control.

I got some pretty good jumps running scripts like Rav Snap and Wiggle, and using the spacebar to make it skip positions that obviously didn't fit the density.

jeff101's picture
User offline. Last seen 12 hours 41 min ago. Offline
Joined: 04/20/2012
Groups: Go Science
Some non-script things to try:

To change the sidechain positions without using scripts,
try wiggle sidechains or shake. Wiggle sidechains
can be done at different wiggle powers and
can be done while varying the clashing importance.

Joined: 06/06/2013
Groups: Gargleblasters
ED teaching still needed

I still can't see anything. I think I might have scored worse on ED for crystallin than on de novo. If this is the wave of the future, I still need a lot more help with puzzles and tools. I'd prefer a small puzzle with no known reload from before. That way I can work at trying to find where to start threading the protein through
Skippy

Joined: 09/24/2012
Groups: Go Science
in the tutorial

A small chain (2-3 segments) with a tryptophan and related ED

Joined: 09/24/2012
Groups: Go Science
tutorial 2

Then the same but with a full cloud but the tryptophan almost in the middle: "find the tryptophan by clicking shift + Q on the chain on the middle to see the shapes in the density cloud".

Then n° 3: "shift + Q on the chain then move it in the cloud and fint the tryptophan position

n°4: "now that you have the starting tryptophan, align the other 2-3 segments in the density by freezing the tryptophan and moving the other segments."

Susume's picture
User offline. Last seen 1 day 21 hours ago. Offline
Joined: 10/02/2011
Let us play with winning solution

Could you post a no-points puzzle with the published solution from 1152 as the starting position, and let us load our saved solutions from 1152, so we can load one as a guide to the other and see where we diverged? This would also let people see what a well-placed protein in a cloud looks like.

Note, the saved solutions loaded as guides would have to load in the right position in space relative to the cloud - there is a feedback about this not currently working: https://fold.it/portal/node/2002559

Susume's picture
User offline. Last seen 1 day 21 hours ago. Offline
Joined: 10/02/2011
Muscular Dystrophy

Much as I would love to see an ED puzzle of the LGMD dysferlin protein, I hope you will postpone that until after you have given us at least 2-3 more small ED practice puzzles, like the recent revisit. That would help people get more used to ED and the ED tools, and can only improve the number and quality of solutions on a hard protein like LGMD.

Sitemap

Developed by: UW Center for Game Science, UW Institute for Protein Design, Northeastern University, Vanderbilt University Meiler Lab, UC Davis
Supported by: DARPA, NSF, NIH, HHMI, Amazon, Microsoft, Adobe, RosettaCommons