High wiggle leaves huge ideality score loss

Case number:699969-997053
Topic:General
Opened by:AsDawnBreaks
Status:Open
Type:Bug
Opened on:Sunday, February 16, 2014 - 02:14
Last modified:Wednesday, February 26, 2014 - 21:38

See attached screenshot. This is on 845. Several segments scoring -300 to -500 in ideality. High wiggle should really be able to resolve something that bad...

Latest devprev.

(Sun, 02/16/2014 - 02:14  |  31 comments)


Joined: 04/15/2012
Groups: Beta Folders

Better after some local wiggles on single segs, but it really should be fine with a global wiggle and local wiggling smaller stretches.

Joined: 04/15/2012
Groups: Beta Folders

Oops, didn't add screenshot. Attached.

Joined: 11/28/2011

After idealize I do a LPW before to switch to high power. My feeling is that HPW is unable to deal properly with low scoring segments whether or not they are ideal.

Joined: 04/15/2012
Groups: Beta Folders

Low power wiggle doesn't deal with ideality.

Joined: 11/28/2011

Sorry, I wasn't clear enough. Idealize first, then do a LPW before to switch back to HPW.
A zero lenght band prior to idealize is sometimes required to avoid the protein to explode.

Joined: 04/15/2012
Groups: Beta Folders

Ah, thanks! Didn't think of ZLBs...

jflat06's picture
User offline. Last seen 21 hours 17 min ago. Offline
Joined: 09/29/2010
Groups: Window Group

Believe it or not, this is actually working as intended.

The Baker Lab has discovered recently that deviating from the ideal values for these degrees of freedom can provide better predictions in some cases.

The idea with high power wiggle is that it lets you 'trade off' ideality for increases in some of the other score terms. So even with the bad ideality terms, you're getting even better gains somewhere else that offset these.

That said, we are evaluating the usefulness of high-power wiggle and we may find it isn't helpful in the case of Foldit.

Susume's picture
User offline. Last seen 2 days 23 hours ago. Offline
Joined: 10/02/2011

I hope you won't take out high power and leave us with just the current low and medium. Consensus on my team seems to be that medium is not good for much, score-wise. We are mostly using low, and high only at certain times. If there is some functionality in medium that you want us to use because it makes better predictions, it has to pay off more in the score function.

jflat06's picture
User offline. Last seen 21 hours 17 min ago. Offline
Joined: 09/29/2010
Groups: Window Group

The functionality of medium-power wiggle is to introduce the degrees of freedom for peptide bonds. Normally you shouldn't have to worry about these, since the should stay at their ideal values and aren't allowed to vary.

However, some of the tools in the game let you modify these degrees of freedom, allowing you to create unidealities. The biggest culprit here is the Cut tool. With the cut tool, you can botch two amino acids together in extremely unideal states that score terribly. Once you get in that state, there are options like the idealize tool that will get rid of these unidealities, but not without global changes.

Because of that, we've included these degrees of freedom in medium-power wiggle. That way, you can wiggle out these poorly idealized regions on the protein into more reasonable states.

This functionality has actually always been in Wiggle. Prior to the New Chapter, these extra degrees of freedom were only added on the even iterations of wiggle. Some players had noticed before that the even iterations would take a lot longer - this is why.

The design was such that even if you managed to get some unidealities into your pose, they would be resolved by Wiggle automatically over time. With the New Chapter, we decided that since you don't always need these peptide degrees of freedom, it would be simpler and faster to let you have the ability to run them only when you needed them.

As for why we kept medium as the default - we wanted to preserve the behavior that the default "Wiggle" of Foldit would address unidealities, just like it did before.

As for high power wiggle, we are evaluating the scientific usefulness. We understand it's great for getting points, but unless it's fulfilling some scientific purpose in Foldit, it might be best to remove it, since it might even be causing some issues as well.

spmm's picture
User offline. Last seen 33 weeks 15 hours ago. Offline
Joined: 08/05/2010
Groups: Void Crushers

Thanks for the explanation jflat. The following paras:

'This functionality has actually always been in Wiggle. Prior to the New Chapter, these extra degrees of freedom were only added on the even iterations of wiggle. Some players had noticed before that the even iterations would take a lot longer - this is why.

The design was such that even if you managed to get some unidealities into your pose, they would be resolved by Wiggle automatically over time.'

This seems to mean that if you were using wiggle manually you would never get to an 'even wiggle' is this still the case?
Personally I am really enjoying being able to do more manually with the protein before it turns to cement, that includes trying different versions of folds in tracks. Can we just leave it as is for a couple of puzzles to see how people use it? Early days yet.

jflat06's picture
User offline. Last seen 21 hours 17 min ago. Offline
Joined: 09/29/2010
Groups: Window Group

The non-script Wiggle still used iterations as well - if you kept it running for more than a few seconds it would fall into an even iteration.

As for keeping it, I think low/medium power will almost certainly exist in some form, but we may experiment with the effects of removing or altering high power wiggle.

Joined: 04/15/2012
Groups: Beta Folders

Jflat, I understand that it gives some freedom, but not being able to resolve a several hundred point loss by itself? That loss is many times the affect of other score terms, even all of them at many times, save when you have a really bad clashing problem.

spmm's picture
User offline. Last seen 33 weeks 15 hours ago. Offline
Joined: 08/05/2010
Groups: Void Crushers

ADB it is not clear from your initial post what the difference in segment ideality or overall score was from the start point to after using HP wiggle. Jeff has explained that there is a trade off with HP wiggle which may reduce the ideality score.

Joined: 04/15/2012
Groups: Beta Folders

I used low, medium, and high, and ideality was -500 on a few segs the whole time until I did a lot of local wiggling (different lengths, sometimes 1-3, sometimes te whole chain). My issue is that there's a -500 seg that should somehow be taken care of.

And I don't get how HP would reduce idealization, as the tooltip says it uses high idealization!

jflat06's picture
User offline. Last seen 21 hours 17 min ago. Offline
Joined: 09/29/2010
Groups: Window Group

"High Idealization" is a bit of a misnomer, I agree. Often times we have to come up with short descriptions for complex things, and inevitably something is miscommunicated.

What is meant by High Idealization in this case is that Wiggle is optimizing a higher number of degrees of freedom which are constrained to ideal values. It doesn't mean that it will necessarily increase the ability to resolve the ideality term.

As for your case where you had -500 Ideality on a particular segment - it could be that by allowing that segment to have bad ideality, the rest of the protein was allowed to be much more compact or form a bunch of hydrogen bonds (or something else). It's obviously better if you can find a similar shape that doesn't have that bad ideality term, but if Wiggle cant find that shape, the messed-up-ideality shape is still the second best option.

Joined: 04/15/2012
Groups: Beta Folders

The issue I'm having is that my score was at ~3000. My score after local wiggles and such was around 7-8000. That's a few THOUSAND lost points wiggle was ignoring. Seems either wiggle is ignoring a huge loss, or the weighting for ideality is a tad off. If the ideality score drops 10,20, even 50 points? Sure. But the only other way to loose 500 points is if the protein was almost on top of itself, in which case my protein would blow apart. A "we'll allow a little freedom" shouldn't result in several completely red segments (which, by the way, are set to turn red at ~-100 points, a fifth of the loss of what I'm seeing).

gitwut's picture
User offline. Last seen 33 weeks 4 days ago. Offline
Joined: 05/18/2012
Groups: Contenders

Is it just me, or does anyone else think this information would've been a lot more useful 2-3 weeks ago?

spmm's picture
User offline. Last seen 33 weeks 15 hours ago. Offline
Joined: 08/05/2010
Groups: Void Crushers

Seth Cooper posted in the blog about Wiggle Power on Tue, 01/28/2014 - 11:59 (not sure which time zone)

gitwut's picture
User offline. Last seen 33 weeks 4 days ago. Offline
Joined: 05/18/2012
Groups: Contenders

I read the blog post about wiggle power. It states nowhere that they are considering removing high wiggle power. For anyone who has spent the last 3 weeks trying to come up with strategies which include the use of High wiggle power, it means that we'll be at square one again if they pull it. The blog states:

We may change precisely which bond geometry low, medium, and high power wiggle correspond to in the future, but plan to keep this general structure.

spmm's picture
User offline. Last seen 33 weeks 15 hours ago. Offline
Joined: 08/05/2010
Groups: Void Crushers

Yes agree gitwut, apologies, misunderstood your comment being solely related to removal of HPW.

gitwut's picture
User offline. Last seen 33 weeks 4 days ago. Offline
Joined: 05/18/2012
Groups: Contenders

No apology necessary. I wasn't very specific with my leading comment, so your reply was quite reasonable.

Joined: 09/21/2011
Groups: Void Crushers

Reading all this, I am starting to wonder if it would be an idea to give us more control on WHAT wiggle does. So no longer LOW, MEDIUM, HIGH. But Look at peptide bonds ON/OFF. (Repair mode after cut closing) And more of those controls. Of course there should be functions to manipulate this.

Joined: 04/15/2012
Groups: Beta Folders

I had thought that a custom waiting of wiggle would be nice (not affecting the score function itself, just telling wiggle what to look at, like CI).

Joined: 09/21/2011
Groups: Void Crushers

That is exactly what I mean. Be it on/off switches or even a dashboard with sliders from 0 to 1.

spmm's picture
User offline. Last seen 33 weeks 15 hours ago. Offline
Joined: 08/05/2010
Groups: Void Crushers

Well maybe but don't forget it has to work with hand folding and low powered endpoints as well - mot just scripts.

frood66's picture
User is online Online
Joined: 09/20/2011
Groups: Marvin's bunch

so if I understand correctly now....LPW is more or less like old odd iteration, MPW similar to old even iteration....and HPW is new (and possibly undoes previous good work).

Firstly, I have to agree with gitwut ^

Secondly, I now fear my car analogy with twin steering axles was more accurate than intended (It was supposed to be just a joke)

alcor29's picture
User offline. Last seen 5 hours 14 sec ago. Offline
Joined: 11/16/2012

My observations so far are neither planned nor methodical or conclusive, but my impression is that sometimes high works best, and sometimes low does. Medium is a mystery and doesn't seem really useful. So I would get rid of medium, and set the default to low.

spmm's picture
User offline. Last seen 33 weeks 15 hours ago. Offline
Joined: 08/05/2010
Groups: Void Crushers

I think they do three different things and some of the same things, and I don't think we should throw any of them out until we have had a chance to experiment more.

Joined: 09/21/2011

I only barely understand this discussion, but let me talk in terms of the behavior of the protein. Wiggle gets stuck. I have sections that never had cutpoints, but didn't wiggle out all the way. Often I can just run a very mild bander for about five seconds (tremor with weak bands)and it dislodges the wiggle and i'm back in business. Also, it will finish wiggle when I jerk the behavior slider down low and back up(sometimes). So, I don't know about something being super ideal somewhere else in the protein that's locking in the unideal portion. It feels to me like wiggle is hanging, not- finding an optimal wiggle based on criteria that I'm not familiar with.
In any event, I doubt anyone is keeping the half complete wiggle and preserving the super optimized portion then carefully retweaking the hot spots so as not to disturb the rest of the protein. I suspect we're all just working around the stuttering wiggle.
We'll probably rewrite drw to have stronger fuses (perhaps a couple GAB bands in the fuse) to push past the stutters and make sure we don't miss a good rebuild because wiggle didn't complete its wiggling.
ps. aside from sticky wiggle, I like the way low and high wiggle function a lot.

Susume's picture
User offline. Last seen 2 days 23 hours ago. Offline
Joined: 10/02/2011

Wiggle does get stuck a lot -- perturb the protein with a low-ci shake, a low-ci wiggle, or a very brief wiggle with bands, and instead of wiggling back up near the previous score, full ci wiggle will get stuck hundreds of points below it. It's leaving points on the table that we know are in the protein, because we just had them a moment ago. It's necessary to cycle through all the different ways you can think of to perturb the protein, hoping that one will knock wiggle loose so the actions you're actually interested in (e.g. rebuilds) can gain again.

Also, local wiggle is now getting lots of points that global wiggle leaves behind. Is this intentional? Or is something wrong with global wiggle? This is how the game was in 2012: local wiggle scripts in end game would find significant points that global wiggle had left behind, and on the other hand would cement the protein, which is why we saved them for end game. But for the last year at least, until newchapter, using local wiggle scripts in end game has been largely ineffective - because global wiggle was finding the points first. Now it is overlooking them again and leaving them for local wiggle to find. Maybe we need to alternate local and global wiggle in our scripts and hand folding.

spmm's picture
User offline. Last seen 33 weeks 15 hours ago. Offline
Joined: 08/05/2010
Groups: Void Crushers

Agree with your first para Susume, and I wondered if shake had changed because the side chains don't seem to move as much, or also get stuck even on low ci.

I haven't noticed big changes in local wiggle script behaviour, ie no significant improvement in end game, improvement if used earlier as usual - but have only been playing pig not testing in other puzzles.

Sitemap

Developed by: UW Center for Game Science, UW Institute for Protein Design, Northeastern University, Vanderbilt University Meiler Lab, UC Davis
Supported by: DARPA, NSF, NIH, HHMI, Amazon, Microsoft, Adobe, RosettaCommons