adding timeouts to scripting api

Case number:699969-995839
Topic:General
Opened by:Seth Cooper
Status:Open
Type:Question
Opened on:Wednesday, August 28, 2013 - 21:04
Last modified:Friday, December 20, 2013 - 22:36

Since we've recently introduced a change to wiggle that affects the length of an iteration, we wanted to update the scripting API to allow more control over how long operations will run for. Here's a proposal for the change.

For each existing function that has an iteration argument, a new function will be added with the suffix 'Timeout'. In the place of the 'integer iterations' argument, two new arguments 'integer noimproveSeconds, integer maxSeconds' will be added. Their meaning is as follows: the operation will stop after either 'maxSeconds' have elapsed, or 'noimproveSeconds' have elapsed without any notable improvements in score.

The following functions will be added:

void structure.LocalWiggleAllTimeout(integer noimproveSeconds, integer maxSeconds[, backbone, sidechains])
void structure.LocalWiggleSelectedTimeout(integer noimproveSeconds, integer maxSeconds[, backbone, sidechains])
void structure.MutateSidechainsAllTimeout(integer noimproveSeconds, integer maxSeconds)
void structure.MutateSidechainsSelectedTimeout(integer noimproveSeconds, integer maxSeconds)
void structure.RebuildSelectedTimeout(integer noimproveSeconds, integer maxSeconds)
void structure.ShakeSidechainsAllTimeout(integer noimproveSeconds, integer maxSeconds)
void structure.ShakeSidechainsSelectedTimeout(integer noimproveSeconds, integer maxSeconds)
void structure.WiggleAllTimeout(integer noimproveSeconds, integer maxSeconds[, boolean backbone, boolean sidechains])
void structure.WiggleSelectedTimeout(integer noimproveSeconds, integer maxSeconds[, boolean backbone, boolean sidechains])

The following functions will be deprecated (they will be left in place for backwards compatibility):

void structure.LocalWiggleAll(integer iterations[, boolean backbone, boolean sidechains])
void structure.LocalWiggleSelected(integer iterations[, boolean backbone, boolean sidechains])
void structure.MutateSidechainsAll(integer iterations)
void structure.MutateSidechainsSelected(integer iterations)
void structure.RebuildSelected(integer iterations)
void structure.ShakeSidechainsAll(integer iterations)
void structure.ShakeSidechainsSelected(integer iterations)
void structure.WiggleAll(integer iterations[, boolean backbone, boolean sidechains])
void structure.WiggleSelected(integer iterations[, boolean backbone, boolean sidechains])

Let us know if you have any feedback.

(Wed, 08/28/2013 - 21:04  |  20 comments)


steveB's picture
User offline. Last seen 2 years 51 weeks ago. Offline
Joined: 01/08/2009
Groups: Void Crushers

This idea was submitted as a feedback in 2009

http://fold.it/portal/node/986279

and received 11 votes at the time.

I think it will make a substantial difference to the efficiency of the scripts.

marie_s's picture
User offline. Last seen 1 year 49 weeks ago. Offline
Joined: 05/18/2008
Groups: None

As people have all sort of computers, the result of recipes will be very different between players.
Also, result will depend on the other tasks of the computer so comparing result between options will be more difficult.
If you want to calibrate the computer to add a multiplier, how can we?

Joined: 11/10/2007
Groups: Window Group

Since the timeout would be set when calling the functions, script authors could account for variations in computer power, protein length, etc., however they like. We could provide some utility functions for calibration, for example, a function that returns how long some operation takes on the current computer.

Joined: 09/21/2011
Groups: Void Crushers

I have several comments (don’t take this as negative pls):
1. I don’t like the term deprecated for the original functions Those function should stay fully supported and the new ones should be just add-ons or an extension on the current function. (See also point 4)
2. I don’t get why the mutate functions are included in the list. The way I understand the current mutate counter is that the number of tries each segment gets is the iteration count. That is exactly what I want from a mutate function. (A little optimization might be hat when the first iteration does not change anything no further tries are done, this optimization might also work for other functions if an extra iteration is expected to do the same)
3. The nogain parameter should actually be a nochange parameter because when running not on CI 1 or having bands enables losses are expected and should be allowed. And also there should be a cutoff point because very tiny changes are not important. However this will lead to a difficult to design algorithm, how to find out if the protein is pretty stable because for instance on low CI it keeps moving.
4. Instead of new functions I would rather like to just adding 2 extra parameters to the current functions.
5. Putting this in makes the result of recipes largely dependent on the speed of the computer and even on the same computer non reproducible. That is a big disadvantage.
6. Instead of adding just more functions to be used to compensate for the size of the protein I would like that to be included in the code itself. For instance the given parameters will be exact if the total number of segments where scorechanges can occur is 50 and autoadjusted to other numbers.
7. A possible optimization in design puzzles like the flu design is to ignore parts of the locked protein that will not change in score in the score computing.
8. This attempt to tackle performance changes when the Rosetta core changes is not targeting the real problem . A quantum leap in performance is needed for the game. If that might be accomplished we might be able to work on very big proteins. I don’t know if the game makes use of the parallel processing power of the graphics processors that are in most computers, but that might be something to look at.

bertro's picture
User offline. Last seen 7 weeks 9 hours ago. Offline
Joined: 05/02/2011
Groups: Beta Folders

I completely agree with Timo. Well said!

brow42's picture
User offline. Last seen 1 day 18 hours ago. Offline
Joined: 09/19/2011
Groups: None

I disagree with Timo's overall opinion. Computer speed always has been and always will be a factor and this just codifies human (im-)patience. I don't like using less precise wiggle but I have to, because it spends too much time doing nothing or going up by 0.01 (in still the first iteration). I don't like running DRW with only 5 rebuilds (what did you change the default to, 40? Jeez!) because it spends 2 minutes stabilizing each rebuild (mostly, shaking sidechains, but not actually changing sidechains, for 1 iteration). I welcome a change that relieves me of the duty of clicking stop (which, I am literally doing, canceling shake to get to the next wiggle).

Basically, the LUA change/change-check/repeat code can't do its job until that first iteration ends.

I understand that my desktop computer will finish finding/losing points before somebody else's netbook. But honestly, I can't sympathize strongly because watching Foldit sit there doing nothing is really really aggravating, and causes an unhealthy increase in blood pressure.

It might be better to keep iterations as the fundamental unit, but with rosetta-level timeouts. The difference is minor (almost non-existent) if coders are using change-check wrappers.

Joined: 09/21/2011
Groups: Void Crushers

1. DRW still has 15 rebuilds as the default, but I changed the max to 40 on request from other folders.
2. It only does a local shake (fast) on each rebuild
3. It only stabelizes the highest in each score category and you can change the number of slots.
4. DRW is highly optimized compared to the original. All shakes are now just 1 cycle because the second or more cycles rarely gained. Fuzing can be controlled, if you wish only if there was a gain.
So pls check the facts first before you comment on DRW.
And if you want to change the stabelizing part to skip the shake just add -- in one line of the recipe.

brow42's picture
User offline. Last seen 1 day 18 hours ago. Offline
Joined: 09/19/2011
Groups: None

The facts are, 1 iteration of shake takes too long, I'm clicking stop by hand while the script is running, and I only run 5 rebuilds because it takes too long overall.

Joined: 04/15/2012
Groups: Beta Folders

First of all, thank you for adding these. The noGain parameter (or as Timo pointed out, nochange) should be very helpful. Also, I agree with Timo, especially the fact both should be supported. They have different uses. Also, ignoring locked protein sections would be helpful if it hasn't already been taken into consideration... Also, I do agree with the last point there as well.

Joined: 04/19/2009

From a more basic (non-scripting) point of view:

Which wiggle change? The very slow, precise one? Or the "less precise" that many of us need to use because of our computers?

Does any of this affect basic wiggle? GUI?

By making these changes, will it mean that those without high computing power (the hottest new computers) may not be able to run certain scripts?

Joined: 04/15/2012
Groups: Beta Folders

Assuming I understand this correctly:

Neither.

No, No.

No.

The wiggle algorithm is not changing, just how the scripters script it to run (instead of using iterations, using time/stop when gain is less than _)

Joined: 09/21/2011
Groups: Void Crushers

I forgot to mention dont take out the less precize wiggle. Period, exclamation point.

Joined: 04/15/2012
Groups: Beta Folders

Also, on a related note, could we set it to sop when there has been x gain/loss, with a timeout (of either iterations or time)?

Joined: 09/21/2011
Groups: Void Crushers

This would mean yet another parameter, but I like it. Would be handy in for instance compressor.

Joined: 11/10/2007
Groups: Window Group

Thanks for all the feedback. Having a "no change" parameter rather than a "no improvement" one makes sense. If the timeouts were just added to the existing functions, would a "no change" parameter still be needed, or would the iterations be enough? That is, is one of these preferable?

A) void structure.LocalWiggleAll(integer iterations[, boolean backbone, boolean sidechains, integer maxSeconds])

or

B) void structure.LocalWiggleAll(integer iterations[, boolean backbone, boolean sidechains, integer noChangeSeconds, integer maxSeconds])

Joined: 04/15/2012
Groups: Beta Folders

B. I wouldn't know how long it would take for there to be no change, so I wouldn't know when to set the timeout. They're different.
As I think of it, would it be possible to specify an amount of change (ex. +60 points), or no?

Also, how would you specify with multiple optional arguments of type integer? Or would both noChangeSeconds and maxSeconds have to be set?

Joined: 09/21/2011
Groups: Void Crushers

Neither of these solve the real problem that wiggle is becomming too slow for comfort.
If forced I would choose option B) but without confidence.
And it really should have an extra parameter boolean uselessprecize.

bertro's picture
User offline. Last seen 7 weeks 9 hours ago. Offline
Joined: 05/02/2011
Groups: Beta Folders

Are these changes on the table?

Joined: 09/24/2012
Groups: Go Science

May be it's too late to react?

My strong preferences:

1) Usual command and only add new arguments
2) Add argument Integer maxSeconds in most listed commands, default infinite
3) Boolean uselessprecize should remain on screen (in order to be applicable to hand work and old recipes). Add new functions:

behavior.GetUseLessPreciseDisabled()
behavior.SetUseLessPreciseDisabled(boolean) -- default =true
4) In addition, add argument: boolean uselessprecize

Example (Option B + uselessprecize)

C) void structure.LocalWiggleAll(integer iterations[, boolean backbone, boolean sidechains, integer noChangeSeconds, integer maxSeconds, boolean uselessprecize])

with default: infinite, true, true, infinite, infinite, GetUseLessPreciseDisabled()
and no minimum ppi for noChangeSeconds

Joined: 11/10/2007
Groups: Window Group

Thanks for all the feedback, we're still considering making this kind of change.

Sitemap

Developed by: UW Center for Game Science, UW Institute for Protein Design, Northeastern University, Vanderbilt University Meiler Lab, UC Davis
Supported by: DARPA, NSF, NIH, HHMI, Amazon, Microsoft, Adobe, RosettaCommons