New Lua functions, any suggestions?

Case number:699969-2004288
Topic:General
Opened by:Timo van der Laan
Status:Open
Type:Suggestion
Opened on:Thursday, October 5, 2017 - 19:13
Last modified:Wednesday, January 10, 2018 - 05:46

Hi all,

In a short while I will be working as a volunteer for Foldit.
One of the first things I will be doing is adding new Lua functions.
Of course I have a list of functions I would like to add for instance:
- bonus functions
- an extra parameter to recentbest to return only valid scoring positions
- some cutpoint functions
- and more
However, I would like to hear the opinion of other script writers, which functions they would like to see.

Happy Folding, Timo

(Thu, 10/05/2017 - 19:13  |  34 comments)


Joined: 04/20/2012
Groups: Go Science

Below are some old Feedbacks listed in
http://fold.it/portal/feedback/filter/all/author:421719%20assigned:421719

Make LUA commands band.SetNote and band.GetNote
http://fold.it/portal/node/993966

recipe.GetName and recipe.GetRecipeID in LUA
http://fold.it/portal/node/2001463

One-directional bands (like bumpers) for Contact Map puzzles
http://fold.it/portal/node/996372

How to tell within LUA recipes if an autosave/quicksave-slot is already occupied?
http://fold.it/portal/node/1998859

Dialog boxes that allow mouse-free input
http://fold.it/portal/node/1998680

Make dialog.AddLogSlider() similar to dialog.AddSlider()
http://fold.it/portal/node/998413

Can ui.AlignGuide() output an RMSD value?
http://fold.it/portal/node/998412

LUA function structure.GetAtom to return atom types
http://fold.it/portal/node/997504

Create hoops (like bands, but the protein can slide through them)
http://fold.it/portal/node/992670

Allow recipes to read keyboard, mouse buttons, mouse position, etc.
http://fold.it/portal/node/992669

Recipe command for "Align Protein to Density"
http://fold.it/portal/node/992656

Improving the "Align Protein to Density" or "Center Protein on Density" button
https://fold.it/portal/node/993959#comment-32465
Having a LUA implementation of the
Nelder-Mead Simplex Direct Search Method
for optimization would be very helpful.

Let us get/set (read/change) the wiggle power
from within scripts.

Enzyme's picture
User offline. Last seen 48 years 4 weeks ago. Offline
Joined: 07/10/2008
Groups: None

In addition to many of jeff101's that I would like o see:

recipebest methods similar to recentbest but without the set. The set is auto when the recipe begins. This would allow the recipe to set recentbest throughout, then recipebest.Restore() at the end

The ability to call recipes(with and without parameters) from within another recipe(including the ability for the recipe to verify it's existence first). Also the ability to call methods from within the other recipe without running the entire recipe.

get/set the rama map from within recipes

the ability to set blueprints from within scripts

+ any other button in the UI which doesn't yet have a script equivalent

Enzyme's picture
User offline. Last seen 48 years 4 weeks ago. Offline
Joined: 07/10/2008
Groups: None

the ability to detect which segments are affected by various filters like the Core, Residue IE, SS Design, Ideal Loop, etc filters

Joined: 09/24/2012
Groups: Go Science

In addition to what jeff101 recalled above:

Pause a recipe for some times (71 votes):
http://fold.it/portal/node/986021
http://fold.it/portal/node/998144
http://fold.it/portal/node/2002242

Timed wiggles and shakes (12 votes):
https://fold.it/portal/node/987798

Wiggle power function (11 votes):
https://fold.it/portal/node/996831

Change status (title of the Output window when a recipe is running)(8 votes)
https://fold.it/portal/node/987953

Ideal SS (7 votes):
https://fold.it/portal/node/997505

Tweak tool (6 votes):
https://fold.it/portal/node/994411

Add "auto" argument to get/set secondary structure (4 votes)
https://fold.it/portal/node/2002998

Meta tool for running recipes from the cookbook (4 votes):
https://fold.it/portal/node/990676

Load, save, share solutions (3 votes):
https://fold.it/portal/node/998025
https://fold.it/portal/node/2000684
https://fold.it/portal/node/2002598

Save.SaveObject() - LoadObject() (or any hidden segment or note)(2 votes):
https://fold.it/portal/node/994274

Joined: 09/24/2012
Groups: Go Science

Dialog display timing:

When a dialog is displayed, if the user doesn't react after some times, the recipe can go on with some default ("ok"). The problem today is that you can run a recipe for night, butwhen you come back, you discover that it was stiffed on a dialog you forgot to click "ok" ...

Joined: 09/24/2012
Groups: Go Science

Drug Design tools:

- first numbering the atoms (like residues are numbered)
- then we can add, delete atoms and structure at atom x the same way we go through all remix possible solutions (no need to know what we add: we just automatically mass try everything that is proposed)

Joined: 09/24/2012
Groups: Go Science

Add / Delete residue

(if it's not possible, nothing happens)

Joined: 09/21/2011
Groups: Void Crushers

Already there.
bool structure.InsertResidue(nr) Inserts after nr and
bool structure.DeleteResidue(nr) Delete the segment nr.
So to reverse the first operation you should use nr+1 in the Delete.

Joined: 09/24/2012
Groups: Go Science

puzzle.GetPuzzleFilterScorePartNames()

and

.GetFilterScore (string FilterScorePart)

where FilterScorePartare any "puzzle level" (not by residue) scores like:

Filters (default): the one we see on the top of the score window (total points from filters)
Core
SS
SS Design
Residue Count
Ideal Loop

etc

Then rewriting the (unused) SetFiltersDisabled(bollean setting) the following way:

behavior.GetFiltersDisabled(string FilterScorePart)
behavior.SetFiltersDisabled(string FilterScorePart)

smortier's picture
User offline. Last seen 4 days 8 hours ago. Offline
Joined: 03/10/2016
Groups: None

From Blipperman on the news post:

"Mutatable VoidCrusher
I have often desired to have the mutate all after crushing and before wiggling."

Joined: 09/21/2011
Groups: Void Crushers

That is about one of my recipes, not about a Lua function.

Joined: 04/20/2012
Groups: Go Science

More ideas:

(1) A LUA way to band between residues in different symmetric subunits.
Like in a trimer, how do you make a band between a residue on monomer 1
& a residue on monomer 2 from within a recipe? I think right now,
you can only do this manually.

(2) Read what residue #'s and atom #'s are at one or both ends of a particular band.
Perhaps use the Smart Bands idea (https://fold.it/portal/node/2002640) to encode this.

(3) A LUA way to get/set the residue pairs selected in the Contact Map.
See https://fold.it/portal/node/998342 for more details.

(4) Let players download from the website two or more recipes at once
by entering their recipe #'s in a dialog box (http://fold.it/portal/node/997032).

brow42's picture
User offline. Last seen 2 days 20 hours ago. Offline
Joined: 09/19/2011
Groups: None

Wow, there were so many things I wanted but I won't be able to think of them now.

I'd really like to get atom-atom distances without banding, although this might provide too much information.

I'd like the functionality of my atom tables script to be replaced with queries into rosetta. It's basically luck that my script works at all. The rosetta query would always be correct, even if a weird molecule / AA / nucleoside shows up, or the pH or salinity is non-standard.

#1 game changer: a way to identify existing h-bonds, and the atoms they pair, so that we can band them, or count them, or something. disulfide bonds should also be detectable in the same way. Anticipate other kinds of bonds that might appear in future puzzles so the scripts don't crash, such as bonds to metal ion. Salt bridges are probably the same as H-bonds. Stretch goal: a way to find potential hbonds. Currently I would select AA in a radius, then do a pairwise check of atom distances by banding. This is so ridiculous I don't even have a script that does this (well, maybe one).

Reading the endpoint information of existing bands as Jeff101 suggests allows bands to be a proper user input method.

We definitely need more methods to query and band mirror residues in symmetry puzzles. Perhaps a one way is to just make these addressable above 1 to N (watch out for ligands in the numbering sequence though). There might need to be a function to map real id -> mirror ids.

There are many LUA functions that might be better returning a list for the entire protein or residue, things that we ALWAYS have to loop over. Just off the top of my head: all SS, all AA, all pair-wise AA contact values, all donor atoms in residue i. But if we just end up looping over the list, then it doesn't matter. I'd have to think hard to find a use case.

This looping paradigm for i=1,GetCount do GetProperty(i) has a problem: a lot of things are potentially invalidated by wiggle, shake, undo, quickload, user actions etc. Examples: rotamers, atom count, hbonds, bands. Perhaps iterators? Perhaps we just need a definitive list of that they are so that we are aware. The existing API is pretty ingrained.

There may be other interesting things in rosetta that would be interesting to expose: if foldit could give us a void, maybe we could collapse it.

More control over valid mutations. Crazy idea: let us dynamically change the residue subscore temporarily (this violates a condition obviously)

Susume's picture
User offline. Last seen 4 hours 3 min ago. Offline
Joined: 10/02/2011

Most of these are mentioned above - just throwing in my vote.

Function to get band endpoints (seg num and atom num).

Get/set contacts in the contact map (so highlighting contacts in the map becomes a form of input rather than just a way to view things).

Find out which residues are causing filter penalties.

Functions to get what SS foldit thinks a residue is (the auto structure if that's the best we can get, but I really want to know what SS the design filters consider it to be) and whether it considers the residue core, borderline, or surface.

Tweak, specifically the straighten portion of the tweak tool - this would allow us to gradually move backbone shapes closer to the ideal SS shape without being as destructive as moving them all the way to ideal SS. I never use tweak manually any more (I just drag the rama dots), but it would be useful in scripts.

A way to limit the list of allowed AAs for a given residue (thereby limiting the AAs the mutate tool will pick from). Use case: foldit has buried a polar atom; limit the allowed AAs for that residue to orange, then let mutate tool try to find a good combination for that residue and its neighbors. Another use case: you have an H bond to the ligand that you don't want to give up; limit the AA for that residue to what it currently is and then let a mutate script run to find good alternatives for the other residues.

I love the idea of getting from Rosetta the endpoints of existing hydrogen bonds. This could be used in conjunction with filter violations and SS to add bands to protect sheet bonds that are getting lost during wiggle, or to add bands and/or limit allowed AAs to protect bonds to the ligand.

spvincent's picture
User offline. Last seen 1 hour 2 min ago. Offline
Joined: 12/07/2007
Groups: Contenders

When remix can't find solutions, use rebuild instead

Joined: 05/19/2009
Groups: Contenders

functions that will:
- release the currently loaded files on disk, so that they can be overwritten by the OS
- reload specific files whose handles were released by the function requested above. (quicksaves, autosaves etc)
- an option to start the client without having to maneuver through the user interface
- an option to start a default script upon starting the client
- an option to save the cookbook without exiting the client.
- an option to release the cookbook so it can be overwritten by the OS
- an option to reload the cookbook from disk (latest version)

Thanks and good luck !
BP

Joined: 05/19/2009
Groups: Contenders

- the possibility to have 9999 quicksaves instead of just 99
- quicksave numbering in the four digit format 0001, 0200 etc instead of 1,4,7,10,12 etc.

Joined: 12/27/2012
Groups: Beta Folders

Treat condition (filter) penalty/bonus as scorepart?

Looking at some of these suggestions, it might make sense to extend puzzle.GetPuzzleSubscoreNames to include the conditions which affect the score. Ideally, all subscores would add up to the current score. Some of the accounting might be complicated. The accounting might differ depending on whether there's a penalty or bonus.

Here are the conditions in effect for puzzle 1437, with a suggestion of how a segment's subscore might be calculated:

* Residue IE Score

This condition is local to a segment, and is a penalty only. A segment's subscore for this condition is just the penalty. (May be zero!)

* Core Existence

This condition applies a global penalty or bonus. The penalty could be averaged across all segments. The bonus could be averaged across the segments regarded as core. (Backdoor method for determining the core.)

* Secondary Structure

This condition usually applies a penalty if there are too many segments in helixes, and a bonus otherwise. The penalty or bonus could be averaged across all helix segments. (Alternately, substitute "sheet" for "helix" if we get another one like that.)

* Secondary Structure Design

This condition applies a penalty for certain residues depending on the secondary structure. There's a bonus if there are no penalties. The bonus could be averaged across all segments. The penalty could be averaged across the segments that are violating the condition.

* Residue Count

This condition penalizes added segments. The penalty could either be averaged across all segments or just across the "added" segments, based on segment number.

* Ideal Loops

This condition applies a penalty for each section of non-ideal loop. Normally, the penalty is 100 points. The penalty could be averaged across the segments in each section, corresponding to the red spheres revealed by the "show" option. (This is another backdoor method to get something shown in the UI.) So a bad loop with three red spheres would have a -33.333 subscore for the marked segments, four spheres would yield -25.

This method seems workable to me, having carefully considered it for the time it took to write it down.

Alternatively or additionally, we could have:

* a new function puzzle.GetPuzzleConditionNames to return the conditions.

* a new function puzzle.GetPuzzleConditionValue ( condname ) to return the global boolean or numeric value for the condition.

This approach doesn't give you the backdoor information that the segment-by-segment approach does, but it lends itself to include the boolean conditions like "solution was evolved" and "cutpoints need closing".

bertro's picture
User offline. Last seen 22 hours 28 min ago. Offline
Joined: 05/02/2011
Groups: Beta Folders

If you are also touching the UI?

- Make the Recipe Output window resizable
- Update the handling of the Recipe Output window text with string.format capabilities
- Save changes to options.txt as they are made, not just when the client closes. Right now when a crash occurs, any changes are lost.
- open the main menu in selection interface over all other objects
- add [B] to the Behavior main button (to harmonize with the other buttons)
- make the Score "window" moveable
- open recipe editor in new ScriptV2 mode instead of new GUI mode
- open the Segment Information window with full required height to show everything the first time it is open

Joined: 12/27/2012
Groups: Beta Folders

The function structure.IsLocked should return two booleans, similar to freeze.IsFrozen.

Currently, there's no way for a recipe to tell whether a sidechain is locked. That shows up on puzzle 1443, where the backbone is completely unlocked, but most of the sidechains are locked. On 1443, it seems like the only unlocked sidechains are the ones where structure.IsMutable is true, but that's not 100% clear.

Joined: 12/27/2012
Groups: Beta Folders

Working on an update to "print protein", I've found that looking at the rotamer count is one way to find unlocked sidechains of locked segments. (A rotamer count greater than one indicates an unlocked sidechain.)

The drawback is that rotamer.GetCount () is really slow. For puzzle 1443, which was 252 segments including the ligands, getting the amino acid or secondary structure for all segments takes less than a second. Getting the rotamer count for all segments takes around 39 seconds.

Puzzle 1443 was also an exception: it has locked sidechains, but unlocked backbone. I think we've seen this type of thing before, but locked backbone with unlocked sidechains is probably more common. For "print protein", I'd like to avoid getting rotamer counts unless needed, but puzzles like 1443 make this difficult.

Susume's picture
User offline. Last seen 4 hours 3 min ago. Offline
Joined: 10/02/2011

Either allow band.AddBetweenSegments to succeed when the segment numbers are the same and the atom numbers are different, or add function band.AddBetweenAtoms for placing a band between two atoms of a single segment.

This will be useful for bendable ligands.

Joined: 12/27/2012
Groups: Beta Folders

Two more candidates, in case you're running out of things to do. Actually, these would make a good start.

1. This humble function should be built-in, along with print() and help():

    function round ( ii )
        return ii - ii % 0.001
    end

2. Either fix recipe.GetRandomSeed or add a new function along the lines of:

    local seed = os.time()
    seed = 1 / seed
    while seed < 10000000 do 
        seed = seed * 10 
    end
    seed = seed - seed % 1
    print ( "Seed is: " .. seed )
    math.randomseed ( seed )
    math.random () 
    math.random ()

I just tested GetRandomSeed again, and it always returns 49 on the first call on Windows. Actually, the first 30 calls return the same value each time on Windows.

brow42's picture
User offline. Last seen 2 days 20 hours ago. Offline
Joined: 09/19/2011
Groups: None

Yes. There is literally no reason not to fix GetRandomSeed. It's not like the existing behavior is worth preserving.

Joined: 04/20/2012
Groups: Go Science

Does the function below round things the same way that
Foldit's Score Window rounds the Score? Does this function
round Scores correctly for both positive and negative Scores?

function round ( ii )
    return ii - ii % 0.001
end
Joined: 09/21/2011
Groups: Void Crushers

I am not that familiar with the software that I can easy find this. What you might do is getting some poses and just run a simple recipe containing s=current.GetEnergyScore() print s - s % 0.001
Then you can find out yourself.

Joined: 09/22/2017
Groups: None

Something like *.importSmiles("SmileString") for drug design puzzles

alcor29's picture
User offline. Last seen 10 hours 30 min ago. Offline
Joined: 11/16/2012

Something to freeze and unfreeze sheets, and something to freeze and unfreeze helices

brow42's picture
User offline. Last seen 2 days 20 hours ago. Offline
Joined: 09/19/2011
Groups: None

Jeff101's comment on the missing math.round (an eternal debate in lualand) reminded me belatedly, Gary Forbis made this long page of documentation and suggestions pre-Lua2: http://foldit.wikia.com/wiki/Lua_Script_Library

It would be a good idea to go through that for ideas that haven't already been covered, particularly the fsl and gary namespaces. I myself put atom tables into fsl because of his suggestions.

Joined: 04/20/2012
Groups: Go Science

In some puzzles (like the aflatoxin puzzles 1440 1445 1450 & 1455),
certain residues can be mutated while others cannot. Is there a
LUA command that lets players choose additional residues to shield
from mutation? Does freezing a residue prevent it from being mutated?

It looks like Foldit already has similar LUA commands:

boolean structure.IsLocked(indx)
boolean structure.IsMutable(indx)
boolean structure.CanMutate(indx,str)

but I think these only tell us what restrictions
Foldit Central has placed into a puzzle, not what
restrictions a player has made.

Perhaps you could make commands like below:

void structure.SetMutable(indx,boolean)
boolean structure.GetMutable(indx)

where SetMutable makes residue indx mutable if true and
non-mutable if false, but SetMutable cannot make a
residue mutable if IsMutable for that residue is false.
Meanwhile, GetMutable returns for residue indx the
boolean (true or false value) set by the player for
that residue. The default values for GetMutable could
be what IsMutable would return.

It would be helpful if the settings for GetMutable
would get saved, shared, and loaded with a structure,
like bands, secondary structure, sequence, & frozen
already get saved.

Thanks!

Joined: 04/20/2012
Groups: Go Science

Can you make a LUA function equivalent to Save Screenshot (Ctrl+H)
that will save a png image of your protein to the desktop?
Perhaps only let it save to a few different png filenames
so that it won't fill up a player's hard drive by accident.

This LUA function could be like below:

void SaveScreenshot(integer)

where integer is a number from 0 to 9 that becomes part
of the png filename (this would let up to 10 different
screenshots be saved to the desktop).

One use for this function could be as follows:
Say you want to use a band to make a hydrogen bond form,
but you don't want to keep stopping a recipe to check on
the hydrogen bond's status. Within the recipe, you could
use LUA to save the structure, remove all of its bands,
post a snapshot of the structure without the bands,
restore the saved structure, post another snapshot of
the structure with the bands, and then continue with
the recipe. Periodically, the recipe could make new
snapshots, reusing the same 2 filenames each time for
the snapshots with and without the bands. Then, as the
recipe runs, the player can open these 2 image files
to examine the latest status of the hydrogen bonds.
The player could also manually copy these 2 image files
for archival purposes, choosing new names for each set
of copied images.

Thanks again!

Joined: 04/20/2012
Groups: Go Science

It might be better to save the screenshots to
the same directory where scriptlog.default.xml
resides. This way if you have Foldit clients in
different folders on the same machine, they won't
overwrite each other's screenshots.

It might also be good to allow the integer in:

void SaveScreenshot(integer)

to be 0 to a number much larger than 9.

If the integer is 0, SaveScreenshot can pick
its own name for the output png file. This
could be a unique name based on the date and
time the image is created, a name a bit like
those used for manually saved and shared puzzle
solution *.ir_solution files.

If the integer is >0, the screenshot
gets saved to a standard filename like
screenshot00001.png.

Having SaveScreenshot like above would let
players adapt recipes like GAB and DRW to
automatically make snapshots every time
the score rises by a certain amount, and
these snapshots could be viewed like a
slideshow to follow the moves taken by
the recipe to raise the protein's score.

It would help if SaveScreenshot would work
for minimized clients as well.

Joined: 04/20/2012
Groups: Go Science

If there were a LUA command equivalent
to pressing the Home key, that would be
helpful too. It could be used with the
SaveScreenshot command above to re-center
and rescale the protein image before
automatically taking a snapshot of the
protein.

Joined: 12/27/2012
Groups: Beta Folders

Expose read-only tables of structure info.

The existing functions like structure.GetSecondaryStructure are slow relative to retrieving the same information from a Lua table. As an example, reading the secondary structure from a table 50,000 times for a 75-segment protein took 8 seconds. Calling the Foldit function instead took 37 seconds. (Not exactly bad, but much slower than using tables.)

Nothing is stopping us from making our own tables. The sample code below will probably go into "print protein 2.8". The Foldit core code could create these tables and update them as needed.

Conversion to tables is simple. A recipe calls protNfo.setNfo early on.

Later in the recipe, the code

local ss = structure.GetSecondaryStructure ( seg )

becomes

local ss = protNfo [ seg ]

The example below doesn't include selects, freezes, and some other possible items of interest. The rotamer.GetCount function is slow, so I've left it out here. The print protein version will include it, since it's used to detect locked sidechains (until someone writes a function for that).

Writers of long-running recipes might want to consider this code as well.

protNfo = {
    aa = {},
    ss = {},
    atom = {},
    phobe = {},
    lock = {},
    
    setNfo = function ()
        for ii = 1, structure.GetCount () do
            protNfo.aa [ #protNfo.aa + 1 ] = structure.GetAminoAcid ( ii )
            protNfo.ss [ #protNfo.ss + 1 ] = structure.GetSecondaryStructure ( ii )
            protNfo.atom [ #protNfo.atom + 1 ] = structure.GetAtomCount ( ii )
            protNfo.phobe [ #protNfo.phobe + 1 ] = structure.IsHydrophobic ( ii )
            protNfo.lock [ #protNfo.lock + 1 ] = structure.IsLocked ( ii )
        end
    end,
}
Sitemap

Developed by: UW Center for Game Science, UW Institute for Protein Design, Northeastern University, Vanderbilt University Meiler Lab, UC Davis
Supported by: DARPA, NSF, NIH, HHMI, Microsoft, Adobe, RosettaCommons