34 replies [Last post]
Tlaloc's picture
User offline. Last seen 15 weeks 5 days ago. Offline
Joined: 08/04/2008
Groups: Mojo Risin'

In the message when foldit starts, it says that the development team is working on lua features. What can we be expecting?

If I had a preference, the first thing I'd want is the ability to easily use an external editor on lua scripts. You guys need to get out of the editor business, because you will never have the resources to do it right. The lua scripts would be in a particular directory on the hard drive with a .lua extension. I should press a button in the UI and edit the script which would fire up whatever was configured as the default editor for .lua files, either in the operating system or a field in the options dialog. When a change was made to the file and saved, the next invocation of the script would use the change.

Next, I want a reimplementation of the get_aa() function. My scripts need this right now.

The next thing would be addition of some of the lua standard library. I started a wiki page http://foldit.wikia.com/wiki/Lua_Standard_Libraries talking about what features I'd like from the library. I have been going over the lua documentation and trying to find which features would be safe to implement, without turning foldit into a virus engine. This page is a work in progress as I get a better understanding of the lua features. The package features probably do need to be implemented, but restricted in some way.

The next thing would be ability to have scripts call other scripts, talked about in the feedback I started here: https://fold.it/portal/node/987800

Then some of the other functions talked about on the wiki page I created here: http://foldit.wikia.com/wiki/Lua_Functions_That_Should_Be_Implemented

This would make working with scripting much more satisfying and powerful, resulting in better scores by the players. I now have a reasonable understanding of lua as a programming language, and I have been surprised at how much I like it considering that I generally hate loosely typed languages. It's small, powerful, and elegant, but many of the power comes from functions in the standard libraries that we don't currently have access to.

Finally, we need some better documentation of the foldit lua functions that are available. Like I still can't figure out how the do_local_rebuild() is supposed to work, and I don't know what the various first argument to get_segment_score_part mean (e.g. what does 'Reference' and 'Other' mean?).

Players: Do you think these are the right priorities, and what other features do you want?

Tlaloc's picture
User offline. Last seen 15 weeks 5 days ago. Offline
Joined: 08/04/2008
Groups: Mojo Risin'
I missed one

At the same level as the get_aa() function (i.e. second on my list), I want a function that returns the score of the save in the 'Save Recent Best' slot. Something like get_recent_best_score().

Joined: 03/20/2010
Two Big Thumbs-UP

External Editor Option: - DEFINITELY!!!
Standard LUA Libraries: - esp math and table object - YES please.
Meta-Scripts: - I've been wanting this for a long while - Great Idea!
"LUA Functions that should be implemented": An amazingly complete compilation of most everything I've seen discussed in chat at various times, plus some great thoughts that I've never yet heard mention of. A "must-read" article for all the developers!
...and finally, "Better Documentation." I stand and applaud for this one!

Will add more ideas to this thread as they occur to me.

Hat's off to you Tlaloc, for so competently and thoroughly initiating this much needed and long overdue discussion. Hope everyone joins in upping this thread

Joined: 09/18/2009
Groups: SETI.Germany

I second what dimension9 said.

Tlaloc, good idea with get_recent_best_score().
This will shrink the script-codes a bit, as we normally have to store the scores "manually".
Maybe a similar function for the save-slots would also help.

Joined: 01/16/2010
Groups: None
These are great ideas

Thank you Tlaloc, the 2 wiki pages are great.

If the editor function is too much work for now, how about upload and download script script buttons. Even that would be a help.

I would also like the ability to have more than one recent best. So I could set a recent best at the start of script that would be a stand-in for the global best and then at least one more that i can use in the normal way. This would help me explore different structures with the scripts.

Joined: 09/18/2009
Groups: SETI.Germany
recent best

@Cartoon Villain:
Maybe I've got you wrong, but there are already 9 quicksave slots, which behave recent best.
Most scripts use only the first ones, so if you use number 9, you will be on the safe side.

Tlaloc's picture
User offline. Last seen 15 weeks 5 days ago. Offline
Joined: 08/04/2008
Groups: Mojo Risin'
10 quicksave slots

There are actually 10 quicksave slots. The first three can be manipulated from the UI.

It is less important to be able to retrieve the score of a quicksave slot, since you have to write the code to store something in the quicksave, you can store the score just before you do into a variable.

But with the Recent Best, if you do a wiggle or a rebuild, foldit fills the recent best automatically. If you do a wiggle function, the score may improve, which causes a recent best to be stored, then drop again before the function returns. So there is currently no way to reliably know what the score is in the recent best without storing the current position to a quicksave slot, retrieving the recent best, getting the score, then restoring the quicksave. This is slow and causes the UI to flash.

I'm sure that foldit has that score being stored somewhere for their internal use, so they just need to give us a function to retrieve it. I'm sure it would be just a few lines of code for them to implement it, and very useful. As Crashguard says, this will reduce the size of our scripts.

Joined: 09/18/2009
Groups: SETI.Germany

"But with the Recent Best, if you do a wiggle or a rebuild, foldit fills the recent best automatically. If you do a wiggle function, the score may improve, which causes a recent best to be stored, then drop again before the function returns."

You mean VERY best, not recent best, or?

"So there is currently no way to reliably know what the score is in the recent best without storing the current position to a quicksave slot, retrieving the recent best, getting the score, then restoring the quicksave. This is slow and causes the UI to flash."

Yes, using score watching, my scripts also cause this behaviour.
This is why I asked for a score threshold which ends shake/wiggle automatically by the "game engine".

Joined: 09/18/2009
Groups: SETI.Germany
I think...

...the very first thing to change is making the get_score_command() returning scores below 0, so recipes can easily be run after threading or rebuild, without this slow pseudo-sum thingy:

I don't know how the variables are handled in the game code, but it shouldn't be a big thing to make in contrast to other features, and in second, make the get_aa() function work again.

Joined: 01/16/2010
Groups: None
About the recent best

Tlaloc's analysis is dead on.

Is the multiple recent best the most important function to implement? No, but it should be trivial to implement and it would help, at least me, a fair bit. Right now my best scripts can only really work from the absolute best, I want to expand on that.

Priorities for me:
1) Editor
2) Math functions
3) freeze functions
4) others listed by Tlaloc et al (some where in there the multiple recent best)

Joined: 06/17/2010
Small but usefull thingy -

Small but usefull thingy - make bands DIRECTLY to hydros, not only to segments.

Joined: 09/18/2009
Groups: SETI.Germany

You mean, applying bands to bonds to maintain them?

Joined: 09/18/2009
Groups: SETI.Germany

Interesting thing how to band into free air (angle rho etc.), I like it!

How about the idea to just fix a segment at its current position, if banding segment parameters of band_add_segment_segment(x,x) are equal?

Joined: 09/18/2009
Groups: SETI.Germany

Now, with get_score(true) we can also get negative scores, and get_aa() works again.
Should we note this here?

Tlaloc's picture
User offline. Last seen 15 weeks 5 days ago. Offline
Joined: 08/04/2008
Groups: Mojo Risin'
wiki updated

I updated the wiki to show get_score and get_aa have been completed. Two down, about 30 left on the list.

My highest priority next function to implement would be get_recent_best_score(). This would significantly speed up and simplify my scripts.

Joined: 09/18/2009
Groups: SETI.Germany

...and save some lines of code.

How about getting all scores of different states by script, also these from quicksave slots and very best?

Joined: 05/09/2008
Groups: Void Crushers
I would like to see lua

I would like to see lua functions that return the selection number and start and end segments numbers when using the new interface. Something like:

get_selections_count() returns number of selections.
get_selection_start(selection number) returns start of selection supplied
get_selection_end(selection number) returns end of selection supplied
get_selections() returns all selections in an array (selection number, selection start, selection end)
is_selected(seg) return true if segment selected

Its all in this feedback thread. http://fold.it/portal/node/987849

Joined: 09/18/2009
Groups: SETI.Germany

How about the possibility of alocal handling for quickload/-save slots or recent best?
Doing it this way, savestates could be handled chunk-/blockwise, so that they never collide.

This would make including other subroutines which also use quickload and save very easy, but I think it will be a big change for the game engine.

Joined: 09/18/2009
Groups: SETI.Germany

Some Programming languages use "randomize" timer.
They take the system time as seed value for pseudo-random algorithms.

If we could call the daytime elapsed seconds with decimal digits (like in VB), we would have an excellent seeding result.

At the moment, starting at the same puzzle state will always give the same results when using pseudo-random numbers.
To get other results, the seed value has to be tweaked manually.

Joined: 09/18/2009
Groups: SETI.Germany
puzzle properties

As posted here

we need to know by script if a puzzle is mutable (even if there is only one segment which can be mutated)
and we need to know by script if this is a ligand puzzle, so the script won't try to mutate the ligand or do some other fatal actions.

Joined: 09/18/2009
Groups: SETI.Germany
Randomize timer again

As I've learned, the standard lua library also has a function to get the system time, called os.time()

So, a vote again for the standard Lua library.

Tlaloc wrote sometime before, that some library functions should be disbled because they could do something bad.

To my opinion, it would do, if all functions according to file writing access (which a virus needs) would be disabled.
All other functions wouldn't do any harm, only crashing the system at worst case.

Joined: 08/06/2010
Groups: Contenders
I was getting ready to do

I was getting ready to do some script work.
A missing feature is to open a dialog box with arbirary controls
such as buttons and sliders. I was thinking a simple vb program
could do the trick using two files for communication between them.
Since it appears one can't create true process threads one could
just have the called thread be the communicator to the external
process that handled the dialog box.

While I could hijack the files fold it writes I'd have to learn
the verification algorithm that generates a number like:

verify: xxxxxxxxxxxxxxxxxxxxxxxxxxxx

Tlaloc's concern about virii makes sense. Restricting IO to
file names generated by foldit would be a good compromise.
This would allow fold it to restrict io to the ProgramData/foldit

Tlaloc's picture
User offline. Last seen 15 weeks 5 days ago. Offline
Joined: 08/04/2008
Groups: Mojo Risin'
Updated Lua Library list

I updated the list of standard Lua library functions in the wiki (http://foldit.wikia.com/wiki/Lua_Standard_Libraries) to show the ones that are available in World of Warcraft (WOW). WOW also uses Lua as its scripting language. Since WOW is a much larger potential potential for being a virus vector, I think you can assume that if they allow a function it is safe.

I'm still investigating what is the right way to allow me to share a library of code across scripts a set of scripts.

Joined: 09/18/2009
Groups: SETI.Germany
Good idea

Nice work, now the developers of FoldIt will have an idea which functions seem to be okay to be included.

Tlaloc's picture
User offline. Last seen 15 weeks 5 days ago. Offline
Joined: 08/04/2008
Groups: Mojo Risin'
Now that CASP is over

Any word from the foldit team on when we can expect to see some new Lua features? I keep finding new things to add to the list on the wiki as I have new ideas for scripts that I can't write.

Joined: 09/18/2009
Groups: SETI.Germany

I'm curious, too.
A simple but effective change would be allowing a higher maxmimum bandwidth, this would make a big difference even for banding recipes which already exist.

Joined: 06/06/2008
Groups: None

I hate to point to a dead link, but http://thewarwiki.com/wiki/WAR_API (defunct) used to point to the Warhammer Online lua api information. As a dig on google found nothing, best I can say is it had all the functions of WoW, as well as a few other things, in a slightly different implementation, different names for functions etc...

Regardless, it had the string, math, and tables libraries implemented.

Its sad that the feedbacks that would be the most benefit to scripting have been downvoted SO much by the people shouting everything should be done by hand. I mean really... I got the feedback for those 3 libraries to get several upvotes today, and others came in and downvoted it. To implement them is a few hour (if things go wrong, which they always do) task.

I really wish they would open the source up. Sure, its sort of available to players, who, if they aren't a research institute can acquire for the low low sum of the price of a luxury car. Even then, all indications are its not the client. If source were opened, and a git or even regulated svn system were added, people could write patches, and take a large load off the devs, and help bring all these features to the client.

beta_helix's picture
User offline. Last seen 11 hours 47 min ago. Offline
Joined: 05/09/2008
Groups: None
FWishbringer, please be patient

We are aware of how important this is for many Foldit players, but we just currently do not have the resources to work on this immediately.

As for the Open Source, we have set up everything for non-academics to be able to download the source files but it is currently stuck in the bureaucracy stage. I really hope that this will be completed soon.

Thank you for your patience.

Joined: 06/06/2008
Groups: None
I'm generally not a patient

I'm generally not a patient person, beta_helix. I admit it. I know its a fault.

The 'big red flag' I see in your statement is 'non-academics to download the source'. Downloading the source is one thing. Look at SecondLife, which was a closed source turned open source. It wasn't until it started _ACCEPTING_ source it really began to benefit.

My notes on using git or a supervised svn weren't so much for the download-to-build as it was for the submitting patches. There's thousands upon thousands of hours of volunteer development hours waiting to be expended to help improve the game. Most of those willing are likely more skilled than I. I'd love to see them given the chance to really push fold.it forward. An open system to support user-made patches would be the best way to forward it.

Right now, its probably a small group of students who work in the lab, getting a small salary and/or some tuition as well as a few professors who oversee and throw in the occasional patch working on the source. All of them cost X amount of dollars for Y work put in.

Open source means the university pays nothing to improve it, but the results, and thereby the science improves.

Sigh... I should stop now. I've said my bit. Opening would allow lua to be improved easier, so this is all relevant. I'm sure you can see the benefits. I wish others would see it as quickly and decisively so as to allow it.

Brick's picture
User offline. Last seen 2 years 25 weeks ago. Offline
Joined: 07/15/2008
Groups: Beta Folders
Open source means the project

Open source means the project people would still have to be involved with analyzing and approving every change submitted to maintain the scientific integrity of the results. Remember, even though this is advertised as a "game", this is serious scientific research, and needs to be controlled as such.

Also, since this project awards points making this a competition, opening the source for changes will attract the usual crop of people who want to bypass the security of the game and submit fraudulent results to "game" the scoreboards. Don't say it doesn't happen - SETI and the other DC projects have all had to fight those battles.

I'm happier having this tightly controlled.

I'm also suspicious that this frantic push to add more functionality to lua for more powerful scripts is all about making the few script writers all-powerful, leaving the rest of the 99.9% of the users unable to compete. that caution could be alleviated by making ALL scripts public - none of these private personal or team-only scripts.

Joined: 09/02/2010
Groups: None
Yes, I do fully agree with

Yes, I do fully agree with your post. Foremost I would like to be able to use the mathematical functions. Second on my list would be a good import possibility, packaging would even be better.
If import of packaging doesn't come, then please a decent cutting and pasting mechanism in the Foldit editor.

fishercat's picture
User offline. Last seen 3 days 8 hours ago. Offline
Joined: 08/05/2010
Allow LUA extensions to PHP and Javascript

This would be the easiest way to open up FoldIt to the masses.

Joined: 06/17/2010

PHP and Java in foldit?? For what reason?
All "external" things we need is few standard libraries and plain text files access.
Internals are well listed by Tlaloc (cant wait for bands over 20...).

fishercat's picture
User offline. Last seen 3 days 8 hours ago. Offline
Joined: 08/05/2010
It would allow all sorts of improvements

Being able to use external scripting languages could allow you to a lot off things. Least of which would be abilities to create interface tweaks. Imagine a choice of Recipe managers or being able to have a choice of editors to create recipes instead of the native kludge editor. With php and javascript (not Java, close but sorta different) you would have file system access to create recipes that save settings and data betweeen instances.

Best of all you could real AI programs like GAs and Neural Nets that could apply from puzzle to puzzle.

The game becomes a whole different entity that way. It changes from a project on how people use foldit to how foldit itself evolves.

Joined: 06/17/2010

1. select/deselet single segment
void select_index(int sn)
void select_index_range(int ss,int se)
void deselect_index(int sn)
void deselect_index_range(int ss,int se)

2. read all selected segments and/or test that segment is selected
table get_selected() {bool,bool,bool...}
bool is_selected(int sn)

3. freeze/unfreeze segement/range
void do_freeze(int sn, bool backbone, bool sidechain)
void do_freeze_range(int ss, int se, bool backbone, bool sidechain)
void do_unfreeze(int sn, bool backbone, bool sidechain)
void do_unfreeze_range(int ss, int se, bool backbone, bool sidechain)

4. read freeze state
bool is_frozen(bool backbone, bool sidechain)
table get_frozen() {{bool bb, bool ss},{...},...}

5. read data form band
bool is_enabled(int bn)
float get_band_str(int bn)
float get_band_ln(int bn)
table get_band_ends(int bn) {ss,se}

6. mass band change
void bands_set_str(float str)
void bands_set_len(float len)

7. slotted load/save bands set - different slots than "normal" saves!
void save_bands(int slotNo)
void load_bands(int slotNo)


Developed by: UW Center for Game Science, UW Institute for Protein Design, Northeastern University, Vanderbilt University Meiler Lab, UC Davis
Supported by: DARPA, NSF, NIH, HHMI, Amazon, Microsoft, Adobe, Boehringer Ingelheim, RosettaCommons