Trouble with symmetry puzzles

Case number:845829-2011400
Topic:Crash/Hang
Opened by:dcrwheeler
Status:Open
Type:Bug
Opened on:Friday, March 26, 2021 - 03:36
Last modified:Thursday, April 15, 2021 - 06:08

Is anyone having trouble with symmetry puzzles? Over the last few months running recipes
have become more and more difficult to keep going due to crashes.
When there was some talk of problems with symmetry puzzles, I didn't have any trouble
then.

First it was only the remix recipes. For a while I just ran it with one segment length
it would go a few rounds then crash. Now nothing runs now without crashing within a minute.
Even cut and wiggle and BandedWormPairs. Bands or remix. Cut and wiggle completes one time
and on the second after inserting cuts the instant wiggle starts...crash.
Some of the linker puzzles also had trouble also with remix.

The only hint is using a recipe away from the other sub-units works. Once it's close
together, nothing runs except Mutate no wiggle. It will do 6 segments at a time but no more.
Various view options didn't change anything. Nor did changing wiggle power.

Graph properties I have use 100% memory, 100 undo
The computer has the power
Windows 10
10 core w/two threads each
64 gigabyte RAM.
liquid cooled

(Fri, 03/26/2021 - 03:36  |  11 comments)


jeff101's picture
User offline. Last seen 12 hours 34 min ago. Offline
Joined: 04/20/2012
Groups: Go Science

I have been playing with Puzzle 1969, a trimer 
puzzle, the past few days. It crashed many times 
this weekend using a fairly up-to-date client (I 
even updated the client a few times and tried both
devprev and main for it). Meanwhile, on an older 
client, Foldit didn't crash much at all. I ended
up switching from the crashing new client to a
second old client, and the crashing seems to have 
stopped. I have some log.txt files from my crashes 
(please DM me if you need them). 

Below lists the clients that have behaved well:

10b has:
{
cmp-bin-49c6f7b81f... 01/05/2021 511pm
cmp-dat-fb137240c5... 11/16/2020 840am
cmp-res-904de2c111... 11/16/2020 841am
options.txt 3/28/2021 224am:
 "update_group" : "devprev"
}

10c has:
{
cmp-bin-49c6f7b81f... 01/06/2021 443am
cmp-dat-fb137240c5... 11/10/2020 914pm
cmp-res-904de2c111... 11/14/2020 613pm
options.txt 2/20/2021 822am:
 "update_group" : "devprev"
}

Joined: 05/03/2009
Groups: Contenders

Yes, I'd like to weigh in on this one too. I'm running the 20210324-2a578da660-win_x86-devprev, and have severe crashes on 1971, trying to run DRemix recipes and sidechain flippers. I had 3 positions to explore and opened 3 clients to DRemix all of them, and my first client crashed before I finished setting up the third. I tried to set a sidechain flipper on the best of them, and it lasted 10 minutes.

For context? I had 6 seperate clients running 1968 on this build for a week with no issue.

frood66's picture
User offline. Last seen 17 hours 29 min ago. Offline
Joined: 09/20/2011
Groups: Marvin's bunch

a real simple question...is the game restricting access so I/O is slowed?

Joined: 12/06/2008
Groups: Contenders

Trying to evolve a teammate's solution. A simple pull-and-wiggle recipe like Banded Worm crashes the puzzle within minutes of starting the script. Has happened multiple times. I'm using the standard "Main" version of the game.

dcrwheeler's picture
User offline. Last seen 10 hours 22 min ago. Offline
Joined: 09/05/2013
Groups: None

Symmetry puzzles are still nearly unplayable.

There have no changes to my system of any kind.

Build ID 20210312-02b96e625d-win_x86

I've been using just one directory since I started playing. I
created a separate directory to get a clean log file.

This is from a very short run of Cut and Wiggle (less than 2 minutes)
This is the last portion of the log file. no debug file

game.util.IRC: Exiting IRC::run
interactive.rosetta_util.rosetta_util: Resymmetrized pose
interactive.application.actions.cart.PoseLoopThreadActionCart: ***** ENDING THREAD ActionGlobalMinimize
interactive.application.shared.Tool: Time: Sat Apr 3 00:11:24 2021 UTC - Pull Mode completed
game.application.boinc.Boinc: Sending SOPs:
interactive.application.actions.cart.PoseLoopThreadActionCart: ***** STARTING THREAD ActionGlobalMinimize
interactive.application.shared.tool_util: RT: 1111111111111111111111111111111111111111111111111111111111111
game.util.IRC: Entering IRC::run
interactive.rosetta_util.rosetta_util: Resymmetrized pose
interactive.application.actions.cart.PoseLoopThreadActionCart: ***** ENDING THREAD ActionGlobalMinimize
interactive.application.shared.Tool: Time: Sat Apr 3 00:11:34 2021 UTC - Pull Mode completed
interactive.rosetta_util.rosetta_util: Resymmetrized pose
interactive.application.shared.Tool: Time: Sat Apr 3 00:11:34 2021 UTC - Pull Mode completed
interactive.application.shared.Tool: Time: Sat Apr 3 00:11:34 2021 UTC - Pull Mode completed
interactive.application.actions.cart.PoseLoopThreadActionCart: ***** STARTING THREAD ActionRepack
interactive.application.shared.tool_util: RT: 1111111111111111111111111111111111111111111111111111111111111
game.util.IRC: IRC::run error, code = 5, desc = Remote connection closed
game.util.IRC: Exiting IRC::run
game.application.BoincThread: SRVR_THRD getting notifications...
game.util.IRC: Entering IRC::run
interactive.rosetta_util.rosetta_util: Resymmetrized pose
interactive.application.actions.cart.PoseLoopThreadActionCart: ***** ENDING THREAD ActionRepack
interactive.application.shared.Tool: Time: Sat Apr 3 00:11:45 2021 UTC - Pull Mode completed
interactive.rosetta_util.rosetta_util: Resymmetrized pose
interactive.application.shared.Tool: Time: Sat Apr 3 00:11:45 2021 UTC - Pull Mode completed
interactive.application.shared.Tool: Time: Sat Apr 3 00:11:45 2021 UTC - Pull Mode completed
interactive.rosetta_util.rosetta_util: Resymmetrized pose
interactive.application.shared.Tool: Time: Sat Apr 3 00:11:47 2021 UTC - Pull Mode completed
interactive.rosetta_util.rosetta_util: Resymmetrized pose
interactive.application.shared.Tool: Time: Sat Apr 3 00:11:48 2021 UTC - Pull Mode completed
interactive.rosetta_util.rosetta_util: Resymmetrized pose
interactive.application.shared.Tool: Time: Sat Apr 3 00:11:49 2021 UTC - Pull Mode completed
interactive.rosetta_util.rosetta_util: Resymmetrized pose
interactive.application.shared.Tool: Time: Sat Apr 3 00:11:50 2021 UTC - Pull Mode completed
interactive.rosetta_util.rosetta_util: Resymmetrized pose
interactive.application.shared.Tool: Time: Sat Apr 3 00:11:52 2021 UTC - Pull Mode completed
game.application.boinc.Boinc: Sending SOPs:
interactive.rosetta_util.rosetta_util: Resymmetrized pose
interactive.application.shared.Tool: Time: Sat Apr 3 00:11:53 2021 UTC - Pull Mode completed
interactive.rosetta_util.rosetta_util: Resymmetrized pose
interactive.application.shared.Tool: Time: Sat Apr 3 00:11:54 2021 UTC - Pull Mode completed
game.util.IRC: IRC::run error, code = 5, desc = Remote connection closed
game.util.IRC: Exiting IRC::run
interactive.rosetta_util.rosetta_util: Resymmetrized pose
interactive.application.shared.Tool: Time: Sat Apr 3 00:11:55 2021 UTC - Pull Mode completed
interactive.rosetta_util.rosetta_util: Resymmetrized pose
interactive.application.shared.Tool: Time: Sat Apr 3 00:11:56 2021 UTC - Pull Mode completed
interactive.rosetta_util.rosetta_util: Resymmetrized pose
interactive.application.shared.Tool: Time: Sat Apr 3 00:11:58 2021 UTC - Pull Mode completed
interactive.rosetta_util.rosetta_util: Resymmetrized pose
interactive.application.shared.Tool: Time: Sat Apr 3 00:11:59 2021 UTC - Pull Mode completed
interactive.rosetta_util.rosetta_util: Resymmetrized pose
interactive.application.shared.Tool: Time: Sat Apr 3 00:12:00 2021 UTC - Pull Mode completed
interactive.rosetta_util.rosetta_util: Resymmetrized pose
interactive.application.shared.Tool: Time: Sat Apr 3 00:12:01 2021 UTC - Pull Mode completed
interactive.rosetta_util.rosetta_util: Resymmetrized pose
interactive.application.shared.Tool: Time: Sat Apr 3 00:12:03 2021 UTC - Pull Mode completed
interactive.rosetta_util.rosetta_util: Resymmetrized pose
interactive.application.shared.Tool: Time: Sat Apr 3 00:12:04 2021 UTC - Pull Mode completed
interactive.rosetta_util.rosetta_util: Resymmetrized pose
interactive.application.shared.Tool: Time: Sat Apr 3 00:12:05 2021 UTC - Pull Mode completed
interactive.rosetta_util.rosetta_util: Resymmetrized pose
interactive.application.shared.Tool: Time: Sat Apr 3 00:12:06 2021 UTC - Pull Mode completed
interactive.rosetta_util.rosetta_util: Resymmetrized pose
interactive.application.shared.Tool: Time: Sat Apr 3 00:12:08 2021 UTC - Pull Mode completed
interactive.rosetta_util.rosetta_util: Resymmetrized pose
interactive.application.shared.Tool: Time: Sat Apr 3 00:12:09 2021 UTC - Pull Mode completed
interactive.rosetta_util.rosetta_util: Resymmetrized pose
interactive.application.shared.Tool: Time: Sat Apr 3 00:12:10 2021 UTC - Pull Mode completed
interactive.application.actions.cart.PoseLoopThreadActionCart: ***** STARTING THREAD ActionGlobalMinimize
interactive.application.shared.tool_util: RT: 1111111111111111111111111111111111111111111111111111111111111
interactive.rosetta_util.rosetta_util: Resymmetrized pose
interactive.application.actions.cart.PoseLoopThreadActionCart: ***** ENDING THREAD ActionGlobalMinimize
interactive.application.shared.Tool: Time: Sat Apr 3 00:12:16 2021 UTC - Pull Mode completed
interactive.application.shared.Tool: Time: Sat Apr 3 00:12:16 2021 UTC - Pull Mode completed
interactive.application.actions.cart.PoseLoopThreadActionCart: ***** STARTING THREAD ActionGlobalMinimize
interactive.application.shared.tool_util: RT: 1111111111111111111111111111111111111111111111111111111111111

UNHANDLED EXCEPTION
[ ERROR ] Issue initializing symbols with path `C:\Foldit test\cmp-binary-b564df3adca3045417c72d34e986c740;.`
[ ERROR ] Could not initialize backtrace symbols: ERR 8 Message:

Joined: 03/28/2020
Groups: Go Science

I am having similar problems. I can only live with them because I have my autohotkey watchdog running which automatically restarts a Foldi-client once it has crashed. This has come in VERY useful the last months.
To me this problem seems to be either filter-related or memory-related. I have been using "Process Lasso" to monitor nemory usage and there I can see that this usage goes up over time when a recipe is running. Once a Foldit client reaches about 1.7GB it typically crashes. Could just be coincidence but I will continue to look at this further.
Also it seems to me that Foldit does not seem to like it that filters are continuously switched on and off. But I am not sure there.
Also I had instances where it crashed when certain HBond configurations were broken or created. But this was hard to reproduce.
I would be very happy if someone would find a way to make Foldit more stable.

Greetings, iwdn

Joined: 03/28/2020
Groups: Go Science

It seems like this is not related to the RAM-occupation of the Foldit-process. I used Process Lasso to trim the Foldit-RAM-occupation every few minutes. It did not go above 800MB or so anymore but it still crashed as frequently as before.
It also does not seem to be filter-related. I had all prominent filters off and it still crashed as frequent as before.
It does though seem to be related to wiggle or bonding/de-bonding scenarios. I had instances where I just issued a global wiggle by hand and it immediately crashed (I assume because bonds were formed/broken). I have seen this multiple times.
Some days back I had a puzzle in which I could reproduce this. Unfortunately I cannot find it again. But there was a HB-bridge between 2 serines. If then when marking one of them and pressing left/right to flip through rotamers (which lead to breaking the bond between them) it crashed instantly and reliably. So it seems to me to be a bonding issue.
I can confirm what dcrwheeler wrote. All log.txt-files of the crashed clients look basically the same:

interactive.rosetta_util.rosetta_util: Resymmetrized pose
interactive.application.actions.cart.PoseLoopThreadActionCart: ***** STARTING THREAD ActionRepack
interactive.application.shared.tool_util: RT: 111111111111111111111111111111111111111111111111111111111111111111111111111111111
game.application.boinc.Boinc: Sending SOPs:
interactive.rosetta_util.rosetta_util: Resymmetrized pose
interactive.application.actions.cart.PoseLoopThreadActionCart: ***** ENDING THREAD ActionRepack
interactive.application.actions.cart.PoseLoopThreadActionCart: ***** STARTING THREAD ActionGlobalMinimizeSidechains
interactive.application.shared.tool_util: RT: 111111111111111111111111111111111111111111111111111111111111111111111111111111111
game.util.IRC: Entering IRC::run

UNHANDLED EXCEPTION
[ ERROR ] Issue initializing symbols with path `C:\Foldit11\cmp-binary-b564df3adca3045417c72d34e986c740;.`
[ ERROR ] Could not initialize backtrace symbols: ERR 8 Message:

They all end with this same unhandled exception. And this crashing does indeed seem to me like an unhandled exception because Foldit crashes at an instant and the process still lingers in the system with very low RAM-usage. So it seems just to "internally break down".
What does this error message in the log.txt mean? It would be interesting to know because it might lead to the origin of this error. Looks to me like some symbol-resource or so is missing? This error message must be issued from somewhere within foldit and it should be possible to trace it down and fix it (I sincerely hope).

Pleeeease try to fix it. It would for sure dramatically enhance game-quality!

Greets, iwdn

bkoep's picture
User offline. Last seen 8 hours 6 min ago. Offline
Joined: 11/15/2012
Groups: Foldit Staff

The greatest roadblock for fixing these crashes is that we are unable to reproduce them. If you can dig up the previous solution that crashed reliably, that would be very helpful.

If you ever encounter a reproducible crash in the future, please let us know and share the steps to reproduce the crash!

Joined: 03/28/2020
Groups: Go Science

Unfortunately it was so long ago that I cannot find it again. But I know that it was a serine-serine-bridge between 2 adjacent strains of a 3-Helix bundle. Tried to re-model it but could not crash Foldit in today's version. Maybe this was fixed in the meantime? But back then I generally broke all bonds between adjacent helices so that they would have more freedom in subsequent global wiggling. I got immediate crashes frequently back then with manual rotamer-switching with left/right key.

Ok. I will try to work on crashing Foldit deliberately :) If that would help then it is worth trying and investigating this further.

Joined: 03/28/2020
Groups: Go Science

I have one instance now where it crashes immediately.
The most crashing-on-me recipe is "Deep Remix Wiggle" (drmxw). I was able to isolate the commands that cause it to crash. You will have to run this:

function ReMix(ss,se,nrresults)
selection.DeselectAll()
selection.SelectRange(ss,se)
local nfound=structure.RemixSelected(30,nrresults)
--local nfound = 30
print("Nr of remix found "..nfound)
print("Loading remixes...")
for i=0,nfound-1 do
save.Quickload(30+i)
print("Loaded: " .. 30+i)
--structure.ShakeSidechainsSelected(1)
end
end

function main()
local Remixes = 30
print("Finding " .. Remixes .. " remixes...")
ReMix(56,58,Remixes)
end

main()

from within a recipe (I simply named it test but it probably does not matter) on the "crashtest" I shared with scientists for puzzle 1977.
If you run this recipe TWICE, then it will crash Foldit instantly at this second execution. Why there should be any difference at the SECOND execution is a mystery to me. It is even possible to reload "crashtest" before the second execution and it will still crash.
If you rather want to have a crash at the first execution, then you can uncomment the line "--structure.ShakeSidechainsSelected(1)" and for me it typically crashes at qs 51 or so. But that seems to vary. This is the full basic command-sequencing which is used in drmxw.

Even stranger is that it is not sufficient to just load the 30 remixes in the second execution (so disabling the line "local nfound=structure.RemixSelected(30,nrresults)" and enabling the line "--local nfound = 30". It seems to have something to do with twice executing the RemixSelected(). It also seems to get worse with higher number of remixes to find but 30 leads to reliable crashing for me. This is probably a generic problem and this crash-recipe can probably be used on designs of other puzzles as well since drmxw crashes on all puzzles for me.

I am pretty certain that this is not the only scenario which causes crashes because other recipes to not use remix but still crash. But it may be a hint to finding the root-cause.

Hope it helps to track this down.

dcrwheeler's picture
User offline. Last seen 10 hours 22 min ago. Offline
Joined: 09/05/2013
Groups: None

In my case, with the monomer by itself, I can will only run Mutate No
Wiggle to completion. Any other recipe crashes in less than three
minutes. Once within bonding range Mutate no wiggle will do 6 segments
but no more. All other recipes crash within a couple of minutes.

I wanted to ask if it was possible to obtain an earlier build.
I think it was this last update or the one before it that would at least
would let recipes run for a while.

As it is now symmetry puzzles are unplayable to completion.

I would like to thank the input from the other players.

I was relieved to know it wasn't just me.

Sitemap

Developed by: UW Center for Game Science, UW Institute for Protein Design, Northeastern University, Vanderbilt University Meiler Lab, UC Davis
Supported by: DARPA, NSF, NIH, HHMI, Amazon, Microsoft, Adobe, Boehringer Ingelheim, RosettaCommons