A Request to Roll Back the NC Client

Case number:845813-996839
Topic:Game: Tools
Opened by:auntdeen
Status:Open
Type:Suggestion
Opened on:Tuesday, February 4, 2014 - 04:36
Last modified:Tuesday, February 11, 2014 - 17:01

This feedback is a general consensus of a conversation in the veteran's chat room.

We all understand how important accuracy is in our game client, and applaud dev efforts to bring us a better client.

Right now, though, NC is simply not ready for prime time. The update just posted to devprev was very anticipated as we all hoped that it would solve one of the biggest bugs, rebuild. Unfortunately it has actually made rebuild worse, if anything (http://fold.it/portal/node/996667).

The community has tried very hard this past week to make this work… but there are too many major bugs and hurdles. Some players are bouncing in & out waiting for a playable client. Some have rage quit. Most have done their best by playing, trying to adapt scripts, trying to find ways to make the software do what it used to do easily.

However - most feel that the fun is completely gone. The new game is barely playable, and not at all enjoyable for many.

There was a proposal in this conversation that the following be done:

Roll back the Main client.

Keep NC in it's own update group until all the major bugs are gone (that would include the script hooks for the wiggle levels, and anything else missing for scripts to be able to work again).

Allow us to use both clients until we can tell you that NC is both playable and enjoyable. This would also allow devs to see if the new accuracy really will work well.

(Tue, 02/04/2014 - 04:36  |  37 comments)


Joined: 12/06/2008
Groups: Contenders

I hope the developers learned a lesson from a prior major update (hello, wiggle sliders!), and will do the right thing this time to prevent another mass exodus of players, as happened with that update.

That update was rolled back, and the idea was abandoned. Most of the players who left never came back.

I'm not suggesting to abandon the ideas presented in NC. We need a more accurate scoring system. I think the current update needs to be recalled and THOROUGHLY tested outside of game play, before it is made the law of the land. A few weeks is not enough time to test for all the quirks that could happen with that radical a change.

So sorry NC probably won't be ready for CASP, but quality over quantity, please.

wisky's picture
User offline. Last seen 9 weeks 2 days ago. Offline
Joined: 07/13/2011

I am third'ing this motion. Too many bugs in current NC client. It needs more work.

Susume's picture
User offline. Last seen 1 week 6 days ago. Offline
Joined: 10/02/2011

I agree that there was not enough time to test newchapter, and for fixes to be put in and tested in turn. When fixing a tool causes it to malfunction even worse, that suggests that the new client is really not ready for production. Relying on fixes being put in while it is in production seems a bad idea for now, given that the development cycle as it currently exists did not uncover some rather glaring defects before they were released. The new devprev client is unplayable in my opinion; I have gone back to main. I hope at least the latest update will be rolled back. Even better to roll back the entire newchapter, so that rebuild could be fixed and player tested as many times as necessary to come up with something playable in competition.

The players really want newchapter to work; we have been trying all kinds of new and old tricks to try to make it workable, and sharing the results in chat. This on top of the testing and publishing of info that we did before newchapter was released. People have been asking all weekend if the update was out yet, wanting to try the improved rebuild. It was a huge letdown to discover it was broken right out of the box.

Joined: 05/28/2012

I know trying to work within the current structure while still adding updates can be challenging. At this time with such a major update, it's time to just hold what you have and try to redo the foldit code from the ground up instead of this patchwork it currently is. With this big of a change you need to redo the engine to suit your needs not just re-purpose whats there. Its the difference between an old car with different doors, a bent in bumper, slightly odd noise from the engine and a new car. they both work, one just works better and looks nicer(albeit more expensive)

If you expect people to take this more seriously you need quicker response times, and better support. thoroughly test your code in game in as many situations as possible looking for what it should do vs what its doing and actually know what the game is supposed to do from a player perspective.

phi16's picture
User offline. Last seen 6 hours 52 min ago. Offline
Joined: 12/23/2008
Topic: General » Game: Tools

With puzzle 841, Denovo Round 2 and Round 1 before it, I tried as best as I could to create a helix where the given structure suggested one, without success. I couldn't create the helix no matter what I tried. Rebuild simply would not move the thing. Banding was clumsy and stupid. I came up with using the Tweak Straighten tool to create the helix, again without success. That attempt began well, but halfway through morphed into something ugly and grotesque.

Without the ability to create this helix there doesn't seem like much purpose in continuing with the puzzle. My score is currently -30,000. I'm used to creating a helix by making the required segments into helices and using the rebuild tool. That doesn't work at all.

I finally tried banding every fourth segment in the hopes that I could muscle my way to a helix. Nope. Nada. Annihilation. Crumbs. Diddily. Scratch. Zero. Zilch. Zippo. Goose-eggs. Nothing.

viosca's picture
User offline. Last seen 1 year 19 weeks ago. Offline
Joined: 10/16/2013

Attached is a screenshot showing a loop with an extreme acute dihedral angle produced by rebuild in dev_prev.
Most of the loop is obscured by a helix in the foreground.
I just happened to quickly catch it during a tvdl drw run.
The area is highlighted in a circle.

-Randy

spmm's picture
User offline. Last seen 41 weeks 3 days ago. Offline
Joined: 08/05/2010
Groups: Void Crushers

Ah yes is that what it is called - they have been becoming more and more frequent over several months, NC has tipped them over. Even on CI 1

Joined: 09/21/2011
Groups: Void Crushers

Would it be possible to ONLY use the new scoring function but keep all the tools the same as they were before NC. Maybe that would solve some of the problems and allow a smoother changeover.

Angus's picture
User offline. Last seen 6 hours 43 min ago. Offline
Joined: 06/04/2008
Groups: Beta Folders

I will echo most of the sentiments above. After 5 1/2 years of this, you (the developers) have managed to drag all the fun out of this game. I wasn't paying attention during the runup to this release due to real world commitments, and it was quit a shock when first trying a new puzzle with this present client.

Handfolding didn't react at all the same, and most of the collection of recipes didn't seem to have much of an effect at all. I couldn't get into the top 100 on an easy puzzle like 839 using any of my regular tools and methods, and if I hadn't caught some some chatter about Idealize would be even further down.

Unless this changes back, I'm afraid my career here has come to an end.

gitwut's picture
User offline. Last seen 41 weeks 6 days ago. Offline
Joined: 05/18/2012
Groups: Contenders

Even if everything else worked perfectly, I would still be unhappy with NC due to the constant client deaths. Whether it be death by wiggle power (particularly high) or death during scripts, there are far, far too many of them. It was suggested that the problem with death by wiggle power was fixed with the 1-26-2014 build, but it wasn't.

It was suggested that the scripting issued was successfully duplicated and passed along to the devs circa 1-29-2014. The only update since then was yesterday and after hearing that rebuild was worse than before, I wasn't about to update to the newer version--so I don't know if either has been fixed yet.

The above is from my feedbacks: http://fold.it/portal/node/996704

As stated in the feedback, I have scaled back on the number of clients I am running as compared to pre-NC.

Although I have no problem creating or re-curling helices (cut points on both ends and allowing the straighten tool to run until it stops on its own), the complaint that it doesn't work with Rebuild is both valid and an important issue.

If these problems can't be resolved quickly, I don't see how using it for CASP is going to be an option--at least not a successful one. Keep in mind that even if these client problems are resolved before CASP starts, the end users still needs time to adjust their play styles and scipters will have to adjust and test their scripts thoroughly for all puzzle categories. I think that the devs are severely underestimating the magnitude of adjustments necessary for the players and scripters--either that or they are not duly concerned about it. I hope it's the former.

The current patch put out yesterday should be rolled back until rebuild functions better than it currently does. I am all for running NC separately from both main and devprev as it did prior to 1-26 until the major problems are solved (and confirmed by the players).

Joined: 11/16/2012
Groups: Beta Folders

as stated above there is to many bugs to work with new chapter, its fine when starting a structure. But when you run a recipe the client crashes, and when it dosent it takes over an hour to do a 15 minute script.

Joined: 04/15/2012
Groups: Beta Folders

In consensus with all-if it don't work, you can't use it.

Interesting point Phallicies made. Every so often, things need to be re-done so what you're working on is all nice and clean again, plus maybe adding in some things to help it work nice (ex. if I upgrade a C# script to use classes, or change the class structure from a garbled mess to something I could actually use).

Also, it seems to me the score weighting is off. I use idealize on a helix, gives me a nice curve, I local wiggle, wiggle, and bam-back to where I started, with a stretched (or malformed) helix, and a lower scoring one too. I'm all in favor of a wiggle that chooses specific score terms to look at.

I wish I could jump in and say, "Hey, I'll come in and help motor along with your coding!" but sadly I don't have the knowledge (I can program, but barely, with next to no experience, and I've never touched C or C++, which I assume you use heavily), plus I'm in school. However, if there is anything I can look at, test, improvise, organize, etc., I would love to know.

There's probably not much I can do now between lack of knowledge and lack of time, but anything I can do, I would love to, and I'm sure many here would love to pitch in as well.

frood66's picture
User offline. Last seen 10 hours 8 min ago. Offline
Joined: 09/20/2011
Groups: Marvin's bunch

I’m not going to repeat any of the above submissions – I think they are perfectly clear enough.

I fully understand the need for improvements in the scoring system and agree that progress is necessary – what I do not understand is the way this is being implemented.

It seems clear that radical change has been on the cards for some considerable time and yet the community was given practically no forewarning. This was followed by one of the shortest ‘testing periods’ we have had even after extension (which the community practically had to beg for and was, it seems, only grudgingly given) and requested information was generally very slow to materialise. Just to cap it all off the NC release was absolutely riddled with bugs, many of which I would describe as blatant. Quite frankly I feel the whole episode so far has shown little but disrespect, nearing on contempt for the community in general and a complete devil may care attitude.

I fail to understand why it has been approached this way – surely a more informed, inclusive and steady introduction, that encourages pro-activity from the community, would yield a stronger game both in terms of solution accuracy and player numbers.

This is not what we have now – indeed, it has the makings of a crisis.

A great shame….and a great opportunity missed.

Joined: 12/27/2012
Groups: Beta Folders

I think the real issue behind the new chapter disaster is how The Powers That Be in Foldit-land treat "the Foldit community". The Foldit community has spent untold thousands of person-hours developing scripts and strategies, and who knows how many millions of person-hours playing the game, teaching others about playing the game, and refining those scripts and strategies. Suddenly, a large part of that effort is "not operational", with little explanation.

It doesn't take a lot of imagination to understand how flushing something like a million person-hours or 500 person-years down the drain might discourage the volunteers who donated all that effort.

Even before this NC debacle hit, there were signs of a lack of respect from TPTB for the community. I've seen a number of chat comments to the effect that opening a feedback is an exercise in futility, and that bugs noted in devprev are routinely pushed out into production despite negative feedback. The forum system on the fold.it site is badly broken, and therefore lightly used. The group forum feature is especially weak, with group leaders unable to create sticky postings, remove postings, or even edit postings. The wikia site takes up some of the slack, but is at least mostly a volunteer effort, with little or no official participation. Also, the wikia site doesn't give groups a private spot to share information. Major features and functions of the game are thinly documented. Try to find information on the structure.IdealizeSelected() function....

Bottom line, The Powers That Be really need to share some of that power. Right now, it seems like nothing the community says is given much weight. (Insert reference to "The Matrix".) A little community input (and a lot more testing time) would have mitigated the pain and discontent that NC has inflicted.

Joined: 04/15/2012
Groups: Beta Folders

Yeah, especially with KatFish gone, we need a better way to have a connection with devs. Whether it be player liaisons, a dev that is often in chat, or whatnot. Perhaps go down the route of Eterna and have bi-weekly dev chats? At this point I think we might need weekly or daily lol.

Any way, there still needs to be some re-structuring done to make sure the problems that matter most get done, on all sides, as opposed to being haphazard, like it looks to be now from what we can tell.

spmm's picture
User offline. Last seen 41 weeks 3 days ago. Offline
Joined: 08/05/2010
Groups: Void Crushers

I can understand the urgency to introduce new wiggle and scoring regimes (PNASPaper from last year) and I completly agree with Timo's comment requesting that this massive upgrade be 'modularised' in some way so that the players can get used to the new wiggle first and any bugs associated with it could then be managed.
Unfortunately adding adverse script and rebuild impacts at the same time makes the game very difficult to play.
Add to that the reported crashing and other difficulties being experienced by many players it becomes difficult to separate 'new standard behaviour' from a possible bug or unintended result.
Is it possible to keep the new wiggle, so we can get used to it, and rollback the other changes. At the same time documenting the changes which are breaking scripts so that it becomes slightly simpler to fix the ones we actually use.
I really don't get why rebuild has to involve cut points, is there a preferred number of segs which rebuild can successfully address?

brow42's picture
User offline. Last seen 5 days 23 hours ago. Offline
Joined: 09/19/2011
Groups: None

Specifically for rebuild, I think that if the secondary structure is not loop, then it should ignore the residue type (except for pro and gly) and make the desired flat/twist shape. In other words, don't use your fragment library (or only the absolute most common fragments).

This would break any player methods that rely on rebuild in structure mode for refinement, but I don't know anybody that does this. It also would not address any fundamental problems with rebuild with no secondary structure. But at least the fundamental step of making helices would work.

I've also noticed that 'straighten helix' on a freshly-rebuilt helix attempt doesn't straighten it all the way. I have to alternate between idealize and tweak to fix the rebuilt helix. This is even with free ends (cuts).

Susume's picture
User offline. Last seen 1 week 6 days ago. Offline
Joined: 10/02/2011

I use both hand rebuild and rebuild scripts with sheets and helices in early game to help keep the overall shape while improving the backbone; I definitely would not want rebuild to ignore the residue types and only form perfect sheets and helices.

momadoc's picture
User offline. Last seen 1 day 16 hours ago. Offline
Joined: 04/03/2010
Groups: None

I am not one of the top scorers, I rarely get to play, but prior versions were so absorbing that I would play whenever I could. The fun is gone with the rebuild tool so nonfunctional. I raise my voice along with these others to reverse the updates and restore the fun of the best game on the internet.

Joined: 09/29/2011
Groups: Contenders

Mixed feelings on this one. From a gaming perspective I just had my worst 2 scores in 7 months and had many of the same problems mentioned above. Since I only get an hour or two a night to mess with the puzzles this was rather frustrating spending one night trying to rebuild a helix and another trying to fix one bad loop (unsuccessfully).

This isn't just a game though, and if the old build wasn't yielding the desired results I would rather take our lumps now and hammer out the bugs in the updated version. We could revert to the prior version but this would just set things back several weeks/months and many people (including myself) may not have the time to assist in the testing.

There does however need to be clear communication on goals, status, and next steps. Right now I don't have a very good feel for those second two.

On a side note, the latest devprev client (from two nights ago) has performed much better for me and eliminated many of the rebuild problems I was having the first few days. Just need some patience to figure out what works and what does not.

Joined: 09/19/2011
Groups: Beta Folders

I'm hoping we don't get any new puzzles until the current ones are over so that we can experiment a bit more with what does or doesn't work.

Joined: 09/24/2012
Groups: Go Science

I agree. Let us some time to discover, test, script and adapt. Limit big puzzles to 2, small puzzles to 3 during some time.

bkoep's picture
User offline. Last seen 2 hours 53 min ago. Offline
Joined: 11/15/2012
Groups: None

Hello all, sorry you haven't heard from any of the Foldit team on this thread, yet.

We have heard you, loud and clear (and I do mean that—you have all been very constructive and descriptive in your feedback, and we thank you for it). There has been a lot of discussion on our end about the feasibility and benefits of a roll-back to "old chapter" Foldit. For the time being, we are focusing on fixing the most serious bugs you have brought to our attention; however, we have not dismissed a roll-back as "out of the question" in the case that things fail to improve. A special thanks to NickyCGS, who has been working like a dog to help us pin down the real errors behind some of these more complex bugs.

Also, I want to stress that your feedback is invaluable to Foldit. We know we're not very good about getting back to you guys, and we'll have to make even more of an effort on that front now that katfish is gone. But even if nobody has posted a response to your feedback thread, rest assured that we read every one of them, and will try to address every issue we can!

Angus's picture
User offline. Last seen 6 hours 43 min ago. Offline
Joined: 06/04/2008
Groups: Beta Folders

Oh boy - a definite "maybe". Nice way to dodge the issue.

bob1928's picture
User offline. Last seen 1 year 5 weeks ago. Offline
Joined: 12/16/2012
Groups: Gargleblasters

We live in interesting times.

It would seem easy enough to “re-activate” old chapter with a separate assortment of puzzles (much as new chapter was in the two week trial). This might help retain interest and the player population. This may be beneficial in the future and perhaps should happen in parallel with fixing New Chapter.

The new scoring is a refinement in the science and a step forward. Fixing the New Chapter problems is clearly a top priority.

However, with its present problems, this player (and others it seems) thinks New Chapter will under-perform Old Chapter. Speed can often outperform accuracy in numerical analysis.

The fool in the corner wants to know; “If the upcoming CASP is important, why not attack it with both tools?” Did you throw out all your inch size wrenches when “the world switched to metric”?

beta_helix's picture
User offline. Last seen 6 days 11 hours ago. Offline
Joined: 05/09/2008
Groups: None

It is a big regret that there was no way to have parallel versions of Foldit running at the same time (with some "old chapter" puzzles running at the same time as "new chapter" puzzles).

The way that Rosetta is coded unfortunately does not allow this, requiring New Chapter to be pushed to main directly because it is not compatible with the "old chapter" code.

It would have been ideal to overlap both versions of the game and phase things over time.
We would initially introduce new puzzles that use new chapter, but keep puzzles that work with old chapter.
Eventually (as the bugs/issues got resolved) we would have fewer old chapter puzzles, and more for new chapter.

That would have been the proper and better way to do it, but sadly the change in the code is so drastic that this was not a possibility.

We really apologize for all of this, and we want to thank each and every one of you for your patience on this one.

We know that this is asking more from you than we ever have, and we thank you so much for all the useful feedbacks you have reported.

Thank you again for staying with us during this difficult transition, we can't thank you all enough!

beta_helix's picture
User offline. Last seen 6 days 11 hours ago. Offline
Joined: 05/09/2008
Groups: None

I should clarify that there was no way to have parallel versions of Foldit running at the same time where you would be able to get credit for playing both versions (ie how you can be playing in devprev and you get credit for a puzzle like you would if you were playing it in main).

Sorry for not being clear about that when trying to address your suggestion, bobandpenny.

Angus's picture
User offline. Last seen 6 hours 43 min ago. Offline
Joined: 06/04/2008
Groups: Beta Folders

Since the scoring has changed so drastically, where people and strategies were competing at a high level before, and now everything is turned upside down, it might be time to do a rest of scores, retire the old Foldit and start over .

Otherwise, commit the resources to figure out how to run two parallel sets - maybe on a new virtual server - and write some easy scripts to combine to scores? Although, it's really mixing apples and oranges to compare the scores from the two systems.

I think a total retirement of Foldit original scores and ranking would be easier. It's a brand new game, make a clean cut and go for it.

Joined: 04/15/2012
Groups: Beta Folders

V_Mulligan posted an explanation: http://fold.it/portal/node/996871

bob1928's picture
User offline. Last seen 1 year 5 weeks ago. Offline
Joined: 12/16/2012
Groups: Gargleblasters

"there was no way to have parallel versions of Foldit running at the same time"

... so the two weeks that the New Chapter and Old Chapter DID run in parallel could not possibly have happened. Very Very interesting times.

Joined: 04/15/2012
Groups: Beta Folders

No, they were running a different puzzle set.

frood66's picture
User offline. Last seen 10 hours 8 min ago. Offline
Joined: 09/20/2011
Groups: Marvin's bunch

I believe spider toxin was run in both 'old' and newchapter. No doubt the the original puzzle had to be edited in some way to make it compatible with newchapter. So it seems to me that, whilst the puzzle set was different in totality, this was as a result of the setters' choice.

Assuming other puzzles can be edited to suit either platform, I see no reason why old and newchapter cannot run alongside each other - with more or less the same puzzle set.

Angus's picture
User offline. Last seen 6 hours 43 min ago. Offline
Joined: 06/04/2008
Groups: Beta Folders

Exactly! Legacy Foldit and New C(*&% running two different code bases, and puzzles set up for each, leave it the the players (your resources) to choose.

Legacy client running older Rosetta, and those that want to fuss with NC can run the crippled new client with new Rosetta.

You'll soon find out which client people prefer, although I don't think that's real mystery right now. The hardcore "doing it for the science regardless of the pain" folks can bang away on the NC client, and the rests can get back to enjoying the game. Both will provide science, maybe not at the level you want, but having only a slow or broken client with fewer people playing with client instances and going nowhere isn't going to get you much.

Joined: 04/15/2012
Groups: Beta Folders

In addition to Jflat's comment below, because most people would just use legacy, the "better" solutions they would make on NC would no longer be produced,.

Angus's picture
User offline. Last seen 6 hours 43 min ago. Offline
Joined: 06/04/2008
Groups: Beta Folders

And why would most people use legacy ? Because it works, and they don't have to re-learn the whole game. Not everyone has all day every day to try to learn a new game, and it's especially offensive to have to throw away years of experience and start over.

And, the new client is nowhere near as productive as the old one, nothing happens quickly. We shouldn't have to buy new 16 core CPUs with gobs of memory to be able to run this.

Joined: 04/15/2012
Groups: Beta Folders

I don't disagree, but that completely gets rid of the point of having NC in the first place.

jflat06's picture
User offline. Last seen 5 hours 42 min ago. Offline
Joined: 09/29/2010
Groups: Window Group

It is impossible to have the same client run puzzles like it did on the old client while still being able to run new-client puzzles. This means that running two sets of puzzles requires two separate clients. We temporarily had two different clients while new chapter was in testing, but that isn't a realistic model for a deployed game.

Two clients require two code bases. Two code bases mean we have to fix bugs in both code bases when they happen. Worse yet, since the very nature of a 'legacy' client means that it doesn't get Rosetta updates, as time goes on, the differences between the two code bases will grow and grow, and the fixes for these bugs and even the bugs themselves will be different. We would also need to track feedback separately for the two clients in a way that is reliable.

We would have two different release cycles to manage. Two sets of puzzles to actually post. Half the time, when we try out a new puzzle internally before players even see it, we discover it doesn't work properly. That requires us to modify the client - or in this case, both clients (probably in different ways). These new puzzles often require new features from Rosetta as well, for which there is no real fix in the legacy client. The traditional scoreboards of the website have no differentiation between new/old clients, so you would have to play both sets in order to get full points. The category scoreboards are a bit better, since you could make separate categories for each client, but that's a hacky solution at best.

Then you have to deal with testing, server communication, analysis, and new player confusion about multiple sets of puzzles and clients. It just isn't sustainable. We would be spending more time just trying to keep afloat than doing actual science.

Sitemap

Developed by: UW Center for Game Science, UW Institute for Protein Design, Northeastern University, Vanderbilt University Meiler Lab, UC Davis
Supported by: DARPA, NSF, NIH, HHMI, Amazon, Microsoft, Adobe, RosettaCommons