Open source

Case number:699969-986352
Topic:General
Opened by:dennisn
Status:Open
Type:Suggestion
Opened on:Saturday, June 27, 2009 - 16:33
Last modified:Wednesday, September 27, 2017 - 10:15

Is there a good reason why this program isn't open-source? I thought it was primarily a scientific thing, and thus should be as open as possible?

(The binary blob also doesn't work here in my x86_64 linux).

(Sat, 06/27/2009 - 16:33  |  24 comments)


ipatrol's picture
User offline. Last seen 4 years 38 weeks ago. Offline
Joined: 12/04/2010
Groups: None

Agreed. I understand the issues with scoring, but since we already have Lua scripting, there is no reason why users should not be able to use their own bioinformatics software. Our goal is first and foremost to develop ways to better predict protein folding, the scoring is for motivation. If we use a copyleft free-software license, any user will be able to pick up these modified clients and modify them further, a process that will lead to many features that can be integrated back into the main program. Bugfixes can be authored by users, and the development of more powerful automation will no doubt improve Rosetta@Home's performance by integrating the rich world of open-source bioinformatics software into the project. Now then, what are you waiting for?

P.S. Read http://catb.org/~esr/writings/quake-cheats.html for a word on cheating prevention in free game clients

beta_helix's picture
User offline. Last seen 6 days 14 hours ago. Offline
Joined: 05/09/2008
Groups: None

assigning...

B_2's picture
User offline. Last seen 3 years 3 weeks ago. Offline
Joined: 11/29/2008
Groups: None

here we go again

Joined: 09/27/2011

Just my input:

Even if there are problems with open-sourcing the scoring and/or networking systems, there's no reason to not open source the user interface. With the user interface open, people can submit patches that fix UI issues, add new UI features, or configure it to allow different types of input devices (each with unique input styles).

Like on google code, patches could first be reviewed. If the user has submitted more than one patch and is deemed a reliable developer, they can be promoted to committer so that their patches do not need to always be reviewed(by an existing foldit developer), and can review/apply patches made by other users.

Not only can this reduce the load on the team, but UI interfacing can be extended, and any new features that are later on the priority list can be implemented without hindering the work on more important feedback. Committers will normally review each others' work and that of new patch submissions, so the UI branch can effectively manage itself, once it gets going.

As a bonus, you will be providing these committers with a valuable resume entry.

Joined: 09/27/2011

I noticed an issue concerning UI macro/scripting, and want to point out that this is NOT the same suggestion.

I forgot to mention, but by UI components I mean anything that isn't network/scoring related, including graphics rendering. I really believe the community can improve on rendering, increasing the performance of the game itself (I'm sure you've noticed larger proteins causing slowdowns).

Also don't forget, the community has some really smart people in it, some being computer science students at universities, others being professional programmers. They can and WILL produce high quality work (if that's what people are worried about, don't be).

beta_helix's picture
User offline. Last seen 6 days 14 hours ago. Offline
Joined: 05/09/2008
Groups: None
Status: Open » Open

re-assigning...

Joined: 06/17/2010

*lurking for foldit git tree* :)

Joined: 05/28/2012

Just felt the need to bump it, open sourcing things is a great way to increase community involvement and reduce the load of programming on your developers. Also you don't need to pay people when its open source, they do it because they want to make it better.

Joined: 10/16/2013
Groups: None
Status: Open » Open

Open source software is great, but for computer games where people compete with each other it's not always a good idea to do so. I know "security by obscurity" is frowned upon by most experts. From a security standpoint, the optimal solution would be if all calculations were done on the server, no savegames on the client, everything server-sided. But for most computer games, this way just isn't a practicable solution. Especially not for FoldIt, which combines gaming and distributed computing. This means the fight against cheaters will always be a cat and mice game. That's why it is important to keep some secrets to be able to keep the upper hand in the fight against cheaters. One might argue that cheaters are no real issue in FoldIt so far, but I would not underestimate the impact of open sourcing the whole code on this issue.

I think it's also a problem if people "improve" their client with new tools and features. I don't think the tools are meant to make it as easy as possible for us. As I understand it they're chosen in a way so that scientists can learn as much as possible from our manual folding. Perhaps there is some super handy tool which is intentionally kept away from us to see how we solve problems without it, to be able to improve some algorithms. If someone could "improve" the client on his own and still participate online and submit scores, that might totally ruin these intentions.

Open Sourcing the UI code only as Heinermann suggested seems like a good idea to me. There really is a lot of room for improvement in the user interface. But open sourcing the "core" of the game seems like a bad idea to me.

spmm's picture
User offline. Last seen 21 weeks 5 days ago. Offline
Joined: 08/05/2010
Groups: Void Crushers

Having read about the recent brouhaha on the Linux kernel it would seem that there would be much more work involved for the foldit team in managing code submissions and maintaining a good quality codebase. Also performing the less than joyful tasks of sorting out dramas and reading bad code.

foldit UI design should be done by UI and UX design 'professionals' in conjunction with scientists who actually use the program; with some help from programmers (ducks) the results also need to be carefully tested with real users. I shudder to think about what an open sourced UI would actually end up like, probably unplayable.

Joined: 02/08/2014
Groups: None

Opening the source is one thing, giving commit power to people is another.
When the UI crashes, currently we can just get a very coarse view of what happened with gdb or strace.
With the source in hand, we could analyze and even fix bugs for ourselves, leaving to the dev team the ultimate right to ignore all patches. Only the Feedback page will be populated with things like

"In this detailed set of conditions, there is a crash on Linux because..."

instead of

"Hi, my client crashes 3 times an hour. What gives ?"

I don't see this as an extra burden for the dev team; rather, a better way of tapping into their population of worshippers, among which hundreds of software engineers lurk ;)
By the way: FoldIt is just awesome !

Joined: 04/15/2012
Groups: Beta Folders

I completely agree. This would be similar to how Eterna is going to do it: http://eternagame.org/web/news/3570131/

Angus's picture
User offline. Last seen 6 weeks 3 days ago. Offline
Joined: 06/04/2008
Groups: Beta Folders

Seriously, after reading this thread and the other "open source" threads that pop up when new players join, you actually decided that you needed to resurrect this AGAIN ?

Oh wait, maybe you just joined yesterday and immediately decided you knew better. Sorry, we didn't realize.

Joined: 02/08/2014
Groups: None

No. I simply offered help, being a software engineer. And I'd really like to hear the developer's thoughts on this offer rather than such sterile rant as "oh, not again".

Joined: 04/15/2012
Groups: Beta Folders

Just thinking, but what about some sort of system layed out like feedback? There are great points about the difficulties and pitfalls, but there should be a way for those who are able to somehow contribute... We have someodds of 60 open feedbacks. A bunch are probably now invalid, but I'm sure many are not. It could really have a tremendous benefit.

v_mulligan's picture
User offline. Last seen 2 years 18 weeks ago. Offline
Joined: 03/04/2009
Groups: None

From the scientists' perspective, I don't think it would be possible for the game to be open-source. There are many things that could be done to improve gameplay at the expense of scientific validity. (One extreme example would be simply omitting computationally-expensive parts of the scoring function. This would make the game much more responsive, but would not result in particularly meaningful protein structures.) When development of the game interface is limited to a few specialists who work closely with the scientists, we can make sure that improvements to gameplay are subject to the constraint that the game is still modelling proteins in a useful way. If it were open to the community, changes would likely creep in that would mean that the game interface was either no longer linking properly to the Rosetta core, or was no longer producing scientifically meaningful results. This would not be due to any malace on the community's part; it's just that development requires quite a bit of expertise in the science of protein folding, meaning that well-intentioned changes could be damaging.

Now, that being said, the source code for Rosetta IS freely available for non-commercial use, with registration: https://www.rosettacommons.org/ . Only registered developers can commit changes to the source code, though, since this does require a certain level of expertise.

Joined: 02/08/2014
Groups: None

If the central server always re-scores proteins submitted to it, there is no "cheating" possible. The client program then is just one way, among others, of producing interesting candidates. A whole ecosystem of alternate clients, possibly forked from the original one, could gracefully compete. Even fully automated search programs, including Rosetta, could join the race. Nobody cares how the winning PDB file was generated...

Joined: 04/15/2012
Groups: Beta Folders

In my mind, devs could just keep an iron hand on bad changes and not commit them. Honestly, with all the thoughts in the feedback section, I would find it beneficial for players to have the option to build a tool and submit it as opposed to the possibility of something that could greatly increase the game and research possibly not getting integrated at all because of limited development resources. Even if there was a way for interested players to say "Hey, I'd like to build this tool", you give them the ok and needed code if you are good with it, they return with their tool, you make any necessary tweaks, then you release it. The possibilities of this tool we call a game could increase substantially. By no means should you let anyone and their brother get access to the code, write whatever they want, then commit it themselves, but there should be some way for those who are able to help be able to do so.

dennisn's picture
User offline. Last seen 4 years 14 weeks ago. Offline
Joined: 06/27/2009
Groups: None

I understand that relinquishing some control (by open sourcing) is not ideal from the sterile scientific lab's perspective, but either way this is not a sterile lab. Reverse engineering and code modification and cheating is still possible. Obfuscation really isn't a good solution. I'm pretty sure there are open sourceable ways to detect/prevent cheating. I mean, isn't the goal of the project simply to solve protein structures? (I haven't played the game, since I refuse to run closed-source software on my machine (a la Richard Stallman), and originally actually couldn't on my 64bit system.) Are the goals of the project not alligned with the scoring system of the game? That is, why is cheating even possible? If somebody finds a clever way to shortcut the solutions, why is that a problem for the project?

v_mulligan's picture
User offline. Last seen 2 years 18 weeks ago. Offline
Joined: 03/04/2009
Groups: None

There are really two types of "cheating": one good, and one bad. If a player "cheats" by finding a very quick and easy way to get a scientifically valid solution (a folded protein structure, or a good protein sequence that will fold into a desired structure), then that's terrific -- and that's not really cheating. It's exactly what we want to encourage. Insofar as looking at the code will help that, yes, there's no reason to prevent players from looking at (though not changing) the code; security through obscurity isn't necessary. And yes, it would be great for players to have a way of coding algorithms that can serve as shortcuts -- but players already have that, in the form of the Lua scripts that you can write, share, modify, and so forth.

The other type of "cheating", though, is exploiting imperfections in the scoring function in order to come up with easy ways to produce structures that score well, but are scientifically meaningless (e.g. the "forests" of surface-exposed tryptophan residues that Foldit and Rosetta designs often used to have before we figured out ways to penalize that). Even this is not malicious cheating -- indeed, when players do this, it helps us to identify the flaws in our scoring function and to fix these. The problem is that we can only improve the scientific validity of the scoring function and the game by having players respond to it without being able to change it. If players had control over the game engine and the scoring function, the natural tendency would be to create more scoring artifacts that could be exploited to produce high-scoring (albeit meaningless) structures. We would inevitably move towards scientifically meaningless results -- again, without any malice whatsoever on players' parts -- since players have no means at their disposal for assessing the scientific validity of the changes that they introduce (something that requires a wet lab).

Joined: 04/15/2012
Groups: Beta Folders

I do agree. But why not let players work, then YOU go in and actually make the changes? Or maybe even let players create tools, and not touch the scoring function *Cough* Pins *Cough*? And if that causes issues, UW changes what they need to change. It would still mean more things getting out the door, and Foldit isn't trashed with scientifically bad code.

Joined: 01/28/2014
Groups: None

Obscurity is never a security feature. Why don't you allow us to fix your mistakes?

acassis's picture
User offline. Last seen 3 years 51 weeks ago. Offline
Joined: 05/26/2014
Groups: None

Please release the source code! The interface is amazing and could help other projects...

fenollp's picture
User offline. Last seen 33 weeks 5 days ago. Offline
Joined: 09/27/2017
Groups: None

Hi there,
Yes, I have not played the game.
Just came here to say I love the idea. Got you a great paper in Nature!
As a software engineer, you really need to collaborate with the open source community.
There are plenty of things you probably don't know that will help you greatly building this game (and others) and free resources as well (I'm thinking about bitbucket, github, circleci, travisci, ...).

If you are scared to be accepting proteins that have been wrongly ranked by a bogus function, then don't trust scoring from clients. Just add a validation pass that you run yourselves (or gently ask Folding@HOME if they'd be ok with lending you guys some computing power to run this).

Scientific Discovery Games appears to have received grants from multiple US departments, I don't see why at least certain parts of the source code shouldn't be accessible to at least the American tax payer.

I don't know what your goal is with this (DARPA stuff? making millions?) I just see this labeled as research and think OMG this is a terrific idea now how does this help science, health, humanity?...

Please make it open. Find the LICENSE that suits you but at least allow non-commercial derivatives.

Sitemap

Developed by: UW Center for Game Science, UW Institute for Protein Design, Northeastern University, Vanderbilt University Meiler Lab, UC Davis
Supported by: DARPA, NSF, NIH, HHMI, Microsoft, Adobe, RosettaCommons