Suggestion: a complete change of the social structure of Foldit

Case number:699969-993125
Topic:General
Opened by:Ignacio
Status:Closed
Type:Suggestion
Opened on:Tuesday, July 10, 2012 - 12:12
Last modified:Tuesday, July 10, 2012 - 20:37

Hi all,

After much thinking, I have considered it is the time for sharing a few ideas on the game. They are quite extreme, I hope you excuse me for that.

A significant previous point is that I have always played only as a soloist and without using any information from the rest of my team. That is, I provide solutions to the team when they are good enough, but I never look at a solution by other player, until the puzzle is finished.

This has given me a good independent idea of how possible is for a single player to compete with multiple players by using just his creative skills. Playing like that, I think I got to the 4th place among soloist once, and now I am 23rd and improving, after I came back, just for the CASP puzzles. I recently was first in two CASP puzzles, showing that single players can sometimes outperform all the other players, even if the are collaborating.

This is a simple demonstration that teams are not absolutely required. On the contrary, I think they can freeze creativity: within a team, people often just copy what other people is doing. By working independently, I can come up with novel structures that nobody else has seen.

True, I often greatly fail, sometimes ending 50th. That is the problem of playing totally alone. Of course, I agree that collaboration/competence may greatly improve overall results, if properly done.

Now, a second important point i that it would have been impossible for me to play at that level without the sophisticated recipes that I have, many of them group-specific. I recognize the important contribution of some people in my group Go Science, which has been critical. They have come up with great novel recipes that makes us a competitive team right now. However, and most significantly, an important group of my recipes, sad is to recognize that, were stolen from another team about some time ago, in an affair that I am certain is well remembered by many. That robbery certainly helped me, I just could not compete with the top players before it. Terrible, but, in my opinion, another good demonstration that teams do not necessarily help.

With this background, I have concluded that the whole structure of our social network is wrong. It is based on the idea that competence between teams will help to improve the overall results, but I think the complete opposite is true: open collaboration among as many players as possible will improve the results. Right now, this is becoming a competence between just a few top members in a few top teams. These players have great advantage over the rest, based on having the privileged information of group-only recipes.

My experience leads me to think that just a few players (clearly less than 50, perhaps less than 25) have the skills required for systematically coming up with intuitive solutions that are good enough as to lead to the top final results. These top players will certainly not win all puzzles, because some other member may be lucky enough as to sometimes hit the jackpot. Also, skilled but still almost trivial use of the recipes may be enough to win some puzzles, especially if they are small/easy. But even with those exceptions, it is obvious that just a few top players win most often, and most often are among the top soloists. This is because they invent the best solutions.

How often these top players are in a group without the right tools to fully explore the solutions that they come up with is for me the main question unsolved in Foldit. That was my experience when our group, Go Science, started: we just did not have the right programs to compete. Once we got them (in part an illicit affair, as I mentioned above), it was much easier. The same happens now, differences among groups exist that make competition unfair.

I am sorry to say that the best thing to do now to be a top Foldit player may be perhaps something like this: go to several groups, steal all recipes, go then to the top group and copy the best players in that group as much as possible. This will make you a top player, without too much effort and with no creativity. Is this reasonable?.

To solve the problems that I right now see, I suggest the following, which is something that would certainly require a change of mentality, but I am also certain would improve overall results:

1) Completely eliminate groups.
2) Completely eliminate private recipes: everybody has the same tools.

These two make all players equal, increasing competence among players, not groups, and avoiding the problems caused by closed groups. It is clear that it is best if all players have the same tools.

3) Players start each puzzle by competing as soloists.
4) The best solutions for each puzzle (let's say the top ten) generated by the soloists, are accessible to all players at all times.
5) However, if a player gets access to any of those solutions, it ends the soloist period and enters the evolver period. From then on, he/she gets only evolver points, no more soloist improvement is possible. Of course, an evolver can move from one solution to another if it turns out to be better.
6) Last hours of a puzzle: all players automatically shifted to evolvers. Timing for this shift related to puzzle difficulty (or even better: how much improvement has occurred in the previous hours, if that was possible to evaluate).
7) Just as it is now, keep independent classifications for top soloists and top evolvers.

3) to 7) allow all players to try to improve the best solutions at all times. If they are clearly failing as soloist, they can try their hands at evolving the top solutions (not the top solutions of a team, that are often quite bad). Also, soloist are rewarded by INDEPENDENT WORK, not by peeking at the top solution of their group buddies and correcting the solo solution (if you want to do that, you must be an evolver, period). All this should lead to a higher degree of diversity in the solutions generated. Finally 6) leads to all players concentrating in just one or a few top solutions for critical last-hour improvement.

7) Create a new ranking for the tools, so it is known how many times each recipe has been downloaded.
8) Generate a ranking for the people that is programming the recipes, according to how many times their recipes are downloaded. They deserve a lot of the credit for our results, that now they are getting only in part privately, within their groups.

7) and 8) Make the programmers useful for everybody, not just his/her teammates. Also, their skills are recognized, something that I think is no completely done now.

I hope this helps. Best.

Ignacio

(Tue, 07/10/2012 - 12:12  |  21 comments)


Joined: 04/19/2009

I can agree with some of your points…

But would like to offer a different perspective on others.

One thing that I can see (that you state) is that you have taken little advantage of the help between teammates. One of the best outcomes of team evolving is a communication between the soloist and those who choose to evolve a solution - not just what scripts to run that may improve, but which areas have responded to what type of handwork. This would likely be an impossible outcome using only global chat.

There are two things about group scripts that I would take issue with. The first is that often, a scriptwriter is posting to group an initial version of a new script, and asks teammates to test it for bugs.

The second is that from what I have seen, the best scripts are a collaboration between someone who has a great idea who does not have the ability to write a script, and a good scriptwriter. Often someone else may chime in with an improvement. This is something that requires less "noise" than global. That kind of collaboration works best in a smaller than global grouping. While it works in a group the size of mine, I suspect that we probably push the outer limits.

I have either given an idea or an improvement that has resulted in a decent recipe a dozen or so times. Without the easy access to someone who could turn those ideas into a useful script, I doubt that I would have "contacted" a scriptwriter to ask them to do something. Since a few of those scripts were among those that you have benefited from, they would not have helped you - or anyone, because they would never have been written.

The system that we have now is imperfect in many ways. As foldit gets more complex, there is no doubt that we need better teaching tools - webinars, videos, etc. We simply don't have these tools at the moment that are good quality and free for use - I have investigated many, and have been trying to work on this for many months. We are lucky to have had so many people contribute to the updating of the wiki, but without good quality video access, imho we have reached a ceiling on what we can "teach" on a large scale.

The team structure at the moment provides an easy means of a one on one help, or a Teamviewer session to demonstrate either a basic concept or a more complex means of handbuilding a protein. While it's irritating that a folder can only get that in a small group, it would be close to impossible at the moment with a "group" the size of global.

One last thought… for many people, it's the social aspects of foldit that keep them folding. There is silliness and bonding and fun and information about many off topic things that go on in the group chat rooms. This is why many people keep folding - through a dry spell, while they are learning, during computer issues. If you cut that off by throwing everyone into the large global group, many folders would simply drift away. Yes, anyone can open an external irc room, but many folders choose to use only the ingame chat - some just don't have the ability or the desire to do an external irc.

I suspect that this may be a lively thread! However, I'm changing the priority to 3 - "1-High" is usually reserved for game-killing bugs.

Ignacio's picture
User offline. Last seen 1 year 43 weeks ago. Offline
Joined: 05/10/2008
Groups: Go Science

I understand what you mean. But this can be easily solved: people may be encouraged to self-organize in not necessarily closed teams, but eliminating group-private recipes (top problem), eliminating group competition (generally useless to make any change about which is the top solution) and eliminating the possibility of copying solutions for improving a soloist score (also a bad problem right now). I am not against communicating information to other players and it is likely, as you said, that communication is better in groups smaller than the whole community. However, the current group structure, being competitive, generates more problems than benefits.

Also, please remember that several of the best AD scripts were kept group-only until they were "transferred" to Go Science. Only then, and after a big fuss, they were shared. What happened is that competition became much more close. A nice natural experiment that suggest that if all recipes in the groups were today shared, the top scores would go up simply because many more players would have the opportunity of improving their results. And, supposedly, we are here to improve the global scores as much as we can, to find out how proteins fold, not to be better than other team or player.

Joined: 04/19/2009

Ummm...

Nah.

I refuse to get into old history here.

Moving along, it took almost a year after that "transference" for GS to become competitive, and I suspect that it had more to do with the soloists who were able to become top folders (and the efforts of one good scriptwriter) who collaborated within the GS team. The recipes themselves didn't magically make anyone a top folder. The reason they were so effective within our team situation was because they are only tools - it takes the collaboration to make them work to their best. What you do in handbuilding first, last and always is the key, recipes are automated ways of doing something that you simply don't have the time to do. When you run a recipe - and what it works on - after what you have done by hand is the important part. That goes for any recipe!

How exactly would you encourage people to self-organize without a team structure?

We have a very unique community here. People are free to choose whether they wish a collaborative experience (by joining a group) or an isolated one. People are free beyond that to decide whether they wish to be social in either a group or in global. Every person crafts their own experience.

Competition is why it's a "game". Removing a part of the game (groups and group competition) would take much of the fun out for many.

If not as much fun - then why bother? We need to find more ways to encourage people to stay, rather than remove one of the components that does keep them here. Imho, more and stronger teams would be a better solution to the problems that you see - enough teams with enough diversity among them to encourage and teach any and all folders that wish to participate.

Ignacio's picture
User offline. Last seen 1 year 43 weeks ago. Offline
Joined: 05/10/2008
Groups: Go Science

I think these can be easily answered:

1) Recipes are required for the top players to be competitive. I have been surprised by how good are some of the new recipes generated in this year, since I last played. Of course, no way I can compete just with my old ones. This means that having a few additional top recipes is critical for success. Tongue in cheek: If you don't think so, why don't you ask AD to put everything into the open repository, e. g. today?. That could set an example for the rest of groups. Also we would see in real practice if your results are the same (your expectation) or go significantly down (my expectation). But, actually the latter already happened in the past... I understood that people in AD was not very happy for that, although I think it was a positive affair for Foldit as a whole.

2) Teams do not mean competition among teams or team-specific recipes, the two things I am criticizing. If people find interesting to self-organize into teams to learn, improve, share experiences about the puzzles or simply chat, it would do so. Teams could be organized around top players or simply players of the same level that want to share experiences, etc.

3) You may bother to play Foldit because it is not only fun, but also scientifically significant. The idea that our minds may (still) outperform computers by finding shorcuts in complex problems is the ONLY reason why Foldit exists. The social aspect of the game may be significant, I understand it is significant for the players, but we should not forget the fact that this is also a truly scientific endeavour, and that our results may have practical significance. Thus, how to improve the top scores is what we should be thinking of, as a community. Hence my post.

Actually, if our results are very bad in CASP this year, then, why bother at all??. Foldit should close as a failed experiment, no matter how well we feel playing with our friends. That is why I am here, to help out, one among many others, and see what happens.

Joined: 09/19/2011
Groups: Beta Folders

A reply button doesn't show up for Ignacio's post, so, I shall reply to this one.

Ignacio,

A change in the group system may or may not be warranted. I really don't want to debate that. But, you NEVER participated as a group member, so, I find it odd for you to comment on the group experience.

I was in Go Science for nine months and never saw you utter one word. I use an external irc client and would have seen anything you said. It was only in the last month or two did I see you ever share a solution. How many did you share? A handful perhaps? With no notes to let us know what you had done. You don't seem to know how to be a team player and that is why you don't see the value of groups.

Also, if you had fully participated you would have realized most of us didn't use those old recipes from AD that you speak of. I tried one out once. Most of us use publicly shared recipes and those written by Paul Dunn.

I'm not still in GS. I left 10 days ago for my own reasons.

However, I learned a lot from my fellow team members. The group experience was a good one and I would have stayed except for a major issue with another member.

Ignacio's picture
User offline. Last seen 1 year 43 weeks ago. Offline
Joined: 05/10/2008
Groups: Go Science

Another ad hominem attack. Please stick to the topic. Thanks.

Joined: 09/19/2011
Groups: Beta Folders

Your experience simply wasn't what the rest of us had due to your lack of collaboration.

Hanto's picture
User offline. Last seen 13 hours 31 min ago. Offline
Joined: 05/10/2008
Groups: None

Response being drafted, will take some time in order to remain rational during the response. Ignacio, it would have been nice if you had broached this subject in group rather then behaving the way you did, you would have learned much that apparently you don't understand now. I'm sure you were put up to this tho, and I have my suspicions about who has set you up. You will be dismayed when you finally realize the truth of the matter..

Joined: 04/19/2009

Please

Whatever is the problem here, this is not a game-killing bug for ALL folders - this is a 3 priority, not a 1.

beta_helix's picture
User offline. Last seen 2 days 10 hours ago. Offline
Joined: 05/09/2008
Groups: None

Suggestions are indeed the lowest priority so please do not change it.

I will also please ask that everyone remain civil in this thread. Any posts containing personal attacks will be deleted immediately and this thread will be closed if it strays off topic.

The initial post suggests many different changes to the game, so please focus your replies on those. Thanks!

beta_helix's picture
User offline. Last seen 2 days 10 hours ago. Offline
Joined: 05/09/2008
Groups: None
Status: Open » Closed

I gave a warning that this topic would be closed if it strayed off topic.

I do not have time to babysit this thread, so it is now closed. Please feel free to air out your dirty laundry in your team's Forum.

The suggestion had been posted and we be evaluated by the Foldit Team based on its priority.

I'm sorry if this sounds rude, but I hope you understand that we have better things to do than monitor/edit/clean up this thread.

Hanto's picture
User offline. Last seen 13 hours 31 min ago. Offline
Joined: 05/10/2008
Groups: None

For once I am enjoyably in agreement with Aunt Deem's response.
However you mention an illicit affair and ashamedly I am guilty. What I am not guilty of tho is not trying to remove those recipes from the GoScience group as soon as I could, but I have to say that this particular failure is not all my fault. It seems that once a recipe is shared it becomes extremely difficult to remove from group listings UNLESS it is done immediately after the initial share.
I remember not even knowing how to share a recipe and itsKimo took me through the process, and in fact it was at his urging that they were shared to begin with. I know he can't defend himself as I haven't seen him in months, possibly even a year or more. But I do remember Foxy Lady and Stardust more or less stopped playing shortly after the share occurred. and one more thing I remember quite clearly. Within a month of me joining the #GoScience group I happened to notice two corruptions of my two client nicks had also joined the group. Yes at that time I had two client nicks, as I did then, and still do believe in what is so quaintly called ' clones '. Two separate machine IDs however forces one to actually evo results between the two machines, wheres using only one client ID allows one to share unlimited amounts data between machines. That IMO is true cloning... This true cloning as I consider it to be allows people with many machines to easily outperform the poor people with only one machine. So that should considered when one talks about fairness.
Now once B2 and pauldunn joined the group because it was so small, at the time I was almost totally alone in the group, with itsKimo only only rarely showing up. I think he was already interested in foldeRNA and I can't say I blame him. Always nicer to be in a more populated, but not to heavily populated group. I figure he probably liked the recipes too, but he never directly mentioned the same to me. Anyway, it quickly became apparent that Paul was capable of writing extremely advanced recipes, not like the primitive stuff I have brought over from AD. B2 and myself started making suggestions to Paul and he started churning out the advanced recipes. There was much troubleshooting of Paul's new recipes with him doing most of that as he more or less recognized his failures far quicker the myself and B2. So again Aunt Deem is correct. Much can be done in a small environment if enough dedication is offered to that environment, and assuming one has the dedication of an advanced Lau programmer, Paul wasn't all that advanced to begin with, but he showed the skill and dedication to keep trying.
During this phase you joined the group in this early stage of Paul's writing, I had never seen you before, we only knew that Dj-P only extremely rarely showed himself to the group. For the longest we didn't even know that you and Dj were related. And then somehow or another you got OPs, we knew that Dj or the admins had to have given to you, but getting a response from you was like pulling teeth on a live mammoth. You might have belonged to the group earlier, and just never noticed in the group listings or you might have joined the group just to keep the appearance of family within the group. At that time we didn't know who you were. Only that you had OPs and were only extremely rarely responsive to our questions/commentary. And along the way, you rarely shared and that was most of the time only if we virtually begged you to. After maybe a couple of months you disappeared from the group entirely, not one of knew where you had gone, but we did know that we didn't have an OP in the channel anymore, thus many issues cropped up that needed the attention of an OP at minimum and more likely the attention of a channel founder..It was during this time that we starting getting new recruits to the group, Smiling being one of them. She and Mark are the only ones from this period of time that stuck around for any lenght of time, although we still see Hylee sometimes.. I still remember teaching Smiling so much about how to get her computers to run more clients then they would normally endure. And she was an avid student, engaging in anything that allowed her to get more and higher scores, even it meant using the rather primitive recipes from AD, half of which never worked anyway. I practically begged her not to use them, but scores were more important then anything else and by that time she had been made an OP herself as Paul was sometimes away for family business. It really was at her urgings that Dj-P was replaced as channel founder. I just thought you should know that. I was pretty much against, but I am guilty of going along with the replacement.
Now I think I have explained myself enough concerning illicit affairs and the history of the group since well before I noticed your existance in the group.

Now I have a few suggestions of my own, concerning Groups/Teams

1.) Groups/Teams should not be founded by any one person as that leaves to much power the hands of one person who either can be helpful or detrimental to the health of the group. My personal belief is that at absolute minimum Groups/Teams need 3 founders and if I had my way, that requirement could be raised to at least 5 to start a group/Team.

2.) That changes to a group/Team be enforced by a two-thirds vote of the founders. That being in Open, moderated, or closed groups..

I get the feeling that you are a little upset with #GoScience becoming a closed group. I'm pretty sure Paul had good reason for his change. He and all of us old men in the group spent much time creating, testing, and finally sometimes only after a year or so, actually learning how to use some of his most important recipes.I still have some ideas about new variants of both his recipes and some public recipes, but I can tell you beyond a shadow of a doubt, I have not used ANY illicit recipes in at least a year and a half. Now that we finally have the team moving higher, you decide without talking to anyone in group to make your own decisions about what is good for the group. Like I said, I believe you had some rather ill-advised help in making your decision to avoid talking to the rest of us. So, in fact, I am not really blaming you. I think you simply made a mistake like I did when I first shared those recipes, at the urging of someone I truly respected at the time and ( I thought ) would not steer me in the wrong direction...

Anyway, Nuff said, but I am truly amazed by my almost total agreement with AuntDeem. Sorry about that Auntie :)

Ignacio's picture
User offline. Last seen 1 year 43 weeks ago. Offline
Joined: 05/10/2008
Groups: Go Science

Nothing in your post has anything to do with my suggestions. And I don't discuss ad hominem attacks.

Hanto's picture
User offline. Last seen 13 hours 31 min ago. Offline
Joined: 05/10/2008
Groups: None

I'm afraid that I will be in trouble because of an honest response, but I feel this needs to be said. Ignacio, in the American legal system there is a right that in most cases is downright mandated for a defendant to respond to an accusation. I have not and neither has Smiling attacked you, we both however are responding to your comments in honesty in open discussion. If anyone has been attacked it is me, and I offered my response which apparently fell on deaf ears/blind eyes. While it's true I deserve some of that attack, as Auntie said, it is truly very old news. While I was in the Navy prior to 1993, I spent some of my best times in the Med on Spanish soil. The people were agreeable and not hard to talk to in many cases, I had two years of Spanish in H/S at that time, although I had by the time of my retirement forgotten an extremely large part of it. If you felt so badly about using those ' stolen ' recipes, why did you? I know for example in my case I was not helped by a single one of them, it was my ignorance of how lau recipes were constructed that often pushed me into the lower 1/3 of scores. The best recipes were always public anyway, part of my boredom with AD and other factors, that I believe neither Auntie nor myself want to get into in a public discussion. My recent better puzzle scores are entirely due to a better understanding of how recipes are constructed, my modifications there-of, and Paul's extremely good reprogramming of my suggestions and the purchase of a computer that allows me to use 8 clients at a time commonly. Auntie is correct, it only through direct and considerable competition between groups/individuals that progress is made. Similar to the theory of Evolution, which most of us believe in even if we do happen to go to church on Sunday.
Now I apologize ahead of time for what will most likely be a double bounce of mouse switches and I hope everyone notices that I changed the priority to 5 which is pretty much exactly what this thread deserves.
I consider my responses gentlemenly and without any attack on anyone. I personally am quite happy that #GoScience has started to do much better then in those early days, when most of the truly high scores came from you. However you seem to be less then happy that #GoScience is doing so well. What you have done tho is to remove the your group's ability to use my scores and that is real shame for both the group you should be loving, and for me to share with MY group.
No accusations, just fact...

Ignacio's picture
User offline. Last seen 1 year 43 weeks ago. Offline
Joined: 05/10/2008
Groups: Go Science

I think you don't get it. This is not about you or what happened in the past. I just suggested some ways that I think can improve Foldit play. Please stick to the topic. Thanks.

Hanto's picture
User offline. Last seen 13 hours 31 min ago. Offline
Joined: 05/10/2008
Groups: None

I think it is you that doesn't get it. I am almost 100% certain that you kicked me out of group, I know Paul didn't and even tho B2 sometimes disagrees with me, I don't think he did either. I think the admins can come to a resolution regarding this issue, it shouldn't be hard for them to find the actual ID that kicked me from group. You have strongly affected your group, which should have been my group also. I hope you get it now. Your entire commentary about what you wish could happen to groups in general is really nothing but a sham for your actions regarding me. I know the admins don't want to see it that way, but I doubt it takes much research to figure it out...
Again I apologize for double bouncing switches.

Hanto's picture
User offline. Last seen 13 hours 31 min ago. Offline
Joined: 05/10/2008
Groups: None

I think it is you that doesn't get it. I am almost 100% certain that you kicked me out of group, I know Paul didn't and even tho B2 sometimes disagrees with me, I don't think he did either. I think the admins can come to a resolution regarding this issue, it shouldn't be hard for them to find the actual ID that kicked me from group. You have strongly affected your group, which should have been my group also. I hope you get it now. Your entire commentary about what you wish could happen to groups in general is really nothing but a sham for your actions regarding me. I know the admins don't want to see it that way, but I doubt it takes much research to figure it out...
Again I apologize for double bouncing switches.

Ignacio's picture
User offline. Last seen 1 year 43 weeks ago. Offline
Joined: 05/10/2008
Groups: Go Science

I am sorry to say that you are off topic again. This is my last answer.

Hanto's picture
User offline. Last seen 13 hours 31 min ago. Offline
Joined: 05/10/2008
Groups: None

Again I apologize for the double bouncing of my rather expensive Logitech mouse switches. Just goes to show that these brand name products don't mind creating problems and then ignoring a person's pleas for rectification, This mouse is only two months old, and the previous were only two months old when they started the same double bouncing. Like a fool, I couldn't believe that a name brand product would do that to a person.

B_2's picture
User offline. Last seen 4 years 29 weeks ago. Offline
Joined: 11/29/2008
Groups: None

Priority back to 3 again.

Hanto - As beta_helix asked, please do not continue changing the priority to "High 1"

Hanto's picture
User offline. Last seen 13 hours 31 min ago. Offline
Joined: 05/10/2008
Groups: None

I think everyone will realize that I did not purposely change the priority and as Auntie suggested, I would only do that with an extreme bug. Sorry for your mis-understanding B2.

Sitemap

Developed by: UW Center for Game Science, UW Institute for Protein Design, Northeastern University, Vanderbilt University Meiler Lab, UC Davis
Supported by: DARPA, NSF, NIH, HHMI, Amazon, Microsoft, Adobe, RosettaCommons