20 replies [Last post]
Joined: 06/06/2008
Groups: Sundodgers

Hey Folders,

There's been a bit of discussion over the past couple of days (feedback thread: http://fold.it/portal/node/985991) on how to best teach newer players about the basics of competition puzzles. We, the devs, have an idea or two for how to implement this (gathered from suggestions from users), and we would like to receive your feedback, or better yet, other ideas that the community has. We would like feedback both from newer players who have just jumped into the game, and veterans who have been teaching beginners for a long time.

In the feedback post, I said we would post beginner puzzles that would last for a month, with a new one coming up every other week. These puzzles would be available only to those with less than 15 or 150 points and would be simpler proteins that would be easier to improve than the regular competition puzzles. One such puzzle (Beginner Puzzle 1: Mouse) was posted yesterday. The downside to this approach, as LennStar pointed out, is that expert players are less capable of assisting newer players in the beginner puzzles. So we've got a second idea:

There would be two permanent beginner puzzles. Each puzzle would be available for competition for those with less than some number of points (say, 150). Every week, points would be given out and the scoreboard would be reset. The puzzle would also be available to those who have more than 150 points, but the scoreboard would never reset and points would never be given out. Experts would be able to see the structure and assist new players, but not have to spend time competing on these puzzles.

Let us know what you think! Our veteran players have always been crucial to initiating new players, so we'd like to know how you think the initiation process can be smoother.

Thanks,

The Dev Team

Joined: 04/19/2008
Groups: None
Why not simply add a few

Why not simply add a few competition-like puzzles to the intro puzzles? You don't give the players any hints anymore and let them fold a protein completely from the open game to the end game with all the tools available in the competition mode. I think this is an important step for new players, as it's the first time they have to decide themselves what to do. The intro puzzles before each concentrated on one specific tool or method. The score limit that has to be reached should be of course not too high.

There are many other unclear things like structure mode, note mode, duels and buddies, guides and the ability to share puzzles with your group. I think we should find a way to explain them properly to new players that finished the intro puzzles. Maybe an ingame FAQ would be an option. At the end of the intro puzzles you show the player where to get help and information about the other tools available in competition mode.

I think this could make it easier for newcomers because they now get a better understanding of how the competition mode works.

Another thing I would like to mention is, that although you give a rough estimation of the difficulty of the proteins in the protein selection dialog, I've hardly seen any puzzles that aren't classified as "Intermediate". I propose to keep the current duration of puzzles, but instead release more puzzles with different difficulty levels at the same time. In that way players can choose more easily the puzzles they want (new players -> "Easy" puzzles, pros -> "Advanced" puzzles) and there's a greater variety. So nobody's getting bored. :)

Migi's picture
User offline. Last seen 5 years 32 weeks ago. Offline
Joined: 12/06/2008
Groups: xkcd
Re: Why not simply...

That's certainly a good idea, but if you leave the new players with just a protein and no instructions whatsoever, what's the benefit compared to trying a real competition right away? I think you should open the puzzle with a note encouraging them to try what they want, but every minute or so give pop-up bubbles like "Try to make the protein more compact", or "Don't forget the sidechains!", or "If you're stuck, you can try to nudge & wiggle" with a link to a video explaining how to nudge & wiggle. Of course it should depend on how well the user is doing, not suggesting to look at the sidechains if those are already properly folded.

The idea about "easy" and "advanced" puzzles is a bit vague. After all, what is an easy puzzle and what is a hard puzzle? Some could argue that easy puzzles are the ones with very low points, because you can easily get more points, and the harder ones are those that are near their native state, because those require things like walking the dog. But you could just as well argue that the ones far off the native state require a lot more insight to fold properly and the high-starting-score ones can't really be screwed up.

So instead of "easy" to "advanced" I would classify puzzles as "low RMSD" to "high RMSD", but with a fancier name.

I would provide some variation, like a maximum number of moves, a time limit, try to match the guide, a game where you get 2 minutes to get a score as high as possible (with bronze, silver and gold medals), a game where you guess which of the two given folds has the highest score, a game where you can only tweak the sidechains, a game where you should try to make a cookbook recipe that solves it, there are a million possibilities.

These "games" should not necessarily be part of the tutorials but they can be a classified as "challenges", unlocked when you finish the tutorial. If possible, it would be awesome if there were multiple challenges in every category (time-based, limited-moves, sidechain-only) and completing one would unlock the next, harder one.

I'll stop now, before I write a book about this. But you got me started and I just love FoldIt.

axcho's picture
User offline. Last seen 7 years 28 weeks ago. Offline
Joined: 12/18/2007
Groups: None
Re: Beginner Puzzles

I think the distinction between easy and hard is less vague than it might appear. It's about what's easy or hard for a beginner to start with, and can be determined more or less empirically. We could even use stats on how people actually do in the puzzle to dynamically mark it easy or hard - not that I'm promising anything! ;)

The pop up tips are an interesting idea. I'm not sure yet if that's the best approach. I think the way to find out would be for us to watch some new players trying the science puzzles for the first time, after the intro, and see where they might need extra help or encouragement.

Varying goals, with awards would be very cool. I like that idea.

Joined: 12/14/2008
hmmm

I don't think the popups are a good idea. I would hate them earlier then they need for every hint to shown one time :D

The part "I would provide..." sounds very good. You can also take that for duels. Propably with user choosen goals like "only 20 moves and only sidechains" or "only 5 wiggles, rest free to use".

Joined: 05/13/2009
Groups: None
100% Newbie... Here are my thoughts...

Hello, I am a 99% completely new to Foldit (just did all the intro puzzles yesterday), and I have a number of thoughts on the beginner experience.

In case it makes a difference, I am using the Linux version.

1) What to do after Intro 4-4???

After doing 4-4 (I've seen a level 5 mentioned somewhere in wiki or something, but was not available), it was not clear what to do next. All I could do was go into the "Science Puzzle"... then found the one that said "Beginner".

My suggestion: Provide a clear progression of puzzles.

2) The Intro Puzzles are incomplete?

The last intro puzzle was 4-4. I just downloaded this yesterday, so I assume I have most complete version (and/or it should be able to get puzzles online). I have seen mentioned somewhere (on the wiki?) "Level 5", but there are no level 5 puzzles available.
When I moved on to "Beginner Puzzle 1 (<15): Mouse", suddenly a great many more buttons, menu items, and so forth appeared that were not available up to 4-4. I had no idea what these were. The only apparent option was to look at each button/menu item, read its tool-tip, and fiddle with things by trial and error. It definitely seems that there should many more "intro puzzles" to introduce all these other features.

My suggestion: A complete series of tutorial puzzles, that progressively introduce all the features of the game, including all tools for manipulating the protein, as well as all the other options, such as changing the view options, using the chat windows and so forth. The sudden jump from 4-4 to the full game interface was rather overwhelming.

Another note: I said 99% new because I actually tried Foldit a year ago (my computer at the time was too slow to handle it). I vaguely recall the Intro Puzzles back then covering more features (like "freezing" pieces of the model, or showing transparent "ghost" versions of the model to try to achieve). So I am wondering if some of those intro puzzles have been eliminated, and if so, why?

3) Could not find Web-based tutorial anywhere.

Maybe it is there, but I haven't found it yet if it exists. When I check out the FAQ, forums, or the wiki, the topics all look rather advanced. There is nothing inviting out there that says "Hey you, New Player!! Come look here if you are trying to figure out what is going on." If there is a tutorial out there, it needs to be *prominently* linked from the Foldit home page. I would likewise recommend such a link on the home page of the wiki. The web-based tutorial could go hand in hand/step by step along with interactive sequence of tutorial puzzles, but could explain in detail and depth the "what", "how", and "why".

4) Better visualization/feedback on score changes.

This does not exactly relate to the choice of new puzzles, but it relates to the experience of a new player and the development of their skill and intuition. Working on the Into Puzzles did almost nothing to help me develop an intuition for how moves affect score. There are three major reasons for this:

a) The score is just a tiny number at the top of the screen. I get little or no "feel" for how moves affect score from this -- I would have to make a tiny adjustment, then take my eyes off the model and look at the number and back to the model. When I explored the "view" options when I got to "Beginner Puzzle 1", I found some options that solve this problem somewhat by highlighting problem areas -- but the tutorial puzzles never taught me about these options. Mind you, I am not saying that there is no visual feedback -- the spiky clashes and bubbly voids are useful -- there is just not enough. It would be good to have a more visual/intuitive depiction of the score that can be sensed in the players peripheral vision, so to speak. The little popup score changes that sometimes occur are only slightly more useful than the score itself.
Suggestion: A large "thermometer style" score indicator that both grows and changes color as the score improves. Because of the huge dynamic range of the scores, a simple linear thermometer may not be sufficient. It may be necessary to have one indicator of overal score (possible logarithmic to accomodate the large range) and another indicator for small variations in the score. The scale of each indicator may need to adapt. These indicators should be the full height or width of the display area, so that the player will perceive the changes without having to actually look at them.

b) The score often changes far too quickly to get any sense of what happened and why it changed. This particularly happens when the score crashes to zero as a results of introducing several new clashes.
Suggestion: I am not sure if clashes (for example) are discrete or continuous score features, but I recommend continuous (this should be possible since the underlying physics is continuous). Then slow down the rate of change in the model to keep the rate of score change reasonable. Basically let the action go into "slow motion" so the player can see the score change along with what is happening to the model. This slow-motion effect could be an adjustable or toggleable feature so advanced players can turn it off (or reduce the effect).

c) Most of the score changes in the Intro Puzzles result from doing "shake" and "wiggle". This is rather unsatisfying. I had little feeling that I had done anything myself to improve the score. I just hit "wiggle" and watch the score climb.
Suggestion: A series of simple puzzles in which "shake" and "wiggle" (and other "automatic" tools) are not available. The player will do everything "manually" in order to see how changes affect the score.

5) The tutorial puzzle sequence needs lots of "toy" puzzles to help better develop intuition AND skill in simply manipulating the model. More models with roughly the complexity of 1-1, but with different tips/objectives for manipulating the model. One interesting twist would be instead of trying to get the highest score possible, to manipulate the model to get a specific sub-optimal score -- this will force player to push/pull, rotate back&forth, and so on, thus developing their skill.
Also, I think there is a feature that shows some kind of "ghost" shape for the model, which can be a target for the player to achieve. This was not in any of the Intro Puzzles, but I saw it mentioned somewhere. Such "ghost" targets need not be "optimal". Just configurations that require a player to push/pull/rotate and use the other tools to get the model into that shape.
I think a good idea would puzzles that had a sequence of such shapes -- the player manipulates the model into that each successive shape (developing their skill manipulating the model), and as each shape is achieved, they are treated with some explanatory text for why that shape has the score that it has.

---

These are my initial impressions and suggestions. I hope you find them useful.

Joined: 05/13/2009
Groups: None
correction...

Just a slight correction to my previous post. When I suggested puzzle "1-1" in section (5)... I meant more like 2-1 or so. Actually for the "sequence of target shapes" idea, even more complex models would be good.

Joined: 05/13/2009
Groups: None
Further feedback...

OK, I just went through the Intro Puzzles again. This time I did it "online", and they somehow seemed a little different, and a little bit more instructive. That might just be because having gone through them once, I picked up on things I missed the first time around. Or are some of the online intro puzzles different from the offline set? In any event the intro puzzles still end at 4-4 and then all there is to do is to jump into the Science Puzzles with no guidance. There are *many* features of the interface which are not covered in the intro puzzles.

One other note. Having gone through the intro puzzles again, I can anticipate that the developers/designers might say that some of my suggestions are already there. My general response would be "not enough or not in the right way". To some extent, just a bit more repetition of the same concept would be useful -- maybe a dozen puzzles for each level instead of four.

One thing off the top of my head. The usefulness of freezing was not really clear in the puzzle that included it. In fact I'm quite uncertain precisely what it does (the frozen object still seemed to move when I did a wiggle).

Here are some things that hit me when I first try the Beginner Science Puzzle. Some explanation of these may be available in the Wiki or other documentation. However, most players, even very intelligent and (initially) interested players are going to throw their hands up in frustration.

Actually, before I get into the puzzle there is the list of Science puzzles, the First thing a new player is going to see after working through intro puzzle 4-4... "Grand Challenge"??? Woah... I don't think I'm quite ready for that... don't they have something easier? Oh I have to scroll down... What does "evolver" mean? What are these "(<150)" after the puzzle names? What do the puzzle numbers mean (if anything)? What are these dates for when the puzzle is "over"? Why would a puzzle be "over"? Should players be able to try any puzzle any time?... Oh there it is, a puzzle that says "Beginner"... that must be the one to try next... Here goes...

Woah... where did all this stuff come from? Most of this was not there before...

Cookbook? Where did that come from? What is it? How do I use it?

What are the red arrows making circles around the... are they circling the voids or the side chains? What do they mean? Are they giving me information about the state of the model? Or are they some kind of tool to manipulate the model? Or both?

Reset Structure? What does that do? How is it different from resetting the whole model? Presumably there is some way to select substructures of the model and save/ reset them independently. How? I have never seen this before.

A view menu? Look at all those options. Some sound like they'll be pretty useful. Would be nice to know what they all mean...

A mode menu? What is structure mode... hmmm... maybe something to do with reset structure? What is node mode?

...and so on.

The combination of all these new unexplained features, along with nothing but trial-and-error guesswork to get started on working on the protein model... means only the most interested and dedicated of players are going to take the time to dig through the web pages, wiki, or just "try and see", to figure out how these things work. Many players who were interested and fairly excited (if still a little confused) up through Intro 4-4 are going to hit a wall when they start "Beginner Puzzle 1".

I hope I have made a good case that there needs to be a long, smooth progression of tutorial puzzles that incrementally cover every feature of the user interface (this includes the online interactive/community features, so the player has a clue about duels, contests, puzzle "expiration", etc.), while progressively giving the new player more "hands on" experience fiddling with the models. When the player first tries "Beginner Puzzle 1", you want them to have some idea what to try first, other than just random fiddling or clicking "wiggle".

axcho's picture
User offline. Last seen 7 years 28 weeks ago. Offline
Joined: 12/18/2007
Groups: None
rednalyerg, I totally agree

rednalyerg, I totally agree with everything you've said here. Foldit is still at a very rudimentary stage.

I'd compare it to World of Goo at three and a half months. Ron Carmel writes:

"at first we wondered if we were showing the game to the right people.  why are they so bad at it?  do they simply suck?  it shouldn’t be so difficult to figure this game out!  it would take months of playtesting to get it feeling good and to not lose non-gamers after 30 seconds of play.  if you play this version, pay attention to your frustration and confusion levels as you try to work through the puzzles.  that should give you an indication of how far we still had to go."

That's about how far Foldit has to go, if not further. Maybe another year of playtesting and polishing and it will feel like a real game? :p

There are some cool things coming up, like a new interface that should make it easier to introduce interface elements gradually. I'm hoping that once the design mode is out, we can really start to focus on the playability of the game. Though I'd hope that before then, we could try some of your suggestions such as puzzles without shake or wiggle, and puzzles based on moving the protein to match a guide.

Thanks a lot for your feedback. :)

 

Joined: 05/19/2009
Groups: Go Science
Re: Beginner Puzzles

As a beginner I would really like to see replays of how advanced folders do it.
It should be possible for closed challenges to have a mode where one can see exactly what a person did to reach those high scores.
Afterwards you would have an idea about what to try.

A replay mode would probably also be good for experts to learn from each other.

axcho's picture
User offline. Last seen 7 years 28 weeks ago. Offline
Joined: 12/18/2007
Groups: None
Re: Beginner Puzzles

As far as I know, we do plan to implement replays. I'm not sure how difficult this would be to do, but it's on our list. ;)

It would be great for the development team, too, in order to see exactly where new players get stuck and what they try.

Joined: 12/14/2008
Re: Beginner Puzzles

I was told that a full replay woukd be too big because of the many informations?

I really would like a something like an .sgf file for replays.

Joined: 05/19/2009
Groups: Go Science
If the tools all work

If the tools all work deterministically you would just need to log all user input into a file and then the computer of the person watching the replay could reconstuct everything.
It could optionally cut out all pathes that lead to a Restore very best to compress the time it takes to watch everything.
Of course having a fast forward button would also be nice and maybe the game could safe every 15 minutes or so, the state of the portein to allow fast forward to those points.

axcho's picture
User offline. Last seen 7 years 28 weeks ago. Offline
Joined: 12/18/2007
Groups: None
Yes, that's probably how

Yes, that's probably how we'd do it. I remember we were talking about doing that, recording everything that players do and having an automatic playback system so that we could watch new players struggling and new players could watch experts playing. I don't know the current plan though, so I can't make any promises.

Joined: 07/04/2009
Groups: None
Introducing the interface

I've only recently gotten stuck into Foldit, and love it to bits.

I think I've now done the intro puzzles (all the way through to the Design puzzles) at least three times, maybe four. Not sure.

One reason I keep going back to the intro puzzles is because the technical gap between those intro puzzles and the full experience is seriously daunting. Suddenly, you go from a set of well-understood tools to having a whole bunch of new tools, and the only place to find information on how to use them is in the Wiki, rather than in the game itself. If I had my druthers, I would introduce all those new tools (Structure Mode, Note Mode, the Cookbook, and all the rest of it) one at a time in new intro puzzles, so that when you finish you are in fact comfortable with the full interface. (Of course, since you're planning to introduce a new interface--w00t! Looks exciting!--you may already have this kind of thing in mind.) I found the jump from the intro material to the full interface deeply daunting/confusing, which is a shame, because I love the whole idea of the game.

On a different note: I'm a Linux user. Getting Foldit running under Linux is tricky, and unsatisfying. First you have to get an archive uncompressed, then get the app folder into the Home Folder, and then to actually play the game, you have to resort to the command line, change directory, and then enter the "./Foldit" command. Hmm. A worthwhile goal for the developers would be to find a way to make available .deb and .rpm (etc) packages, which are easy to install, and lower the the initial barrier to trying the game for users who are reluctant to use the command line (yes, we exist, and we're people, too :). Getting the game into various distribution's repositories, too, would be a big help, and greatly simplify installation and running the game. It would be nice not to have to go near the command line, and have Foldit available as a game in the Games Menu, like any other.

Thanks for your time. Hope this helps.

Adrian :)

taurine's picture
User offline. Last seen 7 years 15 weeks ago. Offline
Joined: 07/30/2009
Groups: None
Very new here, just finished

Very new here, just finished the tutorials yesterday and was dumped into a seemingly unfamiliar world of extra tools and functions that were never touched upon in the tutorials, I have no idea where to start or what I am supposed to be doing or where to even find out what these tools do or how to use them! Couldn't find any video tutorials or even a written list of tools and their functions anywhere, maybe I should look harder, but really if you are looking to improve this for beginners you shouldn't make them have to suffer this unnecessarily steep learning curve, sounds like you are interested in trying to attract gamers not scientists so it should feel more like a game, most games I've played have levels, start you off on level 1 and earn your way up to level two pretty straight forward stuff, the tuturial is seriously lacking. Not to be too negative here I love the concept and what you guys are trying to do here, it is just not keeping my attention, there is only so much I can do with what the tutorials taught me, perhaps a "tips and tricks" section written by pros who have been playing for a while, or perhaps it should be taught like chess, tips for beginning game play, middle game play and end game tips, I have no idea since I just started yesterday, I am only writing in this forum now because I was hoping to find some sticky thread for newbies but....nothing. Every single tool needs to be addressed in the tutorial even if that makes it take four hours to get through it, what is the use of a tool if I can't use it? Also a side note once you scroll way out and lose the view of your protein on the screen there is no key to automatically center it back in view (at least not that I could find) and unfortunately it saves that view for all puzzles which looks like I will need to reinstall the whole darn thing if I want to ever see anything again.

Joined: 12/14/2008
wiki

most of what you are searching is in the wiki, including how the pros fold.

The key for centering back to the protein is home I think. Should stnad in the help in the game.

taurine's picture
User offline. Last seen 7 years 15 weeks ago. Offline
Joined: 07/30/2009
Groups: None
Thanks for telling me about

Thanks for telling me about the wiki, however I probably am not going to read through that whole thing and I suspect neither is any ADHD teenage gamer, call me lazy, but I don't feel your average gamers who are used to built-in tutorials that cover all the basics (every game I've ever played) are going to spend much time outside the game learning, let's be realistic if we are trying to reach gamers as the target audience for crowd sourcing. Lets not forget gamers are the people who most likely have have the proven "intuition" skills for Spacial Relations games/puzzles rather than an interest in professionally designing pharmacueticlas which is what it currently feels like. I just found out about this "game" in Wired Mag and was just trying to contibute some gaming skills to the project and have fun, not make this a full time post-graduate course on building proteins. The Wired article was a bit misleading as to how far along the beta was and how user friendly it is. I still feel any puzzle game (foldit seems to bill itself as a puzzle game) should be self-contained as to not have to spend hours searching for terms (outside of the game itself) that I don't even know I should be searching for. It's got a lot of potential, but I'm sure it's like this in the beginning for any new software. I guess the question is what is the goal here? From the only thing I've read (wired article) it's to tap into the vast resource of millions of gamers and get them interested in playing this game that also hopefully contributes to science eventually. Sorry to sound like souch a newbie but if you are looking for millions of new gamers there are going to be millions of newbies at least for a little while, rather than just saying "read the 1000 page wiki" the focus needs to be primarily on actively educating newbies in an industry-standard way that gamers are used to, that is with fully comprehensive tutorials and progressive levels of difficulty, the only level puzzle that currently shows up in the science puzzle section is "intermediate" absolutely nothing for beginners (that I could find), I really want this to work that is why I am spending my time writing here. Like with any job your employees are only as good as they were trained, if you want to up the performance and productivity from people you need to up their skill set by education, people can usually work around the clunky tools in the beginning if they are taught how to use them.

Joined: 12/14/2008
You can make your life

You can make your life easier, if you just open the wiki main page and click e.g. on "Tutorial" or one of the "sugested methods". There is what you want. You can easily see it from the first page.

And btw there are only 101 pages.

And as a beginner you only need tutorial, since you have to get a feeling first.

Joined: 10/02/2009
Groups: None
Help!

Okay. I just created an account. How do access the puzzles!

Please help,
Espeon

Joined: 06/06/2008
Groups: Sundodgers
Espeon, If you go back to

Espeon,

If you go back to http://fold.it/portal, there are icons to download the installer in the upper right area of the page. Select the appropriate installer for your OS, download and install, and you should be able to start up the application. Log in with your account, and select intro puzzles or competition puzzles to find the puzzles.

Sitemap

Developed by: UW Center for Game Science, UW Institute for Protein Design, Northeastern University, Vanderbilt University Meiler Lab, UC Davis
Supported by: DARPA, NSF, NIH, HHMI, Amazon, Microsoft, Adobe, RosettaCommons