ranking

Case number:671076-654933
Topic:Server
Opened by:DisposableHeart
Status:Closed
Type:Suggestion
Opened on:Monday, December 8, 2008 - 07:34
Last modified:Monday, August 22, 2011 - 20:05

My current ranking is:

Soloist: 14th with 444 points (last 6 puzzles)
Soloist this month: 8th with 634 points (last 8 puzzles)
Soloist this week: 15th with 173 points (last 2 puzzles)

As an example I took ourtown's scores to compare. Nothing personal!


Soloist: 1st with 691 points (last 12 puzzles)
Soloist this month: 2nd with 698 points (last 15 puzzles)
Soloist this week: 105th with 73 points (last 2 puzzles)

Ourtown played <15 and <150 puzzles and scored well. For example on Puzzle 105 he received 100 global points for a score of 10591. With this score we would have ranked 35th with 60 global points in the main arena.

I see multiple problems here with global rank calculation:

- Not the same number of puzzles are considered for each player.
- The top soloist score considers LESS puzzles than the monthly ranking
- The players in the low entry arena are not measured to the global performance, but taking the local rank are added up to the global ranking.

(Mon, 12/08/2008 - 07:34  |  16 comments)


admin's picture
User offline. Last seen 15 weeks 5 days ago. Offline
Joined: 11/10/2007
Groups: vi users

To address the confusion:

1. I only saw this after the scores had updated again, for the last time in the current phaseout, so I have different numbers. Your score now is 282, which is all puzzles since and including 107. ourtown's score is 271, which is all puzzles since and including 107. I very much doubt that this was not the case yesterday.

2. Yes, this was due to the phaseout of old scores, which was mentioned in a news item. It is still on the front page, in fact. The monthly scores will adjust in another week. But if you think it very confusing, I can forcibly them off at puzzle 107 too.

3. Yes. This is a compensatory measure for the amount of time old players have been playing. It is not unfair, as by the time it would make a difference, the players are no longer eligible to play those puzzles.

Joined: 05/19/2008
Groups: None

Dear Admin,
I am fully aware of that post you're referring to. The way I understood it, only the last 4 months' score count into global ranking. This is very good, wise, and fair. The inactive folders disappear from the global ranking, no matter how good they were.

I do not know where you get from that my score is 282. This must be a typo.

At this moment my score is 382 point. My points going backward in time are: 100,81,92,32,77,69,93,98,92,100,100. This means 5 puzzles condisered

Outworn's current score is 277.
6,39,34,66,10,66,(10,66,56),54,(74,61),(100,88,44) (numbers in brackes are the SAME PUZZLE in different group, <15 and <150 and normal)

This means 7 puzzles are considered. The first group of scores in brackets is puzzle 107. 56 was taken into account, that belongs to <150 puzzle.

Another example is davidgn, ranking 2nd with 434 global points. His points are 36,80,79,(39,100),(11,89). This means that on puzzle 108 he made 139 points. How sweet! No wonder he is ranked 2nd now.

Or take toots! Ranked 6th, folding since 17 november.

This is not right. Top soloists should mean TOP SOLOISTS. It is a mirage for beginners to put them in high rank for nothing and fall back in a week because they turned out to be not that "top" after all. This will not help to keep them, even if that was the very intention to give initial "good feeling" and make them stay. They won't.

My suggestions:
- The moving sum to calculate global points and ranks should really take a longer period, like 4 months as it was mentioned.
- The moving sum should be calculated the same way for everyone.
- For beginners, that can take all three versions on the same puzzle, only one result sould be considered: the one that ranks them highest in the global arena.
- Make a "Hall of fame" ranking. (I remember from the above mentioned post this is a plan in anyway.)

Thanks

(ourtown, toots, and davidgn, excuse me using you as examples. Nothing personal!)

admin's picture
User offline. Last seen 15 weeks 5 days ago. Offline
Joined: 11/10/2007
Groups: vi users

"I do not know where you get from that my score is 282. This must be a typo."
No, I simply posted before 111 closed, which netted you another 100 points.

As I said in my earlier post, the global score is not a function of a fixed number of puzzles per player. That is entirely a question of how many they chose to play in a given interval. 7 are considered for outworn, 5 for you, because since puzzle 107, you played 5 and he played 7. This is perfectly fair.

"This is not right. Top soloists should mean TOP SOLOISTS. It is a mirage for beginners to put them in high rank for nothing and fall back in a week because they turned out to be not that "top" after all. This will not help to keep them, even if that was the very intention to give initial "good feeling" and make them stay. They won't."
As I think I've stated clearly in the news item and here, this is an inevitable artifact. There is no intention of making anybody feel specially good about themselves. This... mirage, as you put it, is the consequence of us having discarded all but recent scores. If these players have distinguished themselves within days of joining, then they eminently deserve to be recognised. For example, Steven Pletsch got 100 points on the first two puzzles he played, and was in the top 10 on the next one he worked on. It is quite right that he began climbing in the ranks quickly.

"The moving sum to calculate global points and ranks should really take a longer period, like 4 months as it was mentioned."
Again, I cannot help but repeat what I've said before. The global scores are indeed calculated on a 4-month basis, except that we have cut off all scores prior to puzzle 107. You will only notice this in about 3 and a half months. Conceptually, you can imagine discarding all old scores and starting from scratch.

"The moving sum should be calculated the same way for everyone."
Again, the simplest is to repeat myself: it is calculated exactly the same way for everyone. It is calculated in a fair manner, which could not be said of your idea of "the last X puzzles played by Y".

Your third suggestion is sound and original. I don't consider it a high priority, but I will look into seeing how it can be addressed.

admin

Brick's picture
User offline. Last seen 1 year 32 weeks ago. Offline
Joined: 07/15/2008
Groups: Beta Folders

It's not fair, because the new players have two or three times more puzzles available to them, even though it's for a relatively short time. It still gives them an unfair advantage that doesn't go away for 4 months when those drop off. The existing players do not have the ability to take advantage of this loophole created by the rules changes.

It would be the same if the 'alpha' testers (or whatever they are called) get to use their testing puzzle scores - again unfair because not all players have access to those puzzles.

Perhaps all the players that were here and had global scores above 150 before puzzle 107 should have the chance to play the newbie puzzles until they pass the 15 and 150 points - even out the playing field.

Bozonik's picture
User offline. Last seen 10 years 34 weeks ago. Offline
Joined: 10/27/2008
Groups: xkcd

I think we're only noticing this at the moment, because there aren't very many puzzles being considered (just 107, 108, 109, 110 and 111). So the people who've played the <15 and <150 puzzles in two or three of those have an obvious over everyone else.

But that is just because everything was cut off before 107. Once we have a full 4 months of puzzles counting again, those couple of extra puzzles beginners get won't make so much difference.

admin's picture
User offline. Last seen 15 weeks 5 days ago. Offline
Joined: 11/10/2007
Groups: vi users

Brick, you are contradicting no-one. No-one defended the fact that beginners have the <15 and <150 and normal puzzles all potentially available to them if they are clever. What I said on the subject was, and I'll quote here because reading posts before replying to them seems to have gone out of fashion, "Your third suggestion is sound and original. I don't consider it a high priority, but I will look into seeing how it can be addressed."

Why I don't consider it a high priority is best summed in what Bozonik wrote.

Brick's picture
User offline. Last seen 1 year 32 weeks ago. Offline
Joined: 07/15/2008
Groups: Beta Folders

I *did* read the thread, particularly the part where you said "It is not unfair, ..."

So indeed, I am contradicting you, since it is unfair now, and won't be fair for 4 months - longer than most people have played.


admin's picture
User offline. Last seen 15 weeks 5 days ago. Offline
Joined: 11/10/2007
Groups: vi users

Ah, I see, you only read the first part of the thread. It would have prevented a useless interjection if you had read below the part you chose to address.

Brick's picture
User offline. Last seen 1 year 32 weeks ago. Offline
Joined: 07/15/2008
Groups: Beta Folders

You didn't say anything further along about changing anything soon (ever?), so I guess we all have to live with an unfair scoring system.

On any given puzzle, a user who's above got 150 points can only get a maximum of 100 points,  and a newbie could come along and get 300 points, 200 more than your best folder could ever achieve on one puzzle.

How will that ever be fair?

axcho's picture
User offline. Last seen 8 years 43 weeks ago. Offline
Joined: 12/18/2007
Groups: None

One idea that was suggested in chat is to get rid of the <15 <150 puzzles completely and have a separate set of beginner competitions using expired puzzles and top solutions as guides, like the idea that DisposableHeart mentioned in the other thread.

I personally think that it would be very valuable to keep some sort of permanent metric, like experience points, that is not converted into a ranking list but used to gain levels, like in an MMO or a site like Kongregate. We have talked about this idea, and it's very likely that we will implement something like this in the near future.

Joined: 05/19/2008
Groups: None

Well, I figured what the problem is. The news item referred by our Admin did not mention the scores before Puzzle 107 are ignored. He only mentioned it in his response to me:

"Again, I cannot help but repeat what I've said before. The global scores are indeed calculated on a 4-month basis, except that we have cut off all scores prior to puzzle 107."

This little piece of information puts everything in place.

Soon the best players will have enough points that beginners will not perturb the top positions, even though they can play the same puzzle 3 times.

A side note: this moving sum calculation for the 4 months period could have been done on an ongoing basis, not by resetting the whole scoring system.

Another side note: I feel some impatience from our Admin. Excuse me if my impressions deceived me.

Bozonik's picture
User offline. Last seen 10 years 34 weeks ago. Offline
Joined: 10/27/2008
Groups: xkcd

I guess one option that might make everyone happy would be to only count the best score if people make attempts at more than one of 'same' puzzle.

I think the idea of the <15, <150 puzzles is not really for newcomers to get 300 points per puzzle and launch themselves up to the top of the leaderboard, but more to give them a puzzle they have a chance of competing on without having to beat the experienced players straight away? So just counting the best scoring of the three attempts wouldn't hinder that goal.

Also with the new rolling system, probably some day we'll have experienced players who've been inactive for a while coming back and being able to enter the <15, <150 puzzles. With three active puzzles they can do three times, they could potentially score 800+ over those puzzles.

Bozonik's picture
User offline. Last seen 10 years 34 weeks ago. Offline
Joined: 10/27/2008
Groups: xkcd

Sorry, I see I have just reproduced one of DH's original suggestions.

I'll get my coat :duh:

admin's picture
User offline. Last seen 15 weeks 5 days ago. Offline
Joined: 11/10/2007
Groups: vi users

To clear the record before I move this to the feedback tracker: I had posted this information before too, quite clearly. Following DH's example, I will bold out the section:

"Due to the slight changes to scoring that were posted with the latest update, as well as the fixing of the bug that allowed confusion of soloist and evolver scores, we will be phasing out old scores up to the point the update was released. This will take place over a period of about a month, so the transition will be relatively smooth."

admin's picture
User offline. Last seen 15 weeks 5 days ago. Offline
Joined: 11/10/2007
Groups: vi users
Topic: » Server
Status: Array » Open
Type: Array » Suggestion

Set details.

Joined: 06/17/2010
Status: Open » Closed

We have 4mo scoring window, closing as done.

Sitemap

Developed by: UW Center for Game Science, UW Institute for Protein Design, Northeastern University, Vanderbilt University Meiler Lab, UC Davis
Supported by: DARPA, NSF, NIH, HHMI, Amazon, Microsoft, Adobe, RosettaCommons