By using this site, you agree to our Privacy Policy and our Terms of Use. Close

Forums - Gaming Discussion - The 11th Annual Greatest Games Event - The Discussion Thread

mZuzek said:
The_Liquid_Laser said:

If you want to weight games more by getting a good rank then you could do something like this.  Give 50 extra points every time a game scores in the top 10 of a person's list.  I am thinking normally a game gets 1 pt for #50 and 50 pts for #1.  Using this method points would look like this for each person's list.

#50, 1pt
#49, 2pt
#48, 3pt

#11, 40pt
#10, 91pt
#9, 92pt

#2, 99pt
#1, 100pt

This would boost games that tend to get ranked highly on a several lists, but it wouldn't overly favor a game that only got a couple of votes.  Another thing with this method is that it still makes every person's list still worth the same amount.  On the other hand if you start doing things like multiplying by a median, then the lists of some people are going to have more weight than others.

The thing is...

I want certain votes to have more weight than others.

That is to say, I feel like the average/median of where a game ranks on lists in general, that should be more important than the sheer amount of votes... but only when a game has many votes already. Basically, the more votes a game has, the less each individual vote should affect it; the less votes it has, the more important it'd be for it to get more votes in.

That's my point of view, at least. I think if over 20 people put a game on their lists, that's already enough to determine that it's a popular game among the community, and also enough to have a pretty formed opinion of how the community as a whole feels about the game. From that point on, whether it got 20, or 30, or 40 votes or whatever, I don't think it should matter nearly as much as the average/median ranking of the game (which of course, is still being influenced by every vote).

This is how I've felt since pretty much forever. I always thought about creating some kind of miracle formula that'd properly separate the popular games from the not-so-popular, and then rank the popular games by how much people like them, instead of by how popular. I've wanted to do something like this since my first year doing this, but I'm no math genius so I just stuck with the old formula, and then thought it was fitting to continue using that until the event's first decade came to a close.

I'm still no math genius, but the median/votes thing did look quite nice initially, and I've already figured out how to tweak the formula so as to put more weight on the votes. Then it's just a matter of deciding how much weight is fair. I've been trying to do it without looking at the game names, just the numbers, and am basically trying to fit the ranking within a certain guideline...

Like, every game in the top 20 should have double digit votes; every game in the top 100 should have at least 5 or 6 votes, and so on.

Maybe I'm being biased with this, but I really am trying not to be.

Edit: also, the thing about giving games a specific amount of points based on where they ranked is... really quite difficult to do as things are. The way I compile the lists normally goes like this:

#1, 1pt
#2, 2pts
#3, 3pts...

With #50 getting 50 points, and unlisted games getting 60 points. This is the method Leadified used and I believe it's also how Smeags did it. This is good because it makes the lists easy to compile, as you can just list the numbers as they are. It leads to less mistakes, basically. Right now, it would be a pain to go through all the numbers and change the amount of points for everything I've compiled, but even for future years, it'd inevitably lead to a lot of mistakes if I was writing down numbers that didn't correlate with the numbers on each list. Maybe I could try to think of an automated way of doing it, where it'd automatically turn every 1 into 100 and so on, but I really don't know how to do that.

And then again, this method can also feel arbitrary. How many points should be given for each standing? You have a 51 point gap at the end of the top 10, but does that accurately reflect how each person feels about their own list? Not for me, I feel there's a really small gap there and I could've listed a different game at #10. This is different for each person, and that's why I don't really like the idea of this system.

Heh, this is a lot to reply to, but here goes.

"Maybe I'm being biased with this, but I really am trying not to be."
I don't think you are trying to be biased.  However, you have to be careful using medians, because the results can look really biased (unintentionally).  Here is one example:

Game A has 5 #1 votes and that is it.
Game B has 6 #1 votes and 20 #40 votes.

According to this Game A's median is #1 and Game B's median is #40.  However Game B has more #1 votes.  That means that those 20 other people who voted for the game actually hurt it's ranking.  Game B has far more votes and even more #1 votes and it's still going to get clobbered in the rankings by game A.  Popularity is not just irrelevant here, but it hurts Game B.

On top of that there is an even more important point.  Every vote, by it's nature, is a popularity contest. That's what a vote means.  It means to have a popularity contest.  That does not mean that in our situation we are condemned to a top 20 full of Marios and Zeldas. There are actually ways to solve this problem while having every person's vote count the same (see below).  However, if you do things that clearly negate people's votes, then you are going to create a lot more headaches than you are going to solve.  That is the sort of thing that is going to alienate the regular people who show up to this every year.

So let's talk about Leadified and his method, because that might actually be contributing to the current problem.  I can see now that a low score is good.  However, #50 gets 50pts, and unlisted gets 60pts.  That means just being on someone's list improves it's score by 10.  That means that if you have a lot of Mario/Zelda games that appear in the 30-50 range on many people's lists then they are really going to shoot up in the rankings, even if most of those voters don't really want to see that game in the top 10 and it is on their own list.  So, one simple thing you can do is just make unlisted games worth 51pts instead of 60pts. 

However, I really think games in someone's top 10 should get a lot of weight.  Mnementh was also suggesting something with a similar idea to mine.  I think the method I'm suggesting will give you the sort of results you are looking for, i.e. if a handful of people really love a game than that matters a lot more than a bunch of votes in the 30-50 range.  I am suggesting a bonus of 50 for every game in someone's top 10.  What that looks like with low score being good is this:

#1, 1pt
#2, 2pt
#3, 3pt

...

#10, 10 pt
#11, 61 pt
#12, 62 pt
#13, 63 pt

...

#49, 99pt
#50, 100pt
unlisted, 101pt


I don't really know what kind of program/spreadsheet you are using, but I don't think it would be too difficult to modify it like this.  If you hate the results you can always change it back.  But the thing about this method is that it makes a person's top 10 games really worth a lot.  So if you have a game that is in the top 10 for several people then that will really shoot it up in the rankings even if most people haven't given it a chance.

Last edited by The_Liquid_Laser - on 19 January 2021

Around the Network
mZuzek said:

About the median experiment, well, I am trying to find the right balance. Obviously making the whole ranking based on the median alone isn't gonna turn out any good, I've already seen what the list looks like with that method and it's quite silly. Instead, what I went with yesterday as a test was multiplying the median by the number of votes it got, and the results looked a lot more fitting like that. Different from what they've been in the past, but in a really good way, with some of the more beloved games by not-so-many-people climbing up the list above some games that just get loads of votes but not as high. These games that got loads of votes were still ranked quite highly by this method.

Be aware, that median (by design) does away a lot of information. So two games may have the same number of votes:

A: #1, #2, #3, #10, #10, #10, #10

B: #10, #10, #10, #10, #30, #40, #50

Both games have 7 votes and a median ranking of #10. Yet looking at the single votes they look very differently. The average would have the same problem, but with other constellations.

I think @The_Liquid_Laser has a point, that no game should be worse off, if it is named in one more list, even at lower places. If a game has so far votes of #1, #2, #3, #45, #46, #47, then only one more list with #50 drops the median quite a lot, which probably isn't fixed by having more votes. I fear such a system could be gamed, by users wanting to remove certain games from the top putting it in their lists at the end. So far my TOP 50 consists of games I like, but if the lower places actually may hurt games in the result, I may rethink my approach to ranking, as I probably don't want to hurt some games.



3DS-FC: 4511-1768-7903 (Mii-Name: Mnementh), Nintendo-Network-ID: Mnementh, Switch: SW-7706-3819-9381 (Mnementh)

my greatest games: 2017, 2018, 2019, 2020, 2021, 2022 

10 years greatest game event!

bets: [peak year] [+], [1], [2], [3], [4]

The_Liquid_Laser said:

So let's talk about Leadified and his method, because that might actually be contributing to the current problem.  I can see now that a low score is good.  However, #50 gets 50pts, and unlisted gets 60pts.  That means just being on someone's list improves it's score by 10.  That means that if you have a lot of Mario/Zelda games that appear in the 30-50 range on many people's lists then they are really going to shoot up in the rankings, even if most of those voters don't really want to see that game in the top 10 and it is on their own list.  So, one simple thing you can do is just make unlisted games worth 51pts instead of 60pts.

I agree with this.  All this does is to give an unneeded bonus to games that show up in a lot of lists.  Those games are going to end up with high rankings anyway.  This is probably the biggest factor in hurting games that score high in a few lists, but get completely left off most others.  It would also be a simple change and not involve any complicated math or formulas.



Switch Code: SW-7377-9189-3397 -- Nintendo Network ID: theRepublic -- Steam ID: theRepublic

Now Playing
Switch - The Legend of Zelda: Link's Awakening (2019)
Switch - Bastion (2011/2018)
Switch - Mario Kart 8 Deluxe (2014/2017)
3DS - Pushmo (2011)
Wii U - Darksiders: Warmastered Edition (2010/2017)
Mobile - The Simpson's Tapped Out and Yugioh Duel Links

People just should come to terms with the fact that Nintendo is the best.



Signature goes here!

mZuzek said:

Alright, it's done!


I... gave up, honestly. I tried so many different systems, many of which initially looked good, but all of which ultimately failed at the basic task of the thing: which is to never have a game's score worsened by an additional vote. So, yeah, that's it. I'm going back to the original system, but with a small alteration which will lessen the impact of a game not featuring in a list. This little change will have an impact on the results, but only as little as the change itself. Looking through the list, it's a very minor difference.

Anyways, that's that. After two days of thinking hard about this, I couldn't come up with a definitive solution. Maybe another year, I suppose. Now it's time to move onto the other bits of work to finally get these results out there. Thread will be up in a few days.

I am looking forward to the results and thanks for your work hosting this exciting event each year!



3DS-FC: 4511-1768-7903 (Mii-Name: Mnementh), Nintendo-Network-ID: Mnementh, Switch: SW-7706-3819-9381 (Mnementh)

my greatest games: 2017, 2018, 2019, 2020, 2021, 2022 

10 years greatest game event!

bets: [peak year] [+], [1], [2], [3], [4]

Around the Network
TruckOSaurus said:

People just should come to terms with the fact that Nintendo is the best.

I think it's a close tie between Nintendo and Sony. Nintendo has been around longer and has certainly pushed the envelope more in creativity and innovation, but Sony has just been a powerhouse of consistent quality with tonnes of variety. It's so very hard to chose between Nintendo or Sony first party titles, but I do think Nintendo wins. I concede! 



My Console Library:

PS5, Switch, XSX

PS4, PS3, PS2, PS1, WiiU, Wii, GCN, N64 SNES, XBO, 360

3DS, DS, GBA, Vita, PSP, Android

mZuzek said:

Alright, it's done!


I... gave up, honestly. I tried so many different systems, many of which initially looked good, but all of which ultimately failed at the basic task of the thing: which is to never have a game's score worsened by an additional vote. So, yeah, that's it. I'm going back to the original system, but with a small alteration which will lessen the impact of a game not featuring in a list. This little change will have an impact on the results, but only as little as the change itself. Looking through the list, it's a very minor difference.

Anyways, that's that. After two days of thinking hard about this, I couldn't come up with a definitive solution. Maybe another year, I suppose. Now it's time to move onto the other bits of work to finally get these results out there. Thread will be up in a few days.

I am sure there is some underlying issue, but without looking at the data I can only make blind guesses.  For example, what if next year people could only pick so many games from the same franchise?  Not sure if that would help or hurt.

Thank you for all of the work you are putting into this.  I appreciate and look forward to this year's results.