By using this site, you agree to our Privacy Policy and our Terms of Use. Close

Forums - Gaming Discussion - The 11th Annual Greatest Games Event - The Discussion Thread

Mnementh said:
mZuzek said:

I'm mostly done compiling the lists for this year, and about to begin making preparations for the results threads (coming up soon!), and...

I kinda wanna make a question to y'all. I've been thinking of changing up the scoring system for a while now, but never really took it seriously because I wanted to keep it consistent throughout the decade. We're past the first decade now, though, and I've been giving this a lot of thought. I'm experimenting with median-based rankings and I've been surprised by how much it actually shakes up the results, but it's hard to find the right balance, where to draw the line before things start getting too arbitrary. I don't want it to ever feel like I'm personally influencing the results, but I'm interested in the idea of changing up the system as it's led to very repetitive lists over the years... I mean, at this point anyone could just make a fake mockup ranking of VGC's favorite games and it'd be just as easy to believe as the real thing. These lists are so predictable

Anyways, trying to keeping the question simple here.

Are you interested in a reworked scoring system, or do you feel things should be kept as they are?

Edit: also, the Official Thread will be locked up in a bit as I'm done compiling and will not receive any further lists from now on.

Be careful in reworkign the scoring, but some change would not be wrong. As I see it, the goal of a scoring system for this should be:

  • boost games that often make a list
  • boost games that reach high positions

I'd be invested in seeing this kind of stats. 
Or highlight the games that released that year and showcased how they ranked. 

Sort of graphical summaries of specific stats. 



Around the Network

If you think you have come up with a better system, I say go for it.

But if it is just change for the sake of change, then I think that is probably not a good idea.



Switch Code: SW-7377-9189-3397 -- Nintendo Network ID: theRepublic -- Steam ID: theRepublic

Now Playing
Switch - Super Mario Maker 2 (2019)
Switch - The Legend of Zelda: Link's Awakening (2019)
Switch - Bastion (2011/2018)
Switch - Mario Kart 8 Deluxe (2014/2017)
3DS - Star Fox 64 3D (2011)
3DS - Phoenix Wright: Ace Attorney (Trilogy) (2005/2014)
Wii U - Darksiders: Warmastered Edition (2010/2017)
Mobile - The Simpson's Tapped Out and Yugioh Duel Links

Mnementh said:

Be careful in reworkign the scoring, but some change would not be wrong. As I see it, the goal of a scoring system for this should be:

  • boost games that often make a list
  • boost games that reach high positions

The current system does that kinda already. At least I think it does. The current system is something along the line of 1 point for #100 and 100 points for #1 and adding it all up, right?

Median is very risky. Someone may have an obscure game he likes very much near the top of his list, but nobody else has the game (because nobody else played it). Another game might be well known, well liked and often played by many, so it enters a lot of lists, but not always at the top. The second game has a lower median than the first, and that is something you probably don't want.

I understand that many are frustrated seeing the same games everyone knows and kinda likes at the top, while more obscure games have no chance. But instead of using a median, I would simply give some bonus points for top-placements. Say 1 point extra for TOP 50 games, 2 points for TOP 25, 3 points for TOP 10, 5 points for TOP 5. This would help elevate the games reaching top-placement in some lists rise above often named but lower ranked games. But be careful not to overdo it.

I definitely agree with the points being made here, though you seem to have confused the numbers. Adding 1 extra point for the top 50 highest games in each list... That's all of them :D



Try out my free game on Steam

2024 OpenCritic Prediction Leagues:

Nintendo | PlayStation | Multiplat

UnderwaterFunktown said:
Mnementh said:

Be careful in reworkign the scoring, but some change would not be wrong. As I see it, the goal of a scoring system for this should be:

  • boost games that often make a list
  • boost games that reach high positions

The current system does that kinda already. At least I think it does. The current system is something along the line of 1 point for #100 and 100 points for #1 and adding it all up, right?

Median is very risky. Someone may have an obscure game he likes very much near the top of his list, but nobody else has the game (because nobody else played it). Another game might be well known, well liked and often played by many, so it enters a lot of lists, but not always at the top. The second game has a lower median than the first, and that is something you probably don't want.

I understand that many are frustrated seeing the same games everyone knows and kinda likes at the top, while more obscure games have no chance. But instead of using a median, I would simply give some bonus points for top-placements. Say 1 point extra for TOP 50 games, 2 points for TOP 25, 3 points for TOP 10, 5 points for TOP 5. This would help elevate the games reaching top-placement in some lists rise above often named but lower ranked games. But be careful not to overdo it.

I definitely agree with the points being made here, though you seem to have confused the numbers. Adding 1 extra point for the top 50 highest games in each list... That's all of them :D

You're right. I somehow thought about a TOP 100.



3DS-FC: 4511-1768-7903 (Mii-Name: Mnementh), Nintendo-Network-ID: Mnementh, Switch: SW-7706-3819-9381 (Mnementh)

my greatest games: 2017, 2018, 2019, 2020, 2021, 2022, 2023

10 years greatest game event!

bets: [peak year] [+], [1], [2], [3], [4]

mZuzek said:

I'm mostly done compiling the lists for this year, and about to begin making preparations for the results threads (coming up soon!), and...

I kinda wanna make a question to y'all. I've been thinking of changing up the scoring system for a while now, but never really took it seriously because I wanted to keep it consistent throughout the decade. We're past the first decade now, though, and I've been giving this a lot of thought. I'm experimenting with median-based rankings and I've been surprised by how much it actually shakes up the results, but it's hard to find the right balance, where to draw the line before things start getting too arbitrary. I don't want it to ever feel like I'm personally influencing the results, but I'm interested in the idea of changing up the system as it's led to very repetitive lists over the years... I mean, at this point anyone could just make a fake mockup ranking of VGC's favorite games and it'd be just as easy to believe as the real thing. These lists are so predictable

Anyways, trying to keeping the question simple here.

Are you interested in a reworked scoring system, or do you feel things should be kept as they are?

Edit: also, the Official Thread will be locked up in a bit as I'm done compiling and will not receive any further lists from now on.

I do think there are problems with the rankings and I have a couple of suggestions at fixes/changes.  But I first want to say that predictability is not a bad thing in and of itself.  In fact, since we are talking about the best games ever made then it really shouldn't change too much from year to year.  For example, while I haven't even played either Ocarina of Time or GTA5, I would expect both games to rank fairly high.  If this year Ocarina didn't even make the top 50, then I'd be calling foul even though I personally have no stake in the game.  At the same time, I think a current problem we have is that maybe third party games like GTA5 are not scoring high enough.  If we could somehow come up with a perfect system though, then I wouldn't want it to change much from year to year.  In a good system, the best games of all time this year should look a lot like the best games of all time next year.

So here are two specific issues I have with the rankings and some possible suggestions at a fix:

1) Third party games do not score high enough.  I think the fix here is simple in principle, but I don't know how much work it would put on you.  The simple fix is to automatically combine every possible platform into their score.  Right now GTA5 could be scored on 5 different possible platforms and next year it will be 7.  When the score is split among 5 platforms then this type of game really gets short changed.

2) Mario and Zelda dominate the top slots too much.  Most people end up really loving the Mario and/or Zelda game that was popular when they were a kid.  I really like that many different generations of gamers are participating, but this results in lots of Mario and Zelda games in the top 20, because almost every major entry is there.  I say this as someone who put a Zelda game as their #1 and a Mario game at #6 from the same platform (the popular Mario and Zelda from when I was a kid).  Clearly all of these games deserve to be recognized to some extent, and that is why they are so loved, but maybe not so many Marios and Zeldas deserve to be in the top 20.

My suggested fix is to give all games more points just for making it into anyone's top 50.  IIRC last years method was that a #1 game got 60 points and a #50 game got 11 pts.  Is that correct?  I would try something more like #1 game gets 100 pts and #50 game gets 51 pts.  Or if you want to be even more extreme then #1 game gets 150 pts and #50 game gets 101 pts.  I think this will dilute some of the Mario and Zelda problem.  I don't think all of these games are actually appearing more frequently on lists than every other game out there.  Instead, I think most people have their favorite Mario and Zelda and rank them really highly.

If you make these two changes then I think it will fix the problems with the lists.  Games like GTA5, Minecraft, Dark Souls, etc... will end up moving higher in the rankings.  I suspect there are several third party games that are actually popular with the community here, but they have been ending up with a worse rank just because of how the scoring system is structured.

EDIT: I just want to add, that in light of everything I've just said, there is a lot about the current system to be praised.  One of my pet peeves with other "Top game of all time" lists is when most of the top games were released in the past 10-15 years.  Another thing that irritates me is when 1-2 consoles are represented far more than all of the others.  I don't think the VGChartz system falls into either of these traps.  In making changes we want to hold on to the parts that are good while fixing the parts that are bad.

Last edited by The_Liquid_Laser - on 19 January 2021

Around the Network
The_Liquid_Laser said:
mZuzek said:

I'm mostly done compiling the lists for this year, and about to begin making preparations for the results threads (coming up soon!), and...

I kinda wanna make a question to y'all. I've been thinking of changing up the scoring system for a while now, but never really took it seriously because I wanted to keep it consistent throughout the decade. We're past the first decade now, though, and I've been giving this a lot of thought. I'm experimenting with median-based rankings and I've been surprised by how much it actually shakes up the results, but it's hard to find the right balance, where to draw the line before things start getting too arbitrary. I don't want it to ever feel like I'm personally influencing the results, but I'm interested in the idea of changing up the system as it's led to very repetitive lists over the years... I mean, at this point anyone could just make a fake mockup ranking of VGC's favorite games and it'd be just as easy to believe as the real thing. These lists are so predictable

Anyways, trying to keeping the question simple here.

Are you interested in a reworked scoring system, or do you feel things should be kept as they are?

Edit: also, the Official Thread will be locked up in a bit as I'm done compiling and will not receive any further lists from now on.

2) Mario and Zelda dominate the top slots too much.  Most people end up really loving the Mario and/or Zelda game that was popular when they were a kid.  I really like that many different generations of gamers are participating, but this results in lots of Mario and Zelda games in the top 20, because almost every major entry is there.  I say this as someone who put a Zelda game as their #1 and a Mario game at #6 from the same platform (the popular Mario and Zelda from when I was a kid).  Clearly all of these games deserve to be recognized to some extent, and that is why they are so loved, but maybe not so many Marios and Zeldas deserve to be in the top 20.

I feel this contradicts a bit your opening statement regarding predictability and the top spots no changing much from year to year.  

While I agree with the sentiment, "not so many Marios and Zeldas deserve to be in the top 20" is not for any one person to decide. One thing that this particular message board has shown me is that the Zelda franchise is far more divisive than I originally would've expected. Skyward Sword wouldnt make my top 100, but its in the top 25 of many people. Its not for me to say it shouldnt be there just because its not there for me. 



mZuzek said:
The_Liquid_Laser said:

I do think there are problems with the rankings and I have a couple of suggestions at fixes/changes.  But I first want to say that predictability is not a bad thing in and of itself.

1) Third party games do not score high enough. I think the fix here is simple in principle, but I don't know how much work it would put on you.  The simple fix is to automatically combine every possible platform into their score.  Right now GTA5 could be scored on 5 different possible platforms and next year it will be 7.  When the score is split among 5 platforms then this type of game really gets short changed.

2) Mario and Zelda dominate the top slots too much.  Most people end up really loving the Mario and/or Zelda game that was popular when they were a kid.  I really like that many different generations of gamers are participating, but this results in lots of Mario and Zelda games in the top 20, because almost every major entry is there.  I say this as someone who put a Zelda game as their #1 and a Mario game at #6 from the same platform (the popular Mario and Zelda from when I was a kid).  Clearly all of these games deserve to be recognized to some extent, and that is why they are so loved, but maybe not so many Marios and Zeldas deserve to be in the top 20.

My suggested fix is to give all games more points just for making it into anyone's top 50.  IIRC last years method was that a #1 game got 60 points and a #50 game got 11 pts.  Is that correct?  I would try something more like #1 game gets 100 pts and #50 game gets 51 pts.  Or if you want to be even more extreme then #1 game gets 150 pts and #50 game gets 101 pts.  I think this will dilute some of the Mario and Zelda problem.  I don't think all of these games are actually appearing more frequently on lists than every other game out there.  Instead, I think most people have their favorite Mario and Zelda and rank them really highly.

If you make these two changes then I think it will fix the problems with the lists.  Games like GTA5, Minecraft, Dark Souls, etc... will end up moving higher in the rankings.  I suspect there are several third party games that are actually popular with the community here, but they have been ending up with a worse rank just because of how the scoring system is structured.

EDIT: I just want to add, that in light of everything I've just said, there is a lot about the current system to be praised.  One of my pet peeves with other "Top game of all time" lists is when most of the top games were released in the past 10-15 years.  Another thing that irritates me is when 1-2 consoles are represented far more than all of the others.  I don't think the VGChartz system falls into either of these traps.  In making changes we want to hold on to the parts that are good while fixing the parts that are bad.

I disagree with your idea that Mario and Zelda games will drop down on the list if there are more points given just for appearing on a list. These games rank highly exactly because they get so many votes, if anything, they'd only dominate the list even more like that. As a quick example, if I list the games strictly by the amount of votes they got, there are 10 Mario/Zelda games in the top 12. Not only that, but the end result actually looks very similar to how it looks when ranking them by the traditional scoring system.

Also, I don't think there's any real fix for the issue of third-party games, at least not without a major change in the scoring system. Third-party games do actually have their scores combined across all platforms, that's not the issue. I think it generally boils down to exclusives getting a lot of attention, more than the average third-party games get, so they tend to get a lot more votes. Like, if someone primarily games on Nintendo systems, it's likely that this person will have a relatively large number of Nintendo exclusives on their list - and because the selection of Nintendo exclusives is a lot smaller than the sea of third-party games out there, there's usually far more overlap with exclusives than with third-parties, thus leading the third-parties to get fewer votes in general. Honestly, I'm not sure there's a fix for this with any scoring system.

Anyways, I suppose I agree with you on predictability not being necessarily a bad thing. I'm just tired of these lists looking the same every year, and every year they lead to the same complaints about too many Mario/Zelda games at the top. It's been an issue ever since the beginning of the event, really, and one could argue that it's not really an issue, but personally I think it is. Whenever I go through the top 100 in the results thread, it always bothers me when I get near the top of the list and suddenly it feels like all variety's been thrown right out the window.

Hmmm...it may be that there is no easy fix these issues.  I suspect that Mario and Zelda games also rank high in people's lists as well as being frequent.  For example if you just look at the top 10 games from every person's list then would Mario and Zelda still be just as popular?

It sounds like like the main issue is Zelda and Mario taking all of the top spots.  I don't think there are many people out there that think the best 10 games of all time are either Mario or Zelda.  It's more an issue that people have their own personal Marios and Zeldas they love and that ends up putting lots of them at the top.



mZuzek said:

I thought about this idea of giving bonus points for top X, but ultimately concluded it's not too fair because what it does is basically separate every user's lists into pre-defined "tiers", but in reality, each person's ranking has those tiers at different places. For example in my list, I think there's a really small gap between #10 and #11, with a much bigger gap after #12 or so - so, this scoring system would be creating a gap where I personally don't feel there is one.

About the median experiment, well, I am trying to find the right balance. Obviously making the whole ranking based on the median alone isn't gonna turn out any good, I've already seen what the list looks like with that method and it's quite silly. Instead, what I went with yesterday as a test was multiplying the median by the number of votes it got, and the results looked a lot more fitting like that. Different from what they've been in the past, but in a really good way, with some of the more beloved games by not-so-many-people climbing up the list above some games that just get loads of votes but not as high. These games that got loads of votes were still ranked quite highly by this method.

It's not the definitive method, because I did find some issues with it (games ranked #1 with only 1 vote were still ranked too high). I think I'll try to make the votes a more important factor somehow, but as an initial test, I like where it's going.

Reading through these replies, everyone seems to be on board with whatever I come up with so I think I will change it, then. Just gonna try to find the right balance in the next couple of days.

This definitely seems more reasonable than what I thought and when you think about it it's not so different from the old system, assuming I've understood it right. Before it was the sum which is also equal to average*number of votes. Now instead it's median*number of votes.

The difference would be that games than get most high rankings and a few lower ones would get more points since their median is higher than their average and the opposite for games that get mostly lower rankings and a few high ones. Games that get ranked #1 on a single list should get the same number of points as before though, so I'm not sure I necessarily see that as a problem. I would assume they still wouldn't place too high.



Try out my free game on Steam

2024 OpenCritic Prediction Leagues:

Nintendo | PlayStation | Multiplat

mZuzek said:
UnderwaterFunktown said:

This definitely seems more reasonable than what I thought and when you think about it it's not so different from the old system, assuming I've understood it right. Before it was the sum which is also equal to average*number of votes. Now instead it's median*number of votes.

The difference would be that games than get most high rankings and a few lower ones would get more points since their median is higher than their average and the opposite for games that get mostly lower rankings and a few high ones. Games that get ranked #1 on a single list should get the same number of points as before though, so I'm not sure I necessarily see that as a problem. I would assume they still wouldn't place too high.

With just the strict median/votes formula, games that got 1 vote at #1 were all ranked at the bottom end of the top 40. Personally, I don't feel any game with only 1 vote should be in the top 100 of a community ranking, maybe not even 2 or 3 votes honestly. So I'm trying to figure out a fair solution, but it's gonna take a lot of thought, because this method leads to an otherwise good ranking.

If you want to weight games more by getting a good rank then you could do something like this.  Give 50 extra points every time a game scores in the top 10 of a person's list.  I am thinking normally a game gets 1 pt for #50 and 50 pts for #1.  Using this method points would look like this for each person's list.

#50, 1pt
#49, 2pt
#48, 3pt
...

#11, 40pt
#10, 91pt
#9, 92pt
...

#2, 99pt
#1, 100pt

This would boost games that tend to get ranked highly on a several lists, but it wouldn't overly favor a game that only got a couple of votes.  Another thing with this method is that it still makes every person's list still worth the same amount.  On the other hand if you start doing things like multiplying by a median, then the lists of some people are going to have more weight than others.



mZuzek said:
UnderwaterFunktown said:

This definitely seems more reasonable than what I thought and when you think about it it's not so different from the old system, assuming I've understood it right. Before it was the sum which is also equal to average*number of votes. Now instead it's median*number of votes.

The difference would be that games than get most high rankings and a few lower ones would get more points since their median is higher than their average and the opposite for games that get mostly lower rankings and a few high ones. Games that get ranked #1 on a single list should get the same number of points as before though, so I'm not sure I necessarily see that as a problem. I would assume they still wouldn't place too high.

With just the strict median/votes formula, games that got 1 vote at #1 were all ranked at the bottom end of the top 40. Personally, I don't feel any game with only 1 vote should be in the top 100 of a community ranking, maybe not even 2 or 3 votes honestly. So I'm trying to figure out a fair solution, but it's gonna take a lot of thought, because this method leads to an otherwise good ranking.

It's possible I've just misunderstood the system, but that sound extremely odd to me. Those games would have 50 points right? Assuming 1st place = 50, which multiplied by their one vote should still be 50. Then shouldn't a game a game with a median ranking of let's say 30 only need 3 votes to beat that? 30th place = 21 points multiplied by 3 = 63. I don't see how there could only be around 40 games higher than 50 points if that was the case.



Try out my free game on Steam

2024 OpenCritic Prediction Leagues:

Nintendo | PlayStation | Multiplat