Page 1 of 2 1 2 LastLast
Results 1 to 20 of 37

  Click here to go to the first staff post in this thread.   Thread: Matchmaking: What do we want it to do?

  1. #1
    Backer Slimsy Platypus's Avatar
    Join Date
    Nov 2012
    Location
    Northeast US
    Posts
    368

    Matchmaking: What do we want it to do?

    In lieu of all the comments regarding matchmaking that have come up with the recent build I thought I would attempt to bring some clarity to the issue via some simple organization. I apologize if this is interpreted as a "duplicate thread" but I do think it is very important that we aid Stoic in addressing what we are seeing with the matchmaking and right now I do not think we have a single thread that has provided any type of clear message. I think when we spell out exactly what we want matchmaking to do for us, we then can have measurable indicators of how effectively it is doing that.

    With that being said, it is my understanding that matchmaking is intended to do the following:

    • Firstly, match players that are bringing similarly progressed teams versus each other (i.e. Team Power)
    • Secondly, match players that have similar Elo rankings to each other
    • Thirdly, match players reasonably quickly (done by keeping all the players in one large matchmaking pool and allowing a degree of "mismatching")


    These have been touched upon in the following threads:
    hey Stoic I have just spend 7 minutes looking for a match
    Concerning Ranks and Power Level and ELO
    A new ladder system

    Seems simple enough, right? Well this was exactly what the initial system set out to do and we learned that there are a couple things we want to prevent the system from doing:

    • Not frequently match players with high Elo against teams that have higher "Team Power" (some players are concerned that frequently getting mismatched may negatively affect their Elo)
    • Experienced players (high Elo) want to be able to train Rank 0 units without being matched at an extreme disadvantage
    • New players need to be prevented from playing veterans that have adjusted their teams to be the most effective against them (which puts the new player at two disadvantages: less effective teams and less experience)
    • Not match players that are at such an extreme Team Power disadvantage, that the match isn't worth playing (for example don't match a Team Power: 0 player versus a Team power: 6 player, the limits here are arbitrary for the sake of an example)
    • Prevent matches from occurring where a player had both a significant Team power and Elo advantage


    These concepts have been touched upon in these threads.
    How about a "play same rank bg" button in the matchmaker
    Always matched to higher ranks
    Matchmaking and rankings needs an overhaul
    New Patch and the New Matchmaking

    If you guys think I missed anything I can go back and edit them in. So, if you would like to provide feedback let's decisively say what we want the clear limits to be here

    For example:
    • From your experience: When using a rank 1 team how much Team Power puts you at a disadvantage where you would rather have not even played that match?
    • Similarly, when using a rank 0 team, how much team power puts you at a disadvantage? Is this number different than when using a rank 1 team?
    • Would you rather wait an extra 5 minutes for a match, or play at a disatvantage? What is the longest you are willing to wait?

    Alternatively, if you think one of the fundemental components of the matchmaking should change, simply state what should be omitted and provide a clear description of how the system your propose works.

    Here's to constructive criticism! Thanks all!
    Last edited by Slimsy Platypus; 03-10-2013 at 10:47 PM.

  2. #2
    Tirean and balnoisi have a couple of suggestions, too.

    One problem I think might crop up under the current system: New players being stomped by experienced players dropping below power 6 to get kills and Renown without risking their rankings. The experienced players aren't doing anything wrong by this, but it may be an unpleasant grind for new players (who, if they don't pay cash, will be playing many games under 6 power).

    I used to think uneven matches were okay in ranked play, but have changed my mind. Personally, I can often hold my own 6 v 8 or 9, but I'd rather not risk my rankings on it. I want the matchmaker to let me avoid mis-matches, either with a toggle or, as balnoisi and Tirean suggested, by coarsening the set of power levels at which ranked matches can occur (e.g., 3, 6, 9, ..., 18).

    A note to later posters: stay on point. This is not a thread about costs or rankings, despite my mentioning them above!

  3. #3
    Senior Member sweetjer's Avatar
    Join Date
    Nov 2012
    Location
    Pacific NW, United States
    Posts
    154
    Great presentation of the topic yet again slim. Personally I think a toggle for matching is a good solution. I don't mind (actually rather enjoy) playing matches where I am disadvantaged as an underdog. I would probably opt mostly for fair matches, but I'd like to set a range as well when searching. I'd say between 2 or 3 powerlevel difference has been manageable for me as long as the player is lower on the elo board.
    that which does not kill you often leaves you handicapped

  4. #4
    The main issue with any solution I can think of is this: there may not always be newbies to play against other newbies. What happens then? They have to play against experienced opponents, or not play at all.

    Some possible solutions I can think of to this problem:

    Give newbies their pick of 3-5 starting teams that are known to be balanced and mostly effective. This could counter experienced players building against new teams, especially if the devs updated or rotated out the possible starting teams to make them harder to build against.

    If the devs have already made bots, a train against bots mode could give noobs some training options. Or it could train them to kill bots and not human players, that's always possible.

    You could do what hero academy does and add a puzzle mode. Banner Saga's one unit per turn gameplay would be pretty badly suited to it though unless I'm missing something.

    If splitting up the matchmaker is acceptable, a rank 0 league would make it hard to build against new players (due to having less options), and give experienced players somewhere to train their rank 0s. Right now the gameplay would be very bland with only four unit types, but I hear four more are on the way.

    Give newbies a set amount of renown and no elo change for each of their first ten games. This way they might get beat up at the start but it wouldn't make any difference to their ranking or renown buildup.

    That's all I've got for now. Just throwing ideas out there to see if something sticks.

  5. #5
    Backer Slimsy Platypus's Avatar
    Join Date
    Nov 2012
    Location
    Northeast US
    Posts
    368
    Quote Originally Posted by sweetjer View Post
    I'd say between 2 or 3 powerlevel difference has been manageable for me as long as the player is lower on the elo board.
    Sweetjer - I just had the most epic "eureka!" moment after reading your comment. If the matchmaking was going to match you against against someone with a higher team power, but only would do so if they had a lower Elo, it could potentially solve the "training Rank 0" and "new player farming" issues in itself!

    In the "training Rank 0" scenario the better player would always have a worse or equal team (never stronger). So even if at a bit of a disatvantage, they would at least have more experience to accommidate (as long as the limits of the differential Power Level are appropriate).

    In the "new player farming" scenario, a new player would not be capable of being matched against someone that has both a stronger team and better Elo than them, preventing players from potentially using them as a stepping stone of sorts. (However, I'm not 100% sure how often they would get matched against a veteran that has perhaps only upgraded their warrors)

    I'm not sure what it would do to the "veteren mismatch Elo concern", as a tradeoff exists there between facing less experienced opponnents and potentially losing more Elo. Either way, that in itself may cure some of the matchmaking symptoms players have been observing! Exciting contribution, however I'm not exactly sure how the wait-time variable would play out here; regarless, it's exciting none the less.
    Last edited by Slimsy Platypus; 03-09-2013 at 06:52 PM.

  6. #6
    Developer raven2134's Avatar
    Join Date
    May 2012
    Location
    Manila, Philippines
    Posts
    1,061
    Some things...

    I have not been too happy as well with the latest iteration of the matchmaker. To summarize, we noticed these things regarding the 2 iterations we have had so far:

    1. The old MM matched team power very well, but this could be exploited so that people who at least knew how to play the game, could match newbies at even power for easy wins.
    2. In the new MM, Elo is given more importance in matching to resolve this, but the issue now is the power gaps are not only too wide, but also that the higher skilled player may also be allowed to field the higher power team. Meaning his opponent may be inferior in both skill and team power.
    3. A solution implemented to discourage low power high skill play for the purpose of stat farming was to remove this from recorded/ranked matches. This also has the effect of having "provisional" matches until you manage to make a rank 1 team.
    4. The New matchmaker coincides with rank 2-3 for masterclasses, which complicates our power matching even at power 6 teams.

    Now, one of the things I think which is still warped by the latest matchmaker is it's measure of skill. While ideally Elo and Team power can be used to counterbalance each other, as has been the idea in order to achieve even "total" power in a match, the problem here is how to validate how much 1 power difference is worth.

    We don't need to worry about 1 power difference below rank 6, so the difference between having full advanced v 5 advanced one basic is less of a concern (ability v no ability). We can focus on the gap of power 6 upward (ability v better ability & +1 stat). When looking at things this way, however, it's still largely impossible to pin down.

    This is because at +power 1, different abilities have varying improved levels of utility (such as the shieldmaster or warmaster for example), and different units have different dynamics even while using their level 1 abilities but with the additional stat point (such as the raidmaster for example).


    To address these concerns, I think we need to dig deeper into how we want to be playing and what seems fair. We have these objectives:

    1. We would like to achieve just evaluation and record of a player's skill
    2. We want close matching, which is also crucial to avoid the P2W perception
    3. We want to protect new players but give veteran players a way to train basic units

    I would like to propose these changes to the MM

    1. A player with more skill and higher team power cannot match with a player inferior in both. Having upper hand in both creates too much of an advantage (and while this may still be winnable for the losing player), I think promoting an even playing ground on at least 1 of these fronts provides a better playing experience.

    Corollary to this, a match will only be made if at least one of these is even or counterbalances.

    2. Elo is only recorded if the match is played within 1 power level gap. (6v7, 7v8, 8v9, etc.). I think rating a player is fairest ceteris paribus. Let's say we follow suggestion 1 and the idea of counterbalancing team power and elo. Player A who has Elo 1500 now matches Player B with Elo 1350, but team power is at 6v8 resectively. Yes, Player A has more experience, but is this match still a fair rating of his skill? Can we guarantee that his rating of skill accurately equates to the advantage of 2 power on the unit/s Player B has and the dynamics that creats?

    For the large part we cannot guarantee it, but we also want to have a bit of flexibility, which is why I would like to suggest only a maximum of 1 power gap is allowed for games that will be rated/recorded.

    3. If we are concerned about the diversity of power now open in the game and the speed/fairness in getting matches, including the 2 suggestions above, one good way to address this is to allow the matchmaker to give us an idea of what kind of teams are searching for matches. I do not know how feasible it is, but if the MM were able to tell us: At what power level would I find a match? The player could then adjust accordingly. How to implement this could be thought out further.

    4. Finding a way for vets to train basic units while protecting new players is choosing the lesser of evils.

    a) Allowing for large power gaps to occur when very high Elo players train at power 1, but to not have these matches recorded...

    Note, kills are the objective and this probably creates conflict.

    b) Matching evenly even when the player is new, speeding up the veteran's training but possibly a lopsided match for the new player.

    C) creating a beginner's league that lasts 25-30 games before entering the versus population, at the risk of some playerbase fragmentation, and the possibility that if no/less new players are on, we end up isolating them/preventing them from matching.
    Last edited by raven2134; 03-09-2013 at 11:44 PM.

  7. #7
    I'm not sure about plausibility, but I HEARTILY suggest implementing #2. I think that underdog games would be MUCH more fun if they can't sabotage ELO; at that point, there literally is nothing to win or lose except a bit of renown and bragging rights. (I, at least, do lust a bit after a screen in which I get renown for all 6 bonuses: kills, win, win streak, daily streak, expert mode, and underdog.) Less frustration in that case, and therefore much more fun.

    That said, why not key ELO to only equal-level matches, period?

    #1 also sounds like a good idea, as it lets lower-renown players have a better chance of beating their higher-renown foes, while giving the high-renown folks an interesting (and rewarding) challenge.

  8. #8
    Senior Member sweetjer's Avatar
    Join Date
    Nov 2012
    Location
    Pacific NW, United States
    Posts
    154
    If we go in the direction of being more discerning in what matches count as ranked I would also suggest that we provide visual feedback when the match is starting indicating that it is ranked or unranked, that way players don't need to learn the intricacies of what counts and doesn't count as a ranked match.
    that which does not kill you often leaves you handicapped

  9. #9
    Superbacker netnazgul's Avatar
    Join Date
    May 2012
    Location
    Belarus
    Posts
    456
    Quote Originally Posted by sweetjer View Post
    If we go in the direction of being more discerning in what matches count as ranked I would also suggest that we provide visual feedback when the match is starting indicating that it is ranked or unranked, that way players don't need to learn the intricacies of what counts and doesn't count as a ranked match.
    Many + to that. I was suggesting some indication on enemy rankings display during matchmaking/battle since pre-launch.
    If you don't know where to put it - put it in the pillage

    Steelhammer Tribune issues collected here
    Some of my Factions games can be observed here
    Also possible streaming at http://www.twitch.tv/netnazgul

  10. #10
    Junior Member epfrndz's Avatar
    Join Date
    Mar 2013
    Posts
    5
    I agree to what the OP has said. Please match us with similar-powered teams. My past few games have been match-ups with higher power teams (I once have been matched with a team with +2 power advantage, my friend on teh otehr hand was unfortunately matched to a +3 power advantage). It resulted in protracted losing streaks which should not happen if the intention of the match-making system was to pit you against similarly skilled players.

    The older match-making system was much better in my opinion.
    And I really do hope that this is not the reason for the exodus of factions players.

    And another thing that is the implementation of the underdog bonus, is in my opinion flawed. It places the burden, of the system's inability to match-up against those with similar skills, on the player. Underdog bonus should be rewarded whether or not the player wins/loses the match. It will incentivize the player to stay and try to play the match.

  11. #11
    Developer raven2134's Avatar
    Join Date
    May 2012
    Location
    Manila, Philippines
    Posts
    1,061
    Underdog bonus is awarded win or lose.

  12. #12
    Junior Member epfrndz's Avatar
    Join Date
    Mar 2013
    Posts
    5
    Really? I had games where it was not awarded to me.

  13. #13
    Developer raven2134's Avatar
    Join Date
    May 2012
    Location
    Manila, Philippines
    Posts
    1,061
    Underdog awards 1 renown for every 2 difference in power level. It is highly possible you won/lost with only a 1 power difference. If it did not award it to you while fulfilling these conditions, then it's a bug that Stoic needs to look into .

  14. #14
    Junior Member epfrndz's Avatar
    Join Date
    Mar 2013
    Posts
    5
    thank you for clarifying . Yes I indeed lost to a +1 power advatage team. It still did feel quite unfair and I hope they revise how they award those underdog points.

    Salamat for your prompt reply.

  15. #15
    Backer Slimsy Platypus's Avatar
    Join Date
    Nov 2012
    Location
    Northeast US
    Posts
    368
    I was just waiting in queue for quite some time with my Team Power: 6 group. After approximately 10 minutes I got matched with a Team Power: 0 group. I understand why this happened but I'm not sure whether this should happen. At least if it does, (as stated by others above) it should not have any implication on your visable ranking, win streaks, or win ratio.

    Also, if and when something get's done to address this in this manor - it is very important that the system be transparent enough that players know whether their matches will be detract from their recorded stats or not. If players win but don't get Elo/recorded-wins they will feel "robbed" if they don't know why, even if it was a one sided match.

  16. #16
    Junior Member caine1138's Avatar
    Join Date
    Nov 2012
    Posts
    10
    Ok, a key factor in my posting this is the current population will require different rulings than a higher quantity will. In order to be the fairest we can, I think we need to keep in mind that a higher quantity of players will allow for more complex matchmaking determiners. And in light of the current population, some of that complexity needs to be toned down. In order to facilitate shorter queue times.

    Ive got an idea of how to make the queue fair, while reducing complexity...

    And more importantly, without splitting the playerbase between adept and inexperienced.

    Its essentially this...

    Build a checkbox, similar to expert mode(perhaps even including it, as "experience mode"). That when checked, makes power level the primary determining factor in matchmaking. When unchecked, the primary determiner will be elo. The primary functions as secondary in both alternative formats, and they share the same queue.

    Tweaks to this framework should result in a fair determination of skill, while allowing players to rank up units in the checked format.
    Last edited by caine1138; 03-11-2013 at 07:05 PM.
    Runes from the Web...

  17. #17
    I really think underdog should also apply to rating as well as team power.
    I think the matchmaker should give feedback on the current queue situation (roughly what power levels and ranking are in the queue) so that you can adjust your team to get a match.

    Matchmaker should report underdog status at the start of the match (I think opponents rating and team power should also be displayed at the match-up screen).

    I really really really think the current version of the matchmaking code should be posted and kept up to date so people always know what it going on.

    As others have mentioned I strongly believe that if you are over matched in team power (more than 2 power) you must not also be over matched in ranking (more than 50)

  18. #18
    I believe I have a solution to the match maker problem that will ensure that a) most games are competitive, b) matches are found quickly for most players, and c) players are rewarded when they are matched against stronger players/teams. I will begin by assuming that we have an accurate Elo rating system that accounts for team power and explain how this can be used for matchmaking. I will then propose a way to modify the current Elo system to accommodate differences in player skill and team power.

    All three conditions can be achieved relatively simply if an accurate measure of a player's chance of winning can be determined. Assume for the moment that Elo ratings accurately determine the probability of a player winning any given match. This probability would be defined as
    P_A = 1/(1 + 10^((R_B R_A)/400)),
    where R_A and R_B are the Elo ratings of player A and B, respectively. Now, place the players in a queue. Take the first player in the queue and assign a weight to each other player based on the expected chance of winning. This weight would be defined using a normal Gaussian distribution, thus the weight for playing against player B would be
    W_B = e^(-x^2/(2σ^2)),
    where x is the difference in Elo rating between the two players and σ = 17 (this value will be justified after the algorithm is explained). These weights are then scaled such that their sum is equal to 1.

    Finally, the server chooses a random number from 0 to 1 to select a match. This selection process can be thought of as placing an area proportional to the weights on a number line and choosing a point on that line at random. The selected match would be against the player whose region was chosen.

    The choice of σ = 17 was made to ensure that most matches are competitive. Assuming that the queue contains a uniform distribution of Elo ratings, the chances of being matched against any other player would correspond to a normal distribution. As such, you would have a 95% chance of entering a match where you are within 2σ of your opponent's Elo rating. Thus, 95% of your matches would be against players with an Elo rating within 34 of yours. This corresponds to a 45% to 55% chance of winning for each player. Of course, these probabilities would be different for a non-uniform Elo distribution, but any matchmaking with a finite queue and finite time will have similar issues.

    If a player with very high or low Elo rating were in the queue, that person may need to wait until he or she reaches the front to find a match. Thus, it will be important to keep the queue as large as possible without imposing a long wait. I suspect that a pool of 10-30 players at any given time will give you a reasonable distribution of Elo ratings.

    For the third condition, the same Elo ratings can be used to assign rewards to the underdog player. I propose that the underdog player be awarded 1 renown per 5% below 50% chance of winning the match. For example, if a player was deemed to have a 30% chance of winning, that player would be awarded an additional 4 renown for completing the match. I would also suggest informing the underdog player the size of this bonus at the beginning of the match.

    Until now I have assumed we have a well functioning Elo system that will account for team power in its calculations. To achieve such a system, I propose a week or two long phase where matchmaking is partially disabled. For this phase, the current matchmaking system can be maintained with an exception for teams of power 5 and 6. These teams will form a new pool in which matches are made randomly regardless of power or Elo rating.

    This will allow the developers to collect data on the effect that a difference in team power of 1 has on the likelihood of winning a match. Let the player with power 5 be player A, while the player with power 6 is B. If the probability of player A winning the match is plotted against the difference in Elo rating, it should appear to be a shifted logistic distribution. This can be fitted using
    f(x) = 1/(1 + 10^((Δ x)/400)),
    where x = R_A R_B and Δ is the fitting parameter. This fitting parameter will then be the effective difference in Elo of running a power 6 team instead of a power 5 team.

    Using Δ, we can adjust the Elo rating of player B used during matchmaking and awarding Elo at the end of matches. For example, a player fielding a power 5 team will have no bonus or penalty applied to his or her Elo rating for matchmaking, but the same player would add Δ to their Elo when playing with a power 6 team.

    When this phase is over, the above matchmaking system can be turned on while using Δ to adjust the Elo ratings of each player and team combination. Initially, the trend from power 5 and 6 would have to be extrapolated to higher and lower powers (i.e. power 1 teams would subtract 4Δ from Elo while power 8 teams would add 3Δ). As the developers collect more data, these adjustments can be fine tuned to more accurately determine the probability of winning a match. These may need to be updated periodically as new units and levels become available. In the end, this will generate a table of Elo adjustments for all possible team powers.

    I'm sure some will find a way to game the system, no matter what we try. However, I believe this solution will prevent most mismatches while keeping wait times low. In addition, incentives can be awarded for those mismatches that do occur that should encourage people to continue playing.

    Thank you for your consideration,
    Benjamin O'Neill, PhD

    --------------------

    tl;dr
    By modifying the Elo and matchmaking systems, I believe we can easily ensure that most (95%) matches are competitive (45-55% chance to win) regardless of Elo rating and team power. Those matches that are not competitive would award an underdog bonus relative to how likely you are to win.

  19. #19
    Backer Grits's Avatar
    Join Date
    Oct 2012
    Location
    East Coast
    Posts
    106
    Thought of a solution to miss-matched matchmaking. When two teams are not and exact match, simply allow the players to back out of the VS screen. Fixed. When there is an exact match, do not allow them to back out. Yay!

  20. #20
    Backer Slimsy Platypus's Avatar
    Join Date
    Nov 2012
    Location
    Northeast US
    Posts
    368
    I think the bottom line may be that we need hard limits on the disparity between Power levels during matchmaking. Prior to this matchmaking change I have never been matched against a player with team power 0, and now I have seen it once and at other times with a team power disparity close to 6. I understand that with the higher ranks the breadth of the matchmaking has to increase to accommidate a more diverse pool of units, but all I've noticed since this change was rolled out is that the matchmaker is taking longer to find me worse matches (just going off my gut probably about 85% of my matches are fine, then I get one that just perplexes me how it ever happened).

    I'm not sure if this is just a result of where my current Elo is or because I have not begun using any rank 2 or higher, but from my experience something doesn't appear to be working well. I suppose high Elo could possibly translate to poor matches, so maybe that's what is happening here (since the Elo:Unit-rank ratio may more heavily favor Elo while highly ranked). If that is the case, perhaps the Elo conversion to Team Power should not be linear?

    Personally for me, I don't want to have to "verify" that the matchmaker has done a good job with a backout check, or after getting poor matches push a button that effectively "matches me better", I just want to system to have reasonable limits on its worse case scenarios. I honestly don't mind if the matchmaker takes 10 minutes to match me every once in a while, as long as I didn't spend that ten minutes to fight a team that was a horrendous match.

Page 1 of 2 1 2 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •