PvXwiki talk:Real Vetting/Provisional Vetting

As an important note, this would not retroactively apply to builds that lost all their votes in the Wikia>Curse transfer (where we did not get to keep our rating database). It would retroactively apply to any builds posted after that transfer though (since they obviously never lost votes). -Toraen (talk) 13:43, 9 September 2018 (UTC)

Innovation
I guess this part should be rewritten slightly to reflect that it's yes/no instead of a scale now. --Krschkr (talk) 14:02, 9 September 2018 (UTC)
 * Done. -Toraen (talk) 14:18, 9 September 2018 (UTC)

Provisional quality category of meta builds
With the provisional quality category tags it would probably make sense to introduce the same system for the meta tag. So, instead of waiting for 5 votes to add the quality category to the meta tag there could be a provisional quality category which alredy kicks in with 2–4 ratings. --Krschkr (talk) 14:02, 9 September 2018 (UTC)
 * That might make the meta tag and our vetting system more complicated than it really needs to be. I'd also say that a meta build that has bypassed vetting (GvG/HA & some PvE builds that split off an already meta build) is clearly not in need of a provisional score. It's well known as effective and we just haven't caught up. I've been envisioning provisional rating as a way for PvX to place a tentative endorsement on niche but still functional builds. I'll be honest that I'm not the best wiki coder though and making the meta tag do even more is kinda daunting though. -Toraen (talk) 14:17, 9 September 2018 (UTC)
 * Makes sense, so I agree that there's no need for a quality tag within the meta tag until 5 ratings are reached. --Krschkr (talk) 14:24, 9 September 2018 (UTC)

Build authors and provisional quality categories
It should be mentioned whether the biased build author counts towards the 2–4 ratings which can constitute the provisional quality category or whether there have to be at least two votes from other people. --Krschkr (talk) 14:02, 9 September 2018 (UTC)
 * Well, we don't enforce that standard on the 'real' vetted builds (the 5 vote minimum can consist of the author and 4 more voters). An obviously biased vote is grounds for challenging/removal of the vote anyway, depending on how blatant it is. Though given the lower threshold for provisional, bias would be harder to combat. I'll have to think a bit on that one. -Toraen (talk) 14:09, 9 September 2018 (UTC)

Universality
I've been thinking of changing this criteria slightly, as some people have misunderstood it and voted low in this criteria in the past when they shouldn't have. Basically, I want to change it so users should only consider unexpected situations/areas within the "scope" of the build. See my comments here for an build example. I'm unsure of how to word it, but I'd like the Universality description to be clearer on what PvX has meant (admins/build masters have always enforced universality based on scope). -Toraen (talk) 05:28, 10 September 2018 (UTC)
 * You'll have to elaborate this a bit more. I'm not certain that I know exactly what you mean. --Krschkr (talk) 17:04, 10 September 2018 (UTC)
 * While obviously less of a problem now with much fewer voters, previously we did have many people who would vote zero in universality on a build designed for a specific farm, run or SC. This would take the rating down to Good which does not reflect how well say, the W/N raptor farmer is at farming raptors. Being able to use a build in a different area is a criteria that made more sense when PvX was new and largely composed of 'generalized' builds (such as the generic 55 HP monk). It doesn't actually make much sense to cripple the rating and visibility of a specialized farming build when there isn't actual cost in changing your skill bar. Equipment can be an issue but is often shared across even wildly different farming builds - and is often less critical to success unless you're talking about 55 or 600/Smite farming (which have dirt-cheap equips anyway). In PvP sections, it hasn't caused too many issues to my knowledge. The problem is mostly due to a difference in how the PvE general and hero sections evolved (which DO need to cover a wide amount of game content) when compared to the farming, speed clear, and running sections. -Toraen (talk) 06:41, 11 September 2018 (UTC)
 * As I'm only interested in the hero, general PvE, PvE teams, PvP and PvP teams categories I didn't think so much about farming builds. The universality rating for those is, I guess, odd to handle. It doesn't make sense to tear down area specific builds with low universality ratings, but if you were to look at a general farming build (55hp?) you'd actually have to give a more universal farming build a lower rating in universality because it doesn't work everywhere as a general build, right? If it still was 2007 I'd vote for making the universality rating optional!
 * Anyway, let's take a look at PvP builds. Here I understood the universality rating as the build's ability to (A) compete with meta builds and (B) compete with common gimmick builds and (C) adapt to situations when your team is not in control of the match and can't enforce the strategy which is suggested by the team build ran. A build which can work quite well against the builds you'd expect to face and doesn't auto fail if the other team screws up your match strategy (i.e. by playing a byob) would receive a high rating; a build which only works against most other builds and has quite some trouble to adapt to certain tactics of the opposing team would receive a lower rating.
 * For the PvE categories I mentioned my impression was that it makes sense to give the universality rating dependant on whether the build works with almost its entire power in the vast majority of content or whether there are areas which noticeably cut down the build's usefulness. An example for this would be a minion master, which is highly effective in most game content but barely good for anything in some areas which lack corpses. I probably wouldn't accept build isn't good enough for hard content as a problem for the universality rating, though, as that's part of the effectivity rating. --Krschkr (talk) 07:37, 11 September 2018 (UTC)
 * I too, would have preferred Universality just go away. I feel that changing its weighting or removing it should probably be a later, separate policy proposal though. The description issue really only negatively affects 3 categories (as you've described for PvP and general PvE, the current description works fine). The generic 55 hp monk should actually get a slightly lower effectiveness and higher universality than comparable farming builds though (I would hope a 55 HP monk isn't tagged as general). The end result if we were to religiously stick to the uni definition here is that nearly every farming build would end up with a good rating (specific builds would have very low uni, most generic builds would have at best 4 effectiveness, which is weighted more than uni) and some would be dropped into trash that shouldn't be (the current rating thresholds are a bit narrow). The userbase in general decided to not do that and tried to match the spirit of the policy rather than the letter, but it's not very clear to new users. ORIGINALLY there was going to be a search system that let you filter based on eff and/or uni (and various other qualities) but it never happened. I think even area-specific ratings were proposed at one point. In the end, the split criteria have just caused headaches for the most part without that payoff so PvX ended up just reducing/eliminating their effect on rating (Eff, Uni, and Ino were all equally weighted in the beginning, which led to a backwards definition of Innovation so we didn't trash Shock Axe/Cripshot/etc.). -Toraen (talk) 08:18, 11 September 2018 (UTC)

General feedback on policy change
Considering the landscape of the game and this wiki is much different than years ago when original vetting policies were formed, this type of change is entirely appropriate and looks at first glance to be headed in the right direction (if not already arrived at its destination). Can’t wait to witness all the yet-to-be unleashed innovation take g-dubs by storm >:D Juniper real (talk) 17:28, 10 September 2018 (UTC) Juniper real (talk) 17:28, 10 September 2018 (UTC)
 * While I agree that provisional ratings are the right direction I think that it's barely ever going to happen again that 5 people rate the same build so it can get a proper tag. There's a build from 2014 with two votes (ok, it's an SC team, that might make it not that interesting for most people) and one from 2015 with only the author's vote. Not to mention the recent influx of new/votewiped pages with usually just one rating. Attempts to lure people into PvX didn't result in an influx of active users who edit pages or rate builds either. Possible solutions would be a rating caretaker (like the PvP meta caretaker) but I don't know anyone who is sufficiently experienced and unbiased for this. Or a lowered rating threshold in general, which we had for a few weeks now which didn't solve the problem actually. We could pay people to test builds (wtf?!). O well, I don't think there's a good solution with the tiny community left to the game. --Krschkr (talk) 17:45, 10 September 2018 (UTC)
 * The rating caretaker position was build masters, which, in the words of one of the founding bureaucrats, were a "failed experiment." I don't think reintroducing them would really help much, even if we could find someone to do it. In the end, more users is the only real solution and it may not be attainable without some serious work into promoting ourselves in other communities to get people interested. I unfortunately lack the time and negotiation skills to make that go over well with other places, especially considering that PvX for much of its history had been pretty toxic. And really I'd want to refresh/condense PvX policies (and probably redesign the main page) to make things less opaque first (I think most of the policies are actually common sense when you get down to their essentials, but I've been here a while and PvX:Policy probably looks intimidating to a new user). Otherwise interested users will continue to bounce off the site really quickly. -Toraen (talk) 07:44, 11 September 2018 (UTC)