Page 1 of 3

Revisiting Lock Level Standards - Regional

Posted: Sat Jan 10, 2015 4:24 am
by Fredo-p
REVISITING LOCK LEVEL STANDARDS

With the recent map vandalism that went on on TX, the discussion of heightening lock levels for MH and mH has come up. Many are suggesting to set them to R4.

As it may take some time for a national standard to be reached, it is important for the regions to also take part in the discussion and get the ball rolling. The Great Lakes Region has already been in full discussion with this. The thread can be found here.

Let's get this discussion going and figure out what this region thinks. Invite all your local editors in on this discussion to get everyones opinion.

Re: Revisiting Lock Level Standards - Regional

Posted: Sat Jan 10, 2015 7:51 pm
by dbwiddis
Good conversation, and I'm generally opposed to increasing lock levels. I've been a level 3 "area manager" for 2 years and have been increasingly frustrated at my inability to do simple edits on arterials in my own city, having to resort to asking for unlocks or bugging the heck out of the other editors in my area. Which makes this point very salient:
DwarfLord wrote:There are two sides to the locking coin. One is the lock level. The other is our process for promotion, demotion, and blocking. They go together; we can't talk about one without the other.

Saying that lock levels are too low is identical to saying that promotion is too easy, demotion too rare, and blocking too hard.

I would rather see us evolve better strategies for promotion, demotion, and blocking than to respond to isolated events by placing more of the map out of reach of good editors.
This. This is the key.

From what I understand, only the level 1 to level 2 promotion is currently automatic. In order for someone to become level 3, they need to have actually communicated with someone and demonstrated some level of competence or knowledge of the wiki, etc. I have no problem with locking mH or higher at level 3. But as soon as you talk about locking something at level 4, you're saying you don't trust an Area Manager to manage his/her own area, typically including arterials that he/she is very familiar with. That sort of heavy-handed lack of trust will drive away well intentioned, good editors who haven't accumulated 100K edits to get level 4.

Which is another issue. We need to totally deemphasize number of edits as the measure of editing ability/skill/experience. It may have been appropriate in the early days of fixing the auto-import to a usable map, but once a city has been "fixed up" the number of edits turns into the opposite measure... encouraging someone to tweak stuff that doesn't need tweaking, add more unnecessary PLRs (since there aren't any other roads needing adding), and in general do a lot of editing work that is less important than the NEEDED work (responding to MPs and URs, updating Places, adding house numbers) that generates far too little "edit count". Which makes me agree 100% with this comment:
JJohnston84 wrote:Bad Behavior
As someone who recently got Level 3, and only from mentorship and map raid promotion - not even edit count, this change in my area (Los Angeles) would make me hang up my hat and leave Waze map editing. It is, 100%, the wrong approach to solving this problem.

1.) It won't stop vandalism. It's only a matter of time before a level 4, 5, or 6 goes off the rails and destroys something in a big way. What then, bump major highways to level 5? Keep escalating?
2.) It encourages bad edits. I could sit around wiggling segments to get my edit numbers up. Better yet, I could write a script to do it for me. But I don't, because at least I don't feel it necessary to actually get my level to the place I need it to do meaningful edits. Make it so I need 100k to effectively reach what is, right now, level 3, and such bad behavior is far more tempting.
3.) It alienates smart, well-intentioned editors from participation.
What we can do as a community, immediately, is change our procedures for promotion. Align levels with some measure of experience/participation/community that is unrelated to edit count, but is more appropriate to the "level of trust" we expect people to have before editing roads with higher importance.

Not only that, but totally get rid of the emphasis on edit count. Seriously. It's counter-productive. It encourages mass edits. It encourages useless edits. It encourages cheating. It discourages doing the real, slow, hard work that actually improves the map. Get rid of the shield icons. Get rid of the tie between edit count and promotions. Completely. Not even the automatic level 2 at 1000 (possibly bad) edits.

The ideal solution is out of our hands, but in the hands of developers:
JJohnston84 wrote: it is complete insanity that Waze isn't exposing complete revision history to the relevant parties. Rolling back should be in the toolset. Transactions should be recorded.
In a lot of ways, Waze is like wikipedia. Everyone is encouraged to edit. People "own" certain pages (areas) and have the power to "undo" bad changes. Some pages can be "locked" to prevent vandalism. Unfortunately for us, Waze has not put in fundamental, basic procedures to prevent vandalism. And it should not be OUR overriding concern as editors to stop it, at the cost of making more work for ourselves, and making it harder to recruit and train additional editors (leading to more work for ourselves). No... if road deletion/vandalism is a concern to a company making money with a popular driving app, then it is the for-profit company's responsibility to put in measures to prevent vandalism, not the responsibility of a volunteer community.

Vandalism will happen no matter what we do to prevent it. Waze needs better recovery procedures. If we lock more important roads, people can/will just vandalize the lower locked roads, and perhaps more of them. Think the recovery workload for volunteer editors. will be any easier rebuilding neighborhoods than a long, straight stretch of an arterial? Not likely.

But back to what we can do as a community, this is spot on:
JJohnston84 wrote: 1.) Make mentorship mandatory. No one should hit level 2 without mentor training in my opinion. Make mentorship a formal institution. If you can't communicate with other human beings, we don't want you in the map editor anyway. Likewise, make advancing to the higher levels require mentoring others. You solidify knowledge when you have to start teaching others. This would be a huge win for the community.
Agree 1000%. Level 1 can still be the entry level for "anyone". Level 2 should be a "provisional" temporary level that one can easily get once they jump through a few small hoops that we actually want them to... enroll as a mentee with a mentor, display knowledge that the wiki and forum exists, get in contact (PMs or GHO or whatever) with their local AM/SM/RM, etc. They can learn, edit, get mentored, and then have some sort of "test" of knowledge (when (not) to split roads, functional classification, etc.) in order to be promoted to level 3, where we should TRUST them to actually manage their area and not vandalize it.
JJohnston84 wrote: 2.) Edit counts should be removed from the rank chart on the wiki. It's useful info for someone issuing promotions to have, but it needs to stop being a goal for editors. Rank increases should only come from experienced editors recognizing your understanding and quality of work. For some, that could be in a few edits - others will take thousands. Make having to sell yourself to your peers part of the advancement process. There should be no default edit count to fall back on as a means for promotion.
I would go beyond this and say we should really get rid of the edit count altogether. It's a useless metric anymore. It's too easy to "cheat" and there is no real correlation between editor quality and number of edits. We should get rid of the edit count shields in everyone's signatures.
JJohnston84 wrote: Edit, Another idea: make deletion require 1 rank higher than all other forms of the lock. In other words, if a segment is locked at 2, you would need level 3 to delete. But, all other activities could be done at 2.
Unfortunately this can be easily worked around by a vandal by lowering levels. However, as others have mentioned, some sort of "flag" should prevent mass deletions, or require a particular level to delete independent of the actual lock level of the road. Some restrictions on what level it takes to delete a road could be based on if a segment that has existed longer than X days, or has accumulated Y amount of traffic data on it, or sees Z amount of traffic/routing engine selection per day, etc.
SuperDave1426 wrote:
ply8808 wrote:After seeing firsthand the destruction in this recent issue I feel that a minimum of lock 3 on PS/mH is justifiable, with MH at 4 and Freeway/Ramp at 5.
In urban areas, I agree with you. In rural areas, not so much. :)
This is an important distinction. If "national standards" are developed they need to recognize that locks in rural areas (with fewer (or no) local editors, but less of an impact on overall routing) are different than urban areas (where all collectors are important and the above comment's levels are spot on).
SuperDave1426 wrote:
Locking roads is not the end of editing for junior editors, it is an opportunity to get more involved with senior editors and to gain editing, research, visibility and become great editors with their commitment.
How exactly do they get experience in actual editing if most of the roads are locked above their rank and the ones that aren't don't need editing? You can talk about something until you're blue in the face, but for a LOT of people, actually doing it and applying what is being taught is the way they learn to do it. "Involvement" also means doing editing, not just discussing it. :)
And I think I have an idea that may address that. Have the "area manager" designation and geographical area tied to locking ability. Someone who is a level 2 might become an "area manager" of their hometown, and get level 3 (or even 4, as they display ability) permission in that small geographical area, but still only have level 2 permissions as they drive cross country into other people's territories. This would permit granting of small areas of higher permission... kind of a fast way to let someone display editing ability, even on higher locked roads, in a small scale, controlled manner consistent with their knowledge and experience. The longer someone edits, the more they communicate with their other local AMs, SMs, RMs, etc., the bigger those areas can get.

And this can be tied hand in glove with the formal mentorship program.

Re: Revisiting Lock Level Standards - Regional

Posted: Sat Jan 10, 2015 4:41 am
by DwarfLord
Sorry to hear about the vandalism. I hope something will be written up about it, as it could be a useful case study.

There are two sides to the locking coin. One is the lock level. The other is our process for promotion, demotion, and blocking. They go together; we can't talk about one without the other.

Saying that lock levels are too low is identical to saying that promotion is too easy, demotion too rare, and blocking too hard.

I would rather see us evolve better strategies for promotion, demotion, and blocking than to respond to isolated events by placing more of the map out of reach of good editors.

Re: Revisiting Lock Level Standards - Regional

Posted: Sat Jan 10, 2015 6:26 pm
by DwarfLord
MojaveCactusMonkey wrote:So if an editor is a rank 3 and doesn't communicate at all and got there by mass editing, and not reading the wiki and not communicating with the advanced and senior editors
Such an editor shouldn't be Rank 3 in the first place, and if there already, should be demoted. In the least confrontational way possible, of course, with a clear description of what needs to be demonstrated for (re)promotion. But demoted all the same. Rank 3 should mean something.

To me this is far preferable to putting roads out of reach of all the responsible Rank 3 editors who worked hard to reach that rank, reading the wiki, communicating, and practicing.

Roads should be locked according to the skill, comprehension, and communication level required to edit them, not the fear that an undeservedly-promoted editor will come along and wreck them. Doing otherwise punishes good editors more than it does the clunkers.

Re: Revisiting Lock Level Standards - Regional

Posted: Thu Jan 29, 2015 6:07 pm
by irowiki
I had a thread on this last year or so. This ends up being a catch 22 situation every time! For example, at level 4, I can't approve places that have been submitted by users where the places are attached to level 5 roads... and there's a lot of those in a few cities I've visited because there's either too many roads at 5 that shouldn't be at 5, or not enough level 5 editors.

NM (and AZ I think? Maybe NV?) has been doing Freeway 5, MH 4, mH 3, PS 2-3. Some of it in NM hasn't caught up to the FC thing yet (some SR's being MH now, etc)

So these major roads that were deleted (MH and mH), what were they locked to? As other people have said, where's our rollback, or editing history?

I think the delete lock might have some merit, especially for PS, mH and MH (and Freeway of course). Or how about if it's ranked PS or higher, it has to go into an approval queue just like Places do? The coding is already in place.

<User> Would like to <delete US-50 from XXX to YYY> (Approve or deny?)

Of course, that's for a thread over in the editor forum (or has this been discussed before?)

Even deleting roads sticking around for a month would be helpful (as another layer in the editor) and allow level 3-6's to undelete)



This all boils down to these recent quotes I like:
AlanOfTheBerg wrote:I've read all the previous discussion. I will reiterate my opinion that except for areas of known issues, special circumstances, etc., if we have state managers at L4 whom we are supposed to be trusting with edits across the entire state, that the general standard maximum lock level should be 4. If you have a L4 that you don't trust to edit freeways, then demote them.
DwarfLord wrote:There are two sides to the locking coin. One is the lock level. The other is our process for promotion, demotion, and blocking. They go together; we can't talk about one without the other.
Saying that lock levels are too low is identical to saying that promotion is too easy, demotion too rare, and blocking too hard.

I would rather see us evolve better strategies for promotion, demotion, and blocking than to respond to isolated events by placing more of the map out of reach of good editors.

If everything in a town that mattered was locked at level 3-5, how is a 1-2 ever going to do anything? You end up with a bunch of broken roads and frustrated editors because they can't figure it out. There's a big gulf between "Here start editing, have fun" from Waze, and then the Waze community wanting new editors to be involved.

Is there a way for RC's/LC's to keep track of every level 3, 4, 5 editor? and to track edits on a major scale? Run reports?

Re: Revisiting Lock Level Standards - Regional

Posted: Sat Jan 10, 2015 7:29 am
by JJohnston84
Bad Behavior
As someone who recently got Level 3, and only from mentorship and map raid promotion - not even edit count, this change in my area (Los Angeles) would make me hang up my hat and leave Waze map editing. It is, 100%, the wrong approach to solving this problem.

1.) It won't stop vandalism. It's only a matter of time before a level 4, 5, or 6 goes off the rails and destroys something in a big way. What then, bump major highways to level 5? Keep escalating?
2.) It encourages bad edits. I could sit around wiggling segments to get my edit numbers up. Better yet, I could write a script to do it for me. But I don't, because at least I don't feel it necessary to actually get my level to the place I need it to do meaningful edits. Make it so I need 100k to effectively reach what is, right now, level 3, and such bad behavior is far more tempting.
3.) It alienates smart, well-intentioned editors from participation.

Roll-backs, the right solution to vandalism
I know some of you are not software developers, so the solution to this may not be obvious. But, for those of you who are developers, you should know it is complete insanity that Waze isn't exposing complete revision history to the relevant parties. Rolling back should be in the toolset. Transactions should be recorded.

It'd be utter insanity if I could walk into work and delete all my company's source code. And I can't. I could try, but it would be trivial for any other developer to simply roll back my last changeset and undo all the damage. Waze should be no different. I know there is some nuance in branching/merging data, but this is not an insurmountable obstacle. There is a ton of precedent for dealing with this in the realm of source control, and I don't know why Waze's toolset isn't following suit.

Until we get rollbacks
Here are a few ideas that are better controls to helping alleviate bad edits until we get the tools we need:
1.) Make mentorship mandatory. No one should hit level 2 without mentor training in my opinion. Make mentorship a formal institution. If you can't communicate with other human beings, we don't want you in the map editor anyway. Likewise, make advancing to the higher levels require mentoring others. You solidify knowledge when you have to start teaching others. This would be a huge win for the community.
2.) Edit counts should be removed from the rank chart on the wiki. It's useful info for someone issuing promotions to have, but it needs to stop being a goal for editors. Rank increases should only come from experienced editors recognizing your understanding and quality of work. For some, that could be in a few edits - others will take thousands. Make having to sell yourself to your peers part of the advancement process. There should be no default edit count to fall back on as a means for promotion.

We need help from the Waze devs
Nothing we editors can do actually solves the risk of vandalism. One could easily, with a brand new account, do insane amounts of damage to the Waze map in the period of about a week. I won't detail how in a public forum, but the point is that none of the volunteer editors have the toolset to deal with such vandalism right now. This is a system and infrastructure failure and the only ones that can really fix it are the Waze developers. I'm hoping such a fix is already in the works.

Edit, Another idea: make deletion require 1 rank higher than all other forms of the lock. In other words, if a segment is locked at 2, you would need level 3 to delete. But, all other activities could be done at 2.

Re: Revisiting Lock Level Standards - Regional

Posted: Sat Jan 10, 2015 4:08 pm
by JJohnston84
MojaveCactusMonkey wrote:
Jacob wrote
Edit, Another idea: make deletion require 1 rank higher than all other forms of the lock. In other words, if a segment is locked at 2, you would need level 3 to delete. But, all other activities could be done at 2.
Now I like this idea!!
The more I thought about it, I think that we would need 2 lock levels exposed in the UI: a delete lock level, and an editing lock level. The problem is, you wouldn't want a situation where someone can create a segment by accident and then not delete it (I.e., a level 1 unable to delete his own segment until level 2). This would quickly clutter the map. You also don't want people to be able to simply just lower the lock level and then be able to delete. For example, If locking were set at 2, which meant only a 3 could delete it, then the level 2 editor could just lower it to 1 before deleting. If one merged two segments together - it should inherit the highest delete level defined.

But essentially, how often does one delete non-level 1 segments? It's really rare. Even when Undividing a road, I'm still not deleting segments. What I do need access to on a daily basis, as a level 3, is the ability to connect PLRs, divide, undivide, and modify turn restrictions on MHs. I would be loathe to lose that ability. Thus, I would be totally fine with MHs having a delete capability out of my level range - as long as I could do those other things.

Re: Revisiting Lock Level Standards - Regional

Posted: Sat Jan 10, 2015 5:06 pm
by JJohnston84
I'll just reiterate - does one really think that connecting PLRs to MHs needs to be done by someone at the level of state manager? Because that is the net effect of the proposal on the table.

Re: Revisiting Lock Level Standards - Regional

Posted: Sat Jan 10, 2015 6:02 pm
by JJohnston84
Why not just lock MHs at 4 only after they have been deemed "mature" and otherwise lock at 3?

To me, mature means:
- Every PLR has been connected with at least a stub and turn restrictions set.
- House numbers are completed

My feeling is a lot of these "mature" urban MHs are actually far from complete when you consider those points, at least in Los Angeles

Re: Revisiting Lock Level Standards - Regional

Posted: Sat Jan 10, 2015 10:05 pm
by JJohnston84
taco909 wrote: L4 /= State Manager.
Our SM just happened to be L4 until recently, but we have a LOT of L4 and, as of now, only one SM in California.
Absolutely, I'm aware, I was just pointing out the dichotomy between the simple activity of connecting PLRs and entering house numbers on MHs versus being a state manager - which would both only be available to editors at Level 4.