Page 63 of 72

Re: [Script] WME UR-MP Tracking

PostPosted: Fri Feb 01, 2019 8:33 pm
by iainhouse
This is unlikely anything to do with total numbers of MPs. It sounds like a problem with a high density of MPs in a particular location. It looks like atropicshiner has posted exactly the details needed to investigate. I've been having the same problem myself for a week or so and my console is indicating the scan failing in exactly the same location.

Re: [Script] WME UR-MP Tracking

PostPosted: Sat Feb 02, 2019 8:25 am
by atrophicshiner
Thanks Iain, really thought I was going crazy, as you point out I already had the reason for the failure, the exact location has a stack of at least 499 MPs that is causing this script to stumble.

Re: [Script] WME UR-MP Tracking

PostPosted: Sat Feb 02, 2019 11:18 am
by Twister-UK
As I was concerned that URO might have problems here (it doesn't, only because I'd forgotten that URO backfilling only works on URs :oops: ), I did a little investigation here and I'm pretty sure the stack contains more than 500 markers all located in exactly the same spot - if you reload the WME session with the map view centred on that stack, and with WME zoomed all the way in so that it doesn't pull in the single MP marker located nearby, the stack increases to 500 markers, which seems a little coincidental.

Even if the stack does really contain only 500 markers and not a single one more, having 500 markers located on exactly the same co-ords means that any attempt to get around the 500 marker limit by requesting data from increasingly small areas stands a good chance of failing here - no matter how small the request area becomes, the area containing the co-ords of this point will always return 500 markers...

Re: [Script] WME UR-MP Tracking

PostPosted: Sat Feb 02, 2019 5:40 pm
by iainhouse
OK, UR-MP Devs - we have confirmation for you.

At this location there are a few MPs. I can't be precise, because I lost count, but there are somewhere between 10,000 and 100,000 MPs all at this single point. :o

I've just finished closing them all in batches of 500 at a time. And they are still appearing: it looks like a new MP is being added about once every minute.

Obviously this is a problem with the feed Waze are using to generate these MPs (which are closure warnings). Now I've cleared the stack, it's manageable for a few days. Would you like to experiment in this location before I ask HQ to fix the problem? I expect that, when they do, they will also delete (as opposed to just close) all the MPs.

[EDIT] An AM area can be requested if it will help. Or I'm available for online collaboration :D

Re: [Script] WME UR-MP Tracking

PostPosted: Sat Feb 02, 2019 7:09 pm
by iainhouse
I've had a look at the UR-MP code. As suggested above by Twister-UK, the problem is pretty simple. If a scanned block gives 500 results - the maximum the servers will return - the code subdivides that block into 4 and re-scans. This is a great idea - it lets you scan the map in big chunks and only have to "zoom in" where there are a lot of results.

However, it all falls over if the 500+ results are at a single point, as they are here. The code keeps zooming in but will never get fewer results. Whilst I realise this situation is rare - and probably the result of a Waze mistake - it does happen. This is at least the third time UR-MP has hit this problem in the UK.

Having examined the code, I've also managed to code in a boundary condition that will stop the scan getting stuck. I'm already running a modified version because I incorporated a change someone else suggested a while back to allow filtering on all existing MP types. So I can't provide precise line numbers, but this chunk of code should appear around line 6640:
Code: Select all
    WMEURMPT.scanAreaBoundsList.shift();
    if (MPs != null) {
      if (MPs.hasOwnProperty("error") && MPs.error == 1 ) {
        WMEURMPT.log("Found " + MPs.logMessage.length + " " + MPs.logMessage.obj + ". This is a waze server limit. Enqueuing sub tile...");
        if ( Math.abs(tileBounds.right - tileBounds.left) > 0.01 ) {  //THIS LINE ADDED BY IAINHOUSE
            var newTileBounds = new OpenLayers.Bounds;
            newTileBounds.extend(new OpenLayers.LonLat(tileBounds.left, tileBounds.bottom));
            newTileBounds.extend(new OpenLayers.LonLat((tileBounds.left + tileBounds.right) / 2.0, (tileBounds.bottom + tileBounds.top) / 2.0));
            WMEURMPT.scanAreaBoundsList.unshift(newTileBounds);
            newTileBounds = new OpenLayers.Bounds;
            newTileBounds.extend(new OpenLayers.LonLat((tileBounds.left + tileBounds.right) / 2.0, tileBounds.bottom));
            newTileBounds.extend(new OpenLayers.LonLat(tileBounds.right, (tileBounds.bottom + tileBounds.top) / 2.0));
            WMEURMPT.scanAreaBoundsList.unshift(newTileBounds);
            newTileBounds = new OpenLayers.Bounds;
            newTileBounds.extend(new OpenLayers.LonLat(tileBounds.left, (tileBounds.bottom + tileBounds.top) / 2.0));
            newTileBounds.extend(new OpenLayers.LonLat((tileBounds.left + tileBounds.right) / 2.0, tileBounds.top));
            WMEURMPT.scanAreaBoundsList.unshift(newTileBounds);
            newTileBounds = new OpenLayers.Bounds;
            newTileBounds.extend(new OpenLayers.LonLat((tileBounds.left + tileBounds.right) / 2.0, (tileBounds.bottom + tileBounds.top) / 2.0));
            newTileBounds.extend(new OpenLayers.LonLat(tileBounds.right, tileBounds.top));
            WMEURMPT.scanAreaBoundsList.unshift(newTileBounds);
            WMEURMPT.scanAreaBoundsCount += 4;
        }  //THIS LINE ADDED BY IAINHOUSE
      } else {
        WMEURMPT.log("Found: " + (MPs.hasOwnProperty("mapUpdateRequests") ? MPs.mapUpdateRequests.objects.length + " URs; " : "") + (MPs.hasOwnProperty("problems") ? MPs.problems.objects.length + " MPs; " : "") + (MPs.hasOwnProperty("mapComments") ? MPs.mapComments.objects.length + " MCs; " : "") + (MPs.hasOwnProperty("venues") ? MPs.venues.objects.length + " PURs" : ""));
        WMEURMPT.updateURList(MPs);
        WMEURMPT.updateMPList(MPs);
        WMEURMPT.updateMCList(MPs);
        WMEURMPT.updatePURList(MPs);
        WMEURMPT.removeOldURMP(MPs.area, MPs.filterType, MPs.tile);
      }
    }

I've "wrapped" a chunk of that code with an if statement - the two additional lines have //THIS LINE ADDED BY IAINHOUSE tacked on the end of them. I've just chosen an arbitrary cut off after the original tile has been sub-divided 6 times.

Hopefully this will help those of you facing problems at the moment, until the script itself can be updated. :ugeek:

Re: [Script] WME UR-MP Tracking

PostPosted: Sat Feb 02, 2019 7:36 pm
by Kkervinjones
iainhouse wrote:Having examined the code, I've also managed to code in a boundary condition that will stop the scan getting stuck. I'm already running a modified version because I incorporated a change someone else suggested a while back to allow filtering on all existing MP types. So I can't provide precise line numbers, but this chunk of code should appear around line 6640:


Thanks for this Iain, I can confirm it works for me now with this alteration. I can also confirm it's line 6645 in vanilla UR-MP.

Re: [Script] WME UR-MP Tracking

PostPosted: Thu Feb 28, 2019 2:52 am
by tenetienne
Hopefully this is an easy one after I've just been reading the last couple of pages here in the thread.
I have two URs directly on top of each other and the one that UR-MP shows as not having a response yet is not the one that gets selected. It selects the one with the response. I now have the one I need on top so I can select it manually, but I'm wondering if there is anything I can do about this happening in the future, so that it doesn't happen.
https://i.imgur.com/O8tCy3o.png

Thanks!

Re: [Script] WME UR-MP Tracking

PostPosted: Thu Feb 28, 2019 2:54 am
by subs5
tenetienne wrote:Hopefully this is an easy one after I've just been reading the last couple of pages here in the thread.
I have two URs directly on top of each other and the one that UR-MP shows as not having a response yet is not the one that gets selected. It selects the one with the response. I now have the one I need on top so I can select it manually, but I'm wondering if there is anything I can do about this happening in the future, so that it doesn't happen.
https://i.imgur.com/O8tCy3o.png

Thanks!

You can unstack with URO+ or URComments Enhanced

Re: [Script] WME UR-MP Tracking

PostPosted: Sat Mar 02, 2019 1:17 am
by tenetienne
You can unstack with URO+ or URComments Enhanced

I use that quite often, thanks though. I'm more curious about the fact that selecting the link directly within UR-MP doesn't take me to the exact UR. When I look at the PL of the UR in UR-MP it shows me the correct UR, but then it takes me to the UR on top of the one I actually want.

Re: [Script] WME UR-MP Tracking

PostPosted: Thu Mar 28, 2019 10:29 am
by Caart
Hello,

I always sort URs on Last Comment Date, with "UR age column is last comment age" checked in the settings.

When I comment a virgin UR, the Last Comment Date is setting automatically from "-" to "0", which is good.

But when I comment an already commented UR, the Last Comment Date stays on the old value until I manually re-open the same UR, which is annoying to know if I already checked and asked again details on this UR.

Could you do something about this please ?

Thanks by advance.

Caart