[Script] Live Map UR Layer v2.24 (20160202)

Discussion for the unofficial, community-developed addons, extensions and scripts built for the Waze Map Editor.

The official index of these tools is the Community Plugins, Extensions and Tools wiki page.

Moderators: Unholy, bextein

Forum rules
Discussion for the unofficial, community-developed addons, extensions and scripts built for the Waze Map Editor.

DO NOT START a new thread unless it is about a new idea. Keep discussion of existing tools within the main thread for that tool.

The official index of these tools is the Community Plugins, Extensions and Tools wiki page.

Re: [Script] Live Map UR Layer v2.15 (20141122)

Postby Twister-UK » Sat Nov 22, 2014 6:24 pm

Thanks to a late night hackathon, 2.15 has turned up a bit sooner than expected...

Getting the routine changes out of the way first:

  • Added filter for URs with/without descriptions.
  • Master enable state is saved between sessions.
  • JS console debug output can be toggled on or off by clicking the version number at the top of the LMUR UI - when output is enabled "(dbg)" is shown after the version.

And now for the biggie.

Major reworking of the data retrieval code...

The LMURs of old would try to request as much data as possible in a single trip to the server (which translated into requests for blocks of data spanning a 1x1 degree area of the map), and would also perform all the required requests in a single tight loop, locking out the rest of the script until all the data had been received. With the growth in the amount of data required to service both the existing UR/MP filtering, plus the significant extra data now required for PUR filtering, this "big hit" approach to data retrieval was starting to struggle, with retrieval errors becoming more frequently experienced by more users.

2.15 adopts a two-pronged approach to this problem. Firstly, and quite sensibly, the data requests are now made for much smaller blocks at a time - 0.2x0.2 degree areas. This cuts down the amount of data returned in each request, making it more likely that the server will be able to return something other than an error. Secondly, each block request is now placed in a download queue, so that the required blocks can be streamed back and processed on an as and when available basis, leaving the script free to do other stuff in the meantime.

The most obvious evidence of this change is that, if you're viewing an area of the map wider/taller than 0.2 degrees, you'll now see each block rendered as it arrives, rather than (as in previous versions) seeing no change at all until everything had been downloaded and then having the whole screen change at once.


It's not all entirely sweetness and light however, as the UI is momentarily blocked each time one of the chunks is returned from the server and LMUR runs its rendering task, so if you notice the occasional click not being registered, or the occasional popup not, umm, popping up, then it's almost certainly because the render task is doing its thing. On my two development systems it doesn't cause any real problems, so I'm hoping it'll be acceptable for other users too - now I've got the new streaming code in place I'll look into improving this behaviour, but with all the problems the older "big hit" retrieval code is now causing, I wanted to release the streaming code as soon as I was happy with its core performance, and worry about the UI side of things later on...

The other glitch I'm aware of and intend to fix sooner rather than later is that enabling any of the script sections, or the master enable itself, doesn't intitiate the data download required to show any of the newly-enabled items - you'll have to persuade the download to begin by dragging or zooming the map view.


Given how much of the core code has been modified with this release, I'd really appreciate as much feedback as possible, especially comparing the networking performance of 2.15 against the previous releases. I fully expect there to be a fair bit of fine-tuning required before it's working nicely for everyone, but I hope the performance is good enough for you all to see the potential of this redesign. Having spent the last week or so living with this code as it went through pre-release shake down, I certainly wouldn't want to go back to the pre 2.15 way of doing things...


Firefox+Greasemonkey version: https://greasyfork.org/scripts/1948-livemap-ur-overlay
Chrome Web Store version: https://chrome.google.com/webstore/deta ... niefljookp
Twister-UK
Beta tester
Beta tester
 
Posts: 2573
Joined: Sat Jan 07, 2012 12:00 am
Location: NW London
Has thanked: 380 times
Been thanked: 2228 times

Re: [Script] Live Map UR Layer v2.14 (20141029)

Postby Twister-UK » Wed Nov 19, 2014 12:14 am

Just a quick status update to say that despite the best efforts of RealLifeTM to minimise the amount of time I've had to spend on Waze this month, some progress has been made towards fixing the performance issues, however it's probably safe to assume there won't be a release until some time next month...
Twister-UK
Beta tester
Beta tester
 
Posts: 2573
Joined: Sat Jan 07, 2012 12:00 am
Location: NW London
Has thanked: 380 times
Been thanked: 2228 times

Re: [Script] Live Map UR Layer v2.14 (20141029)

Postby SuperDave1426 » Tue Nov 11, 2014 9:51 pm

While you're doing all this updating of the script, etc., is there any chance you can make it so that once the "enable" button is checked, that it stays checked, at least for the session? I use this as a way to scan my area (the whole state :)) more efficiently since you can't zoom out far enough in the editor. I'll pan around, see a marker, get in close and enter the editor, then go back to Live Map view to zoom back out and continue scrolling around. As it is right now, I have to keep going back and checking the enable button each time I enter the Live Map view again. I understand that maybe you don't want it to be permanently enabled, and I'm fine with checking it when I first get in to do some editing, but it's kind-of a pain to have to keep turning it back on each time I go back to the LM view.... :-)

Thanks, either way!
Last edited by SuperDave1426 on Wed Nov 19, 2014 7:12 am, edited 1 time in total.
SuperDave1426
Country Manager
Country Manager
 
Posts: 866
Joined: Wed Oct 16, 2013 5:27 pm
Location: Nevada, USA
Has thanked: 79 times
Been thanked: 250 times

Re: [Script] Live Map UR Layer v2.14 (20141029)

Postby ragacs » Tue Nov 11, 2014 9:43 pm

Twister-UK wrote:would have meant rewriting the caching code to allow partial reloads.

As a temporary workaround you could skip requesting the places (with the PUR-s) if user not selected PUR view then later drop and refill the whole cache if user decides to need the PUR-s and turn them on. This way you gain speed when only MP-s and UR-s are needed and the script have to double-request only once.
ragacs
Area Manager
Area Manager
 
Posts: 388
Joined: Thu Apr 25, 2013 8:15 am
Location: Hungary
Has thanked: 525 times
Been thanked: 279 times

Re: [Script] Live Map UR Layer v2.14 (20141029)

Postby ditchi56 » Fri Nov 07, 2014 3:33 pm

Twister-UK wrote:the present code is far from optimal it did at least seem to work well enough to be released into the wild...

No worries, I'm sure you've got more important things to do.
ditchi56
Country Manager
Country Manager
 
Posts: 4966
Joined: Tue May 29, 2012 12:35 pm
Has thanked: 1116 times
Been thanked: 2080 times

Re: [Script] Live Map UR Layer v2.14 (20141029)

Postby Twister-UK » Fri Nov 07, 2014 3:05 pm

Unfortunately yes. I did look at loading only the data required for the current settings, but the way the code handles its internal data cache was written back when there wasn't really a problem loading everything up front regardless, so having different bits of data loaded at different times depending on when the user enables/disables each display mode would have meant rewriting the caching code to allow partial reloads.

Essentially this is what I'll end up having to to anyway, but as noted above it's a fairly major upgrade of the code, so although the present code is far from optimal it did at least seem to work well enough to be released into the wild...
Twister-UK
Beta tester
Beta tester
 
Posts: 2573
Joined: Sat Jan 07, 2012 12:00 am
Location: NW London
Has thanked: 380 times
Been thanked: 2228 times

Re: [Script] Live Map UR Layer v2.14 (20141029)

Postby ditchi56 » Fri Nov 07, 2014 1:25 pm

Twister-UK wrote:This problem has always been lurking in the background ready to strike at times when the Waze servers are bogged down, but it now seems to have been exacerbated by the introduction of PUR support, as each server request is now pulling down a hell of a lot more data than almost ever used to be the case - even in areas where URs are spreading like measles, chances are they're still outnumbered by the sheer volume of PURs being generated now.

There may be a partial quick fix, assuming that PUR's are contributing to the problem. I'm getting this problem with PUR filtering disabled; does LMUR load PUR data regardless of whether it's required?
ditchi56
Country Manager
Country Manager
 
Posts: 4966
Joined: Tue May 29, 2012 12:35 pm
Has thanked: 1116 times
Been thanked: 2080 times

Re: [Script] Live Map UR Layer v2.14 (20141029)

Postby Twister-UK » Fri Nov 07, 2014 12:56 pm

Re your point about it trying again when you click OK, this isn't the case - if a block fails to download then LMUR moves onto the next block in the list, so if you're seeing multiple error messages it means the server has failed to serve up multiple different blocks.

Once you've clicked through all the errors for a given set of download requests, LMUR shouldn't then attempt any further downloads unless you move the map beyond the bounds of the already cached data area (i.e. the yellow-highlighted region), or it attempts an auto-refresh of the central block in the current view - this occurs every 300 seconds - so users shouldn't end up being prevented from manually unticking the enabled box.
Twister-UK
Beta tester
Beta tester
 
Posts: 2573
Joined: Sat Jan 07, 2012 12:00 am
Location: NW London
Has thanked: 380 times
Been thanked: 2228 times

Re: [Script] Live Map UR Layer v2.14 (20141029)

Postby Twister-UK » Fri Nov 07, 2014 12:40 pm

This problem has always been lurking in the background ready to strike at times when the Waze servers are bogged down, but it now seems to have been exacerbated by the introduction of PUR support, as each server request is now pulling down a hell of a lot more data than almost ever used to be the case - even in areas where URs are spreading like measles, chances are they're still outnumbered by the sheer volume of PURs being generated now.

Looks like I'm going to need to rewrite the server request code to spread the load a bit more evenly across the whole LMUR session rather than the current "bursty" approach to requesting large chunks in one hit and then going quiet until the next cache update or map relocation. Expect this to happen soon. Waze soon, that is... i.e. it's a fairly big job with plenty of things to get horribly wrong and totally screw up the script, so probably not something I'm going to be able to fix during my usual lunchtime coding sessions.
Twister-UK
Beta tester
Beta tester
 
Posts: 2573
Joined: Sat Jan 07, 2012 12:00 am
Location: NW London
Has thanked: 380 times
Been thanked: 2228 times

Re: [Script] Live Map UR Layer v2.14 (20141029)

Postby ditchi56 » Fri Nov 07, 2014 11:21 am

Quite often, I get the message "Unable to access UR/problem data on server". The only option offered is to press "OK", whereupon it tries again and probably hits the same problem. I usually end up closing the tab.

I'm not sure whether this is under your control, but would it be possible to allow users to shrink the map area or otherwise alter their LMUR options before it tries again. Perhaps the "enabled" box in the highest window could be automatically unchecked when this message appears?
ditchi56
Country Manager
Country Manager
 
Posts: 4966
Joined: Tue May 29, 2012 12:35 pm
Has thanked: 1116 times
Been thanked: 2080 times

PreviousNext

Return to Addons, Extensions, and Scripts

Who is online

Users browsing this forum: rmeatc_2, Yahoo [Bot]