davidg666 wrote: Which browser and OS are you using? I'm using Chrome on a Mac (it was decent in 2015, but it sometimes struggles a bit with WME now).

I'm using Firefox on Windows 10 if I'm at home, or Linux if I'm using my laptop. My PC at home is almost 7 years old now, and WME is still manageable with the generalised townlands loaded in. (8GB RAM and an old, I think 3rd gen, intel core i5)

I've managed to get around the memory issues by loading the ungeneralised file in line by line (which in itself was a challenge because the original file was all on one line!), So the only drawback now is speed as the new file is a couple of million lines long. It takes a good 15 minutes to extract a county.
Python isn't the best language to be using for speed (I didn't think it would be this bad!) So I'll probably port it over to something like C++ if optimisation efforts in python don't improve anything.

In the meantime, I can throw this script onto a server and have it work away at extracting each county. It might take a while! I'll then make these smaller files publicly available (I'll maybe set up a site at waze.cw1998.uk)

If these smaller county sized ungeneralised files are still too big to be managed efficiently in WME, I'll split them in half again. The benefit here being the smaller files should take much less time to process! At this stage it's just a maths problem - working out which townlands need included by coordinates.

I have updated the first post with some more information about this. I also have a server going through the dataset and making a separate file for each county. I'll update you here when this is finished!

davidg666 wrote: When you get a chance, would you mind writing a little cheat sheet telling people what to do to get Tim's script and your datafiles working with WME? Without having tried it yet, I imagine it's straightforward once you've done it once

Yep, it's on the todo list. Just need to wait for a few more county files to get generated (it's at 6% overall at the time of writing). I've also just added a new DNS record to my domain, so waze.cw1998.uk should be live soon (if it isn't already). All the county files will live there.

Just to update this: I made the script half the precision of the coordinates for the points on the townland polygons and remove unneeded polygon properties.
It looks a little bit messier (see this screenshot), but the file size is essentially half of the original, and it certainly is still workable.
I have only run this with Co. Clare so far, and the file size went from 39.3MB to 20.1MB.

davidg666 wrote: So the lower-precision one is the purple line on your screenshot?

Yes.

davidg666 wrote: Does it make any noticeable speed difference in WME?

Yes, zooming in and out is definitely smoother. I don't really have much of a problem with panning around with the larger files loaded, but on a lower end device, I'm guessing it will be smoother.

As I write this the server the files are hosted on is generating "reduced" copies of all the county files. All the files that were there are now in a folder called "full". Full seems to be fine most of the time and is ok for smaller counties, but if you're having trouble, then try the ungeneralised reduced variant.

If anyone is using the script: it has also been vastly improved. I got rid of that awful console menu in favour of passing options in via command line arguments.

Just to log the change here for anyone using this script. I have changed the name of the townland polygon objects to include the string "name", rather than "TD_ENGLISH".

This allows the datasets to be used with WME Geometries (Timbone's script) without having to alter the script in any way.

I have also regenerated all of the available townland geometry files available at https://waze.cw1998.uk to reflect this change. So any files downloaded from there should now work straight away with the script.
If there are any problems with this, let me know here or by PM.

cw1998 wrote:I should probably make a new thread for this...

I took the liberty of moving your post into a new thread...

cw1998 wrote:I have completed one part of my script - county extraction of townland geometries. See the screenshot here.

Looks great! If this works well (and it's easy to set up) I think I'm going to love it.

cw1998 wrote:Performance is good with just one county's worth of geometry loaded. The file is only about 2MB in size for Monaghan, so I image it will be similar for the majority of the rest. Maybe Mayo, Cork and Galway will be larger; testing needs done for this.

The next step is testing this with the ungeneralised data set, which is many times larger than the generalised file...

Edit: Yeah this may be a challenge. I just get memory errors trying to load in the ungeneralised file. Will keep working on this tomorrow.

Which browser and OS are you using? I'm using Chrome on a Mac (it was decent in 2015, but it sometimes struggles a bit with WME now).

++David

cw1998 wrote:I have updated the first post with some more information about this. I also have a server going through the dataset and making a separate file for each county. I'll update you here when this is finished!

Lovely - thank you!

When you get a chance, would you mind writing a little cheat sheet telling people what to do to get Tim's script and your datafiles working with WME? Without having tried it yet, I imagine it's straightforward once you've done it once

++David