Get a sneak peek at whats next for Permanent Hazards on our April 7th Office Hours!
Coordinator: orbitc & ARC: SanzClew
------------------------------------------------------------

Post Reply

New NER Discord Bot for Twitter Closures

Post by itismejoey
Hello New England (and everyone else)!

When I first joined the NER Discord I noticed that there was an unused #twitterfeed channel that appeared to go unused for a while. I could tell that the leadership was most likely using webhooks and most likely a service like IFTTT (If This Then That), which was later confirmed. The downside with using IFTTT is that sometimes there would be up to a 15 minute delay from when the tweet was sent out publicly to when it would be sent to the webhook channel.

With the help of johnninja58 and Elli21486, as well as the support from the NER SMs, I would like to introduce the new way NER will be getting Twitter updates sent straight to its Discord.

NER Traffic Alerts is a Discord bot account that is coded using JS (JavaScript). The bot utilizes Twitter's API to look at specific Twitter accounts and pull certain tweets, based on a filter, and post them in a state's Twitter feed. Because the bot is looking directly at the Twitter API and checking every 60 seconds, the tweet will be sent to the channel usually within 60 seconds from being posted.

Below I have listed the current Twitter accounts that we are following for each state:

Code: Select all

CT: CT_STATE_POLICE, CTDOT_Statewide, TollandAlert

Code: Select all

MA: "MassStatePolice, MassDCR, MassDOT, TotalTrafficBOS, lawrencepolice, HaverhillPolice, TewksburyPD, WBZTraffic, ChelseaScanner, reverescanner, 5_13Dist, LowellPD, WilmingtonMAPD, WorcesterEM, bostonpolice, SpencerMAPolice, BelmontPD, MethuenPolice, DanversPolice, Falmouth_Police

Code: Select all

ME: MEStatePolice, MaineTurnpike, MaineDOT1

Code: Select all

NH: NH_StatePolice, NHDOTRoutes, nhdoti93, NHDOTI293, NHDOTNH101, NHDOTI89, NHDOTEverett, NHDOTSpaulding, manch_nh_hiway, pelhamnhpolice, rocksheriffnh, HamptonNHPD, ConcordNHGS, AtkinsonPolice, DoverNHPolice, nashuadpw

Code: Select all

RI: RIStatePolice, RIDOTNews

Code: Select all

VT: VTStatePolice, VTDMV, 511VT
Most of the above Twitter accounts were used during testing and can be modified in anyway, based on what each SM decides they'd like in their feed. Just let me know and I can change it!

The bot looks for the following words in a tweet to send an alert:

Code: Select all

closed, closure, all lanes closed, ramp blocked, off-ramp, ramp open, ramp reopen, ramp re-open, traffic alert
But will also filter out a tweet if it has the following:

Code: Select all

one lane, two lanes, right lane, right travel lane, rtl, left lane, left travel lane, LTL, except right, except left, center lane, 2 lanes, 1 lane, partial, sidewalk, 3 right lanes, 3 left lanes, shoulder open, breakdown lane open, break down lane open, office, library, clear, cleared, report , service center, school, Tech, Institute, College, University, DMV, RMV, advance notice, advanced notice, overnight, intermittently, temporary
If you have any issues that you notice, such as certain tweets making it through the filter when they shouldn't have, please don't hesitate to reach out!
itismejoey
Map Editor - level 3
Map Editor - level 3
Posts: 7
Has thanked: 2 times
Been thanked: 8 times

POSTER_ID:17606656

1

Send a message
Last edited by itismejoey on Fri Mar 22, 2019 6:03 pm, edited 2 times in total.
Level 3 editor
AM New Hampshire (Derry area)
Southern NH/Merrimack Valley Mass

Post by Inactive user -1649923503-
Thank itismejoey I see that the first tweet has fed into CT. I think this will be a huge help for the community and appreciate your time and energy making this happen.
Inactive user -1649923503-
Posts: 1023
Has thanked: 404 times
Been thanked: 460 times
Send a message
-John
-------
https://j.mp/1BqcgbWhttps://s.waze.tools/c5.png
VT - State Manager
New England - Multi State Manager

Post by Inactive user -1649923503-
If you find examples please bring them to itismejoey attention. While there will be scenarios due to the filtering that a tweet might be missed. That is a longer term optimization task. Right now we have no tweets going into the server, having a reliable feed is critical otherwise people will tune things out. As we get more comfortable with the feeds we can definitely explore adjusting the filtering. It would also be good to evaluate if any twitter accounts should be added.
Inactive user -1649923503-
Posts: 1023
Has thanked: 404 times
Been thanked: 460 times
Send a message
-John
-------
https://j.mp/1BqcgbWhttps://s.waze.tools/c5.png
VT - State Manager
New England - Multi State Manager

Post by Inactive user -1649923503-
Can we add @LaconiaNHPolice

Thank you
Inactive user -1649923503-
Posts: 1023
Has thanked: 404 times
Been thanked: 460 times
Send a message
-John
-------
https://j.mp/1BqcgbWhttps://s.waze.tools/c5.png
VT - State Manager
New England - Multi State Manager

Post by Inactive user -1649923503-
TotalTrafficBOS And WBZtraffic have been removed to evaluate a question that was raised regarding TOS
Inactive user -1649923503-
Posts: 1023
Has thanked: 404 times
Been thanked: 460 times
Send a message
-John
-------
https://j.mp/1BqcgbWhttps://s.waze.tools/c5.png
VT - State Manager
New England - Multi State Manager

Post by dBsooner
Awesome job on on being forward thinking and developing a JS based bot to utilize Twitter API and Discord webhooks!

If you ever have issues with it, @TheChrisK setup a Huginn server that a lot of regions use. It is hosted on a cloud server I donated. Huginn is very versatile and helps get past the limits Discord webhooks and the Twitter API introduce.

Good stuff though, really! I love seeing new technology come out that helps editors do their daily routines!
dBsooner
Map Editor - Level 5
Map Editor - Level 5
Posts: 767
Has thanked: 367 times
Been thanked: 631 times
Send a message
https://s.waze.tools/scrp.pnghttps://s.waze.tools/am.pnghttps://s.waze.tools/betc.pnghttps://s.waze.tools/beta.pnghttps://s.waze.tools/mapr.pnghttps://storage.googleapis.com/wazeopedia-files/7/74/Waze_signature_200k_plain.pnghttps://s.waze.tools/c5s.png
• AM: Eastern Oklahoma, DFW, Southern Florida
• Wiki: WME | Editing Manual | Best Practices | FAQ | Places
• T-Mobile | iPhone 14 Pro | iOS Latest | Waze Latest Beta

Post by Elli21486
itismejoey wrote:
nhmapguy wrote:Who will be monitoring the filtered exclusion tweets to see if tweets that should make it through aren't filtered out?
Great question! So I still have my own semi-private Discord specifically for Waze. It's where I keep track of any current or future projects that I am working on. What John suggested to me is to have a secondary feed that posts tweets without the exclusion filter, just to make sure we aren't missing anything.

We've been looking at this for almost a month and I think we got it pretty good right now. I also monitor over 400 accounts, while at work or on the computer at home, and get alerts for specific filters. Most of the time I'll get alerted on TweetDeck and from the Discord bot at the same time.
Just to continue on this, I am also monitoring Twitter and Facebook so there's more than one user in regards to monitoring social media. There are things however such as weather damage or fires, that would unfortunately need to be manually reviewed due to the nature of those events as they can have many variables.

Overall though, the bot did quite well during our last storm.
Elli21486
Posts: 152
Has thanked: 1 time
Been thanked: 16 times
Send a message

Post by itismejoey
nhmapguy wrote:Who will be monitoring the filtered exclusion tweets to see if tweets that should make it through aren't filtered out?
Great question! So I still have my own semi-private Discord specifically for Waze. It's where I keep track of any current or future projects that I am working on. What John suggested to me is to have a secondary feed that posts tweets without the exclusion filter, just to make sure we aren't missing anything.

We've been looking at this for almost a month and I think we got it pretty good right now. I also monitor over 400 accounts, while at work or on the computer at home, and get alerts for specific filters. Most of the time I'll get alerted on TweetDeck and from the Discord bot at the same time.
Spenceman2023 wrote:Although sometimes they can be vague and not have a reason to why a lane is closed
That's frustrating sometimes, but as Moogonk said once in the Discord, "If authorities are asking for people to avoid a location, I generally put in a closure to give them room to work." Sometimes we just have to mark a road as closed without a reason or we can try and dig deeper to find that reason (I personally like to dig).
itismejoey
Map Editor - level 3
Map Editor - level 3
Posts: 7
Has thanked: 2 times
Been thanked: 8 times
Send a message
Level 3 editor
AM New Hampshire (Derry area)
Southern NH/Merrimack Valley Mass

Post by itismejoey
nhmapguy wrote:Sounds good. I'll have to create a Twitter list and stay on top of it. I was also going to recommend accounts as I saw them.
Also, yes please!

As you can see, I do have a couple local agencies on the list that I know are good about posting closures. I just added WilmingtonMAPD and WorcesterEM, for Mass. I am going to go through my followed list and see who else is good about posting closures.

If you're looking for a good start on accounts to follow, you can look at what I have followed so far. https://twitter.com/itismewaze/following
itismejoey
Map Editor - level 3
Map Editor - level 3
Posts: 7
Has thanked: 2 times
Been thanked: 8 times
Send a message
Level 3 editor
AM New Hampshire (Derry area)
Southern NH/Merrimack Valley Mass

Post by itismejoey
Added "overnight" to the exclude filter, because MassDOT felt like replying to some people this morning and the tweets met all requirements to be posted. I assume that most overnight closures won't result in full lane closures.

Edit: This might be changed at a later date, since MassDOT does overnight closures on ramps. This will still get caught by the bot, but will be posted in a different server/channel if it includes "overnight".
itismejoey
Map Editor - level 3
Map Editor - level 3
Posts: 7
Has thanked: 2 times
Been thanked: 8 times
Send a message
Level 3 editor
AM New Hampshire (Derry area)
Southern NH/Merrimack Valley Mass