Police feeds monitor

Well its that time again, where i release another small project.

Police Feeds Monitor or PFM was designed with automation in mind while still being useful and flexible. Originally designed to help with ArrestTracker.com Project there was soon a realization that this system could be much more useful.

The system is designed to monitor given Police News, Media Release and other special law enforcement feeds and from this it aggregates the data, checks for matches against some filters and marks them where appropriate while saving all content for future usage.

The system is currently offers the ability for registered users to monitor particular keywords. Registered users may list current feeds, entries from each feed or all entries from all feeds as well being able to view all matches from filters even if they are not following them.

For this project i opt'd to build it in ruby on rails, the main majority of work comes from Feedjira but then there is also a selection of sites that do not offer any RSS feeds so this left one option, to scrape the raw contents of the sites and parse them as if they was RSS feeds, to do this i created a custom parser that allows for flexibility in different hosts, news feeds and html contents for parsing easily which in turn allows for a wide selection of news content to be aggregated into a single source. Each feed is checked by background cronjobs and only updated contents is added if found. Each feed under goes a check about every 60 minutes or so.

For now the system is in BETA, once a full working API is in place and its tried and tested the system will be opened up for cheap licensing for all types of commercial entities who wish to be able to follow law enforcement news more accurately, until then if you have any questions contact me on [email protected].

View the site here

Lee Johnstone

Information Security Data Analyst, Web Developer, Data Scraping/Crawling Specialist.

Read More