Automatically update poll data

[Wouter8]

This pull request adds automated polling data updates to the project by introducing a poll data importer tool, updating data structures and UI to display dynamic poll metadata, and integrating a scheduled GitHub Actions workflow. The main themes are automation of poll data updates, codebase and data structure changes, and user interface improvements.

Automated Poll Data Import and Update:

• Added a new poll data importer tool in tools/importer/ that fetches the latest Dutch election poll data and metadata (dates, URLs) from Wikipedia, parses it, and updates src/parties.js accordingly. The importer uses axios and cheerio for scraping and parsing, and includes validation and error handling. (tools/importer/scraper.js, tools/importer/package.json, tools/importer/README.md, tools/importer/.gitignore) [1] [2] [3] [4]
• Introduced a GitHub Actions workflow (.github/workflows/update-polls.yml) to run the importer automatically on a configurable schedule, enabling regular, hands-off updates of poll data. (.github/workflows/update-polls.yml)

Codebase and Data Structure Changes:

• Updated src/parties.js to support dynamic poll data and metadata, replacing hardcoded poll results and static metadata with new polls and pollMetadata objects that are automatically updated by the importer. (src/parties.js)

User Interface Improvements:

• Modified the app to import and use pollMetadata from src/parties.js, and updated the UI to display poll sources, dates, and URLs dynamically, instead of hardcoded poll links. (src/components/App.js) [1] [2]

Documentation Updates:

• Added documentation for the importer tool, its usage, and the automation process, including details on scheduling and output format. (tools/importer/README.md)
• Updated the main README.md to explain the new automated poll update process and reference the importer tool. (README.md)

[vercel]

@Wouter8 is attempting to deploy a commit to the Personal Team on Vercel.

A member of the Team first needs to authorize it.

[larixk]

Wow, amazing approach and this could have saved us some manual work. Thank you so much for contributing to this project!

Unfortunately the timing for this PR is not ideal. The final three predictive polls we are using were published yesterday and I just did a final update of all numbers. Tonight I will be manually switching from using the 3 polls to a single exit poll by Ipsos, and when the official results of the election are made available we will switch to those. This means an automated way to import the predictive polls will not be used for a while...

In the lead-up to the next general elections in the Netherlands, I will be hopefully revisiting this project and it is hard to predict at the moment whether the same polling organizations will be most relevant and whether Wikipedia's way of structuring the data will remain the same. I therefore suggest keeping this PR open for now, until the next elections are planned... Which will take 4 years (at most).

Thank you again for this contribution, looking forward to having this working for the next elections.

[Wouter8]

Totally understand your comment. A couple days ago I stumbled upon your site and sometimes I saw newer poll results on Wikipedia, while the website was not updated yet. Then I saw that you were manually updating these results which I thought could be improved :).

The Wikipedia page should include all polls, including the exit poll, so maybe we should make this more configurable. Currently the 3 organizations are indeed hardcoded. I agree that fine tuning this for the current Wikipedia page is not the most sensible, as it will become obsolete later today.

I'm fine with leaving the PR open and revisiting this for the next elections (let's hope it is in 4 years...). Thanks for the cool site!