Decentralized search engine & automatized press reviews

Version 1.8.16 : source maintenance, headlines and broken JSON

A bug in the headline loading forced us to release a new version, but as a lot of maintenance have also been done with sources it’s not a bad thing.

1. Source maintenance

Mediapart in its 3 languages have been fixed and is now a precise (for many words) source !

AFP Factual and its 21 languages changed their date format, so we had to parse all of them again., and DHnet now serves results in faulty JSON format that we have to fix on the fly.

A bunch of sources have been cleared from "many words" search as they were providing unfocused results. and a few other sources were added, some removed… json/source.json now counts 119 more lines and 4260 of them were edited in 2024.

1.1. 120 more lines, 276 edited in the source definition file

2019 586

2020 879

2021 3004

2022 6910

2023 8735

2024 4260

For a total line number of : 24.374 (+120)

2. Fixed bugs

  • #83 Status line does not report precisely the number of sources we’re waiting for under 30 (a new step has been added at 15 and we now display "> 15" while waiting for the last sources)

  • #84 Inconsistency in request duration digits (now it’s limited to 2 or 3 meaningful digits)

  • #85 [] Broken load_headlines

  • #86 Bad JSON sent by sources (where we explain how we escape in-value double-quotes, based on JSON.parse exceptions).

3. Miscellaneous

The ESlint flat file config of the project have been updated resulting in more const usage.

To finish, the testing tool design have been improved a bit with a compact and colorful statistics.