The system has a three-layer architecture. The collection layer gathers data from
Wikipedia (national election results), the European Parliament Open Data API (political entities and EP
groups), and the EP voting API (sessions, roll-call votes, attendance). Extraction of unstructured
content (Wikipedia HTML) into structured data uses an LLM (Google Gemini) with schema-validated
structured output and a deterministic (temperature=0) configuration. The
persistence layer is a relational database whose central entity links a national party
to its representation in the European Parliament (EP group membership, MEPs, votes, attendance). The
presentation layer is this web application, a tabbed interface over pre-computed
analytical outputs (faction/country cohesion, inter-faction agreement, MEP loyalty and participation,
voting-category correlations, topic profiles, association rules, anomaly detection).
Ground truth covers all 27 EU member states' last two national parliamentary elections
(54 country/year records, 454 party entries), built independently of the pipeline under test: seat
counts and government status from ParlGov where available, official/news sources and Wikipedia
infobox cross-checks otherwise; Wikidata IDs looked up directly, never taken from the pipeline's own
extraction output. All results below reflect a single, consistent pipeline configuration:
gemini-2.5-flash, temperature=0, schema-validated structured output with
up to 5 retries on an invalid response.
The pipeline is given only (country, election_year) and must discover the correct
Wikipedia article itself; this is compared against the ground-truth URL. A raw string-match gives
43/54 (79.6%) exact agreement. Manual, API-verified investigation of all 11 disagreements found that
most are not pipeline errors:
| Category | Count | % of 54 | Genuine pipeline error? |
|---|---|---|---|
| Exact match | 43 | 79.6% | — |
A — Ground-truth slug is a stale redirect; pipeline found the correct current page (verified via the Wikipedia redirects API for all 6 cases) | 6 | 11.1% | No — discovery was correct |
B — Evaluation-harness lookup bug for countries with two elections in the same year (e.g. "2024 (Jun)" vs. "2024 (Oct)") | 3 | 5.6% | No — evaluation-code artifact; the production pipeline is not affected |
| C — Corrupted/truncated cached fetch of the source page (Sweden 2018/2022), confirmed by a fresh, cache-bypassing API call reproducing the identical empty result | 2 | 3.7% | Yes — but in the fetch/cache layer, not the LLM extraction step |
| Functional discovery rate (exact + Category A) | 49 | 90.7% | |
| Genuine pipeline error rate (Category C only) | 2 | 3.7% |
Category A reflects Wikipedia renaming its canonical article title after the ground truth was recorded (e.g. "legislative" → "parliamentary", or the old "Name, year" convention to "year Name"); the pipeline discovered the live canonical page in every case. Category C's root cause was traced past the LLM to a 4.4 kB cached page for Sweden (a comparable country's cache is 13.9 kB) containing a single unrelated navbox table — the correct fix is re-fetching that one page, not changing extraction logic.
Three content-reduction strategies were compared: HTML (cleaned text, the production baseline), Markdown (link/table-preserving conversion), and raw (no reduction). HTML and Markdown were evaluated on the full 54-record set; raw was evaluated on a fixed 10-record subset (Cyprus, Germany, Malta, Portugal, Spain × 2 election years) for cost reasons, with HTML and Markdown re-aggregated over the same 10 records for a matched, like-for-like comparison.
| Metric | HTML (n=54) | Markdown (n=54) |
|---|---|---|
| Existence precision | 60.4% | 56.8% |
| Existence recall | 77.3% | 75.1% |
| Existence F1 | 67.8% | 64.7% |
| Strict precision (existence + seats + government) | 55.7% | 52.1% |
| Strict recall | 73.7% | 71.4% |
| Strict F1 | 63.4% | 60.3% |
| Seat accuracy | 83.4% | 85.3% |
| Government-status accuracy | 96.4% | 95.2% |
| Entity-linking success rate | 98.8% | 98.8% |
| Avg. compression ratio (of original tokens) | 24.2% | 9.8% |
| Phase-3 JSON schema validity | 54/54 | 54/54 |
| Metric | HTML (n=10) | Markdown (n=10) | Raw (n=10) |
|---|---|---|---|
| Existence F1 | 79.7% | 76.5% | 67.1% |
| Strict F1 | 76.6% | 72.9% | 61.5% |
| Seat accuracy | 94.7% | 94.7% | 93.0% |
| Government-status accuracy | 91.0% | 89.4% | 91.1% |
| Entity-linking success rate | 100% | 100% | 100% |
| Avg. compression ratio | 24.0% | 8.5% | 100% (none) |
On the matched subset, HTML > Markdown > raw on every accuracy metric — raw, despite having the most available information, performs worst, suggesting that reduction removes navigation/styling noise rather than useful signal. Markdown achieves roughly 2.5× the compression of HTML at a modest accuracy cost (existence/strict F1 both −3.1pp on n=54).
| Comparison | Metric | Difference | 95% CI | p | Significant? |
|---|---|---|---|---|---|
| HTML − Markdown (n=54) | Existence F1 | +3.1pp | [+0.1, +5.9]pp | ≈0.038 | Yes |
| HTML − Markdown (n=54) | Strict F1 | +3.2pp | [−0.1, +6.2]pp | ≈0.062 | No (borderline) |
| HTML − raw (n=10) | Existence F1 | +12.6pp | [+3.8, +20.3]pp | ≈0.013 | Yes |
| HTML − raw (n=10) | Strict F1 | +15.0pp | [+4.9, +22.2]pp | ≈0.013 | Yes |
| Markdown − raw (n=10) | Existence F1 | +9.4pp | [+0.0, +21.8]pp | ≈0.061 | No (borderline) |
| Markdown − raw (n=10) | Strict F1 | +11.3pp | [+0.0, +23.2]pp | ≈0.055 | No (borderline) |
HTML significantly outperforms Markdown on existence F1, but the strict-F1 gap does not reach conventional significance. HTML significantly outperforms raw on both metrics. Markdown's advantage over raw does not reach significance at this sample size — the raw-mode sample was deliberately kept at n=10 rather than expanded, since compression ratio (an exact measurement, not an estimate) already favors Markdown by a margin far larger than any plausible sampling effect on the accuracy comparison.
In Markdown mode, a results-table cell such as
<a href=".../New_Flemish_Alliance">N-VA</a> becomes
[N-VA](.../New_Flemish_Alliance); the LLM sometimes extracts the URL-derived full name
instead of the visible abbreviation. Checking all 600 extracted party entries across the 54 Markdown
records against the source HTML's visible link text found this pattern in 142 entries (23.7%), but it
degraded matching accuracy against ground truth in only 19 of those (3.2% of all 600 entries) — the
ground-truth dataset itself standardizes on full party names, so most instances of the preference are
harmless. The 19 that did cause a mismatch are concentrated among coalition/alliance parties, where the
English-translated full name does not correspond to any ground-truth alias, rather than single parties
with an established abbreviation.
Party-entity resolution uses a deterministic Wikidata-ID → full-name → abbreviation fallback match with no fuzzy/LLM-based matching. A stated methodological rule was applied throughout this project's ground-truth construction: no matching/lookup heuristic developed for ground-truth or evaluation purposes (multilingual Wikidata aliases, manual alliance/rename overrides) was ever ported into the pipeline's own entity-linking code — doing so would let the system under test converge toward its own answer key for reasons unrelated to genuine extraction capability, the same failure mode as train/test leakage.