✒

LLM-as-Annotator

LREC 2026 · ACL Anthologypaper: Vidal-Gorène et al. 2026, LoResLM →
·
📍 · sentences, tokens · ⚠ disagreements ICL pool · v · ex ·
·

🧠
Mixture-of-Experts is ON. models will run in parallel — answers are voted per token, and contested tokens (⚠) are highlighted for your review.
💡
Single-model mode. Add a 2nd model in to compare per token (Mixture-of-Experts).

Task & label schema

edit fields…

JSON of the internal schema (fields with name/type/values/nullable/aggregator). Click outside to apply.

Models

🧠 OpenRouter supports MoE — you can pick several models. Single-model only. For Mixture-of-Experts (parallel models + vote), use OpenRouter.

API key

🔒 Tab-scoped storage. Keys live only in this browser tab's sessionStorage (one slot per provider). Sent as X-API-Key + X-LLM-Provider headers — never stored on the server.

Get a key → openrouter.ai/keys console.mistral.ai/api-keys platform.openai.com/api-keys ilaas.fr/services-inference

Load your own corpus

Press Enter, , or ; to add. You can also paste PER, LOC, ORG, WORK and press Enter to add them all at once.

Need lemmatization, morphology, or several tag columns? Use one of the POS / Lemma presets above, or open 📋 Task after loading for finer control.

ICL pool · v

Corrected sentences live here and are re-injected as few-shot examples on the next annotation run. The pool filters by (language, schema_hash) so cross-task contamination cannot happen.

# lang source preview
empty — correct a sentence and click 📥 to ICL.
⬇ Download .jsonl

Export corpus

All sentences in the corpus, with their current (possibly corrected) annotations.

⬇TSV round-trip with PIE baseline ⬇JSON schema-conformant ⬇CoNLL-U UD-standard ⬇JSONL fine-tune format

Apply to tokens

⚙ Advanced settings

Help

Keyboard shortcuts

j / k focus next / previous token
e / ↵ edit focused token
1 – 9 (in editor) assign the i-th visible tag
x toggle selection of focused token
r re-annotate the focused sentence
↵ save token edit and advance to next disagreement
Esc close popup

Workflow

  1. Pick a sandbox example (sidebar) or paste your text.
  2. Choose a task preset (top bar → 📋 Task).
  3. Set OpenRouter API key (top bar → 🔑) and one or more models.
  4. Click ▶ Annotate all. Disagreement tokens turn amber.
  5. Click a token to fix it. Click 📥 to ICL to feed corrections back.
  6. Export TSV / JSON / CoNLL-U / JSONL when done.