in which the machine tries to design a solution for synonyms

Hide hard words

This is a genuinely°hard word interesting linguistic°hard word modeling problem. After reviewing°hard word the current data model, here are my thoughts.

The Core°hard word Problem

Each Lemma°hard word currently represents°hard word one concept°hard word anchored°hard word to an English°hard word label°hard word. "hat" and "cap°hard word" are two separate lemmas°hard word, each with one translation°hard word per°hard word language. But languages carve up semantic°hard word space differently:

English°hard word: "hat" (broad brim°hard word) vs°hard word "cap°hard word" (fitted°hard word, no brim°hard word) — distinct°hard word

Lithuanian: "skrybel°hard wordė" (wide brim°hard word) vs°hard word "kepur°hard wordė" (general headwear°hard word/cap°hard word) — different boundary°hard word

Chinese°hard word: "帽子" covers nearly all headwear°hard word generically°hard word

French°hard word: "chapeau°hard word" (hat, broad) vs°hard word "casquette°hard word" (baseball cap°hard word) — yet°hard word another split°hard word

The current model has no way to say "these two lemmas°hard word occupy°hard word overlapping°hard word semantic°hard word territory°hard word, and in language X they might collapse into one word."

⎯⎯⎯⎯⎯

Design Ideas

1. Synonym°hard word Clusters°hard word via°hard word Existing Relation Groups (Minimal°hard word Change)

Add a new relation_type = "synonym°hard word_cluster°hard word" to the existing LemmaRelationGroup°hard word / LemmaRelationMember°hard word system. Group "hat" and "cap°hard word" together under concept°hard word_label°hard word "headwear°hard word".

Pros°hard word: Almost no schema°hard word change. Reuses°hard word existing infrastructure°hard word and release°hard word format°hard word (lemma°hard word_relations/synonym°hard word_cluster°hard word/headwear°hard word.jsonl°hard word).

Cons°hard word: Too blunt°hard word. It says "these are related°hard word" but doesn't capture°hard word how — it doesn't tell the app what to do in Lithuanian vs°hard word Chinese°hard word. A cluster°hard word of 5 near-synonyms°hard word doesn't tell you which ones merge°hard word in which languages.

2. Per-Language°hard word Acceptable°hard word Alternatives°hard word on Translations°hard word

Add an alternatives°hard word field (JSON°hard word list) to LemmaTranslation°hard word. For "hat" in Chinese°hard word, the primary°hard word translation°hard word is "帽子", and for "cap°hard word" in Chinese°hard word, the primary°hard word translation°hard word is also "帽子" (or maybe "鸭舌帽"). The alternatives°hard word field would note that "帽子" is acceptable°hard word for both.

More concretely, a new table like:

TranslationAlternative°hard word:

lemma°hard word_id°hard word → FK°hard word to Lemma°hard word

language_code°hard word → str°hard word

alternative°hard word_text°hard word → str°hard word (the acceptable°hard word alternative°hard word translation°hard word)

source_lemma°hard word_id°hard word → FK°hard word to Lemma°hard word (which lemma°hard word "owns" this word natively°hard word)

notes → str°hard word (why it's acceptable°hard word)

So for "hat" (N09_X) in Chinese°hard word, you'd have: primary°hard word = "帽子", no alternatives°hard word needed (it's the generic°hard word). For "cap°hard word" (N09_Y) in Chinese°hard word, primary°hard word = "鸭舌帽", alternative°hard word = "帽子" (source: hat's°hard word lemma°hard word).

Pros°hard word: Directly solves°hard word the Trakaido°hard word use case — the app knows which answers to accept. Per-language°hard word granularity°hard word. Works well for quiz°hard word/testing scenarios°hard word.

Cons°hard word: Doesn't capture°hard word the why — it's a flat list of "also accept this." Maintenance°hard word burden°hard word grows combinatorially°hard word. Doesn't help with teaching (explaining the distinction°hard word to learners°hard word).

3. Semantic°hard word Overlap°hard word Groups with Language-Specific°hard word Merge°hard word Flags

A hybrid°hard word: create synonym°hard word clusters°hard word (idea 1), but annotate°hard word per°hard word language whether the cluster's°hard word members°hard word are distinct°hard word or merged°hard word in that language.

SynonymCluster°hard word:

id°hard word, concept°hard word_label°hard word ("headwear°hard word")

SynonymClusterMember°hard word:

cluster°hard word_id°hard word, lemma°hard word_id°hard word

SynonymClusterLanguageInfo°hard word:

cluster°hard word_id°hard word

language_code°hard word

merge°hard word_type → "distinct°hard word" | "merged°hard word" | "partial°hard word"

dominant°hard word_lemma°hard word_id°hard word → if merged°hard word, which lemma's°hard word translation°hard word covers the group

notes → free text°hard word ("kepur°hard wordė covers both unless°hard word brimmed°hard word")

For the headwear°hard word cluster°hard word in Chinese°hard word: merge°hard word_type = "merged°hard word", dominant°hard word_lemma°hard word_id°hard word = hat (because 帽子 covers everything). In English°hard word: merge°hard word_type = "distinct°hard word". In Lithuanian: merge°hard word_type = "partial°hard word" with a note.

Pros°hard word: Captures°hard word the actual°hard word linguistic°hard word reality°hard word. The app can use merge°hard word_type to decide behavior (skip testing distinctions°hard word in languages where they're merged°hard word, accept either answer for "partial°hard word"). Useful°hard word for pedagogy°hard word — you can teach learners°hard word "in Chinese°hard word, there's one word for all of these."

Cons°hard word: More complex schema°hard word. "partial°hard word" is vague°hard word and needs the notes field to be useful°hard word. Requires°hard word human curation°hard word per°hard word language per°hard word cluster°hard word.

4. Concept°hard word Hierarchy°hard word with Language-Specific°hard word Lexicalization°hard word Levels

Model a hypernym°hard word/hyponym°hard word tree:

headwear°hard word

├── hat (brimmed°hard word headwear°hard word)

│ ├── sun hat

│ └── top hat

├── cap°hard word (fitted°hard word, no brim°hard word)

│ ├── baseball cap°hard word

│ └── beanie°hard word

└── helmet

Then°hard word annotate°hard word per°hard word language which level has a common word. Chinese°hard word lexicalizes°hard word at "headwear°hard word" level (帽子). English°hard word lexicalizes°hard word at "hat"/"cap°hard word" level. Lithuanian has its own split°hard word point.

Pros°hard word: Linguistically°hard word the most accurate°hard word model. Powerful for curriculum°hard word design — teach the generic°hard word word first, then°hard word the specific°hard word ones at higher difficulty°hard word. Naturally handles the "how many words does this language need for this concept°hard word space" question.

Cons°hard word: Most complex to build and maintain°hard word. Over-engineered°hard word for many cases (do you really need a tree for "big" vs°hard word "large"?). The hierarchy°hard word itself may be English-biased°hard word — who says "headwear°hard word" is the natural top node°hard word?

5. Pragmatic°hard word Recommendation°hard word: Cluster°hard word + Alternatives°hard word (Ideas 1+2 Combined°hard word)

My actual°hard word recommendation°hard word is to combine°hard word the simplest°hard word version°hard word of two ideas:

A) Synonym°hard word clusters°hard word using the existing relation group system (relation_type = "synonym°hard word_cluster°hard word"). This groups related°hard word lemmas°hard word and is cheap to implement°hard word. It lives in lemma°hard word_relations/synonym°hard word_cluster°hard word/ in the release°hard word format°hard word.

B) Acceptable°hard word alternatives°hard word as a lightweight°hard word annotation°hard word on LemmaTranslation°hard word (a JSON°hard word field or a small sibling°hard word table). This gives the app the per-language°hard word "accept this answer too" data it needs.

The cluster°hard word provides°hard word the human/editorial°hard word grouping for curation°hard word. The alternatives°hard word provide°hard word the app-facing°hard word behavior. They're populated°hard word semi-independently°hard word — you might have a cluster°hard word of 4 words but only 2 of them share a translation°hard word in Lithuanian.