Skip to content

Korean tokenizer plugin: garu-orama-tokenizer (1.9MB WASM analyzer)Β #1023

Description

@ongjin

Hi Orama team! πŸ‘‹

I built a Korean morphological tokenizer plugin for Orama: garu-orama-tokenizer.

Korean search with the default tokenizer has a real pain point β€” particles like 은/λŠ”/이/κ°€/을/λ₯Ό glue onto the noun, so searching for 학ꡐ misses 학ꡐ에, 학ꡐλ₯Ό, 학ꡐ가. And verb inflections like λ¨Ήμ—ˆλ‹€ vs λ¨ΉλŠ”λ‹€ never match each other. This plugin runs real morphological analysis under the hood, powered by garu-ko (a 1.9MB WASM analyzer that runs in browsers, Node, and edge).

Usage follows the standard Orama pattern (same as @orama/tokenizers/mandarin):

import { create, insert, search } from '@orama/orama'
import { createTokenizer } from 'garu-orama-tokenizer'

const db = await create({
  schema: { title: 'string', body: 'string' },
  components: { tokenizer: await createTokenizer() }
})

Would you consider linking this from the Tokenizers docs page or your community integrations list? Happy to send a docs PR if that's easier.

Links:

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions