Version 1.1 — 20 July 2010

By Jim Dovey

This project implements a Trie structure for the Go programming language, along with a customization of that structure designed for use with TeX-style hyphenation tables (where values need to be stored with each character).

The structures implemented here use Unicode runes instead of bytes for each item in the trie. This means that strings of Japanese kanji characters will use exactly one trie node per character, rather than one per byte of that character’s UTF-8 representation. It also makes the trie itself relatively encoding-agnostic, since it will store UTF32 rune values, meaning that inputs can be compared against strings using any valid sequence of UTF-8 bytes which would resolve to the same UTF32 runes.

The Trie structure can optionally store a value with each inserted string. The value is of type interface{}, so anything will work. My Hyphenator uses vector.IntVector as the value for each string, for example.

There is one additional file, hyphen_trie.go, which implements a convenience for TeX-style hyphenation: it will accept a pattern string similar to that used by the above hyphenator package and will insert the string while attaching the numeric values at the end. This function might well be moved into hyphenator in the future; at present it’s here because it can be more optimized at this level.

For more information on Tries, read this Wikipedia article. For more information on TeX’s use of tries for hyphenation, refer to the original paper by Franklin Mark Liang.

The simplest way to install is using goinstall:

> goinstall “github.com/AlanQuatermain/go-trie”

This will install the package in such a way that it must be imported using the above quoted string as its import statement, i.e. import trie "github.com/AlanQuatermain/go-trie".

Alternatively you can clone the code directly and install it using make install. This will enable you to import it in your code using just import "trie".