Skip to content

darkhelmet/fetch

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

55 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

fetch

What is it?

fetch is a full text search engine and server written in go

How?

POST '{ "title": "My Blog Post", "body": "Ruby is the bomb..." }' /blog/posts/5 (/:index/:scope/:id)

Pipeline

JSON parts → Tokenizer → Analyzer → Storage

Tokenzier

String -> TokenChan

Analyzer

Tokens maybe have a type/flag/something and analyzers only work on certain filters?

TokenChan -> email -> superstrip(invalid characters, ascii, strip, lowercase) -> stopword -> stemming -> double_metaphone -> ngram? -> TokenChan

Storage

storage.Index(index, scope, id, field, tokens)

storage.Search(index, scope, query) # -> List of ids

Quick start

  • Install go (In the Fetch the repository step, use weekly instead of release)
  • Run ./all.bash from the src directory

Bugs

Submit bugs at my FogBugz

Hacking

  1. Install gospec
  2. Install git-flow
  3. Fork the repo, clone, and run ./setup.bash
  4. Use git-flow to start a feature: git flow feature start my-feature
  5. Write code and tests
  6. Stage files (so git ls-files picks them up) and run ./src/format.bash to format your source files according to gotfmt and my settings. DO THIS BEFORE COMMITTING.
  7. Commit
  8. Repeat steps 5-7 until feature is complete.
  9. Use git-flow to finish the feature: git flow feature finish my-feature
  10. Send a pull request.

The most important parts are tests and gofmt. If you don’t have tests, or didn’t run the formatting script, I’ll tell you to go do that. If you continually ignore this hacking workflow, I’ll start to continually ignore your pull requests!

License and Authors

fetch is licensed under the terms of the Apache License, Version 2.0. See LICENSE for details.

fetch was created by Daniel Huckstep. Type git shortlog -s for a full list of contributors.

The Porter Stemming algorithm in stemming.h is written by Martin Porter.

The Double Metaphone algorithm in double_metaphone.h is written by Maurice Aubrey and used with permission to “do what [I] want with it”.

About

A full text search engine written in Go

Resources

License

Stars

Watchers

Forks

Packages

No packages published