A lean web-crawler library, written in Go.

For an example on how to use this library, see blackbox_test.go

Warning: This is meant to be run on a relatively powerful machine. It potentially spawns a lot of goroutines which may cause resource exhaustion on older machines or mobile devices.

TBD