• Filebeat's prospector/harvester approach is inappropriate for unifiedbeat
  • while unifiedbeat does work as-is today, changes should recognize:
    • the volume of data is a lot less than syslogs (given a well tuned sensor)
    • there's only one folder containing unified2 files
    • there's only one unified2 file that is active and should be tailed
  • of course, the above is all wrong if a server is used to centralize unified2 files from multiple sensors
    • then again, one could just run this version of unifiedbeat for each folder (i.e. each sensor) [KISS]
• add os.Stat(r.reader.File.Name()) to spoolrecordreader.go
  • to deal with renamed or deleted files
• change Offset() in spoolrecordreader.go to return full path not just path.Base
• add sample unified2 files in sample_data
• add examples
  • clsreadu2.go simple reader with counts
  • clsspoolreader.go to test SpoolRecordReader
    • use CloseHook to rename (archive) an indexed file
      • otherwise, an endless reading loop occurs (openNext flaw)
• add new fields to the SpoolRecordReader struct:
  • FileSource - the file currently being tailed (read)
  • FileOffset - the offset position into FileSource
  • these fields allow for a registry file -- to bookmark where we were if interrupted
  • use these fields in openNext to set the offset in the call to NewRecordReader

A Go(lang) Library for decoding unified2 log files as generated by IDS applications such as Snort and Suricata.

go get github.com/jasonish/go-unified2

See https://godoc.org/github.com/jasonish/go-unified2

For more information on the unified2 file format see the Snort Manual.