func (h *HostRow) DataAsFlatKeyValue() map[string]map[string]interface{} { inputData := make(map[string]map[string]interface{}) outputData := make(map[string]map[string]interface{}) err := json.Unmarshal(h.Data, &inputData) if err != nil { return outputData } for path, innerData := range inputData { innerDataJson, err := json.Marshal(innerData) if err != nil { continue } flattenMapJson, err := gojsonexplode.Explodejson(innerDataJson, ".") if err != nil { continue } flattenMap := make(map[string]interface{}) err = json.Unmarshal(flattenMapJson, &flattenMap) if err != nil { continue } outputData[path] = flattenMap } return outputData }
func loop(inChan chan *nsq.Message) { count := 0 unmarshaled := make(map[string]interface{}) distChan := make(chan distMsg, 1000) countChan := make(chan string, 1000) tsChan := make(chan struct { k string v interface{} }) go updateDistribution(distChan) go insertTotal(countChan) go insertData(tsChan) tick := time.NewTicker(10 * time.Second) for { select { case m := <-inChan: exploded, err := gojsonexplode.Explodejson(m.Body, "_") if err != nil { log.Println(err) continue } countChan <- *stream err = json.Unmarshal(exploded, &unmarshaled) if err != nil { log.Println(err) continue } for k, v := range unmarshaled { typStr := "null" //fmt.Println(reflect.TypeOf(v).Name()) typ := reflect.TypeOf(v) if typ != nil { typStr = typ.Name() } distChan <- distMsg{k, typStr} tsChan <- struct { k string v interface{} }{k, v} } count++ m.Finish() case <-tick.C: log.Printf("read %d events\n", count) } } }