func ProcessVisitsLine(dataQueue <-chan []string, facebookUniqueIds map[string]*synced.IntSetSynced, facebookPageViews *synced.MapViewsSynced, waitGroup *sync.WaitGroup) { for line := range dataQueue { visitLine := new(VisitsLine) visitLine.Id = line[0] visitLine.Date = line[1] decompressLine, err := file_processing.Decompress(line[2]) if err != nil { fmt.Println("Failed decompression visit line") } if err := json.Unmarshal(decompressLine, &visitLine.Data); err != nil { fmt.Printf("Failed to unmarshal data") } if visitLine.Id == "facebook.com" { for userId, pageViews := range visitLine.Data { parsedId, _ := strconv.ParseInt(userId, 10, 64) if _, ok := facebookUniqueIds[visitLine.Date]; !ok { facebookUniqueIds[visitLine.Date] = new(synced.IntSetSynced) facebookUniqueIds[visitLine.Date].Set = make(map[int64]bool) } facebookUniqueIds[visitLine.Date].AddToSet(parsedId) for _, pageView := range pageViews { if _, ok := facebookPageViews.Map[visitLine.Date]; !ok { facebookPageViews.Map[visitLine.Date] = 0 } facebookPageViews.AddToViews(visitLine.Date, pageView[1]) } } } } defer waitGroup.Done() }