func LookupFlowsByNodes(s Storage, context graph.GraphContext, hnmap flow.HostNodeIDMap, filter *flow.Filter, interval *flow.Range) ([]*flow.Flow, error) { now := context.Time.Unix() andFilter := &flow.BoolFilter{ Op: flow.BoolFilterOp_AND, Filters: []*flow.Filter{ { LteInt64Filter: &flow.LteInt64Filter{ Key: "Metric.Start", Value: now, }, }, { GteInt64Filter: &flow.GteInt64Filter{ Key: "Metric.Last", Value: now, }, }, }, } if len(hnmap) > 0 { nodeFilter := &flow.BoolFilter{Op: flow.BoolFilterOp_OR} for _, ids := range hnmap { nodeFilter.Filters = append(nodeFilter.Filters, flow.NewFilterForNodes(ids)) } andFilter.Filters = append(andFilter.Filters, &flow.Filter{BoolFilter: nodeFilter}) } if filter != nil { andFilter.Filters = append(andFilter.Filters, filter) } return s.SearchFlows(&flow.Filter{BoolFilter: andFilter}, interval) }
func TestFlowStorage(t *testing.T) { aa := helper.NewAgentAnalyzerWithConfig(t, confStorage, nil) aa.Start() defer aa.Stop() client, err := api.NewCrudClientFromConfig(&http.AuthenticationOpts{}) if err != nil { t.Fatal(err.Error()) } capture := api.NewCapture("G.V().Has('Name', 'br-sflow', 'Type', 'ovsbridge')", "") if err := client.Create("capture", capture); err != nil { t.Fatal(err.Error()) } time.Sleep(3 * time.Second) now := time.Now() setupCmds := []helper.Cmd{ {"ovs-vsctl add-br br-sflow", true}, {"ovs-vsctl add-port br-sflow sflow-intf1 -- set interface sflow-intf1 type=internal", true}, {"ip netns add sflow-vm1", true}, {"ip link set sflow-intf1 netns sflow-vm1", true}, {"ip netns exec sflow-vm1 ip address add 169.254.33.33/24 dev sflow-intf1", true}, {"ip netns exec sflow-vm1 ip link set sflow-intf1 up", true}, {"ovs-vsctl add-port br-sflow sflow-intf2 -- set interface sflow-intf2 type=internal", true}, {"ip netns add sflow-vm2", true}, {"ip link set sflow-intf2 netns sflow-vm2", true}, {"ip netns exec sflow-vm2 ip address add 169.254.33.34/24 dev sflow-intf2", true}, {"ip netns exec sflow-vm2 ip link set sflow-intf2 up", true}, // wait to have everything ready, sflow, interfaces {"sleep 2", false}, {"ip netns exec sflow-vm1 ping -c 10 -s 1024 -I sflow-intf1 169.254.33.34", false}, } tearDownCmds := []helper.Cmd{ {"ip netns del sflow-vm1", true}, {"ip netns del sflow-vm2", true}, {"ovs-vsctl del-br br-sflow", true}, } helper.ExecCmds(t, setupCmds...) defer helper.ExecCmds(t, tearDownCmds...) time.Sleep(time.Second) aa.Flush() // Wait for the flows to be indexed in elasticsearch time.Sleep(3 * time.Second) gh := helper.NewGremlinQueryHelper(&http.AuthenticationOpts{}) node := gh.GetNodeFromGremlinReply(t, `g.V().Has("Name", "br-sflow", "Type", "ovsbridge")`) id := string(node.ID) filters := flow.NewFilterForNodes([]string{id}) flows, err := aa.Analyzer.Storage.SearchFlows(filters, nil) if err != nil { t.Fatalf("Failed to query flows: %s", err.Error()) } if len(flows) == 0 { t.Fatalf("SearchFlows should return at least one flow, got %d", len(flows)) } queryFlowMetrics(t, now.Add(5*time.Second).Unix(), 10) client.Delete("capture", capture.ID()) }
func (s *FlowGremlinTraversalStep) Exec(last traversal.GraphTraversalStep) (traversal.GraphTraversalStep, error) { var graphTraversal *traversal.GraphTraversal var err error flowSearchQuery, err := s.makeFlowSearchQuery() if err != nil { return nil, err } var flowset *flow.FlowSet switch tv := last.(type) { case *traversal.GraphTraversal: graphTraversal = tv context := graphTraversal.Graph.GetContext() // if Since predicate present in a non time context query if s.hasSinceParam() && context.Time == nil { return nil, errors.New("Since predicate has to be used with Context step") } if context.Time != nil { if s.Storage == nil { return nil, storage.NoStorageConfigured } s.addTimeFilter(&flowSearchQuery, context.Time) // We do nothing as the following step is Metrics // and we'll make a request on metrics instead of flows if s.metricsNextStep { return &FlowTraversalStep{GraphTraversal: graphTraversal, Storage: s.Storage, flowSearchQuery: flowSearchQuery, since: s.sinceParam()}, nil } if flowset, err = s.Storage.SearchFlows(flowSearchQuery); err != nil { return nil, err } } else { flowset, err = s.TableClient.LookupFlows(flowSearchQuery) } case *traversal.GraphTraversalV: graphTraversal = tv.GraphTraversal context := graphTraversal.Graph.GetContext() // if Since predicate present in a non time context query if s.hasSinceParam() && context.Time == nil { return nil, errors.New("Since predicate has to be used with Context step") } // not need to get flows from node not supporting capture nodes := captureAllowedNodes(tv.GetNodes()) if len(nodes) != 0 { if context.Time != nil { if s.Storage == nil { return nil, storage.NoStorageConfigured } s.addTimeFilter(&flowSearchQuery, context.Time) // previously selected nodes then need to filter flow belonging to them nodeFilter := flow.NewFilterForNodes(nodes) flowSearchQuery.Filter = flow.NewAndFilter(flowSearchQuery.Filter, nodeFilter) // We do nothing as the following step is Metrics // and we'll make a request on metrics instead of flows if s.metricsNextStep { return &FlowTraversalStep{GraphTraversal: graphTraversal, Storage: s.Storage, flowSearchQuery: flowSearchQuery, since: s.sinceParam()}, nil } if flowset, err = s.Storage.SearchFlows(flowSearchQuery); err != nil { return nil, err } } else { hnmap := topology.BuildHostNodeTIDMap(nodes) flowset, err = s.TableClient.LookupFlowsByNodes(hnmap, flowSearchQuery) } } default: return nil, traversal.ExecutionError } if err != nil { logging.GetLogger().Errorf("Error while looking for flows: %s", err.Error()) return nil, err } if r := s.context.StepContext.PaginationRange; r != nil { flowset.Slice(int(r[0]), int(r[1])) } return &FlowTraversalStep{GraphTraversal: graphTraversal, Storage: s.Storage, flowset: flowset, flowSearchQuery: flowSearchQuery, since: s.sinceParam()}, nil }