Traffic requirements

The Unica Discover Network Capture Application requires a minimum of a bidirectional traffic stream or two unidirectional traffic streams containing all HTTP request and response traffic between your web application(s) and the visitor's browsers that interact with your web applications.

If the data stream coming into the Unica Discover Network Capture Application is not complete (or “clean") you might get the erroneous impression that Discover is not working correctly.

Basic traffic requirements

To capture traffic, the DNCA needs to see the start of all TCP connections.

To allow monitoring of a complete HTTP(S) conversation, , the DNCA requires that the mirrored network traffic be of very high integrity and quality. Any loss of critical network TCP packets can prevent the DNCA from reassembling the TCP traffic into HTTP hits.

Lost TCP packets may result in Discover sessions with missing pages, partial pages or both. In a worst-case scenario, the entire session may be unusable.

Check with your IT team to confirm if HTTP persistent connections have been enabled in the IT infrastructure.

Individual HTTP persistent connections may be used by multiple visitors to your web application and may be deployed by a load balancer such as an F5 network device, a front-end proxy such as an Akamai server or the web server itself.

HTTP persistent connections, which can also be called HTTP keep-alive, or HTTP connection reuse, are the idea of using the same TCP connection to send and receive multiple HTTP requests/responses, as opposed to opening a new one for every single request/response pair.

The Network Capture Application requires that it sees the start of all HTTP/TCP connections. If HTTP persistent connections are enabled then the DNCA will not be able to reassemble hits from in-progress connections.
  • Traffic stream:

    The DNCA requires a minimum of one bidirectional traffic stream or two unidirectional traffic streams containing all HTTP request and response traffic between the web application and the visitor’s browser interacting with it.

  • No errors or dropped packets:

    No errors, dropped packets, or overrun packets at operating system network interface card and network level.

    An ifconfig ethx command on the capture server should display a constant number of dropped packets or errors. The X in "ethx" will be the number of the NIC card, e.g. ifconfig eth0.

    If the number is increasing at a high rate, there may be problems with the fidelity of the traffic sent to the DNCA, inadequate sizing of your DNCA hardware for your traffic volume, or both.

  • Real visitor IPs:

    The capture point can see the real visitor IPs or host address of visitor's IP.

    Access to the real IP address of your visitors is a valuable resource for troubleshooting purposes. For customers using load balancers, this requirement may not be possible.

  • Filtered traffic:

    Spanned traffic is filtered down to the essential traffic only.

    It is recommended that you filter out as much unnecessary traffic as possible at the network level before it is delivered to the DNCA. Filtering off-loads processing resources that the DNCA has to use to filter out traffic.

  • TCP persistent connections disabled:

    If HTTP persistent connections are enabled then the DNCA will not be able to reassemble hits from in-progress connections.

TCP Connections Sources of Traffic

SPAN PORT
A SPAN port is also known as "Port Mirroring".

Port Mirroring is used on a network switch to send a copy of network packets seen on one switch port (or an entire VLAN) to a network monitoring connection on another switch port.

This is commonly used for network appliances that require monitoring of network traffic, such as an intrusion-detection system.

Port mirroring on a Cisco Systems switch is generally referred to as Switched Port Analyzer (SPAN); some other vendors have other names for it e.g. Roving Analysis Port (RAP) on 3Com switches.

NETWORK TAP
A network tap is a hardware device which provides a way to access the data flowing across a computer network.

In many cases, it is desirable for a third party to monitor the traffic between two points in the network. If the network between points A and B consists of a physical cable, a "network tap" may be the best way to accomplish this monitoring.

The network tap has (at least) three ports: an A port, a B port, and a monitor port. A tap inserted between points A and B passes all traffic between A and B through unimpeded but also copies that same data to its monitor port. This enables a third party to listen.

Network taps are commonly used for Discover, network intrusion detection systems, VoIP recording, network probes, RMON probes, packet sniffers, other monitoring and collection devices and software that require access to a network segment. Taps are used in security applications because they are non-obtrusive, are not detectable on the network (having no physical or logical address), can deal with full-duplex and non-shared networks and will usually pass through traffic even if the tap stops working or loses power.

SPAN PORT AGGREGATOR
SPAN Port Aggregation is a technology that combines a bidirectional full duplex data transmission into one single stream of data.

Additionally, aggregation can allow for the combination of data transmitted from multiple networks or SPAN ports.

An identical copy of this single stream of data can then be sent to any connected monitoring device. The connected monitoring device can receive the entire full duplex conversation or aggregate data from multiple networks with a single network interface card (NIC) also without having to reassemble the traffic Port Requirements

Port Definition and Configuration

Your IT team might be required open various TCP ports to enable communication with Unica Discover.

The types of communications can include:
  • Source Request
  • Destination server to handle request
  • Port Number: The port number which needs to be opened.
Table 1. Port details

Source Destination TCP Port

Capture Server (Linux)

Processing Server 1966

Discover End User desktop

Processing Server (for session retrieval) 19000

Discover End User desktop

Reporting/Portal (for the Discover Web Portal) 80

Discover Administrator desktop

to Processing and Portal/Reporting Servers (for Terminal Services access to the server) 3389

Discover Administrator desktop

Network Capture Application (for web interface setup and administration of the Network Capture Application) 8080 and/or 8443

Discover Administrator desktop

Network Capture Application (for SSH setup and administration of the Network Capture Application ) 22

Portal/Reporting Server

SMTP server (for Scorecard, Alerts e-mails) 25

Portal/Reporting Server

Processing Server (for Discover Management Server) 20000