Template is not detected

[KywoSkylake]

Hi,

I wanted to report an issue that I encountered. I am using flowd from mindrot.org to send the netflow data to my python script.
Unfortunately the Netflow implementation is not getting the template.

I checked the packets that are send by flowd and sometimes it send the information containing the template but the netflow implementation doesn't handle it. For reference I included the output and the template that is send by flowid 0.

NetFlow v9 packet detected, but no templates dict was passed! For correct parsing of packets with templates, create a 'templates' dict and pass it into the 'parse_packet' function.
(0, 172, b'\x01\x00\x00\x14\x00\x08\x00\x04\x00\x0c\x00\x04\x00\x0f\x00\x04\x00\n\x00\x02\x00\x0e\x00\x02\x00\x02\x00\x04\x00\x01\x00\x04\x00\x18\x00\x04\x00\x17\x00\x04\x00\x16\x00\x04\x00\x15\x00\x04\x00\x07\x00\x02\x00\x0b\x00\x02\x00\x06\x00\x01\x00\x04\x00\x01\x00\x05\x00\x01\x00\x10\x00\x04\x00\x11\x00\x04\x00\t\x00\x01\x00\r\x00\x01\x01\x03\x00\x14\x00\x1b\x00\x10\x00\x1c\x00\x10\x00>\x00\x10\x00\n\x00\x02\x00\x0e\x00\x02\x00\x02\x00\x04\x00\x01\x00\x04\x00\x18\x00\x04\x00\x17\x00\x04\x00\x16\x00\x04\x00\x15\x00\x04\x00\x07\x00\x02\x00\x0b\x00\x02\x00\x06\x00\x01\x00\x04\x00\x01\x00\x05\x00\x01\x00\x10\x00\x04\x00\x11\x00\x04\x00\t\x00\x01\x00\r\x00\x01')
(1, 0, 0, 20, 0, 8, 0, 4, 0, 12, 0, 4, 0, 15, 0, 4, 0, 10, 0, 2, 0, 14, 0, 2, 0, 2, 0, 4, 0, 1, 0, 4, 0, 24, 0, 4, 0, 23, 0, 4, 0, 22, 0, 4, 0, 21, 0, 4, 0, 7, 0, 2, 0, 11, 0, 2, 0, 6, 0, 1, 0, 4, 0, 1, 0, 5, 0, 1, 0, 16, 0, 4, 0, 17, 0, 4, 0, 9, 0, 1, 0, 13, 0, 1, 1, 3, 0, 20, 0, 27, 0, 16, 0, 28, 0, 16, 0, 62, 0, 16, 0, 10, 0, 2, 0, 14, 0, 2, 0, 2, 0, 4, 0, 1, 0, 4, 0, 24, 0, 4, 0, 23, 0, 4, 0, 22, 0, 4, 0, 21, 0, 4, 0, 7, 0, 2, 0, 11, 0, 2, 0, 6, 0, 1, 0, 4, 0, 1, 0, 5, 0, 1, 0, 16, 0, 4, 0, 17, 0, 4, 0, 9, 0, 1, 0, 13, 0, 1)
NetFlow v9 packet detected, but no templates dict was passed! For correct parsing of packets with templates, create a 'templates' dict and pass it into the 'parse_packet' function.

The rest is send by flowid = 256.

I wanted to define the template manually but I don't know how I could do it.

If someone has an idea on how to fix it. Please let me know.

Thank you in advance
Mike

[bitkeks]

Hi Mike, thanks for your report.

Your script is probably missing a way to store received templates, as your output says: NetFlow v9 packet detected, but no templates dict was passed! For correct parsing of packets with templates, create a 'templates' dict and pass it into the 'parse_packet' function.

This is a bit confusing at first, but you can use the reference collector as an example:

python-netflow-v9-softflowd/netflow/collector.py

Lines 118 to 129 in 71fb316

	templates = {"netflow": {}, "ipfix": {}}
	to_retry = []
	while not self._shutdown.is_set():
	try:
	# 0.5s delay to limit CPU usage while waiting for new packets
	pkt = self.input.get(block=True, timeout=0.5) # type: RawPacket
	except queue.Empty:
	continue
	try:
	# templates is passed as reference, updated in V9ExportPacket
	export = parse_packet(pkt.data, templates)

You therefore need a dict templates = {"netflow": {}, "ipfix": {}} and then pass this dict into the parse_packet function, parse_packet(pkt.data, templates). That way your received templates can be used across multiple incoming streams from flowd.