Strange issue with firewall rules

[buzzzo]

Hi

I'm facing a strange issue with the firewall rule.
I've basically create a simple host firewall rule in wich:
1) i filter ONLY inbound traffic on the lan interface (labeled: LAN) - INPUT CHAIN
2) i let all permitted outgoing - OUTPUT CHAIN
3) i deny all in the FORWARD CHAIN
4) firewall type: iptables.

attached are the screenshot of the configuration .
As you can see nothing fancy: just a bunch of rules to permit/deny.
The catchall rules is set on PERMIT.
I've added rule:9 as catchall deny rule for interface labelled lan.

What i'm expecting is that the rule 9 (id 58) is triggered only for traffic that not match the rule above it.
What happens instead is that i've receive log like that:

[2047966.859210] RULE ID 58 [REJECT] IN=ens18 OUT= MAC=9a:5d:6f:7f:07:ff:9a:56:1a:6a:9d:4a:08:00 SRC=10.0.2.102 DST=172.18.4.99 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=0 DF PROTO=TCP SPT=3306 DPT=53398 WINDOW=65160 RES=0x00 ACK SYN URGP=0

where: dst=172.18.4.99 is the host's firewall ip ens18 (LAN) and the src is another host in lan.
The interesting point in that is the spt=3306, wich is a mysql port.
So basically this is a returning traffic for a connection that host firewall (172.18.4.99) makes to 10.0.2.102 on port 3306 (so: OUTPUT chain).
But why this kind of traffic (outbound) is matched by rule 9/id:58 ? normally the returning traffic should be managed by the stateful feature of the firewall.

Am i Missing something ?

Thx n advance.

 

[Carles Munyoz]

Hi,
This is not OUTBOUND traffic, it is INBOUND.

Look at this part of your log: IN=ens18 OUT=
If this where OUTBOUND traffic the OUT field will have the name of the OUTBOUND traffic interface.

For some reason the original host is using the TCP port 3306 as source port.

 

[buzzzo]

Hi,
This is not OUTBOUND traffic, it is INBOUND.

Look at this part of your log: IN=ens18 OUT=
If this where OUTBOUND traffic the OUT field will have the name of the OUTBOUND traffic interface.

For some reason the original host is using the TCP port 3306 as source port.

Hi

Src host (10.0.2.102) is a mysql server, and the host firewall use it for some services.
As you can see rule are very simple and btw i have the same issue if i set the catchall rule to "deny".
I've added rule 9 as workaround , but it's not working.

So there are two possible reasons causing this issue:
1) i've misunderstood how to use rules
2) something in the logic behind the creation of the firewall rules is broken (maybe a bug??)

Thx

 

[Carles Munyoz]

Is the communication form your firewall with the MySQL server working?
Have you compiled and installed the policy in the firewall?

Can you attach the policy script?

 

[Carles Munyoz]

Hi,
I would like to help you in the resolution of this problem, but for it I need more information and feedback.
It is not normal this problem in a so simple stateful firewall like the one you have.

May you answer the questions of my previous post?

Maybe a tcpdump trace will help.
Can you run this tcpdump command and paste here the output?

tcpdump -n -i ens18 port 3306

 

[buzzzo]

Hi,
I would like to help you in the resolution of this problem, but for it I need more information and feedback.
It is not normal this problem in a so simple stateful firewall like the one you have.

May you answer the questions of my previous post?

Maybe a tcpdump trace will help.
Can you run this tcpdump command and paste here the output?

tcpdump -n -i ens18 port 3306

 

[buzzzo]

The issue seems to be sporadic, eg like if some packets not coming tru the right flow . I let tcpdump opened for a while to catch the dump.
The only thing coming in mind could be the fact the firewall host is a vm on a proxmox host and for some reason the networking stack of the vm "see" some other traffic.

Thx

 

[Carles Munyoz]

We usually use vm as firewalls over Proxmox hypervisors. I think that the problem is not here.

Maybe you are receiving out-of-order packages ...
In the log I see an ACK SYN packet that can come from a previous TCP flow already closed in the firewall, is it possible?
Have you altered the TCP CLOSE_WAIT timeout of your firewall?

 

[buzzzo]

We usually use vm as firewalls over Proxmox hypervisors. I think that the problem is not here.

Maybe you are receiving out-of-order packages ...
In the log I see an ACK SYN packet that can come from a previous TCP flow already closed in the firewall, is it possible?
Have you altered the TCP CLOSE_WAIT timeout of your firewall?

Hi.

The vm is simply a rocky linux with stock options. I don't think I hve modified anything.
Could you please tell me how can i check this ?

Thx

 

[Carles Munyoz]

Forget about it, if it is a fresh Linux install it not should be the origin of the problem.
I think that the tcpdump command output will be useful for analyze the problem.

 

[buzzzo]

Another strange issue related to another hosts
Simple rule to permit inbound port 4343 tcp.
The kernel logs denied (catchall rule) even if in reality is not blocked.
Same issue for port 443 on another hosts
This kingd of behaviour is "cross" hypervisor, happens both on proxmox and on vmware , on two different vm

 

[Carles Munyoz]

Can you put here screenshots of your firewall policy?

 

[buzzzo]

This issue is on vmware.

log:

[17931697.958436] RULE ID 874 [REJECT] IN=ens192 OUT= MAC=00:50:56:a4:16:d2:00:00:5e:00:01:36:08:00 SRC=3.238.30.45 DST=172.16.1.103 LEN=40 TOS=0x00 PREC=0x00 TTL=240 ID=0 DF PROTO=TCP SPT=42526 DPT=443 WINDOW=0 RES=0x00 RST URGP=0

Rule 874 is a custom "catchall" rule based on interface (ens192).
This host have 443 natted on the internet.

 

[Carles Munyoz]

We have firewalls over the VMware hypervisor. The hypervisor should not be the problem.
Why have you created this custom "catchall" rule if you already have the default catch-all rule?
Is this a stateful firewall or stateless?
The screenshot is for the input policy?

 

[buzzzo]

We have firewalls over the VMware hypervisor. The hypervisor should not be the problem.
Why have you created this custom "catchall" rule if you already have the default catch-all rule?
Is this a stateful firewall or stateless?
The screenshot is for the input policy?

I've created the rule to target more specifically the issue:
1) rule 1: permit port (eg: 443 tcp on the interface xyz
2) rule 2: deny all in on interface xyz

But the same happens with default catchall rule.
The firewall is stateful (aka: on the firewall option i specify stateful), i've not specified stateful on single rules.
screenshot is for input policy, output is all permit and forward is reject all.

 

[Carles Munyoz]

That is not normal in a so simple firewall.

Is it possible that you put here the policy load script generated by the firewall compilation process?
And can you copy and paste the output of the ip a command executed in your firewall?

 

[Carles Munyoz]

On question more ... the problem arises with all ports that you have allowed in your firewall or only with some of them?

 

[buzzzo]

Shell script

 

[buzzzo]

ip a output:
ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
2: ens192: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 00:50:56:a4:16:d2 brd ff:ff:ff:ff:ff:ff
inet 172.16.1.103/24 brd 172.16.1.255 scope global ens192
valid_lft forever preferred_lft forever

 

[Carles Munyoz]

Please, don't forget to answer my last question:

On question more ... the problem arises with all ports that you have allowed in your firewall or only with some of them?