linux - ip 转发在 CentOS7 上的 netns 内不起作用

Question

我试图用一台虚拟机和 2 个虚拟路由器构建一个虚拟网络。

VM -> Router1 -> Router2 -> External network

Router1 执行 SNAT 并且运行良好。 Router2 预计会进行 ip 转发，但不起作用。

这是我检查过的 Router2 的详细信息。 (Router2位于netns d3dcb2df-f3ca-4079-a434-491b23f84b5a内部。)

NIC 和地址

[root@controller ~]# ip netns exec qrouter-d3dcb2df-f3ca-4079-a434-491b23f84b5a ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: qr-70aabff6-c8@if60: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast state UP qlen 1000
    link/ether fa:16:3e:29:3b:ea brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 192.168.1.1/24 brd 192.168.1.255 scope global qr-70aabff6-c8
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fe29:3bea/64 scope link 
       valid_lft forever preferred_lft forever
3: qg-30c10598-27@if63: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether fa:16:3e:fc:1b:5b brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.10.52.82/24 brd 10.10.52.255 scope global qg-30c10598-27
       valid_lft forever preferred_lft forever
    inet 10.10.52.158/32 brd 10.10.52.158 scope global qg-30c10598-27
       valid_lft forever preferred_lft forever
    inet 10.10.52.73/32 brd 10.10.52.73 scope global qg-30c10598-27
       valid_lft forever preferred_lft forever
    inet 10.10.52.68/32 brd 10.10.52.68 scope global qg-30c10598-27
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fefc:1b5b/64 scope link 
       valid_lft forever preferred_lft forever

路线规则

[root@controller ~]# ip netns exec qrouter-d3dcb2df-f3ca-4079-a434-491b23f84b5a ip rule
0:	from all lookup local 
32766:	from all lookup main 
32767:	from all lookup default 
[root@controller ~]# ip netns exec qrouter-d3dcb2df-f3ca-4079-a434-491b23f84b5a ip route
default via 10.10.52.1 dev qg-30c10598-27 
10.10.52.0/24 dev qg-30c10598-27  proto kernel  scope link  src 10.10.52.82 
192.168.1.0/24 dev qr-70aabff6-c8  proto kernel  scope link  src 192.168.1.1

转发已开启

[root@controller ~]# ip netns exec qrouter-d3dcb2df-f3ca-4079-a434-491b23f84b5a sysctl net.ipv4.ip_forward
net.ipv4.ip_forward = 1
[root@controller ~]# ip netns exec qrouter-d3dcb2df-f3ca-4079-a434-491b23f84b5a sysctl net.ipv4.conf.qr-70aabff6-c8.forwarding
net.ipv4.conf.qr-70aabff6-c8.forwarding = 1
[root@controller ~]# ip netns exec qrouter-d3dcb2df-f3ca-4079-a434-491b23f84b5a sysctl net.ipv4.conf.qg-30c10598-27.forwarding
net.ipv4.conf.qg-30c10598-27.forwarding = 1

iptables规则已清除

[root@controller ~]# ip netns exec qrouter-d3dcb2df-f3ca-4079-a434-491b23f84b5a iptables -t mangle -F
[root@controller ~]# ip netns exec qrouter-d3dcb2df-f3ca-4079-a434-491b23f84b5a iptables -t nat -F
[root@controller ~]# ip netns exec qrouter-d3dcb2df-f3ca-4079-a434-491b23f84b5a iptables -t filter -F
[root@controller ~]# ip netns exec qrouter-d3dcb2df-f3ca-4079-a434-491b23f84b5a iptables -t mangle -L -n
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination         

Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination         

Chain neutron-l3-agent-FORWARD (0 references)
target     prot opt source               destination         

Chain neutron-l3-agent-INPUT (0 references)
target     prot opt source               destination         

Chain neutron-l3-agent-OUTPUT (0 references)
target     prot opt source               destination         

Chain neutron-l3-agent-POSTROUTING (0 references)
target     prot opt source               destination         

Chain neutron-l3-agent-PREROUTING (0 references)
target     prot opt source               destination         

Chain neutron-l3-agent-float-snat (0 references)
target     prot opt source               destination         

Chain neutron-l3-agent-floatingip (0 references)
target     prot opt source               destination         

Chain neutron-l3-agent-mark (0 references)
target     prot opt source               destination         

Chain neutron-l3-agent-scope (0 references)
target     prot opt source               destination         
[root@controller ~]# ip netns exec qrouter-d3dcb2df-f3ca-4079-a434-491b23f84b5a iptables -t nat -L -n
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination         

Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination         

Chain neutron-l3-agent-OUTPUT (0 references)
target     prot opt source               destination         

Chain neutron-l3-agent-POSTROUTING (0 references)
target     prot opt source               destination         

Chain neutron-l3-agent-PREROUTING (0 references)
target     prot opt source               destination         

Chain neutron-l3-agent-float-snat (0 references)
target     prot opt source               destination         

Chain neutron-l3-agent-snat (0 references)
target     prot opt source               destination         

Chain neutron-postrouting-bottom (0 references)
target     prot opt source               destination         
[root@controller ~]# ip netns exec qrouter-d3dcb2df-f3ca-4079-a434-491b23f84b5a iptables -t filter -L -n
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         

Chain neutron-filter-top (0 references)
target     prot opt source               destination         

Chain neutron-l3-agent-FORWARD (0 references)
target     prot opt source               destination         

Chain neutron-l3-agent-INPUT (0 references)
target     prot opt source               destination         

Chain neutron-l3-agent-OUTPUT (0 references)
target     prot opt source               destination         

Chain neutron-l3-agent-local (0 references)
target     prot opt source               destination         

Chain neutron-l3-agent-scope (0 references)
target     prot opt source               destination

最后，当我从虚拟机 ping 8.8.8.8 时，路由器只能看到收到的数据包，没有转发的数据包。

[root@controller ~]# ip netns exec qrouter-d3dcb2df-f3ca-4079-a434-491b23f84b5a tcpdump -i any -nn icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 65535 bytes
14:00:37.138271 IP 10.10.52.140 > 8.8.8.8: ICMP echo request, id 9616, seq 10258, length 64
14:00:38.139298 IP 10.10.52.140 > 8.8.8.8: ICMP echo request, id 9616, seq 10259, length 64
14:00:39.140488 IP 10.10.52.140 > 8.8.8.8: ICMP echo request, id 9616, seq 10260, length 64

感谢您的帮助。

Answer 1

感谢上帝。在深入研究内核源码后，我终于找到了答案。有一个小开关“rp_filter”告诉内核丢弃“坏”数据包。以下是内核文档的完整描述:

rp_filter - INTEGER
	0 - No source validation.
	1 - Strict mode as defined in RFC3704 Strict Reverse Path
	    Each incoming packet is tested against the FIB and if the interface
	    is not the best reverse path the packet check will fail.
	    By default failed packets are discarded.
	2 - Loose mode as defined in RFC3704 Loose Reverse Path
	    Each incoming packet's source address is also tested against the FIB
	    and if the source address is not reachable via any interface
	    the packet check will fail.

	Current recommended practice in RFC3704 is to enable strict mode
	to prevent IP spoofing from DDos attacks. If using asymmetric routing
	or other complicated routing, then loose mode is recommended.

	The max value from conf/{all,interface}/rp_filter is used
	when doing source validation on the {interface}.

	Default value is 0. Note that some distributions enable it
	in startup scripts.

在我的情况下，像这样关闭它很好:

ip netns exec qrouter-d3dcb2df-f3ca-4079-a434-491b23f84b5a sysctl net.ipv4.conf.all.rp_filter=0
ip netns exec qrouter-d3dcb2df-f3ca-4079-a434-491b23f84b5a sysctl net.ipv4.conf.qr-70aabff6-c8.rp_filter=0