For those of us who write code, proxy is an all-too-familiar term, but it’s often puzzling that sometimes we need to set up a proxy, for example, when using Go, we often have to do something like this
[[email protected].io]# HTTP_PROXY=127.0.0.1:1080 go get github.com/liuliqiang/log4go
However, sometimes when we visit google.com, we can access it directly in the browser, so what’s the difference. This is actually the so-called transparent proxy, the so-called transparent proxy, simply put, is the application does not need to know whether there is a proxy server, for example, we often use the “rule-based global proxy” can probably be considered a transparent proxy, because we access the network, we use applications such as Chrome are unaware of the presence of a proxy.
For example, if you try to set a non-existent HTTP_PROXY variable and write an http client program in Go, you will find that your http client program does not work properly because Go’s default HTTP Client library is concerned about whether your environment variable contains HTTP_PROXY, and if it does, it will actively set it as a proxy for your http client, so setting a wrong proxy will result in different access, which is a problem you often encounter when developing locally.
Under Linux, the kernel has supported transparent proxies since version 2.2. This feature is called TProxy and is also based on Netfilter, common tools are
- iptables: a very common network tool that directly supports TProxy
- nftables is a framework developed by the Netfilter project team, mainly used for packet filtering, NAT and other packet modification operations, however, this tool is mainly used in Debian, it is used in Debain instead of iptables.
Uses of Tproxy
Don’t limit the use of TProxy to our simple, everyday needs of going over walls, there are actually applications for TProxy in industry, here are a few scenarios.
Receiving packets that do not belong to the local machine
In my previous talk on iptables (iptables in depth), I described the processing flow of a datagram after it has been network stacked as follows
|Figure 1: iptables data flow diagram|
After the PreRouting Filter, a target address determination is made to see if it is a local datagram, if not, it is forwarded out and the local application is not informed, then through the tproxy, we can do something to send the datagram to our local application as well.
[[email protected].io]# iptables -t mangle -N DIVERT
[[email protected].io]# iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT
[[email protected].io]# iptables -t mangle -A DIVERT -j MARK --set-mark 1
[[email protected].io]# iptables -t mangle -A DIVERT -j TPROXY --on-port 1080
[[email protected].io]# ip rule add fwmark 1 lookup 100
[[email protected].io]# ip route add local 0.0.0.0/0 dev lo table 100
ok, you can see the procedure here.
- add a custom chain of iptables (my previous introductory article E8%87%AA%E5%AE%9A%E4%B9%89%E9%93%BE))
- if it’s a socket, send it to the custom chain
- mark all packets entering the custom chain
- accept the datagram
- look up the table with ID 100 for all packets with mark 1
- route the packets locally
At this point, the packet may be received by the local application, but to ensure that it is received by the application, you need to set the socket options specifically: ```.
[[email protected].io]# cat main.c
fd = socket(AF_INET, SOCK_STREAM, 0);
int value = 1;
setsockopt(fd, SOL_IP, IP_TRANSPARENT, &value, sizeof(value));
name.sin_family = AF_INET;
name.sin_port = htons(0x1080);
name.sin_addr.s_addr = htonl(0xDEADBEEF);
bind(fd, &name, sizeof(name));
OK, so what does this do? This is a very easy to understand transparent proxy function, if you want to implement a port-based transparent proxy, then you can use this set, for example, to implement Istio similar functions, how to implement it depends on how to write the code of your application part.
This is a DNAT-like feature that works like this, I have three machines.
- 10.0.0.4: 80, 50080
Now I want to access 10.0.0.4:80 through 10.0.0.3, but the application is actually working on port 50080, so it works like this.
[[email protected].io]# iptables -t mangle -A PREROUTING -p tcp --dport 50080 -j TPROXY \
--tproxy-mark 0x1/0x1 --on-port 80
Application direct support
This is a feature of applications such as Squid and Envoy configuration/listeners/listener_filters/original_dst_filter) are both supported.
This one is not introduced, just read someone else’s article: Transparent Proxy: TPROXY