Reverse Proxy Logs and Security
When putting everything behind a reverse proxy, one might assume that it is easy and will not pose any issues whatsoever. In fact, the truth is that it is harder and pose several more issues. One of them is regarding client IP and logging.
Previously I set up fail2ban 1 to ban not only failed SSH attempts, but also for certain web services. It is a rather simple attempt, but it should work for the most people trying to brute force themselves into the system. The problem is that for the web it relies on the logs, and the logs rely on the underlying network system giving us the IP from the client. Introducing a web proxy, basically makes the proxy an agent that represents the client, but the IP changes to that of the proxy, as a web server works on the application layer 2. Therefore the web logs will contain the proxy IP, and no the IP. Why is this a problem? Because if malicious activities are being done, their IP will be masked with the proxy IP, and suddenly fail2ban will ban the proxy, therefore everyone that use that proxy.
To solve the issue of IP masking, the proxy can rely the IP off the client as an HTTP header. Do note, that HTTP headers are not to be trusted, so this can only be done on trusted proxies. But when doing this, one also needs to modify the logging scheme, because we want to use the header as the IP. The result in the following code added as a custom config file, where it retrieves the IP from the header field X-Forwarded-For. This is sort of a standard practice, but it is fully possible to pick something else that fits your own stack.
LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" proxy_combined
In addition, to make slightly more secure, I added faillock for PAM 3 4. It is slightly hardcore change which if done wrong will destroy your login system. You basically add a couple of lines to the configs for PAM and it should enable faillock, which is used to lock accounts for a while if failed attempts reach a threshold.
This kind of system is not perfect, and there are several other systems that might work better, but for now this is a valid approach to mitigate the worst issues.