[GH-ISSUE #1867] [Question] About the "fake" ACK sent by frps #1475

Closed
opened 2026-05-05 12:56:00 -06:00 by gitea-mirror · 4 comments
Owner

Originally created by @database64128 on GitHub (Jun 18, 2020).
Original GitHub issue: https://github.com/fatedier/frp/issues/1867

Issue is only used for submiting bug report and documents typo. If there are same issues or answers can be found in documents, we will close it directly.

Use the commands below to provide key information from your environment:
You do NOT have to include this information if this is a FEATURE REQUEST

What version of frp are you using (./frpc -v or ./frps -v)?
0.32.1

What operating system and processor architecture are you using (go env)?

GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOENV="/root/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/root/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build048169722=/tmp/go-build -gno-record-gcc-switches"

Configures you used:
TCP

Steps to reproduce the issue:

  1. Configure the frpc client to redirect a remote service to frps's port 7016.
  2. Connect to the remote service via frps's port 7016.
  3. The link between frpc and the remote service is broken for some reason. All packets sent from frpc to the remote service are dropped/timed-out without any ACK from the remote service.

Describe the results you received:
The TCP connection between the service client and frps is still established. Because upon receiving a [PSH, ACK] packet from the service client, frps immediately sends an ACK, making the service client think the connection is still alive, whereas in fact, the connection between frpc and the remote service is already dead. The service client, in my case is ssh, hangs forever because of this.

Describe the results you expected:

  1. Why would frps immediately responds with an ACK upon receiving an [PSH, ACK] packet? Shouldn't it just relay the ACK coming from the remote service?
  2. frps just blindly responds with an ACK, even if the remote service is no longer reachable.
  3. If responding with ACK immediately is justified, I would expect it to detect if the connection between frpc and the remote service is still alive, and close the connection if no ACK or any packets are coming from the remote service. Or just do its job of relaying TCP packets, and don't respond with ACK on its own.

Additional information you deem important (e.g. issue happens only occasionally):
The above statements and conclusions are backed by WireShark captures.

Can you point out what caused this issue (optional)
frps blindly responds with an ACK as soon as it received transmission from the service client.

Originally created by @database64128 on GitHub (Jun 18, 2020). Original GitHub issue: https://github.com/fatedier/frp/issues/1867 Issue is only used for submiting bug report and documents typo. If there are same issues or answers can be found in documents, we will close it directly. Use the commands below to provide key information from your environment: You do NOT have to include this information if this is a FEATURE REQUEST **What version of frp are you using (./frpc -v or ./frps -v)?** `0.32.1` **What operating system and processor architecture are you using (`go env`)?** ``` GO111MODULE="" GOARCH="amd64" GOBIN="" GOCACHE="/root/.cache/go-build" GOENV="/root/.config/go/env" GOEXE="" GOFLAGS="" GOHOSTARCH="amd64" GOHOSTOS="linux" GOINSECURE="" GONOPROXY="" GONOSUMDB="" GOOS="linux" GOPATH="/root/go" GOPRIVATE="" GOPROXY="https://proxy.golang.org,direct" GOROOT="/usr/lib/go" GOSUMDB="sum.golang.org" GOTMPDIR="" GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64" GCCGO="gccgo" AR="ar" CC="gcc" CXX="g++" CGO_ENABLED="1" GOMOD="" CGO_CFLAGS="-g -O2" CGO_CPPFLAGS="" CGO_CXXFLAGS="-g -O2" CGO_FFLAGS="-g -O2" CGO_LDFLAGS="-g -O2" PKG_CONFIG="pkg-config" GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build048169722=/tmp/go-build -gno-record-gcc-switches" ``` **Configures you used:** TCP **Steps to reproduce the issue:** 1. Configure the `frpc` client to redirect a remote service to `frps`'s port 7016. 2. Connect to the remote service via `frps`'s port 7016. 3. The link between `frpc` and the remote service is broken for some reason. All packets sent from `frpc` to the remote service are dropped/timed-out without any `ACK` from the remote service. **Describe the results you received:** The TCP connection between the service client and `frps` is still established. Because upon receiving a `[PSH, ACK]` packet from the service client, `frps` immediately sends an `ACK`, making the service client think the connection is still alive, whereas in fact, the connection between `frpc` and the remote service is already dead. The service client, in my case is `ssh`, hangs forever because of this. **Describe the results you expected:** 1. Why would `frps` immediately responds with an `ACK` upon receiving an `[PSH, ACK]` packet? Shouldn't it just relay the `ACK` coming from the remote service? 2. `frps` just blindly responds with an `ACK`, even if the remote service is no longer reachable. 3. If responding with `ACK` immediately is justified, I would expect it to detect if the connection between `frpc` and the remote service is still alive, and close the connection if no `ACK` or any packets are coming from the remote service. Or just do its job of relaying TCP packets, and don't respond with `ACK` on its own. **Additional information you deem important (e.g. issue happens only occasionally):** The above statements and conclusions are backed by WireShark captures. **Can you point out what caused this issue (optional)** `frps` blindly responds with an `ACK` as soon as it received transmission from the service client.
Author
Owner

@fatedier commented on GitHub (Jun 19, 2020):

I'm not sure i have already understand your problem. Can you point out the releated code?

Do you mean that connection between your ssh client and frps is ok when connection between frpc and your ssh service is broken?

There are different situation:

  1. If your ssh service isn't listen on this port, frps will get connection refushed and reset your ssh client connection.
  2. Configure local_ip but no route to it, frpc will always try to dial to it until you cancel your client connection or timeout.
  3. Connection has been established sucessfully but network is broken. It relays on your client to detect the failture by heartbeat.

What's your situation?

<!-- gh-comment-id:646410983 --> @fatedier commented on GitHub (Jun 19, 2020): I'm not sure i have already understand your problem. Can you point out the releated code? Do you mean that connection between your ssh client and frps is ok when connection between frpc and your ssh service is broken? There are different situation: 1. If your ssh service isn't listen on this port, frps will get connection refushed and reset your ssh client connection. 2. Configure `local_ip` but no route to it, frpc will always try to dial to it until you cancel your client connection or timeout. 3. Connection has been established sucessfully but network is broken. It relays on your client to detect the failture by heartbeat. What's your situation?
Author
Owner

@database64128 commented on GitHub (Jun 20, 2020):

I'm not sure i have already understand your problem. Can you point out the releated code?

I'm not familiar with Golang 😅.

Do you mean that connection between your ssh client and frps is ok when connection between frpc and your ssh service is broken?

Yes.

What's your situation?
3. Connection has been established sucessfully but network is broken. It relays on your client to detect the failture by heartbeat.

Case 3 would best describe my situation.

image

This is how a typical SSH connection looks like: Press a key in the terminal. The SSH client sends packet 1. The server sends packet 2 to update the texts in the terminal. And finally an ACK from the client.

If the network is broken for some reason, you would only see packet 1. Then the SSH connection would timeout after 30 seconds without any response from the server.

image

And this is how it looks like when the SSH client connects to a sshd port forwarded by frps. When you press a key in the terminal, the SSH client sends packet 1. Then frps immediately responds with an ACK. But the actual response from sshd only comes in packet 3.

When the network between frpc and the remote sshd is broken:

  1. On the frpc side, it couldn't receive any packet from the remote service.
  2. On the frps side, frps continues responding any PSH packet from the SSH client with an ACK. Because the SSH client still receives these "fake" ACK packets, it thinks the connection is intact, so it wouldn't trigger the timeout mechanism. But no actual data coming from the server means it can't update anything in the terminal, causing the terminal to freeze forever. In this case when capturing packets, you would only see packet 1 and packet 2. This behavior of frps sending ACK on its own is causing the problem.
<!-- gh-comment-id:646920327 --> @database64128 commented on GitHub (Jun 20, 2020): > I'm not sure i have already understand your problem. Can you point out the releated code? I'm not familiar with Golang 😅. > Do you mean that connection between your ssh client and frps is ok when connection between frpc and your ssh service is broken? Yes. > What's your situation? > 3. Connection has been established sucessfully but network is broken. It relays on your client to detect the failture by heartbeat. Case 3 would best describe my situation. ![image](https://user-images.githubusercontent.com/18757988/85188546-fd90e400-b2d9-11ea-8062-118054fed74f.png) This is how a typical SSH connection looks like: Press a key in the terminal. The SSH client sends packet 1. The server sends packet 2 to update the texts in the terminal. And finally an `ACK` from the client. If the network is broken for some reason, you would only see packet 1. Then the SSH connection would timeout after 30 seconds without any response from the server. ![image](https://user-images.githubusercontent.com/18757988/85188621-c242e500-b2da-11ea-96c8-11d6e2b29da8.png) And this is how it looks like when the SSH client connects to a `sshd` port forwarded by `frps`. When you press a key in the terminal, the SSH client sends packet 1. Then `frps` immediately responds with an `ACK`. But the actual response from `sshd` only comes in packet 3. When the network between `frpc` and the remote `sshd` is broken: 1. On the `frpc` side, it couldn't receive any packet from the remote service. 2. On the `frps` side, `frps` continues responding any `PSH` packet from the SSH client with an `ACK`. Because the SSH client still receives these "fake" `ACK` packets, it thinks the connection is intact, so it wouldn't trigger the timeout mechanism. But no actual data coming from the server means it can't update anything in the terminal, causing the terminal to freeze forever. In this case when capturing packets, you would only see packet 1 and packet 2. This behavior of `frps` sending `ACK` on its own is causing the problem.
Author
Owner

@fatedier commented on GitHub (Jun 20, 2020):

The problem is frpc can't detect the network error like this. TCP socket won't return error.

Try this heatlh check to solve the problem?

If network between sshd service and frpc has error, frps will close your ssh connections.

<!-- gh-comment-id:646982709 --> @fatedier commented on GitHub (Jun 20, 2020): The problem is `frpc` can't detect the network error like this. TCP socket won't return error. Try this [heatlh check](https://github.com/fatedier/frp#service-health-check) to solve the problem? If network between sshd service and frpc has error, frps will close your ssh connections.
Author
Owner

@database64128 commented on GitHub (Jun 21, 2020):

Thanks for the suggestion!

<!-- gh-comment-id:647141844 --> @database64128 commented on GitHub (Jun 21, 2020): Thanks for the suggestion!
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/frp#1475
No description provided.