[GH-ISSUE #4798] frpc health checks leak file descriptors on FreeBSD when type = "tcp" is enabled #3788

Closed
opened 2026-05-05 14:25:33 -06:00 by gitea-mirror · 4 comments
Owner

Originally created by @snacsnoc on GitHub (May 16, 2025).
Original GitHub issue: https://github.com/fatedier/frp/issues/4798

Bug Description

I’m encountering a file descriptor leak in frpc 0.61.1 on FreeBSD when using TCP-based health checks.
After ~24 hours of runtime, frpc logs begin to show this repeating message:

health check failed: dial tcp 192.168.1.21:22: socket: too many open files in system

File descriptor usage continues to grow until system-wide limits are exhausted (kern.maxfiles was 20000, increased to 50000 as a workaround).
Once exhausted, even core system services and shells begin failing with "Too many open files".

Expected behaviour:

  • frpc should close the TCP socket after each health check attempt.
  • FD usage should remain stable, especially on failed health checks.

frpc Version

0.61.1

frps Version

0.61.1

System Architecture

freebsd/amd64

Configurations

serverAddr = "X.X.X.X"
serverPort = 7000

auth.method = "token"
auth.token = "REDACTED"
auth.additionalScopes = ["NewWorkConns", "HeartBeats"]

transport.protocol = "kcp"
transport.tcpMux = true

[[proxies]]
name = "ssh_monster"
type = "tcp"
localIP = "192.168.1.21"
localPort = 22
remotePort = 6200
transport.useEncryption = false
transport.useCompression = false

healthCheck.type = "tcp"
healthCheck.intervalSeconds = 10
healthCheck.timeoutSeconds = 3
healthCheck.maxFailed = 3

Logs

easto@monster [~/frp_0.61.1_freebsd_amd64] $ ./frpc -c new_frpc.toml
2025-05-14 22:59:07.905 [I] [sub/root.go:142] start frpc service for config file [new_frpc
.toml]
2025-05-14 22:59:07.905 [I] [client/service.go:295] try to connect to server...
2025-05-14 22:59:08.045 [I] [client/service.go:287] [f7da4460a0c6fe76] login to server suc
cess, get run id [f7da4460a0c6fe76]
2025-05-14 22:59:08.046 [I] [proxy/proxy_manager.go:173] [f7da4460a0c6fe76] proxy added: [
ssh_monster]
2025-05-14 22:59:08.046 [I] [health/health.go:123] [f7da4460a0c6fe76] [ssh_monster] health
 check status change to success
2025-05-14 22:59:08.047 [I] [proxy/proxy_wrapper.go:239] [f7da4460a0c6fe76] [ssh_monster]
health check success
2025-05-14 22:59:08.588 [I] [client/control.go:168] [f7da4460a0c6fe76] [ssh_monster] start
 proxy success
2025-05-15 22:48:27.844 [W] [health/health.go:128] [f7da4460a0c6fe76] [ssh_monster] do one
 health check failed: dial tcp 192.168.1.21:22: socket: too many open files in system
2025-05-15 22:48:37.845 [W] [health/health.go:128] [f7da4460a0c6fe76] [ssh_monster] do one
 health check failed: dial tcp 192.168.1.21:22: socket: too many open files in system
2025-05-15 22:48:47.845 [W] [health/health.go:128] [f7da4460a0c6fe76] [ssh_monster] do one
 health check failed: dial tcp 192.168.1.21:22: socket: too many open files in system
2025-05-15 22:48:47.845 [W] [health/health.go:131] [f7da4460a0c6fe76] [ssh_monster] health
 check status change to failed
2025-05-15 22:48:47.845 [I] [proxy/proxy_wrapper.go:251] [f7da4460a0c6fe76] [ssh_monster]
health check failed
2025-05-15 22:48:57.846 [W] [health/health.go:128] [f7da4460a0c6fe76] [ssh_monster] do one
 health check failed: dial tcp 192.168.1.21:22: socket: too many open files in system
2025-05-15 22:49:07.847 [W] [health/health.go:128] [f7da4460a0c6fe76] [ssh_monster] do one
 health check failed: dial tcp 192.168.1.21:22: socket: too many open files in system
2025-05-15 22:49:17.850 [W] [health/health.go:128] [f7da4460a0c6fe76] [ssh_monster] do one
 health check failed: dial tcp 192.168.1.21:22: socket: too many open files in system

Steps to reproduce

  1. Create a config file (frpc.toml) with the following:
  • A [[proxies]] block using type = "tcp"
  • healthCheck.type = "tcp" enabled with a valid but potentially unreachable localIP:localPort (e.g., 192.168.1.21:22).
  1. Start frpc normally
    ./frpc -c frpc.toml
  2. Let it run for several hours or overnight.
  3. While running, periodically monitor open file descriptors
  4. Observe socket count steadily increasing.

Affected area

  • Docs
  • Installation
  • Performance and Scalability
  • Security
  • User Experience
  • Test and Release
  • Developer Infrastructure
  • Client Plugin
  • Server Plugin
  • Extensions
  • Others
Originally created by @snacsnoc on GitHub (May 16, 2025). Original GitHub issue: https://github.com/fatedier/frp/issues/4798 ### Bug Description I’m encountering a file descriptor leak in frpc 0.61.1 on FreeBSD when using TCP-based health checks. After ~24 hours of runtime, frpc logs begin to show this repeating message: ``` health check failed: dial tcp 192.168.1.21:22: socket: too many open files in system ``` File descriptor usage continues to grow until system-wide limits are exhausted (`kern.maxfiles` was 20000, increased to 50000 as a workaround). Once exhausted, even core system services and shells begin failing with "Too many open files". **Expected behaviour:** * frpc should close the TCP socket after each health check attempt. * FD usage should remain stable, especially on failed health checks. ### frpc Version 0.61.1 ### frps Version 0.61.1 ### System Architecture freebsd/amd64 ### Configurations ``` serverAddr = "X.X.X.X" serverPort = 7000 auth.method = "token" auth.token = "REDACTED" auth.additionalScopes = ["NewWorkConns", "HeartBeats"] transport.protocol = "kcp" transport.tcpMux = true [[proxies]] name = "ssh_monster" type = "tcp" localIP = "192.168.1.21" localPort = 22 remotePort = 6200 transport.useEncryption = false transport.useCompression = false healthCheck.type = "tcp" healthCheck.intervalSeconds = 10 healthCheck.timeoutSeconds = 3 healthCheck.maxFailed = 3 ``` ### Logs ``` easto@monster [~/frp_0.61.1_freebsd_amd64] $ ./frpc -c new_frpc.toml 2025-05-14 22:59:07.905 [I] [sub/root.go:142] start frpc service for config file [new_frpc .toml] 2025-05-14 22:59:07.905 [I] [client/service.go:295] try to connect to server... 2025-05-14 22:59:08.045 [I] [client/service.go:287] [f7da4460a0c6fe76] login to server suc cess, get run id [f7da4460a0c6fe76] 2025-05-14 22:59:08.046 [I] [proxy/proxy_manager.go:173] [f7da4460a0c6fe76] proxy added: [ ssh_monster] 2025-05-14 22:59:08.046 [I] [health/health.go:123] [f7da4460a0c6fe76] [ssh_monster] health check status change to success 2025-05-14 22:59:08.047 [I] [proxy/proxy_wrapper.go:239] [f7da4460a0c6fe76] [ssh_monster] health check success 2025-05-14 22:59:08.588 [I] [client/control.go:168] [f7da4460a0c6fe76] [ssh_monster] start proxy success 2025-05-15 22:48:27.844 [W] [health/health.go:128] [f7da4460a0c6fe76] [ssh_monster] do one health check failed: dial tcp 192.168.1.21:22: socket: too many open files in system 2025-05-15 22:48:37.845 [W] [health/health.go:128] [f7da4460a0c6fe76] [ssh_monster] do one health check failed: dial tcp 192.168.1.21:22: socket: too many open files in system 2025-05-15 22:48:47.845 [W] [health/health.go:128] [f7da4460a0c6fe76] [ssh_monster] do one health check failed: dial tcp 192.168.1.21:22: socket: too many open files in system 2025-05-15 22:48:47.845 [W] [health/health.go:131] [f7da4460a0c6fe76] [ssh_monster] health check status change to failed 2025-05-15 22:48:47.845 [I] [proxy/proxy_wrapper.go:251] [f7da4460a0c6fe76] [ssh_monster] health check failed 2025-05-15 22:48:57.846 [W] [health/health.go:128] [f7da4460a0c6fe76] [ssh_monster] do one health check failed: dial tcp 192.168.1.21:22: socket: too many open files in system 2025-05-15 22:49:07.847 [W] [health/health.go:128] [f7da4460a0c6fe76] [ssh_monster] do one health check failed: dial tcp 192.168.1.21:22: socket: too many open files in system 2025-05-15 22:49:17.850 [W] [health/health.go:128] [f7da4460a0c6fe76] [ssh_monster] do one health check failed: dial tcp 192.168.1.21:22: socket: too many open files in system ``` ### Steps to reproduce 1. Create a config file (frpc.toml) with the following: * A `[[proxies]]` block using `type = "tcp"` * `healthCheck.type = "tcp"` enabled with a valid but potentially unreachable localIP:localPort (e.g., 192.168.1.21:22). 2. Start frpc normally `./frpc -c frpc.toml` 3. Let it run for several hours or overnight. 4. While running, periodically monitor open file descriptors 5. Observe socket count steadily increasing. ### Affected area - [ ] Docs - [ ] Installation - [x] Performance and Scalability - [ ] Security - [ ] User Experience - [ ] Test and Release - [ ] Developer Infrastructure - [x] Client Plugin - [ ] Server Plugin - [ ] Extensions - [ ] Others
gitea-mirror 2026-05-05 14:25:33 -06:00
Author
Owner

@fatedier commented on GitHub (May 19, 2025):

Can you use ss or netstat to check the specific connection status?

<!-- gh-comment-id:2889501745 --> @fatedier commented on GitHub (May 19, 2025): Can you use `ss` or `netstat` to check the specific connection status?
Author
Owner

@fatedier commented on GitHub (May 20, 2025):

From AI Agent:

I've analyzed the fatedier/frp code, specifically client/health/health.go, to understand the reported file descriptor leak.

Explanation of the Bug and Code Involved:

The Issue's Location: The file descriptor leak occurs when frpc uses TCP health checks on FreeBSD. The error "socket: too many open files" eventually appears during the net.Dialer.DialContext call within the doTCPCheck function in client/health/health.go.

frp's Connection Handling: The doTCPCheck function in frp correctly calls conn.Close() for successfully established TCP connections:

func (monitor *Monitor) doTCPCheck(ctx context.Context) error {
    // ...
    var d net.Dialer
    conn, err := d.DialContext(ctx, "tcp", monitor.addr) // Error occurs here
    if err != nil {
        return err // No connection to close if DialContext fails
    }
    conn.Close() // Connection is closed if dial succeeds
    return nil
}

This means frp's explicit code for managing the connection object is not the direct cause of the leak. If DialContext fails, it's the responsibility of DialContext (and the Go runtime/OS) to ensure no file descriptor is leaked for that failed attempt.

<!-- gh-comment-id:2893211797 --> @fatedier commented on GitHub (May 20, 2025): From AI Agent: I've analyzed the fatedier/frp code, specifically client/health/health.go, to understand the reported file descriptor leak. Explanation of the Bug and Code Involved: The Issue's Location: The file descriptor leak occurs when frpc uses TCP health checks on FreeBSD. The error "socket: too many open files" eventually appears during the net.Dialer.DialContext call within the doTCPCheck function in client/health/health.go. frp's Connection Handling: The doTCPCheck function in frp correctly calls conn.Close() for successfully established TCP connections: ```go func (monitor *Monitor) doTCPCheck(ctx context.Context) error { // ... var d net.Dialer conn, err := d.DialContext(ctx, "tcp", monitor.addr) // Error occurs here if err != nil { return err // No connection to close if DialContext fails } conn.Close() // Connection is closed if dial succeeds return nil } ``` This means frp's explicit code for managing the connection object is not the direct cause of the leak. If DialContext fails, it's the responsibility of DialContext (and the Go runtime/OS) to ensure no file descriptor is leaked for that failed attempt.
Author
Owner

@snacsnoc commented on GitHub (May 20, 2025):

Can you use ss or netstat to check the specific connection status?

I am away but will attempt this on the original hardware when I return, thank you. I will also attempt to replicate this in a VM

<!-- gh-comment-id:2895287984 --> @snacsnoc commented on GitHub (May 20, 2025): > Can you use `ss` or `netstat` to check the specific connection status? I am away but will attempt this on the original hardware when I return, thank you. I will also attempt to replicate this in a VM
Author
Owner

@github-actions[bot] commented on GitHub (Jun 4, 2025):

Issues go stale after 14d of inactivity. Stale issues rot after an additional 3d of inactivity and eventually close.

<!-- gh-comment-id:2937886477 --> @github-actions[bot] commented on GitHub (Jun 4, 2025): Issues go stale after 14d of inactivity. Stale issues rot after an additional 3d of inactivity and eventually close.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/frp#3788
No description provided.