[GH-ISSUE #3860] panic: close of closed channel

gitea-mirror commented

2026-05-05 13:59:12 -06:00

Owner

Originally created by @itning on GitHub (Dec 19, 2023).
Original GitHub issue: https://github.com/fatedier/frp/issues/3860

Bug Description

2023-12-19 06:59:25 panic: close of closed channel
2023-12-19 06:59:25 
2023-12-19 06:59:25 goroutine 22 [running]:
2023-12-19 06:59:25 github.com/fatedier/frp/client.(*Service).loopLoginUntilSuccess.func1()
2023-12-19 06:59:25     github.com/fatedier/frp/client/service.go:331 +0x498
2023-12-19 06:59:25 github.com/fatedier/frp/pkg/util/wait.BackoffUntil(0x400031de10, {0x862b40, 0x40003502d0}, 0x1, 0x400032ba40)
2023-12-19 06:59:25     github.com/fatedier/frp/pkg/util/wait/backoff.go:135 +0xe0
2023-12-19 06:59:25 github.com/fatedier/frp/client.(*Service).loopLoginUntilSuccess(0x40001f21c0, 0x4a817c800, 0x0)
2023-12-19 06:59:25     github.com/fatedier/frp/client/service.go:336 +0x164
2023-12-19 06:59:25 github.com/fatedier/frp/client.(*Service).keepControllerWorking.func1()
2023-12-19 06:59:25     github.com/fatedier/frp/client/service.go:198 +0x34
2023-12-19 06:59:25 github.com/fatedier/frp/pkg/util/wait.BackoffUntil(0x400031df98, {0x862b40, 0x400022d0e0}, 0x1, 0x4000114360)
2023-12-19 06:59:25     github.com/fatedier/frp/pkg/util/wait/backoff.go:135 +0xe0
2023-12-19 06:59:25 github.com/fatedier/frp/client.(*Service).keepControllerWorking(0x40001f21c0)
2023-12-19 06:59:25     github.com/fatedier/frp/client/service.go:195 +0x114
2023-12-19 06:59:25 created by github.com/fatedier/frp/client.(*Service).Run
2023-12-19 06:59:25     github.com/fatedier/frp/client/service.go:173 +0x144

frpc Version

0.53.0

frps Version

0.53.0

System Architecture

linux/amd64

Configurations

N/A

Logs

No response

Steps to reproduce

I don` know.

Affected area

Docs
Installation
Performance and Scalability
Security
User Experience
Test and Release
Developer Infrastructure
Client Plugin
Server Plugin
Extensions
Others

Originally created by @itning on GitHub (Dec 19, 2023). Original GitHub issue: https://github.com/fatedier/frp/issues/3860 ### Bug Description ```shell 2023-12-19 06:59:25 panic: close of closed channel 2023-12-19 06:59:25 2023-12-19 06:59:25 goroutine 22 [running]: 2023-12-19 06:59:25 github.com/fatedier/frp/client.(*Service).loopLoginUntilSuccess.func1() 2023-12-19 06:59:25 github.com/fatedier/frp/client/service.go:331 +0x498 2023-12-19 06:59:25 github.com/fatedier/frp/pkg/util/wait.BackoffUntil(0x400031de10, {0x862b40, 0x40003502d0}, 0x1, 0x400032ba40) 2023-12-19 06:59:25 github.com/fatedier/frp/pkg/util/wait/backoff.go:135 +0xe0 2023-12-19 06:59:25 github.com/fatedier/frp/client.(*Service).loopLoginUntilSuccess(0x40001f21c0, 0x4a817c800, 0x0) 2023-12-19 06:59:25 github.com/fatedier/frp/client/service.go:336 +0x164 2023-12-19 06:59:25 github.com/fatedier/frp/client.(*Service).keepControllerWorking.func1() 2023-12-19 06:59:25 github.com/fatedier/frp/client/service.go:198 +0x34 2023-12-19 06:59:25 github.com/fatedier/frp/pkg/util/wait.BackoffUntil(0x400031df98, {0x862b40, 0x400022d0e0}, 0x1, 0x4000114360) 2023-12-19 06:59:25 github.com/fatedier/frp/pkg/util/wait/backoff.go:135 +0xe0 2023-12-19 06:59:25 github.com/fatedier/frp/client.(*Service).keepControllerWorking(0x40001f21c0) 2023-12-19 06:59:25 github.com/fatedier/frp/client/service.go:195 +0x114 2023-12-19 06:59:25 created by github.com/fatedier/frp/client.(*Service).Run 2023-12-19 06:59:25 github.com/fatedier/frp/client/service.go:173 +0x144 ``` ### frpc Version 0.53.0 ### frps Version 0.53.0 ### System Architecture linux/amd64 ### Configurations N/A ### Logs _No response_ ### Steps to reproduce I don` know. ### Affected area - [ ] Docs - [ ] Installation - [ ] Performance and Scalability - [ ] Security - [ ] User Experience - [ ] Test and Release - [ ] Developer Infrastructure - [ ] Client Plugin - [ ] Server Plugin - [ ] Extensions - [X] Others

gitea-mirror

2026-05-05 13:59:12 -06:00

closed this issue
added the
bug
label

gitea-mirror commented

2026-05-05 13:59:14 -06:00

Author

Owner

@fatedier commented on GitHub (Dec 19, 2023):

Please provide as complete as possible the content of the log before.

@fatedier commented on GitHub (Dec 19, 2023): Please provide as complete as possible the content of the log before.

gitea-mirror commented

2026-05-05 13:59:15 -06:00

Author

Owner

@itning commented on GitHub (Dec 19, 2023):

no more info for logs.

@itning commented on GitHub (Dec 19, 2023): no more info for logs. <img width="1382" alt="image" src="https://github.com/fatedier/frp/assets/19759891/0c28c9b6-a753-4ffc-b5d8-9aa9581c7bc9">

gitea-mirror commented

2026-05-05 13:59:16 -06:00

Author

Owner

@im-zhou commented on GitHub (Dec 20, 2023):

Code

// first login to frps
svr.loopLoginUntilSuccess(10*time.Second, lo.FromPtr(svr.common.LoginFailExit))
if svr.ctl == nil {
	cancelCause := cancelErr{}
	_ = errors.As(context.Cause(svr.ctx), &cancelCause)
	return fmt.Errorf("login to the server failed: %v. With loginFailExit enabled, no additional retries will be attempted", cancelCause.Err)
}

go svr.keepControllerWorking()

经测试 loopLoginUntilSuccess 有几率触发重复登录（即显示两次login successful, 相同 runid）, 造成panic ̊ଳ ̊ᵎᵎᵎᵎ

@im-zhou commented on GitHub (Dec 20, 2023): [Code](https://github.com/fatedier/frp/blob/dev/client/service.go#L165-L174) ```go // first login to frps svr.loopLoginUntilSuccess(10*time.Second, lo.FromPtr(svr.common.LoginFailExit)) if svr.ctl == nil { cancelCause := cancelErr{} _ = errors.As(context.Cause(svr.ctx), &cancelCause) return fmt.Errorf("login to the server failed: %v. With loginFailExit enabled, no additional retries will be attempted", cancelCause.Err) } go svr.keepControllerWorking() ``` 经测试 `loopLoginUntilSuccess` 有几率触发重复登录（即显示两次login successful, 相同 runid）, 造成panic ̊ଳ ̊ᵎᵎᵎᵎ

gitea-mirror commented

2026-05-05 13:59:17 -06:00

Author

Owner

@fatedier commented on GitHub (Dec 20, 2023):

@im-zhou 你如果懂代码的话，可以看一下怎么复现和修复。修复比较简单，通过 sync.Once 可以确保只 close 一次，主要是找到原因，从根本上解决。

@fatedier commented on GitHub (Dec 20, 2023): @im-zhou 你如果懂代码的话，可以看一下怎么复现和修复。修复比较简单，通过 sync.Once 可以确保只 close 一次，主要是找到原因，从根本上解决。

gitea-mirror commented

2026-05-05 13:59:18 -06:00

Author

Owner

@alvinkwok1 commented on GitHub (Dec 20, 2023):

我也遇到了同样的问题。我的设备是ubuntu22.04上的，我采用的systemd的形式启动的只有这样才会有问题，如果是直接命令启动没有这个毛病。

c 20 21:43:41 alvinkwok frpc[1103223]: panic: close of closed channel
Dec 20 21:43:41 alvinkwok frpc[1103223]: goroutine 1 [running]:
Dec 20 21:43:41 alvinkwok frpc[1103223]: github.com/fatedier/frp/client.(*Service).loopLoginUntilSuccess.func1()
Dec 20 21:43:41 alvinkwok frpc[1103223]:         github.com/fatedier/frp/client/service.go:331 +0x517
Dec 20 21:43:41 alvinkwok frpc[1103223]: github.com/fatedier/frp/pkg/util/wait.BackoffUntil(0xc0001f7a10, {0xd2f4a0, 0xc000153a70}, 0x1, 0xc000100420)
Dec 20 21:43:41 alvinkwok frpc[1103223]:         github.com/fatedier/frp/pkg/util/wait/backoff.go:135 +0x118
Dec 20 21:43:41 alvinkwok frpc[1103223]: github.com/fatedier/frp/client.(*Service).loopLoginUntilSuccess(0xc0001c61c0, 0x2540be400, 0x1)
Dec 20 21:43:41 alvinkwok frpc[1103223]:         github.com/fatedier/frp/client/service.go:336 +0x1e9
Dec 20 21:43:41 alvinkwok frpc[1103223]: github.com/fatedier/frp/client.(*Service).Run(0xc0001c61c0, {0xd35450?, 0xc000038020?})
Dec 20 21:43:41 alvinkwok frpc[1103223]:         github.com/fatedier/frp/client/service.go:166 +0x10c
Dec 20 21:43:41 alvinkwok frpc[1103223]: github.com/fatedier/frp/cmd/frpc/sub.startService(0xc00019e840, {0xc0000411a0, 0x2, 0x2}, {0x1182358, 0x0, 0x0}, {0x7fff4b505ef3, 0x25})
Dec 20 21:43:41 alvinkwok frpc[1103223]:         github.com/fatedier/frp/cmd/frpc/sub/root.go:159 +0x37d
Dec 20 21:43:41 alvinkwok frpc[1103223]: github.com/fatedier/frp/cmd/frpc/sub.runClient({0x7fff4b505ef3, 0x25})
Dec 20 21:43:41 alvinkwok frpc[1103223]:         github.com/fatedier/frp/cmd/frpc/sub/root.go:129 +0x19c
Dec 20 21:43:41 alvinkwok frpc[1103223]: github.com/fatedier/frp/cmd/frpc/sub.glob..func2(0x1147100?, {0xb8a06f?, 0x2?, 0x2?})
Dec 20 21:43:41 alvinkwok frpc[1103223]:         github.com/fatedier/frp/cmd/frpc/sub/root.go:69 +0x4c
Dec 20 21:43:41 alvinkwok frpc[1103223]: github.com/spf13/cobra.(*Command).execute(0x1147100, {0xc000034190, 0x2, 0x2})
Dec 20 21:43:41 alvinkwok frpc[1103223]:         github.com/spf13/cobra@v1.8.0/command.go:983 +0xaaa
Dec 20 21:43:41 alvinkwok frpc[1103223]: github.com/spf13/cobra.(*Command).ExecuteC(0x1147100)
Dec 20 21:43:41 alvinkwok frpc[1103223]:         github.com/spf13/cobra@v1.8.0/command.go:1115 +0x425
Dec 20 21:43:41 alvinkwok frpc[1103223]: github.com/spf13/cobra.(*Command).Execute(...)
Dec 20 21:43:41 alvinkwok frpc[1103223]:         github.com/spf13/cobra@v1.8.0/command.go:1039
Dec 20 21:43:41 alvinkwok frpc[1103223]: github.com/fatedier/frp/cmd/frpc/sub.Execute()
Dec 20 21:43:41 alvinkwok frpc[1103223]:         github.com/fatedier/frp/cmd/frpc/sub/root.go:100 +0x25
Dec 20 21:43:41 alvinkwok frpc[1103223]: main.main()
Dec 20 21:43:41 alvinkwok frpc[1103223]:         github.com/fatedier/frp/cmd/frpc/main.go:23 +0x17
Dec 20 21:43:41 alvinkwok systemd[1]: frpc.service: Main process exited, code=exited, status=2/INVALIDARGUMENT

@alvinkwok1 commented on GitHub (Dec 20, 2023): 我也遇到了同样的问题。我的设备是ubuntu22.04上的，我采用的systemd的形式启动的只有这样才会有问题，如果是直接命令启动没有这个毛病。 ``` c 20 21:43:41 alvinkwok frpc[1103223]: panic: close of closed channel Dec 20 21:43:41 alvinkwok frpc[1103223]: goroutine 1 [running]: Dec 20 21:43:41 alvinkwok frpc[1103223]: github.com/fatedier/frp/client.(*Service).loopLoginUntilSuccess.func1() Dec 20 21:43:41 alvinkwok frpc[1103223]: github.com/fatedier/frp/client/service.go:331 +0x517 Dec 20 21:43:41 alvinkwok frpc[1103223]: github.com/fatedier/frp/pkg/util/wait.BackoffUntil(0xc0001f7a10, {0xd2f4a0, 0xc000153a70}, 0x1, 0xc000100420) Dec 20 21:43:41 alvinkwok frpc[1103223]: github.com/fatedier/frp/pkg/util/wait/backoff.go:135 +0x118 Dec 20 21:43:41 alvinkwok frpc[1103223]: github.com/fatedier/frp/client.(*Service).loopLoginUntilSuccess(0xc0001c61c0, 0x2540be400, 0x1) Dec 20 21:43:41 alvinkwok frpc[1103223]: github.com/fatedier/frp/client/service.go:336 +0x1e9 Dec 20 21:43:41 alvinkwok frpc[1103223]: github.com/fatedier/frp/client.(*Service).Run(0xc0001c61c0, {0xd35450?, 0xc000038020?}) Dec 20 21:43:41 alvinkwok frpc[1103223]: github.com/fatedier/frp/client/service.go:166 +0x10c Dec 20 21:43:41 alvinkwok frpc[1103223]: github.com/fatedier/frp/cmd/frpc/sub.startService(0xc00019e840, {0xc0000411a0, 0x2, 0x2}, {0x1182358, 0x0, 0x0}, {0x7fff4b505ef3, 0x25}) Dec 20 21:43:41 alvinkwok frpc[1103223]: github.com/fatedier/frp/cmd/frpc/sub/root.go:159 +0x37d Dec 20 21:43:41 alvinkwok frpc[1103223]: github.com/fatedier/frp/cmd/frpc/sub.runClient({0x7fff4b505ef3, 0x25}) Dec 20 21:43:41 alvinkwok frpc[1103223]: github.com/fatedier/frp/cmd/frpc/sub/root.go:129 +0x19c Dec 20 21:43:41 alvinkwok frpc[1103223]: github.com/fatedier/frp/cmd/frpc/sub.glob..func2(0x1147100?, {0xb8a06f?, 0x2?, 0x2?}) Dec 20 21:43:41 alvinkwok frpc[1103223]: github.com/fatedier/frp/cmd/frpc/sub/root.go:69 +0x4c Dec 20 21:43:41 alvinkwok frpc[1103223]: github.com/spf13/cobra.(*Command).execute(0x1147100, {0xc000034190, 0x2, 0x2}) Dec 20 21:43:41 alvinkwok frpc[1103223]: github.com/spf13/cobra@v1.8.0/command.go:983 +0xaaa Dec 20 21:43:41 alvinkwok frpc[1103223]: github.com/spf13/cobra.(*Command).ExecuteC(0x1147100) Dec 20 21:43:41 alvinkwok frpc[1103223]: github.com/spf13/cobra@v1.8.0/command.go:1115 +0x425 Dec 20 21:43:41 alvinkwok frpc[1103223]: github.com/spf13/cobra.(*Command).Execute(...) Dec 20 21:43:41 alvinkwok frpc[1103223]: github.com/spf13/cobra@v1.8.0/command.go:1039 Dec 20 21:43:41 alvinkwok frpc[1103223]: github.com/fatedier/frp/cmd/frpc/sub.Execute() Dec 20 21:43:41 alvinkwok frpc[1103223]: github.com/fatedier/frp/cmd/frpc/sub/root.go:100 +0x25 Dec 20 21:43:41 alvinkwok frpc[1103223]: main.main() Dec 20 21:43:41 alvinkwok frpc[1103223]: github.com/fatedier/frp/cmd/frpc/main.go:23 +0x17 Dec 20 21:43:41 alvinkwok systemd[1]: frpc.service: Main process exited, code=exited, status=2/INVALIDARGUMENT ```

gitea-mirror commented

2026-05-05 13:59:18 -06:00

Author

Owner

@wuqinqiang commented on GitHub (Dec 21, 2023):

粗略看了一下, 问题的关键是BackoffUntil 的逻辑无法保证 ticker 触发时机和对stopCh 信号接收的先后关系,可能stopCh 关闭的同时本轮已经开始走了,导致会多次调用相同的f,直到n轮的时候被检测然后才return

@wuqinqiang commented on GitHub (Dec 21, 2023): 粗略看了一下, 问题的关键是BackoffUntil 的逻辑无法保证 ticker 触发时机和对stopCh 信号接收的先后关系,可能stopCh 关闭的同时本轮已经开始走了,导致会多次调用相同的f,直到n轮的时候被检测然后才return

gitea-mirror commented

2026-05-05 13:59:19 -06:00

Author

Owner

@wuqinqiang commented on GitHub (Dec 21, 2023):

我觉得Backoff应该新增一个函数,区别心跳这种heartbeatWorker成功了也需要继续的逻辑,以及像login 这样的只要成功直接能退出的 @fatedier

@wuqinqiang commented on GitHub (Dec 21, 2023): 我觉得Backoff应该新增一个函数,区别心跳这种heartbeatWorker成功了也需要继续的逻辑,以及像login 这样的只要成功直接能退出的 @fatedier

gitea-mirror commented

2026-05-05 13:59:20 -06:00

Author

Owner

@z357904947 commented on GitHub (Dec 21, 2023):

我使用0.53的版本，在客户端启动的时候也出现了个错误

@z357904947 commented on GitHub (Dec 21, 2023): 我使用0.53的版本，在客户端启动的时候也出现了个错误

gitea-mirror commented

2026-05-05 13:59:20 -06:00

Author

Owner

@im-zhou commented on GitHub (Dec 21, 2023):

是否可以这样优化:

func BackoffUntil(f func() error, exitOnSuccess bool, backoff BackoffManager, sliding bool, stopCh <-chan struct{})
// ...
if err := f(); err != nil {
	previousError = true
} else {
	if exitOnSuccess {
		return
	}
	previousError = false
}

不理会stopCh, 只要首次loginFunc登录成功，就退出

测试几轮貌似没什么问题 :xd:

@im-zhou commented on GitHub (Dec 21, 2023): 是否可以这样优化: ```go func BackoffUntil(f func() error, exitOnSuccess bool, backoff BackoffManager, sliding bool, stopCh <-chan struct{}) // ... if err := f(); err != nil { previousError = true } else { if exitOnSuccess { return } previousError = false } ``` 不理会`stopCh`, 只要**首次**`loginFunc`登录成功，就退出测试几轮貌似没什么问题 :xd:

gitea-mirror commented

2026-05-05 13:59:21 -06:00

Author

Owner

@fatedier commented on GitHub (Dec 21, 2023):

@im-zhou @wuqinqiang 我觉得可以考虑扩展执行函数的返回值，从 f func() error 变更为 f func() (done bool, err error)。

由传入函数的返回决定是否需要立即退出，done 为 true 时，表示无需再继续执行。

@fatedier commented on GitHub (Dec 21, 2023): @im-zhou @wuqinqiang 我觉得可以考虑扩展执行函数的返回值，从 `f func() error` 变更为 `f func() (done bool, err error)`。由传入函数的返回决定是否需要立即退出，done 为 true 时，表示无需再继续执行。

gitea-mirror commented

2026-05-05 13:59:21 -06:00

Author

Owner

@wuqinqiang commented on GitHub (Dec 21, 2023):

@im-zhou @wuqinqiang 我觉得可以考虑扩展执行函数的返回值，从 f func() error 变更为 f func() (done bool, err error)。

由传入函数的返回决定是否需要立即退出，done 为 true 时，表示无需再继续执行。

这样没问额

但是我觉得扩展BackoffUntil(f func() error, backoff BackoffManager, sliding bool, stopCh <-chan struct{},options ...opt) 代价更小,至少api不会发生变化

另外正好请教一下,
https://github.com/fatedier/frp/blob/dev/pkg/util/wait/backoff.go#L130

https://github.com/fatedier/frp/blob/dev/pkg/util/wait/backoff.go#L140

这里为啥这样写,我没看懂,sliding的值不会改变, 感觉是逻辑bug

@wuqinqiang commented on GitHub (Dec 21, 2023): > @im-zhou @wuqinqiang 我觉得可以考虑扩展执行函数的返回值，从 `f func() error` 变更为 `f func() (done bool, err error)`。 > > 由传入函数的返回决定是否需要立即退出，done 为 true 时，表示无需再继续执行。这样没问额但是我觉得扩展BackoffUntil(f func() error, backoff BackoffManager, sliding bool, stopCh <-chan struct{},options ...opt) 代价更小,至少api不会发生变化另外正好请教一下, https://github.com/fatedier/frp/blob/dev/pkg/util/wait/backoff.go#L130 https://github.com/fatedier/frp/blob/dev/pkg/util/wait/backoff.go#L140 这里为啥这样写,我没看懂,sliding的值不会改变, 感觉是逻辑bug

gitea-mirror commented

2026-05-05 13:59:22 -06:00

Author

Owner

@fatedier commented on GitHub (Dec 21, 2023):

@wuqinqiang 同一个函数的参数有多个控制 stop 的语义，会有一点奇怪，另外不太倾向于入参里增加太多的参数，会让一个简单的函数变得越来越复杂，调用的心智负担会很重。

sliding 是额外考虑函数执行时间，一个是包括了函数执行耗时，一个是不包括。

@fatedier commented on GitHub (Dec 21, 2023): @wuqinqiang 同一个函数的参数有多个控制 stop 的语义，会有一点奇怪，另外不太倾向于入参里增加太多的参数，会让一个简单的函数变得越来越复杂，调用的心智负担会很重。 sliding 是额外考虑函数执行时间，一个是包括了函数执行耗时，一个是不包括。

gitea-mirror referenced this issue

2026-05-05 14:46:36 -06:00

[PR #3068] [CLOSED] Added OpExit #4676