[GH-ISSUE #3915] Unable to get https reverse tunnel working #3105

Closed
opened 2026-05-05 14:00:42 -06:00 by gitea-mirror · 3 comments
Owner

Originally created by @Chuckame on GitHub (Jan 7, 2024).
Original GitHub issue: https://github.com/fatedier/frp/issues/3915

Bug Description

When having Caddy as frontend reverse proxy, forwarding traffic to an https client is not working. The traffic is going to the home caddy instance, but showing an error (TLS handshake error from 172.19.0.3:48122: no certificate available for '172.19.0.6').

Here is the infrastructure:

flowchart TD
  aa(remote user) -.->|https://sub.domain.tld| a
  bb(home user) -.->|https://sub.domain.tld| d
subgraph VPS
  a(Caddy server) -->|https://172.27.0.4:84| b(tunnel server)
end
  b ===|https tunnel| c(tunnel client: 172.19.0.3)
subgraph home server behind NAT
  c -->|https://caddy:443| d(Caddy server: 172.19.0.6)
  d -->|http| e(app)
end

frpc Version

0.53.2

frps Version

0.53.2

System Architecture

linux/amd64

Configurations

caddy (server):

{
	admin off
	acme_dns ovh {
		endpoint {env.OVH_ENDPOINT}
		application_key {env.OVH_APP_KEY}
		application_secret {env.OVH_APP_SECRET}
		consumer_key {env.OVH_CONSUMER_KEY}
	}
}
sub.domain.tld {
	reverse_proxy https://172.27.0.4:84 {
		header_up Host sub.domain.tld
	}
}

server:

bindPort = 7003
vhostHTTPSPort = 84
auth.token = ...
log.level = "trace"

client:

serverAddr = "domain.tld"
serverPort = 7003
auth.token = ...
log.level = "trace"

[[proxies]]
name = "https-tunnel"
type = "https"
localIP = "caddy" # even if I put sub.domain.tld, the caddy server keeps failing because of the cert not found for the IP instead of the domain name
localPort = 443
customDomains = ["*"] # not working with sub.domain.tld as the host is empty ...

caddy (client):

{
	admin off
	acme_dns ovh {
		endpoint {env.OVH_ENDPOINT}
		application_key {env.OVH_APP_KEY}
		application_secret {env.OVH_APP_SECRET}
		consumer_key {env.OVH_CONSUMER_KEY}
	}
}
sub.domain.tld {
	respond "pong"
}

Logs

server:

2024/01/07 00:04:57 [I] [service.go:225] frps tcp listen on 0.0.0.0:7003
2024/01/07 00:04:57 [I] [service.go:306] https service listen on 0.0.0.0:84
2024/01/07 00:04:57 [I] [root.go:113] frps started successfully
2024/01/07 00:05:00 [T] [service.go:482] start check TLS connection...
2024/01/07 00:05:00 [T] [service.go:492] check TLS connection success, isTLS: true custom: false internal: false
2024/01/07 00:05:00 [I] [service.go:563] [89bd521ee0e4a758] client login info: ip [<public-ip>:42754] version [0.53.2] hostname [] os [linux] arch [amd64]
2024/01/07 00:05:00 [I] [https.go:67] [89bd521ee0e4a758] [https-tunnel] https proxy listen for host [*]
2024/01/07 00:05:00 [I] [control.go:401] [89bd521ee0e4a758] new proxy [https-tunnel] type [https] success
2024/01/07 00:05:00 [D] [control.go:432] [89bd521ee0e4a758] receive heartbeat
...
# when request is made on https://sub.domain.tld/
2024/01/07 00:05:00 [D] [control.go:243] [89bd521ee0e4a758] new work connection registered
2024/01/07 00:05:08 [D] [vhost.go:247] [89bd521ee0e4a758] [https-tunnel] new request host [] path [] httpUser []
2024/01/07 00:05:08 [I] [proxy.go:204] [89bd521ee0e4a758] [https-tunnel] get a user connection [172.27.0.12:56356]
2024/01/07 00:05:08 [D] [control.go:272] [89bd521ee0e4a758] get work connection from pool
2024/01/07 00:05:08 [D] [proxy.go:131] [89bd521ee0e4a758] [https-tunnel] get a new work connection: [<client-ip>:42754]
2024/01/07 00:05:08 [T] [proxy.go:240] [89bd521ee0e4a758] [https-tunnel] handler user tcp connection, use_encryption: false, use_compression: false
2024/01/07 00:05:08 [D] [proxy.go:261] [89bd521ee0e4a758] [https-tunnel] join connections, workConn(l[172.27.0.4:7003] r[<client-ip>:42754]) userConn(l[172.27.0.4:84] r[172.27.0.12:56356])
2024/01/07 00:05:08 [D] [control.go:243] [89bd521ee0e4a758] new work connection registered
2024/01/07 00:05:08 [D] [proxy.go:271] [89bd521ee0e4a758] [https-tunnel] join connections closed

client:

2024/01/07 00:24:17 [I] [root.go:141] start frpc service for config file [/etc/frp/frpc.toml]
2024/01/07 00:24:17 [I] [service.go:287] try to connect to server...
2024/01/07 00:24:18 [I] [service.go:279] [4f163207fe46d751] login to server success, get run id [4f163207fe46d751]
2024/01/07 00:24:18 [I] [proxy_manager.go:173] [4f163207fe46d751] proxy added: [https-tunnel]
2024/01/07 00:24:18 [T] [proxy_wrapper.go:200] [4f163207fe46d751] [https-tunnel] change status from [new] to [wait start]
2024/01/07 00:24:18 [D] [control.go:243] [4f163207fe46d751] send heartbeat to server
2024/01/07 00:24:18 [I] [control.go:169] [4f163207fe46d751] [https-tunnel] start proxy success
2024/01/07 00:24:18 [D] [control.go:194] [4f163207fe46d751] receive heartbeat from server
2024/01/07 00:24:27 [D] [proxy_wrapper.go:260] [4f163207fe46d751] [https-tunnel] start a new work connection, localAddr: 172.19.0.3:52948 remoteAddr: <public-ip>:7003
2024/01/07 00:24:27 [T] [proxy.go:144] [4f163207fe46d751] [https-tunnel] handle tcp work connection, useEncryption: false, useCompression: false
2024/01/07 00:24:27 [D] [proxy.go:208] [4f163207fe46d751] [https-tunnel] join connections, localConn(l[172.19.0.3:53772] r[172.19.0.6:443]) workConn(l[172.19.0.3:52948] r[<public-ip>:7003])
2024/01/07 00:24:27 [D] [proxy.go:220] [4f163207fe46d751] [https-tunnel] join connections closed

home caddy logs:

DBG ts=1704587067.3488138 logger=tls.handshake msg=no matching certificates and no custom selection logic identifier=172.19.0.6
DBG ts=1704587067.3488328 logger=tls.handshake msg=no certificate matching TLS ClientHello remote_ip=172.19.0.3 remote_port=53772 server_name= remote=172.19.0.3:53772 identifier=172.19.0.6 cipher_suites=[52393,52392,49195,49199,49196,49200,49161,49171,49162,49172,156,157,47,53,49170,10,4867,4865,4866] cert_cache_fill=0.0004 load_or_obtain_if_necessary=true on_demand=false
DBG ts=1704587067.3489401 logger=http.stdlib msg=http: TLS handshake error from 172.19.0.3:53772: no certificate available for '172.19.0.6'

Steps to reproduce

  1. take the config
  2. call https://sub.domain.tld/

Affected area

  • Docs
  • Installation
  • Performance and Scalability
  • Security
  • User Experience
  • Test and Release
  • Developer Infrastructure
  • Client Plugin
  • Server Plugin
  • Extensions
  • Others
Originally created by @Chuckame on GitHub (Jan 7, 2024). Original GitHub issue: https://github.com/fatedier/frp/issues/3915 ### Bug Description When having Caddy as frontend reverse proxy, forwarding traffic to an https client is not working. The traffic is going to the home caddy instance, but showing an error (`TLS handshake error from 172.19.0.3:48122: no certificate available for '172.19.0.6'`). Here is the infrastructure: ```mermaid flowchart TD aa(remote user) -.->|https://sub.domain.tld| a bb(home user) -.->|https://sub.domain.tld| d subgraph VPS a(Caddy server) -->|https://172.27.0.4:84| b(tunnel server) end b ===|https tunnel| c(tunnel client: 172.19.0.3) subgraph home server behind NAT c -->|https://caddy:443| d(Caddy server: 172.19.0.6) d -->|http| e(app) end ``` ### frpc Version 0.53.2 ### frps Version 0.53.2 ### System Architecture linux/amd64 ### Configurations caddy (server): ``` { admin off acme_dns ovh { endpoint {env.OVH_ENDPOINT} application_key {env.OVH_APP_KEY} application_secret {env.OVH_APP_SECRET} consumer_key {env.OVH_CONSUMER_KEY} } } sub.domain.tld { reverse_proxy https://172.27.0.4:84 { header_up Host sub.domain.tld } } ``` server: ```toml bindPort = 7003 vhostHTTPSPort = 84 auth.token = ... log.level = "trace" ``` client: ```toml serverAddr = "domain.tld" serverPort = 7003 auth.token = ... log.level = "trace" [[proxies]] name = "https-tunnel" type = "https" localIP = "caddy" # even if I put sub.domain.tld, the caddy server keeps failing because of the cert not found for the IP instead of the domain name localPort = 443 customDomains = ["*"] # not working with sub.domain.tld as the host is empty ... ``` caddy (client): ``` { admin off acme_dns ovh { endpoint {env.OVH_ENDPOINT} application_key {env.OVH_APP_KEY} application_secret {env.OVH_APP_SECRET} consumer_key {env.OVH_CONSUMER_KEY} } } sub.domain.tld { respond "pong" } ``` ### Logs server: ``` 2024/01/07 00:04:57 [I] [service.go:225] frps tcp listen on 0.0.0.0:7003 2024/01/07 00:04:57 [I] [service.go:306] https service listen on 0.0.0.0:84 2024/01/07 00:04:57 [I] [root.go:113] frps started successfully 2024/01/07 00:05:00 [T] [service.go:482] start check TLS connection... 2024/01/07 00:05:00 [T] [service.go:492] check TLS connection success, isTLS: true custom: false internal: false 2024/01/07 00:05:00 [I] [service.go:563] [89bd521ee0e4a758] client login info: ip [<public-ip>:42754] version [0.53.2] hostname [] os [linux] arch [amd64] 2024/01/07 00:05:00 [I] [https.go:67] [89bd521ee0e4a758] [https-tunnel] https proxy listen for host [*] 2024/01/07 00:05:00 [I] [control.go:401] [89bd521ee0e4a758] new proxy [https-tunnel] type [https] success 2024/01/07 00:05:00 [D] [control.go:432] [89bd521ee0e4a758] receive heartbeat ... # when request is made on https://sub.domain.tld/ 2024/01/07 00:05:00 [D] [control.go:243] [89bd521ee0e4a758] new work connection registered 2024/01/07 00:05:08 [D] [vhost.go:247] [89bd521ee0e4a758] [https-tunnel] new request host [] path [] httpUser [] 2024/01/07 00:05:08 [I] [proxy.go:204] [89bd521ee0e4a758] [https-tunnel] get a user connection [172.27.0.12:56356] 2024/01/07 00:05:08 [D] [control.go:272] [89bd521ee0e4a758] get work connection from pool 2024/01/07 00:05:08 [D] [proxy.go:131] [89bd521ee0e4a758] [https-tunnel] get a new work connection: [<client-ip>:42754] 2024/01/07 00:05:08 [T] [proxy.go:240] [89bd521ee0e4a758] [https-tunnel] handler user tcp connection, use_encryption: false, use_compression: false 2024/01/07 00:05:08 [D] [proxy.go:261] [89bd521ee0e4a758] [https-tunnel] join connections, workConn(l[172.27.0.4:7003] r[<client-ip>:42754]) userConn(l[172.27.0.4:84] r[172.27.0.12:56356]) 2024/01/07 00:05:08 [D] [control.go:243] [89bd521ee0e4a758] new work connection registered 2024/01/07 00:05:08 [D] [proxy.go:271] [89bd521ee0e4a758] [https-tunnel] join connections closed ``` client: ``` 2024/01/07 00:24:17 [I] [root.go:141] start frpc service for config file [/etc/frp/frpc.toml] 2024/01/07 00:24:17 [I] [service.go:287] try to connect to server... 2024/01/07 00:24:18 [I] [service.go:279] [4f163207fe46d751] login to server success, get run id [4f163207fe46d751] 2024/01/07 00:24:18 [I] [proxy_manager.go:173] [4f163207fe46d751] proxy added: [https-tunnel] 2024/01/07 00:24:18 [T] [proxy_wrapper.go:200] [4f163207fe46d751] [https-tunnel] change status from [new] to [wait start] 2024/01/07 00:24:18 [D] [control.go:243] [4f163207fe46d751] send heartbeat to server 2024/01/07 00:24:18 [I] [control.go:169] [4f163207fe46d751] [https-tunnel] start proxy success 2024/01/07 00:24:18 [D] [control.go:194] [4f163207fe46d751] receive heartbeat from server 2024/01/07 00:24:27 [D] [proxy_wrapper.go:260] [4f163207fe46d751] [https-tunnel] start a new work connection, localAddr: 172.19.0.3:52948 remoteAddr: <public-ip>:7003 2024/01/07 00:24:27 [T] [proxy.go:144] [4f163207fe46d751] [https-tunnel] handle tcp work connection, useEncryption: false, useCompression: false 2024/01/07 00:24:27 [D] [proxy.go:208] [4f163207fe46d751] [https-tunnel] join connections, localConn(l[172.19.0.3:53772] r[172.19.0.6:443]) workConn(l[172.19.0.3:52948] r[<public-ip>:7003]) 2024/01/07 00:24:27 [D] [proxy.go:220] [4f163207fe46d751] [https-tunnel] join connections closed ``` home caddy logs: ``` DBG ts=1704587067.3488138 logger=tls.handshake msg=no matching certificates and no custom selection logic identifier=172.19.0.6 DBG ts=1704587067.3488328 logger=tls.handshake msg=no certificate matching TLS ClientHello remote_ip=172.19.0.3 remote_port=53772 server_name= remote=172.19.0.3:53772 identifier=172.19.0.6 cipher_suites=[52393,52392,49195,49199,49196,49200,49161,49171,49162,49172,156,157,47,53,49170,10,4867,4865,4866] cert_cache_fill=0.0004 load_or_obtain_if_necessary=true on_demand=false DBG ts=1704587067.3489401 logger=http.stdlib msg=http: TLS handshake error from 172.19.0.3:53772: no certificate available for '172.19.0.6' ``` ### Steps to reproduce 1. take the config 2. call https://sub.domain.tld/ ### Affected area - [ ] Docs - [ ] Installation - [ ] Performance and Scalability - [ ] Security - [ ] User Experience - [ ] Test and Release - [ ] Developer Infrastructure - [ ] Client Plugin - [X] Server Plugin - [ ] Extensions - [ ] Others
Author
Owner

@Chuckame commented on GitHub (Jan 7, 2024):

Kind of solved (hours passed on it...): I needed the frps container with a dns alias to the same as the request hostname, and remote caddy must target this dns alias instead of the IP.

It would be great that frp sets this hostname automagically, because I don't know how to do with wildcards.

<!-- gh-comment-id:1879896802 --> @Chuckame commented on GitHub (Jan 7, 2024): Kind of solved (hours passed on it...): I needed the frps container with a dns alias to the same as the request hostname, and remote caddy must target this dns alias instead of the IP. It would be great that frp sets this hostname automagically, because I don't know how to do with wildcards.
Author
Owner

@fatedier commented on GitHub (Jan 8, 2024):

Obviously, you should configure the correct SNI name (host) for caddy to forward HTTPS requests, rather than using the IP. I think this is a common configuration for reverse proxy functionality.

For https, frps has no modifications to the request.

<!-- gh-comment-id:1880324316 --> @fatedier commented on GitHub (Jan 8, 2024): Obviously, you should configure the correct SNI name (host) for caddy to forward HTTPS requests, rather than using the IP. I think this is a common configuration for reverse proxy functionality. For https, frps has no modifications to the request.
Author
Owner

@Chuckame commented on GitHub (Jan 8, 2024):

I fixed it with overriding the SNI... The param and the workflow is not that clear when we are new to tls/https world.

<!-- gh-comment-id:1880887168 --> @Chuckame commented on GitHub (Jan 8, 2024): I fixed it with overriding the SNI... The param and the workflow is not that clear when we are new to tls/https world.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/frp#3105
No description provided.