[GH-ISSUE #3298] using nginx reverse proxy with frp and type = tcp #2641

Closed
opened 2026-05-05 13:42:17 -06:00 by gitea-mirror · 2 comments
Owner

Originally created by @zaalgol on GitHub (Feb 9, 2023).
Original GitHub issue: https://github.com/fatedier/frp/issues/3298

Bug Description

In our application, we create a frp tunnel from the local-server to the public server, so the public server can send data to the local-server. In the config we defined the type to be http, and defined a domain.
The public server uses another proxy - nginx reverse proxy to route client requests to the local server address..
It works great.

We have a customer that wants to add his private proxy, that all data from public server to local-server should pass through it. Because frp supports http_proxy only with type=tcp, as a beginning I changed config to type=tcp, but because the type=tpc doesn’t support domains, I get a error in the public server frp:
[http.go:92] do http proxy request error: no such domain: SUBDOMAIN GENERATED FROM SECRET AND RAW SUBDOMAIN.RAW SUB DOMAIN

frpc Version

0.35.1

frps Version

0.35.1

System Architecture

linux VM on a cloud

Configurations

frpc.ini:

[common]
server_addr = <PUBLIC SERVER ADDRESS>
server_port = 5223

#LOGGING
log_file = /var/log/supervisor/frpc.log
log_level = debug
log_max_days = 3
disable_log_color = false

#AUTHEXTRAS
authenticate_heartbeats = false
authenticate_new_work_conns = false
token = TOKEN

#set admin address for control frpc's action by http api such as reload
admin_addr = 0.0.0.0
admin_port = 7400

#connections will be established in advance, default value is zero
pool_count = 2

#if tcp stream multiplexing is used, default is true, it must be same with frps
tcp_mux = true

#decide if exit program when first login failed, otherwise continuous relogin to frps
login_fail_exit = false

#communication protocol used to connect to server
#now it supports tcp, kcp and websocket, default is tcp
protocol = tcp

#TLS
tls_enable = true
tls_cert_file = /certs/client.crt
tls_key_file = /certs/client.key

[<SUBDOMAIN GENERATED FROM SECRET AND RAW SUBDOMAIN>] 
type = http ### error when changed to tcp
remote_port = 55999 ### only for type=tcp
local_ip = gateway-service
local_port = 8081
#remote_port = 55998 ### for type = tcp
use_encryption = true
use_compression = true
subdomain = <SUBDOMAIN GENERATED FROM SECRET AND RAW SUBDOMAIN>

frps.ini:

[common]
bind_addr = 0.0.0.0
bind_port = 5223
vhost_http_port = 8080

dashboard_addr = 0.0.0.0
dashboard_port = 7400
dashboard_user =
dashboard_pwd =
enable_prometheus = false

#console or real logFile path like ./frps.log
log_file = /var/log/supervisor/frps.log
log_level = debug
log_max_days = 3
disable_log_color = false
detailed_errors_to_client = true

#auth token

token = TOKEN
tls_only = true
tls_cert_file = /certs/server.pem
tls_key_file = /certs/server.pem
tls_trusted_ca_file = /certs/ca.crt

#pool_count in each proxy will change to max_pool_count if they exceed the maximum value
max_pool_count = 10000
#max ports can be used for each client, default value is 0 means no limit
max_ports_per_client = 0

#if subdomain_host is not empty, you can set subdomain when type is http or https in frpc's configure file
#when subdomain is test, the host used by routing is test.frps.com
subdomain_host = <RAW SUB DOMAIN>

#if tcp stream multiplexing is used, default is true
tcp_mux = true

#custom 404 page for HTTP requests
custom_404_page = /ops/404.htmlbash-5

Nginx config:

location ~* ^/v1/api/<LOCAL SERVER ID>/(.*) {
    proxy_pass http://gtunnel-server.$kn.svc.cluster.local:8080/$1$is_args$args;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "Upgrade";
    proxy_set_header Host <SUBDOMAIN GENERATED FROM SECRET AND RAW SUBDOMAIN.RAW SUB DOMAIN>;
}

Logs

[http.go:92] do http proxy request error: no such domain: <SUBDOMAIN GENERATED FROM SECRET AND RAW SUBDOMAIN.RAW SUB DOMAIN>

Steps to reproduce

...

Affected area

  • Docs
  • Installation
  • Performance and Scalability
  • Security
  • User Experience
  • Test and Release
  • Developer Infrastructure
  • Client Plugin
  • Server Plugin
  • Extensions
  • Others
Originally created by @zaalgol on GitHub (Feb 9, 2023). Original GitHub issue: https://github.com/fatedier/frp/issues/3298 ### Bug Description In our application, we create a frp tunnel from the local-server to the public server, so the public server can send data to the local-server. In the config we defined the type to be http, and defined a domain. The public server uses another proxy - nginx reverse proxy to route client requests to the local server address.. It works great. We have a customer that wants to add his private proxy, that all data from public server to local-server should pass through it. Because frp supports http_proxy only with type=tcp, as a beginning I changed config to type=tcp, but because the type=tpc doesn’t support domains, I get a error in the public server frp: [http.go:92] do http proxy request error: no such domain: SUBDOMAIN GENERATED FROM SECRET AND RAW SUBDOMAIN.RAW SUB DOMAIN ### frpc Version 0.35.1 ### frps Version 0.35.1 ### System Architecture linux VM on a cloud ### Configurations ### **frpc.ini:** ``` [common] server_addr = <PUBLIC SERVER ADDRESS> server_port = 5223 #LOGGING log_file = /var/log/supervisor/frpc.log log_level = debug log_max_days = 3 disable_log_color = false #AUTHEXTRAS authenticate_heartbeats = false authenticate_new_work_conns = false token = TOKEN #set admin address for control frpc's action by http api such as reload admin_addr = 0.0.0.0 admin_port = 7400 #connections will be established in advance, default value is zero pool_count = 2 #if tcp stream multiplexing is used, default is true, it must be same with frps tcp_mux = true #decide if exit program when first login failed, otherwise continuous relogin to frps login_fail_exit = false #communication protocol used to connect to server #now it supports tcp, kcp and websocket, default is tcp protocol = tcp #TLS tls_enable = true tls_cert_file = /certs/client.crt tls_key_file = /certs/client.key [<SUBDOMAIN GENERATED FROM SECRET AND RAW SUBDOMAIN>] type = http ### error when changed to tcp remote_port = 55999 ### only for type=tcp local_ip = gateway-service local_port = 8081 #remote_port = 55998 ### for type = tcp use_encryption = true use_compression = true subdomain = <SUBDOMAIN GENERATED FROM SECRET AND RAW SUBDOMAIN> ``` ### **frps.ini:** ``` [common] bind_addr = 0.0.0.0 bind_port = 5223 vhost_http_port = 8080 dashboard_addr = 0.0.0.0 dashboard_port = 7400 dashboard_user = dashboard_pwd = enable_prometheus = false #console or real logFile path like ./frps.log log_file = /var/log/supervisor/frps.log log_level = debug log_max_days = 3 disable_log_color = false detailed_errors_to_client = true #auth token token = TOKEN tls_only = true tls_cert_file = /certs/server.pem tls_key_file = /certs/server.pem tls_trusted_ca_file = /certs/ca.crt #pool_count in each proxy will change to max_pool_count if they exceed the maximum value max_pool_count = 10000 #max ports can be used for each client, default value is 0 means no limit max_ports_per_client = 0 #if subdomain_host is not empty, you can set subdomain when type is http or https in frpc's configure file #when subdomain is test, the host used by routing is test.frps.com subdomain_host = <RAW SUB DOMAIN> #if tcp stream multiplexing is used, default is true tcp_mux = true #custom 404 page for HTTP requests custom_404_page = /ops/404.htmlbash-5 ``` ### **Nginx config:** ``` location ~* ^/v1/api/<LOCAL SERVER ID>/(.*) { proxy_pass http://gtunnel-server.$kn.svc.cluster.local:8080/$1$is_args$args; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "Upgrade"; proxy_set_header Host <SUBDOMAIN GENERATED FROM SECRET AND RAW SUBDOMAIN.RAW SUB DOMAIN>; } ``` ### Logs `[http.go:92] do http proxy request error: no such domain: <SUBDOMAIN GENERATED FROM SECRET AND RAW SUBDOMAIN.RAW SUB DOMAIN>` ### Steps to reproduce 1. 2. 3. ... ### Affected area - [ ] Docs - [ ] Installation - [ ] Performance and Scalability - [X] Security - [X] User Experience - [ ] Test and Release - [ ] Developer Infrastructure - [ ] Client Plugin - [ ] Server Plugin - [ ] Extensions - [X] Others
Author
Owner

@Becods commented on GitHub (Feb 10, 2023):

location ~* ^/v1/api/LOCAL SERVER ID/(.*) {
  proxy_pass http://127.0.0.1:8080/$1$is_args$args;
  proxy_http_version 1.1;
  proxy_set_header Upgrade $http_upgrade;
  proxy_set_header Connection "Upgrade";
  proxy_set_header Host SUBDOMAIN GENERATED FROM SECRET AND RAW SUBDOMAIN.RAW SUB DOMAIN;
}

OR

location ~* ^/v1/api/LOCAL SERVER ID/(.*) {
  proxy_pass http://127.0.0.1:8081/$1$is_args$args;
  proxy_http_version 1.1;
  proxy_set_header Upgrade $http_upgrade;
  proxy_set_header Connection "Upgrade";
  proxy_set_header Host SUBDOMAIN GENERATED FROM SECRET AND RAW SUBDOMAIN.RAW SUB DOMAIN;

}
[SUBDOMAIN GENERATED FROM SECRET AND RAW SUBDOMAIN]
type = tcp
remote_port = 8081
local_ip = gateway-service
local_port = 8081
use_encryption = true
use_compression = true
subdomain = SUBDOMAIN GENERATED FROM SECRET AND RAW SUBDOMAIN
<!-- gh-comment-id:1425544361 --> @Becods commented on GitHub (Feb 10, 2023): ``` location ~* ^/v1/api/LOCAL SERVER ID/(.*) { proxy_pass http://127.0.0.1:8080/$1$is_args$args; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "Upgrade"; proxy_set_header Host SUBDOMAIN GENERATED FROM SECRET AND RAW SUBDOMAIN.RAW SUB DOMAIN; } ``` OR ``` location ~* ^/v1/api/LOCAL SERVER ID/(.*) { proxy_pass http://127.0.0.1:8081/$1$is_args$args; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "Upgrade"; proxy_set_header Host SUBDOMAIN GENERATED FROM SECRET AND RAW SUBDOMAIN.RAW SUB DOMAIN; } ``` ``` [SUBDOMAIN GENERATED FROM SECRET AND RAW SUBDOMAIN] type = tcp remote_port = 8081 local_ip = gateway-service local_port = 8081 use_encryption = true use_compression = true subdomain = SUBDOMAIN GENERATED FROM SECRET AND RAW SUBDOMAIN ```
Author
Owner

@zaalgol commented on GitHub (Feb 10, 2023):

Hi @Becods,
Thanks for your response!

I tried the solution you suggested, and changed the 2 configurations.

The local server frp side runs OK. Logs:

2023/02/10 11:42:21 [I] [tcp.go:63] [4b6862736fea4562] [<SUBDOMAIN GENERATED FROM SECRET AND RAW SUBDOMAIN>] tcp proxy listen port [8081]
2023/02/10 11:42:21 [I] [control.go:446] [4b6862736fea4562] new proxy [<SUBDOMAIN GENERATED FROM SECRET AND RAW SUBDOMAIN>] success

Also the public server frp runs well. Logs:

2023/02/10 12:34:43 [I] [tcp.go:63] [5174c5f44d91f23c] [<SUBDOMAIN GENERATED FROM SECRET AND RAW SUBDOMAIN>] tcp proxy listen port [8081]
2023/02/10 12:34:43 [I] [control.go:446] [5174c5f44d91f23c] new proxy [<SUBDOMAIN GENERATED FROM SECRET AND RAW SUBDOMAIN>] success
2023/02/10 12:34:43 [D] [control.go:219] [5174c5f44d91f23c] new work connection registered

The problem is in the nginx reverse proxy.

If I set the port in proxy_pass to 8080, I get the following error, when sending a request from public server to local server:

127.0.0.1 - - [10/Feb/2023:11:53:29 +0000] "GET <REQUEST> HTTP/1.1" 403 162 "-" "python-requests/2.25.1"
<IP> - - [10/Feb/2023:11:53:29 +0000] "GET /v1/api/<LOCAL SERVER ID>/<REQUEST> HTTP/1.1" 403 162 "-" "python-requests/2.25.1"

If I set the port in proxy_pass to 8081, I get the following error, when sending a request from public server to local server:
<IP> - - [10/Feb/2023:12:53:36 +0000] "GET /v1/api/<LOCAL SERVER ID><REQUEST> HTTP/1.1" 502 166 "-" "python-requests/2.25.1"

It is worth noting that in the old logic in which I used http, the log in nginx proxy was:
<IP> - - [13/Feb/2023:09:09:38 +0000] "GET /v1/api/<SUBDOMAIN GENERATED FROM SECRET AND RAW SUBDOMAIN>/<REQUEST>HTTP/1.1" 200 51 "-" " python-requests/2.25.1"
As you can see, it is directed at port 51, unlike the one I changed to tcp, which is directed at other ports.

Maybe need to make a change also in frps.ini?
Or maybe need to do a different change in frpc.ini?

<!-- gh-comment-id:1425777174 --> @zaalgol commented on GitHub (Feb 10, 2023): Hi @Becods, Thanks for your response! I tried the solution you suggested, and changed the 2 configurations. The local server frp side runs OK. Logs: ``` 2023/02/10 11:42:21 [I] [tcp.go:63] [4b6862736fea4562] [<SUBDOMAIN GENERATED FROM SECRET AND RAW SUBDOMAIN>] tcp proxy listen port [8081] 2023/02/10 11:42:21 [I] [control.go:446] [4b6862736fea4562] new proxy [<SUBDOMAIN GENERATED FROM SECRET AND RAW SUBDOMAIN>] success ``` Also the public server frp runs well. Logs: ``` 2023/02/10 12:34:43 [I] [tcp.go:63] [5174c5f44d91f23c] [<SUBDOMAIN GENERATED FROM SECRET AND RAW SUBDOMAIN>] tcp proxy listen port [8081] 2023/02/10 12:34:43 [I] [control.go:446] [5174c5f44d91f23c] new proxy [<SUBDOMAIN GENERATED FROM SECRET AND RAW SUBDOMAIN>] success 2023/02/10 12:34:43 [D] [control.go:219] [5174c5f44d91f23c] new work connection registered ``` The problem is in the nginx reverse proxy. If I set the port in proxy_pass to 8080, I get the following error, when sending a request from public server to local server: ``` 127.0.0.1 - - [10/Feb/2023:11:53:29 +0000] "GET <REQUEST> HTTP/1.1" 403 162 "-" "python-requests/2.25.1" <IP> - - [10/Feb/2023:11:53:29 +0000] "GET /v1/api/<LOCAL SERVER ID>/<REQUEST> HTTP/1.1" 403 162 "-" "python-requests/2.25.1" ``` If I set the port in proxy_pass to 8081, I get the following error, when sending a request from public server to local server: `<IP> - - [10/Feb/2023:12:53:36 +0000] "GET /v1/api/<LOCAL SERVER ID><REQUEST> HTTP/1.1" 502 166 "-" "python-requests/2.25.1"` It is worth noting that in the old logic in which I used http, the log in nginx proxy was: `<IP> - - [13/Feb/2023:09:09:38 +0000] "GET /v1/api/<SUBDOMAIN GENERATED FROM SECRET AND RAW SUBDOMAIN>/<REQUEST>HTTP/1.1" 200 51 "-" " python-requests/2.25.1"` As you can see, it is directed at port 51, unlike the one I changed to tcp, which is directed at other ports. Maybe need to make a change also in frps.ini? Or maybe need to do a different change in frpc.ini?
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/frp#2641
No description provided.