[GH-ISSUE #49] Identify dead connections #26

Closed
opened 2026-05-05 10:57:26 -06:00 by gitea-mirror · 12 comments
Owner

Originally created by @goriccardo on GitHub (Nov 20, 2017).
Original GitHub issue: https://github.com/mmatczuk/go-http-tunnel/issues/49

I'm having issues with long lived connections:

  • the server sometimes receives connection requests from clients that are reportedly already connected, and reject them.
  • when the server restarts, some clients do not try to connect again.

I debugged the issue and found out that once the client is connected to the server this connection is kept open waiting for a user to connect to the server. If the connection dies without notifying the client or server, they would keep listening to dead connections.

If the server believes that a dead connection belonging to an id is open it would reject new (valid) connections from that id.

A client may keep listening to a dead connection forever, effectively preventing the client to retry and open a fresh connection.

My workaround was to turn on TCP keepalive for the client connection, and to let the server ping (http ping) the existing client connection when a connection request with an already connected id is received.

Originally created by @goriccardo on GitHub (Nov 20, 2017). Original GitHub issue: https://github.com/mmatczuk/go-http-tunnel/issues/49 I'm having issues with long lived connections: - the server sometimes receives connection requests from clients that are reportedly already connected, and reject them. - when the server restarts, some clients do not try to connect again. I debugged the issue and found out that once the client is connected to the server this connection is kept open waiting for a user to connect to the server. If the connection dies without notifying the client or server, they would keep listening to dead connections. If the server believes that a dead connection belonging to an id is open it would reject new (valid) connections from that id. A client may keep listening to a dead connection forever, effectively preventing the client to retry and open a fresh connection. My workaround was to turn on TCP keepalive for the client connection, and to let the server ping (http ping) the existing client connection when a connection request with an already connected id is received.
Author
Owner

@mmatczuk commented on GitHub (Nov 20, 2017):

For ping I assume you mean https://godoc.org/golang.org/x/net/http2#ClientConn.Ping.

This seems perfectly valid, could you please send a PR.

<!-- gh-comment-id:345715247 --> @mmatczuk commented on GitHub (Nov 20, 2017): For ping I assume you mean https://godoc.org/golang.org/x/net/http2#ClientConn.Ping. This seems perfectly valid, could you please send a PR.
Author
Owner

@goriccardo commented on GitHub (Nov 20, 2017):

Yes that ping :)

I'll prepare a PR tonight, I have to cleanup my workaround a bit.

<!-- gh-comment-id:345716431 --> @goriccardo commented on GitHub (Nov 20, 2017): Yes that ping :) I'll prepare a PR tonight, I have to cleanup my workaround a bit.
Author
Owner

@mmatczuk commented on GitHub (Nov 20, 2017):

Thanks.

<!-- gh-comment-id:345716877 --> @mmatczuk commented on GitHub (Nov 20, 2017): Thanks.
Author
Owner

@mmatczuk commented on GitHub (Nov 21, 2017):

Do you have an environment where you could easily reproduce the issue?

<!-- gh-comment-id:346124313 --> @mmatczuk commented on GitHub (Nov 21, 2017): Do you have an environment where you could easily reproduce the issue?
Author
Owner

@goriccardo commented on GitHub (Nov 21, 2017):

The only way I found to test the issue was using iptables to drop packets to/from the server and kill either the client or the server.

Unfortunately I did not find an easy way to reproduce the bug without using network filtering techniques.

<!-- gh-comment-id:346160257 --> @goriccardo commented on GitHub (Nov 21, 2017): The only way I found to test the issue was using iptables to drop packets to/from the server and kill either the client or the server. Unfortunately I did not find an easy way to reproduce the bug without using network filtering techniques.
Author
Owner

@mmatczuk commented on GitHub (Nov 21, 2017):

Ok that's fine. I have slightly different implementation idea. Will get back to you shortly.

<!-- gh-comment-id:346160855 --> @mmatczuk commented on GitHub (Nov 21, 2017): Ok that's fine. I have slightly different implementation idea. Will get back to you shortly.
Author
Owner

@mmatczuk commented on GitHub (Nov 22, 2017):

@goriccardo could you check if https://github.com/mmatczuk/go-http-tunnel/pull/52 works for you?

<!-- gh-comment-id:346278786 --> @mmatczuk commented on GitHub (Nov 22, 2017): @goriccardo could you check if https://github.com/mmatczuk/go-http-tunnel/pull/52 works for you?
Author
Owner

@goriccardo commented on GitHub (Nov 22, 2017):

I will test it later today, thanks.

On a side note I was not confident in using the tcpkeepalive package
because apparently there are some known issues reported on its readme. I
did not look at them closely though.

<!-- gh-comment-id:346283494 --> @goriccardo commented on GitHub (Nov 22, 2017): I will test it later today, thanks. On a side note I was not confident in using the tcpkeepalive package because apparently there are some known issues reported on its readme. I did not look at them closely though.
Author
Owner

@mmatczuk commented on GitHub (Nov 22, 2017):

The conclusion is to use SetKeepAlive that shall be safe see https://github.com/felixge/tcpkeepalive/issues/3.

<!-- gh-comment-id:346290356 --> @mmatczuk commented on GitHub (Nov 22, 2017): The conclusion is to use `SetKeepAlive` that shall be safe see https://github.com/felixge/tcpkeepalive/issues/3.
Author
Owner

@goriccardo commented on GitHub (Nov 22, 2017):

#52 seems to work fine, thanks.

<!-- gh-comment-id:346322958 --> @goriccardo commented on GitHub (Nov 22, 2017): #52 seems to work fine, thanks.
Author
Owner

@mmatczuk commented on GitHub (Nov 22, 2017):

Thanks!

<!-- gh-comment-id:346323181 --> @mmatczuk commented on GitHub (Nov 22, 2017): Thanks!
Author
Owner

@mmatczuk commented on GitHub (Nov 22, 2017):

Fixed by #52.

<!-- gh-comment-id:346323548 --> @mmatczuk commented on GitHub (Nov 22, 2017): Fixed by #52.
Sign in to join this conversation.
No labels
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/go-http-tunnel#26
No description provided.