Debugging Git network connection issues using GIT_TRACE

I have been travelling a lot this month and working remotely from cafes and public places during my stay in Europe. And most often I was connected to the internet using the public hot spots or free Wifi provided by the cafes or restaurants.

The quality of the connections were really good I must admit, but one common trouble I had with these connections was when using git.

Most of these connections had some restrictions with using git.

Problem

In one of the instances, I was able to push to github but was not able to fetch/clone/pull from the repo. As I issued the command

git fetch origin

it hangs there forever and didn't respond at all.

Debugging

So let's open our troubleshooting guide and go to Rule 1:

  • Check your connectivity :P
ping google.com
PING google.com (216.58.205.238): 56 data bytes
64 bytes from 216.58.205.238: icmp_seq=0 ttl=46 time=124.577 ms

Okay, so I am connected.!!

  • Check if you have access to the repo

Since I am able to push to repo, I am pretty sure that I have access to the repo, but still :check:

  • Check if ssh was disabled.
$ ssh [email protected]

PTY allocation request failed on channel 0
Hi manusajith! You've successfully authenticated, but GitHub does not provide shell access.
Connection to github.com closed.

Okay.!! so ssh is not blocked on the router for sure now.

  • Replace git:// protocol with http/https

I was pretty sure that this was not the issue, but still lets give it a try.!!

  • Try changing your DNS to 8.8.8.8

I already had that :P

Since all of the above failed, I realised that it needs a bit more serious debugging.

My immediate thoughts was to troubleshoot the network connections and to log the connection attempts to GitHub in a more verbose manner so that I can debug it in depth.

GIT_TRACE

Welcome to GIT_TRACE. This configuration option gives us a more verbose trace to the git network connections and all the internal commands it goes through.

This environment variable can accept the following values:

  • 1, 2, or true

If this variable is set to 1, 2 or true ( the comparison is case insensitive), git will print trace: messages on stderr telling about alias expansion, built-in command execution and external command execution.

  • greater than 1 and less than 10

If this variable is set to an integer value greater than 1 and lower than 10 (strictly) then git will interpret this value as an open file descriptor and will try to write the trace messages into this file descriptor.

  • absolute path

Alternatively, if this variable is set to an absolute path (starting with a / character), git will interpret this as a file path and will try to write the trace messages into it.

$ GIT_TRACE=1 git fetch origin

13:44:47.299097 git.c:350               trace: built-in: git 'fetch' 'origin'
13:44:47.403611 run-command.c:336       trace: run_command: 'ssh' '[email protected]' 'git-upload-pack '\''woumedia/naturalblender-api.git'\'''

With the more in-depth trace, I was able to figure out that it hanged forever at when trying to upload the pack.

Since I know where exactly the problem was I am now able to find an fix for the same, further research and debugging gave me clues as to why it was stuck there forever: It was a type of service issue with TCP packets handling on my router, which unfortunately couldn't do much about.