Git Submodule Accelerated Cloning & Common Network Troubleshooting

When cloning complex open-source projects—such as C++ frameworks, media codecs, machine learning inference engines, or embedded SDKs—repositories frequently include Git Submodules to pull external dependencies.

Because every submodule functions as an independent Git repository, running git clone --recursive over high-latency networks frequently runs into timeouts, broken streams, or SSL handshake failures on any single submodule, failing the entire operation.

This guide explores how Git Submodules work, how to accelerate recursive cloning, and how to resolve the most frequent Git network errors.

---

1. Why Are Submodules Slow to Clone?

Repositories that rely on submodules store dependency references in a .gitmodules file in the root directory:

[submodule "third_party/googletest"]
	path = third_party/googletest
	url = https://github.com/google/googletest.git
[submodule "third_party/benchmark"]
	path = third_party/benchmark
	url = https://github.com/google/benchmark.git

When you execute git clone --recursive <url>, Git completes the parent clone and then contacts GitHub serially for each individual submodule. If any single transfer encounters latency drops or resets, the entire clone command halts with an error.

---

2. Three Techniques to Accelerate Submodules

Technique 1: Global insteadOf Rewriting for Automatic Cascaded Acceleration (Recommended)

The most robust technique is configuring Git native insteadOf directive. When active, both the parent repository and every nested submodule are automatically rewritten to use GH-Proxy:

# Enable global automatic proxy routing
git config --global url."https://gh-proxy.org/https://github.com/".insteadOf "https://github.com/"

Execute your clone command normally:

# Git automatically rewrites and routes all transfers through gh-proxy.org
git clone --recursive https://github.com/commaai/openpilot.git

To revert this configuration at any time:

git config --global --unset url."https://gh-proxy.org/https://github.com/".insteadOf

---

Technique 2: Multi-Threaded Parallel Submodule Pulls with Depth 1

If the parent repository is already present locally, you can speed up submodule initialization using shallow history and concurrent worker jobs:

# 1. Register submodule paths
git submodule init

# 2. Concurrently pull submodules (--jobs 4 spawns 4 worker threads; --depth 1 limits commit history)
git submodule update --init --recursive --jobs 4 --depth 1

---

Technique 3: Batch Rewriting .gitmodules In-Place

If you do not want to change your global Git settings, rewrite submodule URLs within the project itself using sed:

# Prefix all https://github.com entries in .gitmodules
sed -i "s|https://github.com/|https://gh-proxy.org/https://github.com/|g" .gitmodules

# Synchronize and fetch
git submodule sync
git submodule update --init --recursive

---

3. High-Frequency Git Network Errors & Troubleshooting

1. OpenSSL SSL_read: Connection was reset, errno 10054

• Root Cause: The TLS handshake or large payload stream was interrupted by intermediate network resets (TCP RST).
• Fixes:
  1. Route clone requests through GH-Proxy.
  2. If using local proxy software, verify Git local proxy parameters:

     git config --global http.proxy http://127.0.0.1:7890
     git config --global https.proxy http://127.0.0.1:7890
     

2. GnuTLS recv error (-110): The TLS connection was non-properly terminated.

• Root Cause: Common in Ubuntu / Debian environments where the system git package is linked against GnuTLS instead of OpenSSL, causing extreme sensitivity to packet jitter.
• Fixes:
  • Extend Git transfer timeout tolerances:

    git config --global http.lowSpeedLimit 0
    git config --global http.lowSpeedTime 999999
    

  • Or route via GH-Proxy to avoid international route volatility.

3. fatal: early EOF / RPC failed; curl 18 transfer closed with outstanding read data remaining

• Root Cause: When cloning repositories exceeding several gigabytes, packet delivery stall triggers connection teardown.
• Fixes:
  1. Increase the Git HTTP buffer limit (from default 1MB to 500MB):

     git config --global http.postBuffer 524288000
     

  2. Perform an initial shallow clone, then un-shallow as required:

     # Fast clone of depth 1
     git clone --depth 1 https://gh-proxy.org/https://github.com/torvalds/linux.git
     
     # Retrieve complete commit history later if necessary
     cd linux
     git fetch --unshallow
     

4. fatal: unable to access "https://github.com/...": Failed to connect to 127.0.0.1 port 1080: Connection refused

• Root Cause: A stale proxy port was configured in Git, but the corresponding local proxy software is no longer running.
• Fixes: Clear existing Git proxy configurations:

  git config --global --unset http.proxy
  git config --global --unset https.proxy
  

---

4. Summary

1. Daily Development: Enable url."https://gh-proxy.org/...".insteadOf to handle both root projects and submodules transparently.
2. Massive Codebases: Pair --depth 1 with --jobs 4 to minimize clone times and disk consumption.