Git Course 0%

Monorepos, LFS, submodules

Advanced ≈ 9 min

Before this lesson

What you will learn

  • What a monorepo changes about your day
  • What Git LFS is and how to tell a project uses it
  • How submodules behave, and the one command that fixes the usual surprise

After this lesson you can

  • I am not surprised by an empty folder, a text file where an image should be, or a repository with everything in it

Why this matters

These three arrive without warning, usually as something that looks broken: a folder that is empty after a clone, an image that opens as a page of text, a repository with fourteen products in it. None of them is broken, and each has one thing you need to know.

Monorepos

A monorepo is one repository holding many projects: several applications, their shared libraries, the infrastructure and the documentation, all versioned together.

Monorepo Many repositories
A change spanning two projects One merge request Two, coordinated
Finding anything One clone, one search Know which repository first
Clone size and speed Large, sometimes very Small
CI Must work out what changed and run only that Straightforward
Access control Coarse: access is to the whole thing Per repository

What changes for you, in practice:

  • Your path matters more than your repository. Issues, CODEOWNERS and CI rules are all scoped by directory, so "the docs team owns /docs" replaces "the docs team owns the docs repository".
  • The pipeline may not run everything. A change under docs/ may trigger only the documentation jobs, which is why a green pipeline can look suspiciously fast.
  • git log needs a path. git log --oneline -- docs/ is the only way to see your area's history; without the path you get everyone's.
  • Cloning can be slow. git clone --filter=blob:none fetches file contents on demand and makes a large clone much faster, and everything works normally afterwards.

Git LFS

Git stores every version of every file forever, which is fine for text and painful for a 40 MB video. Git LFS (Large File Storage) keeps large files on a separate server and puts a small pointer file in the repository instead.

You can tell a project uses it from .gitattributes:

Text
*.psd filter=lfs diff=lfs merge=lfs -text
*.mp4 filter=lfs diff=lfs merge=lfs -text

The symptom to recognize: you open an image or a video and get a few lines of text like this:

Text
version https://git-lfs.github.com/spec/v1
oid sha256:4d7a2140c1f...
size 41306624

That is the pointer, and it means LFS is not installed or the files were not fetched. The fix is to install Git LFS, run git lfs install once per machine, and then git lfs pull in the repository.

Two things worth knowing beyond that. LFS storage and bandwidth are quota'd and billed on both platforms, which is why teams are careful about what goes into it. And a file must be tracked by LFS before it is committed: adding a rule later does not move existing files out of history, which is a separate and unpleasant operation.

Submodules

A submodule is another Git repository nested inside this one, pinned to a specific commit. .gitmodules records where it comes from:

Text
[submodule "vendor/trailguide-data"]
	path = vendor/trailguide-data
	url = ../sub-child

The surprise: a plain clone leaves the folder empty.

Terminal
$ git clone <url> trailguide
$ ls vendor/trailguide-data
-d3b92ec8073de8841dbb87d25de61309478b897b vendor/trailguide-data

Nothing. And git submodule status shows a leading -, meaning "not initialised":

The fix, and the one command to remember from this lesson:

Terminal
$ git submodule update --init --recursive
Submodule 'vendor/trailguide-data' registered for path 'vendor/trailguide-data'
Cloning into '…/vendor/trailguide-data'...
Submodule path 'vendor/trailguide-data': checked out 'd3b92ec…'

Now the folder has content, and the status line loses its -. Cloning with git clone --recurse-submodules does both steps at once.

What a submodule change looks like

The parent repository does not record the submodule's files, only which commit it points at. Commit something inside the submodule and the parent shows:

Terminal
$ git status --short
 M vendor/trailguide-data
Terminal
$ git diff --submodule=short
-Subproject commit d3b92ec8073de8841dbb87d25de61309478b897b
+Subproject commit fe8516bd408a62cad581c86d33843db1fba54e5b

One line changed: the pinned commit. That is the whole mental model. To update a submodule, you commit inside it and push there, then commit the new pointer in the parent. Two repositories, two pushes; forgetting the second is the most common submodule mistake, and it leaves colleagues pointing at a commit that only exists on your machine.

How to do it

The commands worth knowing, by symptom:

Terminal
$ git submodule status                      # is it initialised?
$ git submodule update --init --recursive   # fill an empty submodule folder
$ git clone --recurse-submodules <url>      # do it at clone time
$ git lfs install && git lfs pull           # a "file" that is three lines of text
$ git clone --filter=blob:none <url>        # a very large repository
$ git log --oneline -- docs/                # a monorepo: your area only

Common mistakes

  • Assuming an empty submodule folder means a broken clone. It means --init was not run.
  • Committing inside a submodule and pushing only the parent, leaving colleagues pointing at a commit that does not exist for them.
  • Committing a large binary directly in a project that uses LFS for that type. Check .gitattributes first.
  • Adding an LFS rule after the file is already in history, which does not move it.
  • Running git log without a path in a monorepo and drowning.
  • Treating a monorepo's fast pipeline as suspicious. Path-based rules are normal.

Try it yourself

Goal: meet a submodule in controlled conditions.

  1. Create two small repositories locally, parent and child.
  2. In parent, run git submodule add ../child vendor/child, then commit.
  3. Clone parent to a new folder and run ls vendor/child. It is empty.
  4. Run git submodule status and note the leading -, then git submodule update --init --recursive.
  5. Commit something inside the submodule, then run git status and git diff --submodule=short in the parent and read what changed.

Expected result: an empty folder that fills with one command, and a parent whose diff is a single "Subproject commit" line.

Show solution

Step 5 is the mental model. The parent repository never contains the submodule's files, only a pointer to one of its commits. Once that is clear, every confusing submodule situation becomes readable: the question is always "which commit is the parent pointing at, and does it exist where everyone can reach it?".

Check yourself

1. You cloned a repository and one folder is empty. What is the likely cause?
2. You open a video from the repository and get three lines of text mentioning oid sha256. What happened?
3. What does the parent repository record about a submodule?

Key terms

Repository (repo) Clone Commit Dependency

Related