Git Submodule By Example


If you have been working with version control system for a while, you would know or speculated about the possibility of including references from other projects into your project. From a programmer’s point of view, this feature is useful especially when you need to include code libraries owned by someone else or common header/footer files to be used across different repositories.

Like Subversion externals, git submodule allows one to work with multiple repositories within one main repository. Unfortunately, git submodule resources were scarce on the web. It could also be confusing for people coming from subversion background because it operates differently from it. Despite sharing the same concept, the general feedback was that many people find it hard to digest at a first glance.

In this tutorial, we will be going through an example of how to use git submodule in a typical software development process. We will also be setting up bare repositories from the start, so feel free to skip the first 3 steps if you already know the basics. A bit of basic Git and command line knowledge is assumed.

Say we have 2 developers working on an app that uses a common library (submodule) and this is the workflow:

  • Scenario 1: dev1 is the lead developer and sets up bare origin repository structure under the /mnt/demo directory.
  • Scenario 2: submodule-origin needs something before the team continue. Dev1 creates a master branch and commits a file to the empty submodule-origin.
  • Scenario 3: app-origin cannot be empty as well. Again, dev1 creates a master branch on app-origin and adds a file to it.
  • Scenario 4: The team now has the essential environment to start playing with git submodule. dev1 adds the submodule to its local repo, commits the file and pushes it to submodule-origin.
  • Scenario 5: dev1 is pretty happy with everything and decides to tag his changes.
  • Scenario 6: dev2 comes along and wants to make some code changes to both the app and the submodule. He does a clone, modify some files and commits.
  • Scenario 7: Dev2 is happy with his changes and decides to tag it with v1.0.1
  • Scenario 8: Dev1 now wants to get dev2 changes and does a pull.
  • Scenario 9: Dev1 is not very happy with dev2 changes and decides to revert everything back to v1.0.0

Hopefully, the snapshots of the command line and comments are self-explainatory…

Scenario 1: dev1 is the lead developer and sets up bare origin repository structure under the /mnt/demo directory.


# init user globals first
dev1@localhost:~$ git config --global user.name "dev1"
dev1@localhost:~$ git config --global user.email dev1.localhost

# initialise bare app and submodule repo
dev1@localhost:~$ cd /mnt
dev1@localhost:/mnt$ sudo mkdir {demo,demo/app-origin,demo/submodule-origin}
dev1@localhost:/mnt$ sudo chmod 777 -R demo
dev1@localhost:~/mnt$ cd demo
dev1@localhost:/mnt/demo$ git init --bare app-origin/
dev1@localhost:/mnt/demo$ git init --bare submodule-origin/

Scenario 2: submodule-origin needs something before the team continue. Dev1 creates a master branch and commits a file to the empty submodule-origin.


dev1@localhost:~$ mkdir submodule
dev1@localhost:~$ cd submodule
dev1@localhost:~/submodule$ git init
Initialized empty Git repository in /home/dev1/submodule/.git/
dev1@localhost:~/submodule$ echo submodule > submodule
dev1@localhost:~/submodule$ git add submodule
dev1@localhost:~/submodule$ git commit -m"init"
[master (root-commit) c61fab7] init
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 submodule
dev1@localhost:~/submodule$ git remote add origin /mnt/demo/submodule-origin/
dev1@localhost:~/submodule$ git push origin master
Counting objects: 3, done.
Writing objects: 100% (3/3), 207 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
To /mnt/demo/submodule-origin/
 * [new branch]      master -> master

Scenario 3: app-origin cannot be empty as well. Again, dev1 creates a master branch on app-origin and adds a file to it.


# initialse the dev1 directory
dev1@localhost:~$ mkdir app
dev1@localhost:~$ cd app
dev1@localhost:~/app$ git init
Initialized empty Git repository in /home/dev1/app/.git/
dev1@localhost:~/app$ echo dev1 > dev1
dev1@localhost:~/app$ git add app
dev1@localhost:~/app$ git commit -m "init"
[master (root-commit) 28eda11] init
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 dev1
dev1@localhost:~/app$ git remote add origin /mnt/demo/app-origin/ 
dev1@localhost:~/app$ git push origin master
Counting objects: 3, done.
Writing objects: 100% (3/3), 199 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
To /mnt/demo/app-origin/
 * [new branch]      master -> master

Scenario 4: The team now has the essential environment to start playing with git submodule. dev1 adds the submodule to its local repo, commits the file and pushes it to submodule-origin.


# add a submodule
dev1@localhost:~/app$ git submodule add /mnt/demo/submodule-origin/ submodule

if you are doing it correctly, you should see a .gitmodules file being created.


dev1@localhost:~/dev1$ cat .gitmodules
[submodule "submodule"]
	path = submodule
	url = /mnt/demo/submodule-origin

dev1@localhost:~/app$ git status
# On branch master
# Changes to be committed:
#   (use "git reset HEAD ..." to unstage)
#
#	new file:   .gitmodules
#	new file:   submodule

Let’s add the new files and push


dev1@localhost:~/app$ git add *
dev1@localhost:~/app$ git commit -m "add submodule"
[master c78abf9] add submodule
 2 files changed, 4 insertions(+), 0 deletions(-)
 create mode 100644 .gitmodules
 create mode 160000 submodule
dev1@localhost:~/app$ git push origin master
Counting objects: 4, done.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 350 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
To /mnt/demo/app-origin/
   a94d038..c78abf9  master -> master

Scenario 5: dev1 is pretty happy with everything and decides to tag his changes.


dev1@localhost:~/app$ git tag v1.0.0
dev1@localhost:~/app$ git push --tags
Total 0 (delta 0), reused 0 (delta 0)
To /mnt/demo/app-origin/
 * [new tag]         v1.0.0 -> v1.0.0

Scenario 6: dev2 comes along and wants to make some code changes to both the app and the submodule. He does a clone, modify some files and commits.


# init user
dev2@localhost:~$ git config --global user.name dev2
dev2@localhost:~$ git config --global user.email dev2@localhost.com

# clone the app-origin first.
dev2@localhost:~$ git clone /mnt/demo/app-origin/ app
Initialized empty Git repository in /home/dev2/app/.git/
dev2@localhost:~/app$ ls
dev1  submodule

dev2 modifies the dev1 file and add it to be committed later.


dev2@localhost:~/app$ echo "dev1 dev2" > dev1
dev2@localhost:~/app$ git add dev1

let’s see what’s in the submodule


dev2@localhost:~/app$ ls submodule

Nothing?? Remember that git submodules are to be treated differently? Cloning the app doesn’t clone the submodule. Dev2 will have to do that manually.


dev2@localhost:~/app$ git submodule init
Submodule 'submodule' (/mnt/demo/submodule-origin) registered for path 'submodule'
dev2@localhost:~/app$ git submodule update
Initialized empty Git repository in /home/dev2/app/submodule/.git/
Submodule path 'submodule': checked out 'c61fab7265554f3e99632376102831e6712622b5'

now, we should see something in the submodule dir.


dev2@localhost:~/app$ ls submodule/
submodule

the “git submodule update” also created a dummy branch for you


dev2@localhost:~/app$ cd submodule/
dev2@localhost:~/app/submodule$ git branch
* (no branch)
  master

Now say dev2 wants to fix a bug in the submodule and he wants everyone using the submodule to get it. To push changes in the submodule, he needs to re-checkout the master branch and push. Note that he needs to push again in the submodule dir because the submodule is now a repo within another repo. Despite the hierarchy, both repo need to be managed as if they are separate.

dev2 now checkout the master branch, modify the submodule file and commits.


dev2@localhost:~/app/submodule$ git checkout master
dev2@localhost:~/app/submodule$ echo "fixed bug" >> submodule 
dev2@localhost:~/app/submodule$ git add submodule
dev2@localhost:~/app/submodule$ git commit -m "fixed bug"
[master 9ddfa96] fixed bug
 1 files changed, 1 insertions(+), 0 deletions(-)
dev2@localhost:~/app/submodule$ git push origin master
Counting objects: 5, done.
Writing objects: 100% (3/3), 260 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
To /mnt/demo/submodule-origin
   c61fab7..9ddfa96  master -> master

back to the root, we should see the files/dir modified.


dev2@localhost:~/app/submodule$ cd ..
dev2@localhost:~/app$ git status
# On branch master
# Changed but not updated:
#   (use "git add ..." to update what will be committed)
#   (use "git checkout -- ..." to discard changes in working directory)
#
#	modified:   dev1
#	modified:   submodule
#

dev1 need to do another commit and push in the top level to include the submodule changes.


dev2@localhost:~/app$ git add dev1 submodule
dev2@localhost:~/app$ git commit -m "dev2 made both app and submodule changes"
[master a542a08] dev2 made both app and submodule changes
 2 files changed, 2 insertions(+), 2 deletions(-)
dev2@localhost:~/app$ git push origin master
Counting objects: 5, done.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 327 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
To /mnt/demo/app-origin/
   c78abf9..a542a08  master -> master

Scenario 7: ev2 is happy with his changes and decides to tag it with v1.0.1dev2 comes along and wants to make some code changes to both the app and the submodule, so he does a clone, modify the files and commit.


dev2@localhost:~/app$ git fetch --tags
From /mnt/demo/app-origin
 * [new tag]         v1.0.0     -> v1.0.0
dev2@localhost:~/app$ git tag v1.0.1
dev2@localhost:~/app$ git push --tags
Total 0 (delta 0), reused 0 (delta 0)
To /mnt/demo/app-origin/
 * [new tag]         v1.0.1 -> v1.0.1

Scenario 8: Dev1 now wants to get dev2 changes and does a pull.


dev1@localhost:~/app$ git pull origin master
remote: Counting objects: 5, done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
From /mnt/demo/app-origin
 * branch            master     -> FETCH_HEAD
Updating c78abf9..a542a08
Fast-forward
 dev1      |    2 +-
 submodule |    2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

Let’s check on the submodule file…


dev1@localhost:~/app$ cat submodule/submodule 
submodule

Opps, where was the change that dev2 made on the submodule/submodule file? Dev1 tries doing a “git submodule update” and got the changes.


dev1@localhost:~/app$ git submodule update
remote: Counting objects: 5, done.
remote: Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
From /mnt/demo/submodule-origin
   c61fab7..9ddfa96  master     -> origin/master
Submodule path 'submodule': checked out '9ddfa961f29c3866cdcbef697811a185af3fc407'

Scenario 9: Dev1 is not very happy with dev2 changes and decides to revert everything back to v1.0.0


dev1@localhost:~/app$ git checkout v1.0.0
M	submodule
Note: checking out 'v1.0.0'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b new_branch_name

HEAD is now at c78abf9... add submodule

dev1@localhost:~/app$ cat dev1
dev1
fixed bug

dev1 file is now reverted. How about the submodule?


dev1@localhost:~/app$ cat submodule/submodule 
submodule

Opps, submodule file has not been reverted. dev1 does another submodule update to grab the old changes for the submodule.


dev1@localhost:~/app$ git submodule update
Submodule path 'submodule': checked out 'c61fab7265554f3e99632376102831e6712622b5'
dev1@localhost:~/app$ cat submodule/submodule 
submodule

Hopefully, this example covers the basic use of git submodule. The trick is to treat the submodule as a separate repo within the top working repo. Remember to do “git submodule update” after you do a pull (if you wish to update the submodule).

Another thing to note is that if you create a branch from a branch containing a submodule, the submodule will remain in the same state irregardless of which module branch you are in. This is important to know if you have separate branches within the submodule itself.

Like it.? Share it:
Tags:

2 Responses to Git Submodule By Example

  1. yeo

    PHP Site Calendar E has installed but when the show and wince at the table just out of date (Please try again later). If the log in the display can be seen. why should a new lon in abstruse see. please, how do you solve this problem. Joomla 1.6.5

  2. Matt

    Thanks for this, I was struggling a bit with submodules, and this really clarified things for me.