Self Updating Git Repos

TL;DR In this post I write a small bash script that updates your Git repos in the background. It is completely hands-free, and pops up a desktop notification whenever it syncs up.

At my workplace I commit to 11 different Git repos. And that doesn’t include my open-source work! If you, like me, have several active projects on Github, it’s not possible to remember to keep them updated. Sure you could `git pull` when you’re working on them, but you’d never know if someone actually committed to a repo, when they did. And sometimes you’re working from home and you just forget to push your updates. Next day at work, it’s missing!

This bugged me enough that i had to come up with a solution that i could just set and forget. This is that solution.

Before we get into code, let’s look at our options. There are a few ways to get notified when there are updates :

  1. GitHub Notifications:  You can setup Github to send you notifications whenever there are updates to your repos. This will require you to set up notifications on all your projects on GitHub. Then you need to write a listener that listens to these notices and updates your repo.
    GitHub notifications are rich, they can notify you across a range of events on git or GitHub (such as users forking your repo or replying to your comment) and they have a place in a team development environment. But for our purposes they’re overkill. For one thing, I don’t want to rely on Github, or any 3rd party service, for something so simple. It requires setup and makes you rely on their APIs, when you don’t need to.
  2. Git hooks. Then there’s Git hooks. Git hooks are a native feature of git. They allow you to run scripts whenever Git detects a certain action. Server-side hooks run on network operations, which is what we need for this solution to work. However most of us don’t run our own git servers. And, this requires us to setup every Git origin repo with hooks. No way!
  3. cron and `git pull`. Yes, write a cron that loops through all your repos, calling git pull on them. This is all we need, plus a nice desktop notice that pops up whenever there’s an update.

In this post I’ll be going with option #3.

It’s really simple

There’s really only two lines of code you need to do this.

git pull;
git push;

Cron that and be gone! Just kidding.

You probably know what git pull origin master does. git pull is even better, it fetches all updates across branches using git fetch, and merges any updates that belong in your checked-out branch. There’s no need to specify the `branch` or `remote`, git will read that from the .git/config. This is enough if you only commit to `master`, but not if you want to keep ALL your branches in sync. This trick here is to loop through your remote-tracking branches, merging any updates that have been fetched.

The issue, then, becomes about intrusiveness. You actually have to git checkout your branch before merging. What we want is a way to merge remote updates transparently, i.e: without having to checkout any branch. It’s possible as I’ll demonstrate below.

The Script

The goals of this script are simple.

  1. It should pull updates in a non-instrusive fashion, i.e: it should not merge on branches I’m working on and it should not require checking out to a different branch.
  2. It should give me a small summary of what’s been synced. Syncing means fetching remote commits, and pushing local commits upstream.
  3. It should inform me on the desktop whenever a repo has been synced. Set it and forget it man.

This script does not depend on GitHub APIs, does not rely on Git hooks, doesn’t have any dependencies and doesn’t require us to touch any of our repos, now, or in the future. It pulls updates for all your git repos in the background, and notifies you on the desktop. The whole script is about 70 lines of code (with comments)! The logic is as follows :

  • Setup a cron to loop through all our git dirs.
  • On each dir do a git fetch to check for server-side updates.
  • If the git log differs, sync the repo and display a popup.

notify-send

Ubuntu, and several other Linux desktops, provide the notify-send command. It triggers a popup on the desktop :

notification.

You can customize the message and, optionally, the icon. It can be triggered with :

notify-send "TITLE" "MESSAGE"

Specify an icon using the -i option. On Ubuntu, see a list of emotes here : /usr/share/icons/gnome/48x48/emotes/

We’ll be using this command in our script to popup desktop notifications whenever our script syncs up successfully.

Now, the entire script

Let’s see the entire script in one go. We’ll break it down into logical chunks below.

Run this script as follows :

git-updater.sh /full/path/to/git/repo

and watch it flow!

Meat and bones

Let’s break down the script.

Is this a git repo?

# Check if $1 is a git repository
    stat "$1" > /dev/null || return 1;
    cd $1; echo "Checking $PWD";
    git status --porcelain || return 1; # A fatal error if $PWD is not a git repo

This code is relatively straightforward. It checks if the dir actually exists, and that it’s a git repo. git status gives a fatal error if this dir isn’t a repo.

Fetch!

# Use `git fetch` to fetch all updates across branches from all remotes. This will also fetch any NEW branches from the remote.
    # Use --tags to fetch any additional tags not associated with these branches.
    git fetch --tags

git fetch fetches remote updates to all configured branches . Add --tags to that, and you get all new tags committed to the repo as well.

Merge all branches

declare -a LOCAL_BRANCHES=$(git branch -l | cut -c3- | tr "\n" " ")

git branch -l lists all local branches. We then loop through each of these branches, filtering out anything that doesn’t track remote.

local REMOTE_COMMITS=$(git log $BRANCH..$REMOTE_BRANCH --oneline | wc -l) # Remote commits not in my local
local LOCAL_COMMITS=$(git log $REMOTE_BRANCH..$BRANCH --oneline | wc -l) # Local commits not pushed to remote

Due to my earlier git fetch, i now have remote updates that haven’t been applied. The above two lines count them up for me, which i will use in the popup summary.

# If on a checkout-out branch, only pull if there are no uncommitted local changes. Skip otherwise
# For other branches, merge remote changes
if [ "$CURR_BRANCH" = "$BRANCH" ]; then
    if [ $(git status --porcelain) = "" ]; then
        git pull
    fi
else
    git fetch origin $BRANCH:$BRANCH
fi 

These lines are key. They check if we’re on a checked-out branch now, and don’t pull anything if we actually have commits here. git pull by default will merge any fast-forward commits. If there is a conflict, this command will simply fail.

If we’re on a non-checkout branch then

git fetch origin $BRANCH:$BRANCH

will merge any commits into it WITHOUT checking out the branch. Cool huh!

Push!

# Push all updates across all "matching" branches
if [ $PUSH -gt 0 ]; then 
    git push origin :
fi

We don’t need to push on every branch. git push origin : will push all changes across “matching” branches . Matching branches are :

The special refspec : (or +: to allow non-fast-forward updates) directs Git to push “matching” branches: for every branch that exists on the local side, the remote side is updated if a branch of the same name already exists on the remote side.

Houston, we have an update!

if [ "$MESSAGE" != "" ]; then
        MESSAGE+="\nLatest commit on '$(git rev-parse --abbrev-ref HEAD)'\n"$(git log --format=format:"%h by %an, %ar%n'%s'%n" -n 1)
        echo $MESSAGE
        notify-send "$PWD Updated" "$MESSAGE" -i /usr/share/icons/gnome/48x48/emotes/face-wink.png -t 2000 
    fi

These last lines take the latest commit from our currently checked-out branch and popup a notification in case we synced up

Cron it

Setup the cron like so:

* * * * * linux_user /path/to/git-updater.sh /path/to/Project1 > /var/log/git-updater.log

The gist of it ..

The entire script is available on GitHub Gist : https://gist.github.com/adilbaig/2e7ad4bf38cfc76afb4d

Conclusion

In this post we have a working solution to a common programmer’s problem.

A few caveats before you leave :

  1. git fetch origin $BRANCH:$BRANCH will only merge fast-forward commits by default, i.e: commits that don’t cause conflicts. In case of a conflict, it simply fails
  2. In some places I have had to forcefully use origin in the script, because the real parameter was next! If you don’t have an ‘origin‘ upstream, this script will fail. There’s small chance of that, as ‘origin’ is default

Leave a comment!

Advertisements

One thought on “Self Updating Git Repos

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s