TL;DR – Setting up automatic deployment to Windows Azure Websites from your source control repository is just about painless. While the free Windows Azure Websites offering is not without it’s shortcomings, and the “pay-as-you-go” model remains a little spendy for most do-it-yourselfers, the Azure websites platform brings a lot to the table. Automatic deployment from source control is very high on that list.
The Windows Azure platform has come a long, long way in the past year. While Azure is far from the only “Infrastructure-As-A-Service” platform out there, it does present a compelling feature set, and the Azure team has demonstrated an aggressive deployment cycle in pushing improvements out the door. Also, pricing for so-called “cloud services” in general have been steadily falling, making platforms such as Windows Azure, Amazon Web Services, and Linode more and more compelling.
One of the most interesting features offered by Azure of interest to the larger community are Azure Websites. Azure provides an easy to use interface for hosting one or more sites, at a variety of service levels, from a free entry level package to dedicated instance hosting.
More than One Way to Skin a Cat
There are any number of ways to manage deployments to Windows Azure Websites. This is only one. There are also numerous integration tools for Visual Studio, and for really quick and dirty solutions, check out the recently released Webmatrix 3 Preview. How you deploy will depend on your needs, and possibly your technical abilities.
Personally, whether I’m working in Visual Studio, Webmatrix, or simple text files, I find the automatic deployment from my remote source control repo the most attractive.
Deployment to Azure from Source Control
The Azure team has made it very simple to set up an Azure website, and in fact, to do so directly from source control. You can automatically publish changes to your Azure-hosted site simply by pushing to the master branch of your repo on various source control service providers, or simply from your local repository on your machine.
In this post, we’ll walk through the basics of deploying a website to Azure directly from Github, and then see how when we push changes to master on our Github repository, the changes are automatically published to our live site.
Getting Started
To follow along, you will need:
For our example, I will use a very basic site I have sitting around, consisting of an index.html file, a CSS file, and an a Jpeg image (this just happens to be a small practice project I had sitting around, so I used it).
First, I will initialize a new local repository for the site, and commit all the files. Then, after creating a corresponding remote repo on my Github account, I push the content up to the new remote:
Create a Git Repository and Push to Remote:
Now that your site files have been pushed to the Github remote, log into your Azure account, navigate to the Websites” tab, and click the Create A Website link:
From here, your should be looking at something like the picture below. Click on the “Custom Create” link:
From here, you will see some options, as below. You will need to provide a URL for your Azure site. Note that the free websites do not allow custom domain names. You can pick any name you want, (so long as it is not already taken) but it will be a sub-domain within .azurewebsites.net. In my case, I entered jattenGitDeploymentExample, and the full URL for my site will be jattenGitDeploymentExample.azurewebsites.net. Not the greatest URL, but it will do for our example.
You will also need to pick a geographical region in which your site will be hosted. This represents the primary datacenter location where your files will live. The most likely choice would be the one closes to where you live.
We don’t need a database for this simple exercise. Last, make sure the “Publish from source control” box is checked. Then click the right-pointing arrow at the bottom of the screen.
Setting Up Deployment Options for a Windows Azure Website:
Next, you need to tell Azure where your repo is. Notice there are a number of available options. In other words, while we are using Github for this example, you could just as easily publish from any of the other listed options, including ( but not visible in the picture here) your local machine.
Select Your Source Code Provider:
If you are not signed in to Github on your local machine, you will be prompted to do so, and taken to your account sign-in at Github in a separate browser window. Once you have signed in to Github, return to the Azure window and select Github again.
Once you select your source control location and click the right-pointing arrow again, the next window asks you which repository you want to publish. Select the repo for the site we just pushed up to master. Also, you can specify a different branch. Here, we are going to stick with master. However, you could just as easily create a special branch (named” production” for example). Just remember, whenever you push changes to the branch specified here, the result will be published to your site:
Select the Remote Repository to Hook Into:
Once you click the check icon in the lower right corner, Azure will hook into your Github repository and deploy your site. This might take a few seconds. When provisioning and deployment are complete, you should see this:
The Site is Deployed and Live:
Clicking on the link in the URL column will open your site in a new browser tab. Congratulations, your site is now deployed, direct from your Github repository. More importantly, your site is now hooked into branch master, and when changes are pushed or merged to that branch, those changes will be automatically deployed to your Azure site.
Opening the Example Site:
Now, if we look in the image above, we can see that there is some problem with my HTML – apparently I am missing the opening bracket on the <!DOCTYPE html> tag. Also, I want to add some text to my main content area.
In doing this, we’ll take a quick look at how changes are automatically deployed when we push to the remote repo on Github.
Pushing Changesets to Automatically Deploy
First, I want to fix that problematic DOCTYPE tag. I’ll open the file, and correct it in my text editor:
Fixing the Broken <!DOCTYPE html> Tag:
Then I will add some text content to the main content area:
Adding Text Content:
Now, I save my changes, then open my Bash window. Checking status, I can see my changes are ready to be stages and committed. So I do that, and then push to origin master:
Commit and Push to Remote:
That done, I hit “Reload” (or hit the F5 key) on my browser window where my site is currently open:
Reload the Site in the Browser Window:
Voila! My changes have been deployed, simply by pushing to the master branch of my Github repository. The broken <!DOCTYPE html> tag has been fixed, and the new text appears in the Main Content area.
Viewing Deployment History in the Azure Portal
Once your site is deployed, Azure keeps a record of the deployment history. If you click on your website in the websites list of the Azure portal:
You will be taken to THIS window. Click on the deployment history link:
Notice, the active deployment is your most recent commit. In fact, if you look closely, you will see that the deployment ID shown just to the left of the blue Github icon matches the commit hash from your repository:
Automatic Deployment to Windows Azure Websites is a slick feature of the Windows Azure platform. While Azure has come a long way, it is clearly just getting started. Hopefully, costs will continue to come down such that deploying one or more sites to this platform matches comparable shared and dedicated hosting plans.
Additional Resources
John on Google CodeProject
This post describes a rather specific use of Git’s rebase command. Rebase is a powerful and complex topic which I am not fully covering here. I will attempt a more thorough look at rebase in general in an upcoming post. If you are newer to Git, as I am, you might want to refer to a few of my previous posts covering some of the basics:
A handy feature of Git is the ability to use the interactive rebase command to combine, re-organize, and otherwise clean up a messy commit history once development of a feature or bug fix is complete. I am a firm believer in the practice of “commit early, commit often” approach. However, this can leave untold number of commits which, when merged back into mainline development, clutter up a repository.
How often do you end up creating a commit which consists of some modified whitespace after cleaning up a code file, or a commit arising from updating some comments or renaming variables?
Oftentimes, when we complete a feature or fix, we can clearly see how (hindsight being, more often than not, 20/20)we would have liked the code to come together. In fact, for myself, I often wish I could just combine all of the relevant commits into one, which I could then merge back into master. For more complex features, or those which may have undergone substantial evolution during development, it can sometimes become clear how the commit history should have occurred, and not necessarily in the order of manner in which it actually did.
For reasons of maintenance, or particularly if we plan to push our code out to a public repository, it would be nice if we could straighten things up a little, and present a commit history to the world which will:
- Allow us to rollback changes in a manner which does not cripple our codebase.
- Make sense to those who might need to maintain or perhaps branch from our repository.
- Not leave us looking like disorganized idiots (“Did you really commit the deletion of a single empty line?”).
Interactive rebase gives us a way to write that history.
A Simple Example
I will start with an overly simple example. Assume a straightforward git repository consisting of a master branch. In order to add a new feature, you create a feature branch, names (appropriately, feature). On the new feature branch, you work up the basic functionality of the new feature, knowing you will be fine-tuning things shortly. However, you commit the work to this point.
While developing the primary feature code, you noticed a bug in another part of the codebase, perhaps impacting the development of your new feature. You quickly spot a solution, fix the bug, and commit.
Once that is done, you fine-tune the new feature code until it is right where you want it. After committing those changes, you decide to do some cleanup. You remove unnecessary whitespace, rename some methods for clarity, and other housekeeping tasks. After committing this last batch of changes, you are ready to merge your work into master.
Except . . . the new feature, in this case, is not overly complex, and you sure wish you could merge it into master as a single commit. Also, that bug fix should remain separate, as that should be applied to master separately from the new feature. That way, if for whatever reason the new feature needs to be rolled back, it can be done so independently of the bug fix. As of right now, the state of your repo is something like this:
Repo After Feature Development
If we were to perform a standard merge at this point, our repo would look like this:
Repo State After Hypothetical Standard Merge:
As I stated, this is a rather simplified example. But you can see where we might want to combine our feature development (commits C, D, and E) into a single commit before merging to our master branch, while leaving our bug fix (commit C) on its own.
Use Git’s Interactive Rebase Feature to Re-Write History
I created a (very contrived) sample project with a main application on master, and a feature branch. After writing some trivial code to illustrate the commit history we are using in our example, I ran Gitk, which now looks like this:
The Feature Branch Shown in Gitk:
Similarly, the git log command, run from within our feature branch, shows us this:
Git Log Contents for Feature Branch:
If we want to re-write the commit history here such that commits B, D, and E are combined into a single new commit, we can tell git to run an interactive rebase which will allow us to provide direction to git with respect to how we would like it to handle each of the commits involved in the rebase.
What we are actually going to do here, consistent with how rebase works, is rewrite one or more commits in the same branch onto the parent commit we specify in the command below. Because we will be using the interactive flag, we will be allowed to reorder the commits we have selected for the operation, as well as provide some other direction as to how we want git to handle the process.
The Command Syntax is like this:
$ git rebase -i HEAD~n
In the above, n represents the number of commits we would like to edit. However, it is important to note that this actually points to the parent of commit n as the rebase target. So, having examined the state of my feature branch, I can see that I would like clean up commits B thru E. I want to combine Commits B, D and E into s single commit, and leave commit C as a stand-alone (the bug fix).
Here is what I will type into the command shell:
$ git rebase -i HEAD~4
After I type that in and hit enter, my editor widow opens, and looks like this:
The Editor Window After Entering the Interactive Rebase Command:
In this case, the first four lines which start with “pick” represent the commits I have chosen to edit. Note the order in which they appear. When we are finished editing, git will process our revised commits from top to bottom. This is significant because we can re-order them if we so choose. Also note the comments below. We are given some options here which we can apply to the editing process. If we replace the pick keyword with one of these flags for any specific commit, that operation will be performed on that commit.
Of particular interest to us in this case is the squash or s flag. When we replace pick with squash, we tell git to combine that commit with the previous one (remember the processing order, top to bottom). Remember what we want to do in this rather contrived example – we want to combine commits B, D, and E into one. However, the current processing order doesn’t allow us to do this, because our bug fix is sandwiched between commits B and D.
So first we will need to change the processing order by moving some commits around. In the editor, we can easily Cut/Paste the top two lines to look like this instead:
Changing the Commit Processing Order in the Editor:
Now, git will process our edited commits as ordered above, starting with C, then B, then D, then E. Next, we tell git to squash commits D and E by replacing the pick flag with s or squash, so that they both roll up into the previous commit (Commit B, in this case):
Change the Commit Processing Flags in the Editor:
Now, having made the changes above, SAVE THE EDITOR CONTENTS (I often forget this step!) and close the editor window. Your command line window will return while Git processes the commits, and then the editor reappears. Git now wants you to sort out the commit messages, so that the newly created commit history will make sense.
In our example case, the first thing that we see is this:
Making Sense of the Commit History and Messages in the Editor:
We can see that the three commit messages in question are from our original commits B, D, and E, related to our new feature. We can clear all of these (and the comments included with them) and type a new, more meaningful commit message. Note that any line prefixed with a hash symbol (“#”) will be ignored:
Modifying the Commit Message for the Combined Feature Commit in the Editor:
Now, SAVE THE EDITOR CONTENT AGAIN and close the editor window. The command line window now looks like this:
Command Window After Successful Interactive Rebase:
If we run git log now, we see the following:
Git Log After Successful Interactive Rebase:
If we run Gitk, we see this:
As we can see, in the above, we now have a single commit representing the implementation of our new feature, and the original bug fix (previously, commit C) where previously there were four distinct commits. We have re-written history, in preparation for a sensible merge into our master branch.
Notice that our bug fix commit (previously commit C) now has a different hash identifier than previously, and the combined commit is brand-new as well. This is important, because again, HISTORY HAS BEEN CHANGED. The implication here is that if the previous history has been pushed to a public repository (or otherwise cloned, perhaps locally) the history of the feature branch will no longer be the same. In other words, we might as well think of Commit C as a new commit F, and the combined commit as a new commit G.
IMPORTANT: This should not be performed on commits which have been pushed to a public repository, if other committers are working with your code!!! It is generally best to reserve this technique for code in your local working tree, before publishing.
Graphically, our repository now looks like THIS:
Repo After Interactive Rebase:
But, What About Merge Conflicts During the Interactive Rebase?
As often as not, you will run into merge conflicts during this process. I contrived the example above specifically to illustrate the ability to combine commits, and studiously made sure conflicts would not occur here.
In cases where there are conflicts, Git will stop the rebase process, and demand that you fix any conflicts before resuming. Once you have resolved any conflicts, type git rebase –continue in the command window, and the process will resume. We’ll take a look at conflict resolution in another post.
Additional Resources
John on Google CodeProject
NOTE: This post is mainly targeted at newer git users who may want a more detailed look at what is happening. If you consider yourself comfortable with git, I have also created a “reference” version that skips most of the narrative:
The Quick Reference
- Link to condensed version HERE
Also, some navigation aids, because this is a long-ish post:
Problems introduced by “Nested” library projects in a Git Repository
Often it is the case I find myself building out a project and incorporating another library I have been working on (such as a custom control) as a dependency. The library in question may, in fact, be consumed by other projects as well (that IS the purpose of creating such libraries, right?). Equally as often, the project I am referencing as a dependency in my main project will itself be in a state of evolution. A problem arises related to either or both of the following:
- The newly introduced dependency (the “sub-project” e.g. custom control or such) is also being actively developed, and I will want the option to pull in any changes as that library evolves.
- I may, in the course of working on my “main” project, make some modifications to the sub-project, some or all of which I decide should be incorporated into the main line of development of the shared library.
John on Google CodeProject 
Set a reference to the binaries of the shared library from within my main project, perform all modifications to the shared library from within that project, and re-build so that changes are reflected in the binary outputs. - Use Git “Sub modules” to refer to a specific commit within the library project, and manage the updating of these commit pointers as the library project evolves.
- Copy the dependency project source into the main project. Yuck. Isn’t the whole idea of libraries to avoid this type of thing?
I have not used Git sub modules much, but my limited experience with them indicates to me that outside of certain circumstances, they are a pain. Equally unsatisfactory under the conditions I described in the first paragraph is the managing of a standard binary reference (especially in cases where I may need to customize the shared library within my main project, while keeping it up to date with the development repo. The third option is not even worth considering.
In searching for an optimal way to handle the “nested libraries” problem, I came across a few articles on the Sub-tree Merge workflow in GIT. In this workflow, we accomplish the following (this represents my understanding of what is going on – if you know differently, please let me know in the comments!):
- Maintain a tracking branch which tracks the remote shared library project, from which we can pull in changes made in mainline development of the shared library. This remote tracking branch tracks and represents the history of the shared library, distinctly form the history of the main project.
- From this remote, create a “subtree” as a subdirectory within the master branch (or whichever branch we designate) of the main project. The new sub-directory contains a copy of the shared library source code. This subtree does not bring with it the history of commits in the actual shared library. Instead, it joins and shares history with the main project.
- Once this is done, we can pull new change sets down from the shared library remote as needed, and merge them into our sub tree (there’s a trick to this, though, so keep reading). Likewise, we can also merge changes we make within our subtree directory out to our remote tracking branch, and push them up to the shared library remote.
Essentially, we will have told Git to read the contents of the remote tracking branch into a sub-directory on branch master, and remember that this sub-directory represents a subtree based on the remote tracking branch. Graphically, I think of it a little like this:
This is How I Picture the Git Subtree Merge Workflow:
Walk Through:
Let’s walk through the process of creating a subtree within an example project, and then we’ll discuss some things to pay attention to when it comes to merging changes to and from the subtree remote.
I have created an example project as the “main” project for use in the following examples. I will be using a Pdf Merge utility project created for another post on this blog as the “shared library.” The main project is called, unsurprisingly, “MainProject.” The shared library is my oh-so-useful iTextToolsExample. We will assume I have already created MainProject, created a local repository on my development machine, and a remote repo on Github.
From the Git command line, we set up a remote that points at the repository for the shared library project. I will be using an example project as the main project, and adding a shared library I created for another post as a subtree.
Git Add Remote Syntax:
$ git add <local_remote_name> <remote_location>
e.g:
$ git remote add itext_remote https://github.com/xivSolutions/iTextToolsExample.git
Next we need to fetch down the shared library into our working directory using the remote we just created. Note, we want to use “fetch” here, as we don’t want to commit what we add to branch master:
Git Fetch Syntax:
$ git fetch <remote_name>
e.g:
$ git fetch itext_remote
Note the warning after we fetch about “no common commits.” That’s ok – we wouldn’t expect there to be an common history here, we are importing a library for that very reason!
Now, in keeping with the illustration above, we want to get the contents of the shared library tracked in their own remote tracking branch, so we can keep things in sync separate from master:
Checkout to New Branch Syntax:
$ git checkout -b <new_branch_name> remote_name/branch_name
e.g:
$ git checkout –b itext_branch itext_remote/master
Now, the new tracking branch points to the root of the shared library project. We can check this by examining the directory contents here in itext_branch:
Directory Listing in iText Branch:
And then checkout master and compare:
Directory Listing in Master:
Now is when we perform the magic. We will use git read-tree to read the contents of itext_branch into a subdirectory of master. This will be essentially the same as copying it all in, with on small exception: Git knows and remembers that this is a subtree related to the remote tracking branch itext_branch.
Git Read-Tree Syntax:
$ git read-tree --prefix=<subdirectory_name>/ –u <shared_library_branch>
e.g:
$ git read-tree –-prefix=itextTools/ –u itext_branch
In the above, <subdirectory_name> can be anything you like – it is going to be the name of a new “folder” or subdirectory within your mater branch. The --prefix flag tells git that this will be the name of a directory. I usually use the same name as the original shared library project. Note it is important to include the forward-slash after this. The –s flag tells git to go ahead and update the working tree with the new changes. Also note, we execute this command from within branch master:
Running Git Read-Tree From Branch Master:
Now, if we run git status, we can see that a bevy of new files have been added to our master branch, in a directory named (not surprisingly) “iTextTools:”
Status of Branch Master After Read-Tree Operation:
After you have performed the read-tree operation, the files have been read into our master branch as described above. The important thing here is that the history they share with the remote source repository (as referenced by the itext_branch) has NOT. We can commit this as a single commit:
Commit the New Directory and Files:
We have now added the shared iTextTools library as a subtree in our master branch. If we check the contents of our project directory on branch master now, we see the new iTextTools subdirectory:
Now, what if there are changes introduced to the main shared library source project? How can keep our sub-project in sync with mainline development on the shared library?
If we need to pull changes down from the original shared library source into our sub-project, we can simply checkout the remote tracking branch itext_branch and use git pull, which will pull down and merge any changes made since we originally fetched the project. Say we (or whoever maintains the itextTools project) added a new class named SomeNewClass and solution file to the main development repo for the itextTools library:
Pulling Changes down from the Shared Library Remote:
While the above pulls the changes down, we need to be careful with this next step – merging the new change set into our subtree in master. The changes we just pulled down bring with them all of the history from the iText remote repository. In almost all cases, we do not want to merge this history with that of our main project, so we can’t simply do a merge or rebase to get the new changes into master. Instead, we want to use the following:
Squash and No-Commit Change Sets into the Subtree (Syntax):
$ git merge --squash -s subtree --no-commit <source_branch>
e.g:
$ git merge --squash –s subtree –-no-commit itext_branch
In the above, the --s flag indicates that you are going to specify which merge strategy git should use in merging changes. This is followed by the strategy itself, which in this case is the subtree strategy. We are also telling git to squash the commits from <source_branch> into a single change set, and to not automatically commit when the merge is complete.
So, now that we have pulled the changes down into our remote tracking branch, we checkout master, and execute the above command:
Merging Changes from Remote Shared Library into Subtree (from branch master):
If we check the status of master again, we see the changes from the remote iText library repo have been merged into the appropriate subdirectory, and staged for commit:
Status of Master After Subtree Merge Strategy with --Squash and --No-Commit:
From here, simply commit (with a descriptive commit message!).
If you made some changes to the shared library code within your subproject and decide you want to push them out to the main development repo, the process is essentially the same, in reverse. You merge from the subtree directory within the main project (again, explicitly specifying the Subtree Merge strategy), out to the tracking branch that points to the shared library remote. Then push the changes up to the remote.
Say we made some changes to SomeNewClass while working in MainProject and we decided they should be incorporated into mainline development for the itextTools project main repo:
- Once you have made changes to the shared library from within branch master of your main project, commit the changes as normal.
- Then, checkout the tracking branch that points to the shared library remote.
Now, we use the same basic merge command from before, in the opposite direction:
Syntax to Merge from Subtree to the Remote Tracking Branch:
$ git merge --squash -s subtree --no-commit <source_branch>
e.g:
$ git merge --squash –s subtree --no-commit master
Merging the itextTools Subtree into the Remote Tracking Branch:
As in the previous example, we can check Git status and see that the changes we made in our sub-tree project have been merged into the remote tracking branch and are now staged for commit:
Now commit the changes. I usually state clearly in the commit message that these are changes from <project>/subtree so that, when added to the shared library, I can clearly identify that this set of changes came from the sub-project. Now it’s time to push the changes back up to the main shared library repo.
Pushing from Tracking Branch to Shared Library Remote Syntax:
$ git push <shared_library_remote> <library_tracking_branch>
e.g:
$ git push itext_remote itext_branch
Pushing to the Remote Repository:
Note that using the syntax above will push the changes into a new branch in my remote repo, named (not surprisingly) itext_branch. This will allow me to apply the changes to the master or production branch of the shared library using a merge or pull request (in the case where I am not the maintainer of the shared library).
There is More To It
I have brushed through the basics here. There are numerous ways to condense some of the commands I used here, and like most things, there are probably dozens of different viewpoints on how to best manage project dependencies in source control. The Subtree Strategy has been working best for me, in my working context, but I welcome your feedback, and especially, corrections where I might have it wrong, or have missed seeing a better way.
Thanks for reading.
Additional Resources
I am not going to pretend there aren’t any number of good resources from which I learned the above. Most of the information here I assimilated from elsewhere. Here I have striven to simply present my own understanding, in a way which might have helped *ME* when I was trying to figure it out. Here are some of those sources:
John on Google CodeProject
The Subtree Merge Workflow offers a relatively painless mechanism for managing shared library source code as a component of a larger project. If you have ever had a shared library, custom control, or other component under development that you also wanted to use within one or more dependent projects, you no doubt understand what I mean.
This post is a condensed version of a longer post with more examples, and aimed at those less familiar with Git. If, like me, you have a little trouble following what is going on here, check out the other post first. This one is more of a reference.
In searching for an optimal way to handle the “nested libraries” problem, I came across a few articles on the Sub-tree Merge workflow in GIT. In this workflow, we accomplish the following (this represents my understanding of what is going on – if you know differently, please let me know in the comments!):
- Maintain a tracking branch which tracks the remote shared library project, from which we can pull in changes made in mainline development of the shared library. This remote tracking branch tracks and represents the history of the shared library, distinctly form the history of the main project.
- From this remote, create a “subtree” as a subdirectory within the master branch (or whichever branch we designate) of the main project. The new sub-directory contains a copy of the shared library source code. This subtree does not bring with it the history of commits in the actual shared library. Instead, it joins and shares history with the main project.
- Once this is done, we can pull new change sets down from the shared library remote as needed, and merge them into our sub tree (there’s a trick to this, though, so keep reading). Likewise, we can also merge changes we make within our subtree directory out to our remote tracking branch, and push them up to the shared library remote.
My Mental Picture of the Subtree Workflow:
In my head, I picture the subtree relationships something like THIS:
Create a Subtree in your Main Project
In the examples below, I will use MainProject as the name of the main project, in which I want to utilize a shared library which is also under development. I will use the iTextTools library from a previous post as the example shared library.
Step 1: Point a Remote in the Main Project at the Shared Library Repository:
$ git remote add <library_remote_name> <library_remote_location>
e.g.
$ git remote add itext_remote https://github.com/xivSolutions/iTextToolsExample.git
Step 2: Fetch Shared Library into the Working Tree:
$ git fetch <library_remote_name>
e.g:
$ git fetch itext_remote
Step 3: Checkout a New Tracking Branch Based on the Shared Library Remote:
$ git checkout -b <new_branch_name> remote_name/branch_name
e.g:
git checkout –b itext_branch itext_remote/master
Step 4: Read the Library Project Into Master as a Subtree Directory:
Before performing this step, switch back to branch master (or whatever your chosen target branch is). The read-tree command is going to read the contents of the remote library tracking branch into the branch you execute the read-tree command from.
- The --prefix option establishes the name of the subdirectory into which the contents of the read will be placed.
- The --u flag tells git to update the contents of the working tree once the read is complete. Files will be staged for commit when the read is complete.
$ git read-tree --prefix=<subdirectory_name>/ -u <shared_library_branch>
e.g:
$ git read-tree -–prefix=iTextTools/ –u itext_branch
Step 5: Commit the New Files:
Commit as normal. From this point forward, the files added in the new subdirectory will be tracked as part of this branch, and share history with MainProject. However, Git also knows about the subtree directory. We can leverage this information to merge changes between the local version of our shared library and the remote library repository, without merging the separate project histories (which have nothing in common other than the use of the library project source code).
As we will see, there are a few things to keep in mind when doing so, but overall it is pretty simple.
Update the Library Sub-Project from the Shared Library Remote
If changes are made to the shared library at the source repository, we will likely want to bring them into our sub-project so that the subproject is up-to-date with the latest code. The process is a little different than the standard pull, fetch/merge, or rebase process we are accustomed to, because most of the time, we don’t want to bring the commit history from the shared library remote into our main project (as they are unrelated and share no common commits).
Before performing this update, I recommend committing any other local changes you may have introduced in MainProject so that, when you commit the imported changes retrieved for the shared library code, it will stand as its own unique commit.
Step 1: Checkout the Shared Library Remote Tracking Branch:
$ git checkout <library_branch_name>
e.g:
$ git checkout itext_branch
Step 2: Pull from Remote Library Repository:
When you set up the remote tracking branch originally, you also pointed it at the the shared library remote repo, so Git understands that git pull from this branch means to fetch/merge from the proper location:
$ git pull
Step 3: Checkout Target Branch
Before we perform any merging, we obviously need to make sure we are back on the branch we want to merge the changes into:
$ git checkout <target_branch>
e.g:
$ git checkout master
Step 4: Merge Changesets without Merging History
This is where things differ a little from a normal merge or rebase. We want to merge all of the changes, but we don’t want to bring any commit history with it. Also, because we are targeting our subtree directory, we need to explicitly specify the subtree merge strategy. In the below, the –s flag tells git we will be explicitly prescribing a merge strategy, followed by the strategy to use (obviously, subtree in this case):
$ git merge --squash -s subtree --no-commit <library_branch>
e.g:
$ git merge --squash -s subtree --no-commit itext_branch
Step 5: Commit:
Now the sub-project version of the shared library is up-to-date with the remote. Commit as normal. I recommend using a descriptive commit message indicating that this commit is specifically related to updating the subtree project.
Update the Shared Library Remote with Changes from the Library Sub-Project
It may happen that while working in your main project, you introduce modifications or additional features to the shared library code that you decide should be integrated into the Shared Library source repository. Doing so is essentially the reverse of the preceding. Simply merge the changes from your working branch out to the shared library remote tracking branch (again, without bringing the local project history along), then push to the shared library remote (and/or submit a pull request, depending upon who maintains the shared library repository).
Obviously, prior to performing the following steps, commit your changes in the working tree on branch master (or whatever your working branch is) in your main project.
Step 1: Checkout the Shared Library Remote Tracking Branch
$ git checkout <library_branch>
e.g:
$ git checkout itext_branch
Step 2: Merge Changes from the Library Sub-Project Without Merging History
As with the previous situation, in all but some very odd circumstance we don’t want to merge the two projects histories together, so we use the --squash option when we merge. Also as before, we explicitly specify the subtree merge strategy:
$ git merge --squash -s subtree --no-commit <source_branch>
e.g:
$ git merge –squash –s subtree –no-commit master
Step 3: Commit the Merged Changes:
Now, assuming there were no merge conflicts, commit as normal, again using a descriptive commit message identifying the commit as a merge from a subtree in an outside project.
Step 4: Push Changes to the Shared Library Repo
When we go to push changes from our remote tracking branch to the shared library remote, the command below will actually push our changes on a new branch (named, not surprisingly, “itext_branch”). For me, this is the behavior I want. This way, I (or whoever is the maintainer of the remote library) can integrate the changes into the mainline of development from within the library remote. In other words, we are not merging directly back into master (rarely how we would want to do things under these circumstances). Obviously, if we are NOT the maintainer of the shared library, this is where a pull request would come in instead of a direct merge.
$ git push <library_remote> <library_branch>
e.g:
$ git push itext_remote itext_branch
Final Thoughts
I do not claim to have the authoritative post on this topic by any means. I have written this, and the longer-form post, as much as a learning exercise and reference for myself as anything else. I have sought to present the information in a way which would have helped me understand ti when I was first learning some of Git’s more advanced features and techniques.
I am aware that there are more condensed version of some of the commands used, and quite possibly better ways of doing this. If you have read this far, and have noted some incredible stupidity on my part herein, please do point it out in the comments.
Additional Resources
As I learned this, I referred frequently to the following, among other sources:
John on Google CodeProject
CodeProject
Up to this point . . .
This is the third part in a series focused on getting familiar with Git for Windows developers. The series assumes little or no experience using the command line. If you missed the first few posts, here is what we have covered so far:
In the previous post, we learned how to navigate directories within the Windows file system, create new directories, and delete directories. Now let’s look at creating files, and performing rudimentary (VERY rudimentary) text editing from the Bash command prompt.
Before We Proceed . . .
Let’s create a directory to use for the following examples. Open your Bash Command Line Interface (CLI) and make sure you are in your default Windows user folder. Create a new directory within your user folder called BashExcercises, and then move into that directory (Review the previous post if you are unsure how to do this):
Create a Directory for the Following Exercises
Also remember, when we are explaining syntax, we will often use square brackets to denote optional items, and angle brackets to denote use-provided content. In the following:
mkdir [options] <FileName>
We do not type the square brackets or angle brackets. The [options] are, well, optional, meaning there may be none or there may be many, and the <Filename> is required and provided by the user.
Bash: Create/Edit Files (touch/echo)
There are a couple of ways to create a new file using the Bash command prompt. The most basic is the touch command. The purpose of the touch command is ostensibly to change/modify the date/time stamps of files. However, it is commonly used to create new files as well, since, when used without any optional arguments, it does just that: creates a new file, with a date stamp.
Using the Touch Command to Create a New Text File
The syntax for the touch command is
Create a new file with a date stamp:
touch [options] <filename>
There are a number of options for use with the touch command, but for now, we are going to focus on simple file creation, and we won’t be using any of the options. If we want to create a simple text file, we can type:
Create a new Text File Named MyNewTextFile:
$ touch MyNewTextFile.txt
Type the above into your Bash window and hit enter:
Remember from the previous post that in many cases, when a Bash command executes successfully, we are rewarded with simply a new command prompt. However, we can use the ls command we learned from the last post to examine the contents of our directory:
What do you know? There is is. Now, you could open the new file using Notepad (or the text editor of your choosing), but you won’t see much. We have created an empty text file. To open the file using Notepad from the Bash prompt, simply type:
Open MyNewTextFile.txt in Notepad using Bash:
$ notepad MyNewTextFile.txt
Or, of course, you could open the file the old-fashioned way - by double clicking on it with your mouse!
Using the echo Command to Write Strings as Output
Before we look at using echo to create a new file, we should understand a little about what echo DOES.
The primary design purpose of echo is apparently to write strings to the standard output (usually, the Bash CLI).
The syntax for echo is:
Echo Command Syntax:
$ echo [options] <string>
Lets try that now. Type the following into Bash:
Output a line of Text to the Bash Screen:
$ echo show this line on the screen
We can also tell the echo command to write a line of text to a file, using the following syntax:
Syntax: Append text to the end of a file:
$ echo [options] <Text string> >> <FileName>
By using the >> (double right-hand angle-brackets) operator, we tell Bash to append the text to the end of the file. Type the following into the Bash window and hit enter:
$ echo This line was added to the end of the file >> MyNewTextFile.txt
Now, if we use our $ notepad MyNewTextFile.txt Command, we see the following:
Close that window, and let’s add another line. Type the following and hit enter:
Adding Another Line of Text:
$ echo This is another line of text >> MyNewTextFile.txt
The use $ notepad MyNewTextFile.txt again:
Huh. Looks like it worked, kind of. Apparently, Bash isn’t kidding about appending the new text to the end of the file. Happily, we can use the > (a single right-hand angle bracket) operator to replace the text in the current file and get rid of that mess. Type this and hit enter. Then open the file in Notepad again:
Replace the Text in a File with a New Text String:
$ echo This line replaced the old mess > MyNewTextFile.txt
Your Bash window should look like THIS:
And you should see this in your Notepad window:
Use the Echo Command to Create a New File with Text Content:
Ok, back to the original subject - creating a new text file. We can use echo to create a new file, and include the specified text string as content. In fact, it works the same as when we appended or replaced the text in an existing file, because, lacking an option argument telling it not to, Bash will assume that if the file specified in our echo command does not exist, that we want to create one. Type the following and hit enter:
$ echo This is a new line in a new file >> DifferentTextFile.txt
Then use ls -1 to display the directory contents. This is what your Bash window should look like:
Note the presence of our new file. Let’s open it in Notepad, and see what there is to see:
Well, well well. Whadd’ya know!
Bash: Remove Files (rm)
Ok, now lets get rid of our first file. To remove (delete) files using Bash, we use the rm command. The syntax for removing files is as follows;
rm [options] <FileName>
SO now, let’s delete that first file, MyNewTextFile.txt. Type the following, hit enter, then use the ls -1 command to display the directory contents:
rm MyNewTextFile.txt
Your Bash window should look like this:
Remove a Directory and all Contents (rm -rf)
In the previous post, we discussed how to remove an empty directory. But what if we want to remove a direcory and all of its contents? We use the rm command with the -r and -f options:
Syntax for Removing a Directory and All of its Contents:
rm -rf <FolderName>
So now we are going to remove the BashExcercises directory we created at the beginning of this post. Of course, we can’t remove a directory if we are currently IN that directory. So first, let’s return to the directory above (in this case, our home folder) using the cd .. Comand (the cd Command, followed by a single space, followed by two periods in a row. This one means “move up one level”).
Next, type the following into your Bash window, hit enter, and then use the ls -d */ command to view the directories within the current directory (which should be, at this point, your Windows User folder):
$ rm -rf BashExcercises
When you are done, your Bash window should resemble this (note that the directories are organized alphabetically in columns, and the BashExcercises directory is no longer there. Also note that for obvious reasons, your user folder will contain different folders than mine!).
Now, let’s use everything we just learned in this post, plus we’ll add one final tidbit at the end which you may find useful. We’re going to step through this right quick. We will:
- Add one more new directory
- Add a file containing a line of text to the new directory
- Remove the directory and its contents
- Have Bash tell us what was removed.
1. Add a New Directory Named OneLastFolder to our current directory:
$ mkdir OneLastFolder
2. Add a New File Named OneLastFile.txt, containing the text “One more text line” to the new folder:
$ echo One more text line >> OneLastFolder/OneMoreFile.txt
Notice in the above, I did not navigate into the new folder to create the new file, but simply used the relative path from my current location.
3. Remove the directory and all its contents, and have Bash tell us what was done:
rm -rfv OneLastFolder
Notice in the above, I added one more optional argument, the -v option (for “verbose”). This tells Bash to print the actions it took to the output window.
When you have done all of that, your window should look like this:
There you have it. You now have the most basic set of tools for using the Bash Command Line Interface. From here, we are ready to examine the basic command line operations required to start using Git to manage our code. We’ll pick that up in the next post in this series.
Additional Resources:
John on Google