GitMigration

From Biopython
(Difference between revisions)
Jump to: navigation, search
(Additional Resources)
m (Link to git page)
 
(6 intermediate revisions by 2 users not shown)
Line 1: Line 1:
We are currently testing the benefits of migration to Git distributed version control.
+
We have recently migrated to git distributed version control (instead of [[CVS]]), as summarised [[git|here]].
  
[http://github.com/ GitHub] currently hosts the Biopython Git repository at http://github.com/biopython/biopython/
+
The process of development of Biopython with git is outlined [[GitUsage|here]].
  
This is the official repository. Currently it is synchronized with the main [[CVS]] trunk every hour, so it should be up to date most of the time. All developers and potential contributors are encouraged to try out this repository. Instructions on developing Biopython with Git follow below.
+
This page contains the information on the technicalities of the transition itself and how the core developers should deal with the changes during the transitional period.
  
= Prerequisites =
+
= Current setup =
  
== Installing Git ==
+
[http://github.com/ GitHub] currently hosts the Biopython Git repository at http://github.com/biopython/biopython/
 
+
You will need to install Git on your computer. Git (http://git-scm.com/) is available for all major operating systems. Please use the appropriate installation method as described below.
+
 
+
=== Linux ===
+
 
+
Git is now packaged in all major Linux distributions, you should find it in your package manager.
+
 
+
==== Ubuntu/Debian ====
+
 
+
You can install Git from the <tt>git-core</tt> package. e.g.,
+
<code>sudo apt-get install git-core</code>
+
 
+
You'll probably also want to install the following packages: <tt>gitk git-gui git-doc</tt>
+
 
+
=== Mac OS X ===
+
 
+
Download the <tt>.dmg</tt> disk image from http://code.google.com/p/git-osx-installer/
+
 
+
=== Windows ===
+
 
+
Two options exist for Windows: [http://code.google.com/p/msysgit/ msysGit] and running Git under [http://www.cygwin.com/ Cygwin].
+
 
+
==== msysGit (Recommended) ====
+
 
+
[http://code.google.com/p/msysgit/ msysGit] is a port of Git that runs natively on Windows via the MinGW library. Because Git was not originally designed to run on Windows, and since this is a port, some bugs exist, though are rarely encountered in everyday use of Git. See their website for download and installation instructions. Additionally, you can watch [http://gitcasts.com/posts/git-on-windows Scott Chacon's screencast on installing msysGit on Windows], and see [http://github.com/guides/using-git-and-github-for-the-windows-for-newbies this GitHub guide].
+
 
+
==== Cygwin ====
+
 
+
[http://www.cygwin.com/ Cygwin] provides a Linux-like environment for Windows. It includes access to repositories of many software packages available commonly in Linux distributions, including Git. You can find the <tt>git</tt> package under the "devel" category.
+
 
+
== Creating a GitHub account (Optional) ==
+
 
+
Once you have Git installed on your machine, you can obtain the code and start developing.
+
Since the code is hosted at GitHub, however, you may wish to take advantage of the site's offered features by signing up for a GitHub account. While a GitHub account is completely optional and not required for obtaining the Biopython code or participating in development, a GitHub account will enable all other Biopython developers to track (and review) your changes to the code base, and will help you track other developers' contributions. This fosters a social, collaborative environment for the Biopython community.
+
 
+
If you don't already have a GitHub account, you can create one [http://github.com/plans here] (the free plan is absolutely enough). Once you have created your account, upload an SSH public key by clicking on '[https://github.com/account/ Account]' after logging in. For more information on generating and uploading an SSH public key, see [http://github.com/guides/providing-your-ssh-key this GitHub guide].
+
 
+
= Obtaining the source code=
+
 
+
If you want to make changes, then there are two ways of getting the source code tree onto your machine: by simply "cloning" the repository, or by "forking" the repository on GitHub. They're not that different, in fact both will result in a directory on your machine containing a full copy of the repository on your machine. However, if you have a GitHub account, you can make your repository a public branch of the project. If you do so, other people will be able to easily review your code, make their own branches from it or merge it back to the trunk.
+
 
+
In fact you can change this later, using the .git/config file, but to make it easier I'll describe the two possibilities separately.
+
 
+
== Cloning the repository without a GitHub account ==
+
 
+
Getting a copy of the repository (called "cloning" in Git terminology) without GitHub account is very simple:
+
+
  git clone git://github.com/biopython/biopython.git
+
 
+
This command creates a local copy of the entire Biopython repository on your machine (your own personal copy of the official repository with its complete history).  You can now make local changes and commit them to this local copy.
+
 
+
If you want other people to see your changes, however, you need to take care of publishing your branch to a public server yourself. Alternatively, using GitHub takes care of this for you.
+
 
+
== Forking Biopython using a GitHub account ==
+
 
+
If you are logged in to GitHub, you can go to the Biopython repository page
+
http://github.com/biopython/biopython/tree/master
+
and click on a button named 'Fork'. This will create a fork (basically a copy) of the official Biopython repository, still on publically viewable on GitHub, but listed under your personal account.
+
 
+
Now, assuming that you have Git installed on your computer, execute the following commands locally on your machine:
+
 
+
git clone git@github.com:<your username>/biopython.git
+
 
+
Where <your username>, not surprisingly, stands for your GitHub username.
+
You have just created a local copy of the biopython repository on your machine.
+
 
+
You may want to also link your branch with the official distribution:
+
 
+
git remote add official_dist git://github.com/biopython/biopython.git
+
 
+
You can find more info here:
+
http://github.com/guides/keeping-a-git-fork-in-sync-with-the-forked-repo
+
 
+
To add additional contributors to your branch on GitHub, select 'edit' and then add them to the 'Repository Collaborators' section. You will need to know their username on GitHub.
+
 
+
= Making changes =
+
 
+
Now you can make changes to your branch. Since your local branch is a full repository, you can commit your changes as often as you like. In fact, you should commit as often as possible, because smaller commits are much better to manage and document. Let us assume you've made changes to the file Bio/x.py. You need to add this file to your change-set:
+
git add Bio/x.py
+
and now you commit:
+
git commit -m "added feature Y in Bio.x"
+
 
+
Your commits in Git are local, i.e. they affect only your working branch on your computer, and not the whole Biopython tree or even your fork on GitHub. You don't need an internet connection to commit, so you can do it very often.
+
  
Once you think your changes are fine and should be reviewed by others, you can push your changes back to the GitHub server:
+
This could become the official repository. Currently it is synchronized with the main [[CVS]] trunk every hour, so it should be up to date most of the time.
git push origin
+
  
If you think you changes are worth including in the main Biopython distribution, then [http://bugzilla.open-bio.org/ file an (enhancement) bug on Bugzilla], and include a link to your updated branch (i.e. your branch on GitHub, or another public Git server).  You could also attach a patch on Bugzilla.  If the changes are accepted, one of the Biopython developers will have to check this code into our [[CVS]] repository, and within the hour this should update the main Biopython branch on GitHub.
+
== Updates from CVS to github branch ==
  
On GitHub itself, you can inform keepers of the main branch of your changes by sending a 'pull request' from the main page of your branch. Once the file has been committed to the main branch, you may want to delete your now redundant bug fix branch on GitHub. Branches can be deleted by selecting 'edit' and then 'delete repository' from the bottom of the edit page.
+
The updates are done with the [http://cvs2svn.tigris.org/ cvs2svn] tool in the [http://cvs2svn.tigris.org/cvs2git.html cvs2git] mode.
  
== Evaluating changes ==
+
This works by extracting commits from the cvs repo and generating input for [http://www.kernel.org/pub/software/scm/git/docs/git-fast-import.html git fast-import]. It seems to work nicely and is really fast.
  
This section describes the steps to evaluate and integrate changes for Biopython core developers, or anyone accepting changes on a branch. The following commands will show you a diff of modifications on another branch. First add a reference to the remote repository, and fetch the changes locally:
+
The updates are performed hourly, using the following configuration [http://bartek.rezolwenta.eu.org/biopython_git_update/].
 +
The scripts obtains the cvs repository via rsync and performs the conversion locally. This has to be done this way at least until we get git installed on dev.open-bio.org.
  
  git remote add username/biopython-mods git://github.com/username/biopython-mods.git
+
After the conversion is done, the updated git branch is pushed to github, so that others can use it. Afterwards, the copy of the git repo is rsynced back to OBF servers for backup purposes.
  git fetch username/biopython-mods
+
 
+
To double check the branch is integrated:
+
  
  git branch -r
 
  
Finally, get the diff of the modifications:
+
== Accepting code contributions ==
  
  git diff origin/master username/biopython-mods/master
+
During the migration, the CVS repo is assumed to be still of higher priority. This means that all code contributions need to go through CVS and then get updated to the github branch. This effectively means, we cannot push to the main biopython branch directly, but instead work on different branches and generate diffs to be applied to CVS.
  
If you later want to remove the reference to the branch:
+
In case of small bug fixes, the core developers can continue to work directly in CVS. The changes will get pushed to github eventually.
  
  git branch -r -d username/biopython-mods/master
+
Since we also want to accept contributions through github, it means that core developer integrating changes will need to do some extra work:
 +
* make sure you have an updated version of cvs source tree
 +
* make sure you have a git repo with the official and contributed branch
 +
* make a diff between the contributed and official branch in git repo (see [[GitUsage]])
 +
* apply this diff to the cvs repository
 +
* commit in cvs with appropriate message
  
Alternatively, from within GitHub you can use the fork-queue to cherry pick commits from other people's forked branches.  See [http://github.com/blog/270-the-fork-queue this github blog post] for details.  While this defaults to applying the changes to your current branch, you would typically do this using a new integration branch, then fetch it to your local machine to test everything, before merging it to your main branch.
 
  
= Additional Resources =
+
== Next steps ==
  
There is a lot of different nice guides to using Git on the web:
+
Once we reach a consensus that git/github serves us well. We will make the final switch. This would include:
* [http://www.eecs.harvard.edu/~cduan/technical/git/ Understanding Git Conceptually]
+
* dropping the cvs support (updating Biopython webpage)
* http://github.com/guides/git-cheat-sheet
+
* shutting down the cvs2git update scripts
* http://zrusin.blogspot.com/2007/09/git-cheat-sheet.html
+
* installing git on the open-bio servers
* http://www.kernel.org/pub/software/scm/git/docs/v1.4.4.4/cvs-migration.html
+
* setting up a synchronization between obf-hosted branch and github branch
 +
* posting an announcement on the dev-mailing list, news server, and twitter account to celebrate!

Latest revision as of 12:23, 24 September 2009

We have recently migrated to git distributed version control (instead of CVS), as summarised here.

The process of development of Biopython with git is outlined here.

This page contains the information on the technicalities of the transition itself and how the core developers should deal with the changes during the transitional period.

Contents

Current setup

GitHub currently hosts the Biopython Git repository at http://github.com/biopython/biopython/

This could become the official repository. Currently it is synchronized with the main CVS trunk every hour, so it should be up to date most of the time.

Updates from CVS to github branch

The updates are done with the cvs2svn tool in the cvs2git mode.

This works by extracting commits from the cvs repo and generating input for git fast-import. It seems to work nicely and is really fast.

The updates are performed hourly, using the following configuration [1]. The scripts obtains the cvs repository via rsync and performs the conversion locally. This has to be done this way at least until we get git installed on dev.open-bio.org.

After the conversion is done, the updated git branch is pushed to github, so that others can use it. Afterwards, the copy of the git repo is rsynced back to OBF servers for backup purposes.


Accepting code contributions

During the migration, the CVS repo is assumed to be still of higher priority. This means that all code contributions need to go through CVS and then get updated to the github branch. This effectively means, we cannot push to the main biopython branch directly, but instead work on different branches and generate diffs to be applied to CVS.

In case of small bug fixes, the core developers can continue to work directly in CVS. The changes will get pushed to github eventually.

Since we also want to accept contributions through github, it means that core developer integrating changes will need to do some extra work:

  • make sure you have an updated version of cvs source tree
  • make sure you have a git repo with the official and contributed branch
  • make a diff between the contributed and official branch in git repo (see GitUsage)
  • apply this diff to the cvs repository
  • commit in cvs with appropriate message


Next steps

Once we reach a consensus that git/github serves us well. We will make the final switch. This would include:

  • dropping the cvs support (updating Biopython webpage)
  • shutting down the cvs2git update scripts
  • installing git on the open-bio servers
  • setting up a synchronization between obf-hosted branch and github branch
  • posting an announcement on the dev-mailing list, news server, and twitter account to celebrate!
Personal tools
Namespaces
Variants
Actions
Navigation
Toolbox