Edit this page on GitHub

How to build a release for Biopython.

Build Biopython in 22 easy steps!!

Setup required for a new release manager

The instructions below require that you have access to a few servers and the code repository. When you start, be sure to have write access to:

  1. GitHub Biopython source code repository
  2. GitHub biopython.org website repository
  3. GitHub biopython.org/DIST/ repository
  4. GitHub biopython wheel repository
  5. OBF WordPress Blog
  6. Biopython on PyPI

If you don’t have any of the above, please ask.

We assume you have cloned these under ~/repositories/ and that the git origin is the official Biopython copy of the repository.

We assume you are running Linux (but macOS should be fine too), and have the following tools installed (plus as many of the Biopython optional dependencies as possible for local testing):

  1. Python (traditionally we’ve used Python 2, but 3 should be fine)
  2. git
  3. twine, installed with pip install twine
  4. epydoc, currently using version 3.0.1
  5. LaTeX, including assorted packages like comments and preprint.
  6. hevea, I am currently using version 1.10+9 of 2008-12-17

Final commit(s)

  1. Using make sure I have the latest code:

    $ cd ~/repositories/biopython
    $ git checkout master
    $ git pull origin master
    
  2. make sure the README.rst file is still up to date

  3. add any important info to NEWS.rst or DEPRECATED.rst - you can get a log of recent git changes like this (adjust the date accordingly):

    $ git log --since="2016/01/01" --reverse --pretty="medium"
    
  4. make sure CONTRIB.rst still current

  5. make sure setup.py and MANIFEST.in are still up to date

    • Are there any new modules/files which should get installed?
  6. bump version numbers and set the release data:

    • Biopython version - edit Bio/__init__.py
    • Biopython Tutorial - update the date/version line in the Doc/Tutorial.tex file
    • Biopython NEWS.rst - fill in the release date
    • Make sure to commit the modified files to github, e.g.
    $ git commit Bio/__init__.py Doc/Tutorial.tex NEWS.rst -m "Call this Biopython 1.68"
    
  7. do last check to make sure things are checked in:

    $ rm -r build
    $ rm Tests/*.pyc
    $ make clean -C Doc
    $ git status
    
  8. build Biopython and do last regression test:

    drevil:~biopython> python setup.py build
    drevil:~biopython> python setup.py test
    

    Running the tests simultaneously is risky (e.g. Python 2 and 3), as two threads may both try to read/write to the same temp files.

  9. Push this to gitub, all being well this commit will be tagged as the release (barring no problems uncovered while building the documentation, or with the manifest while testing the tar-ball):

    $ git push origin master
    

Making and testing the tar-ball

  1. check out clean version somewhere else:

    drevil:~tmp1/> git clone git://github.com/biopython/biopython.git
    drevil:~tmp1/> cd biopython
    
  2. make documentation PDF, text and HTML files in Doc:

    drevil:~tmp1/biopython/> make -C Doc
    drevil:~tmp1/biopython/> make clean -C Doc
    
  3. make the source distribution

    drevil:~tmp1/biopython> python setup.py sdist --formats=gztar,zip
    
  4. untar the file somewhere else

    drevil:~tmp1/biopython/> cd ..
    drevil:~tmp1/> tar -xzvf biopython/dist/biopython-1.71.tar.gz
    drevil:~tmp1/> cd biopython-1.71
    

    Check to make sure it includes the HTML and PDF files under Doc

  5. make sure I can build and test it

    drevil:~tmp1/biopython-1.71/> python setup.py build
    drevil:~tmp1/biopython-1.71/> python setup.py test
    drevil:~tmp1/biopython-1.71/> python setup.py install --prefix /tmp/test-install
    

    A typical source of failure here (on the tests) is the lack of example files being added to the source distribution: add them to MANIFEST.in

Making the API documentation

  1. Update API documentation using Epydoc (this can often report otherwise overlooked errors).

    • If you haven’t already, clone the DIST repository - otherwise first fetch any upstream changes.
    $ cd ~/repositories
    $ git clone git@github.com:biopython/DIST.git
    $ cd ~/repositories/DIST
    
    • Remove version of the API documentation which we’re going to replace:
    $ git rm docs/api/*
    
    • Go to the /tmp/test-install/lib/python2.7/site-packages directory. This is the directory created under your source installation after the install step above. Running epydoc in your git tree works, but can miss some packages due to import errors.
    $ cd /tmp/test-install/lib/python2.7/site-packages
    $ grep "__version__" Bio/__init__.py
    __version__ = "1.71"
    $ epydoc -v -o ~/repositories/DIST/docs/api/ -u http://biopython.org -n Biopython --docformat restructuredtext Bio BioSQL
    $ grep "__version__" ~/repositories/DIST/docs/api/Bio-pysrc.html
    <a name="L13"></a><tt class="py-lineno"> 13</tt>  <tt class="py-line"><tt class="py-name">__version__</tt> <tt class="py-op">=</tt> <tt class="py-string">"1.71"</tt> </tt>
    
    • Assuming no mismatches in version number (which would suggest epydoc is not looking at the new test installation), commit these new HTML files to the gh-pages branch:
    $ cd ~/repositories/DIST
    $ git add docs/api/*
    $ git commit -m "epydoc for Biopython 1.68"
    
    • Update the tutorial too:
    $ cd ~/repositories/DIST/docs/tutorial/
    $ cp .../tmp1/biopython/Doc/Tutorial.html .
    $ cp .../tmp1/biopython/Doc/Tutorial.pdf .
    $ git commit Tutorial.html Tutorial.pdf -m "Tutorial for Biopython 1.71"
    
    • Push this to GitHub Pages to update the website:
    $ git push origin gh-pages
    

Tagging the release, and making wheels

  1. Back in the main repository, tag the release:

    $ cd  .../tmp1/biopython/
    $ git tag biopython-171
    $ git push origin master --tags
    
  2. Now we use https://github.com/biopython/biopython-wheels to build wheels, by updating the BUILD_COMMIT line in .travis.yml and appveyor.yaml to the new release’s git tag.

    $ cd ~/repositories
    $ git clone git@github.com:biopython/biopython-wheels.git
    $ cd biopython-wheels/
    $ git submodule update --init
    $ emacs .travis.yml  # update BUILD_COMMIT=... line
    $ emacs appveyor.yml  # update BUILD_COMMIT: ... line
    $ git commit .travis.yml appveyor.yml -m "Build Biopython 1.xx"
    $ git push origin master
    

    Check the wheels build on TravisCI for Linux and Mac and on AppVeyor for Windows.

    You don’t seem to need to update the biopython git submodule, but if you need to update multibuild this seems to work.

    $ git submodule foreach git pull origin master
    $ git commit -a -m "Update submodules"
    $ git push origin master
    
  3. Successful wheels will be on on Rackspace, download them from there to your ~/repository/biopython/dist folder. We will upload these to PyPI later using Twine.

  4. If you have a Windows machine, remove any prior Biopython installations, and confirm the Windows wheel file(s) work.

Tagging the release, and uploading

  1. Upload the new release tar-ball and zip to the website via GitHub Pages DIST repository.

    $ cp dist/biopython-1.68.* ~/repositories/DIST/
    $ cd ~/repositories/DIST/
    $ git add biopython-1.71.*
    $ git commit biopython-1.71.* -m "Downloads for Biopython 1.71"
    $ shasum -a 256 biopython-1.71.*
    $ md5sum biopython-1.71.*
    $ git commit --amend # paste checksums into comment
    $ git push origin gh-pages
    
  2. Upload to the python package index (except for beta/alpha level releases):

    $ cd  ~/repositories/biopython/
    $ pip install twine
    $ twine upload dist/biopython-1.71.tar.gz
    $ twine upload dist/biopython-1.71-*.whl
    
  3. Update the website:

    • If you haven’t already, clone the biopython.github.io repository, (otherwise make sure your copy is up to date):
    $ cd ~/repositories
    $ git clone git@github.com:biopython/biopython.github.io.git
    
    • Update the website:
    $ cd ~/repositories/biopython.github.io
    $ emacs _config.yml
    $ emacs wiki/Biopython.md
    $ emacs wiki/Download.md
    $ git commit _config.yml wiki/Biopython.md wiki/Download.md -m "Biopython 1.68 released"
    
  4. Annoucment:

    • post the announcement on news.open-bio.org (which will update the news page and twitter via the news feed)
    • send email to biopython-announce@biopython.org (see mailing lists)
    • forward the email to Linux packagers e.g. debian-med@lists.debian.org
  5. Conda-Forge should automatically open a pull request to update the package once it appears on PyPI. Check for a new pull request on github.com/conda-forge/biopython-feedstock which once merged will upload the new release to anaconda.org/conda-forge/biopython

Post release version bump

  1. Bump version numbers again

    • Update Bio/__init__.py version
    • Biopython Tutorial - update the date/version line in the Doc/Tutorial.tex file
    • Make sure to commit the modified files to github.

    Include the suffix .dev0 to indicate this is a development version e.g. if you have __version__ = "1.68", make it 1.69.dev0