Biopython - User contributions [en]

GSOC

Joaor — Wed, 13 Mar 2013 13:21:34 GMT

Joaor: Redirect to main GSOC page

#REDIRECT [[Google_Summer_of_Code]]

Google Summer of Code

Joaor — Wed, 13 Mar 2013 13:20:59 GMT

Joaor: Content update for GSOC 2013

== Introduction ==

The Open Bioinformatics foundation successfully [http://www.open-bio.org/wiki/Google_Summer_of_Code applied to participate in the Google Summer of Code].

Please read the [http://www.open-bio.org/wiki/Google_Summer_of_Code GSoC page at the Open Bioinformatics Foundation] and the [http://code.google.com/soc main Google Summer of Code page] for more details about the program.

== Mentor List ==

Usually, each BioPython proposal has one or more mentors assigned to it. Nevertheless, we encourage potential students/mentors to contact the [http://biopython.org/wiki/Mailing_lists mailing lists] with their own ideas for proposals. There is therefore not a set list of 'available' mentors, since it highly depends on which projects are proposed every year.

Past mentors include:

* [http://casbon.me/ James Casbon]
* [https://github.com/chapmanb Brad Chapman]
* [http://www.hutton.ac.uk/staff/peter-cock Peter Cock]
* [http://wiki.binf.ku.dk/User:Thomas_Hamelryck Thomas Hamelryck]
* [http://www.linkedin.com/in/reece Reece Hart]
* [http://nmr.chem.uu.nl/~joao João Rodrigues]
* [http://etal.myweb.uga.edu/ Eric Talevich]

== Proposals ==
=== 2013 ===

The BioPython proposals for the Google Summer of Code 2013 will be published here once discussed. We encourage potential students and mentors to join the [http://biopython.org/wiki/Mailing_lists BioPython mailing lists] and actively participate in these discussions, either by submitting their own ideas or contributing to improving existing ones.

== Past Proposals ==

=== 2012 ===
==== [http://biopython.org/wiki/SearchIO SearchIO] ====
; Rationale
: Biopython has general APIs for parsing and writing assorted sequence file formats (SeqIO), multiple sequence alignments (AlignIO), phylogenetic trees (Phylo) and motifs (Bio.Motif). An obvious omission is something equivalent to BioPerl's SearchIO. The goal of this proposal is to develop an easy-to-use Python interface in the same style as SeqIO, AlignIO, etc but for pairwise search results. This would aim to cover EMBOSS muscle & water, BLAST XML, BLAST tabular, HMMER, Bill Pearson's FASTA alignments, and so on.
; Approach
: Much of the low level parsing code to handle these file formats already exists in Biopython, and much as the SeqIO and AlignIO modules are linked and share code, similar links apply to the proposed SearchIO module when using pairwise alignment file formats. However, SearchIO will also support pairwise search results where the pairwise sequence alignment itself is not available (e.g. the default BLAST tabular output). A crucial aspect of this work will be to design a pairwise-search-result object heirachy that reflects this, probably with a subclass inheriting from both the pairwise-search-result and the existing MultipleSequenceAlignment object. Beyond the initial challenge of an iterator based parsing and writing framework, random access akin to the Bio.SeqIO.index and index_db functionality would be most desirable for working with large datasets.
; Challenges
: The project will cover a range of important file formats from major Bioinformatics tools, thus will require familiarity with running these tools, and understanding their output and its meaning. Inter-converting file formats is part of this.
; Difficulty and needed skills
: Medium/Hard depending on how many objectives are attempted. The student needs to be fluent in Python and have knowledge of the BioPython codebase. Experience with all of the command line tools listed would be clear advantages, as would first hand experience using BioPerl's SearchIO. You will also need to know or learn the git version control system.
; Mentors
: [http://www.hutton.ac.uk/staff/peter-cock Peter Cock]

==== [http://arklenna.tumblr.com/tagged/gsoc2012 Representation and manipulation of genomic variants] ====
; Rationale
: Computational analysis of genomic variation requires the ability to reliably communicate and manipulate variants. The goal of this project is to provide facilities within BioPython to represent sequence variation objects, convert them to and from common human and file representations, and provide common manipulations on them.
; Approach & Goals
* Object representation
** identify variation types to be represented (SNV, CNV, repeats, inversions, etc)
** develop internal machine representation for variation types
** ensure coverage of essential standards, including HGVS, GFF, VCF
* External representations
** write parser and generators between objects and external string and file formats
* Manipulations
** canonicalize variations with more than one valid representation (e.g., ins versus dup and left shifting repeats).
** develop coordinate mapping between genomic, cDNA, and protein sequences (HGVS)
* Other
** release code to appropriate community efforts and write short manuscript
** implement web service for HGVS conversion
; Difficulty and needed skills
: Easy-to-Medium depending on how many objectives are attempted. The student will need have skills in most or all of: basic molecular biology (genomes, transcripts, proteins), genomic variation, Python, BioPython, Perl, BioPerl, NCBI Eutilities and/or Ensembl API. Experience with computer grammars is highly desirable. You will also need to know or learn the git version control system.
; Mentors
: [http://www.linkedin.com/in/reece Reece Hart]
: [https://github.com/chapmanb Brad Chapman]
: [http://casbon.me/ James Casbon]
=== 2011 ===
==== [http://biopython.org/wiki/GSoC2011_mtrellet Biomolecular Interface Analysis] ====
; Student
: Mikael Trellet
; Rationale
: Analysis of protein-protein complexes interfaces at a residue level yields significant information on the overall binding process. Such information can be broadly used for example in binding affinity studies, interface design, and enzymology. To tap into it, there is a need for tools that systematically and automatically analyze protein structures, or that provide means to this end. Protorop (http://www.bioinformatics.sussex.ac.uk/protorp/) is an example of such a tool and the elevated number of citations the server has had since its publication acknowledge its importance. However, being a webserver, Protorop is not suited for large-scale analysis and it leaves the community dependent on its maintainers to keep the service available. On the other hand, Biopython’s structural biology module, Bio.PDB, provides the ideal parsing machinery and programmatic structures for the development of an offline, open-source library for interface analysis. Such a library could be easily used in large-scale analysis of protein-protein interfaces, for example in the CAPRI experiment evaluation or in benchmark statistics. It would be also reasonable, if time permits, to extend this module to deal with protein-DNA or protein-RNA complexes, as Biopython supports nucleic acids already.
; Approach & Goals
* Add the new module backbone in current Bio.PDB code base
** Evaluate possible code reuse and call it into the new module
** Try simple calculations to be sure that there is stability between the different modules (parsing for example) and functions
* Define a stable benchmark
** Select few PDB files among interface size and proteins size would be different
* Extend IUPAC.Data module with residue information
** Deduce residues weight from Atom instead of direct dictionary storage
** Polar/charge character (dictionary or influenced by pH)
** Hydrophobicity scale(s)
* Implement Extended Residue class as a subclass of Residue
* Implement Interface object and InterfaceAnalysis module
* Develop functions for interface analysis
** Calculation of interface polar character statistics (% of polar residues, apolar, etc)
** Calculation of BSA calling MSMS or HSA
** Calculation of SS element statistics in the interface through DSSP
** Unit tests and use of results as input for further calculations by other tools and scripts
* Develop functions for Interface comparison
* Code organization and final testing

; Difficulty and needed skills
: Easy/Medium. Working knowledge of the Bio.PDB module of BioPython. Knowledge of structural biology in general and associated file formats (PDB).
; Mentors
: [http://nmr.chem.uu.nl/~joao João Rodrigues]
: [http://etal.myweb.uga.edu/ Eric Talevich]

==== [http://biopython.org/wiki/GSOC2011_Mocapy A Python bridge for Mocapy++] ====
; Student
: Michele Silva
; Rationale
: Discovering the structure of biomolecules is one of the biggest problems in biology. Given an amino acid or base sequence, what is the three dimensional structure? One approach to biomolecular structure prediction is the construction of probabilistic models. A Bayesian network is a probabilistic model composed of a set of variables and their joint probability distribution, represented as a directed acyclic graph. A dynamic Bayesian network is a Bayesian network that represents sequences of variables. These sequences can be time-series or sequences of symbols, such as protein sequences. Directional statistics is concerned mainly with observations which are unit vectors in the plane or in three-dimensional space. The sample space is typically a circle or a sphere. There must be special directional methods which take into account the structure of the sample spaces. The union of graphical models and directional statistics allows the development of probabilistic models of biomolecular structures. Through the use of dynamic Bayesian networks with directional output it becomes possible to construct a joint probability distribution over sequence and structure. Biomolecular structures can be represented in a geometrically natural, continuous space. Mocapy++ is an open source toolkit for inference and learning using dynamic Bayesian networks that provides support for directional statistics. Mocapy++ is excellent for constructing probabilistic models of biomolecular structures; it has been used to develop models of protein and RNA structure in atomic detail. Mocapy++ is used in several high-impact publications, and will form the core of the molecular modeling package Phaistos, which will be released soon. The goal of this project is to develop a highly useful Python interface to Mocapy++, and to integrate that interface with the Biopython project. Through the Bio.PDB module, Biopython provides excellent functionality for data mining biomolecular structure databases. Integrating Mocapy++ and Biopython will allow training a probabilistic model using data extracted from a database. Integrating Mocapy++ with Biopython will create a powerful toolkit for researchers to quickly implement and test new ideas, try a variety of approaches and refine their methods. It will provide strong support for the field of biomolecular structure prediction, design, and simulation.
; Approach & Goals
: Mocapy++ is a machine learning toolkit for training and using Bayesian networks. It has been used to develop probabilistic models of biomolecular structures. The goal of this project is to develop a Python interface to Mocapy++ and integrate it with Biopython. This will allow the training of a probabilistic model using data extracted from a database. The integration of Mocapy++ with Biopython will provide a strong support for the field of protein structure prediction, design and simulation.
; Mentors
: [http://etal.myweb.uga.edu/ Eric Talevich]
: [http://wiki.binf.ku.dk/User:Thomas_Hamelryck Thomas Hamelryck]

==== [http://biopython.org/wiki/GSOC2011_MocapyExt MocapyExt] ====
; Student
: Justinas V. Daugmaudis
; Rationale
: BioPython is a very popular library in Bioinformatics and Computational Biology. Mocapy++ is a machine learning toolkit for parameter learning and inference in dynamic Bayesian networks (DBNs), which encode probabilistic relationships among random variables in a domain. Mocapy++ is freely available under the GNU General Public Licence (GPL) from SourceForge. The library supports a wide spectrum of DBN architectures and probability distributions, including distributions from directional statistics. Notably, Kent distribution on the sphere and the bivariate von Mises distribution on the torus, which have proven to be useful in formulating probabilistic models of protein and RNA structure. Such a highly useful and powerful library, which has been used in such projects as TorusDBN, Basilisk, FB5HMM with great success, is the result of the long-term effort. The original Mocapy implementation dates back to 2004, and since then the library has been rewritten in C++. However, C++ is a statically typed and compiled programming language, which does not facilitate rapid prototyping. As a result, currently Mocapy++ has no provisions for dynamic loading of custom node types, and a mechanism to plug-in new node types that would not require to modify and recompile the library is of interest. Such a plug-in interface would assist rapid prototyping by allowing to quickly implement and test new probability distributions, which, in turn, could substantially reduce development time and effort; the user would be empowered to extend Mocapy++ without modifications and subsequent recompilations. Recognizing this need, the project (herein referred as MocapyEXT), with the aim to improve the current Mocapy++ node type extension mechanism, has been proposed by T. Hamelryck.
; Approach & Goals
: The MocapyEXT project is largely an engineering effort to bring a transparent Python plug-in interface to Mocapy++, where built-in and dynamically loaded node types could be used in a uniform manner. Also, externally implemented and dynamically loaded nodes could be modified by a user and these changes will not necessitate the recompilation of the client program, nor the accompanying Mocapy++ library. This will facilitate rapid prototyping, ease the adaptation of currently existing code, and improve the software interoperability whilst introducing minimal changes to the existing Mocapy++ interface, thus facilitating a smooth acceptance of the changes introduced by MocapyEXT.
; Mentors
: [http://etal.myweb.uga.edu/ Eric Talevich]
: [http://wiki.binf.ku.dk/User:Thomas_Hamelryck Thomas Hamelryck]

=== 2010 ===
==== [http://biopython.org/wiki/GSOC2010_Joao Improving Bio.PDB] ====
; Student
: [http://nmr.chem.uu.nl/~joaor João Rodrigues]
; Rationale
: Biopython is a very popular library in Bioinformatics and Computational Biology. Its Bio.PDB module, originally developed by Thomas Hamelryck, is a simple yet powerful tool for structural biologists. Although it provides a reliable PDB parser feature and it allows several calculations (Neighbour Search, RMS) to be made on macromolecules, it still lacks a number of features that are part of a researcher's daily routine. Probing for disulphide bridges in a structure and adding polar hydrogen atoms accordingly are two examples that can be incorporated in Bio.PDB, given the module's clever structure and good overall organisation. Cosmetic operations such as chain removal and residue renaming – to account for the different existing nomenclatures – and renumbering would also be greatly appreciated by the community. Another aspect that can be improved for Bio.PDB is a smooth integration/interaction layer for heavy-weights in macromolecule simulation such as MODELLER, GROMACS, AutoDock, HADDOCK. It could be argued that the easiest solution would be to code hooks to these packages' functions and routines. However, projects such as the recently developed edPDB or the more complete Biskit library render, in my opinion, such interfacing efforts redundant. Instead, I believe it to be more advantageous to include these software' input/output formats in Biopython's SeqIO and AlignIO modules. This, together with the creation of interfaces for model validation/structure checking services/software would allow Biopython to be used as a pre- and post-simulation tool. Eventually, it would pave the way for its inclusion in pipelines and workflows for structure modelling, molecular dynamics, and docking simulations.
; Mentors
: [http://etal.myweb.uga.edu/ Eric Talevich]
: [http://www.hutton.ac.uk/staff/peter-cock Peter Cock]
: Diana Jaunzeikare

=== 2009 ===

==== [http://biopython.org/wiki/PhyloXML PhyloXML] ====
; Rationale
: PhyloXML is an XML format for phylogenetic trees, designed to allow storing information about the trees themselves (such as branch lengths and multiple support values) along with data such as taxonomic and genomic annotations. Connecting these pieces of evolutionary information in a standard format is key for comparative genomics.
A Bioperl driver for phyloXML was created during the 2008 Summer of Code; this project aims to build a similar module for the popular Biopython package.
; Mentors
: [https://github.com/chapmanb Brad Chapman]
: Christian Zmasek

==== [http://biopython.org/wiki/BioGeography Biogeographical Phylogenetics for BioPython] ====
; Rationale
: I developed Bio.Geography, a new module for the bioinformatics programming toolkit Biopython. Bio.Geography expands upon Biopython's traditional capabilities for accessing gene and protein sequences from online databases by allowing automated searching, downloading, and parsing of geographic location records from GBIF, the authoritative aggregator of specimen information from natural history collections worldwide. This will enable analyses of evolutionary biogeography that require the areas inhabited by the species at the tips of the phylogeny, particularly for large-scale analyses where it is necessary to process thousands of specimen occurrence records. The module will also facilitate applications such as species mapping, niche modeling, error-checking of museum records, and monitoring range changes.
; Mentors
: [https://github.com/chapmanb Brad Chapman]
: Stephen Smith
: David Kidd

GSOC

Joaor — Fri, 08 Mar 2013 22:57:49 GMT

Joaor: Added earlier projects (2010, 2009)

== Introduction ==

The Open Bioinformatics foundation successfully [http://www.open-bio.org/wiki/Google_Summer_of_Code applied to participate in the Google Summer of Code].

Please read the [http://www.open-bio.org/wiki/Google_Summer_of_Code GSoC page at the Open Bioinformatics Foundation] and the [http://code.google.com/soc main Google Summer of Code page] for more details about the program.

== Mentor List ==

Usually, each BioPython proposal has one or more mentors assigned to it. Nevertheless, we encourage potential students/mentors to contact the [http://biopython.org/wiki/Mailing_lists mailing lists] with their own ideas for proposals. There is therefore not a set list of 'available' mentors, since it highly depends on which projects are proposed every year.

Past mentors include:

* [http://casbon.me/ James Casbon]
* [https://github.com/chapmanb Brad Chapman]
* [http://www.hutton.ac.uk/staff/peter-cock Peter Cock]
* [http://wiki.binf.ku.dk/User:Thomas_Hamelryck Thomas Hamelryck]
* [http://www.linkedin.com/in/reece Reece Hart]
* [http://nmr.chem.uu.nl/~joao João Rodrigues]
* [http://etal.myweb.uga.edu/ Eric Talevich]

== Proposals ==
=== 2013 ===

The BioPython proposals for 2013 will be published here once discussed. We encourage potential students and mentors to join the [http://biopython.org/wiki/Mailing_lists mailing lists] and actively participate in these discussions, either by submitting their own ideas or contributing to improving existing ones.

== Past Proposals ==

=== 2012 ===
==== [http://biopython.org/wiki/SearchIO SearchIO] ====
; Rationale
: Biopython has general APIs for parsing and writing assorted sequence file formats (SeqIO), multiple sequence alignments (AlignIO), phylogenetic trees (Phylo) and motifs (Bio.Motif). An obvious omission is something equivalent to BioPerl's SearchIO. The goal of this proposal is to develop an easy-to-use Python interface in the same style as SeqIO, AlignIO, etc but for pairwise search results. This would aim to cover EMBOSS muscle & water, BLAST XML, BLAST tabular, HMMER, Bill Pearson's FASTA alignments, and so on.
; Approach
: Much of the low level parsing code to handle these file formats already exists in Biopython, and much as the SeqIO and AlignIO modules are linked and share code, similar links apply to the proposed SearchIO module when using pairwise alignment file formats. However, SearchIO will also support pairwise search results where the pairwise sequence alignment itself is not available (e.g. the default BLAST tabular output). A crucial aspect of this work will be to design a pairwise-search-result object heirachy that reflects this, probably with a subclass inheriting from both the pairwise-search-result and the existing MultipleSequenceAlignment object. Beyond the initial challenge of an iterator based parsing and writing framework, random access akin to the Bio.SeqIO.index and index_db functionality would be most desirable for working with large datasets.
; Challenges
: The project will cover a range of important file formats from major Bioinformatics tools, thus will require familiarity with running these tools, and understanding their output and its meaning. Inter-converting file formats is part of this.
; Difficulty and needed skills
: Medium/Hard depending on how many objectives are attempted. The student needs to be fluent in Python and have knowledge of the BioPython codebase. Experience with all of the command line tools listed would be clear advantages, as would first hand experience using BioPerl's SearchIO. You will also need to know or learn the git version control system.
; Mentors
: [http://www.hutton.ac.uk/staff/peter-cock Peter Cock]

==== [http://arklenna.tumblr.com/tagged/gsoc2012 Representation and manipulation of genomic variants] ====
; Rationale
: Computational analysis of genomic variation requires the ability to reliably communicate and manipulate variants. The goal of this project is to provide facilities within BioPython to represent sequence variation objects, convert them to and from common human and file representations, and provide common manipulations on them.
; Approach & Goals
* Object representation
** identify variation types to be represented (SNV, CNV, repeats, inversions, etc)
** develop internal machine representation for variation types
** ensure coverage of essential standards, including HGVS, GFF, VCF
* External representations
** write parser and generators between objects and external string and file formats
* Manipulations
** canonicalize variations with more than one valid representation (e.g., ins versus dup and left shifting repeats).
** develop coordinate mapping between genomic, cDNA, and protein sequences (HGVS)
* Other
** release code to appropriate community efforts and write short manuscript
** implement web service for HGVS conversion
; Difficulty and needed skills
: Easy-to-Medium depending on how many objectives are attempted. The student will need have skills in most or all of: basic molecular biology (genomes, transcripts, proteins), genomic variation, Python, BioPython, Perl, BioPerl, NCBI Eutilities and/or Ensembl API. Experience with computer grammars is highly desirable. You will also need to know or learn the git version control system.
; Mentors
: [http://www.linkedin.com/in/reece Reece Hart]
: [https://github.com/chapmanb Brad Chapman]
: [http://casbon.me/ James Casbon]
=== 2011 ===
==== [http://biopython.org/wiki/GSoC2011_mtrellet Biomolecular Interface Analysis] ====
; Student
: Mikael Trellet
; Rationale
: Analysis of protein-protein complexes interfaces at a residue level yields significant information on the overall binding process. Such information can be broadly used for example in binding affinity studies, interface design, and enzymology. To tap into it, there is a need for tools that systematically and automatically analyze protein structures, or that provide means to this end. Protorop (http://www.bioinformatics.sussex.ac.uk/protorp/) is an example of such a tool and the elevated number of citations the server has had since its publication acknowledge its importance. However, being a webserver, Protorop is not suited for large-scale analysis and it leaves the community dependent on its maintainers to keep the service available. On the other hand, Biopython’s structural biology module, Bio.PDB, provides the ideal parsing machinery and programmatic structures for the development of an offline, open-source library for interface analysis. Such a library could be easily used in large-scale analysis of protein-protein interfaces, for example in the CAPRI experiment evaluation or in benchmark statistics. It would be also reasonable, if time permits, to extend this module to deal with protein-DNA or protein-RNA complexes, as Biopython supports nucleic acids already.
; Approach & Goals
* Add the new module backbone in current Bio.PDB code base
** Evaluate possible code reuse and call it into the new module
** Try simple calculations to be sure that there is stability between the different modules (parsing for example) and functions
* Define a stable benchmark
** Select few PDB files among interface size and proteins size would be different
* Extend IUPAC.Data module with residue information
** Deduce residues weight from Atom instead of direct dictionary storage
** Polar/charge character (dictionary or influenced by pH)
** Hydrophobicity scale(s)
* Implement Extended Residue class as a subclass of Residue
* Implement Interface object and InterfaceAnalysis module
* Develop functions for interface analysis
** Calculation of interface polar character statistics (% of polar residues, apolar, etc)
** Calculation of BSA calling MSMS or HSA
** Calculation of SS element statistics in the interface through DSSP
** Unit tests and use of results as input for further calculations by other tools and scripts
* Develop functions for Interface comparison
* Code organization and final testing

; Difficulty and needed skills
: Easy/Medium. Working knowledge of the Bio.PDB module of BioPython. Knowledge of structural biology in general and associated file formats (PDB).
; Mentors
: [http://nmr.chem.uu.nl/~joao João Rodrigues]
: [http://etal.myweb.uga.edu/ Eric Talevich]

==== [http://biopython.org/wiki/GSOC2011_Mocapy A Python bridge for Mocapy++] ====
; Student
: Michele Silva
; Rationale
: Discovering the structure of biomolecules is one of the biggest problems in biology. Given an amino acid or base sequence, what is the three dimensional structure? One approach to biomolecular structure prediction is the construction of probabilistic models. A Bayesian network is a probabilistic model composed of a set of variables and their joint probability distribution, represented as a directed acyclic graph. A dynamic Bayesian network is a Bayesian network that represents sequences of variables. These sequences can be time-series or sequences of symbols, such as protein sequences. Directional statistics is concerned mainly with observations which are unit vectors in the plane or in three-dimensional space. The sample space is typically a circle or a sphere. There must be special directional methods which take into account the structure of the sample spaces. The union of graphical models and directional statistics allows the development of probabilistic models of biomolecular structures. Through the use of dynamic Bayesian networks with directional output it becomes possible to construct a joint probability distribution over sequence and structure. Biomolecular structures can be represented in a geometrically natural, continuous space. Mocapy++ is an open source toolkit for inference and learning using dynamic Bayesian networks that provides support for directional statistics. Mocapy++ is excellent for constructing probabilistic models of biomolecular structures; it has been used to develop models of protein and RNA structure in atomic detail. Mocapy++ is used in several high-impact publications, and will form the core of the molecular modeling package Phaistos, which will be released soon. The goal of this project is to develop a highly useful Python interface to Mocapy++, and to integrate that interface with the Biopython project. Through the Bio.PDB module, Biopython provides excellent functionality for data mining biomolecular structure databases. Integrating Mocapy++ and Biopython will allow training a probabilistic model using data extracted from a database. Integrating Mocapy++ with Biopython will create a powerful toolkit for researchers to quickly implement and test new ideas, try a variety of approaches and refine their methods. It will provide strong support for the field of biomolecular structure prediction, design, and simulation.
; Approach & Goals
: Mocapy++ is a machine learning toolkit for training and using Bayesian networks. It has been used to develop probabilistic models of biomolecular structures. The goal of this project is to develop a Python interface to Mocapy++ and integrate it with Biopython. This will allow the training of a probabilistic model using data extracted from a database. The integration of Mocapy++ with Biopython will provide a strong support for the field of protein structure prediction, design and simulation.
; Mentors
: [http://etal.myweb.uga.edu/ Eric Talevich]
: [http://wiki.binf.ku.dk/User:Thomas_Hamelryck Thomas Hamelryck]

==== [http://biopython.org/wiki/GSOC2011_MocapyExt MocapyExt] ====
; Student
: Justinas V. Daugmaudis
; Rationale
: BioPython is a very popular library in Bioinformatics and Computational Biology. Mocapy++ is a machine learning toolkit for parameter learning and inference in dynamic Bayesian networks (DBNs), which encode probabilistic relationships among random variables in a domain. Mocapy++ is freely available under the GNU General Public Licence (GPL) from SourceForge. The library supports a wide spectrum of DBN architectures and probability distributions, including distributions from directional statistics. Notably, Kent distribution on the sphere and the bivariate von Mises distribution on the torus, which have proven to be useful in formulating probabilistic models of protein and RNA structure. Such a highly useful and powerful library, which has been used in such projects as TorusDBN, Basilisk, FB5HMM with great success, is the result of the long-term effort. The original Mocapy implementation dates back to 2004, and since then the library has been rewritten in C++. However, C++ is a statically typed and compiled programming language, which does not facilitate rapid prototyping. As a result, currently Mocapy++ has no provisions for dynamic loading of custom node types, and a mechanism to plug-in new node types that would not require to modify and recompile the library is of interest. Such a plug-in interface would assist rapid prototyping by allowing to quickly implement and test new probability distributions, which, in turn, could substantially reduce development time and effort; the user would be empowered to extend Mocapy++ without modifications and subsequent recompilations. Recognizing this need, the project (herein referred as MocapyEXT), with the aim to improve the current Mocapy++ node type extension mechanism, has been proposed by T. Hamelryck.
; Approach & Goals
: The MocapyEXT project is largely an engineering effort to bring a transparent Python plug-in interface to Mocapy++, where built-in and dynamically loaded node types could be used in a uniform manner. Also, externally implemented and dynamically loaded nodes could be modified by a user and these changes will not necessitate the recompilation of the client program, nor the accompanying Mocapy++ library. This will facilitate rapid prototyping, ease the adaptation of currently existing code, and improve the software interoperability whilst introducing minimal changes to the existing Mocapy++ interface, thus facilitating a smooth acceptance of the changes introduced by MocapyEXT.
; Mentors
: [http://etal.myweb.uga.edu/ Eric Talevich]
: [http://wiki.binf.ku.dk/User:Thomas_Hamelryck Thomas Hamelryck]

=== 2010 ===
==== [http://biopython.org/wiki/GSOC2010_Joao Improving Bio.PDB] ====
; Student
: [http://nmr.chem.uu.nl/~joaor João Rodrigues]
; Rationale
: Biopython is a very popular library in Bioinformatics and Computational Biology. Its Bio.PDB module, originally developed by Thomas Hamelryck, is a simple yet powerful tool for structural biologists. Although it provides a reliable PDB parser feature and it allows several calculations (Neighbour Search, RMS) to be made on macromolecules, it still lacks a number of features that are part of a researcher's daily routine. Probing for disulphide bridges in a structure and adding polar hydrogen atoms accordingly are two examples that can be incorporated in Bio.PDB, given the module's clever structure and good overall organisation. Cosmetic operations such as chain removal and residue renaming – to account for the different existing nomenclatures – and renumbering would also be greatly appreciated by the community. Another aspect that can be improved for Bio.PDB is a smooth integration/interaction layer for heavy-weights in macromolecule simulation such as MODELLER, GROMACS, AutoDock, HADDOCK. It could be argued that the easiest solution would be to code hooks to these packages' functions and routines. However, projects such as the recently developed edPDB or the more complete Biskit library render, in my opinion, such interfacing efforts redundant. Instead, I believe it to be more advantageous to include these software' input/output formats in Biopython's SeqIO and AlignIO modules. This, together with the creation of interfaces for model validation/structure checking services/software would allow Biopython to be used as a pre- and post-simulation tool. Eventually, it would pave the way for its inclusion in pipelines and workflows for structure modelling, molecular dynamics, and docking simulations.
; Mentors
: [http://etal.myweb.uga.edu/ Eric Talevich]
: [http://www.hutton.ac.uk/staff/peter-cock Peter Cock]
: Diana Jaunzeikare

=== 2009 ===

==== [http://biopython.org/wiki/PhyloXML PhyloXML] ====
; Rationale
: PhyloXML is an XML format for phylogenetic trees, designed to allow storing information about the trees themselves (such as branch lengths and multiple support values) along with data such as taxonomic and genomic annotations. Connecting these pieces of evolutionary information in a standard format is key for comparative genomics.
A Bioperl driver for phyloXML was created during the 2008 Summer of Code; this project aims to build a similar module for the popular Biopython package.
; Mentors
: [https://github.com/chapmanb Brad Chapman]
: Christian Zmasek

==== [http://biopython.org/wiki/BioGeography Biogeographical Phylogenetics for BioPython] ====
; Rationale
: I developed Bio.Geography, a new module for the bioinformatics programming toolkit Biopython. Bio.Geography expands upon Biopython's traditional capabilities for accessing gene and protein sequences from online databases by allowing automated searching, downloading, and parsing of geographic location records from GBIF, the authoritative aggregator of specimen information from natural history collections worldwide. This will enable analyses of evolutionary biogeography that require the areas inhabited by the species at the tips of the phylogeny, particularly for large-scale analyses where it is necessary to process thousands of specimen occurrence records. The module will also facilitate applications such as species mapping, niche modeling, error-checking of museum records, and monitoring range changes.
; Mentors
: [https://github.com/chapmanb Brad Chapman]
: Stephen Smith
: David Kidd

GSOC

Joaor — Fri, 08 Mar 2013 22:44:56 GMT

Joaor: /* MocapyExt */

== Introduction ==

The Open Bioinformatics foundation successfully [http://www.open-bio.org/wiki/Google_Summer_of_Code applied to participate in the Google Summer of Code].

Please read the [http://www.open-bio.org/wiki/Google_Summer_of_Code GSoC page at the Open Bioinformatics Foundation] and the [http://code.google.com/soc main Google Summer of Code page] for more details about the program.

== Mentor List ==

Usually, each BioPython proposal has one or more mentors assigned to it. Nevertheless, we encourage potential students to contact the mailing list with their own
ideas for proposals. There is therefore not a set list of 'available' mentors, since it highly depends on which projects are proposed every year.

Past mentors include:

* [http://casbon.me/ James Casbon]
* [https://github.com/chapmanb Brad Chapman]
* [http://www.hutton.ac.uk/staff/peter-cock Peter Cock]
* [http://wiki.binf.ku.dk/User:Thomas_Hamelryck Thomas Hamelryck]
* [http://www.linkedin.com/in/reece Reece Hart]
* [http://nmr.chem.uu.nl/~joao João Rodrigues]
* [http://etal.myweb.uga.edu/ Eric Talevich]

== Proposals ==
=== 2013 ===

The BioPython proposals for 2013 will be published here once discussed. We encourage potential students to join the mailing lists and actively participate in these discussions, either by
submitting their own ideas or contributing to improving existing ones.

== Past Proposals ==

=== 2012 ===
==== [http://biopython.org/wiki/SearchIO SearchIO] ====
; Rationale
: Biopython has general APIs for parsing and writing assorted sequence file formats (SeqIO), multiple sequence alignments (AlignIO), phylogenetic trees (Phylo) and motifs (Bio.Motif). An obvious omission is something equivalent to BioPerl's SearchIO. The goal of this proposal is to develop an easy-to-use Python interface in the same style as SeqIO, AlignIO, etc but for pairwise search results. This would aim to cover EMBOSS muscle & water, BLAST XML, BLAST tabular, HMMER, Bill Pearson's FASTA alignments, and so on.
; Approach
: Much of the low level parsing code to handle these file formats already exists in Biopython, and much as the SeqIO and AlignIO modules are linked and share code, similar links apply to the proposed SearchIO module when using pairwise alignment file formats. However, SearchIO will also support pairwise search results where the pairwise sequence alignment itself is not available (e.g. the default BLAST tabular output). A crucial aspect of this work will be to design a pairwise-search-result object heirachy that reflects this, probably with a subclass inheriting from both the pairwise-search-result and the existing MultipleSequenceAlignment object. Beyond the initial challenge of an iterator based parsing and writing framework, random access akin to the Bio.SeqIO.index and index_db functionality would be most desirable for working with large datasets.
; Challenges
: The project will cover a range of important file formats from major Bioinformatics tools, thus will require familiarity with running these tools, and understanding their output and its meaning. Inter-converting file formats is part of this.
; Difficulty and needed skills
: Medium/Hard depending on how many objectives are attempted. The student needs to be fluent in Python and have knowledge of the BioPython codebase. Experience with all of the command line tools listed would be clear advantages, as would first hand experience using BioPerl's SearchIO. You will also need to know or learn the git version control system.
; Mentors
: [http://www.hutton.ac.uk/staff/peter-cock Peter Cock]

==== [http://arklenna.tumblr.com/tagged/gsoc2012 Representation and manipulation of genomic variants] ====
; Rationale
: Computational analysis of genomic variation requires the ability to reliably communicate and manipulate variants. The goal of this project is to provide facilities within BioPython to represent sequence variation objects, convert them to and from common human and file representations, and provide common manipulations on them.
; Approach & Goals
* Object representation
** identify variation types to be represented (SNV, CNV, repeats, inversions, etc)
** develop internal machine representation for variation types
** ensure coverage of essential standards, including HGVS, GFF, VCF
* External representations
** write parser and generators between objects and external string and file formats
* Manipulations
** canonicalize variations with more than one valid representation (e.g., ins versus dup and left shifting repeats).
** develop coordinate mapping between genomic, cDNA, and protein sequences (HGVS)
* Other
** release code to appropriate community efforts and write short manuscript
** implement web service for HGVS conversion
; Difficulty and needed skills
: Easy-to-Medium depending on how many objectives are attempted. The student will need have skills in most or all of: basic molecular biology (genomes, transcripts, proteins), genomic variation, Python, BioPython, Perl, BioPerl, NCBI Eutilities and/or Ensembl API. Experience with computer grammars is highly desirable. You will also need to know or learn the git version control system.
; Mentors
: [http://www.linkedin.com/in/reece Reece Hart]
: [https://github.com/chapmanb Brad Chapman]
: [http://casbon.me/ James Casbon]
=== 2011 ===
==== [http://biopython.org/wiki/GSoC2011_mtrellet Biomolecular Interface Analysis] ====
; Student
: Mikael Trellet
; Rationale
: Analysis of protein-protein complexes interfaces at a residue level yields significant information on the overall binding process. Such information can be broadly used for example in binding affinity studies, interface design, and enzymology. To tap into it, there is a need for tools that systematically and automatically analyze protein structures, or that provide means to this end. Protorop (http://www.bioinformatics.sussex.ac.uk/protorp/) is an example of such a tool and the elevated number of citations the server has had since its publication acknowledge its importance. However, being a webserver, Protorop is not suited for large-scale analysis and it leaves the community dependent on its maintainers to keep the service available. On the other hand, Biopython’s structural biology module, Bio.PDB, provides the ideal parsing machinery and programmatic structures for the development of an offline, open-source library for interface analysis. Such a library could be easily used in large-scale analysis of protein-protein interfaces, for example in the CAPRI experiment evaluation or in benchmark statistics. It would be also reasonable, if time permits, to extend this module to deal with protein-DNA or protein-RNA complexes, as Biopython supports nucleic acids already.
; Approach & Goals
* Add the new module backbone in current Bio.PDB code base
** Evaluate possible code reuse and call it into the new module
** Try simple calculations to be sure that there is stability between the different modules (parsing for example) and functions
* Define a stable benchmark
** Select few PDB files among interface size and proteins size would be different
* Extend IUPAC.Data module with residue information
** Deduce residues weight from Atom instead of direct dictionary storage
** Polar/charge character (dictionary or influenced by pH)
** Hydrophobicity scale(s)
* Implement Extended Residue class as a subclass of Residue
* Implement Interface object and InterfaceAnalysis module
* Develop functions for interface analysis
** Calculation of interface polar character statistics (% of polar residues, apolar, etc)
** Calculation of BSA calling MSMS or HSA
** Calculation of SS element statistics in the interface through DSSP
** Unit tests and use of results as input for further calculations by other tools and scripts
* Develop functions for Interface comparison
* Code organization and final testing

; Difficulty and needed skills
: Easy/Medium. Working knowledge of the Bio.PDB module of BioPython. Knowledge of structural biology in general and associated file formats (PDB).
; Mentors
: [http://nmr.chem.uu.nl/~joao João Rodrigues]
: [http://etal.myweb.uga.edu/ Eric Talevich]

==== [http://biopython.org/wiki/GSOC2011_Mocapy A Python bridge for Mocapy++] ====
; Student
: Michele Silva
; Rationale
: Discovering the structure of biomolecules is one of the biggest problems in biology. Given an amino acid or base sequence, what is the three dimensional structure? One approach to biomolecular structure prediction is the construction of probabilistic models. A Bayesian network is a probabilistic model composed of a set of variables and their joint probability distribution, represented as a directed acyclic graph. A dynamic Bayesian network is a Bayesian network that represents sequences of variables. These sequences can be time-series or sequences of symbols, such as protein sequences. Directional statistics is concerned mainly with observations which are unit vectors in the plane or in three-dimensional space. The sample space is typically a circle or a sphere. There must be special directional methods which take into account the structure of the sample spaces. The union of graphical models and directional statistics allows the development of probabilistic models of biomolecular structures. Through the use of dynamic Bayesian networks with directional output it becomes possible to construct a joint probability distribution over sequence and structure. Biomolecular structures can be represented in a geometrically natural, continuous space. Mocapy++ is an open source toolkit for inference and learning using dynamic Bayesian networks that provides support for directional statistics. Mocapy++ is excellent for constructing probabilistic models of biomolecular structures; it has been used to develop models of protein and RNA structure in atomic detail. Mocapy++ is used in several high-impact publications, and will form the core of the molecular modeling package Phaistos, which will be released soon. The goal of this project is to develop a highly useful Python interface to Mocapy++, and to integrate that interface with the Biopython project. Through the Bio.PDB module, Biopython provides excellent functionality for data mining biomolecular structure databases. Integrating Mocapy++ and Biopython will allow training a probabilistic model using data extracted from a database. Integrating Mocapy++ with Biopython will create a powerful toolkit for researchers to quickly implement and test new ideas, try a variety of approaches and refine their methods. It will provide strong support for the field of biomolecular structure prediction, design, and simulation.
; Approach & Goals
: Mocapy++ is a machine learning toolkit for training and using Bayesian networks. It has been used to develop probabilistic models of biomolecular structures. The goal of this project is to develop a Python interface to Mocapy++ and integrate it with Biopython. This will allow the training of a probabilistic model using data extracted from a database. The integration of Mocapy++ with Biopython will provide a strong support for the field of protein structure prediction, design and simulation.
; Mentors
: [http://etal.myweb.uga.edu/ Eric Talevich]
: [http://wiki.binf.ku.dk/User:Thomas_Hamelryck Thomas Hamelryck]

==== [http://biopython.org/wiki/GSOC2011_MocapyExt MocapyExt] ====
; Student
: Justinas V. Daugmaudis
; Rationale
: BioPython is a very popular library in Bioinformatics and Computational Biology. Mocapy++ is a machine learning toolkit for parameter learning and inference in dynamic Bayesian networks (DBNs), which encode probabilistic relationships among random variables in a domain. Mocapy++ is freely available under the GNU General Public Licence (GPL) from SourceForge. The library supports a wide spectrum of DBN architectures and probability distributions, including distributions from directional statistics. Notably, Kent distribution on the sphere and the bivariate von Mises distribution on the torus, which have proven to be useful in formulating probabilistic models of protein and RNA structure. Such a highly useful and powerful library, which has been used in such projects as TorusDBN, Basilisk, FB5HMM with great success, is the result of the long-term effort. The original Mocapy implementation dates back to 2004, and since then the library has been rewritten in C++. However, C++ is a statically typed and compiled programming language, which does not facilitate rapid prototyping. As a result, currently Mocapy++ has no provisions for dynamic loading of custom node types, and a mechanism to plug-in new node types that would not require to modify and recompile the library is of interest. Such a plug-in interface would assist rapid prototyping by allowing to quickly implement and test new probability distributions, which, in turn, could substantially reduce development time and effort; the user would be empowered to extend Mocapy++ without modifications and subsequent recompilations. Recognizing this need, the project (herein referred as MocapyEXT), with the aim to improve the current Mocapy++ node type extension mechanism, has been proposed by T. Hamelryck.
; Approach & Goals
: The MocapyEXT project is largely an engineering effort to bring a transparent Python plug-in interface to Mocapy++, where built-in and dynamically loaded node types could be used in a uniform manner. Also, externally implemented and dynamically loaded nodes could be modified by a user and these changes will not necessitate the recompilation of the client program, nor the accompanying Mocapy++ library. This will facilitate rapid prototyping, ease the adaptation of currently existing code, and improve the software interoperability whilst introducing minimal changes to the existing Mocapy++ interface, thus facilitating a smooth acceptance of the changes introduced by MocapyEXT.
; Mentors
: [http://etal.myweb.uga.edu/ Eric Talevich]
: [http://wiki.binf.ku.dk/User:Thomas_Hamelryck Thomas Hamelryck]

=== 2010 ===
=== 2009 ===

GSOC

Joaor — Fri, 08 Mar 2013 22:44:18 GMT

Joaor: Added 2011 proposals

== Introduction ==

The Open Bioinformatics foundation successfully [http://www.open-bio.org/wiki/Google_Summer_of_Code applied to participate in the Google Summer of Code].

Please read the [http://www.open-bio.org/wiki/Google_Summer_of_Code GSoC page at the Open Bioinformatics Foundation] and the [http://code.google.com/soc main Google Summer of Code page] for more details about the program.

== Mentor List ==

Usually, each BioPython proposal has one or more mentors assigned to it. Nevertheless, we encourage potential students to contact the mailing list with their own
ideas for proposals. There is therefore not a set list of 'available' mentors, since it highly depends on which projects are proposed every year.

Past mentors include:

* [http://casbon.me/ James Casbon]
* [https://github.com/chapmanb Brad Chapman]
* [http://www.hutton.ac.uk/staff/peter-cock Peter Cock]
* [http://wiki.binf.ku.dk/User:Thomas_Hamelryck Thomas Hamelryck]
* [http://www.linkedin.com/in/reece Reece Hart]
* [http://nmr.chem.uu.nl/~joao João Rodrigues]
* [http://etal.myweb.uga.edu/ Eric Talevich]

== Proposals ==
=== 2013 ===

The BioPython proposals for 2013 will be published here once discussed. We encourage potential students to join the mailing lists and actively participate in these discussions, either by
submitting their own ideas or contributing to improving existing ones.

== Past Proposals ==

=== 2012 ===
==== [http://biopython.org/wiki/SearchIO SearchIO] ====
; Rationale
: Biopython has general APIs for parsing and writing assorted sequence file formats (SeqIO), multiple sequence alignments (AlignIO), phylogenetic trees (Phylo) and motifs (Bio.Motif). An obvious omission is something equivalent to BioPerl's SearchIO. The goal of this proposal is to develop an easy-to-use Python interface in the same style as SeqIO, AlignIO, etc but for pairwise search results. This would aim to cover EMBOSS muscle & water, BLAST XML, BLAST tabular, HMMER, Bill Pearson's FASTA alignments, and so on.
; Approach
: Much of the low level parsing code to handle these file formats already exists in Biopython, and much as the SeqIO and AlignIO modules are linked and share code, similar links apply to the proposed SearchIO module when using pairwise alignment file formats. However, SearchIO will also support pairwise search results where the pairwise sequence alignment itself is not available (e.g. the default BLAST tabular output). A crucial aspect of this work will be to design a pairwise-search-result object heirachy that reflects this, probably with a subclass inheriting from both the pairwise-search-result and the existing MultipleSequenceAlignment object. Beyond the initial challenge of an iterator based parsing and writing framework, random access akin to the Bio.SeqIO.index and index_db functionality would be most desirable for working with large datasets.
; Challenges
: The project will cover a range of important file formats from major Bioinformatics tools, thus will require familiarity with running these tools, and understanding their output and its meaning. Inter-converting file formats is part of this.
; Difficulty and needed skills
: Medium/Hard depending on how many objectives are attempted. The student needs to be fluent in Python and have knowledge of the BioPython codebase. Experience with all of the command line tools listed would be clear advantages, as would first hand experience using BioPerl's SearchIO. You will also need to know or learn the git version control system.
; Mentors
: [http://www.hutton.ac.uk/staff/peter-cock Peter Cock]

==== [http://arklenna.tumblr.com/tagged/gsoc2012 Representation and manipulation of genomic variants] ====
; Rationale
: Computational analysis of genomic variation requires the ability to reliably communicate and manipulate variants. The goal of this project is to provide facilities within BioPython to represent sequence variation objects, convert them to and from common human and file representations, and provide common manipulations on them.
; Approach & Goals
* Object representation
** identify variation types to be represented (SNV, CNV, repeats, inversions, etc)
** develop internal machine representation for variation types
** ensure coverage of essential standards, including HGVS, GFF, VCF
* External representations
** write parser and generators between objects and external string and file formats
* Manipulations
** canonicalize variations with more than one valid representation (e.g., ins versus dup and left shifting repeats).
** develop coordinate mapping between genomic, cDNA, and protein sequences (HGVS)
* Other
** release code to appropriate community efforts and write short manuscript
** implement web service for HGVS conversion
; Difficulty and needed skills
: Easy-to-Medium depending on how many objectives are attempted. The student will need have skills in most or all of: basic molecular biology (genomes, transcripts, proteins), genomic variation, Python, BioPython, Perl, BioPerl, NCBI Eutilities and/or Ensembl API. Experience with computer grammars is highly desirable. You will also need to know or learn the git version control system.
; Mentors
: [http://www.linkedin.com/in/reece Reece Hart]
: [https://github.com/chapmanb Brad Chapman]
: [http://casbon.me/ James Casbon]
=== 2011 ===
==== [http://biopython.org/wiki/GSoC2011_mtrellet Biomolecular Interface Analysis] ====
; Student
: Mikael Trellet
; Rationale
: Analysis of protein-protein complexes interfaces at a residue level yields significant information on the overall binding process. Such information can be broadly used for example in binding affinity studies, interface design, and enzymology. To tap into it, there is a need for tools that systematically and automatically analyze protein structures, or that provide means to this end. Protorop (http://www.bioinformatics.sussex.ac.uk/protorp/) is an example of such a tool and the elevated number of citations the server has had since its publication acknowledge its importance. However, being a webserver, Protorop is not suited for large-scale analysis and it leaves the community dependent on its maintainers to keep the service available. On the other hand, Biopython’s structural biology module, Bio.PDB, provides the ideal parsing machinery and programmatic structures for the development of an offline, open-source library for interface analysis. Such a library could be easily used in large-scale analysis of protein-protein interfaces, for example in the CAPRI experiment evaluation or in benchmark statistics. It would be also reasonable, if time permits, to extend this module to deal with protein-DNA or protein-RNA complexes, as Biopython supports nucleic acids already.
; Approach & Goals
* Add the new module backbone in current Bio.PDB code base
** Evaluate possible code reuse and call it into the new module
** Try simple calculations to be sure that there is stability between the different modules (parsing for example) and functions
* Define a stable benchmark
** Select few PDB files among interface size and proteins size would be different
* Extend IUPAC.Data module with residue information
** Deduce residues weight from Atom instead of direct dictionary storage
** Polar/charge character (dictionary or influenced by pH)
** Hydrophobicity scale(s)
* Implement Extended Residue class as a subclass of Residue
* Implement Interface object and InterfaceAnalysis module
* Develop functions for interface analysis
** Calculation of interface polar character statistics (% of polar residues, apolar, etc)
** Calculation of BSA calling MSMS or HSA
** Calculation of SS element statistics in the interface through DSSP
** Unit tests and use of results as input for further calculations by other tools and scripts
* Develop functions for Interface comparison
* Code organization and final testing

; Difficulty and needed skills
: Easy/Medium. Working knowledge of the Bio.PDB module of BioPython. Knowledge of structural biology in general and associated file formats (PDB).
; Mentors
: [http://nmr.chem.uu.nl/~joao João Rodrigues]
: [http://etal.myweb.uga.edu/ Eric Talevich]

==== [http://biopython.org/wiki/GSOC2011_Mocapy A Python bridge for Mocapy++] ====
; Student
: Michele Silva
; Rationale
: Discovering the structure of biomolecules is one of the biggest problems in biology. Given an amino acid or base sequence, what is the three dimensional structure? One approach to biomolecular structure prediction is the construction of probabilistic models. A Bayesian network is a probabilistic model composed of a set of variables and their joint probability distribution, represented as a directed acyclic graph. A dynamic Bayesian network is a Bayesian network that represents sequences of variables. These sequences can be time-series or sequences of symbols, such as protein sequences. Directional statistics is concerned mainly with observations which are unit vectors in the plane or in three-dimensional space. The sample space is typically a circle or a sphere. There must be special directional methods which take into account the structure of the sample spaces. The union of graphical models and directional statistics allows the development of probabilistic models of biomolecular structures. Through the use of dynamic Bayesian networks with directional output it becomes possible to construct a joint probability distribution over sequence and structure. Biomolecular structures can be represented in a geometrically natural, continuous space. Mocapy++ is an open source toolkit for inference and learning using dynamic Bayesian networks that provides support for directional statistics. Mocapy++ is excellent for constructing probabilistic models of biomolecular structures; it has been used to develop models of protein and RNA structure in atomic detail. Mocapy++ is used in several high-impact publications, and will form the core of the molecular modeling package Phaistos, which will be released soon. The goal of this project is to develop a highly useful Python interface to Mocapy++, and to integrate that interface with the Biopython project. Through the Bio.PDB module, Biopython provides excellent functionality for data mining biomolecular structure databases. Integrating Mocapy++ and Biopython will allow training a probabilistic model using data extracted from a database. Integrating Mocapy++ with Biopython will create a powerful toolkit for researchers to quickly implement and test new ideas, try a variety of approaches and refine their methods. It will provide strong support for the field of biomolecular structure prediction, design, and simulation.
; Approach & Goals
: Mocapy++ is a machine learning toolkit for training and using Bayesian networks. It has been used to develop probabilistic models of biomolecular structures. The goal of this project is to develop a Python interface to Mocapy++ and integrate it with Biopython. This will allow the training of a probabilistic model using data extracted from a database. The integration of Mocapy++ with Biopython will provide a strong support for the field of protein structure prediction, design and simulation.
; Mentors
: [http://etal.myweb.uga.edu/ Eric Talevich]
: [http://wiki.binf.ku.dk/User:Thomas_Hamelryck Thomas Hamelryck]

==== [http://biopython.org/wiki/GSOC2011_MocapyExt MocapyExt] ====
; Student
: Justinas V. Daugmaudis
; Rationale
: BioPython is a very popular library in Bioinformatics and Computational Biology. Mocapy++ is a machine learning toolkit for parameter learning and inference in dynamic Bayesian networks (DBNs), which encode probabilistic relationships among random variables in a domain. Mocapy++ is freely available under the GNU General Public Licence (GPL) from SourceForge. The library supports a wide spectrum of DBN architectures and probability distributions, including distributions from directional statistics. Notably, Kent distribution on the sphere and the bivariate von Mises distribution on the torus, which have proven to be useful in formulating probabilistic models of protein and RNA structure.
Such a highly useful and powerful library, which has been used in such projects as TorusDBN, Basilisk, FB5HMM with great success, is the result of the long-term effort. The original Mocapy implementation dates back to 2004, and since then the library has been rewritten in C++. However, C++ is a statically typed and compiled programming language, which does not facilitate rapid prototyping. As a result, currently Mocapy++ has no provisions for dynamic loading of custom node types, and a mechanism to plug-in new node types that would not require to modify and recompile the library is of interest. Such a plug-in interface would assist rapid prototyping by allowing to quickly implement and test new probability distributions, which, in turn, could substantially reduce development time and effort; the user would be empowered to extend Mocapy++ without modifications and subsequent recompilations. Recognizing this need, the project (herein referred as MocapyEXT), with the aim to improve the current Mocapy++ node type extension mechanism, has been proposed by T. Hamelryck.
; Approach & Goals
: The MocapyEXT project is largely an engineering effort to bring a transparent Python plug-in interface to Mocapy++, where built-in and dynamically loaded node types could be used in a uniform manner. Also, externally implemented and dynamically loaded nodes could be modified by a user and these changes will not necessitate the recompilation of the client program, nor the accompanying Mocapy++ library. This will facilitate rapid prototyping, ease the adaptation of currently existing code, and improve the software interoperability whilst introducing minimal changes to the existing Mocapy++ interface, thus facilitating a smooth acceptance of the changes introduced by MocapyEXT.
; Mentors
: [http://etal.myweb.uga.edu/ Eric Talevich]
: [http://wiki.binf.ku.dk/User:Thomas_Hamelryck Thomas Hamelryck]

=== 2010 ===
=== 2009 ===

GSOC

Joaor — Fri, 08 Mar 2013 22:17:59 GMT

Joaor: Reorganization. Added more mentors

GSOC

Joaor — Fri, 08 Mar 2013 22:13:02 GMT

Joaor: Page Creation

Participants

Joaor — Mon, 21 Nov 2011 11:02:42 GMT

Joaor: Added João's details

Note: People are listed here alphabetically by surname.
This is only a partial listing, see also the [http://biopython.org/SRC/biopython/CONTRIB contributor listing] in the Biopython source code.

= Tiago Antao =
{| border="0"
|-

| E-mail || [mailto:tiago@gmail.com tiagoantao@gmail.com]

|-

| Affiliation || Liverpool School of Tropical Medicine

|-

| Location || Liverpool, UK

|-

| Uses Python for || Almost all programming stuff

|-

| Work/Research Interests || Population genetics, Infectious diseases (malaria), domain specific languages (DSLs) and declarative programming, pharmacology

|-

| Biopython Contributions || Bio.PopGen

|-

| Relevant URL || http://tiago.org

|}

= Sebastian Bassi =
{| border="0"
|-

| E-mail || [mailto:sbassi@genesdigitales.com sbassi@genesdigitales.com]

|-

| Affiliation || Universidad Nacional de Quilmes

|-

| Location || Balcarce, Buenos Aires, Argentina

|-

| Uses Python for || Bioinformatics and data manipulation

|-

| Work/Research Interests || IT Manager Advanta Seeds in Balcarce Research Station

|-

| Biopython Contributions || LCC and primer Tm calculation function

|-

| Relevant URL || http://www.bioinformatica.info

|}

= Jeffrey Chang =
{| border="0"
|-

| E-mail || [mailto:jchang@smi.stanford.edu jchang@smi.stanford.edu]

|-

| Affiliation || Postdoctoral Fellow, Duke University

|-

| Location || Durham, NC

|-

| Uses Python for || Eating spam

|-

| Work/Research Interests || Bioinformatics

|-

| Biopython Contributions || Co-Founder

|-

| Relevant URL || http://www.jeffchang.com/

|}

= Brad Chapman =
{| border="0"
|-

| Affiliation || Massachusetts General Hospital

|-

| Location || Boston, MA

|-

| Biopython Contributions || Docs, GenBank, [[BioSQL]]

|-

| Relevant URL || http://bcbio.wordpress.com/

|}

= [[User:Peter|Peter Cock]] =
{| border="0"
|-

| E-mail || See my web page

|-

| Affiliation || The James Hutton Institute (formerly SCRI); previously MOAC Doctoral Training Centre, University of Warwick

|-

| Location || Dundee, Scotland, UK

|-

| Uses Python for || Bioinformatics, controlling R with rpy, ...

|-

| Work/Research Interests || Bacterial signalling, genomics, sequencings

|-

| Biopython Contributions || Sequence parsing including [[SeqIO|Bio.SeqIO]], [[AlignIO|Bio.AlignIO]], maintaining the [[BioSQL|BioSQL interface]], and documentation

|-

| Relevant URL || http://www.hutton.ac.uk/staff/peter-cock and http://www.warwick.ac.uk/go/peter_cock/python/

|-

| github || http://github.com/peterjc

|}

= Andrew Dalke =
{| border="0"
|-

| E-mail || [mailto:dalke@dalkescientific.com dalke@dalkescientific.com]

|-

| Affiliation || Dalke Scientific Software, LLC

|-

| Location || Santa Fe, NM

|-

| Uses Python for || Just about anything

|-

| Work/Research Interests || Large-scale usable systems for scientists

|-

| Biopython Contributions || Co-Founder, Seq, Martel, indexing, EUtils, patterns, parsing, ...

|-

| Relevant URL || http://www.dalkescientific.com/

|}

= [[User:Mdehoon|Michiel de Hoon]] =
{| border="0"
|-

| E-mail || See my web page

|-

| Affiliation || RIKEN Omics Science Center

|-

| Location || Yokohama

|-

| Uses Python for || High-throughput data analysis & Scientific visualization

|-

| Work/Research Interests || RNA Genomics

|-

| Biopython Contributions || Bio.Cluster; Bio.Entrez; Windows installer

|-

| Relevant URL || http://bonsai.ims.u-tokyo.ac.jp/~mdehoon

|}

= Iddo Friedberg =
{| border="0"
|-

| E-mail || idoerg "at" gmail.com

|-

| Affiliation || University of California San Diego

|-

| Location || La-Jolla, CA, USA

|-

| Uses Python for || Maintaining World Domination

|-

| Work/Research Interests || Structural Bioinformatics, metagenomics

|-

| Biopython Contributions || SubsMat, FSSP, bits of Align, bits of the Manual, and a lot of silly questions to the lists

|-

| Relevant URL || http://iddo-friedberg.org

|}

= Christian Gunning =
{| border="0"
|-

| E-mail || bioboy at uga dot edu

|-

| Affiliation || human, mountain

|-

| Location || Athens, GA

|-

| Uses Python for || strings, as glue; also on laundry and dirty dishes

|-

| Work/Research Interests || arabidopsis; Biological Sequence Analysis, Durbin et al.; Primer3; www.swig.org; R programming language and rpy.sourceforge.net

|-

| Biopython Contributions ||

|-

| Relevant URL || http://www.botany.uga.edu/courses/bioinformatics/current/index.html

|}

= Thomas Hamelryck =
{| border="0"
|-

| E-mail || thamelry - binf ku dk

|-

| Affiliation || University of Copenhagen

|-

| Location || Copenhagen, Denmark

|-

| Uses Python for || Annoying FORTRAN programmers

|-

| Work/Research Interests || Structural bioinformatics

|-

| Biopython Contributions || Bio.PDB, KDTree, SVDSuperimposer

|-

| Relevant URL || http://www.binf.ku.dk/users/thamelry

|}

= Michael Hoffman =
{| border="0"
|-

| E-mail || grouse at alumni period utexas period net

|-

| Affiliation || The University of Texas at Austin

|-

| Location || Austin, TX, USA

|-

| Uses Python for || Biopython!

|-

| Work/Research Interests || RNA, Genome annotations

|-

| Biopython Contributions || Bio.GFF, Bio.DocSQL

|-

| Relevant URL || http://spice.cc.utexas.edu/~grouse/

|}

= Frank Kauff =
{| border="0"
|-

| E-mail || fkauff at biologie uni-kl de

|-

| Affiliation || University of Kaiserslautern

|-

| Location || Kaiserslautern, Germany

|-

| Uses Python for || Phylogenetics and everything else

|-

| Work/Research Interests || Phylogenetics and all that's related, Fungi, Lichens, Cyanobacteria

|-

| Biopython Contributions || Phd, Ace, Nexus (mostly with C. Cox)

|-

| Relevant URL || http://www.uni-kl.de/wcms/ag-kauff.html

|}

= [[User:lpritc|Leighton Pritchard]] =
{| border="0"
|-

| E-mail || lpritc(squiggly symbol)scri ac uk

|-

| Affiliation || [http://www.scri.ac.uk/ Scottish Crop Research Institute]

|-

| Location || Invergowrie, Scotland

|-

| Uses Python for || Generally when I have to explain to a computer exactly what I want it to do

|-

| Work/Research Interests || Comparative genomics; Systems Biology; Protein sequence-structure-function relationships; Plant host-pathogen interactions and genomics (heavy on the pathogens).

|-

| Biopython Contributions || GenomeDiagram, bits and bobs

|-

| Relevant URL || http://www.scri.ac.uk/staff/leightonpritchard

|}

= João Rodrigues =
{| border="0"
|-

| E-mail || [mailto:anaryin@gmail.com anaryin@gmail.com]

|-

| Affiliation || Bijvoet Center for Biomolecular Research, Utrecht University

|-

| Location || Utrecht, NL

|-

| Uses Python for || Pretty much all my (programming) tasks

|-

| Work/Research Interests || Structural Biology, Biophysics, Molecular Simulations, Protein Docking, Homology Modelling, etc..

|-

| Biopython Contributions || Bio.PDB (here and there)

|-

| Relevant URL || http://nmr.chem.uu.nl/~joaor

|}

= Bartek Wilczyński =
{| border="0"
|-

| E-mail || bartek_AT_rezolwenta.eu.org

|-

| Affiliation || Institute of Mathematics, Polish Academy of Science

|-

| Location || Warsaw, Poland

|-

| Uses Python for || most of his computations

|-

| Work/Research Interests || mathematical models of gene regulation

|-

| Biopython Contributions || Bio.AlignAce

|-

| Relevant URL || http://bartek.rezolwenta.eu.org

|}

= Harry Zuzan =
{| border="0"
|-

| E-mail || [mailto:iliketobicycle@yahoo.ca iliketobicycle@yahoo.ca]

|-

| Affiliation || Genome Quebec

|-

| Location || Montreal

|-

| Uses Python for || you name it

|-

| Work/Research Interests || Statistics applied to molecular biology and genetics

|-

| Biopython Contributions || Affy package for Affymetrix data

|-

| Relevant URL || http://www.oligopython.org

|}

GSoC2011 mtrellet

Joaor — Wed, 08 Jun 2011 08:36:21 GMT

Joaor:

== Author & Mentors ==

[[User:Mtrellet|Mikael Trellet]] mikael.trellet@gmail.com

'''Mentors'''
: João Rodrigues
: Eric Talevich

== Abstract ==

Analysis of protein-protein complexes interfaces at a residue level yields significant information on the overall binding process. Such information can be broadly used for example in binding affinity studies, interface design, and enzymology. To tap into it, there is a need for tools that systematically and automatically analyze protein structures, or that provide means to this end. Protorop (http://www.bioinformatics.sussex.ac.uk/protorp/) is an example of such a tool and the elevated number of citations the server has had since its publication acknowledge its importance. However, being a webserver, Protorop is not suited for large-scale analysis and it leaves the community dependent on its maintainers to keep the service available. On the other hand, Biopython’s structural biology module, Bio.PDB, provides the ideal parsing machinery and programmatic structures for the development of an offline, open-source library for interface analysis. Such a library could be easily used in large-scale analysis of protein-protein interfaces, for example in the CAPRI experiment evaluation or in benchmark statistics. It would be also reasonable, if time permits, to extend this module to deal with protein-DNA or protein-RNA complexes, as Biopython supports nucleic acids already.

== Project Schedule ==

=== Week 1 [23rd May - 31st June] ===

==== Add the new module backbone in current Bio.PDB code base ====

*Evaluate possible code reuse and call it into the new module
*Try simple calculations to be sure that there is stability between the different modules (parsing for example) and functions

==== Define a stable benchmark ====

*Select few PDB files among interface size and proteins size would be different
*Add some basics unit tests

=== Weeks 2-3 [1st - 13th June] ===

==== Extend IUPAC.Data module with residue information ====

* Deduce residues weight from Atom instead of direct dictionnary storage
* Polar/charge character (dictionary or influenced by pH)
* Hydrophobicity scale(s)
** [http://www3.interscience.wiley.com/journal/112117957/abstract pKa values algorithm]

=== Weeks 4 [14th - 21st June] ===

==== Implement Extended Residue class as a subclass of Residue ====

* Build Extended Residue on the fly or have it hard-coded (?)
* Allow regular operations on Residue to be performed seamlessly in Extended Residue (should come with inheritance)
* Unit tests on pdb files containing particular residues

=== Weeks 5-6-7 [22nd June - 11th July] ===

==== Implement InterfaceAnalysis module ====

* Develop Interface class as a subclass of Model
* Develop method to automatically extract Interface from parsed structure upon class instantiation
** e.g. I = Interface(Structure)
** Allow threshold for distance
** Allow chain pairs to ignore (to avoid intra-molecule contacts)
* Unit tests with results from usual scripts, broadly used by scientists

=== Mid term evaluation ===

=== Weeks 7-8 [12th July - 25th July] ===

==== Develop functions for interface analysis ====

* Calculation of interface polar character statistics (% of polar residues, apolar, etc)
* Calculation of BSA calling MSMS or HSA
* Calculation of SS element statistics in the interface through DSSP
* ...
* Unit tests and use of results as input for further calculations by other tools and scripts

=== Weeks 9-10 [26th July - 8th August] ===

==== Develop functions for Interface comparison ====

* Perhaps adapt current RMSD functions to allow usage of Interface Residues
* Otherwise, should be called through something like Ia.rmsd_to(Ib) where Ia and IB are interface objects
* Calculation of iRMSD
* Calculation of FCC (Fraction of Common Contacts)
* Rough Identity and Similarity percentage
* ...
* Unit tests, comparison with specific tools as Profit

=== Weeks 11 [9th July - 8th August] ===

==== Code organization and final testing ====

== Unit tests ==

Unit tests will be perfomed along the project, allowing to do only a larger test at the end gathering every tests already performed.

Then the aim will be to optimized, if possible, some parts of the code in efficiency and rapidity without changes at algorithmic level. Several days will be booked to package code and be sure that everything can communicate with Biopython.

----

== Project Progress ==

=== Implementation of Interface object backbone ===

* Theory

We began to think of an easy way to add the Interface as a new part of the SMCRA scheme. The idea was to have this new scheme = SM-I-CRA. Unfortunately the Interface object is not as well defined as just a child of model and a parent of chains. Indeed, the main part of the interface is residues, and even residues pairs. We want to keep the information of the chain but we can't keep them as they are defined actually, since we will get some overlaps, duplication and miscompatibility between the chains of our model and the chains of our interface. In the same way, our try to link the creation of the interface with existing modules as StructureBuilder and Model wasn't successful.
So, we decided to simplify a bit the concept in adding the classes related to the Interface in an independent way. Obviously links will exist between the different levels of SMCRA but Interface would be considered now as a parallel entity, not integrated completely in the SMCRA scheme.

* Coding

Interface.py is the definition of the Interface object inherited from Entity with the following methods : '''__init__'''(self, id), '''add'''(self, entity) and '''get_chains'''(self).

The add module overrides the add method of Entity in order to have an easy way to class residues according to their respective chains.
The get_chains modules returns the chains involved in the interface defined by the Interface object.

The second class created is InterfaceBuilder.py which deals directly with the interface building (hard to guess..!)
We find these different modules : '''__init__'''(self, model, id=None, threshold=5.0, include_waters=False, *chains), '''_unpack_chains'''(self, list_of_tuples), '''get_interface'''(self), '''_add_residue'''(self, residue), '''_build_interface'''(self, model, id, threshold, include_waters=False, *chains)

'''__init__''' : In order to initialize an interface you need to provide the model for which you want to calculate the interface, that's the only mandatory argument.

'''_unpack_chains''': Method used by __init__ so as to create self.chain_list, variable read in many parts of the class. It transforms a list of tuples (given by the user) in a list of characters representing the chains which will be involved in the definition of the interface.

'''get_interface''': Returns simply the interface

'''_add_residue''': Allows the user to add some specific residues to his interface

'''_build_interface''': The machinery to build the interface, it uses NeighborSearch and Selection in order to define the interface depending on the arguments given by the user.

* Github repository

[https://github.com/mtrellet/biopython/commit/4cfa4359d0f927609c076ed7b66f37add5aabdfb Interface.py]
[https://github.com/mtrellet/biopython/commit/194efe37ac8f88d688e0cf528f1fb896c8441866 InterfaceBuilder.py]

GSoC2011 mtrellet

Joaor — Wed, 08 Jun 2011 08:35:56 GMT

Joaor:

== Author & Mentors ==

[[User:Mtrellet|Mikael Trellet]] mikael.trellet@gmail.com

'''Mentors'''
: João Rodrigues
: Eric Talevich

== Abstract ==

Analysis of protein-protein complexes interfaces at a residue level yields significant information on the overall binding process. Such information can be broadly used for example in binding affinity studies, interface design, and enzymology. To tap into it, there is a need for tools that systematically and automatically analyze protein structures, or that provide means to this end. Protorop (http://www.bioinformatics.sussex.ac.uk/protorp/) is an example of such a tool and the elevated number of citations the server has had since its publication acknowledge its importance. However, being a webserver, Protorop is not suited for large-scale analysis and it leaves the community dependent on its maintainers to keep the service available. On the other hand, Biopython’s structural biology module, Bio.PDB, provides the ideal parsing machinery and programmatic structures for the development of an offline, open-source library for interface analysis. Such a library could be easily used in large-scale analysis of protein-protein interfaces, for example in the CAPRI experiment evaluation or in benchmark statistics. It would be also reasonable, if time permits, to extend this module to deal with protein-DNA or protein-RNA complexes, as Biopython supports nucleic acids already.

== Project Schedule ==

=== Week 1 [23rd May - 31st June] ===

==== Add the new module backbone in current Bio.PDB code base ====

*Evaluate possible code reuse and call it into the new module
*Try simple calculations to be sure that there is stability between the different modules (parsing for example) and functions

==== Define a stable benchmark ====

*Select few PDB files among interface size and proteins size would be different
*Add some basics unit tests

=== Weeks 2-3 [1st - 13th June] ===

==== Extend IUPAC.Data module with residue information ====

* Deduce residues weight from Atom instead of direct dictionnary storage
* Polar/charge character (dictionary or influenced by pH)
* Hydrophobicity scale(s)
** [http://www3.interscience.wiley.com/journal/112117957/abstract pKa values algorithm]

=== Weeks 4 [14th - 21st June] ===

==== Implement Extended Residue class as a subclass of Residue ====

* Build Extended Residue on the fly or have it hard-coded (?)
* Allow regular operations on Residue to be performed seamlessly in Extended Residue (should come with inheritance)
* Unit tests on pdb files containing particular residues

=== Weeks 5-6-7 [22nd June - 11th July] ===

==== Implement InterfaceAnalysis module ====

* Develop Interface class as a subclass of Model
* Develop method to automatically extract Interface from parsed structure upon class instantiation
** e.g. I = Interface(Structure)
** Allow threshold for distance
** Allow chain pairs to ignore (to avoid intra-molecule contacts)
* Unit tests with results from usual scripts, broadly used by scientists

=== Mid term evaluation ===

=== Weeks 7-8 [12th July - 25th July] ===

==== Develop functions for interface analysis ====

* Calculation of interface polar character statistics (% of polar residues, apolar, etc)
* Calculation of BSA calling MSMS or HSA
* Calculation of SS element statistics in the interface through DSSP
* ...
* Unit tests and use of results as input for further calculations by other tools and scripts

=== Weeks 9-10 [26th July - 8th August] ===

==== Develop functions for Interface comparison ====

* Perhaps adapt current RMSD functions to allow usage of Interface Residues
* Otherwise, should be called through something like Ia.rmsd_to(Ib) where Ia and IB are interface objects
* Calculation of iRMSD
* Calculation of FCC (Fraction of Common Contacts)
* Rough Identity and Similarity percentage
* ...
* Unit tests, comparison with specific tools as Profit

=== Weeks 11 [9th July - 8th August] ===

==== Code organization and final testing ====

== Unit tests ==

Unit tests will be perfomed along the project, allowing to do only a larger test at the end gathering every tests already performed.

Then the aim will be to optimized, if possible, some parts of the code in efficiency and rapidity without changes at algorithmic level. Several days will be booked to package code and be sure that everything can communicate with Biopython.

----

== Project Progress ==

=== Implementation of Interface object backbone ===

* Theory

We began to think of an easy way to add the Interface as a new part of the SMCRA scheme. The idea was to have this new scheme = SM-I-CRA. Unfortunately the Interface object is not as well defined as just a child of model and a parent of chains. Indeed, the main part of the interface is residues, and even residues pairs. We want to keep the information of the chain but we can't keep them as they are defined actually, since we will get some overlaps, duplication and miscompatibility between the chains of our model and the chains of our interface. In the same way, our try to link the creation of the interface with existing modules as StructureBuilder and Model wasn't successful.
So, we decided to simplify a bit the concept in adding the classes related to the Interface in an independent way. Obviously links will exist between the different levels of SMCRA but Interface would be considered now as a parallel entity, not integrated completely in the SMCRA scheme.

* Coding

Interface.py is the definition of the Interface object inherited from Entity with the following methods : '''__init__'''(self, id), '''add'''(self, entity) and '''get_chains'''(self).

The add module overrides the add method of Entity in order to have an easy way to class residues according to their respective chains.
The get_chains modules returns the chains involved in the interface defined by the Interface object.

The second class created is InterfaceBuilder.py which deals directly with the interface building (hard to guess..!)
We find these different modules : '''__init__'''(self, model, id=None, threshold=5.0, include_waters=False, *chains), '''_unpack_chains'''(self, list_of_tuples), '''get_interface'''(self), '''_add_residue'''(self, residue), '''_build_interface'''(self, model, id, threshold, include_waters=False, *chains)

'''__init__''' : In order to initialize an interface you need to provide the model for which you want to calculate the interface, that's the only mandatory argument.

'''_unpack_chains''': Method used by __init__ so as to create self.chain_list, variable read in many parts of the class. It transforms a list of tuples (given by the user) in a list of characters representing the chains which will be involved in the definition of the interface.

'''get_interface''': Returns simply the interface

'''_add_residue''': Allows the user to add some specific residues to his interface

'''_build_interface''': The machinery to build the interface, it uses NeighborSearch and Selection in order to define the interface depending on the arguments given by the user.

* Github repository

[https://github.com/mtrellet/biopython/commit/194efe37ac8f88d688e0cf528f1fb896c8441866 InterfaceBuilder.py]

PDBParser

Joaor — Mon, 16 May 2011 13:46:40 GMT

Joaor: /* CATH Dataset */ Corrected statistics for new lengths.

This is a draft page for the PDBParser class.

= Benchmark =

A performance benchmark of the parser was carried out to evaluate wether the development of new features degraded the overall parsing speed.

== Datasets ==

[http://release.cathdb.info/v3.4.0/CathDomainList CATH Domain Collection] - 11330 Structures containing only coordinate information (no Element assigned).

[ftp://ftp.wwpdb.org/pub/pdb/data/structures/divided/pdb/ Protein Data Bank Collection] - 72836 Structures containing both headers and coordinate information.

== Versions Tested ==

[http://biopython.org/DIST/biopython-1.49.zip Biopython 1.49] (Nov. 2008)

[https://github.com/biopython/biopython Biopython 1.57+] (May 2011) | Element column auto-assignment.

[https://github.com/JoaoRodrigues/biopython/tree/pdb_enhancements Biopython PDB branch] (May 2011 @ Github) | Warnings module replaced _handle_pdb_exception() && Other minor changes

== Benchmarking Script ==

The following script was used to benchmark the parser. The garbage collection module - gc - was necessary to avoid dead objects still in memory causing the machine to start swapping.

<python>
#!/usr/bin/env python
""" Script to Benchmark Bio.PDB PDBParser """

import sys, os, warnings

# Parsing Function
def parse_structure(path):
""" Parses a PDB file """

s = P.get_structure('test', path)

return 0

def fancy_output(tps):
""" Outputs the results in a nicer way """

print "# Bio.PDB PDBParser Benchmark"
print
print "Structure \tLenght \tTime Spent (ms)"
for i,s in enumerate(tps):
print " %s\t (%s) \t%3.3f" %(os.path.basename(pdb_library[i]), pdb_length[i], s)
print
print "Total time spent: %5.3fs" %(sum(tps)/1000)
print "Average time per structure: %5.3fms" %(sum(tps)/len(tps))

if __name__=='__main__':

import time, gc
from Bio.PDB import PDBParser
P = PDBParser(PERMISSIVE=1) # For the pdb_enhancements branch benchmarking, PERMISSIVE was set to 2 (silence warnings).

library_path = sys.argv[1]

pdb_library = [os.path.join(library_path, f) for f in os.listdir(library_path)]
pdb_length = [len(set([l[17:26] for l in open(f) if l.startswith('ATOM')])) for f in pdb_library] # Unique counting of residues
sys.stderr.write("Loaded %s structures (Average Length: %4.3f residues)\n" %(len(pdb_length), (sum(pdb_length)/float(len(pdb_length)))))

tps = []
# Run the Test
for i, pdb_file in enumerate(pdb_library):
sys.stderr.write( "[%s] %i Structure(s) Parsed \n" %(os.path.basename(pdb_file), i+1) )
a = time.time()
parse_structure(pdb_file)
b = time.time()-a
tps.append(b*1000)
gc.collect()
# Output Results
fancy_output(tps)
</python>

== Results ==

=== CATH Dataset ===

Average Structure Length: 146 residues

==== Biopython 1.49 ====

Total Time Spent: 530.686s
Average Time per Structure: 46.84ms/structure
Average Structures per Second: 21.38 structures/s
Failed to parse 0 structures due to errors.

Length N. Structures Average Time Spent ms
< 100 3660 25.09
100 =< x < 200 5296 44.67
200 =< x < 500 2330 83.40
500 =< x < 1000 43 177.10
>= 1000 1 320.10

TOTAL 11330 46.84

[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_149.time Link to full results]
[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_149.png Plot of the full results]

==== Biopython 1.57+ ====

Total Time Spent: 686.176s
Average Time per Structure: 60.56ms/structure
Average Structures per Second: 16.51 structures/s
Failed to parse 0 structures due to errors.

Length N. Structures Average Time Spent ms
< 100 3660 32.55
100 =< x < 200 5296 57.75
200 =< x < 500 2330 107.54
500 =< x < 1000 43 236.62
>= 1000 1 486.602

TOTAL 11330 60.56

[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_current.time Link to full results]
[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_current.png Plot of the full results]

==== Biopython PDB Branch ====

Total Time Spent: 695.405s
Average Time per Structure: 61.37ms/structure
Average Structures per Second: 16.29 structures/s
Failed to parse 0 structures due to errors.

Length N. Structures Average Time Spent ms
< 100 3660 33.21
100 =< x < 200 5296 58.45
200 =< x < 500 2330 108.76
500 =< x < 1000 43 234.37
>= 1000 1 797.583

TOTAL 11330 61.38

[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_pdb_enhancements.time Link to full results]
[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_pdb_enhancements.png Plot of the full results]

=== PDB Dataset ===

Average Structure Length: 589 residues

Failed to parse 2 structures due to errors:
1. 3NH3 (negative occupancy) - fix: http://bit.ly/ks6PDN (still being discussed)
2. 2WMW (invalid ANISOU field) - fix: http://bit.ly/ld9BWs

==== Biopython 1.49 ====

Total Time Spent: 27801.934s (7.72h)
Average Time per Structure: 381.706ms/structure
Average Structures per Second: 2.62 structures/s

Length N. Structures Average Time Spent (ms)
< 100 8402 410.270
100 =< x < 200 12226 461.744
200 =< x < 500 26493 182.027
500 =< x < 1000 15878 290.693
>= 1000 9837 942.513

TOTAL 72836 381.706

[http://nmr.chem.uu.nl/~joao/f/benchmark_PDB-biopython_1.49.time Link to full results]

==== Biopython 1.57+ ====

Total Time Spent: 29516.480s (~8.20h)
Average Time per Structure: 405.246 ms/structure
Average Structures per Second: 2.47 structures/s

Length N. Structures Average Time Spent (ms)
< 100 8402 451.819
100 =< x < 200 12226 505.933
200 =< x < 500 26493 190.991
500 =< x < 1000 15878 304.453
>= 1000 9837 980.047

TOTAL 72836 405.246

[http://nmr.chem.uu.nl/~joao/f/benchmark_PDB-biopython_1.57.time Link to full results]

==== Biopython PDB Branch (In progress) ====

In progress

PDBParser

Joaor — Mon, 16 May 2011 13:20:40 GMT

Joaor: /* Biopython 1.57+ (In progress) */

This is a draft page for the PDBParser class.

= Benchmark =

A performance benchmark of the parser was carried out to evaluate wether the development of new features degraded the overall parsing speed.

== Datasets ==

[http://release.cathdb.info/v3.4.0/CathDomainList CATH Domain Collection] - 11330 Structures containing only coordinate information (no Element assigned).

[ftp://ftp.wwpdb.org/pub/pdb/data/structures/divided/pdb/ Protein Data Bank Collection] - 72836 Structures containing both headers and coordinate information.

== Versions Tested ==

[http://biopython.org/DIST/biopython-1.49.zip Biopython 1.49] (Nov. 2008)

[https://github.com/biopython/biopython Biopython 1.57+] (May 2011) | Element column auto-assignment.

[https://github.com/JoaoRodrigues/biopython/tree/pdb_enhancements Biopython PDB branch] (May 2011 @ Github) | Warnings module replaced _handle_pdb_exception() && Other minor changes

== Benchmarking Script ==

The following script was used to benchmark the parser. The garbage collection module - gc - was necessary to avoid dead objects still in memory causing the machine to start swapping.

<python>
#!/usr/bin/env python
""" Script to Benchmark Bio.PDB PDBParser """

import sys, os, warnings

# Parsing Function
def parse_structure(path):
""" Parses a PDB file """

s = P.get_structure('test', path)

return 0

def fancy_output(tps):
""" Outputs the results in a nicer way """

print "# Bio.PDB PDBParser Benchmark"
print
print "Structure \tLenght \tTime Spent (ms)"
for i,s in enumerate(tps):
print " %s\t (%s) \t%3.3f" %(os.path.basename(pdb_library[i]), pdb_length[i], s)
print
print "Total time spent: %5.3fs" %(sum(tps)/1000)
print "Average time per structure: %5.3fms" %(sum(tps)/len(tps))

if __name__=='__main__':

import time, gc
from Bio.PDB import PDBParser
P = PDBParser(PERMISSIVE=1) # For the pdb_enhancements branch benchmarking, PERMISSIVE was set to 2 (silence warnings).

library_path = sys.argv[1]

pdb_library = [os.path.join(library_path, f) for f in os.listdir(library_path)]
pdb_length = [len(set([l[17:26] for l in open(f) if l.startswith('ATOM')])) for f in pdb_library] # Unique counting of residues
sys.stderr.write("Loaded %s structures (Average Length: %4.3f residues)\n" %(len(pdb_length), (sum(pdb_length)/float(len(pdb_length)))))

tps = []
# Run the Test
for i, pdb_file in enumerate(pdb_library):
sys.stderr.write( "[%s] %i Structure(s) Parsed \n" %(os.path.basename(pdb_file), i+1) )
a = time.time()
parse_structure(pdb_file)
b = time.time()-a
tps.append(b*1000)
gc.collect()
# Output Results
fancy_output(tps)
</python>

== Results ==

=== CATH Dataset ===

Average Structure Length: 147 residues

==== Biopython 1.49 ====

Total Time Spent: 530.686s
Average Time per Structure: 46.84ms/structure
Average Structures per Second: 21.38 structures/s
Failed to parse 0 structures due to errors.

Length N. Structures Average Time Spent (ms)
< 100 3663 25.11
100 =< x < 200 5295 44.68
200 =< x < 500 2328 83.41
500 =< x < 1000 44 180.35

TOTAL 11330 46.84

[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_149.time Link to full results]
[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_149.png Plot of the full results]

==== Biopython 1.57+ ====

Total Time Spent: 686.176s
Average Time per Structure: 60.56ms/structure
Average Structures per Second: 16.51 structures/s
Failed to parse 0 structures due to errors.

Length N. Structures Average Time Spent (ms)
< 100 3663 32.57
100 =< x < 200 5295 57.76
200 =< x < 500 2328 107.56
500 =< x < 1000 44 242.30

TOTAL 11330 60.56

[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_current.time Link to full results]
[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_current.png Plot of the full results]

==== Biopython PDB Branch ====

Total Time Spent: 695.405s
Average Time per Structure: 61.37ms/structure
Average Structures per Second: 16.29 structures/s
Failed to parse 0 structures due to errors.

Length N. Structures Average Time Spent (ms)
< 100 3663 33.24
100 =< x < 200 5295 58.46
200 =< x < 500 2328 108.77
500 =< x < 1000 44 247.171

TOTAL 11330 61.38

[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_pdb_enhancements.time Link to full results]
[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_pdb_enhancements.png Plot of the full results]

=== PDB Dataset ===

Average Structure Length: 589 residues

Failed to parse 2 structures due to errors:
1. 3NH3 (negative occupancy) - fix: http://bit.ly/ks6PDN (still being discussed)
2. 2WMW (invalid ANISOU field) - fix: http://bit.ly/ld9BWs

==== Biopython 1.49 ====

Total Time Spent: 27801.934s (7.72h)
Average Time per Structure: 381.706ms/structure
Average Structures per Second: 2.62 structures/s

Length N. Structures Average Time Spent (ms)
< 100 8402 410.270
100 =< x < 200 12226 461.744
200 =< x < 500 26493 182.027
500 =< x < 1000 15878 290.693
>= 1000 9837 942.513

TOTAL 72836 381.706

[http://nmr.chem.uu.nl/~joao/f/benchmark_PDB-biopython_1.49.time Link to full results]

==== Biopython 1.57+ ====

Total Time Spent: 29516.480s (~8.20h)
Average Time per Structure: 405.246 ms/structure
Average Structures per Second: 2.47 structures/s

Length N. Structures Average Time Spent (ms)
< 100 8402 451.819
100 =< x < 200 12226 505.933
200 =< x < 500 26493 190.991
500 =< x < 1000 15878 304.453
>= 1000 9837 980.047

TOTAL 72836 405.246

[http://nmr.chem.uu.nl/~joao/f/benchmark_PDB-biopython_1.57.time Link to full results]

==== Biopython PDB Branch (In progress) ====

In progress

PDBParser

Joaor — Mon, 16 May 2011 13:19:14 GMT

Joaor: /* PDB Dataset */ Added 1.57 results. Minor corrections

This is a draft page for the PDBParser class.

= Benchmark =

A performance benchmark of the parser was carried out to evaluate wether the development of new features degraded the overall parsing speed.

== Datasets ==

[http://release.cathdb.info/v3.4.0/CathDomainList CATH Domain Collection] - 11330 Structures containing only coordinate information (no Element assigned).

[ftp://ftp.wwpdb.org/pub/pdb/data/structures/divided/pdb/ Protein Data Bank Collection] - 72836 Structures containing both headers and coordinate information.

== Versions Tested ==

[http://biopython.org/DIST/biopython-1.49.zip Biopython 1.49] (Nov. 2008)

[https://github.com/biopython/biopython Biopython 1.57+] (May 2011) | Element column auto-assignment.

[https://github.com/JoaoRodrigues/biopython/tree/pdb_enhancements Biopython PDB branch] (May 2011 @ Github) | Warnings module replaced _handle_pdb_exception() && Other minor changes

== Benchmarking Script ==

The following script was used to benchmark the parser. The garbage collection module - gc - was necessary to avoid dead objects still in memory causing the machine to start swapping.

<python>
#!/usr/bin/env python
""" Script to Benchmark Bio.PDB PDBParser """

import sys, os, warnings

# Parsing Function
def parse_structure(path):
""" Parses a PDB file """

s = P.get_structure('test', path)

return 0

def fancy_output(tps):
""" Outputs the results in a nicer way """

print "# Bio.PDB PDBParser Benchmark"
print
print "Structure \tLenght \tTime Spent (ms)"
for i,s in enumerate(tps):
print " %s\t (%s) \t%3.3f" %(os.path.basename(pdb_library[i]), pdb_length[i], s)
print
print "Total time spent: %5.3fs" %(sum(tps)/1000)
print "Average time per structure: %5.3fms" %(sum(tps)/len(tps))

if __name__=='__main__':

import time, gc
from Bio.PDB import PDBParser
P = PDBParser(PERMISSIVE=1) # For the pdb_enhancements branch benchmarking, PERMISSIVE was set to 2 (silence warnings).

library_path = sys.argv[1]

pdb_library = [os.path.join(library_path, f) for f in os.listdir(library_path)]
pdb_length = [len(set([l[17:26] for l in open(f) if l.startswith('ATOM')])) for f in pdb_library] # Unique counting of residues
sys.stderr.write("Loaded %s structures (Average Length: %4.3f residues)\n" %(len(pdb_length), (sum(pdb_length)/float(len(pdb_length)))))

tps = []
# Run the Test
for i, pdb_file in enumerate(pdb_library):
sys.stderr.write( "[%s] %i Structure(s) Parsed \n" %(os.path.basename(pdb_file), i+1) )
a = time.time()
parse_structure(pdb_file)
b = time.time()-a
tps.append(b*1000)
gc.collect()
# Output Results
fancy_output(tps)
</python>

== Results ==

=== CATH Dataset ===

Average Structure Length: 147 residues

==== Biopython 1.49 ====

Total Time Spent: 530.686s
Average Time per Structure: 46.84ms/structure
Average Structures per Second: 21.38 structures/s
Failed to parse 0 structures due to errors.

Length N. Structures Average Time Spent (ms)
< 100 3663 25.11
100 =< x < 200 5295 44.68
200 =< x < 500 2328 83.41
500 =< x < 1000 44 180.35

TOTAL 11330 46.84

[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_149.time Link to full results]
[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_149.png Plot of the full results]

==== Biopython 1.57+ ====

Total Time Spent: 686.176s
Average Time per Structure: 60.56ms/structure
Average Structures per Second: 16.51 structures/s
Failed to parse 0 structures due to errors.

Length N. Structures Average Time Spent (ms)
< 100 3663 32.57
100 =< x < 200 5295 57.76
200 =< x < 500 2328 107.56
500 =< x < 1000 44 242.30

TOTAL 11330 60.56

[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_current.time Link to full results]
[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_current.png Plot of the full results]

==== Biopython PDB Branch ====

Total Time Spent: 695.405s
Average Time per Structure: 61.37ms/structure
Average Structures per Second: 16.29 structures/s
Failed to parse 0 structures due to errors.

Length N. Structures Average Time Spent (ms)
< 100 3663 33.24
100 =< x < 200 5295 58.46
200 =< x < 500 2328 108.77
500 =< x < 1000 44 247.171

TOTAL 11330 61.38

[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_pdb_enhancements.time Link to full results]
[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_pdb_enhancements.png Plot of the full results]

=== PDB Dataset ===

Average Structure Length: 589 residues

Failed to parse 2 structures due to errors:
1. 3NH3 (negative occupancy) - fix: http://bit.ly/ks6PDN (still being discussed)
2. 2WMW (invalid ANISOU field) - fix: http://bit.ly/ld9BWs

==== Biopython 1.49 ====

Total Time Spent: 27801.934s (7.72h)
Average Time per Structure: 381.706ms/structure
Average Structures per Second: 2.62 structures/s

Length N. Structures Average Time Spent (ms)
< 100 8402 410.270
100 =< x < 200 12226 461.744
200 =< x < 500 26493 182.027
500 =< x < 1000 15878 290.693
>= 1000 9837 942.513

TOTAL 72836 381.706

[http://nmr.chem.uu.nl/~joao/f/benchmark_PDB-biopython_1.49.time Link to full results]

==== Biopython 1.57+ (In progress) ====

Total Time Spent: 29516.480s (~8.20h)
Average Time per Structure: 405.246 ms/structure
Average Structures per Second: 2.47 structures/s

Length N. Structures Average Time Spent (ms)
< 100 8402 451.819
100 =< x < 200 12226 505.933
200 =< x < 500 26493 190.991
500 =< x < 1000 15878 304.453
>= 1000 9837 980.047

TOTAL 72836 405.246

[http://nmr.chem.uu.nl/~joao/f/benchmark_PDB-biopython_1.57.time Link to full results]

==== Biopython PDB Branch (In progress) ====

In progress

PDBParser

Joaor — Mon, 16 May 2011 13:17:12 GMT

Joaor: /* Biopython 1.49 */ Removed link to PNG (too much noise) and corrected statistics according to new lengths.

This is a draft page for the PDBParser class.

= Benchmark =

A performance benchmark of the parser was carried out to evaluate wether the development of new features degraded the overall parsing speed.

== Datasets ==

[http://release.cathdb.info/v3.4.0/CathDomainList CATH Domain Collection] - 11330 Structures containing only coordinate information (no Element assigned).

[ftp://ftp.wwpdb.org/pub/pdb/data/structures/divided/pdb/ Protein Data Bank Collection] - 72836 Structures containing both headers and coordinate information.

== Versions Tested ==

[http://biopython.org/DIST/biopython-1.49.zip Biopython 1.49] (Nov. 2008)

[https://github.com/biopython/biopython Biopython 1.57+] (May 2011) | Element column auto-assignment.

[https://github.com/JoaoRodrigues/biopython/tree/pdb_enhancements Biopython PDB branch] (May 2011 @ Github) | Warnings module replaced _handle_pdb_exception() && Other minor changes

== Benchmarking Script ==

The following script was used to benchmark the parser. The garbage collection module - gc - was necessary to avoid dead objects still in memory causing the machine to start swapping.

<python>
#!/usr/bin/env python
""" Script to Benchmark Bio.PDB PDBParser """

import sys, os, warnings

# Parsing Function
def parse_structure(path):
""" Parses a PDB file """

s = P.get_structure('test', path)

return 0

def fancy_output(tps):
""" Outputs the results in a nicer way """

print "# Bio.PDB PDBParser Benchmark"
print
print "Structure \tLenght \tTime Spent (ms)"
for i,s in enumerate(tps):
print " %s\t (%s) \t%3.3f" %(os.path.basename(pdb_library[i]), pdb_length[i], s)
print
print "Total time spent: %5.3fs" %(sum(tps)/1000)
print "Average time per structure: %5.3fms" %(sum(tps)/len(tps))

if __name__=='__main__':

import time, gc
from Bio.PDB import PDBParser
P = PDBParser(PERMISSIVE=1) # For the pdb_enhancements branch benchmarking, PERMISSIVE was set to 2 (silence warnings).

library_path = sys.argv[1]

pdb_library = [os.path.join(library_path, f) for f in os.listdir(library_path)]
pdb_length = [len(set([l[17:26] for l in open(f) if l.startswith('ATOM')])) for f in pdb_library] # Unique counting of residues
sys.stderr.write("Loaded %s structures (Average Length: %4.3f residues)\n" %(len(pdb_length), (sum(pdb_length)/float(len(pdb_length)))))

tps = []
# Run the Test
for i, pdb_file in enumerate(pdb_library):
sys.stderr.write( "[%s] %i Structure(s) Parsed \n" %(os.path.basename(pdb_file), i+1) )
a = time.time()
parse_structure(pdb_file)
b = time.time()-a
tps.append(b*1000)
gc.collect()
# Output Results
fancy_output(tps)
</python>

== Results ==

=== CATH Dataset ===

Average Structure Length: 147 residues

==== Biopython 1.49 ====

Total Time Spent: 530.686s
Average Time per Structure: 46.84ms/structure
Average Structures per Second: 21.38 structures/s
Failed to parse 0 structures due to errors.

Length N. Structures Average Time Spent (ms)
< 100 3663 25.11
100 =< x < 200 5295 44.68
200 =< x < 500 2328 83.41
500 =< x < 1000 44 180.35

TOTAL 11330 46.84

[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_149.time Link to full results]
[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_149.png Plot of the full results]

==== Biopython 1.57+ ====

Total Time Spent: 686.176s
Average Time per Structure: 60.56ms/structure
Average Structures per Second: 16.51 structures/s
Failed to parse 0 structures due to errors.

Length N. Structures Average Time Spent (ms)
< 100 3663 32.57
100 =< x < 200 5295 57.76
200 =< x < 500 2328 107.56
500 =< x < 1000 44 242.30

TOTAL 11330 60.56

[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_current.time Link to full results]
[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_current.png Plot of the full results]

==== Biopython PDB Branch ====

Total Time Spent: 695.405s
Average Time per Structure: 61.37ms/structure
Average Structures per Second: 16.29 structures/s
Failed to parse 0 structures due to errors.

Length N. Structures Average Time Spent (ms)
< 100 3663 33.24
100 =< x < 200 5295 58.46
200 =< x < 500 2328 108.77
500 =< x < 1000 44 247.171

TOTAL 11330 61.38

[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_pdb_enhancements.time Link to full results]
[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_pdb_enhancements.png Plot of the full results]

=== PDB Dataset ===

Average Structure Length: 263 residues

==== Biopython 1.49 ====

Total Time Spent: 27801.934s (7.72h)
Average Time per Structure: 381.706ms/structure
Average Structures per Second: 2.62 structures/s
Failed to parse 2 structures due to errors:
1. 3NH3 (negative occupancy) - fix: http://bit.ly/ks6PDN
2. 2WMW (invalid ANISOU field) - fix: http://bit.ly/ld9BWs

Length N. Structures Average Time Spent (ms)
< 100 8402 410.270
100 =< x < 200 12226 461.744
200 =< x < 500 26493 182.027
500 =< x < 1000 15878 290.693
>= 1000 9837 942.513

TOTAL 72836 381.706

[http://nmr.chem.uu.nl/~joao/f/benchmark_PDB-biopython1.49.time Link to full results]

==== Biopython 1.57+ (In progress) ====

In progress

==== Biopython PDB Branch (In progress) ====

In progress

PDBParser

Joaor — Mon, 16 May 2011 13:15:34 GMT

Joaor: /* Benchmarking Script */ Corrected residue length calculation line..

This is a draft page for the PDBParser class.

= Benchmark =

A performance benchmark of the parser was carried out to evaluate wether the development of new features degraded the overall parsing speed.

== Datasets ==

[http://release.cathdb.info/v3.4.0/CathDomainList CATH Domain Collection] - 11330 Structures containing only coordinate information (no Element assigned).

[ftp://ftp.wwpdb.org/pub/pdb/data/structures/divided/pdb/ Protein Data Bank Collection] - 72836 Structures containing both headers and coordinate information.

== Versions Tested ==

[http://biopython.org/DIST/biopython-1.49.zip Biopython 1.49] (Nov. 2008)

[https://github.com/biopython/biopython Biopython 1.57+] (May 2011) | Element column auto-assignment.

[https://github.com/JoaoRodrigues/biopython/tree/pdb_enhancements Biopython PDB branch] (May 2011 @ Github) | Warnings module replaced _handle_pdb_exception() && Other minor changes

== Benchmarking Script ==

The following script was used to benchmark the parser. The garbage collection module - gc - was necessary to avoid dead objects still in memory causing the machine to start swapping.

<python>
#!/usr/bin/env python
""" Script to Benchmark Bio.PDB PDBParser """

import sys, os, warnings

# Parsing Function
def parse_structure(path):
""" Parses a PDB file """

s = P.get_structure('test', path)

return 0

def fancy_output(tps):
""" Outputs the results in a nicer way """

print "# Bio.PDB PDBParser Benchmark"
print
print "Structure \tLenght \tTime Spent (ms)"
for i,s in enumerate(tps):
print " %s\t (%s) \t%3.3f" %(os.path.basename(pdb_library[i]), pdb_length[i], s)
print
print "Total time spent: %5.3fs" %(sum(tps)/1000)
print "Average time per structure: %5.3fms" %(sum(tps)/len(tps))

if __name__=='__main__':

import time, gc
from Bio.PDB import PDBParser
P = PDBParser(PERMISSIVE=1) # For the pdb_enhancements branch benchmarking, PERMISSIVE was set to 2 (silence warnings).

library_path = sys.argv[1]

pdb_library = [os.path.join(library_path, f) for f in os.listdir(library_path)]
pdb_length = [len(set([l[17:26] for l in open(f) if l.startswith('ATOM')])) for f in pdb_library] # Unique counting of residues
sys.stderr.write("Loaded %s structures (Average Length: %4.3f residues)\n" %(len(pdb_length), (sum(pdb_length)/float(len(pdb_length)))))

tps = []
# Run the Test
for i, pdb_file in enumerate(pdb_library):
sys.stderr.write( "[%s] %i Structure(s) Parsed \n" %(os.path.basename(pdb_file), i+1) )
a = time.time()
parse_structure(pdb_file)
b = time.time()-a
tps.append(b*1000)
gc.collect()
# Output Results
fancy_output(tps)
</python>

== Results ==

=== CATH Dataset ===

Average Structure Length: 147 residues

==== Biopython 1.49 ====

Total Time Spent: 530.686s
Average Time per Structure: 46.84ms/structure
Average Structures per Second: 21.38 structures/s
Failed to parse 0 structures due to errors.

Length N. Structures Average Time Spent (ms)
< 100 3663 25.11
100 =< x < 200 5295 44.68
200 =< x < 500 2328 83.41
500 =< x < 1000 44 180.35

TOTAL 11330 46.84

[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_149.time Link to full results]
[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_149.png Plot of the full results]

==== Biopython 1.57+ ====

Total Time Spent: 686.176s
Average Time per Structure: 60.56ms/structure
Average Structures per Second: 16.51 structures/s
Failed to parse 0 structures due to errors.

Length N. Structures Average Time Spent (ms)
< 100 3663 32.57
100 =< x < 200 5295 57.76
200 =< x < 500 2328 107.56
500 =< x < 1000 44 242.30

TOTAL 11330 60.56

[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_current.time Link to full results]
[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_current.png Plot of the full results]

==== Biopython PDB Branch ====

Total Time Spent: 695.405s
Average Time per Structure: 61.37ms/structure
Average Structures per Second: 16.29 structures/s
Failed to parse 0 structures due to errors.

Length N. Structures Average Time Spent (ms)
< 100 3663 33.24
100 =< x < 200 5295 58.46
200 =< x < 500 2328 108.77
500 =< x < 1000 44 247.171

TOTAL 11330 61.38

[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_pdb_enhancements.time Link to full results]
[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_pdb_enhancements.png Plot of the full results]

=== PDB Dataset ===

Average Structure Length: 263 residues

==== Biopython 1.49 ====

Total Time Spent: 27801.934s (7.72h)
Average Time per Structure: 381.706ms/structure
Average Structures per Second: 2.62 structures/s
Failed to parse 2 structures due to errors:
1. 3NH3 (negative occupancy) - fix: http://bit.ly/ks6PDN
2. 2WMW (invalid ANISOU field) - fix: http://bit.ly/ld9BWs

Length N. Structures Average Time Spent (ms)
< 100 10986 392.227
100 =< x < 200 19641 371.716
200 =< x < 500 35299 301.699
500 =< x < 1000 6291 597.472
>= 1000 619 2881.520

TOTAL 72836 381.706

[http://nmr.chem.uu.nl/~joao/f/benchmark_PDB-biopython1.49.time Link to full results]
[http://nmr.chem.uu.nl/~joao/f/benchmark_PDB-biopython1.49.png Plot of the full results]

==== Biopython 1.57+ (In progress) ====

In progress

==== Biopython PDB Branch (In progress) ====

In progress

PDBParser

Joaor — Thu, 12 May 2011 18:07:38 GMT

Joaor: /* Biopython 1.49 */

This is a draft page for the PDBParser class.

= Benchmark =

A performance benchmark of the parser was carried out to evaluate wether the development of new features degraded the overall parsing speed.

== Datasets ==

[http://release.cathdb.info/v3.4.0/CathDomainList CATH Domain Collection] - 11330 Structures containing only coordinate information (no Element assigned).

[ftp://ftp.wwpdb.org/pub/pdb/data/structures/divided/pdb/ Protein Data Bank Collection] - 72836 Structures containing both headers and coordinate information.

== Versions Tested ==

[http://biopython.org/DIST/biopython-1.49.zip Biopython 1.49] (Nov. 2008)

[https://github.com/biopython/biopython Biopython 1.57+] (May 2011) | Element column auto-assignment.

[https://github.com/JoaoRodrigues/biopython/tree/pdb_enhancements Biopython PDB branch] (May 2011 @ Github) | Warnings module replaced _handle_pdb_exception() && Other minor changes

== Benchmarking Script ==

The following script was used to benchmark the parser. The garbage collection module - gc - was necessary to avoid dead objects still in memory causing the machine to start swapping.

<python>
#!/usr/bin/env python
""" Script to Benchmark Bio.PDB PDBParser """

import sys, os, warnings

# Parsing Function
def parse_structure(path):
""" Parses a PDB file """

s = P.get_structure('test', path)

return 0

def fancy_output(tps):
""" Outputs the results in a nicer way """

print "# Bio.PDB PDBParser Benchmark"
print
print "Structure \tLenght \tTime Spent (ms)"
for i,s in enumerate(tps):
print " %s\t (%s) \t%3.3f" %(os.path.basename(pdb_library[i]), pdb_length[i], s)
print
print "Total time spent: %5.3fs" %(sum(tps)/1000)
print "Average time per structure: %5.3fms" %(sum(tps)/len(tps))

if __name__=='__main__':

import time, gc
from Bio.PDB import PDBParser
P = PDBParser(PERMISSIVE=1) # For the pdb_enhancements branch benchmarking, PERMISSIVE was set to 2 (silence warnings).

library_path = sys.argv[1]

pdb_library = [os.path.join(library_path, f) for f in os.listdir(library_path)]
pdb_length = [len(set([int(l[23:26]) for l in open(f) if l.startswith('ATOM')])) for f in pdb_library] # Unique counting of residues
sys.stderr.write("Loaded %s structures (Average Length: %4.3f residues)\n" %(len(pdb_length), (sum(pdb_length)/float(len(pdb_length)))))

tps = []
# Run the Test
for i, pdb_file in enumerate(pdb_library):
sys.stderr.write( "[%s] %i Structure(s) Parsed \n" %(os.path.basename(pdb_file), i+1) )
a = time.time()
parse_structure(pdb_file)
b = time.time()-a
tps.append(b*1000)
gc.collect()
# Output Results
fancy_output(tps)
</python>

== Results ==

=== CATH Dataset ===

Average Structure Length: 147 residues

==== Biopython 1.49 ====

Total Time Spent: 530.686s
Average Time per Structure: 46.84ms/structure
Average Structures per Second: 21.38 structures/s
Failed to parse 0 structures due to errors.

Length N. Structures Average Time Spent (ms)
< 100 3663 25.11
100 =< x < 200 5295 44.68
200 =< x < 500 2328 83.41
500 =< x < 1000 44 180.35

TOTAL 11330 46.84

[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_149.time Link to full results]
[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_149.png Plot of the full results]

==== Biopython 1.57+ ====

Total Time Spent: 686.176s
Average Time per Structure: 60.56ms/structure
Average Structures per Second: 16.51 structures/s
Failed to parse 0 structures due to errors.

Length N. Structures Average Time Spent (ms)
< 100 3663 32.57
100 =< x < 200 5295 57.76
200 =< x < 500 2328 107.56
500 =< x < 1000 44 242.30

TOTAL 11330 60.56

[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_current.time Link to full results]
[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_current.png Plot of the full results]

==== Biopython PDB Branch ====

Total Time Spent: 695.405s
Average Time per Structure: 61.37ms/structure
Average Structures per Second: 16.29 structures/s
Failed to parse 0 structures due to errors.

Length N. Structures Average Time Spent (ms)
< 100 3663 33.24
100 =< x < 200 5295 58.46
200 =< x < 500 2328 108.77
500 =< x < 1000 44 247.171

TOTAL 11330 61.38

[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_pdb_enhancements.time Link to full results]
[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_pdb_enhancements.png Plot of the full results]

=== PDB Dataset ===

Average Structure Length: 263 residues

==== Biopython 1.49 ====

Total Time Spent: 27801.934s (7.72h)
Average Time per Structure: 381.706ms/structure
Average Structures per Second: 2.62 structures/s
Failed to parse 2 structures due to errors:
1. 3NH3 (negative occupancy) - fix: http://bit.ly/ks6PDN
2. 2WMW (invalid ANISOU field) - fix: http://bit.ly/ld9BWs

Length N. Structures Average Time Spent (ms)
< 100 10986 392.227
100 =< x < 200 19641 371.716
200 =< x < 500 35299 301.699
500 =< x < 1000 6291 597.472
>= 1000 619 2881.520

TOTAL 72836 381.706

[http://nmr.chem.uu.nl/~joao/f/benchmark_PDB-biopython1.49.time Link to full results]
[http://nmr.chem.uu.nl/~joao/f/benchmark_PDB-biopython1.49.png Plot of the full results]

==== Biopython 1.57+ (In progress) ====

In progress

==== Biopython PDB Branch (In progress) ====

In progress

PDBParser

Joaor — Thu, 12 May 2011 18:07:15 GMT

Joaor: /* Biopython 1.57+ */

This is a draft page for the PDBParser class.

= Benchmark =

A performance benchmark of the parser was carried out to evaluate wether the development of new features degraded the overall parsing speed.

== Datasets ==

[http://release.cathdb.info/v3.4.0/CathDomainList CATH Domain Collection] - 11330 Structures containing only coordinate information (no Element assigned).

[ftp://ftp.wwpdb.org/pub/pdb/data/structures/divided/pdb/ Protein Data Bank Collection] - 72836 Structures containing both headers and coordinate information.

== Versions Tested ==

[http://biopython.org/DIST/biopython-1.49.zip Biopython 1.49] (Nov. 2008)

[https://github.com/biopython/biopython Biopython 1.57+] (May 2011) | Element column auto-assignment.

[https://github.com/JoaoRodrigues/biopython/tree/pdb_enhancements Biopython PDB branch] (May 2011 @ Github) | Warnings module replaced _handle_pdb_exception() && Other minor changes

== Benchmarking Script ==

The following script was used to benchmark the parser. The garbage collection module - gc - was necessary to avoid dead objects still in memory causing the machine to start swapping.

<python>
#!/usr/bin/env python
""" Script to Benchmark Bio.PDB PDBParser """

import sys, os, warnings

# Parsing Function
def parse_structure(path):
""" Parses a PDB file """

s = P.get_structure('test', path)

return 0

def fancy_output(tps):
""" Outputs the results in a nicer way """

print "# Bio.PDB PDBParser Benchmark"
print
print "Structure \tLenght \tTime Spent (ms)"
for i,s in enumerate(tps):
print " %s\t (%s) \t%3.3f" %(os.path.basename(pdb_library[i]), pdb_length[i], s)
print
print "Total time spent: %5.3fs" %(sum(tps)/1000)
print "Average time per structure: %5.3fms" %(sum(tps)/len(tps))

if __name__=='__main__':

import time, gc
from Bio.PDB import PDBParser
P = PDBParser(PERMISSIVE=1) # For the pdb_enhancements branch benchmarking, PERMISSIVE was set to 2 (silence warnings).

library_path = sys.argv[1]

pdb_library = [os.path.join(library_path, f) for f in os.listdir(library_path)]
pdb_length = [len(set([int(l[23:26]) for l in open(f) if l.startswith('ATOM')])) for f in pdb_library] # Unique counting of residues
sys.stderr.write("Loaded %s structures (Average Length: %4.3f residues)\n" %(len(pdb_length), (sum(pdb_length)/float(len(pdb_length)))))

tps = []
# Run the Test
for i, pdb_file in enumerate(pdb_library):
sys.stderr.write( "[%s] %i Structure(s) Parsed \n" %(os.path.basename(pdb_file), i+1) )
a = time.time()
parse_structure(pdb_file)
b = time.time()-a
tps.append(b*1000)
gc.collect()
# Output Results
fancy_output(tps)
</python>

== Results ==

=== CATH Dataset ===

Average Structure Length: 147 residues

==== Biopython 1.49 ====

Total Time Spent: 530.686s
Average Time per Structure: 46.84ms/structure
Average Structures per Second: 21.38 structures/s
Failed to parse 0 structures due to errors.

Length N. Structures Average Time Spent (ms)
< 100 3663 25.11
100 =< x < 200 5295 44.68
200 =< x < 500 2328 83.41
500 =< x < 1000 44 180.35

TOTAL 11330 46.84

[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_149.time Link to full results]
[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_149.png Plot of the full results]

==== Biopython 1.57+ ====

Total Time Spent: 686.176s
Average Time per Structure: 60.56ms/structure
Average Structures per Second: 16.51 structures/s
Failed to parse 0 structures due to errors.

Length N. Structures Average Time Spent (ms)
< 100 3663 32.57
100 =< x < 200 5295 57.76
200 =< x < 500 2328 107.56
500 =< x < 1000 44 242.30

TOTAL 11330 60.56

[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_current.time Link to full results]
[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_current.png Plot of the full results]

==== Biopython PDB Branch ====

Total Time Spent: 695.405s
Average Time per Structure: 61.37ms/structure
Average Structures per Second: 16.29 structures/s
Failed to parse 0 structures due to errors.

Length N. Structures Average Time Spent (ms)
< 100 3663 33.24
100 =< x < 200 5295 58.46
200 =< x < 500 2328 108.77
500 =< x < 1000 44 247.171

TOTAL 11330 61.38

[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_pdb_enhancements.time Link to full results]
[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_pdb_enhancements.png Plot of the full results]

=== PDB Dataset ===

Average Structure Length: 263 residues

==== Biopython 1.49 ====

Total Time Spent: 27801.934s (7.72h)
Average Time per Structure: 381.706ms/structure
Average Structures per Second: 2.62 structures/s
Failed to parse 2 structures due to errors:
1. 3NH3 (negative occupancy) - fix: http://bit.ly/ks6PDN
2. 2WMW (invalid ANISOU field) - fix: http://bit.ly/ld9BWs

Length N. Structures Average Time Spent (ms)
< 100 10986 392.227
100 =< x < 200 19641 371.716
200 =< x < 500 35299 301.699
500 =< x < 1000 6291 597.472
>= 1000 619 2881.520

TOTAL 72836 381.706

[http://nmr.chem.uu.nl/~joao/f/benchmark_PDB-biopython1.49.time Link to full results]

==== Biopython 1.57+ (In progress) ====

In progress

==== Biopython PDB Branch (In progress) ====

In progress

PDBParser

Joaor — Thu, 12 May 2011 18:06:42 GMT

Joaor: /* Biopython PDB Branch */

This is a draft page for the PDBParser class.

= Benchmark =

A performance benchmark of the parser was carried out to evaluate wether the development of new features degraded the overall parsing speed.

== Datasets ==

[http://release.cathdb.info/v3.4.0/CathDomainList CATH Domain Collection] - 11330 Structures containing only coordinate information (no Element assigned).

[ftp://ftp.wwpdb.org/pub/pdb/data/structures/divided/pdb/ Protein Data Bank Collection] - 72836 Structures containing both headers and coordinate information.

== Versions Tested ==

[http://biopython.org/DIST/biopython-1.49.zip Biopython 1.49] (Nov. 2008)

[https://github.com/biopython/biopython Biopython 1.57+] (May 2011) | Element column auto-assignment.

[https://github.com/JoaoRodrigues/biopython/tree/pdb_enhancements Biopython PDB branch] (May 2011 @ Github) | Warnings module replaced _handle_pdb_exception() && Other minor changes

== Benchmarking Script ==

The following script was used to benchmark the parser. The garbage collection module - gc - was necessary to avoid dead objects still in memory causing the machine to start swapping.

<python>
#!/usr/bin/env python
""" Script to Benchmark Bio.PDB PDBParser """

import sys, os, warnings

# Parsing Function
def parse_structure(path):
""" Parses a PDB file """

s = P.get_structure('test', path)

return 0

def fancy_output(tps):
""" Outputs the results in a nicer way """

print "# Bio.PDB PDBParser Benchmark"
print
print "Structure \tLenght \tTime Spent (ms)"
for i,s in enumerate(tps):
print " %s\t (%s) \t%3.3f" %(os.path.basename(pdb_library[i]), pdb_length[i], s)
print
print "Total time spent: %5.3fs" %(sum(tps)/1000)
print "Average time per structure: %5.3fms" %(sum(tps)/len(tps))

if __name__=='__main__':

import time, gc
from Bio.PDB import PDBParser
P = PDBParser(PERMISSIVE=1) # For the pdb_enhancements branch benchmarking, PERMISSIVE was set to 2 (silence warnings).

library_path = sys.argv[1]

pdb_library = [os.path.join(library_path, f) for f in os.listdir(library_path)]
pdb_length = [len(set([int(l[23:26]) for l in open(f) if l.startswith('ATOM')])) for f in pdb_library] # Unique counting of residues
sys.stderr.write("Loaded %s structures (Average Length: %4.3f residues)\n" %(len(pdb_length), (sum(pdb_length)/float(len(pdb_length)))))

tps = []
# Run the Test
for i, pdb_file in enumerate(pdb_library):
sys.stderr.write( "[%s] %i Structure(s) Parsed \n" %(os.path.basename(pdb_file), i+1) )
a = time.time()
parse_structure(pdb_file)
b = time.time()-a
tps.append(b*1000)
gc.collect()
# Output Results
fancy_output(tps)
</python>

== Results ==

=== CATH Dataset ===

Average Structure Length: 147 residues

==== Biopython 1.49 ====

Total Time Spent: 530.686s
Average Time per Structure: 46.84ms/structure
Average Structures per Second: 21.38 structures/s
Failed to parse 0 structures due to errors.

Length N. Structures Average Time Spent (ms)
< 100 3663 25.11
100 =< x < 200 5295 44.68
200 =< x < 500 2328 83.41
500 =< x < 1000 44 180.35

TOTAL 11330 46.84

[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_149.time Link to full results]
[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_149.png Plot of the full results]

==== Biopython 1.57+ ====

Total Time Spent: 686.176s
Average Time per Structure: 60.56ms/structure
Average Structures per Second: 16.51 structures/s
Failed to parse 0 structures due to errors.

Length N. Structures Average Time Spent (ms)
< 100 3663 32.57
100 =< x < 200 5295 57.76
200 =< x < 500 2328 107.56
500 =< x < 1000 44 242.30

TOTAL 11330 60.56

[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_current.time Link to full results]

==== Biopython PDB Branch ====

Total Time Spent: 695.405s
Average Time per Structure: 61.37ms/structure
Average Structures per Second: 16.29 structures/s
Failed to parse 0 structures due to errors.

Length N. Structures Average Time Spent (ms)
< 100 3663 33.24
100 =< x < 200 5295 58.46
200 =< x < 500 2328 108.77
500 =< x < 1000 44 247.171

TOTAL 11330 61.38

[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_pdb_enhancements.time Link to full results]
[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_pdb_enhancements.png Plot of the full results]

=== PDB Dataset ===

Average Structure Length: 263 residues

==== Biopython 1.49 ====

Total Time Spent: 27801.934s (7.72h)
Average Time per Structure: 381.706ms/structure
Average Structures per Second: 2.62 structures/s
Failed to parse 2 structures due to errors:
1. 3NH3 (negative occupancy) - fix: http://bit.ly/ks6PDN
2. 2WMW (invalid ANISOU field) - fix: http://bit.ly/ld9BWs

Length N. Structures Average Time Spent (ms)
< 100 10986 392.227
100 =< x < 200 19641 371.716
200 =< x < 500 35299 301.699
500 =< x < 1000 6291 597.472
>= 1000 619 2881.520

TOTAL 72836 381.706

[http://nmr.chem.uu.nl/~joao/f/benchmark_PDB-biopython1.49.time Link to full results]

==== Biopython 1.57+ (In progress) ====

In progress

==== Biopython PDB Branch (In progress) ====

In progress

PDBParser

Joaor — Thu, 12 May 2011 18:06:14 GMT

Joaor: /* Biopython 1.49 */

This is a draft page for the PDBParser class.

= Benchmark =

A performance benchmark of the parser was carried out to evaluate wether the development of new features degraded the overall parsing speed.

== Datasets ==

[http://release.cathdb.info/v3.4.0/CathDomainList CATH Domain Collection] - 11330 Structures containing only coordinate information (no Element assigned).

[ftp://ftp.wwpdb.org/pub/pdb/data/structures/divided/pdb/ Protein Data Bank Collection] - 72836 Structures containing both headers and coordinate information.

== Versions Tested ==

[http://biopython.org/DIST/biopython-1.49.zip Biopython 1.49] (Nov. 2008)

[https://github.com/biopython/biopython Biopython 1.57+] (May 2011) | Element column auto-assignment.

[https://github.com/JoaoRodrigues/biopython/tree/pdb_enhancements Biopython PDB branch] (May 2011 @ Github) | Warnings module replaced _handle_pdb_exception() && Other minor changes

== Benchmarking Script ==

The following script was used to benchmark the parser. The garbage collection module - gc - was necessary to avoid dead objects still in memory causing the machine to start swapping.

<python>
#!/usr/bin/env python
""" Script to Benchmark Bio.PDB PDBParser """

import sys, os, warnings

# Parsing Function
def parse_structure(path):
""" Parses a PDB file """

s = P.get_structure('test', path)

return 0

def fancy_output(tps):
""" Outputs the results in a nicer way """

print "# Bio.PDB PDBParser Benchmark"
print
print "Structure \tLenght \tTime Spent (ms)"
for i,s in enumerate(tps):
print " %s\t (%s) \t%3.3f" %(os.path.basename(pdb_library[i]), pdb_length[i], s)
print
print "Total time spent: %5.3fs" %(sum(tps)/1000)
print "Average time per structure: %5.3fms" %(sum(tps)/len(tps))

if __name__=='__main__':

import time, gc
from Bio.PDB import PDBParser
P = PDBParser(PERMISSIVE=1) # For the pdb_enhancements branch benchmarking, PERMISSIVE was set to 2 (silence warnings).

library_path = sys.argv[1]

pdb_library = [os.path.join(library_path, f) for f in os.listdir(library_path)]
pdb_length = [len(set([int(l[23:26]) for l in open(f) if l.startswith('ATOM')])) for f in pdb_library] # Unique counting of residues
sys.stderr.write("Loaded %s structures (Average Length: %4.3f residues)\n" %(len(pdb_length), (sum(pdb_length)/float(len(pdb_length)))))

tps = []
# Run the Test
for i, pdb_file in enumerate(pdb_library):
sys.stderr.write( "[%s] %i Structure(s) Parsed \n" %(os.path.basename(pdb_file), i+1) )
a = time.time()
parse_structure(pdb_file)
b = time.time()-a
tps.append(b*1000)
gc.collect()
# Output Results
fancy_output(tps)
</python>

== Results ==

=== CATH Dataset ===

Average Structure Length: 147 residues

==== Biopython 1.49 ====

Total Time Spent: 530.686s
Average Time per Structure: 46.84ms/structure
Average Structures per Second: 21.38 structures/s
Failed to parse 0 structures due to errors.

Length N. Structures Average Time Spent (ms)
< 100 3663 25.11
100 =< x < 200 5295 44.68
200 =< x < 500 2328 83.41
500 =< x < 1000 44 180.35

TOTAL 11330 46.84

[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_149.time Link to full results]
[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_149.png Plot of the full results]

==== Biopython 1.57+ ====

Total Time Spent: 686.176s
Average Time per Structure: 60.56ms/structure
Average Structures per Second: 16.51 structures/s
Failed to parse 0 structures due to errors.

Length N. Structures Average Time Spent (ms)
< 100 3663 32.57
100 =< x < 200 5295 57.76
200 =< x < 500 2328 107.56
500 =< x < 1000 44 242.30

TOTAL 11330 60.56

[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_current.time Link to full results]

==== Biopython PDB Branch ====

Total Time Spent: 695.405s
Average Time per Structure: 61.37ms/structure
Average Structures per Second: 16.29 structures/s
Failed to parse 0 structures due to errors.

Length N. Structures Average Time Spent (ms)
< 100 3663 33.24
100 =< x < 200 5295 58.46
200 =< x < 500 2328 108.77
500 =< x < 1000 44 247.171

TOTAL 11330 61.38

[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_pdb_enhancements.time Link to full results]

=== PDB Dataset ===

Average Structure Length: 263 residues

==== Biopython 1.49 ====

Total Time Spent: 27801.934s (7.72h)
Average Time per Structure: 381.706ms/structure
Average Structures per Second: 2.62 structures/s
Failed to parse 2 structures due to errors:
1. 3NH3 (negative occupancy) - fix: http://bit.ly/ks6PDN
2. 2WMW (invalid ANISOU field) - fix: http://bit.ly/ld9BWs

Length N. Structures Average Time Spent (ms)
< 100 10986 392.227
100 =< x < 200 19641 371.716
200 =< x < 500 35299 301.699
500 =< x < 1000 6291 597.472
>= 1000 619 2881.520

TOTAL 72836 381.706

[http://nmr.chem.uu.nl/~joao/f/benchmark_PDB-biopython1.49.time Link to full results]

==== Biopython 1.57+ (In progress) ====

In progress

==== Biopython PDB Branch (In progress) ====

In progress

PDBParser

Joaor — Thu, 12 May 2011 14:04:46 GMT

Joaor:

This is a draft page for the PDBParser class.

= Benchmark =

A performance benchmark of the parser was carried out to evaluate wether the development of new features degraded the overall parsing speed.

== Datasets ==

[http://release.cathdb.info/v3.4.0/CathDomainList CATH Domain Collection] - 11330 Structures containing only coordinate information (no Element assigned).

[ftp://ftp.wwpdb.org/pub/pdb/data/structures/divided/pdb/ Protein Data Bank Collection] - 72836 Structures containing both headers and coordinate information.

== Versions Tested ==

[http://biopython.org/DIST/biopython-1.49.zip Biopython 1.49] (Nov. 2008)

[https://github.com/biopython/biopython Biopython 1.57+] (May 2011) | Element column auto-assignment.

[https://github.com/JoaoRodrigues/biopython/tree/pdb_enhancements Biopython 1.58-] (May 2011 @ Github) | Warnings module replaced _handle_pdb_exception() && Other minor changes

== Benchmarking Script ==

The following script was used to benchmark the parser. The garbage collection module - gc - was necessary to avoid a memory leak that caused the machine to start swapping.

<python>
#!/usr/bin/env python
""" Script to Benchmark Bio.PDB PDBParser """

import sys, os, warnings

# Parsing Function
def parse_structure(path):
""" Parses a PDB file """

s = P.get_structure('test', path)

return 0

def fancy_output(tps):
""" Outputs the results in a nicer way """

print "# Bio.PDB PDBParser Benchmark"
print
print "Structure \tLenght \tTime Spent (ms)"
for i,s in enumerate(tps):
print " %s\t (%s) \t%3.3f" %(os.path.basename(pdb_library[i]), pdb_length[i], s)
print
print "Total time spent: %5.3fs" %(sum(tps)/1000)
print "Average time per structure: %5.3fms" %(sum(tps)/len(tps))

if __name__=='__main__':

import time, gc
from Bio.PDB import PDBParser
P = PDBParser(PERMISSIVE=1) # For the pdb_enhancements branch benchmarking, PERMISSIVE was set to 2 (silence warnings).

library_path = sys.argv[1]

pdb_library = [os.path.join(library_path, f) for f in os.listdir(library_path)]
pdb_length = [len(set([int(l[23:26]) for l in open(f) if l.startswith('ATOM')])) for f in pdb_library] # Unique counting of residues
sys.stderr.write("Loaded %s structures (Average Length: %4.3f residues)\n" %(len(pdb_length), (sum(pdb_length)/float(len(pdb_length)))))

tps = []
# Run the Test
for i, pdb_file in enumerate(pdb_library):
sys.stderr.write( "[%s] %i Structure(s) Parsed \n" %(os.path.basename(pdb_file), i+1) )
a = time.time()
parse_structure(pdb_file)
b = time.time()-a
tps.append(b*1000)
gc.collect()
# Output Results
fancy_output(tps)
</python>

== Results ==

=== CATH Dataset ===

Average Structure Length: 147 residues

==== Biopython 1.49 ====

Total Time Spent: 530.686s
Average Time per Structure: 46.84ms/structure
Average Structures per Second: 21.38 structures/s
Failed to parse 0 structures due to errors.

Length N. Structures Average Time Spent (ms)
< 100 3663 25.11
100 =< x < 200 5295 44.68
200 =< x < 500 2328 83.41
500 =< x < 1000 44 180.35

TOTAL 11330 46.84

[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_149.time Link to full results]

==== Biopython 1.57+ ====

Total Time Spent: 686.176s
Average Time per Structure: 60.56ms/structure
Average Structures per Second: 16.51 structures/s
Failed to parse 0 structures due to errors.

Length N. Structures Average Time Spent (ms)
< 100 3663 32.57
100 =< x < 200 5295 57.76
200 =< x < 500 2328 107.56
500 =< x < 1000 44 242.30

TOTAL 11330 60.56

[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_current.time Link to full results]

==== Biopython 1.58- ====

Total Time Spent: 695.405s
Average Time per Structure: 61.37ms/structure
Average Structures per Second: 16.29 structures/s
Failed to parse 0 structures due to errors.

Length N. Structures Average Time Spent (ms)
< 100 3663 33.24
100 =< x < 200 5295 58.46
200 =< x < 500 2328 108.77
500 =< x < 1000 44 247.171

TOTAL 11330 61.38

[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_pdb_enhancements.time Link to full results]

=== PDB Dataset ===

Average Structure Length: 263 residues

==== Biopython 1.49 ====

Total Time Spent: 27801.934s (7.72h)
Average Time per Structure: 381.706ms/structure
Average Structures per Second: 2.62 structures/s
Failed to parse 2 structures due to errors:
1. 3NH3 (negative occupancy) - fix: http://bit.ly/ks6PDN
2. 2WMW (invalid ANISOU field) - fix: http://bit.ly/ld9BWs

Length N. Structures Average Time Spent (ms)
< 100 10986 392.227
100 =< x < 200 19641 371.716
200 =< x < 500 35299 301.699
500 =< x < 1000 6291 597.472
>= 1000 619 2881.520

TOTAL 72836 381.706

[http://nmr.chem.uu.nl/~joao/f/benchmark_PDB-biopython1.49.time Link to full results]

==== Biopython 1.57+ (In progress) ====

In progress

==== Biopython 1.58- (In progress) ====

In progress

PDBParser

Joaor — Thu, 12 May 2011 14:04:09 GMT

Joaor:

This is a draft page for the PDBParser class.

= Benchmark =

A performance benchmark of the parser was carried out to evaluate wether the development of new features degraded the overall parsing speed.

== Datasets ==

[http://release.cathdb.info/v3.4.0/CathDomainList CATH Domain Collection] - 11330 Structures containing only coordinate information (no Element assigned).

[ftp://ftp.wwpdb.org/pub/pdb/data/structures/divided/pdb/ Protein Data Bank Collection] - 72836 Structures containing both headers and coordinate information.

== Versions Tested ==

[http://biopython.org/DIST/biopython-1.49.zip Biopython 1.49] (Nov. 2008)

[https://github.com/biopython/biopython Biopython 1.57+] (May 2011) | Element column auto-assignment.

[https://github.com/JoaoRodrigues/biopython/tree/pdb_enhancements] Biopython 1.58- (May 2011 @ Github) | Warnings module replaced _handle_pdb_exception() && Other minor changes

== Benchmarking Script ==

The following script was used to benchmark the parser. The garbage collection module - gc - was necessary to avoid a memory leak that caused the machine to start swapping.

<python>
#!/usr/bin/env python
""" Script to Benchmark Bio.PDB PDBParser """

import sys, os, warnings

# Parsing Function
def parse_structure(path):
""" Parses a PDB file """

s = P.get_structure('test', path)

return 0

def fancy_output(tps):
""" Outputs the results in a nicer way """

print "# Bio.PDB PDBParser Benchmark"
print
print "Structure \tLenght \tTime Spent (ms)"
for i,s in enumerate(tps):
print " %s\t (%s) \t%3.3f" %(os.path.basename(pdb_library[i]), pdb_length[i], s)
print
print "Total time spent: %5.3fs" %(sum(tps)/1000)
print "Average time per structure: %5.3fms" %(sum(tps)/len(tps))

if __name__=='__main__':

import time, gc
from Bio.PDB import PDBParser
P = PDBParser(PERMISSIVE=1) # For the pdb_enhancements branch benchmarking, PERMISSIVE was set to 2 (silence warnings).

library_path = sys.argv[1]

pdb_library = [os.path.join(library_path, f) for f in os.listdir(library_path)]
pdb_length = [len(set([int(l[23:26]) for l in open(f) if l.startswith('ATOM')])) for f in pdb_library] # Unique counting of residues
sys.stderr.write("Loaded %s structures (Average Length: %4.3f residues)\n" %(len(pdb_length), (sum(pdb_length)/float(len(pdb_length)))))

tps = []
# Run the Test
for i, pdb_file in enumerate(pdb_library):
sys.stderr.write( "[%s] %i Structure(s) Parsed \n" %(os.path.basename(pdb_file), i+1) )
a = time.time()
parse_structure(pdb_file)
b = time.time()-a
tps.append(b*1000)
gc.collect()
# Output Results
fancy_output(tps)
</python>

== Results ==

=== CATH Dataset ===

Average Structure Length: 147 residues

==== Biopython 1.49 ====

Total Time Spent: 530.686s
Average Time per Structure: 46.84ms/structure
Average Structures per Second: 21.38 structures/s
Failed to parse 0 structures due to errors.

Length N. Structures Average Time Spent (ms)
< 100 3663 25.11
100 =< x < 200 5295 44.68
200 =< x < 500 2328 83.41
500 =< x < 1000 44 180.35

TOTAL 11330 46.84

[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_149.time Link to full results]

==== Biopython 1.57+ ====

Total Time Spent: 686.176s
Average Time per Structure: 60.56ms/structure
Average Structures per Second: 16.51 structures/s
Failed to parse 0 structures due to errors.

Length N. Structures Average Time Spent (ms)
< 100 3663 32.57
100 =< x < 200 5295 57.76
200 =< x < 500 2328 107.56
500 =< x < 1000 44 242.30

TOTAL 11330 60.56

[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_current.time Link to full results]

==== Biopython 1.58- ====

Total Time Spent: 695.405s
Average Time per Structure: 61.37ms/structure
Average Structures per Second: 16.29 structures/s
Failed to parse 0 structures due to errors.

Length N. Structures Average Time Spent (ms)
< 100 3663 33.24
100 =< x < 200 5295 58.46
200 =< x < 500 2328 108.77
500 =< x < 1000 44 247.171

TOTAL 11330 61.38

[http://nmr.chem.uu.nl/~joao/f/benchmark_CATH-biopython_pdb_enhancements.time Link to full results]

=== PDB Dataset ===

Average Structure Length: 263 residues

==== Biopython 1.49 ====

Total Time Spent: 27801.934s (7.72h)
Average Time per Structure: 381.706ms/structure
Average Structures per Second: 2.62 structures/s
Failed to parse 2 structures due to errors:
1. 3NH3 (negative occupancy) - fix: http://bit.ly/ks6PDN
2. 2WMW (invalid ANISOU field) - fix: http://bit.ly/ld9BWs

Length N. Structures Average Time Spent (ms)
< 100 10986 392.227
100 =< x < 200 19641 371.716
200 =< x < 500 35299 301.699
500 =< x < 1000 6291 597.472
>= 1000 619 2881.520

TOTAL 72836 381.706

[http://nmr.chem.uu.nl/~joao/f/benchmark_PDB-biopython1.49.time Link to full results]

==== Biopython 1.57+ (In progress) ====

In progress

==== Biopython 1.58- (In progress) ====

In progress

PDBParser

Joaor — Thu, 12 May 2011 13:58:19 GMT

Joaor:

This is a draft page for the PDBParser class.

= Benchmark =

A performance benchmark of the parser was carried out to evaluate wether the development of new features degraded the overall parsing speed.

== Datasets ==

[http://release.cathdb.info/v3.4.0/CathDomainList CATH Domain Collection] - 11330 Structures containing only coordinate information (no Element assigned).

[ftp://ftp.wwpdb.org/pub/pdb/data/structures/divided/pdb/ Protein Data Bank Collection] - 72836 Structures containing both headers and coordinate information.

== Versions Tested ==

[http://biopython.org/DIST/biopython-1.49.zip Biopython 1.49] (Nov. 2008)

[https://github.com/biopython/biopython Biopython 1.57+] (May 2011) | Element column auto-assignment.

[https://github.com/JoaoRodrigues/biopython/tree/pdb_enhancements] Biopython 1.58- (May 2011 @ Github) | Warnings module replaced _handle_pdb_exception() && Other minor changes

== Benchmarking Script ==

The following script was used to benchmark the parser. The garbage collection module - gc - was necessary to avoid a memory leak that caused the machine to start swapping.

<python>
#!/usr/bin/env python
""" Script to Benchmark Bio.PDB PDBParser """

import sys, os, warnings

# Parsing Function
def parse_structure(path):
""" Parses a PDB file """

s = P.get_structure('test', path)

return 0

def fancy_output(tps):
""" Outputs the results in a nicer way """

print "# Bio.PDB PDBParser Benchmark"
print
print "Structure \tLenght \tTime Spent (ms)"
for i,s in enumerate(tps):
print " %s\t (%s) \t%3.3f" %(os.path.basename(pdb_library[i]), pdb_length[i], s)
print
print "Total time spent: %5.3fs" %(sum(tps)/1000)
print "Average time per structure: %5.3fms" %(sum(tps)/len(tps))

if __name__=='__main__':

import time, gc
from Bio.PDB import PDBParser
P = PDBParser(PERMISSIVE=1) # For the pdb_enhancements branch benchmarking, PERMISSIVE was set to 2 (silence warnings).

library_path = sys.argv[1]

pdb_library = [os.path.join(library_path, f) for f in os.listdir(library_path)]
pdb_length = [len(set([int(l[23:26]) for l in open(f) if l.startswith('ATOM')])) for f in pdb_library] # Unique counting of residues
sys.stderr.write("Loaded %s structures (Average Length: %4.3f residues)\n" %(len(pdb_length), (sum(pdb_length)/float(len(pdb_length)))))

tps = []
# Run the Test
for i, pdb_file in enumerate(pdb_library):
sys.stderr.write( "[%s] %i Structure(s) Parsed \n" %(os.path.basename(pdb_file), i+1) )
a = time.time()
parse_structure(pdb_file)
b = time.time()-a
tps.append(b*1000)
gc.collect()
# Output Results
fancy_output(tps)
</python>

== Results ==

=== CATH Dataset ===

Average Structure Length: 147 residues

==== Biopython 1.49 ====

Total Time Spent: 530.686s
Average Time per Structure: 46.84ms/structure
Average Structures per Second: 21.38 structures/s
Failed to parse 0 structures due to errors.

Length N. Structures Average Time Spent (ms)
< 100 3663 25.11
100 =< x < 200 5295 44.68
200 =< x < 500 2328 83.41
500 =< x < 1000 44 180.35

TOTAL 11330 46.84

==== Biopython 1.57+ ====

Total Time Spent: 686.176s
Average Time per Structure: 60.56ms/structure
Average Structures per Second: 16.51 structures/s
Failed to parse 0 structures due to errors.

Length N. Structures Average Time Spent (ms)
< 100 3663 32.57
100 =< x < 200 5295 57.76
200 =< x < 500 2328 107.56
500 =< x < 1000 44 242.30

TOTAL 11330 60.56

==== Biopython 1.58- ====

Total Time Spent: 695.405s
Average Time per Structure: 61.37ms/structure
Average Structures per Second: 16.29 structures/s
Failed to parse 0 structures due to errors.

Length N. Structures Average Time Spent (ms)
< 100 3663 33.24
100 =< x < 200 5295 58.46
200 =< x < 500 2328 108.77
500 =< x < 1000 44 247.171

TOTAL 11330 61.38

=== PDB Dataset ===

Average Structure Length: 263 residues

==== Biopython 1.49 ====

Total Time Spent: 27801.934s (7.72h)
Average Time per Structure: 381.706ms/structure
Average Structures per Second: 2.62 structures/s
Failed to parse 2 structures due to errors:
1. 3NH3 (negative occupancy) - fix: http://bit.ly/ks6PDN
2. 2WMW (invalid ANISOU field) - fix: http://bit.ly/ld9BWs

Length N. Structures Average Time Spent (ms)
< 100 10986 392.227
100 =< x < 200 19641 371.716
200 =< x < 500 35299 301.699
500 =< x < 1000 6291 597.472
>= 1000 619 2881.520

TOTAL 72836 381.706

==== Biopython 1.57+ (In progress) ====

In progress

==== Biopython 1.58- (In progress) ====

In progress

PDBParser

Joaor — Thu, 12 May 2011 13:57:33 GMT

Joaor: Creation

This is a draft page for the PDBParser class.

= Benchmark =

A performance benchmark of the parser was carried out to evaluate wether the development of new features degraded the overall parsing speed.

== Datasets ==

[http://release.cathdb.info/v3.4.0/CathDomainList CATH Domain Collection] - 11330 Structures containing only coordinate information (no Element assigned).

[ftp://ftp.wwpdb.org/pub/pdb/data/structures/divided/pdb/ Protein Data Bank Collection] - 72836 Structures containing both headers and coordinate information.

== Versions Tested ==

[http://biopython.org/DIST/biopython-1.49.zip Biopython 1.49] (Nov. 2008)

[https://github.com/biopython/biopython Biopython 1.57+] (May 2011) | Element column auto-assignment.

[https://github.com/JoaoRodrigues/biopython/tree/pdb_enhancements] Biopython 1.58- (May 2011 @ Github) | Warnings module replaced _handle_pdb_exception() && Other minor changes

== Benchmarking Script ==

The following script was used to benchmark the parser. The garbage collection module - gc - was necessary to avoid a memory leak that caused the machine to start swapping.

<python>
#!/usr/bin/env python
""" Script to Benchmark Bio.PDB PDBParser """

import sys, os, warnings

# Parsing Function
def parse_structure(path):
""" Parses a PDB file """

s = P.get_structure('test', path)

return 0

def fancy_output(tps):
""" Outputs the results in a nicer way """

print "# Bio.PDB PDBParser Benchmark"
print
print "Structure \tLenght \tTime Spent (ms)"
for i,s in enumerate(tps):
print " %s\t (%s) \t%3.3f" %(os.path.basename(pdb_library[i]), pdb_length[i], s)
print
print "Total time spent: %5.3fs" %(sum(tps)/1000)
print "Average time per structure: %5.3fms" %(sum(tps)/len(tps))

if __name__=='__main__':

import time, gc
from Bio.PDB import PDBParser
P = PDBParser(PERMISSIVE=1) # For the pdb_enhancements branch benchmarking, PERMISSIVE was set to 2 (silence warnings).

library_path = sys.argv[1]

pdb_library = [os.path.join(library_path, f) for f in os.listdir(library_path)]
pdb_length = [len(set([int(l[23:26]) for l in open(f) if l.startswith('ATOM')])) for f in pdb_library] # Unique counting of residues
sys.stderr.write("Loaded %s structures (Average Length: %4.3f residues)\n" %(len(pdb_length), (sum(pdb_length)/float(len(pdb_length)))))

tps = []
# Run the Test
for i, pdb_file in enumerate(pdb_library):
sys.stderr.write( "[%s] %i Structure(s) Parsed \n" %(os.path.basename(pdb_file), i+1) )
a = time.time()
parse_structure(pdb_file)
b = time.time()-a
tps.append(b*1000)
gc.collect()
# Output Results
fancy_output(tps)
</python>

== Results ==

=== CATH Dataset ===

Average Structure Length: 147 residues

==== Biopython 1.49 ====

Total Time Spent: 530.686s
Average Time per Structure: 46.84ms/structure
Average Structures per Second: 21.38 structures/s
Failed to parse 0 structures due to errors.

Length N. Structures Average Time Spent (ms)
< 100 3663 25.11
100 =< x < 200 5295 44.68
200 =< x < 500 2328 83.41
500 =< x < 1000 44 180.35

TOTAL 11330 46.84

==== Biopython 1.57+ ====

Total Time Spent: 686.176s
Average Time per Structure: 60.56ms/structure
Average Structures per Second: 16.51 structures/s
Failed to parse 0 structures due to errors.

Length N. Structures Average Time Spent (ms)
< 100 3663 32.57
100 =< x < 200 5295 57.76
200 =< x < 500 2328 107.56
500 =< x < 1000 44 242.30

TOTAL 11330 60.56

==== Biopython 1.58- ====

Total Time Spent: 695.405s
Average Time per Structure: 61.37ms/structure
Average Structures per Second: 16.29 structures/s
Failed to parse 0 structures due to errors.

Length N. Structures Average Time Spent (ms)
< 100 3663 33.24
100 =< x < 200 5295 58.46
200 =< x < 500 2328 108.77
500 =< x < 1000 44 247.171

TOTAL 11330 61.38

=== PDB Dataset ===

Average Structure Length: 263 residues

==== Biopython 1.49 ====

Total Time Spent: 27801.934s (7.72h)
Average Time per Structure: 381.706ms/structure
Average Structures per Second: 2.62 structures/s
Failed to parse 2 structures due to errors:
1. 3NH3 (negative occupancy) - fix: http://bit.ly/ks6PDN
2. 2WMW (invalid ANISOU field) - fix: http://bit.ly/ld9BWs

Length N. Structures Average Time Spent (ms)
< 100 10986 392.227
100 =< x < 200 19641 371.716
200 =< x < 500 35299 301.699
500 =< x < 1000 6291 597.472
>= 1000 619 2881.520

TOTAL 72836 381.706

==== Biopython 1.57+ ====

In progress

==== Biopython 1.58- ====

Queued...

Google Summer of Code

Joaor — Thu, 10 Mar 2011 17:24:08 GMT

Joaor: Added GSOC 2010 project and updated the dates to 2011

As part of the Open Bioinformatics Foundation, Biopython is participating in Google Summer of Code (GSoC) again in 2010. We are supporting João Rodrigues in his project, "[[GSOC2010_Joao|Extending Bio.PDB: broadening the usefulness of BioPython's Structural Biology module]]."

In 2009, Biopython was involved with GSoC in collaboration with our friends at [https://www.nescent.org/wg_phyloinformatics/Main_Page NESCent], and had two projects funded:

* Nick Matzke worked on [https://www.nescent.org/wg_phyloinformatics/Phyloinformatics_Summer_of_Code_2009#Biogeographical_Phylogenetics_for_BioPython Biogeographical Phylogenetics].
* Eric Talevich added support for [https://www.nescent.org/wg_phyloinformatics/Phyloinformatics_Summer_of_Code_2009#Biopython_support_for_parsing_and_writing_phyloXML parsing and writing phyloXML].

In 2010, another project was funded:

* João Rodrigues worked on [http://www.biopython.org/wiki/GSOC2010_Joao the Structural Biology module Bio.PDB] adding several features used in everyday structural bioinformatics.

Please read the [http://www.open-bio.org/wiki/Google_Summer_of_Code GSoC page at the Open Bioinformatics Foundation] and the main [http://code.google.com/soc Google Summer of Code] page for more details about the program. If you are interested in contributing as a mentor or student next year, please introduce yourself on the [http://biopython.org/wiki/Mailing_lists mailing list].

== 2011 Project ideas ==

=== Biopython and PyCogent interoperability ===

; Rationale : [http://pycogent.sourceforge.net/ PyCogent] and [http://biopython.org/wiki/Main_Page Biopython] are two widely used toolkits for performing computational biology and bioinformatics work in Python. The libraries have had traditionally different focuses: with Biopython focusing on sequence parsing and retrieval and PyCogent on evolutionary and phylogenetic processing. Both user communities would benefit from increased interoperability between the code bases, easing the developing of complex workflows.

; Approach : The student would focus on soliciting use case scenarios from developers and the larger communities associated with both projects, and use these as the basis for adding glue code and documentation to both libraries. Some use cases of immediate interest as a starting point are:

:* Allow round-trip conversion between biopython and pycogent core objects (sequence, alignment, tree, etc.).
:* Building workflows using Codon Usage analyses in PyCogent with clustering code in Biopython.
:* Connecting Biopython acquired sequences to PyCogent's alignment, phylogenetic tree preparation and tree visualization code.
:* Integrate Biopython's [http://biopython.org/wiki/Phylo phyloXML support], developed during GSoC 2009, with PyCogent.
:* Develop a standardised controller architecture for interrogation of genome databases by extending PyCogent's Ensembl code, including export to Biopython objects.

; Challenges : This project provides the student with a lot of freedom to create useful interoperability between two feature rich libraries. As opposed to projects which might require churning out more lines of code, the major challenge here will be defining useful APIs and interfaces for existing code. High level inventiveness and coding skill will be required for generating glue code; we feel library integration is an extremely beneficial skill. We also value clear use case based documentation to support the new interfaces.

; Involved toolkits or projects :

:* [http://biopython.org/wiki/Main_Page Biopython]
:* [http://pycogent.sourceforge.net/ PyCogent]

; Degree of difficulty and needed skills : Medium to Hard. At a minimum, the student will need to be highly competent in Python and become familiar with core objects in PyCogent and Biopython. Sub-projects will require additional expertise, for instance: familiarity with concepts in phylogenetics and genome biology; understanding SQL dialects.

; Mentors : [http://jcsmr.anu.edu.au/org/dmb/compgen/ Gavin Huttley], [http://chem.colorado.edu/index.php?option=com_content&view=article&id=263:rob-knight Rob Knight], [http://bcbio.wordpress.com Brad Chapman], [[User:EricTalevich|Eric Talevich]]

=== Galaxy phylogenetics pipeline development ===

; Rationale : [http://main.g2.bx.psu.edu/ Galaxy] is a popular web based interface for integrating biological tools and analysis pipelines. It is widely used by bench biologists for their analysis work, and by computational biologists for building interfaces to developed tools. [http://hyphy.org HyPhy] provides a popular package for molecular evolution and sequence statistical analysis, and the [http://www.datamonkey.org/ datamonkey.org] server provides web based workflows to perform a number of common tasks with HyPhy. This project bridges these two complementary projects by bringing HyPhy workflows into the Galaxy system, standardizing these analyses on a widely used platform.

; Approach : The student would bring existing workflows from datamonkey.org to Galaxy. The general approach would be to pick a datamonkey.org workflow, wrap the relevant tools using [http://bitbucket.org/galaxy/galaxy-central/wiki/AddToolTutorial Galaxy's XML tool definition language], and implement a shared pipeline with [http://screencast.g2.bx.psu.edu/galaxy/flash/WorkflowFromHistory.html Galaxy's workflow system]. Functional tests will be developed for tools and workflows, along with high level documentation for end users.

; Challenges : This project requires the student to become comfortable working in the existing Galaxy framework. This is a useful practical skill as Galaxy is widely used in the biological community. Similarly, the student should become familiar with the statistical evolutionary methods in HyPhy to feel comfortable wrapping and testing them in Galaxy. Since the tools would be widely used from the main Galaxy website and installed instances, we place a strong emphasis on students who feel comfortable building tests and examples that would ensure the developed workflows function as expected.

; Involved toolkits or projects :

:* [http://bitbucket.org/galaxy/galaxy-central/wiki/Home Galaxy]
:* [http://hyphy.org HyPhy]
:* [http://www.datamonkey.org Adaptive Evolution Server]

; Degree of difficulty and needed skills : Medium to Hard. As envisioned, the project would involve implementing full phylogenetic pipelines with the Galaxy toolkits. This would require becoming familiar with the Galaxy tool integration framework as well as being comfortable with HyPhy tools and current pipelines. This would involve comfort with XML for developing the tool interfaces, and Python for integrating scripts and tests with Galaxy and HyPhy.

; Mentors : [http://www.hyphy.org/sergei/ Sergei L Kosakovsky Pond], [http://bcbio.wordpress.com Brad Chapman], [http://www.bx.psu.edu/~anton/ Anton Nekrutenko]

=== Accessing R phylogenetic tools from Python ===

; Rationale : The [http://www.r-project.org/ R statistical language] is a powerful open-source environment for statistical computation and visualization. [http://www.python.org/ Python] serves as an excellent complement to R since it has a wide variety of available libraries to make data processing, analysis, and web presentation easier. The two can be smoothly interfaced using [http://bitbucket.org/lgautier/rpy2/ Rpy2], allowing programmers to leverage the best features of each language. Here we propose to build Rpy2 library components to help ease access to phylogenetic and biogeographical libraries in R.

; Approach : Rpy2 contains higher level interfaces to popular R libraries. For instance, the [http://rpy.sourceforge.net/rpy2/doc-2.1/html/graphics.html#package-ggplot2 ggplot2 interface] allows python users to access powerful plotting functionality in R with an intuitive API. Providing similar high level APIs for biological toolkits available in R would help expose these toolkits to a wider audience of Python programmers. A nice introduction to phylogenetic analysis in R is available from Rich Glor at the [http://bodegaphylo.wikispot.org/Phylogenetics_and_Comparative_Methods_in_R Bodega Bay Marine Lab wiki]. Some examples of R libraries for which integration would be welcomed are:

:* [http://ape.mpl.ird.fr/ ape (Analysis of Phylogenetics and Evolution)] -- an interactive library environment for phylogenetic and evolutionary analyses
:* [http://pbil.univ-lyon1.fr/ADE-4/home.php?lang=eng ade4] -- Data Analysis functions to analyse Ecological and Environmental data in the framework of Euclidean Exploratory methods
:* [http://cran.r-project.org/web/packages/geiger/index.html geiger] -- Running macroevolutionary simulation, and estimating parameters related to diversification from comparative phylogenetic data.
:* [http://picante.r-forge.r-project.org/ picante] -- R tools for integrating phylogenies and ecology
:* [http://mefa.r-forge.r-project.org/ mefa] -- multivariate data handling for ecological and biogeographical data

; Challenges : The student would have the opportunity to learn an available R toolkit, and then code in Python and R to make this available via an intuitive API. This will involve digging into the R code examples to discover the most useful parts for analysis, and then projecting this into a library that is intuitive to Python coders. Beyond the coding and design aspects, the student should feel comfortable writing up use case documentation to support the API and encourage its adoption.

; Involved toolkits or projects :

:* [http://ape.mpl.ird.fr/ ape (Analysis of Phylogenetics and Evolution)]
:* [http://bitbucket.org/lgautier/rpy2/ Rpy2]
:* [http://biopython.org/wiki/Main_Page Biopython]

; Degree of difficulty and needed skills : Moderate. The project requires familiarity with coding in Python and R, and knowledge of phylogeny or biogeography. The student has plenty of flexibility to define the project based on their biological interests (e.g. [http://www.warwick.ac.uk/go/peter_cock/python/heatmap/ microarrays and heatmaps]); there is also the possibility to venture far into data visualization once access to analysis methods is made. [http://kiwi.cs.dal.ca/GenGIS/Main_Page GenGIS] and can give ideas about what is possible.

; Mentors : [http://dk.linkedin.com/pub/laurent-gautier/8/81/869 Laurent Gautier], [http://bcbio.wordpress.com Brad Chapman], [http://www.scri.ac.uk/staff/petercock Peter Cock]

Struct

Joaor — Fri, 13 Aug 2010 23:41:35 GMT

Joaor: Added Struct Content

This module extends Bio.PDB, providing additional features that prove useful to structural biologists using Biopython.

=== Availability ===

This module is only available (for now) in [http://github.com/JoaoRodrigues/biopython/tree/GSOC2010 João's GSOC2010 branch in Github].

After getting the branch, it can be accessed easily by an import statement. It should suffice to access all of the functions. Importing specific parts of the module is not supposed to be necessary unless likewise specific operations are sought after.

<python>from Bio import Struct</python>

=== I/O Functions ===

Bio.Struct provides a simple I/O interface to read and write structures in PDB format. It has two methods '''read''' and '''write''' that wrap Bio.PDB.PDBParser and Bio.PDB.PDBIO respectively. It does not intend to replace these methods, merely being a simpler way of performing I/O tasks.

==== read() ====

<python>s = Struct.read('protein_A.pdb')</python>

The name of the resulting Structure object is based on the filename.

It has an additional ''id'' argument that allows specification of the Structure object name, much like the first argument in ''PDBParser.get_structure()''.

==== write() ====

<python>Struct.write(s)</python>

The name of the resulting output is based on the Structure object id.

It has an additional ''name'' argument that allows naming the output file (much like the main argument in ''PDBIO.save()''). If a file already exists by that name, the write() function automatically renames the output adding ''_0'' (or _1, etc).

=== Protein Class ===

We created a Structure-based class named Protein to confer protein specific methods to a given SMCRA object. A method was also added to Bio.PDB.Structure that allows easy interconversion between the two classes: '''as_protein()'''. We will discuss each method in detail below.

==== as_protein() ====

Converts a Structure object to the Protein class. The conversion also filters all residues and excludes all those that are not aminoacids (HETATMs are also excluded). This filtering can be disabled by setting the optional ''filter_residues'' argument to False.

<python>s = Struct.read('protein_A.pdb')

dir(s)
# Edited for shortening purposes
['__doc__', ... , 'renumber_residues', 'set_parent', 'xtra']

p = s.as_protein()

dir(p)

['__doc__', ... , 'renumber_residues', 'search_ss_bonds', 'set_parent', 'xtra']</python>

==== search_ss_bonds() ====

The function returns an iterator with tuples of pairs of Cysteine residues in close enough proximity to be forming a SS bond. The threshold for a S-S contact to be defined as a SS bond is 3.0A, but it can be manually specified through the ''threshold'' argument.

<python>for bond in p.search_ss_bonds():
print bond

(<Residue CYS het= resseq=5 icode= >, <Residue CYS het= resseq=55 icode= >)
(<Residue CYS het= resseq=14 icode= >, <Residue CYS het= resseq=38 icode= >)
(<Residue CYS het= resseq=30 icode= >, <Residue CYS het= resseq=51 icode= >)</python>

==== coarse_grain() ====

Despite coarse-graining being general to all molecules, the current implemented methods concern proteins only. To ease the introduction of new CG-models, a CG_models.py class is present that defines how each residue should be coarse grained. As of now, three models are supported.

===== Example Usage: MARTINI =====

<python>cg_martini = p.coarse_grain('MARTINI')

for residue in cg_martini.get_residues():
print residue.resname, residue.child_list

ARG [<Atom BB>, <Atom S1>, <Atom S2>]
PRO [<Atom BB>, <Atom S1>]
ASP [<Atom BB>, <Atom S1>]
PHE [<Atom BB>, <Atom S1>, <Atom S2>, <Atom S3>]
......
GLY [<Atom BB>]
ALA [<Atom BB>]</python>

===== Supported Models =====

====== CA Trace ======

Protein residues are reduced to their alpha carbon atom. This is the default method.

====== ENCAD 3pt Model ======

ENCAD is the Energy Calculation and Dynamics program, developed by Michal Levitt since the 80s. Its 3pt coarse grained model reduces each protein residue to 3 points (some exceptions are reduced to 4 points): Ca, O, and a side chain bead centered on a particular pre-defined atom.

====== MARTINI Model ======

MARTINI is a well known CG model that, as ENCAD, reduces protein residues to 3/4 beads: Ca, O, and a side chain bead in a particular position.

==== check_missing_atoms() ====

Compares the residues in the Protein object with a pre-defined topology (derived from AMBER) to check for missing atoms. Outputs a dictionary of tuples for each incomplete residue. Automatically ignores Hydrogen atoms (''ha_only'' argument can be set to False to override this) and allows the usage of a particular template through the ''template'' argument (default None). Templates are dictionaries with residue names as keys and lists with atom names as values.

<python>p.check_missing_atoms()</python>

==== find_seq_homologues() ====

Bridges Bio.Blast.NCBIWWW qblast() function. It allows a direct sequence homology search through that function using the Protein object's sequence. Auto-adjusts parameters for short sequences. For more complex homology searches, use the Bio.Blast.NCBIWWW module directly as this is supposed to be just a convenience function.

It returns a list ranked by Expectation Value with some informational values (e-value, identities, positives, gaps), the PDB code of the match, and the alignment.

Allows an argument, ''raw_output'', that replaces the default parsed results with raw XML output from the BLAST search.

<python>seq_homologues = p.find_seq_homologues()

for homologues in seq_homologues:
print homologues[0], homologues[1]
print homologues[-1]
print

2BUO 1.82482e-31
DIRQGPKEPFRDYVDRFYKTLRAEQASQEVKNW-TETLLVQNANPDCKTILKALGPGATLEE--TACQG
DIRQGPKEPFRDYVDRFYKTLRAEQASQEVKNW TETLLVQNANPDCKTILKALGPGATLEE TACQG
DIRQGPKEPFRDYVDRFYKTLRAEQASQEVKNWMTETLLVQNANPDCKTILKALGPGATLEEMMTACQG

.....</python>

Struct

Joaor — Fri, 13 Aug 2010 00:52:03 GMT

Joaor: Creation

This page describes Bio.Struct, a new module based on [[Bio.PDB]] that adds further structural biology features.

== Aims ==

Some thing here...

GSOC2010 Joao

Joaor — Fri, 13 Aug 2010 00:38:45 GMT

Joaor: /* Support for MODELLER PIR format in SeqIO */ Edited mpir to pir-modeller

== Author & Mentors ==

[[User:Joaor|João Rodrigues]] anaryin@gmail.com

'''Mentors'''
: Eric Talevich
: Diana Jaunzeikare
: Peter Cock

== Abstract ==

Biopython is a very popular library in Bioinformatics and Computational Biology. Its Bio.PDB module, originally developed by Thomas Hamelryck, is a simple yet powerful tool for structural biologists. Although it provides a reliable PDB parser feature and it allows several calculations (Neighbour Search, RMS) to be made on macromolecules, it still lacks a number of features that are part of a researcher's daily routine. Probing for disulphide bridges in a structure and adding polar hydrogen atoms accordingly are two examples that can be incorporated in Bio.PDB, given the module's clever structure and good overall organisation. Cosmetic operations such as chain removal and residue renaming – to account for the different existing nomenclatures – and renumbering would also be greatly appreciated by the community.

Another aspect that can be improved for Bio.PDB is a smooth integration/interaction layer for heavy-weights in macromolecule simulation such as MODELLER, GROMACS, AutoDock, HADDOCK. It could be argued that the easiest solution would be to code hooks to these packages' functions and routines. However, projects such as the recently developed [http://sbcb.bioch.ox.ac.uk/oliver/software/GromacsWrapper/html/edpdb.html edPDB] or the more complete [http://biskit.pasteur.fr/ Biskit library] render, in my opinion, such interfacing efforts redundant. Instead, I believe it to be more advantageous to include these software' input/output formats in Biopython's SeqIO and AlignIO modules. This, together with the creation of interfaces for model validation/structure checking services/software would allow Biopython to be used as a pre- and post-simulation tool. Eventually, it would pave the way for its inclusion in pipelines and workflows for structure modelling, molecular dynamics, and docking simulations.

== Project Schedule ==

The schedule below was organised to be flexible, which means that some features will likely be done early. Also, the weeks include documentation and unit testing efforts for the features, with extended periods for reviewing these efforts at the two points during the project (halfway, final week).

=== Community Bonding Period ===

*Getting familiar with development environment (Git Hub account, Git, Biopython's repository, Bug tracking system, etc)

*Gather scientific literature and discuss some of the to-be-implemented methods.

=== Week 1 [31st May - 6th June] ===

==== Renumbering residues of a structure ====

*Read SEQRES record to account for gaps
*Alternatively read ATOM records.

==== Probe disulphide bridges in the structure ====

*Via NeighbourSearch class
*Also use SSBOND in header

==== Extract Biological Unit ====

*REMARK350 contains rotation and translation information
*If REMARK is absent, do nothing.

=== Week 2 [7th – 13th June] ===

==== Structure Hydrogenation ====

*Add all/polar hydrogens through interface with WHATIF server.
*Optionally define a set pH
** [http://www3.interscience.wiley.com/journal/112117957/abstract pKa values algorithm]

==== Hydrogenation Report ====

*Produces a brief list of polar hydrogen atoms in the structure.
** Chain | Residue [number] | Atom

=== Weeks 3-5 [14th June- 4th July] ===

==== Removal of disordered atoms ====

*[[Remove_PDB_disordered_atoms|Solution proposed in the Biopython wiki]]

==== Residue name normalisation ====

*Build conversion table from different nomenclatures (research them during c.bonding period )
*Write function to make a given structure compliant with a given software nomenclature:
** Amber
** CNS/HADDOCK
** GROMACS

==== Coarse Grain Structure ====

*Implement function to reduce complexity of a structure
** 1pt*c-alpha
** 2pt*c-alpha / c-beta
** 3pt*c-alpha / c-beta / side-chain pseudo-centroid OR side-chain centroid

=== Week 6 (Mid-Term) [5th - 11th July] ===

: Testing and consolidating the features thoroughly.
: Write documentation & examples for each feature, to be included in Biopython's Wiki and Bio.PDB's FAQ.
: Mid-term Evaluations. Discussing with mentors current state of project and adjust following schedule to comply with project's needs.

=== Week 7 [12th - 19th July] ===

==== Add support for MODELLER's PIR format to Biopython ====

*[http://www.salilab.org/modeller/manual/node445.html#alignmentformat Format Description]
*SeqIO
*AlignIO

==== Allow conversion of Structure Object to Sequence Object ====

*Based on Bio.PDB.Polypeptide function

=== Weeks 8-10 [20th July - 9th August] ===

==== Add Sequence/Structure Homology functions ====

*Create call to Biopython's BLAST interfaces
** Allow direct blast from structure object ( e.g. protein.find_homoseq() )
** Returns list of tuples with E-Value *Dictionary (name, length of alignment, etc..)
*Create interface with structural homology web services
** e.g. [http://ekhidna.biocenter.helsinki.fi/dali_server/ Dali server]
** Return list of tuples with Z-Score*Dictionary (name, etc...)

==== Implement basic structure validation checks ====

*Via NeighbourSearch class
** Same Charge contacts
** Atom Clashes
*Via ResidueDepth Class
** Buried Charges
*Interface WHATIF PDBReport web service
** Parse WARNING and ERROR messages

=== Week 11 [10th - 17th August] ===

==== Reviewing documentation, code, write tests for new functions. ====

== Project Code ==

Hosted at [http://github.com/JoaoRodrigues/biopython/tree/GSOC2010 this GitHub branch]

== Project Progress ==

Since I'm adding some methods that are useful/logical only for proteins, having them exposed in Structure.py for every molecule could be misleading. We decided then to add a 'as_protein()' method that allows protein-specific methods to be accessed. The following example demonstrates how this call works. Note how the "search_ss_bonds" method is absent from dir(s) but not from dir(prot).

<python>
from Bio.PDB import PDBParser

p = PDBParser()
s = p.get_structure('example', '4PTI.pdb')

dir(s)
# Cut for viewing purposes
['__doc__', ... , 'renumber_residues', 'set_parent', 'xtra']

prot = s.as_protein()

dir(prot)

['__doc__', ... , 'renumber_residues', 'search_ss_bonds', 'set_parent', 'xtra']
</python>

=== Renumbering residues of a structure ===

Since parse_pdb_header is far from optimal and is likely to change in the future, I opted to forfeit reading SEQREQ records to account for gaps. However, ignoring this information and renumbering based on ATOM records would make us lose information on gaps. I opted to subtract the first residue number-1 to all residues thus making the numbering start in 1 and still keep gaps. I also added an argument (start) to allow the user to set which number to start the counting from.

Example:

<python>
from Bio.PDB import PDBParser

p = PDBParser()
s = p.get_structure('example', '1IHM.pdb')

print list(s.get_residues())[0]
<Residue ASP het= resseq=1029 icode= >

s.renumber_residues()
print list(s.get_residues())[0]
<Residue ASP het= resseq=1 icode= >
</python>

=== Probe disulphide bridges in the structure ===

The same rationale from SEQRES applies for the exclusion of looking up SSBOND. Also, instead of using NeighborSearch to look for pairs of cysteins in bond distance, I instead used the minus operator since it has been overloaded to return the distance between two atoms (Page 10 of the [http://www.biopython.org/DIST/docs/cookbook/biopdb_faq.pdf FAQ]). The average distance cited in the literature is 2.05A but other software packages and my own tests set 3.0A as a good threshold. Still, the user can set his own threshold manually.

The function returns an iterator with tuples of pairs of residues.

<python>
from Bio.PDB import PDBParser

p = PDBParser()
s = p.get_structure('example', '4PTI.pdb')

prot = s.as_protein()

for bond in prot.search_ss_bonds():
print bond

(<Residue CYS het= resseq=5 icode= >, <Residue CYS het= resseq=55 icode= >)
(<Residue CYS het= resseq=14 icode= >, <Residue CYS het= resseq=38 icode= >)
(<Residue CYS het= resseq=30 icode= >, <Residue CYS het= resseq=51 icode= >)
</python>

=== Extract Biological Unit ===

Added parsing for REMARK350 to parse_pdb_header since there was already a bit written for another REMARK section. This extracts the transformation matrices and the translation vector from the header, that is then fed to the Structure function. Each new rotated structure is created as a new MODEL. I chose this because crystal structures very rarely have more than one MODEL instance and also because NMR models don't have REMARK 350 that often (at least to my knowledge).

<python>
from Bio.PDB import PDBParser

p = PDBParser()

s1 = p.get_structure('a', '4PTI.pdb')
s1.build_biological_unit()
'Processed 0 transformations on the structure.' # Identity matrix is ignored.

s2 = p.get_structure('b', 'homol_1bd8.pdb') # A homology model
s2.build_biological_unit()
'PDB File lacks appropriate REMARK 350 entries to build Biological Unit.'

s3 = p.get_structure('c', '1IHM.pdb')
s3.build_biological_unit()
'Processed 59 transformations on the structure.'
</python>

=== Hydrogenation of PDB files ===

Following discussion between the mentors and me, we decided that maybe it was better to not only include a webserver for this purpose but also a local algorithm. This would not limit the user when there he/she lacks an internet connection.

The interface for the WHATIF Protonation service has been implemented, although it should be regarded as **highly experimental** for now. Interfacing this server included writing a small parser for a PDBXML-like format, which is expected to have serious bugs in its initial versions. I ran some simple tests and it works. It doesn't support water molecules yet, nor any other molecules other than proteins. Such issues will be hopefully solved later on..

For those brave enough to want to test it (and help me debug it), here's an example usage.

<python>
from Bio.Struct.WWW import WHATIF
from Bio import Struct

server = WHATIF.WHATIF() # Performs a sort of PING to the server. Gracefully exits if the servers are down.

# Get the protein structure
structure = Struct.read('4PTI.pdb')
protein = structure.as_protein() # This excludes water molecules

# Upload the structure to the WHATIF server
# This should convert the structure from a Structure object to a string via tempfile and PDBIO
# I was having some issues uploading structures...

id = server.UploadPDB(protein)

# Protonate
# Returns a Structure Object / WARNING! Bug prone for now.

protein_h = server.PDBasXMLwithSymwithPolarH(id)

</python>

Regarding the local implementation, after much reading I settled on using PyMol's algorithm. It seems to allow for protonation of any structure, regardless of its nature (protein, DNA, etc). Its vectorial and matrix operations can likely be optimized with Numpy and Biopython's Vector.py module. This first implementation works for proteins only. I'll add general molecule support later.

<python>
from Bio import Struct
from Bio.Struct import Hydrogenate as H

s = Struct.read('1ctf.pdb')
p = s.as_protein()

prot = H.Hydrogenate_Protein()
prot.add_hydrogens(p)
</python>

=== Coarse Grain Structure ===

A Center of Mass function was developed first as part of a new module Bio.Struct.Geometry. It allows for calculation of the center of geometry (all masses are equal) and center of mass (taking into account elemental masses for the atoms). The masses are a new Atom object feature derived from [http://www.chem.qmul.ac.uk/iupac/AtWt/ this list] and from PyMol. Essentially, all atoms of a structure now get their mass defined when the Structure is created (check Atom.py and [http://lists.open-bio.org/pipermail/biopython-dev/2010-June/007880.html this thread] for details). This is obviously experimental.

To calculate the center of mass of any Entity (Structure, Model, Chain, Residue) or a List of Atoms:

<python>
from Bio.Struct.Geometry import center_of_mass
from Bio import Struct

s = Struct.read('4PTI.pdb')

print center_of_mass.__doc__

Returns gravitic or geometric center of mass of an Entity.
Geometric assumes all masses are equal (geometric=True)
Defaults to Gravitic.

print center_of_mass(s)
[14.833301303933874, 21.431581746366263, 4.1218478418007134]

print center_of_mass(s, geometric=True)
[14.805324902127458, 21.365571977563405, 4.1108949403803985]
</python>

As of now, 3 CG models are supported.

1) CA-Trace
2) ENCAD 3-point model (CA, O, Side Chain bead)
3) MARTINI protein model (BB, Side Chain points [S1 to S4])

An example, picking up the s Structure from above:

<python>
p = s.as_protein() # To expose the CG method

ca_trace = p.coarse_grain()

# One atom per residue
print ( len(list(p.get_residues())) == len(list(ca_trace.get_atoms())) )
True

cg_encad = p.coarse_grain('ENCAD_3P')

for residue in cg_encad.get_residues():
print residue.resname, residue.child_list

ARG [<Atom CA>, <Atom O>, <Atom CMA>]
PRO [<Atom CA>, <Atom O>, <Atom CMA>]
ASP [<Atom CA>, <Atom O>, <Atom CMA>]
PHE [<Atom CA>, <Atom O>, <Atom CMA>]
CYS [<Atom CA>, <Atom O>, <Atom CMA>]
LEU [<Atom CA>, <Atom O>, <Atom CMA>]
GLU [<Atom CA>, <Atom O>, <Atom CMA>]
PRO [<Atom CA>, <Atom O>, <Atom CMA>]
PRO [<Atom CA>, <Atom O>, <Atom CMA>]
TYR [<Atom CA>, <Atom O>, <Atom CMA>]
.....
CYS [<Atom CA>, <Atom O>, <Atom CMA>]
GLY [<Atom CA>, <Atom O>]
GLY [<Atom CA>, <Atom O>]
ALA [<Atom CA>, <Atom O>, <Atom CMA>]

cg_martini = p.coarse_grain('MARTINI')

for residue in cg_martini.get_residues():
print residue.resname, residue.child_list

ARG [<Atom BB>, <Atom S1>, <Atom S2>]
PRO [<Atom BB>, <Atom S1>]
ASP [<Atom BB>, <Atom S1>]
PHE [<Atom BB>, <Atom S1>, <Atom S2>, <Atom S3>]
CYS [<Atom BB>, <Atom S1>]
LEU [<Atom BB>, <Atom S1>]
GLU [<Atom BB>, <Atom S1>]
PRO [<Atom BB>, <Atom S1>]
PRO [<Atom BB>, <Atom S1>]
TYR [<Atom BB>, <Atom S1>, <Atom S2>, <Atom S3>]
......
CYS [<Atom BB>, <Atom S1>]
GLY [<Atom BB>]
GLY [<Atom BB>]
ALA [<Atom BB>]
</python>

=== Removal of disordered atoms ===

Implement as part of Structure.py and based loosely on the [http://www.biopython.org/wiki/Remove_PDB_disordered_atoms contribution of Ramon Crehuet]. The DisorderedAtom objects are removed from the residue and a single Atom object is added corresponding to the location of the user's choice (keep_loc argument) which defaults to A.

An example, still keeping s from above:

<python>
s = s.remove_disordered_atoms(verbose=True)
0 residues were modified

# Now if we load a structure with disordered atoms
ds = Struct.read('1MC2.pdb')
ds.remove_disordered_atoms(verbose=True)
Residue TRP:1010 has 8 disordered atoms: CD1/CD2/NE1/CE2/CE3/CZ2/CZ3/CH2
Residue VAL:1018 has 3 disordered atoms: CB/CG1/CG2
Residue LEU:1024 has 4 disordered atoms: CB/CG/CD1/CD2
Residue ARG:1043 has 7 disordered atoms: CB/CG/CD/NE/CZ/NH1/NH2
Residue MET:1092 has 4 disordered atoms: CB/CG/SD/CE
Residue ARG:1107 has 7 disordered atoms: CB/CG/CD/NE/CZ/NH1/NH2
Residue GLU:1108 has 4 disordered atoms: CG/CD/OE1/OE2
Residue ASP:1111 has 4 disordered atoms: CB/CG/OD1/OD2
Residue SER:1116 has 1 disordered atoms: OG
Residue SER:1131 has 1 disordered atoms: O
10 residues were modified
</python>

=== Sequence Homologues from Structures ===

Biopython supports BLAST (local and remote through NCBI servers). We bridged both Bio.PDB and Bio.Blast modules to allow an easier search for sequence homologues. For now, it supports remote BLAST through Bio.Blast.NCBIWWW and functions as a blackbox - i.e. users cannot change any search parameter. If one wants to fully use BLAST he/she should use the regular BLAST module. This is just a convenience function.

It is accessible only to Protein objects. It queries the PDB subset database of NCBI BLAST servers with the Structure object's sequence, auto-adjusting parameters for short sequences (less than 15 residues).

It returns a list ranked by Expectation Value with some informational values (e-value, identities, positives, gaps), the PDB code of the match, and the alignment.

<python>from Bio import Struct

s = Struct.read('1A8O.pdb')
p = s.as_protein()

seq_homologues = p.find_seq_homologues()

for homologues in seq_homologues:
print homologues[0], homologues[1]
print homologues[-1]
print

2BUO 1.82482e-31
DIRQGPKEPFRDYVDRFYKTLRAEQASQEVKNW-TETLLVQNANPDCKTILKALGPGATLEE--TACQG
DIRQGPKEPFRDYVDRFYKTLRAEQASQEVKNW TETLLVQNANPDCKTILKALGPGATLEE TACQG
DIRQGPKEPFRDYVDRFYKTLRAEQASQEVKNWMTETLLVQNANPDCKTILKALGPGATLEEMMTACQG

1AUM 1.82482e-31
DIRQGPKEPFRDYVDRFYKTLRAEQASQEVKNW-TETLLVQNANPDCKTILKALGPGATLEE--TACQG
DIRQGPKEPFRDYVDRFYKTLRAEQASQEVKNW TETLLVQNANPDCKTILKALGPGATLEE TACQG
DIRQGPKEPFRDYVDRFYKTLRAEQASQEVKNWMTETLLVQNANPDCKTILKALGPGATLEEMMTACQG

.....</python>

=== Support for MODELLER PIR format in SeqIO ===

MODELLER PIR format support was added to SeqIO as 'pir-modeller'. Currently, the format can be read but not written. An example of the format follows, as well as an example of the parser's usage.

<pre>
>P1;5fd1
structureX:5fd1:1 :A:106 :A:ferredoxin:Azotobacter vinelandii: 1.90: 0.19
AFVVTDNCIKCKYTDCVEVCPVDCFYEGPNFLVIHPDECIDCALCEPECPAQAIFSEDEVPEDMQEFIQLNAELA
EVWPNITEKKDPLPDAEDWDGVKGKLQHLER*
</pre>

<python>from Bio import SeqIO

handle = open('test_pir.txt')

records = SeqIO.parse(handle, 'pir-modeller')

for i in records:
print i

ID: 5fd1
Name: 5fd1
Description: ferredoxin
Number of features: 0
/r_factor= 0.19
/end_residue=106
/initial_chain=a
/end_chain=a
/record_type=X-Ray Structure
/initial_residue=1
/resolution= 1.90
/source_organism=Azotobacter vinelandii
Seq('AFVVTDNCIKCKYTDCVEVCPVDCFYEGPNFLVIHPDECIDCALCEPECPAQAI...LER', ProteinAlphabet())</python>

GSOC2010 Joao

Joaor — Thu, 12 Aug 2010 04:53:37 GMT

Joaor: /* Support for MODELLER PIR format in SeqIO */

== Author & Mentors ==

[[User:Joaor|João Rodrigues]] anaryin@gmail.com

'''Mentors'''
: Eric Talevich
: Diana Jaunzeikare
: Peter Cock

== Abstract ==

Biopython is a very popular library in Bioinformatics and Computational Biology. Its Bio.PDB module, originally developed by Thomas Hamelryck, is a simple yet powerful tool for structural biologists. Although it provides a reliable PDB parser feature and it allows several calculations (Neighbour Search, RMS) to be made on macromolecules, it still lacks a number of features that are part of a researcher's daily routine. Probing for disulphide bridges in a structure and adding polar hydrogen atoms accordingly are two examples that can be incorporated in Bio.PDB, given the module's clever structure and good overall organisation. Cosmetic operations such as chain removal and residue renaming – to account for the different existing nomenclatures – and renumbering would also be greatly appreciated by the community.

Another aspect that can be improved for Bio.PDB is a smooth integration/interaction layer for heavy-weights in macromolecule simulation such as MODELLER, GROMACS, AutoDock, HADDOCK. It could be argued that the easiest solution would be to code hooks to these packages' functions and routines. However, projects such as the recently developed [http://sbcb.bioch.ox.ac.uk/oliver/software/GromacsWrapper/html/edpdb.html edPDB] or the more complete [http://biskit.pasteur.fr/ Biskit library] render, in my opinion, such interfacing efforts redundant. Instead, I believe it to be more advantageous to include these software' input/output formats in Biopython's SeqIO and AlignIO modules. This, together with the creation of interfaces for model validation/structure checking services/software would allow Biopython to be used as a pre- and post-simulation tool. Eventually, it would pave the way for its inclusion in pipelines and workflows for structure modelling, molecular dynamics, and docking simulations.

== Project Schedule ==

The schedule below was organised to be flexible, which means that some features will likely be done early. Also, the weeks include documentation and unit testing efforts for the features, with extended periods for reviewing these efforts at the two points during the project (halfway, final week).

=== Community Bonding Period ===

*Getting familiar with development environment (Git Hub account, Git, Biopython's repository, Bug tracking system, etc)

*Gather scientific literature and discuss some of the to-be-implemented methods.

=== Week 1 [31st May - 6th June] ===

==== Renumbering residues of a structure ====

*Read SEQRES record to account for gaps
*Alternatively read ATOM records.

==== Probe disulphide bridges in the structure ====

*Via NeighbourSearch class
*Also use SSBOND in header

==== Extract Biological Unit ====

*REMARK350 contains rotation and translation information
*If REMARK is absent, do nothing.

=== Week 2 [7th – 13th June] ===

==== Structure Hydrogenation ====

*Add all/polar hydrogens through interface with WHATIF server.
*Optionally define a set pH
** [http://www3.interscience.wiley.com/journal/112117957/abstract pKa values algorithm]

==== Hydrogenation Report ====

*Produces a brief list of polar hydrogen atoms in the structure.
** Chain | Residue [number] | Atom

=== Weeks 3-5 [14th June- 4th July] ===

==== Removal of disordered atoms ====

*[[Remove_PDB_disordered_atoms|Solution proposed in the Biopython wiki]]

==== Residue name normalisation ====

*Build conversion table from different nomenclatures (research them during c.bonding period )
*Write function to make a given structure compliant with a given software nomenclature:
** Amber
** CNS/HADDOCK
** GROMACS

==== Coarse Grain Structure ====

*Implement function to reduce complexity of a structure
** 1pt*c-alpha
** 2pt*c-alpha / c-beta
** 3pt*c-alpha / c-beta / side-chain pseudo-centroid OR side-chain centroid

=== Week 6 (Mid-Term) [5th - 11th July] ===

: Testing and consolidating the features thoroughly.
: Write documentation & examples for each feature, to be included in Biopython's Wiki and Bio.PDB's FAQ.
: Mid-term Evaluations. Discussing with mentors current state of project and adjust following schedule to comply with project's needs.

=== Week 7 [12th - 19th July] ===

==== Add support for MODELLER's PIR format to Biopython ====

*[http://www.salilab.org/modeller/manual/node445.html#alignmentformat Format Description]
*SeqIO
*AlignIO

==== Allow conversion of Structure Object to Sequence Object ====

*Based on Bio.PDB.Polypeptide function

=== Weeks 8-10 [20th July - 9th August] ===

==== Add Sequence/Structure Homology functions ====

*Create call to Biopython's BLAST interfaces
** Allow direct blast from structure object ( e.g. protein.find_homoseq() )
** Returns list of tuples with E-Value *Dictionary (name, length of alignment, etc..)
*Create interface with structural homology web services
** e.g. [http://ekhidna.biocenter.helsinki.fi/dali_server/ Dali server]
** Return list of tuples with Z-Score*Dictionary (name, etc...)

==== Implement basic structure validation checks ====

*Via NeighbourSearch class
** Same Charge contacts
** Atom Clashes
*Via ResidueDepth Class
** Buried Charges
*Interface WHATIF PDBReport web service
** Parse WARNING and ERROR messages

=== Week 11 [10th - 17th August] ===

==== Reviewing documentation, code, write tests for new functions. ====

== Project Code ==

Hosted at [http://github.com/JoaoRodrigues/biopython/tree/GSOC2010 this GitHub branch]

== Project Progress ==

Since I'm adding some methods that are useful/logical only for proteins, having them exposed in Structure.py for every molecule could be misleading. We decided then to add a 'as_protein()' method that allows protein-specific methods to be accessed. The following example demonstrates how this call works. Note how the "search_ss_bonds" method is absent from dir(s) but not from dir(prot).

<python>
from Bio.PDB import PDBParser

p = PDBParser()
s = p.get_structure('example', '4PTI.pdb')

dir(s)
# Cut for viewing purposes
['__doc__', ... , 'renumber_residues', 'set_parent', 'xtra']

prot = s.as_protein()

dir(prot)

['__doc__', ... , 'renumber_residues', 'search_ss_bonds', 'set_parent', 'xtra']
</python>

=== Renumbering residues of a structure ===

Since parse_pdb_header is far from optimal and is likely to change in the future, I opted to forfeit reading SEQREQ records to account for gaps. However, ignoring this information and renumbering based on ATOM records would make us lose information on gaps. I opted to subtract the first residue number-1 to all residues thus making the numbering start in 1 and still keep gaps. I also added an argument (start) to allow the user to set which number to start the counting from.

Example:

<python>
from Bio.PDB import PDBParser

p = PDBParser()
s = p.get_structure('example', '1IHM.pdb')

print list(s.get_residues())[0]
<Residue ASP het= resseq=1029 icode= >

s.renumber_residues()
print list(s.get_residues())[0]
<Residue ASP het= resseq=1 icode= >
</python>

=== Probe disulphide bridges in the structure ===

The same rationale from SEQRES applies for the exclusion of looking up SSBOND. Also, instead of using NeighborSearch to look for pairs of cysteins in bond distance, I instead used the minus operator since it has been overloaded to return the distance between two atoms (Page 10 of the [http://www.biopython.org/DIST/docs/cookbook/biopdb_faq.pdf FAQ]). The average distance cited in the literature is 2.05A but other software packages and my own tests set 3.0A as a good threshold. Still, the user can set his own threshold manually.

The function returns an iterator with tuples of pairs of residues.

<python>
from Bio.PDB import PDBParser

p = PDBParser()
s = p.get_structure('example', '4PTI.pdb')

prot = s.as_protein()

for bond in prot.search_ss_bonds():
print bond

(<Residue CYS het= resseq=5 icode= >, <Residue CYS het= resseq=55 icode= >)
(<Residue CYS het= resseq=14 icode= >, <Residue CYS het= resseq=38 icode= >)
(<Residue CYS het= resseq=30 icode= >, <Residue CYS het= resseq=51 icode= >)
</python>

=== Extract Biological Unit ===

Added parsing for REMARK350 to parse_pdb_header since there was already a bit written for another REMARK section. This extracts the transformation matrices and the translation vector from the header, that is then fed to the Structure function. Each new rotated structure is created as a new MODEL. I chose this because crystal structures very rarely have more than one MODEL instance and also because NMR models don't have REMARK 350 that often (at least to my knowledge).

<python>
from Bio.PDB import PDBParser

p = PDBParser()

s1 = p.get_structure('a', '4PTI.pdb')
s1.build_biological_unit()
'Processed 0 transformations on the structure.' # Identity matrix is ignored.

s2 = p.get_structure('b', 'homol_1bd8.pdb') # A homology model
s2.build_biological_unit()
'PDB File lacks appropriate REMARK 350 entries to build Biological Unit.'

s3 = p.get_structure('c', '1IHM.pdb')
s3.build_biological_unit()
'Processed 59 transformations on the structure.'
</python>

=== Hydrogenation of PDB files ===

Following discussion between the mentors and me, we decided that maybe it was better to not only include a webserver for this purpose but also a local algorithm. This would not limit the user when there he/she lacks an internet connection.

The interface for the WHATIF Protonation service has been implemented, although it should be regarded as **highly experimental** for now. Interfacing this server included writing a small parser for a PDBXML-like format, which is expected to have serious bugs in its initial versions. I ran some simple tests and it works. It doesn't support water molecules yet, nor any other molecules other than proteins. Such issues will be hopefully solved later on..

For those brave enough to want to test it (and help me debug it), here's an example usage.

<python>
from Bio.Struct.WWW import WHATIF
from Bio import Struct

server = WHATIF.WHATIF() # Performs a sort of PING to the server. Gracefully exits if the servers are down.

# Get the protein structure
structure = Struct.read('4PTI.pdb')
protein = structure.as_protein() # This excludes water molecules

# Upload the structure to the WHATIF server
# This should convert the structure from a Structure object to a string via tempfile and PDBIO
# I was having some issues uploading structures...

id = server.UploadPDB(protein)

# Protonate
# Returns a Structure Object / WARNING! Bug prone for now.

protein_h = server.PDBasXMLwithSymwithPolarH(id)

</python>

Regarding the local implementation, after much reading I settled on using PyMol's algorithm. It seems to allow for protonation of any structure, regardless of its nature (protein, DNA, etc). Its vectorial and matrix operations can likely be optimized with Numpy and Biopython's Vector.py module. This first implementation works for proteins only. I'll add general molecule support later.

<python>
from Bio import Struct
from Bio.Struct import Hydrogenate as H

s = Struct.read('1ctf.pdb')
p = s.as_protein()

prot = H.Hydrogenate_Protein()
prot.add_hydrogens(p)
</python>

=== Coarse Grain Structure ===

A Center of Mass function was developed first as part of a new module Bio.Struct.Geometry. It allows for calculation of the center of geometry (all masses are equal) and center of mass (taking into account elemental masses for the atoms). The masses are a new Atom object feature derived from [http://www.chem.qmul.ac.uk/iupac/AtWt/ this list] and from PyMol. Essentially, all atoms of a structure now get their mass defined when the Structure is created (check Atom.py and [http://lists.open-bio.org/pipermail/biopython-dev/2010-June/007880.html this thread] for details). This is obviously experimental.

To calculate the center of mass of any Entity (Structure, Model, Chain, Residue) or a List of Atoms:

<python>
from Bio.Struct.Geometry import center_of_mass
from Bio import Struct

s = Struct.read('4PTI.pdb')

print center_of_mass.__doc__

Returns gravitic or geometric center of mass of an Entity.
Geometric assumes all masses are equal (geometric=True)
Defaults to Gravitic.

print center_of_mass(s)
[14.833301303933874, 21.431581746366263, 4.1218478418007134]

print center_of_mass(s, geometric=True)
[14.805324902127458, 21.365571977563405, 4.1108949403803985]
</python>

As of now, 3 CG models are supported.

1) CA-Trace
2) ENCAD 3-point model (CA, O, Side Chain bead)
3) MARTINI protein model (BB, Side Chain points [S1 to S4])

An example, picking up the s Structure from above:

<python>
p = s.as_protein() # To expose the CG method

ca_trace = p.coarse_grain()

# One atom per residue
print ( len(list(p.get_residues())) == len(list(ca_trace.get_atoms())) )
True

cg_encad = p.coarse_grain('ENCAD_3P')

for residue in cg_encad.get_residues():
print residue.resname, residue.child_list

ARG [<Atom CA>, <Atom O>, <Atom CMA>]
PRO [<Atom CA>, <Atom O>, <Atom CMA>]
ASP [<Atom CA>, <Atom O>, <Atom CMA>]
PHE [<Atom CA>, <Atom O>, <Atom CMA>]
CYS [<Atom CA>, <Atom O>, <Atom CMA>]
LEU [<Atom CA>, <Atom O>, <Atom CMA>]
GLU [<Atom CA>, <Atom O>, <Atom CMA>]
PRO [<Atom CA>, <Atom O>, <Atom CMA>]
PRO [<Atom CA>, <Atom O>, <Atom CMA>]
TYR [<Atom CA>, <Atom O>, <Atom CMA>]
.....
CYS [<Atom CA>, <Atom O>, <Atom CMA>]
GLY [<Atom CA>, <Atom O>]
GLY [<Atom CA>, <Atom O>]
ALA [<Atom CA>, <Atom O>, <Atom CMA>]

cg_martini = p.coarse_grain('MARTINI')

for residue in cg_martini.get_residues():
print residue.resname, residue.child_list

ARG [<Atom BB>, <Atom S1>, <Atom S2>]
PRO [<Atom BB>, <Atom S1>]
ASP [<Atom BB>, <Atom S1>]
PHE [<Atom BB>, <Atom S1>, <Atom S2>, <Atom S3>]
CYS [<Atom BB>, <Atom S1>]
LEU [<Atom BB>, <Atom S1>]
GLU [<Atom BB>, <Atom S1>]
PRO [<Atom BB>, <Atom S1>]
PRO [<Atom BB>, <Atom S1>]
TYR [<Atom BB>, <Atom S1>, <Atom S2>, <Atom S3>]
......
CYS [<Atom BB>, <Atom S1>]
GLY [<Atom BB>]
GLY [<Atom BB>]
ALA [<Atom BB>]
</python>

=== Removal of disordered atoms ===

Implement as part of Structure.py and based loosely on the [http://www.biopython.org/wiki/Remove_PDB_disordered_atoms contribution of Ramon Crehuet]. The DisorderedAtom objects are removed from the residue and a single Atom object is added corresponding to the location of the user's choice (keep_loc argument) which defaults to A.

An example, still keeping s from above:

<python>
s = s.remove_disordered_atoms(verbose=True)
0 residues were modified

# Now if we load a structure with disordered atoms
ds = Struct.read('1MC2.pdb')
ds.remove_disordered_atoms(verbose=True)
Residue TRP:1010 has 8 disordered atoms: CD1/CD2/NE1/CE2/CE3/CZ2/CZ3/CH2
Residue VAL:1018 has 3 disordered atoms: CB/CG1/CG2
Residue LEU:1024 has 4 disordered atoms: CB/CG/CD1/CD2
Residue ARG:1043 has 7 disordered atoms: CB/CG/CD/NE/CZ/NH1/NH2
Residue MET:1092 has 4 disordered atoms: CB/CG/SD/CE
Residue ARG:1107 has 7 disordered atoms: CB/CG/CD/NE/CZ/NH1/NH2
Residue GLU:1108 has 4 disordered atoms: CG/CD/OE1/OE2
Residue ASP:1111 has 4 disordered atoms: CB/CG/OD1/OD2
Residue SER:1116 has 1 disordered atoms: OG
Residue SER:1131 has 1 disordered atoms: O
10 residues were modified
</python>

=== Sequence Homologues from Structures ===

Biopython supports BLAST (local and remote through NCBI servers). We bridged both Bio.PDB and Bio.Blast modules to allow an easier search for sequence homologues. For now, it supports remote BLAST through Bio.Blast.NCBIWWW and functions as a blackbox - i.e. users cannot change any search parameter. If one wants to fully use BLAST he/she should use the regular BLAST module. This is just a convenience function.

It is accessible only to Protein objects. It queries the PDB subset database of NCBI BLAST servers with the Structure object's sequence, auto-adjusting parameters for short sequences (less than 15 residues).

It returns a list ranked by Expectation Value with some informational values (e-value, identities, positives, gaps), the PDB code of the match, and the alignment.

<python>from Bio import Struct

s = Struct.read('1A8O.pdb')
p = s.as_protein()

seq_homologues = p.find_seq_homologues()

for homologues in seq_homologues:
print homologues[0], homologues[1]
print homologues[-1]
print

2BUO 1.82482e-31
DIRQGPKEPFRDYVDRFYKTLRAEQASQEVKNW-TETLLVQNANPDCKTILKALGPGATLEE--TACQG
DIRQGPKEPFRDYVDRFYKTLRAEQASQEVKNW TETLLVQNANPDCKTILKALGPGATLEE TACQG
DIRQGPKEPFRDYVDRFYKTLRAEQASQEVKNWMTETLLVQNANPDCKTILKALGPGATLEEMMTACQG

1AUM 1.82482e-31
DIRQGPKEPFRDYVDRFYKTLRAEQASQEVKNW-TETLLVQNANPDCKTILKALGPGATLEE--TACQG
DIRQGPKEPFRDYVDRFYKTLRAEQASQEVKNW TETLLVQNANPDCKTILKALGPGATLEE TACQG
DIRQGPKEPFRDYVDRFYKTLRAEQASQEVKNWMTETLLVQNANPDCKTILKALGPGATLEEMMTACQG

.....</python>

=== Support for MODELLER PIR format in SeqIO ===

MODELLER PIR format support was added to SeqIO as 'mpir'. Currently, the format can be read but not written. An example of the format follows, as well as an example of the parser's usage.

<pre>
>P1;5fd1
structureX:5fd1:1 :A:106 :A:ferredoxin:Azotobacter vinelandii: 1.90: 0.19
AFVVTDNCIKCKYTDCVEVCPVDCFYEGPNFLVIHPDECIDCALCEPECPAQAIFSEDEVPEDMQEFIQLNAELA
EVWPNITEKKDPLPDAEDWDGVKGKLQHLER*
</pre>

<python>from Bio import SeqIO

handle = open('test_pir.txt')

records = SeqIO.parse(handle, 'mpir')

for i in records:
print i

ID: 5fd1
Name: 5fd1
Description: ferredoxin
Number of features: 0
/r_factor= 0.19
/end_residue=106
/initial_chain=a
/end_chain=a
/record_type=X-Ray Structure
/initial_residue=1
/resolution= 1.90
/source_organism=Azotobacter vinelandii
Seq('AFVVTDNCIKCKYTDCVEVCPVDCFYEGPNFLVIHPDECIDCALCEPECPAQAI...LER', ProteinAlphabet())</python>

GSOC2010 Joao

Joaor — Thu, 12 Aug 2010 04:52:50 GMT

Joaor: /* Support for MODELLER PIR format in SeqIO */

== Author & Mentors ==

[[User:Joaor|João Rodrigues]] anaryin@gmail.com

'''Mentors'''
: Eric Talevich
: Diana Jaunzeikare
: Peter Cock

== Abstract ==

Biopython is a very popular library in Bioinformatics and Computational Biology. Its Bio.PDB module, originally developed by Thomas Hamelryck, is a simple yet powerful tool for structural biologists. Although it provides a reliable PDB parser feature and it allows several calculations (Neighbour Search, RMS) to be made on macromolecules, it still lacks a number of features that are part of a researcher's daily routine. Probing for disulphide bridges in a structure and adding polar hydrogen atoms accordingly are two examples that can be incorporated in Bio.PDB, given the module's clever structure and good overall organisation. Cosmetic operations such as chain removal and residue renaming – to account for the different existing nomenclatures – and renumbering would also be greatly appreciated by the community.

Another aspect that can be improved for Bio.PDB is a smooth integration/interaction layer for heavy-weights in macromolecule simulation such as MODELLER, GROMACS, AutoDock, HADDOCK. It could be argued that the easiest solution would be to code hooks to these packages' functions and routines. However, projects such as the recently developed [http://sbcb.bioch.ox.ac.uk/oliver/software/GromacsWrapper/html/edpdb.html edPDB] or the more complete [http://biskit.pasteur.fr/ Biskit library] render, in my opinion, such interfacing efforts redundant. Instead, I believe it to be more advantageous to include these software' input/output formats in Biopython's SeqIO and AlignIO modules. This, together with the creation of interfaces for model validation/structure checking services/software would allow Biopython to be used as a pre- and post-simulation tool. Eventually, it would pave the way for its inclusion in pipelines and workflows for structure modelling, molecular dynamics, and docking simulations.

== Project Schedule ==

The schedule below was organised to be flexible, which means that some features will likely be done early. Also, the weeks include documentation and unit testing efforts for the features, with extended periods for reviewing these efforts at the two points during the project (halfway, final week).

=== Community Bonding Period ===

*Getting familiar with development environment (Git Hub account, Git, Biopython's repository, Bug tracking system, etc)

*Gather scientific literature and discuss some of the to-be-implemented methods.

=== Week 1 [31st May - 6th June] ===

==== Renumbering residues of a structure ====

*Read SEQRES record to account for gaps
*Alternatively read ATOM records.

==== Probe disulphide bridges in the structure ====

*Via NeighbourSearch class
*Also use SSBOND in header

==== Extract Biological Unit ====

*REMARK350 contains rotation and translation information
*If REMARK is absent, do nothing.

=== Week 2 [7th – 13th June] ===

==== Structure Hydrogenation ====

*Add all/polar hydrogens through interface with WHATIF server.
*Optionally define a set pH
** [http://www3.interscience.wiley.com/journal/112117957/abstract pKa values algorithm]

==== Hydrogenation Report ====

*Produces a brief list of polar hydrogen atoms in the structure.
** Chain | Residue [number] | Atom

=== Weeks 3-5 [14th June- 4th July] ===

==== Removal of disordered atoms ====

*[[Remove_PDB_disordered_atoms|Solution proposed in the Biopython wiki]]

==== Residue name normalisation ====

*Build conversion table from different nomenclatures (research them during c.bonding period )
*Write function to make a given structure compliant with a given software nomenclature:
** Amber
** CNS/HADDOCK
** GROMACS

==== Coarse Grain Structure ====

*Implement function to reduce complexity of a structure
** 1pt*c-alpha
** 2pt*c-alpha / c-beta
** 3pt*c-alpha / c-beta / side-chain pseudo-centroid OR side-chain centroid

=== Week 6 (Mid-Term) [5th - 11th July] ===

: Testing and consolidating the features thoroughly.
: Write documentation & examples for each feature, to be included in Biopython's Wiki and Bio.PDB's FAQ.
: Mid-term Evaluations. Discussing with mentors current state of project and adjust following schedule to comply with project's needs.

=== Week 7 [12th - 19th July] ===

==== Add support for MODELLER's PIR format to Biopython ====

*[http://www.salilab.org/modeller/manual/node445.html#alignmentformat Format Description]
*SeqIO
*AlignIO

==== Allow conversion of Structure Object to Sequence Object ====

*Based on Bio.PDB.Polypeptide function

=== Weeks 8-10 [20th July - 9th August] ===

==== Add Sequence/Structure Homology functions ====

*Create call to Biopython's BLAST interfaces
** Allow direct blast from structure object ( e.g. protein.find_homoseq() )
** Returns list of tuples with E-Value *Dictionary (name, length of alignment, etc..)
*Create interface with structural homology web services
** e.g. [http://ekhidna.biocenter.helsinki.fi/dali_server/ Dali server]
** Return list of tuples with Z-Score*Dictionary (name, etc...)

==== Implement basic structure validation checks ====

*Via NeighbourSearch class
** Same Charge contacts
** Atom Clashes
*Via ResidueDepth Class
** Buried Charges
*Interface WHATIF PDBReport web service
** Parse WARNING and ERROR messages

=== Week 11 [10th - 17th August] ===

==== Reviewing documentation, code, write tests for new functions. ====

== Project Code ==

Hosted at [http://github.com/JoaoRodrigues/biopython/tree/GSOC2010 this GitHub branch]

== Project Progress ==

Since I'm adding some methods that are useful/logical only for proteins, having them exposed in Structure.py for every molecule could be misleading. We decided then to add a 'as_protein()' method that allows protein-specific methods to be accessed. The following example demonstrates how this call works. Note how the "search_ss_bonds" method is absent from dir(s) but not from dir(prot).

<python>
from Bio.PDB import PDBParser

p = PDBParser()
s = p.get_structure('example', '4PTI.pdb')

dir(s)
# Cut for viewing purposes
['__doc__', ... , 'renumber_residues', 'set_parent', 'xtra']

prot = s.as_protein()

dir(prot)

['__doc__', ... , 'renumber_residues', 'search_ss_bonds', 'set_parent', 'xtra']
</python>

=== Renumbering residues of a structure ===

Since parse_pdb_header is far from optimal and is likely to change in the future, I opted to forfeit reading SEQREQ records to account for gaps. However, ignoring this information and renumbering based on ATOM records would make us lose information on gaps. I opted to subtract the first residue number-1 to all residues thus making the numbering start in 1 and still keep gaps. I also added an argument (start) to allow the user to set which number to start the counting from.

Example:

<python>
from Bio.PDB import PDBParser

p = PDBParser()
s = p.get_structure('example', '1IHM.pdb')

print list(s.get_residues())[0]
<Residue ASP het= resseq=1029 icode= >

s.renumber_residues()
print list(s.get_residues())[0]
<Residue ASP het= resseq=1 icode= >
</python>

=== Probe disulphide bridges in the structure ===

The same rationale from SEQRES applies for the exclusion of looking up SSBOND. Also, instead of using NeighborSearch to look for pairs of cysteins in bond distance, I instead used the minus operator since it has been overloaded to return the distance between two atoms (Page 10 of the [http://www.biopython.org/DIST/docs/cookbook/biopdb_faq.pdf FAQ]). The average distance cited in the literature is 2.05A but other software packages and my own tests set 3.0A as a good threshold. Still, the user can set his own threshold manually.

The function returns an iterator with tuples of pairs of residues.

<python>
from Bio.PDB import PDBParser

p = PDBParser()
s = p.get_structure('example', '4PTI.pdb')

prot = s.as_protein()

for bond in prot.search_ss_bonds():
print bond

(<Residue CYS het= resseq=5 icode= >, <Residue CYS het= resseq=55 icode= >)
(<Residue CYS het= resseq=14 icode= >, <Residue CYS het= resseq=38 icode= >)
(<Residue CYS het= resseq=30 icode= >, <Residue CYS het= resseq=51 icode= >)
</python>

=== Extract Biological Unit ===

Added parsing for REMARK350 to parse_pdb_header since there was already a bit written for another REMARK section. This extracts the transformation matrices and the translation vector from the header, that is then fed to the Structure function. Each new rotated structure is created as a new MODEL. I chose this because crystal structures very rarely have more than one MODEL instance and also because NMR models don't have REMARK 350 that often (at least to my knowledge).

<python>
from Bio.PDB import PDBParser

p = PDBParser()

s1 = p.get_structure('a', '4PTI.pdb')
s1.build_biological_unit()
'Processed 0 transformations on the structure.' # Identity matrix is ignored.

s2 = p.get_structure('b', 'homol_1bd8.pdb') # A homology model
s2.build_biological_unit()
'PDB File lacks appropriate REMARK 350 entries to build Biological Unit.'

s3 = p.get_structure('c', '1IHM.pdb')
s3.build_biological_unit()
'Processed 59 transformations on the structure.'
</python>

=== Hydrogenation of PDB files ===

Following discussion between the mentors and me, we decided that maybe it was better to not only include a webserver for this purpose but also a local algorithm. This would not limit the user when there he/she lacks an internet connection.

The interface for the WHATIF Protonation service has been implemented, although it should be regarded as **highly experimental** for now. Interfacing this server included writing a small parser for a PDBXML-like format, which is expected to have serious bugs in its initial versions. I ran some simple tests and it works. It doesn't support water molecules yet, nor any other molecules other than proteins. Such issues will be hopefully solved later on..

For those brave enough to want to test it (and help me debug it), here's an example usage.

<python>
from Bio.Struct.WWW import WHATIF
from Bio import Struct

server = WHATIF.WHATIF() # Performs a sort of PING to the server. Gracefully exits if the servers are down.

# Get the protein structure
structure = Struct.read('4PTI.pdb')
protein = structure.as_protein() # This excludes water molecules

# Upload the structure to the WHATIF server
# This should convert the structure from a Structure object to a string via tempfile and PDBIO
# I was having some issues uploading structures...

id = server.UploadPDB(protein)

# Protonate
# Returns a Structure Object / WARNING! Bug prone for now.

protein_h = server.PDBasXMLwithSymwithPolarH(id)

</python>

Regarding the local implementation, after much reading I settled on using PyMol's algorithm. It seems to allow for protonation of any structure, regardless of its nature (protein, DNA, etc). Its vectorial and matrix operations can likely be optimized with Numpy and Biopython's Vector.py module. This first implementation works for proteins only. I'll add general molecule support later.

<python>
from Bio import Struct
from Bio.Struct import Hydrogenate as H

s = Struct.read('1ctf.pdb')
p = s.as_protein()

prot = H.Hydrogenate_Protein()
prot.add_hydrogens(p)
</python>

=== Coarse Grain Structure ===

A Center of Mass function was developed first as part of a new module Bio.Struct.Geometry. It allows for calculation of the center of geometry (all masses are equal) and center of mass (taking into account elemental masses for the atoms). The masses are a new Atom object feature derived from [http://www.chem.qmul.ac.uk/iupac/AtWt/ this list] and from PyMol. Essentially, all atoms of a structure now get their mass defined when the Structure is created (check Atom.py and [http://lists.open-bio.org/pipermail/biopython-dev/2010-June/007880.html this thread] for details). This is obviously experimental.

To calculate the center of mass of any Entity (Structure, Model, Chain, Residue) or a List of Atoms:

<python>
from Bio.Struct.Geometry import center_of_mass
from Bio import Struct

s = Struct.read('4PTI.pdb')

print center_of_mass.__doc__

Returns gravitic or geometric center of mass of an Entity.
Geometric assumes all masses are equal (geometric=True)
Defaults to Gravitic.

print center_of_mass(s)
[14.833301303933874, 21.431581746366263, 4.1218478418007134]

print center_of_mass(s, geometric=True)
[14.805324902127458, 21.365571977563405, 4.1108949403803985]
</python>

As of now, 3 CG models are supported.

1) CA-Trace
2) ENCAD 3-point model (CA, O, Side Chain bead)
3) MARTINI protein model (BB, Side Chain points [S1 to S4])

An example, picking up the s Structure from above:

<python>
p = s.as_protein() # To expose the CG method

ca_trace = p.coarse_grain()

# One atom per residue
print ( len(list(p.get_residues())) == len(list(ca_trace.get_atoms())) )
True

cg_encad = p.coarse_grain('ENCAD_3P')

for residue in cg_encad.get_residues():
print residue.resname, residue.child_list

ARG [<Atom CA>, <Atom O>, <Atom CMA>]
PRO [<Atom CA>, <Atom O>, <Atom CMA>]
ASP [<Atom CA>, <Atom O>, <Atom CMA>]
PHE [<Atom CA>, <Atom O>, <Atom CMA>]
CYS [<Atom CA>, <Atom O>, <Atom CMA>]
LEU [<Atom CA>, <Atom O>, <Atom CMA>]
GLU [<Atom CA>, <Atom O>, <Atom CMA>]
PRO [<Atom CA>, <Atom O>, <Atom CMA>]
PRO [<Atom CA>, <Atom O>, <Atom CMA>]
TYR [<Atom CA>, <Atom O>, <Atom CMA>]
.....
CYS [<Atom CA>, <Atom O>, <Atom CMA>]
GLY [<Atom CA>, <Atom O>]
GLY [<Atom CA>, <Atom O>]
ALA [<Atom CA>, <Atom O>, <Atom CMA>]

cg_martini = p.coarse_grain('MARTINI')

for residue in cg_martini.get_residues():
print residue.resname, residue.child_list

ARG [<Atom BB>, <Atom S1>, <Atom S2>]
PRO [<Atom BB>, <Atom S1>]
ASP [<Atom BB>, <Atom S1>]
PHE [<Atom BB>, <Atom S1>, <Atom S2>, <Atom S3>]
CYS [<Atom BB>, <Atom S1>]
LEU [<Atom BB>, <Atom S1>]
GLU [<Atom BB>, <Atom S1>]
PRO [<Atom BB>, <Atom S1>]
PRO [<Atom BB>, <Atom S1>]
TYR [<Atom BB>, <Atom S1>, <Atom S2>, <Atom S3>]
......
CYS [<Atom BB>, <Atom S1>]
GLY [<Atom BB>]
GLY [<Atom BB>]
ALA [<Atom BB>]
</python>

=== Removal of disordered atoms ===

Implement as part of Structure.py and based loosely on the [http://www.biopython.org/wiki/Remove_PDB_disordered_atoms contribution of Ramon Crehuet]. The DisorderedAtom objects are removed from the residue and a single Atom object is added corresponding to the location of the user's choice (keep_loc argument) which defaults to A.

An example, still keeping s from above:

<python>
s = s.remove_disordered_atoms(verbose=True)
0 residues were modified

# Now if we load a structure with disordered atoms
ds = Struct.read('1MC2.pdb')
ds.remove_disordered_atoms(verbose=True)
Residue TRP:1010 has 8 disordered atoms: CD1/CD2/NE1/CE2/CE3/CZ2/CZ3/CH2
Residue VAL:1018 has 3 disordered atoms: CB/CG1/CG2
Residue LEU:1024 has 4 disordered atoms: CB/CG/CD1/CD2
Residue ARG:1043 has 7 disordered atoms: CB/CG/CD/NE/CZ/NH1/NH2
Residue MET:1092 has 4 disordered atoms: CB/CG/SD/CE
Residue ARG:1107 has 7 disordered atoms: CB/CG/CD/NE/CZ/NH1/NH2
Residue GLU:1108 has 4 disordered atoms: CG/CD/OE1/OE2
Residue ASP:1111 has 4 disordered atoms: CB/CG/OD1/OD2
Residue SER:1116 has 1 disordered atoms: OG
Residue SER:1131 has 1 disordered atoms: O
10 residues were modified
</python>

=== Sequence Homologues from Structures ===

Biopython supports BLAST (local and remote through NCBI servers). We bridged both Bio.PDB and Bio.Blast modules to allow an easier search for sequence homologues. For now, it supports remote BLAST through Bio.Blast.NCBIWWW and functions as a blackbox - i.e. users cannot change any search parameter. If one wants to fully use BLAST he/she should use the regular BLAST module. This is just a convenience function.

It is accessible only to Protein objects. It queries the PDB subset database of NCBI BLAST servers with the Structure object's sequence, auto-adjusting parameters for short sequences (less than 15 residues).

It returns a list ranked by Expectation Value with some informational values (e-value, identities, positives, gaps), the PDB code of the match, and the alignment.

<python>from Bio import Struct

s = Struct.read('1A8O.pdb')
p = s.as_protein()

seq_homologues = p.find_seq_homologues()

for homologues in seq_homologues:
print homologues[0], homologues[1]
print homologues[-1]
print

2BUO 1.82482e-31
DIRQGPKEPFRDYVDRFYKTLRAEQASQEVKNW-TETLLVQNANPDCKTILKALGPGATLEE--TACQG
DIRQGPKEPFRDYVDRFYKTLRAEQASQEVKNW TETLLVQNANPDCKTILKALGPGATLEE TACQG
DIRQGPKEPFRDYVDRFYKTLRAEQASQEVKNWMTETLLVQNANPDCKTILKALGPGATLEEMMTACQG

1AUM 1.82482e-31
DIRQGPKEPFRDYVDRFYKTLRAEQASQEVKNW-TETLLVQNANPDCKTILKALGPGATLEE--TACQG
DIRQGPKEPFRDYVDRFYKTLRAEQASQEVKNW TETLLVQNANPDCKTILKALGPGATLEE TACQG
DIRQGPKEPFRDYVDRFYKTLRAEQASQEVKNWMTETLLVQNANPDCKTILKALGPGATLEEMMTACQG

.....</python>

=== Support for MODELLER PIR format in SeqIO ===

MODELLER PIR format support was added to SeqIO as 'mpir'. Currently, the format can be read but not written. An example of the format follows, as well as an example of the parser's usage.

<code>
>P1;5fd1
structureX:5fd1:1 :A:106 :A:ferredoxin:Azotobacter vinelandii: 1.90: 0.19
AFVVTDNCIKCKYTDCVEVCPVDCFYEGPNFLVIHPDECIDCALCEPECPAQAIFSEDEVPEDMQEFIQLNAELA
EVWPNITEKKDPLPDAEDWDGVKGKLQHLER*
</code>

<python>from Bio import SeqIO

handle = open('test_pir.txt')

records = SeqIO.parse(handle, 'mpir')

for i in records:
print i

ID: 5fd1
Name: 5fd1
Description: ferredoxin
Number of features: 0
/r_factor= 0.19
/end_residue=106
/initial_chain=a
/end_chain=a
/record_type=X-Ray Structure
/initial_residue=1
/resolution= 1.90
/source_organism=Azotobacter vinelandii
Seq('AFVVTDNCIKCKYTDCVEVCPVDCFYEGPNFLVIHPDECIDCALCEPECPAQAI...LER', ProteinAlphabet())</python>

GSOC2010 Joao

Joaor — Thu, 12 Aug 2010 04:49:44 GMT

Joaor: Added MODELLER PIR support to SeqIO

== Author & Mentors ==

[[User:Joaor|João Rodrigues]] anaryin@gmail.com

'''Mentors'''
: Eric Talevich
: Diana Jaunzeikare
: Peter Cock

== Abstract ==

Biopython is a very popular library in Bioinformatics and Computational Biology. Its Bio.PDB module, originally developed by Thomas Hamelryck, is a simple yet powerful tool for structural biologists. Although it provides a reliable PDB parser feature and it allows several calculations (Neighbour Search, RMS) to be made on macromolecules, it still lacks a number of features that are part of a researcher's daily routine. Probing for disulphide bridges in a structure and adding polar hydrogen atoms accordingly are two examples that can be incorporated in Bio.PDB, given the module's clever structure and good overall organisation. Cosmetic operations such as chain removal and residue renaming – to account for the different existing nomenclatures – and renumbering would also be greatly appreciated by the community.

Another aspect that can be improved for Bio.PDB is a smooth integration/interaction layer for heavy-weights in macromolecule simulation such as MODELLER, GROMACS, AutoDock, HADDOCK. It could be argued that the easiest solution would be to code hooks to these packages' functions and routines. However, projects such as the recently developed [http://sbcb.bioch.ox.ac.uk/oliver/software/GromacsWrapper/html/edpdb.html edPDB] or the more complete [http://biskit.pasteur.fr/ Biskit library] render, in my opinion, such interfacing efforts redundant. Instead, I believe it to be more advantageous to include these software' input/output formats in Biopython's SeqIO and AlignIO modules. This, together with the creation of interfaces for model validation/structure checking services/software would allow Biopython to be used as a pre- and post-simulation tool. Eventually, it would pave the way for its inclusion in pipelines and workflows for structure modelling, molecular dynamics, and docking simulations.

== Project Schedule ==

The schedule below was organised to be flexible, which means that some features will likely be done early. Also, the weeks include documentation and unit testing efforts for the features, with extended periods for reviewing these efforts at the two points during the project (halfway, final week).

=== Community Bonding Period ===

*Getting familiar with development environment (Git Hub account, Git, Biopython's repository, Bug tracking system, etc)

*Gather scientific literature and discuss some of the to-be-implemented methods.

=== Week 1 [31st May - 6th June] ===

==== Renumbering residues of a structure ====

*Read SEQRES record to account for gaps
*Alternatively read ATOM records.

==== Probe disulphide bridges in the structure ====

*Via NeighbourSearch class
*Also use SSBOND in header

==== Extract Biological Unit ====

*REMARK350 contains rotation and translation information
*If REMARK is absent, do nothing.

=== Week 2 [7th – 13th June] ===

==== Structure Hydrogenation ====

*Add all/polar hydrogens through interface with WHATIF server.
*Optionally define a set pH
** [http://www3.interscience.wiley.com/journal/112117957/abstract pKa values algorithm]

==== Hydrogenation Report ====

*Produces a brief list of polar hydrogen atoms in the structure.
** Chain | Residue [number] | Atom

=== Weeks 3-5 [14th June- 4th July] ===

==== Removal of disordered atoms ====

*[[Remove_PDB_disordered_atoms|Solution proposed in the Biopython wiki]]

==== Residue name normalisation ====

*Build conversion table from different nomenclatures (research them during c.bonding period )
*Write function to make a given structure compliant with a given software nomenclature:
** Amber
** CNS/HADDOCK
** GROMACS

==== Coarse Grain Structure ====

*Implement function to reduce complexity of a structure
** 1pt*c-alpha
** 2pt*c-alpha / c-beta
** 3pt*c-alpha / c-beta / side-chain pseudo-centroid OR side-chain centroid

=== Week 6 (Mid-Term) [5th - 11th July] ===

: Testing and consolidating the features thoroughly.
: Write documentation & examples for each feature, to be included in Biopython's Wiki and Bio.PDB's FAQ.
: Mid-term Evaluations. Discussing with mentors current state of project and adjust following schedule to comply with project's needs.

=== Week 7 [12th - 19th July] ===

==== Add support for MODELLER's PIR format to Biopython ====

*[http://www.salilab.org/modeller/manual/node445.html#alignmentformat Format Description]
*SeqIO
*AlignIO

==== Allow conversion of Structure Object to Sequence Object ====

*Based on Bio.PDB.Polypeptide function

=== Weeks 8-10 [20th July - 9th August] ===

==== Add Sequence/Structure Homology functions ====

*Create call to Biopython's BLAST interfaces
** Allow direct blast from structure object ( e.g. protein.find_homoseq() )
** Returns list of tuples with E-Value *Dictionary (name, length of alignment, etc..)
*Create interface with structural homology web services
** e.g. [http://ekhidna.biocenter.helsinki.fi/dali_server/ Dali server]
** Return list of tuples with Z-Score*Dictionary (name, etc...)

==== Implement basic structure validation checks ====

*Via NeighbourSearch class
** Same Charge contacts
** Atom Clashes
*Via ResidueDepth Class
** Buried Charges
*Interface WHATIF PDBReport web service
** Parse WARNING and ERROR messages

=== Week 11 [10th - 17th August] ===

==== Reviewing documentation, code, write tests for new functions. ====

== Project Code ==

Hosted at [http://github.com/JoaoRodrigues/biopython/tree/GSOC2010 this GitHub branch]

== Project Progress ==

Since I'm adding some methods that are useful/logical only for proteins, having them exposed in Structure.py for every molecule could be misleading. We decided then to add a 'as_protein()' method that allows protein-specific methods to be accessed. The following example demonstrates how this call works. Note how the "search_ss_bonds" method is absent from dir(s) but not from dir(prot).

<python>
from Bio.PDB import PDBParser

p = PDBParser()
s = p.get_structure('example', '4PTI.pdb')

dir(s)
# Cut for viewing purposes
['__doc__', ... , 'renumber_residues', 'set_parent', 'xtra']

prot = s.as_protein()

dir(prot)

['__doc__', ... , 'renumber_residues', 'search_ss_bonds', 'set_parent', 'xtra']
</python>

=== Renumbering residues of a structure ===

Since parse_pdb_header is far from optimal and is likely to change in the future, I opted to forfeit reading SEQREQ records to account for gaps. However, ignoring this information and renumbering based on ATOM records would make us lose information on gaps. I opted to subtract the first residue number-1 to all residues thus making the numbering start in 1 and still keep gaps. I also added an argument (start) to allow the user to set which number to start the counting from.

Example:

<python>
from Bio.PDB import PDBParser

p = PDBParser()
s = p.get_structure('example', '1IHM.pdb')

print list(s.get_residues())[0]
<Residue ASP het= resseq=1029 icode= >

s.renumber_residues()
print list(s.get_residues())[0]
<Residue ASP het= resseq=1 icode= >
</python>

=== Probe disulphide bridges in the structure ===

The same rationale from SEQRES applies for the exclusion of looking up SSBOND. Also, instead of using NeighborSearch to look for pairs of cysteins in bond distance, I instead used the minus operator since it has been overloaded to return the distance between two atoms (Page 10 of the [http://www.biopython.org/DIST/docs/cookbook/biopdb_faq.pdf FAQ]). The average distance cited in the literature is 2.05A but other software packages and my own tests set 3.0A as a good threshold. Still, the user can set his own threshold manually.

The function returns an iterator with tuples of pairs of residues.

<python>
from Bio.PDB import PDBParser

p = PDBParser()
s = p.get_structure('example', '4PTI.pdb')

prot = s.as_protein()

for bond in prot.search_ss_bonds():
print bond

(<Residue CYS het= resseq=5 icode= >, <Residue CYS het= resseq=55 icode= >)
(<Residue CYS het= resseq=14 icode= >, <Residue CYS het= resseq=38 icode= >)
(<Residue CYS het= resseq=30 icode= >, <Residue CYS het= resseq=51 icode= >)
</python>

=== Extract Biological Unit ===

Added parsing for REMARK350 to parse_pdb_header since there was already a bit written for another REMARK section. This extracts the transformation matrices and the translation vector from the header, that is then fed to the Structure function. Each new rotated structure is created as a new MODEL. I chose this because crystal structures very rarely have more than one MODEL instance and also because NMR models don't have REMARK 350 that often (at least to my knowledge).

<python>
from Bio.PDB import PDBParser

p = PDBParser()

s1 = p.get_structure('a', '4PTI.pdb')
s1.build_biological_unit()
'Processed 0 transformations on the structure.' # Identity matrix is ignored.

s2 = p.get_structure('b', 'homol_1bd8.pdb') # A homology model
s2.build_biological_unit()
'PDB File lacks appropriate REMARK 350 entries to build Biological Unit.'

s3 = p.get_structure('c', '1IHM.pdb')
s3.build_biological_unit()
'Processed 59 transformations on the structure.'
</python>

=== Hydrogenation of PDB files ===

Following discussion between the mentors and me, we decided that maybe it was better to not only include a webserver for this purpose but also a local algorithm. This would not limit the user when there he/she lacks an internet connection.

The interface for the WHATIF Protonation service has been implemented, although it should be regarded as **highly experimental** for now. Interfacing this server included writing a small parser for a PDBXML-like format, which is expected to have serious bugs in its initial versions. I ran some simple tests and it works. It doesn't support water molecules yet, nor any other molecules other than proteins. Such issues will be hopefully solved later on..

For those brave enough to want to test it (and help me debug it), here's an example usage.

<python>
from Bio.Struct.WWW import WHATIF
from Bio import Struct

server = WHATIF.WHATIF() # Performs a sort of PING to the server. Gracefully exits if the servers are down.

# Get the protein structure
structure = Struct.read('4PTI.pdb')
protein = structure.as_protein() # This excludes water molecules

# Upload the structure to the WHATIF server
# This should convert the structure from a Structure object to a string via tempfile and PDBIO
# I was having some issues uploading structures...

id = server.UploadPDB(protein)

# Protonate
# Returns a Structure Object / WARNING! Bug prone for now.

protein_h = server.PDBasXMLwithSymwithPolarH(id)

</python>

Regarding the local implementation, after much reading I settled on using PyMol's algorithm. It seems to allow for protonation of any structure, regardless of its nature (protein, DNA, etc). Its vectorial and matrix operations can likely be optimized with Numpy and Biopython's Vector.py module. This first implementation works for proteins only. I'll add general molecule support later.

<python>
from Bio import Struct
from Bio.Struct import Hydrogenate as H

s = Struct.read('1ctf.pdb')
p = s.as_protein()

prot = H.Hydrogenate_Protein()
prot.add_hydrogens(p)
</python>

=== Coarse Grain Structure ===

A Center of Mass function was developed first as part of a new module Bio.Struct.Geometry. It allows for calculation of the center of geometry (all masses are equal) and center of mass (taking into account elemental masses for the atoms). The masses are a new Atom object feature derived from [http://www.chem.qmul.ac.uk/iupac/AtWt/ this list] and from PyMol. Essentially, all atoms of a structure now get their mass defined when the Structure is created (check Atom.py and [http://lists.open-bio.org/pipermail/biopython-dev/2010-June/007880.html this thread] for details). This is obviously experimental.

To calculate the center of mass of any Entity (Structure, Model, Chain, Residue) or a List of Atoms:

<python>
from Bio.Struct.Geometry import center_of_mass
from Bio import Struct

s = Struct.read('4PTI.pdb')

print center_of_mass.__doc__

Returns gravitic or geometric center of mass of an Entity.
Geometric assumes all masses are equal (geometric=True)
Defaults to Gravitic.

print center_of_mass(s)
[14.833301303933874, 21.431581746366263, 4.1218478418007134]

print center_of_mass(s, geometric=True)
[14.805324902127458, 21.365571977563405, 4.1108949403803985]
</python>

As of now, 3 CG models are supported.

1) CA-Trace
2) ENCAD 3-point model (CA, O, Side Chain bead)
3) MARTINI protein model (BB, Side Chain points [S1 to S4])

An example, picking up the s Structure from above:

<python>
p = s.as_protein() # To expose the CG method

ca_trace = p.coarse_grain()

# One atom per residue
print ( len(list(p.get_residues())) == len(list(ca_trace.get_atoms())) )
True

cg_encad = p.coarse_grain('ENCAD_3P')

for residue in cg_encad.get_residues():
print residue.resname, residue.child_list

ARG [<Atom CA>, <Atom O>, <Atom CMA>]
PRO [<Atom CA>, <Atom O>, <Atom CMA>]
ASP [<Atom CA>, <Atom O>, <Atom CMA>]
PHE [<Atom CA>, <Atom O>, <Atom CMA>]
CYS [<Atom CA>, <Atom O>, <Atom CMA>]
LEU [<Atom CA>, <Atom O>, <Atom CMA>]
GLU [<Atom CA>, <Atom O>, <Atom CMA>]
PRO [<Atom CA>, <Atom O>, <Atom CMA>]
PRO [<Atom CA>, <Atom O>, <Atom CMA>]
TYR [<Atom CA>, <Atom O>, <Atom CMA>]
.....
CYS [<Atom CA>, <Atom O>, <Atom CMA>]
GLY [<Atom CA>, <Atom O>]
GLY [<Atom CA>, <Atom O>]
ALA [<Atom CA>, <Atom O>, <Atom CMA>]

cg_martini = p.coarse_grain('MARTINI')

for residue in cg_martini.get_residues():
print residue.resname, residue.child_list

ARG [<Atom BB>, <Atom S1>, <Atom S2>]
PRO [<Atom BB>, <Atom S1>]
ASP [<Atom BB>, <Atom S1>]
PHE [<Atom BB>, <Atom S1>, <Atom S2>, <Atom S3>]
CYS [<Atom BB>, <Atom S1>]
LEU [<Atom BB>, <Atom S1>]
GLU [<Atom BB>, <Atom S1>]
PRO [<Atom BB>, <Atom S1>]
PRO [<Atom BB>, <Atom S1>]
TYR [<Atom BB>, <Atom S1>, <Atom S2>, <Atom S3>]
......
CYS [<Atom BB>, <Atom S1>]
GLY [<Atom BB>]
GLY [<Atom BB>]
ALA [<Atom BB>]
</python>

=== Removal of disordered atoms ===

Implement as part of Structure.py and based loosely on the [http://www.biopython.org/wiki/Remove_PDB_disordered_atoms contribution of Ramon Crehuet]. The DisorderedAtom objects are removed from the residue and a single Atom object is added corresponding to the location of the user's choice (keep_loc argument) which defaults to A.

An example, still keeping s from above:

<python>
s = s.remove_disordered_atoms(verbose=True)
0 residues were modified

# Now if we load a structure with disordered atoms
ds = Struct.read('1MC2.pdb')
ds.remove_disordered_atoms(verbose=True)
Residue TRP:1010 has 8 disordered atoms: CD1/CD2/NE1/CE2/CE3/CZ2/CZ3/CH2
Residue VAL:1018 has 3 disordered atoms: CB/CG1/CG2
Residue LEU:1024 has 4 disordered atoms: CB/CG/CD1/CD2
Residue ARG:1043 has 7 disordered atoms: CB/CG/CD/NE/CZ/NH1/NH2
Residue MET:1092 has 4 disordered atoms: CB/CG/SD/CE
Residue ARG:1107 has 7 disordered atoms: CB/CG/CD/NE/CZ/NH1/NH2
Residue GLU:1108 has 4 disordered atoms: CG/CD/OE1/OE2
Residue ASP:1111 has 4 disordered atoms: CB/CG/OD1/OD2
Residue SER:1116 has 1 disordered atoms: OG
Residue SER:1131 has 1 disordered atoms: O
10 residues were modified
</python>

=== Sequence Homologues from Structures ===

Biopython supports BLAST (local and remote through NCBI servers). We bridged both Bio.PDB and Bio.Blast modules to allow an easier search for sequence homologues. For now, it supports remote BLAST through Bio.Blast.NCBIWWW and functions as a blackbox - i.e. users cannot change any search parameter. If one wants to fully use BLAST he/she should use the regular BLAST module. This is just a convenience function.

It is accessible only to Protein objects. It queries the PDB subset database of NCBI BLAST servers with the Structure object's sequence, auto-adjusting parameters for short sequences (less than 15 residues).

It returns a list ranked by Expectation Value with some informational values (e-value, identities, positives, gaps), the PDB code of the match, and the alignment.

<python>from Bio import Struct

s = Struct.read('1A8O.pdb')
p = s.as_protein()

seq_homologues = p.find_seq_homologues()

for homologues in seq_homologues:
print homologues[0], homologues[1]
print homologues[-1]
print

2BUO 1.82482e-31
DIRQGPKEPFRDYVDRFYKTLRAEQASQEVKNW-TETLLVQNANPDCKTILKALGPGATLEE--TACQG
DIRQGPKEPFRDYVDRFYKTLRAEQASQEVKNW TETLLVQNANPDCKTILKALGPGATLEE TACQG
DIRQGPKEPFRDYVDRFYKTLRAEQASQEVKNWMTETLLVQNANPDCKTILKALGPGATLEEMMTACQG

1AUM 1.82482e-31
DIRQGPKEPFRDYVDRFYKTLRAEQASQEVKNW-TETLLVQNANPDCKTILKALGPGATLEE--TACQG
DIRQGPKEPFRDYVDRFYKTLRAEQASQEVKNW TETLLVQNANPDCKTILKALGPGATLEE TACQG
DIRQGPKEPFRDYVDRFYKTLRAEQASQEVKNWMTETLLVQNANPDCKTILKALGPGATLEEMMTACQG

.....</python>

=== Support for MODELLER PIR format in SeqIO ===

MODELLER PIR format support was added to SeqIO as 'mpir'. Currently, the format can be read but not written. An example of the format follows, as well as an example of the parser's usage.

>P1;5fd1
structureX:5fd1:1 :A:106 :A:ferredoxin:Azotobacter vinelandii: 1.90: 0.19
AFVVTDNCIKCKYTDCVEVCPVDCFYEGPNFLVIHPDECIDCALCEPECPAQAIFSEDEVPEDMQEFIQLNAELA
EVWPNITEKKDPLPDAEDWDGVKGKLQHLER*

<python>from Bio import SeqIO

handle = open('test_pir.txt')

records = SeqIO.parse(handle, 'mpir')

for i in records:
print i

ID: 5fd1
Name: 5fd1
Description: ferredoxin
Number of features: 0
/r_factor= 0.19
/end_residue=106
/initial_chain=a
/end_chain=a
/record_type=X-Ray Structure
/initial_residue=1
/resolution= 1.90
/source_organism=Azotobacter vinelandii
Seq('AFVVTDNCIKCKYTDCVEVCPVDCFYEGPNFLVIHPDECIDCALCEPECPAQAI...LER', ProteinAlphabet())</python>

GSOC2010 Joao

Joaor — Thu, 12 Aug 2010 04:44:12 GMT

Joaor: Added find_seq_homologues description

GSOC2010 Joao

Joaor — Thu, 12 Aug 2010 04:29:48 GMT

Joaor: /* Weeks 2-5 */

== Author & Mentors ==

[[User:Joaor|João Rodrigues]] anaryin@gmail.com

'''Mentors'''
: Eric Talevich
: Diana Jaunzeikare
: Peter Cock

== Abstract ==

Biopython is a very popular library in Bioinformatics and Computational Biology. Its Bio.PDB module, originally developed by Thomas Hamelryck, is a simple yet powerful tool for structural biologists. Although it provides a reliable PDB parser feature and it allows several calculations (Neighbour Search, RMS) to be made on macromolecules, it still lacks a number of features that are part of a researcher's daily routine. Probing for disulphide bridges in a structure and adding polar hydrogen atoms accordingly are two examples that can be incorporated in Bio.PDB, given the module's clever structure and good overall organisation. Cosmetic operations such as chain removal and residue renaming – to account for the different existing nomenclatures – and renumbering would also be greatly appreciated by the community.

Another aspect that can be improved for Bio.PDB is a smooth integration/interaction layer for heavy-weights in macromolecule simulation such as MODELLER, GROMACS, AutoDock, HADDOCK. It could be argued that the easiest solution would be to code hooks to these packages' functions and routines. However, projects such as the recently developed [http://sbcb.bioch.ox.ac.uk/oliver/software/GromacsWrapper/html/edpdb.html edPDB] or the more complete [http://biskit.pasteur.fr/ Biskit library] render, in my opinion, such interfacing efforts redundant. Instead, I believe it to be more advantageous to include these software' input/output formats in Biopython's SeqIO and AlignIO modules. This, together with the creation of interfaces for model validation/structure checking services/software would allow Biopython to be used as a pre- and post-simulation tool. Eventually, it would pave the way for its inclusion in pipelines and workflows for structure modelling, molecular dynamics, and docking simulations.

== Project Schedule ==

The schedule below was organised to be flexible, which means that some features will likely be done early. Also, the weeks include documentation and unit testing efforts for the features, with extended periods for reviewing these efforts at the two points during the project (halfway, final week).

=== Community Bonding Period ===

*Getting familiar with development environment (Git Hub account, Git, Biopython's repository, Bug tracking system, etc)

*Gather scientific literature and discuss some of the to-be-implemented methods.

=== Week 1 [31st May - 6th June] ===

==== Renumbering residues of a structure ====

*Read SEQRES record to account for gaps
*Alternatively read ATOM records.

==== Probe disulphide bridges in the structure ====

*Via NeighbourSearch class
*Also use SSBOND in header

==== Extract Biological Unit ====

*REMARK350 contains rotation and translation information
*If REMARK is absent, do nothing.

=== Week 2 [7th – 13th June] ===

==== Structure Hydrogenation ====

*Add all/polar hydrogens through interface with WHATIF server.
*Optionally define a set pH
** [http://www3.interscience.wiley.com/journal/112117957/abstract pKa values algorithm]

==== Hydrogenation Report ====

*Produces a brief list of polar hydrogen atoms in the structure.
** Chain | Residue [number] | Atom

=== Weeks 3-5 [14th June- 4th July] ===

==== Removal of disordered atoms ====

*[[Remove_PDB_disordered_atoms|Solution proposed in the Biopython wiki]]

==== Residue name normalisation ====

*Build conversion table from different nomenclatures (research them during c.bonding period )
*Write function to make a given structure compliant with a given software nomenclature:
** Amber
** CNS/HADDOCK
** GROMACS

==== Coarse Grain Structure ====

*Implement function to reduce complexity of a structure
** 1pt*c-alpha
** 2pt*c-alpha / c-beta
** 3pt*c-alpha / c-beta / side-chain pseudo-centroid OR side-chain centroid

=== Week 6 (Mid-Term) [5th - 11th July] ===

: Testing and consolidating the features thoroughly.
: Write documentation & examples for each feature, to be included in Biopython's Wiki and Bio.PDB's FAQ.
: Mid-term Evaluations. Discussing with mentors current state of project and adjust following schedule to comply with project's needs.

=== Week 7 [12th - 19th July] ===

==== Add support for MODELLER's PIR format to Biopython ====

*[http://www.salilab.org/modeller/manual/node445.html#alignmentformat Format Description]
*SeqIO
*AlignIO

==== Allow conversion of Structure Object to Sequence Object ====

*Based on Bio.PDB.Polypeptide function

=== Weeks 8-10 [20th July - 9th August] ===

==== Add Sequence/Structure Homology functions ====

*Create call to Biopython's BLAST interfaces
** Allow direct blast from structure object ( e.g. protein.find_homoseq() )
** Returns list of tuples with E-Value *Dictionary (name, length of alignment, etc..)
*Create interface with structural homology web services
** e.g. [http://ekhidna.biocenter.helsinki.fi/dali_server/ Dali server]
** Return list of tuples with Z-Score*Dictionary (name, etc...)

==== Implement basic structure validation checks ====

*Via NeighbourSearch class
** Same Charge contacts
** Atom Clashes
*Via ResidueDepth Class
** Buried Charges
*Interface WHATIF PDBReport web service
** Parse WARNING and ERROR messages

=== Week 11 [10th - 17th August] ===

==== Reviewing documentation, code, write tests for new functions. ====

== Project Code ==

Hosted at [http://github.com/JoaoRodrigues/biopython/tree/GSOC2010 this GitHub branch]

== Project Progress ==

Since I'm adding some methods that are useful/logical only for proteins, having them exposed in Structure.py for every molecule could be misleading. We decided then to add a 'as_protein()' method that allows protein-specific methods to be accessed. The following example demonstrates how this call works. Note how the "search_ss_bonds" method is absent from dir(s) but not from dir(prot).

<python>
from Bio.PDB import PDBParser

p = PDBParser()
s = p.get_structure('example', '4PTI.pdb')

dir(s)
# Cut for viewing purposes
['__doc__', ... , 'renumber_residues', 'set_parent', 'xtra']

prot = s.as_protein()

dir(prot)

['__doc__', ... , 'renumber_residues', 'search_ss_bonds', 'set_parent', 'xtra']
</python>

=== Renumbering residues of a structure ===

Since parse_pdb_header is far from optimal and is likely to change in the future, I opted to forfeit reading SEQREQ records to account for gaps. However, ignoring this information and renumbering based on ATOM records would make us lose information on gaps. I opted to subtract the first residue number-1 to all residues thus making the numbering start in 1 and still keep gaps. I also added an argument (start) to allow the user to set which number to start the counting from.

Example:

<python>
from Bio.PDB import PDBParser

p = PDBParser()
s = p.get_structure('example', '1IHM.pdb')

print list(s.get_residues())[0]
<Residue ASP het= resseq=1029 icode= >

s.renumber_residues()
print list(s.get_residues())[0]
<Residue ASP het= resseq=1 icode= >
</python>

=== Probe disulphide bridges in the structure ===

The same rationale from SEQRES applies for the exclusion of looking up SSBOND. Also, instead of using NeighborSearch to look for pairs of cysteins in bond distance, I instead used the minus operator since it has been overloaded to return the distance between two atoms (Page 10 of the [http://www.biopython.org/DIST/docs/cookbook/biopdb_faq.pdf FAQ]). The average distance cited in the literature is 2.05A but other software packages and my own tests set 3.0A as a good threshold. Still, the user can set his own threshold manually.

The function returns an iterator with tuples of pairs of residues.

<python>
from Bio.PDB import PDBParser

p = PDBParser()
s = p.get_structure('example', '4PTI.pdb')

prot = s.as_protein()

for bond in prot.search_ss_bonds():
print bond

(<Residue CYS het= resseq=5 icode= >, <Residue CYS het= resseq=55 icode= >)
(<Residue CYS het= resseq=14 icode= >, <Residue CYS het= resseq=38 icode= >)
(<Residue CYS het= resseq=30 icode= >, <Residue CYS het= resseq=51 icode= >)
</python>

=== Extract Biological Unit ===

Added parsing for REMARK350 to parse_pdb_header since there was already a bit written for another REMARK section. This extracts the transformation matrices and the translation vector from the header, that is then fed to the Structure function. Each new rotated structure is created as a new MODEL. I chose this because crystal structures very rarely have more than one MODEL instance and also because NMR models don't have REMARK 350 that often (at least to my knowledge).

<python>
from Bio.PDB import PDBParser

p = PDBParser()

s1 = p.get_structure('a', '4PTI.pdb')
s1.build_biological_unit()
'Processed 0 transformations on the structure.' # Identity matrix is ignored.

s2 = p.get_structure('b', 'homol_1bd8.pdb') # A homology model
s2.build_biological_unit()
'PDB File lacks appropriate REMARK 350 entries to build Biological Unit.'

s3 = p.get_structure('c', '1IHM.pdb')
s3.build_biological_unit()
'Processed 59 transformations on the structure.'
</python>

=== Hydrogenation of PDB files ===

Following discussion between the mentors and me, we decided that maybe it was better to not only include a webserver for this purpose but also a local algorithm. This would not limit the user when there he/she lacks an internet connection.

The interface for the WHATIF Protonation service has been implemented, although it should be regarded as **highly experimental** for now. Interfacing this server included writing a small parser for a PDBXML-like format, which is expected to have serious bugs in its initial versions. I ran some simple tests and it works. It doesn't support water molecules yet, nor any other molecules other than proteins. Such issues will be hopefully solved later on..

For those brave enough to want to test it (and help me debug it), here's an example usage.

<python>
from Bio.Struct.WWW import WHATIF
from Bio import Struct

server = WHATIF.WHATIF() # Performs a sort of PING to the server. Gracefully exits if the servers are down.

# Get the protein structure
structure = Struct.read('4PTI.pdb')
protein = structure.as_protein() # This excludes water molecules

# Upload the structure to the WHATIF server
# This should convert the structure from a Structure object to a string via tempfile and PDBIO
# I was having some issues uploading structures...

id = server.UploadPDB(protein)

# Protonate
# Returns a Structure Object / WARNING! Bug prone for now.

protein_h = server.PDBasXMLwithSymwithPolarH(id)

</python>

Regarding the local implementation, after much reading I settled on using PyMol's algorithm. It seems to allow for protonation of any structure, regardless of its nature (protein, DNA, etc). Its vectorial and matrix operations can likely be optimized with Numpy and Biopython's Vector.py module. This first implementation works for proteins only. I'll add general molecule support later.

<python>
from Bio import Struct
from Bio.Struct import Hydrogenate as H

s = Struct.read('1ctf.pdb')
p = s.as_protein()

prot = H.Hydrogenate_Protein()
prot.add_hydrogens(p)
</python>

=== Coarse Grain Structure ===

A Center of Mass function was developed first as part of a new module Bio.Struct.Geometry. It allows for calculation of the center of geometry (all masses are equal) and center of mass (taking into account elemental masses for the atoms). The masses are a new Atom object feature derived from [http://www.chem.qmul.ac.uk/iupac/AtWt/ this list] and from PyMol. Essentially, all atoms of a structure now get their mass defined when the Structure is created (check Atom.py and [http://lists.open-bio.org/pipermail/biopython-dev/2010-June/007880.html this thread] for details). This is obviously experimental.

To calculate the center of mass of any Entity (Structure, Model, Chain, Residue) or a List of Atoms:

<python>
from Bio.Struct.Geometry import center_of_mass
from Bio import Struct

s = Struct.read('4PTI.pdb')

print center_of_mass.__doc__

Returns gravitic or geometric center of mass of an Entity.
Geometric assumes all masses are equal (geometric=True)
Defaults to Gravitic.

print center_of_mass(s)
[14.833301303933874, 21.431581746366263, 4.1218478418007134]

print center_of_mass(s, geometric=True)
[14.805324902127458, 21.365571977563405, 4.1108949403803985]
</python>

As of now, 3 CG models are supported.

1) CA-Trace
2) ENCAD 3-point model (CA, O, Side Chain bead)
3) MARTINI protein model (BB, Side Chain points [S1 to S4])

An example, picking up the s Structure from above:

<python>
p = s.as_protein() # To expose the CG method

ca_trace = p.coarse_grain()

# One atom per residue
print ( len(list(p.get_residues())) == len(list(ca_trace.get_atoms())) )
True

cg_encad = p.coarse_grain('ENCAD_3P')

for residue in cg_encad.get_residues():
print residue.resname, residue.child_list

ARG [<Atom CA>, <Atom O>, <Atom CMA>]
PRO [<Atom CA>, <Atom O>, <Atom CMA>]
ASP [<Atom CA>, <Atom O>, <Atom CMA>]
PHE [<Atom CA>, <Atom O>, <Atom CMA>]
CYS [<Atom CA>, <Atom O>, <Atom CMA>]
LEU [<Atom CA>, <Atom O>, <Atom CMA>]
GLU [<Atom CA>, <Atom O>, <Atom CMA>]
PRO [<Atom CA>, <Atom O>, <Atom CMA>]
PRO [<Atom CA>, <Atom O>, <Atom CMA>]
TYR [<Atom CA>, <Atom O>, <Atom CMA>]
.....
CYS [<Atom CA>, <Atom O>, <Atom CMA>]
GLY [<Atom CA>, <Atom O>]
GLY [<Atom CA>, <Atom O>]
ALA [<Atom CA>, <Atom O>, <Atom CMA>]

cg_martini = p.coarse_grain('MARTINI')

for residue in cg_martini.get_residues():
print residue.resname, residue.child_list

ARG [<Atom BB>, <Atom S1>, <Atom S2>]
PRO [<Atom BB>, <Atom S1>]
ASP [<Atom BB>, <Atom S1>]
PHE [<Atom BB>, <Atom S1>, <Atom S2>, <Atom S3>]
CYS [<Atom BB>, <Atom S1>]
LEU [<Atom BB>, <Atom S1>]
GLU [<Atom BB>, <Atom S1>]
PRO [<Atom BB>, <Atom S1>]
PRO [<Atom BB>, <Atom S1>]
TYR [<Atom BB>, <Atom S1>, <Atom S2>, <Atom S3>]
......
CYS [<Atom BB>, <Atom S1>]
GLY [<Atom BB>]
GLY [<Atom BB>]
ALA [<Atom BB>]
</python>

=== Removal of disordered atoms ===

Implement as part of Structure.py and based loosely on the [http://www.biopython.org/wiki/Remove_PDB_disordered_atoms contribution of Ramon Crehuet]. The DisorderedAtom objects are removed from the residue and a single Atom object is added corresponding to the location of the user's choice (keep_loc argument) which defaults to A.

An example, still keeping s from above:

<python>
s = s.remove_disordered_atoms(verbose=True)
0 residues were modified

# Now if we load a structure with disordered atoms
ds = Struct.read('1MC2.pdb')
ds.remove_disordered_atoms(verbose=True)
Residue TRP:1010 has 8 disordered atoms: CD1/CD2/NE1/CE2/CE3/CZ2/CZ3/CH2
Residue VAL:1018 has 3 disordered atoms: CB/CG1/CG2
Residue LEU:1024 has 4 disordered atoms: CB/CG/CD1/CD2
Residue ARG:1043 has 7 disordered atoms: CB/CG/CD/NE/CZ/NH1/NH2
Residue MET:1092 has 4 disordered atoms: CB/CG/SD/CE
Residue ARG:1107 has 7 disordered atoms: CB/CG/CD/NE/CZ/NH1/NH2
Residue GLU:1108 has 4 disordered atoms: CG/CD/OE1/OE2
Residue ASP:1111 has 4 disordered atoms: CB/CG/OD1/OD2
Residue SER:1116 has 1 disordered atoms: OG
Residue SER:1131 has 1 disordered atoms: O
10 residues were modified
</python>

=== Current Development ===

Unit Tests are being developed for all these previous features. Some other features were already developed and are being tested. Additionally, other unplanned features have been developed and tested (check missing atoms).

Current work is focusing on the Nomenclature Normalization and the Homology features. We have established contact with the Dali Server developers and are working towards a solution with the Structural Homology feature. However, it seems that since results take too long to run, this is not a valuable addition to the project..

GSOC2010 Joao

Joaor — Thu, 12 Aug 2010 04:26:48 GMT

Joaor: /* Week 1 */

== Author & Mentors ==

[[User:Joaor|João Rodrigues]] anaryin@gmail.com

'''Mentors'''
: Eric Talevich
: Diana Jaunzeikare
: Peter Cock

== Abstract ==

Biopython is a very popular library in Bioinformatics and Computational Biology. Its Bio.PDB module, originally developed by Thomas Hamelryck, is a simple yet powerful tool for structural biologists. Although it provides a reliable PDB parser feature and it allows several calculations (Neighbour Search, RMS) to be made on macromolecules, it still lacks a number of features that are part of a researcher's daily routine. Probing for disulphide bridges in a structure and adding polar hydrogen atoms accordingly are two examples that can be incorporated in Bio.PDB, given the module's clever structure and good overall organisation. Cosmetic operations such as chain removal and residue renaming – to account for the different existing nomenclatures – and renumbering would also be greatly appreciated by the community.

Another aspect that can be improved for Bio.PDB is a smooth integration/interaction layer for heavy-weights in macromolecule simulation such as MODELLER, GROMACS, AutoDock, HADDOCK. It could be argued that the easiest solution would be to code hooks to these packages' functions and routines. However, projects such as the recently developed [http://sbcb.bioch.ox.ac.uk/oliver/software/GromacsWrapper/html/edpdb.html edPDB] or the more complete [http://biskit.pasteur.fr/ Biskit library] render, in my opinion, such interfacing efforts redundant. Instead, I believe it to be more advantageous to include these software' input/output formats in Biopython's SeqIO and AlignIO modules. This, together with the creation of interfaces for model validation/structure checking services/software would allow Biopython to be used as a pre- and post-simulation tool. Eventually, it would pave the way for its inclusion in pipelines and workflows for structure modelling, molecular dynamics, and docking simulations.

== Project Schedule ==

The schedule below was organised to be flexible, which means that some features will likely be done early. Also, the weeks include documentation and unit testing efforts for the features, with extended periods for reviewing these efforts at the two points during the project (halfway, final week).

=== Community Bonding Period ===

*Getting familiar with development environment (Git Hub account, Git, Biopython's repository, Bug tracking system, etc)

*Gather scientific literature and discuss some of the to-be-implemented methods.

=== Week 1 [31st May - 6th June] ===

==== Renumbering residues of a structure ====

*Read SEQRES record to account for gaps
*Alternatively read ATOM records.

==== Probe disulphide bridges in the structure ====

*Via NeighbourSearch class
*Also use SSBOND in header

==== Extract Biological Unit ====

*REMARK350 contains rotation and translation information
*If REMARK is absent, do nothing.

=== Week 2 [7th – 13th June] ===

==== Structure Hydrogenation ====

*Add all/polar hydrogens through interface with WHATIF server.
*Optionally define a set pH
** [http://www3.interscience.wiley.com/journal/112117957/abstract pKa values algorithm]

==== Hydrogenation Report ====

*Produces a brief list of polar hydrogen atoms in the structure.
** Chain | Residue [number] | Atom

=== Weeks 3-5 [14th June- 4th July] ===

==== Removal of disordered atoms ====

*[[Remove_PDB_disordered_atoms|Solution proposed in the Biopython wiki]]

==== Residue name normalisation ====

*Build conversion table from different nomenclatures (research them during c.bonding period )
*Write function to make a given structure compliant with a given software nomenclature:
** Amber
** CNS/HADDOCK
** GROMACS

==== Coarse Grain Structure ====

*Implement function to reduce complexity of a structure
** 1pt*c-alpha
** 2pt*c-alpha / c-beta
** 3pt*c-alpha / c-beta / side-chain pseudo-centroid OR side-chain centroid

=== Week 6 (Mid-Term) [5th - 11th July] ===

: Testing and consolidating the features thoroughly.
: Write documentation & examples for each feature, to be included in Biopython's Wiki and Bio.PDB's FAQ.
: Mid-term Evaluations. Discussing with mentors current state of project and adjust following schedule to comply with project's needs.

=== Week 7 [12th - 19th July] ===

==== Add support for MODELLER's PIR format to Biopython ====

*[http://www.salilab.org/modeller/manual/node445.html#alignmentformat Format Description]
*SeqIO
*AlignIO

==== Allow conversion of Structure Object to Sequence Object ====

*Based on Bio.PDB.Polypeptide function

=== Weeks 8-10 [20th July - 9th August] ===

==== Add Sequence/Structure Homology functions ====

*Create call to Biopython's BLAST interfaces
** Allow direct blast from structure object ( e.g. protein.find_homoseq() )
** Returns list of tuples with E-Value *Dictionary (name, length of alignment, etc..)
*Create interface with structural homology web services
** e.g. [http://ekhidna.biocenter.helsinki.fi/dali_server/ Dali server]
** Return list of tuples with Z-Score*Dictionary (name, etc...)

==== Implement basic structure validation checks ====

*Via NeighbourSearch class
** Same Charge contacts
** Atom Clashes
*Via ResidueDepth Class
** Buried Charges
*Interface WHATIF PDBReport web service
** Parse WARNING and ERROR messages

=== Week 11 [10th - 17th August] ===

==== Reviewing documentation, code, write tests for new functions. ====

== Project Code ==

Hosted at [http://github.com/JoaoRodrigues/biopython/tree/GSOC2010 this GitHub branch]

== Project Progress ==

Since I'm adding some methods that are useful/logical only for proteins, having them exposed in Structure.py for every molecule could be misleading. We decided then to add a 'as_protein()' method that allows protein-specific methods to be accessed. The following example demonstrates how this call works. Note how the "search_ss_bonds" method is absent from dir(s) but not from dir(prot).

<python>
from Bio.PDB import PDBParser

p = PDBParser()
s = p.get_structure('example', '4PTI.pdb')

dir(s)
# Cut for viewing purposes
['__doc__', ... , 'renumber_residues', 'set_parent', 'xtra']

prot = s.as_protein()

dir(prot)

['__doc__', ... , 'renumber_residues', 'search_ss_bonds', 'set_parent', 'xtra']
</python>

=== Renumbering residues of a structure ===

Since parse_pdb_header is far from optimal and is likely to change in the future, I opted to forfeit reading SEQREQ records to account for gaps. However, ignoring this information and renumbering based on ATOM records would make us lose information on gaps. I opted to subtract the first residue number-1 to all residues thus making the numbering start in 1 and still keep gaps. I also added an argument (start) to allow the user to set which number to start the counting from.

Example:

<python>
from Bio.PDB import PDBParser

p = PDBParser()
s = p.get_structure('example', '1IHM.pdb')

print list(s.get_residues())[0]
<Residue ASP het= resseq=1029 icode= >

s.renumber_residues()
print list(s.get_residues())[0]
<Residue ASP het= resseq=1 icode= >
</python>

=== Probe disulphide bridges in the structure ===

The same rationale from SEQRES applies for the exclusion of looking up SSBOND. Also, instead of using NeighborSearch to look for pairs of cysteins in bond distance, I instead used the minus operator since it has been overloaded to return the distance between two atoms (Page 10 of the [http://www.biopython.org/DIST/docs/cookbook/biopdb_faq.pdf FAQ]). The average distance cited in the literature is 2.05A but other software packages and my own tests set 3.0A as a good threshold. Still, the user can set his own threshold manually.

The function returns an iterator with tuples of pairs of residues.

<python>
from Bio.PDB import PDBParser

p = PDBParser()
s = p.get_structure('example', '4PTI.pdb')

prot = s.as_protein()

for bond in prot.search_ss_bonds():
print bond

(<Residue CYS het= resseq=5 icode= >, <Residue CYS het= resseq=55 icode= >)
(<Residue CYS het= resseq=14 icode= >, <Residue CYS het= resseq=38 icode= >)
(<Residue CYS het= resseq=30 icode= >, <Residue CYS het= resseq=51 icode= >)
</python>

=== Extract Biological Unit ===

Added parsing for REMARK350 to parse_pdb_header since there was already a bit written for another REMARK section. This extracts the transformation matrices and the translation vector from the header, that is then fed to the Structure function. Each new rotated structure is created as a new MODEL. I chose this because crystal structures very rarely have more than one MODEL instance and also because NMR models don't have REMARK 350 that often (at least to my knowledge).

<python>
from Bio.PDB import PDBParser

p = PDBParser()

s1 = p.get_structure('a', '4PTI.pdb')
s1.build_biological_unit()
'Processed 0 transformations on the structure.' # Identity matrix is ignored.

s2 = p.get_structure('b', 'homol_1bd8.pdb') # A homology model
s2.build_biological_unit()
'PDB File lacks appropriate REMARK 350 entries to build Biological Unit.'

s3 = p.get_structure('c', '1IHM.pdb')
s3.build_biological_unit()
'Processed 59 transformations on the structure.'
</python>

=== Weeks 2-5 ===

==== Hydrogenation of PDB files ====

Following discussion between the mentors and me, we decided that maybe it was better to not only include a webserver for this purpose but also a local algorithm. This would not limit the user when there he/she lacks an internet connection.

The interface for the WHATIF Protonation service has been implemented, although it should be regarded as **highly experimental** for now. Interfacing this server included writing a small parser for a PDBXML-like format, which is expected to have serious bugs in its initial versions. I ran some simple tests and it works. It doesn't support water molecules yet, nor any other molecules other than proteins. Such issues will be hopefully solved later on..

For those brave enough to want to test it (and help me debug it), here's an example usage.

<python>
from Bio.Struct.WWW import WHATIF
from Bio import Struct

server = WHATIF.WHATIF() # Performs a sort of PING to the server. Gracefully exits if the servers are down.

# Get the protein structure
structure = Struct.read('4PTI.pdb')
protein = structure.as_protein() # This excludes water molecules

# Upload the structure to the WHATIF server
# This should convert the structure from a Structure object to a string via tempfile and PDBIO
# I was having some issues uploading structures...

id = server.UploadPDB(protein)

# Protonate
# Returns a Structure Object / WARNING! Bug prone for now.

protein_h = server.PDBasXMLwithSymwithPolarH(id)

</python>

Regarding the local implementation, after much reading I settled on using PyMol's algorithm. It seems to allow for protonation of any structure, regardless of its nature (protein, DNA, etc). Its vectorial and matrix operations can likely be optimized with Numpy and Biopython's Vector.py module. This first implementation works for proteins only. I'll add general molecule support later.

<python>
from Bio import Struct
from Bio.Struct import Hydrogenate as H

s = Struct.read('1ctf.pdb')
p = s.as_protein()

prot = H.Hydrogenate_Protein()
prot.add_hydrogens(p)
</python>

==== Coarse Grain Structure ====

A Center of Mass function was developed first as part of a new module Bio.Struct.Geometry. It allows for calculation of the center of geometry (all masses are equal) and center of mass (taking into account elemental masses for the atoms). The masses are a new Atom object feature derived from [http://www.chem.qmul.ac.uk/iupac/AtWt/ this list] and from PyMol. Essentially, all atoms of a structure now get their mass defined when the Structure is created (check Atom.py and [http://lists.open-bio.org/pipermail/biopython-dev/2010-June/007880.html this thread] for details). This is obviously experimental.

To calculate the center of mass of any Entity (Structure, Model, Chain, Residue) or a List of Atoms:

<python>
from Bio.Struct.Geometry import center_of_mass
from Bio import Struct

s = Struct.read('4PTI.pdb')

print center_of_mass.__doc__

Returns gravitic or geometric center of mass of an Entity.
Geometric assumes all masses are equal (geometric=True)
Defaults to Gravitic.

print center_of_mass(s)
[14.833301303933874, 21.431581746366263, 4.1218478418007134]

print center_of_mass(s, geometric=True)
[14.805324902127458, 21.365571977563405, 4.1108949403803985]
</python>

As of now, 3 CG models are supported.

1) CA-Trace
2) ENCAD 3-point model (CA, O, Side Chain bead)
3) MARTINI protein model (BB, Side Chain points [S1 to S4])

An example, picking up the s Structure from above:

<python>
p = s.as_protein() # To expose the CG method

ca_trace = p.coarse_grain()

# One atom per residue
print ( len(list(p.get_residues())) == len(list(ca_trace.get_atoms())) )
True

cg_encad = p.coarse_grain('ENCAD_3P')

for residue in cg_encad.get_residues():
print residue.resname, residue.child_list

ARG [<Atom CA>, <Atom O>, <Atom CMA>]
PRO [<Atom CA>, <Atom O>, <Atom CMA>]
ASP [<Atom CA>, <Atom O>, <Atom CMA>]
PHE [<Atom CA>, <Atom O>, <Atom CMA>]
CYS [<Atom CA>, <Atom O>, <Atom CMA>]
LEU [<Atom CA>, <Atom O>, <Atom CMA>]
GLU [<Atom CA>, <Atom O>, <Atom CMA>]
PRO [<Atom CA>, <Atom O>, <Atom CMA>]
PRO [<Atom CA>, <Atom O>, <Atom CMA>]
TYR [<Atom CA>, <Atom O>, <Atom CMA>]
.....
CYS [<Atom CA>, <Atom O>, <Atom CMA>]
GLY [<Atom CA>, <Atom O>]
GLY [<Atom CA>, <Atom O>]
ALA [<Atom CA>, <Atom O>, <Atom CMA>]

cg_martini = p.coarse_grain('MARTINI')

for residue in cg_martini.get_residues():
print residue.resname, residue.child_list

ARG [<Atom BB>, <Atom S1>, <Atom S2>]
PRO [<Atom BB>, <Atom S1>]
ASP [<Atom BB>, <Atom S1>]
PHE [<Atom BB>, <Atom S1>, <Atom S2>, <Atom S3>]
CYS [<Atom BB>, <Atom S1>]
LEU [<Atom BB>, <Atom S1>]
GLU [<Atom BB>, <Atom S1>]
PRO [<Atom BB>, <Atom S1>]
PRO [<Atom BB>, <Atom S1>]
TYR [<Atom BB>, <Atom S1>, <Atom S2>, <Atom S3>]
......
CYS [<Atom BB>, <Atom S1>]
GLY [<Atom BB>]
GLY [<Atom BB>]
ALA [<Atom BB>]
</python>

==== Removal of disordered atoms ====

Implement as part of Structure.py and based loosely on the [http://www.biopython.org/wiki/Remove_PDB_disordered_atoms contribution of Ramon Crehuet]. The DisorderedAtom objects are removed from the residue and a single Atom object is added corresponding to the location of the user's choice (keep_loc argument) which defaults to A.

An example, still keeping s from above:

<python>
s = s.remove_disordered_atoms(verbose=True)
0 residues were modified

# Now if we load a structure with disordered atoms
ds = Struct.read('1MC2.pdb')
ds.remove_disordered_atoms(verbose=True)
Residue TRP:1010 has 8 disordered atoms: CD1/CD2/NE1/CE2/CE3/CZ2/CZ3/CH2
Residue VAL:1018 has 3 disordered atoms: CB/CG1/CG2
Residue LEU:1024 has 4 disordered atoms: CB/CG/CD1/CD2
Residue ARG:1043 has 7 disordered atoms: CB/CG/CD/NE/CZ/NH1/NH2
Residue MET:1092 has 4 disordered atoms: CB/CG/SD/CE
Residue ARG:1107 has 7 disordered atoms: CB/CG/CD/NE/CZ/NH1/NH2
Residue GLU:1108 has 4 disordered atoms: CG/CD/OE1/OE2
Residue ASP:1111 has 4 disordered atoms: CB/CG/OD1/OD2
Residue SER:1116 has 1 disordered atoms: OG
Residue SER:1131 has 1 disordered atoms: O
10 residues were modified
</python>

=== Current Development ===

Unit Tests are being developed for all these previous features. Some other features were already developed and are being tested. Additionally, other unplanned features have been developed and tested (check missing atoms).

Current work is focusing on the Nomenclature Normalization and the Homology features. We have established contact with the Dali Server developers and are working towards a solution with the Structural Homology feature. However, it seems that since results take too long to run, this is not a valuable addition to the project..

GSOC2010 Joao

Joaor — Thu, 05 Aug 2010 01:33:24 GMT

Joaor: /* Project Progress */

== Author & Mentors ==

[[User:Joaor|João Rodrigues]] anaryin@gmail.com

'''Mentors'''
: Eric Talevich
: Diana Jaunzeikare
: Peter Cock

== Abstract ==

Biopython is a very popular library in Bioinformatics and Computational Biology. Its Bio.PDB module, originally developed by Thomas Hamelryck, is a simple yet powerful tool for structural biologists. Although it provides a reliable PDB parser feature and it allows several calculations (Neighbour Search, RMS) to be made on macromolecules, it still lacks a number of features that are part of a researcher's daily routine. Probing for disulphide bridges in a structure and adding polar hydrogen atoms accordingly are two examples that can be incorporated in Bio.PDB, given the module's clever structure and good overall organisation. Cosmetic operations such as chain removal and residue renaming – to account for the different existing nomenclatures – and renumbering would also be greatly appreciated by the community.

Another aspect that can be improved for Bio.PDB is a smooth integration/interaction layer for heavy-weights in macromolecule simulation such as MODELLER, GROMACS, AutoDock, HADDOCK. It could be argued that the easiest solution would be to code hooks to these packages' functions and routines. However, projects such as the recently developed [http://sbcb.bioch.ox.ac.uk/oliver/software/GromacsWrapper/html/edpdb.html edPDB] or the more complete [http://biskit.pasteur.fr/ Biskit library] render, in my opinion, such interfacing efforts redundant. Instead, I believe it to be more advantageous to include these software' input/output formats in Biopython's SeqIO and AlignIO modules. This, together with the creation of interfaces for model validation/structure checking services/software would allow Biopython to be used as a pre- and post-simulation tool. Eventually, it would pave the way for its inclusion in pipelines and workflows for structure modelling, molecular dynamics, and docking simulations.

== Project Schedule ==

The schedule below was organised to be flexible, which means that some features will likely be done early. Also, the weeks include documentation and unit testing efforts for the features, with extended periods for reviewing these efforts at the two points during the project (halfway, final week).

=== Community Bonding Period ===

*Getting familiar with development environment (Git Hub account, Git, Biopython's repository, Bug tracking system, etc)

*Gather scientific literature and discuss some of the to-be-implemented methods.

=== Week 1 [31st May - 6th June] ===

==== Renumbering residues of a structure ====

*Read SEQRES record to account for gaps
*Alternatively read ATOM records.

==== Probe disulphide bridges in the structure ====

*Via NeighbourSearch class
*Also use SSBOND in header

==== Extract Biological Unit ====

*REMARK350 contains rotation and translation information
*If REMARK is absent, do nothing.

=== Week 2 [7th – 13th June] ===

==== Structure Hydrogenation ====

*Add all/polar hydrogens through interface with WHATIF server.
*Optionally define a set pH
** [http://www3.interscience.wiley.com/journal/112117957/abstract pKa values algorithm]

==== Hydrogenation Report ====

*Produces a brief list of polar hydrogen atoms in the structure.
** Chain | Residue [number] | Atom

=== Weeks 3-5 [14th June- 4th July] ===

==== Removal of disordered atoms ====

*[[Remove_PDB_disordered_atoms|Solution proposed in the Biopython wiki]]

==== Residue name normalisation ====

*Build conversion table from different nomenclatures (research them during c.bonding period )
*Write function to make a given structure compliant with a given software nomenclature:
** Amber
** CNS/HADDOCK
** GROMACS

==== Coarse Grain Structure ====

*Implement function to reduce complexity of a structure
** 1pt*c-alpha
** 2pt*c-alpha / c-beta
** 3pt*c-alpha / c-beta / side-chain pseudo-centroid OR side-chain centroid

=== Week 6 (Mid-Term) [5th - 11th July] ===

: Testing and consolidating the features thoroughly.
: Write documentation & examples for each feature, to be included in Biopython's Wiki and Bio.PDB's FAQ.
: Mid-term Evaluations. Discussing with mentors current state of project and adjust following schedule to comply with project's needs.

=== Week 7 [12th - 19th July] ===

==== Add support for MODELLER's PIR format to Biopython ====

*[http://www.salilab.org/modeller/manual/node445.html#alignmentformat Format Description]
*SeqIO
*AlignIO

==== Allow conversion of Structure Object to Sequence Object ====

*Based on Bio.PDB.Polypeptide function

=== Weeks 8-10 [20th July - 9th August] ===

==== Add Sequence/Structure Homology functions ====

*Create call to Biopython's BLAST interfaces
** Allow direct blast from structure object ( e.g. protein.find_homoseq() )
** Returns list of tuples with E-Value *Dictionary (name, length of alignment, etc..)
*Create interface with structural homology web services
** e.g. [http://ekhidna.biocenter.helsinki.fi/dali_server/ Dali server]
** Return list of tuples with Z-Score*Dictionary (name, etc...)

==== Implement basic structure validation checks ====

*Via NeighbourSearch class
** Same Charge contacts
** Atom Clashes
*Via ResidueDepth Class
** Buried Charges
*Interface WHATIF PDBReport web service
** Parse WARNING and ERROR messages

=== Week 11 [10th - 17th August] ===

==== Reviewing documentation, code, write tests for new functions. ====

== Project Code ==

Hosted at [http://github.com/JoaoRodrigues/biopython/tree/GSOC2010 this GitHub branch]

== Project Progress ==

Since I'm adding some methods that are useful/logical only for proteins, having them exposed in Structure.py for every molecule could be misleading. We decided then to add a 'as_protein()' method that allows protein-specific methods to be accessed. The following example demonstrates how this call works. Note how the "search_ss_bonds" method is absent from dir(s) but not from dir(prot).

<python>
from Bio.PDB import PDBParser

p = PDBParser()
s = p.get_structure('example', '4PTI.pdb')

dir(s)
# Cut for viewing purposes
['__doc__', ... , 'renumber_residues', 'set_parent', 'xtra']

prot = s.as_protein()

dir(prot)

['__doc__', ... , 'renumber_residues', 'search_ss_bonds', 'set_parent', 'xtra']
</python>

=== Week 1 ===

==== Renumbering residues of a structure ====

Since parse_pdb_header is far from optimal and is likely to change in the future, I opted to forfeit reading SEQREQ records to account for gaps. However, ignoring this information and renumbering based on ATOM records would make us lose information on gaps. I opted to subtract the first residue number-1 to all residues thus making the numbering start in 1 and still keep gaps. I also added an argument (start) to allow the user to set which number to start the counting from.

Example:

<python>
from Bio.PDB import PDBParser

p = PDBParser()
s = p.get_structure('example', '1IHM.pdb')

print list(s.get_residues())[0]
<Residue ASP het= resseq=1029 icode= >

s.renumber_residues()
print list(s.get_residues())[0]
<Residue ASP het= resseq=1 icode= >
</python>

==== Probe disulphide bridges in the structure ====

The same rationale from SEQRES applies for the exclusion of looking up SSBOND. Also, instead of using NeighborSearch to look for pairs of cysteins in bond distance, I instead used the minus operator since it has been overloaded to return the distance between two atoms (Page 10 of the [http://www.biopython.org/DIST/docs/cookbook/biopdb_faq.pdf FAQ]). The average distance cited in the literature is 2.05A but other software packages and my own tests set 3.0A as a good threshold. Still, the user can set his own threshold manually.

The function returns an iterator with tuples of pairs of residues.

<python>
from Bio.PDB import PDBParser

p = PDBParser()
s = p.get_structure('example', '4PTI.pdb')

prot = s.as_protein()

for bond in prot.search_ss_bonds():
print bond

(<Residue CYS het= resseq=5 icode= >, <Residue CYS het= resseq=55 icode= >)
(<Residue CYS het= resseq=14 icode= >, <Residue CYS het= resseq=38 icode= >)
(<Residue CYS het= resseq=30 icode= >, <Residue CYS het= resseq=51 icode= >)
</python>

==== Extract Biological Unit ====

Added parsing for REMARK350 to parse_pdb_header since there was already a bit written for another REMARK section. This extracts the transformation matrices and the translation vector from the header, that is then fed to the Structure function. Each new rotated structure is created as a new MODEL. I chose this because crystal structures very rarely have more than one MODEL instance and also because NMR models don't have REMARK 350 that often (at least to my knowledge).

<python>
from Bio.PDB import PDBParser

p = PDBParser()

s1 = p.get_structure('a', '4PTI.pdb')
s1.build_biological_unit()
'Processed 0 transformations on the structure.' # Identity matrix is ignored.

s2 = p.get_structure('b', 'homol_1bd8.pdb') # A homology model
s2.build_biological_unit()
'PDB File lacks appropriate REMARK 350 entries to build Biological Unit.'

s3 = p.get_structure('c', '1IHM.pdb')
s3.build_biological_unit()
'Processed 59 transformations on the structure.'
</python>

=== Weeks 2-5 ===

==== Hydrogenation of PDB files ====

Following discussion between the mentors and me, we decided that maybe it was better to not only include a webserver for this purpose but also a local algorithm. This would not limit the user when there he/she lacks an internet connection.

The interface for the WHATIF Protonation service has been implemented, although it should be regarded as **highly experimental** for now. Interfacing this server included writing a small parser for a PDBXML-like format, which is expected to have serious bugs in its initial versions. I ran some simple tests and it works. It doesn't support water molecules yet, nor any other molecules other than proteins. Such issues will be hopefully solved later on..

For those brave enough to want to test it (and help me debug it), here's an example usage.

<python>
from Bio.Struct.WWW import WHATIF
from Bio import Struct

server = WHATIF.WHATIF() # Performs a sort of PING to the server. Gracefully exits if the servers are down.

# Get the protein structure
structure = Struct.read('4PTI.pdb')
protein = structure.as_protein() # This excludes water molecules

# Upload the structure to the WHATIF server
# This should convert the structure from a Structure object to a string via tempfile and PDBIO
# I was having some issues uploading structures...

id = server.UploadPDB(protein)

# Protonate
# Returns a Structure Object / WARNING! Bug prone for now.

protein_h = server.PDBasXMLwithSymwithPolarH(id)

</python>

Regarding the local implementation, after much reading I settled on using PyMol's algorithm. It seems to allow for protonation of any structure, regardless of its nature (protein, DNA, etc). Its vectorial and matrix operations can likely be optimized with Numpy and Biopython's Vector.py module. This first implementation works for proteins only. I'll add general molecule support later.

<python>
from Bio import Struct
from Bio.Struct import Hydrogenate as H

s = Struct.read('1ctf.pdb')
p = s.as_protein()

prot = H.Hydrogenate_Protein()
prot.add_hydrogens(p)
</python>

==== Coarse Grain Structure ====

A Center of Mass function was developed first as part of a new module Bio.Struct.Geometry. It allows for calculation of the center of geometry (all masses are equal) and center of mass (taking into account elemental masses for the atoms). The masses are a new Atom object feature derived from [http://www.chem.qmul.ac.uk/iupac/AtWt/ this list] and from PyMol. Essentially, all atoms of a structure now get their mass defined when the Structure is created (check Atom.py and [http://lists.open-bio.org/pipermail/biopython-dev/2010-June/007880.html this thread] for details). This is obviously experimental.

To calculate the center of mass of any Entity (Structure, Model, Chain, Residue) or a List of Atoms:

<python>
from Bio.Struct.Geometry import center_of_mass
from Bio import Struct

s = Struct.read('4PTI.pdb')

print center_of_mass.__doc__

Returns gravitic or geometric center of mass of an Entity.
Geometric assumes all masses are equal (geometric=True)
Defaults to Gravitic.

print center_of_mass(s)
[14.833301303933874, 21.431581746366263, 4.1218478418007134]

print center_of_mass(s, geometric=True)
[14.805324902127458, 21.365571977563405, 4.1108949403803985]
</python>

As of now, 3 CG models are supported.

1) CA-Trace
2) ENCAD 3-point model (CA, O, Side Chain bead)
3) MARTINI protein model (BB, Side Chain points [S1 to S4])

An example, picking up the s Structure from above:

<python>
p = s.as_protein() # To expose the CG method

ca_trace = p.coarse_grain()

# One atom per residue
print ( len(list(p.get_residues())) == len(list(ca_trace.get_atoms())) )
True

cg_encad = p.coarse_grain('ENCAD_3P')

for residue in cg_encad.get_residues():
print residue.resname, residue.child_list

ARG [<Atom CA>, <Atom O>, <Atom CMA>]
PRO [<Atom CA>, <Atom O>, <Atom CMA>]
ASP [<Atom CA>, <Atom O>, <Atom CMA>]
PHE [<Atom CA>, <Atom O>, <Atom CMA>]
CYS [<Atom CA>, <Atom O>, <Atom CMA>]
LEU [<Atom CA>, <Atom O>, <Atom CMA>]
GLU [<Atom CA>, <Atom O>, <Atom CMA>]
PRO [<Atom CA>, <Atom O>, <Atom CMA>]
PRO [<Atom CA>, <Atom O>, <Atom CMA>]
TYR [<Atom CA>, <Atom O>, <Atom CMA>]
.....
CYS [<Atom CA>, <Atom O>, <Atom CMA>]
GLY [<Atom CA>, <Atom O>]
GLY [<Atom CA>, <Atom O>]
ALA [<Atom CA>, <Atom O>, <Atom CMA>]

cg_martini = p.coarse_grain('MARTINI')

for residue in cg_martini.get_residues():
print residue.resname, residue.child_list

ARG [<Atom BB>, <Atom S1>, <Atom S2>]
PRO [<Atom BB>, <Atom S1>]
ASP [<Atom BB>, <Atom S1>]
PHE [<Atom BB>, <Atom S1>, <Atom S2>, <Atom S3>]
CYS [<Atom BB>, <Atom S1>]
LEU [<Atom BB>, <Atom S1>]
GLU [<Atom BB>, <Atom S1>]
PRO [<Atom BB>, <Atom S1>]
PRO [<Atom BB>, <Atom S1>]
TYR [<Atom BB>, <Atom S1>, <Atom S2>, <Atom S3>]
......
CYS [<Atom BB>, <Atom S1>]
GLY [<Atom BB>]
GLY [<Atom BB>]
ALA [<Atom BB>]
</python>

==== Removal of disordered atoms ====

Implement as part of Structure.py and based loosely on the [http://www.biopython.org/wiki/Remove_PDB_disordered_atoms contribution of Ramon Crehuet]. The DisorderedAtom objects are removed from the residue and a single Atom object is added corresponding to the location of the user's choice (keep_loc argument) which defaults to A.

An example, still keeping s from above:

<python>
s = s.remove_disordered_atoms(verbose=True)
0 residues were modified

# Now if we load a structure with disordered atoms
ds = Struct.read('1MC2.pdb')
ds.remove_disordered_atoms(verbose=True)
Residue TRP:1010 has 8 disordered atoms: CD1/CD2/NE1/CE2/CE3/CZ2/CZ3/CH2
Residue VAL:1018 has 3 disordered atoms: CB/CG1/CG2
Residue LEU:1024 has 4 disordered atoms: CB/CG/CD1/CD2
Residue ARG:1043 has 7 disordered atoms: CB/CG/CD/NE/CZ/NH1/NH2
Residue MET:1092 has 4 disordered atoms: CB/CG/SD/CE
Residue ARG:1107 has 7 disordered atoms: CB/CG/CD/NE/CZ/NH1/NH2
Residue GLU:1108 has 4 disordered atoms: CG/CD/OE1/OE2
Residue ASP:1111 has 4 disordered atoms: CB/CG/OD1/OD2
Residue SER:1116 has 1 disordered atoms: OG
Residue SER:1131 has 1 disordered atoms: O
10 residues were modified
</python>

=== Current Development ===

Unit Tests are being developed for all these previous features. Some other features were already developed and are being tested. Additionally, other unplanned features have been developed and tested (check missing atoms).

Current work is focusing on the Nomenclature Normalization and the Homology features. We have established contact with the Dali Server developers and are working towards a solution with the Structural Homology feature. However, it seems that since results take too long to run, this is not a valuable addition to the project..

GSOC2010 Joao

Joaor — Thu, 05 Aug 2010 01:28:02 GMT

Joaor: /* Coarse Grain Structure */ Added MARTINI description

== Author & Mentors ==

[[User:Joaor|João Rodrigues]] anaryin@gmail.com

'''Mentors'''
: Eric Talevich
: Diana Jaunzeikare
: Peter Cock

== Abstract ==

Biopython is a very popular library in Bioinformatics and Computational Biology. Its Bio.PDB module, originally developed by Thomas Hamelryck, is a simple yet powerful tool for structural biologists. Although it provides a reliable PDB parser feature and it allows several calculations (Neighbour Search, RMS) to be made on macromolecules, it still lacks a number of features that are part of a researcher's daily routine. Probing for disulphide bridges in a structure and adding polar hydrogen atoms accordingly are two examples that can be incorporated in Bio.PDB, given the module's clever structure and good overall organisation. Cosmetic operations such as chain removal and residue renaming – to account for the different existing nomenclatures – and renumbering would also be greatly appreciated by the community.

Another aspect that can be improved for Bio.PDB is a smooth integration/interaction layer for heavy-weights in macromolecule simulation such as MODELLER, GROMACS, AutoDock, HADDOCK. It could be argued that the easiest solution would be to code hooks to these packages' functions and routines. However, projects such as the recently developed [http://sbcb.bioch.ox.ac.uk/oliver/software/GromacsWrapper/html/edpdb.html edPDB] or the more complete [http://biskit.pasteur.fr/ Biskit library] render, in my opinion, such interfacing efforts redundant. Instead, I believe it to be more advantageous to include these software' input/output formats in Biopython's SeqIO and AlignIO modules. This, together with the creation of interfaces for model validation/structure checking services/software would allow Biopython to be used as a pre- and post-simulation tool. Eventually, it would pave the way for its inclusion in pipelines and workflows for structure modelling, molecular dynamics, and docking simulations.

== Project Schedule ==

The schedule below was organised to be flexible, which means that some features will likely be done early. Also, the weeks include documentation and unit testing efforts for the features, with extended periods for reviewing these efforts at the two points during the project (halfway, final week).

=== Community Bonding Period ===

*Getting familiar with development environment (Git Hub account, Git, Biopython's repository, Bug tracking system, etc)

*Gather scientific literature and discuss some of the to-be-implemented methods.

=== Week 1 [31st May - 6th June] ===

==== Renumbering residues of a structure ====

*Read SEQRES record to account for gaps
*Alternatively read ATOM records.

==== Probe disulphide bridges in the structure ====

*Via NeighbourSearch class
*Also use SSBOND in header

==== Extract Biological Unit ====

*REMARK350 contains rotation and translation information
*If REMARK is absent, do nothing.

=== Week 2 [7th – 13th June] ===

==== Structure Hydrogenation ====

*Add all/polar hydrogens through interface with WHATIF server.
*Optionally define a set pH
** [http://www3.interscience.wiley.com/journal/112117957/abstract pKa values algorithm]

==== Hydrogenation Report ====

*Produces a brief list of polar hydrogen atoms in the structure.
** Chain | Residue [number] | Atom

=== Weeks 3-5 [14th June- 4th July] ===

==== Removal of disordered atoms ====

*[[Remove_PDB_disordered_atoms|Solution proposed in the Biopython wiki]]

==== Residue name normalisation ====

*Build conversion table from different nomenclatures (research them during c.bonding period )
*Write function to make a given structure compliant with a given software nomenclature:
** Amber
** CNS/HADDOCK
** GROMACS

==== Coarse Grain Structure ====

*Implement function to reduce complexity of a structure
** 1pt*c-alpha
** 2pt*c-alpha / c-beta
** 3pt*c-alpha / c-beta / side-chain pseudo-centroid OR side-chain centroid

=== Week 6 (Mid-Term) [5th - 11th July] ===

: Testing and consolidating the features thoroughly.
: Write documentation & examples for each feature, to be included in Biopython's Wiki and Bio.PDB's FAQ.
: Mid-term Evaluations. Discussing with mentors current state of project and adjust following schedule to comply with project's needs.

=== Week 7 [12th - 19th July] ===

==== Add support for MODELLER's PIR format to Biopython ====

*[http://www.salilab.org/modeller/manual/node445.html#alignmentformat Format Description]
*SeqIO
*AlignIO

==== Allow conversion of Structure Object to Sequence Object ====

*Based on Bio.PDB.Polypeptide function

=== Weeks 8-10 [20th July - 9th August] ===

==== Add Sequence/Structure Homology functions ====

*Create call to Biopython's BLAST interfaces
** Allow direct blast from structure object ( e.g. protein.find_homoseq() )
** Returns list of tuples with E-Value *Dictionary (name, length of alignment, etc..)
*Create interface with structural homology web services
** e.g. [http://ekhidna.biocenter.helsinki.fi/dali_server/ Dali server]
** Return list of tuples with Z-Score*Dictionary (name, etc...)

==== Implement basic structure validation checks ====

*Via NeighbourSearch class
** Same Charge contacts
** Atom Clashes
*Via ResidueDepth Class
** Buried Charges
*Interface WHATIF PDBReport web service
** Parse WARNING and ERROR messages

=== Week 11 [10th - 17th August] ===

==== Reviewing documentation, code, write tests for new functions. ====

== Project Code ==

Hosted at [http://github.com/JoaoRodrigues/biopython/tree/GSOC2010 this GitHub branch]

== Project Progress ==

Since I'm adding some methods that are useful/logical only for proteins, having them exposed in Structure.py for every molecule could be misleading. We decided then to add a 'as_protein()' method that allows protein-specific methods to be accessed. The following example demonstrates how this call works. Note how the "search_ss_bonds" method is absent from dir(s) but not from dir(prot).

<python>
from Bio.PDB import PDBParser

p = PDBParser()
s = p.get_structure('example', '4PTI.pdb')

dir(s)
# Cut for viewing purposes
['__doc__', ... , 'renumber_residues', 'set_parent', 'xtra']

prot = s.as_protein()

dir(prot)

['__doc__', ... , 'renumber_residues', 'search_ss_bonds', 'set_parent', 'xtra']
</python>

=== Week 1 ===

==== Renumbering residues of a structure ====

Since parse_pdb_header is far from optimal and is likely to change in the future, I opted to forfeit reading SEQREQ records to account for gaps. However, ignoring this information and renumbering based on ATOM records would make us lose information on gaps. I opted to subtract the first residue number-1 to all residues thus making the numbering start in 1 and still keep gaps. I also added an argument (start) to allow the user to set which number to start the counting from.

Example:

<python>
from Bio.PDB import PDBParser

p = PDBParser()
s = p.get_structure('example', '1IHM.pdb')

print list(s.get_residues())[0]
<Residue ASP het= resseq=1029 icode= >

s.renumber_residues()
print list(s.get_residues())[0]
<Residue ASP het= resseq=1 icode= >
</python>

==== Probe disulphide bridges in the structure ====

The same rationale from SEQRES applies for the exclusion of looking up SSBOND. Also, instead of using NeighborSearch to look for pairs of cysteins in bond distance, I instead used the minus operator since it has been overloaded to return the distance between two atoms (Page 10 of the [http://www.biopython.org/DIST/docs/cookbook/biopdb_faq.pdf FAQ]). The average distance cited in the literature is 2.05A but other software packages and my own tests set 3.0A as a good threshold. Still, the user can set his own threshold manually.

The function returns an iterator with tuples of pairs of residues.

<python>
from Bio.PDB import PDBParser

p = PDBParser()
s = p.get_structure('example', '4PTI.pdb')

prot = s.as_protein()

for bond in prot.search_ss_bonds():
print bond

(<Residue CYS het= resseq=5 icode= >, <Residue CYS het= resseq=55 icode= >)
(<Residue CYS het= resseq=14 icode= >, <Residue CYS het= resseq=38 icode= >)
(<Residue CYS het= resseq=30 icode= >, <Residue CYS het= resseq=51 icode= >)
</python>

==== Extract Biological Unit ====

Added parsing for REMARK350 to parse_pdb_header since there was already a bit written for another REMARK section. This extracts the transformation matrices and the translation vector from the header, that is then fed to the Structure function. Each new rotated structure is created as a new MODEL. I chose this because crystal structures very rarely have more than one MODEL instance and also because NMR models don't have REMARK 350 that often (at least to my knowledge).

<python>
from Bio.PDB import PDBParser

p = PDBParser()

s1 = p.get_structure('a', '4PTI.pdb')
s1.build_biological_unit()
'Processed 0 transformations on the structure.' # Identity matrix is ignored.

s2 = p.get_structure('b', 'homol_1bd8.pdb') # A homology model
s2.build_biological_unit()
'PDB File lacks appropriate REMARK 350 entries to build Biological Unit.'

s3 = p.get_structure('c', '1IHM.pdb')
s3.build_biological_unit()
'Processed 59 transformations on the structure.'
</python>

=== Weeks 2-5 ===

==== Hydrogenation of PDB files ====

Following discussion between the mentors and me, we decided that maybe it was better to not only include a webserver for this purpose but also a local algorithm. This would not limit the user when there he/she lacks an internet connection.

The interface for the WHATIF Protonation service has been implemented, although it should be regarded as **highly experimental** for now. Interfacing this server included writing a small parser for a PDBXML-like format, which is expected to have serious bugs in its initial versions. I ran some simple tests and it works. It doesn't support water molecules yet, nor any other molecules other than proteins. Such issues will be hopefully solved later on..

For those brave enough to want to test it (and help me debug it), here's an example usage.

<python>
from Bio.Struct.WWW import WHATIF
from Bio import Struct

server = WHATIF.WHATIF() # Performs a sort of PING to the server. Gracefully exits if the servers are down.

# Get the protein structure
structure = Struct.read('4PTI.pdb')
protein = structure.as_protein() # This excludes water molecules

# Upload the structure to the WHATIF server
# This should convert the structure from a Structure object to a string via tempfile and PDBIO
# I was having some issues uploading structures...

id = server.UploadPDB(protein)

# Protonate
# Returns a Structure Object / WARNING! Bug prone for now.

protein_h = server.PDBasXMLwithSymwithPolarH(id)

</python>

Regarding the local implementation, after much reading I settled on using PyMol's algorithm. It seems to allow for protonation of any structure, regardless of its nature (protein, DNA, etc). Its vectorial and matrix operations can likely be optimized with Numpy and Biopython's Vector.py module. This first implementation works for proteins only. I'll add general molecule support later.

<python>
from Bio import Struct
from Bio.Struct import Hydrogenate as H

s = Struct.read('1ctf.pdb')
p = s.as_protein()

prot = H.Hydrogenate_Protein()
prot.add_hydrogens(p)
</python>

==== Coarse Grain Structure ====

A Center of Mass function was developed first as part of a new module Bio.Struct.Geometry. It allows for calculation of the center of geometry (all masses are equal) and center of mass (taking into account elemental masses for the atoms). The masses are a new Atom object feature derived from [http://www.chem.qmul.ac.uk/iupac/AtWt/ this list] and from PyMol. Essentially, all atoms of a structure now get their mass defined when the Structure is created (check Atom.py and [http://lists.open-bio.org/pipermail/biopython-dev/2010-June/007880.html this thread] for details). This is obviously experimental.

To calculate the center of mass of any Entity (Structure, Model, Chain, Residue) or a List of Atoms:

<python>
from Bio.Struct.Geometry import center_of_mass
from Bio import Struct

s = Struct.read('4PTI.pdb')

print center_of_mass.__doc__

Returns gravitic or geometric center of mass of an Entity.
Geometric assumes all masses are equal (geometric=True)
Defaults to Gravitic.

print center_of_mass(s)
[14.833301303933874, 21.431581746366263, 4.1218478418007134]

print center_of_mass(s, geometric=True)
[14.805324902127458, 21.365571977563405, 4.1108949403803985]
</python>

As of now, 3 CG models are supported.

1) CA-Trace
2) ENCAD 3-point model (CA, O, Side Chain bead)
3) MARTINI protein model (BB, Side Chain points [S1 to S4])

An example, picking up the s Structure from above:

<python>
p = s.as_protein() # To expose the CG method

ca_trace = p.coarse_grain()

# One atom per residue
print ( len(list(p.get_residues())) == len(list(ca_trace.get_atoms())) )
True

cg_encad = p.coarse_grain('ENCAD_3P')

for residue in cg_encad.get_residues():
print residue.resname, residue.child_list

ARG [<Atom CA>, <Atom O>, <Atom CMA>]
PRO [<Atom CA>, <Atom O>, <Atom CMA>]
ASP [<Atom CA>, <Atom O>, <Atom CMA>]
PHE [<Atom CA>, <Atom O>, <Atom CMA>]
CYS [<Atom CA>, <Atom O>, <Atom CMA>]
LEU [<Atom CA>, <Atom O>, <Atom CMA>]
GLU [<Atom CA>, <Atom O>, <Atom CMA>]
PRO [<Atom CA>, <Atom O>, <Atom CMA>]
PRO [<Atom CA>, <Atom O>, <Atom CMA>]
TYR [<Atom CA>, <Atom O>, <Atom CMA>]
.....
CYS [<Atom CA>, <Atom O>, <Atom CMA>]
GLY [<Atom CA>, <Atom O>]
GLY [<Atom CA>, <Atom O>]
ALA [<Atom CA>, <Atom O>, <Atom CMA>]

cg_martini = p.coarse_grain('MARTINI')

for residue in cg_martini.get_residues():
print residue.resname, residue.child_list

ARG [<Atom BB>, <Atom S1>, <Atom S2>]
PRO [<Atom BB>, <Atom S1>]
ASP [<Atom BB>, <Atom S1>]
PHE [<Atom BB>, <Atom S1>, <Atom S2>, <Atom S3>]
CYS [<Atom BB>, <Atom S1>]
LEU [<Atom BB>, <Atom S1>]
GLU [<Atom BB>, <Atom S1>]
PRO [<Atom BB>, <Atom S1>]
PRO [<Atom BB>, <Atom S1>]
TYR [<Atom BB>, <Atom S1>, <Atom S2>, <Atom S3>]
......
CYS [<Atom BB>, <Atom S1>]
GLY [<Atom BB>]
GLY [<Atom BB>]
ALA [<Atom BB>]
</python>

==== Removal of disordered atoms ====

Implement as part of Structure.py and based loosely on the [http://www.biopython.org/wiki/Remove_PDB_disordered_atoms contribution of Ramon Crehuet]. The DisorderedAtom objects are removed from the residue and a single Atom object is added corresponding to the location of the user's choice (keep_loc argument) which defaults to A.

An example, still keeping s from above:

<python>
s = s.remove_disordered_atoms(verbose=True)
0 residues were modified

# Now if we load a structure with disordered atoms
ds = Struct.read('1MC2.pdb')
ds.remove_disordered_atoms(verbose=True)
Residue TRP:1010 has 8 disordered atoms: CD1/CD2/NE1/CE2/CE3/CZ2/CZ3/CH2
Residue VAL:1018 has 3 disordered atoms: CB/CG1/CG2
Residue LEU:1024 has 4 disordered atoms: CB/CG/CD1/CD2
Residue ARG:1043 has 7 disordered atoms: CB/CG/CD/NE/CZ/NH1/NH2
Residue MET:1092 has 4 disordered atoms: CB/CG/SD/CE
Residue ARG:1107 has 7 disordered atoms: CB/CG/CD/NE/CZ/NH1/NH2
Residue GLU:1108 has 4 disordered atoms: CG/CD/OE1/OE2
Residue ASP:1111 has 4 disordered atoms: CB/CG/OD1/OD2
Residue SER:1116 has 1 disordered atoms: OG
Residue SER:1131 has 1 disordered atoms: O
10 residues were modified
</python>

GSOC2010 Joao

Joaor — Fri, 02 Jul 2010 18:52:20 GMT

Joaor: /* Hydrogenation of PDB files */ Added local implementation

== Author & Mentors ==

[[User:Joaor|João Rodrigues]] anaryin@gmail.com

'''Mentors'''
: Eric Talevich
: Diana Jaunzeikare
: Peter Cock

== Abstract ==

Biopython is a very popular library in Bioinformatics and Computational Biology. Its Bio.PDB module, originally developed by Thomas Hamelryck, is a simple yet powerful tool for structural biologists. Although it provides a reliable PDB parser feature and it allows several calculations (Neighbour Search, RMS) to be made on macromolecules, it still lacks a number of features that are part of a researcher's daily routine. Probing for disulphide bridges in a structure and adding polar hydrogen atoms accordingly are two examples that can be incorporated in Bio.PDB, given the module's clever structure and good overall organisation. Cosmetic operations such as chain removal and residue renaming – to account for the different existing nomenclatures – and renumbering would also be greatly appreciated by the community.

Another aspect that can be improved for Bio.PDB is a smooth integration/interaction layer for heavy-weights in macromolecule simulation such as MODELLER, GROMACS, AutoDock, HADDOCK. It could be argued that the easiest solution would be to code hooks to these packages' functions and routines. However, projects such as the recently developed [http://sbcb.bioch.ox.ac.uk/oliver/software/GromacsWrapper/html/edpdb.html edPDB] or the more complete [http://biskit.pasteur.fr/ Biskit library] render, in my opinion, such interfacing efforts redundant. Instead, I believe it to be more advantageous to include these software' input/output formats in Biopython's SeqIO and AlignIO modules. This, together with the creation of interfaces for model validation/structure checking services/software would allow Biopython to be used as a pre- and post-simulation tool. Eventually, it would pave the way for its inclusion in pipelines and workflows for structure modelling, molecular dynamics, and docking simulations.

== Project Schedule ==

The schedule below was organised to be flexible, which means that some features will likely be done early. Also, the weeks include documentation and unit testing efforts for the features, with extended periods for reviewing these efforts at the two points during the project (halfway, final week).

=== Community Bonding Period ===

*Getting familiar with development environment (Git Hub account, Git, Biopython's repository, Bug tracking system, etc)

*Gather scientific literature and discuss some of the to-be-implemented methods.

=== Week 1 [31st May - 6th June] ===

==== Renumbering residues of a structure ====

*Read SEQRES record to account for gaps
*Alternatively read ATOM records.

==== Probe disulphide bridges in the structure ====

*Via NeighbourSearch class
*Also use SSBOND in header

==== Extract Biological Unit ====

*REMARK350 contains rotation and translation information
*If REMARK is absent, do nothing.

=== Week 2 [7th – 13th June] ===

==== Structure Hydrogenation ====

*Add all/polar hydrogens through interface with WHATIF server.
*Optionally define a set pH
** [http://www3.interscience.wiley.com/journal/112117957/abstract pKa values algorithm]

==== Hydrogenation Report ====

*Produces a brief list of polar hydrogen atoms in the structure.
** Chain | Residue [number] | Atom

=== Weeks 3-5 [14th June- 4th July] ===

==== Removal of disordered atoms ====

*[[Remove_PDB_disordered_atoms|Solution proposed in the Biopython wiki]]

==== Residue name normalisation ====

*Build conversion table from different nomenclatures (research them during c.bonding period )
*Write function to make a given structure compliant with a given software nomenclature:
** Amber
** CNS/HADDOCK
** GROMACS

==== Coarse Grain Structure ====

*Implement function to reduce complexity of a structure
** 1pt*c-alpha
** 2pt*c-alpha / c-beta
** 3pt*c-alpha / c-beta / side-chain pseudo-centroid OR side-chain centroid

=== Week 6 (Mid-Term) [5th - 11th July] ===

: Testing and consolidating the features thoroughly.
: Write documentation & examples for each feature, to be included in Biopython's Wiki and Bio.PDB's FAQ.
: Mid-term Evaluations. Discussing with mentors current state of project and adjust following schedule to comply with project's needs.

=== Week 7 [12th - 19th July] ===

==== Add support for MODELLER's PIR format to Biopython ====

*[http://www.salilab.org/modeller/manual/node445.html#alignmentformat Format Description]
*SeqIO
*AlignIO

==== Allow conversion of Structure Object to Sequence Object ====

*Based on Bio.PDB.Polypeptide function

=== Weeks 8-10 [20th July - 9th August] ===

==== Add Sequence/Structure Homology functions ====

*Create call to Biopython's BLAST interfaces
** Allow direct blast from structure object ( e.g. protein.find_homoseq() )
** Returns list of tuples with E-Value *Dictionary (name, length of alignment, etc..)
*Create interface with structural homology web services
** e.g. [http://ekhidna.biocenter.helsinki.fi/dali_server/ Dali server]
** Return list of tuples with Z-Score*Dictionary (name, etc...)

==== Implement basic structure validation checks ====

*Via NeighbourSearch class
** Same Charge contacts
** Atom Clashes
*Via ResidueDepth Class
** Buried Charges
*Interface WHATIF PDBReport web service
** Parse WARNING and ERROR messages

=== Week 11 [10th - 17th August] ===

==== Reviewing documentation, code, write tests for new functions. ====

== Project Code ==

Hosted at [http://github.com/JoaoRodrigues/biopython/tree/GSOC2010 this GitHub branch]

== Project Progress ==

Since I'm adding some methods that are useful/logical only for proteins, having them exposed in Structure.py for every molecule could be misleading. We decided then to add a 'as_protein()' method that allows protein-specific methods to be accessed. The following example demonstrates how this call works. Note how the "search_ss_bonds" method is absent from dir(s) but not from dir(prot).

<python>
from Bio.PDB import PDBParser

p = PDBParser()
s = p.get_structure('example', '4PTI.pdb')

dir(s)
# Cut for viewing purposes
['__doc__', ... , 'renumber_residues', 'set_parent', 'xtra']

prot = s.as_protein()

dir(prot)

['__doc__', ... , 'renumber_residues', 'search_ss_bonds', 'set_parent', 'xtra']
</python>

=== Week 1 ===

==== Renumbering residues of a structure ====

Since parse_pdb_header is far from optimal and is likely to change in the future, I opted to forfeit reading SEQREQ records to account for gaps. However, ignoring this information and renumbering based on ATOM records would make us lose information on gaps. I opted to subtract the first residue number-1 to all residues thus making the numbering start in 1 and still keep gaps. I also added an argument (start) to allow the user to set which number to start the counting from.

Example:

<python>
from Bio.PDB import PDBParser

p = PDBParser()
s = p.get_structure('example', '1IHM.pdb')

print list(s.get_residues())[0]
<Residue ASP het= resseq=1029 icode= >

s.renumber_residues()
print list(s.get_residues())[0]
<Residue ASP het= resseq=1 icode= >
</python>

==== Probe disulphide bridges in the structure ====

The same rationale from SEQRES applies for the exclusion of looking up SSBOND. Also, instead of using NeighborSearch to look for pairs of cysteins in bond distance, I instead used the minus operator since it has been overloaded to return the distance between two atoms (Page 10 of the [http://www.biopython.org/DIST/docs/cookbook/biopdb_faq.pdf FAQ]). The average distance cited in the literature is 2.05A but other software packages and my own tests set 3.0A as a good threshold. Still, the user can set his own threshold manually.

The function returns an iterator with tuples of pairs of residues.

<python>
from Bio.PDB import PDBParser

p = PDBParser()
s = p.get_structure('example', '4PTI.pdb')

prot = s.as_protein()

for bond in prot.search_ss_bonds():
print bond

(<Residue CYS het= resseq=5 icode= >, <Residue CYS het= resseq=55 icode= >)
(<Residue CYS het= resseq=14 icode= >, <Residue CYS het= resseq=38 icode= >)
(<Residue CYS het= resseq=30 icode= >, <Residue CYS het= resseq=51 icode= >)
</python>

==== Extract Biological Unit ====

Added parsing for REMARK350 to parse_pdb_header since there was already a bit written for another REMARK section. This extracts the transformation matrices and the translation vector from the header, that is then fed to the Structure function. Each new rotated structure is created as a new MODEL. I chose this because crystal structures very rarely have more than one MODEL instance and also because NMR models don't have REMARK 350 that often (at least to my knowledge).

<python>
from Bio.PDB import PDBParser

p = PDBParser()

s1 = p.get_structure('a', '4PTI.pdb')
s1.build_biological_unit()
'Processed 0 transformations on the structure.' # Identity matrix is ignored.

s2 = p.get_structure('b', 'homol_1bd8.pdb') # A homology model
s2.build_biological_unit()
'PDB File lacks appropriate REMARK 350 entries to build Biological Unit.'

s3 = p.get_structure('c', '1IHM.pdb')
s3.build_biological_unit()
'Processed 59 transformations on the structure.'
</python>

=== Weeks 2-5 ===

==== Hydrogenation of PDB files ====

Following discussion between the mentors and me, we decided that maybe it was better to not only include a webserver for this purpose but also a local algorithm. This would not limit the user when there he/she lacks an internet connection.

The interface for the WHATIF Protonation service has been implemented, although it should be regarded as **highly experimental** for now. Interfacing this server included writing a small parser for a PDBXML-like format, which is expected to have serious bugs in its initial versions. I ran some simple tests and it works. It doesn't support water molecules yet, nor any other molecules other than proteins. Such issues will be hopefully solved later on..

For those brave enough to want to test it (and help me debug it), here's an example usage.

<python>
from Bio.Struct.WWW import WHATIF
from Bio import Struct

server = WHATIF.WHATIF() # Performs a sort of PING to the server. Gracefully exits if the servers are down.

# Get the protein structure
structure = Struct.read('4PTI.pdb')
protein = structure.as_protein() # This excludes water molecules

# Upload the structure to the WHATIF server
# This should convert the structure from a Structure object to a string via tempfile and PDBIO
# I was having some issues uploading structures...

id = server.UploadPDB(protein)

# Protonate
# Returns a Structure Object / WARNING! Bug prone for now.

protein_h = server.PDBasXMLwithSymwithPolarH(id)

</python>

Regarding the local implementation, after much reading I settled on using PyMol's algorithm. It seems to allow for protonation of any structure, regardless of its nature (protein, DNA, etc). Its vectorial and matrix operations can likely be optimized with Numpy and Biopython's Vector.py module. This first implementation works for proteins only. I'll add general molecule support later.

<python>
from Bio import Struct
from Bio.Struct import Hydrogenate as H

s = Struct.read('1ctf.pdb')
p = s.as_protein()

prot = H.Hydrogenate_Protein()
prot.add_hydrogens(p)
</python>

==== Coarse Grain Structure ====

A Center of Mass function was developed first as part of a new module Bio.Struct.Geometry. It allows for calculation of the center of geometry (all masses are equal) and center of mass (taking into account elemental masses for the atoms). The masses are a new Atom object feature derived from [http://www.chem.qmul.ac.uk/iupac/AtWt/ this list] and from PyMol. Essentially, all atoms of a structure now get their mass defined when the Structure is created (check Atom.py and [http://lists.open-bio.org/pipermail/biopython-dev/2010-June/007880.html this thread] for details). This is obviously experimental.

To calculate the center of mass of any Entity (Structure, Model, Chain, Residue) or a List of Atoms:

<python>
from Bio.Struct.Geometry import center_of_mass
from Bio import Struct

s = Struct.read('4PTI.pdb')

print center_of_mass.__doc__

Returns gravitic or geometric center of mass of an Entity.
Geometric assumes all masses are equal (geometric=True)
Defaults to Gravitic.

print center_of_mass(s)
[14.833301303933874, 21.431581746366263, 4.1218478418007134]

print center_of_mass(s, geometric=True)
[14.805324902127458, 21.365571977563405, 4.1108949403803985]
</python>

Coarse-Graining makes use of this function when reducing each residue to 3 points (ENCAD-like): Ca, peptide-bond O, and center of mass of side chain. The default CG function reduces the protein to a Ca-trace. I am still working on implementing perhaps some other CG-representations such as that from CABS.

An example, picking up the s Structure from above:

<python>
p = s.as_protein() # To expose the CG method

ca_trace = p.coarse_grain()

# One atom per residue
print ( len(list(p.get_residues())) == len(list(ca_trace.get_atoms())) )
True

cg_3pt = p.coarse_grain('3pt')

for residue in cg_3pt.get_residues():
print residue.resname, residue.child_list

ARG [<Atom CA>, <Atom O>, <Atom CMA>]
PRO [<Atom CA>, <Atom O>, <Atom CMA>]
ASP [<Atom CA>, <Atom O>, <Atom CMA>]
PHE [<Atom CA>, <Atom O>, <Atom CMA>]
CYS [<Atom CA>, <Atom O>, <Atom CMA>]
LEU [<Atom CA>, <Atom O>, <Atom CMA>]
GLU [<Atom CA>, <Atom O>, <Atom CMA>]
PRO [<Atom CA>, <Atom O>, <Atom CMA>]
PRO [<Atom CA>, <Atom O>, <Atom CMA>]
TYR [<Atom CA>, <Atom O>, <Atom CMA>]
.....
CYS [<Atom CA>, <Atom O>, <Atom CMA>]
GLY [<Atom CA>, <Atom O>]
GLY [<Atom CA>, <Atom O>]
ALA [<Atom CA>, <Atom O>, <Atom CMA>]
</python>

==== Removal of disordered atoms ====

Implement as part of Structure.py and based loosely on the [http://www.biopython.org/wiki/Remove_PDB_disordered_atoms contribution of Ramon Crehuet]. The DisorderedAtom objects are removed from the residue and a single Atom object is added corresponding to the location of the user's choice (keep_loc argument) which defaults to A.

An example, still keeping s from above:

<python>
s = s.remove_disordered_atoms(verbose=True)
0 residues were modified

# Now if we load a structure with disordered atoms
ds = Struct.read('1MC2.pdb')
ds.remove_disordered_atoms(verbose=True)
Residue TRP:1010 has 8 disordered atoms: CD1/CD2/NE1/CE2/CE3/CZ2/CZ3/CH2
Residue VAL:1018 has 3 disordered atoms: CB/CG1/CG2
Residue LEU:1024 has 4 disordered atoms: CB/CG/CD1/CD2
Residue ARG:1043 has 7 disordered atoms: CB/CG/CD/NE/CZ/NH1/NH2
Residue MET:1092 has 4 disordered atoms: CB/CG/SD/CE
Residue ARG:1107 has 7 disordered atoms: CB/CG/CD/NE/CZ/NH1/NH2
Residue GLU:1108 has 4 disordered atoms: CG/CD/OE1/OE2
Residue ASP:1111 has 4 disordered atoms: CB/CG/OD1/OD2
Residue SER:1116 has 1 disordered atoms: OG
Residue SER:1131 has 1 disordered atoms: O
10 residues were modified
</python>

GSOC2010 Joao

Joaor — Fri, 25 Jun 2010 22:12:18 GMT

Joaor: Week 3 and 4 Update.

== Author & Mentors ==

[[User:Joaor|João Rodrigues]] anaryin@gmail.com

'''Mentors'''
: Eric Talevich
: Diana Jaunzeikare
: Peter Cock

== Abstract ==

Biopython is a very popular library in Bioinformatics and Computational Biology. Its Bio.PDB module, originally developed by Thomas Hamelryck, is a simple yet powerful tool for structural biologists. Although it provides a reliable PDB parser feature and it allows several calculations (Neighbour Search, RMS) to be made on macromolecules, it still lacks a number of features that are part of a researcher's daily routine. Probing for disulphide bridges in a structure and adding polar hydrogen atoms accordingly are two examples that can be incorporated in Bio.PDB, given the module's clever structure and good overall organisation. Cosmetic operations such as chain removal and residue renaming – to account for the different existing nomenclatures – and renumbering would also be greatly appreciated by the community.

Another aspect that can be improved for Bio.PDB is a smooth integration/interaction layer for heavy-weights in macromolecule simulation such as MODELLER, GROMACS, AutoDock, HADDOCK. It could be argued that the easiest solution would be to code hooks to these packages' functions and routines. However, projects such as the recently developed [http://sbcb.bioch.ox.ac.uk/oliver/software/GromacsWrapper/html/edpdb.html edPDB] or the more complete [http://biskit.pasteur.fr/ Biskit library] render, in my opinion, such interfacing efforts redundant. Instead, I believe it to be more advantageous to include these software' input/output formats in Biopython's SeqIO and AlignIO modules. This, together with the creation of interfaces for model validation/structure checking services/software would allow Biopython to be used as a pre- and post-simulation tool. Eventually, it would pave the way for its inclusion in pipelines and workflows for structure modelling, molecular dynamics, and docking simulations.

== Project Schedule ==

The schedule below was organised to be flexible, which means that some features will likely be done early. Also, the weeks include documentation and unit testing efforts for the features, with extended periods for reviewing these efforts at the two points during the project (halfway, final week).

=== Community Bonding Period ===

*Getting familiar with development environment (Git Hub account, Git, Biopython's repository, Bug tracking system, etc)

*Gather scientific literature and discuss some of the to-be-implemented methods.

=== Week 1 [31st May - 6th June] ===

==== Renumbering residues of a structure ====

*Read SEQRES record to account for gaps
*Alternatively read ATOM records.

==== Probe disulphide bridges in the structure ====

*Via NeighbourSearch class
*Also use SSBOND in header

==== Extract Biological Unit ====

*REMARK350 contains rotation and translation information
*If REMARK is absent, do nothing.

=== Week 2 [7th – 13th June] ===

==== Structure Hydrogenation ====

*Add all/polar hydrogens through interface with WHATIF server.
*Optionally define a set pH
** [http://www3.interscience.wiley.com/journal/112117957/abstract pKa values algorithm]

==== Hydrogenation Report ====

*Produces a brief list of polar hydrogen atoms in the structure.
** Chain | Residue [number] | Atom

=== Weeks 3-5 [14th June- 4th July] ===

==== Removal of disordered atoms ====

*[[Remove_PDB_disordered_atoms|Solution proposed in the Biopython wiki]]

==== Residue name normalisation ====

*Build conversion table from different nomenclatures (research them during c.bonding period )
*Write function to make a given structure compliant with a given software nomenclature:
** Amber
** CNS/HADDOCK
** GROMACS

==== Coarse Grain Structure ====

*Implement function to reduce complexity of a structure
** 1pt*c-alpha
** 2pt*c-alpha / c-beta
** 3pt*c-alpha / c-beta / side-chain pseudo-centroid OR side-chain centroid

=== Week 6 (Mid-Term) [5th - 11th July] ===

: Testing and consolidating the features thoroughly.
: Write documentation & examples for each feature, to be included in Biopython's Wiki and Bio.PDB's FAQ.
: Mid-term Evaluations. Discussing with mentors current state of project and adjust following schedule to comply with project's needs.

=== Week 7 [12th - 19th July] ===

==== Add support for MODELLER's PIR format to Biopython ====

*[http://www.salilab.org/modeller/manual/node445.html#alignmentformat Format Description]
*SeqIO
*AlignIO

==== Allow conversion of Structure Object to Sequence Object ====

*Based on Bio.PDB.Polypeptide function

=== Weeks 8-10 [20th July - 9th August] ===

==== Add Sequence/Structure Homology functions ====

*Create call to Biopython's BLAST interfaces
** Allow direct blast from structure object ( e.g. protein.find_homoseq() )
** Returns list of tuples with E-Value *Dictionary (name, length of alignment, etc..)
*Create interface with structural homology web services
** e.g. [http://ekhidna.biocenter.helsinki.fi/dali_server/ Dali server]
** Return list of tuples with Z-Score*Dictionary (name, etc...)

==== Implement basic structure validation checks ====

*Via NeighbourSearch class
** Same Charge contacts
** Atom Clashes
*Via ResidueDepth Class
** Buried Charges
*Interface WHATIF PDBReport web service
** Parse WARNING and ERROR messages

=== Week 11 [10th - 17th August] ===

==== Reviewing documentation, code, write tests for new functions. ====

== Project Code ==

Hosted at [http://github.com/JoaoRodrigues/biopython/tree/GSOC2010 this GitHub branch]

== Project Progress ==

Since I'm adding some methods that are useful/logical only for proteins, having them exposed in Structure.py for every molecule could be misleading. We decided then to add a 'as_protein()' method that allows protein-specific methods to be accessed. The following example demonstrates how this call works. Note how the "search_ss_bonds" method is absent from dir(s) but not from dir(prot).

<python>
from Bio.PDB import PDBParser

p = PDBParser()
s = p.get_structure('example', '4PTI.pdb')

dir(s)
# Cut for viewing purposes
['__doc__', ... , 'renumber_residues', 'set_parent', 'xtra']

prot = s.as_protein()

dir(prot)

['__doc__', ... , 'renumber_residues', 'search_ss_bonds', 'set_parent', 'xtra']
</python>

=== Week 1 ===

==== Renumbering residues of a structure ====

Since parse_pdb_header is far from optimal and is likely to change in the future, I opted to forfeit reading SEQREQ records to account for gaps. However, ignoring this information and renumbering based on ATOM records would make us lose information on gaps. I opted to subtract the first residue number-1 to all residues thus making the numbering start in 1 and still keep gaps. I also added an argument (start) to allow the user to set which number to start the counting from.

Example:

<python>
from Bio.PDB import PDBParser

p = PDBParser()
s = p.get_structure('example', '1IHM.pdb')

print list(s.get_residues())[0]
<Residue ASP het= resseq=1029 icode= >

s.renumber_residues()
print list(s.get_residues())[0]
<Residue ASP het= resseq=1 icode= >
</python>

==== Probe disulphide bridges in the structure ====

The same rationale from SEQRES applies for the exclusion of looking up SSBOND. Also, instead of using NeighborSearch to look for pairs of cysteins in bond distance, I instead used the minus operator since it has been overloaded to return the distance between two atoms (Page 10 of the [http://www.biopython.org/DIST/docs/cookbook/biopdb_faq.pdf FAQ]). The average distance cited in the literature is 2.05A but other software packages and my own tests set 3.0A as a good threshold. Still, the user can set his own threshold manually.

The function returns an iterator with tuples of pairs of residues.

<python>
from Bio.PDB import PDBParser

p = PDBParser()
s = p.get_structure('example', '4PTI.pdb')

prot = s.as_protein()

for bond in prot.search_ss_bonds():
print bond

(<Residue CYS het= resseq=5 icode= >, <Residue CYS het= resseq=55 icode= >)
(<Residue CYS het= resseq=14 icode= >, <Residue CYS het= resseq=38 icode= >)
(<Residue CYS het= resseq=30 icode= >, <Residue CYS het= resseq=51 icode= >)
</python>

==== Extract Biological Unit ====

Added parsing for REMARK350 to parse_pdb_header since there was already a bit written for another REMARK section. This extracts the transformation matrices and the translation vector from the header, that is then fed to the Structure function. Each new rotated structure is created as a new MODEL. I chose this because crystal structures very rarely have more than one MODEL instance and also because NMR models don't have REMARK 350 that often (at least to my knowledge).

<python>
from Bio.PDB import PDBParser

p = PDBParser()

s1 = p.get_structure('a', '4PTI.pdb')
s1.build_biological_unit()
'Processed 0 transformations on the structure.' # Identity matrix is ignored.

s2 = p.get_structure('b', 'homol_1bd8.pdb') # A homology model
s2.build_biological_unit()
'PDB File lacks appropriate REMARK 350 entries to build Biological Unit.'

s3 = p.get_structure('c', '1IHM.pdb')
s3.build_biological_unit()
'Processed 59 transformations on the structure.'
</python>

=== Weeks 2-5 ===

==== Hydrogenation of PDB files ====

Following discussion between the mentors and me, we decided that maybe it was better to not only include a webserver for this purpose but also a local algorithm. This would not limit the user when there he/she lacks an internet connection.

The interface for the WHATIF Protonation service has been implemented, although it should be regarded as **highly experimental** for now. Interfacing this server included writing a small parser for a PDBXML-like format, which is expected to have serious bugs in its initial versions. I ran some simple tests and it works. It doesn't support water molecules yet, nor any other molecules other than proteins. Such issues will be hopefully solved later on..

For those brave enough to want to test it (and help me debug it), here's an example usage.

<python>
from Bio.Struct.WWW import WHATIF
from Bio import Struct

server = WHATIF.WHATIF() # Performs a sort of PING to the server. Gracefully exits if the servers are down.

# Get the protein structure
structure = Struct.read('4PTI.pdb')
protein = structure.as_protein() # This excludes water molecules

# Upload the structure to the WHATIF server
# This should convert the structure from a Structure object to a string via tempfile and PDBIO
# I was having some issues uploading structures...

id = server.UploadPDB(protein)

# Protonate
# Returns a Structure Object / WARNING! Bug prone for now.

protein_h = server.PDBasXMLwithSymwithPolarH(id)

</python>

Regarding the local implementation, after much reading I settled on using PyMol's algorithm. It seems to allow for protonation of any structure, regardless of its nature (protein, DNA, etc). Its vectorial and matrix operations can likely be optimized with Numpy and Biopython's Vector.py module.

==== Coarse Grain Structure ====

A Center of Mass function was developed first as part of a new module Bio.Struct.Geometry. It allows for calculation of the center of geometry (all masses are equal) and center of mass (taking into account elemental masses for the atoms). The masses are a new Atom object feature derived from [http://www.chem.qmul.ac.uk/iupac/AtWt/ this list] and from PyMol. Essentially, all atoms of a structure now get their mass defined when the Structure is created (check Atom.py and [http://lists.open-bio.org/pipermail/biopython-dev/2010-June/007880.html this thread] for details). This is obviously experimental.

To calculate the center of mass of any Entity (Structure, Model, Chain, Residue) or a List of Atoms:

<python>
from Bio.Struct.Geometry import center_of_mass
from Bio import Struct

s = Struct.read('4PTI.pdb')

print center_of_mass.__doc__

Returns gravitic or geometric center of mass of an Entity.
Geometric assumes all masses are equal (geometric=True)
Defaults to Gravitic.

print center_of_mass(s)
[14.833301303933874, 21.431581746366263, 4.1218478418007134]

print center_of_mass(s, geometric=True)
[14.805324902127458, 21.365571977563405, 4.1108949403803985]
</python>

Coarse-Graining makes use of this function when reducing each residue to 3 points (ENCAD-like): Ca, peptide-bond O, and center of mass of side chain. The default CG function reduces the protein to a Ca-trace. I am still working on implementing perhaps some other CG-representations such as that from CABS.

An example, picking up the s Structure from above:

<python>
p = s.as_protein() # To expose the CG method

ca_trace = p.coarse_grain()

# One atom per residue
print ( len(list(p.get_residues())) == len(list(ca_trace.get_atoms())) )
True

cg_3pt = p.coarse_grain('3pt')

for residue in cg_3pt.get_residues():
print residue.resname, residue.child_list

ARG [<Atom CA>, <Atom O>, <Atom CMA>]
PRO [<Atom CA>, <Atom O>, <Atom CMA>]
ASP [<Atom CA>, <Atom O>, <Atom CMA>]
PHE [<Atom CA>, <Atom O>, <Atom CMA>]
CYS [<Atom CA>, <Atom O>, <Atom CMA>]
LEU [<Atom CA>, <Atom O>, <Atom CMA>]
GLU [<Atom CA>, <Atom O>, <Atom CMA>]
PRO [<Atom CA>, <Atom O>, <Atom CMA>]
PRO [<Atom CA>, <Atom O>, <Atom CMA>]
TYR [<Atom CA>, <Atom O>, <Atom CMA>]
.....
CYS [<Atom CA>, <Atom O>, <Atom CMA>]
GLY [<Atom CA>, <Atom O>]
GLY [<Atom CA>, <Atom O>]
ALA [<Atom CA>, <Atom O>, <Atom CMA>]
</python>

==== Removal of disordered atoms ====

Implement as part of Structure.py and based loosely on the [http://www.biopython.org/wiki/Remove_PDB_disordered_atoms contribution of Ramon Crehuet]. The DisorderedAtom objects are removed from the residue and a single Atom object is added corresponding to the location of the user's choice (keep_loc argument) which defaults to A.

An example, still keeping s from above:

<python>
s = s.remove_disordered_atoms(verbose=True)
0 residues were modified

# Now if we load a structure with disordered atoms
ds = Struct.read('1MC2.pdb')
ds.remove_disordered_atoms(verbose=True)
Residue TRP:1010 has 8 disordered atoms: CD1/CD2/NE1/CE2/CE3/CZ2/CZ3/CH2
Residue VAL:1018 has 3 disordered atoms: CB/CG1/CG2
Residue LEU:1024 has 4 disordered atoms: CB/CG/CD1/CD2
Residue ARG:1043 has 7 disordered atoms: CB/CG/CD/NE/CZ/NH1/NH2
Residue MET:1092 has 4 disordered atoms: CB/CG/SD/CE
Residue ARG:1107 has 7 disordered atoms: CB/CG/CD/NE/CZ/NH1/NH2
Residue GLU:1108 has 4 disordered atoms: CG/CD/OE1/OE2
Residue ASP:1111 has 4 disordered atoms: CB/CG/OD1/OD2
Residue SER:1116 has 1 disordered atoms: OG
Residue SER:1131 has 1 disordered atoms: O
10 residues were modified
</python>

GSOC2010 Joao

Joaor — Thu, 17 Jun 2010 20:39:16 GMT

Joaor: Weeks 2 and 3. Hydrogenation Discussion and Coarse Graining.

GSOC2010 Joao

Joaor — Fri, 11 Jun 2010 16:01:15 GMT

Joaor: Added progress

GSOC2010 Joao

Joaor — Thu, 20 May 2010 01:23:05 GMT

Joaor:

User:Joaor

Joaor — Wed, 19 May 2010 15:13:37 GMT

Joaor:

Hello :)

''' About Me '''

This is the Wiki page of João Rodrigues, currently doing research in protein structure refinement with [http://csb.stanford.edu Prof. Michael Levitt] at Stanford University. I have previously worked with [http://haddock.chem.uu.nl Prof. Alexandre Bonvin] in protein-protein docking at Utrecht University.

''' Contributions to Biopython '''

I have been a member of the Biopython community since 2007 but only became an active participant with the GSOC 2010 programme. My project is on the enrichment of the Bio.PDB module with several features that will hopefully make it more attractive and useful to the structural biology community.

A detailed layout of my project is to be presented [[GSOC2010_Joao|elsewhere]].

----

[http://stanford.edu/~joaor My personal page] || Last updated: 05.2010

GSOC2010 Joao

Joaor — Thu, 13 May 2010 08:18:36 GMT

Joaor: Creation

Active projects

Joaor — Thu, 13 May 2010 07:25:37 GMT

Joaor: /* Current projects */ Added GSOC 2010 project

This page provides a central location to collect references to active projects. This is a good place to start if you are interested in contributing to Biopython and want to find larger projects in progress. For developers, use this to reference [[git]] branches or other projects which you will be working on for an extended period of time. Please keep it up to date as projects are finished and integrated into Biopython.

== Current projects ==

=== Population Genetics development ===

Giovanni and Tiago are working on expanding population genetics code in Biopython. See the [[PopGen_dev|PopGen development page]] for more details.

=== GFF parser ===

Brad is working on a Biopython GFF parser. Source code is available from [http://github.com/chapmanb/bcbb/tree/master/gff git hub]. Documentation is in progress at [[GFF Parsing]]. See blog posts on the [http://bcbio.wordpress.com/2009/03/08/initial-gff-parser-for-biopython/ initial implementation] and [http://bcbio.wordpress.com/2009/03/22/mapreduce-implementation-of-gff-parsing-for-biopython/ MapReduce parallel version].

=== Phylo ===

[[User:EricTalevich|Eric]] is working on a new module for phylogenetics, [[Phylo|Bio.Phylo]]. It grew out of a [[Google Summer of Code]] 2009 [http://socghop.appspot.com/student_project/show/google/gsoc2009/nescent/t124022798969 project], mentored by Brad, to add support for [http://www.phyloxml.org/ phyloXML] to Biopython; it also refactors part of Bio.Nexus. Most of the code has been pushed to the main development branch on GitHub already, but new features appear first on Eric's [http://github.com/etal/biopython/tree/phyloxml phyloxml branch].

=== Biogeography ===

[[Matzke|Nick]] is working on developing a Biogeography module for BioPython. This work was funded by a [[Google Summer of Code]] 2009 [http://socghop.appspot.com/program/home/google/gsoc2009 project] through NESCENT's [https://www.nescent.org/wg_phyloinformatics/Phyloinformatics_Summer_of_Code_2009 Phyloinformatics Summer of Code 2009]. See the project proposal at: [http://socghop.appspot.com/student_project/show/google/gsoc2009/nescent/t124022798250 Biogeographical Phylogenetics for BioPython]. The mentors are [http://blackrim.org/ Stephen Smith] (primary), [http://bcbio.wordpress.com/ Brad Chapman], and [http://evoviz.nescent.org/ David Kidd]. The code currently lives at the Bio/Geography directory of the [http://github.com/nmatzke/biopython/tree/Geography Geography fork of the nmatzke branch on GitHub], and you can see a timeline and other info about ongoing development [[BioGeography|here]]. The new module is being documented on this wiki as [[BioGeography]].

=== Roche 454 SFF parsing in Bio.SeqIO ===

See [http://bugzilla.open-bio.org/show_bug.cgi?id=2837 Bug 2837], based on code from Jose Blanca. Recently merged into the trunk and should be included in Biopython 1.54 onwards.

=== Multiple Sequence Alignments ===

Peter is working on a new alignment class for sequence alignments (not the kind in next gen sequencing), the core of which was recently merged into the trunk and should be included in Biopython 1.54 onwards.

=== Open Enhancement Bugs ===

This [http://bugzilla.open-bio.org/buglist.cgi?product=Biopython&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_severity=enhancement Bugzilla Search] will list all open enhancement bugs (any filed by core developers are fairly likely to be integrated, some are just wish list entries).

=== (GSOC 2010) Extending Bio.PDB ===

GSOC 2010 project that aims to introduce several new features to the Bio.PDB structural biology module. Includes functions to add polar hydrogens to structures, probing for SS bridges based on structural information and annotations, renumbering residues, coarse-graining a structure, etc. A more comprehensive layout of the project is available [[GSOC2010_Joao|here]].

== Project ideas ==

Please add any ideas or proposals for new additions to Biopython. Bugs and enhancements for current code should be discussed though our bugzilla interface.

* Build a general tool to filter sequences containing ambiguous or low quality bases. Chris Fields from BioPerl is interested in coordinating the BioPerl/Biopython implementations. See these threads on the mailing lists for discussion: http://lists.open-bio.org/pipermail/biopython/2009-July/005355.html, http://lists.open-bio.org/pipermail/biopython/2009-July/005342.html

* Use SQLAlchemy, an object relational mapper, for BioSQL internals. This would add an additional external dependency to Biopython, but provides ready support for additional databases like SQLite. It also would provide a raw object interface to BioSQL databases when the SeqRecord-like interface is not sufficient. Brad and Kyle have some initial code for this.

* Revamp the GEO SOFT parser, drawing on the ideas used in [http://www.bioconductor.org/packages/bioc/html/GEOquery.html Sean Davis' GEOquery parser in R/Bioconductor]. See also [http://www.warwick.ac.uk/go/peter_cock/r/geo/ this page].

== Enhancement list ==

Maintaining software involves incremental improvements for new format changes and removal of bugs. Please see our [http://bugzilla.open-bio.org/ bugzilla] page for a current list. Post to the developer mailing list if you are interested in tackling any open issues.

User:Joaor

Joaor — Thu, 13 May 2010 07:16:54 GMT

Joaor: Creation.

Hello :)

''' About Me '''

This is the Wiki page of João Rodrigues, currently doing research in protein structure refinement with [http://csb.stanford.edu Prof. Michael Levitt] at Stanford University. I have previously worked with [http://haddock.chem.uu.nl Prof. Alexandre Bonvin] in protein-protein docking at Utrecht University.

''' Contributions to Biopython '''

I have been a member of the Biopython community since 2007 but only became an active participant with the GSOC 2010 programme. My project involves enrich the Bio.PDB module with several features that will hopefully make it more attractive and useful to the structural biology community.

A detailed layout of my project is to be presented [[GSOC2010_Joao|elsewhere]].

----

[http://stanford.edu/~joaor My personal page] || Last updated: 05.2010