NEW: SidechainNet v1.0 introduces new features (e.g., OpenMM interaction, hydrogen support) and several breaking changes. Please see our release notes for more information. An updated walkthrough is available below.
Colab Walkthrough v1.0.1, SidechainNet Paper, OpenMM-Loss Paper
SidechainNet is a protein structure prediction dataset that directly extends ProteinNet2 by Mohammed AlQuraishi.
Specifically, SidechainNet adds measurements for protein angles and coordinates that describe the complete, all-atom protein structure (backbone and sidechain, excluding hydrogens) instead of the protein backbone alone.
This repository provides the following:
- SidechainNet datasets stored as pickled Python dictionaries.
- Methods for loading and batching SidechainNet data efficiently in PyTorch.
- Methods for generating protein structure visualizations (
.pdb,3Dmol,.gltf) from model predictions. - Methods for augmenting SidechainNet to include new proteins and specify dataset organization.
- Interfacing with OpenMM to compute protein energy and atomic forces.
conda install -c conda-forge openmm
pip install sidechainnet
These attributes can be accessed via SCNProtein or ProteinBatch objects.
| Entry | Dimensionality* | Attribute name | ProteinNet | SidechainNet |
|---|---|---|---|---|
| Primary sequence§ | L | seq |
X | X |
| DSSP Secondary structure**,§ | L | secondary_structure |
X | X |
| PSSM + Information content | L x 21 | evolutionary |
X | X |
| Missing residue mask§ | L | mask |
X | X |
| Backbone coordinates | L x 4*** x 3 | coords, subset [0:5] |
X | X |
| Backbone torsion angles | L x 3 | angles, subset [0:3] |
X | |
| Backbone bond angles | L x 3 | angles, subset [3:6] |
X | |
| Sidechain torsion angles | L x 6 | angles, subset [6:12] |
X | |
| Sidechain coordinates | L x 10 x 3 | coords, subset [5:15] |
X | |
| Experimental resolution | 1 | resolution |
X | |
| Unmodified (non-standardized) sequence | L | unmodified_seq |
X |
*L reperesents the length of any given protein in the dataset.
**Secondary structure is acquired from ProteinNet for training sets only. Blank characters are added by SidechainNet to refer to unmatched amino acids after alignment.(Added January 2021)
***SidechainNet explicitly includes oxygen atoms as part of the backbone coordinate data in contrast to ProteinNet, which only includes the primary N, C_alpha, C atoms.
§Stored as string values in the underlying SidechainNet data dictionary.
For hands on learning to use SidechainNet, see our walkthrough above. Here are the basics:
The easiest way to interact with SidechainNet is most likely by using the SCNDataset and
SCNProtein objects.
>>> data = scn.load("debug", scn_dataset=True)
>>> data
SCNDataset(n=461)
>>> data["1HD1_1_A"]
SCNProtein(1HD1_1_A, len=75, missing=0, split='train')
>>> data[0]
SCNProtein(2CMY_d2cmyb1, len=23, missing=2, split='train')Available features:
SCNDatasetis iterable,- proteins (
SCNProteins) can selected from the dataset by name or index, - proteins can be visualized with
.to_3Dmol()and writable to PDBs with.to_pdb(). - non-terminal hydrogens can be added with
SCNProtein.add_hydrogens(),
Additionally, all of the attributes below are available directly from the SCNProtein object:
coords, angles, seq, unmodified_seq, mask, evolutionary, secondary_structure, resolution, is_modified, id, split
The load function can also be used to load SidechainNet data as a dictionary of torch.utils.data.DataLoader objects. PyTorch DataLoaders make it simple to iterate over dataset items for training machine learning models. This method is recommended for using SidechainNet data with PyTorch.
Iterating over our Dataloaders in this manner yields ProteinBatch objects that facilitate
training and data collation/padding.
>>> dataloaders = scn.load(casp_version=12, casp_thinning=30, with_pytorch="dataloaders")
>>> dataloaders.keys()
['train', 'train_eval', 'valid-10', ..., 'valid-90', 'test']
>>> for protein_batch in dataloaders['train']:
.... pred_protein = model(protein_batch.seqs)
.... loss = compute_loss(protein_batch.angles, protein_batch.coords, # True values
.... pred_protein.angles, pred_protein.coords) # Predicted values
.... ...An important component of this work is the inclusion of both angular and 3D coordinate representations of each protein. Researchers who develop methods that rely on angular representations may be interested in converting this information into 3D coordinates. For this reason, SidechainNet provides a method to convert the angles it provides into Cartesian coordinates.
>>> d = scn.load(casp_version=12, casp_thinning=30)
>>> p = d[100]
>>> p.fastbuild(inplace=True)Gradient-friendly methods for adding hydrogens to protein structures.
>>> p.add_hydrogens()
>>> # OR
>>> p.fastbuild(add_hydrogens=True, inplace=True) # builds atoms (including Hs) from anglesSidechainNet also makes it easy to visualize both existing and predicted all-atom protein structures. These visualizations are available as PDB files, py3Dmol.view objects, and Graphics Library Transmission Format (gLTF) files.
>>> sb.to_pdb("example.pdb")
>>> sb.to_gltf("example.gltf")>>> p.get_energy()
>>> # OR
>>> p.fastbuild(add_hydrogens=True, inplace=True) # builds atoms (including Hs) from angles
Quantity(value=-3830.165405454086, unit=kilojoule/mole)If you would like to reproduce our work or make modifications/additions to the dataset, please see
the example we provide in our walkthrough (above). In simple terms, you will need to call scn.create
with the desired CASP/ProteinNet information or provide a list of ProteinNet-formatted IDs to
scn.create_custom. Please note that since some data is acquired from ProteinNet directly (e.g., Position Specific Scoring Matrices), protein entries will exclude this data if it was not previously available in ProteinNet.
# Reproduce SidechainNet
scn.create(casp_version=12, training_set=30)
# Create a custom version of SidechainNet
custom_ids = scn.get_proteinnet_ids(casp_version=12, split="train", thinning=30)
# Include a protein released in April 2021 (not included in SidechainNet)
custom_ids += ['7C3K_1_A']
scn.create_custom(pnids=custom_ids,
output_filename="custom.pkl",
short_description="Custom SidechainNet.")- Python 3
- ProDy (
pip install ProDy)- Biopython
- numpy
- scipy
- OpenMM v8 or greater (
conda install -c conda-forge openmm) - PyTorch
- tqdm
- py3Dmol (
pip install py3Dmol) - pymol (optional, for
gltfsupport, repo, linux install requireslibxml2) - pytorch_lightning (optional, for model training examples)
Thanks to Mohammed AlQuraishi for building ProteinNet, upon which our work is built. Thanks, also, to Jeppe Hallgren for his development of a ProteinNet text record parser, which I have used in part here.
This work is supported by R01GM108340 from the National Institute of General Medical Sciences, is supported in part by the University of Pittsburgh Center for Research Computing through the resources provided, and by NIH T32 training grant T32 EB009403 as part of the HHMI-NIBIB Interfaces Initiative.
Project structure (continuous integration, docs, testing) based on the Computational Molecular Science Python Cookiecutter version 1.1.
- SidechainNet: An All-Atom Protein Structure Dataset for Machine Learning. J.E. King, D. Koes. Proteins: Structure, Function, and Bioinformatics (Vol. 89, Issue 11, pp. 1489–1496). Wiley. (2021).
- ProteinNet: a standardized data set for machine learning of protein structure.. M. AlQuraishi. BMC Bioinformatics 20, 311 (2019).
- 3dmol.js: molecular visualization with WebGL. N. Rego and D. Koes. Bioinformatics, 31(8):1322–1324, (2014).
Computational Biology Skills Seminar, U.C. Berkeley, May 13, 2021
Please cite our paper(s) if you find SidechainNet useful in your work.
@article{https://doi.org/10.1002/prot.26169,
author = {King, Jonathan Edward and Koes, David Ryan},
title = {SidechainNet: An all-atom protein structure dataset for machine learning},
journal = {Proteins: Structure, Function, and Bioinformatics},
volume = {89},
number = {11},
pages = {1489-1496},
keywords = {dataset, deep learning, machine learning, protein structure, proteins, software},
doi = {https://doi.org/10.1002/prot.26169},
url = {https://onlinelibrary.wiley.com/doi/abs/10.1002/prot.26169},
eprint = {https://onlinelibrary.wiley.com/doi/pdf/10.1002/prot.26169},
abstract = {Abstract Despite recent advancements in deep learning methods for protein structure prediction and representation, little focus has been directed at the simultaneous inclusion and prediction of protein backbone and sidechain structure information. We present SidechainNet, a new dataset that directly extends the ProteinNet dataset. SidechainNet includes angle and atomic coordinate information capable of describing all heavy atoms of each protein structure and can be extended by users to include new protein structures as they are released. In this article, we provide background information on the availability of protein structure data and the significance of ProteinNet. Thereafter, we argue for the potentially beneficial inclusion of sidechain information through SidechainNet, describe the process by which we organize SidechainNet, and provide a software package (https://github.com/jonathanking/sidechainnet) for data manipulation and training with machine learning models.},
year = {2021}
}@article {King2023.10.03.560775,
author = {Jonathan Edward King and David Ryan Koes},
title = {Interpreting Molecular Dynamics Forces as Deep Learning Gradients Improves Quality Of Predicted Protein Structures},
elocation-id = {2023.10.03.560775},
year = {2023},
doi = {10.1101/2023.10.03.560775},
publisher = {Cold Spring Harbor Laboratory},
URL = {https://www.biorxiv.org/content/early/2023/10/05/2023.10.03.560775},
eprint = {https://www.biorxiv.org/content/early/2023/10/05/2023.10.03.560775.full.pdf},
journal = {bioRxiv}
}
Copyright (c) 2023 Jonathan King and the University of Pittsburgh