CAPRI or: What is the State of Protein-Protein Docking?

This is the first post in a series, summarizing the CAPRI (Critical Assessment of PRediction of Interactions) 4th Evaluation meeting. In this post I’ll try to give a more personal perspective of the experiment results, the state and trends of computational protein-protein docking and the vibes behind the scenes. The next posts in the series will shortly summarize select talks from the meeting, kindly provided by the speakers.

The Experiment

The protein-protein docking community gathered in the beginning of December ’09 in an old town-house right at the heart of downtown Barcelona, to evaluate the results of the on-going, now almost 10 years old CAPRI (critical assessment of prediction of interactions) blind experiment. In CAPRI much as in CASP experimentalists that solve the structure of a protein-protein complex, graciously delay the publication of the coordinates and give the participating groups the chance to predict the structure of the complex. In order to assess the ability to select correct models, following each “prediction” round, there is a scoring round in which the participants are asked to select the 10 best models out of a pool of models uploaded by the different predictors.

The Numbers

This evaluation meeting covered 15 targets from rounds 13-19 of the CAPRI experiment originating from 11 complexes for most of which an unbound structure for one or both of the partners was available. All in all 67 predictor groups participated, 10 automatic servers and 31 scorers. On average there were ~30-40 participants per target.

The Conclusions

* Easy targets are easy:
For several targets there was an overwhelming success by almost all groups. These were Enzyme-Inhibitor complexes in which there is almost no conformational change between the bound and unbound monomer. This is in good agreement with the fact that the performance of ‘unbound’ docking, approaches that of ‘bound’ docking when there is not much conformational change. Besides this fact, for some of these targets an already solved homologue complex existed and could be easily used to predict the binding mode. For the hard cases that involved large conformational changes, usually no one succeeded in predicting a correct model.

* No dramatic improvement since the last evaluation:
As mentioned before, several targets with large conformational changes were still too difficult for current approaches. Here lie the benefit of CAPRI as it drives the community to develop new techniques. In previous rounds the big challenge was to model correctly side-chain conformations. Today, side-chain modeling during docking is standard and development is focused on backbone flexibility (see below). The percentage of successful targets in CAPRI rounds 13-19 is about the same as that of CAPRI rounds 1-12 (about 70%) indicating that there was no big breakthrough, although the targets did seem to be harder. The collective failure rate (targets for which no correct model was submitted) is also about the same (about 15%). Similar to what is happening in CASP, automatic servers are beginning to show good results although they still do worse than the human predictor groups.

* Hierarchical approach for docking:  (A good review on the subject)
It seems that many of the groups partition the docking into two parts: the first is the global search for the binding mode of the two partners. This is usually done using FFT based approaches (such as “ZDock” and “Piper“) or Geometric Hashing (“PatchDock“). These approaches are fast and show high success rate in finding an approximate binding mode (especially when there is no conformational change). The second step is high-resolution refinement, usually including side-chain flexibility (and backbone flexibility in some approaches) for which many groups use “RosettaDock” or a similar approach. Indeed, analysis of the best solutions for all targets showed that the best models display the least number of clashes – this is the result of the high-resolution refinement which can also improve the ranking of the models.

* NMA (Normal Modes Analysis) for modeling backbone flexibility:
NMA is the method of choice for many groups to sample conformational backbone variability. Whether the usage of the normal modes is integrated into the docking simulation (Such as “Attract” and “FiberDock“), investigated before the docking (E.g. work by M. Sternberg, work by I. Bahar) or used to create a starting ensemble of structures (one of the possible input ensembles for “Rosetta EnsembleDock“). The usage of normal modes for modeling backbone flexibility is on the rise.

* Binding site predictions and biological information:
Most of the groups usually conduct a literary search on each target during the prediction round. This biological data could be used either as a guide for the docking software during the simulation, or as a filter to get rid of false positives a-posteriori. Biological information turned out to be a double-edged sword. Indeed in many cases it improves the prediction by minimizing the number of false positives and narrowing the search space, but for at least one target incorrect biological information mislead the entire community – the only successful predictions were made by “automatic” software.

Many groups incorporate binding site predictions (biological or bioinformatics) into their search such as “HADDOCK“. There are different approaches to define the binding site, one that is gaining popularity is to use the results of an exhaustive global docking run (or the decoys themselves) to identify residues that have high propensity to be at the interface (E.g. Vajda, Fernandez-Recio, Weng). In fact, since most groups rely on binding sites predictions at some level, it has been proposed that from the next round binding site prediction will be a category in itself.

* RNA docking:
For the first time Nucleic acids (RNA) docking was introduced into CAPRI. The round consisted of one target in which the unbound state of the RNA was provided followed by a target in which the bound state of the RNA was provided. As in Protein-Protein docking when there is a great conformational change in one of the partners (as indeed was the case with the RNA molecule) no program was able to predict the correct complex. However when the bound RNA was provided several groups produced successful predictions. This is very promising for the development of such Nucleic acid docking methods.

The “Winners”

Just to satisfy the readers curiosity, below is a list of the 10 “best” groups. This ranking should be taken with a grain of salt as: A) it is not official (it’s based largely on number of successful predictions) and B) the differences between the leading groups are really not that dramatic. No group got ALL the targets correctly (though all groups got the easy targets right 😉 ) Most of the leading groups (human groups) had about 6 successes.

# Human groups: Automatic Servers: Scoring Exp.:
1 Sandor Vajda CLUSPRO Alexandre Bonvin
2 Martin Zacharias HADDOCK Paul Bates
3 Xiaoqin Zou GRAMM-X Xiaoqin Zou
4 Haim Wolfson, Miriam Eisenstein SKE-DOCK Zhiping Weng
5 Huan-Xiang Zhou, Zhiping Weng PatchDock, FireDock, FiberDock Wang Cunxin
6 Alexandre Bonvin TOP-DOWN Juan Fernandez-Recio
7 Juan Fernandez-Recio Haim Wolfson
8 Jeffrey Gray Haliloglu, Camacho,Takeda-Shitaka

The Acknowledgments

This post is largely based on talks by Joël Janin, Marc Lensink and Shoshana Wodak and my personal impressions. I’d also like to thank the CAPRI management team { Kim Henrick (EBI-EMBL, Hinxton, UK), Joël Janin (LEBS-CNRS, Gif-sur-Yvette, France), John Moult (CARB, Rockville, USA), Lynn Ten Eyck (University of California San Diego, USA), Michael Sternberg (Imperial College London, UK), Sandor Vajda (Boston University, Boston, USA), Ilya Vakser (SUNY at Stony Brook, USA), Shoshana Wodak (The Hospital for Sick Children, University of Toronto, CA) } the evaluation meeting organizing committee { Juan Fernandez-Recio (Barcelona Supercomputing Center, Spain), Michael Sternberg (Imperial College London, UK), Joël Janin (LEBS-CNRS, Gif-sur-Yvette, France), Modesto Orozco (Univ. Barcelona, IRB, BSC, Spain), Josep Lluis Gelpí (Univ. Barcelona, IRB, BSC, Spain), Patrick Aloy (ICREA, IRB, Spain) } Sameer Velankar from the PDB, and the experamentalists which made the CAPRI possible by providing the coordinates:

T29 N. Leulliot, H. van Tilbeurgh IBBMC, Orsay, France
T30 W. Tempel, Y. Tong SGC, University of Toronto, Canada
T31 M. Buck Case Western, Cleveland, Ohio
T32 L. de Maria Novozymes, Bagsvaerd DK
T33/34 L. Renault LEBS, Gif-sur-Yvette, France
T35/36 S. Najmudin, C Fontes University of Lisbon, Portugal
T37 J. Ménétrey Institut Curie, Paris, France
T38/39 H. Park SGC, University of Toronto, Canada
T40 R. Bao, Y. Chen USTC, Hefei, China
T41 N. Meenan York University, UK
T42 C. Kleanthous York University, UK

Some CAPRI related citations:
Wodak SJ (2007). From the Mediterranean coast to the shores of Lake Ontario: CAPRI’s premiere on the American continent. Proteins, 69 (4), 697-8 PMID: 17912754

Janin J (2005). Sailing the route from Gaeta, Italy, to CAPRI. Proteins, 60 (2) PMID: 15981247

Janin, J. (2003). Docking in CAPRI Proteins: Structure, Function, and Genetics, 52 (1), 1-1 DOI: 10.1002/prot.10398

Janin J, & Wodak S (2007). The third CAPRI assessment meeting Toronto, Canada, April 20-21, 2007. Structure (London, England : 1993), 15 (7), 755-9 PMID: 17637336

Janin J (2005). Assessing predictions of protein-protein interaction: the CAPRI experiment. Protein science : a publication of the Protein Society, 14 (2), 278-83 PMID: 15659362

Vajda S, Vakser IA, Sternberg MJ, & Janin J (2002). Modeling of protein interactions in genomes. Proteins, 47 (4), 444-6 PMID: 12001222

Be Sociable, Share!
Written by Nir London in: Events,Weird science | Tags: , , , ,

Enjoyed this Post ?

Subscribe by E-mail:

Subscribe in a reader. Follow us on twitter.

Powered by WordPress | Aeros Theme | TheBuckmaker.com WordPress Themes
© 2009 Rosetta Design Group LLC