Crystallographic image processing
This article has multiple issues. Please help improve it or discuss these issues on the talk page. (Learn how and when to remove these template messages)
|
Crystallographic image processing (CIP) is traditionally understood as being a set of key steps in the determination of the atomic structure of crystalline matter from high-resolution electron microscopy (
HREM image contrasts and crystal potential reconstruction methods
Many beam HREM images of extremely thin samples are only directly interpretable in terms of a projected
Brief history of crystallographic image processing
Aaron Klug suggested in 1979 that a technique that was originally developed for structure determination of membrane protein structures can also be used for structure determination of inorganic crystals.[5][6] This idea was picked up by the research group of Sven Hovmöller which proved that the metal framework partial structure of the K8−xNb16−xW12+xO80 heavy-metal oxide could be determined from single HREM images recorded at Scherzer defocus.[7] (Scherzer defocus ensures within the weak-phase object approximation a maximal contribution to the image of elastically scattered electrons that were scattered just once while contributions of doubly elastically scattered electrons to the image are optimally suppressed.)
In later years the methods became more sophisticated so that also non-Scherzer images could be processed.[8] One of the most impressive applications at that time was the determination of the complete structure of the complex compound Ti11Se4, which has been inaccessible by X-ray crystallography.[9] Since CIP on single HREM images works only smoothly for layer-structures with at least one short (3 to 5 Å) crystal axis, the method was extended to work also with data from different crystal orientations (= atomic resolution electron tomography). This approach was used in 1990 to reconstruct the 3D structure of the mineral staurolite HFe2Al9Si4O4 [10][11] and more recently to determine the structures of the huge quasicrystal approximant phase ν-AlCrFe [12] and the structures of the complex zeolites TNU-9 [13] and IM-5.[14] As mentioned below in the section on crystallographic processing of images that were recorded from 2D periodic arrays with other types of microscopes, the CIP techniques were taken up since 2009 by members of the scanning transmission electron microscopy, scanning probe microscopy and applied crystallography communities.
Contemporary robotics and computer vision researchers also deal with the topic of "computational symmetry",[15][16][17][18][19] but have so far failed to utilize the spatial distribution of site symmetries that result from crystallographic [20] origin conventions. In addition, a well known statistician noted in his comments on "Symmetry as a continuous feature" [21] that symmetry groups possess inclusion relations (are not disjoint in other words) so that conclusions about which symmetry is most likely present in an image need to be based on "geometric inferences".[22] Such inferences are deeply rooted in information theory, where one is not trying to model empirical data, but extracts and models the information content of the data.[23][24] The key difference between geometric inference and all kinds of traditional statistical inferences is that the former merely states the co-existence of a set of definitive (and exact geometrical) constraints and noise, whereby noise is nothing else but an unknown characteristic of the measurement device and data processing operations. From this follows that "in comparing two" (or more) "geometric models we must take into account the fact that the noise is identical (but unknown) and has the same characteristic for both" (all) "models".[25] Because many of these approaches use linear approximations, the level of random noise needs to be low to moderate, or in other words, the measuring devices must be very well corrected for all kinds of known systematic errors.
These kinds of ideas have, however, only been taken up by a tiny minority of researchers within the computational symmetry [26] and scanning probe microscopy / applied crystallography [27][28] communities. It is fair to say that the members of computational symmetry community are doing crystallographic image processing under a different name and without utilization of its full mathematical framework (e.g. ignorance to the proper choice of the origin of a unit cell and preference for direct space analyses). Frequently, they are working with artificially created 2D periodic patterns, e.g. wallpapers, textiles, or building decoration in the Moorish/Arabic/Islamic tradition. The goals of these researchers are often related to the identification of point and translation symmetries by computational means and the subsequent classifications of patterns into groups. Since their patterns were artificially created, they do not need to obey all of the restrictions that nature typically imposes on long range periodic ordered arrays of atoms or molecules.
Computational geometry takes a broader view on this issue and concluded already in 1991 that the problem of testing approximate point symmetries in noisy images is in general
Crystallographic image processing of high-resolution TEM images
The principal steps for solving a structure of an inorganic crystal from HREM images by CIP are as follows (for a detailed discussion see [31]).
- Selecting the area of interest and calculation of the Fourier transform (= power spectrum consisting of a 2D periodic array of complex numbers)
- Determining the defocus value and compensating for the contrast changes imposed by the objective lens (done in Fourier space)
- Indexing and refining the lattice (done in Fourier space)
- Extracting amplitudes and phase values at the refined lattice positions (done in Fourier space)
- Determining the origin of the projected unit cell and determining the projected (plane group) symmetry
- Imposing constrains of the most likely plane group symmetry on the amplitudes an phases. At this step the image phases are converted into the phases of the structure factors.
- Calculating the pseudo-potential map by Fourier synthesiswith corrected (structure factor) amplitudes and phases (done in real space)
- Determining 2D (projected) atomic co-ordinates (done in real space)
A few computer programs are available which assist to perform the necessary steps of processing. The most popular programs used by materials scientists (electron crystallographers) are CRISP,[32][33][34] VEC,[35][36] and the EDM package.[37] There is also the recently developed crystallographic image processing program EMIA,[38] but so far there do not seem to be reports by users of this program.
Structural biologists achieve resolutions of a few ångströms (up from a to few nanometers in the past when samples used to be negatively stained) for membrane forming proteins in regular two-dimensional arrays, but prefer the usage of the programs 2dx,
Inorganic crystals are much stiffer than 2D periodic protein membrane arrays so that there is no need for the unbending of images that were taken from suitably thinned parts of these crystals. Consequently, the CRISP program does not possess the unbending image processing feature but offers superior performance in the so-called phase origin refinement.
The latter feature is particularly important for electron crystallographers as their samples may possess any space group out of the 230 possible groups types that exist in three dimensions. The regular arrays of membrane forming proteins that structural biologists deal with are, on the other hand, restricted to possess one out of only 17 (two-sided/black-white) layer group types (of which there are 46 in total and which are periodic only in 2D) due to the chiral nature of all (naturally occurring) proteins. Different crystallographic settings of four of these layer group types increase the number of possible layer group symmetries of regular arrays of membrane forming proteins to just 21.
All 3D space groups and their subperiodic 2D periodic layer groups (including the above-mentioned 46 two-sided groups) project to just 17 plane space group types, which are genuinely 2D periodic and are sometimes referred to as the wallpaper groups. (Although quite popular, this is a misnomer because wallpapers are not restricted to possess these symmetries by nature.)
All individual transmission electron microscopy images are projections from the three-dimensional space of the samples into two dimensions (so that spatial distribution information along the projection direction is unavoidably lost). Projections along prominent (i.e. certain low-index) zone axes of 3D crystals or along the layer normal of a membrane forming protein sample ensure the projection of 3D symmetry into 2D. (Along arbitrary high-index zone axes and inclined to the layer normal of membrane forming proteins, there will be no useful projected symmetry in transmission images.) The recovery of 3D structures and their symmetries relies on electron tomography techniques, which use sets of transmission electron microscopy images.
The origin refinement part of CIP relies on the definition of the plane symmetry group types as provided by the International Tables of Crystallography, where all symmetry equivalent positions in the unit cell and their respective site symmetries are listed along with systematic absences in reciprocal space. Besides plane symmetry groups p1, p3, p3m1 and p31m, all other plane group symmetries are centrosymmetric so that the origin refinement simplifies to the determination of the correct signs of the amplitudes of the Fourier coefficients.
When crystallographic image processing is utilized in scanning probe microscopy, the symmetry groups to be considered are just the 17 plane space group types in their possible 21 settings.
Crystallographic processing of images that were recorded from 2D periodic arrays with other types of microscopes
Because digitized 2D periodic images are in the information theoretical approach just data organized in 2D arrays of pixels, core features of Crystallographic Image Processing can be utilized independent of the type of microscope with which the images/data were recorded. The CIP technique has, accordingly been applied (on the basis of the 2dx program) to atomic resolution Z-contrast images of Si-clathrates, as recorded in an aberration-corrected scanning transmission electron microscope.[48] Images of 2D periodic arrays of flat lying molecules on a substrate as recorded with scanning tunneling microscopes [49][50] were also crystallographic processed utilizing the program CRISP.
References
- ^ T. E. Weirich, From Fourier series towards crystal structures - a survey of conventional methods for solving the phase problem; in: Electron Crystallography - Novel Approaches for Structure Determination of Nanosized Materials, T. E. Weirich, J. L. Lábár, X. Zou, (Eds.), Springer 2006, 235 - 257.
- .
- PMID 15219694.
- .
- ^ Klug, A. (1979). "Image Analysis and Reconstruction in the Electron Microscopy of Biological Macromolecules". Chemica Scripta. 14: 245–256.
- PMID 6289376.
- doi:10.1038/311238a0.
- PMID 22666921.
- S2CID 4327149.
- S2CID 4340756.
- PMID 1445681.
- PMID 14581752.
- S2CID 4396820.
- S2CID 98380733.
- ^ Y. Liu, "Computational Symmetry" in: Symmetry 2000, Part I, eds. I. Hargittai and T. C. Laurent, chapter 21, p. 231–245, Portland Press, London, 2002, (Wenner-Gren International Series, vol. 80), https://www.cs.cmu.edu/~yanxi/images/computationalSymmetry.pdf
- ^ Y. Liu, H. Hel-Or, C. S. Kaplan, and L. V. Gool (2009) "Foundations and Trends in Computer Graphics and Vision" vol. 5, Nos. 1–2, p. 1–195, open access: http://vision.cse.psu.edu/publications/pdfs/liuCSinCV.pdf, also published as a book by Now Publishers Inc., Boston and Delft, 2010
- S2CID 13894016.
- ^ Y. Liu (2014) "Computational Symmetry", Computer Vision, A Reference Guide, Ikeuchi, K. (ed.), Springer, 2014
- S2CID 207060370.
- ^ Hahn T. (2005) International Tables for Crystallography, Brief Teaching Edition of Volume A, Space-group symmetry. 5th revised edition, Chester: International Union of Crystallography
- .
- ^ K. Kanatani, (1996) Statistical Optimization for Geometric Computation: Theory and Practice, Dover Books on Mathematics, Mineola, New York
- ^ K. P. Burnham and D R Anderson (2002) Model selection and multimodel inference: a practical information-theoretic approach, 2nd edition. Springer, New York
- S2CID 3354490.
- S2CID 19248991.
- S2CID 2394019.
- S2CID 94543563.
- S2CID 51783244.
- .
- ^ C. Dieckmann (2012) Symmetry Detection and Approximation, Dissertation zur Erlangung des Doktorgrades, Fachbereich Mathematik und Informatik der Freien Universität Berlin
- ISBN 978-3-540-67681-2
- .
- .
- ISBN 9780444521781.
- S2CID 96860861.
- S2CID 250873737.
- PMID 15639355.
- ^ "Home". analitex.com.
- PMID 17055742.
- ^ "EMAN2 - EMAN Wiki".
- PMID 14643205.
- PMID 8742717.
- ^ "Image Processing Software".
- ^ "Home". ccpem.ac.uk.
- PMID 16997575.
- .
- ISBN 9780470016176.
- PMID 19297343.
- ^ Moeck, P., Toader, M., Abdel-Hafiez, M., Michael Hietschold, M. (2009) "Quantifying and enforcing two-dimensional symmetries in scanning probe microscopy images", in: "Frontiers of Characterization and Metrology for Nanoelectronics", edited by D. G. Seiler, A. C. Diebold, R. McDonald, C. M. Garner, D. Herr, R. P. Khosla, and E. M. Secula, American Institute of Physics, 978-0-7354-0712-1/09.
- arXiv:2011.13102
External links
- MRC Image Processing programs, the classical standard for structural biology, free for academic usages (Fortran source code)
- CRISP commercial, but superior for inorganic electron crystallography (for Windows PCs)
- VEC free for academic usages, particularly useful for the analysis of incommensurately modulated structures (for Windows PCs)
- IPLT open source, structural biology (for Mac PCs, Linux, and a Windows PC demo version)
- EMAN Vers. 2, open source, structural biology including single particle reconstruction (Linux)
- 2dx open source, mainly for structural biology (for Mac PCs, Linux)
- EDM open source, free for non-commercial purposes - a ready to go version of EDM is implemented on the elmiX Linux live CD
Further reading
see also the Wiki on Electron Crystallography