Abstract
Light-sheet microscopes enable rapid high-resolution imaging of biological specimens; however, biological processes span spatiotemporal scales. Moreover, long-term phenotypes are often instigated by rare or fleeting biological events that are difficult to capture with a single imaging modality. Here, to overcome this limitation, we present smartLLSM, a microscope that incorporates artificial intelligence-based instrument control to autonomously switch between epifluorescent inverted imaging and lattice light-sheet microscopy (LLSM). We apply this approach to two unique processes: cell division and immune synapse formation. In each context, smartLLSM provides population-level statistics across thousands of cells and autonomously captures multicolor three-dimensional datasets or four-dimensional time-lapse movies of rare events at rates that dramatically exceed human capabilities. From this, we quantify the effects of Taxol dose on spindle structure and kinetochore dynamics in dividing cells and of antigen strength on cytotoxic T lymphocyte engagement and lytic granule polarization at the immune synapse. Overall, smartLLSM efficiently detects rare events within heterogeneous cell populations and records these processes with high spatiotemporal four-dimensional imaging over statistically significant replicates.
Similar content being viewed by others
Data availability
Due to the inordinate size of the image data (∼40 TB), it is not currently feasible to deposit this into a central repository; however, all datasets underlying the results in this paper are available from the corresponding author upon request. To the extent possible, the authors will try to meet all requests for data sharing within 2 weeks from the original request. Source data are provided with this paper.
Code availability
The source code, annotation GUI, the library of annotated training data and the trained YOLOv5 network generated in the current study are available at https://github.com/legantlab/smartLLSM. Code is provided under The MIT License for open source software, a permissive license approved by the Open Source Initiative. Specific terms can be found at https://opensource.org/licenses/MIT.
References
Power, R. M. & Huisken, J. A guide to light-sheet fluorescence microscopy for multiscale imaging. Nat. Methods 14, 360–373 (2017).
Hobson, C. M. et al. Practical considerations for quantitative light sheet fluorescence microscopy. Nat. Methods 19, 1538–1549 (2022).
Liu, Z. et al. 3D imaging of Sox2 enhancer clusters in embryonic stem cells. eLife 3, e04236 (2014).
Gustavsson, A.-K., Petrov, P. N., Lee, M. Y., Shechtman, Y. & Moerner, W. E. 3D single-molecule super-resolution microscopy with a tilted light sheet. Nat. Commun. 9, 123 (2018).
Legant, W. R. et al. High-density three-dimensional localization microscopy across large volumes. Nat. Methods 13, 359–365 (2016).
Chen, B.-C. et al. Lattice light-sheet microscopy: imaging molecules to embryos at high spatiotemporal resolution. Science 346, 1257998 (2014).
Chen, J. et al. Single-molecule dynamics of enhanceosome assembly in embryonic stem cells. Cell 156, 1274–1285 (2014).
Keller, P. J., Schmidt, A. D., Wittbrodt, J. & Stelzer, E. H. K. Reconstruction of zebrafish early embryonic development by scanned light sheet microscopy. Science 322, 1065–1069 (2008).
McDole, K. et al. In toto imaging and reconstruction of post-implantation mouse development at the single-cell level. Cell 175, 859–876.e33 (2018).
Huisken, J., Swoger, J., Del Bene, F., Wittbrodt, J. & Stelzer, E. H. K. Optical sectioning deep inside live embryos by selective plane illumination microscopy. Science 305, 1007–1009 (2004).
Liu, T.-L. et al. Observing the cell in its native state: Imaging subcellular dynamics in multicellular organisms. Science 360, eaaq1392 (2018).
Reynaud, E. G., Peychl, J., Huisken, J. & Tomancak, P. Guide to light-sheet microscopy for adventurous biologists. Nat. Methods 12, 30–34 (2015).
Laissue, P. P., Alghamdi, R. A., Tomancak, P., Reynaud, E. G. & Shroff, H. Assessing phototoxicity in live fluorescence imaging. Nat. Methods 14, 657–661 (2017).
Sapoznik, E. et al. A versatile oblique plane microscope for large-scale and high-resolution imaging of subcellular dynamics. eLife 9, e57681 (2020).
Kumar, A. et al. Dual-view plane illumination microscopy for rapid and spatially isotropic imaging. Nat. Protoc. 9, 2555–2573 (2014).
Dean, K. M., Roudot, P., Welf, E. S., Danuser, G. & Fiolka, R. Deconvolution-free subcellular imaging with axially swept light sheet microscopy. Biophys. J. 108, 2807–2815 (2015).
Conrad, C. et al. Micropilot: automation of fluorescence microscopy-based imaging for systems biology. Nat. Methods 8, 246–249 (2011).
André, O., Ahnlide, J. K., Norlin, N., Swaminathan, V. & Nordenfelt, P. Data-driven microscopy allows for automated context-specific acquisition of high-fidelity image data. Cell Rep. Methods. 3, 100419 (2023).
Barentine, A. E. S. et al. An integrated platform for high-throughput nanoscopy. Nat. Biotechnol. https://doi.org/10.1038/s41587-023-01702-1 (2023).
Mahecic, D. et al. Event-driven acquisition for content-enriched microscopy. Nat. Methods. https://doi.org/10.1038/s41592-022-01589-x (2022).
Alvelid, J., Damenti, M., Sgattoni, C. & Testa, I. Event-triggered STED imaging. Nat. Methods. https://doi.org/10.1038/s41592-022-01588-y (2022).
Bochkovskiy, A., Wang, C.-Y. & Liao, H.-Y. M. YOLOv4: Optimal speed and accuracy of object detection. Preprint at https://doi.org/10.48550/arXiv.2004.10934 (2020).
Redmon, J., Divvala, S., Girshick, R. & Farhadi, A. You only look once: unified, real-time object detection. in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 779–788 https://doi.org/10.1109/CVPR.2016.91 (IEEE, 2016).
Jocher, G. YOLOv5 SOTA realtime instance segmentation. Ultralytics https://doi.org/10.5281/zenodo.3908559 (2020).
Yamashita, N. et al. Three-dimensional tracking of plus-tips by lattice light-sheet microscopy permits the quantification of microtubule growth trajectories within the mitotic apparatus. J. Biomed. Opt. 20, 101206 (2015).
Pamula, M. C. et al. High-resolution imaging reveals how the spindle midzone impacts chromosome movement. J. Cell Biol. 218, 2529–2544 (2019).
David, A. F. et al. Augmin accumulation on long-lived microtubules drives amplification and kinetochore-directed growth. J. Cell Biol. 218, 2150–2168 (2019).
Ritter, A. T. et al. Actin depletion initiates events leading to granule secretion at the immunological synapse. Immunity 42, 864–876 (2015).
Ikui, A. E., Yang, C.-P. H., Matsumoto, T. & Horwitz, S. B. Low concentrations of taxol cause mitotic delay followed by premature dissociation of p55CDC from Mad2 and BubR1 and abrogation of the spindle checkpoint, leading to aneuploidy. Cell Cycle 4, 1385–1388 (2005).
Brito, D. A. & Rieder, C. L. The ability to survive mitosis in the presence of microtubule poisons differs significantly between human nontransformed (RPE-1) and cancer (U2OS, HeLa) cells. Cell Motil. Cytoskelet. 66, 437–447 (2009).
Denton, A. E. et al. Affinity thresholds for naive CD8+ CTL activation by peptides and engineered influenza A viruses. J. Immunol. 187, 5733–5744 (2011).
Frazer, G. L., Gawden-Bone, C. M., Dieckmann, N. M. G., Asano, Y. & Griffiths, G. M. Signal strength controls the rate of polarization within CTLs during killing. J. Cell Biol. 220, e202104093 (2021).
Yachi, P. P., Ampudia, J., Zal, T. & Gascoigne, N. R. J. Altered peptide ligands induce delayed CD8–T cell receptor interaction—a role for CD8 in distinguishing antigen quality. Immunity 25, 203–211 (2006).
Jenkins, M. R., Tsun, A., Stinchcombe, J. C. & Griffiths, G. M. The strength of T cell receptor signal controls the polarization of cytotoxic machinery to the immunological synapse. Immunity 31, 621–631 (2009).
Magidson, V. et al. The spatial arrangement of chromosomes during prometaphase facilitates spindle assembly. Cell 146, 555–567 (2011).
Chaly, N. & Brown, D. L. The prometaphase configuration and chromosome order in early mitosis. J. Cell Sci. 91, 325–335 (1988).
Sen, O., Harrison, J. U., Burroughs, N. J. & McAinsh, A. D. Kinetochore life histories reveal an Aurora-B-dependent error correction mechanism in anaphase. Dev. Cell https://doi.org/10.1016/j.devcel.2021.10.007 (2021).
Klaasen, S. J. et al. Nuclear chromosome locations dictate segregation error frequencies. Nature https://doi.org/10.1038/s41586-022-04938-0 (2022).
Royer, L. A. et al. Adaptive light-sheet microscopy for long-term, high-resolution imaging in living organisms. Nat. Biotechnol. 34, 1267–1278 (2016).
Eisenstein, M. Smart solutions for automated imaging. Nat. Methods 17, 1075–1079 (2020).
Moore, R. P. et al. A multi-functional microfluidic device compatible with widefield and light sheet microscopy. Lab Chip 22, 136–147 (2021).
Ding, S. et al. Efficient transposition of the piggyBac (PB) transposon in mammalian cells and mice. Cell 122, 473–483 (2005).
Yeo, N. C. et al. An enhanced CRISPR repressor for targeted mammalian gene regulation. Nat. Methods 15, 611–616 (2018).
Kirk, J. M. et al. Functional classification of long non-coding RNAs by k-mer content. Nat. Genet. 50, 1474–1482 (2018).
Shao, L., Milkie, D. & Lambert, T. cudadecon. Zenodo https://doi.org/10.5281/zenodo.7659013 (2023).
Jaqaman, K. et al. Robust single-particle tracking in live-cell time-lapse sequences. Nat. Methods 5, 695–702 (2008).
Acknowledgements
We thank K. Heath, M. Clynes and V. Augoustides for assistance with annotating images. We thank G. Upadhyayula for assistance with the single particle tracking code and T. Kapoor, M. Emanuele and A. Palmer for helpful discussions and feedback on the manuscript. This work was funded in part by grants from the National Institutes of Health (1DP2GM136653) awarded to W.R.L. W.R.L. acknowledges additional support from the Searle Scholars program, the Beckman Young Investigator Program and the Packard Fellowship for Science and Engineering.
Author information
Authors and Affiliations
Contributions
W.R.L. conceived the project. A.G. and J.S.T. lead the development of the DL-based cell detection and classification pipeline with input and contribution from Y.S. and W.R.L. D.E.M assisted with integrating the YOLO network together with the microscope control software. T.A.D, C.Q.Y. and A.T.R assisted with sample preparation. Y.S. and W.R.L performed the imaging experiments, analyzed data and wrote the paper with feedback from all authors. W.R.L. supervised and directed the project.
Corresponding authors
Ethics declarations
Competing interests
W.R.L. and D.E.M. are authors on patents related to lattice light-sheet microscopy and its applications including US Patent numbers: US 11,221,476 B2 and US 10,795,144 B2 issued to W.R.L., D.E.M. and co-authors, and assigned to Howard Hughes Medical Institute. The other authors declare no competing interests.
Peer review
Peer review information
Nature Methods thanks Mathieu Ducros, Ilaria Testa and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Primary Handling Editor: Rita Strack, in collaboration with the Nature Methods team. Peer reviewer reports are available.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Supplementary Information
List of Supplementary Movies and Captions 1–6, Tables 1–2, Figs. and Captions 1–21, Notes 1–3 and references.
Supplementary Data 1
Source data for supplementary figures.
Supplementary Video 1
Supplementary Movie 1.
Supplementary Video 2
Supplementary Movie 2.
Supplementary Video 3
Supplementary Movie 3.
Supplementary Video 4
Supplementary Movie 4.
Supplementary Video 5
Supplementary Movie 5.
Supplementary Video 6
Supplementary Movie 6.
Source data
Source Data
Source data for all the main figures.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Shi, Y., Tabet, J.S., Milkie, D.E. et al. Smart lattice light-sheet microscopy for imaging rare and complex cellular events. Nat Methods 21, 301–310 (2024). https://doi.org/10.1038/s41592-023-02126-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/s41592-023-02126-0
- Springer Nature America, Inc.
This article is cited by
-
Deep 3D histology powered by tissue clearing, omics and AI
Nature Methods (2024)
-
3D live imaging and phenotyping of CAR-T cell mediated-cytotoxicity using high-throughput Bessel oblique plane microscopy
Nature Communications (2024)
-
Navigate: an open-source platform for smart light-sheet microscopy
Nature Methods (2024)
-
Where imaging and metrics meet
Nature Methods (2024)