Abstract
Ecological studies are increasingly using video image data to study the distribution and behaviour of organisms. Particularly in marine sciences cameras are utilised to access underwater environments. Up till now image data has been processed by human observers which is costly and often represents repetitive mundane work. Deep learning techniques that can automatically classify objects can increase the speed and the amounts of data that can be processed. This ultimately will make image processing in ecological studies more cost effective, allowing studies to invest in larger, more robust sampling designs. As such, deep learning will be a game changer for ecological research helping to improve the quality and quantity of the data that can be collected. Within this chapter we introduce two case studies to demonstrate the application of deep learning techniques in marine ecological studies. The first example demonstrates the use of deep learning in the detection and classification of an important underwater ecosystem in the Mediterranean (Posidonia oceanica seagrass meadows), the other showcases the automatic identification of several jellyfish species in coastal areas. Both applications showed high levels of accuracy in the detection and identification of the study organisms, which represents encouraging results for the applicability of these methodologies in marine ecological studies. Despite its potential, deep learning has yet not been widely adopted in ecological studies. Information technologists and natural scientists alike need to more actively collaborate to move forward in this field of science. Cost-effective data collection solutions are desperately needed in a time when large amounts of data are required to detect and adapt to global environmental change.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Caughlan, L.: Cost considerations for long-term ecological monitoring. Ecol. Indic. 1(2), 123–134 (2001)
Barrio Froján, C.R.S., Cooper, K.M., Bolam, S.G.: Towards an integrated approach to marine benthic monitoring. Mar. Pollut. Bull. 104(1–2), 20–28 (2016)
Del Vecchio, S., Fantinato, E., Silan, G., Buffa, G.: Trade-offs between sampling effort and data quality in habitat monitoring. Biodivers. Conserv. 28(1), 55–73 (2018)
Bennett, M., Acott, C., Richardson, K., Bowen, S., Smart, D., Smith, P., Sharp, F., Bryson, P., Goble, S.: Recreational technical diving part 1: an introduction to technical diving methods and activities. J. S. Pac. Underw. Med. Soc. Eur. Underw. Baromedical Soc. 43(4), 86–93 (2013)
Hissmann, K., Schauer, J.: Manned submersible JAGO. J. Large-Scale Res. Facil. JLSRF 3, A110 (2017)
Sagalevich, A.M.: 30 years experience of Mir submersibles for the ocean operations. Deep Sea Res. Part II Top. Stud. Ocean. 155(2017), 83–95 (2017)
Rosenkranz, G.E., Byersdorfer, S.C.: Video scallop survey in the eastern Gulf of Alaska, USA. Fish. Res. 69(1), 131–140 (2004)
Lambert, G.I., Jennings, S., Hinz, H., Murray, L.G., Parrott, L., Kaiser, M.J., Hiddink, J.G.: A comparison of two techniques for the rapid assessment of marine habitat complexity. Methods Ecol. Evol. 4(3), 226–235 (2013)
Santana-Garcon, J., Newman, S.J., Harvey, E.S.: Development and validation of a mid-water baited stereo-video technique for investigating pelagic fish assemblages. J. Exp. Mar. Biol. Ecol. 452, 82–90 (2014)
Gallo, N.D., Cameron, J., Hardy, K., Fryer, P., Bartlett Douglas, H., Levin Lisa, A.: Submersible- and lander-observed community patterns in the Mariana and New Britain trenches: influence of productivity and depth on epibenthic and scavenging communities. Deep Sea Res. Part I Ocean. Res. Pap. 99, 119–133 (2015)
Sheehan, E., Vaz, S., Pettifer, E., Foster, N., Nancollas, S., Cousens, S., Holmes, L., Facq, J.-V., Germain, G., Attrill, M.: An experimental comparison of three towed underwater video systems using species metrics, benthic impact and performance. Methods Ecol. Evol. 7(07) (2016)
Langlois, T.J., Harvey, E.S., Fitzpatrick, B., Meeuwig, J.J., Shedrawi, G., Watson, D.L.: Cost-efficient sampling of fish assemblages: comparison of baited video stations and diver video transects. Aquat. Biol. 9(2), 155–168 (2010)
Holmes, T.H., Wilson, S.K., Travers, M.J., Langlois, T.J., Evans, R.D., Moore, G.I., Douglas, R.A., Shedrawi, G., Harvey, E.S., Hickey, K.: A comparison of visual- and stereo-video based fish community assessment methods in tropical and temperate marine waters of Western Australia. Limnol. Ocean. Methods 11(7), 337–350 (2013)
Martin-Abadal, M., Guerrero-Font, E., Bonin-Font, F., Gonzalez-Cid, Y.: Deep semantic segmentation in an AUV for online posidonia oceanica meadows identification. IEEE Access 6, 60956–60967 (2018)
Gray, P.C., Fleishman, A.B., Klein, D.J., McKown, M.W., Bézy, V.S., Lohmann, K.J., Johnston, D.W.: A convolutional neural network for detecting sea turtles in drone imagery. Methods Ecol. Evol., 1–11 (2018). In Review
Kotta, J., Valdivia, N., Kutser, T., Toming, K., Rätsep, M., Orav-Kotta, H.: Predicting the cover and richness of intertidal macroalgae in remote areas: a case study in the Antarctic Peninsula. Ecol. Evol. 8(17), 9086–9094 (2018)
Wäldchen, J., Mäder, P.: Machine learning for image based species identification. Methods Ecol. Evol. 9(11), 2216–2225 (2018)
Weinstein, B.G.: Scene-specific convolutional neural networks for video-based biodiversity detection. Methods Ecol. Evol. 9(6), 1435–1441 (2018)
Willi, M., Pitman, R.T., Cardoso, A.W., Locke, C., Swanson, A., Boyer, A., Veldthuis, M., Fortson, L.: Identifying animal species in camera trap images using deep learning and citizen science. Methods Ecol. Evol., 1–12 (2018)
Mieszkowska, N., Sugden, H., Firth, L.B., Hawkins, S.J.: The role of sustained observations in tracking impacts of environmental change on marine biodiversity and ecosystems. Philos. Trans. R. Soc. A Math. Phys. Eng. Sci. 372(2025) (2014)
Gouraguine, A., Moranta, J., Ruiz-Frau, A., Hinz, H, Reñones, O., Ferse, S.C.A., Jompa, J., Smith, D.J.: Citizen science in data and resource-limited areas: a tool to detect long-term ecosystem changes. Plos One 14(1), e0210007 (2019)
Cun, L., Cun, L., Boser, B., Denker, J.S., Henderson, D., Howard, R.E., Hubbard, W., Jackel, L.D.: Handwritten digit recognition with a back-propagation network. Adv. Neural Inf. Process. Syst. 2, 396–404 (1990)
Russakovsky, O., Deng, J., Hao, S., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A.C., Fei-Fei, L.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115(3), 211–252 (2015)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. ArXiv:1409.1556, Sept 2014
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. ArXiv:1512.03385, Dec 2015
Huang, G., Liu, Z., van der Maaten, L., Weinberger, K.Q.: Densely Connected Convolutional Networks, Aug 2016
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. ArXiv:1409.4842, Sept 2014
Chollet, F.: Xception: deep learning with depthwise separable convolutions. ArXiv:1610.02357, Oct 2016
Geisser, S.: The predictive sample reuse method with applications. J. Am. Stat. Assoc. 70(350), 320–328 (1975)
Diaz-Almela, E., Duarte, C.: Management of Natura 2000 Habitats 1120, (Posidonia Oceanicae). Technical report, European Commission (2008)
Ruiz-Frau, A., Gelcich, S., Hendriks, I.E., Duarte, C.M., Marbà, N.: Current state of seagrass ecosystem services: research and policy integration. Ocean. Coast. Manag. 149, 107–115 (2017)
Marba, N., Duarte, C.: Mediterranean warming triggers seagrass (posidonia oceanica) shoot mortality. Glob. Chang. Biol. 16(8), 2366–2375 (2010)
Telesca, L., Belluscio, A., Criscoli, A., Ardizzone, G., Apostolaki, E.T., Fraschetti, S., Gristina, M., Knittweis, L., Martin, C.S., Pergent, G., Alagna, A., Badalamenti, F., Garofalo, G., Gerakaris, V., Pace, M.L., Pergent-Martini, C., Salomidi, M.: Seagrass meadows (posidonia oceanica) distribution and trajectories of change. Scientific reports (2015)
y Royo, C.L., Pergent, G., Pergent-Martini, C., Casazza, G.: Seagrass (posidonia oceanica) monitoring in western mediterranean: implications for management and conservation. Environ. Monit. Assess. 171, 365–380 (2010)
Sagawa, T., Komatsu, T.: Simulation of seagrass bed mapping by satellite images based on the radiative transfer model. Ocean. Sci. J. 50(2), 335–342 (2015)
Montefalcone, M., Rovere, A., Parravicini, V., Albertelli, G., Morri, C., Bianchi, C.N.: Evaluating change in seagrass meadows: a time-framed comparison of side scan sonar maps. Aquat. Bot. 104, 204–212 (2013)
Vasilijevic, A., Miskovic, N., Vukic, Z., Mandic, F.: Monitoring of seagrass by lightweight AUV: a posidonia oceanica case study surrounding Murter island of Croatia. In: Mediterranean Conference on Control and Automation, pp. 758–763, June 2014
Rende, F.S., Irving, A.D., Lagudi, A., Bruno, F., Scalise, S., Cappa, P., Montefalcone, M., Bacci, T., Penna, M., Trabucco, B., Di Mento, R., Cicero, A.M.: Pilot application of 3D underwater imaging techniques for mapping Posidonia oceanica (L.) delile meadows. ISPRS - Int. Arch. Photogramm. Remote. Sens. Spat. Inf. Sci., 177–181 (2015)
Bonin-Font, F., Burguera, A., Lisani, J.-L.: Visual discrimination and large area mapping of posidonia oceanica using a lightweight AUV. IEEE Access 5, 24479–24494 (2017)
Gonzalez-Cid, Y., Burguera, A., Bonin-Font, F., Matamoros, A.: Machine learning and deep learning strategies to identify posidonia meadows in underwater images. IEEE Oceans, 1–5 (2017)
Dumoulin, V., Visin, F.: A guide to convolution arithmetic for deep learning. arXiv:1603.07285, Mar 2016
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3431–3440, June 2015
Teichmann, M., Weber, M., Zoellner, M., Cipolla, R., Urtasun, R.: MultiNet: real-time joint semantic reasoning for autonomous driving. arXiv:1612.07695, Dec 2016
Buja, A., Stuetzle, W., Shen, Y.: Loss functions for binary class probability estimation and classification: structure and applications. Technical report, Department of Statistics of University of Pennsylvania, Jan 2005
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv:1412.6980, Dec 2014
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15, 1929–1958 (2014)
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Li, F.-F.: Imagenet: a large-scale hierarchical image database. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255, June 2009
Taylor, L., Nitschke, G.: Improving deep learning using generic data augmentation. arXiv:1708.06020, Aug 2017
Bengio, Y.: Practical recommendations for gradient-based training of deep architectures. In: Neural Networks: Tricks of the Trade (2012)
Provost, F., Fawcett, T., Kohavi, R.: The case against accuracy estimation for comparing induction algorithms. In: Proceedings of the Fifteenth International Conference on Machine Learning, Apr 2001
Powers, D.M.W.: Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation. Int. J. Mach. Learn. Technol. 2, 37–63 (2011)
European Commission: Assessment of jellyfish socioeconomic impacts in the mediterranean: implications for management, horizon 2020, May 2017
Condon, R.H., Graham, W.M., Duarte, C.M., Pitt, K.A., Lucas, C.H., Haddock, S.H.D., Sutherland, K.R., Robinson, K.L., Dawson, M.N., Decker, M.B., Mills, C.E., Purcell, J.E., Malej, A., Mianzan, H., Uye, S.-I., Gelcich, S., Madin, L.P.: Questioning the rise of gelatinous zooplankton in the world’s oceans. Bioscience 62(2), 160–169 (2012)
Richardson, A.J., Bakun, A., Hays, G.C., Gibbons, M.J.: The jellyfish joyride: causes, consequences and management responses to a more gelatinous future. Trends Ecol. Evol. 24(6), 312–322 (2009)
Condon, R.H., Duarte, C.M., Pitt, K.A., Robinson, K.L., Lucas, C.H., Sutherland, K.R., Mianzan, H.W., Bogeberg, M., Purcell, J.E., Decker, M.B., Uye, S.-I., Madin, L.P., Brodeur, R.D., Haddock, S.H.D., Malej, A., Parry, G.D., Eriksen, E., Quinones, J., Acha, M., Harvey, M., Arthur, J.M., Graham, W.M.: Recurrent jellyfish blooms are a consequence of global oscillations. Proc. Natl. Acad. Sci. USA 110(3), 1000–1005 (2013)
United Nations: Factsheet: people and oceans, pp. 1–2. https://www.un.org/sustainabledevelopment/wp-content/uploads/2017/05/Ocean-fact-sheet-package.pdf (2017)
Macrokanis, C., Hall, N., Mein, J.: Irukandji syndrome in Northern Western Australia: an emerging health problem. Med. J. Aust. 181, 699–702 (2004)
Fenner, P.J., Lippmann, J., Gershwin, L.A.: Fatal and nonfatal severe jellyfish stings in Thai waters. J. Travel. Med. 17(2), 133–138 (2010)
Purcell, J.E., Ichi Uye, S.-I., Tseng Lo, W.-T.: Anthropogenic causes of jellyfish blooms and their direct consequences for humans: a review. Mar. Ecol. Prog. Ser. 350, 153–174 (2007)
Purcell, J.E., Baxter, E.J., Fuentes, V.L.: Jellyfish as products and problems of aquaculture. In: Advances in Aquaculture Hatchery Technology, pp. 404–430. Elsevier (2013)
Merceron, M., Le Fevre-Lehoerff, G., Bizouarn, Y., Kempf, M.: Fish and jellyfish in Brittany (France). Equinoxe 56, 6–8 (1995)
Lee, J.H., Choi, H.W., Chae, J., Kim, D.S., Lee, S.B.: Performance analysis of intake screens in power plants on mass impingement of marine organisms. Ocean. Polar Res. 28, 385–393 (2006)
Matsumura, K., Kamiya, K., Yamashita, K., Hayashi, F., Watanabe, I., Murao, Y., Miyasaka, H., Kamimura, N., Nogami, M.: Genetic polymorphism of the adult medusae invading an electric power station and wild polyps of Aurelia aurita in Wakasa Bay, Japan. J. Mar. Biol. Assoc. UK 85(3), 563–568 (2005)
Ferraris, M., Berline, L., Lombard, F., Guidi, L., Elineau, A., Mendoza-Vera, J.M., Lilley, M.K.S., Taillandier, V., Gorsky, G.: Distribution of Pelagia noctiluca (Cnidaria, Scyphozoa) in the Ligurian Sea (NW Mediterranean Sea). J. Plankton Res. 34(10), 874–885 (2012)
Barrado, C., Fuentes, J.A., Salamí, E., Royo, P., Olariaga, A.D., López, J., Fuentes, V.L., Gili, J.M., Pastor, E.: Jellyfish monitoring on coastlines using remote piloted aircraft. In: IOP Conference Series: Earth and Environmental Science, vol. 17, p. 12195 (2014)
Kim, Donghoon, Shin, J.U., Kim, H., Kim, H., Lee, D., Lee, S.M., Myung, H.: Development and experimental testing of an autonomous jellyfish detection and removal robot system. Int. J. Control Autom. Syst. 14(1), 312–322 (2016)
Matsuura, F., Fujisawa, N., Ishikawa: Detection and removal of jellyfish using underwater image analysis. J. Vis. 10(3), 259–260 (2007)
Szegedy, C., Ioffe, S., Vanhoucke, V.: Inception-v4, inception-resnet and the impact of residual connections on learning. In: AAAI Conference on Artificial Intelligence, Feb 2016
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–9, June 2015
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016)
Pascanu, R., Mikolov, T., Bengio, Y.: On the difficulty of training recurrent neural networks. In: Proceedings of the 30th International Conference on International Conference on Machine Learning, vol. 28, pp. III–1310–III–1318 (2013)
Lin, T.-Y., Maire, M., Belongie, S.J., Bourdev, L.D., Girshick, R.B., Hays, J., Perona, P., Ramanan, D., Dollár, P., Lawrence Zitnick, C.: Microsoft coco: common objects in context. In: ECCV (2014)
Tzutalin, D.: Labelimg. https://github.com/tzutalin/labelImg (2018)
Zhu, M.: Recall, precision and average precision. Technical report, Department of Statistics and Actuarial Science, University of Waterloo, Waterloo, Canada, Sept 2004
Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The PASCAL visual object classes (VOC) challenge. Int. J. Comput. Vis. 88(2), 303–338 (2010)
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: CVPR09 (2009)
Martin-Abadal, M.: Video: online posidonia oceanica segmentation. http://srv.uib.es/po-identification/, May 2018
Martin-Abadal, M.: Video: jellyfish object detection. http://srv.uib.es/jellyfish-object-detection/, Dec 2018
Acknowledgements
This work is partially supported by Ministry of Economy and Competitiveness (AEI, FEDER, UE), under contracts TIN2017-85572-P, DPI2017-86372-C3-1-R. H Hinz was supported by the Ramón y Cajal Fellowship (grant by the Ministerio de Economía y Competitividad de España and the Conselleria d’Educació, Cultura i Universitats Comunidad Autonoma de las Islas Baleares). We would like to thank Charlotte Jennings for her help in identifying and labelling jellyfish in underwater images.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Martin-Abadal, M., Ruiz-Frau, A., Hinz, H., Gonzalez-Cid, Y. (2020). The Application of Deep Learning in Marine Sciences. In: Pedrycz, W., Chen, SM. (eds) Deep Learning: Algorithms and Applications. Studies in Computational Intelligence, vol 865. Springer, Cham. https://doi.org/10.1007/978-3-030-31760-7_7
Download citation
DOI: https://doi.org/10.1007/978-3-030-31760-7_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-31759-1
Online ISBN: 978-3-030-31760-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)