The “Mathemusical” Dynamics Between Music and Mathematics (via Computer Science)

In western tradition, mathematics and music have been connected for more than 2000 years. Despite this long history of relations between the two fields, the interest of “working mathematicians” in this research domain is a rather new phenomenon [9]. Whilst the power of applying mathematics to music has been acknowledged for a long time, it is only thanks to more recent developments that also music begins to occupy an important place in the development of mathematics. In fact, music has shown to provide a number of difficult theoretical problems, in particular for what concerns their constructive formalization and algorithmic solution. This asks for a permanent feedback between musical thought, mathematical formalisation and computational modelling, a dynamic movement between different disciplines that I suggested to call “mathemusical” [3]. This feedback between music and mathematics via computer science is illustrated in the diagram of Fig. 1.

Fig. 1
figure 1

A diagram showing the “mathemusical” dynamics between music and mathematics via computer-science

Full size image

Among the music-theoretical problems showing a remarkable link with interesting mathematical constructions and open conjectures, one may quote the following ones that have provided the content of a considerable number of Master and PhD dissertations:Footnote 1

  • Tiling rhythmic canons and their spectral dimensions (via the Fuglede Conjecture)

  • Z-relation in music theory and the study of homometric structures in crystallography

  • Transformational music theory and the categorical classification of direct musical graphs

  • Neo-Riemannian music analysis, spatial computing and Formal Concept Analysis (FCA)

  • Diatonic Theory, Maximally Even Sets and the Discrete Fourier Transform

  • Periodic sequences and finite difference calculus

  • Chord classification and combinatorial block-designs in music composition

Interestingly, most of these problems are deeply interrelated showing the existence of a remarkable interplay between algebraic formalization and geometric representations of musical structures and processes.Footnote 2 Moreover, the mathemusical dynamics (from music to mathematics to music via computer science and the possible epistemological and cognitive implications) constitutes a radical change of perspective with respect to the traditional application of mathematics in the musical domain. Mathemusical problems are characterised by the fact that settling them in an appropriate mathematical framework not only gives rise to new musical applications, but also paves the way to new mathematical constructions. By analysing more carefully the different steps of this mathemusical dynamics, one observes that it can be decomposed into the following three stages:

  • Formalization: the initial music-theoretical problem is approached by means of a combination of mathematical tools enabling to formalize it and revealing its computational character

  • Generalization: the formalized problem is generalized by using a panoply of mathematical constructions, ranging from abstract algebra to topology and category theory and leading to general statements (or theorems)

  • Application: once a generalized result has been obtained, it can be applied to music by focusing on one of the three main aspects, i.e. the theoretical, the analytical and the compositional one.Footnote 3

This decomposition of the mathemusical dynamics, together with the triple perspective of the possible musical applications of a general result is shown in Fig. 2.

Fig. 2
figure 2

A more detailed perspective on the “mathemusical” research diagram of Fig. 1, with the indication of the three main ingredients of the dynamics (formalisation, generalisation and application). The computer science is represented with the icon of OpenMusic, a visual programming language for computer-aided music theory, analysis and composition currently developed at IRCAM and integrating—in its MathTools environment—most of the computational constructions derived from the “mathemusical” research

Full size image

It is this fruitful double movement, from music to mathematics and backwards, which is at the heart of a growing international research activity where computer science is positioned in the middle of this feedback, as an interface for connecting the musical and mathematical domains. We simplify the picture by considering a homogeneous intermediate level corresponding to the place occupied by computer science with respect to music theoretical and mathematical research. By analyzing more carefully the different music theoretical problems, one may nevertheless distinguish the cases in which the computer-aided models are directly built in the formalization process as in the case of problems asking for a computational exploration of the solution space, more than a search of a general underlying mathematical theory. To this family belong, for example, typical enumeration problems such as the classification of all possible Hamiltonian paths and cycles within music-theoretical geometric spaces, such as the Tonnetz and their multi-dimensional extensions [8]. Conversely, there are cases in which the computational models are built starting from some general algebraic results, as in the case of the construction of tiling rhythmic canons corresponding to the decomposition of a cyclic group of order \(n\) into a direct sum of two non-periodic factors (i.e. two subsets of periodicity equal to \(n\)).Footnote 4 The ubiquitous role of computational modeling in the mathemusical dynamics clearly show that one may balance the usual Leibnizian perspective of music as an exercitium arithmeticae by proposing that also the reverse hypothesis holds, according to which mathematics can be considered, in some special cases, as an exercitium musicae [4].

The new interplay which I propose to establish among algebra, topology and category theory in the service of computational musicology is also necessary to successfully tackle difficult “mathemusical” problems which are linked to open conjectures in mathematics. This is the case of two major problems that have been the object of study in the last fifteen years and which can be approached in a new way: the construction of tiling rhythmic canons and the classification of homometric musical structures. Tiling canons are special rhythmic canons having the property of tiling the time axis by temporal translation of a given rhythmic pattern [5]. This compositional process is deeply connected with an open conjecture in mathematics dating from the 1970s, i.e. Fuglede or “Spectral Conjecture” [7]. This conjecture turns out to be also linked to homometry theory, a field in mathematical combinatorics that originates in crystallography, where one may find crystals having the same X-ray spectrum without being isometric. Analogously, composition naturally provides examples of musical homometric structures having the same distributions of intervals but being not equivalent up to elementary musical transformations such as transpositions or inversions. The deep connection between tiling and homometry comes from the observation that if a rhythmic pattern tiles the musical line by translation (i.e. it generates a tiling rhythmic canon), so does any rhythmic pattern that is homometric to the initial one. Moreover, in this tiling process one only has to consider tiling canons associated to factorizations of a cyclic group as a direct sum of two non-periodic subsets, since all the other canons verify Fuglede’s Conjecture [2].

This new perspective on the mathematical relevance of many music-theoretical constructions, with an emphasis on their computational character, probably played an important role in the change of perspective by mathematicians on music and mathematics as a research field. This led in 2007 to the constitution of an international society (the “Society for Mathematics and Computation in Music”)Footnote 5 and the launching of the first mathematical Journal devoted to “mathemusical” research (the “Journal of Mathematics and Music”, edited by Taylor and Francis).Footnote 6 The recognition of the mathematical dimension of the research carried on in this domain enabled the inscription in 2010 of “Mathematics and Music” as an official research field within the Mathematics Subject Classification of the American Mathematical Society (under the code 00A66). Although the research conducted by the members of this community mainly focused on classical or contemporary art music, there is a growing interest on popular music, whose theoretical problems are often as deep as those belonging to contemporary music. The SMIR project clearly suggests the necessity to push the boundaries between music genres and to look at new possibilities of interaction between contemporary art music and popular music, both providing rich music-theoretical constructions for possible collaborations between mathematicians, computer scientists, musicologists and composers.

The Originality of a Structural Approach in MIR (Music Information Research)

Following the Roadmap described in [41], we prefer to consider MIR as the field of Music Information Research instead of limiting the scope of purely Music Information Retrieval.

More specifically, the SMIR project proposes to approach “mathemusical” research by pushing further the fruitful interplay between algebraic computations, topological representations and categorical formalisations. It has two interdisciplinary levels. The first interdisciplinary degree arises within the field of music itself by treating simultaneously, as we have seen, a wide spectrum of different musical genres, ranging from contemporary art music to popular music, including rock, pop, jazz and chanson. Surprisingly, far from being easy to formalize, the compositional process in popular music can be a very rich research domain, particularly once the harmonic structures are represented in a topological way through the formalization of the Tonnetz and their generalized versions as simplicial complexes [13, 14]. Figure 3 shows the construction of the Tonnetz as a simplicial complex starting from the topological representation of notes (0-cells), intervals (1-cells), 3-note chords (2-cells) and their self-assembly by identification of a common edge.

Fig. 3
figure 3

The Tonnetz as a simplicial complex obtained by self-assembly of major and minor chords viewed as 2-cells. See [13]

Full size image

The second degree of interdisciplinarity of the project relies in the use of different mathematical concepts in a structural approach which is in contrast to the current state of Music Information Research (MIR), mainly relying on the application of statistical methods in signal processing. Instead of focusing on the signal content, our approach emphasizes the symbolic aspects of musical representations and their structural algebraic, topological and categorical formalisations. For example, a musical chord progression can be seen as a family of subsets of a cyclic or dihedral group, as a path in a simplicial complex space or, in a more abstract way, as a collection of functors and natural transformations. As an example, let us take the generating hexagonal shape of the Tonnetz structure (see Fig. 4). The simplest way to describe this shape is to consider it as a collection of symmetries relying major and minor chords. More precisely, according to the neo-Riemannian music analysis,Footnote 7 there are three ways of transforming a major chord into a minor chord by preserving two common notes and these three symmetries correspond to the \(\mathbf{R}\), \(\mathbf{P}\) and \(\mathbf{L}\) transformations. The \(\mathbf{R}\) transformation (as “relative”), changes for example the C major chord into the A minor chord, whereas the \(\mathbf{P}\) transformation (as “parallel”) and the \(\mathbf{L}\) transformation (as “Leading-Tone operator”) change respectively the C major chord into the C minor and the E minor chord. In a categorical framework, generalizing the K-net theory (or Klumpenhouwer Networks), the neo-Riemannian transformations are viewed as natural transformations between major and minor chords represented as labelled graphs with vertices corresponding to the notes and arrows corresponding to transposition and inversion operations [35]. By definition a transposition by \(h \)semi-tons is an operation indicated by \(T _{h}\) that sends a generic element \(x \)of the cyclic group of order 12 (i.e. a pitch-class in the musical set-theoretical terminology) into \(x+h\) (modulo 12). Similarly, one may define the generic inversion operation indicated by \(I _{k}\) that sends a pitch-class \(x\) into \(k-x\) (always modulo 12).

Fig. 4
figure 4

The hexagonal cycle generating the Tonnetz and represented in a categorical framework displaying the transformations between major and minor chords viewed as labelled graphs with vertices corresponding to the notes and arrows corresponding to transposition operations. See [36]

Full size image

One of the interests of using a categorical approach instead of a group-action based approach in the representation of musical structures such as chords or melodic patterns relies in the possibility to provide a flexible notion of equivalence relation, not necessarily linked to an underlying group action. More precisely, in the usual enumeration and classification of harmonic structures, two chords \(A \) and \(B \) are considered as equivalent if there exists a musically-relevantFootnote 8 group \(G \)acting on the family of all possible chord structures (i.e. the family of subsets of the cyclic group of order 12 seen as the space of pitch-classes where the algebraic structure has been forgotten) and such that \(A \)and \(B \)are related via a transformation \(f \) belonging to \(G\). Music offers many example where one would feel the need to establish a formal equivalence relation between chords which are not orbits with respect to an underlying group action. The categorical approachFootnote 9 enables precisely to overcome some limitations of the paradigmatic approach by establishing isomorphic relations between configurations of elements (i.e. objects and morphisms between objects) instead of pre-existing “types” of chords. An example is shown in Fig. 5 where three chords are “strongly isographic” (i.e. they have the same configuration of arrows) despite the fact that they do not belong to the same orbit under the action of any the three “musically-relevant” groups we mentioned before (cyclic, dihedral and affine groups).

Fig. 5
figure 5

Three chords belonging to Anton Webern’s Drei Kleine Stücke, Op. 11/2 which literally share the same type of transpositions and inversions without being in the same orbit under the action of the cyclic, dihedral or affine groups. See [36]

Full size image

A (Very) Short Journey Through Some Tools and Research Axes of the SMIR Project

As we have seen by briefly describing some examples of the interplay between algebraic, topological and categorical approaches, the SMIR project proposes to use structural approaches in Music Information Researches based on advanced mathematics. These approaches include the search of algebraic invariants to classify musical structures, the use of simplicial complexes to represent musical spaces, the use of Galois lattices and ordered structures, together with persistent homological tools and functorial approaches to describe graph-theoretical musical constructions. These theoretical concepts are systematically accompanied by computational modelling including spatial computing, a non-conventional paradigm in computer science aiming to reformulate in spatial terms the data structures and their formal manipulations, as in the case of the simplicial representation of the Tonnetz and its possible generalisations. The panoply of tools and theoretical constructions which are potentially useful within a structural approach to music information research can be classified into the three main research axes that will be shortly described in the following sections.

The Interplay Between Mathematical Morphology and Formal Concept Analysis in Computational Musicology

By using some recent results that we obtained concerning the connections between topology, music and Formal Concept Analysis [19, 39], on the one hand, and some existing relations between Mathematical Morphology [40] and Formal Concept Analysis [20] on the other hand [10], the project will investigate how to combine Formal Concept Analysis and Mathematical Morphology in order to approach in a new way some classical problems of Music Information Research. One of the most prominent examples is the automatic retrieval of musical structure. Some promising results have been obtained by Pierre Relaño in his Master dissertation devoted to the application of techniques developed within the field of mathematical morphology to the lattice representation of musical structures, with a special focus on harmony [38].

Generalized Tonnetze, Persistent Homology and Automatic Classification of Musical Styles

The SMIR project will build on some recent findings concerning the simplicial complexes and neo-Riemannian music analysis [13, 14] in order to study the problem of automatic classification of musical styles via a purely topological approach based on techniques such as persistent homology, which appears as a fundamental tool in the field of Topological Data Analysis. As in the case of this new research area that emerged from the application of computational topology in data analysis, by computing persistent homology from musical data sets it is possible to characterise the underlying musical space with a “topological signature” that reveals its structural properties. Some interesting results have been obtained by Mattia Bergomi in his PhD dissertation that is the first doctoral thesis devoted to the application of persistent homology to automatic style classification [11]. This approach has been successively applied to a structural computational analysis of popular music [12].

Category Theory and Transformational (Computer-Aided) Music Analysis

Several other “mathemusical” problems emerged in the recent years, which ask for extending the mathematical framework by also including tools and constructions belonging to other fields of mathematics, in particular category theory. As a theory of abstract mathematical structures, category theory is in fact particularly suitable for unifying different music-theoretical constructions. As we have shown, it constitutes the natural mathematical framework for the so-called “transformational music theory” [35]. Following our first attempt at developing a category-based approach to creativity [6], we will explore in this project different categorical constructions at the basis of the creative process in music analysis and composition. As we suggested elsewhere [1], category theory also provides very powerful conceptual tools that can have crucial theoretical implications for cognitive sciences and mathematical psychology. We strongly believe that the fact of coupling an algebraically formalized geometrical approach, such as the transformational one, with a computational perspective has some crucial theoretical implications for cognitive sciences and mathematical psychology. One simple way to have the intuition of this change is to compare the transformational approach in music analysis with some mathematically-oriented directions in developmental psychology and cognition, such as Halford and Wilson’s neostructuralistic approach [22], Ehresmann and Vanbremeersch’s Model of Memory Evolutive Systems [16] and Phillips and Wilson’s Categorical Compositionality [32]. From an epistemological point of view, transformational analysis provides an instantiation, in the music domain, of Gilles-Gaston Granger’s articulation between the “objectal” and the “operational” dimensions [21]. This duality was considered by the French epistemologist as the foundational basis for the very notion of “concept” in philosophy.Footnote 10 From the perspective of developmental psychology, among the three problematics which—according to the psychologist Olivier Houdé—mark the renewal of Piaget’s genetic epistemology, category theory occupies a central place [24]. Differently from the structural approach which Piaget developed starting from his logical treatise [33] and which also constitutes the conceptual framework of his researches on the “abstraction réfléchissante”, category theory introduces, according to Houdé, a new element in the operational thinking. Morphisms enable to take into account an “aspect of logical-mathematical cognition which does not proceed from the transformation of the reality (operations and grouping of operations) but which takes into account the simple relational activity [mise en relation]”.Footnote 11 Being capable of integrating these epistemological and cognitive aspects within a theoretical research will be one of the major challenges of the SMIR project. Starting from the reflections of mathematicians on the phenomenological account of contemporary mathematics, and comparing these authors with some more epistemological orientations on the cognitive aspects of the phenomenological method, the structural Music Information Researcher might find the way to constitute a new conceptual space within which some mathematical problems raised by music open new perspectives enabling to enrich the philosophical quest. This would surely lead to a better understanding of the interplay between algebraic formalizations, topological descriptions and categorical representations of musical structures and processes.