Keywords

Text. In memoriam of P r Arthur I Karshmer, 1945 to 2015, the founder of long serving chair of this S T S.

Twenty years ago, our dear colleague, mentor and Computer Science guru Professor Arthur I. Karshmer, organised a thematic session, which became a series, about access to Mathematics and Science at ICCHP in 2002 in Linz. In his introduction [3], he described the aim of this session papers as “new approaches to offer blind students a better access to math, to provide tools for doing math as well as to support teachers in teaching math.”. It has been observed many times that people with visual impairment are under-represented in STEM studies (STEM stands for Science, Technology, Engineering, and Mathematics), and we agreed that there is no specific reason in the nature of scientific content in itself. Most often STEM topics require mathematical calculation and the difficulty comes from the access to the content itself and not in the content itself [1]. This is why many research works have been carried out in the last 35 years on this domain and this series of session aims at presenting them.

Art left us in 2015 and it needed no less than 4 colleagues to take over the chair of new sessions in this series, that we entitled after Art’s name from the ICCHP 2016 conference. We try to keep up according to his inspiration. It’s now the 11\(^\mathrm{th}\) thematic session on this subject, making it one of the longest lasting STS of ICCHP, with a total of 106 papers published within the proceedings, and 10 more in the current volume. We believe that it constitutes a fair knowledge base about the topic of access to STEM content.

We are happy to see that many papers from our session this year were proposed by teams that contribute to this series since a long time (some since the very first [2]). But we are as happy to see that new researchers also join at every session of the series.

Since a few sessions, we have more and more papers about access to STEM graphical content, which was considered – twenty years ago – as more or less impossible. Indeed this type of content is very important to understand scientific topics. Recent technologies, including devices allowing for tactile visualisation but also computer vision and machine learning, allow to imagine more and more assistive tools. This year we will have 4 papers about this kind of subjects.

As for the access to mathematical equations content we had identified [1] three main areas : Accessing, Understanding and Doing calculations. These three areas are all represented in this year’s session.

1 Accessing

This was the topic of the first research works of our domain: to work on mathematical formulas, one needs to access to actual content in a format that he or she can handle. Mathematical formats such as and MathML can be used. Many Braille specific codes for mathematics are available, while many users use speech synthesis. This implied works about conversion of format but also, lately, works on accessibility of mainstream Mathematical formats.

Currently, two formats – namely PDF and PPT/PPTx – are widely spread for distributing documents to students, but their accessibility is usually not optimal, especially when they contain mathematical content. The 2 first papers, from colleagues of the Science Accessibility Net non profit organisation in Japan, are proposing solutions for these two formats, respectively [A-1] and [A-2].

Two other papers use different techniques including machine learning to help improving the accessibility of scientific documents, especially in the case of equations [A-3], and chemical formulas [A-4].

  1. A-1

    Katsuhito Yamaguchi and Masakazu Suzuki, from Nihon and Kyushu Universities: “Conversion of Multi-Lingual STEM Documents in E-Born PDF into Various Accessible E-Formats”

  2. A-2

    Toshihiko Komada, Katsuhito Yamaguchi and Masakazu Suzuki, from Nihon and Kyushu Universities: “An Efficient Method to Produce Accessible Contents for Online STEM Education”

  3. A-3

    Sanjeev Kumar Sharma, Shivansh Juyal, Neha Jadhav, Volker Sorge and M Balakrishnan, from Indian Institute of Technology and University of Birmingham: “Making Equations Accessible in Scientific Documents”

  4. A-4

    Merlin Knaeble, Zihan Chen, Thorsten Schwarz, Gabriel Sailer, Kailun Yang, Rainer Stiefelhagen and Alexander Maedche, from Karlsruhe Institute of Technology: “Accessible Chemical Structural Formulas through Interactive Document Labeling”

2 Understanding

When a user have a document in an accessible format, the next issue is to understand it. The complex structure of mathematical content make it difficult to handle it using a linear modality. Therefore tools allowing to enhance the way it is said by a speech synthesizer or to navigate within the formulas are necessary to improve their understanding. [U-1] present a system, also using machine learning, to enhance the audio rendering of equations.

  1. U-1

    Akashdeep Bansal, Volker Sorge, M Balakrishnan and Aayush Aggarwal, from Indian Institute of Technology and University of Birmingham: “Towards Semantically Enhanced Audio Rendering of Equations”

3 Doing

Finally understanding an equation is necessary but it is also needed to perform calculations, to actually do maths ! This means we need to have editors allowing to work with the content. Many projects have been presented in the past during this series of sessions. [D-1] use a well known theorem prover to give some hints to the user in order to help her or him on each step of the calculation.

  1. D-1

    Bernhard Stöger, Walther Neuper, Klaus Miesenberger and Makarius Wenzel, from Johannes Kepler Universität Linz: “Designing an Inclusive and Accessible Mathematical Learning Environment Based on a Theorem Prover”

4 Graphical Content

As mentioned earlier, we have several papers addressing the question of graphical representations, which are very important in science understanding.

In a very pragmatic way, [G-1] propose to create a corpus of images taken from a selection of schoolbooks and to classify them. [G-2] and [G-3] focus ways to interact with graphical data. In [G-2] the user interacts with Natural Language. Summaries are processed and presented to the user, and then the interface will allow the user to submit queries to the system. In a totally different perspective, [G-3] propose an interface using a Computer Algebra System to enable the user to create tactile representations of data. Finally [G-4] present a audio-tactile system, where tactile overlays are placed on the touch screen of a device allowing to play audio descriptions. The particularities of this system is that it is mobile, and able to adapt the level of content to the level of the user. Additionally it can be used to test the level of knowledge of user.

  1. G-1

    Theodora Antonakopoulou, Paraskevi Riga and Georgios Kouroupetroglou, from National and Kapodistrian University of Athens: “Developing a corpus of hierarchically classified STEM images for accessibility purposes”

  2. G-2

    Tomas Murillo-Morales and Klaus Miesenberger, from Johannes Kepler Universität Linz: “Accessible Non-visual Diagrams through Natural Language”

  3. G-3

    Thorsten Schwarz, Giuseppe Melfi, Stefan Scheiffele, and Rainer Stiefelhagen from from Study Centre for the Visually Impaired in Karlsruhe Institute of Technology: “Interface for automatic tactile display of data plots”

  4. G-4

    Michał Maćkowski, Piotr Brzoza, Mateusz Kawulok and Tomasz Knura, from Silesian University of Technology in Gliwice: “Mobile e-Learning Platform for Audio-Tactile Graphics Presentation”

This set of 10 papers, written by 31 authors and co-authors, gives a good overview of the current trends that are actively explored by researchers of this domain. It is remarkable that this year several works are using high level techniques such as machine learning, computer algebra systems, or natural language processing. We already look forward the next steps that these exciting papers will stimulate in the near future.