Keywords

1 Introduction

The interactive art installation, “Data Shed,” is designed to facilitate understanding and experiential learning about data. It conceptualizes data shedding as a “natural” process, drawing an analogy from the shedding of a snake’s skin. This representation captures the implicit generation and shedding of latent data in our digital interactions. Data shedding occurs in various interactions with technology or even when we are near it, from location tracking, sensor systems, facial recognition, search engines, to online shopping.

As data has become ubiquitous in our lives; data are all around us, and we constantly generate data without even realizing it. The rapid progress in artificial intelligence and machine learning, fueled by data, has given rise to new algorithms and technologies that are accelerated by the rapid breakthroughs in the field. Despite having all these data and access to powerful tools we understand so little about how data is collected and how data directs our decisions and choices in this well woven digital environment and economy. Data fluency is the key to unlocking knowledge and understanding how data has become the main ingredient of the digital age.Footnote 1

Data are used to make many of our day-to-day activities easier. For instance, a person purchasing a transit ticket in many places around the world provides the data from the station they are boarding and the destination station. The data trade-off gives insights such as identifying the stations that see higher footfall, which can then be used to design stations and transit schedules, responding to the footfall at different stations. This is an example of how a small interaction can lead to data generation in an implicit manner, where the users are not aware they generated data through their interactions. These examples can be described as latent data that exists around the user without them knowing about it.

As more organizations start collecting personal data, there is a growing need for people to understand this hidden force acting on their behalf, making decisions for them, and how all this data is being collected with the use of new algorithmic systems.

Given this context it is essential for us to seek more ethical and conscientious methods of utilizing the vast amount of data collected. “Data Shed” fills this gap in knowledge by actively addressing the imperative for ethical data practices. “Data Shed” responds to the call for ethical data practices by empowering users with insights into the often-unnoticed world of data shedding, thereby contributing to a more informed and responsible approach to data utilization.

2 Related Work

The literature review draws on a wide range of sources, including studies on data collection, algorithmic bias, and critical data studies to form the foundation for understanding the multifaceted landscape of data in our digital age. The exploration extends to the rise of machine learning, addressing concerns related to surveillance and data shedding. Related work includes generative artworks and human-AI collaboration, contributing to the theoretical groundwork for understanding the intersections of technology, art, and data. This theoretical framework forms the backdrop against which the “Data Shed” project unfolds.

2.1 Navigating Data’s Dominance in the Modern Age: Algorithms, Acceleration and Data Shedding Dynamics

In the past decade, the world has seen a tremendous increase in the use of artificial intelligence (AI). AI systems are evolving so fast that it has become increasingly possible to build an intelligent computer system that is programmed to do a single task. For example, a camera can extract data regarding what people are wearing to analyze the number of people who wear a tie during the day. Intelligent machines accelerate the data collection process several folds, often automatically, creating a gap in the knowledge of the data being generated, collected, and analyzed. Efficiency and quantity push data to the forefront of our digital economy as “The amount of data in the world doubles every two years” [1]. Clive Humby’s statement “Data is the new oil’’ has echoed relentlessly in recent years as people have started to harness and understand the value of data [2, 3]. Traditionally, one way of using this data was to sell more things through targeted advertisements [4]. Unrefined data is meaningless; it makes sense when connected and analyzed [5], which makes it very crucial to start looking at data as a system [6, 7].

Algorithms within the field, like computer vision, can produce data from multimedia which can be used for object recognition, facial recognition, and motion recognition. In addition, these algorithms can filter data and find the needed information quickly. Although latent to us, data with the help of algorithms are making decisions for many of our day-to-day activities [8].

2.2 Data Bias

Modern algorithmic systems are plagued by bias, as these systems are actively trained on historical data to help the algorithm understand context based on historic decisions [9]. The datasets often contain data that is biased, compounded by bias that engineers bring to choices in the data that they use to solve questions, giving rise to discrimination in algorithms [10]. Cathy O’Neil scrutinizes the ways that algorithmic decision making can worsen existing inequalities and keep them in place [11]. As these systems are built for predicting outcomes, they are more likely to give biased predictions due to the nature of dataset used to train them. As Julia Angwin proves risk scores in states such as Florida discriminate against people by race and label those with higher scores as being more likely to be criminals when empirical evidence contradicts machine learning outcomes [12].

The more data these algorithms access, the more accurate they can be in their predictions. Yet this has not been the case. Rather, as Wendy Hui Kyong Chun demonstrates, it is “slippery identifications-mis-and missed identifications- that form the basis for recognition and correlation” [13, pg. 228]. Bias in datasets can arise from various sources, including historical inequalities, under representation of certain groups, or systemic prejudices embedded in the data collection process [14]. Thereby these models do not work well with outliers, eliminating small data sets, or with data that was not widely available to the model in the first place while training, which further adds to the problem of data bias in such systems [15].

“Data Shed” is motivated by this growing recognition of data biases. As we are working with vision-based ML (Machine Learning) models, it becomes important to establish that these models are not perfect and that they can be biased. “Data Shed” is a real-time system that does not directly address the issue of data bias, but grapples with it indirectly through interactive artworks to raise awareness of latent data.

2.3 Data Fluency and Critical Data Studies

Data fluency is the ability to understand structural meaning of data and be able to comprehend and explain data through a more granular level of detail [16]. It involves being able to recognize patterns and connections between data to make better decisions and translate these to others [17]. Although many people have become more concerned about how their data is handled, managed, and used by organizations, their behavior still does not reflect this concern as individuals continue to tick consent boxes as though they do not matter [18]. This behavior is known as the “Privacy Paradox” [19].

The privacy paradox can be attributed to lack of understanding of data-driven systems compounded by the limitation of services if individuals do not comply and share data. The organizations that hold data are an integral part of daily life and have few incentives to offer control to their users to control their own data and even if they do, it is buried under the many layers of settings that an average user is not aware of. Technology users struggle to comprehend data and protect their privacy.

One can say, “Data is intangible” and “We are data,” as anything that users do online generates data. Data subtly influences recommendations and decisions, often operating beneath users’ explicit awareness. [18]. The revelation of these concealed data points is vividly demonstrated through targeted advertisements. An illustrative example is presented in the New York Times article “These Ads Think They Know You,” which scrutinizes ads from a data-centric perspective, unraveling how user data triggers specific advertisements [20]. Examining a sample ad reveals a keen understanding, stating, “This ad thinks you’re trying to lose weight but still love bakeries,” drawing insights from the user’s browsing and credit card history.

As data has become building blocks of everyday life, Dalton and Thatcher advocate for a critical perspective on the overwhelming dominance of data in modern society [21, 22]. This dominance has given rise to “Algorithmic Culture” where human thoughts are expressed through big data and computation [23]. It has become complicated for individuals to analyze and interpret data in a meaningful way, as there is a divide between the person that is generating the data and the person that is reading the data [21, 22]. Dalton and Thatcher argue that data production is place-dependent, giving rise to data biomes and illustrating the challenge of accurately reading big data sets out of context, compounded by missing data. Available data could not portray the full picture to the reader who might be unknowingly making decisions at some remote location.

The whole data generation and collection process is often seen as a flat process that does not take into consideration the other hidden aspects that influence data production; there is a growing business and institutional faith in ‘big data’ as a solution for all problems and that data is the key to unlocking human understanding. Data’s role in targeted marketing and the surveillance state are clear, but what other purposes could it serve?” [21, 22]. Can this data help us to understand people through artworks and artifacts, but how does this data relate to people in real life?

2.4 Rise of Machine Learning and Artificial Intelligence in Addressing Surveillance and Data-Shedding Concerns

The rapid advancement of machine learning and artificial intelligence has ushered into a transformative era, impacting various facets of society, with particular significance in the context of surveillance and challenges surrounding data-shedding. The ability to detect, classify and analyze complex data is often enabled by classifier systems, predictive models like linear regression, decision trees, random forests, gradient boosting, and neural networks in machine learning algorithms. As these systems become more common and complex, the associated risks and impacts will increase significantly making it possible for malicious actors to access both the entire model and the training datasets, through a malicious ML provider posing a significant security concern [24].

Classification without permission, location data being shared to advertisers, geolocation services used to track users in the real world, user profiling and aggregation of data without user consent are all examples of privacy concerns arising from the increased reliance on these technologies [25,26,27,28].

Facial Recognition Technology has integrated itself into the fabric of our lives as it is so widely used and scalable, living in a time where we are constantly being monitored and tracked using CCTV’s and other surveillance cameras [29]. A byproduct of these systems are people losing their sense of privacy; trying to figure out how to evade these technologies to avoid being identified or misidentified. In 2019 Hong Kong police used face detection technology to identify protestors against a controversial extradition bill [30]. Facebook used facial recognition to track users based on the pictures they posted but then later removed this feature, reflecting an acknowledgment of the privacy implications associated with classifying individuals without their explicit consent [31]. This feature could have been used to protect users’ privacy by limiting the impersonation of users on Facebook. Apple introduced Face ID in 2017 as a replacement to the fingerprint scanner on the iPhone X which holds biometric data that includes depth mapping of a user’s face and infrared images of their face [32]. Grave damage to users could occur in a data breach, as a face cannot be changed as opposed to passwords that can be changed.

This context sets the stage for the significance of “Data Shed,” which addresses the ethical dimensions of data use and sheds light on the implications of unconstrained data collection. It serves as a proactive response to the growing concerns outlined in the literature review, offering an interactive platform to educate users about the intricacies of data dynamics and encouraging responsible data practices.

2.5 Exploring User Understanding: Generative Artworks and Human-AI Collaboration

Human and Artificial Intelligence Collaboration has ushered in a new era of creativity, where the synergy between human ingenuity and advanced AI technologies fosters unprecedented innovation. As AI systems evolve, their integration with human capabilities forms super teams, transforming traditional workflows. This collaboration yields unique expressions and innovative creation methods. One example is the Casual Creator, an interactive system promoting fast and joyful exploration of creative possibilities, resulting in the generation of artifacts that instill pride and creativity in users [33].

The synthesis of workflows between tools and artists has given rise to systems like MidjourneyFootnote 2 and DalleFootnote 3, enabling artists to co-create with AI. This collaborative approach varies, with artists either inputting parameters or the AI system leading the creation process. Examples such as “Move Mirror” and “Body, Movement, Language: AI Sketches With Bill T. Jones” showcase diverse collaborative workflows, blending human expression with AI capabilities to produce engaging and boundary-pushing content. Defining collaboration in the realm of human-AI interaction may be nuanced, but the potential for exciting possibilities in creativity remains a constant driving force for both sides.

In the ever-evolving landscape of technology and human interaction, the realm of generative artworks stands as a captivating and relevant piece with immense potential for further exploration. Parikh’s study indicates predictability in user preferences through choices made while creating interactive generative art as subtle interactions with generative art can unveil unforeseen data regarding user preferences [34]. In the synergy of generative art and AI, “Data Shed” facilitates interaction between the AI-driven generative art and the audience by intentionally releasing and utilizing data. This intentional sharing of data contributes to a continuous feedback loop, allowing the generative art to adapt and evolve based on user interactions. The collaboration becomes an iterative exchange where data shed refines the creative potential, ensuring the artworks resonate meaningfully and personally with the audience.

2.6 Related Works

The works discussed in this study seek a common goal of visualizing latent data in a visual format. These works played a crucial role in conceptualizing ideas for an interactive installation, focusing on interactivity, data fluency, live data, and the interplay of artwork with dynamic data. “Data Shed” builds on precedents such as “Thermal Drift Density Map” which uses thermal data as a metaphor for life force and military precision technologies (Lozano-Hemmer, 2022), “How Normal am I” which compares and distorts facial data (Schep, 2020), “Herald/ Harbinger” that draws real time bedrock data from the Bow Glacier to create an interactive sound and image work in Calgary, the Centre of Canadian extraction industries (Ruben & Thorp, 2018) and lastly “Visualizing Algorithms” which slows down algorithmic processing to indicate data flow in decision tree classifiers (Catherine Griffiths, 2020). These are interactive artworks that display and analyze data in experiential and moving ways, and at the same time encourage audiences to think critically about the sources of data behind these works.

Through a comparative analysis, the table below indicates levels of interactivity, intention to address data fluency, approaches to data and algorithms, and approaches to interactive media art (Table 1).

Table 1. Analyzing Contextual Works: Exploring Similarities and Differences, Anchored by the Artwork ‘Data Shed.
Full size table

2.7 Gap in the Study and Need for “Data Shed.”

The literature review underscored critical gaps in understanding and addressing the profound implications of big data, particularly in the context of surveillance and decision-making trades offs. While the increasing reliance on machine learning algorithms and artificial intelligence has accelerated data collection, there is a notable lack of awareness among individuals regarding the depth and breadth of data capture. The privacy paradox, as discussed in the literature, highlights the inconsistency between concerns about data handling and the actual behavior of individuals who may unwittingly sacrifice privacy for convenience. Additionally, the literature points out that the overwhelming dominance of data in modern society, often viewed as a solution to various problems, tends to oversimplify the complex nature of data generation and collection. The challenge lies in recognizing the inherent biases in datasets, the limitations of data fluency among users, and the potential societal implications of surveillance technologies. Thus, literature emphasizes the need for a critical perspective on data, fostering awareness of the trade-offs involved in decision-making processes influenced by vast and often indiscriminate data capture.

All the above concerns point towards a need for an interactive installation that educates about data by combining elements of interactivity, data fluency, and live data, addressing a space where these aspects are not fully explored or integrated in existing works. The comparative analysis of contextual works emphasizes this opportunity for a unique approach to teaching about data through an engaging and interactive installation.

3 Methodology

3.1 Process Overview

The creation of “Data Shed” adopted Research-Creation as the chosen methodology, a practice-based method that intertwines artistic creation with research. The interactive installation becomes the canvas, providing a unique space to explore and visualize the intricate dynamics between data, algorithms, and their societal repercussions.

Challenging the assumption that knowledge is primarily conveyed through verbal or numerical means, “Data Shed” employs data visualization techniques to transcend these conventional boundaries [35].

Fig. 1.
figure 1

The conceptual framework to apply Research Creation to create an interactive installation.

Full size image

The interactive installation serves as a tangible and immersive experience, offering the audience an opportunity to actively engage with and contribute to generative artworks using their pose data (Fig. 1).

3.2 User Engagement and Interaction

Movement-sensing installations engage audiences in actively influencing computer-generated images and sounds through natural movement. These interactive setups require no special training, fostering collaborative and participatory experiences [36]. This unique dynamic leads to a greater acceptance and ownership of the creative results by the participants. In addition, these types of installations can be embedded into environments, making them a promising tool for educational purposes.

User engagement and interaction with the installation is the central aspect of the process as the more engagement the user has with the installation the more information, that is data, they can provide to the underlying models to improve the accuracy of the models and algorithms. The goal is to make the processes more visible to the user and aid the audience in gaining a deeper understanding of the workings of the digital environment through interaction, building upon the taxonomy from “A Common Framework for Audience Interactivity” paraphrased below [37].

Observe Passively:

  • At the foundational level, users may choose to observe passively, merely taking in the visual stimuli of the installation without actively participating.

  • Passive observation allows individuals to absorb the atmosphere and artistic elements without direct involvement, providing a space for initial comfort and exploration (Fig. 2).

Fig. 2.
figure 2

Levels of audience interactivity, adapted from [37]

Full size image

React to Other Audiences:

  • Moving a step beyond passive observation, users may engage in a reactive mode, responding to the presence and actions of other audience members.

  • This level of interaction creates a dynamic environment where individuals become part of a collective experience, sharing reactions and responses with those around them.

React to Group Experience:

  • As user engagement deepens, participants may collectively react to the group experience, forming a shared response to the installation.

  • This level fosters a sense of community and shared connection, as individuals synchronize their reactions based on the overall atmosphere created by the group.

Influence Performance:

  • At a more involved stage, users may actively influence the installation’s performance through direct input or manipulation of elements.

  • This level empowers individuals to shape the artistic output, allowing for a personalized and co-creative experience that blurs the lines between audience and performer.

Become a Performer and Take Over Performance:

  • The highest level of audience interactivity occurs when participants transition from observers to performers, taking over aspects of the installation’s performance.

  • Users, in this scenario, become active contributors, directly shaping the narrative or visual elements of the installation, transforming the space into a collaborative and participatory platform.

The goal of “Data Shed” is to encourage audience experiences that encompass all these levels of interactivity.

3.3 Data Visualization

Data Visualization plays a fundamental role in data collection by transforming raw and complex datasets into accessible and interpretable visual representations. In the modern era of information overload, where massive amounts of data are generated daily, visualization serves as a powerful tool to distill intricate patterns, trends, and insights from the data. In this way, complicated data relationships and data-driven insights are communicated in an easily comprehensible manner. Moreover, interactive visualization enhances transparency and accessibility, making data-driven insights more comprehensible to a wider audience, thereby fostering a deeper understanding of the underlying information, and promoting data fluency in various fields [38]. While building a data visualization system, several factors are necessary to ensure a user-centered design approach be it diverse types of visualization or approaches to presenting data [39].

Individuals increasingly learn about themselves through mobile telephones and personal informatics, which visualize the user’s data [40]. A goal of “Data Shed” was to arrange the data collected and its analysis to help the audience understand the essence of their data in an artistic format of color and shapes rather than the algorithmic outputs of numbers and statistics. The visualization was designed into two stages: (1) data collection and classification and (2) the analysis and visualization of the collected data. The abstract visual language of the system is intended to promote self-discovery in understanding the meaning of the data collected and the abstractions derived from the data [34].

4 Prototypes

Iterative incremental prototyping was used to design and improve the artifact and the interaction with the audience based on their feedback. Prototyping allowed different ideas to be tested, as well as the feasibility and functionality of the technology.

Prototype 1 tested different machine learning models combined with different visualization generation techniques. The basic technology deployed for this prototype were web-based machine learning libraries and P5.JsFootnote 4 to visualize an interface deployed in a web browser. A small web application in a browser started latent data collection culled from interaction with an interactive artwork that appeared on a large screen.

Prototype 2 extended functionality and complexity leveraged from the framework of the first prototype and incorporating a LED (Light Emitting Diode) array-based installation that would allow the user to not only interact with the installation but also have a richer visual experience. The second prototype moved from a simple web-based approach to the use of VVVVFootnote 5, a visual scripting tool based on.NET platform.

For audience privacy, the artifacts never used cloud-based solutions to compute audience data. All captured data were processed locally and not stored. They were solely used to create artworks displayed on an LED wall. The artifacts from the interaction ensure no disclosure of confidential information or profiling, capturing only non-intrusive data points without facial profiling or unnecessary retention.

4.1 Prototype 1 Details

The emphasis was on collecting latent user data, exploring the method of implicit data collection through sensor and machine learning interactions. Implicit collection involves observing user interactions and engagements with artifacts. The choices contribute to building a decision table, covering aspects like color preferences and user curiosity during interactions. The sketches were built with P5.Js and incorporated PoseNetFootnote 6 to detect the user’s face and the body moment data to make interactions possible and collect latent data.

Color Preference.

The first artwork was designed around a preconfigured scheme of color swatches. The interaction involves three phases: idle, where the artwork waits for someone to enter the interaction parameters; change, triggered by clapping within the interaction parameter, leading to a shift in color scheme; and engage, where continuous engagement generates a series of artworks based on the chosen color scheme, creating a new artwork every second of the interaction (Fig. 3).

Fig. 3.
figure 3

Generative Artworks for Interaction with Color

Full size image

Left-Right Preference.

The second artwork was centered around creating bold circles with varying weights. The interaction involves three phases: Idle, where circles move randomly on the screen, and new weights are assigned to each circle, generating a new artwork every second; Left Push, where the user uses their left hand to arrange circles based on weights, with higher weights on the left; and Right Push, where the user uses their right hand to arrange circles with higher weights on the right. Both interaction states result in a visually distinctive presentation of circles based on the applied weights (Fig. 4).

Fig. 4.
figure 4

Generative Artwork for interaction with weight

Full size image

Prototype 1 provided valuable insights into the interplay of generative art, AI, and data shedding. Users were observed to actively shape their preferences through interactions with the generative artworks, often without conscious awareness. Simultaneously, users also displayed an understanding of the underlying interactivity that played a crucial role in molding their overall experience. The intentional release and use of data in the generative art process highlights the potential for dynamic and engaging interaction between the system and the audience. Prototype 1 set the stage for further exploration and refinement, emphasizing the promising avenues for creative collaboration and innovation at the intersection of technology and human interaction.

4.2 Prototype 2 Details

At this stage of prototyping the emphasis was on creating a functional but visually rich interaction which demonstrated the concept of collecting data and projecting them in a visual way. Prototype 1 focused on simple interactions which could be used to generate latent data by the user. Prototype 2 developed the visual design of the final installation; the LED curtain made a dynamic and responsive output. As LEDs (Light Emitting Diode) can be controlled and adjusted individually these display different animations and visuals with specific brightness patterns and colors. The curtain was made up of two parts with the first part being the curtain wall and the second part being the LEDs scattered on the ground. The interface was divided into different interactions and animations. The first part contained data visualizations that were generated from the user’s movement and other data (such as clothing) while the second part of the curtain contained the visuals and animations that showed the flow of latent data shed in response to the user’s movements.

The installation was built with VVVV, visual programming software, to develop the software and hardware solutions for our installation. VVVV enabled the use of different.NET libraries and custom code to implement machine learning models and data visualization techniques. ArtNetFootnote 7, a protocol for data transmission over Ethernet controlled the LED strips that form the main visual element of the installation. Microsoft Azure Kinect DKFootnote 8 was chosen as the sensor, as it provides a wide range of features such as depth, RGB and infrared cameras, and a 7-microphone array. These features captured various data points from the user, such as pose, body index, segmentation, point cloud, IR and RGB data. Machine learning algorithms processed and analyzed these data using them to generate the visual output on the LED wall (Fig. 5).

Fig. 5.
figure 5

Photos of the prototype 2 with viewers engaging with it.

Full size image

Prototype 2 marks a significant advancement in the exploration of generative art, AI collaboration, and data shedding in this research project. The integration of a LED wall appeared to empower users to play a more active role in shaping the evolving artwork. The technical goals were met. The user experience appeared to be seamless and immersive. Challenges occurred in achieving a balance between user influence and showing the underlying layers of data shedding in a simplified manner. Prototype 2 served as a steppingstone for continued exploration in the creation of a final installation and exhibition of “Data Shed”.

5 Installation

The final iteration of the installation was developed to house a data double of the person derived from interaction. The visual output was dynamic and responsive to the user’s movements. The system housed more refined models and algorithms to differentiate between the user who was interacting and the audience in the scene. The installation also included a UI (User Interface) to allow the user to interact with a secondary display that acted as an output for the data that was being read by the machine learning algorithms. The users could select and play with the data to view and experiment with the data in a unique way by changing the values or disabling different data streams to create different or broken visualizations. Viewers were encouraged to understand the data collected and its representation by clicking a picture of the artwork they created while collaborating with the installation (Fig. 6).

Fig. 6.
figure 6

Photo of the final” Data Shed” installation, including the labeled hardware and software; latent data is captured and presented within the piece dynamically.

Full size image

Building upon the successes and lessons from previous prototypes, the final version presented a refined experience for users. Users engaged with the installation in playful ways and sought to manipulate patterns on the LED curtain. However, most audience members, while aware of their role in driving the installation did not perceive that their data was captured, analyzed, and were moving the images on the LED curtains. Only those who engaged with the data display and algorithm visualization monitor and the LED curtain fully understood the lessons of “Data Shed.” Refinement of the installation would provide a larger role for the data and algorithm capture display.

6 Evaluation, Conclusions and Future Work

6.1 Evaluation

The evaluation was an Observational study, a method for meticulously observing the behavior of individuals within a specific environment [41, pg. 86]. This approach was chosen to gain insights into user interactions with the “Data Shed” installation, ensuring a comprehensive understanding of how participants engaged with the project’s technical features and interactive elements (Fig. 7).

Fig. 7.
figure 7

Photo of a people engaging with the installation

Full size image

Technical Goals.

The observational study provided a firsthand opportunity to scrutinize the effectiveness of technical functionalities, specifically the integration of the Microsoft Azure Kinect DK sensor and machine learning algorithms. The sensor captured diverse data points such as poses, body indices, and segmentation precisely, acknowledging its pivotal role in the project’s functionality. The real-time processing and adaptability of machine learning algorithms became apparent as users actively engaged with the system, ensuring a responsive and dynamic user experience.

Interactivity Goals.

“Data Shed” was designed to offer a spectrum of interactive engagement drawing from Striner et. al.’s (2015) Common Framework for Audience Interactivity, ranging from passive observation to active participation, fostering a diverse and immersive user experience. Observations during the interactive sessions highlighted the success of distinct levels of interactivity. Users, at the passive observation level, absorbed the visual and auditory stimuli without active engagement, highlighting the installation’s ability to captivate audiences even in a non-interactive mode. Moving up the interactivity spectrum, users reacted to other audience members, adjusting their interactions based on the collective experience. As the interactivity levels progressed, participants actively engaging with a group experience, collaborating with others to shape the overall performance expressed by LED visualizations. Users actively influenced the visual and auditory outputs, taking on a more participatory role. Some audience members realized that the system was blind to certain colors and returned over several days to test its response to their clothing. The take-over level allowed users to become performers themselves, exercising full control over the interactive elements, indicating the project’s versatility in accommodating varying degrees of user agency, manipulating the data and algorithmic screen.

Data Fluency Goal.

The data fluency goal of “Data Shed” aimed to enhance users’ understanding of their personal data by translating complex data concepts into visually accessible and comprehensible forms. The project partially met its data fluency goal. Users engaged with dynamic visualizations that represented latent data shedding processes, offering a tangible and intuitive representation of how data is generated and interacted with in the digital realm. While some played with the installation to manipulate its images, they did not necessarily understand the connection between the data that they shed and the machine learning algorithms driving the images. ML (Machine Learning) Future iterations of the installation would provide a large screen display of data collection and algorithms at work to emphasize the path between the user’s data, ML (Machine Learning) algorithm, and LED display.

6.2 Conclusions

  • “Data Shed” was designed to encourage users to understand this data with the help of computer vision machine learning models and data visualization through interactive installation. The installation’s aim was to encourage users to take control of their data and understand how the system in place collects it. Below are further takeaways:

  • Interdisciplinary Collaboration: The project underscores the importance of interdisciplinary collaboration in creating meaningful and impactful work. Artists, data scientists, and engineers can work together to create installations that are not only aesthetically powerful but also informative and thought-provoking.

  • User Engagement: The interactive nature of the installation can lead to higher user engagement. It shows that when users are actively involved in the process, they are more likely to understand and appreciate the underlying message and reflect on the future of data and the ways it shapes our lives.

  • Ethical Considerations: The project serves as a reminder of the ethical considerations in data collection and usage. It highlights the need for transparency and consent in all stages of data handling.

  • Public Education: Installation art can be an effective tool for public education and can be used to communicate complex concepts in a simple and engaging manner.

  • Innovation in Art: The project shows how modern technologies like machine learning and computer vision can be used in art to create innovative and exciting new forms of expression.

6.3 Future Work

The future stages of the installation are:

  • Future development would program machine learning models and algorithms to respond to the interactions triggered by the users and not just the automatic collection of data. These would give the user the ability to choose the models and algorithms that they want to learn about. For example, if the user wants to learn about the voice models the install should be able to trigger the specific voice model and present to the user the information.

  • Adding more visuals and graphical representations of the data as a second screen or set of annotations to help the user understand their data shedding process.

  • Theoretical research in Human Computer Interaction to better understand the ways that interactions as a framework combined with artworks and machine learning models represent a novel approach to design learning experiences that allow people to explore and gain a deeper understanding of data through interaction.

In summary, this work has presented “Data Shed” as an approach toward raising awareness of the latent data dynamic, by leveraging interactive art exhibition, machine learning, generative design, and physical LED visualizations. It is hoped that such an approach can become a catalyst to shift perspectives on this central aspect of modern life.