Keywords

1 Introduction

Training in lighting design for theater is increasingly grounded in new technologies [7]. A growing momentum towards the incorporation of new digital tools including computer-based “magic sheets” and digital lighting consoles simplifies the work of lighting designers while also supporting diverse talent through accessibility offerings [4]. As the industry also moves away from traditional classroom education, there is a need for alternative options that will allow future lighting designers to practice their trade [4].

1.1 Background

Access to resources, time, and spaces in which theater artists can practice lighting design is an issue for junior theatrical lighting designers and theater students [5]. Tight budgets, small turn-around times between shows, and a lack of equipment and other training resources limits the amount of hands-on experience aspiring designers can have [4].

When training through the classroom, lighting students learn the basics alongside many peers, sharing scarce resources and equipment [5]. Teachers will expose students to lighting equipment, terminology, and basic practice. After learning the basics, students will then be asked to demonstrate their skills through design documentation, written and practical tests, or sometimes even by working on small shows [11]. Experienced staff will expect junior lighting designers to exhibit good design and communication of their ideas, using industry standard terminology [11]; however, many junior lighting designers have few opportunities to practice verbal communication of design in working scenarios [5]. The ability to communicate to a board operator is an integral skill for lighting designers [11], but it can be difficult for students to have access to spaces, peers, and equipment with which to practice these skills in a meaningful way [5]. This tooling provides a new medium through which designers can practice and test their communication skills without said limitations.

1.2 Approach

This project combines virtual reality technology with Alexa voice commands to address gaps in the technical education of lighting designers. The Alexa user interface takes on the role of a board operator. In theater, board operators listen to commands made by the lighting designer and enter each command into a board that controls the lights in the theater. Designers must learn to work effectively with their board operator to create looks and cues for the stage. Designers therefore must learn to communicate their designs. By training with a lighting visualization tool that leverages voice control, designers gain experience creating lighting looks using the same industry standard terminology and commands that they would employ in a real theater.

Fig. 1.
figure 1

The Keylight stage with a lighting design. The icons show where lights are hung in the space, with different types for the house and stage lights. A large camera icon intersects with a table light near the bottom of the image. This shows where the user sits in virtual reality, next to the board operator at a table in the house.

Full size image

The use of VR with visualization software does produce some challenges. Computers with powerful graphics cards are currently needed to be able to run VR programs without disruptive frame drops. Not everyone has access to such computers, and the adoption rate of many virtual reality headsets is still relatively low. To address this challenge, KeyLight was designed to also functions without virtual reality equipment (Fig. 1).

1.3 Lighting Terminology

Users of the software must have basic theater lighting knowledge. Here are some common definitions for the concepts that appear throughout this paper:

  • Channel: An organizational number, assigned by a user of a lighting control board. Essentially serves as a name for some number of dimmers.

  • Dimmers: Allow users to change a light’s intensity. Connected to lighting fixtures.

  • Groups: A group of channel numbers. Allows control of multiple channels at once.

  • Light plot: A diagram showing the placement, type, orientation, color, and more of all the lights being use for a production.

  • Cyc: A large, blank surface placed on the back of the stage on which lights are focused to create a background effect.

  • House: The area where the audience sits in the theater.

  • Half: Setting a fixture to fifty percent intensity.

  • Full: Setting a fixture to one hundred percent intensity.

1.4 The Magic Sheet

The magic sheet is a “cheat sheet” for a light plot. It provides designers with information about the lights that can be gathered at a quick glance. Groups of lights are separated into multiple miniature plots of the stage. The stage is oriented on the magic sheet as you would see it from the house. Outlines of the set are visible in each square to help orient the designer. The name and number for each group is given above each plot, and available colors are shown as well. Arrows show the direction of the lights, and channel numbers are placed onto the spots where their fixtures will affect the stage. An example of a magic sheet is seen in Fig. 2.

Fig. 2.
figure 2

The magic sheet provided in KeyLight. This is placed on a table in front of the user. Each square shows the stage in miniature with dotted lines indicating the placement of set pieces. Group numbers are next to the name of each group (i.e. the Up Lights are Group 20 and include channels 21–25). Each channel’s focus is indicated by its placement on each plot.

Full size image

2 Related Work

Many lighting visualizers already exist on the market. Software is used by theater professionals to pre-program shows, plan lighting designs, make decisions about where to hang or focus lights, and more. However, popular lighting design tools in the market today do not incorporate a vocal interface [11]. Designs for systems that use voice interfaces to control physical lighting fixtures have already been patented but not incorporated into digital tooling [2]. Designers today use physical or digital inputs to control their lighting consoles or visualization tools. The addition of a vocal input innovates on top of these previously designed and proven systems.

Virtual reality and other digital tools are becoming more prevalent in theater spaces. Virtual reality is being explored as a tool for visualizing lighting design before events and productions [1]. Large companies such as Vectorworks have created VR visualization tools which can be used to view designs for anything from architecture to set design [12]. Other projects have used VR as a visualization tool, using mechanical controls and motions within VR to control the space [1].

Novel lighting software, which include digital magic sheets or accessibility features such as touch controls and auditory responses, are being explored to support the diverse community of lighting designers [4]. While researching related work, however, the authors found no existing tools that combined lighting visualization, design tools, and voice controls into one program. Hence the need for a system such as KeyLight is apparent.

Fig. 3.
figure 3

Users sit at a table in the audience and look out upon the stage they are designing.

Full size image

3 Key Light System

3.1 Overview

KeyLight is a system that provides visualization for lighting within a theater hall (Fig. 3).

The system is designed using Amazon Alexa, AWS Lambda, Amazon Simple Queue Service (SQS), and the Unity Game Engine. Many of the limitations of the system, discussed throughout the paper, could potentially be improved by switching to a different vocal processing tool. A designer utilizing a virtual magic sheet similar to the one shown in Fig. 4a vocalizes a command to a personification of Alexa as seen in Fig. 4b. Alexa then processes the speech as an intent, sending information about the command through the model created in the Developer Console.

Fig. 4.
figure 4

KeyLight work environment

Full size image

Slots in each intent are filled with keywords, such as the numeric value of a group number as seen in Fig. 5.

Fig. 5.
figure 5

An Alexa Intent sheet example, as displayed in the Alexa developer portal.

Full size image

The Alexa service has a custom AWS Lambda endpoint; all code for handling activated Alexa intents is stored in AWS Lambda and is written in Python. As depicted in Fig. 6 When a valid command is received, such as changing a group of lights to a new color, the command is parsed and sent into an AWS SQS FIFO queue. Unity continually polls the queue to check for new jobs. If one is found, the Unity code removes it from the queue, processes it, and sends the command as an event. Lights in the Unity scene listen for these commands and change their values as directed.

Fig. 6.
figure 6

KeyLight Trainer’s backend is built on interactions between Amazon Web Services, user input, Amazon Alexa, and the Unity Engine.

Full size image

3.2 Current Steps for Use of KeyLight

The steps below describe how the system works in its prototype phase. Future developments would bypass steps I–IV by using any Alexa device; however, Amazon certification for a completed application is needed for this functionality so this will have to be implemented after all development is completed for the project.

  1. I.

    Sign into the Alexa developer portal using the username and password given.

  2. II.

    Under Alexa skills, click the skill labeled as “KeyLight”.

  3. III.

    Go to the Test tab.

  4. IV.

    To interact with Alexa, either hold the microphone button down and talk or type into the terminal.

  5. V.

    Launch the Alexa component of KeyLight by saying or typing “key light trainer”.

  6. VI.

    Open the Unity executable.

  7. VII.

    Optional: Put on VR equipment.

  8. VIII.

    Begin directing Alexa using the lighting commands in Sect. 3.

3.3 Design Choices

Polling Design: Due to Santa Clara University network restrictions on incoming traffic, it was necessary to create a system in which Unity asks the queue for new jobs and receives a JSON response. After lots of trial and error, other models, such as using the Unity Networking packages or using client-server communications, proved unsuitable for this task.

Custom Endpoint: Rather than hosting the Alexa code in the Alexa development console itself, KeyLight uses a custom AWS endpoint in Lambda code. This allowed for easier editing and management of the actual code running behind the scenes to handle the Alexa requests, and the use of Amazon’s Simple Queue Service (SQS).

Intents: The commands discussed in Sect. 3.4 have been created as separate intents to clarify their purpose.

Linguistic Limitations

  1. I.

    At the time of this writing, Alexa is unable to differentiate between intents, based on the input provided. If one intent is formatted to accept channel_number color_name with no other surrounding words, and another is formatted to accept channel_number light_level, Alexa’s voice parsers will sometimes plug a command such as “ten red” into the latter of those commands, throwing errors. This could possibly be circumvented with code, but after some time working through this problem, it was decided that KeyLight would enforce a few rules about each command that would still be applicable in traditional lighting design communication. Levels are adjusted by saying “at” (ex. “channel ten at fifty”) and colors are changed by saying “to” (ex. “channel ten to red”). If neither word is present, the intent will be processed as a color change, as it is more common to say, “channel ten red” than, “channel ten fifty”.

  2. II.

    Alexa has difficulty processing combined words that should fit into one slot. Thus support for colors such as “red-brown” and “green-blue” were added, but in tests Alexa only processed “red” or “green” and ignored any inputs following that.

HTC VIVE: The program is designed for optional use with the HTC Vive, but as the controllers are not used in KeyLight, it can be easily extended to work on other devices.

Future Simplification of Code: For speed during the prototyping phase, an event listener model was used. The Unity code can be simplified in future versions to use a game object with all the light objects attached, rather than an events system.

3.4 Commands

Each command is associated with a separate intent. The phrase “start key light trainer” invokes the Alexa skill (Table 1).

Table 1. Important Alexa Intents.
Full size table

Intents are comprised of defined phrasing that combines specific utterances with custom slots, which allow for variable speech inputs in an utterance (Table 2).

Table 2. Alexa Slots.
Full size table

Each of these intents has many possible associated commands that Alexa will mark as valid. In the current prototype, users have the ability to change the level (intensity) and color of groups of fixtures or individual fixtures (channels). Supported colors include red, orange, yellow, green, blue, purple, pink, and white.

  1. I.

    To change a level, a user would say: [Group/Channel] [#] at [level #]. For Example:

    • “Group [thirty] at [fifty]”

    • “Group [thirty] at [half]”

    • “Channel [twenty-two] at [one hundred]”

    • “Channel [twenty-two] at [full]”

    • “Set group [one hundred] to [forty three]”

  2. II.

    To change a color, a user would say: [Group/Channel] [#] to [color]. For Example:

    • “Group [ten] [blue]”

    • “Channel [twelve] [green]”

    • “Change group [ten] to [red]”

    • “Set channel [ten] to [yellow]”

    • “Let’s make channel [thirty one] [purple]”

3.5 Future Developments

These are the planned improvements for the KeyLight Trainer:

  • The ability to save stage looks as cues and replay (or load) cues into the scene.

  • The ability to ask Alexa to change the set or scenery.

  • The ability to choose between different light plots (configurations of lighting fixtures).

  • The ability for users to view the current state on an interface within the trainer.

  • The ability for users to select additional colors and also make adjustments to those colors.

  • The ability for users to change the numbering of channels or groups.

  • Further support for transitions and fades.

  • Better natural language recognition for communication not relating to commands.

  • The use of realistic types of fixtures.

  • Expanded dialogue options for the board operator and Alexa.

  • Adding additional animations to the model representing Alexa.

  • Expanding terminology to match that used outside of the western United States.

  • Improvements to visual displays, including communication of state of the fixtures.

4 Results

4.1 Summary of Findings

Four theater industry professionals were presented with a demonstration of the KeyLight Trainer. Those interviewed included Participant 1) a senior product manager at a professional stage lighting company, Participant 2) a Theater Manager for a school district, Participant 3) the Technical Director of Performing Arts at a high school, and Participant 4) a lighting designer.

In a separate demo with Participant 5, who was not formally interviewed and who had no prior lighting experience, testimony indicated that it is particularly important that users have some background knowledge about lighting design and terminology. Participant 5 communicated that they felt uncertainty about how to use the software and a lack of clarity on what might make a design “good”. These results focus on the testimony of the four industry professionals, who are closer to the intended audience of this tool (Fig. 7).

Fig. 7.
figure 7

A participant using the Keylight Trainer with a Vive headset.

Full size image

Of the participants who saw KeyLight in use, 100% thought KeyLight Trainer was a useful tool for lighting designers. All also thought the KeyLight Trainer had novel value. “I think the thing that’s different here, is that you’re not building the look. You’re asking someone else to build the look, and you have to be able to communicate that,” said Participant 1 [11]. Participant 3 called KeyLight the “missing link” for students looking to cross over from classroom education to hands-on experience [5]. Participant 3 also noted that the KeyLight project would be beneficial to both professors and students who are often unable to use professional visualization software due to its cost prohibitive nature.

Mentioning the many visualizers available in the market, those interviewed instead reported the success of KeyLight as a communication trainer. “The communication part is something that we don’t have, that none of these things have.... The other stuff is already out there. And having a way to talk to a visualizer isn’t really that exciting. You already have the visualizer. A way to just learn to talk to a programmer, yeah, that’s different,” said Participant 2 [11].

All four further discussed the importance of communication skills as a lighting designer, and that the VUI implemented in this software would help designers build upon those skills. “To be able to translate what’s in your head into the stage when you’re limited by time... when you’re trying to explain it to another entity, that means your thoughts have to be organized in such a way that you can communicate them,” said Participant 1 [11].

Best applications of the model for this trainer were also discussed. All four participants brought up the different patterns of language and commands used by lighting designers across different lighting boards. There was consensus that the program, if tailored specifically to the language of each board, would provide immense value for learning not only how to communicate but also how to work with specific lighting technologies. They suggested that this would make the trainer more useful for professionals as well. Discussing the generality of the magic sheet and its associated commands, Participant 2 stated, “I like the idea of that in that it’s realistic and yet generic. It’s not platform specific, it’s not Eos but it follows the reality of how those types of software work close enough that you’re teaching the concept without getting bogged down” [11]. All four expressed that they liked that the commands programmed in the KeyLight Trainer were more general for student use.

All four also reported a lack of interest in the VR component of KeyLight. While all four had either used or encountered lighting visualizers before, none of them reported using the VR features for any such product.

Only one of the four participants thought that KeyLight was close to bringing value to students and educators in its current state. This participant, a high school teacher and a department head of the theater program at his school, stated that the lights in the virtual space needed to be focused better (reducing lights bleeding unrealistically into areas of the stage) before he would use it with students. This minor improvement is the only feature that he felt impeded effective use of the product, and he stated that he would use the trainer with his theater students if that were to be fixed. The other three participants felt that some major features needed to be added to bring KeyLight into a state where it adds value to students. Describing the value of KeyLight as an educational tool in its current state, Participant 1 stated, “I think it has value. I think it’s value is a little bit limited... I think it’s cool” [11]. The changes they pointed out are discussed below.

4.2 Improvements Needed

All four participants reported that the trainer needs to include relative brightness to be useful as a teaching tool. This means that the VUI must allow users to communicate level changes such as “Bump up channel thirty by five points” or “Bring down group sixty a little.” They felt it was important that future versions also include the ability to change groups of lights according to their names, such as “Fronts at fifty.”

State visibility was another critical component that all four professionals felt the current program lacked. There was no way for a designer to check the current levels of any group or channel or to determine which were turned on. The second group of participants suggested that the trainer include another interface on the magic sheet with a visual display of active channels and their levels, as well as the ability to ask the Alexa board operator about these levels.

All four also mentioned color palettes. They expressed that it would be best if users could choose the color of their lights prior to using the tool, or otherwise have a larger selection of basic colors and variations in light warmth.

All four agreed that there should be multiple human targets on the digital stage, and that the ability to switch between set designs would add to the experience. Three expressed interest in the ability to pre-program a set design for potential students and then have the students work in that created space. One thought it was important that the humanoids on the stage also have eye sockets and slightly more realistic features so that students can learn more about the effect of different lighting angles on a subject. Three out of four liked the simplicity of the “cartoony” environment and models.

Debating what would be needed for KeyLight to provide value to theater students, all four also agreed that cues need to be implemented in KeyLight. Users must have the ability to save, replay, and transition between cues or looks on the stage. They agreed that once relative brightness, a display of the system state, cues, and the color palette were expanded, KeyLight would provide great value as an educational tool.

5 Discussion

While pointing out some critical improvements to KeyLight, each professional interviewed also acknowledged the value of KeyLight as both an educational and professional tool. In one testimonial, an interviewee speaking to the immediate application of KeyLight said, “You could go into the theater that’s in your [KeyLight] visualizer for the last week of your class and light things using the plot, using the everything. That would be helpful” [11]. Speaking to future applications of the VUI for professionals, another said, “If you had something that would let me voice control a console out there, I would buy it in 30 s. In my world I spend a lot of time at a lighting desk, and I don’t have operators that are really very good. I end up having to write my own cues mostly. If I could just sit there and talk into it while I’m watching the stage, suddenly I’m twice as fast. I think there’s a product there possibly quite powerful and useful. It’s not educational. It’s for people who are in smaller venues where your lighting designer has to write cues... I’ve wanted that for a long time” [11]. In its current state, those interviewed see KeyLight as being a powerful educational tool with future possibilities for professional expansion.

By using a VUI with industry standard language, KeyLight bypasses some challenges posited by voice user interfaces [8, 10] by enforcing a model that is both naturally learnable and discoverable for users with training in lighting design. KeyLight also goes beyond this model by requiring users to speak in patterns that reflect those required in their profession. Other research on VUI’s has focused on the reliability of a mental model that users build for the program [3], and by focusing on a niche audience KeyLight evades some of the challenges users face while forming one.

Personification of Alexa as the board operator, another aspect of VUI’s explored by other researchers [6, 9], helps to ground the voice technology and add a sense of realism to this simulation according to those interviewed.

With budget, time constraints, or public health limitations preventing aspiring theater designers from practicing their craft, KeyLight solves access problems in theater education. “Most high schools don’t have visualizers,” said Participant 2. “Colleges are going to have them but they’re probably going to be restricted access as to who gets to do it when. So the idea that you’ve got something simple built into [KeyLight] is really appealing because anyone can use it and they can see, even if it isn’t super fancy visualization, there’s something to see that’s there and its part of the system” [11]. KeyLight fills a rift in theater education. Many students lack access to the software, spaces, or equipment that will allow them to develop their professional communication skills.

As Keylight continues into later phases, more rigorous evaluation of its efficacy will be needed. After the improvements discussed in this paper are implemented, the intention is to compare both the qualitative and quantitative impact of Keylight versus other lighting software and tooling on junior lighting designers learning lighting through the classroom.

6 Conclusion

KeyLight moves beyond the functionality of similar software in several important ways. As a training program for lighting designers that leverages voice control, KeyLight Trainer helps designers gain experience with an actual method for creating lighting looks, using industry standard terminology and commands. An Alexa voice assistant sits beside the user and takes on the role of a board operator, adding to the immersion of KeyLight Trainer. Finally, while this program uses virtual reality to help designers get comfortable with giving commands in a simulated space, it can also run on any computer without VR equipment.

With KeyLight, junior lighting designers can strengthen their fundamental skills. Users can learn to be more confident communicators as they increase the hours of experience they have designing for the stage. While future expansions to KeyLight have the potential to prepare the trainer for professional use, testimony suggests that this program will increase the comfort, confidence, and competency of both traditional theater students and people entering the industry through new, nontraditional paths.