Clare, A. S.; Macbeth, J. C,; Cummings, M. L.
Advances in autonomy have made it possible to invert the typical operator-to-unmanned vehicle ratio so that a single operator can now control multiple heterogeneous Unmanned Vehicles (UVs). Real-time scheduling and task assignment for multiple UVs in uncertain environments will require the computational ability of optimization algorithms combined with the judgment and adaptability of human supervisors through mixed-initiative systems. The goal of this paper is to analyze the interactions between operators and scheduling algorithms in two human- in-the-loop multiple UV control experiments. The impact of real-time operator modifications to the objective function of an optimization algorithm for multi-UV scheduling is described....
Cummings, M. L.; Tappan, J.; Mikkelsen, C.
d observations, among other techniques. Given the time and resources required, we examine how to generalize a work domain analysis technique, namely the hybrid Cognitive Task Analysis (hCTA) method across two domains in order to generate a common set of display information requirements. The two domains of interest are field workers troubleshooting low voltage distribution networks and telecommunication problems. Results show that there is a high degree of similarity between the two domains due to their service call nature, particularly in tasking and decision-making. While the primary differences were due to communication protocols and equipment requirements, the basic overall mission...
Gao, F.; Cummings, M. L.; Bertuccelli, L. F.
Simultaneously controlling increasing numbers of robots requires multiple operators working together as a team. Helping operators allocate attention among different robots and determining how to construct the human-robot team to promote performance and reduce workload are critical questions that must be answered in these settings. To this end, we investigated the effect of team structure and search guidance on operators’ performance,
subjective workload, work processes and communication. To investigate team structure in an urban search and rescue setting, we compared a pooled condition, in which team members shared control of 24 robots, with a sector condition, in which each team member...
Macbeth, J. C.; Cummings, M. L.; Bertuccelli, L. F.; Surana, A.
While there is currently significant interest in developing Unmanned Aerial Systems (UASs) that can be supervised by a single operator, the majority of these systems focus on Intelligence, Surveillance, and Reconnaissance
(ISR) domains. One domain that has received significantly less attention is the use of multiple UASs to insert or extract supplies or people. To this end, MAVIES (Multi-Autonomous Vehicle Insertion-Extraction System) was developed to allow a single operator the ability to supervise a primary cargo Unmanned Aerial Vehicle (UAV) along with multiple scouting UAVs. This paper will detail the development of the design requirements generated through a Hybrid Cognitive Task...
Stimpson, A. J.; Buinhas, L. S.; Bezek, S.; Boussemart, Y.; Cummings, M. L.
Procedures play an important role in domains where humans interact with critical, complex systems. In such environments, the operator’s ability to correctly follow a given set of procedures can directly impact system safety. A quantitative measure of procedural adherence during training for complex system opera-tion would be useful to assess trainee performance and evaluate a training program. This paper presents a novel model-based objective metric for quantifying procedural adherence in training. This metric is sensi-tive to both the number and nature of procedural deviations, and can be used with cluster analysis to classi-fy trainee performance based on adherence. The metric...
Thornburg, K. M.; Peterse, H.P.M.; Liu, A.M.
Long duration, low task load environments are typical for nuclear power plant control rooms, where operators, after hours of operating under a low task load situation, may have to shift to a high task load situation. The effects of time-on-task and boredom due to low task load will be an important consideration for the design of new nuclear power plant control rooms, which will rely more heavily on automation. This paper describes a research study of performance in a simulated nuclear control room environment, where
36 participants responded to an alarm during a 4 hour long experiment where the alarm onset...
Mkrtchyan, A. A.; Macbeth, J. C.; Solovey, E. T.; Ryan, J. C.; Cummings, M. L.
A low task load, long duration experiment was conducted to evaluate the impact of cyclical attention switching strategies on operator performance in supervisory domains. The impetus for such a study stems from the lack of prior work to improve human-system performance in low task load supervisory domains through the use of design interventions. In this study, a design intervention in the form of auditory alerts is introduced and the effects of the alerts are examined. The test bed consists of a video game-like simulation environment, which allows a single opera-tor the ability to supervise multiple unmanned vehicles. Each participant in...
Gao, F.; Cummings, M. L.
With the increasing need for teams of operators in controlling multiple robots, it is important to understand how to construct the team and support team processes. While running experiments can be time consuming and expensive, the use of simulation models is an alternative method. In this
study, we built a discrete event simulation model that represents multi-robot multi-operator teamwork. Preliminary results show that the model can generate performance measures consistent with experimental results.
Teamwork is important when humans work together with automated agents to perform tasks requiring monitoring, coordination, and complex decision making. While human-agent teams can bring many benefits such as higher productivity, adaptability and creativity, they may also fail for various reasons. It is important to understand the tradeoffs in teamwork. The purpose of this research is to investigate the process and outcomes of human-agent teamwork by running experiments and building quantitative simulation
models. Preliminary results are discussed as well as future directions. We expect this research to deepen the under-standing of human-agent teamwork and provide recommendations for the design of teams...
Gao, F.; Clare, A. S.; Macbeth, J. C.; Cummings, M. L.
We developed a system dynamics model to simulate the impact of operator trust on performance in multiple robot control. Analysis of a simulated urban search and rescue experiment showed that operators decided to manually control the robots when they lost trust in the autonomous planner that was directing the robots. Operators who rarely used manual control performed the worst. However, the operators who most frequently used manual control reported higher workload and did not perform any better than operators with
moderate manual control usage. Based on these findings, we implemented a model where trust and performance form a feedback loop, in...
Berardi, C.; Solovey, E. T.; Cummings, M. L.
There is an increasing requirement for advanced analytical methodologies to help military intelligence analysts cope with the growing amount of data they are saturated with on a daily basis. Specifically, within the context of terror network analysis, one of the largest problems is the transformation of raw tabular data into a visualization that is easily and effectively exploited by intelligence analysts. Currently, the primary method within the intelligence domain is the node-link visualization, which encodes data sets by depicting
the ties between nodes as lines between objects in a plane. This method, although useful, has limitations when the size and complexity...
Ryan, J. C.; Banerjee, A. G.; Cummings, M. L.; Roy, N.
Planning operations across a number of domains can be considered as resource allocation problems with timing constraints. An unexplored instance of such a problem domain is the aircraft carrier flight deck, where, in current operations, replanning is done without the aid of any computerized decision
support. Rather, veteran operators employ a set of experience based
heuristics to quickly generate new operating schedules. These expert user heuristics are neither codified nor evaluated by the United States Navy; they have grown solely from the convergent experiences of supervisory staff. As unmanned aerial vehicles (UAVs) are introduced in the aircraft carrier domain,
these heuristics may require...
Cummings, M. L.
In the recent development of a human-in-the-loop simulation test bed designed to examine human performance issues for supervisory control of the Navy’s new Tactical Tomahawk missile, measurements of operator situation awareness (SA) and workload through secondary tasking were taken through an embedded instant messaging program. Instant message interfaces (otherwise known as “chat”), already a means of communication between Navy ships, allow researchers to query users in real-time in a
natural, ecologic setting, and thus provide more realistic and unobtrusive measurements. However, in the course of this testing, results revealed that some subjects fixated on the real-time instant messaging secondary task instead...
Cummings, M. L.; Tsonis, C.
Cognitive complexity is a term that appears frequently in air traffic control (ATC) research literature, yet there is little principled investigation of the potential sources of cognitive complexity. Three distinctly different sources of
cognitive complexity are proposed which are environmental, organizational, and display. Two experiments were conducted to explore whether or not these proposed components of complexity could be effectively partitioned,
measured, and compared. The findings demonstrate that sources of complexity can be decomposed and measured and furthermore, the use of color in displays, a display design intervention meant to reduce environmental complexity, can actually contribute to it.
Cummings, M. L.
When the human element is introduced into decision support system design, entirely new layers of social and ethical issues emerge but are not always recognized as such. This paper discusses those ethical and social impact issues specific to decision support systems and highlights areas that interface designers should consider during design with an emphasis on military applications. Because of the inherent complexity of socio-technical
systems, decision support systems are particularly vulnerable to certain potential ethical pitfalls that encompass automation and accountability issues. If computer systems diminish a user’s sense of moral agency and responsibility, an erosion of accountability could result. In...
Cummings, M. L.; Mitchell, P. J.
The application of network centric operations to time-constrained command and control environments
will mean that human operators will be increasingly responsible for multiple simultaneous supervisory
control tasks. One such futuristic application will be the control of multiple unmanned aerial vehicles
(UAVs) by a single operator. To achieve such performance in complex, time critical, and high risk
settings, automated systems will be required both to guarantee rapid system response as well as
manageable workload for operators. Through the development of a simulation test bed for human
supervisory control of multiple independent UAVs by a single operator, this paper presents recent
efforts to investigate workload mitigation strategies as a...
Cummings, M. L.; Kilgore, R. M.; Wang, E.; Tijerina, L.; Kochhar, D. S.
Objective: To explore how a single master alarm system affects drivers’ responses when compared to multiple, distinct warnings. Background: Advanced driver warning systems are intended to improve safety, yet inappropriate integration may increase the complexity of driving, especially in high workload situations. This study investigated the effects of auditory alarm scheme, reliability, and collision event-type on driver performance. Method: A 2x2x4 mixed factorial design investigated the impact of two alarm schemes (master vs. individual) and two levels of alarm reliability (high and low) on distracted drivers’ performance across four collision event-types (frontal collision warnings, left and right lane departure warnings,...
Cummings, M. L.; Brzezinski, A. S.; Lee, J. D.
There is increasing interest in designing systems such that the current many-to-one ratio of operators to unmanned vehicles (UVs) can be inverted. Instead of lower-level tasks performed by today’s UV teams, the sole operator would focus on high-level supervisory control tasks. A key challenge in the design of such single-operator systems will be the need to minimize periods of excessive workload that arise when critical tasks for several UVs occur simultaneously. Thus some kind of decision support is needed that facilitates an operator’s ability to evaluate different action alternatives for managing a multiple UV mission schedule in real-time. This paper...
Crandall, J. W.; Cummings, M. L.
In recent years, much research has focused on making possible single operator control of multiple robots. In these high workload situations, many questions arise including how many robots should be in the team, which autonomy levels should they employ, and when should these autonomy levels
change? To answer these questions, sets of metric classes should be identified that capture these aspects of the human-robot team. Such a set of metric classes should have three properties. First, it should contain the key performance parameters of the system. Second, it should identify the limitations of the agents in the system. Third, it should...
Cummings, M. L.; Bruni, S.; Mercier, S.; Mitchell, P. J.
In light of the Office of the Secretary Defense’s Roadmap for unmanned aircraft systems (UASs), there is a critical need for research examining human interaction with heterogeneous unmanned vehicles. The OSD Roadmap clearly delineates the need to investigate the “appropriate conditions and requirements under which a single pilot would be allowed to control multiple airborne UA (unmanned aircraft) simultaneously.” Toward this
end, in this paper, we provide a meta-analysis of research studies across unmanned aerial and ground vehicle domains that investigated single operator control of multiple vehicles. As a result, a hierarchical control model for single operator control of multiple unmanned...