ABSTRACT

DoD has a requirement for C4I systems that are mobile, collaborative, dispersed, interactive and real-time.  Advanced electronic communications is the key to meeting these requirements, but the current generation of secure, digital, military radios has limited bandwidth and capacity.  The practical constraints on our communication networks require multimedia visualization and collaborative work environment based on hybrid communication technologies and distributed databases. 

The Virtual Command Post (VCP) is an experiment in the use of distributed simulation and virtual conferencing technologies to provide a “you are there” collaborative C4I environment. An initial version of the VCP was developed by Lionhearth Technologies for the US Army Communications and Electronics Command (CECOM) under a Phase II Small Business Innovative Research (SBIR) contract [1][i].  Commanders and war fighters in dispersed locations (See Figure 1) can use the VCP to view battlefield information on portable or man wearable devices including laptops, personal digital assistants (PDAs), or head mounted displays (HMDs).  The VCP captures each commander’s “presence” in the form of compressed voice, line-of-sight, and body position and can transmit his/her voice and animated iconic representation over a 4800 baud radio network, leaving bandwidth available for maintenance of a distributed battlefield management database.  This database is updated locally and converted at each node into 3D-battlefield visualization with interactive planning, "what-if", and data drill-down capabilities.  In parallel, satellite broadcasts or video-teleconferences can be captured and displayed in the VCP on video walls. A user interface shell is available for generic, collaborative white board interactions.  When refined and coupled into the appropriate C4I infrastructure, the result will be a complete, interactive, portable, command post system for battle planning, mission briefing, and tactical monitoring applications.

This paper discusses the current prototype, our planned future enhancements, and the lessons learned from the current implementation. 

PROJECT OBJECTIVES

The goals for the Virtual Command Post project were defined as follows:

·        Construct a demonstration system          

·        Demonstrate and evaluate the VCP System concept      

·        Build a compelling battlefield model

·        Explore the adequacy of the system architecture 

·        Explore system sensitivity to critical technical components          

The completed Phase II effort accomplished all of the objectives along with the delivery of a two-node Virtual Command Post prototype to CECOM.  The overall result of the VCP effort was the development of a system architecture and the implementation of the Virtual Command Post system.  This system is capable of capturing and transmitting voice, body language and movements, providing operator control of the environment and battle management data exchange between conferencing nodes using a simulated single 9,600 BPS SINCGARS radio net.  At each node, a collaborating commander views a virtual reality simulation of a command center, including 3D models of each participant in the conference.  Each participant’s face is rendered in multiple polygons, and may be made realistic by wrapping the polygons in a texture map taken from an actual photo.  As a future option, during the conference, facial expressions for the active speaker may be generated by digitally processing real-time images of the speaker’s face at the transmitting node to extract key parameters of facial expression changes. These parameters are then transmitted using approximately 1/4 of the available 9600 BPS bandwidth, and converted back to texture maps at the receiving nodes, after which the maps are used to animate the speaker’s face in real-time.  An illustration of the Virtual Command Post environment created by the system at each node and made visible to the local commander is shown in Figure 2.

Each transmitting commander’s body language and gestures are also captured by motion capture devices and optional glove peripherals, converted to a set of gestures, and transmitted as a code using an additional 600 BPS. These codes are used to render the speaker’s 3D-body model, or avatar at the receiving node, and also to select and manipulate objects within the virtual environments. The system components necessary for the implementation of a VCP node are shown in Figure 3.

A transmitting speaker’s voice is captured by a microphone, compressed to 4800 BPS, and combined with all other conference data before transmission to the other conferences. It was observed that CODECs operating at 1200 BPS in real-time may provide minimum latency.  With the imminent release of COTS software, incoming voice can be spatialized in real time to create aural localization information consistent with the transmitted speaker’s position within the receiving conference’s virtual command post at each receiving VCP node. In addition, voice recognition may be added as required.

To use the resulting system for battle planning and management, each virtual environment includes virtual video walls to display incoming satellite video or computer applications running at that node’s site.  Any application could potentially be viewed or controlled by the user through a combination of gesturing, voice commands, writing on a pen tablet, and/or using a keyboard.  Capturing applications’ outputs into memory as texture maps and rendering the texture maps in real time within the conference simulation generates the images on each of the video walls.

In addition to providing an environment in which to use a wide range of conventional computer applications, commanders are also provided with an interactive 3D-battlefield map decision tool generated and manipulated in real-time.  Proposed future versions of the battlefield map will be linked to a distributed fact base (DFB) of op­erational data describing battle plans and status. This database will be generated at each node by software known as Information Distri­bution Technology (IDT)[2][ii].  The IDT system defines a terse protocol for distributing all relevant facts about a planned or actual battle to operational units, using a rule-based approach that aggregates information at each command level to reduce bandwidth and data volume requirements. This system is designed to operate using 1200 BPS, leaving adequate bandwidth for conferencing.

Figure 4: VCP COTS Infrastructure

In order to provide a true analog to the template-based military decision-making process that is central to battle planning and management doctrine, the 3D battlefield map provides a backdrop for 3D virtual templates which effectively combine proven and familiar planning processes with immersive visualization capability and a distributed information data­base.

In the future, users will be capable of interfacing with the VCP in one of three ways: 1.) In stationary command post installations, users will be immersed within a projection display providing 180-degree horizontal field-of-view, and will wear Liquid Crystal Display (LCD) shutter glasses to create stereoscopy.  A small, high-resolution auxiliary display with a pen input overlay is provided for high-resolution application control; 2.) In vehicular installations, the projection display will be replaced by a wall-mounted display 3.) For dismounted use, a man-portable interface will be provided using a head-mounted display and a high-bandwidth link to the vehicle-based system computer.

The VCP Demonstration System

he demonstration system consists of two VCP nodes or stations connected by a 9600 baud serial line or 14.4 modems to emulate the SINCGARS radios.  Optionally, nodes may be connected via Ethernet for higher bandwidth communications. In order to demonstrate both a stationary/vehicular configuration and a man-portable configuration of the VCP for each combination, each station was equipped with an HMD device and high-resolution video display.

The demonstration Virtual Command Post was constructed using Commercial-Off-The-Shelf (COTS) components.  The hardware system was composed of Pentium Pro based computers equipped with high performance 3-D graphics accelerator boards.  The software system was developed on top of Microsoft Windows and the Microsoft Direct X library, enabling maximum performance for real time 3-D graphical animation.  3-D Scenes and modeling was implemented with the Microsoft Softimage 3D modeling tools, running on the Windows NT system. (See Figure 4.)

The VCP System Concept

The original and final System Concepts are shown in Figure 5.  The Virtual Command Post will be useful commanders, staff, and their subordinates for dispersed collaboration in the process of gathering battlefield information, planning, and decision making.  This final system was be used to evaluate this system concept.

Figure 6: VCP Sand Table and 3D Planning Area

Before the utility of the VCP system concept could be assessed, a man-machine interface needed to be designed that would allow users to perform specific collaborative tasks within the VCP.  The mission-critical functions necessary for assessing the overall system usefulness were prototyped to suggest the capabilities that a completely developed implementation might provide.  Commanders and developers were then invited to initiate virtual command center briefings, display 3D battlefield maps, physically manipulate virtual armaments, view video walls, interact with other battlefield commanders at remote locations and use whiteboard technologies for virtual conferencing. Evaluation feedback resulted in further modifications and enhancements of the VCP.

To be fully useful for battlefield collaboration as shown in Figure 6, the VCP demonstration system must have the proper “look and feel” of a battlefield command post.  Commanders engaged in tactical warfare will not use equipment that seems toy-like or unrealistic.  Professional graphic designers carefully storyboarded the VCP system, modeling both the interior of the command and the appearance of commander’s avatars.  Army Commanders and knowledgeable developers were solicited to aid in the design and selection of the environment and models. Texture-mapped graphics were designed to facilitate a realistic, collaborative environment acceptable to commanders involved in battlefield management activities.

A Compelling Battlefield Model

Communication of the Commander’s “Vision of the Battlefield” was one of the most important concepts this program addressed.  The VCP constructed in this Phase II effort has a working interface to a terrain database model.  This allowed the design team to construct a three dimensional model of the battlefield within the command post environment.

Once commanders experience the ability to visualize the battlefield model and to collaborate with their staff on evaluation of threats, terrain, weather, and battlefield conditions, it is anticipated that additional complex capabilities and requirements will be requested.

The following interactions were chosen and implemented because they clearly demonstrate the usefulness of the battlefield model.

·        Display the Battlefield Model

·        Move Around and Zoom Into the Battlefield Model

·        Provide Physical Manipulation of Individual Armaments and Units

·        Provide Manipulation of Planning Templates

·        Allow first-person, virtual walk-through of the Battlefield Model

Adequacy of the System Architecture

An Architecture Control Document was developed and maintained throughout the development of the VCP.  Past experience teaches that modular, well-defined subsystems are necessary both for the management and construction of complex systems.  The software architecture proposed in Phase I report defines a system with well-defined modules linked in a hierarchical manner.  In order to preserve the modularity of the architecture throughout the design process, an architecture control discipline was used.  Nevertheless, it is likely that some portions of the system hardware or software architecture will require improvements prior to Phase III deployment.  Construction of the Phase II prototype has allowed the assessment of the strengths and weakness of the first design and will allow the enhancement and refinement of the design in the next generation system.

Sensitivity to Technical Components

Initially, the degree of system performance sensitivity to critical technical components was difficult to evaluate without development of the demonstration system.  Because this system comprised such a large step forward in VR system capabilities, it was difficult to “see” how the performance of each component would effect the performance of the system prior to prototype development. 

Our design approach was to concentrate on the behavior of the overall system and to select components for each subsystem that could be made robust within the time allowed for the project.  Because we gave system level performance high precedence, performance of each technical component was secondary. Construction of the prototype system demonstrates where future refinement or improvement of any subsystem is needed to achieve improved system performance and cost-effectiveness.

As a whole, the system performance is excellent.  This is especially true in light of the usage of standard personal computers and relatively low-cost graphics accelerators.  The system performance has met all of the program objectives.  Due to a careful selection of system components and software optimization, each subsystem runs at performance levels meeting or exceeding what would be expected of such an application running on a dedicated platform.

A proposed future version of the system will focus on the following:

·           Improved component performance (5X) and ease of use

·           Improved motion capture devices

·           Improved Demonstration of the VCP Concept- Collaboration, Visualization and Communications

·           Improved story board and integration with existing C4I structures

·           Higher resolution graphics display devices

SUMMARY

Evaluation feedback confirmed the value-added of a VCP environment as an integral part of a multi-modal Human Computer interface environment.  Overall, the VCP has been well received.  The results of the SBIR effort have demonstrated an effective immersive work environment for performance of collaborative planning and management tasks.  Several specific advances were accomplished during this effort as described in the following sections.

Demonstration of Virtual Conferencing

This project is probably one of the most significant demonstrations of virtual conferencing to date.  Other experiments have demonstrated the idea, but every effort was made to improve production qualities and software stability within the VCP.[3][4][5]  The prototype delivered for this project is robust, fully functional and capable of demonstrating the virtual conference concept to its full potential at 100-1,000 times less communications bandwidth than required in earlier demonstrations.

Validation of the VCP Concept

The Phase II prototype provides a platform for validating the effectiveness of the Army’s dispersed command post concept. Commanders are able to interact with the 3D-battlefield model, view satellite weather data on virtual video walls, and collaborate with each other in the virtual command post environment. Through continued testing, the user will be able to compare and evaluate process performance, travel savings, and survivability improvements achievable to current co-located work environments.

Low Bandwidth Communications

Independent of the need for a virtual command post, the proposed effort has generated an architecture capable of providing virtual communications for other battlefield tasks.  Any soldier currently using radios may find a low-bandwidth, virtual communications capability providing data and multiparty collaboration useful. The audio and communications capabilities necessary for this task have been implemented in this demonstration system.

Advanced VR Audio Subsystem

The Army uses hundreds of VR systems in its modernized equipment and training simulators, and has indicated its need for advanced VR-capable audio subsystems.  The Audio Subsystem integrated under this effort provides sophisticated set of low-bandwidth digital audio capabilities. These capabilities are available to the Army for its simulator programs.

FUTURE ENHANCEMENTS

Current efforts are continuing in activities designed to improve the utility of the VCP both for Military Command and Control and for commercial applications.  One such effort is to develop an HLA based VCP that uses the DoD's new High Level Architecture to control a distributed simulation.  A diagram of this proposed system is illustrated in Figure 7 and 8. 

Additionally, progress is being made in developing commercial variants of the VCP for use in Internet Commerce [6] (Figure 9), distance learning (Figure 10), and tele-medicine. The capabilities of the VCP show that virtual conferencing may be at the forefront of the mobile computing field and should offer a favorable alternative to telecommuting, video-conferencing and other forms of dispersed collaboration. Once a user becomes familiar with the VCP, it becomes self-evident how virtual tele-presence will lead to immersive communication devices that will someday seem as fundamental and ubiquitous as the telephone and television.

These devices will provide environments equipped with virtual tools, displays, and information that will enable applications such as tele-commuting, remote training, entertainment, and distributed simulation. By systematically pursuing opportunities to develop the sensors and environments that enable early VCP dual use spin-off, VCP technology may very well find its first operational home as a commercial product.

ACKNOWLEDGMENTS

This work was supported by the Army SBIR Program and CECOM.  The authors would like to thank the many reviewers of the VCP prototype who have made valuable comments that helped to make it more relevant and useful.  In particular, Lisa Tran and Richard Wong provided invaluable assistance in installation and on-call demonstrations of the VCP, Jim Salton was responsible for initiating this topic and provided invaluable technical guidance throughout the first two phases of the project, and finally, Dr. Dirk Klose provided much encouragement and support in leveraging the technology in other programs.  The authors would also like to thank the entire Lionhearth team, including Sasha Sokolov and programmers, Douglas Swalen, and Jan and Andy Filo.

REFERENCES

[ Home ] [ Up ]

Copyright 2001-2005 RubyQuest Technologies
Last modified: 12/31/04