MATRIX Lab Workshop Focuses on Fielding Autonomous Systems

The University of Maryland (UMD) A. James Clark School of Engineering MATRIX Lab and the UMD Applied Research Laboratory for Intelligence and Security (ARLIS) are spearheading an effort to ensure that autonomous systems are safe and reliable for use in our everyday lives. As part of this effort, on February 4th, 2026, the MATRIX Lab, in coordination with ARLIS, hosted a workshop focusing on the current research gaps and workforce needs that must be fulfilled to promote the development and fielding of autonomous systems.

“Future autonomous systems will be defined by how we shape them. I am proud that my colleagues at the University of Maryland are leading this conversation and actively working to improve the field of autonomy,” said Dr. Reza Ghodssi, UMD Distinguished University Professor and MATRIX Lab Executive Director of Research and Innovation. “Events like these keep the University at the forefront of impactful research and education.”

The Development, Test, Evaluation, Verification & Validation (DTEVV) Workshop identified research and workforce gaps hindering the ability to field autonomous systems safely and efficiently. The workshop was organized by Dr. Donald “Bucket” Costello, the Director of Test and Evaluation of Autonomous Systems at the MATRIX Lab, and Dr. Brian A. Weiss, a Research Engineer at ARLIS. Drs. Costello and Weiss focus on the development and execution of tests and evaluation (T&E) of technologies with intelligent/autonomous capabilities in their research.

“The MATRIX and ARLIS teams collectively bring together a wealth of expertise in the verification and validation of emergent, intelligent technologies across the military, law enforcement, first response, and commercial sectors. This personnel experience is complemented with MATRIX’s numerous state-of-the-art measurement facilities, particularly with small, unmanned aerial systems. Likewise, ARLIS’ capability to perform CLASSIFIED research, including technology prototype development and T&E, to protect and advance national security further backbones the value the team provides to the defense community,” notes Dr. Weiss.

The DTEVV process is becoming increasingly crucial as technologies advance, especially to the Department of War (DoW). The U.S. Navy is working toward dramatically increasing the number of unmanned systems in its inventory to help ensure the organization is able to move quickly when needed. Before the systems are fielded, however, they must be rigorously evaluated for safety, reliability, and effectiveness. Current standards and methods of compliance used for crewed systems do not always translate to autonomous systems when a human is not in or on the loop. This is especially troubling when unexpected conditions present themselves. New processes must be created, then taught to the emerging workforce. The DTEVV Workshop gathered experts across industry, government, and academia to help identify how to make that happen.

“The future of transportation is uncrewed and ultimately autonomous. This goes for commercial and military vehicles. However, taking a human out of the loop poses new risks – ones that current DTEVV practices can’t address,” Dr. Costello said. “We thank all workshop participants – the speakers, panelists, and attendees – for contributing their ideas for solving this crucial issue.”

Risk and Trust in DTEVV

The first half of the DTEVV workshop focused on open research gaps that need to be filled to enable the fielding of autonomous systems. The second half focused on the needs of the workforce to do the same.

In his morning keynote presentation, Dr. Kamel "Kam" Saidi, Program Manager and Group Leader of the Intelligent Systems Division at the National Institute of Standards and Technology (NIST), focused on T&E. He pointed out that systems may test well in controlled environments, but that does not mean they will succeed in unpredictable, real-world settings.

The workshop’s morning panel included experts from the DoW, NAWCAD, and NIST. Much of the conversation focused on how much testing is enough testing. The panelists agreed that risk will always be there, but how much risk will be accepted must be defined, and we need to have methods that we can use to define them as technology evolves.

Following the morning keynote and panel, the 100+ workshop attendees engaged on an interactive activity, generating roadmaps to fill research gaps. Some of the proposals included, but were not limited to: infrastructure needs for the T&E community to evaluate new technologies, lack of metrics and methods of compliance for the evaluation of systems without interaction, and lack of universally agreed upon definitions for commonly used terms such as autonomy and AI.

During a break in the program, Dr. Costello outlined his efforts to close DTEVV research and education gaps, which include a DTEVV education program that provides graduate-level education on how to evaluate autonomy using an engineering-based risk management process. The first graduate certificates of this program will be given out in 2027.

The afternoon keynote speaker represented NAWCAD, the Navy's largest warfare center and busiest flight test facility. Executive Director Stephen Cricchi stressed that the workforce is more important than technology, saying it’s about the people. He recognized that autonomy requires deep technical knowledge, and academic partnerships are critical to giving people the knowledge they need to succeed.

The afternoon panel featured experts from industry, multiple government agencies, and a retired DoW Senior Executive Service official whose focus had been workforce development. Panelists identified the expected workforce skills that will be needed to develop, implement, and assess the technologies of tomorrow.

Following the afternoon keynote and panel, the attendees once again had a chance to develop roadmaps. This time, the focus was on what skillsets the current and future workforce need to properly assess the risk associated with evaluating autonomous systems. Some of the proposals included but were not limited to: defining the formal education needs of the current and future workforce that could be filled by a graduate education program at UMD, defining the skillsets that the workforce could obtain through short courses versus formal education, and defining the skillsets that are lacking under the current workforce training paradigm.

What’s Ahead

The MATRIX/ARLIS team is currently analyzing all of the roadmaps created by workshop attendees and will use those ideas to create recommendations for improving DTEVV research and education. These recommendations will be included in a report summarizing the workshop that will be distributed to thought leaders and changemakers across the country. Several follow-up events scheduled for 2026 and 2027 will check on progress and keep momentum strong. Join our mailing list to be notified of future events.

This event was made possible by a grant from the Office of Naval Research and support from the University of Maryland A. James Clark School of Engineering and the Clark School of Engineering Institute for Systems Research.

Published February 12, 2026