Advancing AI-Automated Tiered Reporting

PI: Helen Armstrong.

Master of Graphic & Experience Design (MGXD) students recently collaborated with the NC State Laboratory of Analytic Sciences (LAS) to explore the potential of AI to assist intelligence reporters as they move through the tiered reporting process.

Background: “Tiered reporting is a structured method of disseminating intelligence or sensitive information, designed to tailor the message to different audiences based on their needs, expertise, and security levels. Its primary goal is to streamline communication while safeguarding critical sources, methods, and intelligence. By adjusting the detail, sanitization, and focus of reports, tiered reporting ensures that the right information reaches the right stakeholders at the right time.”

Working in groups, the design students dug into the current tiered reporting workflow, identified pain points, and then developed UX/UI concepts that seamlessly integrated AI to enhance the efficiency and effectiveness of intelligence reporting. The goal was to reduce the burden on analysts by providing a more intuitive, automated reporting system, ensuring that sensitive information was accurately sanitized and delivered across appropriate tiers without compromising security. Students focused on discovering a range of future-facing interface design possibilities for the novel integration of AI. 

The core research question of this project:

How might the design of an interface automate the tiered reporting process, so that reporters might more efficiently and knowledgeably team with AI to sanitize and deliver sensitive information across appropriate tiers?

Students were asked to consider:

    • OVERSIGHT:  How might this interface effectively communicate A.I. recommendations and data modifications (detail, sanitization, and focus) across multiple classification tiers as content moves through this reporting system, thus ensuring efficient and effective analyst oversight?
    • AGENCY: How might this interface support analyst agency and involvement throughout the tiered reporting process, thus encouraging analysts to remain active, deliberative participants?
    • TRUST CALIBRATION: How might this interface clearly convey the capabilities and limitations of the AI system to analysts of varying experience levels, thus assisting with trust calibration?

Prototypes

Designers: Willow Ahrens, Clara Matthews, William Whitley, Graphic & Experience Design, © NC State University, All Rights Reserved

Drawing from Jordan’s source database, ARC proactively suggests custom report sections. The platform provides a clear rationale for each section and a breakdown of the applicable tiers.
Using Jordan’s manually authored tier as a foundation, ARC can generate the remaining tiers automatically. ARC then steps back, giving Jordan full freedom to revise.
ARC’s language slider puts the tone of the report directly in Jordan’s hands. Jordan can fluidly adjust the writing style to fit his intent. For example, slider interactions can shift report language between formal and accessible, avoiding tedious rewrites.

Designers: Donté Coleman, Amaya Hush, Katie Kirke, Graphic & Experience Design, © NC State University, All Rights Reserved

Provides intelligence reporters with context on detected issues, including edits made, risk concerns, relevant policies, and helpful examples through a video format. Key chapters allow quick navigation, with the option to view key insights when time is limited.
A built-in AI feature that supports analysts during virtual meetings by listening in real time and bringing in relevant sources, policies, and reports based on the conversation.
To easily edit the custom agent, an adjustable slider controls the balance between user involvement and agent autonomy, ranging from the reporter monitoring autonomous activity to actively collaborating to plan, delegate, and execute tasks alongside the agent. Changing this level shapes the agent’s behavior, goals, and how it supports tasks.

Designers: Mia Biehler, Tushita Kaul, Sean Ran, Graphic & Experience Design, © NC State University, All Rights Reserved

Aisha can view a full node map of each report, showing its history of revisions and external decisions across tiers. The nodes also connect related reports and preserve AI-human interactions, giving Aisha full transparency into how decisions and edits were made.
When Aisha highlights content in one tier, the system automatically reflects corresponding changes across all opened tiers. She can also query the AI to compare relevant sections, and add annotations that route back to other analysts.
While reviewing a report, Aisha is able to pull up to five tiers beside each other to run comparisons with the AI.

Designers: Colton Hendrix, Diya Franklin, Swatik Salinera Parthasarathy, Graphic & Experience Design, © NC State University, All Rights Reserved

The Research Phase

Students completing exercises to understand tiered reporting