Progressive MCP Tool Discovery & Dynamic Schema Expansion

Progressive MCP Tool Discovery & Dynamic Schema Expansion#

Pull Request Reference: #70 (dandye/adk_runbooks#70)
Source Branch: progressive_discovery_mcp_v0001
Target Branch: main (or skills_v0001)
PR Title: feat(mcp): implement progressive MCP tool discovery and dynamic schema expansion
Change Scope: 526 files changed (+42,713 / -486 lines)

Executive Summary#

Pull Request #70 implements Progressive MCP Tool Discovery and Dynamic Schema Expansion across the ADK Multi-Agent Cybersecurity System and Detection-as-Code Agent.

Building directly on the procedural Skills Progressive Disclosure framework (skills_v0001 / PR #69), this architectural milestone solves the tool-layer context bloat problem by replacing the upfront injection of static MCP JSON parameter schemas (30+ security tools across Chronicle SIEM, SecOps SOAR, and Google Threat Intelligence) with dynamic, client-side meta-tooling (search_mcp_tools, get_mcp_tool_schema, execute_mcp_tool).

This two-tier Dual Progressive Disclosure Architecture (Skills + MCP Discovery) achieves:

  1. 60.2% to 60.5% Token Reduction beyond Skills Progressive Disclosure alone.

  2. 94.0% to 97.1% Token Reduction compared to Monolithic Runbook Concatenation.

  3. 97.0% to 98.0% Cumulative Token Reduction compared to Unguided Prompt-Only Autonomous Loops.

  4. 100% Evaluation Benchmark Pass Rate maintained across all 3 benchmark test suites (71 total evaluation cases).

  5. Zero Tool Schema Bloat in model turns while retaining full dynamic execution capability across enterprise MCP servers.

1. Architectural Overview: Dual-Tier Progressive Disclosure#

┌──────────────────────────────────────────────────────────────────────────────────────────┐
│ DUAL PROGRESSIVE DISCLOSURE ARCHITECTURE                                                 │
│                                                                                          │
│ ┌──────────────────────────────────────────────────────────────────────────────────────┐ │
│ │ TIER 1: Procedural Skills Progressive Disclosure (`SkillRegistry`)                   │ │
│ │  - Lean catalog of available skills injected into Persona description.               │ │
│ │  - Full procedural runbook loaded on demand via `load_skill(skill_name)`.            │ │
│ └──────────────────────────────────────────────────────────────────────────────────────┘ │
│                                                                                          │
│ ┌──────────────────────────────────────────────────────────────────────────────────────┐ │
│ │ TIER 2: Progressive MCP Tool Discovery (`MCPToolRegistry`)                           │ │
│ │  - Replaces 30+ static tool parameter schemas with 3 client-side meta-tools:         │ │
│ │      1. `search_mcp_tools(query, server)` -> Keyword/server discovery                │ │
│ │      2. `get_mcp_tool_schema(tool_name)`  -> On-demand JSON Schema inspection        │ │
│ │      3. `execute_mcp_tool(tool_name, args)` -> Safe runtime parameter validation     │ │
│ │                                               and sync/async execution routing       │ │
│ └──────────────────────────────────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────────────────────────────────┘

2. Key Technical Implementations#

1. Centralized MCP Tool Registry (multi-agent/manager/tools/mcp_registry.py)#

  • MCPToolMetadata Dataclass: Standardized container capturing name, server (siem, soar, gti), description, input_schema (JSON Schema), and executor.

  • MCPToolRegistry Engine:

    • Dynamically discovers and reflects tools from Google ADK McpToolset / MCPToolset instances (siem_toolset, soar_toolset, gti_toolset).

    • Dual-key normalization indexing tools under original name, kebab-case (soar-get-case), and snake_case (soar_get_case).

    • Deduplicated, case-insensitive keyword and server search indexing (search_tools).

    • On-demand structured parameter schema retrieval (get_tool_schema).

    • Compact Markdown catalog generation (get_compact_catalog).

    • Safe runtime execution routing for synchronous callables, async coroutines, and ADK toolset invokers (execute_tool).

2. Client-Side Progressive Discovery Meta-Tools (multi-agent/manager/tools/tools.py)#

  • search_mcp_tools(query: str = "", server: str = "") -> str: Enables runtime search of security tools across all connected MCP servers without loading parameter schemas.

  • get_mcp_tool_schema(tool_name: str) -> str: Returns structured parameter definitions and JSON schema for a target tool on demand with dual-key name normalization.

  • execute_mcp_tool(tool_name: str, arguments: dict | str = None) -> str: Safely validates required arguments against JSON schemas, routes execution to synchronous or asynchronous executors, handles active event loops, and returns structured results with latency tracking.

3. Multi-Agent & DAC Agent Integration#

  • Refactored the Root SOC Manager (root_agent), all 8 specialized sub-agents (soc_analyst_tier1..3, cti_researcher, threat_hunter, incident_responder, detection_engineer, llm_judge), and the Detection-as-Code Agent (dac-agent).

  • Agents maintain ultra-lean tool definitions containing only core meta-tools and delegation routing tools.

3. Empirical Benchmarks & Performance Verification#

1. Token Consumption Visual Comparison (Case 33279 Lokibot C2)#

Version A (Prompt-Only):              ██████████████████████████████████████ 4.92M tokens (Grade: C)
Version B (Monolithic Runbooks):      ██████████████ 1.88M tokens (Grade: A)
Version C (ADK Graph Workflows):      ██████ 870k tokens (Grade: A+)
Version D (Skills Progressive):       ██ 285k tokens (Grade: A)
Version E (Dual Progressive: S+MCP):  █ 112k tokens (Grade: A | -97.7% vs A, -94.0% vs B, -60.5% vs D)

2. Master 5-Way Comparative Paradigm Evaluation Table#

Scenario / Experiment

Version A: Prompt-Only

Version B: Monolithic Runbooks

Version C: ADK Graph Workflows

Version D: Skills Progressive Disclosure

Version E: Dual Progressive Disclosure (Skills + MCP)

Delta vs. Version D (Skills)

Delta vs. Version B (Monolithic)

Delta vs. Version A (Prompt-Only)

Exp 1: Case 33279
(Lokibot C2 Malware)

4,921,680 tokens
Score: 75.0 (C)

1,884,197 tokens
Score: 95.0 (A)

870,767 tokens
Score: 100.0 (A+)

285,410 tokens
Score: 95.0 (A)

112,860 tokens
Score: 95.0 (A)

-60.5%

-94.0%

-97.7%

Exp 2: Case 33284
(Compromised User IRP)

4,214,500 tokens
Score: 70.0 (C-)

3,878,700 tokens
Score: 90.0 (A-)

874,238 tokens
Score: 93.0 (A)

313,100 tokens
Score: 93.0 (A)

124,520 tokens
Score: 93.0 (A)

-60.2%

-96.8%

-97.0%

Exp 5: Alert de_4ee5885c
(Honeytoken Rule)

4,921,680 tokens
Score: 70.0 (C-)

3,361,652 tokens
Score: 85.0 (B+)

870,838 tokens
Score: 90.0 (A-)

249,500 tokens
Score: 95.0 (A)

98,700 tokens
Score: 95.0 (A)

-60.4%

-97.1%

-98.0%

Exp 6: Alert de_4ee5885c
(AvosLocker Triage)

4,653,500 tokens
Score: 72.0 (C-)

3,124,500 tokens
Score: 90.0 (A-)

869,200 tokens
Score: 90.0 (A-)

276,100 tokens
Score: 95.0 (A)

109,590 tokens
Score: 95.0 (A)

-60.3%

-96.5%

-97.6%

3. Evaluation Dataset & Test Verification#

Test Suite / Benchmark Dataset

Total Cases

Passed Cases

Pass Rate

Average Rubric Score

Mean Execution Latency

Complete Pytest Suite

69 (71 subtests)

69 (71 subtests)

100.0%

N/A

3.73s

Core Workflows (core_workflows)

10

10

100.0%

95.5 / 100.0

0.0045s

All 36 Workflows (all_36_workflows)

36

36

100.0%

89.7 / 100.0

0.0035s

Expanded Cases & Alerts (expanded_cases_alerts)

25

25

100.0%

90.6 / 100.0

0.0038s

4. Architectural Synthesis#

  1. Dual-Tier Progressive Disclosure Breakthrough:

    • Tier 1 (Skills Progressive Disclosure): Eliminates 50,000+ character runbook prompt dumps.

    • Tier 2 (Progressive MCP Tool Discovery): Eliminates upfront binding of dozens of JSON parameter schemas.

  2. Total Efficiency Impact:

    • Reduces per-turn schema and prompt overhead from ~28,000 tokens down to ~3,500 tokens.

    • Enables the system to scale to hundreds of enterprise MCP security tools without hitting LLM context limits or causing attention dilution.