Advanced Design

Learn advanced patterns for building production-ready MCP environments. These techniques are used in environments like remote_browser and enable sophisticated agent interactions.

The Hub Pattern

The hub pattern uses BaseHub, which is actually an inner MCP server that can be mounted to your main server. This provides namespace separation and modular organization.

How Hubs Work

A hub is an MCP server that exposes a single tool. When you call this tool, you specify which function within the hub to execute:

{
    "name": "setup",
    "arguments": {
        "name": "navigate_to_page",
        "arguments": {
            "url": "https://example.com"
        }
    }
}

The outer name is the hub tool, inner name is the specific function, and inner arguments are passed to that function.

Implementation

# setup/__init__.py
from hud.tools.base import BaseHub

setup = BaseHub("setup")

# Import all setup modules
from . import navigation, data, auth

__all__ = ["setup"]

# setup/navigation.py
from . import setup

@setup.tool()
async def navigate_to_page(url: str):
    """Navigate browser to specified URL."""
    env = setup.env  # Access shared environment
    await env.browser.goto(url)
    return {"status": "success", "url": url}

@setup.tool()
async def reset_browser():
    """Clear cookies and return to blank page."""
    env = setup.env
    await env.browser.clear_cookies()
    await env.browser.goto("about:blank")

# server.py
from .setup import setup as setup_hub
from .evaluate import evaluate as evaluate_hub

mcp = MCPServer(name="my-environment")

@mcp.initialize
async def initialize_environment(ctx):
    # Create environment context
    env = await create_environment()
    
    # Set environment on hubs
    setup_hub.env = env
    evaluate_hub.env = env
    
    # Mount hubs to server
    mcp.mount(setup_hub)
    mcp.mount(evaluate_hub)

Benefits:

Modular organization: Each function in its own file
Namespace separation: Tools grouped logically
Shared state: Access environment via hub.env
Auto-discovery: Import registers all decorated functions

Progress Notifications

For long-running initialization, provide real-time feedback:

@mcp.initialize
async def initialize_environment(ctx):
    """Initialize with progress updates."""
    # Extract progress token from context
    progress_token = getattr(ctx.meta, "progressToken", None) if ctx.meta else None
    
    async def send_progress(progress: int, message: str):
        if progress_token:
            await ctx.session.send_progress_notification(
                progress_token=progress_token,
                progress=progress,
                total=100,
                message=message
            )
    
    await send_progress(10, "Starting services...")
    await start_x11_server()
    
    await send_progress(40, "Launching browser...")
    browser = await launch_browser()
    
    await send_progress(70, "Connecting to cloud provider...")
    await connect_provider()
    
    await send_progress(100, "Environment ready!")

This shows live in the HUD platform during container startup.

Dynamic Configuration via Agent Metadata

Agents can pass metadata that gets injected into the MCP server’s initialization context using patch_mcp_config:

# In your agent
from hud.agents import ClaudeAgent

agent = ClaudeAgent(
    metadata={
        "display_width": 1280,
        "display_height": 720,
        "browser_provider": "anchorbrowser"
    }
)

# This metadata is patched into mcp_config via patch_mcp_config
# See hud/agents/base.py#L476-L479

How It Works

The agent’s _setup_config method uses patch_mcp_config to inject metadata:

from hud.utils.mcp import MCPConfigPatch, patch_mcp_config

async def _setup_config(self, mcp_config: dict[str, dict[str, Any]]) -> None:
    """Inject metadata into the metadata of the initialize request."""
    if self.metadata:
        patch_mcp_config(
            mcp_config,
            MCPConfigPatch(meta=self.metadata),
        )

Using Metadata in Your Environment

@mcp.initialize
async def initialize_environment(ctx):
    # Access metadata from context
    metadata = ctx.meta
    
    # Adjust display resolution
    width = metadata.get("display_width", 1920)
    height = metadata.get("display_height", 1080)
    
    # Select browser provider
    provider = metadata.get("browser_provider", "chromium")
    
    # Configure performance
    headless = metadata.get("headless", False)
    
    # Apply configuration
    browser_config = {
        "viewport": {"width": width, "height": height},
        "headless": headless,
        "args": ["--disable-gpu"] if headless else []
    }
    
    browser = await launch_browser(browser_config)

See patch_mcp_config implementation for details.

Live Telemetry

Expose real-time data via resources:

@mcp.resource("telemetry://live")
async def get_telemetry():
    """Provide live environment status."""
    return {
        "status": "running",
        "live_url": get_live_view_url(),  # VNC or browser URL
        "metrics": {
            "memory_usage": get_memory_usage(),
            "active_connections": count_connections(),
            "uptime": get_uptime()
        },
        "logs": get_recent_logs(lines=50)
    }

This appears in the HUD platform UI for debugging.

VNC Server Integration

For GUI environments, provide visual access:

# Start from base image with VNC
FROM hudpython/novnc-base:latest

# Your GUI app installation
RUN apt-get update && apt-get install -y your-gui-app

# VNC is already configured on ports 5901 (VNC) and 8080 (noVNC)

In your MCP server:

@mcp.resource("telemetry://live")
async def get_telemetry():
    # Provide noVNC URL for browser viewing
    return {
        "live_url": "http://localhost:8080/vnc.html",
        "vnc_port": 5901,
        "password": "vncpass"  # Set via VNC_PW env var
    }

Using Base Images

HUD provides optimized base images:

hudpython/novnc-base

Ubuntu with XFCE desktop
VNC and noVNC pre-configured
Common utilities installed

FROM hudpython/novnc-base:latest

# Add your GUI application
RUN apt-get update && apt-get install -y \
    libreoffice \
    firefox \
    your-app

# VNC auto-starts with your app

hudpython/browser-base

Chromium and Playwright installed
Optimized for browser automation

FROM hudpython/browser-base:latest

# Add your browser automation code
COPY src/ /app/src/
CMD ["python", "-m", "src.server"]

Phase 6: Automated Improvement

Enable Cursor Agent to improve your environment:

1. Development Configuration

Use reloaderoo for hot-reload development:

// .cursor/mcp.json
{
  "mcpServers": {
    "my-env-dev": {
      "command": "npx",
      "args": [
        "reloaderoo", "--",  // Enables hot-reload without restarting client
        "docker", "run", "-i", "--rm",
        "-v", "$(pwd)/src:/app/src:rw",
        "-e", "PYTHONDEBUG=1",
        "my-env:dev"
      ]
    }
  }
}

2. Cursor Rules

Create .cursor/rules/mcp_environment_iteration.mdc:

---
description: Improve an MCP environment
alwaysApply: false
---

Setup
1. Ensure dev mode MCP config exists
2. Explore environment structure
3. Clarify improvement objectives

Iteration
1. Test current functionality via tools
2. Note errors and improvements in TODOs
3. Fix issues in source code
4. Call restart_server to reload
5. Verify fixes work correctly
6. Continue until all TODOs complete

3. Improvement Workflow

Prompt Cursor:

Context: I have a browser environment in my-env folder
Tools: Use my-env-dev to interact with it
Task: Test all evaluators and fix any that fail

The agent will:

Explore available tools
Test each function
Fix issues in your code
Reload and verify
Continue iterating

Example: Production Browser Environment

Here’s how remote_browser implements these patterns:

# Modular structure
src/hud_controller/
├── server.py          # Main MCP server with progress
├── providers/         # Cloud browser integrations
├── setup/            # Hub with navigation, cookies
├── evaluate/         # Hub with page checks
└── tools/            # Playwright and computer tools

# Advanced features
- Progress notifications during startup
- Live browser URL in telemetry
- Dynamic resolution from metadata
- Multiple provider support
- Automatic cleanup on shutdown

Get Started

Core Concepts

Evaluate Agents

Build Environments

Train Agents

CLI Reference

SDK Reference

Advanced Design

The Hub Pattern

How Hubs Work

Implementation

Progress Notifications

Dynamic Configuration via Agent Metadata

How It Works

Using Metadata in Your Environment

Live Telemetry

VNC Server Integration

Using Base Images

hudpython/novnc-base

hudpython/browser-base

Phase 6: Automated Improvement

1. Development Configuration

2. Cursor Rules

3. Improvement Workflow

Example: Production Browser Environment

Next Steps

Environment README

Reference Examples

Get Started

Core Concepts

Evaluate Agents

Build Environments

Train Agents

CLI Reference

SDK Reference

​The Hub Pattern

​How Hubs Work

​Implementation

​Progress Notifications

​Dynamic Configuration via Agent Metadata

​How It Works

​Using Metadata in Your Environment

​Live Telemetry

​VNC Server Integration

​Using Base Images

​hudpython/novnc-base

​hudpython/browser-base

​Phase 6: Automated Improvement

​1. Development Configuration

​2. Cursor Rules

​3. Improvement Workflow

​Example: Production Browser Environment

​Next Steps

Environment README

Reference Examples

The Hub Pattern

How Hubs Work

Implementation

Progress Notifications

Dynamic Configuration via Agent Metadata

How It Works

Using Metadata in Your Environment

Live Telemetry

VNC Server Integration

Using Base Images

hudpython/novnc-base

hudpython/browser-base

Phase 6: Automated Improvement

1. Development Configuration

2. Cursor Rules

3. Improvement Workflow

Example: Production Browser Environment

Next Steps