What is PCP?

Abstract

Portal Control Protocol (PCP) is a wire protocol and architecture for embedding System Intelligence directly into an operating system’s compositor. It redefines how machine intelligence interacts with a graphical desktop: not as an external agent peering in through accessibility APIs or screen-scraping, but as a first-class subsystem that shares the same process space, data structures, and event loop as the compositor itself.

The distinction matters. External agent architectures sit outside the OS, communicating through IPC, HTTP, or accessibility bridges. Each crossing of a process boundary adds latency, drops context, and forces serialization of data that the compositor already holds in memory. PCP eliminates that gap. System Intelligence and the PCP Core run inside the compositor. Capabilities are invoked through direct language-level trait methods. State is shared, not copied. Events are streamed, not polled.

This embedded model yields an architecture where the intelligence layer has authoritative, real-time knowledge of every surface, window, and input event on the desktop. No discovery heuristic, no OCR fallback, no coordinate guessing. The system declares what exists, and SI acts on declared state with semantic precision.

PCP is an open standard. Its message format, data model, and wire protocol are documented and implementable by any compositor that wishes to embed System Intelligence. This specification defines the protocol, its design principles, and the architecture that makes embedded intelligence possible.

The Embedded Intelligence Model

Traditional approaches to desktop automation treat the intelligence layer as a separate entity. Accessibility frameworks expose widget trees over IPC. Screen readers query text content through serialized objects. Agent frameworks simulate keyboard and mouse input, then observe pixel changes to infer what happened. Each of these crossings introduces latency, imprecision, and fragility.

PCP inverts this relationship. Rather than reaching into the OS from outside, System Intelligence lives inside it. The intelligence layer is not a process communicating with the compositor through a socket or a DBus interface. It is a compositor subsystem, compiled into the same binary, running on the same event loop, sharing the same heap.

This co-location carries concrete benefits. When an application exposes a button, the compositor already knows that button’s label, bounds, semantic role, and activation method. PCP re-exports that data, enriched with accessibility metadata, to the intelligence layer through direct trait implementations. No serialization. No copy. No round-trip.

The compositor is the single authoritative source of desktop state. It sees every surface, tracks every input event, and manages the lifecycle of every application window. PCP anchors the intelligence layer to that authority. SI does not discover the desktop by scraping or probing. It receives a continuously updated, structurally complete model of what the system is doing, right now.

Because SI runs in-process, operations that would require multiple IPC round-trips in an external architecture collapse into single function calls. Activating a menu item, reading a text buffer, or subscribing to window-state changes are all direct operations on compositor data structures. The result is an intelligence layer that responds at the speed of the desktop itself.

Architecture Overview

The PCP stack consists of four layers, each with a defined responsibility:

graph TD
    subgraph Compositor
        SI["System Intelligence"]
        Core["PCP Core"]
        Adapters["Adapters"]
    end
    Apps["Applications"]

    SI -->|"invoke capabilities"| Core
    Core -->|"read state"| Adapters
    Adapters -->|"surface declarations"| Apps
    Apps -->|"expose semantics"| Adapters
    Adapters -->|"enriched model"| Core
    Core -->|"events + state"| SI

System Intelligence is the reasoning layer. It receives the enriched desktop model from PCP Core, decides what to do, and invokes capabilities to act. SI runs inside the compositor process and communicates with PCP Core through direct language-level calls.

PCP Core is the protocol engine. It maintains the desktop model, dispatches events, manages capability invocations, and handles session state. PCP Core sits between SI and the adapter layer, translating between semantic operations and compositor internals.

Adapters bridge the protocol to application surfaces. Each adapter knows how to extract semantic data from a specific surface type: Wayland surfaces, X11 windows, accessibility nodes, or application-specific interfaces. Adapters feed declared state upward into the PCP Core model.

Applications sit below the adapter layer. They expose their UI semantics through standard interfaces: accessibility APIs, semantic markup, or protocol-specific declarations. Applications do not communicate with SI directly.

The entire stack above the adapter boundary runs inside the compositor process. Only the adapter-to-application crossing involves any form of IPC, and even that uses native display protocol mechanisms rather than custom serialization.

Design Principles

Principle	Description
Agent-native	Capabilities are semantic operations, not pixel coordinates. The intelligence layer invokes named capabilities (“activate the compose button in Mail”) rather than simulating clicks at screen positions. The protocol is built around what things mean, not where pixels land.
Compositor-authoritative	The compositor sees every surface, application, and input event on the desktop. PCP re-exports that state, enriched with accessibility data, to the intelligence layer. Nothing is guessed or inferred. The system declares what exists.
Progressive fidelity	Integration depth determines capability depth. Tier 1 applications expose full semantic access, letting SI read, write, and interact with structured content. Tier 2 applications expose structural accessibility data. Tier 3 applications fall back to input simulation. Better integration yields richer interaction, but basic control is always available.
Real-time and bidirectional	PCP streams events, maintains persistent sessions, and handles continuous data flows. This is not a request-response protocol. It is a persistent, bidirectional connection between the intelligence layer and the desktop. State changes flow upward; capability invocations flow downward.
System-internal by default	SI and PCP Core share the compositor process. No daemon, no Unix socket, no serialization layer for internal operations. Communication between SI and the protocol engine happens through direct language-level calls on shared data structures.
Open protocol	The message format, data model, and wire protocol are fully documented and implementable by any compositor. PCP is not tied to a single display server or windowing system. Any compositor that can embed a subsystem can implement PCP.
Audit-everything	Every capability invocation is logged with a timestamp, target, parameters, and result. System-privileged operations carry the same audit trail as user-initiated actions. Full auditability is a protocol-level invariant, not an optional extension.

What PCP Is Not

Not an external API. PCP is the internal interface between System Intelligence and the compositor. It is invoked through direct language-level trait methods, not HTTP endpoints, not DBus methods, not CLI tools. External tooling may speak PCP over a wire, but the primary interface is in-process.

Not a daemon. PCP Core is a compositor subsystem, not a background service. It does not listen on a socket. It does not run as a separate process. It is part of the compositor binary, started with it and stopped with it.

Not MCP. The Model Context Protocol serves a different purpose: it connects language models to external tools through a request-response abstraction. PCP handles streaming events, persistent sessions, bidirectional communication, continuous hardware data, and in-process state sharing. Where MCP is a tool-calling protocol, PCP is an embedded intelligence protocol. The two address different problems at different layers.

Not UI automation. Input simulation exists in PCP as Tier 3, the fallback of last resort for applications that expose no semantic interface. The protocol’s primary operating mode is capability-level interaction: invoking named operations on declared UI elements. Clicking at coordinates is the exception, not the design intent.