← Engineering log
AI Agents LangChain.js Security

Bring Your Own Agent CLI: An Optional API Mode, and Why We Removed the Harness Router

RobOS keeps running the coding-agent CLIs you already use. We added an opt-in API mode through LangChain.js for quick requests, local Ollama models and semantic Knowledge Graph search, and deleted the Unified Harness Protocol router that nothing actually used.

The question was whether RobOS should adopt LangChain. Almost all of its AI work already goes to Claude Code, Codex, Gemini CLI / Antigravity or GitHub Copilot, which bring their own tool loops, models and sign-in. Putting a second agent runtime underneath them would duplicate that. So the rule is: install a CLI, sign in, and RobOS works. Direct model APIs are an option, never a requirement.

  • One switch point: every one-shot AI request in the apps goes through robos-lib/ai-agent, which runs the CLI you set as active in RobOS Agents. Gemini CLI / Antigravity joins Copilot, Claude Code and Codex as a supported provider, and Software Center can now install it.
  • Optional API mode (packages/robos-llm): off by default. Turn it on in RobOS Preferences and quick requests go to Anthropic, OpenAI, Google Gemini or a local Ollama server through LangChain.js, falling back to your CLI if the call fails. The LangChain packages are not shipped with RobOS; Preferences installs them on request into ~/.local/share/robos/llm-deps. API keys live only in pass, keys that older versions saved as plain text are moved there, and LangSmith tracing is forced off.
  • Stubs that now do real work: the ? quick-ask button in the AI text box was not wired into any app; it now answers in AI Prompt Studio, Git Projects and the Knowledge Graph Explorer. The Graph Explorer's AI Co-Pilot used to pick nodes by keyword; it now asks your agent and checks the answer against the SHACL shapes. Import & Refine's Ask AI agent works with any signed-in CLI instead of needing a router server.
  • Semantic Knowledge Graph search: with API mode on, kgraph search --engine semantic and the robos_kgraph_semantic_search MCP tool rank nodes by meaning, using embeddings from OpenAI, Gemini or Ollama and a local vector index that only re-embeds changed nodes.
  • Removing the harness router: no app ran real work through robos-agent-client/harness-router.js. Its in-process fallback passed prompts through a shell, so quotes or $(...) in a prompt could run commands, and it reported success for three of its six harnesses without starting anything. We deleted it, the unused session layer and the Agents Manager panel, about 3,600 lines of code and docs. The prompt security scan moved into ai-agent, so every one-shot prompt is checked before it reaches a CLI or an API.

Still open: answers arrive in one piece rather than streaming, and Task Implementer and Dev Central start their CLIs without the prompt scan.

RobOS Preferences, AI Agents and API Mode section, with API mode on, Google Gemini selected, a key saved in pass, and Ollama embeddings for semantic search
The new AI Agents & API Mode section in RobOS Preferences. Key fields save straight to pass.