Skip to content

Conversation

@wachterjohannes
Copy link

Q A
Bug fix? yes/no
New feature? yes/no
Docs? yes/no
Issues Fix #...
License MIT

Implements the Model Router architecture, enabling intelligent model selection based on input characteristics. This allows agents to automatically route requests to appropriate models (e.g., vision models for images, speech models for audio).

Core Infrastructure:

  • RouterInterface: Universal contract for all router types
  • RoutingResult: Result with model name, optional transformer, reason, and confidence
  • TransformerInterface: Contract for input transformation
  • RouterContext: Provides platform access, model catalog, and metadata
  • SimpleRouter: Callable-based router for flexible routing logic
  • ChainRouter: Composite router for trying multiple routers in sequence
  • ModelRouterInputProcessor: Integration with Agent's input processing pipeline

Input Enhancement:

  • Add optional platform property to Input class (backward compatible)
  • Update Agent to pass platform when creating Input
  • Platform available to input processors via Input.getPlatform()

Includes 6 examples demonstrating different routing patterns and comprehensive test coverage (14 router tests, all 187 agent tests passing).

Key Features:

  • No breaking changes - all changes are backward compatible
  • Extensible design - custom routers in <20 lines
  • Multi-provider support - works with any Platform implementation
  • Transformation support - routers can transform input before invocation
  • Default model fallback - routers can access agent's default model
  • Capability-based routing - find models supporting specific capabilities

Architecture Decisions:

  • Single RouterInterface for all router types enables composition
  • Transformer in RoutingResult (router decides both model and transformation)
  • Platform passed via Input (no breaking changes to interfaces)
  • RouterContext provides metadata including default model
  • Start simple with SimpleRouter, add complexity gradually

Implements Phase 1 of the Model Router architecture, enabling intelligent
model selection based on input characteristics. This allows agents to
automatically route requests to appropriate models (e.g., vision models
for images, speech models for audio).

Core Infrastructure:
- RouterInterface: Universal contract for all router types
- RoutingResult: Result with model name, optional transformer, reason, and confidence
- TransformerInterface: Contract for input transformation
- RouterContext: Provides platform access, model catalog, and metadata
- SimpleRouter: Callable-based router for flexible routing logic
- ChainRouter: Composite router for trying multiple routers in sequence
- ModelRouterInputProcessor: Integration with Agent's input processing pipeline

Input Enhancement:
- Add optional platform property to Input class (backward compatible)
- Update Agent to pass platform when creating Input
- Platform available to input processors via Input.getPlatform()

Includes 6 examples demonstrating different routing patterns and comprehensive
test coverage (14 router tests, all 187 agent tests passing).

Key Features:
- No breaking changes - all changes are backward compatible
- Extensible design - custom routers in <20 lines
- Multi-provider support - works with any Platform implementation
- Transformation support - routers can transform input before invocation
- Default model fallback - routers can access agent's default model
- Capability-based routing - find models supporting specific capabilities

Architecture Decisions:
- Single RouterInterface for all router types enables composition
- Transformer in RoutingResult (router decides both model and transformation)
- Platform passed via Input (no breaking changes to interfaces)
- RouterContext provides metadata including default model
- Start simple with SimpleRouter, add complexity gradually
@wachterjohannes
Copy link
Author

@chr-hertel this is my first protoype of the router - don't look to deep into the code - just the architecture stuff and there will be more as soon as we merged the first part.

following things i have planned:

  1. Basic routing
  2. Rule-based declarative routing (basically modelflow-ai decission tree)
  3. AI-powered routing

Another time - i have to refactor this code as soon as we are OK with the patterns behind it :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant