Skip to content

README.md

Latest commit

 

History

History
276 lines (211 loc) · 7.21 KB

README.md

File metadata and controls

276 lines (211 loc) · 7.21 KB
Raw

Chronicle Integration Tests

Quick Start

Start containers and run tests:

cd tests
make test           # Start containers + run tests (excludes slow/sdk/GPU)

Validate suite tags without starting containers:

make validate-tags

Or step by step:

make start          # Start test containers
make all            # Run all tests (excludes slow/sdk/GPU)
make stop           # Stop containers

Note: Default test runs exclude slow tests (backend restarts, long timeouts) and sdk tests (unreleased SDK features) for faster feedback. Run these explicitly with make test-slow or make test-sdk when needed.

Test Suites

Run specific test suites:

make endpoints          # API endpoint tests
make integration        # End-to-end workflows
make infra              # Infrastructure resilience tests
make configuration      # Container-independent configuration tests

Special Test Categories

Slow Tests (excluded by default for faster feedback):

make test-slow    # Run ONLY slow tests (backend restarts, long timeouts)
  • Backend restart tests (service stop/start cycles)
  • Connection resilience tests
  • Tests requiring >30s timeouts
  • Excluded from default make test runs

SDK Tests (excluded until SDK is released):

make test-sdk     # Run ONLY SDK tests (unreleased features)
  • SDK client library tests
  • SDK authentication tests
  • SDK upload/retrieval tests
  • Excluded from default make test runs until SDK is published

All Tests Including Excluded:

make test-all-with-slow-and-sdk    # Run everything including slow and SDK tests

Container Management

All container operations are available through simple Makefile targets:

Command What it does
make start Start test containers (or reuse if healthy)
make stop Stop containers (saves logs automatically)
make restart Restart containers (keep same images)
make rebuild Rebuild images and restart (for code changes)
make containers-clean Saves logs → stops → removes everything
make status Show container health and ports
make logs SERVICE=<name> View logs for specific service

Important: Containers are NEVER removed without saving logs first!

Logs are automatically saved to: tests/logs/YYYY-MM-DD_HH-MM-SS/

Available Services for Logs

make logs SERVICE=chronicle-backend-test   # Main backend service
make logs SERVICE=workers-test              # RQ workers
make logs SERVICE=mongo-test                # MongoDB
make logs SERVICE=redis-test                # Redis
make logs SERVICE=speaker-service-test      # Speaker recognition

Test Workflows

Full Test Run (Clean Slate)

make containers-clean   # Clean previous state (saves logs)
make test               # Start fresh + run all tests

Quick Iteration (Reuse Containers)

make start              # Start containers once
make test-quick         # Run tests (fast, no container startup)
make test-quick         # Run again (even faster)

Code Changes (Rebuild Required)

# After modifying Python code
make rebuild            # Rebuild images with latest code
make test-quick         # Run tests on new build

Test Environment

Test services run on separate ports from production to avoid conflicts:

Service Test Port Production Port
Backend API 8001 8000
MongoDB 27018 27017
Redis 6380 6379

Test Database: Uses test_db database (isolated from production)

Test Credentials:

  • Admin Email: test-admin@example.com
  • Admin Password: test-admin-password-123
  • JWT Secret: test-jwt-signing-key-for-integration-tests

Troubleshooting

Port Conflicts

make status         # See what's running
make stop           # Stop test containers

If ports are still in use by other services:

lsof -i :8001       # Find what's using port 8001
# Kill the process or stop the conflicting service

Test Failures

# View backend logs
make logs SERVICE=chronicle-backend-test

# View worker logs
make logs SERVICE=workers-test

# Check container health
make status

Clean Slate

make containers-clean    # Saves logs + full cleanup
make start               # Fresh start

Container Issues

Containers won't start:

make status                  # Check current state
make containers-clean        # Full cleanup (saves logs)
make start                   # Start fresh

Health checks failing:

make logs SERVICE=chronicle-backend-test   # Check backend logs
# Common issues: MongoDB not ready, Redis connection failed

Tests hang or timeout:

# Check if services are healthy
make status

# View logs for stuck service
make logs SERVICE=workers-test

Log Preservation

All cleanup operations preserve logs automatically!

When you run make containers-clean or make clean-all:

  1. Step 1: Logs are saved to tests/logs/YYYY-MM-DD_HH-MM-SS/
  2. Step 2: Containers are stopped and removed
  3. Step 3: Volumes are removed

Each log directory contains:

  • Service logs for all containers
  • Container status snapshot
  • Container resource usage stats
  • Test results (if available)

View saved logs:

ls -lh tests/logs/                          # List all log archives
cat tests/logs/2026-01-17_14-30-45/chronicle-backend-test.log

API Key Separation

Chronicle tests are separated into two execution paths:

1. No API Keys Required

These tests run without external API dependencies:

  • Endpoint tests (CRUD operations, permissions)
  • Infrastructure tests (workers, queues, health checks)
  • Basic integration tests

Configuration: Uses configs/mock-services.yml (no transcription/LLM)

2. API Keys Required (~30% of tests)

These tests require external services:

  • Full E2E tests with transcription (Deepgram)
  • Memory extraction tests (OpenAI)
  • Transcript quality verification

Configuration: Uses configs/deepgram-openai.yml

Setup:

# Copy template
cp setup/.env.test.template setup/.env.test

# Add API keys
DEEPGRAM_API_KEY=your-key-here
OPENAI_API_KEY=your-key-here

Development Tips

Faster iteration:

  1. Start containers once: make start
  2. Run specific test suite: make endpoints
  3. Keep containers running between test runs
  4. Only rebuild when code changes: make rebuild

Debugging specific tests:

# Run Robot Framework directly for a single test file
cd tests
uv run --with-requirements test-requirements.txt robot \
    --outputdir results \
    --test "Specific Test Name" \
    endpoints/test_user.robot

Clean iteration cycle:

# 1. Make code changes
# 2. Rebuild containers
make rebuild

# 3. Run specific test suite
make endpoints

# 4. View logs if needed
make logs SERVICE=chronicle-backend-test

# 5. Repeat

Technical Details: Tests use Robot Framework for end-to-end validation, but you don't need to know Robot Framework to run tests. Just use the Makefile commands above.

For Robot Framework test development guidelines, see:

  • TESTING_GUIDELINES.md - Comprehensive testing patterns and standards
  • tags.md - Approved test tags and usage