Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions .github/workflows/compat-nightly.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
name: compat-nightly

# Drift-detection + real-launch conformance job. Per
# docs/design/20260707-python-connector-sdk.md §3/§4/AC#2: regenerates the
# gRPC stubs from the current conduit-connector-protocol BSR HEAD and, once
# Lane D/F land (examples/http-poll-source, a real Conduit build), launches
# the example connector as a subprocess via a real, unmodified Conduit binary
# and asserts handshake + record flow + graceful GRPCController.Shutdown.
#
# STATUS (week 1): stub only. The stub-regeneration step is real and runs
# today; the "launch a real Conduit binary" step is not implemented yet —
# it depends on Lane D (example connector) and Lane F (harness), which are not
# week-1 scope. Do not claim this job proves AC#2/#3 until that step exists.
on:
schedule:
- cron: "0 6 * * *"
workflow_dispatch:

jobs:
drift-check:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: astral-sh/setup-uv@v5
with:
enable-cache: true
- uses: bufbuild/buf-setup-action@v1
- run: uv sync --all-extras
- name: Regenerate stubs from current BSR HEAD
run: ./tools/generate-stubs.sh
- name: Fail if regenerated stubs differ from committed stubs
run: |
git diff --exit-code -- src/conduit/_grpc || {
echo "::error::conduit-connector-protocol has drifted from the vendored stubs. Regenerate and commit."
exit 1
}

# TODO (Lane D/F, not week 1): real-conduit-launch job — build/download a
# Conduit binary, launch examples/http-poll-source as a standalone plugin,
# assert the handshake line, record flow, and GRPCController.Shutdown path
# (not the 2s force-kill fallback). Tracked in the design doc's AC #2/#3.
28 changes: 28 additions & 0 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
name: lint

on:
push:
branches: [main]
pull_request:

jobs:
ruff:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: astral-sh/setup-uv@v5
with:
enable-cache: true
- run: uv sync --all-extras
- run: uv run ruff format --check .
- run: uv run ruff check .

mypy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: astral-sh/setup-uv@v5
with:
enable-cache: true
- run: uv sync --all-extras
- run: uv run mypy
33 changes: 33 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
name: release

# Manual release process for v0.x, per the v0.19 workstream scope
# (conduit-v019-plans/workstreams/python-connector-sdk.md §2, item 9):
# trusted-publisher CI automation to PyPI is an explicit fast-follow, not
# blocking. This workflow builds and checks the package on a tag push; the
# actual `pypi-publish` step is intentionally left commented out until the
# trusted-publisher OIDC setup is done (requires PyPI project configuration
# DeVaris has not yet performed — do not uncomment without that).
on:
push:
tags: ["v*"]
workflow_dispatch:

jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: astral-sh/setup-uv@v5
with:
enable-cache: true
- run: uv build
- run: uv run --with twine twine check dist/*
- uses: actions/upload-artifact@v4
with:
name: dist
path: dist/

# Fast-follow (v0.20, not this workflow's current scope): trusted
# publisher OIDC push to PyPI. See docs/design/20260707-python-connector-sdk.md §3.
# - name: Publish to PyPI
# uses: pypa/gh-action-pypi-publish@release/v1
25 changes: 25 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
name: test

on:
push:
branches: [main]
pull_request:

jobs:
test:
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
# Two most recent CPython minors on the 3.11+ floor, per
# docs/design/20260707-python-connector-sdk.md §3 (CI matrix).
python-version: ["3.12", "3.13"]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- uses: astral-sh/setup-uv@v5
with:
enable-cache: true
python-version: ${{ matrix.python-version }}
- run: uv sync --all-extras
- run: uv run pytest -v
30 changes: 30 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Python
__pycache__/
*.py[cod]
*.egg-info/
.eggs/
build/
dist/
.venv/
venv/

# Tooling caches
.mypy_cache/
.ruff_cache/
.pytest_cache/
.hypothesis/
htmlcov/
.coverage
.coverage.*

# uv
.uv/

# Editors / OS
.vscode/
.idea/
.DS_Store

# Generated protobuf/grpc stubs are checked in (vendored, see docs/design/) —
# do NOT ignore src/conduit/_grpc/**, but ignore ad hoc regeneration scratch.
buf-generate.log
21 changes: 21 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Changelog

All notable changes to this project are documented in this file. Format follows
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versions follow
[Semantic Versioning](https://semver.org/) starting at v1.0 (see the design
doc's Upgrade/rollback section for the pre-1.0 caveat).

## [Unreleased]

### Added

- Repo scaffold: `pyproject.toml` (uv/hatchling), lint/type config (ruff,
mypy), CI workflow stubs, package layout.
- Design doc (`docs/design/20260707-python-connector-sdk.md`) landed with the
Phase-1 review's must-fixes folded in.
- Generated gRPC/protobuf stubs for `conduit-connector-protocol` v2
(`SourcePlugin`, `DestinationPlugin`, `SpecifierPlugin`).
- Handshake implementation (`_handshake.py`): magic-cookie check, protocol
version negotiation, stdout handshake line.

No release has been tagged yet; nothing here is installable from PyPI.
52 changes: 52 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Contributing

Thanks for considering a contribution to `conduit-connector-sdk-python`.

## Tier and review bar

This repo is **Tier 1 (data path)** in the ConduitIO org's review taxonomy
(see [`ConduitIO/conduit`'s `CLAUDE.md`](https://github.com/ConduitIO/conduit/blob/main/CLAUDE.md)):
it originates and acknowledges records on one side of the exact gRPC boundary
Conduit's engine treats as authoritative. Concretely:

- Any change touching the wire adapter (`_grpc/`, `_handshake.py`, `serve.py`),
the ack/nack logic in `destination.py`/`source.py`, position handling, or the
`Record`/`Config` codec requires a design doc or an explicit waiver for small
fixes, per `docs/design/`.
- Human maintainer sign-off is required on Tier-1 changes; automated review
alone is never sufficient.
- Bug fixes ship with the regression test that would have caught the bug.
- PR descriptions include a failure-mode analysis: what could this break, what
would show it, how do we roll back.

## Local setup

```bash
uv sync --all-extras
# or: python3 -m venv .venv && . .venv/bin/activate && pip install -e '.[dev]'
```

## Before opening a PR

```bash
ruff format --check .
ruff check .
mypy
pytest
```

Regenerating the gRPC stubs (only needed when `conduit-connector-protocol` or
the `opencdc`/`config` messages in `conduit-commons` change):

```bash
./tools/generate-stubs.sh
```

Review the diff under `src/conduit/_grpc/` before committing — this is the
one directory in the repo that's generated, vendored output (see
`src/conduit/_grpc/__init__.py` for why it's structured the way it is).

## Commit style

Conventional commits (`feat:`, `fix:`, `docs:`, `refactor:`, `chore:`), matching
the rest of the ConduitIO org.
Loading
Loading