Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions Dockerfile.integrationtest
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
FROM golang:1.23 AS builder
FROM golang:1.24 AS builder
WORKDIR /go/src/github.com/infobloxopen/hotload/
COPY . .

# build integration test binary
RUN make build-test

FROM golang:1.23 AS runner
FROM golang:1.24 AS runner
COPY --from=builder /go/src/github.com/infobloxopen/hotload/integrationtests.test .

2 changes: 1 addition & 1 deletion Dockerfile.test
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM golang:1.23
FROM golang:1.24
WORKDIR /go/src/github.com/infobloxopen/hotload
COPY . .
RUN make ci-test
124 changes: 124 additions & 0 deletions k8ssecret/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
# hotload/k8ssecret

A [hotload](https://github.com/infobloxopen/hotload) strategy that watches Kubernetes Secrets for connection string changes via the K8s API.

Use this when the Secret containing your database credentials is in a **different namespace** than your application — where a volume mount isn't possible.

## Installation

```bash
go get github.com/infobloxopen/hotload/k8ssecret
```

This is a separate Go module. It pulls in `k8s.io/client-go` but does **not** add those dependencies to the root `hotload` module.

## Usage

```go
import (
"database/sql"

_ "github.com/infobloxopen/hotload"
_ "github.com/infobloxopen/hotload/k8ssecret"
_ "github.com/jackc/pgx/v5/stdlib" // or any sql driver
)

func main() {
// The Secret "orders-db" in namespace "orders-ns" has a key "dsn.txt"
// containing: postgres://statexfer:s3cret@orders-pg:5432/orders?sslmode=require
db, err := sql.Open("hotload", "k8ssecret://pgx/orders-db?namespace=orders-ns&dsn=dsn.txt")
if err != nil {
log.Fatal(err)
}
db.Query("SELECT 1")
}
```

When the Secret is updated (e.g., credential rotation), hotload detects the change and transparently creates new connections with the updated DSN.

## DSN Format

```
k8ssecret://<driver>/<secret-name>?namespace=<ns>&dsn=<key>
```

| Component | Description |
|-----------|-------------|
| `<driver>` | The registered database driver name (e.g., `pgx`, `postgres`, `mysql`) |
| `<secret-name>` | Kubernetes Secret name |
| `namespace` | Namespace containing the Secret. Defaults to the pod's namespace (from service account mount) or `"default"`. |
| `dsn` | Data key within the Secret that holds the connection string. Defaults to `"dsn.txt"`. |

## Kubernetes Secret

The Secret holds connection strings as data keys. Each key is a file when volume-mounted, or a watchable field via this strategy:

```yaml
apiVersion: v1
kind: Secret
metadata:
name: orders-db
namespace: orders-ns
type: Opaque
stringData:
dsn.txt: "postgres://statexfer:s3cret@orders-pg:5432/orders?sslmode=require"
```

The `dsn` parameter selects which key to read:

```
k8ssecret://pgx/orders-db?namespace=orders-ns&dsn=dsn.txt
```

## When to Use This vs fsnotify

| Scenario | Strategy | Why |
|----------|----------|-----|
| Secret in the **same** namespace | `fsnotify` | Mount the Secret as a volume. Cheaper — no API calls. |
| Secret in a **different** namespace | `k8ssecret` | Volume mounts can't cross namespaces. This strategy watches via the K8s API. |

For same-namespace Secrets mounted as volumes:

```go
// Secret mounted at /var/run/secrets/myapp/dsn.txt
db, err := sql.Open("hotload", "fsnotify://pgx/var/run/secrets/myapp/dsn.txt")
```

## RBAC

The service account running your application needs `get` and `watch` permissions on the target Secret:

```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: secret-reader
namespace: orders-ns
rules:
- apiGroups: [""]
resources: ["secrets"]
resourceNames: ["orders-db"]
verbs: ["get", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: statexd-reads-orders-db
namespace: orders-ns
subjects:
- kind: ServiceAccount
name: statexd
namespace: statexfer-system
roleRef:
kind: Role
name: secret-reader
apiGroup: rbac.authorization.k8s.io
```

## How It Works

1. On first `Watch()` call, the strategy fetches the Secret via `Secrets(namespace).Get()`.
2. It reads the value of the specified `key` and returns it as the initial connection string.
3. A background goroutine establishes a K8s watch on the Secret.
4. When the Secret is modified, the new value is pushed to hotload, which transparently rotates connections.
5. If the watch is interrupted (API server restart, network partition), it reconnects automatically with a 2-second backoff.
59 changes: 59 additions & 0 deletions k8ssecret/go.mod
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
module github.com/infobloxopen/hotload/k8ssecret

go 1.23.0

require (
github.com/infobloxopen/hotload v1.7.4
k8s.io/api v0.32.3
k8s.io/apimachinery v0.32.3
k8s.io/client-go v0.32.3
)

require (
github.com/beorn7/perks v1.0.1 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/colega/gaugefuncvec v0.1.0 // indirect
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
github.com/emicklei/go-restful/v3 v3.11.0 // indirect
github.com/fxamacker/cbor/v2 v2.7.0 // indirect
github.com/go-logr/logr v1.4.3 // indirect
github.com/go-openapi/jsonpointer v0.21.0 // indirect
github.com/go-openapi/jsonreference v0.20.2 // indirect
github.com/go-openapi/swag v0.23.0 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/protobuf v1.5.4 // indirect
github.com/google/gnostic-models v0.6.8 // indirect
github.com/google/go-cmp v0.7.0 // indirect
github.com/google/gofuzz v1.2.0 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/josharian/intern v1.0.0 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/kylelemons/godebug v1.1.0 // indirect
github.com/mailru/easyjson v0.7.7 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/prometheus/client_golang v1.20.0 // indirect
github.com/prometheus/client_model v0.6.1 // indirect
github.com/prometheus/common v0.55.0 // indirect
github.com/prometheus/procfs v0.15.1 // indirect
github.com/teivah/onecontext v1.3.0 // indirect
github.com/x448/float16 v0.8.4 // indirect
golang.org/x/net v0.43.0 // indirect
golang.org/x/oauth2 v0.23.0 // indirect
golang.org/x/sys v0.35.0 // indirect
golang.org/x/term v0.34.0 // indirect
golang.org/x/text v0.28.0 // indirect
golang.org/x/time v0.7.0 // indirect
google.golang.org/protobuf v1.36.7 // indirect
gopkg.in/evanphx/json-patch.v4 v4.12.0 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
k8s.io/klog/v2 v2.130.1 // indirect
k8s.io/kube-openapi v0.0.0-20241105132330-32ad38e42d3f // indirect
k8s.io/utils v0.0.0-20241104100929-3ea5e8cea738 // indirect
sigs.k8s.io/json v0.0.0-20241010143419-9aa6b5e7a4b3 // indirect
sigs.k8s.io/structured-merge-diff/v4 v4.4.2 // indirect
sigs.k8s.io/yaml v1.4.0 // indirect
)
Loading
Loading