fix(rotation): default NULL-policy certs to the 'default' rotation policy

[evan-datadog]

Summary

Certs created without an explicit rotation_policy (the import/discovery and ACME/pending-cert paths) end up with rotation_policy_id = NULL. That's the root cause behind the ~559 NULL-policy certs in prod (and 231 in sandbox) — and because of the old cross-join bug they were effectively being rotated on a 70-day window.

This is the simple fix (vs. the over-engineered #355): instead of touching the rotation-window SQL, it makes the Certificate constructor fall back to the named default RotationPolicy so no newly created cert can ever be NULL-policy.

The default row is kept in sync with LEMUR_DEFAULT_ROTATION_INTERVAL — and the sync fires right before the rotation-candidate query (get_all_pending_reissue), so a config change takes effect on the next rotation pass regardless of whether any cert is created.

Change

File	Change
lemur/certificates/models.py	_get_default_rotation_policy() — pure getter (fetch, create-if-missing with config days, no config-sync). Certificate.__init__ uses kwargs.get("rotation_policy") or _get_default_rotation_policy().
lemur/policies/service.py	New sync_default_rotation_policy() — creates or updates the default row's days to LEMUR_DEFAULT_ROTATION_INTERVAL.
lemur/certificates/service.py	get_all_pending_reissue() calls sync_default_rotation_policy() at the top, so the row is in sync before the in-rotation-window query runs.
lemur/default.conf.py	Add LEMUR_DEFAULT_ROTATION_INTERVAL = 60 (canonical config).
lemur/tests/conf.py	Add LEMUR_DEFAULT_ROTATION_INTERVAL = 60.
lemur/manage.py	Align the init-time policy-seeding fallback to 60 — both the seed and the NULL-policy fallback read the same single flag (LEMUR_DEFAULT_ROTATION_INTERVAL).

⚠️ One-off scripts to run (manual, after deploy)

Scripts: experimental/users/evan.mcelheny/local/scripts/lemur/

lemur-rota.sh copies rotation_policy.py into a lemur pod and runs it against the pod's Postgres. Safe to re-run (all actions are idempotent). Tested in sandbox.

Run in order:

# 1. Set the "default" policy to 60 days (sandbox already done; prod is currently 35)
lemur-rota.sh prod set --days 60

# 2. Backfill existing NULL-policy certs onto "default" (sandbox already done — 231 → 0)
lemur-rota.sh prod backfill

# 3. Add unique index on rotation_policies.name (prevents duplicate "default" rows)
lemur-rota.sh prod add-index

Sandbox validation

set --days 45: default 60 → 45   ✓
set --days 60: default 45 → 60   ✓
add-index:     unique index created   ✓
add-index (2nd run): already exists, handled gracefully   ✓
duplicate insert after index: rejected by UniqueViolation   ✓
NULL-policy certs after backfill: 0 / 296   ✓

Audit (pre-change)

Env	default	NULL-policy certs	valid	within 60d
Sandbox	60	231/296	40	12
Prod	35	559/2060	89	25

Fixes CLOUDR-2089

[maperu]

i dont think this function is worthwhile, let's just keep this service.py file simple. I think we should just read the value directly in the caller

[evan-datadog]

Read what? the config value? we would have to update everywhere that RotationPolicy.days is referenced. Makes more sense; it is after all just a number and doesnt really need ot be a whole table in the dabase but it is currently and thats a much larger work of art.

[maperu]

im confused as to why were not calling policy.get in the other file? why do we have two getters in the policy file now?

[maperu]

why do we call this every time we reissue a cert?

[evan-datadog]

its not two but three; but I get your point (heh). Would you be ok if I just renamed it to update_default_rotation_policy ?

If thats the case do you want me to also put it in the manage.py instead of the whole

_DEFAULT_ROTATION_INTERVAL = "default"
default_rotation_interval = policy_service.get_by_name(
  _DEFAULT_ROTATION_INTERVAL
)

if default_rotation_interval:
  sys.stdout.write("[-] Default rotation interval policy already created, skipping...!\n")
else:
  days = current_app.config.get("LEMUR_DEFAULT_ROTATION_INTERVAL", 30)
  days = current_app.config.get("LEMUR_DEFAULT_ROTATION_INTERVAL", 60)
  sys.stdout.write("[+] Creating default certificate rotation policy of {days} days before issuance.\n".format(days=days))
  policy_service.create(days=days, name=_DEFAULT_ROTATION_INTERVAL)

[maperu]

isn't the default policy supposed to be set once at policy creation? why is it set at cert issuance?

[evan-datadog]

I am assuming that we are going to need to update this as cert lifespan goes down (a rota policy of 60 days makes zero sense when the certs only last for a max of 49 days, we would be reissuing our entire fleet every day). I want a sane way to tune this policy via a config change instead of sshing into the pod and hand writing to the database.

My plan is to have update_default_rotation_policy get called everytime we calculate what certs need to be rotated so if we decide to rollout a config change that sets the policy to 45 days or 30 it will get rolled out to every cert that uses the default policy.

That way we just have a single policy that the vast majority of certs use; if there is some strange subset that need a different policy they can have that and be on their own.

[maperu]

yes but the default policy is a config value, why do we need to query it at cert issuance? why not query it and set it at lemur boot?

[evan-datadog]

With the new changes it has been repushed to sandbox and is fully working.