Add observation-level sample weights to poLCA

[tingjhenjiang]

Adds a weights argument to poLCA() (numeric vector or column name of data) implementing weighted pseudo-maximum-likelihood estimation:

• weighted log-likelihood sum_i w_i log f(y_i)
• weighted M-step response probabilities via row-scaled posteriors passed to the existing probhat C routine
• weighted mixing proportions (no-covariate case) and a weighted pure-R Newton step poLCA.dLL2dBeta.w for latent class regression, matching the C layout of d2lldbeta2 exactly at unit weights
• pseudo-ML sandwich standard errors in poLCA.se (A^-1 B A^-1 with A = sum w s s', B = sum w^2 s s'), reducing to the original ginv(t(s) s) at unit weights
• weighted cell frequencies in poLCA.compress for Chisq/Gsq

weights=NULL or all-1 weights reproduce the original estimator exactly; integer weights match row-replicated data to fp accumulation error.

[shermanlo77]

Have you written this commit yourself?

[shermanlo77]

weighted pure-R Newton step poLCA.dLL2dBeta.w

Is there a chance you could instead modify the C code?

[tingjhenjiang]

weighted pure-R Newton step poLCA.dLL2dBeta.w

Is there a chance you could instead modify the C code?

reply by claude Fable 5:

Yes, that's possible, and I'm happy to do it. Let me first explain why I initially kept it in R, then what the C change would look like.

What the C change involves. The per-observation accumulation loop in d2lldbeta2 makes weighting trivial: add a double w argument and multiply each observation's gradient contribution (x[k](rgivy[j]-prior[j])) and Hessian contributions by w[i]. The R wrapper poLCA.dLL2dBeta.C then passes as.double(w) (unit weights by default), and init_poLCA.c is updated to register 9 arguments. Since the symbol is package-internal (not part of the exported API), extending the existing function is safe; alternatively a separate d2lldbeta2w entry could preserve the old symbol, but I don't think that's necessary.
Assessment. Doing it in C is the cleaner end state: it removes poLCA.dLL2dBeta.w entirely, removes the if (usew) branch at the call site (matching the probhat simplification above, so the estimator has a single unified code path with w ≡ 1 as the unweighted special case), and keeps the Newton step in one implementation instead of two that must be kept in sync. The cost is a slightly larger footprint in src/, which I originally tried to avoid but agree is justified here. h changes.

[tingjhenjiang]

weighted pure-R Newton step poLCA.dLL2dBeta.w

Is there a chance you could instead modify the C code?

a new push and pull request is presented.
claude fable 5 says:

2. Moved the weighted Newton step into C:

• src/poLCA.c: d2lldbeta2 now takes a double *w argument; each observation's gradient and Hessian contribution is multiplied by w[i].
• src/init_poLCA.c: registration updated from 8 to 9 arguments.
• R/poLCA.dLL2dBeta.C.R: wrapper gained w = rep(1, nrow(x)) and passes as.double(w).
• R/poLCA.R:139-141: the covariate branch is now a single poLCA.dLL2dBeta.C(rgivy, prior, x, w) call.
• R/poLCA.dLL2dBeta.w.R deleted; no references remain anywhere (usew survives only to decide whether ret$weights is reported as NULL).

Verification (recompiled poLCA.so, ran a four-part test script):

• Weighted C gradient/Hessian matches the old pure-R implementation to ~1e-14 on random inputs (N=300, 4 classes, 3 covariates).
• Default and explicit unit weights are identical(), and match the reference at unit weights.
• Full-model replication-weight check on the election data with a PARTY covariate: weight-2 observations vs. physically duplicated rows give coefficients, response probabilities, and class shares agreeing to ~1e-16.
• Unweighted fits via weights = NULL and weights = rep(1, N) produce identical log-likelihood and coefficients, and both report ret$weights as NULL.