Compare model robustness across treatment and missing outcome adjustments
Guide for cdid 1.0.0.0 (Stata). Recorded examples retain their stated versions.
On this page
When baseline covariates influence which units enter treatment, how untreated outcomes evolve over time, and which outcomes are recorded across dates, chained difference-in-differences (cdid, Bellégo, Benatia and Dortet-Bernadet 2025; Callaway and Sant’Anna 2021) separates estimation into two modeling layers: treatment adjustment (method(ipw|reg|dr)) and handling missing outcomes (selection(none|conditional_mtar) and selmethod(ipw|imputation|dr)).
This article defines the four nuisance models across the two layers, summarizes the large-sample consistency conditions for each supported combination, and walks through five model-correctness scenarios. For full mathematical definitions, see Treatment adjustment and Handling missing outcomes.
1. The four labeled nuisance models across two layers
To recover average treatment effects for the study population $F$ in frame(), cdid uses up to four labeled nuisance models across two layers:
| Symbol | Model name | Layer and Stata options | What it models and where it is fitted |
|---|---|---|---|
| $p$ | Treatment propensity model | Treatment adjustment (method(ipw) or method(dr), using covars() / indepvars) |
For each distinct risk set, a logit model of membership in target cohort $g$ versus controlgroup(), weighted according to the correction for missing outcomes. |
| $m$ | Counterfactual untreated change regression | Treatment adjustment (method(reg) or method(dr), with outcomepooling(separate|pooled), using covars() / indepvars) |
Weighted least squares regression fitted on control units per comparison block $b = (e, K)$ by default (outcomepooling(separate): $m_b(X) = \alpha_b + L(e), z(X)'\beta_b$) or jointly across blocks (outcomepooling(pooled): $m_b(X) = \alpha_b + L(e), z(X)'\beta$). |
| $q$ | Observation pair probability model | Handling missing outcomes (selection(conditional_mtar) with selmethod(ipw) or selmethod(dr), using selcovars()) |
For each needed cohort and date pair cell $c = (k, e)$, a logit model $q_c(W)$ for observing both dates of pair $e$, either directly (selmodel(pair_logit)) or as the product of two date logits (selmodel(independent_period_logit)). |
| $j$ | Actual cohort change imputer | Handling missing outcomes (selection(conditional_mtar) with selmethod(imputation) or selmethod(dr), using covars() / indepvars) |
For each needed cohort and date pair cell $c = (k, e)$, a weighted least squares linear model $j_c(X) = \alpha_{k,e} + L(e), z(X)'\beta_r$ fitted to observed changes of cohort $k$, treatment effects included, and evaluated across every unit of cohort $k$ in $F$. |
2. Correctness conditions across both layers
Every consistency result below is a large-sample property that requires:
- Scientifically sufficient conditioning: Supplying variables in
covars()orselcovars()does not prove sufficiency; the covariates must justify conditional parallel trends and conditional mean trends missing at random ($E[Y_t - Y_s \mid W, G = k, S_{s,t} = 1] = E[Y_t - Y_s \mid W, G = k]$) in the population. - Overlap and support in the population, with the probability bounds not binding: Treatment propensities stay strictly below one, observation probabilities stay strictly positive where $q$ is fitted, imputation cells have positive training mass and prediction support where $j$ is fitted, the fitted models converge to regular interior limits, and all comparisons combined by GMM satisfy valid moment conditions for the same parameter.
| Correction for missing outcomes | method(ipw) |
method(reg) |
method(dr) |
|---|---|---|---|
selection(none) with representative pair mean changes (no $q$ or $j$) |
$p$ correct | $m$ correct across all blocks | $p$ correct or $m$ correct |
selection(conditional_mtar) selmethod(ipw) (fits $q$) |
$q$ correct in every cell and $p$ correct | $q$ correct in every cell and $m$ correct | $q$ correct in every cell and ($p$ correct or $m$ correct) |
selection(conditional_mtar) selmethod(imputation) (fits $j$) |
$j$ correct in every cell/group and $p$ correct | $j$ correct in every cell/group and $m$ correct | $j$ correct in every cell/group and ($p$ correct or $m$ correct) |
selection(conditional_mtar) selmethod(dr) (fits $q$ and $j$) |
For every needed cell $c$, ($q_c$ correct or $j_c$ correct), and $p$ correct | For every needed cell $c$, ($q_c$ correct or $j_c$ correct), and $m$ correct | For every needed cell $c$, ($q_c$ correct or $j_c$ correct), and ($p$ correct or $m$ correct) |
Four qualifications
- Treatment DR supplies no general correction for a wrong imputer: Under pure imputation (
selmethod(imputation)), the sufficient condition requires a correct $j$ within each needed cell/pooling group. Choosingmethod(dr)provides the $p$-or-$m$ alternative after cohort changes are reconstructed; it does not generally repair biased reconstructed changes from a misspecified $j$. (These conditions are sufficient, not necessary: special projection or aggregation cancellations can still occur.) - Coupling through shared fits: When
outcomepooling(pooled)shares one slope vector $\beta$ across all comparison blocks in $m$, and whenimppooling(),selpooling(), orselmodel(independent_period_logit)shares parameters across cells or dates, misspecification in one cell or block can shift a shared slope everywhere it is used (whereas defaultoutcomepooling(separate)isolates $\beta_b$ to block $b$). - Common conditioning set $W$ and covariates only in
selcovars(): Under the augmented correction (selmethod(dr)),covars()builds $j$, $m$, and $p$ from covariates $X$, whereasselcovars()builds $q$ from $W = (X, Z)$. Variables $Z$ listed only inselcovars()never enter $j$, $m$, or $p$. If $Z$ shifts the conditional mean of actual changes given $W$ but is omitted fromcovars(), $j_c(X)$ cannot generally represent that $W$-conditional mean, so the condition that the imputer is correct fails; a correct $q_c(W)$ together with valid treatment adjustment on $W$ provides the alternative condition. Adding extra variables toselcovars()is therefore not automatically harmless. - Every model used must be estimable: the logits must pass their rank checks and have interior estimates, and the regressions must pass their rank, conditioning and prediction-support checks, even when the other model of a doubly robust pair is correct;
cdiddoes not fall back to another estimator. A completely observed or completely unobserved cell with a free intercept in $q$ stops estimation withr(498). Pure imputation (selmethod(imputation)) runs on fully observed cells because it fits no $q$, whereasselmethod(dr)stops whenever any model it uses ($q$, $j$, and $p$ or $m$ as required bymethod()) cannot be estimated.
3. Five misspecification scenarios
These are theoretical examples of sufficient conditions; they do not describe the simulated data of the hands-on do-file. Consider five specifications on a study population with treatment covariates x_bin x_cont and an additional observation-shifting covariate z_sel:
fit_none_dr:selection(none) method(dr)fit_obs_or:selection(conditional_mtar) selmethod(ipw) method(reg) selcovars(x_bin x_cont z_sel)fit_obs_dr:selection(conditional_mtar) selmethod(ipw) method(dr) selcovars(x_bin x_cont z_sel)fit_imp_dr:selection(conditional_mtar) selmethod(imputation) imppooling(separate) method(dr)fit_aug_dr:selection(conditional_mtar) selmethod(dr) imppooling(separate) method(dr) selcovars(x_bin x_cont z_sel)
| Scenario | Correct vs misspecified models | fit_none_dr |
fit_obs_or |
fit_obs_dr |
fit_imp_dr |
fit_aug_dr |
|---|---|---|---|---|---|---|
1. Baseline design: z_sel shifts observation only; linear changes in x_bin x_cont; logit $p(x)$ and pair-observation logits $q(x, z)$ (the default selmodel(pair_logit)) |
$p, m, q, j$ all correct | Generally biased | Consistent | Consistent | Consistent | Consistent |
2. Nonlinear untreated trend: untreated changes depend on $x_{\mathrm{cont}}^2$, omitted from covars(x_bin x_cont) |
$p, q$ correct; linear $m, j$ misspecified | No guarantee | No guarantee | Consistent | No guarantee | Consistent |
3. Wrong observation model $q$: observation depends on $z_{\mathrm{sel}}^2$, omitted from selcovars(x_bin x_cont z_sel) |
$j$ correct; $p, m$ valid on $W$; fitted $q$ misspecified | No guarantee | No guarantee | No guarantee | Consistent | Consistent if the fitted $q$ has a regular interior limit |
4. Mispooled imputer $j$: set imppooling(within_cohort) or across_cohorts when treatment effects vary with $x$ |
$q$ correct and $m, p$ valid; imposed $j$ pooling misspecified | No guarantee | Consistent | Consistent | No guarantee | Consistent |
5. z_sel also shifts outcomes: z_sel shifts outcome trends but is omitted from covars(x_bin x_cont) while included in selcovars(x_bin x_cont z_sel) |
$q$ and $p$ correct on $W$; $j$ and $m$ omit outcome-shifting z_sel |
No guarantee | No guarantee from the conditions on $W$ | Consistent through $q$ and $p$ | No guarantee | Consistent through $q$ and $p$ |
“Consistent” means that a sufficient large-sample condition holds under all stated assumptions and regular limits of the fitted models; “No guarantee” means those sufficient conditions no longer establish consistency (it does not prove bias for a specific cell or exclude special cancellation).
4. Specifications in Stata
Sections 3, 4 and 5 of the hands-on do-file estimate the models without correction, with observation IPW, with pure imputation and with the augmented correction. See Treatment adjustment, Observation models and Change imputation for the commands. The scenarios above are reasoning about sufficient conditions, not results of a simulation. The qualifications above on shared fits and on models that cannot be estimated apply.
Reading path, step 16 of 22 · Previous: GMM weighting and covariance regularization · Next: Clustered inference, confidence bands, and pretrend tests