Choose IPW, OR or DR in an incomplete panel
Guide for cdid 1.3.0.0 (R). Recorded examples retain their stated versions.
On this page
Results below were produced on October 29, 2026 with cdid 1.1.0.0, did 2.5.1 and DRDID 1.3.0 under R 4.5.2, using the displayed seeds where applicable. These are individual examples; they do not establish repeated-sample coverage. OR/DR calls shown here explicitly retain the historical pooled specification; the current default uses separate comparison regressions.
In chained difference in differences (cdid, extending Bellégo, Benatia and Dortet-Bernadet 2025 and Callaway and Sant’Anna 2021), est_method selects the treatment adjustment method: inverse probability weighting (IPW, "ipw"), outcome regression (OR, "reg"), or doubly robust estimation (DR, "dr"). That choice is separate from handling missing outcomes (selection and selection_est_method), which governs how observed outcome changes represent the study population in the unit table (frame).
This worked example compares IPW, OR, and DR on the same incomplete panel, study population, and target effects, holding selection = "none", control_group = "nevertreated", pt_assumption = "post", base_period = "varying", and the inference flags fixed. Add PT-all restrictions reuses the exact panel and unit table generated here.
Question
When an incomplete panel tracks a study population in a unit table (frame) with a baseline covariate x that shifts both cohort membership and untreated outcome trends, and missingness is independent of covariates, cohort, and potential outcomes, how do est_method = "ipw", est_method = "reg", and est_method = "dr" compare on the same data, why does this data generating law satisfy both the treatment propensity model and the shared counterfactual outcome regression, and how should an analyst choose among them in practice?
Data
Consider a study population of $N = 2000$ units in units (frame) across $T = 6$ stored calendar dates $t \in {1, 2, 3, 4, 5, 6}$. Each unit $i$ has an independent baseline covariate $X_i \sim \text{Uniform}(-1, 1)$. Cohort $G_i \in {0, 3, 5}$ (first_treat) is assigned by multinomial weights $1$ for never treated controls ($G_i = 0$), $\text{odds}_3(X_i) = \exp(0.2 + 0.6 X_i)$ for cohort $3$, and $\text{odds}_5(X_i) = \exp(-0.2 - 0.4 X_i)$ for cohort $5$. With $\text{total}(X_i) = 1 + \text{odds}_3(X_i) + \text{odds}_5(X_i)$, a uniform draw $U_i \sim \text{Uniform}(0, 1)$ allocates unit $i$ with probabilities $p_3(X_i) = \text{odds}_3(X_i) / \text{total}(X_i)$, $p_5(X_i) = \text{odds}_5(X_i) / \text{total}(X_i)$, and $p_0(X_i) = 1 / \text{total}(X_i)$.
Untreated potential outcomes at date $t \in {1, \dots, 6}$ are $Y_{i,t}(0) = \alpha_i + 0.1 t + 0.3 X_i t + \varepsilon_{i,t}$, with independent unit effect $\alpha_i \sim \mathcal{N}(0, 1)$ and shock $\varepsilon_{i,t} \sim \mathcal{N}(0, 0.5^2)$ independent of $(G_i, X_i, \alpha_i)$. Actual latent outcomes are $Y_{i,t} = Y_{i,t}(0) + \mathbf{1}{G_i > 0, t \ge G_i}(1 + 0.25(t - G_i))$. No anticipation holds ($\text{anticipation} = 0$). For posttreatment dates $t \ge g$ of cohort $g \in {3, 5}$, the population target is $\text{ATT}(g, t) = 1 + 0.25(t - g)$, and the dynamic event time target at $e = t - g \ge 0$ is the same function $1 + 0.25 e$.
Each unit and date outcome is observed independently with probability $0.5$. The panel panel retains all $N \times T = 12000$ rows across dates $1$ to $6$ with NA in y when unobserved, and units retains one row per study unit (id, first_treat, x), including units with zero or one observed outcome.
Assumptions
The three fits vary only est_method (Options).
| Argument | Setting | Purpose |
|---|---|---|
yname, tname, idname, gname |
"y", "period", "id", "first_treat" |
Outcome, stored calendar date ($1$ to $6$), unit ID, and cohort ($0, 3, 5$). |
xformla, data, frame |
~ x, panel, units |
Covariate design, $N \times T$ panel with NA outcomes, and $N$ row unit table (Options). |
control_group, est_method |
"nevertreated", "ipw" / "reg" / "dr" |
Compares treated cohorts with $G_i = 0$ across IPW, OR, and DR (Treatment adjustment). |
selection, pt_assumption, base_period |
"none", "post", "varying" |
Representative observed pair changes, PT-post, and consecutive preperiod reports (Parallel trends). |
bstrap, cband, compute_inffunc |
FALSE, FALSE, TRUE |
Analytical standard errors, pointwise normal intervals, and complete unit influence functions (Options). |
Why this law satisfies both treatment models and baseline missingness
Passing xformla = ~ x to IPW and OR does not automatically make both models match the population law: IPW fits risk set cohort logits, whereas the explicit outcome_pooling = "pooled" used in this example fits one regression of control outcome changes across all comparison blocks (Treatment adjustment). Here both hold by construction:
- Cancellation of $\alpha_i$ and conditional parallel trends: Differencing across any stored calendar pair $e = (s, t)$ with $s < t$ cancels $\alpha_i$, giving $Y_{i,t}(0) - Y_{i,s}(0) = 0.1(t - s) + 0.3 X_i (t - s) + (\varepsilon_{i,t} - \varepsilon_{i,s})$. Since $\varepsilon_{i,t}$ has mean zero given $(G_i, X_i)$, conditional parallel trends hold at all dates $1$ to $6$.
- Representative baseline missingness (
selection = "none"): Because observation has probability $0.5$ independently of $(G_i, X_i, \alpha_i, \varepsilon_{i,t})$, observed pair changes within each cohort are mean representative of that cohort inunits. The estimator uses empirical cohort pair response rates without fitting an observation model; this is not an exact finite sample recovery of the population mean change (Handling missing outcomes). - Correct pairwise treatment propensity logits ($p$): Conditioning on $G_i \in {3, 0}$, the odds of cohort $3$ versus never treated controls are $p_3(X_i) / p_0(X_i) = \exp(0.2 + 0.6 X_i)$, with linear log odds $0.2 + 0.6 X_i$. Conditioning on $G_i \in {5, 0}$, the odds of cohort $5$ versus never treated controls are $p_5(X_i) / p_0(X_i) = \exp(-0.2 - 0.4 X_i)$, with linear log odds $-0.2 - 0.4 X_i$. Compact support $X_i \in [-1, 1]$ bounds the population probabilities away from 0 and 1. The independent observation mask preserves these linear odds in observed pair populations; multiplying by cohort specific response-rate constants can change the intercept, without introducing nonlinear terms in $X_i$. This does not guarantee support or a finite logit root in a realized sample.
- Correct shared counterfactual outcome regression ($m$): With all dates $1, \dots, 6$ stored, the calendar step count is $L(e) = t - s$. The control mean change is $E[Y_{i,t}(0) - Y_{i,s}(0) \mid X_i, G_i = 0] = 0.1 L(e) + 0.3 L(e) X_i$, matching
cdid’s single pooled control regression $m_b(X_i) = \alpha(b) + L(e) X_i \beta$ with local block intercept $\alpha(b) = 0.1 L(e)$ and shared global slope $\beta = 0.3$ (n_regression_fits = 1). - Why
"notyettreated"alters the pooled odds: For cohort $3$ before period $5$,"notyettreated"pools $G_i = 0$ and $G_i = 5$. Under this multinomial law, the odds of $G_i = 3$ against $G_i \in {0, 5}$ are $\exp(0.2 + 0.6 X_i) / (1 + \exp(-0.2 - 0.4 X_i))$, whose log odds are nonlinear in $X_i$. Linear pairwise log odds againstnevertreateddo not imply linear risk set log odds for anotyettreatedmixture. This is a model specification issue for that mixture, rather than a claim that NYT by itself forces a difference betweencdidanddid. The distinct fixed-horizon control-pool moments retained by chained estimation are explained in Moving from did.
Code
set.seed(20261028)
n_units <- 2000L
n_periods <- 6L
x <- stats::runif(n_units, min = -1, max = 1)
odds3 <- exp(0.2 + 0.6 * x)
odds5 <- exp(-0.2 - 0.4 * x)
total_weight <- 1 + odds3 + odds5
p3 <- odds3 / total_weight
p5 <- odds5 / total_weight
p0 <- 1 / total_weight
u_cohort <- stats::runif(n_units)
first_treat <- ifelse(u_cohort < p3, 3L, ifelse(u_cohort < p3 + p5, 5L, 0L))
units <- data.frame(id = seq_len(n_units), first_treat = first_treat, x = x)
panel <- data.frame(
id = rep(units$id, each = n_periods),
period = rep(seq_len(n_periods), times = n_units),
first_treat = rep(units$first_treat, each = n_periods),
x = rep(units$x, each = n_periods)
)
alpha <- stats::rnorm(n_units, mean = 0, sd = 1)
eps <- stats::rnorm(nrow(panel), mean = 0, sd = 0.5)
y0 <- rep(alpha, each = n_periods) + 0.1 * panel$period + 0.3 * panel$x * panel$period + eps
treated_now <- panel$first_treat > 0L & panel$period >= panel$first_treat
effect <- ifelse(treated_now, 1 + 0.25 * (panel$period - panel$first_treat), 0)
y_complete <- y0 + effect
observed <- stats::runif(nrow(panel)) < 0.5
panel$y <- ifelse(observed, y_complete, NA_real_)
methods <- c(ipw = "ipw", or = "reg", dr = "dr")
fits <- lapply(methods, function(m) {
cdid::att_gt(
outcome_pooling = "pooled",
yname = "y", tname = "period", idname = "id", gname = "first_treat",
xformla = ~ x, data = panel, frame = units,
control_group = "nevertreated", est_method = m, selection = "none",
pt_assumption = "post", base_period = "varying", bstrap = FALSE, cband = FALSE
)
})
common_cols <- c("target.id", "group", "time", "role", "available", "estimate", "std.error")
cell_tables <- Map(function(fit, method_name) {
tab <- cdid::tidy(fit)[, common_cols]
tab$method <- method_name
tab$estimator_label <- fit$chained$estimator
tab$population_truth <- ifelse(tab$role == "post", 1 + 0.25 * (tab$time - tab$group), 0)
tab
}, fits, names(fits))
per_cell_comparison <- do.call(rbind, cell_tables)
per_cell_comparison[per_cell_comparison$role == "post", ]
dynamic_fits <- lapply(fits, cdid::aggte, type = "dynamic")
dynamic_comparison <- do.call(rbind, Map(function(agg, method_name) {
data.frame(
method = method_name, event_time = agg$egt,
estimate = agg$att.egt, std.error = agg$se.egt,
population_truth = ifelse(agg$egt >= 0, 1 + 0.25 * agg$egt, 0)
)
}, dynamic_fits, names(dynamic_fits)))
dynamic_comparison
gt_plots <- lapply(fits, cdid::ggdid)
dynamic_plots <- lapply(dynamic_fits, cdid::ggdid)
Console output
target.id group time role available estimate std.error method
ipw.2 ATT(g=3,t=3) 3 3 post TRUE 1.011155 0.05960259 ipw
ipw.3 ATT(g=3,t=4) 3 4 post TRUE 1.305540 0.06123456 ipw
ipw.4 ATT(g=3,t=5) 3 5 post TRUE 1.538407 0.05912689 ipw
ipw.5 ATT(g=3,t=6) 3 6 post TRUE 1.744085 0.06290892 ipw
ipw.9 ATT(g=5,t=5) 5 5 post TRUE 1.053864 0.06549822 ipw
ipw.10 ATT(g=5,t=6) 5 6 post TRUE 1.139585 0.06709454 ipw
or.2 ATT(g=3,t=3) 3 3 post TRUE 1.022526 0.05780193 or
or.3 ATT(g=3,t=4) 3 4 post TRUE 1.290286 0.05976722 or
or.4 ATT(g=3,t=5) 3 5 post TRUE 1.529845 0.05911358 or
or.5 ATT(g=3,t=6) 3 6 post TRUE 1.723973 0.06180484 or
or.9 ATT(g=5,t=5) 5 5 post TRUE 1.047170 0.06530943 or
or.10 ATT(g=5,t=6) 5 6 post TRUE 1.136599 0.06704396 or
dr.2 ATT(g=3,t=3) 3 3 post TRUE 1.007945 0.05946479 dr
dr.3 ATT(g=3,t=4) 3 4 post TRUE 1.298627 0.06084339 dr
dr.4 ATT(g=3,t=5) 3 5 post TRUE 1.528495 0.05908227 dr
dr.5 ATT(g=3,t=6) 3 6 post TRUE 1.728774 0.06184471 dr
dr.9 ATT(g=5,t=5) 5 5 post TRUE 1.053470 0.06544368 dr
dr.10 ATT(g=5,t=6) 5 6 post TRUE 1.143902 0.06712190 dr
estimator_label population_truth
ipw.2 chained IPW-GMM 1.00
ipw.3 chained IPW-GMM 1.25
ipw.4 chained IPW-GMM 1.50
ipw.5 chained IPW-GMM 1.75
ipw.9 chained IPW-GMM 1.00
ipw.10 chained IPW-GMM 1.25
or.2 chained outcome regression/control-mean GMM 1.00
or.3 chained outcome regression/control-mean GMM 1.25
or.4 chained outcome regression/control-mean GMM 1.50
or.5 chained outcome regression/control-mean GMM 1.75
or.9 chained outcome regression/control-mean GMM 1.00
or.10 chained outcome regression/control-mean GMM 1.25
dr.2 chained doubly robust GMM 1.00
dr.3 chained doubly robust GMM 1.25
dr.4 chained doubly robust GMM 1.50
dr.5 chained doubly robust GMM 1.75
dr.9 chained doubly robust GMM 1.00
dr.10 chained doubly robust GMM 1.25
method event_time estimate std.error population_truth
ipw.1 ipw -3 -0.0036005082 0.07624331 0.00
ipw.2 ipw -2 0.1276100192 0.07024728 0.00
ipw.3 ipw -1 0.0207240855 0.05430503 0.00
ipw.4 ipw 0 1.0279996411 0.04434251 1.00
ipw.5 ipw 1 1.2400861096 0.04047772 1.25
ipw.6 ipw 2 1.5384065010 0.05912689 1.50
ipw.7 ipw 3 1.7440852287 0.06290892 1.75
or.1 or -3 0.0099445749 0.07412692 0.00
or.2 or -2 0.1330753901 0.07008427 0.00
or.3 or -1 0.0213751385 0.05350763 0.00
or.4 or 0 1.0322457015 0.04343659 1.00
or.5 or 1 1.2296705225 0.03912655 1.25
or.6 or 2 1.5298454025 0.05911358 1.50
or.7 or 3 1.7239726289 0.06180484 1.75
dr.1 dr -3 -0.0003672074 0.07605146 0.00
dr.2 dr -2 0.1281262590 0.07029130 0.00
dr.3 dr -1 0.0193646340 0.05420753 0.00
dr.4 dr 0 1.0259005597 0.04426626 1.00
dr.5 dr 1 1.2376021460 0.04021797 1.25
dr.6 dr 2 1.5284947291 0.05908227 1.50
dr.7 dr 3 1.7287739283 0.06184471 1.75
Result
Each entry in fits is a cdid_MP object (Results). In per_cell_comparison and dynamic_comparison, population_truth records the known population parameter, kept distinct from sample estimates (estimate) and analytical standard errors (std.error):
- Cohort $3$ posttreatment cells $(3, 3), (3, 4), (3, 5), (3, 6)$ at event times $e = 0, 1, 2, 3$ have population truth $1.00, 1.25, 1.50, 1.75$; cohort $5$ cells $(5, 5), (5, 6)$ at $e = 0, 1$ have truth $1.00, 1.25$.
fit$chained$estimatoris documented on every chained fit, whereascdid::tidy(fit)includes the optionalestimatorcolumn for OR and DR fits and for imputation or augmented missing outcome routes. Selectingcommon_colsavoids assuming identical optional tidy columns across methods.fit$nequals $2000$ for all three fits becauseframe = unitskeeps every study unit infit$inffuncand in cohort shares.
Interpretation
Why finite sample estimates can differ across IPW, OR, DR, and did
Even when both $p$ and $m$ are correct in the population, IPW, OR, and DR solve different sample equations before two step GMM combination: IPW uses normalized Hajek weights from risk set logits, OR subtracts predictions $\hat{\alpha}(b) + L(e) X_i \hat{\beta}$ from one pooled control regression across all blocks, and DR applies IPW to the control regression residuals (Treatment adjustment). Their finite sample estimates and standard errors can therefore differ. Covariate imbalance does not force a difference: fitted corrections can coincide or cancel in particular samples.
Passing the same xformla = ~ x formula to cdid and did likewise does not imply identical finite sample OR or DR estimates: cdid and did both fit separate control regressions by default. This example explicitly uses outcome_pooling = "pooled", which shares a per-step slope across comparisons. Chaining can also combine a different set of comparisons, even with separate slopes. If the full fitted procedure coincides, including coefficient restrictions, nuisance fitting, retained comparisons and weights, point estimates should agree up to numerical rounding; method labels alone establish neither equality nor a difference. With incomplete outcomes and explicit allow_unbalanced_panel = TRUE, ordinary mode uses did’s RCS routines based on levels. With FALSE or its ordinary mode default, it instead keeps complete cases and estimates changes within units. Each route has its own sampling conditions; the independent mask here is compatible with them, but their fitted constructions differ. Use mode = "ordinary" when exact did behavior is required (Moving from did).
Practical choice among IPW, OR, and DR
- IPW (
est_method = "ipw"): Needs a credible risk set treatment logit $p$ and strict overlap for every active comparison, while leaving period specific covariate effects on untreated changes unrestricted. - OR (
est_method = "reg"): Needs a credible control change regression $m$, without fitting treatment logits. Default separate fits allow each comparison its own coefficients. The explicit pooled specification used here instead requires a constant per step slope $\beta$ on $L(e) z(X_i)$ across its blocks; misspecification in one block can then move $\hat{\beta}$ elsewhere. - DR (
est_method = "dr", default): Consistent for each comparison if either $p$ or $m$ is correct, under valid missing outcome handling, parallel trends, no anticipation, overlap and the usual large sample regularity conditions. Both active fits must still satisfy their computation and support checks. If an active treatment logit separates or is rank deficient, or if a required reduced control regression remains rank deficient, is badly conditioned or fails required prediction reconstruction,cdidstops with an explicit error rather than falling back to OR or IPW. Exact aliases can first be removed when every required prediction is preserved; local rank deficiency alone need not invalidate an explicitly pooled fit, but the package never switches to pooling automatically.
There is no general finite sample unbiasedness guarantee for IPW, OR, or DR, and none is guaranteed to have the lowest variance across all designs. In any random finite draw, unsupported cells (available == FALSE), unavailable Wald pretrend tests (fit$W, fit$Wpval), or numerical and conditioning refusals remain possible without automatic retry, redraw, or dropping of units or moments. See Robustness and How to compare precision.
Next step
- Keep
units,panel,y_complete, andobservedin your R session and continue to Add PT-all restrictions, Treatment adjustment, Options, and Results.
References
- Bellégo, C., Benatia, D. and Dortet-Bernadet, V. (2025). The Chained Difference-in-Differences. Journal of Econometrics. https://doi.org/10.1016/j.jeconom.2024.105783
- Callaway, B. and Sant’Anna, P. H. C. (2021). Difference-in-Differences with Multiple Time Periods. Journal of Econometrics 225(2). https://doi.org/10.1016/j.jeconom.2020.12.001

Reading path, step 8 of 22 · Previous: Treatment adjustment · Next: Handling missing outcomes
All steps of the reading path
| Step | Page |
|---|---|
| 1 | Get started |
| 2 | Moving from did |
| 3 | Incomplete panels |
| 4 | Identify effects when the observed units change composition |
| 5 | How to compare precision |
| 6 | Prepare your panel and unit table |
| 7 | Treatment adjustment |
| 8 | Choose IPW, OR or DR in an incomplete panel (this page) |
| 9 | Handling missing outcomes |
| 10 | Sampling information for the observation model |
| 11 | Choose an observation model |
| 12 | Impute actual changes |
| 13 | Parallel trends |
| 14 | Add PT-all restrictions |
| 15 | GMM weighting and covariance regularization |
| 16 | Robustness |
| 17 | Inference |
| 18 | Options |
| 19 | Results |
| 20 | Diagnose errors and unavailable results |
| 21 | Cross-pair chaining and control parallel trends |
| 22 | Short pre-treatment windows |