Estimation Bias Adjustment for Adaptively Collected Data

Abstract

In many scientific experiments involving the multi-arm bandits, the data is collectedand are genuinely dependent. As a result, many commonly used the statistical inference methods could be problematic. For example, the ordinary least square estimator, which is widely used in the regression literature, will produce a biased estimator in the contextual disjoint linear models. As a result, any further statistical inference methods such as the hypothesis testing and confidence interval based on this biased estimator could be either invalid or conservative. We develop approaches in two stages: pre-data bias mitigation (pre-BM) and post-data bias mitigation (post- BM) to correct this. In Chapter 2, we propose an alternative approach named the randomized Multi-Arm Bandits (rMAB) that combines a randomization step with a chosen MAB algorithm. The proposed rMAB can achieve the optimal regret asymptotically if choosing randomization probability appropriately. It is shown numerically that the magnitude of the bias of the sample mean based on the rMAB is substantially smaller than that of competing methods. In Chapter 3, we first explicitly derive the exact bias formula for a family of estimators. It is shown that the bias term depends on the average number of the times that a particular arm is pulled and the covariance between the estimator and this number. To get a data-driven version method, we introduce the RBA, a Resampling-based Bias Adjustment method, to calculate this bias term. It is numerically shown that the RBA performs better than its competitors.