Small Area Estimation WILEY SERIES IN SURVEY METHODOLOGY Established in part by WALTEAR . SHEWHAARNTD SAMUESL. WILKS Editors: Robert M. Groves, Graham KaIton, J N. K. Rao, Norbert Schwarz, Christopher Skinner A complete list of the titles in this series appears at the end of this volume. Small Area Estimation J. N. K. RAO Carleton University @EZiciENCE A JOHN WILEY & SONS, INC., PUBLICATION Copyright 0 2003 by John Wiley & Sons, Inc. All rights reserved Published by John Wiley & Sons, Inc., Hoboken, New Jersey Published simultaneously in Canada. No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 750-4744, or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 1 I I River Street, Hoboken, NJ 07030, (20 I) 7,184011 , fax (201) 748-6008, e-mail: [email protected]. Limit of LiabilityiDisclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representation or wamnties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages. For general information on our other products and services please contact our Customer Care Department within the U.S. at 877-762-2974, outside the U.S. at 317-572-3993 or fax 317-572-4002. Wiley also publishes its books in a variety of electronic formats. Some content that appears in print, however, may not be available in electronic format. Library of Congress Cataloging-in-Publication Data: Rao, J. N. K., 1937- Small area estimation / J.N.K. Rao. p. cm. -- (Wiley series in survey methodology) lncludes bibliographical references and index. lSBN 0-471-41374-7 (cloth) 1. Sampling (Statistics) 2. Estimation theoty. I. Title. 11. Series. QA27b.b .R344 2003 5 19.5'2--dc2 I 2002033197 Printed in the United States of America. 10 9 8 7 6 5 4 3 2 To my mother, Sakuntalamma and to my wife, Neela Contents List of Figures xiii List of Tables xv Foreword xvii Preface xxi 1 Introduction 1 1.1 What is a Small Area? . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Demand for Small Area Statistics . . . . . . . . . . . . . . . . . 3 1.3 Traditional Indirect Estimators . . . . . . . . . . . . . . . . . . 3 1.4 Small Area Models . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.5 Model-Based Estimation . . . . . . . . . . . . . . . . . . . . . . 4 1.6 Some Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Direct Domain Estimation 9 2 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.2 Design-based Approach . . . . . . . . . . . . . . . . . . . . . . 10 2.3 Estimation of Totals . . . . . . . . . . . . . . . . . . . . . . . . 11 2.3.1 Design-unbiased Estimator . . . . . . . . . . . . . . . . 11 2.3.2 Generalized Regression Estimator . . . . . . . . . . . . 13 2.4 Domain Estimation . . . . . . . . . . . . . . . . . . . . . . . . . 15 2.4.1 Case of no Auxiliary Information . . . . . . . . . . . . . 15 2.4.2 GREG Estimation . . . . . . . . . . . . . . . . . . . . . 17 2.4.3 Domain-specific Auxiliary Information . . . . . . . . . . 17 2.5 Modified Direct Estimators . . . . . . . . . . . . . . . . . . . . 20 2.6 Design Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 2.7 Proofs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 2.7.1 Proof of YGR(X)= X . . . . . . . . . . . . . . . . . . . . 25 2.7.2 Derivation of Calibration Weights w; . . . . . . . . . . 25 2.7.3 Proof of Y = XTB when cj = vTxj . . . . . . . . . . . 25 vii viii CONTENTS 3 Traditional Demographic Methods 27 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 3.2 Symptomatic Accounting Techniques . . . . . . . . . . . . . . . 28 3.2.1 Vital Rates Method . . . . . . . . . . . . . . . . . . . . 28 3.2.2 Composite Method . . . . . . . . . . . . . . . . . . . . . 30 3.2.3 Component Methods . . . . . . . . . . . . . . . . . . . . 30 3.2.4 Housing Unit Method . . . . . . . . . . . . . . . . . . . 30 3.3 Regression Symptomatic Procedures . . . . . . . . . . . . . . . 31 3.3.1 Ratio Correlation and Difference Correlation Methods . 31 3.3.2 Sample Regression Method . . . . . . . . . . . . . . . . 33 3.4 Dual-system Estimation of Total Population . . . . . . . . . . . 37 3.4.1 Dual-system Model . . . . . . . . . . . . . . . . . . . . . 37 3.4.2 Post-enumeration Surveys . . . . . . . . . . . . . . . . . 39 3.5 Derivation of Average MSEs . . . . . . . . . . . . . . . . . . . . 42 4 Indirect Domain Estimation 45 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 4.2 Synthetic Estimation . . . . . . . . . . . . . . . . . . . . . . . . 46 4.2.1 No Auxiliary Information . . . . . . . . . . . . . . . . . 46 4.2.2 Auxiliary Information Available . . . . . . . . . . . . . . 46 4.2.3 Regression-adjusted Synthetic Estimator . . . . . . . . . 51 4.2.4 Estimation of MSE . . . . . . . . . . . . . . . . . . . . . 51 4.2.5 Structure Preserving Estimation . . . . . . . . . . . . . 5 3 4.3 Composite Estimation . . . . . . . . . . . . . . . . . . . . . . . 57 4.3.1 Optimal Estimator . . . . . . . . . . . . . . . . . . . . . 57 4.3.2 Sample Size Dependent Estimators . . . . . . . . . . . . GO 4.4 James-Stein Method . . . . . . . . . . . . . . . . . . . . . . . . 63 4.4.1 Common Weight . . . . . . . . . . . . . . . . . . . . . . 63 4.4.2 Equal Variances $, = . . . . . . . . . . . . . . . . . . 64 $J 4.4.3 Estimation of Component MSE . . . . . . . . . . . . . . 68 4.4.4 Unequal Variances 4% . . . . . . . . . . . . . . . . . . . 7 1 4.4.5 Extensions . . . . . . . . . . . . . . . . . . . . . . . . . 71 4.5 Proofs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 5 Small Area Models 75 5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 5.2 Basic Area Level (Type A) Model . . . . . . . . . . . . . . . . 76 5.3 Basic Unit Level (Type B) Model . . . . . . . . . . . . . . . . . 78 5.4 Extensions: Type A Models . . . . . . . . . . . . . . . . . . . . 81 5.4.1 Multivariate Fay-Herriot Model . . . . . . . . . . . . . . 81 5.4.2 Model with Correlated Sampling Errors . . . . . . . . . 82 5.4.3 Time Series and Cross-sectional Models . . . . . . . . . 83 5.4.4 Spatial Models . . . . . . . . . . . . . . . . . . . . . . . 86 5.5 Extensions: Type B Models . . . . . . . . . . . . . . . . . . . . 87 5.5.1 Multivariate Nested Error Regression Model . . . . . . . 87 5.5.2 Random Error Variance Linear Model . . . . . . . . . . 88 CONTENTS ix 5.5.3 Two-fold Nested Error Regression Model . . . . . . . . 88 5.5.4 Two-level Model . . . . . . . . . . . . . . . . . . . . . . 89 5.5.5 General Linear Mixed Model . . . . . . . . . . . . . . . 90 5.6 Generalized Linear Mixed Models . . . . . . . . . . . . . . . . . 91 5.6.1 Logistic Regression Models . . . . . . . . . . . . . . . . 91 5.6.2 Models for Mortality and Disease Rates . . . . . . . . . 92 5.6.3 Exponential Family Models . . . . . . . . . . . . . . . . 93 5.6.4 Semi-parametric Models . . . . . . . . . . . . . . . . . . 93 6 Empirical Best Linear Unbiased Prediction: Theory 95 6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 6.2 General Linear Mixed Model . . . . . . . . . . . . . . . . . . . 96 6.2.1 BLUP Estimator . . . . . . . . . . . . . . . . . . . . . . 96 6.2.2 MSE of BLUP . . . . . . . . . . . . . . . . . . . . . . . 98 6.2.3 EBLUP Estimator . . . . . . . . . . . . . . . . . . . . . 99 6.2.4 ML and REML Estimators . . . . . . . . . . . . . . . .1 00 6.2.5 MSE of EBLUP . . . . . . . . . . . . . . . . . . . . . . 103 6.2.6 Estimation of MSE of EBLUP . . . . . . . . . . . . . .1 04 6.2.7 Software . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 6.3 Block Diagonal Covariance Structure . . . . . . . . . . . . . . .1 07 6.3.1 EBLUP Estimator . . . . . . . . . . . . . . . . . . . . . 107 6.3.2 Estimation of MSE . . . . . . . . . . . . . . . . . . . . . 108 6.3.3 Extension . . . . . . . . . . . . . . . . . . . . . . . . . . 110 6.3.4 Model Diagnostics . . . . . . . . . . . . . . . . . . . . . 110 6.4 Proofs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 6.4.1 Derivation of BLUP . . . . . . . . . . . . . . . . . . . . 112 6.4.2 Equivalence of BLUP and Best Predictor E(mTvlATy) 113 6.4.3 Derivation of the Decomposition (6.2.26) . . . . . . . . 113 7 EBLUP: Basic Models 115 7.1 Basic Area Level Model . . . . . . . . . . . . . . . . . . . . . . 115 7.1.1 BLUP Estimator . . . . . . . . . . . . . . . . . . . . . . 116 7.1.2 Estimation of . . . . . . . . . . . . . . . . . . . . . . 118 0; 7.1.3 Relative Efficiency of Estimators of . . . . . . . . . . 1 20 0,” 7.1.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 121 7.1.5 MSE Estimation . . . . . . . . . . . . . . . . . . . . . . 128 7.1.6 Conditional MSE . . . . . . . . . . . . . . . . . . . . . . 131 7.1.7 Mean Product Error of Two Estimators . . . . . . . . . 1 32 7.1.8 Estimation of Small Area Means . . . . . . . . . . . . . 133 7.1.9 Weighted Estimator . . . . . . . . . . . . . . . . . . . . 134 7.2 Basic Unit Level Model . . . . . . . . . . . . . . . . . . . . . . 134 7.2.1 BLUP Estimator . . . . . . . . . . . . . . . . . . . . . . 135 7.2.2 Estimation of 0,”a nd u: . . . . . . . . . . . . . . . . . .1 38 7.2.3 MSE of EBLUP . . . . . . . . . . . . . . . . . . . . . . 139 7.2.4 MSE Estimation . . . . . . . . . . . . . . . . . . . . . . 140 7.2.5 Non-negligible Sampling Rates . . . . . . . . . . . . . .1 41 X CONTENTS 7.2.6 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 142 7.2.7 Pseudo-EBLUP Estimation . . . . . . . . . . . . . . . .1 48 8 EBLUP: Extensions 153 8.1 Multivariate Fay-Herriot Model . . . . . . . . . . . . . . . . . .1 53 8.2 Correlated Sampling Errors . . . . . . . . . . . . . . . . . . . . 155 8.3 Time Series and Cross-sectional Models . . . . . . . . . . . . .1 58 8.3.1 Rao-Yu Model . . . . . . . . . . . . . . . . . . . . . . . 158 8.3.2 State Space Models . . . . . . . . . . . . . . . . . . . . 162 8.4 Spatial Models . . . . . . . . . . . . . . . . . . . . . . . . . . . 168 8.5 Multivariate Nested Error Regression Model . . . . . . . . . . . 1 69 8.6 Random Error Variances Linear Model . . . . . . . . . . . . . . 1 71 8.7 Two-fold Nested Error Regression Model . . . . . . . . . . . . . 1 72 8.8 Two-level Model . . . . . . . . . . . . . . . . . . . . . . . . . . 176 9 Empirical Bayes (EB) Method 179 9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 9.2 Basic Area Level Model . . . . . . . . . . . . . . . . . . . . . . 180 9.2.1 EB Estimator . . . . . . . . . . . . . . . . . . . . . . . . 181 9.2.2 MSE Estimation . . . . . . . . . . . . . . . . . . . . . . 182 9.2.3 Approximation to Posterior Variance . . . . . . . . . . . 1 85 9.2.4 EB Confidence Intervals . . . . . . . . . . . . . . . . . .1 91 9.3 Linear Mixed Models . . . . . . . . . . . . . . . . . . . . . . . . 194 9.3.1 EB Estimation . . . . . . . . . . . . . . . . . . . . . . . 194 9.3.2 MSE Estimation . . . . . . . . . . . . . . . . . . . . . . 195 9.3.3 Approximations to the Posterior Variance . . . . . . . . 1 96 9.4 Binary Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197 9.4.1 Case of no Covariates . . . . . . . . . . . . . . . . . . . 197 9.4.2 Models with Covariates . . . . . . . . . . . . . . . . . . 202 9.5 Disease Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . 205 9.5.1 Poisson-Gamma Model . . . . . . . . . . . . . . . . . .2 06 9.5.2 Log-normal Models . . . . . . . . . . . . . . . . . . . . . 208 9.5.3 Extensions . . . . . . . . . . . . . . . . . . . . . . . . . 209 9.6 Triple-goal Estimation . . . . . . . . . . . . . . . . . . . . . . . 211 9.6.1 Constrained EB . . . . . . . . . . . . . . . . . . . . . . 211 9.6.2 Histogram . . . . . . . . . . . . . . . . . . . . . . . . . . 213 9.6.3 Ranks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213 9.7 Empirical Linear Bayes . . . . . . . . . . . . . . . . . . . . . . 214 9.7.1 LB Estimation . . . . . . . . . . . . . . . . . . . . . . . 214 9.7.2 Posterior Linearity . . . . . . . . . . . . . . . . . . . . 217 9.8 Constrained LB . . . . . . . . . . . . . . . . . . . . . . . . . . . 219 9.9 Proofs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220 9.9.1 Proof of (9.2.11) . . . . . . . . . . . . . . . . . . . . . . 220 9.9.2 Proof of (9.2.30) . . . . . . . . . . . . . . . . . . . . . . 221 9.9.3 Proof of (9.6.6) . . . . . . . . . . . . . . . . . . . . . . . 221 9.9.4 Proof of (9.7.1) . . . . . . . . . . . . . . . . . . . . . . . 222