August 12, 2010

Problems associated with surveying small demographic groups

In the course of discussing our recent update on home broadband adoption, I was asked on Twitter why Pew Internet doesn’t provide data for Asian / Pacific Islanders as part of our standard demographic splits. Several years ago we asked our lead pollster, Evans Witt (principal and CEO of Princeton Survey Research Associates International) to provide a survey methodologist’s take on this question. His response hopefully sheds some light on the challenges associated with polling the Asian population in the U.S.:

The short answer is that Asian-Americans make up a very small slice of the population, 3.7 percent in the 2000 Census. In addition, for a good portion of that population, there are complex language barriers. Since most of Pew Internet’s work is done only in English [ed. note: we have included a Spanish-language option on several recent studies, and will do so as a standard deliverable on all future research], all language barriers reduce the number of completes with the non-English speaking minorities. The diversity of the Asian-American population and the languages they speak makes offering interviews in those native languages very difficult and very, very expensive.

Typically, an RDD survey will obtain interviews with between 1.3 percent and 2.3 percent of respondents who describe themselves as Asian. For a survey of 2,000 completes, that translates into 26 to 46 interviews. That is far too few to analyze with any statistical accuracy.

So where should folks go for this information if not us? One great place to start is the NTIA, who recently sponsored a special internet use supplement to the Census Bureau’s Current Population Survey. With a total sample size of around 54,000 households, their data set offers the ability to conduct statistically valid measurements of not just Asian-Americans but other relatively small groups such as Native Americans. Their most recent findings are available at http://www.ntia.doc.gov/reports/2010/NTIA_internet_use_report_Feb2010.pdf.

Another potential resource is a recent survey by the Public Policy Institute of California. While not national in scope, it does provide racial/ethnic comparisons for Asians. The PPIC survey can be found at http://www.ppic.org/content/pubs/jtf/JTF_DigitalDivideJTF.pdf.

For researchers with a specific interest in young adults, Eszter Hargittai’s surveys of incoming University of Illinois-Chicago students offer another valuable resource. Conducted in 2007, 2009 and 2010, these surveys also offer a large enough Asian-American respondent pool to enable robust analysis within this group. For more information, see http://webuse.org/pubs/.