April 6, 2005

We Stand By Our Data

The Pew Internet & American Life Project is seeking a correction to two stories which appeared regarding our recent podcasting data memo.

Newsfactor and Washingtonpost.com not only mischaracterized our data, but suggested that we have now “backtracked.” To the contrary, we stand by our data and urge anyone concerned about it to examine the actual survey question and to read the data memo.

Some have noted that our question wording did not make the distinction between getting an MP3 file via RSS or by learning about the file and downloading it another way.

All respondents in our survey were asked, “Do you have an iPod or other MP3 player that stores and plays music files, or do you not have one of these?”

  • 11% of American adults said yes
  • 88% of American adults said no
  • 1% of American adults said they don’t know

Of the 208 survey respondents who answered yes, we asked, “Have you ever downloaded a podcast or internet radio program so you could listen to it on your digital audio player at a later time?”

  • 29% of iPod or MP3 player owners said yes
  • 70% of iPod or MP3 player owners said no
  • 1% of iPod or MP3 player owners said they don’t know

As we crafted this question, we debated the wording and decided not to be very technical in our description of podcasting because technical language often creates its own problems and misunderstandings in phone surveys like ours. Our intention was to see how many people had done something that could be functionally defined as podcasting, even though it didn’t specifically use all the technical specifications that some people use to download podcasts. Thus, we decided we did not want to limit the question only to those who use RSS to learn about and capture the files. It was more useful, we thought, to learn about people time-shifting the consumption of audio files in the way they use their MP3 players, rather than risk confusion with a question that was pretty technical.

We also thought that the likelihood that people would say “yes” to our podcasting question even though they might only have streamed audio was very small because this question specifically used the word “podcast” and came immediately after a question asking about ownership of iPods and MP3 players.

One other concern about our report relates to our sample and margin of error. We felt that this was a decent enough representative sample of iPod/MP3 owners that it was worth reporting, even if it does have a high margin of error. This margin-of-error means, of course, that those who think we got too high a number could be right because the low end of our margin of error suggests that the podcast-downloader number might be as much as 25% lower than we reported. That is a legitimate issue that we should be more careful to highlight. In future reports we issue of this nature, we will explain more fully what the range of answers might mean.

We also thought that it was useful to report these findings even on a relatively small sample because there is growing interest in podcasting and we at the Pew Internet Project are being asked more frequently if we have any data on this subject. We thought these numbers were legitimate to report because they are the first from a national sample to take a stab at defining the podcast-user universe. We plan to keep sampling on this subject because it’s an important new internet application and because our guess is that the incidence of podcasting and consumption of podcasts will grow over time as more people purchase MP3 players and more organizations take advantage of the ease and low expense of creating podcasts. It was good to try now to capture a baseline reading of the general phenomenon, even if we did it in a way that doesn’t limit itself to a technical description of podcatching through RSS aggregators.