Friday, May 12, 2006

Study a Study!

In the last week, there have been two 'loss statements' made by two prominent industry bodies. On May 3rd, the MPAA issued a press release saying they lost $6.1billion in 2005 to 'pirates' whilst on the 10th, the ESA's Chunnie Wright, senior anti-privacy council, announced that the videogames industry had lost more than $3Billion.

Staggering figures – some $9 billion dollars (£4.8 billion) lost, stolen, if you will, from their coffers. Was it though? How was that figure arrived at? These we don't know. The problems are three-fold
  • how the figures are calculated – I've never found the determination of the number of units given in any one of these press releases, citing loses, and costs to this industry, or that industry. With no conclusive determination of the global units distributed in all mediums, you can't get anywhere near a reasonable determination of total units. The segment you use might be representative (ie average) and so can be scaled up to match the world, or it could be non-representative, either quieter than the norm, meaning the study is understating, or they've gotten a particularly high concentration of infringement in their sample, which makes their study overstate the case.
  • How the cost price is given. DVDs, movie tickets, computer games, CDs, they all cost different amounts. A Star Trek box-set can cost $99 whereas a store like Walmart will have other DVDs on sale for $5.50. There is also the different pricing between US and UK, just for the same film. On the DVD is available for £9.09 ($16.96) whilst on the UK site, its available for £11.96 ($22.30) – that's more than $5 difference for the same film, by the same company in different areas. So when calculating the lost price, is it all the at highest price, the lowest price, or are they just assigning an arbitrary value for these discs.
  • Thirdly, the assumption that a download is a lost sale. This isn't the case, and has never been. Its a cost analysis. If I (or anyone else) like a movie, TV show, or game, we will pay for it and own it, providing and here's the rub, the cost isn't greater than our interest in the product. Add that into the quality problems and that of of product compatibility, you have a large problem. Its hard to find objective reviews of a film, tv series, game or application nowadays. If it were a car, I would be able to go to the place i was buying, test drive one, see if its what I want, and if not, look at something else. Then when I buy it, if I'm not satisfied with it, I can refuse it and get my money back. If are considering Mission Impossible 3, what have you to go on – the advert/trailer (which only shows the best bits of the film, and sometimes bits not even in the film) a review featuring as much personal relevance as the reviewers medical most times. And nothing else. If I then pay my money and see it and am not happy with the film, finding it a turgid rehash of previous films with sub-standard plot, can I get my money back? Nope. Such trial methods can be successful in software, but has become more of a lost art in games, and a downloaded copy, can be as much a validation that is not what was wanted (a potential lost sale, depending on level of initial interest) or if it was a potential buyer encouraged by what they saw to buy it (or a sale creation, rather than one lost)

That, in a rub, are the main problems facing the movie, music and software industries and their studies at present. We don't get to see the studies. We don't get to see how these facts, as ascertained by the study's commissioners, are arrived at. A friend of mine (the my editor and sometime proofreader) is a fan of military vehicles of all types. If I then said to him that I could make a new tank gun for only £200 that would defeat any amour out there, he'd ask him for proof of his claim. Heck, if I said I could do one for £200,000 he'd still ask to see how I arrived at my determination. There's been a big fuss in the US over the last few years, about the WMDs in Iraq. People are asking to see Bush's evidence now, and questioning the blind assentation. Questioning statements of fact (and sometimes statements by FACT) until sufficient proof has been rendered is the basis of progress, and civilisation. Fermat's last theorum was an assentation without proof, and people fussed over it for hundreds of years.
So, what's to be done? Well, I've done all that can be done, and sent a request t the MPAA asking if I can see how LEX came by their report's conclusion. After all, a 1½ page report, mostly filled with quotes is hardly informative. Until then, use your brain, and trust the facts that they can actually substantiate with proof.

“But Wait!” I hear the MPAA say, The data is released – and what good data it is too. Let me summarise it for you in much less than 1½ pages.
Pg1) Statement by Dan Glickman, a statement that the data is to be used to prod governments in a direction for lawmaking, a statement that previous studies were incomplete, and that one other study made an estimate of some losses, and finally, 31 words as to how the study was conducted and its comparison to previous studies
Pg2) Another Glickman statement, a summery of the findings of the study, and a graph showing the US percentage, and the MPAA's definition of terms (including that they consider fair use copies to be 'illegal'
Pg3) 'Chart of calculated losses in countries, to both MPAA members and non-MPAA members. Then another raph, this time showing the study's estimation of the market lost to piracy.
Pg4) Another graph showing MPA members and their estimated losses, with a caveat that different markets were analysed differently, and then a table showing guesses for losses in tax revenue in certain countries to different reasons.
Pg5) The really weird stuff. Their profile of a pirate. Basically, they identify the most common trends in the data they collected, by age, by gender, and then by a mathematically incomplete graph about college students.
Pg6) A timeline showing certain selected MPAA enforcement activities. This is followed bya list of 'education' programs which have almost without limit, bent and twisted the truth.
Pg7) Certain (again selective) highlights of litigation and legislative efforts, plus a list of new rights enforcement methods that are being looked into

that's it. Not a single explanation of how ANY data was actually collected, information about sample sizes, statistical variation, or grouping. Two interesting things to note though. On page 4, it was noted that some of the 22 countries were surveyed in different ways (which I would generally think means its non-comparative). Its nice to see them actually having the guts to remark that some of their data is nonsense, even if it is in passing, and then ignored. The second is an inference of mine based on what's written on page 5. It seems to be that they say you're more likely to pay to see their films, if you're not at college. Are they drawing some sort of link between a persons education level, and the perceived quality and value of a movie?

Ben jones

External links
MPAA's press release statement on the study 78kb PDF
MPAA's summery of the study, as described above 118kb PDF
report of ESA's speech at E3

(Footnote – the currency conversion used was the rate of 1 GBP = 1.86432 USD as given by for May 10th)

1 comment:

  1. Then another raph, this time showing the study's estimation of the market lost to piracy.