Agency rulemaking decisions must be grounded on a sound factual basis. Studies on which an agency relies must be based on accurate facts, for even a perfect methodology will produce useless results if applied to erroneous data. It has now become clear that the report prepared by Dr. Marc Rysman for use in this rulemaking, as well as nearly all other analyses submitted into the record, were based on an irretrievably flawed data set that severely understated cable providers’ ability to provision true business data services (“BDS”). The record now shows that the major cable providers were able to provide Metro Ethernet – not what the Commission calls “best efforts” service – in 22 times as many census blocks in 2013 as was reflected in the original data set on which the Rysman Report and many other analyses were based. The Administrative Procedure Act, the Data Quality Act, and bedrock principles of evidence require that these materials be stricken from the record. Indeed, given the central role the Rysman Report plays in the Commission’s proposals, the agency should rescind the aspects of the May 2 FNPRM that cited or relied upon them, allowing the Commission and parties to conduct new analyses reflecting accurate data regarding the state of the marketplace. The Commission should then develop and seek comment on new proposals as appropriate.
The Commission has for years placed its mandatory data collection at the heart of its efforts to reform the BDS regulatory regime, repeatedly underscoring its interest in a data-driven framework reflecting the true state of competitive deployment. Chairman Wheeler, Acting Chairwoman Clyburn, and Chairman Genachowski all touted the data collection’s centrality to this proceeding. The May 2 FNPRM fulfilled these promises – its analysis and proposals were based entirely on the initial data set and analyses of that data set.
It recently became apparent, however, that the original data set analyzed by Dr. Rysman and others had radically undercounted the availability of true Ethernet cable offerings. Indeed, data placed into the record just last week show that the four largest cable providers in 2013 were able to provide Metro Ethernet in 22 times as many census blocks as had previously been reported. The Commission and the Wireline Competition Bureau have thus far downplayed or misunderstood the significance of the flawed data set, but there is no basis for doing so: The record now shows that the largest cable providers had upgraded their facilities to provide Metro Ethernet service (which the Commission has been clear is a true substitute for ILEC BDS) virtually everywhere in 2013. Even absent legal compulsion, the Commission should recognize that the public would be best served by granting the relief sought herein. Here, though, the Commission also does face an overwhelming legal compulsion to grant this relief.
First, the Administrative Procedure Act (“APA”) demands that the evidence at issue be stricken from the record. In the D.C. Circuit’s words, “[i]t is not consonant with the purpose of a rule-making proceeding to promulgate rules on the basis of inadequate data.” Thus, courts have regularly invalidated agency actions that are based on flawed data or studies, or that lack a valid factual basis. The Commission itself has often been reversed for reliance on flawed evidence or analyses, in contexts ranging from the cable subscribership cap to the computation of the pricecap “X Factor” to the interference standards for broadband-over-powerline to broadcast ownership limits. These precedents make clear that the Commission may not rely on the Rysman Report or other analyses based on the original data set, which understated cable providers’ ability to provision Metro Ethernet service by a factor of more than 20. Moreover, the Bureau’s attempt to respond to this point in its June 8 Extension Denial utterly fails: The Bureau wrongly assumed that the new data concerned so-called best efforts service, and was thus (under the Commission’s theory) not pertinent to the agency’s analysis. In fact, as discussed herein, the new data’s most important revelation concerns true Metro Ethernet service. Nor is there any basis for the Bureau’s suggestion that parties can adequately analyze the new data set in the three weeks between when it was made available for review and when comments are due: It took Dr. Rysman himself six months to analyze the original data set, even with help from Commission staff. Simply put, the APA precludes reliance on the Rysman Report and similar analyses, and requires that parties be provided meaningful opportunity to review the new data and any Commission analyses or proposals based on those data.
Second, reliance on the Rysman Report and related studies is barred by the Data Quality Act (“DQA”). That statute requires the Commission to ensure that it maximizes the “quality, objectivity, utility, and integrity of information (including statistical information)” on which it relies in its decisions and demands that studies on which the agency relies be subjected to peer review. The Commission has declared that it is “dedicated to ensuring that all data it disseminates reflect a level of quality commensurate with technical information” and information and analyses on which it relies “shall be generated [and] developed using sound statistical and research methods.” It should go without saying that analyses accounting for only one out of every 22 census blocks in which ILECs’ principal competitors in the BDS marketplace were able to provide Metro Ethernet service in 2013 are fundamentally unsound. In addition, the Rysman Report has not been peer reviewed. While the Commission has committed to such review, it is not clear when it will be completed, or whether parties will be afforded a meaningful opportunity to comment on the review(s), as the Commission previously has recognized they must. The DQA thus bars reliance on the materials at issue.
Third, the materials at issue here would be inadmissible in federal court under the Federal Rules of Evidence, and should be excluded here as well. The courts have held that an expert opinion based on inaccurate facts is inadmissible under those rules, even where the methodology would be sound if applied to accurate data, because (in the Supreme Court’s words) expert testimony must be “sufficiently tied to the facts of the case,” such that it will assist the trier of fact to understand the evidence or determine a fact in issue. The principles reflected by the rules and court decisions applying them forbid introduction of the Rysman Report and similar analyses here. These materials obviously are not based on sufficient facts or data and are have been rendered unreliable by the underlying data on which they are premised. While the Commission is not a federal court, it has consulted the Federal Rules of Evidence in previous rulemaking proceedings, and should follow their guidance here.
For the reasons discussed above, the foundation on which the Commission intends to set its BDS framework is irredeemably flawed. The Commission can and must take remedial action to ensure that whatever regime it adopts serves the public interest and can withstand legal scrutiny. For the reasons presented herein, the Commission should (1) strike from the record the Rysman Report and other studies based on the flawed data set; (2) rescind the portions of the FNPRM that have been compromised by reliance on the Rysman Report and the flawed data set; (3) prepare or commission a new analysis to replace the Rysman Report, reflecting the corrected data set, and allow parties to do the same; and (4) put those new proposals and analyses out for comment.