Broadband speed tests conducted by research firm comScore, cited by the FCC in its National Broadband Plan, aren't always accurate according to Netforcast, a network engineering consulting firm. The FCC cited comScore's broadband speed tests to justify a statement that actual broadband speeds are only 40-50% of speeds advertised by service providers.
comScore's results are derived from a group of survey participants. Each participant has a comScore speed testing application installed on their computer which takes random samples of broadband speeds by downloading a file ranging from 1 MB to 15 MB. The application then calculates the size of the file divided by the time it took to download to determine the broadband speed.
But Netforcast has uncovered several errors in comScore's testing methodology that create lower broadband speed results. Those errors include the fact that most broadband connections are shared via wi-fi, with many systems and devices within a home accessing a broadband connection simultaneously. If one of those systems is using the broadband connection at the same time as comScore's testing, a lower speed would result due to the sharing of the broadband connection. comScore's tests do not compensate for this variable.
The comScore testing also "severely limits the accuracy of its results" because it only uses a single TCP connection for each test. Netforcast contends that most speed testing services use multiple TCP connections.
comScore's test results may also be flawed due to a fundamental miscalculation of the size of a megabyte of data. Netforcast believes that comScore may be calculating a megabyte at one million bytes of data rather than actual size of 1,048,576 bytes per megabyte. Such an error would result in a miscalculation of broadband speeds by 4.5 percent in every case.