Carson Block, Carson Block Consulting
Measuring & Monitoring Internet Service
Testing the speed and quality of the tribal library’s internet service can help build understanding of:
- The general performance of the internet connection
- Whether the library’s internet connection is performing as expected
- Potential issues with the library user’s experience using the network [1]
By running one or more internet measurement tests once or several times per day and keeping records of the measurements, a tribal library is taking action to collect data that can be useful:
- In conversations with the library’s Internet Service Provider or tribal IT
- To demonstrate the need for improving internet service in grant applications, board presentations, etc.
- To compare the library’s internet service with other institutions nearby, and with home internet service in the community
Options for Measuring Internet Service
The US Federal Communications Commission (FCC) uses the speed of an internet connection to identify whether it is “broadband”. The current definition of broadband is 100 Mbps download and 20 Mbps upload.[2] But speed isn’t the only aspect of an internet connection that tribal libraries should think about when it comes to measuring their internet service. Recent developments among internet researchers and the FCC have identified latency as an important measurement to consider as well.[3] [4] For example, if an internet connection has high download and upload speeds, but also high latency and buffering, people using the internet may still experience problems.
Depending on what goals the tribal library has for measuring internet service, staff may decide to use one or several tests. Here are a few popular tests and a brief description of what they measure:
Tests used to measure internet service
Company | Test | Use Cases |
Ookla | Speedtest.net | Measure service from the tribal library to the edge of the ISP’s network. This is the best test to compare with advertised speeds because the test methodology is most closely aligned with typical ISP terms of service. |
Measurement Lab (M-Lab) | Network Diagnostic Tool (NDT) | Measure service from the tribal library to internet exchanges and Google Cloud locations. This test can be helpful to identify whether an ISP’s network is well connected to the internet. |
Waveform | Bufferbloat and Internet Speed Test and Bufflerbloat Grade | Measure the tribal library’s internet speed, and latency under load. This test is designed to identify “bufferbloat” and grades the connection based on speeds and latency. |
Cloudflare | Speed Test and Network Quality Score | Similar test to Waveform, but measures to servers in Cloudflare’s Content Delivery Network (CDN). Network Quality Score rates the connection for use with video streaming, online gaming, and video chatting. |
When choosing options for measuring internet service, tribal libraries should consider their goals for measuring and their available resources to conduct measurements. The tests themselves should provide measurements that align with the intended goal.
In addition to selecting the right measurements, the library should also consider how frequently to run them, if and when the process should be repeated, and consider whether continuous automated measurements would help achieve the library’s goals. Here are a few examples. A series of measurements before and after an ISP upgrade of service could provide data to confirm the change. An annual assessment combined with data about the number of people using the library’s Wi-Fi or public computers could be used to determine if additional bandwidth is needed in the next year. Continuous automated measurement could be helpful to monitor the connection and provide a source of data to confirm the impact of network outages or to diagnose issues reported by library users.
Best Practices for Measuring Broadband Service
- Consult with any IT staff managing or providing the internet connection to the library about the measurement initiative. This can help build shared understanding of the library’s need for measurement data to support its mission. IT staff may have additional suggestions, could have existing monitoring data to support the library’s need, or could help implement changes to network configurations or equipment if needed.
- Run internet measurement tests when no one else is using the connection, such as early in the morning before the library opens for the public and before other staff might be using the connection.
- Test the connection using a computer connected by an ethernet cable – a wired connection instead of wireless.
- The testing computer should be connected to the broadband router itself, or to a switch as near to the router as possible.
- Consider repeating this once per day over a week or a month, and taking the average or median of the measured values.[5]
- Consider dedicating a measurement computer to run tests automatically, using one of the suggested methods below, Automatically Running Tests.
- Measure and Mitigate High Latency with Active/Smart Queue Management as outlined in the final section of this chapter.
Running Tests Manually
To run tests manually, tribal library staff would connect a computer or laptop to the router or switch, open the tests web pages one at a time, run each test, and record the results.
Automatically Running Tests
There are different ways to automatically run internet measurement tests, from commercial services to a “do-it-yourself” (DIY) approach. Using a system to automatically run tests gives the tribal library ongoing measurements without the need to dedicate staff time, provides ongoing monitoring of the library’s connection, and allows the library to see trends and anomalies in service that might not be seen in single tests. Below are some options for both commercial and Do-It-Yourself (DIY) systems to automatically run tests.
Netrics
Netrics is an open source product, built and maintained by academic researchers at the University of Chicago’s Internet Equity Initiative. This DIY system can be used by anyone, including for commercial purposes. Netrics runs these internet measurement tests: M-Lab’s NDT, Ookla’s Speedtest.net, bufferbloat, DNS latency, latency/speed tests to a list of user-entered websites. Netrics can also be extended by adding other tests to their open framework. The system also can be used to assess Wi-Fi bottlenecks. To use Netrics, tribal libraries would purchase devices and install the Netrics code on them. As a DIY solution, tribal libraries would also need to allocate staff time to visualize test data collected by each device running Netrics.
Netrics Links:
» About the Internet Equity Initiative: https://internetequity.uchicago.edu/
» Netrics code:
» https://github.com/internet-equity/netrics
» https://github.com/internet-equity/netrics-docker
M-Lab’s Murakami Tool
Murakami is an open source product, built by M-Lab. Murakami is software that runs internet measurement tests using a randomized schedule a configurable number of times a day. Murakami supports M-Lab’s NDT, Ookla’s Speedtest.net, and other tests. It can be configured to export each test result to the computer it runs on, to a local or remote server, or to Google Cloud Storage. As a DIY solution, tribal libraries would need to allocate staff time to visualize test data collected by each device running Murakami. https://github.com/m-lab/murakami/
Ready
Ready is a commercial service providing internet measurement tests and surveys to state broadband offices and tribal governments. Ready supports M-Lab’s NDT, Ookla’s Speedtest.net, Cloudflare’s Speed Test, and RST. Focused on providing data and mapping internet service for communities to use in broadband service area challenges submitted to the FCC, and for use by states and tribes to support funding applications. https://ready.net/
Comparing Library Broadband Measurements to Other Organizations and Locations
Tribal libraries may find it helpful to compare internet measurements run from the library to other community anchors around them, such as schools, businesses, or tribal government offices. It also is useful to compare internet service at these locations with the service people receive in their homes. Below are links to resources that tribal libraries can use to compare their internet service with others around them.
- The FCC National Broadband Map provides access to two datasets:
» the “Broadband Fabric” – residential, business, and other locations where internet service may be provided;
» and internet “Availability data” – areas of the country where ISPs provide service now or can deliver service.
» States and tribal governments may request special access to the National Broadband Map from the FCC, which allows them to present challenges to the FCC where locations are incorrect, missing, or misclassified; and/or to challenge the ISP submitted availability data presented in the map.
» A commercial service from Ookla providing “up-to-date insights into global fixed broadband and mobile performance”
Measure and Mitigate High Latency with Active/Smart Queue Management
While measuring internet service using speed tests has historically provided support for library advocacy, internet researchers cite “working latency” as “a critical factor in providing a high-performance internet connection.”[6] In plain terms, if a high speed internet connection also has high latency, library users can experience “lag” or buffering. “Bufferbloat is a high latency (or lag) that occurs when there’s other traffic on your network. This means that your network isn’t responsive under normal working conditions.”[7]
First, measure the internet connection’s latency underload using the Waveform Bufferbloat Test or Speedtest.net. The connection likely has bufferbloat if Speedtest.net measures greater than 75 milliseconds latency for either download or upload, or if the Waveform test grades the connection lower than a B.
Network equipment that can manage network buffers intelligently using Active or Smart Queue Management (AQM/SQM) is the recommended solution to bufferbloat. Replacing the library’s router or firewall with one that employs AQM or SQM will enable smart management of all users’ requests on the network that are competing for available bandwidth and working latency. The Bufferbloat.net website recommends several off-the-shelf routers and firewalls that come with AQM/SQM. See https://www.bufferbloat.net/projects/bloat/wiki/What_can_I_do_about_Bufferbloat/ for more details.
[1] Institute for Museum and Library Services, Internet 2 Community Anchor Program, and Carson Block Consulting. “Toward Gigabit Libraries Toolkit.” Toward Gigabit Libraries – Internet 2. January 1, 2024. https://internet2.edu/tgl.
[2] Federal Communications Commission. “FCC INCREASES BROADBAND SPEED BENCHMARK.” FCC News. March 14, 2024. https://docs.fcc.gov/public/attachments/DOC-401205A1.pdf.
[3] Ohlsen, Lai Yi. “How Should Internet Quality Be Measured? Join Us for Our Community Call on February 1, 2024.” Measurement Lab. January 23, 2024. https://www.measurementlab.net/blog/feb24-community-call/.
[4] “FCC FACT SHEET – Inquiry Concerning Deployment of Advanced Telecommunications Capability to All Americans in a Reasonable and Timely Fashion 2024 Section 706 Report, GN Docket No. 22-270.” Federal Communications Commission. February 22, 2024. https://docs.fcc.gov/public/attachments/DOC-400675A1.pdf.
[5] Institute of Museum and Library Services, Internet 2 Community Anchor Program, and Carson Block Consulting. “Toward Gigabit Libraries Toolkit.” Toward Gigabit Libraries – Internet 2. January 1, 2024. https://internet2.edu/tgl.
[6] “Latency Explained.” Broadband Internet Technical Advisory Group (BITAG), January 10, 2022. https://bitag.org/documents/BITAG_latency_explained.pdf.
[7] “What Can I Do About Bufferbloat?” Bufferbloat.Net, October 20, 2023. https://www.bufferbloat.net/projects/bloat/wiki/What_can_I_do_about_Bufferbloat/. service may be provided; and internet “Availability data” – areas of the country where ISPs provide service now or can deliver service.
» States and tribal governments may request special access to the National Broadband Map from the FCC, which allows them to present challenges to the FCC where locations are incorrect, missing, or misclassified; and/or to challenge the ISP submitted availability data presented in the map.
» Ookla Speedtest Intelligence service A commercial service from Ookla providing “up-to-date insights into global fixed broadband and mobile performance”
Measure and Mitigate High Latency with Active/Smart Queue Management
While measuring internet service using speed tests has historically provided support for library advocacy, internet researchers cite “working latency” as “a critical factor in providing a high performance internet connection.”[6] In plain terms, if a high speed internet connection also has high latency, library users can experience “lag” or buffering. “Bufferbloat is high latency (or lag) that occurs when there’s other traffic on your network. This means that your network isn’t responsive under normal working conditions.”[7]
First, measure the internet connection’s latency under load using the Waveform Bufferbloat Test or Speedtest.net. The connection likely has bufferbloat if Speedtest.net measures greater than 75 milliseconds latency for either download or upload, or if the Waveform test grades the connection lower than a B.
Network equipment that can manage network buffers intelligently using Active or Smart Queue Management (AQM/SQM) is the recommended solution to bufferbloat. Replacing the library’s router or firewall with one that employs AQM or SQM will enable smart management of all users’ requests on the network that are competing for available bandwidth and working latency. The Bufferbloat.net website recommends several off-the-shelf routers and firewalls that come with AQM/SQM. See https://www.bufferbloat.net/projects/bloat/wiki/What_can_I_do_about_Bufferbloat/ for more details.
[1] Institute for Museum and Library Services, Internet 2 Community Anchor Program, and Carson Block Consulting. “Toward Gigabit Libraries Toolkit.” Toward Gigabit Libraries – Internet 2. January 1, 2024. https://internet2.edu/tgl.
[2] Federal Communications Commission. “FCC INCREASES BROADBAND SPEED BENCHMARK.” FCC News. March 14, 2024. https://docs.fcc.gov/public/attachments/DOC-401205A1.pdf.
[3] Ohlsen, Lai Yi. “How Should Internet Quality Be Measured? Join Us for Our Community Call on February 1, 2024.” Measurement Lab. January 23, 2024. https://www.measurementlab.net/blog/feb24-community-call/.
[4] “FCC FACT SHEET – Inquiry Concerning Deployment of Advanced Telecommunications Capability to All Americans in a Reasonable and Timely Fashion 2024 Section 706 Report, GN Docket No. 22-270.” Federal Communications Commission. February 22, 2024. https://docs.fcc.gov/public/attachments/DOC-400675A1.pdf.
[5] Institute of Museum and Library Services, Internet 2 Community Anchor Program, and Carson Block Consulting. “Toward Gigabit Libraries Toolkit.” Toward Gigabit Libraries – Internet 2. January 1, 2024. https://internet2.edu/tgl.
[6] “Latency Explained.” Broadband Internet Technical Advisory Group (BITAG), January 10, 2022. https://bitag.org/documents/BITAG_latency_explained.pdf.
[7] “What Can I Do About Bufferbloat?” Bufferbloat.Net, October 20, 2023. https://www.bufferbloat.net/projects/b