Download from the the Registry of Open Data on AWS

19 Oct 2020 ᛫ 2 min read

SHARE:

Recently, Ookla released Global Fixed and Mobile Network Performance Maps based on Speedtest on the Registry of Open Data on AWS. There’s actually quite a lot of data in the said registry and they can easily be downloaded to your machine, if needed. Here’s how:

Download and install the AWS CLI on your machine

The first thing you need is to install the AWS CLI on your machine: Windows, MacOS, or Linux.

Once you’ve installed the AWS CLI on your machine, you should be able to access the data using simple terminal commands.

Access the data from the terminal

First, you need to know where the data is on AWS. Thankfully, you can easily find this in the homepage of the dataset. Look for AWS CLI Access (No AWS account required).

Registry of Open Data on AWS AWS CLI Access
AWS CLI Access (No AWS account required)

For this dataset, we find:

aws s3 ls s3://ookla-open-data/ --no-sign-request

This command just lists (ls) the contents of the directory. You need some basic terminal command knowledge to download the data you need. I run a Linux machine so I use cp to copy data from a bucket. For me, that command is:

aws s3 cp <file-i-need-to-download> <file-to-store-it-on-my-computer> --no-sign-request

Copy the file you need from the AWS bucket

From the documentation of the Ookla open data, the data files are are “provided in both Shapefile format as well as Apache Parquet, and can be accessed on Amazon Web Services (AWS) object storage service, S3, at the following paths:”

  • s3://ookla-open-data/parquet/performance/
  • s3://ookla-open-data/shapefiles/performance/

using the key names for the type, year, quarter, and data layer.

For example, to access mobile tiles in the parquet format for the second quarter of the year 2020, one would use the bucket,

s3://ookla-open-data/parquet/performance/type=mobile/year=2020/quarter=2/2020-04-01_performance_mobile_tiles.parquet

Since I will be downloading the shapefile version, I will use the bucket:

s3://ookla-open-data/shapefiles/performance/type=mobile/year=2020/quarter=2/2020-04-01_performance_mobile_tiles.zip

So, to download the mobile tiles in the zipped shapefile format for the second quarter of the year 2020, I would use:

aws s3 cp s3://ookla-open-data/shapefiles/performance/type=mobile/year=2020/quarter=2/2020-04-01_performance_mobile_tiles.zip output.zip --no-sign-request

This should start the download process and save the downloaded data in the output.zip file.

Since the downloaded data is a zipped shapefile, I should be able to open it in QGIS.

Ookla Global Fixed and Mobile Network Performance Maps in QGIS
Ookla Global Fixed and Mobile Network Performance Maps

That’s it! There’s a lot of other datasets available in AWS that we can check out, download, and use. Good luck!

Like and follow BNHR on Facebook and Twitter for more #FOSS4G and #QGIS stuff. :)

SHARE:
comments powered by Disqus

You may also like:

Strictly by the Numbers: The dominance of Ateneo in round 1 of #UAAPSeason82

02 Oct 2019 ᛫ 4 min read

Should we be worried about the UP Men's Basketball Team? #UAAPSeason82

26 Sep 2019 ᛫ 5 min read

BNHR

[email protected]

Creative Commons License
Except when explicitly stated otherwise, this work and its contents by Ben Hur S. Pintor is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Other works (software, source code, etc.) referenced in this website are under their own respective licenses.
This site is powered by Jekyll and hosted on Github (view source)