Remote Data Sets

A remote data set is a data set that is stored on one or more remote servers. It may be a single grid file or a collection of subset tiles making up a larger grid. They are not distributed with GMT or installed during the installation procedures. GMT offers several remote global data grids that you can access via our remote file mechanism. The first time you access one of these files, GMT will download the file (or a subset tile) from the selected GMT server and save it to the server directory under your GMT user directory [~/.gmt]. From then on we read the local file from there.

By using the remote file mechanism you should know that these files, on the server, will change from time to time (i.e., new versions are released, a problem in one file is fixed, or a dataset becomes obsolete), and GMT will take actions accordingly. It is our policy to only supply the latest version of any dataset that undergoes revisions. If you require previous versions for your work you will need to get those data from the data provider separately. Unless you deactivate the remote data service, GMT will do the following when you request a remote file in a GMT command:

  1. We check if the locally cached catalog with information about the data available from the server is up-to-date or if it needs to be refreshed. If the file is older that the GMT_DATA_UPDATE_INTERVAL limit then we refresh the catalog.

  2. When the catalog is refreshed, we determine the publication date for each dataset on the server, and if any local copies you may have are now obsolete we will remove them to force a re-download from the server.

Usage

We have processed and reformatted publicly available global data sets (grids and images) and standardized their file names. In GMT, you may access such data (or a subset only by using the -R option) by specifying the special name

@remote_name_rru[_reg]

where the leading @ symbol identifies the file as a remote data set, the remote_name_ is specific to the dataset and the rr code is a 2-digit integer specifying the grid/image resolution in the unit u, where u is either d, m or s for arc degree, arc minute or arc second, respectively. Optionally, you can append _g or _p to specifically get the gridline-registered or pixel-registered version (if they both exist). If reg is not specified we will return the pixel-registered version unless only the gridline-registered file is available. If you do specify a specific registration and that version is not available you will get an error message. The codes for rru and the optional reg that are supported will be listed in the sections below describing each of the available data sets.

Currently, GMT provides the following datasets (with their special names in parentheses)

Many of the remote datasets have a preferred, default color table that will be used unless you override that default by giving your desired CPT information.

Controlling the Process

There are several ways you can control the remote data process and the amount of space taken up by your own server directory:

  1. You can select the GMT data server closest to you to minimize download time [GMT_DATA_SERVER].

  2. You can set an upper limit on the file sizes that may be downloaded [GMT_DATA_SERVER_LIMIT].

  3. You can turn of the automatic download temporarily [GMT_AUTO_DOWNLOAD].

  4. You can control how often GMT will refresh the catalog of information on your computer [GMT_DATA_UPDATE_INTERVAL]

  5. You can clear the server directory, or perhaps just some subsets, any time via gmt clear.

Offline Usage

If you anticipate to be without an Internet connection (or have a very slow one), you can download all (or some) of the remote files prior to losing connection with the module gmtget. You can choose which data to download and limit it to node spacings larger or equal to a limit, and you can minimize space on your computer by requesting that any JPEG2000 tiles not be converted until GMT is accessing them. Here are some examples of usage. Download the entire cache directory used in examples and tests:

gmt get -Dcache

Get all the data for Earth but only for 1 arc minute and coarser, and leave tiles in JPEG2000 format:

gmt get -Ddata=earth -I1m -N

As shown in the tables below, the largest datasets may take some time to download the data from GMT server, so be patient!

File Compression

Typically, a dataset is released by the data provider in a single, high-resolution format. To optimize use of these data in GMT and to prevent download bottlenecks we have downsampled them via Cartesian Gaussian filtering to prevent aliasing while preserving the latitude-dependent resolution in the original grid or image. To improve responsiveness, the larger files (i.e., currently for node spacings 05m and smaller) have been split into smaller tiles. When the 06m or lower resolution files are accessed the first time we download the entire file, regardless of your selected region (-R). However, for the tiled data sets we only download the tiles that intersect your selected region the first time they are referenced.

Single grids are provided as netCDF-4 maximum-lossless compressed short int grids, making the files much smaller than their original source files without any loss of precision. To minimize download speed, the dataset tiles are all stored as JPEG2000 images on the GMT server due to superior compression, but once downloaded to your server directory they are converted to the same short int compressed netCDF4 format for easier access. This step uses our GDAL bridge and thus requires that you have built GMT with GDAL support and that your GDAL distribution was built with openjpeg support.

../_images/srtm1.png

Histogram of compression rates for the SRTM 1x1 arc second tiles. 100% reflects the full short integer size of an uncompressed tile (~25 Mb). As can be seen, on average a JPEG2000 tile is only half the size of the corresponding fully compressed (level 9) netCDF short int grid. This is why we have chosen the JP2 format for tiles on the server.

Cache File Updates

Remote cache files are our collection of miscellaneous files that are used throughout the GMT examples, man pages, and test suite. There is no system nor catalog and files come and go as we need them. The cache files are subject to similar rules as the remote data set when it comes to refreshing or deleting them. If any of these files is precious to you we suggest you make a copy somewhere.

Getting a single grid

Should you need a single grid from any of our tiled dataset, e.g., to feed into other programs that do not depend on GMT, you can create that via grdcut. For instance, to make a global grid from the eight tiles that make up the 2m x 2m gridline-registered data, try:

gmt grdcut @earth_relief_02m_g -Gearth_at_2m.grd

Global Earth Relief Grids

../_images/igpp.png
../_images/dem.jpg

Usage

You access a global relief grid by specifying the special name

@earth_relief_rru[_reg]

The following codes for rru and the optional reg are supported (dimensions are listed for pixel-registered grids; gridline-registered grids increment dimensions by one):

Code

Dimensions

Reg

Size

Description

01d

360 x 180

g,p

128 KB

1 arc degree global relief (SRTM15+V2.1 @ 111 km)

30m

720 x 360

g,p

435 KB

30 arc minute global relief (SRTM15+V2.1 @ 55 km)

20m

1080 x 540

g,p

918 KB

20 arc minute global relief (SRTM15+V2.1 @ 37 km)

15m

1440 x 720

g,p

1.6 MB

15 arc minute global relief (SRTM15+V2.1 @ 28 km)

10m

2160 x 1080

g,p

3.4 MB

10 arc minute global relief (SRTM15+V2.1 @ 18 km)

06m

3600 x 1800

g,p

8.8 MB

6 arc minute global relief (SRTM15+V2.1 @ 10 km)

05m

4320 x 2160

g,p

13 MB

5 arc minute global relief (SRTM15+V2.1 @ 9 km)

04m

5400 x 2700

g,p

19 MB

4 arc minute global relief (SRTM15+V2.1 @ 7.5 km)

03m

7200 x 3600

g,p

33 MB

3 arc minute global relief (SRTM15+V2.1 @ 5.6 km)

02m

10800 x 5400

g,p

71 MB

2 arc minute global relief (SRTM15+V2.1 @ 3.7 km)

01m

21600 x 10800

g,p

258 MB

1 arc minute global relief (SRTM15+V2.1 @ 1.9 km)

30s

43200 x 21600

g,p

935 MB

30 arc second global relief (SRTM15+V2.1 @ 1.0 km)

15s

86400 x 43200

p

3.2 GB

15 arc second global relief (SRTM15+V2.1)

03s

432000 x 216000

g

6.8 GB

3 arc second global relief (SRTM3S)

01s

1296000 x 432000

g

41 GB

1 arc second global relief (SRTM1S)

All of these data will, when downloaded, be placed in your ~/.gmt/server directory, with the earth_relief files being placed in an earth/earth_relief sub-directory. If you do not specify a CPT then this dataset default to the GMT master geo.

Technical Information

As you see, the 30s and lower resolutions are all derivatives of Scripps’ SRTM15+V2.1 grid (Tozer et al., 2019). We have downsampled it via Cartesian Gaussian filtering to prevent aliasing while preserving the latitude-dependent resolution in the original 15 arc sec grid. The full (6 sigma) filter-widths are indicated in parenthesis. The 3 and 1 arc second data are the SRTM 1x1 degree tiles from NASA. Note: The 3 and 1 arc second grids only extend to latitudes ±60˚ and are only available over land. When these grids are accessed as @earth_relief_01s or @earth_relief_03s we will automatically up-sample the relevant @earth_relief_15s tiles to fill in the missing ocean values. If you just want the original land-only SRTM tiles you may use the special names @srtm_relief_03s or @srtm_relief_01s instead. Almost all grids are available in both gridline- and pixel-registered formats except the original pixel-registered SRTM15+V2.1 (here called @earth_relief_15s) and the gridline-registered SRTM tiles.

Global Earth Seafloor Crustal Age Grids

../_images/EarthByte_logo_small.png
../_images/GMT_agefig.png

Numerous geodynamic and geophysical studies consider both the seafloor depth and the seafloor crustal age. EarthByte has pioneered the creation of crustal age grids since 1997 and we offer their latest version for remote use in GMT.

Usage

You access a global crustal age grid by specifying the special name

@earth_age_rru[_reg]

The following codes for rru and the optional reg are supported (dimensions are listed for pixel-registered grids; gridline-registered grids increment dimensions by one):

Code

Dimensions

Reg

Size

Description

01d

360 x 180

g,p

125 KB

1 arc degree global seafloor ages (1 min @ 111 km)

30m

720 x 360

g,p

402 KB

30 arc minute global relief (1 min @ 55 km)

20m

1080 x 540

g,p

827 KB

20 arc minute global relief (1 min @ 37 km)

15m

1440 x 720

g,p

1.4 MB

15 arc minute global relief (1 min @ 28 km)

10m

2160 x 1080

g,p

2.9 MB

10 arc minute global relief (1 min @ 18 km)

06m

3600 x 1800

g,p

7.3 MB

6 arc minute global relief (1 min @ 10 km)

05m

4320 x 2160

g,p

10 MB

5 arc minute global relief (1 min @ 9 km)

04m

5400 x 2700

g,p

15 MB

4 arc minute global relief (1 min @ 7.5 km)

03m

7200 x 3600

g,p

26 MB

3 arc minute global relief (1 min @ 5.6 km)

02m

10800 x 5400

g,p

56 MB

2 arc minute global relief (1 min @ 3.7 km)

01m

21600 x 10800

g,p

188 MB

1 arc minute global relief (1 min original)

All of these data will, when downloaded, be placed in your ~/.gmt/server directory, with the earth_age files being placed in an earth/earth_age sub-directory. If you do not specify a CPT, the default CPT for this data set will be used (@age_chrons_GTS2012_2020.cpt)

Technical Information

We scale and reformat the original data to take up very little space so that downloads from the servers are as fast as possible. For the seafloor crustal age grid this means we chose 0.01 My as the smallest data unit, which is well below the uncertainties in the model. Data are scaled and shifted to fit in a short integer grid that is highly compressed by netCDF lossless compression and chunking. The data are reported in Myr relative to the 2012 Geological Time Scale.

Data References

  1. Seton et al. [2020, in review] [https://www.earthbyte.org/category/resources/data-models/seafloor-age/].

Global Earth Day/Night Images

../_images/nasa-logo-web-rgb.png
../_images/daynight.jpg

We serve two NASA image products: The Blue and Black marble mosaics. We have filtered and downsampled those as well at the same resolutions that are available for the Earth DEMs. However, all images are pixel-registered only. These images may be plotted with grdimage or grdview and manipulated by grdmix. The above example mixes both images according to a day-night mask and adds illumination from a corresponding Earth DEM.

Usage

You access a global daytime image by specifying the special names

@earth_day_rru

Similarly for the nighttime view:

@earth_night_rru

The following codes for rru and the optional reg are supported (dimensions are listed for pixel-registered grids; gridline-registered grids increment dimensions by one):

Code

Dimensions

Size

Description

01d

360 x 180

128 KB

1 arc degree image view (Blue|Black Marble mosaic @ 111 km)

30m

720 x 360

435 KB

30 arc minute image view (Blue|Black Marble mosaic @ 55 km)

20m

1080 x 540

918 KB

20 arc minute image view (Blue|Black Marble mosaic @ 37 km)

15m

1440 x 720

1.6 MB

15 arc minute image view (Blue|Black Marble mosaic @ 28 km)

10m

2160 x 1080

3.4 MB

10 arc minute image view (Blue|Black Marble mosaic @ 18 km)

06m

3600 x 1800

8.8 MB

6 arc minute image view (Blue|Black Marble mosaic @ 10 km)

05m

4320 x 2160

13 MB

5 arc minute image view (Blue|Black Marble mosaic @ 9 km)

04m

5400 x 2700

19 MB

4 arc minute image view (Blue|Black Marble mosaic @ 7.5 km)

03m

7200 x 3600

33 MB

3 arc minute image view (Blue|Black Marble mosaic @ 5.6 km)

02m

10800 x 5400

71 MB

2 arc minute image view (Blue|Black Marble mosaic @ 3.7 km)

01m

21600 x 10800

258 MB

1 arc minute image view (Blue|Black Marble mosaic @ 1.9 km)

30s

43200 x 21600

935 MB

30 arc second image view (Blue|Black Marble mosaic original)

All of these images will, when downloaded, be placed in your ~/.gmt/server director under the earth/earth_day and earth/earth_night sub-directories.

Technical Information

The 01m and lower resolution images are derivatives of NASA’s Blue and Black marble image mosaics. We have downsampled them via Cartesian Gaussian filtering to prevent aliasing while preserving the latitude-dependent resolution in the original images. The full (6 sigma) filter-widths are indicated in parenthesis. The dimensions above reflect the number of nodes covered by the global images and the sizes refer to the geotiff files on the remote server. Note: This data set is experimental and the format and delivery is likely to change in the future (e.g., via image tiles). To make the files as small as possible we have also downgraded them from 24-bit to 8-bit indexed images.

Global Earth Mask Grids

../_images/GMT_earthmask.png

Many users need ocean/land mask grids to isolate features that should only be set over the ocean or on land. While you can prevent such areas from being plotted via the clipping in coast, you may still want to perform calculations on gridded data and apply masks via grdmath. Such mask grids can be computed via grdlandmask but they can take a long time to compute for large regions, the full GSHHG resolution, and small grid spacings. For these reasons we offer precalculated mask grids via the remote server mechanism.

Usage

You access a global mask grid by specifying the special name

@earth_mask_rru[_reg]

The following codes for rru and the optional reg are supported (dimensions are listed for pixel-registered grids; gridline-registered grids increment dimensions by one):

Code

Dimensions

Reg

Size

Description

01d

360 x 180

g,p

20 KB

1 arc degree global mask (skip features < 12364 km^2)

30m

720 x 360

g,p

24 KB

30 arc minute global mask (skip features < 3091 km^2)

20m

1080 x 540

g,p

30 KB

20 arc minute global mask (skip features < 1374 km^2)

15m

1440 x 720

g,p

35 KB

15 arc minute global mask (skip features < 773 km^2)

10m

2160 x 1080

g,p

57 KB

10 arc minute global mask (skip features < 343 km^2)

06m

3600 x 1800

g,p

106 KB

6 arc minute global mask (skip features < 124 km^2)

05m

4320 x 2160

g,p

117 KB

5 arc minute global mask (skip features < 86 km^2)

04m

5400 x 2700

g,p

149 KB

4 arc minute global mask (skip features < 55 km^2)

03m

7200 x 3600

g,p

209 KB

3 arc minute global mask (skip features < 31 km^2)

02m

10800 x 5400

g,p

393 KB

2 arc minute global mask (skip features < 14 km^2)

01m

21600 x 10800

g,p

998 KB

1 arc minute global mask (skip features < 3 km^2)

30s

43200 x 21600

g,p

2.6 MB

30 arc second global mask (skip features < 1 km^2)

15s

86400 x 43200

g,p

7.5 MB

15 arc second global (all features)

All of these data will, when downloaded, be placed in your ~/.gmt/server directory, with the Earth mask files being placed in an earth/earth_mask sub-directory.

Technical Information

Given that GSHHG contains 5 levels (ocean, land, lake, island-in-lake, pond-in-island-in-lake) corresponding to levels 0-4, the mask grids were computed to reflect those 5 levels. This means that if you only want a mask for the ocean or one for wet versus dry, you will need to run a simple grdmath command to adjust the mask for your use. We give some common examples here. To create a gridline-registered mask that is 0 in the ocean and 1 everywhere else (even in lakes, etc.), try:

gmt grdmath @earth_mask_01m_g 0 GT = ocean_land.grd

To make a pixel-registered mask that has NaNs on land and 1 in the oceans, use:

gmt grdmath @earth_mask_01m_p 0 LE 0 NAN = land_NaN.grd

Finally, to make a pixel-registered mask that is 1 for lakes and NaN elsewhere, try:

gmt grdmath @earth_mask_01m 2 EQ 0 NAN = lakes_and_NaN.grd