thelper.data.geo package

Geospatial dataset parsing/loading package.

This package contains classes and functions whose role is to fetch the data required to train, validate, and test a model on geospatial data. Importing the modules inside this package requires GDAL.

Submodules

thelper.data.geo.agrivis module

Agricultural Semantic Segentation Challenge Dataset Interface

Original author: David Landry (david.landry@crim.ca) Updated by Pierre-Luc St-Charles (April 2020)

thelper.data.geo.bigearthnet module

thelper.data.geo.gdl module

Data parsers & utilities for cross-framework compatibility with Geo Deep Learning (GDL).

Geo Deep Learning (GDL) is a machine learning framework initiative for geospatial projects lead by the wonderful folks at NRCan’s CCMEO. See https://github.com/NRCan/geo-deep-learning for more information.

The classes and functions defined here were used for the exploration of research topics and for the validation and testing of new software components.

thelper.data.geo.infer module

class thelper.data.geo.infer.SlidingWindowTester(session_name, session_dir, model, task, loaders, config, ckptdata=None)

Bases: thelper.infer.base.Tester

Tester that satisfies the requirements of the Tester in order to run classification inference

__init__(session_name, session_dir, model, task, loaders, config, ckptdata=None)

Receives the trainer configuration dictionary, parses it, and sets up the session.

eval_epoch(model, epoch, dev, loader, metrics, output_path)

Computes the pixelwise prediction on an image.

It does the prediction per batch size of N pixels. It returns the class predicted and its probability. The results are saved into two images created with the same size and projection info as the input rasters.

The class image gives the class id, a number between 1 and the number of classes for corresponding pixels. Class id 0 is reserved for nodata.

The probs image contains N-class channels with the probability values of the pixels for each class. The probabilities by default are normalised.

Also, a config-classes.json file is created listing the name-to-class-id mapping that was used to generate the values in the class image (i.e.: class names defined by the pre-trained model).

Parameters

• model – the model with which to run inference that is already uploaded to the target device(s).

• epoch – the epoch index we are training for (0-based, and should normally only be 0 for single test pass).

• dev – the target device that tensors should be uploaded to (corresponding to model’s device(s)).

• loader – the data loader used to get transformed test samples.

• metrics – the dictionary of metrics/consumers to report inference results (mostly loggers and basic report generator in this case since there shouldn’t be ground truth labels to validate against).

• output_path – directory where output files should be written, if necessary.

supports_classification = True

thelper.data.geo.ogc module

Data parsers & utilities module for OGC-related projects.

class thelper.data.geo.ogc.TB15D104

Bases: object

Wrapper class for OGC Testbed-15 (D104) identifiers.

BACKGROUND_ID = 0

LAKE_ID = 1

TYPECE_LAKE = '21'

TYPECE_RIVER = '10'

class thelper.data.geo.ogc.TB15D104DetectLogger(conf_threshold=0.5)

Bases: thelper.train.utils.DetectLogger

__init__(conf_threshold=0.5)

Receives the logging parameters & the optional class label names used to decorate the log.

report_geojson()

thelper.data.geo.ogc.postproc_features(input_file, bboxes_srs, orig_geoms_path, output_file, final_srs=None, write_shapefile_copy=False)

Post-processes bounding box detections produced during an evaluation session into a GeoJSON file.

thelper.data.geo.parsers module

Geospatial data parser & utilities module.

thelper.data.geo.utils module

thelper.data.geo.utils.export_geojson_with_crs(features, srs_target)

Exports a list of features along with their SRS into a GeoJSON-compat string.

thelper.data.geo.utils.export_geotiff(filepath, crop, srs, geotransform)

thelper.data.geo.utils.get_feature_bbox(geom, offsets=None)

thelper.data.geo.utils.get_feature_roi(geom, px_size, skew, roi_buffer=None, crop_img_size=None, crop_real_size=None)

thelper.data.geo.utils.get_geocoord(geotransform, x, y)

thelper.data.geo.utils.get_geoextent(geotransform, x, y, cols, rows)

thelper.data.geo.utils.get_pxcoord(geotransform, x, y)

thelper.data.geo.utils.open_rasterfile(raster_data, keep_rasters_open=False)

thelper.data.geo.utils.parse_geojson(geojson, srs_target=None, roi=None, allow_outlying=False, clip_outlying=False)

thelper.data.geo.utils.parse_geojson_crs(body)

Imports a coordinate reference system (CRS) from a GeoJSON tree.

thelper.data.geo.utils.parse_raster_metadata(raster_metadata, raster_dataset=None)

Parses the provided raster metadata and updates it by adding extra details required for later use.

The provided raster metadata is updated directly. Metadata is validated against the matching data storage. If any important, required or requested (bands) metadata is missing, the function raises the issue immediately.

Parameters

• raster_metadata (dict) – raster metadata dictionary with minimally a file ‘path’ and list of ‘bands’ indices to process.

• raster_dataset (gdal.Dataset) – (optional) preloaded dataset object corresponding to the raster metadata.

Raises

• ValueError – at least one input raster was missing a required metadata parameter or a parameter is erroneous.

• IOError – the raster path could not be found or reading it did not generate a valid raster using GDAL.

thelper.data.geo.utils.parse_rasters(raster_paths, srs_target=None, reproj=False)

thelper.data.geo.utils.parse_roi(roi_path, srs_target=None)

thelper.data.geo.utils.parse_shapefile(shapefile_path, srs_target=None, roi=None, allow_outlying=False, clip_outlying=False, layer_id=0)

thelper.data.geo.utils.reproject_coords(coords, src_srs, tgt_srs)

thelper.data.geo.utils.reproject_crop(raster, crop_raster, crop_size, crop_datatype, crop_nodataval=None, reproj_opt=None, fill_nodata=False)