aisdb.denoising_encoder module

class aisdb.denoising_encoder.InlandDenoising(data_dir, land_cache='land.pkl', water_cache='water.pkl')

Bases: object

data_url = 'http://bigdata5.research.cs.dal.ca/geo_land_water_NorthAmerica.7z'

filter_noisy_points(tracks: iter) → dict

Filter points that fall in land but not in water features.

aisdb.denoising_encoder.encode_greatcircledistance(tracks, *, distance_threshold, speed_threshold=50, minscore=1e-06)

Partitions tracks where delta speeds exceed speed_threshold or delta_meters exceeds distance_threshold. concatenates track segments with the highest likelihood of being sequential, as encoded by the encode_score function

Parameters:

• tracks (aisdb.track_gen.TrackGen) – track vectors generator

• distance_threshold (int) – distance in meters that will be used as a speed score numerator

• speed_threshold (float) – maximum speed in knots that should be considered a continuous trajectory

• minscore (float) – minimum score threshold at which to allow track segments to be linked. Value range: (0, 1). A minscore closer to 0 will be less restrictive towards trajectory grouping. A reasonable value for this is 1e-6. This score is computed by the function aisdb.denoising_encoder.encode_score()

>>> import os
>>> from datetime import datetime, timedelta
>>> from aisdb import SQLiteDBConn, DBQuery, TrackGen
>>> from aisdb import decode_msgs, encode_greatcircledistance, sqlfcn_callbacks

>>> # create example database file
>>> dbpath = 'encoder_test.db'
>>> filepaths = ['aisdb/tests/testdata/test_data_20210701.csv',
...              'aisdb/tests/testdata/test_data_20211101.nm4']

>>> with SQLiteDBConn(dbpath) as dbconn:
...     decode_msgs(filepaths=filepaths, dbconn=dbconn,
...                 source='TESTING', verbose=False)

>>> with SQLiteDBConn(dbpath) as dbconn:
...     q = DBQuery(callback=sqlfcn_callbacks.in_timerange_validmmsi,
...             dbconn=dbconn,
...             start=datetime(2021, 7, 1),
...             end=datetime(2021, 7, 7))
...     tracks = TrackGen(q.gen_qry(), decimate=True)
...     for track in encode_greatcircledistance(
...             tracks,
...             distance_threshold=250000,  # metres
...             speed_threshold=50,         # knots
...             minscore=0,
...         ):
...         print(track['mmsi'])
...         print(track['lon'], track['lat'])
...         break

aisdb.denoising_encoder.encode_score(track, distance_threshold, speed_threshold, minscore)

Encodes likelihood of persistent track membership when given distance, speed, and score thresholds, using track speed deltas computed using distance computed by haversine function divided by elapsed time

A higher distance threshold will increase the maximum distance in meters allowed between pings for same trajectory membership. A higher speed threshold will allow vessels travelling up to this value in knots to be kconsidered for persistent track membership. The minscore assigns a minimum score needed to be considered for membership, typically 0 or very close to 0 such as 1e-5.

For example: a vessel travelling at a lower speed with short intervals between pings will have a higher likelihood of persistence. A trajectory with higher average speed or long intervals between pings may indicate two separate trajectories and will be segmented forming alternate trajectories according to highest likelihood of membership.

aisdb.denoising_encoder.encoder_score_fcn(x1, y1, t1, x2, y2, t2, speed_thresh, dist_thresh)

This function is used internally by aisdb.denoising_encoder.encode_score().

Assigns a score for likelihood of two points being part of a sequential vessel trajectory. A hard cutoff will be applied at distance_threshold, after which all scores will be set to -1.

Parameters:

• x1 (float) – longitude of coordinate pair 1

• y1 (float) – latitude of coordinate pair 1

• t1 (float) – Timestamp for coordinate pair 1 in epoch seconds

• x2 (float) – longitude of coordinate pair 2

• y2 (float) – latitude of coordinate pair 2

• t2 (float) – Timestamp for coordinate pair 2 in epoch seconds

• speed_threshold (float) – Tracks will be segmented between points where computed speed values exceed this threshold. Segmented tracks will be scored for reconnection. Measured in knots

• distance_threshold (float) – Used as a numerator when determining score; this value is divided by the distance between xy1 and xy2. If the distance between xy1 and xy2 exceeds this value, the score will be set to -1. Measured in meters

Returns:

f64)

Return type:

score (float

aisdb.denoising_encoder.remove_pings_wrt_speed(tracks, speed_threshold)

Remove pings from tracks where the speed of a vessel is lesser than equal to speed_threshold. In most cases, the archored vessel tracks are removed through this technique

Parameters:

tracks (aisdb.track_gen.TrackGen) – track vectors generator

return generator