Help on hausdorff distance error in cuspatial

[nithinme3]

Hi,
Thanks for the amazing repo.

I was trying to do trajectory clustering as given in https://medium.com/rapids-ai/releasing-cuspatial-to-accelerate-geospatial-and-spatiotemporal-processing-b686d8b32a9

I also found the github repo for the same
https://github.com/wkelongws/trajectory_clustering_with_cuspatial/
In the above repo they are first converting the trajectories to image points then they are doing hausdorff distance calculation latter clustering using db scan also.

I was wondering if we can do hausdorff distance calculation on lat long values (or at least on corresponding utm values) directly instead on image points.
My plan was to do the clustering in lat-long/utm coordinates and plot using folium.

Infact I tried hausdorff distance calculation on both utm and lat long but am getting following error,

(rapids) [root@nithin-rapids cuspatial]# python ./python/cuspatial/demos/hausdorff_clustering_test_toy.py
Traceback (most recent call last):
File "/share/adas_coe_159/nithin/cuspatial/./python/cuspatial/demos/hausdorff_clustering_test_toy.py", line 39, in
distance = cuspatial.directed_hausdorff_distance(pnt_x, pnt_y, cnt)
File "/opt/conda/envs/rapids/lib/python3.10/site-packages/cuspatial/core/spatial/distance.py", line 97, in directed_hausdorff_distance
return DataFrame(result)
File "/opt/conda/envs/rapids/lib/python3.10/contextlib.py", line 79, in inner
return func(*args, **kwds)
File "/opt/conda/envs/rapids/lib/python3.10/site-packages/cudf/core/dataframe.py", line 679, in init
new_df = self._from_arrays(data, index=index, columns=columns)
File "/opt/conda/envs/rapids/lib/python3.10/contextlib.py", line 79, in inner
return func(*args, **kwds)
File "/opt/conda/envs/rapids/lib/python3.10/site-packages/cudf/core/dataframe.py", line 5404, in _from_arrays
df._data[k] = column.as_column(
File "/opt/conda/envs/rapids/lib/python3.10/site-packages/cudf/core/column/column.py", line 1887, in as_column
arbitrary = cupy.ascontiguousarray(arbitrary)
File "/opt/conda/envs/rapids/lib/python3.10/site-packages/cupy/_creation/from_data.py", line 107, in ascontiguousarray
return _core.ascontiguousarray(a, dtype)
File "cupy/_core/core.pyx", line 2673, in cupy._core.core.ascontiguousarray
File "cupy/_core/core.pyx", line 2689, in cupy._core.core.ascontiguousarray
File "cupy/_core/core.pyx", line 136, in cupy._core.core.ndarray.new
File "cupy/_core/core.pyx", line 224, in cupy._core.core._ndarray_base._init
File "cupy/cuda/memory.pyx", line 742, in cupy.cuda.memory.alloc
File "/opt/conda/envs/rapids/lib/python3.10/site-packages/rmm/rmm.py", line 230, in rmm_cupy_allocator
buf = librmm.device_buffer.DeviceBuffer(size=nbytes, stream=stream)
File "device_buffer.pyx", line 85, in rmm._lib.device_buffer.DeviceBuffer.cinit
MemoryError: std::bad_alloc: CUDA error at: /opt/conda/envs/rapids/include/rmm/mr/device/cuda_memory_resource.hpp
(rapids) [root@nithin-rapids cuspatial]# python ./python/cuspatial/demos/hausdorff_clustering_test_toy.py

What could be the issue?

[jarmak-nv]

Hi @nithinme3! Glad you're checking out cuSpatial!

Per our docs - Hausdorff distance is cartesian only so you will indeed need to convert to UTM prior to doing the calculations.

A few questions for you:

• What installation method and version are you using? It looks like a docker install, but want to confirm
• Hardware/OS you're running?

I made a barebones example and ran it successfully on the 23.02 Ubuntu 20.04 py3.10 container a moment ago, does this run on your system? If yes, something is probably either going wrong as you form the inputs for directed_hausdorff_distance or you may be running out of memory.

import numpy as np
from cudf import Series
import cuspatial

in_trajs = []
in_trajs.append(np.array([[1, 0], [2, 1], [3, 2], [5, 3], [7, 1]]))
in_trajs.append(np.array([[0, 3], [2, 5], [3, 6], [6, 5]]))
in_trajs.append(np.array([[1, 4], [3, 7], [6, 4]]))

out_trajs = np.concatenate([np.asarray(traj) for traj in in_trajs], 0)
py_x = np.array(out_trajs[:, 0])
py_y = np.array(out_trajs[:, 1])
py_cnt = []

for traj in in_trajs:
    py_cnt.append(len(traj))
    
pnt_x = Series(py_x)
pnt_y = Series(py_y)
cnt = Series(py_cnt)

distance = cuspatial.directed_hausdorff_distance(pnt_x, pnt_y, cnt)
distance

0	1	2
0	-1.000000e+00	-1.000000e+00	-1.0
1	-1.000000e+00	-1.000000e+00	-1.0
2	1.340781e+154	1.340781e+154	0.0

[nithinme3]

Hi @jarmak-nv ,

Thanks for the response,

Yes am running a docker install. I pulled, nvcr.io/nvidia/rapidsai/rapidsai:cuda11.2-runtime-centos7-py3.10.
I have nvidia v100 GPU. Running on a Virtual machine with ubuntu 20.4.

I had already ran the sample code above which is running fine.

I have used the dataset file provided in the repo, https://github.com/wkelongws/trajectory_clustering_with_cuspatial/ instead of dummy data in the above code and tried to run
There are more than 10k trajectories in the dataset. I filtered the trajectories with points between 30 and 500 which gives around 9930 trajectories, but I am getting above mentioned error.

One observation is,

When I tried to run with a reduced dataset size (around 25 trajectories), it is running. But for 50 its giving the same error above.

any suggestion?

Thank you,
Nithin

[jarmak-nv]

Yeah interesting - I turned that notebook into a barebones version and ran on an RTX3070 without issue. Does the code below break for you as well?

You might also try running your container with the following commands below if you're not already. We've seen the --ulimit commands fix some errors in the past.

docker run --gpus all --rm -it \
    --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 \
    -p 8888:8888 -p 8787:8787 -p 8786:8786 \
    nvcr.io/nvidia/rapidsai/rapidsai-core:23.02-cuda11.2-runtime-centos7-py3.10

---

import numpy as np
import cudf
import cuspatial
import pickle

with open('trajectories_extracted.p','rb') as f:
    trajectories = pickle.load(f)

def convert_latlon_2_image_for_vis_wrongway_intersection(lat,long):

    x_anchor = 436.369
    y_anchor = 126.613
    long_anchor = -90.666714
    lat_anchor = 42.492466

    long_1 = -90.663183
    lat_1 = 42.491671
    x_1 = 971.026
    y_1 = 291.955

    long_2_x_rate = (x_1-x_anchor)/(long_1-long_anchor)
    lat_2_y_rate = (y_1-y_anchor)/(lat_1-lat_anchor)

    y = (lat-lat_anchor)*lat_2_y_rate+y_anchor
    x = (long-long_anchor)*long_2_x_rate+x_anchor

    return x, y

def convert_traj_latlon_2_image_wrongway_intersection(traj):
    traj_image = []
    for pt in traj:
        lat = pt[0]
        long = pt[1]
        x,y = convert_latlon_2_image_for_vis_wrongway_intersection(lat,long)
        traj_image.append((x,y))

    return np.asarray(traj_image)

def traj_filter_num_points(trajectories,num_min):
    max_length = 0
    min_length = 999
    lengths = []
    long_trajectories = []
    for traj in trajectories:
        if len(traj)>=num_min:
            long_trajectories.append(traj)
            if len(traj)<min_length:
                min_length = len(traj)
            lengths.append(len(traj))
        if len(traj) > max_length:
            max_length = len(traj)
    return long_trajectories

def pick_traj_with_label(label,labels,trajectories):
    selected_trajectories = []
    for i in range(len(labels)):
        if labels[i] == label:
            selected_trajectories.append(trajectories[i])
    return selected_trajectories  

trajectories = traj_filter_num_points(trajectories,num_min=30)
trajectories = [convert_traj_latlon_2_image_wrongway_intersection(trajectory) for trajectory in trajectories]

cnt = []
for traj in trajectories:
    cnt.append(len(traj))
cnt = np.asarray(cnt)

trajs = np.concatenate([np.asarray(traj) for traj in trajectories],0)

x_s = trajs[:,0]
y_s = trajs[:,1]

pnt_x=cudf.Series(x_s)
pnt_y=cudf.Series(y_s)
cnt=cudf.Series(cnt)

dist=cuspatial.directed_hausdorff_distance(pnt_x,pnt_y,cnt)

dist

0	1	2	3	4	5	6	7	8	9	...	10060	10061	10062	10063	10064	10065	10066	10067	10068	10069
-1.000000	-1.000000	-1.000000e+00	-1.000000	-1.000000e+00	-1.000000e+00	-1.000000	-1.000000e+00	-1.000000	-1.000000	...	-1.000000e+00	-1.000000	-1.000000e+00	-1.000000e+00	-1.000000	-1.000000e+00	-1.000000e+00	-1.000000	-1.000000	-1.000000
-1.000000	-1.000000	-1.000000e+00	-1.000000	-1.000000e+00	-1.000000e+00	-1.000000	-1.000000e+00	-1.000000	-1.000000	...	-1.000000e+00	-1.000000	-1.000000e+00	-1.000000e+00	-1.000000	-1.000000e+00	-1.000000e+00	-1.000000	-1.000000	-1.000000
-1.000000	-1.000000	-1.000000e+00	-1.000000	-1.000000e+00	-1.000000e+00	-1.000000	-1.000000e+00	-1.000000	-1.000000	...	-1.000000e+00	-1.000000	-1.000000e+00	-1.000000e+00	-1.000000	-1.000000e+00	-1.000000e+00	-1.000000	-1.000000	-1.000000
-1.000000	-1.000000	-1.000000e+00	-1.000000	-1.000000e+00	-1.000000e+00	-1.000000	-1.000000e+00	-1.000000	-1.000000	...	-1.000000e+00	-1.000000	-1.000000e+00	-1.000000e+00	-1.000000	-1.000000e+00	-1.000000e+00	-1.000000	-1.000000	-1.000000
-1.000000	-1.000000	-1.000000e+00	-1.000000	-1.000000e+00	-1.000000e+00	-1.000000	-1.000000e+00	-1.000000	-1.000000	...	-1.000000e+00	-1.000000	-1.000000e+00	-1.000000e+00	-1.000000	-1.000000e+00	-1.000000e+00	-1.000000	-1.000000	-1.000000
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
-1.000000	-1.000000	-1.000000e+00	-1.000000	-1.000000e+00	-1.000000e+00	-1.000000	-1.000000e+00	-1.000000	-1.000000	...	-1.000000e+00	-1.000000	-1.000000e+00	-1.000000e+00	-1.000000	-1.000000e+00	-1.000000e+00	-1.000000	-1.000000	-1.000000
-1.000000	-1.000000	-1.000000e+00	-1.000000	-1.000000e+00	-1.000000e+00	-1.000000	-1.000000e+00	-1.000000	-1.000000	...	-1.000000e+00	-1.000000	-1.000000e+00	-1.000000e+00	-1.000000	-1.000000e+00	-1.000000e+00	-1.000000	-1.000000	-1.000000
-1.000000	-1.000000	-1.000000e+00	-1.000000	-1.000000e+00	-1.000000e+00	-1.000000	-1.000000e+00	-1.000000	-1.000000	...	-1.000000e+00	-1.000000	-1.000000e+00	-1.000000e+00	-1.000000	-1.000000e+00	-1.000000e+00	-1.000000	-1.000000	-1.000000
574.116873	0.163471	1.340781e+154	310.504367	1.340781e+154	1.340781e+154	402.505328	1.340781e+154	453.047834	4.264833	...	1.340781e+154	322.407233	1.340781e+154	1.340781e+154	419.828392	1.340781e+154	1.340781e+154	310.504367	0.000000	0.218549
985.509696	411.574239	1.340781e+154	720.802275	1.340781e+154	1.340781e+154	814.420754	1.340781e+154	864.874414	411.574239	...	1.340781e+154	733.073818	1.340781e+154	1.340781e+154	831.774165	1.340781e+154	1.340781e+154	720.802275	411.574239	0.000000

[jarmak-nv]

Hey @nithinme3 just following up:

• Did the docker commands above help?
• Did the reduced code run?

Just want to make sure we're helping how we can. Thanks!