how to draw the corresponding vehicle track

[jagodki]

Hello, I am a graduate student. I would like to ask you how to draw the corresponding vehicle track on the map according to the existing GPS data set of vehicles (including longitude and latitude).Thank you. Looking forward to your reply.

Originally posted by @love-freedom777 in #22 (comment)

[jagodki]

Hi,

I am not sure if I understand your question correctly. Do you want to draw the GPS trajectory on the map (the road network)? This is the main functionality of my plugin and is described in the README on GitHub, also in a nutshell directly in the GUI of the plugin and in the paper I published (link at the end of the README, but just in german ^^). If your data has coordinates with lon/lat, does it mean, that the data has the CRS EPSG:4326? Than you should project it in a metric CRS (e.g. with UTM coordinates). If you are not familiar with QGIS and you need help to do it, please tell me exactly what do you want to do and what do you not know to reach your goals.

Sincerely,
Christoph

[love-freedom777]

Hello,
Thank you very much for your kind reply to my question!Your reply is of great help to me, but there seems to be a error when I use the plug-in you said(Maybe I don't konw how to use it
attachment.zip
), so I can't use it (I cut off picture 1)and I want to know why.Therefore, some points in the trajectory drawn by my GPS data set are problematic. For example, in the attachment, I cut off picture 2 of the trajectory, some of the points above are not on the road or do not exactly match the road.My goal is to accurately match the relevant GPS data to the map, and then the next step is to plot the corresponding track at intervals to show the status of the traffic flow (which is the next goal, of course).Thanks again for your help. I hope to keep further communication and learn from you.
Sincerely,
Qi He

[jagodki]

Thank you for the attachment. I will have a look on it, but I need some days to find the time for it (the plugin is a free time project and my free time is currently very rare, sorry ^^). I will inform you, when I find something out.

Sincerely,
Christoph

[love-freedom777]

Ok, thank you very much for your reply!I look forward to your answers to my questions.
Sincerely，
He

[love-freedom777]

Hello, I don't know what is the cause of this plug-in running error, do you know how to solve it?Thank you very much for your reply!

[jagodki]

You can read in the QGIS log, that the input parameter NETWORK is not correct. You did not provide a network layer with the geometry type LINESTRING. Such a layer is mandatory for the plugin, because the trajectory will be mapped on this network layer. Your OSM-background map cannot be used for the map matching. Try to get the road network of this area from a public service, from OSM or something else and then try tu run the plugin again with this network layer.

[jagodki]

For the next version (maybe in autumn or at the end of this year) I will implement a better error handling if parameters are not given by the user.

[love-freedom777]

Thank you very much for your reply!Do you mean I need to add another network layer?As a beginner, I don't know how to add a network layer. I don't understand how to" get the road network of this area from a public service, from OSM or something else."Add a network layer as shown below?

If you can, can you use the following simple data to try to display the algorithmic track image on QGIS?For this I am extremely grateful!
Taxi_105.txt

What's more, my QGIS software seems to fail to respond all of a sudden.Just like this. Do you know why?

Thanks again for your kind reply!

[jagodki]

Hi,

I do not know why your QGIS freezes. On my machine it sometimes happens too. A GIS (not important which specific) is a very complex software, so it does not wonder me, that such big and special software freezes from time to time.

I did not know, that you are a GIS-beginner, my fault :)
Because of this, some GIS basics:
The background map in your QGIS-project is just an image, i.e. a raster layer. This image is rendered by a server (in this case from OpenStreetMap). Your trajectory is a so called vector layer, because this layer contains single datasets for each geometry. QGIS reads the features (that is the GIS-name for datasets of a vector layer) and renders a picture for your monitor with the features into it.

My plugin needs two vector layers: a trajectory and a network layer. The network layer contains datasets/features for the edges of a road network. You can get such data e.g. from OpenStreetMap. You can download them via the plugins QuickOSM or OSMDownloader. I downloaded a network for you:
network.gpkg.zip
Because I do not know which type of streets do you need (Highways, residential streets etc.) and because your trajectory is extremely large, the network I downloaded is very huge. My advice: split your trajectory into smaller parts (e.g. maximum 500m or 1000m for each sub trajectory), this will improve the computation time a lot. I filtered the network too, maybe some necessary edges are missing, but the network looks a lot like a chess board, so it should be okay.

Some information about this network I attached:
the CRS is EPSG:4326, i.e. it has lon/lat as coordinates (like your trajectory). You can use this data as it is with my plugin, but I cannot say, which result you can expect. Please reproject your data into a metric CRS, than my plugin should work. I am not familiar with maps of China, so I do not know which CRS is the best for your area of interest. If you need more information about this, let me know (but my help will not guarantee success ^^).

Enough text at the moment 😸

Sincerely,
Christoph

[love-freedom777]

Thank you very much for your patient reply!Because of your solution, I have a deeper understanding of this software.I used the network layer you downloaded for me, but after successfully running your plug-in, I did not get the result I expected.My expectation is that the result will match the original data point to the line shown in the network layer you gave me, but it won't show up, I don't know why.Can you run the plugin and match the original data point to the line of the network layer you gave me?

These two screenshots show the layers running with your plug-in, but it doesn't show the matching tracks.

Are there any steps I haven't done or have done wrong?Thank you very much for your reply!
Sincerely,
He

[jagodki]

If you start the plugin from the toolbox, a result layer will be created every time, also when the calculations will fail (thats the reason why the result layer is empty).

There was a problem with the calculations of the transition probabilities. The calculations failed, if the network contained a street, that is not connected to the other streets of the network. I recently uploaded a new version of the plugin (version 2.2.2). Please upload the plugin via the plugin download manager and try it again. I would expect, that the result layer will not be empty anymore. Let's see if it works now 😸

[love-freedom777]

After I use the updated plug-in you said, it seems that the plug-in will run slowly and will stay stuck at 37%, but I also reduce the data, I don't know why.This is shown below：

[CateDeku]

Same here. I think that instead of stopping the algorithm and showing the error message regarding the transition probabilities, it remains stuck

[jagodki]

@love-freedom777 @CateDeku The plugin does not freeze or stuck or fail, but it calculates the transition probabilities anymore. This step can take a huge amount of time. If you start the plugin via the toolbox, the progress bar indicates only the processing steps described in the help text and in the QGIS log. It does not indicates the progress of the calculations during the individual processing steps. The control over the progress bar of the toolbox is very limited for a developer. Maybe I can improve it in a future version.

If you want more information about the process of each processing step, than you have to use the main GUI of the plugin via vector menu - Offline-MapMatching. But if you start the calculations via the main GUI, you cannot use QGIS until the processing finished.

@love-freedom777 I think the problem is, that your trajectory is very noisy compared to the density of roads in the network. That means that you have to choose a high maximum search distance to find at least one candidate on the network per trajectory point. The high search distance will produce a lot of candidates. And a lot of candidates result in a lot of transitions. For each transition the dijkstra algorithm will be calculated. This is the bottleneck of the plugin.

Today I reduced your network more and after an half of an hour, the plugin finished with a non empty result layer 😄

Please download this file:
model.model3.zip
It contains a model which runs the following tasks:

• create a path from your trajectory (use the field with the timestamps for order the points of your trajectory)
• buffer this path (please use a buffer radius of 150 m, it is the default value)
• clip the network using the buffer as overlay
• create single geometries from the clipped network

All described parameters has to be inserted after starting the model. To load the model, first open the graphic modeler:

Than open the downloaded model file (1) and run the model (2):

A dialog comparable to the processing tools will open. Just choose your network, your trajectory, the field of the trajectory to order your points (use the timestamps) and choose a buffer radius of 150 m. After running the model, a new layer should appear in QGIS. Use this layer as the network in the plugin.

A lot of text ^^ Now please try it and let me know, if the map matching now works. Remember, it takes a lot of time (30 minutes on my machine).

Sincerely,
Christoph

[CateDeku]

Ok, good to know. Thank you for your answers and your patience! :D

[love-freedom777]

OK，thank you very much!Do you mean to download the zip pack you sent, then follow your steps to generate a new reduced network layer, then use this network layer to run your map matching plug-in, and finally generate the expected matching path?Maybe my computer is not well configured, it will still get stuck and run for a long time without any result.Later I will switch to a better configured computer to run it.What does it look like when you run it? Can you take a look at it?Thanks again for your reply.

[jagodki]

Yes, that is the way you should use the zip pack. I will post a screenshot of the successful map matching on my machine during the next days.

[love-freedom777]

OK,thank you very much!

[jagodki]

Sorry, I forgot to send you the screenshot and I have no access to the machine I tested your data on until monday :(
But I created a reminder in my mobile phone, i.e. next monday I will post it with some more information :)

[love-freedom777]

That's OK.:)

[jagodki]

Now I can show you a screenshot of the successful (i.e. the plugin calculated a matched trajectory, I did not try to adjust the parameters for getting a better/realistic result) matching:

I did not matched the whole trajectory, because it is very very long. Maybe too long for a good result. So try to split your trajectory into smaller parts. Your data has a field with timestamps. Maybe you could use this field for splitting your trajectory.

Sincerely,
Christoph

[love-freedom777]

Hello, thank you very much!I came across a question again, which was how your reduced_network was generated only around the locus point?The network layer you gave me before is very large, does this lead to very large computation?Just like this,

Can you tell me how to add the street network layer only near the track point？I'm really sorry to keep asking you so many questions.Thank you very much again！：）
Sincerely，
he

[love-freedom777]

I downloaded part of the OSM road map with the track points through the OSMDownloader plug-in, but when you run your plug-in it will look like this.The software crashes directly and can't get the results you send.I don't know which step went wrong?

[love-freedom777]

I will describe my detailed operation process. Could you please help me see which step is wrong with me?First add the trace data set
taxi105.txtto the layer, just like this,

Next, use the OSMDownload plug-in to get the road network layer near the track point.Like this,

The new reduced network is then generated using the model 3 you sent to me earlier,like this,


Finally, when you run your plugin, you get this problem: the software crashes, and it doesn't work.
Just like the result of my previous comment.

[jagodki]

Hi,

excuse me for my late respond. I was on the countryside during the last days with a very very poor internet access ^^

You are right, a big network results in a long computation time. The “problem” is, that the plugin calculates possible points on each dataset of the network in the range of the maximum search distance for each trajectory point, i.e. (count of trajectory points) x (possible points on the network). This results in a big number And i think, that the computation time increases exponentially with an increase of this number. Because of this I recommend to reduce the network (and/or preprocess the trajectory, but this is more complicated).

I had the same experience like you when I used big datasets. The plugin freezes because it runs in the main thread of the QGIS application. During the calculations, the user interface will not be updated on every second, but after a calculation step. If the calculations need a lot of time, the user interface freezes for a long time...

On your screenshots I do not see a difference between the basic and the reduced network (after using the model I provided). Did you used the default buffer distance from the model? If yes, please reduce this value to get a difference between the basic network and the reduced one. My reduced network is the result of this model. If the adjustment of the buffer radius does not result in a smaller network, than please open the QGIS log and run it again. Errors should be logged there.

Sincerely,
Christoph

[love-freedom777]

Thank you very much for your kind reply！As you mentioned, after running your model 3, the output reduced network is no different from the basic network. I don't know why. I changed the default buffer radius to 50 as you mentioned, but the output reduced network layer is still the same.I opened the QGIS log and saw no errors, but a few warnings, as shown in the screenshot.I don't know if these warnings will affect the results of the run.So I probably know the reason why I can't get the result I want, which is that the network layer of road network data is not generated. Ok.I guess so. How do I fix it?


Sincerely,
He

[jagodki]

Really strange 🤔

Maybe I am wrong, but something on your last screenshot is not correct in my opinion. Your QGIS project has the CRS EPSG:32651, i.e. a CRS with metric coordinates. But the field with the coordinates of your cursor seems like it shows geographic coordinates. I will check your trajectory the next days, maybe my guess is wrong...

[love-freedom777]

Hello, I would like to ask how the road network data in your example is generated and I would like to know where you downloaded your road network data?That's it, as shown below.

I have seen the contents of your road network data, which seems to be different from the contents of the road network data format I generated.As shown below,
Yours,

my,

the contents of the key-value pair are not quite the same, especially the key-value, your road network data is also generated in QGIS, right?I would like to know if the content is different in your plug-in will also have an impact?Thank you !

[jagodki]

Hi,

first please update the plugin. I released a new version with the model I sent you before is integrated as a preprocessing function.

The network data I provide at GitHub as a test data is downloaded via the QuickOSM plugin and is processed in PostGIS/pgRouting for getting a clean node-edge-network. But this task is not necessary. The differences between your and my network are just the attributes. But the attributes of the network will not be used by my plugin.

I tested again your data. I downloaded the whole network of your city that I uploaded in this thread and I downloaded your track data too. Now my processing steps:

1. reproject both data sets to EPSG:32651
2. filter for the first 36 entries in your trajectory (ordered by the timestamp) with the following filter:

"field_2" LIKE '2007-02-20 00:1%' OR "field_2" LIKE '2007-02-20 00:0%'

3. run the new function of my plugin "Clip Network" with a buffer radius of 500 meters to reduce the network.
4. now run the main function of my plugin ("Match Trajectory") with the following settings:
   

After ca. 1 minute (on my 8-9 years old laptop ^^) I got a result:

Please use this data with my settings above for reproducing my results:
test_data.zip

I hope you can reproduce my results...

Sincerely,
Christoph

[love-freedom777]

Thank you very much for your answer!I downloaded the data compression package you sent, maybe you compressed the wrong file in the road network when you compressed the file, you can check that the file in the road network is empty.

[jagodki]

Oh, I am sorry. I copied the wrong file into the zip ^^

Now the complete test data:
test_data.zip

[love-freedom777]

Hello!Using your data can actually generate the trajectory I want.I think I have solved the problem of road network data, but it seems that the data of track points always go wrong.I want to ask you what you did to the GPS track of TXT file I sent to you?Because I looked at the track data that you processed it seems to have a few more columns of key-value pairs, like "fid": 1, "timestamp": "2007/02/20" are these two added by you?And a key pair like this' coordinates' : [354337.172447311808355, 3462929.829346186015755] has it gone through a coordinate projection?How is it generated?Because when I open up my data, it's still the same latitude and longitude.Thanks again for your answer!

[jagodki]

That are good news, because now we come closer to the problem :)

The column fid is inserted automatically by QGIS after saving your trajectory in the geopackage file type. It is an internal id for the features.

The timestamp column was a try to convert the timestamps of the original trajectory into a Date/Time format for a better filtering. Both columns can be ignored and will not be used by the plugin.

I just imported the trajectory into QGIS as a csv file and exported it to geopackage and added the filter. Not more. The coordinates looks a bit strange. But at the moment I am in holidays and have no QGIS or laptop here. Maybe in one or two weeks I can have a closer look on it.

[love-freedom777]

Ok, I have successfully run your plug-in and can generate the desired trajectory diagram.Thank you very much for your patience all the time.I can also generate the corresponding trajectory by intercepting the data inside, as shown in the figure below.

Once again, I would like to express my heartfelt thanks to you. I hope I can consult you modestly if I encounter any relevant problems in the future.

[jagodki]

Congratulations, great to hear :)

I am happy if my plugin can help some people around the world 😺
I know that it can be improved in some things, but it is just a freetime project of mine and time is limited ^^. So if you have any questions or problems in the future, just open a new issue and I will help you as soon as possible (I will close this issue).

[wk2020csu]

This issue is very helpful to me. Thank you very much！