Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extends shapes to store bounding boxes #148

Merged
merged 62 commits into from
Nov 12, 2024
Merged
Show file tree
Hide file tree
Changes from 32 commits
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
1929425
Move bounding box calculation into nigiri
MichaelKutzner Oct 23, 2024
21c6bde
Make bounding boxes for segments optional
MichaelKutzner Oct 23, 2024
ceee6db
Fix duplicated entries
MichaelKutzner Oct 23, 2024
67b6bac
Make filtering more explicit
MichaelKutzner Oct 23, 2024
3fe6905
Delay value calculation until needed
MichaelKutzner Oct 23, 2024
a70323c
Simplify route iteration
MichaelKutzner Oct 23, 2024
1aa404a
Update geo dependency
MichaelKutzner Oct 23, 2024
2a9b6bd
Fix out of bounds shape offset
MichaelKutzner Oct 24, 2024
121b67d
Add test case for invalid offset
MichaelKutzner Oct 24, 2024
c195ca9
Improve comment
MichaelKutzner Oct 24, 2024
2923d91
Merge branch 'main' into bounding_boxes
MichaelKutzner Oct 31, 2024
6044729
Unify boxes size
MichaelKutzner Oct 31, 2024
2a34524
Store stops per shapes to process
MichaelKutzner Oct 31, 2024
9dfc2ed
Move code into struct
MichaelKutzner Oct 31, 2024
e8cc9a8
WIP: Perform parallel calculations
MichaelKutzner Nov 4, 2024
d38704c
Make methods const
MichaelKutzner Nov 4, 2024
8642d1b
Update progress bar
MichaelKutzner Nov 4, 2024
3d82cd7
Move nullptr check into struct
MichaelKutzner Nov 4, 2024
e704874
Check for execution policy support
MichaelKutzner Nov 4, 2024
7d81cb5
Remove unused code
MichaelKutzner Nov 4, 2024
46e1bc5
Merge branch 'main' into bounding_boxes
MichaelKutzner Nov 4, 2024
22ba329
Make nullopt more explicit
MichaelKutzner Nov 4, 2024
60dab28
Fix MSVC build
MichaelKutzner Nov 5, 2024
b09d4c0
Fix formatting
MichaelKutzner Nov 5, 2024
ec3bbdd
Fix formatting
MichaelKutzner Nov 5, 2024
eda90d5
Add test for routes containing sequential trips
MichaelKutzner Nov 5, 2024
339ba6f
Rename shape result objects
MichaelKutzner Nov 5, 2024
b1973f7
Make intended usage more obvious
MichaelKutzner Nov 5, 2024
a2c7d75
Move data on last usage
MichaelKutzner Nov 5, 2024
85d4638
Update variable names
MichaelKutzner Nov 5, 2024
c37e792
Renane variable
MichaelKutzner Nov 5, 2024
b6a1b97
Add missing const
MichaelKutzner Nov 5, 2024
289a000
Apply suggested newline
MichaelKutzner Nov 6, 2024
ecf11c6
Improve readability
MichaelKutzner Nov 6, 2024
38b55a6
Update parallel computations
MichaelKutzner Nov 6, 2024
55925f9
Improve method names
MichaelKutzner Nov 6, 2024
b3272d8
Split shape boxes into two files
MichaelKutzner Nov 6, 2024
e1e8578
Simplify parallel usage
MichaelKutzner Nov 6, 2024
53e9a9e
Rename variables
MichaelKutzner Nov 6, 2024
ec8f3ed
WIP: Change code to tasks
MichaelKutzner Nov 7, 2024
7234e20
Fix use after free
MichaelKutzner Nov 7, 2024
e1e003a
Create offset entries
MichaelKutzner Nov 7, 2024
ea7cddf
Create bounding boxes
MichaelKutzner Nov 7, 2024
20dc322
Remove mutex
MichaelKutzner Nov 7, 2024
ee37475
Delete old code
MichaelKutzner Nov 7, 2024
05ce7c7
Rename variables
MichaelKutzner Nov 7, 2024
d75fb44
Remove intermediate struct
MichaelKutzner Nov 7, 2024
ced2e8d
Fix compile issues
MichaelKutzner Nov 7, 2024
4f09b87
Fix invalid range
MichaelKutzner Nov 7, 2024
edbfeda
Fix empty bound boxes
MichaelKutzner Nov 8, 2024
16ed908
Fix empty bound boxes for sequential trips
MichaelKutzner Nov 8, 2024
dd18872
Reduce stored data for simple sequential trips
MichaelKutzner Nov 8, 2024
94b2af4
Fix formatting
MichaelKutzner Nov 8, 2024
1fac9fb
Fix size type
MichaelKutzner Nov 8, 2024
1437f02
Attempt to fix macos build
MichaelKutzner Nov 8, 2024
244a1c0
wip
felixguendling Nov 11, 2024
ce3dcc8
Apply pending changes
MichaelKutzner Nov 11, 2024
584318d
Delete old code
MichaelKutzner Nov 11, 2024
426dd8e
Remove unused includes
MichaelKutzner Nov 12, 2024
f39b5e4
Cleanup code
MichaelKutzner Nov 12, 2024
f5fdca7
Add missing check for end element
MichaelKutzner Nov 12, 2024
3d83c8e
Update progress status text
MichaelKutzner Nov 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .pkg
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
[geo]
[email protected]:motis-project/geo.git
branch=master
commit=0a14addf42e91b267906a156c9c2564935c03eaf
commit=cee17208dba98f5f3ddaf78d0bfff9c083557ba3
[utl]
[email protected]:motis-project/utl.git
branch=master
Expand Down
4 changes: 2 additions & 2 deletions include/nigiri/common/interval.h
Original file line number Diff line number Diff line change
Expand Up @@ -47,8 +47,8 @@ struct interval {
t_ -= x;
return *this;
}
iterator operator+(difference_type const x) const { return *this += x; }
iterator operator-(difference_type const x) const { return *this -= x; }
iterator operator+(difference_type const x) const { return {t_ + x}; }
iterator operator-(difference_type const x) const { return {t_ - x}; }
friend difference_type operator-(iterator const& a, iterator const& b) {
return static_cast<difference_type>(cista::to_idx(a.t_) -
cista::to_idx(b.t_));
Expand Down
35 changes: 30 additions & 5 deletions include/nigiri/loader/gtfs/shape_prepare.h
Original file line number Diff line number Diff line change
@@ -1,18 +1,43 @@
#pragma once

#include "nigiri/loader/gtfs/shape.h"
#include "geo/box.h"

#include "nigiri/loader/gtfs/trip.h"
#include "nigiri/types.h"

namespace nigiri {
struct shape_loader_state;
struct shapes_storage;
struct timetable;
} // namespace nigiri

namespace nigiri::loader::gtfs {

void calculate_shape_offsets(timetable const&,
shapes_storage&,
vector_map<gtfs_trip_idx_t, trip> const&,
shape_loader_state const&);
struct shape_prepare {
struct shape_results {
struct result {
explicit result(stop_seq_t const* stop_seq,
std::vector<double> const* distances);
stop_seq_t const* stop_seq_;
std::vector<double> const* distances_;
shape_offset_idx_t offset_idx_;
std::vector<geo::box> boxes_;
};

shape_idx_t shape_idx_;
std::vector<result> results_;
};

shape_prepare(shape_loader_state const&,
vector_map<gtfs_trip_idx_t, trip> const&,
shapes_storage&);
void calculate_results(timetable const&, shape_loader_state const&);
void write_trip_shape_offsets(vector_map<gtfs_trip_idx_t, trip> const&) const;
void write_route_boxes(timetable const&) const;
shape_idx_t index_offset_;
MichaelKutzner marked this conversation as resolved.
Show resolved Hide resolved
std::vector<shape_results> shape_results_;
shapes_storage& shapes_;
bool results_ready_;
};

} // namespace nigiri::loader::gtfs
5 changes: 5 additions & 0 deletions include/nigiri/shapes_storage.h
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
#pragma once

#include <filesystem>
#include <optional>
#include <span>

#include "cista/containers/pair.h"

#include "geo/box.h"
#include "geo/latlng.h"

#include "nigiri/types.h"
Expand All @@ -23,6 +25,8 @@ struct shapes_storage {
shape_offset_idx_t add_offsets(std::vector<shape_offset_t> const&);
void add_trip_shape_offsets(
trip_idx_t, cista::pair<shape_idx_t, shape_offset_idx_t> const&);
geo::box get_bounding_box(route_idx_t) const;
std::optional<geo::box> get_bounding_box(route_idx_t, std::size_t) const;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

std::size_t ohne Parametername? nicht klar


cista::mmap::protection mode_;
std::filesystem::path p_;
Expand All @@ -31,6 +35,7 @@ struct shapes_storage {
mm_vecvec<shape_offset_idx_t, shape_offset_t, std::uint64_t> offsets_;
mm_vec_map<trip_idx_t, cista::pair<shape_idx_t, shape_offset_idx_t>>
trip_offset_indices_;
mm_vecvec<route_idx_t, geo::box, std::uint64_t> boxes_;
};

} // namespace nigiri
17 changes: 15 additions & 2 deletions src/loader/gtfs/load_timetable.cc
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
#include <charconv>
#include <filesystem>
#include <numeric>
#include <optional>
felixguendling marked this conversation as resolved.
Show resolved Hide resolved
#include <string>

#include "utl/get_or_create.h"
Expand Down Expand Up @@ -261,8 +262,15 @@ void load_timetable(loader_config const& config,
{source_file_idx, trp.from_line_, trp.to_line_},
train_nr, stop_seq_numbers);
}
if (shapes_data != nullptr) {
calculate_shape_offsets(tt, *shapes_data, trip_data.data_, shape_states);

auto shape_results =
(shapes_data == nullptr)
? std::nullopt
: std::make_optional(
shape_prepare{shape_states, trip_data.data_, *shapes_data});
if (shape_results.has_value()) {
shape_results->calculate_results(tt, std::move(shape_states));
shape_results->write_trip_shape_offsets(trip_data.data_);
}

auto const timer = scoped_timer{"loader.gtfs.routes.build"};
Expand Down Expand Up @@ -367,6 +375,11 @@ void load_timetable(loader_config const& config,
progress_tracker->increment();
}

// Build bounding boxes
if (shape_results.has_value()) {
shape_results->write_route_boxes(tt);
}

// Build location_routes map
for (auto l = tt.location_routes_.size(); l != tt.n_locations(); ++l) {
tt.location_routes_.emplace_back(location_routes[location_idx_t{l}]);
Expand Down
Loading
Loading