Crowding data confidence determination #673

PaulJKim · 2023-08-16T19:57:14Z

Summary of changes

Asana Ticket: 🟠🧑‍🤝‍🧑🔊 "High Confidence" filter logic

This PR begins calculation of crowding data confidence level. High confidence crowding data is currently defined as

It's an OL prediction
The train has left the prior stop and is en route to the prediction's stop

…fidence level

github-actions · 2023-08-16T20:24:22Z

Coverage of commit `db7d146`

Summary coverage rate:
  lines......: 76.3% (1918 of 2514 lines)
  functions..: 77.5% (520 of 671 functions)
  branches...: no data found

Files changed coverage rate:
                                                             |Lines       |Functions  |Branches    
  Filename                                                   |Rate     Num|Rate    Num|Rate     Num
  =================================================================================================
  lib/content/audio/predictions.ex                           |83.1%     59| 100%     1|    -      0
  lib/content/message/predictions.ex                         |94.4%     72| 100%    10|    -      0
  lib/engine/locations.ex                                    |58.3%     48|69.2%    13|    -      0
  lib/engine/predictions.ex                                  |96.7%     30| 100%     8|    -      0
  lib/fake/httpoison.ex                                      |72.1%     43|57.1%     7|    -      0
  lib/locations/location.ex                                  |50.0%      2| 100%     2|    -      0
  lib/realtime_signs.ex                                      | 100%     13| 100%     3|    -      0
  lib/signs/utilities/audio.ex                               |87.7%     73| 100%     7|    -      0

Download coverage report

github-actions · 2023-08-17T19:15:05Z

Coverage of commit `165b961`

Summary coverage rate:
  lines......: 76.8% (1935 of 2518 lines)
  functions..: 77.5% (520 of 671 functions)
  branches...: no data found

Files changed coverage rate:
                                                             |Lines       |Functions  |Branches    
  Filename                                                   |Rate     Num|Rate    Num|Rate     Num
  =================================================================================================
  lib/content/audio/predictions.ex                           |83.1%     59| 100%     1|    -      0
  lib/content/message/predictions.ex                         |94.4%     72| 100%    10|    -      0
  lib/engine/locations.ex                                    |61.5%     52|69.2%    13|    -      0
  lib/locations/location.ex                                  |50.0%      2| 100%     2|    -      0
  lib/signs/utilities/audio.ex                               |89.0%     73| 100%     7|    -      0

Download coverage report

github-actions · 2023-08-17T19:26:56Z

Coverage of commit `1dd530d`

Summary coverage rate:
  lines......: 76.9% (1934 of 2516 lines)
  functions..: 77.5% (520 of 671 functions)
  branches...: no data found

Files changed coverage rate:
                                                             |Lines       |Functions  |Branches    
  Filename                                                   |Rate     Num|Rate    Num|Rate     Num
  =================================================================================================
  lib/content/audio/predictions.ex                           |83.1%     59| 100%     1|    -      0
  lib/content/message/predictions.ex                         |95.7%     70| 100%    10|    -      0
  lib/engine/locations.ex                                    |61.5%     52|69.2%    13|    -      0
  lib/locations/location.ex                                  |50.0%      2| 100%     2|    -      0
  lib/signs/utilities/audio.ex                               |89.0%     73| 100%     7|    -      0

Download coverage report

github-actions · 2023-08-17T19:36:43Z

Coverage of commit `a5f1f17`

Summary coverage rate:
  lines......: 76.9% (1934 of 2516 lines)
  functions..: 77.5% (520 of 671 functions)
  branches...: no data found

Files changed coverage rate:
                                                             |Lines       |Functions  |Branches    
  Filename                                                   |Rate     Num|Rate    Num|Rate     Num
  =================================================================================================
  lib/content/audio/predictions.ex                           |83.1%     59| 100%     1|    -      0
  lib/content/message/predictions.ex                         |95.7%     70| 100%    10|    -      0
  lib/engine/locations.ex                                    |61.5%     52|69.2%    13|    -      0
  lib/locations/location.ex                                  |50.0%      2| 100%     2|    -      0
  lib/signs/utilities/audio.ex                               |89.0%     73| 100%     7|    -      0

Download coverage report

panentheos · 2023-08-28T16:52:52Z

lib/content/audio/predictions.ex

+        if predictions.crowding_data_confidence == :high do
+          # TODO: Pass along crowding data classification once available
+          [
+            %Approaching{
+              destination: predictions.destination,
+              trip_id: predictions.trip_id,
+              platform: predictions.platform,
+              route_id: predictions.route_id,
+              new_cars?: predictions.new_cars?
+            }
+          ]
+        else
+          [
+            %Approaching{
+              destination: predictions.destination,
+              trip_id: predictions.trip_id,
+              platform: predictions.platform,
+              route_id: predictions.route_id,
+              new_cars?: predictions.new_cars?
+            }
+          ]
+        end


Thinking out loud: Not sure how this will end up looking in the following PRs but hopefully there's an opportunity to unify some of this logic a bit.

Yep! This does get unified down the line

panentheos · 2023-08-28T16:58:07Z

lib/content/message/predictions.ex

+    crowding_data_confidence =
+      calculate_crowding_data_confidence(
+        prediction,
+        Engine.Locations.for_vehicle(prediction.vehicle_id)


Doing this lookup here means we'll end up grabbing location info for every prediction message, every tick, even ones far away or on other lines. Granted, it's an indexed operation, so it may be fast enough, but it's not needed the vast majority of the time. What about storing the vehicle_id in the Message, and then performing this calculation only when we actually need the result?

Yeah that's a good point. I did end up thinking about that later down the line in a later PR and changed the logic so that we only fetch location if it's an Orange Line prediction. Take a look at this logic for do_crowding/1 and let me know what you think

That does improve things. And the lookup should be fast enough, so it's probably not worth worrying about.

panentheos · 2023-08-28T17:01:00Z

lib/engine/locations.ex

    }
  end

-  defp status_to_atom(status) do
+  defp map_multi_carriage_details(multi_carriage_details) do


nit: This might be better named something like parse_carriage_details(carriage_details)

panentheos · 2023-08-28T17:04:55Z

lib/locations/location.ex

-            trip_id: nil,
-            consist: []
+defmodule Locations do
+  defmodule CarriageDetails do


There's a pretty strong pattern of one-module-per-file in non-test code. Perhaps this should live in a new locations/carriage_details.ex file?

PaulJKim · 2023-08-28T20:21:01Z

Whoops sorry @panentheos didn't realize I forgot to push the changes from your last two comments. Just did that

panentheos

Looks good.

panentheos · 2023-08-28T20:26:59Z

lib/locations/location.ex

@@ -14,6 +15,7 @@ defmodule Locations.Location do
          timestamp: DateTime.t() | nil,
          route_id: String.t() | nil,
          trip_id: String.t() | nil,
-          consist: list()
+          consist: list(),
+          multi_carriage_details: list(CarriageDetails.t())


This type needs the fully qualified name now.

PaulJKim added 9 commits August 15, 2023 17:24

Add locations feed engine

373e511

Add unit test for 'deletion' logic

e075fba

Add a few more tests

67d4d85

remmove io puts

abf5778

fix typedef

7b85ec7

Begin reading in multi_carriage_details field

8e7d492

Calculate crowding data confidence level

7e98cfd

Branch arrival and approaching audio logic based on crowding data con…

f6ce48b

…fidence level

Add a todo for where to incorporate read loop selection logic

db7d146

Merge branch 'main' into pk/crowding-data-confidence-determination

165b961

Clean up calculation logic

1dd530d

adjust mapping logic to match actual schema

a5f1f17

PaulJKim requested a review from panentheos August 17, 2023 19:39

panentheos reviewed Aug 28, 2023

View reviewed changes

Address PR comments

edd033c

panentheos approved these changes Aug 28, 2023

View reviewed changes

panentheos reviewed Aug 28, 2023

View reviewed changes

Use fully qualified name for CarriageDetails type

fe97d8d

PaulJKim merged commit f441dd1 into main Aug 28, 2023
1 check passed

panentheos deleted the pk/crowding-data-confidence-determination branch February 21, 2024 19:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Crowding data confidence determination #673

Crowding data confidence determination #673

PaulJKim commented Aug 16, 2023

github-actions bot commented Aug 16, 2023

github-actions bot commented Aug 17, 2023

github-actions bot commented Aug 17, 2023

github-actions bot commented Aug 17, 2023

panentheos Aug 28, 2023

PaulJKim Aug 28, 2023

panentheos Aug 28, 2023

PaulJKim Aug 28, 2023

panentheos Aug 28, 2023

panentheos Aug 28, 2023

panentheos Aug 28, 2023

PaulJKim commented Aug 28, 2023

panentheos left a comment

panentheos Aug 28, 2023

Crowding data confidence determination #673

Crowding data confidence determination #673

Conversation

PaulJKim commented Aug 16, 2023

Summary of changes

github-actions bot commented Aug 16, 2023

Coverage of commit db7d146

github-actions bot commented Aug 17, 2023

Coverage of commit 165b961

github-actions bot commented Aug 17, 2023

Coverage of commit 1dd530d

github-actions bot commented Aug 17, 2023

Coverage of commit a5f1f17

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

PaulJKim commented Aug 28, 2023

panentheos left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Coverage of commit `db7d146`

Coverage of commit `165b961`

Coverage of commit `1dd530d`

Coverage of commit `a5f1f17`