[Bugfix] pycharm debug loading project modules as standard library mo…

…dules (#2126)

* Bugfix types module for pycharm.

* Fix sstudio.types module imports.

* Fix envision.types.

* Make format.

* Rename math to core_math.

* Allow compatibility import of envision.types.

* Allow compatibility import of smarts.sstudio.types

* Make format.

* Update utils.math to utils.core_math.

* Rename logging to core_logging.

* Improve compatability.

* Make format.

* Update changelog.

* Fix tests and type test.

* Fix format test.

CHANGELOG.md

@@ -12,6 +12,7 @@ Copy and pasting the git commit messages is __NOT__ enough.
 ### Added
 ### Changed
 - `VehicleIndex.build_agent_vehicle()` no longer has `filename` and `surface_patches` parameters.
+- The following modules have been renamed: `envision.types` -> `envision.etypes`, `smarts.core.utils.logging` -> `smarts.core.utils.core_logging`, `smarts.core.utils.math` -> `smarts.core.utils.core_math`, `smarts.sstudio.types` -> `smarts.sstudio.sstypes`. For compatibility reasons they can still be imported by their original module name.
 ### Deprecated
 ### Fixed
 - Fixed an issue where `AgentInterface.vehicle_type` would not affect agent vehicles when attempting to take over an existing vehicle.
@@ -26,6 +27,7 @@ Copy and pasting the git commit messages is __NOT__ enough.
 - Fixed a `pkg_resources` deprecation warning in `python3.10` and up.
 - Fixed the envision camera to center on the map which was broken due to a change in the `gltf` metadata hierarchy.
 - Fixed an issue where you would need to install `waymo` in order to use any kind of dataset histories.
+- Fixed an issue where Pycharm would load `smarts/sstudio/types` as the integrated `types` module. See #2125.
 ### Removed
 ### Security
 

docs/sim/bubbles.rst

@@ -22,7 +22,7 @@ Bubbles can be fixed to a static location defined either as an edge or a positio
 
 .. code-block:: python
 
-    import smarts.sstudio.types as t
+    import smarts.sstudio.sstypes as t
     zoo_agent_actor = t.SocialAgentActor(
         # Unique agent name
         name="zoo-agent",
@@ -45,7 +45,7 @@ Bubbles that are vehicle-relative can be attached to specific actors by specifyi
 
 .. code-block:: python
 
-    import smarts.sstudio.types as t
+    import smarts.sstudio.sstypes as t
     t.Bubble(
         ...,
         # The target of the bubble to follow the given actor
@@ -61,7 +61,7 @@ There is currently no interface for dynamically-created bubbles. However, if the
 
 .. code-block:: python
     
-    import smarts.sstudio.types as t
+    import smarts.sstudio.sstypes as t
     scenario_iter = Scenario.scenario_variations(path)
     scenario = next(scenario)
     scenario.bubbles.append(

docs/sim/minimal_scenario_studio.py

@@ -1,6 +1,6 @@
 from pathlib import Path
 
-from smarts.sstudio import types as t
+from smarts.sstudio import sstypes as t
 from smarts.sstudio import gen_scenario
 
 traffic_actor = t.TrafficActor(

docs/sim/scenario_studio.rst

@@ -7,9 +7,9 @@
 Scenario Studio
 ===============
 
-The Scenario Studio ``sstudio`` of SMARTS is a stand alone package :mod:`~smarts.sstudio` that supports flexible and expressive scenario specification. The ``sstudio`` domain specific language (DSL) has a simple ontology defined by :mod:`~smarts.sstudio.types` to help express your scenario needs in SMARTS.
+The Scenario Studio ``sstudio`` of SMARTS is a stand alone package :mod:`~smarts.sstudio` that supports flexible and expressive scenario specification. The ``sstudio`` domain specific language (DSL) has a simple ontology defined by :mod:`~smarts.sstudio.sstypes` to help express your scenario needs in SMARTS.
 
-A scenario is formed by combining several inputs, such as (i) road map, (ii) traffic, (iii) social agents, (iv) ego agents, (v) bubbles, (vi) friction patches, and (vii) traffic histories. Refer to :class:`~smarts.sstudio.types.scenario.Scenario`.
+A scenario is formed by combining several inputs, such as (i) road map, (ii) traffic, (iii) social agents, (iv) ego agents, (v) bubbles, (vi) friction patches, and (vii) traffic histories. Refer to :class:`~smarts.sstudio.sstypes.scenario.Scenario`.
 
 SMARTS interacts with road maps abstractly (via polymorphism) through the :class:`~smarts.core.road_map.RoadMap` interface in order to support multiple map formats and allow for extending to support new map formats.
 
@@ -21,10 +21,10 @@ A typical scenario creation workflow is as follows.
 2. Design the map.
 
    * For a SUMO road network, create the map using `netedit <https://sumo.dlr.de/docs/NETEDIT.html>`_ and save it in the new scenario folder.
-   * For other road networks, define and instantiate a :class:`~smarts.sstudio.types.map_spec.MapSpec` object in ``scenario.py``. Pass the ``MapSpec`` object to :class:`~smarts.sstudio.types.scenario.Scenario` in ``scenario.py``.
+   * For other road networks, define and instantiate a :class:`~smarts.sstudio.sstypes.map_spec.MapSpec` object in ``scenario.py``. Pass the ``MapSpec`` object to :class:`~smarts.sstudio.sstypes.scenario.Scenario` in ``scenario.py``.
 
-3. Add traffic, social agents, ego agents, friction patches, map spec, etc, to :class:`~smarts.sstudio.types.scenario.Scenario` in ``scenario.py``.
-4. Feed the created :class:`~smarts.sstudio.types.scenario.Scenario` object to :func:`~smarts.sstudio.genscenario.gen_scenario` in ``scenario.py`` to generate the scenario. 
+3. Add traffic, social agents, ego agents, friction patches, map spec, etc, to :class:`~smarts.sstudio.sstypes.scenario.Scenario` in ``scenario.py``.
+4. Feed the created :class:`~smarts.sstudio.sstypes.scenario.Scenario` object to :func:`~smarts.sstudio.genscenario.gen_scenario` in ``scenario.py`` to generate the scenario. 
 
    .. code:: python
 
@@ -48,7 +48,7 @@ Following sections below explain how to handle and edit (i) traffic, (ii) social
 Traffic
 -------
 
-A minimal ``scenario.py`` instantiating a :class:`~smarts.sstudio.types.traffic.Traffic` object.
+A minimal ``scenario.py`` instantiating a :class:`~smarts.sstudio.sstypes.traffic.Traffic` object.
 
 .. literalinclude:: ./minimal_scenario_studio.py
    :language: python
@@ -61,17 +61,17 @@ Simply run the ``scenario.py`` file as a regular Python script to generate the s
 
 Traffic vehicles are controlled by either ``SUMO`` or ``SMARTS`` engine. Defaults to ``engine="SUMO"``. ``engine="SUMO"`` can only be used on SUMO road networks. For other map types use ``engine="SMARTS"``.
 
-:class:`~smarts.sstudio.types.traffic.Flow` is used to generate repeated vehicle runs on the same route. Vehicle route, departure rate, and behavior, can be configured here.
+:class:`~smarts.sstudio.sstypes.traffic.Flow` is used to generate repeated vehicle runs on the same route. Vehicle route, departure rate, and behavior, can be configured here.
 
 The example above simply uses a random route ``route=RandomRoute()``. A more specific route may be used such as :python:`Route(begin=("gneE72", 0, "random"), end=("edge2", 1, "max"))` which defines the edge id, lane id, and offset into the lane, to designate the start and end vehicle positions.
 
-:class:`~smarts.sstudio.types.actor.traffic_actor.TrafficActor` is used to specify a spec for traffic actors (e.g. Vehicles, Pedestrians, etc). The defaults provided are for a car.
+:class:`~smarts.sstudio.sstypes.actor.traffic_actor.TrafficActor` is used to specify a spec for traffic actors (e.g. Vehicles, Pedestrians, etc). The defaults provided are for a car.
 Acceleration, deceleration, speed distribution, imperfection distribution, and other configurations, can be specified for the traffic.
 
 When :func:`~smarts.sstudio.genscenario.gen_scenario` is executed, a dir named "traffic" will be created under ``output_dir`` which contains background vehicle and route definitions.
 
 .. important::
-    If you want to train a policy on one scenario, remember to set the ``end`` time of :class:`~smarts.sstudio.types.traffic.Flow` larger or equal to your expected training time, since SMARTS will continue the flow after each ``reset`` call. 
+    If you want to train a policy on one scenario, remember to set the ``end`` time of :class:`~smarts.sstudio.sstypes.traffic.Flow` larger or equal to your expected training time, since SMARTS will continue the flow after each ``reset`` call. 
 
     However, if there are multiple scenarios to train for one worker, you can relax this restriction since after the scenario change, the flow will also be reset to the beginning time.
 
@@ -143,7 +143,7 @@ Scenario Studio also allows generation of *missions* for ego agents to complete.
 Friction patches
 ----------------
 
-The Scenario Studio of SMARTS also allows the generation of *friction patches* which consists of a list of *surface patches* for ego agents and social agents. These surface patches uses :class:`~smarts.sstudio.types.zone.PositionalZone` as in the case of bubbles. When we run :func:`~smarts.sstudio.genscenario.gen_scenario` passing in ``friction_maps``, a `"`friction_map.pkl"`` file will be created under the output dir.
+The Scenario Studio of SMARTS also allows the generation of *friction patches* which consists of a list of *surface patches* for ego agents and social agents. These surface patches uses :class:`~smarts.sstudio.sstypes.zone.PositionalZone` as in the case of bubbles. When we run :func:`~smarts.sstudio.genscenario.gen_scenario` passing in ``friction_maps``, a `"`friction_map.pkl"`` file will be created under the output dir.
 
 .. code-block:: python
 

envision/__init__.py

@@ -17,3 +17,9 @@
 # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
 # THE SOFTWARE.
+
+import sys
+
+from . import etypes as types
+
+sys.modules[f"{__name__}.{types=}".partition("=")[0]] = types

envision/client.py

@@ -32,11 +32,11 @@
 
 import numpy as np
 
-from envision import types
+from envision import etypes
 from envision.client_config import EnvisionStateFilter
 from envision.data_formatter import EnvisionDataFormatter, EnvisionDataFormatterArgs
+from smarts.core.utils.core_logging import suppress_websocket
 from smarts.core.utils.file import unpack
-from smarts.core.utils.logging import suppress_websocket
 
 
 @dataclass
@@ -219,7 +219,7 @@ def _connect(
         if data_formatter_args:
             data_formatter = EnvisionDataFormatter(data_formatter_args)
 
-        def optionally_serialize_and_write(state: Union[types.State, str], ws):
+        def optionally_serialize_and_write(state: Union[etypes.State, str], ws):
             # if not already serialized
             if not isinstance(state, str):
                 if data_formatter:
@@ -307,7 +307,7 @@ def run_socket(endpoint, wait_between_retries):
         except (BrokenPipeError, KeyboardInterrupt, EOFError):
             pass
 
-    def send(self, state: Union[types.State, types.Preamble]):
+    def send(self, state: Union[etypes.State, etypes.Preamble]):
         """Send the given envision state to the remote as the most recent state."""
         if not self._headless and self._process.is_alive():
             self._state_queue.put(state)

envision/data_formatter.py

@@ -41,7 +41,7 @@
 
 import numpy as np
 
-from envision.types import (
+from envision.etypes import (
     SignalLightState,
     SignalState,
     State,

envision/tests/test_data_formatter.py

@@ -27,15 +27,15 @@
 import numpy as np
 import pytest
 
-import smarts.sstudio.types as t
+import smarts.sstudio.sstypes as t
 from envision.client import Client
 from envision.data_formatter import (
     EnvisionDataFormatter,
     EnvisionDataFormatterArgs,
     Operation,
     _formatter_map,
 )
-from envision.types import State, TrafficActorState, TrafficActorType, VehicleType
+from envision.etypes import State, TrafficActorState, TrafficActorType, VehicleType
 from smarts.core import seed
 from smarts.core.agent_interface import AgentInterface
 from smarts.core.controllers import ActionSpaceType
@@ -302,10 +302,10 @@ def sim_data():
         [],
         [4],
         {
-            0: "car-west_0_0-east_0_max--41457-668134-0-0.0",
+            0: "car-west_0_0-east_0_max--22530-248058-0-0.0",
             1: None,
-            2: "car-west_1_0-east_1_max--31231--18481-1-0.0",
-            3: "car-west_2_0-east_2_max-674625--72317-2-0.0",
+            2: "car-west_1_0-east_1_max-810328-635068-1-0.0",
+            3: "car-west_2_0-east_2_max--22462--73609-2-0.0",
             4: "AGENT_1",
         },
         [],
@@ -398,7 +398,7 @@ def side_effect(state: State):
         es = EnvisionDataFormatter(EnvisionDataFormatterArgs(None))
         assert state.scenario_id is not None
         with mock.patch(
-            "envision.types.State.scenario_id", new_callable=PropertyMock
+            "envision.etypes.State.scenario_id", new_callable=PropertyMock
         ) as state_scenario_id:
             state_scenario_id.return_value = None
             es.add_any(state)

examples/direct/bubble_example.py

@@ -15,8 +15,8 @@
 from smarts.core.observations import Observation
 from smarts.core.scenario import Scenario
 from smarts.core.smarts import SMARTS
-from smarts.core.utils.logging import timeit
-from smarts.sstudio.types import Bubble, PositionalZone, SocialAgentActor
+from smarts.core.utils.core_logging import timeit
+from smarts.sstudio.sstypes import Bubble, PositionalZone, SocialAgentActor
 from smarts.zoo.agent_spec import AgentSpec
 from smarts.zoo.registry import register
 
@@ -177,7 +177,7 @@ class ObservationState:
         def __init__(self) -> None:
             self.last_observations: Dict[str, Observation] = {}
 
-        def observation_callback(self, obs: Observation):
+        def observation_callback(self, obs: Dict[str, Observation]):
             self.last_observations = obs
 
     obs_state = ObservationState()

examples/direct/traffic_histories_vehicle_replacement.py

@@ -16,7 +16,7 @@
 from smarts.core.smarts import SMARTS
 from smarts.core.traffic_history import TrafficHistory
 from smarts.core.traffic_history_provider import TrafficHistoryProvider
-from smarts.core.utils.math import radians_to_vec, rounder_for_dt
+from smarts.core.utils.core_math import radians_to_vec, rounder_for_dt
 from smarts.zoo.agent_spec import AgentSpec
 
 logging.basicConfig(level=logging.INFO)

scenarios/open_drive/od_4lane/scenario.py

@@ -1,7 +1,7 @@
 from pathlib import Path
 
 from smarts.sstudio.genscenario import gen_scenario
-from smarts.sstudio.types import Scenario
+from smarts.sstudio.sstypes import Scenario
 
 gen_scenario(
     Scenario(),

scenarios/open_drive/od_newmarket/scenario.py

@@ -1,7 +1,7 @@
 from pathlib import Path
 
 from smarts.sstudio.genscenario import gen_scenario
-from smarts.sstudio.types import Scenario
+from smarts.sstudio.sstypes import Scenario
 
 gen_scenario(
     Scenario(),

scenarios/sumo/figure_eight/scenario.py

@@ -1,7 +1,7 @@
 from pathlib import Path
 
 from smarts.sstudio.genscenario import gen_scenario
-from smarts.sstudio.types import Scenario
+from smarts.sstudio.sstypes import Scenario
 
 gen_scenario(
     scenario=Scenario(),

scenarios/sumo/intersections/1_to_1lane_left_turn_c_agents_1/scenario.py

@@ -23,7 +23,7 @@
 from pathlib import Path
 
 from smarts.sstudio.genscenario import gen_scenario
-from smarts.sstudio.types import (
+from smarts.sstudio.sstypes import (
     Flow,
     MapZone,
     Mission,

scenarios/sumo/intersections/1_to_2lane_left_turn_c_agents_1/scenario.py

@@ -23,7 +23,7 @@
 from pathlib import Path
 
 from smarts.sstudio import gen_scenario
-from smarts.sstudio.types import (
+from smarts.sstudio.sstypes import (
     Flow,
     Mission,
     Route,

scenarios/sumo/intersections/1_to_2lane_left_turn_t_agents_1/scenario.py

@@ -23,7 +23,7 @@
 from pathlib import Path
 
 from smarts.sstudio import gen_scenario
-from smarts.sstudio.types import (
+from smarts.sstudio.sstypes import (
     Flow,
     Mission,
     Route,

scenarios/sumo/intersections/1_to_3lane_left_turn_middle_lane_c_agents_1/scenario.py

@@ -23,7 +23,7 @@
 from pathlib import Path
 
 from smarts.sstudio import gen_scenario
-from smarts.sstudio.types import (
+from smarts.sstudio.sstypes import (
     Flow,
     Mission,
     Route,

scenarios/sumo/intersections/1_to_3lane_left_turn_sumo_c_agents_1/scenario.py

@@ -23,7 +23,7 @@
 from pathlib import Path
 
 from smarts.sstudio import gen_scenario
-from smarts.sstudio.types import (
+from smarts.sstudio.sstypes import (
     Flow,
     Mission,
     Route,

scenarios/sumo/intersections/2lane/scenario.py

@@ -1,7 +1,7 @@
 from pathlib import Path
 
 from smarts.sstudio.genscenario import gen_scenario
-from smarts.sstudio.types import (
+from smarts.sstudio.sstypes import (
     Distribution,
     Flow,
     JunctionModel,

scenarios/sumo/intersections/2lane_circle/scenario.py

@@ -1,6 +1,6 @@
 from pathlib import Path
 
-import smarts.sstudio.types as t
+import smarts.sstudio.sstypes as t
 from smarts.sstudio import gen_scenario
 
 laner_agent = t.SocialAgentActor(

scenarios/sumo/intersections/4lane/scenario.py

@@ -1,7 +1,7 @@
 from pathlib import Path
 
 from smarts.sstudio.genscenario import gen_scenario
-from smarts.sstudio.types import (
+from smarts.sstudio.sstypes import (
     EndlessMission,
     Flow,
     JunctionEdgeIDResolver,

scenarios/sumo/intersections/6lane/scenario.py

@@ -2,10 +2,10 @@
 from pathlib import Path
 
 from smarts.sstudio import gen_scenario
-from smarts.sstudio.types import Mission, Route, Scenario, SocialAgentActor, Traffic
-from smarts.sstudio.types.entry_tactic import IdEntryTactic
-from smarts.sstudio.types.route import RandomRoute
-from smarts.sstudio.types.traffic import Trip
+from smarts.sstudio.sstypes import Mission, Route, Scenario, SocialAgentActor, Traffic
+from smarts.sstudio.sstypes.entry_tactic import IdEntryTactic
+from smarts.sstudio.sstypes.route import RandomRoute
+from smarts.sstudio.sstypes.traffic import Trip
 
 
 def actor_gen(id_):

scenarios/sumo/merge/3lane_agents_1/scenario.py

@@ -23,7 +23,7 @@
 from pathlib import Path
 
 from smarts.sstudio import gen_scenario
-from smarts.sstudio.types import (
+from smarts.sstudio.sstypes import (
     Flow,
     Mission,
     Route,

scenarios/sumo/merge/3lane_agents_2/scenario.py

@@ -23,7 +23,7 @@
 from pathlib import Path
 
 from smarts.sstudio import gen_scenario
-from smarts.sstudio.types import (
+from smarts.sstudio.sstypes import (
     Flow,
     Mission,
     Route,

scenarios/sumo/straight/3lane_cruise_agents_1/scenario.py

@@ -23,7 +23,7 @@
 from pathlib import Path
 
 from smarts.sstudio import gen_scenario
-from smarts.sstudio.types import (
+from smarts.sstudio.sstypes import (
     Flow,
     Mission,
     Route,

scenarios/sumo/straight/3lane_cruise_agents_3/scenario.py

@@ -23,7 +23,7 @@
 from pathlib import Path
 
 from smarts.sstudio import gen_scenario
-from smarts.sstudio.types import (
+from smarts.sstudio.sstypes import (
     Flow,
     Mission,
     Route,

scenarios/sumo/straight/3lane_cut_in_agents_1/scenario.py

@@ -25,7 +25,7 @@
 from numpy import random
 
 from smarts.sstudio import gen_scenario
-from smarts.sstudio.types import (
+from smarts.sstudio.sstypes import (
     Distribution,
     Flow,
     Mission,