--visualize_world not displaying complete visualisation

[JCSadeghi]

I am having trouble using the --visualize_world option as described here: https://pylot.readthedocs.io/en/stable/visualization.html

However I do not see the colourful map shown on the documentation - instead I see the following:

In addition my command line shows many errors such as the following:

Traceback (most recent call last):
  File "/home/erdos/.local/lib/python3.8/site-packages/erdos/streams.py", line 188, in send
    return self._py_write_stream.send(internal_msg)
WriteStreamError.TimestampError: Error sending message on a7015556-7321-9352-b74e-a6694460cab1

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/erdos/workspace/pylot/pylot/simulation/carla_operator.py", line 251, in send_actor_data
    self.__send_hero_vehicle_data(self.pose_stream, timestamp)
  File "/home/erdos/workspace/pylot/pylot/simulation/carla_operator.py", line 366, in __send_hero_vehicle_data
    stream.send(erdos.Message(timestamp, pose))
  File "/home/erdos/.local/lib/python3.8/site-packages/erdos/streams.py", line 190, in send
    raise Exception("Exception on stream {} ({})".format(
Exception: Exception on stream pose_stream (a7015556-7321-9352-b74e-a6694460cab1)

The other visualisations work correctly for me (e.g. --visualize_detected_obstacles)
If you can reproduce the problem, please could you share a known commit hash when this feature was working?

Here is my config, please let me know if you see any mistakes:

--scenario_runner
--simulator_mode=synchronous
--random_seed=1337
###### Perception config #####
--perfect_obstacle_detection
--nosimulator_obstacle_detection
--simulator_traffic_light_detection
--obstacle_tracking
--min_matching_iou=0.2
--tracker_type=sort
--obstacle_track_max_age=3
--ignore_obstacles_with_short_history=3
--dynamic_obstacle_distance_threshold=50
--static_obstacle_distance_threshold=70
######### Prediction config #########
--prediction
--prediction_type=linear
--prediction_num_past_steps=5
--prediction_num_future_steps=30
######### Planning config #########
--planning_type=frenet_optimal_trajectory
--target_speed=10
--max_speed=20
--d_road_w=0.3
--d_t_s=0.25
--dt=0.1
--obstacle_clearance_fot=0.7
--obstacle_radius=1.5
--maxt=6.0
--mint=4.0
--ko=100
--max_curvature=10.0
--max_accel=7.0
--max_road_width_l=5.0
--max_road_width_r=0.85
--num_waypoints_ahead=50
--goal_location=47.73, 327.07, 0.5
###### Control config #####
--control=pid
--stop_for_vehicles=False
--stop_for_people=False
--stop_for_traffic_lights=False
--steer_gain=1.0
######### Other config #########
--visualize_world

[Morphlng]

Hello! I'm facing the same problem and even more, --visualize_detected_obstacles doesn't work for me too.

Are you using the Carla integrated inside the docker container? I'm using a outside Carla 0.9.13 and when toggle on --visualize_detected_obstacles, this line of code always raise an exception:

pylot/pylot/perception/detection/obstacle.py

Lines 158 to 165 in a71ae92

	elif isinstance(self.bounding_box, BoundingBox3D):
	if self.bounding_box.corners is None:
	raise ValueError(
	'Obstacle {} does not have bbox corners'.format(self.id))
	corners = self.bounding_box.to_camera_view(
	None, frame.camera_setup.get_extrinsic_matrix(),
	frame.camera_setup.get_intrinsic_matrix())
	frame.draw_3d_box(corners, color)

I investigated how exactly the corners are calculated and tried to replace this raising with the calculation process. However, the result doesn't seem right. I also tried to simply modify this line:

corners = self.bounding_box.to_camera_view( 
   ego_transform, frame.camera_setup.get_extrinsic_matrix(), 
   frame.camera_setup.get_intrinsic_matrix()) 

This will show some point, yet not in the place of the vehicles.

[Morphlng]

Here is my config, please let me know if you see any mistakes:

######### Other config #########
--visualize_world

I've solved the problem of --visualize_world not working properly. It's because you have to pass also --visualize_prediction to show the semantic information. The code logic is written in here:

pylot/pylot/debug/visualizer_operator.py

Lines 365 to 376 in a71ae92

	elif sensor_to_display == "PlanningWorld":
	if prediction_camera_msg is None:
	# Top-down prediction is not available. Show planning
	# world on a black image.
	black_img = np.zeros((self._bird_eye_camera_setup.height,
	self._bird_eye_camera_setup.width, 3),
	dtype=np.dtype("uint8"))
	frame = CameraFrame(black_img, 'RGB',
	self._bird_eye_camera_setup)
	else:
	frame = prediction_camera_msg.frame
	frame.transform_to_cityscapes()

To have that message, you must create the stream:

pylot/pylot/operator_creator.py

Lines 888 to 890 in a71ae92

	if prediction_camera_stream is None or not FLAGS.visualize_prediction:
	prediction_camera_stream = erdos.IngestStream()
	streams_to_send_top_on.append(prediction_camera_stream)