Skip to content

feat: add default cam properties to vision service#862

Open
HipsterBrown wants to merge 3 commits into
mainfrom
nhehr/add-default-cam-properties-to-vision-service
Open

feat: add default cam properties to vision service#862
HipsterBrown wants to merge 3 commits into
mainfrom
nhehr/add-default-cam-properties-to-vision-service

Conversation

@HipsterBrown

Copy link
Copy Markdown
Member

To resolve the issue of floating ObjectPointCloud geometries in the 3D scene, this PR adds a default_camera field to the GetPropertiesResponse for the vision service in order to assign the correct reference frame to the GeometriesInFrame struct created by the vision service builder in the RDK (as recommended by @bhaney https://viaminc.slack.com/archives/C071CRAU7EJ/p1779824550593229?thread_ts=1779816238.540499&cid=C071CRAU7EJ ). Otherwise, the vision service server response for GetObjectPointClouds only assigns the CameraName from the request rather than falling back to the default camera like the implemented service methods do internally: https://github.com/viamrobotics/rdk/blob/01c3f4d4ab1524bc2f0f8a83a2c5aba2bf6b668d/services/vision/server.go#L168-L216

@micheal-parks

micheal-parks commented Jun 8, 2026

Copy link
Copy Markdown
Member

The problem that I see with this solution - if the visualizer were to use it to establish the pointcloud object's reference frame - is that it opens up the door for conflicting information. What if for a specific module the default camera is "camera_1" but in the GeometriesInFrame object the referenceFrame is some other resource, or an empty string, which we canonically accept as "world"? Because of this problem - it will likely cause very confusing breakage to someone in the future - the visualizer isn't going to be able to adopt it. I suggest that we instead invest in making referenceFrame accurate - it should be camera_1 if the referenceFrame is camera_1 cc @DTCurrie.

@HipsterBrown

Copy link
Copy Markdown
Member Author

@micheal-parks I should have been more clear in the PR description. The next step for this feature is to incorporate the default_camera info into the RDK so the vision service assigns it as the reference frame for the GeometriesInFrame if no camera name is passed in the request to GetObjectPointClouds: https://github.com/viamrobotics/rdk/blob/main/services/vision/server.go#L168-L189
The internal vision service implementation already falls back to the default camera but the server doesn't have that information to do the same.

@micheal-parks

Copy link
Copy Markdown
Member

Ah gotcha, that makes sense, thanks for clearing it up!

@github-actions github-actions Bot added the safe to test committer is a member of this org label Jun 9, 2026
@viamrobotics-overwatch

Copy link
Copy Markdown

Hey @HipsterBrown — this PR has been approved and CI has been green for 3+ business days. Ready to merge?

Auto-comment from overwatch. Will not re-nudge for 7 days.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

safe to test committer is a member of this org

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants