Skip to content

Display classification annotations in the annotations grid #1704

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
LeonardoRosaa merged 12 commits into from
Jul 21, 2026
Merged

Display classification annotations in the annotations grid #1704

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
8ae03ec
add classification annotation grid item component
LeonardoRosaa Jul 17, 2026
1059a39
- factor out thumbnail logic
LeonardoRosaa Jul 17, 2026
128fb16
display classification on the annotations grid view
LeonardoRosaa Jul 20, 2026
3b15735
Merge branch 'main' into leonardo-lig-10112-classification-grid-view-…
LeonardoRosaa Jul 21, 2026
68c2b87
fix files path
LeonardoRosaa Jul 21, 2026
bf780aa
update changelog
LeonardoRosaa Jul 21, 2026
b9ac18e
use typed parameter object
LeonardoRosaa Jul 21, 2026
4beb598
add buildClassificationDragData tests
LeonardoRosaa Jul 21, 2026
1c881e0
remove classification pills mock
LeonardoRosaa Jul 21, 2026
d0d3c4b
remove addAnnotationLabelChangeToUndoStackFn mock
LeonardoRosaa Jul 21, 2026
74e1295
await fireEvent
LeonardoRosaa Jul 21, 2026
9c2102f
remove comment
LeonardoRosaa Jul 21, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Add a selector for the numeric metadata histogram bin count.
- Introduce button to see expanded distribution for numeric metadata values.
- Add metadata filter chips to the left sidebar
- Display classification annotations in the annotations grid.


### Changed
Expand Down
0 ...otationClassificationGridItem.mock.svelte → ...otationClassificationGridItem.mock.svelte
View file
Open in desktop
File renamed without changes.
16 changes: 9 additions & 7 deletions .../AnnotationsGrid/AnnotationClassificationGridItem/AnnotationClassificationGridItem.svelte
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,6 @@
containerWidth: number;
/** Height of the grid container tile in pixels. */
containerHeight: number;
/** Whether text labels are visible globally. */
showLabel: boolean;
/** Whether this tile is currently selected. */
selected?: boolean;
/** Collection version cache-buster (same as AnnotationImageGridItem). */
Expand All @@ -26,7 +24,6 @@
annotation,
containerWidth,
containerHeight,
showLabel,
selected = false,
cachedCollectionVersion = '',
onCropWindowChange
Expand Down Expand Up @@ -73,8 +70,13 @@
aria-selected={selected}
style="width: {containerWidth}px; height: {containerHeight}px; background-image: url('{thumbnailUrl}'); background-size: cover; background-position: center;"
>
{#if showLabel}
<!-- One tile shows exactly one label — [annotation.annotation] wraps a single classification. -->
<SampleClassificationPills sample={{ annotations: [annotation.annotation] }} />
{/if}
<!-- One tile shows exactly one label — [annotation.annotation] wraps a single classification. -->
<!-- selectedCollectionIds=[] bypasses the images-grid source filter; the tile already
represents one annotation so no further filtering is needed. -->
<!-- The badge is always visible: for classification there is no bounding-box equivalent,
Comment thread
LeonardoRosaa marked this conversation as resolved.
Outdated
so the badge is the primary visual indicator regardless of the showLabel setting. -->
<SampleClassificationPills
sample={{ annotations: [annotation.annotation] }}
selectedCollectionIds={[]}
Comment thread
LeonardoRosaa marked this conversation as resolved.
/>
</div>
11 changes: 7 additions & 4 deletions ...AnnotationsGrid/AnnotationClassificationGridItem/AnnotationClassificationGridItem.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,6 @@ function buildAnnotation(
const defaultProps = {
containerWidth: 200,
containerHeight: 150,
showLabel: true,
cachedCollectionVersion: 'v1'
};

Expand Down Expand Up @@ -94,10 +93,14 @@ describe('AnnotationClassificationGridItem', () => {
expect(container.firstElementChild).toHaveAttribute('aria-selected', 'true');
});

it('hides SampleClassificationPills when showLabel is false', () => {
renderItem(buildAnnotation(), { showLabel: false });
it('always renders SampleClassificationPills regardless of label visibility settings', async () => {
renderItem(buildAnnotation());

await tick();

expect(screen.queryByTestId('mock-classification-pills')).not.toBeInTheDocument();
// The badge is the primary visual indicator for classification — it shows unconditionally,
// unlike OD where the bounding box can be shown without the text label.
expect(screen.getByTestId('mock-classification-pills')).toBeInTheDocument();
});

it('passes only the single annotation to SampleClassificationPills', async () => {
Expand Down
36 changes: 31 additions & 5 deletions lightly_studio_view/src/lib/components/AnnotationsGrid/AnnotationsGrid.helpers.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,12 @@ import type { AnnotationView } from '$lib/api/lightly_studio_local';
import type { GridItemDragData } from '$lib/components/GridItem';
import type { CropWindow } from './AnnotationItem/renderCropObjectUrl';

interface AnnotationDragDataParams {
annotation: AnnotationView;
cropWindow: CropWindow | undefined;
cropUrl: string | undefined;
}

/**
* Build the drag-to-search payload for an annotation tile.
*
Expand All @@ -11,11 +17,11 @@ import type { CropWindow } from './AnnotationItem/renderCropObjectUrl';
* crop itself. The actual search uses the stored embedding looked up on drop via
* `annotationSampleId`/`annotationCollectionId`.
*/
export function buildAnnotationDragData(
annotation: AnnotationView,
cropWindow: CropWindow | undefined,
cropUrl: string | undefined
): GridItemDragData | undefined {
export function buildAnnotationDragData({
annotation,
cropWindow,
cropUrl
}: AnnotationDragDataParams): GridItemDragData | undefined {
if (!cropWindow) return undefined;
return {
url: cropUrl ?? cropWindow.sourceUrl,
Expand All @@ -24,3 +30,23 @@ export function buildAnnotationDragData(
annotationCollectionId: annotation.annotation_collection_id
};
}

/**
* Build drag-to-search payload for a classification annotation tile.
*
* Intentionally omits `annotationSampleId` — classification annotations have no
* per-annotation crop embedding, so `readAnnotationEmbedding` would fail. Omitting it
* makes the drop handler fall through to the image-upload search path (`search.setImage`),
* which sends the thumbnail as the query image.
*/
export function buildClassificationDragData({
annotation,
cropWindow,
cropUrl
}: AnnotationDragDataParams): GridItemDragData | undefined {
if (!cropWindow) return undefined;
return {
url: cropUrl ?? cropWindow.sourceUrl,
fileName: `${annotation.annotation_label.annotation_label_name}-crop.png`
};
Comment thread
LeonardoRosaa marked this conversation as resolved.
}
102 changes: 54 additions & 48 deletions lightly_studio_view/src/lib/components/AnnotationsGrid/AnnotationsGrid.svelte
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,11 @@
import { GridContainer } from '$lib/components/GridContainer';
import { Grid } from '$lib/components/Grid';
import { GridItem } from '$lib/components/GridItem';
import { buildAnnotationDragData } from './AnnotationsGrid.helpers';
import {
buildAnnotationDragData,
buildClassificationDragData
} from './AnnotationsGrid.helpers';
import AnnotationClassificationGridItem from './AnnotationClassificationGridItem/AnnotationClassificationGridItem.svelte';
import { renderCropObjectUrl, type CropWindow } from './AnnotationItem/renderCropObjectUrl';

type AnnotationsProps = {
Expand Down Expand Up @@ -181,15 +185,8 @@
}
}

// Skip the classification annotations
// because we don't have support for the annotation views
const annotations: AnnotationWithPayloadView[] = $derived(
infiniteAnnotations.data?.pages.flatMap((page) =>
page.data.filter(
(annotation) =>
annotation.annotation.annotation_type != AnnotationType.CLASSIFICATION
)
) || []
infiniteAnnotations.data?.pages.flatMap((page) => page.data) ?? []
);

function handleLoadMore() {
Expand Down Expand Up @@ -304,47 +301,42 @@
>
{#snippet gridItem({ index, style, width, height })}
{#if annotations[index]}
{#key annotations[index].annotation.sample_id}
{@const ann = annotations[index]}
{@const annotationId = ann.annotation.sample_id}
{@const isClassification =
ann.annotation.annotation_type === AnnotationType.CLASSIFICATION}
{#key annotationId}
<GridItem
{width}
{height}
{style}
dataTestId="annotation-grid-item"
tag={false}
ariaLabel={`Edit annotation: ${annotations[index].annotation.sample_id}`}
dragData={buildAnnotationDragData(
annotations[index].annotation,
cropWindowByAnnotationId[
annotations[index].annotation.sample_id
],
cropUrlByAnnotationId[
annotations[index].annotation.sample_id
]
)}
onDragStart={() =>
handleAnnotationDragStart(
annotations[index].annotation.sample_id
)}
ariaLabel={`Edit annotation: ${annotationId}`}
dragData={isClassification
? buildClassificationDragData({
annotation: ann.annotation,
cropWindow: cropWindowByAnnotationId[annotationId],
cropUrl: cropUrlByAnnotationId[annotationId]
})
: buildAnnotationDragData({
annotation: ann.annotation,
cropWindow: cropWindowByAnnotationId[annotationId],
cropUrl: cropUrlByAnnotationId[annotationId]
})}
onDragStart={() => handleAnnotationDragStart(annotationId)}
onSelect={(event) =>
handleGridItemSelect(
event,
annotations[index].annotation.sample_id,
index
)}
ondblclick={() =>
handleOnDoubleClick(
annotations[index].annotation.sample_id
)}
handleGridItemSelect(event, annotationId, index)}
ondblclick={() => handleOnDoubleClick(annotationId)}
>
<div
class="annotation-grid-item relative h-full w-full"
data-annotation-id={annotations[index].annotation.sample_id}
data-annotation-id={annotationId}
data-annotation-index={index}
data-sample-id={annotations[index].annotation
.parent_sample_id}
data-sample-id={ann.annotation.parent_sample_id}
data-index={index}
>
{#if hasMinimumRole(user?.role, 'labeler') && $pickedAnnotationIds[collection_id]?.has(annotations[index].annotation.sample_id)}
{#if hasMinimumRole(user?.role, 'labeler') && $pickedAnnotationIds[collection_id]?.has(annotationId)}
<div
class="pointer-events-none absolute right-2 top-1.5 z-10"
inert
Expand All @@ -356,17 +348,31 @@
</div>
{/if}

<AnnotationsGridItem
annotation={annotations[index]}
{width}
{height}
cachedCollectionVersion={collectionVersion}
showLabel={showLabels}
selected={$pickedAnnotationIds[collection_id]?.has(
annotations[index].annotation.sample_id
)}
onCropWindowChange={handleCropWindowChange}
/>
{#if isClassification}
<!-- One classification annotation = one tile (1:1 mapping, same as OD/seg). -->
<AnnotationClassificationGridItem
Comment thread
LeonardoRosaa marked this conversation as resolved.
annotation={ann}
containerWidth={width}
containerHeight={height}
selected={$pickedAnnotationIds[collection_id]?.has(
annotationId
)}
cachedCollectionVersion={collectionVersion}
onCropWindowChange={handleCropWindowChange}
/>
{:else}
<AnnotationsGridItem
annotation={ann}
{width}
{height}
cachedCollectionVersion={collectionVersion}
showLabel={showLabels}
selected={$pickedAnnotationIds[collection_id]?.has(
annotationId
)}
onCropWindowChange={handleCropWindowChange}
/>
{/if}
</div>
</GridItem>
{/key}
Expand Down
Loading
Loading