Object Detection — Level 6: Grounding DINO

Approach

A stronger, transformer-based open-vocabulary detector — better at nuanced phrases, meaningfully heavier to run.

Cross-attention between text and image regions, via Hugging Face transformers (IDEA-Research/grounding-dino-tiny). Several seconds to over a minute per image on CPU, versus sub-second for YOLO-World in Level 5 — a real compute tradeoff, not just a quality upgrade.

Dependency conflict

Latest transformers (5.x) requires torch>=2.5, which conflicts directly with this Intel Mac's torch==2.2.2 ceiling (see Environment on the Summary page). Pinned transformers==4.46.3 instead — it predates that requirement and still supports Grounding DINO. API differences between versions also required a single flat text string ("helmet. person. safety vest.") rather than the newer nested-list input format.

processor = AutoProcessor.from_pretrained("IDEA-Research/grounding-dino-tiny")
model = AutoModelForZeroShotObjectDetection.from_pretrained("IDEA-Research/grounding-dino-tiny")
text_query = "helmet. person. safety vest. electrical cabinet. hand pallet truck. cardboard box."
inputs = processor(images=image, text=text_query, return_tensors="pt")
results = processor.post_process_grounded_object_detection(
    outputs, input_ids=inputs.input_ids, box_threshold=0.22, text_threshold=0.2, target_sizes=[image.size[::-1]]
)[0]

Custom additions

Cross-class duplicate suppression (a custom IoU function) — different independent text queries could each match the same physical object, producing stacked duplicate boxes with different labels; sorted by confidence and dropped the lower-confidence overlaps.
Per-class color coding plus label-collision avoidance — labels step down vertically instead of overprinting when boxes sit close together.

Standout result

Original, unannotated synthetic 3D rendered forklift warehouse image — **Before** — the original, unannotated synthetic 3D render.

Grounding DINO correctly finding cardboard boxes, a wooden crate, a hand pallet truck, and an electrical cabinet on the same synthetic image — **After** — cardboard box, wooden crate, hand pallet truck, and electrical cabinet all correctly found.

Terminal output

warehouse3.jpg -> Total objects detected: 8
  hand pallet truck: 1
  cardboard box wooden crate: 2
  electrical cabinet: 4
  cardboard box: 1
Saved annotated image as output3.jpg

For reference: Level 5's YOLO-World returned zero detections on this exact same image — same scene, two different open-vocabulary detectors, very different outcomes.

The model that identifies objects best also demands the most compute — several seconds to a minute per image here, versus sub-second for Level 5. That tradeoff is a direct, honest segue into local AI hardware positioning (Levels 9 and 10).