Hand Detector¶

Finds hands in an image and returns 21 landmarks for each, along with whether it is a left or right hand.

Hand landmarks drawn on a moving hand

Basic use¶

import cv2 as cv
from detectors_world import create_detector

cap = cv.VideoCapture(0)

with create_detector("hand", max_hands=2) as hand:
    while True:
        ok, frame = cap.read()
        if not ok:
            break

        hands = hand.detect(frame)
        for found in hands:
            print(found.label, found[8])

        cv.imshow("Hand Detection", frame)
        if cv.waitKey(1) & 0xFF == ord("q"):
            break

cap.release()
cv.destroyAllWindows()

Options¶

Option	Default	Meaning
`max_hands`	`2`	Largest number of hands to look for
`detection_confidence`	`0.5`	Minimum confidence to report a new hand
`tracking_confidence`	`0.5`	Minimum confidence to keep following a hand between frames
`static_image`	`False`	Set to `True` for unrelated photos
`model_path`	`None`	Use your own copy of `hand_landmarker.task`

Landmark map¶

Each hand has 21 landmarks, numbered from the wrist out to each fingertip. The numbers are the same for left and right hands.

Map of the 21 hand landmarks

Id	Name	Description
0	`WRIST`	Wrist
1	`THUMB_CMC`	Thumb base
2	`THUMB_MCP`	Thumb knuckle
3	`THUMB_IP`	Thumb middle joint
4	`THUMB_TIP`	Thumb tip
5	`INDEX_FINGER_MCP`	Index knuckle
6	`INDEX_FINGER_PIP`	Index lower middle joint
7	`INDEX_FINGER_DIP`	Index upper middle joint
8	`INDEX_FINGER_TIP`	Index tip
9	`MIDDLE_FINGER_MCP`	Middle knuckle
10	`MIDDLE_FINGER_PIP`	Middle lower middle joint
11	`MIDDLE_FINGER_DIP`	Middle upper middle joint
12	`MIDDLE_FINGER_TIP`	Middle tip
13	`RING_FINGER_MCP`	Ring knuckle
14	`RING_FINGER_PIP`	Ring lower middle joint
15	`RING_FINGER_DIP`	Ring upper middle joint
16	`RING_FINGER_TIP`	Ring tip
17	`PINKY_MCP`	Pinky knuckle
18	`PINKY_PIP`	Pinky lower middle joint
19	`PINKY_DIP`	Pinky upper middle joint
20	`PINKY_TIP`	Pinky tip

Fingertips are 4, 8, 12, 16 and 20.

Results¶

hand.detect(frame) returns a list of Landmarks, one per hand:

hands = hand.detect(frame)

for found in hands:
    found.label        # "Left" or "Right"
    found.score        # confidence of the label
    found.points       # 21 (x, y) pixel positions
    found.normalized   # 21 (x, y, z) values
    found[4]           # the thumb tip, as (x, y)

Left and right¶

Labels name the person's own hands in a normal camera image. A mirrored frame, such as one from cv.flip(frame, 1), swaps them, so a person's left hand is reported as "Right".

If you flip frames for a mirror-like view, either run detect() before flipping, or swap the labels yourself.

Finding landmarks with locate()¶

hand.locate() returns one hand as [[id, x, y], ...], and draws a dot on each landmark:

hand.detect(frame)
landmarks = hand.locate(frame, hand=0)

if landmarks:
    id_number, x, y = landmarks[8]

Argument	Default	Meaning
`hand`	`0`	Which hand to return, counting from 0
`draw`	`True`	Draw a dot on each landmark
`color`	`(0, 255, 0)`	Dot color as `(blue, green, red)`
`radius`	`6`	Dot radius in pixels

It returns [] if that hand isn't visible.

Recipes¶

Is a finger raised?¶

With the hand upright, a fingertip above its middle joint has a smaller y:

index_raised = found[8][1] < found[6][1]

The Finger Counter extends this to the whole hand.

Distance between two landmarks¶

import math

pinch = math.dist(found[4], found[8])

The Pinch Distance example turns this into a control.

Follow one hand only¶

hand = create_detector("hand", max_hands=1)

Looking for fewer hands is also a little faster.