1 year ago

#330423

test-img

jkjk5979

Good face features for clustering can be used with pytorch or opencv.dnn module

I'm searching about good feature extractor (library or pretrained architecture for pytorch), except dlib, for implementing face clustering with DBSCAN.

I tried openface pretrained features via opencv's dnn module by next code :

import cv2 
import numpy as np 
from pathlib import Path
from sklearn.cluster import DBSCAN
import multiprocessing
 
dataset_path = 'datasetpath'
embedder = cv2.dnn.readNetFromTorch('openface.nn4.small2.v1.t7')

face_list = []
vec_list = []

for i in Path(dataset_path).glob("*.jpg"):
    frame = cv2.imread(str(i))
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    faces = face_cascade.detectMultiScale(gray, 1.3, 5, minSize=None, maxSize=None)
    for coords in faces:
        face = frame[coords[1]:coords[1]+coords[3], coords[0]:coords[0]+coords[2]]
        face = cv2.resize(face, (96, 96))
        faceBlob = cv2.dnn.blobFromImage(face, 1.0 / 255,
            (96, 96),
            (0, 0, 0), 
            swapRB=True,
            crop=False)
        embedder.setInput(faceBlob)
        vec = embedder.forward().flatten()
        face_list.append(face)
        vec_list.append(vec)


clt = DBSCAN(eps=0.5, metric="euclidean", min_samples=5, n_jobs=multiprocessing.cpu_count())
clt.fit_predict(vec_list)
clt.labels_

and results are :

array([-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
       -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
       -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
       -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
       -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
       -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
       -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
       -1, -1])

It is clear that DBSCAN failed to find no dense cluster, all faces clustered as just outlier. My dataset is same with the This post. I used same parameters on DBSCAN with post, so i doubt the openface feature is disperse or respond to a subtle difference on face, do not agregated enough to initial seed for DBSCAN. I changed eps and min_samples but the results are same. Is there anything that the good face feature extractor or pretrained models, that especially can be used with pytorch or cv2.dnn module?

opencv

pytorch

feature-extraction

0 Answers

Your Answer

Accepted video resources