with Google Cloud Vision API, how do you detect if the face is not covered and clearly visible?
example here… it seems to detect a face but my biz rules are to detect “faces that are clearly visible and uncovered and unmasked”…
is there anything in the API response that can help detect this?
never used it but FWIW https://cloud.google.com/vision/docs/detecting-faces#detect_faces_in_a_local_image,
shows a “landmarks” section fo the response body that includes eye coordinates etc
{
"type": "LEFT_EYE",
"position": {
"x": 1368.748,
"y": 739.0957,
"z": 0.0024604797
}
},
{
"type": "RIGHT_EYE",
"position": {
"x": 1660.6105,
"y": 751.5844,
"z": -117.06496
}
},
{
"type": "LEFT_OF_LEFT_EYEBROW",
"position": {
"x": 1284.3208,
"y": 666.61487,
"z": 63.41506
}
},
{
"type": "RIGHT_OF_LEFT_EYEBROW",
"position": {
"x": 1418.9249,
"y": 671.49414,
"z": -83.82396
}
},
{
"type": "LEFT_OF_RIGHT_EYEBROW",
"position": {
"x": 1556.9579,
"y": 672.2199,
"z": -139.39935
}
},
{
"type": "RIGHT_OF_RIGHT_EYEBROW",
"position": {
"x": 1771.4799,
"y": 682.65845,
"z": -131.66716
}
},
{
"type": "MIDPOINT_BETWEEN_EYES",
"position": {
"x": 1479.6194,
"y": 741.87305,
"z": -114.84635
}
},
{
"type": "NOSE_TIP",
"position": {
"x": 1443.3151,
"y": 917.5109,
"z": -194.49301
}
},
{
"type": "UPPER_LIP",
"position": {
"x": 1466.7897,
"y": 1025.3483,
"z": -130.1202
}
},
{
"type": "LOWER_LIP",
"position": {
"x": 1467.2588,
"y": 1147.0403,
"z": -109.24505
}
},
{
"type": "MOUTH_LEFT",
"position": {
"x": 1376.8649,
"y": 1066.0856,
"z": -6.8136826
}
},
{
"type": "MOUTH_RIGHT",
"position": {
"x": 1652,
"y": 1079.3108,
"z": -106.93649
}
},
{
"type": "MOUTH_CENTER",
"position": {
"x": 1485.5554,
"y": 1087.2388,
"z": -110.68126
}
},
{
"type": "NOSE_BOTTOM_RIGHT",
"position": {
"x": 1571.9475,
"y": 944.9213,
"z": -124.11806
}
},
{
"type": "NOSE_BOTTOM_LEFT",
"position": {
"x": 1395.2339,
"y": 938.12787,
"z": -58.072197
}
},
{
"type": "NOSE_BOTTOM_CENTER",
"position": {
"x": 1468.4205,
"y": 968.8732,
"z": -132.09975
}
},
{
"type": "LEFT_EYE_TOP_BOUNDARY",
"position": {
"x": 1357.8658,
"y": 711.2427,
"z": -14.618992
}
},
{
"type": "LEFT_EYE_RIGHT_CORNER",
"position": {
"x": 1423.6936,
"y": 750.4164,
"z": -23.540215
}
},
{
"type": "LEFT_EYE_BOTTOM_BOUNDARY",
"position": {
"x": 1360.5627,
"y": 762.87415,
"z": -1.2607727
}
},
{
"type": "LEFT_EYE_LEFT_CORNER",
"position": {
"x": 1313.72,
"y": 739.443,
"z": 50.216393
}
},
{
"type": "RIGHT_EYE_TOP_BOUNDARY",
"position": {
"x": 1661.6622,
"y": 718.6839,
"z": -134.17404
}
},
{
"type": "RIGHT_EYE_RIGHT_CORNER",
"position": {
"x": 1730.0901,
"y": 763.57104,
"z": -116.365845
}
},
{
"type": "RIGHT_EYE_BOTTOM_BOUNDARY",
"position": {
"x": 1660.8823,
"y": 777.3474,
"z": -120.8635
}
},
{
"type": "RIGHT_EYE_LEFT_CORNER",
"position": {
"x": 1590.8903,
"y": 753.5044,
"z": -91.84842
}
},
{
"type": "LEFT_EYEBROW_UPPER_MIDPOINT",
"position": {
"x": 1345.7522,
"y": 640.18243,
"z": -27.887913
}
},
{
"type": "RIGHT_EYEBROW_UPPER_MIDPOINT",
"position": {
"x": 1660.5848,
"y": 648.36145,
"z": -153.73691
}
},
{
"type": "LEFT_EAR_TRAGION",
"position": {
"x": 1274.1006,
"y": 826.2645,
"z": 422.6642
}
},
{
"type": "RIGHT_EAR_TRAGION",
"position": {
"x": 2014.8041,
"y": 908.56537,
"z": 149.61232
}
},
{
"type": "FOREHEAD_GLABELLA",
"position": {
"x": 1476.2395,
"y": 669.9625,
"z": -120.59111
}
},
{
"type": "CHIN_GNATHION",
"position": {
"x": 1477.3256,
"y": 1269.3269,
"z": -67.748795
}
},
{
"type": "CHIN_LEFT_GONION",
"position": {
"x": 1336.8848,
"y": 1096.2242,
"z": 286.73004
}
},
{
"type": "CHIN_RIGHT_GONION",
"position": {
"x": 1863.2197,
"y": 1128.6213,
"z": 68.90431
}
},
{
"type": "LEFT_CHEEK_CENTER",
"position": {
"x": 1317.8549,
"y": 940.8025,
"z": 50.863163
}
},
{
"type": "RIGHT_CHEEK_CENTER",
"position": {
"x": 1733.4912,
"y": 964.073,
"z": -112.43947
}
}
],
"rollAngle": 1.5912293,
"panAngle": -22.01964,
"tiltAngle": -1.4997566,
"detectionConfidence": 0.9310801,
"landmarkingConfidence": 0.5775582,
"joyLikelihood": "VERY_LIKELY",
"sorrowLikelihood": "VERY_UNLIKELY",
"angerLikelihood": "VERY_UNLIKELY",
"surpriseLikelihood": "VERY_UNLIKELY",
"underExposedLikelihood": "VERY_UNLIKELY",
"blurredLikelihood": "VERY_UNLIKELY",
"headwearLikelihood": "POSSIBLE"
},```hmm interesting
so like if there is no eyes or mouth it could tell me that the face is obscured by something?
it kinda seems to still have the eyes in there 
so struggling to see a difference in response between a clear face vs hidden face 
oh yeah it’s probably picking up on knuckles being eyebrows/eyes and back of hands being cheek/bone …
are you able to do anything like validate presence of:
- left eye
- right eye
- nose
- mouth
yea thats what im trying to see a pattern… like understand when the response tells me there are no eyes, nose, mouth…
but i dont get it haha
it’s just not good? tells me it sees a mouth here too
does that also see eyes? wondering if the eyes are beeing seen as a mouth
i think all pictures have these 34 landmarks…
sorry didnt catch this
does that also see eyes? wondering if the eyes are beeing seen as a mouth
that’s where I’m wondering if you’re able to make it confirm all 4 exist (both eyes, nose and mouth)
i’d hope position is null/nil/undefined or something on some of those
yea all these test pictures have all 4
but you’re also not trying to have to post-process each image result it gives you to validate the JSON contains the desired values and repeat the API call if it doesn’t…
yea i was hoping there would be no mouth attribute if it cant see one at all
or yea at least with NULLs
call the API twice? what scenario would that be?
Oh I just meant that you don’t want to have your app/code call the api, see the response, check for the values, and try again if they’re not present… like if you weren’t able to say “Hey, I require ____” and had to confirm the conditions
like if you had an app that said “fetch me an image” and then your app said “hmmm that payload doesn’t have a mouth, try again”
my main goal is to help the user create a clean profile, where the first picture has to be a clearly visible face
so i think there will be some kinda error says “sorry this does not seem like a face, try again”
