Hello, Nate! Sorry for late reply. I already saw yours 2 weeks ago, but I wanted to reply after seeing the result by myself.
Your advices helped me a lot, first letting me know this :
Nate लिखाTo get a specific bone there is Skeleton findBone, but this compares the name of each bone, so you probably don't want to call it every frame. Instead you can use it to find the bones when the app starts and store the references somewhere for use later, when you need to set their rotation.
I also checked API reference page you linked, and later checked spine-webgl.js code to see methods that were not introduced well in API reference(like, camera.viewportWidth/Height/position, resize(fit/expand))
You are genius, Nate! Thanks a lot! :grinteeth:
And... Here's a video that shows what I have done so far..if you can notice.. there are 'some' issues, and I want to get advice from you(everyone in this forum) again! Please watch it carefully.
it seems to detects arm movement and leg movement well, but almost all bones are too much shaky.
This javascript codes first detects my pose, returns coords of 12 body part keypoints. Then a function i wrote will transform each spine bones. its parameters are :
-a Spine character's bone
-two body part keypoints that correspond to character bone
-Spine skeleton object to update
Here's the code for the function :
//Webcam canvan size : 600 x 500, spine runtime canvas : 1200 x 1000
// Get two keypoints, a character bone and minConfidence
// Animate bone if minimum confidence condition is met
function animateDetectedBone(chaBone, initKeyPt, termKeyPt, minConfidence, skeleton) {
if (initKeyPt.score > minConfidence && termKeyPt.score > minConfidence) {
var chaScale;
chaBone.rotation = chaBone.worldToLocalRotation(getAngle(initKeyPt.position, termKeyPt.position));
if (["head", "torso1", "torso2"].includes(chaBone.data.name)) {
if(initKeyPt.position.x < termKeyPt.position.x){
chaBone.rotation += 90;
}
else {
chaBone.rotation -= 90;
}
}
if (["torso1"].includes(chaBone.data.name)) {
var distanceKeys = Math.sqrt((initKeyPt.position.x - termKeyPt.position.x) ** 2 + (initKeyPt.position.y - termKeyPt.position.y) ** 2);
chaScale = (distanceKeys * 2 / 100);
debug.camera.viewportWidth = 1200 / chaScale;
debug.camera.viewportHeight = 1000 / chaScale;
debug.camera.position.set(debug.camera.viewportWidth / 2, debug.camera.viewportHeight / 2,0);
}
if ("torso1" == chaBone.data.name) {
var posCha = {x : (initKeyPt.position.x + termKeyPt.position.x), y : 1000 - (initKeyPt.position.y + termKeyPt.position.y)};
chaBone.x = (posCha.x * (1 / chaScale));
chaBone.y = (posCha.y * (1 / chaScale));
}
skeleton.updateWorldTransform();
}
}
Note that webcam canvas has 600 x 500px size, and spine runtime canvas has 1200 x 1000px size.
Another issue i found that is, I'm not sure if I'm precisely translating my character upon my position in webcam. Second and Third if condition in code above is the part that deal with this, but i'm doubful if this is done correctly.. 🙁
Damn. I still have no idea what causes this character moving around so randomly :think:
My next step is to fix those shaky viewport camera movement, fixing the ratio differences between two canvases, improving pre-trained PoseNet model, and adding more features like detecting turn-around, facial bones.. whatever that sounds awesome.
But I have to solve those randomly moving bones first. Do you have any idea how to solve this...?
