यह साइट एक आधुनिक ब्राउज़र में सबसे अच्छी तरह से देखी जा सकती है जिसमें जावास्क्रिप्ट सक्षम है।

[canvas/webGL] Mouseover Bounding Box?

kyttias

I'd like to create "touch" events so that when a user mouses over a bounding box attachment - one named "good" and the other "bad" - an animation will play. I'm familiar with mouse over events (in general) and changing animations (with Spine) in basic Javascript. But I'm not sure how to load and access the vertices I need for these bounding boxes? I've looked over http://esotericsoftware.com/spine-api-reference#BoundingBoxAttachment, but I'm super not sure how to reference these vertices in practice.

Any helpful thoughts? Again, using WebGL and you can safely assume all I have at a base is roughly what comes in the example runtime folders, so a simplified version of this - https://github.com/EsotericSoftware/spine-runtimes/blob/master/spine-ts/webgl/example/index.html - just without so many different skeletons loaded at once. Assume default skin, of course. What I need is raw Javascript to access the array of vertices.

And hopefully not something that looks like this:

var getBoundingBoxVertices = function() {
   var skeleton = skeletons[activeSkeleton].skeleton;
   var skinAttachments = skeleton.skin.attachments;
   for (var i = 0; i < skinAttachments.length; i++) {
      var slotAttachments = skinAttachments[i];
      if (!slotAttachments) continue;
      var slot = skeleton.slots[i];
      if (slot.data.name == "good" || slot.data.name == "bad" ){ 
         for (var j in slotAttachments) {
            // ????????????
         }

  }
   }
}    

getBoundingBoxVertices();

Because I'm not having a lot of fun traversing down a JSON tree while scratching my head at the available documentation.
(And I'm just plain sleepy right now... :sleepy: )

Mario

Here's an (untested) snippet that should do what you want. It assumes that you added bounding box attachments to your skeleton in the editor for the areas you want to hit test.

var skeleton = ... // whereever your skeleton came from
var animState = ... // whereever your animation state came from
var bounds = new spine.SkeletonBounds(); // hold on to this, no need to recreate it all the time

// update skeleton first, e.g. apply animation, update world transform, etc.
animState.update(deltaTime);
animState.apply(skeleton);
skeleton.updateWorldTransform();

// then update the skeleton bounds that will calculate the bounding box attachment
// vertex positions and axis aligned bounding boxes derrived from the attachments etc.
bounds.update(skeleton, true)

// Finally, do hit detection (in the coordinate space of the skeleton, that is, 
// you may have to substract the skeleton position from your mouse coordinates)
//  using the methods on SkeletonBounds
var hitBoundingBoxAttachment = bounds.containsPoint(adjustedMouseX, adjustedMouseY);
if (hitBoundingBoxAttachment != null) {
   // do whatever you need to do when an attachment is hit
}

kyttias

That last bit, about subtracting the skeleton position from the mouse coordinates? I think I might need that? Either that or something entirely bizarre is going on otherwise?
Loading Image

Here's the little guy in Spine. The green area is "bb_good" and is what the above thinks my mouse is touching?
Loading Image

/* This gets the mouse coordinates:  */
function getMousePos(canvas, e) {
    var rect = canvas.getBoundingClientRect();
    return {
      x: e.clientX - rect.left,
      y: e.clientY - rect.top
    };
  }
  canvas.addEventListener('mousemove', function(e) {
    var mousePos = getMousePos(canvas, e);
    console.log('Mouse is at ' + mousePos.x + ',' + mousePos.y);
    checkMouseOver(mousePos.x,mousePos.y);
  }, false);

/* Basically what was provided before (with some renamed variables for consistency and setting the delta): */ 
function checkMouseOver(mouseX, mouseY){
   var skeleton = skeletons[activeSkeleton].skeleton;
   var state = skeletons[activeSkeleton].state;
   var bounds = new spine.SkeletonBounds(); 
   var now = Date.now() / 1000;
   var delta = now - lastFrameTime;
   lastFrameTime = now;
   state.update(delta);
   state.apply(skeleton);
   skeleton.updateWorldTransform();

   bounds.update(skeleton, true);

   var hitBoundingBoxAttachment = bounds.containsPoint(mouseX, mouseY);
   console.log(hitBoundingBoxAttachment); // For testing purposes.
   if (hitBoundingBoxAttachment != null){
      console.log("Touched at "+mouseX+", "+mouseY+"!");
   }
}

Thanks so much for helping. :yes: I think I'm close, but there's definitely an offset going on somehow.

Nate

The 0,0 world position is just under his feet, where the black lines cross in Spine. When the mouse is over this point, the coordinates you should pass to containsPoint is 0,0. Try bounds.containsPoint(mouseX - skeleton.x, mouseY - skeleton.y).

kyttias

Oh, right. That would make a lot of sense! Unfortunately, changing that line didn't make a change in the behavior I'm seeing. (And me physically moving the skeleton in Spine so that 0,0 would match the canvas 0,0 definitely isn't the solution - afterall, this little guy could be positioned anywhere on a canvas of varying size?) What am I missing here?

The 0,0 for canvas will always be the top left and I can't change that. (So... the canvas isn't what's considered my "world"??) I'm going need to know the skeleton's 0,0 position relative to the canvas' 0,0 position to line this up. Right now, the skeleton loads at the default position - which is the center, no matter how big or small I make the canvas. If skeleton.x and skeleton.y are supposed to compensate for this, that's not what's happening - both of these values are 0? D;

The truth is I just don't know where the skeleton's 0,0 is on my canvas. It's wherever it loaded by default.

Also:

When the mouse is over this point, the coordinates you should pass to containsPoint is 0,0.

^ This doesn't make sense. When my mouse is over that point the coordinates will be whatever my mouse coordinates are relative to the canvas. I don't pass anything to that function manually, my mouse does. And the mouse will only read 0,0 at the canvas' 0,0 which I cannot change from the top left. You see, if I make my canvas twice as wide the coordinates will be different when I hover over this point. However, the area that the hittest activates on (from the example image) stays relative to the left wall of the canvas:
Loading Image

And when I make the canvas twice as tall instead:
Loading Image

What I need to do is compensate for this, and I'm not sure how without knowing the skeleton's position relative to the canvas 0,0. I'm honestly a little lost because of the way it keeps moving around what it thinks is relative.

Nate

kyttias लिखा
When my mouse is over that point the coordinates will be whatever my mouse coordinates are relative to the canvas.

SkeletonBounds does not accept a point in mouse coordinates. It accepts a point in the skeleton's world coordinates. You need to transform your mouse coordinates to skeleton world coordinates.

If your mouse coordinates use Y axis down, you'll need to convert them to Y up. Try something like:

bounds.containsPoint(mouseX - skeleton.x, (canvasHeight - mouseY) - skeleton.y)

There is a spine.webgl.OrthoCamera class which has a screenToWorld method, but the simple example you linked above doesn't use it. The spine-ts webgl demos use it, eg:
spine-runtimes/hoverboard.js at master
However, it should not be hard to transform your mouse coordinates to skeleton coordinates without needing to rewrite all your stuff.

kyttias

Pointing me toward the hoverboard example helped quite a bit! However, in order to use screenToWorld() I had to compensate for the skeleton being centered on the canvas at run time, I was able to reverse engineer this solution:

var skeletalBounds = skeletons[activeSkeleton].bounds;
var adjustedX = mouseX - (3*skeletalBounds.offset.x + 2*skeletalBounds.size.x)/8;
var adjustedY = mouseY - (skeletalBounds.offset.y + skeletalBounds.size.y)/2;
var actualLocation = SceneRenderer.camera.screenToWorld(coords.set(adjustedX, adjustedY, 0), w, h);
var hitBoundingBoxAttachment = bounds.containsPoint(actualLocation.x, actualLocation.y);

(It's worth noting that the canvas centers the skeleton based on its bounds, not on where the root bone is located relative to its bounds. To center the skeleton by its bounds above 0,0 point, it was as simple as selecting all the images in Spine, and using Translate to place them there with the World Axes and Images Compensate on - luckily this part was easy enough to figure out!)

However, my solution does not compensate for the canvas arbitrarily scaling things down if it can't fit within the canvas. It tries its very hardest to make a nice aspect ratio! Is there a function to assist with compensating for this? Everything is being centered and resized in the resize() function. (I'm still working off code provided with this demo: https://github.com/EsotericSoftware/spine-runtimes/blob/master/spine-ts/webgl/example/index.html and it's using said resize() function - so it's what I've used, too!)

So long as I don't make my canvas smaller and have this scaling issue, I should be fine. I will need to make my canvas smaller, but I usually won't need the mouse/touch functionality on the smaller canvas. However, I may one day be working with mobile devices, and I'd be facing the scaling problem all over again. I'd like to tackle it now, if you'd be willing to help me make this one last leap. :$

Secondly, and this one should be easy, how can I get the name of which bounding box attachment I've found with bounds.containsPoint()? I want to play a different animation based on which one I've touched, afterall.

Thanks so much for your patience and help!

Xelnath

I had trouble with scaling too in my UI - I worked around it by forcing it to maintain a 'me' defined aspect ratio. Then I was responsible for adjusting the ratio for various resolutions.

This was how I managed it.

Nate

containsPoint returns the BoundingBoxAttachment that was hit, or null if none were hit.

A few of the demos have touch interaction. They work at any size, so you could use one of them as a base. They use a few utility classes to handle common functionality. The way they size and position the skeleton is maybe a bit complex, but necessary to handle showing the demo at any size.

If you want to stick to the simpler example, again you should be able to transform the coordinates without needing SceneRenderer or a camera. I gave it a try, modifying the simple example to have a SkeletonBounds (none of the demo skeletons have bounding boxes though):
http://n4te.com/x/1014-b5aH.txt
The example leaves the root bone at 0,0 in GL coordinates, instead adjusting the matrix to place the skeleton on the canvas:

var centerX = bounds.offset.x + bounds.size.x / 2;
var centerY = bounds.offset.y + bounds.size.y / 2;
var scaleX = bounds.size.x / canvas.width;
var scaleY = bounds.size.y / canvas.height;
var scale = Math.max(scaleX, scaleY) * 1.2;
if (scale < 1) scale = 1;
var width = canvas.width * scale;
var height = canvas.height * scale;

mvp.ortho2d(centerX - width / 2, centerY - height / 2, width, height);
gl.viewport(0, 0, canvas.width, canvas.height);

You need to do similar to transform mouse coordinates to GL coordinates:

canvas.addEventListener('mousemove', function (e) {
   var rect = canvas.getBoundingClientRect();
   var x = e.clientX - rect.left;
   var y = e.clientY - rect.top;

   var bounds = skeletons[activeSkeleton].bounds;
   var centerX = bounds.offset.x + bounds.size.x / 2;
   var centerY = bounds.offset.y + bounds.size.y / 2;
   var scaleX = bounds.size.x / canvas.width;
   var scaleY = bounds.size.y / canvas.height;
   var scale = Math.max(scaleX, scaleY) * 1.2;
   if (scale < 1) scale = 1;

   // This is the important part:
   x = x - canvas.width / 2 + centerX / scale; 
   y = canvas.height / 2 - y + centerY / scale;

   var skeleton = skeletons[activeSkeleton].skeleton;
   var skeletonBounds = skeletons[activeSkeleton].skeletonBounds;
   skeletonBounds.update(skeleton);
   var box = skeletonBounds.containsPoint(x, y);

   console.log('Mouse is at: ' + x + ', ' + y);
   if (box) console.log('Hit: ' + box.data.name);
}, false);

kyttias

That's much cleaner! However, var skeletonBounds above needs to be new spine.SkeletonBounds(); and not skeletons[activeSkeleton].bounds; if you want access to update(). Additionally, box.data.name should be simply box.name. Otherwise

this is exactly what I needed from the start, thank you! :heart:

Here's the full thing again with those edits, in case it helps anyone else in the future:

canvas.addEventListener('mousemove', function (e) {
    var rect = canvas.getBoundingClientRect();
    var x = e.clientX - rect.left;
    var y = e.clientY - rect.top;

var bounds = skeletons[activeSkeleton].bounds;
var centerX = bounds.offset.x + bounds.size.x / 2;
var centerY = bounds.offset.y + bounds.size.y / 2;
var scaleX = bounds.size.x / canvas.width;
var scaleY = bounds.size.y / canvas.height;
var scale = Math.max(scaleX, scaleY) * 1.2;
if (scale < 1) scale = 1;

x = x - canvas.width / 2 + centerX / scale; 
y = canvas.height / 2 - y + centerY / scale;

var skeleton = skeletons[activeSkeleton].skeleton;
var skeletonBounds = new spine.SkeletonBounds(); 
skeletonBounds.update(skeleton);
var box = skeletonBounds.containsPoint(x, y);

console.log('Mouse is at: ' + x + ', ' + y);
if (box) console.log('Hit: ' + box.name);
}, false);

Thanks so much! :party:

Nate

Glad that helps. 🙂 You shouldn't create a new SkeletonBounds every time though. This line:

var skeletonBounds = skeletons[activeSkeleton].skeletonBounds;

Gets the instance stored in the object that holds the skeleton, bounds, etc (line 178).

truelion07

This would of taken days to figure out, thanks. Should be turned into an article!