I don't think there's a way so it INCLUDES the file extension. It just removes it by default.
For example, this is how the coin project's atlas looks. Note how it says "coin" and "shine" and not "coin.png" and "shine.png"
size: 1024,256
format: RGBA8888
filter: Linear,Linear
repeat: none
coin
rotate: false
xy: 2, 2
size: 259, 245
orig: 259, 245
offset: 0, 0
index: -1
coin-invert
rotate: false
xy: 263, 2
size: 259, 245
orig: 259, 245
offset: 0, 0
index: -1
shine
rotate: false
xy: 524, 2
size: 72, 245
orig: 72, 245
offset: 0, 0
index: -1
In the Spine project, you may have to use the "Find and Replace" function in the tree if your attachments have that in their names. But why did you have them in their names in the first place? Was it a different plugin?