dimension error

[YJonmo]

Hi there,

Thanks for this work.

I am trying to get the "kornia.feature.LightGlue" or "kornia.feature.LightGlueMatcher" running but I keep getting dimension error.

I could run the LighGlue repository running and integrated into my pipeline. Now I am trying to replace it with kornia.

Here is how I am calling the LighGlue:

img0, scales0 = load_image(images / 'image3_2.PNG', resize=128, grayscale=False)

img1, scales1 = load_image(images / 'image2_2.PNG', resize=128, grayscale=False)

img1 = img1[None, :]
img0 = img0[None, :]
disk = kornia.feature.DISK.from_pretrained('depth')
features1 = disk(img1)
features0 = disk(img0)
kp0 = features0[0].keypoints
kp1 = features1[0].keypoints
des0= features0[0].descriptors
des1= features1[0].descriptors
image0 = {'keypoints': kp0[None,:].type(torch.float32).to(device),
          'descriptors': des0[None,:].type(torch.float32).to(device),
          'image' : img0
         }
image1 = {
    'keypoints': kp1[None, :].type(torch.float32).to(device),
    'descriptors': des1[None, :].type(torch.float32).to(device),
    'image': img1
         }
input = {'image0':image0, 'image1':image0}

matcher = kornia.feature.LightGlue(features='disk', params={})
output = matcher(input)

Which is partially taken from LighGlue repo as shown here.

But I get following error:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/bv/anaconda3/envs/kornia/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/bv/anaconda3/envs/kornia/lib/python3.10/site-packages/kornia/feature/lightglue.py", line 327, in forward
    return self._forward(data)
  File "/home/bv/anaconda3/envs/kornia/lib/python3.10/site-packages/kornia/feature/lightglue.py", line 346, in _forward
    kpts0 = normalize_keypoints(kpts0_, size=size0)
  File "/home/bv/anaconda3/envs/kornia/lib/python3.10/site-packages/torch/cuda/amp/autocast_mode.py", line 105, in decorate_fwd
    return fwd(*args, **kwargs)
  File "/home/bv/anaconda3/envs/kornia/lib/python3.10/site-packages/kornia/feature/lightglue.py", line 36, in normalize_keypoints
    kpts = (kpts - shift[:, None]) / scale[:, None, None]
RuntimeError: The size of tensor a (2) must match the size of tensor b (128) at non-singleton dimension 3

Is there any example of LighGlue matcher together with DISK or SuperPoint s that I could use?

Cheers,
Yaqub

[ducha-aiki]

Hi,

I have created the example here, both with LightGlue and LightGlueMatcher objects.
https://github.com/kornia/tutorials/blob/lightglue/nbs/image_matching_lightglue.ipynb