keras.ops.image.map_coordinates fails on uint8 input with TensorFlow backend

[sergiud]

Consider the following simple example

import keras

image = keras.ops.ones((1, 1, 3), dtype='uint8')

coordinates = keras.ops.convert_to_tensor([-1., 0., 0.])[..., None, None]
interp = keras.ops.image.map_coordinates(image, coordinates, order=1, fill_mode='constant')

that is expected to yield [[0]]. However, with KERAS_BACKEND=tensorflow this code snippet results in

2024-12-08 16:04:24.790791: W tensorflow/core/framework/op_kernel.cc:1841] OP_REQUIRES failed at gather_nd_op.cc:65 : INVALID_ARGUMENT: indices[0,0] = [-1, 0, 0] does not index into param shape [1,1,3], node name: GatherNd
2024-12-08 16:04:24.790814: I tensorflow/core/framework/local_rendezvous.cc:405] Local rendezvous is aborting with status: INVALID_ARGUMENT: indices[0,0] = [-1, 0, 0] does not index into param shape [1,1,3], node name: GatherNd
Traceback (most recent call last):
  File "<home>/tfmapc.py", line 11, in <module>
    interp = keras.ops.image.map_coordinates(image, coordinates, order=1, fill_mode='constant')
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<home>/.env/lib/python3.12/site-packages/keras/src/ops/image.py", line 787, in map_coordinates
    return backend.image.map_coordinates(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<home>/.env/lib/python3.12/site-packages/keras/src/backend/tensorflow/image.py", line 485, in map_coordinates
    contribution = tf.cond(tf.reduce_all(validities), fast_path, slow_path)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<home>/.env/lib/python3.12/site-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "<home>/.env/lib/python3.12/site-packages/keras/src/backend/tensorflow/image.py", line 481, in slow_path
    tf.transpose(tf.gather_nd(input_arr, indices)),
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tensorflow.python.framework.errors_impl.InvalidArgumentError: {{function_node __wrapped__GatherNd_device_/job:localhost/replica:0/task:0/device:CPU:0}} indices[0,0] = [-1, 0, 0] does not index into param shape [1,1,3], node name: GatherNd [Op:GatherNd] name:

The problem does not occur if I change the dtype of image from uint8 to float32 or switch either to the jax or torch backends. Also changing the fill_mode from constant to nearest avoids the issue.

Keras version: 3.7.0

[mehtamansi29]

Hi @sergiud -

Thanks for reporting this issue. I am able to reproduce this issue and found that seems like issue is related to only tensorflow backend where dtype mismatch scenarios(uint8 for image and float32 for coordinates) is not handled in tensorflow backend.
Attached gist for the reference.
We will dig more into the issue and update on the same.