The model was converted successfully, but the output differs for each execution. The same does not happen for the non-converted TensorFlow model, which always yields the same results. I've attached a notebook that shows how to reproduce the problem and how to detect it. After a few runs, the output gets more consistent. It only happens when using LSTMs and seems to get even worse when using return_sequences=True.
Notebook Link: https://github.com/BernardoGO/tflite_convert_bug/blob/main/BiLSTM%20Conversion%20Test.ipynb
My model:
new_model = Sequential()
new_model.add(Input(name='input', batch_size=1, shape=(925, 3)) )
new_model.add(Dense(8))
new_model.add(LSTM(64, return_sequences=True))
new_model.add(LSTM(64, return_sequences=True))
new_model.add(LSTM(64, return_sequences=True))
new_model.add(LSTM(64, return_sequences=True))
new_model.add(Activation('softmax', name='softmax'))
new_model.summary()
Command used to run the converter
converter = tf.lite.TFLiteConverter.from_keras_model(new_model)
#converter.experimental_new_converter = True
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS, tf.lite.OpsSet.SELECT_TF_OPS]
#converter.allow_custom_ops=True
tflite_model = converter.convert()
open("tf_test.tflite", "wb").write(tflite_model)
The output from the converter invocation
INFO:tensorflow:Assets written to: /tmp/tmp4xjrqwpb/assets
Out[9]: 476052
- I've used this function to check if the output is the same after each execution:
def equal(tensor1, tensor2):
for i, j in zip(tensor1.reshape(-1), tensor2.reshape(-1)):
if abs(i - j) > 0.001:
return False
return True
System information
- OS Platform: Ubuntu 20.04
- TensorFlow installed Binary
- TensorFlow version: 2.3.1
