I am implementing a Handwritten Text Recognition system in Tensorflow using the Keras interface. In prediction phase, my system would take as an input the image of a line (.jpg format) and return as an output the corresponding transcription. So this is not a classification problem: it is more akin to encoding-decoding, where the input is an image and the output is text.

To sum up, my training samples are couples of

• x: an image (read with cv2 from a .jpg file)
• y: a transcription (read from a .txt file)

By the way this is how they stored in the filesystem:

In almost all the tutorials I have found, for the sake of simplicity all the training samples are loaded at once in memory at the beginning of the training process and then scanned using generator methods. With large datasets, however, loading all the training data in memory may lead to OOM before the training even begins.

To avoid this risk I would like to load batches in memory on the fly; so I am wondering if TF or Keras provide any built-in solution that loads the training data loading them progressively.

I have found that the Keras method tf.keras.preprocessing.image_dataset_from_directory does something pretty close to what I am thinking about, but apparently it only works for classification tasks (it requires your directory to have a sub-folder for each class; in my scenario, I do not have any classes).

I believe I can implement my own solution zipping an ImageDataGenerator and a... TextDataGenerator-something (not a real class, I should implement this one on my own). I found an example that zips and scans two ImageDataGenerators in the documentation.

However my scenario should be quite common, so I have a hard time believing that it is not already handled by any pre-existing TF/Keras classes or methods. If possible, I would avoid re-inventing the wheel; do you have any suggestions?

Thank you for your help. I really appreciate it!