I am not a programmer and I am trying to implement a real-time monitoring application for a sensor system. The file is too big to upload to GitHub, but I will try my best to explain the main problem.

The data
x: time in seconds
y1: comma-separated list of intensities
y2: comma-separated list of wavelengths.

So for each datapoint/time, the sensor saves a complete spectrum and closes the txt file. Then it opens it again and saves the next datapoint/time after a very short integration time of milliseconds.

My goal
I want to choose a wavelength (or multiple ones) from the spectrum and plot the time/intensity curve in real-time.

My problem
The efficiency of my code is very bad. It needs a lot of time to convert the raw data, which essentially has two different delimiters (tab and comma), but also to find the wavelengths close to my chosen value (I choose an integer, but the list only contains decimals).

My code

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

folder = '...'
file = '...txt'

wavelength1 = 940

wavelength2 = 930
wavelength3 = 950

df = pd.read_table(folder+file, skiprows=[1], index_col=0)
time = df['Elapsed Time (Seconds)']
intensity = df['Spectra (Spectra Intensity)'].str.split(',', expand=False)
spectrum = df['Spectra Wavelength (nanometers)'].str.split(',', expand=False)

x = []
y1 = []
y2 = []
y3 = []
for i in range(len(df)):
    A = np.array([float(j) for j in spectrum[i]])
    index1 = np.where(A == min(A, key=lambda k:abs(k-wavelength1)))
    index2 = np.where(A == min(A, key=lambda k:abs(k-wavelength2)))
    index3 = np.where(A == min(A, key=lambda k:abs(k-wavelength3)))
    x.append(float(time[i]))
    y1.append(float(intensity[i][int(index1[0])]))
    y2.append(float(intensity[i][int(index2[0])]))
    y3.append(float(intensity[i][int(index3[0])]))

plt.plot(x, y1, label='940 nm')
plt.plot(x, y2, label='930 nm')
plt.plot(x, y3, label='950 nm')
plt.show()

If you have experience with such implementations, I would also appreciate tips for efficient live plotting libraries, etc. :)

Thank you very much.

Update
I made a test file here A2422_Spectra so you can see for yourself.