1 year ago
#352336
Ben
Tensorboard with Azure Blob Storage. I always get "No dashboards are active for the current data set."
I can get list of model ckeckpoint directory with tensorflow gfile and tensorflow-io.
tf.io.gfile.listdir('az://rndstoragesample/containersample/efficientdet-finetune/ckpt')
['best_objective.txt', 'checkpoint', 'config.yaml', 'events.out.tfevents.1648598968.my-pipeline-ckh9c-3384085500', 'events.out.tfevents.1648609048.my-pipeline-ks2sj-2126585855', 'graph.pbtxt', 'model.ckpt-0.data-00000-of-00001' ... ... ]
But When I try to set the Azure blob storage path to logdir of tensorboard, I always get "No dashboards are active for the current data set."
root@a601721015eb:~# tensorboard --logdir az://rndstoragesample/containersample/efficientdet-finetune/ckpt --bind_all
TensorBoard 2.8.0 at http://a601721015eb:6006/ (Press CTRL+C to quit)
W0330 03:53:26.889187 140443207591680 projector_plugin.py:489] Failed reading "az://rndstoragesample/containersample/efficientdet-finetune/ckpt/model.ckpt-178"
I downloaded tensorflow-io with pip and set accesskey on env var
pip install tensorflow-io
export TF_AZURE_STORAGE_KEY="<my-key>"
Then I check the connection with storage through python script.
import tensorflow_io
import tensorflow
account_name = 'rndstoragesample'
pathname = 'az://{}/aztest'.format(account_name)
tf.io.gfile.mkdir(pathname)
tf.io.gfile.listdir('az://rndstoragesample/containersample/efficientdet-finetune/ckpt')
The connection is looks good, so I try to log the blob directory with tensorboard.
tensorboard --logdir az://rndstoragesample/containersample/efficientdet-finetune/ckpt --bind_all
I got the log below, and face to "No dashboards are active for the current data set." page.
TensorBoard 2.8.0 at http://a601721015eb:6006/ (Press CTRL+C to quit)
W0330 03:53:26.889187 140443207591680 projector_plugin.py:489] Failed reading "az://rndstoragesample/containersample/efficientdet-finetune/ckpt/model.ckpt-178"
Do you guys have any idea for this problem?
Thanks in advance.
azure
tensorflow
azure-blob-storage
tensorboard
0 Answers
Your Answer