1 year ago

#352336

test-img

Ben

Tensorboard with Azure Blob Storage. I always get "No dashboards are active for the current data set."

I can get list of model ckeckpoint directory with tensorflow gfile and tensorflow-io.

tf.io.gfile.listdir('az://rndstoragesample/containersample/efficientdet-finetune/ckpt')
['best_objective.txt', 'checkpoint', 'config.yaml', 'events.out.tfevents.1648598968.my-pipeline-ckh9c-3384085500', 'events.out.tfevents.1648609048.my-pipeline-ks2sj-2126585855', 'graph.pbtxt', 'model.ckpt-0.data-00000-of-00001' ... ... ]

But When I try to set the Azure blob storage path to logdir of tensorboard, I always get "No dashboards are active for the current data set."

root@a601721015eb:~# tensorboard --logdir az://rndstoragesample/containersample/efficientdet-finetune/ckpt --bind_all
TensorBoard 2.8.0 at http://a601721015eb:6006/ (Press CTRL+C to quit)
W0330 03:53:26.889187 140443207591680 projector_plugin.py:489] Failed reading "az://rndstoragesample/containersample/efficientdet-finetune/ckpt/model.ckpt-178"

img1

I downloaded tensorflow-io with pip and set accesskey on env var

pip install tensorflow-io
export TF_AZURE_STORAGE_KEY="<my-key>"

Then I check the connection with storage through python script.

import tensorflow_io
import tensorflow

account_name = 'rndstoragesample'
pathname = 'az://{}/aztest'.format(account_name)
tf.io.gfile.mkdir(pathname)
tf.io.gfile.listdir('az://rndstoragesample/containersample/efficientdet-finetune/ckpt')

The connection is looks good, so I try to log the blob directory with tensorboard.

tensorboard --logdir az://rndstoragesample/containersample/efficientdet-finetune/ckpt --bind_all

I got the log below, and face to "No dashboards are active for the current data set." page.

TensorBoard 2.8.0 at http://a601721015eb:6006/ (Press CTRL+C to quit)
W0330 03:53:26.889187 140443207591680 projector_plugin.py:489] Failed reading "az://rndstoragesample/containersample/efficientdet-finetune/ckpt/model.ckpt-178"

Do you guys have any idea for this problem?

Thanks in advance.

azure

tensorflow

azure-blob-storage

tensorboard

0 Answers

Your Answer

Accepted video resources