Only Coders

python (65.1k questions)

javascript (44.2k questions)

reactjs (22.7k questions)

java (20.8k questions)

c# (17.4k questions)

html (16.3k questions)

r (13.7k questions)

android (12.9k questions)

Questions - tensorflow-transform

Add reserved tokens to `tft.vocabulary`

I would like to append words to the vocabulary created by tft.vocabulary that are not a part of the training samples (i.e. <mask> and <pad> tokens). I see in the docs that the tft.vocabula...

Zach Robertson

python

tensorflow

tfx

mlops

tensorflow-transform

Votes: 0

Answers: 1

Latest Answer

What is it that you're trying to achieve? I don't think that key_fn is related as it only affects the ordering of the vocabulary (and top k when provided) Could you compute the vocabulary after append...

Zohar

apache beam rows to tfrecord in order to GenerateStatistics

I have built a pipeline that read some data, does some manipulations and create some apache beam Row objects (Steps 1 and 2 in the code below). I then would like to generate statistic and write them t...

DarioB

apache-beam

apache-beam-io

tensorflow-transform

tensorflow-data-validation

Votes: 0

Answers: 1

Latest Answer

Unfortunately there’s no other way to do it without writing in a file. We need to do this process because Machine Learning frameworks consume training data as sequence of examples, this file formats f...

Raul Saucedo

join datasets with tfx tensorflow transform

I am trying to replicate some data preprocessing that I have done in pandas into tensorflow transform. I have a few CSV files, which I joined and aggregated with pandas to produce a training dataset. ...

DarioB

apache-beam

tfx

tensorflow-transform

Votes: 0

Answers: 1

Latest Answer

You can use the Beam Dataframes API to do the join and other preprocessing exactly as you would have in Pandas. You can then use to_pcollection to get a PCollection that you can pass directly to your ...

robertwb

How to get vocabulary size in tensorflow_transform before apply_vocabulary?

Also posted the question at https://github.com/tensorflow/transform/issues/261 I am using tft in TFX and needs to transform string list class labels into multi-hot indicators inside preprocesing_fn. E...

ynait

tensorflow

tfx

tensorflow-transform

Votes: 0

Answers: 1

Latest Answer

As per this comment in the github issue, You can use tft.experimental.get_vocabulary_size_by_name (link) to achieve the same.

user11530462

Posts

Questions

Blogs

Jobs