python (65.1k questions)

javascript (44.2k questions)

reactjs (22.7k questions)

java (20.8k questions)

c# (17.4k questions)

html (16.3k questions)

r (13.7k questions)

android (12.9k questions)

Questions - transformer-model

Using the encoder part only from T5 model

I want to build a classification model that needs only the encoder part of language models. I have tried Bert, Roberta, xlnet, and so far I have been successful. I now want to test the encoder part on...
test-img

ls_grep

deep-learning

pytorch

huggingface-transformers

transformer-model

encoder-decoder

Votes: 0

Answers: 1

Latest Answer

Load T5 encoder checkpoint only: from transformers import T5EncoderModel T5EncoderModel._keys_to_ignore_on_load_unexpected = ["decoder.*"] auto_model = T5EncoderModel.from_pretrained("t...
test-img

Dương Xuân Hiếu

Vision transformer binary classifier is only predicting one class

I wrote a code for a vision transformer to classify mammograms into benign and malignant. After training for 30 epochs, the model is, however, predicting only one class(benign). All the final predicti...
test-img

James Albert

python

keras

deep-learning

classification

transformer-model

Votes: 0

Answers: 0

Why DETR need to set a empty class?

Why DETR need to set a empty class? It has set a "Background" class, which means non-object, why?
test-img

wwzzyy

pytorch

transformer-model

Votes: 0

Answers: 2

Latest Answer

TL;DR By default DETR always predict 100 bounding boxes. Empty class is used as a condition to filter out meaningless bounding boxes. Full explanation If you look at the source code, the transformer d...
test-img

Raven Cheuk

Is it possible to access hugging face transformer embedding layer?

I want to use a pretrained hugging face transformer language model as an encoder in a sequence to sequence model. The task is grammatical error correction, so both input and output come from the same ...
test-img

Aleksander Misztal

python

machine-learning

nlp

huggingface-transformers

transformer-model

Votes: 0

Answers: 2

Latest Answer

Taking bert as example if you load BertModel from transformers import BertModel model = BertModel.from_pretrained("bert-base-uncased") print(model.embeddings) # output is BertEmbeddings( ...
test-img

Bugface

Posts

Questions

Blogs

Jobs