python (65.1k questions)

javascript (44.2k questions)

reactjs (22.7k questions)

java (20.8k questions)

c# (17.4k questions)

html (16.3k questions)

r (13.7k questions)

android (12.9k questions)

Questions - drop

How to drop pandas dataframe columns containing special characters

How do I drop pandas dataframe columns that contains special characters such as @ / ] [ } { - _ etc.? For example I have the following dataframe (called df): I need to drop the columns Name and Matc...
test-img

Giampaolo Levorato

pandas

special-characters

drop

Votes: 0

Answers: 1

Latest Answer

One option is to use a regex with str.contains and apply, then use boolean indexing to drop the columns: import re chars = '¬`!"£$£#/\\' regex = f'[{"".join(map(re.escape, chars))}]' # ...
test-img

mozway

How to drop records containing cell values equals to the header in pandas

I have read in this dataframe (called df): As you can see there is a record that contains the same values as the header (ltv and age). How do I drop that record in pandas? Data: df = pd.DataFrame({'l...
test-img

Giampaolo Levorato

python-3.x

pandas

dataframe

header

drop

Votes: 0

Answers: 2

Latest Answer

One way is to just filter it out (assuming the strings match the column name they are in): out = df[df['ltv']!='ltv'] Another could be to use to_numeric + dropna: out = df.apply(pd.to_numeric, errors...
test-img

user7864386

How to drop entire record if more than 90% of features have missing value in pandas

I have a pandas dataframe called df with 500 columns and 2 million records. I am able to drop columns that contain more than 90% of missing values. But how can I drop in pandas the entire record if 90...
test-img

Giampaolo Levorato

python-3.x

pandas

dataframe

missing-data

drop

Votes: 0

Answers: 2

Latest Answer

You could use isna + mean on axis=1 to find the percentage of NaN values for each row. Then select the rows where it's less than 0.9 (i.e. 90%) using loc: out = df.loc[df.isna().mean(axis=1)<0.9]
test-img

user7864386

Drop pandas column with constant alphanumeric values

I have a dataframe df that contains around 2 million records. Some of the columns contain only alphanumeric values (e.g. "wer345", "gfer34", "123fdst"). Is there a python...
test-img

Giampaolo Levorato

pandas

alphanumeric

drop

Votes: 0

Answers: 1

Latest Answer

Apply Series.str.isalnum column-wise to mask all the alphanumeric values of the DataFrame. Then use DataFrame.all to find the columns that only contain alphanumeric values. Invert the resulting boolea...
test-img

Rodalm

Posts

Questions

Blogs

Jobs