Questions about gcs
Read more about gcs
python (65.1k questions)
javascript (44.2k questions)
reactjs (22.7k questions)
java (20.8k questions)
c# (17.4k questions)
html (16.3k questions)
r (13.7k questions)
android (12.9k questions)
Python - Apache-beam outputs an empty file using dataflow runner, works fine with direct runner. Dataflow does not raise any errors
I've been trying to run this apache-beam script. This script runs nightly through an airflow DAG, and works perfectly fine that way so I'm (reasonably) confident that the script is correct. I think th...
Nyrb
Votes: 0
Answers: 0
Unzipping files with Apache Beam (Python) but when using WriteToText it puts all columns as lines
I am very new to programming and Apache Beam, and I am trying to read plenty zip files on a a GCS bucket and unzip them and save again as csv on GCS.
with beam.Pipeline() as pipeline:
readable_files =...
Eveline Marques Guimarães
Votes: 0
Answers: 1
Can you trigger Python Scripts from Dataproc?
I am experimenting with GCP. I have a local environment with Hadoop. It consists of files stored on HDFS and a bunch of python scripts which make API calls and trigger pig jobs. These python jobs are ...
JYosen
Votes: 0
Answers: 2
Is there a way to provide schema or auto-detect schema when uploading csv from GCS to BigQuery?
I am trying to upload a csv file from Google Cloud Storage (GCS) to BigQuery (BQ) and auto-detect schema.
What I tried to do is enable auto-detect schema and enter the number of rows to skip in "...
R.A
Votes: 0
Answers: 3