1 year ago

#192805

test-img

Priyanka2304

How to write .dsv file into dynamic folders in U-SQL script

I am new to U-SQL and am trying to read multiple .dsv files from a folder and write the output as .dsv files in dynamic folder/sub folder. So far I have successfully read multiple .dsv files and am able to write the files in a single location(pre-defined location) but am not able to write the .dsv files into multiple folders/subfolders. The folders and subfolders are defined based on the file name. As an example, If the file name is ABC_20200421@015814.dsv, it should be reading data from the .dsv file and write it into folder system: test/ABC/2020/04/21/File_Data.dsv

The code I have so far is :

    DECLARE @ImportFile string = "*.dsv"; 

-- To read all the .dsv files and move them to folder locations 
DECLARE  @Code String = @ImportFile.Substring(0, 3); 
DECLARE  @CommanName String = @ImportFile.Substring(4, 14); File_Contents
DECLARE  @Year String = @ImportFile.Substring(18, 4); 
DECLARE  @Month String = @ImportFile.Substring(22, 2); 
DECLARE  @File_Date String = @ImportFile.Substring(24, 2); 

 @result=
SELECT col1,col2,col3 
FROM Table1

//Writing to dsv file:

OUTPUT @result
TO "test/"+@Code+"/"+@Year+"/"+@Month+"/"+@File_Date+"/File_Data.dsv"
USING Outputters.Text(delimiter : '|', quoting: false);

On running the code, I get an error:

E_CSC_USER_EXPRESSIONNOTCONSTANTFOLDABLE: Expression cannot be constant folded.

Details:

at token '"/test/"', near the ###:

OUTPUT @result TO ### "/test/"+@Code+"/"+@Year+"/"+@Month+"/"+@File_Date+"/STAY_REVENUES.dsv" USING Outputters.Text(delimiter : '|', quoting : false)

Any help would be appreciated. Thanks in advance.

azure

azure-data-lake

u-sql

0 Answers

Your Answer

Accepted video resources