Date of previous month

Comments

Post a Comment

Popular posts from this blog

Exponential Value parsing in SQL

 If you have exponential value in you column and and you want to concert into number or string then please use below function. Cast the value in double ( in some sql  double is not supported then please try with float) but if the  length  is higher like Bigint or Long Int the use double. Use format_number function  to convert exponential value to normal number. with precision. Cast again to convert into string to remove precision value. Use replace function to remove comma(',')  between string .     Problem: col -  5.498342022637685E15 Solution: SELECT  replace(cast(format_number(cast(col_name as double),0) as string),',','') as alias   from table_name    Result :- 5498342022637680
Read more

Hive Partition sub folders HIVE_UNION_SUBDIR_1,HIVE_UNION_SUBDIR_2,HIVE_UNION_SUBDIR_8

 Hive Partition have sub folders like HIVE_UNION_SUBDIR_1,HIVE_UNION_SUBDIR_2,HIVE_UNION_SUBDIR_3 Problem : When you use UNION ALL in query with Hive version 1.2.0 onward. UNION ALL will not supported and we have TEZ engine setup in hive-site.xml or hive configuration file which is responsible to create  sub folders like HIVE_UNION_SUBDIR_1 etc  on HDFS. When you use spark sql or other sql query to read Partition data then your resultant partition created with blank or 0 value. Solution: Please change the configuration file of hive and set below property. Its work for me in dataiku. hive.execution.engine=mr It will invoke Map Reduce process which will  bit slow to process the job but it will help to stop creating the extra sub folder on HDFS.  
Read more