help : raw table in hive

[vendettaa]

i have data ingested into hive from Kafka that is raw table in which each record is Json object

and i have to convert it to table in hive ,

need to generate surrogate key for each record 

 

How to do that ? 

Help please

 

[jpismahatma]

Write custom plugin ..

[vendettaa]

9 minutes ago, jpismahatma said:

Write custom plugin ..

u mean UDF?

[mettastar]

1 hour ago, vendettaa said:

i have a raw table in hive which contains json objects as records (kafka output)

and i have to convert it to table in hive ,

need to generate primary key for each record

 

How to do that ? 

Help please

 

Hive lo inbuilt function undi Json read chryadaniki .. 

[mettastar]

Also serde lu unayi Hcatalog vi use it

[vendettaa]

4 minutes ago, mettastar said:

Hive lo inbuilt function undi Json read chryadaniki .. 

reference links unte post it anniyya

 

[former]

DB lo post vesthe evadu help chesthadu. Post inti Kafka or Spark forums. or try to find in the google or bolg posts

[mettastar]

1 hour ago, vendettaa said:

reference links unte post it anniyya

 

https://stackoverflow.com/questions/11479247/how-do-you-make-a-hive-table-out-of-json-data

Read one with 25 votes.. i used similar in one of projects .. thats the easiest way i found .. no need to use any serdes 

[mettastar]

Lateral view antaru

[vendettaa]

2 hours ago, former said:

DB lo post vesthe evadu help chesthadu. Post inti Kafka or Spark forums. or try to find in the google or bolg posts

Naku telsu ikda vati mida working vallu unnaru 

[vendettaa]

1 hour ago, mettastar said:

Lateral view antaru

Json tuple and lateral view column ni string ga convert chesi kada

ikda file anta Json untundi ,ok spark use chesta Inka Json ni dataframe chesi tables Ki pampista

[mettastar]

30 minutes ago, vendettaa said:

Json tuple and lateral view column ni string ga convert chesi kada

ikda file anta Json untundi ,ok spark use chesta Inka Json ni dataframe chesi tables Ki pampista

File antha json untene i used and it worked .. and i used it on a big file 

[kasi]

4 hours ago, vendettaa said:

i have a raw table in hive which contains json objects as records (kafka output)

and i have to convert it to table in hive ,

need to generate primary key for each record

 

How to do that ? 

Help please

 

😂😂😂😂😂😂

Akka hive lo primary key lu unav.....

Btw asalu nuvvu em cheyali anukuntunav???

 

Naku nuvu em chepputhunavo 1% kuda ardham kala

[kasi]

2 hours ago, mettastar said:

https://stackoverflow.com/questions/11479247/how-do-you-make-a-hive-table-out-of-json-data

Read one with 25 votes.. i used similar in one of projects .. thats the easiest way i found .. no need to use any serdes 

Kaka antha avasaram kuda ledu....Schema create chesetappudu.....Adi stored as json antea chalu

 

Adi aa Schema ki folder create chesthundhi..... Just aa folder lo json file save chesthea...... Hive table ga read cheyochu

[kasi]

5 hours ago, vendettaa said:

i have a raw table in hive which contains json objects as records (kafka output)

and i have to convert it to table in hive ,

need to generate primary key for each record

 

How to do that ? 

Help please

 

😂😂😂😂😂😂🤣😂🤣😂🤣