On 7/17/2020 at 11:24 PM, jajjanaka_jandri said:

nenu 5 years nunchi working baa.

You have to know the following.

1. hadoop framework tools (sqoop, oozie, hive, pig, hbase, hdfs etc).

2. sql querying, performance tuning

3. python, java/scala languages

4. APIs

5. linux env hands on/ scripting 

6. No sql like: mangodb

7.inka aah pina vache latest tools flunk, beam, kafka and the list goes on.

 

 

 

Bro did u work on parquet files..in aws or azure or spark?

On 7/17/2020 at 9:02 PM, SharkTank said:

try to implement everything using dataframes single shot . Spark sql usage enta limit cheste good, coz view occupies space. Use spark 2 . Ipdu indulo emundi . Error post chey . As i said give 

spark.conf.set("spark.sql.shuffle.partitions",2000) on your sparkscala console . This should fix.

spark.conf.set("spark.sql.shuffle.partitions",100)

Bro.. parquet files meeda konchem help needed in azure or aws ..u have any idea 

On 7/21/2020 at 1:54 PM, kevinUsa said:

Em scripts raste vachinatee na 

Bro...spark help needed with parquet files..jara pyspark script rayali ...

6 minutes ago, Sarvapindi said:

Bro did u work on parquet files..in aws or azure or spark?

hive lo parquet tables create chesi, data store chesa antha kante ekkuva work cheyla

4 minutes ago, Sarvapindi said:

Bro...spark help needed with parquet files..jara pyspark script rayali ...

I don't know

 

Ltt

Hi, can someone please share some resume’s on “azure data engineer “ and “big data “ skills . Thanks 🙏

9 hours ago, SwamyRaRa said:

Hi, can someone please share some resume’s on “azure data engineer “ and “big data “ skills . Thanks 🙏

Ltt

On 8/11/2020 at 9:12 AM, Sarvapindi said:

Bro...spark help needed with parquet files..jara pyspark script rayali ...

నాకు సహాయం చేయగలి.