Spark cassandra doubt

[vendettaa]

fixed 

[jefferson1]

Calling regina Cassandra to answer this question 

[tacobell fan]

Open vendor ticket and ask them know. Your company must be paying support cost for every product they use. 

[Katara]

Mode overwrite teesi insert vadali

[panipoori]

28 minutes ago, vendettaa said:

Am trying to save dataframe into cassandra

so dataframe has 2000 records 

while saving to cassandra am able to save only one record 

for example 

dataframe is like below i.e dfout

emp_id   emp_score

1          12222 

1          33334

1           44556

1          89909

2         7889090

2     67788

2      55678

 

In cassandra i have table  as

 structuredstreaming.emp_1 (emp_id text,emp_score text,    PRIMARY KEY (emp_id)

below is the code for saving into cassandra from spark (dataframe)

dfout.write.format("org.apache.spark.sql.cassandra").mode("overwrite").option("confirm.truncate","true").option("spark.cassandra.connection.host","1000.000.00").
  option("spark.cassandra.connection.port","1111").option("keyspace","structuredstreaming").option("table"," emp_1").save()

with this code am able to save only one record per primary key in cassandra that is 

it is saving as 

emp_id emp_score

1              12222

2              67788

I want complete data frame into cassandra .how

??????

How much is your billing rate for working on Apache Spark

[Katara]

9 minutes ago, panipoori said:

How much is your billing rate for working on Apache Spark

Niku percentage adugutava?

[vendettaa]

19 minutes ago, Katara said:

Mode overwrite teesi insert vadali

not working, mode overwrite is for table overwrite

ikda data save avatledu mottam

[vendettaa]

26 minutes ago, tacobell fan said:

Open vendor ticket and ask them know. Your company must be paying support cost for every product they use. 

POC idi 

 

[panipoori]

Just now, Katara said:

Niku percentage adugutava?

yes bro need money pocket tight so will move to this

 

[panipoori]

23 minutes ago, panipoori said:

How much is your billing rate for working on Apache Spark

How much is your billing rate for working on Apache Spark

[vendettaa]

fix chesa its row key issue in cassandra

[bavaluu]

[Idassamed]

12 minutes ago, vendettaa said:

fix chesa its row key issue in cassandra

Nice 

[tacobell fan]

24 minutes ago, vendettaa said:

fix chesa its row key issue in cassandra

congrats 

[soodhilodaaram]

3 hours ago, vendettaa said:

Am trying to save dataframe into cassandra

so dataframe has 2000 records 

while saving to cassandra am able to save only one record 

for example 

dataframe is like below i.e dfout

emp_id   emp_score

1          12222 

1          33334

1           44556

1          89909

2         7889090

2     67788

2      55678

 

In cassandra i have table  as

 structuredstreaming.emp_1 (emp_id text,emp_score text,    PRIMARY KEY (emp_id)

below is the code for saving into cassandra from spark (dataframe)

dfout.write.format("org.apache.spark.sql.cassandra").mode("overwrite").option("confirm.truncate","true").option("spark.cassandra.connection.host","1000.000.00").
  option("spark.cassandra.connection.port","1111").option("keyspace","structuredstreaming").option("table"," emp_1").save()

with this code am able to save only one record per primary key in cassandra that is 

it is saving as 

emp_id emp_score

1              12222

2              67788

I want complete data frame into cassandra .how

??????

primary key is unique, will alow only one record per key

you may have to check non unique primary key options in cassandra, i dont know cassandra..logically by data modelling thoughts