Health checks on spark jar

[JavaBava]

On 2/5/2019 at 5:08 PM, vendettaa said:

my manager wants me to use something like jmeter to test the jar

Test cheyyala monitor cheyyala .. ? Did u look into new relic or appdynamics agents?

[WeBeliveInTigerman]

50 minutes ago, vendettaa said:

Looking to schedule jobs that run on spark scheduler .Oozie is not better option so what are best options to schedule jobs?

Dont say cron jobs

Apache Livy, Luigi, Celery

[vendettaa]

2 hours ago, WeBeliveInTigerman said:

Apache Livy, Luigi, Celery

Livy 2.4.0 spark mida work cheydu

luigi is python based and am looking for scala based

[vendettaa]

2 hours ago, JavaBava said:

Test cheyyala monitor cheyyala .. ? Did u look into new relic or appdynamics agents?

Ledu enti relic

[mettastar]

3 hours ago, vendettaa said:

Looking to schedule jobs that run on spark scheduler .Oozie is not better option so what are best options to schedule jobs?

Dont say cron jobs

https://nifi.apache.org/docs/nifi-docs/html/user-guide.html 

[mettastar]

jar health check ante emchestharu usual ga ?

[bavaluu]

On 2/5/2019 at 6:32 PM, Kalam_Youtheman said:

odiniyaavaaa .    u know what jmeter is used for ?  its purely for load testing

 

basics telskoni ask questions baaalu

valla manager jmeter ante ee vendetta bava auto ekki meter meter ani thiruguthondi kada ee db antha 

[MuPaGuNa]

10 minutes ago, bavaluu said:

valla manager jmeter ante ee vendetta bava auto ekki meter meter ani thiruguthondi kada ee db antha 

[WeBeliveInTigerman]

2 hours ago, vendettaa said:

Livy 2.4.0 spark mida work cheydu

luigi is python based and am looking for scala based

Did you try??. Livy works for 2.4. If you don't have latest.. download n build agaisnt 2.4..

Also for scheduling why do u care if its python or scala.. Build a jar and Luigi can do spark-submit or trigger any number of spark jobs

[ranku_mogudu]

enduku bhayya alaaga antaru @vendettaa ni

[vendettaa]

15 hours ago, WeBeliveInTigerman said:

Did you try??. Livy works for 2.4. If you don't have latest.. download n build agaisnt 2.4..

Also for scheduling why do u care if its python or scala.. Build a jar and Luigi can do spark-submit or trigger any number of spark jobs

ok will try with Luigi 

 

I did find this helpful for doing health checks

https://metrics.dropwizard.io/3.1.0/getting-started/

[vendettaa]

15 hours ago, bavaluu said:

valla manager jmeter ante ee vendetta bava auto ekki meter meter ani thiruguthondi kada ee db antha 

finally dorkindi  using Dropwizard Metrics

[vendettaa]

15 hours ago, WeBeliveInTigerman said:

Did you try??. Livy works for 2.4. If you don't have latest.. download n build agaisnt 2.4..

Also for scheduling why do u care if its python or scala.. Build a jar and Luigi can do spark-submit or trigger any number of spark jobs

metrics display cheyali programmatically so Livy lite , Have to explore Luigi, how efficient is Luigi for scheduling ? ok python use chestanu to schedule that jar

[vendettaa]

17 hours ago, mettastar said:

jar health check ante emchestharu usual ga ?

database health checks, connections check  ala 

final HealthCheckRegistry healthChecks = new HealthCheckRegistry();

public class DatabaseHealthCheck extends HealthCheck {
    private final Database database;

    public DatabaseHealthCheck(Database database) {
        this.database = database;
    }

    @Override
    public HealthCheck.Result check() throws Exception {
        if (database.isConnected()) {
            return HealthCheck.Result.healthy();
        } else {
            return HealthCheck.Result.unhealthy("Cannot connect to " + database.getUrl());
        }
    }
}

 

Like this

[vendettaa]

@psycopk @MOD23 kasta e thread lo oka ID asahyamaina pics vestu spam chestunnaru

can u delete that filthy stuff and warn him or ban him