cosmopolitan Posted June 6, 2020 Report Share Posted June 6, 2020 2 minutes ago, superhit3 said: niku yem kavali scale up & down proportional to DB instances n pool gurinchi ...grafana, CPU utilization n gO Router perf ...nodes and services, their relocation etc...consumer count for topics n queues etc. inka bouncing stopping...response times per multiple foundations kondaru organization ani kuda antaru cloud matter ...abbo inka chalane vunnay......my turn comes every 3 weeks...appudu vucha naku. ante beginnig year lo naku yekam ga potty ye vachedi. now a lot better All rounder lekka unav ga bro 1 Quote Link to comment Share on other sites More sharing options...
odhu_le_macha Posted June 6, 2020 Report Share Posted June 6, 2020 2 minutes ago, superhit3 said: niku yem kavali scale up & down proportional to DB instances n pool gurinchi ...grafana, CPU utilization n gO Router perf ...nodes and services, their relocation etc...consumer count for topics n queues etc. inka bouncing stopping...response times per multiple foundations kondaru organization ani kuda antaru cloud matter ...abbo inka chalane vunnay......my turn comes every 3 weeks...appudu vucha naku. ante beginnig year lo naku yekam ga potty ye vachedi. now a lot better Quote Link to comment Share on other sites More sharing options...
JaiTDP Posted June 6, 2020 Report Share Posted June 6, 2020 2 minutes ago, superhit3 said: prod server nu 3 days ....how? peddaga visibility and need leni app? not degrading bhayya ...just questioning how ani ante Haha.. luckily it’s backend data loading server bro not the UI. The clients didn’t have visibility abt this but we notified saying there will be some delay in processing data. total server crashed , Microsoft vaadu kuda hands up annaadu after reviewing logs. Thella manager(chaala manchodu) pushing me on UAT stuff saying client was waiting where I am fighting with this prod server !! Oka 1 month back up dhi vunte dhaanitho laaginchesaamu. Quote Link to comment Share on other sites More sharing options...
Jombo0108 Posted June 6, 2020 Report Share Posted June 6, 2020 3 minutes ago, JaiTDP said: Haha.. luckily it’s backend data loading server bro not the UI. The clients didn’t have visibility abt this but we notified saying there will be some delay in processing data. total server crashed , Microsoft vaadu kuda hands up annaadu after reviewing logs. Thella manager(chaala manchodu) pushing me on UAT stuff saying client was waiting where I am fighting with this prod server !! Skip Oka 1 month back up dhi vunte dhaanitho laaginchesaamu. @lokesh_rocks Quote Link to comment Share on other sites More sharing options...
Jombo0108 Posted June 6, 2020 Report Share Posted June 6, 2020 43 minutes ago, superhit3 said: full time bhayya...that too mana DB lo kondaru hate chese QA nenu. dabbulu yentho ivvaru...anthe yentho lontha matrame......kani On-Call rotation doing. PCF clloud....instances...scaling...bouncing ...memory...grafana...cpu utiliztion... appd dashboards anni chusi...DB connections yenni vunnay yenta yekkuvaku ika scale up cheyyachau yenta pcf space vundi....app logs...Grafana...AppD.... naa bonda....testing ku no link. Same day they patched DB and OS...nodes nd services configuration lo gadabida chesaru...network path s issue...so, we r getting hit millions of evetns backlog Financial domain ahh bro nuvvu work chesedi .. ma daggara kuda same protocol follow chestaru Quote Link to comment Share on other sites More sharing options...
JaiTDP Posted June 6, 2020 Report Share Posted June 6, 2020 4 minutes ago, Jombo0108 said: @lokesh_rocks https://gfycat.com/candidfirsthectorsdolphin 1 Quote Link to comment Share on other sites More sharing options...
superhit3 Posted June 6, 2020 Author Report Share Posted June 6, 2020 6 minutes ago, Unixer said: You can always UNDO patches Kno? May take some time but might eliminate if patch related.. problem is not with Patches this afternoon -> evening -> night...they messed up with network paths resulted into huge backligs at queues...then issu with d onnections to process them Quote Link to comment Share on other sites More sharing options...
superhit3 Posted June 6, 2020 Author Report Share Posted June 6, 2020 11 minutes ago, JaiTDP said: Haha.. luckily it’s backend data loading server bro not the UI. The clients didn’t have visibility abt this but we notified saying there will be some delay in processing data. total server crashed , Microsoft vaadu kuda hands up annaadu after reviewing logs. Thella manager(chaala manchodu) pushing me on UAT stuff saying client was waiting where I am fighting with this prod server !! Oka 1 month back up dhi vunte dhaanitho laaginchesaamu. + time sensitive data kademo ayithe Quote Link to comment Share on other sites More sharing options...
superhit3 Posted June 6, 2020 Author Report Share Posted June 6, 2020 15 minutes ago, cosmopolitan said: All rounder lekka unav ga bro chesaru, thanks to them. But stress is high bro Quote Link to comment Share on other sites More sharing options...
JaiTDP Posted June 6, 2020 Report Share Posted June 6, 2020 1 minute ago, superhit3 said: + time sensitive data kademo ayithe ya kind of. 1 Quote Link to comment Share on other sites More sharing options...
cosmopolitan Posted June 6, 2020 Report Share Posted June 6, 2020 1 minute ago, superhit3 said: chesaru, thanks to them. But stress is high bro yeah.. weekend support full torture untadhi.. kani ee knowledge future lo help avuthundhi .. Quote Link to comment Share on other sites More sharing options...
superhit3 Posted June 6, 2020 Author Report Share Posted June 6, 2020 2 minutes ago, JaiTDP said: ya kind of. gotcha, u 1 lucky fellow Quote Link to comment Share on other sites More sharing options...
superhit3 Posted June 6, 2020 Author Report Share Posted June 6, 2020 21 minutes ago, cosmopolitan said: yeah.. weekend support full torture untadhi.. kani ee knowledge future lo help avuthundhi .. yemo, QA antente QA skills yemunaya ani amtrame looking....ivanni chesam ante aa*t care antaremoga? Quote Link to comment Share on other sites More sharing options...
superhit3 Posted June 6, 2020 Author Report Share Posted June 6, 2020 33 minutes ago, Jombo0108 said: Financial domain ahh bro nuvvu work chesedi .. ma daggara kuda same protocol follow chestaru domain yendukule bhayya mana vallu domain septe alane anni dig chestaru. but, very time sensitive and live 1 Quote Link to comment Share on other sites More sharing options...
kathanayaka Posted June 6, 2020 Report Share Posted June 6, 2020 2 hours ago, superhit3 said: full time bhayya...that too mana DB lo kondaru hate chese QA nenu. dabbulu yentho ivvaru...anthe yentho lontha matrame......kani On-Call rotation doing. PCF clloud....instances...scaling...bouncing ...memory...grafana...cpu utiliztion... appd dashboards anni chusi...DB connections yenni vunnay yenta yekkuvaku ika scale up cheyyachau yenta pcf space vundi....app logs...Grafana...AppD.... naa bonda....testing ku no link. Same day they patched DB and OS...nodes nd services configuration lo gadabida chesaru...network path s issue...so, we r getting hit millions of evetns backlog orinee U tagalayya docker kubernetes ECS EKS Serverless auto scaling inni tagaladdai kadara. inka "night 2 ki server crash ayyinid" story lu cheptav endira kuyya Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.