@@@@calling All Datastage Developers@@@@

[qwerty1204]

ee logic ni parallel job lo ela implement cheyali baa

 

I am writing to a fixed width sequential file, lets say of size 3 bytes and below is the sample data in the file

 

abc

pqr

mnc

xyz

4 

 

last line of the file lo aa file lo enni records unanyi anedi rayali in this example 4

 

and this has to be implemented with in the job scope (not using any command activity stage or after job routine)

 

 

 

[qwerty1204]

LTT

[Silver_mani]

ee logic ni parallel job lo ela implement cheyali baa

 

I am writing to a fixed width sequential file, lets say of size 3 bytes and below is the sample data in the file

 

abc

pqr

mnc

xyz

4 

 

last line of the file lo aa file lo enni records unanyi anedi rayali in this example 4

 

and this has to be implemented with in the job scope (not using any command activity stage or after job routine)

its not that straight forward ... You have to create Routine .. Here is the Routine

 

$INCLUDE DSINCLUDE JOBCONTROL.H

      err = ''
      RtnSuccess = 0
      RtnFailure = 1
      RtnError = -1
      RtnStatus = RtnError
      Call DSLogInfo(" JOBNAME ":JOBNAME, ProgramName)
      Call DSLogInfo("STAGELINKS ":STAGELINKS, ProgramName)
      Call DSLogInfo(" JOBRUNID":JOBRUNID, ProgramName)
      ProgramName = "TPUBNPPXXR000GetLinkCnt"
      myJOB = Trim(JOBNAME):".":Trim(JOBRUNID)

      Call DSLogInfo(" JobName: ":myJOB, ProgramName)
      counter = DCount(STAGELINKS,",")
      For i = 1 to counter

         LinkToMonitor = Trim(Field(Field(STAGELINKS,",",i),":",2))
         StageToMonitor= Trim(Field(Field(STAGELINKS,",",i),":",1))
         JobHandle = DSAttachJob (myJOB, DSJ.ERRWARN )
         result=DSGetLinkInfo( JobHandle,StageToMonitor,LinkToMonitor,DSJ.LINKROWCOUNT)
         print "ForLoop, i : ":i

         print "LinkName  ":LinkToMonitor
         print "StageName  ":StageToMonitor
         print " Link Info for ":LinkToMonitor:"is   " :result

         Call DSLogInfo("LinkName  ":LinkToMonitor, ProgramName)
         Call DSLogInfo("StageName  ":StageToMonitor , ProgramName)
         Call DSLogInfo(" Link Info for ":LinkToMonitor:"is   " :result , ProgramName)
         *Check result for error or value

         Begin Case
            Case result = DSJE.BADHANDLE
               err = ' *JobHandle was invalid. Supplied job name: ':JOBNAME
               RtnStatus = RtnError
            Case result =DSJE.BADTYPE
               err = ' * InfoType was unrecognized.'
               RtnStatus = RtnError
            Case result =DSJE.BADSTAGE
               err = ' *StageName (':StageToMonitor:') does not refer to a known stage in the job (':JOBNAME:').'
               RtnStatus = RtnError
            Case result =DSJE.NOTINSTAGE
               err = ' *StageName was DSJ.ME and the caller is not running within a stage.'
               RtnStatus = RtnError
            Case result =DSJE.BADLINK
               err = ' *LinkName (':LinkToMonitor:') does not refer to a known link for the stage in question (':StageToMonitor:') for the job in question(':JOBNAME:').'
               RtnStatus = RtnError
            Case result = 0
               RtnStatus = RtnFailure
            Case result > 0
               *for debugging only!*
               PRINT "Link count for ":JOBNAME:":":StageToMonitor:":":LinkToMonitor:"=":result
               RtnStatus = RtnSuccess
            Case @TRUE
               err = ' *Unexpected result from DSGetLinkInfo! Result=':result
               RtnStatus = RtnError
         End Case
         print "EndCase, RtnStatus :":RtnStatus
         print "         RtnSuccess:":RtnSuccess
         If RtnStatus <> RtnSuccess Then
            Goto ExitRoutine
         End

      Next i

 

ExitRoutine:

      If RtnStatus = RtnError Then
         *for debugging only!*
         PRINT err
         Call DSLogWarn(err,ProgramName )
         Call DSLogInfo("Routine returned error, aborting", ProgramName)
         *Abort
      End

      Ans = RtnStatus
      if Ans = 0 then
         *  Msg_Success = "The Sequencer ran Successfully"
         *print "The Routine Status Success:   ": Ans
         Call DSLogInfo("The Routine Status : ": Ans , ProgramName)

      end

      if Ans <> 0 then
         *Msg_Failure = "The Sequencer has the ":"Stage:** ":StageToMonitor: " ** and the Link :** ": LinkToMonitor :" ** has the RowCount :   " : result :"(Zero)"
         * print "The Routine Status Failed  ": Msg_Failure
         Call DSLogWarn("The Routine Status :  ": Ans , ProgramName)

 

      end

[Silver_mani]

deeniki 3 arguements pass cheyali
JOBNAME  : The job to check the link counts

STAGELINKS :Comma delimted list of stage and links, where the stage and link are separated by a colon.

JOBRUNID :

 

[McDowellMurthy]

ee logic ni parallel job lo ela implement cheyali baa

I am writing to a fixed width sequential file, lets say of size 3 bytes and below is the sample data in the file

abc
pqr
mnc
xyz
4

last line of the file lo aa file lo enni records unanyi anedi rayali in this example 4

and this has to be implemented with in the job scope (not using any command activity stage or after job routine)

Transformer lo function untadi kada record count check cheyadaniki..not sure why we have to write a complex routine for this..please throw some light..

[qwerty1204]

deeniki 3 arguements pass cheyali
JOBNAME  : The job to check the link counts

STAGELINKS :Comma delimted list of stage and links, where the stage and link are separated by a colon.

JOBRUNID :

 

intha fedda routine rayali antava baa :3D_Smiles:

[qwerty1204]

Transformer lo function untadi kada record count check cheyadaniki..not sure why we have to write a complex routine for this..please throw some light..

 

@INROWNUM ane transformer function untadi kani..naaku job execuation lo ne last lo ila no.of record sin the file ni rayali baa as a lst row in that file.

[mettastar]

Transformer Sequential lo run chesi @outrownum generate cheyi.. Split it to two O/P links one with all the data fields and other with OutRowNum.

Take that Outrownum to Aggregator and get the max or Sort OutRowNum in Desc order and get the first record

Then funnel this record to the original flow in sequence so that this record will be inserted at the end.. 

[qwerty1204]

Transformer Sequential lo run chesi @outrownum generate cheyi..

Take that Outrownum to Aggregator and get the max or Sort OutRowNum in Desc order and get the first record

Then funnel this record to the original flow in sequence so that this record will be inserted at the end.. 

 

nice baa... kakpothe oka doubt..... aggregator lo group by column emani ivali? naaku common value unde column emi ledu across the rows

[mettastar]

nice baa... kakpothe oka doubt..... aggregator lo group by column emani ivali? naaku common value unde column emi ledu across the rows

create a dummy column and hard code it to '1' or something.. 

[qwerty1204]

create a dummy column and hard code it to '1' or something.. 

 

okk..approach konchem lenghty ayna job level lo acheive cheyali ante edhee best laga undi..thanx baa

[Silver_mani]

Transformer Sequential lo run chesi @outrownum generate cheyi.. Split it to two O/P links one with all the data fields and other with OutRowNum.

Take that Outrownum to Aggregator and get the max or Sort OutRowNum in Desc order and get the first record

Then funnel this record to the original flow in sequence so that this record will be inserted at the end.. 

 generally to count the no of rows, OutRownum suggest cheyaru ...what if there are constraints on the transformer ? it will throw of  ur source  row count ...anduke routine use chestaru ...reusable and applicable to all scenarios

[Silver_mani]

baa , mettastar cheppina design work avutundi ... only if ur writing to a database as an update ...what if your target is not a database....  funnel cheyalevu gaa ... :lol:  

[mettastar]

 generally to count the no of rows, OutRownum suggest cheyaru ...what if there are constraints on the transformer ? it will throw of  ur source  row count ...anduke routine use chestaru ...reusable and applicable to all scenarios

 

Baa idhi @OUTROWNUM -- outrownum is generated after constraints in the transformer derivation... nuvvu cheppedi @INROWNUM... @OUTROWNUM shud work..

 

baa , mettastar cheppina design work avutundi ... only if ur writing to a database as an update ...what if your target is not a database....  funnel cheyalevu gaa ... :lol:  

 

Funnel lo three types untayi.. Sequence.. Sorted and some other type.. Sequence funnel chesthe based on your link order it will write the records to output.. so ikkada mainstream link first then count link second unte.. Count last lo vasthundhi.. Funnel o/p ni direct Seq File stage ki ivvaatame..

[mettastar]

inkoka way kuda untadhi.. if you want to explore.. external target stage.. use that as o/p stage instead of sequential file stage.. deentlo nuvvu unix commands kuda execute cheyachu... na paatha proj lo chesaa.. ipudu no more Datastage...