express Posted July 19, 2012 Report Posted July 19, 2012 [color=#000000][font=arial]I have a scenario like this. users will dump .csv files into a folder daily basis. I need to load them into Oracle table. There is no limit of source files daily. one day they might give 10 files, some other day 12 files .so [/font][/color][url="http://on.so/"]on..........so[/url][color=#000000][font=arial] there is no fixed number of files daily. [/font][/color] [color=#000000][font=arial]I need to create a mapping where I need to extract these files and load into Oracle DB.[/font][/color] [color=#000000][font=arial]As we know, I can use Indirect method to load into table. means I can create a list file and I will create names of files into that list file. Once these files loaded into Table I need to delete from the source folder. So, in each run I will have new files ready to load. [/font][/color] [color=#000000][font=arial]We use only Perl scrips ...so please give me an idea that in each run I need to have only newly added files ( according to my requirement I need to delete previous files in folder) in list file so that I can use indirect method to load into table. please assist me.[/font][/color] [color=#000000][font=arial]My idea is like, I need a perl script to create list file in each run which has .csv files in it? after successful load into table I need another script to delete those files in source file directory.[/font][/color] [color=#000000][font=arial]Correct me if I am wrong.[/font][/color] [color=#000000][font=arial]Thank you amigos....!!![/font][/color]
express Posted July 19, 2012 Author Report Posted July 19, 2012 [quote name='Terminator II' timestamp='1342673353' post='1302159682'] perl script ante no idea [/quote] No Prob amigo....will see...!!!
vokatonumberkurrodu Posted July 19, 2012 Report Posted July 19, 2012 [quote name='express' timestamp='1342673266' post='1302159673'] [color=#000000][font=arial]I have a scenario like this. users will dump .csv files into a folder daily basis. I need to load them into Oracle table. There is no limit of source files daily. one day they might give 10 files, some other day 12 files .so [/font][/color][url="http://on.so/"]on..........so[/url][color=#000000][font=arial] there is no fixed number of files daily. [/font][/color] [color=#000000][font=arial]I need to create a mapping where I need to extract these files and load into Oracle DB.[/font][/color] [color=#000000][font=arial]As we know, I can use Indirect method to load into table. means I can create a list file and I will create names of files into that list file. Once these files loaded into Table I need to delete from the source folder. So, in each run I will have new files ready to load. [/font][/color] [color=#000000][font=arial]We use only Perl scrips ...so please give me an idea that in each run I need to have only newly added files ( according to my requirement I need to delete previous files in folder) in list file so that I can use indirect method to load into table. please assist me.[/font][/color] [color=#000000][font=arial]My idea is like, I need a perl script to create list file in each run which has .csv files in it? after successful load into table I need another script to delete those files in source file directory.[/font][/color] [color=#000000][font=arial]Correct me if I am wrong.[/font][/color] [color=#000000][font=arial]Thank you amigos....!!![/font][/color] [/quote] No idea about perl scripting.... But the standard process is to archive the files into a different folder at the end of a successful run. Doing so will make sure the input folder is always empty when it recieved files for a day. Your logic is correct. if you ensure that the input folder doesnt have any files before start of run..
ganesh Posted July 19, 2012 Report Posted July 19, 2012 nuvvu implement cheyyali anukunnadi correste...post session command lo script ni ayina call cheyyu....ledante medeggara emanna scheduler unte dani dwara ayina script ni call cheyyu...
ganesh Posted July 19, 2012 Report Posted July 19, 2012 [quote name='vokatonumberkurrodu' timestamp='1342707160' post='1302160683'] No idea about perl scripting.... But the standard process is to archive the files into a different folder at the end of a successful run. Doing so will make sure the input folder is always empty when it recieved files for a day. Your logic is correct. if you ensure that the input folder doesnt have any files before start of run.. [/quote]aa files store sesukovali anukunte archive cheyyali kani adi avasaramledu anukunte archive deniki?
vokatonumberkurrodu Posted July 19, 2012 Report Posted July 19, 2012 [quote name='ganesh' timestamp='1342707396' post='1302160697'] aa files store sesukovali anukunte archive cheyyali kani adi avasaramledu anukunte archive deniki? [/quote] production lo emaina issue vachi data antha pothe...future lo files anni reload ki paniki vastai kada.... also production support ki avasaram avutundhi...if they want to go back and check what the source data was.
executor Posted July 19, 2012 Report Posted July 19, 2012 load complete avagane move command to vere folder ki move cheyochu kadaaa... new files vachinapudu ave matrame read chestadi appuudu
ganesh Posted July 19, 2012 Report Posted July 19, 2012 [quote name='vokatonumberkurrodu' timestamp='1342707545' post='1302160705'] production lo emaina issue vachi data antha pothe...future lo files anni reload ki paniki vastai kada.... also production support ki avasaram avutundhi...if they want to go back and check what the source data was. [/quote]but valla requirement entoo teliyadu kadaa....i guess files ni remove seyyalanna requirement lekapote manodu adigevadu kademoo
vokatonumberkurrodu Posted July 19, 2012 Report Posted July 19, 2012 [quote name='ganesh' timestamp='1342707656' post='1302160710'] but valla requirement entoo teliyadu kadaa....i guess files ni remove seyyalanna requirement lekapote manodu adigevadu kademoo [/quote] business requirements are always vague....archiving is not always a business requirement but a technical requirement so that they may help us during restartability, recoverability and support. good to have. not mandatory though.
yankees Posted July 19, 2012 Report Posted July 19, 2012 [quote name='express' timestamp='1342673266' post='1302159673'] [color=#000000][font=arial]I have a scenario like this. users will dump .csv files into a folder daily basis. I need to load them into Oracle table. There is no limit of source files daily. one day they might give 10 files, some other day 12 files .so [/font][/color][url="http://on.so/"]on..........so[/url][color=#000000][font=arial] there is no fixed number of files daily. [/font][/color] [color=#000000][font=arial]I need to create a mapping where I need to extract these files and load into Oracle DB.[/font][/color] [color=#000000][font=arial]As we know, I can use Indirect method to load into table. means I can create a list file and I will create names of files into that list file. Once these files loaded into Table I need to delete from the source folder. So, in each run I will have new files ready to load. [/font][/color] [color=#000000][font=arial]We use only Perl scrips ...so please give me an idea that in each run I need to have only newly added files ( according to my requirement I need to delete previous files in folder) in list file so that I can use indirect method to load into table. please assist me.[/font][/color] [color=#000000][font=arial]My idea is like, I need a perl script to create list file in each run which has .csv files in it? after successful load into table I need another script to delete those files in source file directory.[/font][/color] [color=#000000][font=arial]Correct me if I am wrong.[/font][/color] [color=#000000][font=arial]Thank you amigos....!!![/font][/color] [/quote] Amigo.................Nenu chesanu ee scenario...............Neeku kavali ante nenu real time lo chopistanu..............
yankees Posted July 19, 2012 Report Posted July 19, 2012 [quote name='express' timestamp='1342673266' post='1302159673'] [color=#000000][font=arial]I have a scenario like this. users will dump .csv files into a folder daily basis. I need to load them into Oracle table. There is no limit of source files daily. one day they might give 10 files, some other day 12 files .so [/font][/color][url="http://on.so/"]on..........so[/url][color=#000000][font=arial] there is no fixed number of files daily. [/font][/color] [color=#000000][font=arial]I need to create a mapping where I need to extract these files and load into Oracle DB.[/font][/color] [color=#000000][font=arial]As we know, I can use Indirect method to load into table. means I can create a list file and I will create names of files into that list file. Once these files loaded into Table I need to delete from the source folder. So, in each run I will have new files ready to load. [/font][/color] [color=#000000][font=arial]We use only Perl scrips ...so please give me an idea that in each run I need to have only newly added files ( according to my requirement I need to delete previous files in folder) in list file so that I can use indirect method to load into table. please assist me.[/font][/color] [color=#000000][font=arial]My idea is like, I need a perl script to create list file in each run which has .csv files in it? after successful load into table I need another script to delete those files in source file directory.[/font][/color] [color=#000000][font=arial]Correct me if I am wrong.[/font][/color] [color=#000000][font=arial]Thank you amigos....!!![/font][/color] [/quote] wat i will do is to first i will delete the previous files in the folder and then archive the present files. i will keep the files the present files until the next run happens so that we have the archive files and the present files. In case tomorrow any files doesn't came the workflow should be set up such that it succeeds by emailing the user the files are not present............
express Posted July 19, 2012 Author Report Posted July 19, 2012 [quote name='executor' timestamp='1342707584' post='1302160707'] load complete avagane move command to vere folder ki move cheyochu kadaaa... new files vachinapudu ave matrame read chestadi appuudu [/quote] Baa...mari each run lo naaku listfile kavali kada....before run start , aa folder lo vunna files anni list file lo vundali...so that I can use indirect method....any idea?
.thumb.jpg.ce42bb16ec102475a78c2087ce269c7d.jpg)
.thumb.gif.343fe49c7e1c507ae0056326bd8cf77e.gif)
Recommended Posts