CRAB for CMS-HI Dilepton group

Prerequisites

• DongHo summarize the prerequisites, please visit here.

Preparation

• crab.cfg file in CVS : http://cmssw.cvs.cern.ch/cgi-bin/cmssw.cgi/UserCode/CmsHi/Z0Ana/JulyExercise10/skimscripts/
  • for castor : crab_castor.cfg
  • for MIT : crab_MIT.cfg
• pset file - python config file
  • for skim0 :

Real CRAB

• Step 1 : Store data at castor (not publishing on DBS)

(Notice : If you access the data in castor by crab, please read this.)

• Step 2 : Store data at MIT and publish on DBS

Tips for effective work

• If you want to store data at castor, before submit CRAB you should do following first. (ex: stored directory : /castor/cern.ch/user/h/hckim/JulyExercise10/JulyExercise10_HardEnriched_Dilepton_skim0)
  • rfrm -r /castor/cern.ch/user/h/hckim/JulyExercise10/JulyExercise10_HardEnriched_Dilepton_skim0
  • rfmkdir /castor/cern.ch/user/h/hckim/JulyExercise10/JulyExercise10_HardEnriched_Dilepton_skim0
  • rfchmod 775 /castor/cern.ch/user/h/hckim/JulyExercise10/JulyExercise10_HardEnriched_Dilepton_skim0

(If the directory doesn't have the right to write by group, you will face to exit code 60307.)

• At MIT, maximum capacity to process job is about 2000, so it will be good to reduce the number of job per task. In my experience, less than 200 is OK.

• Because of unknown reason, sometimes crab submit can be failed with exit code 60307. That means the folder to store data doesn't have permit to write by group. At that time, please remove that folder, kill that job and create and submit new job. (Thay may depend on which machine do a job..I guess.)
• If you face to exit code 8018, do following list. (ex : trouble job : 13, 119)
  • crab -get(or -getoutput) 13, 119
  • crab -resubmit 13, 119

• When dataset which you accessed with CRAB is extended(that is, addes more event to same dataset), you need to do like following. * crab -extend : That created extended CRAB job added to already created jobs * crab -submit #extended job number : From this step, same as usual jobs

Check whether all the data is stored and published successfully

• Check the storage at castor
• Check the storage at MIT
  • store at /net/pstore01/d00/scratch/hckim/PrimaryDataset/User_Dataset_Name/PsetHash/
    • ex. /pnfs/cmsaf.mit.edu/t2bat/cms/store/user/hckim/MinimumBiasHI/JulyExercise10_MinimumBiasHI_dilepton_skim0/bde1f0b06d4fc6b6d44b106f0c9b396a
• Check the publishment on DBS

• (NEW) If you want submit multiple root file, add following in crab.cfg
  • remark output_files = test1.root
  • add get_edm_output = 1

Useful link

• Related with job monitoring
  • CMS Dashboard Task monitoring : http://dashb-cms-sam.cern.ch/dashboard/request.py/taskmonitoring
  • CMS MIT T2 Monitoring : http://www.cmsaf.mit.edu/condor4web/

• Related with general CRAB information
  • CMS Remote Analysis Builder - CRAB : https://twiki.cern.ch/twiki/bin/view/CMS/SWGuideCrab
  • CRAB Frequently asked questions : https://twiki.cern.ch/twiki/bin/view/CMS/SWGuideCrabFaq
  • CRAB publication : https://twiki.cern.ch/twiki/bin/view/CMS/SWGuideCrabForPublication
  • Error codes currently sent from CMS jobs to the dashboard via ML : https://twiki.cern.ch/twiki/bin/view/CMS/JobExitCodes
  • Detailed list of URLs and allowed VOMS Roles : https://twiki.cern.ch/twiki/bin/view/CMS/DBSInstanceAccessList#Detailed_list_of_URLs_and_allowe
  • Workbook CRAB tutorial : https://twiki.cern.ch/twiki/bin/view/CMS/WorkBookCRABTutorial
  • CMS Grid tutorial (15 April 2010) : http://indico.cern.ch/conferenceDisplay.py?confId=90089

-- HyunChulKim - 07 Jul 2010