WMS test report

Summary

  • Component: WMS 3.3.4

  • ETICS Subsystem Configuration Name: emi-wms_R_3_3_6_3 emi-jdl_R_3_2_6_1

  • VCS Tag: emi-ice_R_3_3_4_1, emi-wms-manager_R_3_3_1_4, emi-jobsubmission_R_3_3_2_1, emi-wmproxy_R_3_3_4_2, emi-wms-brokerinfo_R_3_3_2_1, emi-wms-configuration_R_3_3_2_1, emi-wms-ism_R_3_3_2_1, emi-wms-matchmaking_R_3_3_2_1, emi-wms-purger_R_3_3_2_1, emi-yaim-wms_R_4_1_4_3, emi-jdl-api-cpp_R_3_2_6_1, emi-wms-nagios_R_1_0_0_0, emi-wms-mp_R_1_0_2_0

  • EMI Major Release: EMI 1 (Kebnekaise)

  • Platform: sl5 epel

  • Certification report: not yet available

  • Outcome: Certification under way

Deployment Tests

Clean installation

  • Starting from a SL5 x86_64 kernel machine:

[root@emi-demo11 ~]# cat /etc/redhat-release 
Scientific Linux SL release 5.7 (Boron)
[root@emi-demo11 ~]# uname -a
Linux emi-demo11.cnaf.infn.it 2.6.18-274.12.1.el5 #1 SMP Tue Nov 29 11:14:39 EST 2011 x86_64 x86_64 x86_64 GNU/Linux

  • Import EMI rpm gpg key
[root@emi-demo11 ~]# wget http://emisoft.web.cern.ch/emisoft/dist/EMI/1/RPM-GPG-KEY-emi -O /tmp/emi-key_gd.asc
[root@emi-demo11 ~]# rpm --import /tmp/emi-key_gd.asc

Update

[root@emi-demo12 yum.repos.d]# rpm -qa|grep wms-manager
glite-wms-manager-3.3.1-3.sl5

Proxy mode

The same update procedure has been applied on a wms configured in proxy mode. emi-demo12 being the wms and emi-demo11 the LB server. In order to configure emi-demo12 in proxy mode the following lines have been defined in the site-info.def file:
# Hostname of the WMS
WMS_HOST=emi-demo12.$MY_DOMAIN
LB_HOST=emi-demo11.$MY_DOMAIN

and the following lines in siteinfo/services/glite-wms :

LB_HOST="emi-demo11.cnaf.infn.it:9000"
#GLITE_LB_TYPE=both
GLITE_LB_TYPE=proxy

On emi-demo11 the following lines have been defined in the site-info.def file:

GLITE_LB_WMS_DN="/C=IT/O=INFN/OU=Host/L=CNAF/CN=emi-demo12.cnaf.infn.it"
GLITE_LB_AUTHZ_REGISTER_JOBS=".*"

Unit Tests

N/A: Tests will be available with EMI-2.

Basic Functionality tests

Submission

Where possible the tests should use internal testbed CEs (located in Padova). This is accomplished by adding the following line to wms-command.conf:
DEFAULTREQ= other.GlueCEStateStatus == "Testing"
This is not possible, for example, for parallel jobs.

Normal

[capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 1 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 1: Submit a normal job (3 cases LCG-CE, CREAM, general) 


 Test 1A: Submit to an LCG-CE 


 Test 1B: Submit to a CREAM CE 


 Test 1C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 17:09:32
===> Ended  : 17:21:55
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111228170932.log

[capannini@devel15 WMS-service]$ voms-proxy-init --voms dteam:/dteam/NGI_IT:operator
Enter GRID pass phrase:
Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini
Creating temporary proxy .................................................................. Done
Contacting  voms2.hellasgrid.gr:15004 [/C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr] "dteam" Done
Creating proxy ........................................................................... Done

Your proxy is valid until Thu Dec 29 23:31:47 2011
[capannini@devel15 WMS-service]$ voms-proxy-info --all
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy
issuer    : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini
identity  : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini
type      : proxy
strength  : 1024 bits
path      : /tmp/x509up_u504
timeleft  : 11:59:51
key usage : Digital Signature, Key Encipherment, Data Encipherment
=== VO dteam extension information ===
VO        : dteam
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini
issuer    : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr
attribute : /dteam/NGI_IT/Role=operator/Capability=NULL
attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL
attribute : /dteam/Role=NULL/Capability=NULL
timeleft  : 11:59:51
uri       : voms2.hellasgrid.gr:15004
[capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 1 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 1: Submit a normal job (3 cases LCG-CE, CREAM, general) 


 Test 1A: Submit to an LCG-CE 


 Test 1B: Submit to a CREAM CE 


 Test 1C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 11:32:22
===> Ended  : 11:40:10
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111229113222.log

[capannini@devel15 WMS-service]$ voms-proxy-init --voms testers.eu-emi.eu
Enter GRID pass phrase:
Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini
Creating temporary proxy ................................... Done
Contacting  emitestbed07.cnaf.infn.it:15002 [/C=IT/O=INFN/OU=Host/L=CNAF/CN=emitestbed07.cnaf.infn.it] "testers.eu-emi.eu" Done
Creating proxy ...................................................... Done

[capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -t 1 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 1: Submit a normal job (3 cases LCG-CE, CREAM, general) 


 Test 1A: Submit to an LCG-CE 


 Test 1B: Submit to a CREAM CE 


 Test 1C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 11:45:01
===> Ended  : 11:52:20
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111229114501.log

[fabiok@ui WMS-service]$ voms-proxy-init --voms infngrid
Enter GRID pass phrase:
Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini
Creating temporary proxy ..................................................... Done
Contacting  voms.cnaf.infn.it:15000 [/C=IT/O=INFN/OU=Host/L=CNAF/CN=voms.cnaf.infn.it] "infngrid" Done
Creating proxy .................................................... Done
Your proxy is valid until Thu Dec 29 23:45:58 2011
[fabiok@ui WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V infngrid -d 3 -l -t 1 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 1: Submit a normal job (3 cases LCG-CE, CREAM, general) 


 Test 1A: Submit to an LCG-CE 


 Test 1B: Submit to a CREAM CE 


 Test 1C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 11:46:25
===> Ended  : 11:52:40
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111229114625.log
See submit.log.tgz: Normal submission logs

Proxy mode
There is a problem with the configuration of emi-demo12 (wms in proxy mode) pointing to emi-demo11 (wms in both mode): the events Running,ReallyRunning and Done Ok are not logged by emi-demo11 (and don't appear in the logging-info), even if they are present in the lbserver20 db of emi-demo12. These events are instead correctly logged if emi-demo11 is configured as LB in server mode. Thus, in order to properly test a wms in proxy mode, then it should point to a LB server not in both mode.

ARC CE
Job submitted on a production CE using the cms VO.
[capannini@cert-17 regression_tests]$ lcg-infosites --is egee-bdii.cnaf.infn.it --vo cms ce |grep arc
     80         0           95         5        95   arc.univ.kiev.ua:2811/nordugrid-torque-arc
    768       707          196        61       196   jade-cms.hip.fi:2811/nordugrid-GE-arc
    256         0            0         0         0   korundi.grid.helsinki.fi:2811/nordugrid-GE-arc

[cristofori@ui ~]$ glite-wms-job-submit -e https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server -a -r jade-cms.hip.fi:2811/nordugrid-GE-arc firsttest.jdl

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://emi-demo11.cnaf.infn.it:9000/e4f1VtGXy_I_2MS2dVUs4g

==========================================================================




[cristofori@ui ~]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/e4f1VtGXy_I_2MS2dVUs4g


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/e4f1VtGXy_I_2MS2dVUs4g
Current Status:     Done (Success)
Exit code:          0
Status Reason:      Job terminated successfully
Destination:        jade-cms.hip.fi:2811/nordugrid-GE-arc
Submitted:          Tue Mar 27 14:27:13 2012 CEST
==========================================================================




[cristofori@ui ~]$ glite-wms-job-output --dir ./ https://emi-demo11.cnaf.infn.it:9000/e4f1VtGXy_I_2MS2dVUs4g

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server


================================================================================

                         JOB GET OUTPUT OUTCOME

Output sandbox files for the job:
https://emi-demo11.cnaf.infn.it:9000/e4f1VtGXy_I_2MS2dVUs4g
have been successfully retrieved and stored in the directory:
/home/GUEST/cristofori/cristofori_e4f1VtGXy_I_2MS2dVUs4g

================================================================================


[cristofori@ui ~]$ cat /home/GUEST/cristofori/cristofori_e4f1VtGXy_I_2MS2dVUs4g/stderr.log
[cristofori@ui ~]$ cat /home/GUEST/cristofori/cristofori_e4f1VtGXy_I_2MS2dVUs4g/stdout.log
cmsuser07

Jdl with DataRequirements
In case of problems with lcg-ce, it i possible to create replicas of the file on other SEs close to different lcg-ces.
[capannini@devel15 WMS-service]$ lcg-rep --vo dteam -d srm://gridsrm.pi.infn.it:8444/dteam/fcapanni/e-2M.txt lfn:/grid/dteam/fcapanni/e-2M.txt
GSIFTP: source: set up FTP mode. DCAU disabled. Streams =  1, Tcp BS = 0
GSIFTP: dest: set up FTP mode. DCAU disabled. Streams =  1, Tcp BS = 0
Transfer completed succesfully
[capannini@devel15 WMS-service]$ lcg-rep --vo dteam -d srm://gridsrm.pi.infn.it:8444/dteam/fcapanni/PI_1M.txt lfn:/grid/dteam/fcapanni/PI_1M.txt
GSIFTP: source: set up FTP mode. DCAU disabled. Streams =  1, Tcp BS = 0
GSIFTP: dest: set up FTP mode. DCAU disabled. Streams =  1, Tcp BS = 0
Transfer completed succesfully

The ce/se couple can be found looking for "/jobmanager" in the output of the following command:
[capannini@devel15 WMS-service]$ lcg-infosites --vo dteam closeSE

The GlueVOInfoPath of the storage area to be used in the srm surl after the endpoint can be found with the following search:
ldapsearch -x -LLL -H  ldap://egee-bdii.cnaf.infn.it:2170 -b mds-vo-name=local,o=grid '(&(objectClass=GlueVOInfo)(GlueChunkKey=GlueSEUniqueID=gridsrm.pi.infn.it))' GlueVOInfoPath
(/dteam was picked up in this test)

the lcg-ce to be excluded can be specified in the testsuite configuration file (wms-command.conf) with the following attribute:
DEFAULTREQ=!regexp(".*ce130.*", other.GlueCEUniqueID) && !regexp(".*ce131.*", other.GlueCEUniqueID) && !regexp(".*ce133.*", other.GlueCEUniqueID)

In the following test the lcg-ce was the following (by inspecting the log):
Destination CE is ce132.cern.ch:2119/jobmanager-lcglsf-grid_2nh_dteam

[capannini@devel15 WMS-service]$ cat JDLs/data-req.jdl 
###########################################
#      JDL with Data Requirements         #
###########################################

Executable = "/bin/ls";
Arguments = "-la";
StdOutput = "std.out";
StdError = "std.err";
Prologue = "prologue.sh";

FuzzyRank = true;
Environment={"LFC_HOST=prod-lfc-shared-central.cern.ch", "LFC_HOME=prod-lfc-shared-central.cern.ch:/grid/dteam/aleph"};
InputSandbox = {"JDLs/prologue.sh"};
OutputSandbox = {"std.out", "std.err","prologue.out", "PI_1M.txt"};

DataRequirements = {
[
DataCatalogType = "DLI";
DataCatalog ="http://prod-lfc-shared-central.cern.ch:8085/"; 
InputData = { "lfn:/grid/dteam/fcapanni/PI_1M.txt" };
]
};

DataAccessProtocol = "gsiftp";
AllowZippedISB=true;
RetryCount = 1;
ShallowRetryCount = 2;

[capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 1 -i -j JDLs/data-req.jdl
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 1: Submit a normal job (3 cases LCG-CE, CREAM, general) 


 Test 1A: Submit to an LCG-CE 


 Test 1B: Submit to a CREAM CE 


 Test 1C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 12:24:32
===> Ended  : 12:36:28
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120110122432.log

Jdl with ISBBaseURI and OSBDestURI

Test on CREAM ce:

[capannini@devel15 certificazione]$ cat URI.jdl 
###########################################
#      JDL with ISBURI & OSBDestURI       #
###########################################

Executable = "exe.sh";
StdOutput = "std.out";
StdError = "std.err";

InputSandboxBaseURI = "gsiftp://devel11.cnaf.infn.it:2811/tmp";

FuzzyRank = true;
InputSandbox = {"exe.sh", "test1", "test2"};
OutputSandbox = {"std.out", "std.err"};
OutputSandboxDestURI = {"gsiftp://devel09.cnaf.infn.it:2811/tmp/myjobOutput", "std.err"};


#requirements = other.GlueCEStateStatus == "Testing" &&  regexp("/cream", other.GlueCEUniqueID);
requirements = regexp("/cream", other.GlueCEUniqueID);
DataAccessProtocol = "gsiftp";
RetryCount = 1;
ShallowRetryCount = 2;

[capannini@devel15 certificazione]$ cat tmp/exe.sh 
#!/bin/sh

NAME=`echo $GLITE_WMS_JOBID | sed -e 's/.*\///'`

echo $NAME 

ls -l

[capannini@devel15 certificazione]$ glite-wms-job-submit -a -c ~/wms_emi-demo11.conf URI.jdl 

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://emi-demo11.cnaf.infn.it:9000/NAwVzPxlPqGYz-Kf2xWLNg

==========================================================================

[capannini@devel15 certificazione]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/NAwVzPxlPqGYz-Kf2xWLNg


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/NAwVzPxlPqGYz-Kf2xWLNg
Current Status:     Done (Success)
Exit code:          0
Status Reason:      Job Terminated Successfully
Destination:        inaf-ce-01.ct.pi2s2.it:8443/cream-lsf-cert
Submitted:          Thu Dec 29 16:20:11 2011 CET
==========================================================================

[capannini@devel15 certificazione]$ glite-wms-job-output https://emi-demo11.cnaf.infn.it:9000/NAwVzPxlPqGYz-Kf2xWLNg

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server


================================================================================

         JOB GET OUTPUT OUTCOME

Output sandbox files for the job:
https://emi-demo11.cnaf.infn.it:9000/NAwVzPxlPqGYz-Kf2xWLNg
have been successfully retrieved and stored in the directory:
/tmp/jobOutput/capannini_NAwVzPxlPqGYz-Kf2xWLNg

================================================================================


[capannini@devel15 certificazione]$ ls /tmp/jobOutput/capannini_NAwVzPxlPqGYz-Kf2xWLNg
std.err

Log on devel09.cnaf.infn.it:

[root@devel09 ~]# cat /tmp/myjobOutput 
NAwVzPxlPqGYz-Kf2xWLNg
total 16
-rwxr-xr-x 1 dteam002 dteam 80 Dec 29 16:20 exe.sh
-rw-r--r-- 1 dteam002 dteam  0 Dec 29 16:20 std.err
-rw-r--r-- 1 dteam002 dteam 23 Dec 29 16:20 std.out
-rw-r--r-- 1 dteam002 dteam  6 Dec 29 16:20 test1
-rw-r--r-- 1 dteam002 dteam  6 Dec 29 16:20 test2

===>   test PASSED


Same process with an LCG CE:

[capannini@devel15 certificazione]$ glite-wms-job-submit -a -c ~/wms_emi-demo11.conf URI.jdl 

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://emi-demo11.cnaf.infn.it:9000/d4akLe8H2jynZpCPe5jGxA

==========================================================================

[capannini@devel15 certificazione]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/d4akLe8H2jynZpCPe5jGxA


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/d4akLe8H2jynZpCPe5jGxA
Current Status:     Done (Success)
Logged Reason(s):
    - Job got an error while in the CondorG queue.
    - Job terminated successfully
Exit code:          0
Status Reason:      Job terminated successfully
Destination:        gridce.sns.it:2119/jobmanager-lcgpbs-cert
Submitted:          Fri Dec 30 11:43:28 2011 CET
==========================================================================

[capannini@devel15 certificazione]$ glite-wms-job-output https://emi-demo11.cnaf.infn.it:9000/d4akLe8H2jynZpCPe5jGxA

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server


================================================================================

         JOB GET OUTPUT OUTCOME

Output sandbox files for the job:
https://emi-demo11.cnaf.infn.it:9000/d4akLe8H2jynZpCPe5jGxA
have been successfully retrieved and stored in the directory:
/tmp/jobOutput/capannini_d4akLe8H2jynZpCPe5jGxA

================================================================================


[capannini@devel15 certificazione]$ ls -l /tmp/jobOutput/capannini_d4akLe8H2jynZpCPe5jGxA
total 0
-rw-r--r-- 1 capannini emitestbed 0 Dec 30 13:08 std.err

Log on devel09.cnaf.infn.it:

[root@devel09 tmp]# ls -l myjobOutput 
-rw-r--r-- 1 dteam042 dteam 714 30 dic 13:01 myjobOutput
[root@devel09 tmp]# cat myjobOutput 
d4akLe8H2jynZpCPe5jGxA
total 36
-rwxr-xr-x 1 dteam003 dteam  80 Dec 30 13:00 exe.sh
-rw-r--r-- 1 dteam003 dteam 154 Dec 30 13:01 https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fd4akLe8H2jynZpCPe5jGxA.output
-rw-r--r-- 1 dteam003 dteam   0 Dec 30 13:01 std.err
-rw-r--r-- 1 dteam003 dteam  23 Dec 30 13:01 std.out
-rw-r--r-- 1 dteam003 dteam   6 Dec 30 13:00 test1
-rw-r--r-- 1 dteam003 dteam   6 Dec 30 13:00 test2
-rw------- 1 dteam003 dteam   2 Dec 30 13:01 tmp.GPdus14280
-rw------- 1 dteam003 dteam   2 Dec 30 13:01 tmp.OAref14000
-rw------- 1 dteam003 dteam 103 Dec 30 13:01 tmp.QtSZI14001
-rw------- 1 dteam003 dteam   0 Dec 30 13:01 tmp.vFipq14279
-rw------- 1 dteam003 dteam 103 Dec 30 13:01 tmp.YUTON14281

===>   test PASSED

Jdl with ISBBaseURI and OSBBaseDestURI

Submit to CREAM CE

[capannini@devel15 certificazione]$ cat BaseURI.jdl 
###########################################
#  JDL with ISBURI & OSBBaseDestURI       #
###########################################

Executable = "exe.sh";
StdOutput = "std.out";
StdError = "std.err";

InputSandboxBaseURI = "gsiftp://devel11.cnaf.infn.it:2811/tmp";

FuzzyRank = true;
InputSandbox = {"exe.sh", "test1", "test2"};
OutputSandbox = {"std.out", "std.err"};
OutputSandboxBaseDestURI="gsiftp://devel09.cnaf.infn.it:2811/tmp";

#requirements = other.GlueCEStateStatus == "Testing" &&  regexp("/cream", other.GlueCEUniqueID);
requirements = regexp("/cream", other.GlueCEUniqueID);

RetryCount = 1;
ShallowRetryCount = 2;

[capannini@devel15 certificazione]$ glite-wms-job-submit -a -c ~/wms_emi-demo11.conf BaseURI.jdl 

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://emi-demo11.cnaf.infn.it:9000/bRxuYfMUA79tvRXdrpgJdQ

==========================================================================

[capannini@devel15 certificazione]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/bRxuYfMUA79tvRXdrpgJdQ


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/bRxuYfMUA79tvRXdrpgJdQ
Current Status:     Done (Success)
Exit code:          0
Status Reason:      Job Terminated Successfully
Destination:        t2-ce-01.to.infn.it:8443/cream-pbs-short
Submitted:          Fri Dec 30 12:47:43 2011 CET
==========================================================================

[capannini@devel15 certificazione]$ glite-wms-job-output https://emi-demo11.cnaf.infn.it:9000/bRxuYfMUA79tvRXdrpgJdQ

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server


================================================================================

         JOB GET OUTPUT OUTCOME

No output files to be retrieved for the job:
https://emi-demo11.cnaf.infn.it:9000/bRxuYfMUA79tvRXdrpgJdQ

================================================================================

[capannini@devel15 certificazione]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/bRxuYfMUA79tvRXdrpgJdQ


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/bRxuYfMUA79tvRXdrpgJdQ
Current Status:     Cleared 
Status Reason:      user retrieved output sandbox
Destination:        t2-ce-01.to.infn.it:8443/cream-pbs-short
Submitted:          Fri Dec 30 12:47:43 2011 CET
==========================================================================

Log on devel09.cnaf.infn.it:

[root@devel09 tmp]# ls std*
std.err  std.out

[root@devel09 tmp]# cat std.out 
bRxuYfMUA79tvRXdrpgJdQ
total 16
-rwxr-xr-x 1 dteam029 dteam 80 Dec 30 12:48 exe.sh
-rw-r--r-- 1 dteam029 dteam  0 Dec 30 12:48 std.err
-rw-r--r-- 1 dteam029 dteam 23 Dec 30 12:48 std.out
-rw-r--r-- 1 dteam029 dteam  6 Dec 30 12:48 test1
-rw-r--r-- 1 dteam029 dteam  6 Dec 30 12:48 test2

===>     === test PASSED === 

Submit to an LCG CE:

[capannini@devel15 certificazione]$ glite-wms-job-submit -a -c ~/wms_emi-demo11.conf BaseURI.jdl 

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://emi-demo11.cnaf.infn.it:9000/pX2yRjupthmRSr8bqf5_Bg

==========================================================================


[capannini@devel15 certificazione]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/pX2yRjupthmRSr8bqf5_Bg


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/pX2yRjupthmRSr8bqf5_Bg
Current Status:     Done (Success)
Exit code:          0
Status Reason:      Job terminated successfully
Destination:        grid-ce.lns.infn.it:2119/jobmanager-lcgpbs-short
Submitted:          Fri Dec 30 13:18:38 2011 CET
==========================================================================

[capannini@devel15 certificazione]$ glite-wms-job-output https://emi-demo11.cnaf.infn.it:9000/pX2yRjupthmRSr8bqf5_Bg

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server


================================================================================

         JOB GET OUTPUT OUTCOME

No output files to be retrieved for the job:
https://emi-demo11.cnaf.infn.it:9000/pX2yRjupthmRSr8bqf5_Bg

================================================================================


[capannini@devel15 certificazione]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/pX2yRjupthmRSr8bqf5_Bg


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/pX2yRjupthmRSr8bqf5_Bg
Current Status:     Cleared 
Status Reason:      user retrieved output sandbox
Destination:        grid-ce.lns.infn.it:2119/jobmanager-lcgpbs-short
Submitted:          Fri Dec 30 13:18:38 2011 CET
==========================================================================

Log on devel09.cnaf.infn.it:

[root@devel09 tmp]# ls -l std*
-rw-r--r-- 1 dteam042 dteam   0 30 dic 13:19 std.err
-rw-r--r-- 1 dteam042 dteam 727 30 dic 13:19 std.out

[root@devel09 tmp]# cat std.out
pX2yRjupthmRSr8bqf5_Bg
total 36
-rwxr-xr-x  1 dteam050 dteam  80 Dec 30 13:19 exe.sh
-rw-r--r--  1 dteam050 dteam 118 Dec 30 13:19 https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fpX2yRjupthmRSr8bqf5_5fBg.output
-rw-r--r--  1 dteam050 dteam   0 Dec 30 13:19 std.err
-rw-r--r--  1 dteam050 dteam  23 Dec 30 13:19 std.out
-rw-r--r--  1 dteam050 dteam   6 Dec 30 13:19 test1
-rw-r--r--  1 dteam050 dteam   6 Dec 30 13:19 test2
-rw-------  1 dteam050 dteam 103 Dec 30 13:19 tmp.csGdY18153
-rw-------  1 dteam050 dteam   0 Dec 30 13:19 tmp.FBSue18151
-rw-------  1 dteam050 dteam   2 Dec 30 13:19 tmp.FZDLk18152
-rw-------  1 dteam050 dteam   2 Dec 30 13:19 tmp.HdslV18137
-rw-------  1 dteam050 dteam 103 Dec 30 13:19 tmp.hSlSL18138

Jdl with AllowZippedISB

[capannini@devel15 certificazione]$ cat AllowZIP.jdl 
###########################################
#   JDL with AllowZippedISB attribute     #
###########################################

Executable = "/bin/ls";
Arguments = "-la";
StdOutput = "std.out";
StdError = "std.err";

InputSandbox = {"fileA", "fileB", "fileC", "fileD", "fileE"};
OutputSandbox = {"std.out", "std.err"};

#requirements = other.GlueCEStateStatus == "Testing" &&  regexp("/cream", other.GlueCEUniqueID);
requirements = regexp("/cream", other.GlueCEUniqueID);

AllowZippedISB=true;

RetryCount = 1;
ShallowRetryCount = 2;

[capannini@devel15 certificazione]$ glite-wms-job-submit --debug -a -c ~/wms_emi-demo11.conf AllowZIP.jdl 
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - Vo read from proxy certificate extension
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - VirtualOrganisation value : dteam
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - Configuration file: /home/capannini/wms_emi-demo11.conf - VirtualOrganisation, RetryCount, rank, requirements attribute(s) no more supported outside JDL Default Attributes section "JdlDefaultAttributes"
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - Loaded user configuration file:
 /home/capannini/wms_emi-demo11.conf
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - Configuration file: //etc/glite-wms/dteam/glite_wmsui.conf - VirtualOrganisation, MyProxyServer attribute(s) no more supported outside JDL Default Attributes section "JdlDefaultAttributes"
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - Loaded Vo specific configuration file:
 //etc/glite-wms/dteam/glite_wmsui.conf
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - Loaded Configuration values: 
 [
  JdlDefaultAttributes = 
   [
    PerusalFileEnable = false; 
    ShallowRetryCount = 10; 
    SignificantAttributes = { "Requirements","Rank","FuzzyRank" }; 
    RetryCount = 3; 
    AllowZippedISB = true; 
    VirtualOrganisation = "dteam"; 
    rank =  -other.GlueCEStateEstimatedResponseTime; 
    MyProxyServer = "myproxy.cnaf.infn.it"; 
    requirements = other.GlueCEStateStatus == "Production"
   ]; 
  OutputStorage = "/tmp"; 
  ListenerStorage = "/tmp"; 
  ErrorStorage = "/var/tmp"; 
  WmProxyEndPoints = { "https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server" }
 ]
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - Getting Endpoint URL from configuration file 
-----------------------------------------

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server

-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - Calling the WMProxy getVersion service
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - WMProxy Version: 3.3.4 
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - Sending Proxy Request to https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - Calling the WMProxy delegationns__getProxyReq service
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - delegationns__getProxyReq -  The proxy has been successfully retrieved
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - Calling the WMProxy delegationns__putProxyReq service
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - The proxy has been successfully delegated with the identifier: oHGkZ5pxjZH7-LDxL2NIMw
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - Calling the WMProxy getTransferProtocols service
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - getTransferProtocols -  Available protocols: gsiftp, https
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - FileTransferProtocol not specified; using the default protocol: gsiftp
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - The JDL file is: /home/capannini/certificazione/AllowZIP.jdl
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - Retrieving the list of the local ISB files from the user JDL 
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - Total size of the ISB file(s) to be transferred to: 1126406
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - Max single job size of the ISB file(s) to be transferred to: 1024000
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - Checking the User-FreeQuota on the server 
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - Calling the WMProxy getFreeQuota service
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - getFreeQuota -  User freequota not set on the server
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - Getting the max ISB size from the server https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - Calling the WMProxy getMaxInputSandboxSize service
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - getMaxInputSandboxSize -  Max ISB size information successfully retrieved
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - The max job size (1024000 bytes) doesn't exceed the max size limit of 100000000 bytes: File transfer is allowed
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - File archiving and file compression allowed by user in the JDL
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - Registering JDL [ StdOutput = "std.out"; ShallowRetryCount = 2; SignificantAttributes = { "Requirements","Rank","FuzzyRank" }; RetryCount = 1; Executable = "/bin/ls"; Type = "job"; Arguments = "-la"; AllowZippedISB = true; VirtualOrganisation = "dteam"; JobType = "normal"; DefaultRank =  -other.GlueCEStateEstimatedResponseTime; ZippedISB = { "ISBfiles_y8ovpm9W5-W_kUqpeNiCKA_0.tar.gz" }; OutputSandbox = { "std.out","std.err" }; InputSandbox = { "file:///home/capannini/certificazione/fileA","file:///home/capannini/certificazione/fileB","file:///home/capannini/certificazione/fileC","file:///home/capannini/certificazione/fileD","file:///home/capannini/certificazione/fileE" }; StdError = "std.err"; rank =  -other.GlueCEStateEstimatedResponseTime; MyProxyServer = "myproxy.cnaf.infn.it"; requirements = ( regexp("/cream",other.GlueCEUniqueID) ) && ( other.GlueCEStateStatus == "Production" ) ]
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - Registering the job to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - Calling the WMProxy jobRegister service
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - The JobId is:  https://emi-demo11.cnaf.infn.it:9000/nGo2GsPtKD3urj66sBnHLA
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - Archiving the ISB files: /tmp/ISBfiles_y8ovpm9W5-W_kUqpeNiCKA_0.tar
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - This archive file has been successfully created: /tmp/ISBfiles_y8ovpm9W5-W_kUqpeNiCKA_0.tar
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - Compressing the file (.gz): /tmp/ISBfiles_y8ovpm9W5-W_kUqpeNiCKA_0.tar
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - ISB ZIPPED file successfully created: /tmp/ISBfiles_y8ovpm9W5-W_kUqpeNiCKA_0.tar.gz
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - Getting the SandboxDestinationURI from the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - Calling the WMProxy getSandboxDestURI service with gsiftp protocol 
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - getSandboxDestURI -  Destination URIs sucessfully retrieved
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - DestinationURI: gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/nG/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fnGo2GsPtKD3urj66sBnHLA/input
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - FileTransfer (gsiftp): using globus-url-copy to transfer the local InputSandBox file(s) to the submission endpoint
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - File Transfer (gsiftp) 
 Command: /usr/bin/globus-url-copy
Source: file:///tmp/ISBfiles_y8ovpm9W5-W_kUqpeNiCKA_0.tar.gz
Destination: gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/nG/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fnGo2GsPtKD3urj66sBnHLA/input/ISBfiles_y8ovpm9W5-W_kUqpeNiCKA_0.tar.gz
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - File Transfer (gsiftp) Transfer successfully done
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - Starting the job:  https://emi-demo11.cnaf.infn.it:9000/nGo2GsPtKD3urj66sBnHLA
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - Calling the WMProxy jobStart service
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - jobStart -  The job has been successfully started
-----------------------------------------

====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://emi-demo11.cnaf.infn.it:9000/nGo2GsPtKD3urj66sBnHLA

==========================================================================

      *** Log file created ***
Possible Errors and Debug messages have been printed in the following file:
/var/tmp/glite-wms-job-submit_504_23649_20111230_16-54-29.log

Check if AllowZippedISB has been included in the jdl:

[capannini@devel15 certificazione]$ glite-wms-job-info --jdl https://emi-demo11.cnaf.infn.it:9000/nGo2GsPtKD3urj66sBnHLA|grep Allow
  AllowZippedISB = true;

[capannini@devel15 certificazione]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/nGo2GsPtKD3urj66sBnHLA


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/nGo2GsPtKD3urj66sBnHLA
Current Status:     Done (Success)
Exit code:          0
Status Reason:      Job Terminated Successfully
Destination:        cremino.cnaf.infn.it:8443/cream-pbs-cert
Submitted:          Fri Dec 30 16:54:30 2011 CET
==========================================================================

[capannini@devel15 certificazione]$ 
[capannini@devel15 certificazione]$ 
[capannini@devel15 certificazione]$ 
[capannini@devel15 certificazione]$ glite-wms-job-output https://emi-demo11.cnaf.infn.it:9000/nGo2GsPtKD3urj66sBnHLA

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server


================================================================================

         JOB GET OUTPUT OUTCOME

Output sandbox files for the job:
https://emi-demo11.cnaf.infn.it:9000/nGo2GsPtKD3urj66sBnHLA
have been successfully retrieved and stored in the directory:
/tmp/jobOutput/capannini_nGo2GsPtKD3urj66sBnHLA

================================================================================


[capannini@devel15 certificazione]$ ls -l /tmp/jobOutput/capannini_nGo2GsPtKD3urj66sBnHLA
total 4
-rw-r--r-- 1 capannini emitestbed   0 Dec 30 17:00 std.err
-rw-r--r-- 1 capannini emitestbed 623 Dec 30 17:00 std.out
[capannini@devel15 certificazione]$ cat /tmp/jobOutput/capannini_nGo2GsPtKD3urj66sBnHLA/std.out 
total 1132
drwxr-xr-x 2 dteam011 dteam    4096 Dec 30 16:54 .
drwxr-xr-x 3 dteam011 dteam    4096 Dec 30 16:54 ..
-rw-r--r-- 1 dteam011 dteam    4037 Dec 30 16:54 .BrokerInfo
-rw-r--r-- 1 dteam011 dteam       2 Dec 30 16:54 fileA
-rw-r--r-- 1 dteam011 dteam       2 Dec 30 16:54 fileB
-rw-r--r-- 1 dteam011 dteam 1024000 Dec 30 16:54 fileC
-rw-r--r-- 1 dteam011 dteam  102400 Dec 30 16:54 fileD
-rw-r--r-- 1 dteam011 dteam       2 Dec 30 16:54 fileE
-rw-r--r-- 1 dteam011 dteam       0 Dec 30 16:54 std.err
-rw-r--r-- 1 dteam011 dteam       0 Dec 30 16:54 std.out
-rw-r--r-- 1 dteam011 dteam       0 Dec 30 16:54 .tmp_file

===>     === test PASSED === 

Jdl with ExpiryTime

[capannini@devel15 certificazione]$ cat ExpiryTime.jdl 
###########################################
#       JDL with ExpiryTime attribute     #
###########################################

Executable = "/bin/ls";
Arguments = "-la";
StdOutput = "std.out";
StdError = "std.err";

InputSandbox = {"fileA", "fileB", "fileC", "fileD", "fileE"};
OutputSandbox = {"std.out", "std.err"};

requirements = false;

ExpiryTime = 1325261887;

RetryCount = 1;
ShallowRetryCount = 2;

[capannini@devel15 certificazione]$ date +%s
1325261324

[capannini@devel15 certificazione]$ glite-wms-job-submit -a -c ~/wms_emi-demo11.conf ExpiryTime.jdl 

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://emi-demo11.cnaf.infn.it:9000/O6MWj0UPsVQblxDjgmo-gQ

==========================================================================

[capannini@devel15 certificazione]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/O6MWj0UPsVQblxDjgmo-gQ


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/O6MWj0UPsVQblxDjgmo-gQ
Current Status:     Waiting 
Status Reason:      BrokerHelper: no compatible resources
Submitted:          Fri Dec 30 17:08:58 2011 CET
==========================================================================

[capannini@devel15 certificazione]$ date +%s
1325261928
[capannini@devel15 certificazione]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/O6MWj0UPsVQblxDjgmo-gQ


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/O6MWj0UPsVQblxDjgmo-gQ
Current Status:     Aborted 
Status Reason:      request expired
Submitted:          Fri Dec 30 17:08:58 2011 CET
==========================================================================

===>     === test PASSED === 

Jdl with ShortDeadlineJob
To test SDJs the following bdii must be used: egee-bdii.cnaf.infn.it

[capannini@devel15 certificazione]$ cat SDLJ.jdl 
###########################################
# JDL with ShortDeadlineJob attribute     #
###########################################

Executable = "/bin/ls";
Arguments = "-la";
StdOutput = "std.out";
StdError = "std.err";

InputSandbox = {"fileA", "fileB", "fileC", "fileD", "fileE"};
OutputSandbox = {"std.out", "std.err"};

requirements = true;

ShortDeadlineJob = true;

RetryCount = 1;
ShallowRetryCount = 2;

[capannini@devel15 certificazione]$ glite-wms-job-list-match -a -c ~/wms_emi-demo11.conf SDLJ.jdl

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server

==========================================================================

           COMPUTING ELEMENT IDs LIST 
 The following CE(s) matching your job requirements have been found:

   *CEId*
 - ce01.marie.hellasgrid.gr:2119/jobmanager-pbs-sdj
 - cream-ce01.marie.hellasgrid.gr:8443/cream-pbs-sdj
 - grid4.mif.vu.lt:2119/jobmanager-lcgpbs-sdj
 - grive02.ibcp.fr:8443/cream-pbs-sdj
 - mario.univ-lille1.fr:8443/cream-pbs-sdj
 - ce01.ariagni.hellasgrid.gr:2119/jobmanager-pbs-dteam.sdj
 - cream-ce01.ariagni.hellasgrid.gr:8443/cream-pbs-dteam.sdj

==========================================================================

[capannini@devel15 certificazione]$ python ~/wms_tests/WMS-service/WMS-job-cycle.py -c ~/wms_tests/WMS-service/wms-command.conf -V dteam -d 3 -l -t 1 -i -j SDLJ.jdl 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 1: Submit a normal job (3 cases LCG-CE, CREAM, general) 


 Test 1A: Submit to an LCG-CE 


 Test 1B: Submit to a CREAM CE 


 Test 1C: Submit without restrictions 

===> 
===> Test: /home/capannini/wms_tests/WMS-service/WMS-job-cycle.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 11:58:16
===> Ended  : 12:04:06
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120116115816.log

Log is here: WMSService-TS_20120116115816.log: sdj log

Collection

Bulk submission

[capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 2 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 2: Submit a bulk of jobs (3 cases LCG-CE, CREAM, general) 


 Test 2A: Submit to an LCG-CE 


 Test 2B: Submit to a CREAM CE 


 Test 2C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 15:15:00
===> Ended  : 15:24:31
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120116151500.log
See log: WMSService-TS_20120116151500.log: bulk collection log

[capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -t 2 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 2: Submit a bulk of jobs (3 cases LCG-CE, CREAM, general) 


 Test 2A: Submit to an LCG-CE 


 Test 2B: Submit to a CREAM CE 


 Test 2C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 15:46:24
===> Ended  : 15:56:56
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120116154624.log
See log: WMSService-TS_20120116154624.log: bulk collection log

[capannini@devel15 WMS-service]$ myproxy-init -d
Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini
Enter GRID pass phrase for this identity:
Creating proxy .................................................................. Done
Proxy Verify OK
Your proxy is valid until: Mon Jan 23 16:04:28 2012
Enter MyProxy pass phrase:
Verifying - Enter MyProxy pass phrase:
A proxy valid for 168 hours (7.0 days) for user /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini now exists on myproxy.cnaf.infn.it.

[capannini@devel15 WMS-service]$ myproxy-logon -d --voms dteam
Enter MyProxy pass phrase:
Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy
Contacting  voms.hellasgrid.gr:15004 [/C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr] "dteam" Done
Creating proxy ......................................................................................................................... Done

Your proxy is valid until Tue Jan 17 03:32:47 2012
A credential has been received for user /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini in /tmp/x509up_u504.

[capannini@devel15 WMS-service]$ voms-proxy-info -all
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy/CN=proxy
issuer    : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy
identity  : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy
type      : proxy
strength  : 2048 bits
path      : /tmp/x509up_u504
timeleft  : 10:54:56
key usage : Digital Signature, Key Encipherment, Data Encipherment
=== VO dteam extension information ===
VO        : dteam
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini
issuer    : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr
attribute : /dteam/Role=NULL/Capability=NULL
attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL
timeleft  : 11:53:55
uri       : voms.hellasgrid.gr:15004

[capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -d 3 -l -t 2
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

 Set 2: Submit a bulk of jobs (3 cases LCG-CE, CREAM, general) 


 Test 2A: Submit to an LCG-CE 


 Test 2B: Submit to a CREAM CE 


 Test 2C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 16:32:57
===> Ended  : 16:41:22
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120116163257.log

See log: WMSService-TS_20120116163257.log: bulk collection log

All in a jdl
[capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 7 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 7: Submit a bulk of jobs (3 cases LCG-CE, CREAM, general) using a single jdl with al the jdls of nodes 


 Test 7A: Submit to an LCG-CE 


 Test 7B: Submit to a CREAM CE 


 Test 7C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 10:04:17
===> Ended  : 10:36:27
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120117100417.log

[capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -t 7 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 7: Submit a bulk of jobs (3 cases LCG-CE, CREAM, general) using a single jdl with al the jdls of nodes 


 Test 7A: Submit to an LCG-CE 


 Test 7B: Submit to a CREAM CE 


 Test 7C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 10:24:08
===> Ended  : 10:38:44
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120117102408.log

[capannini@devel15 WMS-service]$ myproxy-init -d
Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini
Enter GRID pass phrase for this identity:
Creating proxy ..................................................................................................................................................................................................... Done
Proxy Verify OK
Your proxy is valid until: Tue Jan 24 11:18:37 2012
Enter MyProxy pass phrase:
Verifying - Enter MyProxy pass phrase:
A proxy valid for 168 hours (7.0 days) for user /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini now exists on myproxy.cnaf.infn.it.
[capannini@devel15 WMS-service]$ myproxy-logon -d --voms dteam
Enter MyProxy pass phrase:
Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy
Contacting  voms2.hellasgrid.gr:15004 [/C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr] "dteam" Done
Creating proxy ............................................................................................................................................................................................................................................. Done

Your proxy is valid until Tue Jan 17 22:19:16 2012
A credential has been received for user /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini in /tmp/x509up_u504.
[capannini@devel15 WMS-service]$ voms-proxy-info -all
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy/CN=proxy
issuer    : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy
identity  : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy
type      : proxy
strength  : 2048 bits
path      : /tmp/x509up_u504
timeleft  : 10:59:33
key usage : Digital Signature, Key Encipherment, Data Encipherment
=== VO dteam extension information ===
VO        : dteam
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini
issuer    : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr
attribute : /dteam/Role=NULL/Capability=NULL
attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL
timeleft  : 11:58:32
uri       : voms2.hellasgrid.gr:15004

[capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 7
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

 Set 7: Submit a bulk of jobs (3 cases LCG-CE, CREAM, general) using a single jdl with al the jdls of nodes 


 Test 7A: Submit to an LCG-CE 


 Test 7B: Submit to a CREAM CE 


 Test 7C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 11:19:59
===> Ended  : 11:29:26
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120117111959.log

See logs: coll.tgz: all in a jdl collection logs

Parametric job

[capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 3 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 3: Submit a parametric job (3 cases LCG-CE, CREAM, general) 


 Test 3A: Submit to an LCG-CE 


 Test 3B: Submit to a CREAM CE 


 Test 3C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 10:46:35
===> Ended  : 10:53:22
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120118104635.log

[capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V testers.eu-emi.eu  -d 3 -l -t 3 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 3: Submit a parametric job (3 cases LCG-CE, CREAM, general) 


 Test 3A: Submit to an LCG-CE 


 Test 3B: Submit to a CREAM CE 


 Test 3C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 10:47:30
===> Ended  : 10:53:46
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120118104730.log

[capannini@devel15 WMS-service]$ myproxy-init -d
Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini
Enter GRID pass phrase for this identity:
Creating proxy ............... Done
Proxy Verify OK
Your proxy is valid until: Wed Jan 25 10:48:21 2012
Enter MyProxy pass phrase:
Verifying - Enter MyProxy pass phrase:
A proxy valid for 168 hours (7.0 days) for user /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini now exists on myproxy.cnaf.infn.it.
[capannini@devel15 WMS-service]$ 
[capannini@devel15 WMS-service]$ 
[capannini@devel15 WMS-service]$ 
[capannini@devel15 WMS-service]$ myproxy-logon -d --voms dteam
Enter MyProxy pass phrase:
Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy
Contacting  voms.hellasgrid.gr:15004 [/C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr] "dteam" Done
Creating proxy ................................................................................................................................................ Done

Your proxy is valid until Wed Jan 18 21:48:49 2012
A credential has been received for user /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini in /tmp/x509up_u504.
[capannini@devel15 WMS-service]$ 
[capannini@devel15 WMS-service]$ 
[capannini@devel15 WMS-service]$ voms-proxy-info -all
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy/CN=proxy
issuer    : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy
identity  : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy
type      : proxy
strength  : 2048 bits
path      : /tmp/x509up_u504
timeleft  : 10:59:51
key usage : Digital Signature, Key Encipherment, Data Encipherment
=== VO dteam extension information ===
VO        : dteam
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini
issuer    : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr
attribute : /dteam/Role=NULL/Capability=NULL
attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL
timeleft  : 11:58:50
uri       : voms.hellasgrid.gr:15004
[capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 3
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

 Set 3: Submit a parametric job (3 cases LCG-CE, CREAM, general) 


 Test 3A: Submit to an LCG-CE 


 Test 3B: Submit to a CREAM CE 


 Test 3C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 10:49:51
===> Ended  : 10:55:10
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120118104951.log

See logs: parametric.tgz: parametric logs

DAG job

[capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 4 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 4: Submit a DAG job 

===> 
===> Test: WMS-job-cycle.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 12:05:40
===> Ended  : 12:10:56
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120118120540.log

[capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V testers.eu-emi.eu  -d 3 -l -t 4 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 4: Submit a DAG job 

===> 
===> Test: WMS-job-cycle.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 12:06:00
===> Ended  : 12:11:15
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120118120600.log

[capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 4
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

 Set 4: Submit a DAG job 

===> 
===> Test: WMS-job-cycle.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 12:06:30
===> Ended  : 12:12:39
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120118120630.log
See logs: dag.tgz: dag logs

Parallel job

Setting II_Contact = "egee-bdii.cnaf.infn.it"; in glite_wms.conf and commenting #DEFAULTREQ= other.GlueCEStateStatus == "Testing"
[capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 5 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 5: Submit a MPI job (3 cases LCG-CE, CREAM, general) 


 Test 5A: Submit to an LCG-CE 


 Test 5B: Submit to a CREAM CE 


 Test 5C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 16:23:09
===> Ended  : 16:27:29
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120118162309.log

[capannini@devel15 WMS-service]$ voms-proxy-info -all
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy/CN=proxy
issuer    : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy
identity  : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy
type      : proxy
strength  : 2048 bits
path      : /tmp/x509up_u504
timeleft  : 5:59:20
key usage : Digital Signature, Key Encipherment, Data Encipherment
=== VO dteam extension information ===
VO        : dteam
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini
issuer    : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr
attribute : /dteam/Role=NULL/Capability=NULL
attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL
timeleft  : 6:58:19
uri       : voms.hellasgrid.gr:15004
[capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 5
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

 Set 5: Submit a MPI job (3 cases LCG-CE, CREAM, general) 


 Test 5A: Submit to an LCG-CE 


 Test 5B: Submit to a CREAM CE 


 Test 5C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 16:23:22
===> Ended  : 16:29:12
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120118162322.log
See logs: parallel.tgz: parallel job logs
Testing forwarding parameters for parallel jobs
In order to test the passing of mpi parameters (wholenodes,smpgranularity,cpunumber,hostnumber) to the ce, an mpi jdl is submitted with varying values for the parameters, specifying a cream CE as destination. Then the cream jobid is retrieved via a glite-wms-job-logging-info -v 2 ("Dest jobid" field of a Transfer event).Finally the cream jdl is found via glite-ce-job-status -L 2 <cream-jobid>.

The following jdls were submitted:

1)

Executable = "openmpi-wrapper.sh";
Arguments="hello";
StdOutput  = "hello.out";
StdError   = "hello.err";
OutputSandbox = {"hello.out","hello.err"};
InputSandbox = {"/home/capannini/certification/mpi/openmpi-wrapper.sh","/home/capannini/certification/mpi/hello.c","/home/capannini/certification/mpi/Makefile"};
Requirements = Member ("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*", other.GlueCEUniqueID);
WholeNodes = true;
SMPGranularity = 2;
CpuNumber=2;
Hostnumber = 1;
Resulting cream JDL:
   JDL            = [[ CEInfoHostName = "ce-qamar.utmgrid.utm.my"; OutputSandboxPath = "/var/SandboxDir/iY/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fiYM9Zl0NcKDqT1HqZiYN_5fg/output"; StdOutput = "hello.out"; BatchSystem = "pbs"; GlobusResourceContactString = "ce-qamar.utmgrid.utm.my:8443/cream-pbs"; QueueName = "dteam"; ShallowRetryCount = 10; SignificantAttributes = { "Requirements","Rank","FuzzyRank" }; RetryCount = 3; Executable = "openmpi-wrapper.sh"; Type = "job"; LB_sequence_code = "UI=000000:NS=0000000004:WM=000011:BH=0000000000:JSS=000004:LM=000012:LRMS=000000:APP=000000:LBS=000000"; Arguments = "hello"; AllowZippedISB = true; maxOutputSandboxSize = -1.000000000000000E+00; VirtualOrganisation = "dteam"; JobType = "normal"; DefaultRank =  -other.GlueCEStateEstimatedResponseTime; ce_id = "ce-qamar.utmgrid.utm.my:8443/cream-pbs-dteam"; ZippedISB = { "ISBfiles_Gmr4VWTb4-nGOH4-Bd092w_0.tar.gz" }; OutputSandboxDestURI = { "gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/iY/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fiYM9Zl0NcKDqT1HqZiYN_5fg/output/hello.out","gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/iY/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fiYM9Zl0NcKDqT1HqZiYN_5fg/output/hello.err" }; OutputSandbox = { "hello.out","hello.err" }; CeRequirements = "( true && ( true && ( true && ( true && ( true && ( true && Member(\"OPENMPI\",other.GlueHostApplicationSoftwareRunTimeEnvironment) ) ) ) ) ) ) && ( true && ( ( ShortDeadlineJob is true ) ? false : true ) && ( EnableWmsFeedback is true ? true : true ) )"; InputSandbox = { "gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/iY/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fiYM9Zl0NcKDqT1HqZiYN_5fg/input/openmpi-wrapper.sh","gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/iY/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fiYM9Zl0NcKDqT1HqZiYN_5fg/input/hello.c","gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/iY/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fiYM9Zl0NcKDqT1HqZiYN_5fg/input/Makefile","gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/iY/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fiYM9Zl0NcKDqT1HqZiYN_5fg/input/.BrokerInfo" }; edg_jobid = "https://emi-demo11.cnaf.infn.it:9000/iYM9Zl0NcKDqT1HqZiYN_g"; VOMS_FQAN = "/dteam/Role=NULL/Capability=NULL"; CertificateSubject = "/C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini"; StdError = "hello.err"; SMPGranularity = 2; InputSandboxPath = "/var/SandboxDir/iY/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fiYM9Zl0NcKDqT1HqZiYN_5fg/input"; CeApplicationDir = "/opt/exp_soft/dteam"; ReallyRunningToken = "gsiftp://emi-demo11.cnaf.infn.it/var/SandboxDir/iY/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fiYM9Zl0NcKDqT1HqZiYN_5fg/token.txt_0"; rank =  -other.GlueCEStateEstimatedResponseTime; WholeNodes = true; CpuNumber = 2; edg_previous_matches = { "gb-ce-emc.erasmusmc.nl" }; ReplansCount = 0; WMSHostname = "emi-demo11.cnaf.infn.it"; lrms_type = "torque"; MyProxyServer = "myproxy.cnaf.infn.it"; edg_previous_matches_ex = { [ timestamp = 1326965618; ce_id = "gb-ce-emc.erasmusmc.nl" ] }; X509UserProxy = "/var/SandboxDir/iY/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fiYM9Zl0NcKDqT1HqZiYN_5fg/user.proxy"; InputSandboxDestFileName = { "openmpi-wrapper.sh","hello.c","Makefile" }; Hostnumber = 1; requirements = ( ( ( ( ( ( Member("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*",other.GlueCEUniqueID) ) && ( other.GlueCEStateStatus == "Production" ) ) && other.GlueCEInfoTotalCPUs >= 2 ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && other.GlueCEInfoTotalCPUs >= 2 ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && ( ( ( ShortDeadlineJob is true ) ? RegExp(".*sdj$",other.GlueCEUniqueID) :  !RegExp(".*sdj$",other.GlueCEUniqueID) ) && ( other.GlueCEPolicyMaxTotalJobs == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs ) && ( EnableWmsFeedback is true ? RegExp("cream",other.GlueCEImplementationName,"i") : true ) ); WMPInputSandboxBaseURI = "gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/iY/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fiYM9Zl0NcKDqT1HqZiYN_5fg" ]]
2)
Executable = "openmpi-wrapper.sh";
Arguments="hello";
StdOutput  = "hello.out";
StdError   = "hello.err";
OutputSandbox = {"hello.out","hello.err"};
InputSandbox = {"/home/capannini/certification/mpi/openmpi-wrapper.sh","/home/capannini/certification/mpi/hello.c","/home/capannini/certification/mpi/Makefile"};
Requirements = Member ("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*", other.GlueCEUniqueID);
WholeNodes = true;
SMPGranularity = 2;
Resulting cream jdl:
   JDL            = [[ CEInfoHostName = "ce-02.roma3.infn.it"; OutputSandboxPath = "/var/SandboxDir/mR/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fmRQj0IPxfJ5ZjuBWrcDn1g/output"; StdOutput = "hello.out"; BatchSystem = "pbs"; GlobusResourceContactString = "ce-02.roma3.infn.it:8443/cream-pbs"; QueueName = "cert"; ShallowRetryCount = 10; SignificantAttributes = { "Requirements","Rank","FuzzyRank" }; RetryCount = 3; Executable = "openmpi-wrapper.sh"; Type = "job"; LB_sequence_code = "UI=000000:NS=0000000004:WM=000017:BH=0000000000:JSS=000006:LM=000022:LRMS=000000:APP=000000:LBS=000000"; Arguments = "hello"; AllowZippedISB = true; maxOutputSandboxSize = -1.000000000000000E+00; VirtualOrganisation = "dteam"; JobType = "normal"; DefaultRank =  -other.GlueCEStateEstimatedResponseTime; ce_id = "ce-02.roma3.infn.it:8443/cream-pbs-cert"; ZippedISB = { "ISBfiles_zpOEExFPS0a9q8TGstx7hQ_0.tar.gz" }; OutputSandboxDestURI = { "gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/mR/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fmRQj0IPxfJ5ZjuBWrcDn1g/output/hello.out","gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/mR/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fmRQj0IPxfJ5ZjuBWrcDn1g/output/hello.err" }; OutputSandbox = { "hello.out","hello.err" }; CeRequirements = "( true && ( true && ( true && ( true && Member(\"OPENMPI\",other.GlueHostApplicationSoftwareRunTimeEnvironment) ) ) ) ) && ( true && ( ( ShortDeadlineJob is true ) ? false : true ) && ( EnableWmsFeedback is true ? true : true ) )"; InputSandbox = { "gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/mR/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fmRQj0IPxfJ5ZjuBWrcDn1g/input/openmpi-wrapper.sh","gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/mR/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fmRQj0IPxfJ5ZjuBWrcDn1g/input/hello.c","gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/mR/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fmRQj0IPxfJ5ZjuBWrcDn1g/input/Makefile","gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/mR/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fmRQj0IPxfJ5ZjuBWrcDn1g/input/.BrokerInfo" }; edg_jobid = "https://emi-demo11.cnaf.infn.it:9000/mRQj0IPxfJ5ZjuBWrcDn1g"; VOMS_FQAN = "/dteam/Role=NULL/Capability=NULL"; CertificateSubject = "/C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini"; StdError = "hello.err"; SMPGranularity = 2; InputSandboxPath = "/var/SandboxDir/mR/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fmRQj0IPxfJ5ZjuBWrcDn1g/input"; CeApplicationDir = "/storage/local/exp_soft/dteam"; ReallyRunningToken = "gsiftp://emi-demo11.cnaf.infn.it/var/SandboxDir/mR/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fmRQj0IPxfJ5ZjuBWrcDn1g/token.txt_0"; rank =  -other.GlueCEStateEstimatedResponseTime; WholeNodes = true; edg_previous_matches = { "ce.scope.unina.it","grisuce.scope.unina.it" }; ReplansCount = 0; WMSHostname = "emi-demo11.cnaf.infn.it"; lrms_type = "pbs"; MyProxyServer = "myproxy.cnaf.infn.it"; edg_previous_matches_ex = { [ timestamp = 1326967114; ce_id = "ce.scope.unina.it" ],[ timestamp = 1326967117; ce_id = "grisuce.scope.unina.it" ] }; X509UserProxy = "/var/SandboxDir/mR/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fmRQj0IPxfJ5ZjuBWrcDn1g/user.proxy"; InputSandboxDestFileName = { "openmpi-wrapper.sh","hello.c","Makefile" }; requirements = ( ( ( ( Member("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*",other.GlueCEUniqueID) ) && ( other.GlueCEStateStatus == "Production" ) ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && ( ( ( ShortDeadlineJob is true ) ? RegExp(".*sdj$",other.GlueCEUniqueID) :  !RegExp(".*sdj$",other.GlueCEUniqueID) ) && ( other.GlueCEPolicyMaxTotalJobs == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs ) && ( EnableWmsFeedback is true ? RegExp("cream",other.GlueCEImplementationName,"i") : true ) ); WMPInputSandboxBaseURI = "gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/mR/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fmRQj0IPxfJ5ZjuBWrcDn1g" ]]
3)
Executable = "openmpi-wrapper.sh";
Arguments="hello";
StdOutput  = "hello.out";
StdError   = "hello.err";
OutputSandbox = {"hello.out","hello.err"};
InputSandbox = {"/home/capannini/certification/mpi/openmpi-wrapper.sh","/home/capannini/certification/mpi/hello.c","/home/capannini/certification/mpi/Makefile"};
Requirements = Member ("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*", other.GlueCEUniqueID);
WholeNodes = false;
SMPGranularity = 3;
CpuNumber=3;
Hostnumber = 1;

Submission result:

[capannini@devel15 mpi]$ glite-wms-job-submit -a -c ~/wms_emi-demo11.conf mpi.jdl 

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server


Error - 
SMPGranularity and HostNumber are mutually exclusive when WholeNodes allocation is not requested: wrong combination of values
4)
Executable = "openmpi-wrapper.sh";
Arguments="hello";
StdOutput  = "hello.out";
StdError   = "hello.err";
OutputSandbox = {"hello.out","hello.err"};
InputSandbox = {"/home/capannini/certification/mpi/openmpi-wrapper.sh","/home/capannini/certification/mpi/hello.c","/home/capannini/certification/mpi/Makefile"};
Requirements = Member ("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*", other.GlueCEUniqueID);
WholeNodes = false;
SMPGranularity = 3;
CpuNumber=3;
Resulting cream jdl:
        JDL            = [[ CEInfoHostName = "ce-02.roma3.infn.it"; OutputSandboxPath = "/var/SandboxDir/l-/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fl-na1nvxx06q_5fy6klpEnQQ/output"; StdOutput = "hello.out"; BatchSystem = "pbs"; GlobusResourceContactString = "ce-02.roma3.infn.it:8443/cream-pbs"; QueueName = "cert"; ShallowRetryCount = 10; SignificantAttributes = { "Requirements","Rank","FuzzyRank" }; RetryCount = 3; Executable = "openmpi-wrapper.sh"; Type = "job"; LB_sequence_code = "UI=000000:NS=0000000004:WM=000017:BH=0000000000:JSS=000006:LM=000022:LRMS=000000:APP=000000:LBS=000000"; Arguments = "hello"; AllowZippedISB = true; maxOutputSandboxSize = -1.000000000000000E+00; VirtualOrganisation = "dteam"; JobType = "normal"; DefaultRank =  -other.GlueCEStateEstimatedResponseTime; ce_id = "ce-02.roma3.infn.it:8443/cream-pbs-cert"; ZippedISB = { "ISBfiles_gQrxvLonwNUGPUr_WRZxrg_0.tar.gz" }; OutputSandboxDestURI = { "gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/l-/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fl-na1nvxx06q_5fy6klpEnQQ/output/hello.out","gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/l-/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fl-na1nvxx06q_5fy6klpEnQQ/output/hello.err" }; OutputSandbox = { "hello.out","hello.err" }; CeRequirements = "( true && ( true && ( true && ( true && ( true && ( true && Member(\"OPENMPI\",other.GlueHostApplicationSoftwareRunTimeEnvironment) ) ) ) ) ) ) && ( true && ( ( ShortDeadlineJob is true ) ? false : true ) && ( EnableWmsFeedback is true ? true : true ) )"; InputSandbox = { "gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/l-/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fl-na1nvxx06q_5fy6klpEnQQ/input/openmpi-wrapper.sh","gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/l-/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fl-na1nvxx06q_5fy6klpEnQQ/input/hello.c","gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/l-/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fl-na1nvxx06q_5fy6klpEnQQ/input/Makefile","gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/l-/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fl-na1nvxx06q_5fy6klpEnQQ/input/.BrokerInfo" }; edg_jobid = "https://emi-demo11.cnaf.infn.it:9000/l-na1nvxx06q_y6klpEnQQ"; VOMS_FQAN = "/dteam/Role=NULL/Capability=NULL"; CertificateSubject = "/C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini"; StdError = "hello.err"; SMPGranularity = 3; InputSandboxPath = "/var/SandboxDir/l-/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fl-na1nvxx06q_5fy6klpEnQQ/input"; CeApplicationDir = "/storage/local/exp_soft/dteam"; ReallyRunningToken = "gsiftp://emi-demo11.cnaf.infn.it/var/SandboxDir/l-/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fl-na1nvxx06q_5fy6klpEnQQ/token.txt_0"; rank =  -other.GlueCEStateEstimatedResponseTime; WholeNodes = false; CpuNumber = 3; edg_previous_matches = { "grisuce.scope.unina.it","ce.scope.unina.it" }; ReplansCount = 0; WMSHostname = "emi-demo11.cnaf.infn.it"; lrms_type = "pbs"; MyProxyServer = "myproxy.cnaf.infn.it"; edg_previous_matches_ex = { [ timestamp = 1326968771; ce_id = "grisuce.scope.unina.it" ],[ timestamp = 1326968773; ce_id = "ce.scope.unina.it" ] }; X509UserProxy = "/var/SandboxDir/l-/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fl-na1nvxx06q_5fy6klpEnQQ/user.proxy"; InputSandboxDestFileName = { "openmpi-wrapper.sh","hello.c","Makefile" }; requirements = ( ( ( ( ( ( Member("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*",other.GlueCEUniqueID) ) && ( other.GlueCEStateStatus == "Production" ) ) && other.GlueCEInfoTotalCPUs >= 3 ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && other.GlueCEInfoTotalCPUs >= 3 ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && ( ( ( ShortDeadlineJob is true ) ? RegExp(".*sdj$",other.GlueCEUniqueID) :  !RegExp(".*sdj$",other.GlueCEUniqueID) ) && ( other.GlueCEPolicyMaxTotalJobs == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs ) && ( EnableWmsFeedback is true ? RegExp("cream",other.GlueCEImplementationName,"i") : true ) ); WMPInputSandboxBaseURI = "gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/l-/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fl-na1nvxx06q_5fy6klpEnQQ" ]]
5)
Executable = "openmpi-wrapper.sh";
Arguments="hello";
StdOutput  = "hello.out";
StdError   = "hello.err";
OutputSandbox = {"hello.out","hello.err"};
InputSandbox = {"/home/capannini/certification/mpi/openmpi-wrapper.sh","/home/capannini/certification/mpi/hello.c","/home/capannini/certification/mpi/Makefile"};
Requirements = Member ("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*", other.GlueCEUniqueID);
WholeNodes = true;
SMPGranularity = 8;
CpuNumber=8;
Hostnumber = 2;
Resulting ce jdl:
   JDL            = [[ CEInfoHostName = "ce-02.roma3.infn.it"; OutputSandboxPath = "/var/SandboxDir/Oc/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fOc3Gbgez4On6im_5fB63zOkQ/output"; StdOutput = "hello.out"; BatchSystem = "pbs"; GlobusResourceContactString = "ce-02.roma3.infn.it:8443/cream-pbs"; QueueName = "cert"; ShallowRetryCount = 10; SignificantAttributes = { "Requirements","Rank","FuzzyRank" }; RetryCount = 3; Executable = "openmpi-wrapper.sh"; Type = "job"; LB_sequence_code = "UI=000000:NS=0000000004:WM=000005:BH=0000000000:JSS=000002:LM=000002:LRMS=000000:APP=000000:LBS=000000"; Arguments = "hello"; AllowZippedISB = true; maxOutputSandboxSize = -1.000000000000000E+00; VirtualOrganisation = "dteam"; JobType = "normal"; DefaultRank =  -other.GlueCEStateEstimatedResponseTime; ce_id = "ce-02.roma3.infn.it:8443/cream-pbs-cert"; ZippedISB = { "ISBfiles_dz-M5EA3o9uj857y8CkqMg_0.tar.gz" }; OutputSandboxDestURI = { "gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/Oc/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fOc3Gbgez4On6im_5fB63zOkQ/output/hello.out","gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/Oc/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fOc3Gbgez4On6im_5fB63zOkQ/output/hello.err" }; OutputSandbox = { "hello.out","hello.err" }; CeRequirements = "( true && ( true && ( true && ( true && ( true && ( true && Member(\"OPENMPI\",other.GlueHostApplicationSoftwareRunTimeEnvironment) ) ) ) ) ) ) && ( true && ( ( ShortDeadlineJob is true ) ? false : true ) && ( EnableWmsFeedback is true ? true : true ) )"; InputSandbox = { "gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/Oc/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fOc3Gbgez4On6im_5fB63zOkQ/input/openmpi-wrapper.sh","gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/Oc/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fOc3Gbgez4On6im_5fB63zOkQ/input/hello.c","gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/Oc/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fOc3Gbgez4On6im_5fB63zOkQ/input/Makefile","gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/Oc/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fOc3Gbgez4On6im_5fB63zOkQ/input/.BrokerInfo" }; edg_jobid = "https://emi-demo11.cnaf.infn.it:9000/Oc3Gbgez4On6im_B63zOkQ"; VOMS_FQAN = "/dteam/Role=NULL/Capability=NULL"; CertificateSubject = "/C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini"; StdError = "hello.err"; SMPGranularity = 8; InputSandboxPath = "/var/SandboxDir/Oc/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fOc3Gbgez4On6im_5fB63zOkQ/input"; CeApplicationDir = "/storage/local/exp_soft/dteam"; ReallyRunningToken = "gsiftp://emi-demo11.cnaf.infn.it/var/SandboxDir/Oc/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fOc3Gbgez4On6im_5fB63zOkQ/token.txt_0"; rank =  -other.GlueCEStateEstimatedResponseTime; WholeNodes = true; CpuNumber = 8; WMSHostname = "emi-demo11.cnaf.infn.it"; lrms_type = "pbs"; MyProxyServer = "myproxy.cnaf.infn.it"; X509UserProxy = "/var/SandboxDir/Oc/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fOc3Gbgez4On6im_5fB63zOkQ/user.proxy"; InputSandboxDestFileName = { "openmpi-wrapper.sh","hello.c","Makefile" }; Hostnumber = 2; requirements = ( ( ( ( ( ( Member("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*",other.GlueCEUniqueID) ) && ( other.GlueCEStateStatus == "Production" ) ) && other.GlueCEInfoTotalCPUs >= 8 ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && other.GlueCEInfoTotalCPUs >= 8 ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && ( ( ( ShortDeadlineJob is true ) ? RegExp(".*sdj$",other.GlueCEUniqueID) :  !RegExp(".*sdj$",other.GlueCEUniqueID) ) && ( other.GlueCEPolicyMaxTotalJobs == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs ) && ( EnableWmsFeedback is true ? RegExp("cream",other.GlueCEImplementationName,"i") : true ) ); WMPInputSandboxBaseURI = "gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/Oc/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fOc3Gbgez4On6im_5fB63zOkQ" ]]

--> Test PASSED

Perusal

[capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 6 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 6: Submit a perusal job 


 BEWARE default min perusal interval is 1000 secs, so this phase could take many minutes 

===> 
===> Test: WMS-job-cycle.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 10:12:07
===> Ended  : 10:46:35
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120120101207.log
[capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V testers.eu-emi.eu  -d 3 -l -t 6 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 6: Submit a perusal job 


 BEWARE default min perusal interval is 1000 secs, so this phase could take many minutes 

===> 
===> Test: WMS-job-cycle.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 14:28:05
===> Ended  : 14:55:34
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120120142805.log
[capannini@devel15 WMS-service]$ voms-proxy-info -all
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy/CN=proxy
issuer    : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy
identity  : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy
type      : proxy
strength  : 2048 bits
path      : /tmp/x509up_u504
timeleft  : 6:31:39
key usage : Digital Signature, Key Encipherment, Data Encipherment
=== VO dteam extension information ===
VO        : dteam
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini
issuer    : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr
attribute : /dteam/Role=NULL/Capability=NULL
attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL
timeleft  : 7:30:39
uri       : voms2.hellasgrid.gr:15004
[capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -d 3 -l -t 6
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

 Set 6: Submit a perusal job 


 BEWARE default min perusal interval is 1000 secs, so this phase could take many minutes 

===> 
===> Test: WMS-job-cycle.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 14:58:43
===> Ended  : 15:26:03
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120120145843.log
See logs: perusal.tgz: perusal logs

Delegation

[capannini@devel15 WMS-service]$ python WMS-delegation-test.py -c wms-command.conf -V dteam -d 3 -l -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test delegation operation 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Test 1: Check --autm-delegation option 


 Test 2: Check --delegationid option 


 Test 3: Try to delegate with a short proxy and check the validity 


 Test 4: Works with expired proxy 

===> 
===> Test: WMS-delegation-test.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 10:22:35
===> Ended  : 10:23:49
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120120102235.log

[capannini@devel15 WMS-service]$ python WMS-delegation-test.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test delegation operation 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Test 1: Check --autm-delegation option 


 Test 2: Check --delegationid option 


 Test 3: Try to delegate with a short proxy and check the validity 


 Test 4: Works with expired proxy 

===> 
===> Test: WMS-delegation-test.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 10:25:00
===> Ended  : 10:26:14
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120120102500.log

[capannini@devel15 WMS-service]$ myproxy-init -d
Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini
Enter GRID pass phrase for this identity:
Creating proxy ............................... Done
Proxy Verify OK
Your proxy is valid until: Fri Jan 27 10:27:27 2012
Enter MyProxy pass phrase:
Verifying - Enter MyProxy pass phrase:
A proxy valid for 168 hours (7.0 days) for user /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini now exists on myproxy.cnaf.infn.it.
[capannini@devel15 WMS-service]$ 
[capannini@devel15 WMS-service]$ 
[capannini@devel15 WMS-service]$ myproxy-logon -d --voms dteam
Enter MyProxy pass phrase:
Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy
Contacting  voms2.hellasgrid.gr:15004 [/C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr] "dteam" Done
Creating proxy .................... Done

Your proxy is valid until Fri Jan 20 21:27:59 2012
A credential has been received for user /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini in /tmp/x509up_u504.
[capannini@devel15 WMS-service]$ 
[capannini@devel15 WMS-service]$ 
[capannini@devel15 WMS-service]$ voms-proxy-info -all
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy/CN=proxy
issuer    : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy
identity  : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy
type      : proxy
strength  : 2048 bits
path      : /tmp/x509up_u504
timeleft  : 10:59:47
key usage : Digital Signature, Key Encipherment, Data Encipherment
=== VO dteam extension information ===
VO        : dteam
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini
issuer    : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr
attribute : /dteam/Role=NULL/Capability=NULL
attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL
timeleft  : 11:58:47
uri       : voms2.hellasgrid.gr:15004
[capannini@devel15 WMS-service]$ 
[capannini@devel15 WMS-service]$ 
[capannini@devel15 WMS-service]$ python WMS-delegation-test.py -c wms-command.conf -d 3 -l
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test delegation operation 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

 Test 1: Check --autm-delegation option 


 Test 2: Check --delegationid option 


 There are other two tests which require the user proxy password. Use -i option to enable them 

===> 
===> Test: WMS-delegation-test.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 10:28:43
===> Ended  : 10:28:46
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120120102843.log
See logs: delegation.tgz: delegation logs

Job List-match

Without data
[capannini@devel15 WMS-service]$ python WMS-listmatch-test.py -c wms-command.conf -V dteam -d 3 -l -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS Job ListMatch Testing 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Test 1: check if glite-wms-job-list-match command exists 


 Test 2: test a simple matching 


 Test 3: check --delegationid option 


 Test 4: exclude a CE from the match 


 Test 5: try a failure matching (Requirements == false) 


 Test 6: try a matching with EnableWMSFeedback attribute true 

===> 
===> Test: WMS-listmatch-test.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 10:49:06
===> Ended  : 10:49:18
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120123104906.log
See log: WMSService-TS_20120123104906.log: list-match log

[capannini@devel15 WMS-service]$ python WMS-listmatch-test.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS Job ListMatch Testing 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Test 1: check if glite-wms-job-list-match command exists 


 Test 2: test a simple matching 


 Test 3: check --delegationid option 


 Test 4: exclude a CE from the match 


 Test 5: try a failure matching (Requirements == false) 


 Test 6: try a matching with EnableWMSFeedback attribute true 

===> 
===> Test: WMS-listmatch-test.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 12:10:55
===> Ended  : 12:11:07
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120123121055.log
See logs: WMSService-TS_20120123121055.log: list-match log

[capannini@devel15 WMS-service]$ myproxy-init -d
Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini
Enter GRID pass phrase for this identity:
Creating proxy .................................................................. Done
Proxy Verify OK
Your proxy is valid until: Mon Jan 30 12:13:56 2012
Enter MyProxy pass phrase:
Verifying - Enter MyProxy pass phrase:
A proxy valid for 168 hours (7.0 days) for user /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini now exists on myproxy.cnaf.infn.it.
[capannini@devel15 WMS-service]$ myproxy-logon -d --voms dteam
Enter MyProxy pass phrase:
Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy
Contacting  voms.hellasgrid.gr:15004 [/C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr] "dteam" Done
Creating proxy ....................... Done

Your proxy is valid until Mon Jan 23 23:14:19 2012
A credential has been received for user /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini in /tmp/x509up_u504.
[capannini@devel15 WMS-service]$ 
[capannini@devel15 WMS-service]$ voms-proxy-info -all
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy/CN=proxy
issuer    : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy
identity  : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy
type      : proxy
strength  : 2048 bits
path      : /tmp/x509up_u504
timeleft  : 10:59:52
key usage : Digital Signature, Key Encipherment, Data Encipherment
=== VO dteam extension information ===
VO        : dteam
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini
issuer    : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr
attribute : /dteam/Role=NULL/Capability=NULL
attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL
timeleft  : 11:58:52
uri       : voms.hellasgrid.gr:15004

[capannini@devel15 WMS-service]$ python WMS-listmatch-test.py -c wms-command.conf -d 3 -l
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS Job ListMatch Testing 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

 Test 1: check if glite-wms-job-list-match command exists 


 Test 2: test a simple matching 


 Test 3: check --delegationid option 


 Test 4: exclude a CE from the match 


 Test 5: try a failure matching (Requirements == false) 


 Test 6: try a matching with EnableWMSFeedback attribute true 

===> 
===> Test: WMS-listmatch-test.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 12:15:09
===> Ended  : 12:15:15
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120123121509.log
See logs: WMSService-TS_20120123121509.log: list-match log

With data
[fabiok@ui ~]$ export LFC_HOST=prod-lfc-shared-central.cern.ch
[fabiok@ui ~]$ lcg-lr --vo dteam lfn:/grid/dteam/fcapanni/e-2M.txt
srm://gridsrm.pi.infn.it/dteam/fcapanni/e-2M.txt
srm://srm-public.cern.ch/castor/cern.ch/grid/dteam/fcapanni/e-2M.txt
The two SEs where the file can be found are gridsrm.pi.infn.it and srm-public.cern.ch.

[capannini@devel15 WMS-service]$ cat JDLs/data-req.jdl 
###########################################
#      JDL with Data Requirements         #
###########################################

Executable = "/bin/ls";
Arguments = "-la";
StdOutput = "std.out";
StdError = "std.err";
Prologue = "prologue.sh";

FuzzyRank = true;
Environment={"LFC_HOST=prod-lfc-shared-central.cern.ch", "LFC_HOME=prod-lfc-shared-central.cern.ch:/grid/dteam/aleph"};
InputSandbox = {"JDLs/prologue.sh"};
OutputSandbox = {"std.out", "std.err","prologue.out", "PI_1M.txt"};

DataRequirements = {
[
DataCatalogType = "DLI";
DataCatalog ="http://prod-lfc-shared-central.cern.ch:8085/"; 
InputData = { "lfn:/grid/dteam/fcapanni/PI_1M.txt" };
]
};

DataAccessProtocol = "gsiftp";
AllowZippedISB=true;
RetryCount = 1;
ShallowRetryCount = 2;

[capannini@devel15 WMS-service]$ glite-wms-job-list-match -a -c ~/wms_emi-demo11.conf JDLs/data-req.jdl 

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server

==========================================================================

           COMPUTING ELEMENT IDs LIST 
 The following CE(s) matching your job requirements have been found:

   *CEId*
 - ce130.cern.ch:2119/jobmanager-lcglsf-grid_2nh_dteam
 - ce130.cern.ch:2119/jobmanager-lcglsf-grid_dteam
 - ce131.cern.ch:2119/jobmanager-lcglsf-grid_2nh_dteam
 - ce131.cern.ch:2119/jobmanager-lcglsf-grid_dteam
 - ce132.cern.ch:2119/jobmanager-lcglsf-grid_2nh_dteam
 - ce132.cern.ch:2119/jobmanager-lcglsf-grid_dteam
 - ce133.cern.ch:2119/jobmanager-lcglsf-grid_2nh_dteam
 - ce133.cern.ch:2119/jobmanager-lcglsf-grid_dteam
 - ce203.cern.ch:8443/cream-lsf-grid_2nh_dteam
 - ce203.cern.ch:8443/cream-lsf-grid_dteam
 - ce204.cern.ch:8443/cream-lsf-grid_2nh_dteam
 - ce204.cern.ch:8443/cream-lsf-grid_dteam
 - ce205.cern.ch:8443/cream-lsf-grid_2nh_dteam
 - ce205.cern.ch:8443/cream-lsf-grid_dteam
 - ce206.cern.ch:8443/cream-lsf-grid_2nh_dteam
 - ce206.cern.ch:8443/cream-lsf-grid_dteam
 - ce207.cern.ch:8443/cream-lsf-grid_2nh_dteam
 - ce207.cern.ch:8443/cream-lsf-grid_dteam
 - ce208.cern.ch:8443/cream-lsf-grid_2nh_dteam
 - ce208.cern.ch:8443/cream-lsf-grid_dteam
 - cream.sns.it:8443/cream-pbs-cert
 - gridce0.pi.infn.it:8443/cream-lsf-cert
 - gridce1.pi.infn.it:2119/jobmanager-lcglsf-cert
 - gridce2.pi.infn.it:2119/jobmanager-lcglsf-cert
 - gridce3.pi.infn.it:8443/cream-lsf-certmpi
 - gridce.sns.it:2119/jobmanager-lcgpbs-cert

==========================================================================

In order to verify that these CEs are close to the above SEs:

[fabiok@ui ~]$ lcg-infosites --vo dteam closeSE|grep -A 5 ce130
Name of the CE: ce130.cern.ch:2119/jobmanager-lcglsf-grid_2nh_dteam
   srm-alice.cern.ch
   srm-atlas.cern.ch
   srm-cms.cern.ch
   srm-lhcb.cern.ch
   srm-public.cern.ch
--
Name of the CE: ce130.cern.ch:2119/jobmanager-lcglsf-grid_dteam
   srm-alice.cern.ch
   srm-atlas.cern.ch
   srm-cms.cern.ch
   srm-lhcb.cern.ch
   srm-public.cern.ch

[fabiok@ui ~]$ lcg-infosites --vo dteam closeSE|grep -A 5 ce208.cern.ch
Name of the CE: ce208.cern.ch:8443/cream-lsf-grid_2nh_dteam
   srm-alice.cern.ch
   srm-atlas.cern.ch
   srm-cms.cern.ch
   srm-lhcb.cern.ch
   srm-public.cern.ch
--
Name of the CE: ce208.cern.ch:8443/cream-lsf-grid_dteam
   srm-alice.cern.ch
   srm-atlas.cern.ch
   srm-cms.cern.ch
   srm-lhcb.cern.ch
   srm-public.cern.ch

[fabiok@ui ~]$ lcg-infosites --vo dteam closeSE|grep -A 5 cream.sns.it
Name of the CE: cream.sns.it:8443/cream-pbs-cert
   gridsrm.pi.infn.it

[fabiok@ui ~]$ lcg-infosites --vo dteam closeSE|grep -A 5 gridce.sns.it
Name of the CE: gridce.sns.it:2119/jobmanager-lcgpbs-cert
   gridsrm.pi.infn.it

===> Test PASSED

Gangmatching
[capannini@devel15 certification]$ cat gang.jdl
[
executable="ls.sh";
myproxyserver="";
SErequirements = anyMatch(other.storage.CloseSEs, target.GlueSAStateAvailableSpace > 20);
CErequirements = other.GlueCEStateStatus == "Production" && other.GlueCEInfoTotalCPUs >= 2;
requirements = CErequirements && SErequirements;
inputsandbox={"ls.sh"};
stdoutput="out3.out";
stderror="err2.err";
outputsandbox={"out3.out", "err2.err", "ls.sh", "a"};
]

[capannini@devel15 certification]$ glite-wms-job-list-match -a -c ~/wms_emi-demo11.conf gang.jdl 

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server

==========================================================================

           COMPUTING ELEMENT IDs LIST 
 The following CE(s) matching your job requirements have been found:

   *CEId*
 - cccreamceli05.in2p3.fr:8443/cream-sge-long
 - cccreamceli05.in2p3.fr:8443/cream-sge-medium
 - cccreamceli05.in2p3.fr:8443/cream-sge-short
 - cccreamceli09.in2p3.fr:8443/cream-sge-long
 - cccreamceli09.in2p3.fr:8443/cream-sge-medium
 - cccreamceli09.in2p3.fr:8443/cream-sge-short
 - cmsce01.na.infn.it:2119/jobmanager-lcgpbs-cert
 - cmsrm-ce01.roma1.infn.it:2119/jobmanager-lcglsf-cmsgcert
 - cmsrm-cream01.roma1.infn.it:8443/cream-lsf-cmsgcert
 - grid010.ct.infn.it:2119/jobmanager-lcgpbs-cert
 - inaf-ce-01.ct.pi2s2.it:8443/cream-lsf-cert
 - unict-dmi-ce-01.ct.pi2s2.it:2119/jobmanager-lcglsf-cert
 - virgo-ce.roma1.infn.it:2119/jobmanager-lcgpbs-cert
 - ce-01.grid.sissa.it:8443/cream-pbs-blade
 - cccreamceli10.in2p3.fr:8443/cream-sge-long
 - cccreamceli10.in2p3.fr:8443/cream-sge-medium
 - cccreamceli10.in2p3.fr:8443/cream-sge-short
 - cmsrm-ce02.roma1.infn.it:2119/jobmanager-lcglsf-cmsgcert
 - cccreamceli06.in2p3.fr:8443/cream-sge-long
 - cccreamceli06.in2p3.fr:8443/cream-sge-medium
 - cccreamceli06.in2p3.fr:8443/cream-sge-short
 - cream-ce.pg.infn.it:8443/cream-pbs-cert

==========================================================================

In order to verify that the SErequirements are met:

[capannini@devel15 certification]$ lcg-infosites closeSE|grep -A 5 cccreamceli05.in2p3.fr
Name of the CE: cccreamceli05.in2p3.fr:8443/cream-sge-long
   ccsrm.in2p3.fr

[capannini@devel15 certification]$ lcg-infosites se |egrep 'Avail|ccsrm.in2p3.fr'
 Avail Space(kB)  Used Space(kB)  Type    SE
     11385302023      3647083513  SRM     ccsrm.in2p3.fr

[capannini@devel15 certification]$ lcg-infosites closeSE|grep -A 5 cmsce01.na.infn.it
Name of the CE: cmsce01.na.infn.it:2119/jobmanager-lcgpbs-cert
   cmsse02.na.infn.it

Name of the CE: cmsrm-ce01.roma1.infn.it:2119/jobmanager-lcglsf-cmsgcert
   cmsrm-se01.roma1.infn.it

[capannini@devel15 certification]$ lcg-infosites se |egrep 'Avail|cmsse02.na.infn.it'
 Avail Space(kB)  Used Space(kB)  Type    SE
     25271855845       690913963  SRM     cmsse02.na.infn.it
[capannini@devel15 certification]$ lcg-infosites closeSE|grep -A 5 cmsrm-ce02.roma1.infn.it
Name of the CE: cmsrm-ce02.roma1.infn.it:2119/jobmanager-lcglsf-cmsgcert
   cmsrm-se01.roma1.infn.it

Name of the CE: cmsrm-cream01.roma1.infn.it:8443/cream-lsf-cmsgcert
   cmsrm-se01.roma1.infn.it

[capannini@devel15 certification]$ lcg-infosites se |egrep 'Avail|cmsrm-se01.roma1.infn.it'
 Avail Space(kB)  Used Space(kB)  Type    SE
            1000          999999  SRM     cmsrm-se01.roma1.infn.it

Moreover, setting SErequirements = true in the jdl, the list of matching CEs is far longer.
===> Test PASSED

Prologue and Epilogue jobs

 [capannini@devel15 WMS-service]$ python WMS-prologue-epilogue.py -c wms-command.conf -V dteam -d 3 -l  -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS test prologue and epilogue attributes 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Test 1: Test prologue attribute 


 Test 2: Test epilogue attribute 


 Test 3: Test prologue and epilogue attributes 

===> 
===> Test: WMS-prologue-epilogue.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 16:05:57
===> Ended  : 16:11:17
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120123160557.log
[capannini@devel15 WMS-service]$ python WMS-prologue-epilogue.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l  -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS test prologue and epilogue attributes 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Test 1: Test prologue attribute 


 Test 2: Test epilogue attribute 


 Test 3: Test prologue and epilogue attributes 

===> 
===> Test: WMS-prologue-epilogue.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 16:18:49
===> Ended  : 16:25:09
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120123161849.log
[capannini@devel15 WMS-service]$ 
[capannini@devel15 WMS-service]$ 
[capannini@devel15 WMS-service]$ voms-proxy-info -all
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy/CN=proxy
issuer    : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy
identity  : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy
type      : proxy
strength  : 2048 bits
path      : /tmp/x509up_u504
timeleft  : 6:43:33
key usage : Digital Signature, Key Encipherment, Data Encipherment
=== VO dteam extension information ===
VO        : dteam
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini
issuer    : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr
attribute : /dteam/Role=NULL/Capability=NULL
attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL
timeleft  : 7:42:33
uri       : voms.hellasgrid.gr:15004
[capannini@devel15 WMS-service]$ 
[capannini@devel15 WMS-service]$ python WMS-prologue-epilogue.py -c wms-command.conf -d 3 -l 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS test prologue and epilogue attributes 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

 Test 1: Test prologue attribute 


 Test 2: Test epilogue attribute 


 Test 3: Test prologue and epilogue attributes 

===> 
===> Test: WMS-prologue-epilogue.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 16:30:56
===> Ended  : 16:37:13
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120123163056.log

See logs: prolepilog.tgz: prologue epilog logs

Shallow and deep resubmission

[capannini@devel15 WMS-service]$ python WMS-job-resubmit.py -c wms-command.conf -V dteam -d 3 -l -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS Job Resubmission Testing 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Test 1: Try a shallow resubmission 


 Test 2: Try a deep resubmission 

===> 
===> Test: WMS-job-resubmit.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 10:16:38
===> Ended  : 10:28:49
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120124101638.log
[capannini@devel15 WMS-service]$ python WMS-job-resubmit.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS Job Resubmission Testing 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Test 1: Try a shallow resubmission 


 Test 2: Try a deep resubmission 

===> 
===> Test: WMS-job-resubmit.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 10:32:35
===> Ended  : 10:45:18
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120124103235.log
[capannini@devel15 WMS-service]$ myproxy-init -d
Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini
Enter GRID pass phrase for this identity:
Creating proxy ............................................................................................................................................. Done
Proxy Verify OK
Your proxy is valid until: Tue Jan 31 10:39:47 2012
Enter MyProxy pass phrase:
Verifying - Enter MyProxy pass phrase:
A proxy valid for 168 hours (7.0 days) for user /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini now exists on myproxy.cnaf.infn.it.
[capannini@devel15 WMS-service]$ 
[capannini@devel15 WMS-service]$ 
[capannini@devel15 WMS-service]$ 
[capannini@devel15 WMS-service]$ myproxy-logon -d --voms dteam
Enter MyProxy pass phrase:
Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy
Contacting  voms.hellasgrid.gr:15004 [/C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr] "dteam" Done
Creating proxy ................................................................................................................................................. Done

Your proxy is valid until Tue Jan 24 21:40:11 2012
A credential has been received for user /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini in /tmp/x509up_u504.
[capannini@devel15 WMS-service]$ 
[capannini@devel15 WMS-service]$ 
[capannini@devel15 WMS-service]$ voms-proxy-info -all
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy/CN=proxy
issuer    : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy
identity  : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy
type      : proxy
strength  : 2048 bits
path      : /tmp/x509up_u504
timeleft  : 10:59:51
key usage : Digital Signature, Key Encipherment, Data Encipherment
=== VO dteam extension information ===
VO        : dteam
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini
issuer    : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr
attribute : /dteam/Role=NULL/Capability=NULL
attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL
timeleft  : 11:58:50
uri       : voms.hellasgrid.gr:15004
[capannini@devel15 WMS-service]$ python WMS-job-resubmit.py -c wms-command.conf -d 3 -l
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS Job Resubmission Testing 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

 Test 1: Try a shallow resubmission 


 Test 2: Try a deep resubmission 

===> 
===> Test: WMS-job-resubmit.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 10:40:39
===> Ended  : 10:53:15
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120124104039.log
See logs: resub.tgz: resubmission logs

Job cancel

[capannini@devel15 WMS-service]$ python WMS-job-cancel.py -c wms-command.conf -V dteam -d 3 -l  -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS Job Cancel Testing 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Test 1: Try to cancel a normal job 


 Test 2: Try to cancel a DAG job 


 Test 3: Try to cancel one node of a DAG job 


 Test 4: Try to cancel a collection job 


 Test 5: Try to cancel one node of a collection job 


 Test 6: Try to cancel a parametric job 


 Test 7: Try to cancel one node of a parametric job 


 Test 8: Try to cancel a MPI job 

===> 
===> Test: WMS-job-cancel.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 10:53:04
===> Ended  : 11:02:03
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120124105304.log
[capannini@devel15 WMS-service]$ python WMS-job-cancel.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l  -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS Job Cancel Testing 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Test 1: Try to cancel a normal job 


 Test 2: Try to cancel a DAG job 


 Test 3: Try to cancel one node of a DAG job 


 Test 4: Try to cancel a collection job 


 Test 5: Try to cancel one node of a collection job 


 Test 6: Try to cancel a parametric job 


 Test 7: Try to cancel one node of a parametric job 


 Test 8: Try to cancel a MPI job 

===> 
===> Test: WMS-job-cancel.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 10:43:32
===> Ended  : 10:52:08
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120124104332.log
[capannini@devel15 WMS-service]$ python WMS-job-cancel.py -c wms-command.conf -d 3 -l
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS Job Cancel Testing 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

 Test 1: Try to cancel a normal job 


 Test 2: Try to cancel a DAG job 


 Test 3: Try to cancel one node of a DAG job 


 Test 4: Try to cancel a collection job 


 Test 5: Try to cancel one node of a collection job 


 Test 6: Try to cancel a parametric job 


 Test 7: Try to cancel one node of a parametric job 


 Test 8: Try to cancel a MPI job 

===> 
===> Test: WMS-job-cancel.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 10:52:46
===> Ended  : 10:57:43
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120124105246.log

See logs: cancel.tgz: Job cancel logs

Proxy renewal

Setting MYPROXYSERVER=emitb2.ics.muni.cz in wms-command.conf

[capannini@devel15 ~]$ voms-proxy-init --voms dteam
Enter GRID pass phrase:
Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini
Creating temporary proxy .............................................. Done
Contacting  voms2.hellasgrid.gr:15004 [/C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr] "dteam" Done
Creating proxy ............................................... Done

Your proxy is valid until Thu Jan 26 22:42:50 2012

[capannini@devel15 ~]$ myproxy-init -s emitb2.ics.muni.cz -d -n
Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini
Enter GRID pass phrase for this identity:
Creating proxy .......................................................................................................................................................................................................................................................................... Done
Proxy Verify OK
Your proxy is valid until: Thu Feb  2 10:44:17 2012
A proxy valid for 168 hours (7.0 days) for user /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini now exists on emitb2.ics.muni.cz.

[capannini@devel15 WMS-service]$ python WMS-proxy-renewal.py -c wms-command.conf -V dteam -d 3 -l  -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS test proxy renewal operation 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Test 1: Test proxy renewal with LCG CE 


 Test 2: Test proxy renewal with CREAM CE 


 Test 3: Test proxy renewal with LCG CE (without setting MYPROXYSERVER) 


 Test 4: Test proxy renewal with CREAM CE (without setting MYPROXYSERVER) 

===> 
===> Test: WMS-proxy-renewal.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 10:47:52
===> Ended  : 11:29:27
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120126104752.log
See log: WMSService-TS_20120126104752.log: proxy renewal log

Configuration file

Verified that the following entries are present in /etc/glite-wms/glite_wms.conf :

ICE section

ICE =  [
    job_cancellation_threshold_time   =   300;
    max_logfile_size   =   100*1024*1024;
    cream_url_prefix   =   "https://";
    start_subscription_updater   =   true;
    max_ice_mem  =  2096000;
    InputType   =   "jobdir";
    subscription_update_threshold_time   =   3600;
    ice_log_level   =   700;
    listener_enable_authn   =   true;
    listener_enable_authz   =   true;
    poller_status_threshold_time   =   30*60;
    lease_update_frequency   =   20*60;
    lease_delta_time   =   0;
    cream_url_postfix   =   "/ce-cream/services/CREAM2";
    subscription_duration   =   86400;
    start_lease_updater   =   false;
    notification_frequency   =   3*60;
    creamdelegation_url_postfix   =   "/ce-cream/services/gridsite-delegation";
    ice_host_cert   =   "/home/glite/.certs/hostcert.pem";
    cemon_url_postfix   =   "/ce-monitor/services/CEMonitor";
    ice_topic   =   "CREAM_JOBS";
    start_proxy_renewer   =   true;
    Input   =   "/var/ice/jobdir";
    max_logfile_rotations   =   10;
    start_poller   =   true;
    listener_port   =   7010;
    poller_delay   =   2*60;
    creamdelegation_url_prefix   =   "https://";
    purge_jobs   =   false;
    start_listener   =   false;
    proxy_renewal_frequency   =   600;
    bulk_query_size   =   100;
    soap_timeout   =   60;
    persist_dir   =   "/var/ice/persist_dir";
    log_on_file  =  true;
    logfile   =   "${WMS_LOCATION_LOG}/ice.log";
    ice_host_key   =   "/home/glite/.certs/hostkey.pem";
    max_ice_threads   =   10;
    log_on_console  =  false;
    cemon_url_prefix   =   "https://";
    ice_empty_threshold  =  600;
    ];
WorkloadManagerProxy:
WorkloadManagerProxy =  [
    EnableServiceDiscovery  =  false;
    ListMatchRootPath  =  "/tmp";
    OperationLoadScripts =  [
          jobSubmit  =  "${WMS_LOCATION_SBIN}/glite_wms_wmproxy_load_monitor --oper jobSubmit --load1 22 --load5 20 --load15 18 --memusage 99
 --diskusage 95 --fdnum 1000 --jdnum 1500  --ftpconn 300";
          jobRegister  =  "${WMS_LOCATION_SBIN}/glite_wms_wmproxy_load_monitor --oper jobRegister --load1 22 --load5 20 --load15 18 --memusag
e 99 --diskusage 95 --fdnum 1000 --jdnum 1500 --ftpconn 300";
          ];
    MinPerusalTimeInterval  =  1000;
    MaxServedRequests  =  50;
    ServiceDiscoveryInfoValidity  =  3600;
    LBServer  =  {"emi-demo11.cnaf.infn.it:9000"};
    LBServiceDiscoveryType  =  "org.glite.lb.server";
    LogLevel  =  5;
    SandboxStagingPath  =  "${WMS_LOCATION_VAR}/SandboxDir";
    LBLocalLogger  =  "localhost:9002";
    WeightsCacheValidity  =  86400;
    LogFile  =  "${WMS_LOCATION_LOG}/wmproxy.log";
    AsyncJobStart  =  true;
    GridFTPPort  =  2811;
    MaxInputSandboxSize  =  100000000;
    ];
NetworkServer:
NetworkServer =  [
    II_Timeout  =  100;
    EnableQuotaManagement  =  false;
    QuotaInsensibleDiskPortion  =  2.0;
    SandboxStagingPath  =  "${WMS_LOCATION_VAR}/SandboxDir";
    II_DN  =  "mds-vo-name=local, o=grid";
    MasterThreads  =  8;
    ConnectionTimeout  =  300;
    LogFile  =  "${WMS_LOCATION_LOG}/networkserver_events.log";
    LogLevel  =  5;
    II_Contact  =  "cert-bdii-04.cnaf.infn.it";
    //II_Contact  =  "egee-bdii.cnaf.infn.it";
    II_Port   =  2170;
    QuotaAdjustmentAmount  =  10000;
    DispatcherThreads  =  10;
    Gris_DN  =  "mds-vo-name=local, o=grid";
    Gris_Port  =  2170;
    DLI_SI_CatalogTimeout  =  60;
    EnableDynamicQuotaAdjustment  =  false;
    ListMatchParadise  =  "${WMS_LOCATION_TMP}/MatchArea";
    Gris_Timeout  =  20;
    BacklogSize  =  64;
    ListeningPort  =  7772;
    MaxInputSandboxSize  =  10000000;
    ];
WorkloadManager:
WorkloadManager =  [
    DisablePurchasingFromGris  =  true;
    IsmBlackList  =  {};
    ExpiryPeriod  =  7200;
    PipeDepth  =  200;
    JobWrapperTemplateDir  =  "${WMS_JOBWRAPPER_TEMPLATE}";
    IsmDump  =  "${WMS_LOCATION_VAR}/workload_manager/ismdump.fl";
    IsmIiPurchasingRate  =  480;
    MatchRetryPeriod  =  600;
    MaxReplansCount  =  5;
    EnableRecovery  =  true;
    DispatcherType  =  "jobdir";
    LogLevel  =  5;
    LogFile   =  "${WMS_LOCATION_LOG}/workload_manager_events.log";
    DliServiceName  =  "data-location-interface";
    RuntimeMalloc  =  "/usr/lib64/libjemalloc.so.1";
    IsmUpdateRate  =  600;
    WmsRequirements   =  ((ShortDeadlineJob =?= TRUE) ? RegExp(".*sdj$", other.GlueCEUniqueID) : !RegExp(".*sdj$", other.GlueCEUniqueID)) && (other.GlueCEPolicyMaxTotalJobs == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs) && (EnableWmsFeedback =?= TRUE ? RegExp("cream", other.GlueCEImplementationName, "i") : true);
    CeMonitorAsynchPort  =  0;
    QueueSize  =  1000;
    EnableBulkMM  =  true;
    MaxRetryCount  =  10;
    SiServiceName  =  "org.glite.SEIndex";
    IsmThreads  =  false;
    MaxOutputSandboxSize  =  -1;
    CeForwardParameters  =  {"GlueHostMainMemoryVirtualSize","GlueHostMainMemoryRAMSize","GlueCEPolicyMaxCPUTime"};
    CeMonitorServices  =  {};
    IsmIiLDAPCEFilterExt  =  "(|(GlueCEAccessControlBaseRule=VO:atlas)(GlueCEAccessControlBaseRule=VOMS:/atlas/*)(GlueCEAccessControlBaseRule=VO:alice)(GlueCEAccessControlBaseRule=VOMS:/alice/*)(GlueCEAccessControlBaseRule=VO:lhcb)(GlueCEAccessControlBaseRule=VOMS:/lhcb/*)(GlueCEAccessControlBaseRule=VO:cms)(GlueCEAccessControlBaseRule=VOMS:/cms/*)(GlueCEAccessControlBaseRule=VO:dteam)(GlueCEAccessControlBaseRule=VOMS:/dteam/*)(GlueCEAccessControlBaseRule=VO:biomed)(GlueCEAccessControlBaseRule=VOMS:/biomed/*)(GlueCEAccessControlBaseRule=VO:ops)(GlueCEAccessControlBaseRule=VOMS:/ops/*)(GlueCEAccessControlBaseRule=VO:infngrid)(GlueCEAccessControlBaseRule=VOMS:/infngrid/*)(GlueCEAccessControlBaseRule=VO:testers.eu-emi.eu)(GlueCEAccessControlBaseRule=VOMS:/testers.eu-emi.eu/*))";
    ReplanGracePeriod  =  3600;
    Input  =  "${WMS_LOCATION_VAR}/workload_manager/jobdir";
    WorkerThreads  =  5;
    ];
Common section:
Common =  [
    HostProxyFile  =  "${WMS_LOCATION_VAR}/glite/wms.proxy";
    LBProxy  =  true;
    DGUser  =  "${GLITE_WMS_USER}";
    ];
Logmonitor section:
LogMonitor =  [
    CondorLogRecycleDir  =  "${WMS_LOCATION_VAR}/logmonitor/CondorG.log/recycle";
    LockFile  =  "${WMS_LOCATION_VAR}/logmonitor/lock";
    AbortedJobsTimeout  =  600;
    CondorLogDir  =  "${WMS_LOCATION_VAR}/logmonitor/CondorG.log";
    MonitorInternalDir  =  "${WMS_LOCATION_VAR}/logmonitor/internal";
    MainLoopDuration  =  5;
    GlobusDownTimeout  =  7200;
    LogLevel  =  5;
    JobsPerCondorLog  =  1000;
    LogFile  =  "${WMS_LOCATION_LOG}/logmonitor_events.log";
    ExternalLogFile  =  "${WMS_LOCATION_LOG}/logmonitor_external.log";
    RemoveJobFiles  =  true;
    IdRepositoryName  =  "irepository.dat";
    ForceCancellationRetries  =  2;
    ];
JobController section:
JobController =  [
    Input  =  "${WMS_LOCATION_VAR}/jobcontrol/jobdir/";
    ContainerRefreshThreshold  =  1000;
    CondorQuery  =  "${CONDORG_INSTALL_PATH}/bin/condor_q";
    LogFile  =  "${WMS_LOCATION_LOG}/jobcontoller_events.log";
    CondorDagman  =  "${CONDORG_INSTALL_PATH}/bin/condor_dagman";
    MaximumTimeAllowedForCondorMatch  =  1800;
    CondorRelease  =  "${CONDORG_INSTALL_PATH}/bin/condor_release";
    OutputFileDir  =  "${WMS_LOCATION_VAR}/jobcontrol/condorio";
    InputType  =  "jobdir";
    CondorRemove  =  "${CONDORG_INSTALL_PATH}/bin/condor_rm";
    DagmanMaxPre  =  10;
    LockFile  =  "${WMS_LOCATION_VAR}/jobcontrol/lock";
    SubmitFileDir  =  "${WMS_LOCATION_VAR}/jobcontrol/submit";
    CondorSubmit  =  "${CONDORG_INSTALL_PATH}/bin/condor_submit";
    LogLevel  =  5;
    ];
Verified that:

  • The attribute II_Contact of NetworkServer section matches the value of the yaim variable BDII_HOST
  • The attribute WMExpiryPeriod of WorkloadManager section matches the value of yaim variable WMS_EXPIRY_PERIOD
  • The attribute MatchRetryPeriod of WorkloadManager section matches the value of yaim variable WMS_MATCH_RETRY_PERIOD
  • The attribute IsmIiLDAPCEFilterExt of WorkloadManager section is (|(GlueCEAccessControlBaseRule=VO:vo1)(GlueCEAccessControlBaseRule=VOMS:/vo1/*)(GlueCEAccessControlBaseRule=VO=vo2...
  • The attribute LBServer of the WorkloadManagerProxy section matches the value of yaim variable LB_HOST

===> test PASSED

WMS feedback

TBD

Limiter mechanism

[capannini@devel15 WMS-service]$ python WMS-limiter-mechanism.py -c wms-command.conf -V dteam -d 3 -l -t 1,4,6-10
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS Limiter Mechanism 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

 Test 1: Test option --load1 threshold for load average (1 min) 


 Test 4: Test option --memusage threshold for memory usage (%) 


 Test 6: Test option --ftpconn threshold for number of FTP connections 


 Test 7: Test option --diskusage threshold for disk usage (%) 


 Test 8: Test option --fdnum threshold for used file descriptor 


 Test 9: Test option --jdnum threshold for number of unprocessed jobs (for jobdir) 


 Test 10: Test option --jdsize threshold for input jobdir size (KB) 

===> 
===> Test: WMS-limiter-mechanism.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 15:17:37
===> Ended  : 15:18:11
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120124151737.log
See log: WMSService-TS_20120124151737.log: limiter log

Purge

Normal job cycle
[capannini@devel15 WMS-service]$ python WMS-check-purger.py -c wms-command.conf -V dteam -d 3 -l
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS Purge Mechanism 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

 Test 1: Test purge , normal job cycle ( Submit to LCG CE ) 


 Test 2: Test purge, normal job cycle ( Submit to CREAM CE ) 


 Test 3: Test proxy cache purging 

===> 
===> Test: WMS-check-purger.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 12:07:54
===> Ended  : 12:11:47
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120125120754.log
See log: WMSService-TS_20120125120754.log: purge logs
SandboxDir cron purger
The purge of the SandBox dir is done by a cron job

- Submit a job and force its purge before it finishes:

[capannini@devel15 WMS-service]$ glite-wms-job-submit -a -c ~/wms_emi-demo11.conf ~/ls.jdl 

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://emi-demo11.cnaf.infn.it:9000/I-xR4xvAHsoHfhl7OplqLQ

==========================================================================


[capannini@devel15 WMS-service]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/I-xR4xvAHsoHfhl7OplqLQ


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/I-xR4xvAHsoHfhl7OplqLQ
Current Status:     Scheduled 
Status Reason:      unavailable
Destination:        ce208.cern.ch:8443/cream-lsf-grid_dteam
Submitted:          Wed Jan 25 12:26:31 2012 CET
==========================================================================

[root@emi-demo11 ~]# su - glite
[glite@emi-demo11 ~]$ 
[glite@emi-demo11 ~]$ /usr/sbin/glite-wms-purgeStorage.sh -p /var/SandboxDir/I- -s 
25 Jan, 12:27:33 -I: [Info] operator()(/home/condor/execute/dir_26089/userdir/emi.wms.wms-purger/build/../src/purger.cpp:447): https://emi-demo11.cnaf.infn.it:9000/I-xR4xvAHsoHfhl7OplqLQ: removed SCHEDULED job

[capannini@devel15 WMS-service]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/I-xR4xvAHsoHfhl7OplqLQ


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/I-xR4xvAHsoHfhl7OplqLQ
Current Status:     Cleared 
Status Reason:      timed out, resource purge forced
Destination:        ce208.cern.ch:8443/cream-lsf-grid_dteam
Submitted:          Wed Jan 25 12:26:31 2012 CET
==========================================================================

===>     === test PASSED === 

- Purge a done job:

[capannini@devel15 WMS-service]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/T7uROEeHs0MVTbeiMYDRJg


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/T7uROEeHs0MVTbeiMYDRJg
Current Status:     Done (Success)
Exit code:          0
Status Reason:      Job Terminated Successfully
Destination:        ce208.cern.ch:8443/cream-lsf-grid_dteam
Submitted:          Wed Jan 25 12:31:26 2012 CET
==========================================================================

[glite@emi-demo11 ~]$ /usr/sbin/glite-wms-purgeStorage.sh -p /var/SandboxDir/T7 -s 
25 Jan, 14:09:45 -I: [Info] operator()(/home/condor/execute/dir_26089/userdir/emi.wms.wms-purger/build/../src/purger.cpp:447): https://emi-demo11.cnaf.infn.it:9000/T7uROEeHs0MVTbeiMYDRJg: removed DONE job

[capannini@devel15 WMS-service]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/T7uROEeHs0MVTbeiMYDRJg


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/T7uROEeHs0MVTbeiMYDRJg
Current Status:     Cleared 
Status Reason:      timed out, resource purge forced
Destination:        ce208.cern.ch:8443/cream-lsf-grid_dteam
Submitted:          Wed Jan 25 12:31:26 2012 CET
==========================================================================

- Purge only jobs older than 12 hours:

Find some directories to be purged with the command:
# find /var/SandboxDir/ -maxdepth 2 -type d -cmin +720
/var/SandboxDir/W6/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fW65OfHAN_5fLrcvyoohMGzWQ
/var/SandboxDir/ez/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fezKOPmT5HcqCRjmO1ttSZA

Try to purge:

[glite@emi-demo11 ~]$ date
Wed Jan 25 14:15:03 CET 2012

[glite@emi-demo11 ~]$ ls -l /var/SandboxDir/ez
total 4
drwxrwx--- 5 dteam045 glite 4096 Jan 24 10:38 https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fezKOPmT5HcqCRjmO1ttSZA
[glite@emi-demo11 ~]$ /usr/sbin/glite-wms-purgeStorage.sh -p /var/SandboxDir/ez -t 43200 
25 Jan, 14:20:43 -I: [Info] operator()(/home/condor/execute/dir_26089/userdir/emi.wms.wms-purger/build/../src/purger.cpp:447): https://emi-demo11.cnaf.infn.it:9000/ezKOPmT5HcqCRjmO1ttSZA: removed ABORTED job
[glite@emi-demo11 ~]$ ls -l /var/SandboxDir/ez
total 0
[glite@emi-demo11 ~]$ ls -lr /var/SandboxDir/W6
total 4
drwxrwx--- 5 dteam045 glite 4096 Jan 24 14:24 https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fW65OfHAN_5fLrcvyoohMGzWQ
[glite@emi-demo11 ~]$ /usr/sbin/glite-wms-purgeStorage.sh -p /var/SandboxDir/W6 -t 43200 
25 Jan, 14:21:02 -I: [Info] operator()(/home/condor/execute/dir_26089/userdir/emi.wms.wms-purger/build/../src/purger.cpp:447): https://emi-demo11.cnaf.infn.it:9000/W65OfHAN_LrcvyoohMGzWQ: removed DONE job
[glite@emi-demo11 ~]$ ls -lr /var/SandboxDir/W6
total 0


Find some directories that should not be purged:
# find /var/SandboxDir/ -maxdepth 2 -type d -cmin -240
/var/SandboxDir/St/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fStNSoiVR3S47qnaFFHEOzw
/var/SandboxDir/T5/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fT5gK96CZD8DbV8Oi1hSNcA

Try to purge:

[glite@emi-demo11 ~]$ date
Wed Jan 25 14:15:03 CET 2012

[glite@emi-demo11 ~]$ /usr/sbin/glite-wms-purgeStorage.sh -p /var/SandboxDir/T5 -t 43200 
[glite@emi-demo11 ~]$ ls -l /var/SandboxDir/T5
total 4
drwxrwx--- 5 dteam045 glite 4096 Jan 25 10:29 https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fT5gK96CZD8DbV8Oi1hSNcA
[glite@emi-demo11 ~]$ /usr/sbin/glite-wms-purgeStorage.sh -p /var/SandboxDir/St -t 43200 
[glite@emi-demo11 ~]$ ls -l /var/SandboxDir/St
total 4
drwxrwx--- 5 dteam045 glite 4096 Jan 25 12:59 https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fStNSoiVR3S47qnaFFHEOzw

===> Test PASSED

- Purge a job whose proxy is already expired

[glite@emi-demo11 ~]$ openssl x509 -in /var/SandboxDir/z6/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fz6gxfYkbU1h8hZIyOXEOUg/user.proxy -noout -dates
notBefore=Jan 17 15:52:12 2012 GMT
notAfter=Jan 17 21:19:12 2012 GMT

[glite@emi-demo11 ~]$ date
Wed Jan 25 15:18:49 CET 2012

[capannini@devel15 WMS-service]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/z6gxfYkbU1h8hZIyOXEOUg


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/z6gxfYkbU1h8hZIyOXEOUg
Current Status:     Submitted 
Submitted:          Tue Jan 17 16:57:12 2012 CET
Parent Job:         https://emi-demo11.cnaf.infn.it:9000/79cGVuy9XVzQxwvn_Z2Hgw
==========================================================================

[glite@emi-demo11 ~]$ /usr/sbin/glite-wms-purgeStorage.sh -p /var/SandboxDir/z6 -s
25 Jan, 15:21:10 -I: [Info] operator()(/home/condor/execute/dir_26089/userdir/emi.wms.wms-purger/build/../src/purger.cpp:447): https://emi-demo11.cnaf.infn.it:9000/z6gxfYkbU1h8hZIyOXEOUg: removed SUBMITTED job

[capannini@devel15 WMS-service]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/z6gxfYkbU1h8hZIyOXEOUg


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/z6gxfYkbU1h8hZIyOXEOUg
Current Status:     Cleared 
Status Reason:      timed out, resource purge forced
Submitted:          Tue Jan 17 16:57:12 2012 CET
Parent Job:         https://emi-demo11.cnaf.infn.it:9000/79cGVuy9XVzQxwvn_Z2Hgw
==========================================================================

===>     === test PASSED ===

- Purge a dag job

[capannini@devel15 dag]$ glite-wms-job-submit -a -c ~/wms_emi-demo11.conf dag.jdl 

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://emi-demo11.cnaf.infn.it:9000/Ab_cMtMLDPkCNKsL-5CFaw

==========================================================================

[capannini@devel15 dag]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/Ab_cMtMLDPkCNKsL-5CFaw


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/Ab_cMtMLDPkCNKsL-5CFaw
Current Status:     Running 
Status Reason:      unavailable
Destination:        dagman
Submitted:          Wed Jan 25 15:33:46 2012 CET
==========================================================================

- Nodes information for: 
    Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/HnZfPHnH3xgxmhl_NrRr-g
    Current Status:     Submitted 
    Submitted:          Wed Jan 25 15:33:46 2012 CET
==========================================================================
    
    Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/X2-G_Tr9eyWgs88vjfnL2g
    Current Status:     Submitted 
    Submitted:          Wed Jan 25 15:33:46 2012 CET
==========================================================================
    
    Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/Yc4eO097SXSBrXIvhsIqyw
    Current Status:     Submitted 
    Submitted:          Wed Jan 25 15:33:46 2012 CET
==========================================================================
    
    Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/b8bBqrceSFYU1mWEfpH-fQ
    Current Status:     Submitted 
    Submitted:          Wed Jan 25 15:33:46 2012 CET
==========================================================================
    
    Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/hGHl6d1B1D8xPAmNXdLdsw
    Current Status:     Submitted 
    Submitted:          Wed Jan 25 15:33:46 2012 CET
==========================================================================

[glite@emi-demo11 ~]$ /usr/sbin/glite-wms-purgeStorage.sh -p /var/SandboxDir/Ab -s25 Jan, 15:34:30 -I: [Info] operator()(/home/condor/execute/dir_26089/userdir/emi.wms.wms-purger/build/../src/purger.cpp:354): https://emi-demo11.cnaf.infn.it:9000/Ab_cMtMLDPkCNKsL-5CFaw: 5/5 nodes removed
25 Jan, 15:34:30 -I: [Info] operator()(/home/condor/execute/dir_26089/userdir/emi.wms.wms-purger/build/../src/purger.cpp:366): https://emi-demo11.cnaf.infn.it:9000/Ab_cMtMLDPkCNKsL-5CFaw: removed RUNNING dag 

[capannini@devel15 dag]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/Ab_cMtMLDPkCNKsL-5CFaw


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/Ab_cMtMLDPkCNKsL-5CFaw
Current Status:     Cleared 
Status Reason:      timed out, resource purge forced
Destination:        dagman
Submitted:          Wed Jan 25 15:33:46 2012 CET
==========================================================================

- Nodes information for: 
    Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/HnZfPHnH3xgxmhl_NrRr-g
    Current Status:     Cleared 
    Status Reason:      timed out, resource purge forced
    Submitted:          Wed Jan 25 15:33:46 2012 CET
==========================================================================
    
    Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/X2-G_Tr9eyWgs88vjfnL2g
    Current Status:     Cleared 
    Status Reason:      timed out, resource purge forced
    Submitted:          Wed Jan 25 15:33:46 2012 CET
==========================================================================
    
    Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/Yc4eO097SXSBrXIvhsIqyw
    Current Status:     Cleared 
    Status Reason:      timed out, resource purge forced
    Submitted:          Wed Jan 25 15:33:46 2012 CET
==========================================================================
    
    Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/b8bBqrceSFYU1mWEfpH-fQ
    Current Status:     Cleared 
    Status Reason:      timed out, resource purge forced
    Submitted:          Wed Jan 25 15:33:46 2012 CET
==========================================================================
    
    Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/hGHl6d1B1D8xPAmNXdLdsw
    Current Status:     Cleared 
    Status Reason:      timed out, resource purge forced
    Destination:        argoce01.na.infn.it:2119/jobmanager-lcgpbs-cert
    Submitted:          Wed Jan 25 15:33:46 2012 CET
==========================================================================

===>     === test PASSED === 

Published information

bdb database information

[capannini@devel15 ~]$ ldapsearch -x -H ldap://emi-demo11.cnaf.infn.it:2170 -b 'o=grid'
# extended LDIF
#
# LDAPv3
# base <o=grid> with scope subtree
# filter: (objectclass=*)
# requesting: ALL
#

# grid
dn: o=grid
objectClass: organization
o: grid

# local, grid
dn: Mds-Vo-name=local,o=grid
Mds-Vo-name: local
objectClass: Mds

# resource, grid
dn: Mds-Vo-name=resource,o=grid
Mds-Vo-name: resource
objectClass: Mds

# emi-demo11.cnaf.infn.it_org.glite.lb.Server_2310828617, resource, grid
dn: GlueServiceUniqueID=emi-demo11.cnaf.infn.it_org.glite.lb.Server_2310828617
 ,Mds-Vo-name=resource,o=grid
objectClass: GlueTop
objectClass: GlueService
objectClass: GlueKey
objectClass: GlueSchemaVersion
GlueServiceUniqueID: emi-demo11.cnaf.infn.it_org.glite.lb.Server_2310828617
GlueServiceEndpoint: https://emi-demo11.cnaf.infn.it:9003/
GlueServiceVersion: 2.2.10
GlueSchemaVersionMinor: 3
GlueServiceName: INFN-CREAMTEST-Server
GlueServiceType: org.glite.lb.Server
GlueServiceWSDL: http://egee.cesnet.cz/cms/export/sites/egee/en/WSDL/3.1/LB.ws
 dl
GlueServiceSemantics: https://edms.cern.ch/file/571273/2/LB-guide.pdf
GlueForeignKey: GlueSiteUniqueID=INFN-CREAMTEST
GlueSchemaVersionMajor: 1
GlueServiceStatus: OK
GlueServiceStartTime: 2012-01-19T12:26:49+01:00
GlueServiceStatusInfo: glite-lb-notif-interlogd running as 9582 glite-lb-inter
 logd for proxy running as 9617 glite-lb-bkserverd running as 9536 glite-jp-im
 porter not running (disabled)

# emi-demo11.cnaf.infn.it_org.glite.wms.WMProxy_841415637, resource, grid
dn: GlueServiceUniqueID=emi-demo11.cnaf.infn.it_org.glite.wms.WMProxy_84141563
 7,Mds-Vo-name=resource,o=grid
GlueServiceAccessControlBaseRule: VO:atlas
GlueServiceAccessControlBaseRule: VO:alice
GlueServiceAccessControlBaseRule: VO:lhcb
GlueServiceAccessControlBaseRule: VO:cms
GlueServiceAccessControlBaseRule: VO:dteam
GlueServiceAccessControlBaseRule: VO:biomed
GlueServiceAccessControlBaseRule: VO:ops
GlueServiceAccessControlBaseRule: VO:infngrid
GlueServiceAccessControlBaseRule: VO:testers.eu-emi.eu
GlueServiceStatus: Other
GlueServiceStatusInfo: ??
objectClass: GlueTop
objectClass: GlueService
objectClass: GlueKey
objectClass: GlueSchemaVersion
GlueServiceUniqueID: emi-demo11.cnaf.infn.it_org.glite.wms.WMProxy_841415637
GlueServiceAccessControlRule: atlas
GlueServiceAccessControlRule: alice
GlueServiceAccessControlRule: lhcb
GlueServiceAccessControlRule: cms
GlueServiceAccessControlRule: dteam
GlueServiceAccessControlRule: biomed
GlueServiceAccessControlRule: ops
GlueServiceAccessControlRule: infngrid
GlueServiceAccessControlRule: testers.eu-emi.eu
GlueServiceEndpoint: https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_se
 rver
GlueServiceVersion: 3.3.4
GlueSchemaVersionMinor: 3
GlueServiceName: INFN-CREAMTEST-WMProxy
GlueServiceType: org.glite.wms.WMProxy
GlueServiceWSDL: http://web.infn.it/gLiteWMS/images/WMS/Docs/wmproxy.wsdl
GlueServiceSemantics: http://web.infn.it/gLiteWMS/images/WMS/Docs/wmproxy-guid
 e.pdf
GlueForeignKey: GlueSiteUniqueID=INFN-CREAMTEST
GlueServiceOwner: atlas
GlueServiceOwner: alice
GlueServiceOwner: lhcb
GlueServiceOwner: cms
GlueServiceOwner: dteam
GlueServiceOwner: biomed
GlueServiceOwner: ops
GlueServiceOwner: infngrid
GlueServiceOwner: testers.eu-emi.eu
GlueSchemaVersionMajor: 1
GlueServiceStartTime: 2012-01-26T00:00:02+01:00

# DN, emi-demo11.cnaf.infn.it_org.glite.wms.WMProxy_841415637, resource, grid
dn: GlueServiceDataKey=DN,GlueServiceUniqueID=emi-demo11.cnaf.infn.it_org.glit
 e.wms.WMProxy_841415637,Mds-Vo-name=resource,o=grid
GlueSchemaVersionMajor: 1
objectClass: GlueTop
objectClass: GlueServiceData
objectClass: GlueKey
objectClass: GlueSchemaVersion
GlueSchemaVersionMinor: 3
GlueServiceDataKey: DN
GlueChunkKey: GlueServiceUniqueID=emi-demo11.cnaf.infn.it_org.glite.wms.WMProx
 y_841415637
GlueServiceDataValue: /C=IT/O=INFN/OU=Host/L=CNAF/CN=emi-demo11.cnaf.infn.it

# glite-info-service_version, emi-demo11.cnaf.infn.it_org.glite.lb.Server_231
 0828617, resource, grid
dn: GlueServiceDataKey=glite-info-service_version,GlueServiceUniqueID=emi-demo
 11.cnaf.infn.it_org.glite.lb.Server_2310828617,Mds-Vo-name=resource,o=grid
GlueSchemaVersionMajor: 1
objectClass: GlueTop
objectClass: GlueServiceData
objectClass: GlueKey
objectClass: GlueSchemaVersion
GlueSchemaVersionMinor: 3
GlueServiceDataKey: glite-info-service_version
GlueChunkKey: GlueServiceUniqueID=emi-demo11.cnaf.infn.it_org.glite.lb.Server_
 2310828617
GlueServiceDataValue: 1.6

# glite-info-service_version, emi-demo11.cnaf.infn.it_org.glite.wms.WMProxy_8
 41415637, resource, grid
dn: GlueServiceDataKey=glite-info-service_version,GlueServiceUniqueID=emi-demo
 11.cnaf.infn.it_org.glite.wms.WMProxy_841415637,Mds-Vo-name=resource,o=grid
GlueSchemaVersionMajor: 1
objectClass: GlueTop
objectClass: GlueServiceData
objectClass: GlueKey
objectClass: GlueSchemaVersion
GlueSchemaVersionMinor: 3
GlueServiceDataKey: glite-info-service_version
GlueChunkKey: GlueServiceUniqueID=emi-demo11.cnaf.infn.it_org.glite.wms.WMProx
 y_841415637
GlueServiceDataValue: 1.6

# glite-info-service_hostname, emi-demo11.cnaf.infn.it_org.glite.lb.Server_23
 10828617, resource, grid
dn: GlueServiceDataKey=glite-info-service_hostname,GlueServiceUniqueID=emi-dem
 o11.cnaf.infn.it_org.glite.lb.Server_2310828617,Mds-Vo-name=resource,o=grid
GlueSchemaVersionMajor: 1
objectClass: GlueTop
objectClass: GlueServiceData
objectClass: GlueKey
objectClass: GlueSchemaVersion
GlueSchemaVersionMinor: 3
GlueServiceDataKey: glite-info-service_hostname
GlueChunkKey: GlueServiceUniqueID=emi-demo11.cnaf.infn.it_org.glite.lb.Server_
 2310828617
GlueServiceDataValue: emi-demo11.cnaf.infn.it

# glite-info-service_hostname, emi-demo11.cnaf.infn.it_org.glite.wms.WMProxy_
 841415637, resource, grid
dn: GlueServiceDataKey=glite-info-service_hostname,GlueServiceUniqueID=emi-dem
 o11.cnaf.infn.it_org.glite.wms.WMProxy_841415637,Mds-Vo-name=resource,o=grid
GlueSchemaVersionMajor: 1
objectClass: GlueTop
objectClass: GlueServiceData
objectClass: GlueKey
objectClass: GlueSchemaVersion
GlueSchemaVersionMinor: 3
GlueServiceDataKey: glite-info-service_hostname
GlueChunkKey: GlueServiceUniqueID=emi-demo11.cnaf.infn.it_org.glite.wms.WMProx
 y_841415637
GlueServiceDataValue: emi-demo11.cnaf.infn.it

# search result
search: 2
result: 0 Success

# numResponses: 11
# numEntries: 10

Stats database definitions

[capannini@devel15 ~]$ ldapsearch -x -H ldap://emi-demo11.cnaf.infn.it:2170 -b 'o=infosys'
# extended LDIF
#
# LDAPv3
# base <o=infosys> with scope subtree
# filter: (objectclass=*)
# requesting: ALL
#

# infosys
dn: o=infosys
objectClass: organization
o: infosys

# zip, infosys
dn: CompressionType=zip,o=infosys
objectClass: CompressedContent
Hostname: localhost
CompressionType: zip
Data: file:///var/run/bdii/archive/12-01-26-11-02-24-snapshot.gz

# localhost, infosys
dn: Hostname=localhost,o=infosys
objectClass: UpdateStats
Hostname: localhost
FailedDeletes: 0
ModifiedEntries: 8
DeletedEntries: 0
UpdateTime: 1
FailedAdds: 0
FailedModifies: 0
TotalEntries: 20
QueryTime: 0
NewEntries: 1
DBUpdateTime: 0
ReadTime: 0
PluginsTime: 0
ProvidersTime: 1

# search result
search: 2
result: 0 Success

# numResponses: 4
# numEntries: 3

Stats database definitions

[capannini@devel15 ~]$ ldapsearch -x -H ldap://emi-demo11.cnaf.infn.it:2170 -b 'o=glue'
# extended LDIF
#
# LDAPv3
# base <o=glue> with scope subtree
# filter: (objectclass=*)
# requesting: ALL
#

# glue
dn: o=glue
objectClass: organization
o: glue

# grid, glue
dn: GLUE2GroupID=grid,o=glue
objectClass: GLUE2Group
GLUE2GroupID: grid

# resource, glue
dn: GLUE2GroupID=resource,o=glue
objectClass: GLUE2Group
GLUE2GroupID: resource

# emi-demo11.cnaf.infn.it_lbserver_201164012, resource, glue
dn: GLUE2ServiceID=emi-demo11.cnaf.infn.it_lbserver_201164012,GLUE2GroupID=res
 ource,o=glue
GLUE2EntityValidity: 3600
GLUE2ServiceType: org.glite.lb.Server
GLUE2ServiceID: emi-demo11.cnaf.infn.it_lbserver_201164012
objectClass: GLUE2Service
GLUE2ServiceQualityLevel: production
GLUE2EntityOtherInfo: InfoProviderName=glite-info-glue2-service
GLUE2EntityOtherInfo: InfoProviderVersion=1.1
GLUE2EntityOtherInfo: InfoProviderHost=emi-demo11.cnaf.infn.it
GLUE2ServiceComplexity: endpointType=1, share=0, resource=0
GLUE2ServiceCapability: information.logging
GLUE2EntityName: INFN-CREAMTEST-Server
GLUE2ServiceAdminDomainForeignKey: INFN-CREAMTEST
GLUE2EntityCreationTime: 2012-01-26T10:04:25Z

# emi-demo11.cnaf.infn.it_wmproxy_2598325561, resource, glue
dn: GLUE2ServiceID=emi-demo11.cnaf.infn.it_wmproxy_2598325561,GLUE2GroupID=res
 ource,o=glue
GLUE2EntityValidity: 3600
GLUE2ServiceType: org.glite.wms.WMProxy
GLUE2ServiceID: emi-demo11.cnaf.infn.it_wmproxy_2598325561
objectClass: GLUE2Service
GLUE2ServiceQualityLevel: production
GLUE2EntityOtherInfo: InfoProviderName=glite-info-glue2-service
GLUE2EntityOtherInfo: InfoProviderVersion=1.1
GLUE2EntityOtherInfo: InfoProviderHost=emi-demo11.cnaf.infn.it
GLUE2ServiceComplexity: endpointType=1, share=0, resource=0
GLUE2ServiceCapability: executionmanagement.candidatesetgenerator
GLUE2ServiceCapability: executionmanagement.jobdescription
GLUE2ServiceCapability: executionmanagement.jobmanager
GLUE2EntityName: INFN-CREAMTEST-WMProxy
GLUE2ServiceAdminDomainForeignKey: INFN-CREAMTEST
GLUE2EntityCreationTime: 2012-01-26T10:04:26Z

# emi-demo11.cnaf.infn.it_lbserver_201164012_org.glite.lb.Server_201164012, e
 mi-demo11.cnaf.infn.it_lbserver_201164012, resource, glue
dn: GLUE2EndpointID=emi-demo11.cnaf.infn.it_lbserver_201164012_org.glite.lb.Se
 rver_201164012,GLUE2ServiceID=emi-demo11.cnaf.infn.it_lbserver_201164012,GLUE
 2GroupID=resource,o=glue
GLUE2EntityValidity: 3600
GLUE2EndpointQualityLevel: production
GLUE2EndpointImplementor: gLite
GLUE2EntityOtherInfo: InfoProviderName=glite-info-glue2-endpoint
GLUE2EntityOtherInfo: InfoProviderVersion=1.2
GLUE2EntityOtherInfo: InfoProviderHost=emi-demo11.cnaf.infn.it
GLUE2EntityOtherInfo: MiddlewareName=EMI
GLUE2EntityOtherInfo: MiddlewareVersion=1.7.0-1
GLUE2EndpointCapability: information.logging
GLUE2EndpointServiceForeignKey: emi-demo11.cnaf.infn.it_lbserver_201164012
GLUE2EntityName: Server endpoint for Service emi-demo11.cnaf.infn.it_lbserver_
 201164012
GLUE2EndpointTechnology: webservice
GLUE2EndpointWSDL: http://egee.cesnet.cz/cms/export/sites/egee/en/WSDL/3.1/LB.
 wsdl
GLUE2EndpointInterfaceName: org.glite.lb.Server
GLUE2EndpointURL: https://emi-demo11.cnaf.infn.it:9003/
GLUE2EndpointDowntimeInfo: See the GOC DB for downtimes: https://goc.egi.eu/
GLUE2EndpointImplementationVersion: 2.2.10
GLUE2EndpointSemantics: https://edms.cern.ch/file/571273/2/LB-guide.pdf
GLUE2EndpointIssuerCA: /C=IT/O=INFN/CN=INFN CA
GLUE2EndpointServingState: production
objectClass: GLUE2Endpoint
GLUE2EndpointInterfaceVersion: 3.2.4
GLUE2EndpointImplementationName: LB
GLUE2EndpointTrustedCA: IGTF
GLUE2EndpointID: emi-demo11.cnaf.infn.it_lbserver_201164012_org.glite.lb.Serve
 r_201164012
GLUE2EndpointStartTime: 2012-01-19T11:26:49Z
GLUE2EndpointHealthStateInfo: glite-lb-notif-interlogd running as 9582 glite-l
 b-interlogd for proxy running as 9617 glite-lb-bkserverd running as 9536 glit
 e-jp-importer not running (disabled)
GLUE2EndpointHealthState: ok
GLUE2EntityCreationTime: 2012-01-26T10:04:25Z

# emi-demo11.cnaf.infn.it_wmproxy_2598325561_org.glite.wms.WMProxy_2598325561
 , emi-demo11.cnaf.infn.it_wmproxy_2598325561, resource, glue
dn: GLUE2EndpointID=emi-demo11.cnaf.infn.it_wmproxy_2598325561_org.glite.wms.W
 MProxy_2598325561,GLUE2ServiceID=emi-demo11.cnaf.infn.it_wmproxy_2598325561,G
 LUE2GroupID=resource,o=glue
GLUE2EntityValidity: 3600
GLUE2EndpointQualityLevel: production
GLUE2EndpointImplementor: gLite
GLUE2EntityOtherInfo: InfoProviderName=glite-info-glue2-endpoint
GLUE2EntityOtherInfo: InfoProviderVersion=1.2
GLUE2EntityOtherInfo: InfoProviderHost=emi-demo11.cnaf.infn.it
GLUE2EntityOtherInfo: MiddlewareName=EMI
GLUE2EntityOtherInfo: MiddlewareVersion=1.7.0-1
GLUE2EndpointCapability: executionmanagement.candidatesetgenerator
GLUE2EndpointCapability: executionmanagement.jobdescription
GLUE2EndpointCapability: executionmanagement.jobmanager
GLUE2EndpointHealthState: ok
GLUE2EndpointServiceForeignKey: emi-demo11.cnaf.infn.it_wmproxy_2598325561
GLUE2EntityName: WMProxy endpoint for Service emi-demo11.cnaf.infn.it_wmproxy_
 2598325561
GLUE2EndpointTechnology: webservice
GLUE2EndpointWSDL: http://web.infn.it/gLiteWMS/images/WMS/Docs/wmproxy.wsdl
GLUE2EndpointInterfaceName: org.glite.wms.WMProxy
GLUE2EndpointURL: https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_serve
 r
GLUE2EndpointDowntimeInfo: See the GOC DB for downtimes: https://goc.egi.eu/
GLUE2EndpointImplementationVersion: 3.3.4
GLUE2EndpointSemantics: http://web.infn.it/gLiteWMS/images/WMS/Docs/wmproxy-gu
 ide.pdf
GLUE2EndpointIssuerCA: /C=IT/O=INFN/CN=INFN CA
GLUE2EndpointServingState: production
objectClass: GLUE2Endpoint
GLUE2EndpointInterfaceVersion: 1.0
GLUE2EndpointImplementationName: WMS
GLUE2EndpointTrustedCA: IGTF
GLUE2EndpointID: emi-demo11.cnaf.infn.it_wmproxy_2598325561_org.glite.wms.WMPr
 oxy_2598325561
GLUE2EndpointStartTime: 2012-01-25T23:00:02Z
GLUE2EndpointHealthStateInfo: WMProxy httpd listening on port 7443 httpd is ru
 nning .... === WMProxy Server running instances: UID PID PPID C STIME TTY TIM
 E CMD glite 13421 30902 0 09:28 ? 00:00:00 /usr/bin/glite_wms_wmproxy_server
GLUE2EntityCreationTime: 2012-01-26T10:04:26Z

# DN_1, emi-demo11.cnaf.infn.it_wmproxy_2598325561_org.glite.wms.WMProxy_2598
 325561, emi-demo11.cnaf.infn.it_wmproxy_2598325561, resource, glue
dn: GLUE2ExtensionLocalID=DN_1,GLUE2EndpointID=emi-demo11.cnaf.infn.it_wmproxy
 _2598325561_org.glite.wms.WMProxy_2598325561,GLUE2ServiceID=emi-demo11.cnaf.i
 nfn.it_wmproxy_2598325561,GLUE2GroupID=resource,o=glue
GLUE2ExtensionLocalID: DN_1
GLUE2ExtensionKey: DN
objectClass: GLUE2Extension
GLUE2ExtensionValue: /C=IT/O=INFN/OU=Host/L=CNAF/CN=emi-demo11.cnaf.infn.it
GLUE2ExtensionEntityForeignKey: emi-demo11.cnaf.infn.it_wmproxy_2598325561_org
 .glite.wms.WMProxy_2598325561

# emi-demo11.cnaf.infn.it_lbserver_201164012_org.glite.lb.Server_201164012_Po
 licy, emi-demo11.cnaf.infn.it_lbserver_201164012_org.glite.lb.Server_2011640
 12, emi-demo11.cnaf.infn.it_lbserver_201164012, resource, glue
dn: GLUE2PolicyID=emi-demo11.cnaf.infn.it_lbserver_201164012_org.glite.lb.Serv
 er_201164012_Policy,GLUE2EndpointID=emi-demo11.cnaf.infn.it_lbserver_20116401
 2_org.glite.lb.Server_201164012,GLUE2ServiceID=emi-demo11.cnaf.infn.it_lbserv
 er_201164012,GLUE2GroupID=resource,o=glue
GLUE2EntityValidity: 3600
objectClass: GLUE2Policy
objectClass: GLUE2AccessPolicy
GLUE2PolicyRule: ALL
GLUE2PolicyScheme: org.glite.standard
GLUE2EntityOtherInfo: InfoProviderName=glite-info-glue2-endpoint
GLUE2EntityOtherInfo: InfoProviderVersion=1.2
GLUE2EntityOtherInfo: InfoProviderHost=emi-demo11.cnaf.infn.it
GLUE2AccessPolicyEndpointForeignKey: emi-demo11.cnaf.infn.it_lbserver_20116401
 2_org.glite.lb.Server_201164012
GLUE2PolicyID: emi-demo11.cnaf.infn.it_lbserver_201164012_org.glite.lb.Server_
 201164012_Policy
GLUE2EntityName: Access control rules for Endpoint emi-demo11.cnaf.infn.it_lbs
 erver_201164012_org.glite.lb.Server_201164012
GLUE2EntityCreationTime: 2012-01-26T10:04:25Z

# emi-demo11.cnaf.infn.it_wmproxy_2598325561_org.glite.wms.WMProxy_2598325561
 _Policy, emi-demo11.cnaf.infn.it_wmproxy_2598325561_org.glite.wms.WMProxy_25
 98325561, emi-demo11.cnaf.infn.it_wmproxy_2598325561, resource, glue
dn: GLUE2PolicyID=emi-demo11.cnaf.infn.it_wmproxy_2598325561_org.glite.wms.WMP
 roxy_2598325561_Policy,GLUE2EndpointID=emi-demo11.cnaf.infn.it_wmproxy_259832
 5561_org.glite.wms.WMProxy_2598325561,GLUE2ServiceID=emi-demo11.cnaf.infn.it_
 wmproxy_2598325561,GLUE2GroupID=resource,o=glue
GLUE2AccessPolicyEndpointForeignKey: emi-demo11.cnaf.infn.it_wmproxy_259832556
 1_org.glite.wms.WMProxy_2598325561
GLUE2EntityValidity: 3600
objectClass: GLUE2Policy
objectClass: GLUE2AccessPolicy
GLUE2PolicyRule: VO:atlas
GLUE2PolicyRule: VO:alice
GLUE2PolicyRule: VO:lhcb
GLUE2PolicyRule: VO:cms
GLUE2PolicyRule: VO:dteam
GLUE2PolicyRule: VO:biomed
GLUE2PolicyRule: VO:ops
GLUE2PolicyRule: VO:infngrid
GLUE2PolicyRule: VO:testers.eu-emi.eu
GLUE2PolicyScheme: org.glite.standard
GLUE2EntityOtherInfo: InfoProviderName=glite-info-glue2-endpoint
GLUE2EntityOtherInfo: InfoProviderVersion=1.2
GLUE2EntityOtherInfo: InfoProviderHost=emi-demo11.cnaf.infn.it
GLUE2PolicyUserDomainForeignKey: atlas
GLUE2PolicyUserDomainForeignKey: alice
GLUE2PolicyUserDomainForeignKey: lhcb
GLUE2PolicyUserDomainForeignKey: cms
GLUE2PolicyUserDomainForeignKey: dteam
GLUE2PolicyUserDomainForeignKey: biomed
GLUE2PolicyUserDomainForeignKey: ops
GLUE2PolicyUserDomainForeignKey: infngrid
GLUE2PolicyUserDomainForeignKey: testers.eu-emi.eu
GLUE2PolicyID: emi-demo11.cnaf.infn.it_wmproxy_2598325561_org.glite.wms.WMProx
 y_2598325561_Policy
GLUE2EntityName: Access control rules for Endpoint emi-demo11.cnaf.infn.it_wmp
 roxy_2598325561_org.glite.wms.WMProxy_2598325561
GLUE2EntityCreationTime: 2012-01-26T10:04:26Z

# search result
search: 2
result: 0 Success

# numResponses: 11
# numEntries: 10

Performance test

The tests are carried out using emi-demo11, a virtual machine with wms installed, as this has never been tried before. The real certification process should use devel20, a real machine with wms installed, instead.

Collection of 1000 nodes

Submission of a collection of 1000 nodes
[capannini@cert-17 StressTest]$ ./StressTest.sh -t 1 -n 1000 -c ~/wms_emi-demo11.conf -j ls.jdl
Test starts on mer feb 1 11:58:27 CET 2012: submit 1 collections (of 1000 nodes) with a frequency of 60 seconds using 1 users.

Submission n. 1:
mer feb 1 11:58:27 CET 2012: user 0 is submitting...

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://emi-demo11.cnaf.infn.it:9000/Cw4IByQC5DYhcHcEguYxuQ

The job identifier has been saved in the following file:
/home/capannini/certification/StressTest/Test_1328093897/output/USER0_0.jobid

==========================================================================


Submission takes 7 seconds.
1 jobs submitted in 7 seconds: 7/7/7 (min/avg/max)
Test finishes at mer feb 1 11:58:34 CET 2012

The collection stays in status 'Waiting'.

======> Test NOT PASSED

Stress test

  • 2880 collection of 25 nodes each (total: 72000 jobs)
  • One collection every 60 seconds
  • one user
  • Use LCG-CEs and CREAM-CEs (with different batch systems)
  • Use automatic-delegation
  • The job is a `ls -l`
  • Resubmission is enabled
  • Enable proxy renewal

[capannini@cert-17 StressTest]$ ./StressTest.sh -t 2880 -n 25 -c ~/wms_emi-demo11.conf -j ls.jdl
Test starts on mer feb 1 14:19:49 CET 2012: submit 2880 collections (of 25 nodes) with a frequency of 60 seconds using 1 users.

Results

[capannini@cert-17 output]$ ../../loginfo.sh -e 54

 Results:
DONE   :      47192 
NOTDone:          8 
Aborted:       1775 
------------------------
Total  :      48975 

done jobs on CREAM CEs: 29819 (63%)
done jobs on LCG CEs: 17373 (37%)

The above statistics is obatined in the following way:
[capannini@cert-17 output]$ cat *.done|wc -l
47192
[capannini@cert-17 output]$ cat *.done|grep '8443/cream' |wc -l
29819
[capannini@cert-17 output]$ cat *.done|grep '2119/jobmanager'|wc -l
17373


====> Test NOT PASSED (Aborted/Total = 3.6% probably too high percentage, but a general rule should be agreed upon)

Regression tests

Bug #81376 (glite-wms-wmproxy.restart.cron MUST support graceful wmp restart) FIXED

[capannini@devel15 regression_tests]$ python execute_regTest.py -d 3 -b 81376 -l
+++++++++++++++++++++++++++++++++++++++++++++++++++++
+ Regression TestSuite ....                         +
+++++++++++++++++++++++++++++++++++++++++++++++++++++

 Testing bug 81376 


 All tests PASSED 
See log: RegressionTest_20120126111141.log: bug 81376 log

Bug #81568 (some inconsistencies in locations for logs and configuration) FIXED

[capannini@devel15 regression_tests]$ python execute_regTest.py -d 3 -b 81568 -l
+++++++++++++++++++++++++++++++++++++++++++++++++++++
+ Regression TestSuite ....                         +
+++++++++++++++++++++++++++++++++++++++++++++++++++++

 Testing bug 81568 


 All tests PASSED 
See log: RegressionTest_20120126112001.log: bug 81568 log

Verified that all wms logs are in this directory:

[root@emi-demo11 ~]# ls $WMS_LOCATION_LOG
glite-wms-purgeStorage.log              ice.log                  logmonitor_events.log          workload_manager_events.log.2
glite-wms-wmproxy-purge-proxycache.log  ice.log.1                wmproxy.log                    workload_manager_events.log.3
glite-wms-wmproxy.restart.cron.log      ice.log.2                wmproxy_logrotate.log
httpd-wmproxy-access.log                ice.log.3                workload_manager_events.log
httpd-wmproxy-errors.log                jobcontoller_events.log  workload_manager_events.log.1

Bug #83062 (A different template should be used for glue2 publication) FIXED

[capannini@devel15 regression_tests]$ python execute_regTest.py -d 3 -b 83062 -l
+++++++++++++++++++++++++++++++++++++++++++++++++++++
+ Regression TestSuite ....                         +
+++++++++++++++++++++++++++++++++++++++++++++++++++++

 Testing bug 83062 


 All tests PASSED 
See log: RegressionTest_20120126112643.log: bug 83062 log

Bug #85327 (glite-wms-wmproxy.restart.cron missing HOSTNAME environment variable) FIXED

[capannini@devel15 regression_tests]$ python execute_regTest.py -d 3 -b 85327 -l
+++++++++++++++++++++++++++++++++++++++++++++++++++++
+ Regression TestSuite ....                         +
+++++++++++++++++++++++++++++++++++++++++++++++++++++

 Testing bug 85327 


 All tests PASSED 
See log: RegressionTest_20120126113216.log: bug 85327 log

Bug #86485 (ICE doesn't send the iceId to CREAM) FIXED

Submitted a job through the WMS to a CREAM CE in debug mode.

Verified that in the CREAM log file the iceid is reported:

# grep selectToRetrieveJobStatusQuery /var/log/cream/glite-ce-cream.log 
20 Oct 2011 07:11:43,186 DEBUG org.glite.ce.cream.jobmanagement.db.table.JobStatusTable (JobStatusTable.java:419) - (TP-Processor25) selectToRetrieveJobStatusQuery = select job_status.type AS type,job_status.exitCode AS exitCode,job_status.failureReason AS failureReason,job_status.description AS description,job_status.time_stamp AS time_stamp,job_status.jobId AS jobId,job_status.id AS id,job.gridJobId AS gridJobId,job.workerNode AS workerNode from job_status, job where job_status.jobId = job.id and job_status.type IN ( '10', '5', '8', '7', '6', '4', '3') and job.userId = '_C_IT_O_INFN_OU_Personal_Certificate_L_Padova_CN_Massimo_Sgaravatto_dteam_Role_NULL_Capability_NULL' and job.iceId = 'C_IT_O_INFN_OU_Host_L_Padova_CN_cream-44.pd.infn.it' and job_status.id >= 0 and time_stamp >= '2011-10-19 15:46:36.0' ORDER BY job_status.id limit 500

Verified also that in the CREAMDB the iceid is set for this job:

mysql> use creamdb;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> select iceId from job where id="CREAM806900601"
    -> ;
+-----------------------------------------------------+
| iceId                                               |
+-----------------------------------------------------+
| C_IT_O_INFN_OU_Host_L_Padova_CN_cream-44.pd.infn.it | 
+-----------------------------------------------------+
1 row in set (0.00 sec)

===> test PASSED

Bug #86490 (EMI WMS stops accepting jobs after 31999th submission for the same DN (on ext3)) FIXED

[capannini@devel15 regression_tests]$ python execute_regTest.py -d 3 -b 86490 -l
+++++++++++++++++++++++++++++++++++++++++++++++++++++
+ Regression TestSuite ....                         +
+++++++++++++++++++++++++++++++++++++++++++++++++++++

 Testing bug 86490 


 All tests PASSED 
See logs: RegressionTest_20120126120305.log: bug 86490 log

Bug #86682 ( yaim changes ownership of /var) FIXED

[capannini@devel15 regression_tests]$ python execute_regTest.py -d 3 -b 86682 -l
+++++++++++++++++++++++++++++++++++++++++++++++++++++
+ Regression TestSuite ....                         +
+++++++++++++++++++++++++++++++++++++++++++++++++++++

 Testing bug 86682 


 All tests PASSED 
See log: RegressionTest_20120126120624.log: bug 86682 log

Bug #86769 (EMI WMS Update 6: WMS 'hangs' after several hours of operation) FIX NOT CERTIFIED

It is not easy to verify this bug, but the WMS has passed some stress test of one or two days of submissions. It also has passed a test done by the user who posted the bug.

Bug #86956 (Match-matching of requests with access to catalogues can cause the WM to crash) FIX NOT CERTIFIED

To use the function specified in the bug we need a jdl with DataRequirements. In any case the error is not deterministic, so you can only set this bug as hopefully fixed.

[ale@cream-12 DataReq]$ cat data-req.jdl 
###########################################
#      JDL with Data Requirements         #
###########################################

Executable = "/bin/ls";
Arguments = "-la";
StdOutput = "std.out";
StdError = "std.err";
Prologue = "prologue.sh";

FuzzyRank = true;
Environment={"LFC_HOST=prod-lfc-shared-central.cern.ch", "LFC_HOME=prod-lfc-shared-central.cern.ch:/grid/dteam/aleph"};
InputSandbox = {"prologue.sh"};
OutputSandbox = {"std.out", "std.err","prologue.out", "file.txt"};

requirements = true;

DataRequirements = {
[
DataCatalogType = "DLI";
DataCatalog ="http://prod-lfc-shared-central.cern.ch:8085/"; 
InputData = { "guid:17c667b7-9622-4c95-be68-3d15d1eacadb" };
]
};

DataAccessProtocol = "gsiftp";
AllowZippedISB=true;
RetryCount = 1;
ShallowRetryCount = 2;

[ale@cream-12 DataReq]$ glite-wms-job-list-match  -a   -c /home/ale/UI/etc/wmp_devel19.conf  data-req.jdl 

Connecting to the service https://devel19.cnaf.infn.it:7443/glite_wms_wmproxy_server

==========================================================================

           COMPUTING ELEMENT IDs LIST 
 The following CE(s) matching your job requirements have been found:

   *CEId*
 - cert-15.pd.infn.it:8443/cream-lsf-cert
 - cert-37.pd.infn.it:8443/cream-lsf-cert
 - cream-19.pd.infn.it:8443/cream-lsf-cert
 - cream-19.pd.infn.it:8443/cream-lsf-creamcert1
 - cream-19.pd.infn.it:8443/cream-lsf-creamcert2
 - cream-26.pd.infn.it:2119/jobmanager-lcgpbs-cert
 - cream-26.pd.infn.it:2119/jobmanager-lcgpbs-creamtest1
 - cream-26.pd.infn.it:2119/jobmanager-lcgpbs-creamtest2
 - cream-31.pd.infn.it:8443/cream-pbs-creamtest1
 - cream-31.pd.infn.it:8443/cream-pbs-creamtest2
 - cream-38.pd.infn.it:8443/cream-pbs-creamtest1
 - cream-38.pd.infn.it:8443/cream-pbs-creamtest2
 - prod-ce-01.pd.infn.it:8443/cream-lsf-cert
 - prod-ce-02.pd.infn.it:2119/jobmanager-lcglsf-cert
 - cream-24.pd.infn.it:8443/cream-lsf-cert
 - cream-24.pd.infn.it:8443/cream-lsf-testbedB_1
 - cream-24.pd.infn.it:8443/cream-lsf-testbedB_2
 - cream-41.pd.infn.it:8443/cream-pbs-creamtest1
 - cream-32.pd.infn.it:8443/cream-pbs-cert
 - cream-32.pd.infn.it:8443/cream-pbs-creamtest1
 - cream-32.pd.infn.it:8443/cream-pbs-creamtest2
 - ce130.cern.ch:2119/jobmanager-lcglsf-grid_2nh_dteam
 - ce130.cern.ch:2119/jobmanager-lcglsf-grid_dteam
 - ce131.cern.ch:2119/jobmanager-lcglsf-grid_2nh_dteam
 - ce131.cern.ch:2119/jobmanager-lcglsf-grid_dteam
 - ce132.cern.ch:2119/jobmanager-lcglsf-grid_2nh_dteam
 - ce132.cern.ch:2119/jobmanager-lcglsf-grid_dteam
 - ce133.cern.ch:2119/jobmanager-lcglsf-grid_2nh_dteam
 - ce133.cern.ch:2119/jobmanager-lcglsf-grid_dteam
 - ce203.cern.ch:8443/cream-lsf-grid_2nh_dteam
 - ce203.cern.ch:8443/cream-lsf-grid_dteam
 - ce204.cern.ch:8443/cream-lsf-grid_2nh_dteam
 - ce204.cern.ch:8443/cream-lsf-grid_dteam
 - ce205.cern.ch:8443/cream-lsf-grid_2nh_dteam
 - ce205.cern.ch:8443/cream-lsf-grid_dteam
 - ce206.cern.ch:8443/cream-lsf-grid_2nh_dteam
 - ce206.cern.ch:8443/cream-lsf-grid_dteam
 - ce207.cern.ch:8443/cream-lsf-grid_2nh_dteam
 - ce207.cern.ch:8443/cream-lsf-grid_dteam
 - ce208.cern.ch:8443/cream-lsf-grid_2nh_dteam
 - ce208.cern.ch:8443/cream-lsf-grid_dteam
 - cream-11.pd.infn.it:2119/jobmanager-lcglsf-cert
 - cream-11.pd.infn.it:2119/jobmanager-lcglsf-creamtest1
 - cream-11.pd.infn.it:2119/jobmanager-lcglsf-creamtest2
 - cream-20.pd.infn.it:8443/cream-lsf-cert
 - cream-20.pd.infn.it:8443/cream-lsf-creamcert1
 - cream-20.pd.infn.it:8443/cream-lsf-creamcert2
 - cream-31.pd.infn.it:8443/cream-pbs-cert
 - cream-41.pd.infn.it:8443/cream-pbs-cert
 - cream-41.pd.infn.it:8443/cream-pbs-creamtest2

==========================================================================

Bug #87259 (WMS purger errors in jobcontroller/logmonitor) FIXED

For a job submitted to a LCG-CE, verified that in /var/log/glite/logmonitor_events.log the removing of job directory, submit file, classad file, and wrapper file is logged:


28 Dec, 16:39:49 -I- EventAborted::process_event(): Job id = https://emi-demo11.cnaf.infn.it:9000/vHF_dDR46zwWtOPwA3O8Vw
28 Dec, 16:39:49 -I- JobFilePurger::do_purge(...): Removing job directory: /var/jobcontrol/condorio/vH/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fvHF_5fdDR46zwWtOPwA3O8Vw
28 Dec, 16:39:49 -U- JobFilePurger::do_purge(...): Removed 1 files.
28 Dec, 16:39:49 -I- JobFilePurger::do_purge(...): Removing submit file: /var/jobcontrol/submit/vH/Condor.https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fvHF_5fdDR46zwWtOPwA3O8Vw.submit
28 Dec, 16:39:49 -U- JobFilePurger::do_purge(...): Removed...
28 Dec, 16:39:49 -I- JobFilePurger::do_purge(...): Removing classad file: /var/jobcontrol/submit/vH/ClassAd.https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fvHF_5fdDR46zwWtOPwA3O8Vw
28 Dec, 16:39:49 -U- JobFilePurger::do_purge(...): Removed...
28 Dec, 16:39:49 -I- JobFilePurger::do_purge(...): Removing wrapper file: /var/jobcontrol/submit/vH/JobWrapper.https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fvHF_5fdDR46zwWtOPwA3O8Vw.sh
28 Dec, 16:39:49 -U- JobFilePurger::do_purge(...): Removed...

Verified that they have been actually removed:
[root@emi-demo11 ~]# ls /var/jobcontrol/condorio/vH/
[root@emi-demo11 ~]# ls /var/jobcontrol/submit/vH/

Submitted a job:

[capannini@devel15 WMS-service]$ glite-wms-job-submit -a -c ~/wms_emi-demo11.conf ~/ls.jdl

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://emi-demo11.cnaf.infn.it:9000/-iL7RrvLDu1Z9jFultvSLA

==========================================================================

Copied the sandbox dir:

[root@emi-demo11 ~]# cd /var/SandboxDir/-i/
[root@emi-demo11 -i]# ls
https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2f-iL7RrvLDu1Z9jFultvSLA
[root@emi-demo11 -i]# cp -r -p https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2f-iL7RrvLDu1Z9jFultvSLA https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2f-iL7RrvLDu1Z9jFultvSLA.old
[root@emi-demo11 -i]# 
[root@emi-demo11 -i]# 
[root@emi-demo11 -i]# ls -l 
totale 8
drwxrwx--- 5 dteam045 glite 4096 26 gen 12:23 https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2f-iL7RrvLDu1Z9jFultvSLA
drwxrwx--- 5 dteam045 glite 4096 26 gen 12:23 https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2f-iL7RrvLDu1Z9jFultvSLA.old

Retrieved the OSB when the job is done:

[capannini@devel15 WMS-service]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/-iL7RrvLDu1Z9jFultvSLA


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/-iL7RrvLDu1Z9jFultvSLA
Current Status:     Done (Success)
Exit code:          0
Status Reason:      Job Terminated Successfully
Destination:        atlasce2.lnf.infn.it:8443/cream-pbs-cert
Submitted:          Thu Jan 26 12:23:29 2012 CET
==========================================================================

[capannini@devel15 WMS-service]$ glite-wms-job-output https://emi-demo11.cnaf.infn.it:9000/-iL7RrvLDu1Z9jFultvSLA

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server


================================================================================

         JOB GET OUTPUT OUTCOME

Output sandbox files for the job:
https://emi-demo11.cnaf.infn.it:9000/-iL7RrvLDu1Z9jFultvSLA
have been successfully retrieved and stored in the directory:
/tmp/jobOutput/capannini_-iL7RrvLDu1Z9jFultvSLA

================================================================================

[capannini@devel15 WMS-service]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/-iL7RrvLDu1Z9jFultvSLA


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/-iL7RrvLDu1Z9jFultvSLA
Current Status:     Cleared 
Status Reason:      user retrieved output sandbox
Destination:        atlasce2.lnf.infn.it:8443/cream-pbs-cert
Submitted:          Thu Jan 26 12:23:29 2012 CET
==========================================================================

Restored the sandbox dir in the WMS:

[root@emi-demo11 -i]# mv https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2f-iL7RrvLDu1Z9jFultvSLA.old https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2f-iL7RrvLDu1Z9jFultvSLA

Purged the job from LBProxy and from LBServer:

[root@emi-demo11 ~]# echo 'https://emi-demo11.cnaf.infn.it:9000/-iL7RrvLDu1Z9jFultvSLA' >jobid.txt
[root@emi-demo11 ~]# 
[root@emi-demo11 ~]# /usr/bin/glite-lb-purge -m emi-demo11.cnaf.infn.it -x -j jobid.txt
The jobs were not dumped.
[root@emi-demo11 ~]# /usr/bin/glite-lb-purge -m emi-demo11.cnaf.infn.it -j jobid.txt
The jobs were not dumped.

Verified that job status returns =Identifier removed= (i.e. error =EIDRM=):

[capannini@devel15 WMS-service]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/-iL7RrvLDu1Z9jFultvSLA



**** Error: API_NATIVE_ERROR ****  
Error while calling the "Job:getStatus" native api 
Unable to retrieve the status for: https://emi-demo11.cnaf.infn.it:9000/-iL7RrvLDu1Z9jFultvSLA
glite.lb.Exception: edg_wll_JobStatus: Identifier removed: matching job already purged
   at glite::lb::Job::status[./src/Job.cpp:87]

On the WMS, purged the sandboxdir, as user glite, invoking the command =glite-wms-purgeStorage.sh= and verifying that =Identifier removed= (i.e. error 43) is returned:

[glite@emi-demo11 ~]$ /usr/sbin/glite-wms-purgeStorage.sh -p /var/SandboxDir/-i
26 Jan, 15:05:17 -E: [Error] query_job_status(/home/condor/execute/dir_26089/userdir/emi.wms.wms-purger/build/../src/purger.cpp:134): https://emi-demo11.cnaf.infn.it:9000/-iL7RrvLDu1Z9jFultvSLA: edg_wll_JobStat [43] Identifier removed(matching job already purged)
26 Jan, 15:05:17 -I: [Info] operator()(/home/condor/execute/dir_26089/userdir/emi.wms.wms-purger/build/../src/purger.cpp:306): https://emi-demo11.cnaf.infn.it:9000/-iL7RrvLDu1Z9jFultvSLA: forced removal, unknown/removed L&B job

Verified that the sandboxdir has been removed:

[glite@emi-demo11 ~]$ ls -l /var/SandboxDir/-i
total 0

===> test PASSED

Bug #87994 yaim-wms doesn't fill the attributes get_acbr and get_owner in the conf file for Glue2 FIXED

Check the attributes get_acbr and get_owner in /etc/glite/info/service/glite-info-glue2-wmproxy.conf [root@emi-demo11 ~]# grep get_acbr /etc/glite/info/service/glite-info-glue2-wmproxy.conf get_acbr = echo -e "VO:atlas\nVO:alice\nVO:lhcb\nVO:cms\nVO:dteam\nVO:biomed\nVO:ops\nVO:infngrid\nVO:testers.eu-emi.eu" [root@emi-demo11 ~]# grep get_owner /etc/glite/info/service/glite-info-glue2-wmproxy.conf get_owner = echo -e "atlas\nalice\nlhcb\ncms\ndteam\nbiomed\nops\ninfngrid\ntesters.eu-emi.eu"

===> test PASSED

Bug #88072 EMI-1 WMS cannot submit to ARC CEs FIXED

-bash-3.2$ glite-wms-job-status https://devel19.cnaf.infn.it:9000/pl6ZSNy7MNWNSH6x8650DQ

======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://devel19.cnaf.infn.it:9000/pl6ZSNy7MNWNSH6x8650DQ
Current Status:     Done (Success)
Exit code:          0
Status Reason:      Job terminated successfully
Destination:        jade-cms.hip.fi:2811/nordugrid-GE-arc
Submitted:          Mon Nov 21 16:01:15 2011 CET
==========================================================================



-bash-3.2$ glite-wms-job-output https://devel19.cnaf.infn.it:9000/pl6ZSNy7MNWNSH6x8650DQ

Connecting to the service https://devel19.cnaf.infn.it:7443/glite_wms_wmproxy_server

================================================================================

                       JOB GET OUTPUT OUTCOME

Output sandbox files for the job:
https://devel19.cnaf.infn.it:9000/pl6ZSNy7MNWNSH6x8650DQ
have been successfully retrieved and stored in the directory:
/tmp/jobOutput/traldi_pl6ZSNy7MNWNSH6x8650DQ

================================================================================



-bash-3.2$ cat /tmp/jobOutput/traldi_pl6ZSNy7MNWNSH6x8650DQ/out.txt
total 32
drwx------ 2 cmsuser43 cmsuser 4096 Nov 21 17:02 .
drwx------ 3 cmsuser43 cmsuser 4096 Nov 21 17:02 ..
-rw------- 1 cmsuser43 cmsuser  118 Nov 21 17:02 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2fpl6ZSNy7MNWNSH6x8650DQ.output
-rw-r--r-- 1 cmsuser43 cmsuser    0 Nov 21 17:02 out.txt
-rw-r--r-- 1 cmsuser43 cmsuser 1359 Nov 21 17:02 pippo.jdl
-rw------- 1 cmsuser43 cmsuser    2 Nov 21 17:02 tmp.bUujW32646
-rw------- 1 cmsuser43 cmsuser  103 Nov 21 17:02 tmp.cNkaG32632
-rw------- 1 cmsuser43 cmsuser  103 Nov 21 17:02 tmp.exhlZ32647
-rw------- 1 cmsuser43 cmsuser    2 Nov 21 17:02 tmp.gZIii32631
-rw------- 1 cmsuser43 cmsuser    0 Nov 21 17:02 tmp.SdwGF32645 

[ale@cream-12 WMS-service]$ glite-wms-job-status https://devel19.cnaf.infn.it:9000/pl6ZSNy7MNWNSH6x8650DQ 


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://devel19.cnaf.infn.it:9000/pl6ZSNy7MNWNSH6x8650DQ
Current Status:     Cleared 
Status Reason:      user retrieved output sandbox
Destination:        jade-cms.hip.fi:2811/nordugrid-GE-arc
Submitted:          Mon Nov 21 16:01:15 2011 CET
==========================================================================


===> test PASSED

Bug #88096 CA certificate verification failed FIXED NOT CERTIFIED

It is not easy to verify this bug, but the WMS has passed some stress test of one or two days of submissions.

Bug #88097 Collection not terminated FIXED NOT CERTIFIED

It is not easy to verify this bug, but the WMS has passed some stress test of one or two days of submissions.

Bug #88450 Ice configuration section doesn't use environment variable FIXED

[root@cream-44 ~]# cat /etc/glite-wms/glite_wms.conf | grep -i " logfile"
    logfile   =   "${WMS_LOCATION_LOG}/ice.log";
    LogFile  =  "${WMS_LOCATION_LOG}/wmproxy.log";
    LogFile  =  "${WMS_LOCATION_LOG}/networkserver_events.log";
    LogFile   =  "${WMS_LOCATION_LOG}/workload_manager_events.log";
    LogFile  =  "${WMS_LOCATION_LOG}/logmonitor_events.log";
    LogFile  =  "${WMS_LOCATION_LOG}/jobcontoller_events.log";

===> test PASSED

Bug #88558 WM creates submit requests with old sequence code upon a replan FIXED

Event: Resubmission
- Arrived                    =    Wed Nov 23 19:16:16 2011 CET
- Host                       =    devel19.cnaf.infn.it
- Level                      =    SYSTEM
- Priority                   =    synchronous
- Reason                     =    token still exists
- Result                     =    SHALLOW
- Seqcode                    =    UI=000000:NS=0000000004:WM=000007:BH=0000000000:JSS=000002:LM=000000:LRMS=000000:APP=000000:LBS=000000
- Source                     =    WorkloadManager
- Src instance               =    13159
- Tag                        =    job will be replanned
- Timestamp                  =    Wed Nov 23 19:16:16 2011 CET
- User                       =    /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy

[...]

Event: EnQueued
- Arrived                    =    Wed Nov 23 19:16:16 2011 CET
- Host                       =    devel19.cnaf.infn.it
- Level                      =    SYSTEM
- Priority                   =    synchronous
- Queue                      =    /var/workload_manager/jobdir
- Result                     =    OK
- Seqcode                    =    UI=000000:NS=0000000004:WM=000009:BH=0000000000:JSS=000002:LM=000000:LRMS=000000:APP=000000:LBS=000000
- Source                     =    WorkloadManager
- Src instance               =    13159
- Timestamp                  =    Wed Nov 23 19:16:16 2011 CET
- User                       =    /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy
- Job            =   
      
       [
        version = "1.0.0"; 
        arguments = 
         [
          ad = 
           [
            NodeName = "Node_jdl_953"; 
            FuzzyRank = true; 
            CEInfoHostName = "cream-31.pd.infn.it"; 
            OutputSandboxPath = "/var/SandboxDir/ji/https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2fjiHL26jdUVDO7Bl6BJlhEQ/output"; 
            StdOutput = "out.log"; 
            GlobusResourceContactString = "cream-31.pd.infn.it:8443/cream-pbs"; 
            QueueName = "creamtest2"; 
            ShallowRetryCount = 3; 
            RetryCount = 2; 
            Executable = "/bin/sleep"; 
            Type = "job"; 
            LB_sequence_code = "UI=000000:NS=0000000004:WM=000009:BH=0000000000:JSS=000002:LM=000000:LRMS=000000:APP=000000:LBS=000000"; 


===> test PASSED

Bug #88569 WMS: LCMAPS should log on file only and not on syslog FIXED

Check that no messages about LCMAPS have been logged by wmproxy on syslog:

[root@cream-44 ~]# grep glite_wms_wmproxy_server /var/log/messages* | grep -i "LCMAPS"
[root@cream-44 ~]# 

And also that there is a log file for lcmaps:

[root@cream-44 ~]# ls -l /var/log/glite/lcmaps.log 
-rw-r--r-- 1 glite glite 659344 Nov 21 15:43 /var/log/glite/lcmaps.log

===> test PASSED

Nagios probes test

Clean installation

Environment

Clean SL5 installation plus an EMI-UI according to EMI guidelines.

[root@devel15 ~]# cat /etc/emi-version
1.6.0-1

Repositories

Install

yum install emi-wms-nagios

[root@devel15 ~]# yum install emi-wms-nagios
Loaded plugins: downloadonly, kernel-module, priorities, protect-packages, protectbase, security, verify, versionlock
Repository EGI-trustanchors is listed more than once in the configuration
EMI-1-base                                                                                                                            | 1.9 kB     00:00     
EMI-1-testing-base                                                                                                                    | 1.9 kB     00:00     
EMI-1-third-party                                                                                                                     | 1.9 kB     00:00     
EMI-1-updates                                                                                                                         | 1.9 kB     00:00     
Oracle                                                                                                                                | 1.9 kB     00:00     
core                                                                                                                                  | 1.1 kB     00:00     
egi-sam                                                                                                                               | 1.9 kB     00:00     
extras                                                                                                                                | 3.7 kB     00:00     
4168 packages excluded due to repository priority protections
23 packages excluded due to repository protections
Reading version lock configuration
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package emi-wms-nagios.noarch 0:1.0.0-1.el5 set to be updated
--> Processing Dependency: nagios-submit-conf >= 0.2 for package: emi-wms-nagios
--> Processing Dependency: python-GridMon >= 1.1.10 for package: emi-wms-nagios
--> Processing Dependency: python-ldap for package: emi-wms-nagios
--> Running transaction check
---> Package nagios-submit-conf.noarch 0:0.2-2.el5 set to be updated
---> Package python-GridMon.noarch 0:1.1.12-1.el5 set to be updated
---> Package python-ldap.x86_64 0:2.2.0-2.1 set to be updated
--> Finished Dependency Resolution
Beginning Kernel Module Plugin
Finished Kernel Module Plugin

Dependencies Resolved

=============================================================================================================================================================
 Package                                   Arch                          Version                             Repository                                 Size
=============================================================================================================================================================
Installing:
 emi-wms-nagios                            noarch                        1.0.0-1.el5                         EMI-1-testing-base                         61 k
Installing for dependencies:
 nagios-submit-conf                        noarch                        0.2-2.el5                           egi-sam                                   3.4 k
 python-GridMon                            noarch                        1.1.12-1.el5                        egi-sam                                   153 k
 python-ldap                               x86_64                        2.2.0-2.1                           sl-base                                   123 k

Transaction Summary
=============================================================================================================================================================
Install       4 Package(s)
Upgrade       0 Package(s)

Total download size: 340 k
Is this ok [y/N]: y
Downloading Packages:
(1/4): nagios-submit-conf-0.2-2.el5.noarch.rpm                                                                                        | 3.4 kB     00:00     
(2/4): emi-wms-nagios-1.0.0-1.el5.noarch.rpm                                                                                          |  61 kB     00:00     
(3/4): python-ldap-2.2.0-2.1.x86_64.rpm                                                                                               | 123 kB     00:00     
(4/4): python-GridMon-1.1.12-1.el5.noarch.rpm                                                                                         | 153 kB     00:00     
-------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                         85 kB/s | 340 kB     00:04     
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing     : python-ldap                                                                                                                           1/4 
  Installing     : nagios-submit-conf                                                                                                                    2/4 
  Installing     : python-GridMon                                                                                                                        3/4 
  Installing     : emi-wms-nagios                                                                                                                        4/4 

Installed:
  emi-wms-nagios.noarch 0:1.0.0-1.el5                                                                                                                        

Dependency Installed:
  nagios-submit-conf.noarch 0:0.2-2.el5                 python-GridMon.noarch 0:1.1.12-1.el5                 python-ldap.x86_64 0:2.2.0-2.1                

Complete!

Upgrade from production

N/A: This is the initial release. No previous version to upgrade from.

Unit Tests

N/A: The nagios probes are interpretted python scripts. Unit tests do not apply.

Regression Tests

N/A: This is the initial release. No associated bugs exist.

Functional Tests

Preparation

Log as a user and create a valid voms proxy

WMSProbe

  • State + Monit
[capannini@devel15 ~]$ /usr/libexec/grid-monitoring/probes/emi.wms/WMS-probe --vo dteam -x /tmp/x509up_u504 -H emi-demo11.cnaf.infn.it -m emi.wms.WMS-JobState
OK: 
OK: 
Testing from: devel15.cnaf.infn.it
DN: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy
VOMS FQANs: /dteam/Role=NULL/Capability=NULL, /dteam/NGI_IT/Role=NULL/Capability=NULL


Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://emi-demo11.cnaf.infn.it:9000/nc4oU4nTtQMspwuhs0CK2A

The job identifier has been saved in the following file:
/var/lib/gridprobes/dteam/emi.wms/WMS/emi-demo11.cnaf.infn.it/jobID

==========================================================================

[capannini@devel15 ~]$ /usr/libexec/grid-monitoring/probes/emi.wms/WMS-probe --vo dteam -x /tmp/x509up_u504 -H emi-demo11.cnaf.infn.it -m emi.wms.WMS-JobMonit --pass-check-dest active
metric results >>> <emi-demo11.cnaf.infn.it,emi.wms.WMS-JobState-dteam>
OK: [Running] https://emi-demo11.cnaf.infn.it:9000/nc4oU4nTtQMspwuhs0CK2A
OK: [Running] https://emi-demo11.cnaf.infn.it:9000/nc4oU4nTtQMspwuhs0CK2A
glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/nc4oU4nTtQMspwuhs0CK2A


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/nc4oU4nTtQMspwuhs0CK2A
Current Status:     Running 
Status Reason:      Job successfully submitted to Globus
Destination:        cmsce01.na.infn.it:2119/jobmanager-lcgpbs-cert
Submitted:          Thu Jan 26 16:14:33 2012 CET
==========================================================================
OK: Jobs processed - 1
OK: Jobs processed - 1
[Running] : 1|jobs_processed=1;; DONE=0;; RUNNING=1;; SCHEDULED=0;; SUBMITTED=0;; READY=0;; WAITING=0;; ABORTED=0;; CANCELLED=0;; CLEARED=0;; MISSED=0;; UNDETERMINED=0;; unknown=0;1;2

[capannini@devel15 ~]$ /usr/libexec/grid-monitoring/probes/emi.wms/WMS-probe --vo dteam -x /tmp/x509up_u504 -H emi-demo11.cnaf.infn.it -m emi.wms.WMS-JobState
OK: Active job - Running [2012-01-26T15:16:24Z]
OK: Active job - Running [2012-01-26T15:16:24Z]
Testing from: devel15.cnaf.infn.it
DN: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy
VOMS FQANs: /dteam/Role=NULL/Capability=NULL, /dteam/NGI_IT/Role=NULL/Capability=NULL
Active job - Running [2012-01-26T15:16:24Z]

[capannini@devel15 ~]$ /usr/libexec/grid-monitoring/probes/emi.wms/WMS-probe --vo dteam -x /tmp/x509up_u504 -H emi-demo11.cnaf.infn.it -m emi.wms.WMS-JobMonit --pass-check-dest active
metric results >>> <emi-demo11.cnaf.infn.it,emi.wms.WMS-JobSubmit-dteam>
OK: success.

glite-wms-job-output --noint --nosubdir --dir /var/lib/gridprobes/dteam/emi.wms/WMS/emi-demo11.cnaf.infn.it/jobOutput https://emi-demo11.cnaf.infn.it:9000/nc4oU4nTtQMspwuhs0CK2A 2>&1

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server


Warning - option --nosubdir specified: 
output files with same name will be overridden


Warning - Directory already exists: 
/var/lib/gridprobes/dteam/emi.wms/WMS/emi-demo11.cnaf.infn.it/jobOutput


================================================================================

         JOB GET OUTPUT OUTCOME

Output sandbox files for the job:
https://emi-demo11.cnaf.infn.it:9000/nc4oU4nTtQMspwuhs0CK2A
have been successfully retrieved and stored in the directory:
/var/lib/gridprobes/dteam/emi.wms/WMS/emi-demo11.cnaf.infn.it/jobOutput

================================================================================
metric results >>> <emi-demo11.cnaf.infn.it,emi.wms.WMS-JobState-dteam>
OK: success.

glite-wms-job-output --noint --nosubdir --dir /var/lib/gridprobes/dteam/emi.wms/WMS/emi-demo11.cnaf.infn.it/jobOutput https://emi-demo11.cnaf.infn.it:9000/nc4oU4nTtQMspwuhs0CK2A 2>&1

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server


Warning - option --nosubdir specified: 
output files with same name will be overridden


Warning - Directory already exists: 
/var/lib/gridprobes/dteam/emi.wms/WMS/emi-demo11.cnaf.infn.it/jobOutput


================================================================================

         JOB GET OUTPUT OUTCOME

Output sandbox files for the job:
https://emi-demo11.cnaf.infn.it:9000/nc4oU4nTtQMspwuhs0CK2A
have been successfully retrieved and stored in the directory:
/var/lib/gridprobes/dteam/emi.wms/WMS/emi-demo11.cnaf.infn.it/jobOutput

================================================================================
OK: Jobs processed - 1
OK: Jobs processed - 1
Done : 1|jobs_processed=1;; DONE=1;; RUNNING=0;; SCHEDULED=0;; SUBMITTED=0;; READY=0;; WAITING=0;; ABORTED=0;; CANCELLED=0;; CLEARED=0;; MISSED=0;; UNDETERMINED=0;; unknown=0;1;2

[capannini@devel15 ~]$ cat /var/lib/gridprobes/dteam/emi.wms/WMS/emi-demo11.cnaf.infn.it/jobOutput/gridjob.out 
cmswn04.na.infn.it

Summary: PASSED

  • State + Monit + Cancel


[capannini@devel15 ~]$ /usr/libexec/grid-monitoring/probes/emi.wms/WMS-probe --vo dteam -x /tmp/x509up_u504 -H emi-demo11.cnaf.infn.it -m emi.wms.WMS-JobState
OK: 
OK: 
Testing from: devel15.cnaf.infn.it
DN: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy
VOMS FQANs: /dteam/Role=NULL/Capability=NULL, /dteam/NGI_IT/Role=NULL/Capability=NULL


Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://emi-demo11.cnaf.infn.it:9000/HnfZcXc--G7TtJ0fzhYGcg

The job identifier has been saved in the following file:
/var/lib/gridprobes/dteam/emi.wms/WMS/emi-demo11.cnaf.infn.it/jobID

==========================================================================

[capannini@devel15 ~]$ /usr/libexec/grid-monitoring/probes/emi.wms/WMS-probe --vo dteam -x /tmp/x509up_u504 -H emi-demo11.cnaf.infn.it -m emi.wms.WMS-JobMonit --pass-check-dest active
metric results >>> <emi-demo11.cnaf.infn.it,emi.wms.WMS-JobState-dteam>
OK: [Scheduled] https://emi-demo11.cnaf.infn.it:9000/HnfZcXc--G7TtJ0fzhYGcg
OK: [Scheduled] https://emi-demo11.cnaf.infn.it:9000/HnfZcXc--G7TtJ0fzhYGcg
glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/HnfZcXc--G7TtJ0fzhYGcg


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/HnfZcXc--G7TtJ0fzhYGcg
Current Status:     Scheduled 
Status Reason:      unavailable
Destination:        inaf-ce-01.ct.pi2s2.it:8443/cream-lsf-cert
Submitted:          Thu Jan 26 16:27:27 2012 CET
==========================================================================
OK: Jobs processed - 1
OK: Jobs processed - 1
[Scheduled] : 1|jobs_processed=1;; DONE=0;; RUNNING=0;; SCHEDULED=1;; SUBMITTED=0;; READY=0;; WAITING=0;; ABORTED=0;; CANCELLED=0;; CLEARED=0;; MISSED=0;; UNDETERMINED=0;; unknown=0;1;2

[capannini@devel15 ~]$ /usr/libexec/grid-monitoring/probes/emi.wms/WMS-probe --vo dteam -x /tmp/x509up_u504 -H emi-demo11.cnaf.infn.it -m emi.wms.WMS-JobCancel
OK: job cancelled
OK: job cancelled
Testing from: devel15.cnaf.infn.it
DN: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy
VOMS FQANs: /dteam/Role=NULL/Capability=NULL, /dteam/NGI_IT/Role=NULL/Capability=NULL
Job cancellation request sent:
glite-wms-job-cancel --noint  -i /var/lib/gridprobes/dteam/emi.wms/WMS/emi-demo11.cnaf.infn.it/jobID
Job bookkeeping files deleted.

[capannini@devel15 ~]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/HnfZcXc--G7TtJ0fzhYGcg


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/HnfZcXc--G7TtJ0fzhYGcg
Current Status:     Cancelled 
Destination:        inaf-ce-01.ct.pi2s2.it:8443/cream-lsf-cert
Submitted:          Thu Jan 26 16:27:27 2012 CET
==========================================================================

Summary: PASSED

-- FabioCapannini - 2011-12-28

Topic attachments
I Attachment Action SizeSorted ascending Date Who Comment
Unknown file formatlog RegressionTest_20120126111141.log manage 1.6 K 2012-01-26 - 10:14 FabioCapannini bug 81376 log
Unknown file formatlog RegressionTest_20120126120624.log manage 1.8 K 2012-01-26 - 11:07 FabioCapannini bug 86682 log
Unknown file formatlog RegressionTest_20120126112001.log manage 1.9 K 2012-01-26 - 10:24 FabioCapannini bug 81568 log
Unknown file formatlog RegressionTest_20120126113216.log manage 1.9 K 2012-01-26 - 10:40 FabioCapannini bug 85327 log
Compressed Zip archivetgz delegation.tgz manage 1.9 K 2012-01-20 - 09:34 FabioCapannini delegation logs
Compressed Zip archivetgz dag.tgz manage 2.2 K 2012-01-18 - 11:17 FabioCapannini dag logs
Compressed Zip archivetgz parallel.tgz manage 2.3 K 2012-01-18 - 15:37 FabioCapannini parallel job logs
Unknown file formatlog RegressionTest_20120126112643.log manage 2.5 K 2012-01-26 - 10:28 FabioCapannini bug 83062 log
Compressed Zip archivetgz parametric.tgz manage 3.1 K 2012-01-18 - 11:03 FabioCapannini parametric logs
Compressed Zip archivetgz prolepilog.tgz manage 3.2 K 2012-01-23 - 15:45 FabioCapannini prologue epilog logs
Compressed Zip archivetgz resub.tgz manage 3.8 K 2012-01-24 - 09:55 FabioCapannini resubmission logs
Unknown file formatlog WMSService-TS_20120123121509.log manage 3.9 K 2012-01-23 - 11:16 FabioCapannini list-match log
Unknown file formatlog WMSService-TS_20120123104906.log manage 4.1 K 2012-01-23 - 09:58 FabioCapannini list-match log
Unknown file formatlog WMSService-TS_20120123121055.log manage 4.1 K 2012-01-23 - 11:12 FabioCapannini list-match log
Compressed Zip archivetgz perusal.tgz manage 4.1 K 2012-01-20 - 15:16 FabioCapannini perusal logs
Compressed Zip archivetgz submit.log.tgz manage 5.1 K 2011-12-29 - 11:06 FabioCapannini Normal submission logs
Compressed Zip archivetgz coll.tgz manage 5.8 K 2012-01-17 - 10:35 FabioCapannini all in a jdl collection logs
Compressed Zip archivetgz cancel.tgz manage 8.8 K 2012-01-24 - 10:09 FabioCapannini Job cancel logs
Unknown file formatlog yaim-install.log manage 9.0 K 2011-12-28 - 11:30 FabioCapannini yaim install log
Unknown file formatlog WMSService-TS_20120116115816.log manage 13.0 K 2012-01-16 - 11:18 FabioCapannini sdj log
Unknown file formatlog WMSService-TS_20120116163257.log manage 16.1 K 2012-01-16 - 15:42 FabioCapannini bulk collection log
Unknown file formatlog RegressionTest_20120126120305.log manage 16.8 K 2012-01-26 - 11:04 FabioCapannini bug 86490 log
Unknown file formatlog WMSService-TS_20120116151500.log manage 17.8 K 2012-01-16 - 14:45 FabioCapannini bulk collection log
Unknown file formatlog WMSService-TS_20120116154624.log manage 19.2 K 2012-01-16 - 15:01 FabioCapannini bulk collection log
Unknown file formatlog WMSService-TS_20120124151737.log manage 31.1 K 2012-01-24 - 14:20 FabioCapannini limiter log
Unknown file formatlog update.log manage 37.6 K 2011-12-28 - 14:31 FabioCapannini Update log
Unknown file formatlog WMSService-TS_20120125120754.log manage 46.7 K 2012-01-25 - 11:18 FabioCapannini purge logs
Unknown file formatlog WMSService-TS_20120126104752.log manage 69.0 K 2012-01-26 - 10:37 FabioCapannini proxy renewal log
Unknown file formatlog install.log manage 124.2 K 2011-12-28 - 11:04 FabioCapannini Clean install log
Edit | Attach | PDF | History: r34 < r33 < r32 < r31 < r30 | Backlinks | Raw View | More topic actions
Topic revision: r34 - 2012-03-27 - FabioCapannini
 
This site is powered by the TWiki collaboration platformCopyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback